2024-09-18T11:19:40.331 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-18T11:19:40.353 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-18T11:19:40.444 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910958 branch: wip-bharath11-testing-2024-09-18-1115-quincy description: rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/dedup-io-snaps} email: skanta@redhat.com first_in_suite: false job_id: '7910958' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-bharath11-testing-2024-09-18-1115-quincy ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 1 ms inject socket failures: 5000 ms type: async osd_async_recovery_min_cost: 1 osd_object_clean_region_max_num_intervals: 1000 osd_pool_default_min_size: 2 osd_pool_default_size: 2 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bluefs allocator: hybrid bluestore allocator: hybrid bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd backoff on degraded: true osd backoff on peering: true osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd heartbeat use min delay socket: true osd inject bad map crc probability: 0.1 osd map cache size: 1 osd max backfills: 6 osd max markdown count: 1000 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub during recovery: false osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(REQUEST_STUCK\) - failed to encode map - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - osd_map_cache_size - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - mons down - mon down - out of quorum - \(POOL_APP_NOT_ENABLED\) sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: enable_coprs: - ceph/el9 flavor: default sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath11-testing-2024-09-18-1115-quincy sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 owner: scheduled_skanta@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 4631 sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath11-testing-2024-09-18-1115-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_7061ea6ca50908d104e9f3979dc615e118d9e153/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 targets: smithi119.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGeluLGInTfcczQVTIBdUwFHyIAxfqoH5hSiTxx2kZGTqjQyUB7X/7WCg37rkGRGbYbe4gbjP7Ht/cGkDfU3hUY= smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPEvG5PCDgOEEu1FawCRpv/8Hrtl/9DnDENp5gwRmc/UV6DTBdL1at90zbJx627NMpbATe7k4wAJQnApNGlsTlk= tasks: - install: null - ceph: conf: osd: debug monc: 20 - exec: mon.a: - while ! ceph balancer status ; do sleep 1 ; done - ceph balancer mode crush-compat - ceph balancer on - thrashosds: chance_pgnum_grow: 0.25 chance_pgnum_shrink: 0.25 chance_pgpnum_fix: 0.25 chance_test_map_discontinuity: 2 timeout: 1800 - exec: client.0: - sudo ceph osd pool create low_tier 4 - rados: clients: - client.0 dedup_chunk_algo: fastcdc dedup_chunk_size: '131072' enable_dedup: true low_tier_pool: low_tier objects: 50 op_weights: read: 100 rollback: 10 set_chunk: 30 snap_create: 10 snap_remove: 10 tier_evict: 10 tier_flush: 5 tier_promote: 10 write: 50 ops: 1500 set_chunk: true teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_09:13:00 tube: smithi user: skanta verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-18T11:19:40.445 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_7061ea6ca50908d104e9f3979dc615e118d9e153/qa; will attempt to use it 2024-09-18T11:19:40.445 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_7061ea6ca50908d104e9f3979dc615e118d9e153/qa/tasks 2024-09-18T11:19:40.446 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-18T11:19:40.447 INFO:teuthology.task.internal:Checking packages... 2024-09-18T11:19:40.468 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '7061ea6ca50908d104e9f3979dc615e118d9e153' 2024-09-18T11:19:40.468 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-18T11:19:40.468 INFO:teuthology.packaging:ref: None 2024-09-18T11:19:40.468 INFO:teuthology.packaging:tag: None 2024-09-18T11:19:40.468 INFO:teuthology.packaging:branch: wip-bharath11-testing-2024-09-18-1115-quincy 2024-09-18T11:19:40.468 INFO:teuthology.packaging:sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T11:19:40.468 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-bharath11-testing-2024-09-18-1115-quincy 2024-09-18T11:19:40.608 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1645.g7061ea6c 2024-09-18T11:19:40.610 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-18T11:19:40.617 INFO:teuthology.task.internal:no buildpackages task found 2024-09-18T11:19:40.617 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-18T11:19:40.658 INFO:teuthology.task.internal:Saving configuration 2024-09-18T11:19:40.675 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-18T11:19:40.683 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-18T11:19:40.701 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910958', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-18 11:14:42.628448', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGeluLGInTfcczQVTIBdUwFHyIAxfqoH5hSiTxx2kZGTqjQyUB7X/7WCg37rkGRGbYbe4gbjP7Ht/cGkDfU3hUY='} 2024-09-18T11:19:40.719 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910958', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-18 11:14:42.629936', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPEvG5PCDgOEEu1FawCRpv/8Hrtl/9DnDENp5gwRmc/UV6DTBdL1at90zbJx627NMpbATe7k4wAJQnApNGlsTlk='} 2024-09-18T11:19:40.719 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-18T11:19:40.725 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-18T11:19:40.725 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-18T11:19:40.725 INFO:teuthology.run_tasks:Running task console_log... 2024-09-18T11:19:40.778 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f402b8b7ac0>, signals=[15]) 2024-09-18T11:19:40.778 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-18T11:19:40.784 INFO:teuthology.task.internal:Opening connections... 2024-09-18T11:19:40.784 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2024-09-18T11:19:40.785 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T11:19:40.861 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2024-09-18T11:19:40.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T11:19:40.941 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-18T11:19:40.947 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2024-09-18T11:19:40.965 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2024-09-18T11:19:40.965 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2024-09-18T11:19:41.023 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="CentOS Stream" 2024-09-18T11:19:41.024 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="9" 2024-09-18T11:19:41.024 INFO:teuthology.orchestra.run.smithi119.stdout:ID="centos" 2024-09-18T11:19:41.024 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="rhel fedora" 2024-09-18T11:19:41.024 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="9" 2024-09-18T11:19:41.024 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el9" 2024-09-18T11:19:41.024 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-18T11:19:41.024 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2024-09-18T11:19:41.024 INFO:teuthology.orchestra.run.smithi119.stdout:LOGO="fedora-logo-icon" 2024-09-18T11:19:41.024 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-18T11:19:41.024 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://centos.org/" 2024-09-18T11:19:41.024 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-18T11:19:41.024 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-18T11:19:41.025 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-18T11:19:41.025 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2024-09-18T11:19:41.067 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2024-09-18T11:19:41.085 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2024-09-18T11:19:41.086 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2024-09-18T11:19:41.146 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2024-09-18T11:19:41.146 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="9" 2024-09-18T11:19:41.146 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2024-09-18T11:19:41.146 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2024-09-18T11:19:41.146 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="9" 2024-09-18T11:19:41.146 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el9" 2024-09-18T11:19:41.146 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-18T11:19:41.146 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2024-09-18T11:19:41.146 INFO:teuthology.orchestra.run.smithi148.stdout:LOGO="fedora-logo-icon" 2024-09-18T11:19:41.146 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-18T11:19:41.146 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2024-09-18T11:19:41.147 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-18T11:19:41.147 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-18T11:19:41.147 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-18T11:19:41.147 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2024-09-18T11:19:41.175 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-18T11:19:41.185 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-18T11:19:41.190 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-18T11:19:41.191 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2024-09-18T11:19:41.195 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2024-09-18T11:19:41.211 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-18T11:19:41.217 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-18T11:19:41.217 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2024-09-18T11:19:41.252 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2024-09-18T11:19:41.540 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-18T11:19:41.589 INFO:teuthology.run_tasks:Running task kernel... 2024-09-18T11:19:41.603 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-18T11:19:41.604 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-18T11:19:41.604 DEBUG:teuthology.orchestra.run.smithi119:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T11:19:41.605 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T11:19:41.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T11:19:41.622 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2024-09-18T11:19:41.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T11:19:41.623 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-09-18T11:19:41.680 INFO:teuthology.orchestra.run.smithi119.stdout:5.14.0-503.el9.x86_64 2024-09-18T11:19:41.680 INFO:teuthology.task.kernel:Running kernel on smithi119: 5.14.0-503.el9.x86_64 2024-09-18T11:19:41.680 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum install -y kernel 2024-09-18T11:19:41.681 INFO:teuthology.orchestra.run.smithi148.stdout:5.14.0-503.el9.x86_64 2024-09-18T11:19:41.681 INFO:teuthology.task.kernel:Running kernel on smithi148: 5.14.0-503.el9.x86_64 2024-09-18T11:19:41.681 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum install -y kernel 2024-09-18T11:19:44.059 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.3 MB 00:00 2024-09-18T11:19:44.729 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 5.9 MB/s | 8.3 MB 00:01 2024-09-18T11:19:46.689 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-18T11:19:48.476 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 6.1 MB/s | 20 MB 00:03 2024-09-18T11:19:52.148 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.5 MB 00:01 2024-09-18T11:19:54.340 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:00 2024-09-18T11:19:55.018 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-09-18T11:19:55.577 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-18T11:19:56.718 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-18T11:19:57.898 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-18T11:20:00.958 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-18T11:20:02.848 INFO:teuthology.orchestra.run.smithi119.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-18T11:20:02.848 INFO:teuthology.orchestra.run.smithi119.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-18T11:20:02.934 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:20:02.944 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:20:02.944 INFO:teuthology.orchestra.run.smithi119.stdout: Package Architecture Version Repository Size 2024-09-18T11:20:02.944 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:20:02.944 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-09-18T11:20:02.944 INFO:teuthology.orchestra.run.smithi119.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-18T11:20:02.945 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-09-18T11:20:02.945 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-18T11:20:02.945 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-18T11:20:02.945 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-18T11:20:02.945 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:20:02.945 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-09-18T11:20:02.945 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:20:02.945 INFO:teuthology.orchestra.run.smithi119.stdout:Install 4 Packages 2024-09-18T11:20:02.945 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:20:02.945 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 79 M 2024-09-18T11:20:02.945 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 126 M 2024-09-18T11:20:02.945 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-09-18T11:20:03.268 INFO:teuthology.orchestra.run.smithi119.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 130 kB/s | 36 kB 00:00 2024-09-18T11:20:03.320 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-18T11:20:05.235 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-18T11:20:05.235 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-18T11:20:05.316 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:20:05.325 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:20:05.325 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-09-18T11:20:05.325 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:20:05.325 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-18T11:20:05.325 INFO:teuthology.orchestra.run.smithi148.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-18T11:20:05.325 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-18T11:20:05.325 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-18T11:20:05.325 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-18T11:20:05.325 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-18T11:20:05.325 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:20:05.325 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-18T11:20:05.325 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:20:05.326 INFO:teuthology.orchestra.run.smithi148.stdout:Install 4 Packages 2024-09-18T11:20:05.326 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:20:05.326 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 79 M 2024-09-18T11:20:05.326 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 126 M 2024-09-18T11:20:05.326 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-18T11:20:05.446 INFO:teuthology.orchestra.run.smithi148.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 494 kB/s | 36 kB 00:00 2024-09-18T11:20:05.959 INFO:teuthology.orchestra.run.smithi119.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 5.3 MB/s | 16 MB 00:02 2024-09-18T11:20:05.997 INFO:teuthology.orchestra.run.smithi148.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-09-18T11:20:06.762 INFO:teuthology.orchestra.run.smithi119.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 9.2 MB/s | 35 MB 00:03 2024-09-18T11:20:06.914 INFO:teuthology.orchestra.run.smithi148.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 19 MB/s | 29 MB 00:01 2024-09-18T11:20:07.247 INFO:teuthology.orchestra.run.smithi148.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 18 MB/s | 35 MB 00:01 2024-09-18T11:20:07.247 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:20:07.248 INFO:teuthology.orchestra.run.smithi148.stdout:Total 41 MB/s | 79 MB 00:01 2024-09-18T11:20:07.932 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-18T11:20:08.308 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-18T11:20:08.308 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-18T11:20:09.270 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-18T11:20:09.271 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-18T11:20:11.695 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-18T11:20:12.566 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-18T11:20:12.664 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T11:20:14.870 INFO:teuthology.orchestra.run.smithi119.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 2.5 MB/s | 29 MB 00:11 2024-09-18T11:20:14.870 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:20:14.870 INFO:teuthology.orchestra.run.smithi119.stdout:Total 6.6 MB/s | 79 MB 00:11 2024-09-18T11:20:15.406 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T11:20:15.540 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T11:20:15.549 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-09-18T11:20:15.914 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-09-18T11:20:15.914 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-09-18T11:20:16.855 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-09-18T11:20:16.856 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-09-18T11:20:19.282 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-09-18T11:20:19.921 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-18T11:20:20.012 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T11:20:20.393 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T11:20:20.491 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T11:20:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T11:20:22.915 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T11:20:25.346 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T11:20:27.751 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T11:20:27.881 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T11:20:32.742 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T11:21:03.448 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T11:21:03.488 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-18T11:21:03.758 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T11:21:03.758 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-18T11:21:03.758 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T11:21:03.758 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T11:21:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T11:21:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:21:04.596 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-18T11:21:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-18T11:21:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-18T11:21:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-18T11:21:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-18T11:21:04.597 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:21:04.597 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:21:04.982 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T11:21:05.062 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T11:21:05.063 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T11:21:05.063 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-18T11:21:05.063 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T11:21:05.063 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T11:21:05.063 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-18T11:21:05.063 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi148.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T11:21:05.063 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T11:21:05.123 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T11:21:05.123 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T11:21:05.123 DEBUG:teuthology.orchestra.run.smithi148:> sudo rpm -qi grub2-tools 2024-09-18T11:21:05.167 INFO:teuthology.orchestra.run.smithi148.stdout:Name : grub2-tools 2024-09-18T11:21:05.168 INFO:teuthology.orchestra.run.smithi148.stdout:Epoch : 1 2024-09-18T11:21:05.168 INFO:teuthology.orchestra.run.smithi148.stdout:Version : 2.06 2024-09-18T11:21:05.168 INFO:teuthology.orchestra.run.smithi148.stdout:Release : 82.el9 2024-09-18T11:21:05.168 INFO:teuthology.orchestra.run.smithi148.stdout:Architecture: x86_64 2024-09-18T11:21:05.168 INFO:teuthology.orchestra.run.smithi148.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-18T11:21:05.169 INFO:teuthology.orchestra.run.smithi148.stdout:Group : Unspecified 2024-09-18T11:21:05.169 INFO:teuthology.orchestra.run.smithi148.stdout:Size : 8274670 2024-09-18T11:21:05.169 INFO:teuthology.orchestra.run.smithi148.stdout:License : GPLv3+ 2024-09-18T11:21:05.169 INFO:teuthology.orchestra.run.smithi148.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-18T11:21:05.169 INFO:teuthology.orchestra.run.smithi148.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-18T11:21:05.170 INFO:teuthology.orchestra.run.smithi148.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-18T11:21:05.170 INFO:teuthology.orchestra.run.smithi148.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-18T11:21:05.170 INFO:teuthology.orchestra.run.smithi148.stdout:Packager : builder@centos.org 2024-09-18T11:21:05.170 INFO:teuthology.orchestra.run.smithi148.stdout:Vendor : CentOS 2024-09-18T11:21:05.170 INFO:teuthology.orchestra.run.smithi148.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-18T11:21:05.171 INFO:teuthology.orchestra.run.smithi148.stdout:Summary : Support tools for GRUB. 2024-09-18T11:21:05.171 INFO:teuthology.orchestra.run.smithi148.stdout:Description : 2024-09-18T11:21:05.171 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:21:05.171 INFO:teuthology.orchestra.run.smithi148.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-18T11:21:05.171 INFO:teuthology.orchestra.run.smithi148.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-18T11:21:05.171 INFO:teuthology.orchestra.run.smithi148.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-18T11:21:05.171 INFO:teuthology.orchestra.run.smithi148.stdout:hardware devices. 2024-09-18T11:21:05.171 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:21:05.171 INFO:teuthology.orchestra.run.smithi148.stdout:This subpackage provides tools for support of all platforms. 2024-09-18T11:21:05.173 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-18T11:21:05.173 INFO:teuthology.task.kernel:Updating grub on smithi148 to boot 5.14.0-508.el9.x86_64 2024-09-18T11:21:05.173 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-18T11:21:05.815 INFO:teuthology.orchestra.run.smithi148.stderr:Generating grub configuration file ... 2024-09-18T11:21:07.136 INFO:teuthology.orchestra.run.smithi148.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-18T11:21:07.175 INFO:teuthology.orchestra.run.smithi148.stderr:done 2024-09-18T11:21:07.177 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2024-09-18T11:21:07.192 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.5TTd2xV7V3 2024-09-18T11:21:07.193 DEBUG:teuthology.orchestra.run.smithi148:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.5TTd2xV7V3 2024-09-18T11:21:07.260 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.5TTd2xV7V3 2024-09-18T11:21:07.396 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.5TTd2xV7V3 is 6KB 2024-09-18T11:21:07.448 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.5TTd2xV7V3 2024-09-18T11:21:07.464 DEBUG:teuthology.orchestra.run.smithi148:> sudo /bin/ls /boot/loader/entries || true 2024-09-18T11:21:07.536 INFO:teuthology.orchestra.run.smithi148.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-18T11:21:07.536 INFO:teuthology.orchestra.run.smithi148.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-18T11:21:07.536 INFO:teuthology.orchestra.run.smithi148.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-18T11:21:07.536 INFO:teuthology.orchestra.run.smithi148.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-18T11:21:07.538 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-18T11:21:07.692 DEBUG:teuthology.orchestra.run.smithi148:> sudo shutdown -r now 2024-09-18T11:21:10.747 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T11:21:10.792 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-18T11:21:11.201 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T11:21:11.201 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-18T11:21:11.202 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T11:21:11.202 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T11:21:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T11:21:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:21:12.083 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-09-18T11:21:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-18T11:21:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-18T11:21:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-18T11:21:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-18T11:21:12.084 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:21:12.084 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:21:12.481 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T11:21:12.567 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T11:21:12.568 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T11:21:12.568 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-18T11:21:12.568 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T11:21:12.568 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T11:21:12.568 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-18T11:21:12.568 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi119.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T11:21:12.568 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T11:21:12.627 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T11:21:12.627 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T11:21:12.628 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -qi grub2-tools 2024-09-18T11:21:12.659 INFO:teuthology.orchestra.run.smithi119.stdout:Name : grub2-tools 2024-09-18T11:21:12.660 INFO:teuthology.orchestra.run.smithi119.stdout:Epoch : 1 2024-09-18T11:21:12.660 INFO:teuthology.orchestra.run.smithi119.stdout:Version : 2.06 2024-09-18T11:21:12.660 INFO:teuthology.orchestra.run.smithi119.stdout:Release : 82.el9 2024-09-18T11:21:12.660 INFO:teuthology.orchestra.run.smithi119.stdout:Architecture: x86_64 2024-09-18T11:21:12.660 INFO:teuthology.orchestra.run.smithi119.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-18T11:21:12.660 INFO:teuthology.orchestra.run.smithi119.stdout:Group : Unspecified 2024-09-18T11:21:12.660 INFO:teuthology.orchestra.run.smithi119.stdout:Size : 8274670 2024-09-18T11:21:12.661 INFO:teuthology.orchestra.run.smithi119.stdout:License : GPLv3+ 2024-09-18T11:21:12.661 INFO:teuthology.orchestra.run.smithi119.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-18T11:21:12.661 INFO:teuthology.orchestra.run.smithi119.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-18T11:21:12.661 INFO:teuthology.orchestra.run.smithi119.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-18T11:21:12.661 INFO:teuthology.orchestra.run.smithi119.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-18T11:21:12.661 INFO:teuthology.orchestra.run.smithi119.stdout:Packager : builder@centos.org 2024-09-18T11:21:12.661 INFO:teuthology.orchestra.run.smithi119.stdout:Vendor : CentOS 2024-09-18T11:21:12.661 INFO:teuthology.orchestra.run.smithi119.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-18T11:21:12.661 INFO:teuthology.orchestra.run.smithi119.stdout:Summary : Support tools for GRUB. 2024-09-18T11:21:12.661 INFO:teuthology.orchestra.run.smithi119.stdout:Description : 2024-09-18T11:21:12.661 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:21:12.661 INFO:teuthology.orchestra.run.smithi119.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-18T11:21:12.661 INFO:teuthology.orchestra.run.smithi119.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-18T11:21:12.662 INFO:teuthology.orchestra.run.smithi119.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-18T11:21:12.662 INFO:teuthology.orchestra.run.smithi119.stdout:hardware devices. 2024-09-18T11:21:12.662 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:21:12.662 INFO:teuthology.orchestra.run.smithi119.stdout:This subpackage provides tools for support of all platforms. 2024-09-18T11:21:12.663 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-18T11:21:12.663 INFO:teuthology.task.kernel:Updating grub on smithi119 to boot 5.14.0-508.el9.x86_64 2024-09-18T11:21:12.663 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-18T11:21:13.349 INFO:teuthology.orchestra.run.smithi119.stderr:Generating grub configuration file ... 2024-09-18T11:21:14.690 INFO:teuthology.orchestra.run.smithi119.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-18T11:21:14.727 INFO:teuthology.orchestra.run.smithi119.stderr:done 2024-09-18T11:21:14.729 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2024-09-18T11:21:14.745 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.dFB1ObQbCH 2024-09-18T11:21:14.746 DEBUG:teuthology.orchestra.run.smithi119:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.dFB1ObQbCH 2024-09-18T11:21:14.812 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.dFB1ObQbCH 2024-09-18T11:21:14.951 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.dFB1ObQbCH is 6KB 2024-09-18T11:21:14.998 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.dFB1ObQbCH 2024-09-18T11:21:15.014 DEBUG:teuthology.orchestra.run.smithi119:> sudo /bin/ls /boot/loader/entries || true 2024-09-18T11:21:15.076 INFO:teuthology.orchestra.run.smithi119.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-18T11:21:15.076 INFO:teuthology.orchestra.run.smithi119.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-18T11:21:15.076 INFO:teuthology.orchestra.run.smithi119.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-18T11:21:15.076 INFO:teuthology.orchestra.run.smithi119.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-18T11:21:15.078 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-18T11:21:15.225 DEBUG:teuthology.orchestra.run.smithi119:> sudo shutdown -r now 2024-09-18T11:21:37.717 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-18T11:21:37.718 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-18T11:21:37.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T11:21:45.235 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-18T11:21:45.235 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-18T11:21:45.236 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T11:21:56.285 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-09-18T11:22:03.709 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-09-18T11:22:05.287 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-18T11:22:05.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T11:22:08.349 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-09-18T11:22:12.713 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-18T11:22:12.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T11:22:15.773 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-09-18T11:22:20.353 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-18T11:22:20.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T11:22:27.780 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-18T11:22:27.780 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T11:22:38.781 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-09-18T11:22:53.797 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-18T11:22:53.797 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T11:22:54.216 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-09-18T11:22:54.629 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-18T11:22:54.629 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-18T11:22:54.629 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-09-18T11:22:54.644 INFO:teuthology.orchestra.run.smithi148.stdout:5.14.0-508.el9.x86_64 2024-09-18T11:22:54.644 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-18T11:22:54.644 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T11:22:54.645 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-18T11:22:55.646 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-18T11:22:55.646 DEBUG:teuthology.orchestra.run.smithi148:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T11:22:55.737 INFO:teuthology.orchestra.run.smithi148.stdout:ttyS1 2024-09-18T11:22:55.759 DEBUG:teuthology.parallel:result is None 2024-09-18T11:22:59.600 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-09-18T11:23:00.138 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-18T11:23:00.138 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-18T11:23:00.138 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2024-09-18T11:23:00.178 INFO:teuthology.orchestra.run.smithi119.stdout:5.14.0-508.el9.x86_64 2024-09-18T11:23:00.179 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-18T11:23:00.179 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T11:23:00.179 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-18T11:23:01.179 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-18T11:23:01.180 DEBUG:teuthology.orchestra.run.smithi119:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T11:23:01.258 INFO:teuthology.orchestra.run.smithi119.stdout:ttyS1 2024-09-18T11:23:01.470 DEBUG:teuthology.parallel:result is None 2024-09-18T11:23:01.470 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-18T11:23:01.478 INFO:teuthology.task.internal:Creating test directory... 2024-09-18T11:23:01.479 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T11:23:01.481 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T11:23:01.498 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-18T11:23:01.505 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-18T11:23:01.515 INFO:teuthology.task.internal:Creating archive directory... 2024-09-18T11:23:01.515 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T11:23:01.536 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T11:23:01.572 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-18T11:23:01.578 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-18T11:23:01.579 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T11:23:01.600 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T11:23:01.624 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T11:23:01.634 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T11:23:01.634 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T11:23:01.644 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T11:23:01.645 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-18T11:23:01.652 INFO:teuthology.task.internal:Configuring sudo... 2024-09-18T11:23:01.653 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T11:23:01.678 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T11:23:01.710 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-18T11:23:01.718 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-18T11:23:01.719 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T11:23:01.743 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T11:23:01.765 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T11:23:01.840 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T11:23:01.890 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T11:23:01.972 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T11:23:02.011 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:23:02.011 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T11:23:02.075 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T11:23:02.119 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T11:23:02.169 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T11:23:02.251 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T11:23:02.290 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-18T11:23:02.291 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T11:23:02.356 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2024-09-18T11:23:02.358 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2024-09-18T11:23:02.398 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T11:23:02.448 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T11:23:02.952 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-18T11:23:02.959 INFO:teuthology.task.internal:Starting timer... 2024-09-18T11:23:02.959 INFO:teuthology.run_tasks:Running task pcp... 2024-09-18T11:23:02.972 INFO:teuthology.run_tasks:Running task selinux... 2024-09-18T11:23:02.981 DEBUG:teuthology.orchestra.run.smithi119:> sudo service auditd rotate 2024-09-18T11:23:03.057 INFO:teuthology.orchestra.run.smithi119.stdout:Rotating logs: 2024-09-18T11:23:03.059 DEBUG:teuthology.orchestra.run.smithi148:> sudo service auditd rotate 2024-09-18T11:23:03.098 INFO:teuthology.orchestra.run.smithi148.stdout:Rotating logs: 2024-09-18T11:23:03.100 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-18T11:23:03.100 DEBUG:teuthology.orchestra.run.smithi119:> /usr/sbin/getenforce 2024-09-18T11:23:03.122 INFO:teuthology.orchestra.run.smithi119.stdout:Permissive 2024-09-18T11:23:03.122 DEBUG:teuthology.orchestra.run.smithi148:> /usr/sbin/getenforce 2024-09-18T11:23:03.173 INFO:teuthology.orchestra.run.smithi148.stdout:Permissive 2024-09-18T11:23:03.173 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi119.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi148.front.sepia.ceph.com': 'permissive'} 2024-09-18T11:23:03.174 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-18T11:23:03.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T11:23:03.200 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-18T11:23:03.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T11:23:03.239 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-18T11:23:03.239 DEBUG:teuthology.orchestra.run.smithi119:> sudo /usr/sbin/setenforce permissive 2024-09-18T11:23:03.270 DEBUG:teuthology.orchestra.run.smithi148:> sudo /usr/sbin/setenforce permissive 2024-09-18T11:23:03.312 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-18T11:23:03.321 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-18T11:23:03.334 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-18T11:23:03.335 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi119.front.sepia.ceph.com,smithi148.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-18T11:26:49.963 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi119.front.sepia.ceph.com'), Remote(name='ubuntu@smithi148.front.sepia.ceph.com')] 2024-09-18T11:26:49.964 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-18T11:26:49.965 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T11:26:50.049 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-09-18T11:26:50.124 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-18T11:26:50.124 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-18T11:26:50.125 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T11:26:50.202 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-09-18T11:26:50.281 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-18T11:26:50.282 INFO:teuthology.run_tasks:Running task clock... 2024-09-18T11:26:50.292 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-18T11:26:50.293 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T11:26:50.293 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T11:26:50.296 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T11:26:50.296 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T11:26:50.324 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-18T11:26:50.339 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-18T11:26:50.353 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-18T11:26:50.364 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: ntpd: command not found 2024-09-18T11:26:50.367 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-18T11:26:50.375 INFO:teuthology.orchestra.run.smithi119.stdout:506 Cannot talk to daemon 2024-09-18T11:26:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-18T11:26:50.393 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: ntpd: command not found 2024-09-18T11:26:50.403 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-18T11:26:50.405 INFO:teuthology.orchestra.run.smithi148.stdout:506 Cannot talk to daemon 2024-09-18T11:26:50.419 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-18T11:26:50.433 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-18T11:26:50.456 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-09-18T11:26:50.460 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T11:26:50.460 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-09-18T11:26:50.460 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T11:26:50.460 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T11:26:50.460 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T11:26:50.460 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T11:26:50.490 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-09-18T11:26:50.497 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T11:26:50.497 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-09-18T11:26:50.497 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T11:26:50.497 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T11:26:50.497 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T11:26:50.497 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T11:26:50.498 INFO:teuthology.run_tasks:Running task install... 2024-09-18T11:26:50.507 DEBUG:teuthology.task.install:project ceph 2024-09-18T11:26:50.507 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}} 2024-09-18T11:26:50.507 DEBUG:teuthology.task.install:config {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'} 2024-09-18T11:26:50.507 INFO:teuthology.task.install:Using flavor: default 2024-09-18T11:26:50.513 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-18T11:26:50.513 INFO:teuthology.task.install:extra packages: [] 2024-09-18T11:26:50.513 DEBUG:teuthology.orchestra.run.smithi119:> sudo dnf -y install 'dnf-command(copr)' 2024-09-18T11:26:50.514 DEBUG:teuthology.orchestra.run.smithi148:> sudo dnf -y install 'dnf-command(copr)' 2024-09-18T11:26:51.088 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:00:48 ago on Wed 18 Sep 2024 11:26:02 AM UTC. 2024-09-18T11:26:51.121 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:47 ago on Wed 18 Sep 2024 11:26:03 AM UTC. 2024-09-18T11:26:51.173 INFO:teuthology.orchestra.run.smithi119.stdout:Package dnf-plugins-core-4.3.0-16.el9.noarch is already installed. 2024-09-18T11:26:51.208 INFO:teuthology.orchestra.run.smithi148.stdout:Package dnf-plugins-core-4.3.0-16.el9.noarch is already installed. 2024-09-18T11:26:51.215 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:26:51.216 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-09-18T11:26:51.216 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:26:51.250 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:26:51.251 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-18T11:26:51.252 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:26:51.283 DEBUG:teuthology.orchestra.run.smithi119:> sudo dnf -y copr enable ceph/el9 2024-09-18T11:26:51.318 DEBUG:teuthology.orchestra.run.smithi148:> sudo dnf -y copr enable ceph/el9 2024-09-18T11:26:51.461 INFO:teuthology.orchestra.run.smithi119.stderr:Enabling a Copr repository. Please note that this repository is not part 2024-09-18T11:26:51.461 INFO:teuthology.orchestra.run.smithi119.stderr:of the main distribution, and quality may vary. 2024-09-18T11:26:51.461 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-18T11:26:51.461 INFO:teuthology.orchestra.run.smithi119.stderr:The Fedora Project does not exercise any power over the contents of 2024-09-18T11:26:51.461 INFO:teuthology.orchestra.run.smithi119.stderr:this repository beyond the rules outlined in the Copr FAQ at 2024-09-18T11:26:51.461 INFO:teuthology.orchestra.run.smithi119.stderr:, 2024-09-18T11:26:51.461 INFO:teuthology.orchestra.run.smithi119.stderr:and packages are not held to any quality or security level. 2024-09-18T11:26:51.462 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-18T11:26:51.462 INFO:teuthology.orchestra.run.smithi119.stderr:Please do not file bug reports about these packages in Fedora 2024-09-18T11:26:51.462 INFO:teuthology.orchestra.run.smithi119.stderr:Bugzilla. In case of problems, contact the owner of this repository. 2024-09-18T11:26:51.489 INFO:teuthology.orchestra.run.smithi148.stderr:Enabling a Copr repository. Please note that this repository is not part 2024-09-18T11:26:51.489 INFO:teuthology.orchestra.run.smithi148.stderr:of the main distribution, and quality may vary. 2024-09-18T11:26:51.490 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-09-18T11:26:51.490 INFO:teuthology.orchestra.run.smithi148.stderr:The Fedora Project does not exercise any power over the contents of 2024-09-18T11:26:51.490 INFO:teuthology.orchestra.run.smithi148.stderr:this repository beyond the rules outlined in the Copr FAQ at 2024-09-18T11:26:51.490 INFO:teuthology.orchestra.run.smithi148.stderr:, 2024-09-18T11:26:51.490 INFO:teuthology.orchestra.run.smithi148.stderr:and packages are not held to any quality or security level. 2024-09-18T11:26:51.490 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-09-18T11:26:51.490 INFO:teuthology.orchestra.run.smithi148.stderr:Please do not file bug reports about these packages in Fedora 2024-09-18T11:26:51.490 INFO:teuthology.orchestra.run.smithi148.stderr:Bugzilla. In case of problems, contact the owner of this repository. 2024-09-18T11:26:51.514 INFO:teuthology.orchestra.run.smithi119.stdout:Repository successfully enabled. 2024-09-18T11:26:51.541 INFO:teuthology.orchestra.run.smithi148.stdout:Repository successfully enabled. 2024-09-18T11:26:51.550 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153', 'tag': None, 'wait_for_package': False} 2024-09-18T11:26:51.551 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T11:26:51.586 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153', 'tag': None, 'wait_for_package': False} 2024-09-18T11:26:51.586 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T11:26:51.694 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/centos/9/flavors/default/ 2024-09-18T11:26:51.694 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1645.g7061ea6c 2024-09-18T11:26:51.733 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/centos/9/flavors/default/ 2024-09-18T11:26:51.733 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1645.g7061ea6c 2024-09-18T11:26:51.803 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-18T11:26:51.803 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:26:51.804 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-18T11:26:51.829 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-18T11:26:51.830 DEBUG:teuthology.orchestra.run.smithi119:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7061ea6ca50908d104e9f3979dc615e118d9e153/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-18T11:26:51.839 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-18T11:26:51.839 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-18T11:26:51.839 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-18T11:26:51.865 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-18T11:26:51.865 DEBUG:teuthology.orchestra.run.smithi148:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7061ea6ca50908d104e9f3979dc615e118d9e153/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-18T11:26:51.894 DEBUG:teuthology.orchestra.run.smithi119:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-18T11:26:51.933 DEBUG:teuthology.orchestra.run.smithi148:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-18T11:26:51.970 DEBUG:teuthology.orchestra.run.smithi119:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-18T11:26:52.010 DEBUG:teuthology.orchestra.run.smithi148:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-18T11:26:52.034 INFO:teuthology.orchestra.run.smithi119.stdout:check_obsoletes = 1 2024-09-18T11:26:52.035 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum clean all 2024-09-18T11:26:52.074 INFO:teuthology.orchestra.run.smithi148.stdout:check_obsoletes = 1 2024-09-18T11:26:52.076 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum clean all 2024-09-18T11:26:52.303 INFO:teuthology.orchestra.run.smithi119.stdout:45 files removed 2024-09-18T11:26:52.304 INFO:teuthology.orchestra.run.smithi148.stdout:45 files removed 2024-09-18T11:26:52.331 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-radosgw 2024-09-18T11:26:52.332 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-radosgw 2024-09-18T11:26:52.890 INFO:teuthology.orchestra.run.smithi119.stdout:ceph packages for x86_64 280 kB/s | 77 kB 00:00 2024-09-18T11:26:52.894 INFO:teuthology.orchestra.run.smithi148.stdout:ceph packages for x86_64 281 kB/s | 77 kB 00:00 2024-09-18T11:26:53.178 INFO:teuthology.orchestra.run.smithi148.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-18T11:26:53.216 INFO:teuthology.orchestra.run.smithi119.stdout:ceph noarch packages 38 kB/s | 11 kB 00:00 2024-09-18T11:26:53.454 INFO:teuthology.orchestra.run.smithi148.stdout:ceph source packages 7.5 kB/s | 1.8 kB 00:00 2024-09-18T11:26:53.492 INFO:teuthology.orchestra.run.smithi119.stdout:ceph source packages 7.5 kB/s | 1.8 kB 00:00 2024-09-18T11:26:53.621 INFO:teuthology.orchestra.run.smithi148.stdout:Copr repo for el9 owned by ceph 58 kB/s | 7.6 kB 00:00 2024-09-18T11:26:53.659 INFO:teuthology.orchestra.run.smithi119.stdout:Copr repo for el9 owned by ceph 58 kB/s | 7.6 kB 00:00 2024-09-18T11:26:54.292 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.3 MB 00:00 2024-09-18T11:26:55.596 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.3 MB 00:01 2024-09-18T11:26:57.280 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 33 MB/s | 20 MB 00:00 2024-09-18T11:26:57.502 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 9.6 MB/s | 20 MB 00:02 2024-09-18T11:27:01.761 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-18T11:27:02.598 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.5 MB 00:01 2024-09-18T11:27:03.170 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 105 kB/s | 19 kB 00:00 2024-09-18T11:27:04.373 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-18T11:27:04.424 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-18T11:27:05.402 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-18T11:27:09.825 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-09-18T11:27:10.833 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-09-18T11:27:12.162 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:27:12.173 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:27:12.173 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-09-18T11:27:12.174 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:27:12.174 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-09-18T11:27:12.174 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 14 M 2024-09-18T11:27:12.174 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2024-09-18T11:27:12.174 INFO:teuthology.orchestra.run.smithi119.stdout: librados2 x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 3.1 M 2024-09-18T11:27:12.174 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1 x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 2.9 M 2024-09-18T11:27:12.174 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-09-18T11:27:12.174 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 5.0 M 2024-09-18T11:27:12.174 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 22 M 2024-09-18T11:27:12.174 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 25 k 2024-09-18T11:27:12.174 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-18T11:27:12.175 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2 x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 637 k 2024-09-18T11:27:12.175 INFO:teuthology.orchestra.run.smithi119.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-18T11:27:12.175 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-18T11:27:12.175 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-18T11:27:12.175 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1 x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 458 k 2024-09-18T11:27:12.175 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-18T11:27:12.175 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2 x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 4.9 M 2024-09-18T11:27:12.175 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-18T11:27:12.175 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-18T11:27:12.175 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 45 k 2024-09-18T11:27:12.175 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 120 k 2024-09-18T11:27:12.175 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 156 k 2024-09-18T11:27:12.175 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-18T11:27:12.175 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 320 k 2024-09-18T11:27:12.176 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 296 k 2024-09-18T11:27:12.176 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 99 k 2024-09-18T11:27:12.176 INFO:teuthology.orchestra.run.smithi119.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-18T11:27:12.176 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:27:12.176 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-09-18T11:27:12.176 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:27:12.176 INFO:teuthology.orchestra.run.smithi119.stdout:Install 22 Packages 2024-09-18T11:27:12.176 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 2 Packages 2024-09-18T11:27:12.176 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:27:12.177 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 58 M 2024-09-18T11:27:12.177 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-09-18T11:27:12.831 INFO:teuthology.orchestra.run.smithi119.stdout:(1/24): ceph-base-17.2.7-1645.g7061ea6c.el9.x86 8.2 MB/s | 5.0 MB 00:00 2024-09-18T11:27:12.889 INFO:teuthology.orchestra.run.smithi119.stdout:(2/24): ceph-selinux-17.2.7-1645.g7061ea6c.el9. 432 kB/s | 25 kB 00:00 2024-09-18T11:27:12.998 INFO:teuthology.orchestra.run.smithi119.stdout:(3/24): libcephfs2-17.2.7-1645.g7061ea6c.el9.x8 5.7 MB/s | 637 kB 00:00 2024-09-18T11:27:13.081 INFO:teuthology.orchestra.run.smithi119.stdout:(4/24): libradosstriper1-17.2.7-1645.g7061ea6c. 5.4 MB/s | 458 kB 00:00 2024-09-18T11:27:13.135 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:27:13.142 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:27:13.142 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-18T11:27:13.142 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:27:13.142 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-18T11:27:13.142 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 14 M 2024-09-18T11:27:13.143 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2024-09-18T11:27:13.143 INFO:teuthology.orchestra.run.smithi148.stdout: librados2 x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 3.1 M 2024-09-18T11:27:13.143 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1 x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 2.9 M 2024-09-18T11:27:13.143 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-18T11:27:13.143 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 5.0 M 2024-09-18T11:27:13.143 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 22 M 2024-09-18T11:27:13.143 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 25 k 2024-09-18T11:27:13.143 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-18T11:27:13.143 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2 x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 637 k 2024-09-18T11:27:13.143 INFO:teuthology.orchestra.run.smithi148.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-18T11:27:13.143 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-18T11:27:13.143 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-18T11:27:13.143 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1 x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 458 k 2024-09-18T11:27:13.144 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-18T11:27:13.144 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2 x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 4.9 M 2024-09-18T11:27:13.144 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-18T11:27:13.144 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-18T11:27:13.144 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 45 k 2024-09-18T11:27:13.144 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 120 k 2024-09-18T11:27:13.144 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 156 k 2024-09-18T11:27:13.144 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-18T11:27:13.144 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 320 k 2024-09-18T11:27:13.144 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 296 k 2024-09-18T11:27:13.144 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 99 k 2024-09-18T11:27:13.144 INFO:teuthology.orchestra.run.smithi148.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-18T11:27:13.144 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:27:13.145 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-18T11:27:13.145 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:27:13.145 INFO:teuthology.orchestra.run.smithi148.stdout:Install 22 Packages 2024-09-18T11:27:13.145 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 2 Packages 2024-09-18T11:27:13.145 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:27:13.145 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 58 M 2024-09-18T11:27:13.145 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-18T11:27:13.232 INFO:teuthology.orchestra.run.smithi119.stdout:(5/24): ceph-radosgw-17.2.7-1645.g7061ea6c.el9. 14 MB/s | 14 MB 00:01 2024-09-18T11:27:13.457 INFO:teuthology.orchestra.run.smithi119.stdout:(6/24): ceph-common-17.2.7-1645.g7061ea6c.el9.x 18 MB/s | 22 MB 00:01 2024-09-18T11:27:13.482 INFO:teuthology.orchestra.run.smithi119.stdout:(7/24): python3-ceph-argparse-17.2.7-1645.g7061 180 kB/s | 45 kB 00:00 2024-09-18T11:27:13.516 INFO:teuthology.orchestra.run.smithi119.stdout:(8/24): python3-ceph-common-17.2.7-1645.g7061ea 2.0 MB/s | 120 kB 00:00 2024-09-18T11:27:13.583 INFO:teuthology.orchestra.run.smithi119.stdout:(9/24): librgw2-17.2.7-1645.g7061ea6c.el9.x86_6 9.8 MB/s | 4.9 MB 00:00 2024-09-18T11:27:13.608 INFO:teuthology.orchestra.run.smithi119.stdout:(10/24): python3-cephfs-17.2.7-1645.g7061ea6c.e 1.2 MB/s | 156 kB 00:00 2024-09-18T11:27:13.700 INFO:teuthology.orchestra.run.smithi119.stdout:(11/24): python3-rgw-17.2.7-1645.g7061ea6c.el9. 1.1 MB/s | 99 kB 00:00 2024-09-18T11:27:13.793 INFO:teuthology.orchestra.run.smithi148.stdout:(1/24): ceph-base-17.2.7-1645.g7061ea6c.el9.x86 8.9 MB/s | 5.0 MB 00:00 2024-09-18T11:27:13.843 INFO:teuthology.orchestra.run.smithi148.stdout:(2/24): ceph-selinux-17.2.7-1645.g7061ea6c.el9. 504 kB/s | 25 kB 00:00 2024-09-18T11:27:13.935 INFO:teuthology.orchestra.run.smithi148.stdout:(3/24): libcephfs2-17.2.7-1645.g7061ea6c.el9.x8 6.8 MB/s | 637 kB 00:00 2024-09-18T11:27:13.975 INFO:teuthology.orchestra.run.smithi119.stdout:(12/24): python3-rados-17.2.7-1645.g7061ea6c.el 697 kB/s | 320 kB 00:00 2024-09-18T11:27:14.035 INFO:teuthology.orchestra.run.smithi148.stdout:(4/24): libradosstriper1-17.2.7-1645.g7061ea6c. 4.5 MB/s | 458 kB 00:00 2024-09-18T11:27:14.250 INFO:teuthology.orchestra.run.smithi119.stdout:(13/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 291 kB/s | 160 kB 00:00 2024-09-18T11:27:14.284 INFO:teuthology.orchestra.run.smithi119.stdout:(14/24): python3-rbd-17.2.7-1645.g7061ea6c.el9. 422 kB/s | 296 kB 00:00 2024-09-18T11:27:14.343 INFO:teuthology.orchestra.run.smithi119.stdout:(15/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 123 kB/s | 45 kB 00:00 2024-09-18T11:27:14.411 INFO:teuthology.orchestra.run.smithi148.stdout:(5/24): ceph-common-17.2.7-1645.g7061ea6c.el9.x 19 MB/s | 22 MB 00:01 2024-09-18T11:27:14.543 INFO:teuthology.orchestra.run.smithi119.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 211 kB/s | 42 kB 00:00 2024-09-18T11:27:14.569 INFO:teuthology.orchestra.run.smithi148.stdout:(6/24): ceph-radosgw-17.2.7-1645.g7061ea6c.el9. 11 MB/s | 14 MB 00:01 2024-09-18T11:27:14.593 INFO:teuthology.orchestra.run.smithi119.stdout:(17/24): librdkafka-1.6.1-102.el9.x86_64.rpm 1.9 MB/s | 662 kB 00:00 2024-09-18T11:27:14.594 INFO:teuthology.orchestra.run.smithi148.stdout:(7/24): python3-ceph-argparse-17.2.7-1645.g7061 246 kB/s | 45 kB 00:00 2024-09-18T11:27:14.620 INFO:teuthology.orchestra.run.smithi148.stdout:(8/24): python3-ceph-common-17.2.7-1645.g7061ea 2.3 MB/s | 120 kB 00:00 2024-09-18T11:27:14.627 INFO:teuthology.orchestra.run.smithi119.stdout:(18/24): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.6 MB/s | 308 kB 00:00 2024-09-18T11:27:14.645 INFO:teuthology.orchestra.run.smithi148.stdout:(9/24): python3-cephfs-17.2.7-1645.g7061ea6c.el 3.0 MB/s | 156 kB 00:00 2024-09-18T11:27:14.652 INFO:teuthology.orchestra.run.smithi119.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 2.8 MB/s | 67 kB 00:00 2024-09-18T11:27:14.670 INFO:teuthology.orchestra.run.smithi148.stdout:(10/24): python3-rados-17.2.7-1645.g7061ea6c.el 6.2 MB/s | 320 kB 00:00 2024-09-18T11:27:14.677 INFO:teuthology.orchestra.run.smithi119.stdout:(20/24): liboath-2.6.7-2.el9.x86_64.rpm 582 kB/s | 49 kB 00:00 2024-09-18T11:27:14.736 INFO:teuthology.orchestra.run.smithi119.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-18T11:27:14.737 INFO:teuthology.orchestra.run.smithi148.stdout:(11/24): librgw2-17.2.7-1645.g7061ea6c.el9.x86_ 7.0 MB/s | 4.9 MB 00:00 2024-09-18T11:27:14.762 INFO:teuthology.orchestra.run.smithi148.stdout:(12/24): python3-rbd-17.2.7-1645.g7061ea6c.el9. 2.5 MB/s | 296 kB 00:00 2024-09-18T11:27:14.787 INFO:teuthology.orchestra.run.smithi148.stdout:(13/24): python3-rgw-17.2.7-1645.g7061ea6c.el9. 850 kB/s | 99 kB 00:00 2024-09-18T11:27:14.871 INFO:teuthology.orchestra.run.smithi148.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-09-18T11:27:14.903 INFO:teuthology.orchestra.run.smithi119.stdout:(22/24): librados2-17.2.7-1645.g7061ea6c.el9.x8 14 MB/s | 3.1 MB 00:00 2024-09-18T11:27:14.921 INFO:teuthology.orchestra.run.smithi148.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 871 kB/s | 160 kB 00:00 2024-09-18T11:27:14.928 INFO:teuthology.orchestra.run.smithi119.stdout:(23/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 454 kB/s | 292 kB 00:00 2024-09-18T11:27:14.963 INFO:teuthology.orchestra.run.smithi148.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-18T11:27:14.988 INFO:teuthology.orchestra.run.smithi148.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-18T11:27:15.039 INFO:teuthology.orchestra.run.smithi148.stdout:(18/24): gperftools-libs-2.9.1-3.el9.x86_64.rpm 4.0 MB/s | 308 kB 00:00 2024-09-18T11:27:15.054 INFO:teuthology.orchestra.run.smithi119.stdout:(24/24): librbd1-17.2.7-1645.g7061ea6c.el9.x86_ 9.3 MB/s | 2.9 MB 00:00 2024-09-18T11:27:15.058 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:27:15.058 INFO:teuthology.orchestra.run.smithi119.stdout:Total 20 MB/s | 58 MB 00:02 2024-09-18T11:27:15.065 INFO:teuthology.orchestra.run.smithi148.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 2.8 MB/s | 67 kB 00:00 2024-09-18T11:27:15.081 INFO:teuthology.orchestra.run.smithi148.stdout:(20/24): liboath-2.6.7-2.el9.x86_64.rpm 528 kB/s | 49 kB 00:00 2024-09-18T11:27:15.124 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-09-18T11:27:15.148 INFO:teuthology.orchestra.run.smithi148.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-18T11:27:15.157 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-09-18T11:27:15.157 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-09-18T11:27:15.173 INFO:teuthology.orchestra.run.smithi148.stdout:(22/24): librdkafka-1.6.1-102.el9.x86_64.rpm 1.7 MB/s | 662 kB 00:00 2024-09-18T11:27:15.265 INFO:teuthology.orchestra.run.smithi148.stdout:(23/24): librados2-17.2.7-1645.g7061ea6c.el9.x8 17 MB/s | 3.1 MB 00:00 2024-09-18T11:27:15.349 INFO:teuthology.orchestra.run.smithi148.stdout:(24/24): librbd1-17.2.7-1645.g7061ea6c.el9.x86_ 15 MB/s | 2.9 MB 00:00 2024-09-18T11:27:15.354 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:27:15.354 INFO:teuthology.orchestra.run.smithi148.stdout:Total 26 MB/s | 58 MB 00:02 2024-09-18T11:27:15.417 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-18T11:27:15.451 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-18T11:27:15.451 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-18T11:27:15.812 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-09-18T11:27:15.813 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-09-18T11:27:16.103 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-18T11:27:16.103 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-18T11:27:17.045 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-09-18T11:27:17.384 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-18T11:27:17.392 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-18T11:27:17.481 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librados2-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/26 2024-09-18T11:27:17.573 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/26 2024-09-18T11:27:17.659 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rados-2:17.2.7-1645.g7061ea6c.el9.x86_64 3/26 2024-09-18T11:27:17.682 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs2-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/26 2024-09-18T11:27:17.726 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-18T11:27:17.771 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs2-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/26 2024-09-18T11:27:17.824 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-18T11:27:17.836 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librados2-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/26 2024-09-18T11:27:17.853 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-18T11:27:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/26 2024-09-18T11:27:18.007 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rados-2:17.2.7-1645.g7061ea6c.el9.x86_64 3/26 2024-09-18T11:27:18.037 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs2-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/26 2024-09-18T11:27:18.108 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs2-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/26 2024-09-18T11:27:18.162 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-18T11:27:18.201 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-18T11:27:18.291 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-18T11:27:18.318 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librgw2-2:17.2.7-1645.g7061ea6c.el9.x86_64 8/26 2024-09-18T11:27:18.373 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librgw2-2:17.2.7-1645.g7061ea6c.el9.x86_64 8/26 2024-09-18T11:27:18.410 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-18T11:27:18.452 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-argparse-2:17.2.7-1645.g7061ea6c.el9. 10/26 2024-09-18T11:27:18.490 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cephfs-2:17.2.7-1645.g7061ea6c.el9.x86_64 11/26 2024-09-18T11:27:18.562 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 12/26 2024-09-18T11:27:18.588 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libradosstriper1-2:17.2.7-1645.g7061ea6c.el9.x86_6 13/26 2024-09-18T11:27:18.638 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-18T11:27:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1645.g7061ea6c.el9.x86_6 13/26 2024-09-18T11:27:18.665 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librgw2-2:17.2.7-1645.g7061ea6c.el9.x86_64 8/26 2024-09-18T11:27:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-18T11:27:18.728 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librgw2-2:17.2.7-1645.g7061ea6c.el9.x86_64 8/26 2024-09-18T11:27:18.738 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/26 2024-09-18T11:27:18.765 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-18T11:27:18.800 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-18T11:27:18.808 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-argparse-2:17.2.7-1645.g7061ea6c.el9. 10/26 2024-09-18T11:27:18.846 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cephfs-2:17.2.7-1645.g7061ea6c.el9.x86_64 11/26 2024-09-18T11:27:18.924 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 12/26 2024-09-18T11:27:18.951 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libradosstriper1-2:17.2.7-1645.g7061ea6c.el9.x86_6 13/26 2024-09-18T11:27:19.017 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1645.g7061ea6c.el9.x86_6 13/26 2024-09-18T11:27:19.057 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-18T11:27:19.093 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/26 2024-09-18T11:27:19.107 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-18T11:27:19.126 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-18T11:27:19.150 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librbd1-2:17.2.7-1645.g7061ea6c.el9.x86_64 18/26 2024-09-18T11:27:19.231 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:17.2.7-1645.g7061ea6c.el9.x86_64 18/26 2024-09-18T11:27:19.305 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 19/26 2024-09-18T11:27:19.336 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-common-2:17.2.7-1645.g7061ea6c.el9.x8 20/26 2024-09-18T11:27:19.419 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-18T11:27:19.463 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librbd1-2:17.2.7-1645.g7061ea6c.el9.x86_64 18/26 2024-09-18T11:27:19.537 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:17.2.7-1645.g7061ea6c.el9.x86_64 18/26 2024-09-18T11:27:19.611 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 19/26 2024-09-18T11:27:19.641 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-common-2:17.2.7-1645.g7061ea6c.el9.x8 20/26 2024-09-18T11:27:21.725 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 21/26 2024-09-18T11:27:21.765 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 21/26 2024-09-18T11:27:22.081 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 21/26 2024-09-18T11:27:22.121 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 21/26 2024-09-18T11:27:22.292 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 21/26 2024-09-18T11:27:22.327 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-base-2:17.2.7-1645.g7061ea6c.el9.x86_64 22/26 2024-09-18T11:27:22.407 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-base-2:17.2.7-1645.g7061ea6c.el9.x86_64 22/26 2024-09-18T11:27:22.407 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-18T11:27:22.407 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-18T11:27:22.407 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:27:22.436 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-selinux-2:17.2.7-1645.g7061ea6c.el9.x86_64 23/26 2024-09-18T11:27:22.646 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 21/26 2024-09-18T11:27:22.682 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-base-2:17.2.7-1645.g7061ea6c.el9.x86_64 22/26 2024-09-18T11:27:22.776 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-base-2:17.2.7-1645.g7061ea6c.el9.x86_64 22/26 2024-09-18T11:27:22.777 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-18T11:27:22.777 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-18T11:27:22.777 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:27:22.808 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-selinux-2:17.2.7-1645.g7061ea6c.el9.x86_64 23/26 2024-09-18T11:27:32.127 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1645.g7061ea6c.el9.x86_64 23/26 2024-09-18T11:27:32.127 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /sys 2024-09-18T11:27:32.127 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /proc 2024-09-18T11:27:32.127 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /mnt 2024-09-18T11:27:32.127 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /var/tmp 2024-09-18T11:27:32.127 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /home 2024-09-18T11:27:32.127 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /root 2024-09-18T11:27:32.127 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /tmp 2024-09-18T11:27:32.128 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:27:32.628 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1645.g7061ea6c.el9.x86_64 23/26 2024-09-18T11:27:32.628 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /sys 2024-09-18T11:27:32.628 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /proc 2024-09-18T11:27:32.628 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /mnt 2024-09-18T11:27:32.628 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /var/tmp 2024-09-18T11:27:32.628 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /home 2024-09-18T11:27:32.628 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /root 2024-09-18T11:27:32.628 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /tmp 2024-09-18T11:27:32.628 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:27:33.429 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-radosgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 24/26 2024-09-18T11:27:33.490 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 24/26 2024-09-18T11:27:33.490 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T11:27:33.490 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-18T11:27:33.490 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T11:27:33.490 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T11:27:33.490 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:27:33.492 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-18T11:27:33.589 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-18T11:27:33.590 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-18T11:27:33.967 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-radosgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 24/26 2024-09-18T11:27:34.045 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 24/26 2024-09-18T11:27:34.045 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T11:27:34.045 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-18T11:27:34.045 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T11:27:34.046 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T11:27:34.046 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:27:34.047 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-18T11:27:34.137 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-18T11:27:34.137 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-18T11:27:34.982 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-18T11:27:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-base-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/26 2024-09-18T11:27:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/26 2024-09-18T11:27:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-radosgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 3/26 2024-09-18T11:27:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-selinux-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/26 2024-09-18T11:27:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs2-2:17.2.7-1645.g7061ea6c.el9.x86_64 5/26 2024-09-18T11:27:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libradosstriper1-2:17.2.7-1645.g7061ea6c.el9.x86_6 6/26 2024-09-18T11:27:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librgw2-2:17.2.7-1645.g7061ea6c.el9.x86_64 7/26 2024-09-18T11:27:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1645.g7061ea6c.el9. 8/26 2024-09-18T11:27:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-common-2:17.2.7-1645.g7061ea6c.el9.x8 9/26 2024-09-18T11:27:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cephfs-2:17.2.7-1645.g7061ea6c.el9.x86_64 10/26 2024-09-18T11:27:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rados-2:17.2.7-1645.g7061ea6c.el9.x86_64 11/26 2024-09-18T11:27:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 12/26 2024-09-18T11:27:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 13/26 2024-09-18T11:27:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-18T11:27:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-18T11:27:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-18T11:27:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-18T11:27:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-18T11:27:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 19/26 2024-09-18T11:27:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-18T11:27:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-18T11:27:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-18T11:27:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:17.2.7-1645.g7061ea6c.el9.x86_64 23/26 2024-09-18T11:27:34.985 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-18T11:27:34.985 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:17.2.7-1645.g7061ea6c.el9.x86_64 25/26 2024-09-18T11:27:35.360 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-18T11:27:35.360 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-base-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/26 2024-09-18T11:27:35.360 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/26 2024-09-18T11:27:35.360 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-radosgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 3/26 2024-09-18T11:27:35.360 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-selinux-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/26 2024-09-18T11:27:35.361 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs2-2:17.2.7-1645.g7061ea6c.el9.x86_64 5/26 2024-09-18T11:27:35.361 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libradosstriper1-2:17.2.7-1645.g7061ea6c.el9.x86_6 6/26 2024-09-18T11:27:35.361 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librgw2-2:17.2.7-1645.g7061ea6c.el9.x86_64 7/26 2024-09-18T11:27:35.361 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1645.g7061ea6c.el9. 8/26 2024-09-18T11:27:35.361 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-common-2:17.2.7-1645.g7061ea6c.el9.x8 9/26 2024-09-18T11:27:35.361 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cephfs-2:17.2.7-1645.g7061ea6c.el9.x86_64 10/26 2024-09-18T11:27:35.361 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rados-2:17.2.7-1645.g7061ea6c.el9.x86_64 11/26 2024-09-18T11:27:35.361 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 12/26 2024-09-18T11:27:35.361 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 13/26 2024-09-18T11:27:35.361 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-18T11:27:35.361 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-18T11:27:35.361 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-18T11:27:35.362 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-18T11:27:35.362 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-18T11:27:35.362 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 19/26 2024-09-18T11:27:35.362 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-18T11:27:35.362 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-18T11:27:35.362 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-18T11:27:35.362 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:17.2.7-1645.g7061ea6c.el9.x86_64 23/26 2024-09-18T11:27:35.362 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-18T11:27:35.362 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:17.2.7-1645.g7061ea6c.el9.x86_64 25/26 2024-09-18T11:27:35.989 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-18T11:27:35.989 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:27:35.989 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2024-09-18T11:27:35.989 INFO:teuthology.orchestra.run.smithi119.stdout: librados2-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:35.989 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:35.989 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-09-18T11:27:35.989 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:35.989 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:35.989 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:35.989 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:35.989 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-18T11:27:35.990 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:35.990 INFO:teuthology.orchestra.run.smithi119.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-18T11:27:35.990 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-18T11:27:35.990 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-18T11:27:35.990 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:35.990 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-18T11:27:35.990 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:35.990 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-18T11:27:35.990 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-18T11:27:35.990 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:35.990 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:35.990 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:35.990 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-18T11:27:35.991 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:35.991 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:35.991 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:35.991 INFO:teuthology.orchestra.run.smithi119.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-18T11:27:35.991 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:27:35.991 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:27:36.317 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-18T11:27:36.317 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:27:36.318 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2024-09-18T11:27:36.318 INFO:teuthology.orchestra.run.smithi148.stdout: librados2-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:36.318 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:36.318 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-18T11:27:36.318 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:36.318 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:36.318 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:36.318 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:36.318 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-18T11:27:36.318 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:36.318 INFO:teuthology.orchestra.run.smithi148.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-18T11:27:36.318 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-18T11:27:36.318 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-18T11:27:36.319 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:36.319 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-18T11:27:36.319 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:36.319 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-18T11:27:36.319 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-18T11:27:36.319 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:36.319 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:36.319 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:36.319 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-18T11:27:36.319 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:36.319 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:36.319 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:36.319 INFO:teuthology.orchestra.run.smithi148.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-18T11:27:36.320 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:27:36.320 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:27:36.333 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-test 2024-09-18T11:27:36.690 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-test 2024-09-18T11:27:36.915 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:00:27 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:27:37.045 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:27:37.046 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:27:37.046 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-09-18T11:27:37.046 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:27:37.046 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-09-18T11:27:37.046 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 48 M 2024-09-18T11:27:37.046 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-09-18T11:27:37.046 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 163 k 2024-09-18T11:27:37.047 INFO:teuthology.orchestra.run.smithi119.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-18T11:27:37.047 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-18T11:27:37.047 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:27:37.047 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-09-18T11:27:37.047 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:27:37.047 INFO:teuthology.orchestra.run.smithi119.stdout:Install 4 Packages 2024-09-18T11:27:37.047 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:27:37.048 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 48 M 2024-09-18T11:27:37.048 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 193 M 2024-09-18T11:27:37.048 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-09-18T11:27:37.269 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:27 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:27:37.336 INFO:teuthology.orchestra.run.smithi119.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.9 MB/s | 304 kB 00:00 2024-09-18T11:27:37.378 INFO:teuthology.orchestra.run.smithi119.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-18T11:27:37.407 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:27:37.407 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:27:37.408 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-18T11:27:37.408 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:27:37.408 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-18T11:27:37.408 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 48 M 2024-09-18T11:27:37.408 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-18T11:27:37.408 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 163 k 2024-09-18T11:27:37.408 INFO:teuthology.orchestra.run.smithi148.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-18T11:27:37.408 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-18T11:27:37.408 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:27:37.408 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-18T11:27:37.408 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:27:37.408 INFO:teuthology.orchestra.run.smithi148.stdout:Install 4 Packages 2024-09-18T11:27:37.408 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:27:37.409 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 48 M 2024-09-18T11:27:37.409 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 193 M 2024-09-18T11:27:37.409 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-18T11:27:37.419 INFO:teuthology.orchestra.run.smithi119.stdout:(3/4): libcephsqlite-17.2.7-1645.g7061ea6c.el9. 668 kB/s | 163 kB 00:00 2024-09-18T11:27:37.641 INFO:teuthology.orchestra.run.smithi148.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-18T11:27:37.692 INFO:teuthology.orchestra.run.smithi148.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-18T11:27:37.717 INFO:teuthology.orchestra.run.smithi148.stdout:(3/4): libcephsqlite-17.2.7-1645.g7061ea6c.el9. 568 kB/s | 163 kB 00:00 2024-09-18T11:27:38.695 INFO:teuthology.orchestra.run.smithi119.stdout:(4/4): ceph-test-17.2.7-1645.g7061ea6c.el9.x86_ 32 MB/s | 48 MB 00:01 2024-09-18T11:27:38.697 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:27:38.697 INFO:teuthology.orchestra.run.smithi119.stdout:Total 29 MB/s | 48 MB 00:01 2024-09-18T11:27:38.725 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-09-18T11:27:38.745 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-09-18T11:27:38.745 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-09-18T11:27:39.113 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-09-18T11:27:39.114 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-09-18T11:27:39.647 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-09-18T11:27:39.729 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-18T11:27:39.769 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-18T11:27:39.797 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephsqlite-2:17.2.7-1645.g7061ea6c.el9.x86_64 3/4 2024-09-18T11:27:40.041 INFO:teuthology.orchestra.run.smithi148.stdout:(4/4): ceph-test-17.2.7-1645.g7061ea6c.el9.x86_ 18 MB/s | 48 MB 00:02 2024-09-18T11:27:40.043 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:27:40.044 INFO:teuthology.orchestra.run.smithi148.stdout:Total 18 MB/s | 48 MB 00:02 2024-09-18T11:27:40.072 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-18T11:27:40.090 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-18T11:27:40.090 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-18T11:27:40.457 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-18T11:27:40.458 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-18T11:27:40.984 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-18T11:27:41.060 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-18T11:27:41.099 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-18T11:27:41.127 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephsqlite-2:17.2.7-1645.g7061ea6c.el9.x86_64 3/4 2024-09-18T11:27:44.211 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1645.g7061ea6c.el9.x86_64 3/4 2024-09-18T11:27:44.295 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-test-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/4 2024-09-18T11:27:45.650 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1645.g7061ea6c.el9.x86_64 3/4 2024-09-18T11:27:45.734 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-test-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/4 2024-09-18T11:27:46.498 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-test-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/4 2024-09-18T11:27:46.498 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-test-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/4 2024-09-18T11:27:46.498 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephsqlite-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/4 2024-09-18T11:27:46.499 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-18T11:27:46.828 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-18T11:27:46.828 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:27:46.828 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-09-18T11:27:46.828 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:46.828 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:46.829 INFO:teuthology.orchestra.run.smithi119.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-18T11:27:46.829 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-18T11:27:46.829 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:27:46.829 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:27:47.143 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph 2024-09-18T11:27:47.719 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:00:38 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:27:47.858 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:27:47.861 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:27:47.861 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-09-18T11:27:47.861 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:27:47.861 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-09-18T11:27:47.861 INFO:teuthology.orchestra.run.smithi119.stdout: ceph x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 6.5 k 2024-09-18T11:27:47.861 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-09-18T11:27:47.861 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 2.1 M 2024-09-18T11:27:47.861 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 1.4 M 2024-09-18T11:27:47.861 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 240 k 2024-09-18T11:27:47.861 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 3.9 M 2024-09-18T11:27:47.862 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 15 M 2024-09-18T11:27:47.862 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-18T11:27:47.862 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-18T11:27:47.862 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-18T11:27:47.862 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-18T11:27:47.862 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-18T11:27:47.862 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-18T11:27:47.862 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-18T11:27:47.862 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections 2024-09-18T11:27:47.862 INFO:teuthology.orchestra.run.smithi119.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-18T11:27:47.862 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-18T11:27:47.862 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-18T11:27:47.862 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-18T11:27:47.863 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-18T11:27:47.863 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-18T11:27:47.863 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-18T11:27:47.863 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-18T11:27:47.863 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-18T11:27:47.863 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-18T11:27:47.863 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-18T11:27:47.863 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-18T11:27:47.863 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-18T11:27:47.863 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-18T11:27:47.863 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-18T11:27:47.863 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-18T11:27:47.863 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-18T11:27:47.864 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-18T11:27:47.864 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-18T11:27:47.864 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-18T11:27:47.864 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-18T11:27:47.864 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:27:47.864 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-09-18T11:27:47.864 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:27:47.864 INFO:teuthology.orchestra.run.smithi119.stdout:Install 34 Packages 2024-09-18T11:27:47.864 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:27:47.865 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 27 M 2024-09-18T11:27:47.865 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 96 M 2024-09-18T11:27:47.865 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-09-18T11:27:47.980 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-test-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/4 2024-09-18T11:27:47.980 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-test-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/4 2024-09-18T11:27:47.981 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephsqlite-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/4 2024-09-18T11:27:47.981 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-18T11:27:48.288 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-18T11:27:48.289 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:27:48.289 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-18T11:27:48.289 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:48.289 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:48.289 INFO:teuthology.orchestra.run.smithi148.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-18T11:27:48.289 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-18T11:27:48.289 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:27:48.289 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:27:48.536 INFO:teuthology.orchestra.run.smithi119.stdout:(1/34): ceph-17.2.7-1645.g7061ea6c.el9.x86_64.r 52 kB/s | 6.5 kB 00:00 2024-09-18T11:27:48.593 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph 2024-09-18T11:27:48.745 INFO:teuthology.orchestra.run.smithi119.stdout:(2/34): ceph-mds-17.2.7-1645.g7061ea6c.el9.x86_ 6.2 MB/s | 2.1 MB 00:00 2024-09-18T11:27:48.787 INFO:teuthology.orchestra.run.smithi119.stdout:(3/34): ceph-mgr-17.2.7-1645.g7061ea6c.el9.x86_ 3.7 MB/s | 1.4 MB 00:00 2024-09-18T11:27:48.895 INFO:teuthology.orchestra.run.smithi119.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1645.g7061 2.2 MB/s | 240 kB 00:00 2024-09-18T11:27:49.088 INFO:teuthology.orchestra.run.smithi119.stdout:(5/34): ceph-mon-17.2.7-1645.g7061ea6c.el9.x86_ 7.1 MB/s | 3.9 MB 00:00 2024-09-18T11:27:49.112 INFO:teuthology.orchestra.run.smithi119.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-18T11:27:49.181 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:39 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:27:49.280 INFO:teuthology.orchestra.run.smithi119.stdout:(7/34): ceph-osd-17.2.7-1645.g7061ea6c.el9.x86_ 29 MB/s | 15 MB 00:00 2024-09-18T11:27:49.305 INFO:teuthology.orchestra.run.smithi119.stdout:(8/34): python3-ply-3.11-14.el9.noarch.rpm 555 kB/s | 106 kB 00:00 2024-09-18T11:27:49.325 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:27:49.327 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:27:49.327 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-18T11:27:49.327 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:27:49.327 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-18T11:27:49.327 INFO:teuthology.orchestra.run.smithi148.stdout: ceph x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 6.5 k 2024-09-18T11:27:49.327 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-18T11:27:49.327 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 2.1 M 2024-09-18T11:27:49.327 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 1.4 M 2024-09-18T11:27:49.328 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 240 k 2024-09-18T11:27:49.328 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 3.9 M 2024-09-18T11:27:49.328 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 15 M 2024-09-18T11:27:49.328 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-18T11:27:49.328 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-18T11:27:49.328 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-18T11:27:49.328 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-18T11:27:49.328 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-18T11:27:49.328 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-18T11:27:49.328 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-18T11:27:49.328 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections 2024-09-18T11:27:49.328 INFO:teuthology.orchestra.run.smithi148.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-18T11:27:49.328 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-18T11:27:49.329 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-18T11:27:49.329 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-18T11:27:49.329 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-18T11:27:49.329 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-18T11:27:49.329 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-18T11:27:49.329 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-18T11:27:49.329 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-18T11:27:49.329 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-18T11:27:49.329 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-18T11:27:49.329 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-18T11:27:49.329 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-18T11:27:49.329 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-18T11:27:49.329 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-18T11:27:49.330 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-18T11:27:49.330 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-18T11:27:49.330 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-18T11:27:49.330 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-18T11:27:49.330 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-18T11:27:49.330 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-18T11:27:49.330 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:27:49.330 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-18T11:27:49.330 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:27:49.330 INFO:teuthology.orchestra.run.smithi148.stdout:Install 34 Packages 2024-09-18T11:27:49.330 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:27:49.331 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 27 M 2024-09-18T11:27:49.331 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 96 M 2024-09-18T11:27:49.331 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-18T11:27:49.413 INFO:teuthology.orchestra.run.smithi119.stdout:(9/34): python3-requests-2.25.1-8.el9.noarch.rp 1.1 MB/s | 125 kB 00:00 2024-09-18T11:27:49.439 INFO:teuthology.orchestra.run.smithi119.stdout:(10/34): python3-pycparser-2.20-6.el9.noarch.rp 852 kB/s | 135 kB 00:00 2024-09-18T11:27:49.522 INFO:teuthology.orchestra.run.smithi119.stdout:(11/34): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-18T11:27:49.547 INFO:teuthology.orchestra.run.smithi119.stdout:(12/34): python3-urllib3-1.26.5-6.el9.noarch.rp 1.6 MB/s | 215 kB 00:00 2024-09-18T11:27:49.573 INFO:teuthology.orchestra.run.smithi119.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 692 kB/s | 35 kB 00:00 2024-09-18T11:27:49.623 INFO:teuthology.orchestra.run.smithi119.stdout:(14/34): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-18T11:27:49.648 INFO:teuthology.orchestra.run.smithi119.stdout:(15/34): python3-packaging-20.9-5.el9.noarch.rp 767 kB/s | 77 kB 00:00 2024-09-18T11:27:49.682 INFO:teuthology.orchestra.run.smithi119.stdout:(16/34): python3-toml-0.10.2-6.el9.noarch.rpm 711 kB/s | 42 kB 00:00 2024-09-18T11:27:49.724 INFO:teuthology.orchestra.run.smithi119.stdout:(17/34): python3-cryptography-36.0.1-4.el9.x86_ 1.9 MB/s | 1.2 MB 00:00 2024-09-18T11:27:49.749 INFO:teuthology.orchestra.run.smithi119.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 432 kB/s | 43 kB 00:00 2024-09-18T11:27:49.783 INFO:teuthology.orchestra.run.smithi119.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 446 kB/s | 11 kB 00:00 2024-09-18T11:27:49.816 INFO:teuthology.orchestra.run.smithi119.stdout:(20/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-18T11:27:49.850 INFO:teuthology.orchestra.run.smithi119.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 271 kB/s | 18 kB 00:00 2024-09-18T11:27:49.867 INFO:teuthology.orchestra.run.smithi119.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-18T11:27:49.884 INFO:teuthology.orchestra.run.smithi119.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 583 kB/s | 19 kB 00:00 2024-09-18T11:27:49.901 INFO:teuthology.orchestra.run.smithi119.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 588 kB/s | 20 kB 00:00 2024-09-18T11:27:49.926 INFO:teuthology.orchestra.run.smithi119.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 1.1 MB/s | 46 kB 00:00 2024-09-18T11:27:49.960 INFO:teuthology.orchestra.run.smithi119.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-09-18T11:27:49.985 INFO:teuthology.orchestra.run.smithi119.stdout:(27/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-09-18T11:27:50.022 INFO:teuthology.orchestra.run.smithi119.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 267 kB/s | 16 kB 00:00 2024-09-18T11:27:50.044 INFO:teuthology.orchestra.run.smithi119.stdout:(29/34): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-09-18T11:27:50.069 INFO:teuthology.orchestra.run.smithi119.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 760 kB/s | 36 kB 00:00 2024-09-18T11:27:50.095 INFO:teuthology.orchestra.run.smithi119.stdout:(31/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-18T11:27:50.117 INFO:teuthology.orchestra.run.smithi148.stdout:(1/34): ceph-17.2.7-1645.g7061ea6c.el9.x86_64.r 54 kB/s | 6.5 kB 00:00 2024-09-18T11:27:50.120 INFO:teuthology.orchestra.run.smithi119.stdout:(32/34): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-18T11:27:50.145 INFO:teuthology.orchestra.run.smithi119.stdout:(33/34): python3-zc-lockfile-2.0-10.el9.noarch. 399 kB/s | 20 kB 00:00 2024-09-18T11:27:50.170 INFO:teuthology.orchestra.run.smithi119.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-18T11:27:50.174 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:27:50.174 INFO:teuthology.orchestra.run.smithi119.stdout:Total 12 MB/s | 27 MB 00:02 2024-09-18T11:27:50.292 INFO:teuthology.orchestra.run.smithi148.stdout:(2/34): ceph-mgr-17.2.7-1645.g7061ea6c.el9.x86_ 4.8 MB/s | 1.4 MB 00:00 2024-09-18T11:27:50.310 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-09-18T11:27:50.342 INFO:teuthology.orchestra.run.smithi148.stdout:(3/34): ceph-mds-17.2.7-1645.g7061ea6c.el9.x86_ 6.0 MB/s | 2.1 MB 00:00 2024-09-18T11:27:50.349 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-09-18T11:27:50.349 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-09-18T11:27:50.426 INFO:teuthology.orchestra.run.smithi148.stdout:(4/34): ceph-mon-17.2.7-1645.g7061ea6c.el9.x86_ 13 MB/s | 3.9 MB 00:00 2024-09-18T11:27:50.451 INFO:teuthology.orchestra.run.smithi148.stdout:(5/34): ceph-mgr-modules-core-17.2.7-1645.g7061 2.2 MB/s | 240 kB 00:00 2024-09-18T11:27:50.744 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-09-18T11:27:50.744 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-09-18T11:27:50.793 INFO:teuthology.orchestra.run.smithi148.stdout:(6/34): ceph-osd-17.2.7-1645.g7061ea6c.el9.x86_ 31 MB/s | 15 MB 00:00 2024-09-18T11:27:50.818 INFO:teuthology.orchestra.run.smithi148.stdout:(7/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 646 kB/s | 253 kB 00:00 2024-09-18T11:27:50.886 INFO:teuthology.orchestra.run.smithi148.stdout:(8/34): python3-pycparser-2.20-6.el9.noarch.rpm 2.0 MB/s | 135 kB 00:00 2024-09-18T11:27:50.952 INFO:teuthology.orchestra.run.smithi148.stdout:(9/34): python3-requests-2.25.1-8.el9.noarch.rp 1.8 MB/s | 125 kB 00:00 2024-09-18T11:27:50.986 INFO:teuthology.orchestra.run.smithi148.stdout:(10/34): python3-cryptography-36.0.1-4.el9.x86_ 2.3 MB/s | 1.2 MB 00:00 2024-09-18T11:27:51.011 INFO:teuthology.orchestra.run.smithi148.stdout:(11/34): python3-ply-3.11-14.el9.noarch.rpm 490 kB/s | 106 kB 00:00 2024-09-18T11:27:51.036 INFO:teuthology.orchestra.run.smithi148.stdout:(12/34): python3-urllib3-1.26.5-6.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-09-18T11:27:51.087 INFO:teuthology.orchestra.run.smithi148.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 463 kB/s | 35 kB 00:00 2024-09-18T11:27:51.129 INFO:teuthology.orchestra.run.smithi148.stdout:(14/34): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-18T11:27:51.154 INFO:teuthology.orchestra.run.smithi148.stdout:(15/34): python3-packaging-20.9-5.el9.noarch.rp 658 kB/s | 77 kB 00:00 2024-09-18T11:27:51.171 INFO:teuthology.orchestra.run.smithi148.stdout:(16/34): python3-pytz-2021.1-5.el9.noarch.rpm 607 kB/s | 51 kB 00:00 2024-09-18T11:27:51.188 INFO:teuthology.orchestra.run.smithi148.stdout:(17/34): python3-toml-0.10.2-6.el9.noarch.rpm 706 kB/s | 42 kB 00:00 2024-09-18T11:27:51.239 INFO:teuthology.orchestra.run.smithi148.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 520 kB/s | 43 kB 00:00 2024-09-18T11:27:51.264 INFO:teuthology.orchestra.run.smithi148.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 458 kB/s | 11 kB 00:00 2024-09-18T11:27:51.289 INFO:teuthology.orchestra.run.smithi148.stdout:(20/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-18T11:27:51.314 INFO:teuthology.orchestra.run.smithi148.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-09-18T11:27:51.339 INFO:teuthology.orchestra.run.smithi148.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 462 kB/s | 23 kB 00:00 2024-09-18T11:27:51.364 INFO:teuthology.orchestra.run.smithi148.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 388 kB/s | 19 kB 00:00 2024-09-18T11:27:51.390 INFO:teuthology.orchestra.run.smithi148.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-18T11:27:51.415 INFO:teuthology.orchestra.run.smithi148.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 922 kB/s | 46 kB 00:00 2024-09-18T11:27:51.440 INFO:teuthology.orchestra.run.smithi148.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-18T11:27:51.465 INFO:teuthology.orchestra.run.smithi148.stdout:(27/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-18T11:27:51.491 INFO:teuthology.orchestra.run.smithi148.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-09-18T11:27:51.516 INFO:teuthology.orchestra.run.smithi148.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-18T11:27:51.542 INFO:teuthology.orchestra.run.smithi148.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 709 kB/s | 36 kB 00:00 2024-09-18T11:27:51.567 INFO:teuthology.orchestra.run.smithi148.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-18T11:27:51.592 INFO:teuthology.orchestra.run.smithi148.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 805 kB/s | 20 kB 00:00 2024-09-18T11:27:51.617 INFO:teuthology.orchestra.run.smithi148.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-18T11:27:51.642 INFO:teuthology.orchestra.run.smithi148.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-18T11:27:51.647 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:27:51.647 INFO:teuthology.orchestra.run.smithi148.stdout:Total 12 MB/s | 27 MB 00:02 2024-09-18T11:27:51.777 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-18T11:27:51.814 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-18T11:27:51.814 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-18T11:27:52.004 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-09-18T11:27:52.060 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-18T11:27:52.092 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-18T11:27:52.125 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-18T11:27:52.209 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-18T11:27:52.209 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-18T11:27:52.217 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-18T11:27:52.304 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-18T11:27:52.369 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-18T11:27:52.405 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-18T11:27:52.443 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-18T11:27:52.477 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-18T11:27:52.511 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-18T11:27:52.551 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-18T11:27:52.586 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-18T11:27:52.621 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-18T11:27:52.670 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-18T11:27:52.710 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-18T11:27:52.783 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-18T11:27:52.931 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-18T11:27:53.009 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-18T11:27:53.063 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-18T11:27:53.101 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-18T11:27:53.159 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-18T11:27:53.221 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-18T11:27:53.417 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-18T11:27:53.471 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-18T11:27:53.495 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-18T11:27:53.551 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-18T11:27:53.562 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-18T11:27:53.594 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-18T11:27:53.627 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-18T11:27:53.641 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-18T11:27:53.704 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-18T11:27:53.718 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-18T11:27:53.805 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-18T11:27:53.823 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-18T11:27:53.846 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-18T11:27:53.881 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-18T11:27:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-18T11:27:53.945 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-18T11:27:53.977 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1645.g7061ea6c.el9. 29/34 2024-09-18T11:27:53.978 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-18T11:27:54.000 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-2:17.2.7-1645.g7061ea6c.el9.x86_64 30/34 2024-09-18T11:27:54.019 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-18T11:27:54.026 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1645.g7061ea6c.el9.x86_64 30/34 2024-09-18T11:27:54.027 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T11:27:54.027 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-18T11:27:54.027 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T11:27:54.027 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T11:27:54.027 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:27:54.058 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-18T11:27:54.092 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-18T11:27:54.140 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-18T11:27:54.182 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-18T11:27:54.262 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-18T11:27:54.407 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-18T11:27:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-18T11:27:54.541 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-18T11:27:54.580 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-18T11:27:54.629 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-18T11:27:54.684 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-18T11:27:54.880 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-18T11:27:54.926 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-18T11:27:55.003 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-18T11:27:55.101 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-18T11:27:55.150 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-18T11:27:55.265 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-18T11:27:55.351 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-osd-2:17.2.7-1645.g7061ea6c.el9.x86_64 31/34 2024-09-18T11:27:55.378 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-osd-2:17.2.7-1645.g7061ea6c.el9.x86_64 31/34 2024-09-18T11:27:55.379 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T11:27:55.379 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-18T11:27:55.379 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T11:27:55.379 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T11:27:55.379 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:27:55.412 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1645.g7061ea6c.el9. 29/34 2024-09-18T11:27:55.444 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-2:17.2.7-1645.g7061ea6c.el9.x86_64 30/34 2024-09-18T11:27:55.471 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1645.g7061ea6c.el9.x86_64 30/34 2024-09-18T11:27:55.471 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T11:27:55.471 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-18T11:27:55.471 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T11:27:55.471 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T11:27:55.471 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:27:55.751 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mon-2:17.2.7-1645.g7061ea6c.el9.x86_64 32/34 2024-09-18T11:27:55.775 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mon-2:17.2.7-1645.g7061ea6c.el9.x86_64 32/34 2024-09-18T11:27:55.776 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T11:27:55.776 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-18T11:27:55.776 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T11:27:55.776 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T11:27:55.776 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:27:55.977 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mds-2:17.2.7-1645.g7061ea6c.el9.x86_64 33/34 2024-09-18T11:27:56.001 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mds-2:17.2.7-1645.g7061ea6c.el9.x86_64 33/34 2024-09-18T11:27:56.001 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T11:27:56.001 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-18T11:27:56.001 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T11:27:56.001 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T11:27:56.001 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:27:56.067 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-2:17.2.7-1645.g7061ea6c.el9.x86_64 34/34 2024-09-18T11:27:56.778 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-osd-2:17.2.7-1645.g7061ea6c.el9.x86_64 31/34 2024-09-18T11:27:56.806 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-osd-2:17.2.7-1645.g7061ea6c.el9.x86_64 31/34 2024-09-18T11:27:56.806 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T11:27:56.806 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-18T11:27:56.806 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T11:27:56.806 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T11:27:56.806 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:27:57.179 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mon-2:17.2.7-1645.g7061ea6c.el9.x86_64 32/34 2024-09-18T11:27:57.202 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mon-2:17.2.7-1645.g7061ea6c.el9.x86_64 32/34 2024-09-18T11:27:57.203 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T11:27:57.203 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-18T11:27:57.203 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T11:27:57.203 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T11:27:57.203 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:27:57.404 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mds-2:17.2.7-1645.g7061ea6c.el9.x86_64 33/34 2024-09-18T11:27:57.428 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mds-2:17.2.7-1645.g7061ea6c.el9.x86_64 33/34 2024-09-18T11:27:57.428 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T11:27:57.428 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-18T11:27:57.428 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T11:27:57.428 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T11:27:57.429 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:27:57.492 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-2:17.2.7-1645.g7061ea6c.el9.x86_64 34/34 2024-09-18T11:27:57.733 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-2:17.2.7-1645.g7061ea6c.el9.x86_64 34/34 2024-09-18T11:27:57.733 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/34 2024-09-18T11:27:57.733 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mds-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/34 2024-09-18T11:27:57.733 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-2:17.2.7-1645.g7061ea6c.el9.x86_64 3/34 2024-09-18T11:27:57.733 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mon-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/34 2024-09-18T11:27:57.733 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-osd-2:17.2.7-1645.g7061ea6c.el9.x86_64 5/34 2024-09-18T11:27:57.733 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1645.g7061ea6c.el9. 6/34 2024-09-18T11:27:57.733 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-18T11:27:57.733 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-18T11:27:57.733 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-18T11:27:57.733 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-18T11:27:57.734 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-18T11:27:57.734 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-18T11:27:57.734 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-18T11:27:57.734 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-18T11:27:57.734 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-18T11:27:57.734 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-18T11:27:57.734 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-18T11:27:57.734 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-18T11:27:57.734 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-18T11:27:57.734 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-18T11:27:57.734 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-18T11:27:57.734 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-18T11:27:57.734 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-18T11:27:57.735 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-18T11:27:57.735 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-18T11:27:57.735 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-18T11:27:57.735 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-18T11:27:57.735 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-18T11:27:57.735 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-18T11:27:57.735 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-18T11:27:57.735 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-18T11:27:57.735 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-18T11:27:57.735 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-18T11:27:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-18T11:27:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:27:58.620 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-09-18T11:27:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T11:27:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:27:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-18T11:27:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-18T11:27:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-18T11:27:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-18T11:27:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-18T11:27:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-18T11:27:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-18T11:27:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-18T11:27:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-18T11:27:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-18T11:27:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-18T11:27:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-18T11:27:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-18T11:27:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-18T11:27:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-18T11:27:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-18T11:27:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply-3.11-14.el9.noarch 2024-09-18T11:27:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-18T11:27:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-18T11:27:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-18T11:27:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-18T11:27:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-18T11:27:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-18T11:27:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-18T11:27:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-18T11:27:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-18T11:27:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-18T11:27:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-18T11:27:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:27:58.624 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:27:59.032 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-base 2024-09-18T11:27:59.162 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-2:17.2.7-1645.g7061ea6c.el9.x86_64 34/34 2024-09-18T11:27:59.162 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/34 2024-09-18T11:27:59.162 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mds-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/34 2024-09-18T11:27:59.162 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-2:17.2.7-1645.g7061ea6c.el9.x86_64 3/34 2024-09-18T11:27:59.162 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mon-2:17.2.7-1645.g7061ea6c.el9.x86_64 4/34 2024-09-18T11:27:59.163 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-osd-2:17.2.7-1645.g7061ea6c.el9.x86_64 5/34 2024-09-18T11:27:59.163 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1645.g7061ea6c.el9. 6/34 2024-09-18T11:27:59.163 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-18T11:27:59.163 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-18T11:27:59.163 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-18T11:27:59.163 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-18T11:27:59.163 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-18T11:27:59.163 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-18T11:27:59.163 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-18T11:27:59.163 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-18T11:27:59.163 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-18T11:27:59.163 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-18T11:27:59.164 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-18T11:27:59.164 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-18T11:27:59.164 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-18T11:27:59.164 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-18T11:27:59.164 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-18T11:27:59.164 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-18T11:27:59.164 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-18T11:27:59.164 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-18T11:27:59.164 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-18T11:27:59.164 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-18T11:27:59.164 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-18T11:27:59.164 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-18T11:27:59.164 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-18T11:27:59.165 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-18T11:27:59.165 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-18T11:27:59.165 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-18T11:27:59.165 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-18T11:27:59.615 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:00:50 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:27:59.704 INFO:teuthology.orchestra.run.smithi119.stdout:Package ceph-base-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T11:27:59.745 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:27:59.746 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-09-18T11:27:59.747 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:27:59.814 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install cephadm 2024-09-18T11:28:00.166 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-18T11:28:00.167 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:00.167 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-18T11:28:00.167 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:28:00.167 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:28:00.167 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:28:00.167 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T11:28:00.167 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:28:00.167 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:28:00.167 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-18T11:28:00.167 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-18T11:28:00.167 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-18T11:28:00.167 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-18T11:28:00.168 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-18T11:28:00.168 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-18T11:28:00.168 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-18T11:28:00.168 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-18T11:28:00.168 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-18T11:28:00.168 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-18T11:28:00.168 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-18T11:28:00.168 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-18T11:28:00.168 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-18T11:28:00.168 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-18T11:28:00.168 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-18T11:28:00.169 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-18T11:28:00.169 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply-3.11-14.el9.noarch 2024-09-18T11:28:00.169 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-18T11:28:00.169 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-18T11:28:00.169 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-18T11:28:00.169 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-18T11:28:00.169 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-18T11:28:00.169 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-18T11:28:00.169 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-18T11:28:00.169 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-18T11:28:00.169 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-18T11:28:00.169 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-18T11:28:00.169 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-18T11:28:00.170 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:00.170 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:00.394 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:00:51 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:28:00.528 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:28:00.530 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:00.530 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-09-18T11:28:00.530 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:00.530 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-09-18T11:28:00.530 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 83 k 2024-09-18T11:28:00.530 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:00.530 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-09-18T11:28:00.530 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:00.530 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-09-18T11:28:00.530 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:00.530 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 83 k 2024-09-18T11:28:00.531 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 375 k 2024-09-18T11:28:00.531 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-09-18T11:28:00.531 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-base 2024-09-18T11:28:00.709 INFO:teuthology.orchestra.run.smithi119.stdout:cephadm-17.2.7-1645.g7061ea6c.el9.noarch.rpm 463 kB/s | 83 kB 00:00 2024-09-18T11:28:00.709 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:00.710 INFO:teuthology.orchestra.run.smithi119.stdout:Total 459 kB/s | 83 kB 00:00 2024-09-18T11:28:00.710 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-09-18T11:28:00.715 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-09-18T11:28:00.715 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-09-18T11:28:00.717 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-09-18T11:28:00.717 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-09-18T11:28:00.792 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-09-18T11:28:01.124 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:51 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:01.214 INFO:teuthology.orchestra.run.smithi148.stdout:Package ceph-base-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T11:28:01.249 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T11:28:01.257 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:01.258 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-18T11:28:01.258 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:01.330 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install cephadm 2024-09-18T11:28:01.354 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T11:28:01.703 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T11:28:01.903 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:51 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:01.967 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T11:28:01.967 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:01.967 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-09-18T11:28:01.967 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T11:28:01.967 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:01.967 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:28:02.034 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:02.035 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:02.035 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-18T11:28:02.035 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:02.035 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-18T11:28:02.035 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 83 k 2024-09-18T11:28:02.035 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:02.036 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-18T11:28:02.036 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:02.036 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-18T11:28:02.036 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:02.036 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 83 k 2024-09-18T11:28:02.036 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 375 k 2024-09-18T11:28:02.036 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-18T11:28:02.145 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-immutable-object-cache 2024-09-18T11:28:02.221 INFO:teuthology.orchestra.run.smithi148.stdout:cephadm-17.2.7-1645.g7061ea6c.el9.noarch.rpm 447 kB/s | 83 kB 00:00 2024-09-18T11:28:02.221 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:02.222 INFO:teuthology.orchestra.run.smithi148.stdout:Total 444 kB/s | 83 kB 00:00 2024-09-18T11:28:02.222 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-18T11:28:02.227 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-18T11:28:02.227 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-18T11:28:02.229 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-18T11:28:02.229 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-18T11:28:02.308 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-18T11:28:02.732 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:00:53 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:28:02.745 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T11:28:02.850 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T11:28:02.864 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:28:02.865 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repo Size 2024-09-18T11:28:02.865 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:02.865 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-09-18T11:28:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 137 k 2024-09-18T11:28:02.865 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:02.866 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-09-18T11:28:02.866 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:02.866 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-09-18T11:28:02.866 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:02.866 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 137 k 2024-09-18T11:28:02.866 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 379 k 2024-09-18T11:28:02.866 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-09-18T11:28:03.073 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-immutable-object-cache-17.2.7-1645.g7061ea 664 kB/s | 137 kB 00:00 2024-09-18T11:28:03.074 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:03.074 INFO:teuthology.orchestra.run.smithi119.stdout:Total 658 kB/s | 137 kB 00:00 2024-09-18T11:28:03.074 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-09-18T11:28:03.081 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-09-18T11:28:03.082 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-09-18T11:28:03.138 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-09-18T11:28:03.138 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-09-18T11:28:03.200 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T11:28:03.287 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-09-18T11:28:03.374 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1645.g7061ea6c. 1/1 2024-09-18T11:28:03.398 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1645.g7061ea6c. 1/1 2024-09-18T11:28:03.399 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T11:28:03.399 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-18T11:28:03.399 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:03.452 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T11:28:03.453 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:03.453 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-18T11:28:03.453 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T11:28:03.453 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:03.453 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:03.647 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-immutable-object-cache 2024-09-18T11:28:04.243 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:54 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:04.342 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1645.g7061ea6c. 1/1 2024-09-18T11:28:04.342 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:04.342 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-09-18T11:28:04.343 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:28:04.343 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:04.343 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:28:04.381 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:04.381 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:04.381 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-09-18T11:28:04.381 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:04.381 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-18T11:28:04.381 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 137 k 2024-09-18T11:28:04.382 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:04.382 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-18T11:28:04.382 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:04.382 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-18T11:28:04.382 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:04.382 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 137 k 2024-09-18T11:28:04.383 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 379 k 2024-09-18T11:28:04.383 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-18T11:28:04.577 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-immutable-object-cache-17.2.7-1645.g7061ea 707 kB/s | 137 kB 00:00 2024-09-18T11:28:04.577 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:04.578 INFO:teuthology.orchestra.run.smithi148.stdout:Total 701 kB/s | 137 kB 00:00 2024-09-18T11:28:04.578 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-18T11:28:04.585 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-18T11:28:04.586 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-18T11:28:04.593 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-mgr 2024-09-18T11:28:04.642 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-18T11:28:04.642 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-18T11:28:04.792 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-18T11:28:04.886 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1645.g7061ea6c. 1/1 2024-09-18T11:28:04.910 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1645.g7061ea6c. 1/1 2024-09-18T11:28:04.910 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T11:28:04.911 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-18T11:28:04.911 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:05.177 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:00:56 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:28:05.268 INFO:teuthology.orchestra.run.smithi119.stdout:Package ceph-mgr-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T11:28:05.311 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:28:05.312 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-09-18T11:28:05.312 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:28:05.381 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-mgr-dashboard 2024-09-18T11:28:05.825 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1645.g7061ea6c. 1/1 2024-09-18T11:28:05.826 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:05.826 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-18T11:28:05.826 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:28:05.826 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:05.826 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:05.951 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:00:56 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:28:06.037 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr 2024-09-18T11:28:06.081 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:28:06.082 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:06.082 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-09-18T11:28:06.082 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:06.082 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-09-18T11:28:06.082 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 3.4 M 2024-09-18T11:28:06.082 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-09-18T11:28:06.082 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 24 k 2024-09-18T11:28:06.083 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 15 k 2024-09-18T11:28:06.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-18T11:28:06.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-18T11:28:06.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-18T11:28:06.083 INFO:teuthology.orchestra.run.smithi119.stdout:Installing weak dependencies: 2024-09-18T11:28:06.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-18T11:28:06.083 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:06.083 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-09-18T11:28:06.083 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:06.083 INFO:teuthology.orchestra.run.smithi119.stdout:Install 7 Packages 2024-09-18T11:28:06.084 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:06.085 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 3.7 M 2024-09-18T11:28:06.085 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 80 M 2024-09-18T11:28:06.085 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-09-18T11:28:06.393 INFO:teuthology.orchestra.run.smithi119.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1645.g706 159 kB/s | 24 kB 00:00 2024-09-18T11:28:06.418 INFO:teuthology.orchestra.run.smithi119.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1645.g7061 82 kB/s | 15 kB 00:00 2024-09-18T11:28:06.460 INFO:teuthology.orchestra.run.smithi119.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 135 kB/s | 9.0 kB 00:00 2024-09-18T11:28:06.485 INFO:teuthology.orchestra.run.smithi119.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-18T11:28:06.511 INFO:teuthology.orchestra.run.smithi119.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 443 kB/s | 41 kB 00:00 2024-09-18T11:28:06.536 INFO:teuthology.orchestra.run.smithi119.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-18T11:28:06.624 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:56 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:06.644 INFO:teuthology.orchestra.run.smithi119.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1645.g7061ea6c 8.3 MB/s | 3.4 MB 00:00 2024-09-18T11:28:06.648 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:06.648 INFO:teuthology.orchestra.run.smithi119.stdout:Total 6.5 MB/s | 3.7 MB 00:00 2024-09-18T11:28:06.649 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-09-18T11:28:06.656 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-09-18T11:28:06.657 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-09-18T11:28:06.716 INFO:teuthology.orchestra.run.smithi148.stdout:Package ceph-mgr-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T11:28:06.741 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-09-18T11:28:06.741 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-09-18T11:28:06.759 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:06.760 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-18T11:28:06.760 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:06.841 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-dashboard 2024-09-18T11:28:07.048 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-09-18T11:28:07.112 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-18T11:28:07.146 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-18T11:28:07.201 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-18T11:28:07.233 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-18T11:28:07.270 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1645.g7061ea6c.el9.n 5/7 2024-09-18T11:28:07.413 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:57 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:07.544 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:07.545 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:07.545 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-18T11:28:07.545 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:07.545 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-18T11:28:07.545 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 3.4 M 2024-09-18T11:28:07.545 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-18T11:28:07.545 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 24 k 2024-09-18T11:28:07.545 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 15 k 2024-09-18T11:28:07.545 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-18T11:28:07.546 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-18T11:28:07.546 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-18T11:28:07.546 INFO:teuthology.orchestra.run.smithi148.stdout:Installing weak dependencies: 2024-09-18T11:28:07.546 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-18T11:28:07.546 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:07.546 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-18T11:28:07.546 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:07.546 INFO:teuthology.orchestra.run.smithi148.stdout:Install 7 Packages 2024-09-18T11:28:07.546 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:07.547 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 3.7 M 2024-09-18T11:28:07.547 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 80 M 2024-09-18T11:28:07.547 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-18T11:28:07.755 INFO:teuthology.orchestra.run.smithi148.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1645.g706 176 kB/s | 24 kB 00:00 2024-09-18T11:28:07.772 INFO:teuthology.orchestra.run.smithi148.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1645.g7061 94 kB/s | 15 kB 00:00 2024-09-18T11:28:07.806 INFO:teuthology.orchestra.run.smithi148.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 181 kB/s | 9.0 kB 00:00 2024-09-18T11:28:07.831 INFO:teuthology.orchestra.run.smithi148.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-18T11:28:07.856 INFO:teuthology.orchestra.run.smithi148.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 488 kB/s | 41 kB 00:00 2024-09-18T11:28:07.881 INFO:teuthology.orchestra.run.smithi148.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-18T11:28:07.990 INFO:teuthology.orchestra.run.smithi148.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1645.g7061ea6c 9.0 MB/s | 3.4 MB 00:00 2024-09-18T11:28:07.993 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:07.994 INFO:teuthology.orchestra.run.smithi148.stdout:Total 8.2 MB/s | 3.7 MB 00:00 2024-09-18T11:28:07.994 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-18T11:28:08.002 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-18T11:28:08.002 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-18T11:28:08.085 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-18T11:28:08.086 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-18T11:28:08.257 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1645.g7061ea6c.el9. 6/7 2024-09-18T11:28:08.299 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1645.g7061ea6c.el9.noarc 7/7 2024-09-18T11:28:08.390 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-18T11:28:08.466 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-18T11:28:08.500 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-18T11:28:08.546 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-18T11:28:08.579 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-18T11:28:08.626 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1645.g7061ea6c.el9.n 5/7 2024-09-18T11:28:09.351 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1645.g7061ea6c.el9.noarc 7/7 2024-09-18T11:28:09.351 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1645.g7061ea6c.el9. 1/7 2024-09-18T11:28:09.351 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1645.g7061ea6c.el9.noarc 2/7 2024-09-18T11:28:09.351 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1645.g7061ea6c.el9.n 3/7 2024-09-18T11:28:09.351 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-18T11:28:09.351 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-18T11:28:09.351 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-18T11:28:09.619 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1645.g7061ea6c.el9. 6/7 2024-09-18T11:28:09.625 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-18T11:28:09.625 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:09.625 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-09-18T11:28:09.625 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T11:28:09.625 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T11:28:09.625 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T11:28:09.625 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-18T11:28:09.625 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-18T11:28:09.625 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-18T11:28:09.625 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-18T11:28:09.626 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:09.626 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:28:09.663 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1645.g7061ea6c.el9.noarc 7/7 2024-09-18T11:28:09.854 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-18T11:28:10.433 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:01 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:28:10.566 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:28:10.567 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-09-18T11:28:10.567 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-09-18T11:28:10.567 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-09-18T11:28:10.567 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-09-18T11:28:10.567 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 7.4 M 2024-09-18T11:28:10.567 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-09-18T11:28:10.567 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-18T11:28:10.567 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-18T11:28:10.567 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-18T11:28:10.568 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-18T11:28:10.568 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-18T11:28:10.568 INFO:teuthology.orchestra.run.smithi119.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-18T11:28:10.568 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-18T11:28:10.568 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-18T11:28:10.568 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-18T11:28:10.568 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-18T11:28:10.568 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-18T11:28:10.568 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:10.568 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-09-18T11:28:10.568 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-09-18T11:28:10.568 INFO:teuthology.orchestra.run.smithi119.stdout:Install 12 Packages 2024-09-18T11:28:10.569 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:10.569 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 38 M 2024-09-18T11:28:10.569 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 212 M 2024-09-18T11:28:10.569 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-09-18T11:28:10.676 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1645.g7061ea6c.el9.noarc 7/7 2024-09-18T11:28:10.677 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1645.g7061ea6c.el9. 1/7 2024-09-18T11:28:10.677 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1645.g7061ea6c.el9.noarc 2/7 2024-09-18T11:28:10.677 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1645.g7061ea6c.el9.n 3/7 2024-09-18T11:28:10.677 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-18T11:28:10.677 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-18T11:28:10.677 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-18T11:28:10.950 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-18T11:28:10.950 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:10.950 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-18T11:28:10.950 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T11:28:10.950 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T11:28:10.951 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T11:28:10.951 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-18T11:28:10.951 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-18T11:28:10.951 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-18T11:28:10.951 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-18T11:28:10.951 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:10.951 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:11.094 INFO:teuthology.orchestra.run.smithi119.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-18T11:28:11.127 INFO:teuthology.orchestra.run.smithi119.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.9 MB/s | 801 kB 00:00 2024-09-18T11:28:11.173 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-18T11:28:11.202 INFO:teuthology.orchestra.run.smithi119.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-09-18T11:28:11.253 INFO:teuthology.orchestra.run.smithi119.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 364 kB/s | 18 kB 00:00 2024-09-18T11:28:11.345 INFO:teuthology.orchestra.run.smithi119.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 418 kB/s | 38 kB 00:00 2024-09-18T11:28:11.479 INFO:teuthology.orchestra.run.smithi119.stdout:(6/12): ceph-mgr-diskprediction-local-17.2.7-16 14 MB/s | 7.4 MB 00:00 2024-09-18T11:28:11.671 INFO:teuthology.orchestra.run.smithi119.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2024-09-18T11:28:11.759 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:01 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:11.893 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:11.894 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-09-18T11:28:11.894 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-18T11:28:11.895 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-09-18T11:28:11.895 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-18T11:28:11.895 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 7.4 M 2024-09-18T11:28:11.895 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-18T11:28:11.895 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-18T11:28:11.895 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-18T11:28:11.895 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-18T11:28:11.895 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-18T11:28:11.895 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-18T11:28:11.895 INFO:teuthology.orchestra.run.smithi148.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-18T11:28:11.895 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-18T11:28:11.895 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-18T11:28:11.896 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-18T11:28:11.896 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-18T11:28:11.896 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-18T11:28:11.896 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:11.896 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-18T11:28:11.896 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-09-18T11:28:11.896 INFO:teuthology.orchestra.run.smithi148.stdout:Install 12 Packages 2024-09-18T11:28:11.896 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:11.897 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 38 M 2024-09-18T11:28:11.897 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 212 M 2024-09-18T11:28:11.897 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-18T11:28:12.113 INFO:teuthology.orchestra.run.smithi119.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.0 MB/s | 3.0 MB 00:00 2024-09-18T11:28:12.221 INFO:teuthology.orchestra.run.smithi119.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.3 MB/s | 250 kB 00:00 2024-09-18T11:28:12.341 INFO:teuthology.orchestra.run.smithi148.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 911 kB/s | 191 kB 00:00 2024-09-18T11:28:12.508 INFO:teuthology.orchestra.run.smithi148.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.1 MB/s | 801 kB 00:00 2024-09-18T11:28:12.609 INFO:teuthology.orchestra.run.smithi148.stdout:(3/12): ceph-mgr-diskprediction-local-17.2.7-16 15 MB/s | 7.4 MB 00:00 2024-09-18T11:28:12.721 INFO:teuthology.orchestra.run.smithi119.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 3.6 MB/s | 4.9 MB 00:01 2024-09-18T11:28:12.958 INFO:teuthology.orchestra.run.smithi148.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 53 kB/s | 33 kB 00:00 2024-09-18T11:28:13.092 INFO:teuthology.orchestra.run.smithi148.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 38 kB/s | 18 kB 00:00 2024-09-18T11:28:13.255 INFO:teuthology.orchestra.run.smithi119.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.2 MB/s | 5.1 MB 00:01 2024-09-18T11:28:13.525 INFO:teuthology.orchestra.run.smithi148.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 68 kB/s | 38 kB 00:00 2024-09-18T11:28:13.584 INFO:teuthology.orchestra.run.smithi148.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.8 MB/s | 3.0 MB 00:01 2024-09-18T11:28:14.185 INFO:teuthology.orchestra.run.smithi148.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 4.5 MB/s | 4.9 MB 00:01 2024-09-18T11:28:14.293 INFO:teuthology.orchestra.run.smithi148.stdout:(9/12): python3-devel-3.9.19-8.el9.x86_64.rpm 319 kB/s | 245 kB 00:00 2024-09-18T11:28:14.693 INFO:teuthology.orchestra.run.smithi148.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.6 MB/s | 5.1 MB 00:01 2024-09-18T11:28:14.864 INFO:teuthology.orchestra.run.smithi119.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.2 MB/s | 16 MB 00:02 2024-09-18T11:28:14.864 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:14.865 INFO:teuthology.orchestra.run.smithi119.stdout:Total 8.9 MB/s | 38 MB 00:04 2024-09-18T11:28:15.010 INFO:teuthology.orchestra.run.smithi148.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 304 kB/s | 250 kB 00:00 2024-09-18T11:28:15.228 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-09-18T11:28:15.254 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-09-18T11:28:15.254 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-09-18T11:28:15.750 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-09-18T11:28:15.750 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-09-18T11:28:15.786 INFO:teuthology.orchestra.run.smithi148.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-18T11:28:15.787 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:15.788 INFO:teuthology.orchestra.run.smithi148.stdout:Total 9.8 MB/s | 38 MB 00:03 2024-09-18T11:28:16.148 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-18T11:28:16.175 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-18T11:28:16.175 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-18T11:28:16.521 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-09-18T11:28:16.600 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-18T11:28:16.644 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-18T11:28:16.690 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-18T11:28:16.691 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-18T11:28:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-18T11:28:16.796 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-18T11:28:16.988 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-18T11:28:17.020 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-18T11:28:17.096 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-18T11:28:17.457 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-18T11:28:17.518 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-18T11:28:17.559 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-18T11:28:17.659 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-18T11:28:17.701 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-18T11:28:17.757 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-18T11:28:17.874 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-18T11:28:17.887 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-18T11:28:17.917 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-18T11:28:17.993 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-18T11:28:18.666 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-18T11:28:18.789 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-18T11:28:19.145 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T11:28:20.089 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T11:28:20.181 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-18T11:28:20.219 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1645.g7061e 12/12 2024-09-18T11:28:21.133 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-18T11:28:21.166 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1645.g7061e 12/12 2024-09-18T11:28:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1645.g7061e 12/12 2024-09-18T11:28:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1645.g7061e 1/12 2024-09-18T11:28:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-18T11:28:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-18T11:28:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-18T11:28:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-18T11:28:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-18T11:28:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-18T11:28:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-18T11:28:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-18T11:28:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T11:28:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-18T11:28:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-18T11:28:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:23.276 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-09-18T11:28:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T11:28:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-18T11:28:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-18T11:28:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-18T11:28:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-18T11:28:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-18T11:28:23.276 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-18T11:28:23.277 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-18T11:28:23.277 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-18T11:28:23.277 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-18T11:28:23.277 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-18T11:28:23.277 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-18T11:28:23.277 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:23.277 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:28:23.675 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-mgr-rook 2024-09-18T11:28:23.767 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1645.g7061e 12/12 2024-09-18T11:28:23.767 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1645.g7061e 1/12 2024-09-18T11:28:23.767 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-18T11:28:23.767 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-18T11:28:23.767 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-18T11:28:23.767 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-18T11:28:23.767 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-18T11:28:23.767 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-18T11:28:23.767 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-18T11:28:23.767 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-18T11:28:23.767 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T11:28:23.767 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-18T11:28:24.219 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-18T11:28:24.220 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:24.220 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-18T11:28:24.220 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T11:28:24.220 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-18T11:28:24.220 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-18T11:28:24.220 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-18T11:28:24.220 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-18T11:28:24.220 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-18T11:28:24.220 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-18T11:28:24.220 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-18T11:28:24.220 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-18T11:28:24.220 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-18T11:28:24.220 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-18T11:28:24.221 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-18T11:28:24.221 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:24.221 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:24.260 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:15 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:28:24.391 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:28:24.392 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:24.393 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-09-18T11:28:24.393 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:24.393 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-09-18T11:28:24.393 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 47 k 2024-09-18T11:28:24.393 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-09-18T11:28:24.393 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-18T11:28:24.393 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-18T11:28:24.393 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-18T11:28:24.393 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-18T11:28:24.393 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-18T11:28:24.393 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-18T11:28:24.393 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-18T11:28:24.393 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-18T11:28:24.394 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-18T11:28:24.394 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-18T11:28:24.394 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-18T11:28:24.394 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-18T11:28:24.394 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:24.394 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-09-18T11:28:24.394 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:24.394 INFO:teuthology.orchestra.run.smithi119.stdout:Install 13 Packages 2024-09-18T11:28:24.394 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:24.395 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 2.2 M 2024-09-18T11:28:24.395 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 26 M 2024-09-18T11:28:24.395 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-09-18T11:28:24.612 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-rook 2024-09-18T11:28:24.612 INFO:teuthology.orchestra.run.smithi119.stdout:(1/13): ceph-mgr-rook-17.2.7-1645.g7061ea6c.el9 301 kB/s | 47 kB 00:00 2024-09-18T11:28:24.746 INFO:teuthology.orchestra.run.smithi119.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 66 kB/s | 19 kB 00:00 2024-09-18T11:28:24.771 INFO:teuthology.orchestra.run.smithi119.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 81 kB/s | 26 kB 00:00 2024-09-18T11:28:25.122 INFO:teuthology.orchestra.run.smithi119.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 423 kB/s | 159 kB 00:00 2024-09-18T11:28:25.163 INFO:teuthology.orchestra.run.smithi119.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 403 kB/s | 222 kB 00:00 2024-09-18T11:28:25.205 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:15 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:25.205 INFO:teuthology.orchestra.run.smithi119.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 772 kB/s | 32 kB 00:00 2024-09-18T11:28:25.231 INFO:teuthology.orchestra.run.smithi119.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 607 kB/s | 279 kB 00:00 2024-09-18T11:28:25.247 INFO:teuthology.orchestra.run.smithi119.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 336 kB/s | 14 kB 00:00 2024-09-18T11:28:25.273 INFO:teuthology.orchestra.run.smithi119.stdout:(9/13): python3-requests-oauthlib-1.3.0-12.el9. 355 kB/s | 54 kB 00:00 2024-09-18T11:28:25.323 INFO:teuthology.orchestra.run.smithi119.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.3 MB/s | 220 kB 00:00 2024-09-18T11:28:25.338 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:25.339 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:25.339 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-18T11:28:25.339 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:25.340 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-18T11:28:25.340 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 47 k 2024-09-18T11:28:25.340 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-18T11:28:25.340 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-18T11:28:25.340 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-18T11:28:25.340 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-18T11:28:25.340 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-18T11:28:25.340 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-18T11:28:25.340 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-18T11:28:25.340 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-18T11:28:25.340 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-18T11:28:25.340 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-18T11:28:25.341 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-18T11:28:25.341 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-18T11:28:25.341 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-18T11:28:25.341 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:25.341 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-18T11:28:25.341 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:25.341 INFO:teuthology.orchestra.run.smithi148.stdout:Install 13 Packages 2024-09-18T11:28:25.341 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:25.342 INFO:teuthology.orchestra.run.smithi119.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 906 kB/s | 59 kB 00:00 2024-09-18T11:28:25.342 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 2.2 M 2024-09-18T11:28:25.342 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 26 M 2024-09-18T11:28:25.342 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-18T11:28:25.365 INFO:teuthology.orchestra.run.smithi119.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 2.1 MB/s | 90 kB 00:00 2024-09-18T11:28:25.399 INFO:teuthology.orchestra.run.smithi119.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.1 MB/s | 1.0 MB 00:00 2024-09-18T11:28:25.401 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:25.402 INFO:teuthology.orchestra.run.smithi119.stdout:Total 2.2 MB/s | 2.2 MB 00:01 2024-09-18T11:28:25.465 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-09-18T11:28:25.479 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-09-18T11:28:25.480 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-09-18T11:28:25.558 INFO:teuthology.orchestra.run.smithi148.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 169 kB/s | 19 kB 00:00 2024-09-18T11:28:25.566 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-09-18T11:28:25.569 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-09-18T11:28:25.583 INFO:teuthology.orchestra.run.smithi148.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 184 kB/s | 26 kB 00:00 2024-09-18T11:28:25.609 INFO:teuthology.orchestra.run.smithi148.stdout:(3/13): ceph-mgr-rook-17.2.7-1645.g7061ea6c.el9 289 kB/s | 47 kB 00:00 2024-09-18T11:28:25.701 INFO:teuthology.orchestra.run.smithi148.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-18T11:28:25.726 INFO:teuthology.orchestra.run.smithi148.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.3 MB/s | 222 kB 00:00 2024-09-18T11:28:25.768 INFO:teuthology.orchestra.run.smithi148.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 802 kB/s | 54 kB 00:00 2024-09-18T11:28:25.802 INFO:teuthology.orchestra.run.smithi148.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.4 MB/s | 279 kB 00:00 2024-09-18T11:28:25.852 INFO:teuthology.orchestra.run.smithi148.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 255 kB/s | 32 kB 00:00 2024-09-18T11:28:25.877 INFO:teuthology.orchestra.run.smithi148.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 129 kB/s | 14 kB 00:00 2024-09-18T11:28:25.902 INFO:teuthology.orchestra.run.smithi148.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-18T11:28:25.928 INFO:teuthology.orchestra.run.smithi148.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.9 MB/s | 220 kB 00:00 2024-09-18T11:28:25.953 INFO:teuthology.orchestra.run.smithi148.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-18T11:28:25.995 INFO:teuthology.orchestra.run.smithi148.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.3 MB/s | 1.0 MB 00:00 2024-09-18T11:28:25.997 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:25.997 INFO:teuthology.orchestra.run.smithi148.stdout:Total 3.4 MB/s | 2.2 MB 00:00 2024-09-18T11:28:26.060 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-18T11:28:26.075 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-18T11:28:26.075 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-18T11:28:26.089 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-09-18T11:28:26.164 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-18T11:28:26.165 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-18T11:28:26.253 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-18T11:28:26.308 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-18T11:28:26.372 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-18T11:28:26.418 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-18T11:28:26.459 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-18T11:28:26.561 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-18T11:28:26.664 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-18T11:28:26.685 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-18T11:28:26.719 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-18T11:28:26.860 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-18T11:28:26.922 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-18T11:28:26.976 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-18T11:28:27.013 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-18T11:28:27.055 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-18T11:28:27.157 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-18T11:28:27.267 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-18T11:28:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-18T11:28:27.442 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-18T11:28:27.500 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-18T11:28:27.533 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-18T11:28:27.579 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-18T11:28:27.603 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-rook-2:17.2.7-1645.g7061ea6c.el9.noarch 13/13 2024-09-18T11:28:28.044 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-18T11:28:28.104 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-18T11:28:28.137 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-18T11:28:28.184 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-18T11:28:28.199 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-rook-2:17.2.7-1645.g7061ea6c.el9.noarch 13/13 2024-09-18T11:28:28.373 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1645.g7061ea6c.el9.noarch 13/13 2024-09-18T11:28:28.373 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1645.g7061ea6c.el9.noarch 1/13 2024-09-18T11:28:28.373 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-18T11:28:28.373 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-18T11:28:28.373 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-18T11:28:28.374 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-18T11:28:28.374 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-18T11:28:28.374 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-18T11:28:28.374 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-18T11:28:28.374 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-18T11:28:28.374 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-18T11:28:28.374 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-18T11:28:28.374 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-18T11:28:28.712 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-18T11:28:28.712 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:28.713 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-09-18T11:28:28.713 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T11:28:28.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-18T11:28:28.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-18T11:28:28.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-18T11:28:28.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-18T11:28:28.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-18T11:28:28.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-18T11:28:28.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-18T11:28:28.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-18T11:28:28.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-18T11:28:28.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-18T11:28:28.713 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-18T11:28:28.714 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-18T11:28:28.714 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:28.714 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:28:28.943 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1645.g7061ea6c.el9.noarch 13/13 2024-09-18T11:28:28.943 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1645.g7061ea6c.el9.noarch 1/13 2024-09-18T11:28:28.943 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-18T11:28:28.943 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-18T11:28:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-18T11:28:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-18T11:28:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-18T11:28:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-18T11:28:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-18T11:28:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-18T11:28:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-18T11:28:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-18T11:28:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-18T11:28:28.963 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-mgr-cephadm 2024-09-18T11:28:29.281 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-18T11:28:29.281 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:29.281 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-18T11:28:29.281 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T11:28:29.281 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-18T11:28:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-18T11:28:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-18T11:28:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-18T11:28:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-18T11:28:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-18T11:28:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-18T11:28:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-18T11:28:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-18T11:28:29.283 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-18T11:28:29.283 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-18T11:28:29.283 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-18T11:28:29.283 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:29.283 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:29.536 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-cephadm 2024-09-18T11:28:29.542 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:20 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:28:29.674 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:28:29.675 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:29.675 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-09-18T11:28:29.675 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:29.675 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-09-18T11:28:29.675 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 124 k 2024-09-18T11:28:29.675 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-09-18T11:28:29.675 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-18T11:28:29.675 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-18T11:28:29.675 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-18T11:28:29.676 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-18T11:28:29.676 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-18T11:28:29.676 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:29.676 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-09-18T11:28:29.676 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:29.676 INFO:teuthology.orchestra.run.smithi119.stdout:Install 6 Packages 2024-09-18T11:28:29.676 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:29.677 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 7.0 M 2024-09-18T11:28:29.677 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 33 M 2024-09-18T11:28:29.677 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-09-18T11:28:30.115 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:20 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:30.201 INFO:teuthology.orchestra.run.smithi119.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.8 MB/s | 248 kB 00:00 2024-09-18T11:28:30.248 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:30.249 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:30.249 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-18T11:28:30.249 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:30.249 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-18T11:28:30.249 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 124 k 2024-09-18T11:28:30.249 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-18T11:28:30.249 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-18T11:28:30.249 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-18T11:28:30.249 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-18T11:28:30.250 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-18T11:28:30.250 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-18T11:28:30.250 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:30.250 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-18T11:28:30.250 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:30.250 INFO:teuthology.orchestra.run.smithi148.stdout:Install 6 Packages 2024-09-18T11:28:30.250 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:30.251 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 7.0 M 2024-09-18T11:28:30.251 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 33 M 2024-09-18T11:28:30.251 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-18T11:28:30.268 INFO:teuthology.orchestra.run.smithi119.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 8.1 MB/s | 548 kB 00:00 2024-09-18T11:28:30.293 INFO:teuthology.orchestra.run.smithi119.stdout:(3/6): ceph-mgr-cephadm-17.2.7-1645.g7061ea6c.e 537 kB/s | 124 kB 00:00 2024-09-18T11:28:30.318 INFO:teuthology.orchestra.run.smithi119.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-18T11:28:30.427 INFO:teuthology.orchestra.run.smithi119.stdout:(5/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-18T11:28:30.461 INFO:teuthology.orchestra.run.smithi119.stdout:(6/6): python3-typing-extensions-4.12.2-2.el9.n 445 kB/s | 74 kB 00:00 2024-09-18T11:28:30.464 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:30.464 INFO:teuthology.orchestra.run.smithi119.stdout:Total 8.9 MB/s | 7.0 MB 00:00 2024-09-18T11:28:30.534 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-09-18T11:28:30.545 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-09-18T11:28:30.546 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-09-18T11:28:30.563 INFO:teuthology.orchestra.run.smithi148.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-09-18T11:28:30.588 INFO:teuthology.orchestra.run.smithi148.stdout:(2/6): ceph-mgr-cephadm-17.2.7-1645.g7061ea6c.e 548 kB/s | 124 kB 00:00 2024-09-18T11:28:30.638 INFO:teuthology.orchestra.run.smithi148.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-18T11:28:30.644 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-09-18T11:28:30.644 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-09-18T11:28:30.664 INFO:teuthology.orchestra.run.smithi148.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 769 kB/s | 58 kB 00:00 2024-09-18T11:28:30.689 INFO:teuthology.orchestra.run.smithi148.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-18T11:28:30.951 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-09-18T11:28:31.126 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-18T11:28:31.179 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-18T11:28:31.705 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-18T11:28:31.807 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-18T11:28:31.864 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-18T11:28:31.891 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 6/6 2024-09-18T11:28:31.997 INFO:teuthology.orchestra.run.smithi148.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.6 MB/s | 6.0 MB 00:01 2024-09-18T11:28:32.001 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:32.001 INFO:teuthology.orchestra.run.smithi148.stdout:Total 4.0 MB/s | 7.0 MB 00:01 2024-09-18T11:28:32.077 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-18T11:28:32.089 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-18T11:28:32.089 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-18T11:28:32.189 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-18T11:28:32.190 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-18T11:28:32.493 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-18T11:28:32.675 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-18T11:28:32.724 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 6/6 2024-09-18T11:28:32.725 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 1/6 2024-09-18T11:28:32.725 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-18T11:28:32.725 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-18T11:28:32.725 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-18T11:28:32.725 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-18T11:28:32.725 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-18T11:28:33.003 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-18T11:28:33.004 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:33.004 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-09-18T11:28:33.004 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T11:28:33.004 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-18T11:28:33.004 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-18T11:28:33.004 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-18T11:28:33.004 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-18T11:28:33.004 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-18T11:28:33.004 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:33.004 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:28:33.258 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-18T11:28:33.270 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-fuse 2024-09-18T11:28:33.344 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-18T11:28:33.408 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-18T11:28:33.436 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 6/6 2024-09-18T11:28:33.841 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:24 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:28:33.972 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:28:33.973 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:33.973 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-09-18T11:28:33.973 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:33.973 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-09-18T11:28:33.973 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 813 k 2024-09-18T11:28:33.973 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-09-18T11:28:33.973 INFO:teuthology.orchestra.run.smithi119.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-18T11:28:33.973 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:33.973 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-09-18T11:28:33.974 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:33.974 INFO:teuthology.orchestra.run.smithi119.stdout:Install 2 Packages 2024-09-18T11:28:33.974 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:33.974 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 893 k 2024-09-18T11:28:33.974 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 2.6 M 2024-09-18T11:28:33.974 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-09-18T11:28:34.154 INFO:teuthology.orchestra.run.smithi119.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 610 kB/s | 80 kB 00:00 2024-09-18T11:28:34.192 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 6/6 2024-09-18T11:28:34.192 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 1/6 2024-09-18T11:28:34.192 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-18T11:28:34.192 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-18T11:28:34.192 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-18T11:28:34.193 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-18T11:28:34.304 INFO:teuthology.orchestra.run.smithi119.stdout:(2/2): ceph-fuse-17.2.7-1645.g7061ea6c.el9.x86_ 2.8 MB/s | 813 kB 00:00 2024-09-18T11:28:34.305 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:34.305 INFO:teuthology.orchestra.run.smithi119.stdout:Total 2.6 MB/s | 893 kB 00:00 2024-09-18T11:28:34.316 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-09-18T11:28:34.338 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-09-18T11:28:34.338 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-09-18T11:28:34.417 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-09-18T11:28:34.418 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-09-18T11:28:34.462 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-18T11:28:34.462 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:34.462 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-18T11:28:34.462 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T11:28:34.462 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-18T11:28:34.462 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-18T11:28:34.462 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-18T11:28:34.462 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-18T11:28:34.462 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-18T11:28:34.462 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:34.463 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:34.600 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-09-18T11:28:34.717 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-fuse 2024-09-18T11:28:34.750 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-18T11:28:34.819 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/2 2024-09-18T11:28:35.300 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:25 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:35.432 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:35.432 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-18T11:28:35.433 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:35.433 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-18T11:28:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 813 k 2024-09-18T11:28:35.433 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-18T11:28:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-18T11:28:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:35.433 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-18T11:28:35.433 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:35.433 INFO:teuthology.orchestra.run.smithi148.stdout:Install 2 Packages 2024-09-18T11:28:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:35.434 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 893 k 2024-09-18T11:28:35.434 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 2.6 M 2024-09-18T11:28:35.434 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-18T11:28:35.454 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/2 2024-09-18T11:28:35.454 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/2 2024-09-18T11:28:35.607 INFO:teuthology.orchestra.run.smithi148.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 522 kB/s | 80 kB 00:00 2024-09-18T11:28:35.705 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-18T11:28:35.705 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:35.705 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-09-18T11:28:35.706 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-18T11:28:35.706 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:35.706 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:28:35.741 INFO:teuthology.orchestra.run.smithi148.stdout:(2/2): ceph-fuse-17.2.7-1645.g7061ea6c.el9.x86_ 2.8 MB/s | 813 kB 00:00 2024-09-18T11:28:35.741 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:35.742 INFO:teuthology.orchestra.run.smithi148.stdout:Total 2.8 MB/s | 893 kB 00:00 2024-09-18T11:28:35.753 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-18T11:28:35.775 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-18T11:28:35.776 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-18T11:28:35.855 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-18T11:28:35.855 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-18T11:28:36.034 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-volume 2024-09-18T11:28:36.042 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-18T11:28:36.172 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-18T11:28:36.240 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/2 2024-09-18T11:28:36.639 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:27 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:28:36.774 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:28:36.775 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:36.775 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-09-18T11:28:36.775 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:36.775 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-09-18T11:28:36.775 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 270 k 2024-09-18T11:28:36.775 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:36.776 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-09-18T11:28:36.776 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:36.776 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-09-18T11:28:36.776 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:36.776 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 270 k 2024-09-18T11:28:36.776 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 1.3 M 2024-09-18T11:28:36.776 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-09-18T11:28:36.902 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 2/2 2024-09-18T11:28:36.902 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/2 2024-09-18T11:28:37.000 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-volume-17.2.7-1645.g7061ea6c.el9.noarch.rp 1.2 MB/s | 270 kB 00:00 2024-09-18T11:28:37.001 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:37.001 INFO:teuthology.orchestra.run.smithi119.stdout:Total 1.2 MB/s | 270 kB 00:00 2024-09-18T11:28:37.001 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-09-18T11:28:37.010 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-09-18T11:28:37.010 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-09-18T11:28:37.039 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-09-18T11:28:37.040 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-09-18T11:28:37.171 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-18T11:28:37.171 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:37.171 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-18T11:28:37.171 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-18T11:28:37.171 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:37.171 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:37.253 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-09-18T11:28:37.319 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-volume-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T11:28:37.336 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-volume-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T11:28:37.336 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T11:28:37.336 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-18T11:28:37.336 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:37.495 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-volume 2024-09-18T11:28:38.085 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:28 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:38.222 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:38.223 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:38.223 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-18T11:28:38.223 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:38.223 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-18T11:28:38.223 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume noarch 2:17.2.7-1645.g7061ea6c.el9 ceph-noarch 270 k 2024-09-18T11:28:38.223 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:38.223 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-18T11:28:38.223 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:38.223 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-18T11:28:38.223 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:38.223 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 270 k 2024-09-18T11:28:38.223 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 1.3 M 2024-09-18T11:28:38.223 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-18T11:28:38.339 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-volume-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T11:28:38.340 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:38.340 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-09-18T11:28:38.340 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T11:28:38.340 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:38.340 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:28:38.437 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-volume-17.2.7-1645.g7061ea6c.el9.noarch.rp 1.2 MB/s | 270 kB 00:00 2024-09-18T11:28:38.438 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:38.438 INFO:teuthology.orchestra.run.smithi148.stdout:Total 1.2 MB/s | 270 kB 00:00 2024-09-18T11:28:38.438 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-18T11:28:38.446 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-18T11:28:38.446 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-18T11:28:38.471 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-18T11:28:38.471 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-18T11:28:38.616 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install librados-devel 2024-09-18T11:28:38.676 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-18T11:28:38.731 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-volume-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T11:28:38.746 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-volume-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T11:28:38.746 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T11:28:38.747 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-18T11:28:38.747 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:39.205 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:30 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:28:39.343 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:28:39.344 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:39.344 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repo Size 2024-09-18T11:28:39.344 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:39.344 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-09-18T11:28:39.344 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 126 k 2024-09-18T11:28:39.344 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:39.344 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-09-18T11:28:39.345 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:39.345 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-09-18T11:28:39.345 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:39.345 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 126 k 2024-09-18T11:28:39.345 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 448 k 2024-09-18T11:28:39.345 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-09-18T11:28:39.531 INFO:teuthology.orchestra.run.smithi119.stdout:librados-devel-17.2.7-1645.g7061ea6c.el9.x86_64 680 kB/s | 126 kB 00:00 2024-09-18T11:28:39.531 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:39.531 INFO:teuthology.orchestra.run.smithi119.stdout:Total 675 kB/s | 126 kB 00:00 2024-09-18T11:28:39.532 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-09-18T11:28:39.537 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-09-18T11:28:39.537 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-09-18T11:28:39.599 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-09-18T11:28:39.599 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-09-18T11:28:39.704 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-volume-2:17.2.7-1645.g7061ea6c.el9.noarch 1/1 2024-09-18T11:28:39.704 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:39.704 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-18T11:28:39.704 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume-2:17.2.7-1645.g7061ea6c.el9.noarch 2024-09-18T11:28:39.705 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:39.705 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:39.749 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-09-18T11:28:39.853 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librados-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:39.975 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install librados-devel 2024-09-18T11:28:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:40.553 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:40.553 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:40.554 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-09-18T11:28:40.554 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:28:40.554 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:40.554 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:28:40.570 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:30 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:40.703 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:40.704 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:40.704 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-09-18T11:28:40.704 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:40.705 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-18T11:28:40.705 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 126 k 2024-09-18T11:28:40.705 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:40.705 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-18T11:28:40.705 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:40.705 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-18T11:28:40.705 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:40.705 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 126 k 2024-09-18T11:28:40.705 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 448 k 2024-09-18T11:28:40.705 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-18T11:28:40.799 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install libcephfs2 2024-09-18T11:28:40.892 INFO:teuthology.orchestra.run.smithi148.stdout:librados-devel-17.2.7-1645.g7061ea6c.el9.x86_64 678 kB/s | 126 kB 00:00 2024-09-18T11:28:40.892 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:40.892 INFO:teuthology.orchestra.run.smithi148.stdout:Total 672 kB/s | 126 kB 00:00 2024-09-18T11:28:40.892 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-18T11:28:40.898 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-18T11:28:40.898 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-18T11:28:40.961 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-18T11:28:40.962 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-18T11:28:41.116 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-18T11:28:41.222 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librados-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:41.391 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:32 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:28:41.483 INFO:teuthology.orchestra.run.smithi119.stdout:Package libcephfs2-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T11:28:41.527 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:28:41.528 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-09-18T11:28:41.529 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:28:41.589 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install libcephfs-devel 2024-09-18T11:28:41.662 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:41.956 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:41.956 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:41.956 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-18T11:28:41.956 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:28:41.957 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:41.957 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:42.167 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:33 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:28:42.210 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install libcephfs2 2024-09-18T11:28:42.298 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:28:42.298 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:42.298 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repo Size 2024-09-18T11:28:42.299 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:42.299 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-09-18T11:28:42.299 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 25 k 2024-09-18T11:28:42.299 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:42.299 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-09-18T11:28:42.299 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:42.299 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-09-18T11:28:42.299 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:42.299 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 25 k 2024-09-18T11:28:42.299 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 110 k 2024-09-18T11:28:42.299 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-09-18T11:28:42.451 INFO:teuthology.orchestra.run.smithi119.stdout:libcephfs-devel-17.2.7-1645.g7061ea6c.el9.x86_6 165 kB/s | 25 kB 00:00 2024-09-18T11:28:42.451 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:42.451 INFO:teuthology.orchestra.run.smithi119.stdout:Total 163 kB/s | 25 kB 00:00 2024-09-18T11:28:42.452 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-09-18T11:28:42.455 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-09-18T11:28:42.455 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-09-18T11:28:42.456 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-09-18T11:28:42.457 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-09-18T11:28:42.545 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-09-18T11:28:42.688 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:42.811 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:32 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:42.902 INFO:teuthology.orchestra.run.smithi148.stdout:Package libcephfs2-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T11:28:42.948 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:42.949 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-18T11:28:42.949 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:43.027 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install libcephfs-devel 2024-09-18T11:28:43.070 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:43.301 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:43.301 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:43.301 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-09-18T11:28:43.301 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:28:43.301 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:43.301 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:28:43.481 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install librados2 2024-09-18T11:28:43.605 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:33 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:43.738 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:43.739 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:43.739 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-09-18T11:28:43.739 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:43.739 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-18T11:28:43.739 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 25 k 2024-09-18T11:28:43.739 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:43.740 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-18T11:28:43.740 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:43.740 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-18T11:28:43.740 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:43.740 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 25 k 2024-09-18T11:28:43.740 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 110 k 2024-09-18T11:28:43.741 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-18T11:28:43.895 INFO:teuthology.orchestra.run.smithi148.stdout:libcephfs-devel-17.2.7-1645.g7061ea6c.el9.x86_6 161 kB/s | 25 kB 00:00 2024-09-18T11:28:43.895 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:43.895 INFO:teuthology.orchestra.run.smithi148.stdout:Total 160 kB/s | 25 kB 00:00 2024-09-18T11:28:43.896 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-18T11:28:43.899 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-18T11:28:43.899 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-18T11:28:43.901 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-18T11:28:43.901 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-18T11:28:43.986 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-18T11:28:44.053 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:35 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:28:44.099 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:44.145 INFO:teuthology.orchestra.run.smithi119.stdout:Package librados2-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T11:28:44.185 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:28:44.186 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-09-18T11:28:44.186 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:28:44.255 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install librbd1 2024-09-18T11:28:44.444 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:44.719 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:44.720 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:44.720 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-18T11:28:44.720 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:28:44.720 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:44.720 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:44.834 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:35 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:28:44.899 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install librados2 2024-09-18T11:28:44.923 INFO:teuthology.orchestra.run.smithi119.stdout:Package librbd1-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T11:28:44.969 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:28:44.969 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-09-18T11:28:44.969 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:28:45.031 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install python3-rados 2024-09-18T11:28:45.477 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:35 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:45.567 INFO:teuthology.orchestra.run.smithi148.stdout:Package librados2-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T11:28:45.610 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:45.611 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-18T11:28:45.611 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:45.612 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:36 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:28:45.698 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install librbd1 2024-09-18T11:28:45.702 INFO:teuthology.orchestra.run.smithi119.stdout:Package python3-rados-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T11:28:45.745 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:28:45.746 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-09-18T11:28:45.746 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:28:45.813 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install python3-rgw 2024-09-18T11:28:46.278 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:36 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:46.367 INFO:teuthology.orchestra.run.smithi148.stdout:Package librbd1-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T11:28:46.389 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:37 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:28:46.410 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:46.411 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-18T11:28:46.411 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:46.478 INFO:teuthology.orchestra.run.smithi119.stdout:Package python3-rgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T11:28:46.493 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-rados 2024-09-18T11:28:46.520 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:28:46.520 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-09-18T11:28:46.521 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:28:46.586 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install python3-cephfs 2024-09-18T11:28:47.074 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:37 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:47.158 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:38 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:28:47.164 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-rados-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T11:28:47.207 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:47.208 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-18T11:28:47.208 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:47.246 INFO:teuthology.orchestra.run.smithi119.stdout:Package python3-cephfs-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T11:28:47.285 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-rgw 2024-09-18T11:28:47.289 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:28:47.290 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-09-18T11:28:47.290 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:28:47.351 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install python3-rbd 2024-09-18T11:28:47.868 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:37 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:47.935 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:38 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:28:47.958 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-rgw-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T11:28:48.002 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:48.003 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-18T11:28:48.003 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:48.023 INFO:teuthology.orchestra.run.smithi119.stdout:Package python3-rbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T11:28:48.066 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:28:48.067 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-09-18T11:28:48.067 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:28:48.084 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-cephfs 2024-09-18T11:28:48.134 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install rbd-fuse 2024-09-18T11:28:48.667 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:38 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:48.711 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:39 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:28:48.757 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-cephfs-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T11:28:48.801 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:48.802 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-18T11:28:48.802 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:48.842 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:28:48.842 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:48.843 INFO:teuthology.orchestra.run.smithi119.stdout: Package Architecture Version Repository Size 2024-09-18T11:28:48.843 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:48.843 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-09-18T11:28:48.843 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 82 k 2024-09-18T11:28:48.843 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:48.843 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-09-18T11:28:48.843 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:48.843 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-09-18T11:28:48.843 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:48.843 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 82 k 2024-09-18T11:28:48.843 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 222 k 2024-09-18T11:28:48.843 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-09-18T11:28:48.884 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-rbd 2024-09-18T11:28:49.037 INFO:teuthology.orchestra.run.smithi119.stdout:rbd-fuse-17.2.7-1645.g7061ea6c.el9.x86_64.rpm 423 kB/s | 82 kB 00:00 2024-09-18T11:28:49.038 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:49.038 INFO:teuthology.orchestra.run.smithi119.stdout:Total 420 kB/s | 82 kB 00:00 2024-09-18T11:28:49.038 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-09-18T11:28:49.045 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-09-18T11:28:49.045 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-09-18T11:28:49.106 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-09-18T11:28:49.107 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-09-18T11:28:49.264 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-09-18T11:28:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:49.462 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:39 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:49.553 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-rbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 is already installed. 2024-09-18T11:28:49.595 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:49.596 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-09-18T11:28:49.596 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:49.675 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install rbd-fuse 2024-09-18T11:28:49.734 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:49.979 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:49.980 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:49.980 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-09-18T11:28:49.980 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:28:49.980 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:49.980 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:28:50.250 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:40 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:50.254 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install rbd-mirror 2024-09-18T11:28:50.381 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:50.382 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:50.382 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-09-18T11:28:50.382 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:50.382 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-18T11:28:50.382 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 82 k 2024-09-18T11:28:50.382 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:50.382 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-18T11:28:50.382 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:50.382 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-18T11:28:50.383 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:50.383 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 82 k 2024-09-18T11:28:50.383 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 222 k 2024-09-18T11:28:50.383 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-18T11:28:50.573 INFO:teuthology.orchestra.run.smithi148.stdout:rbd-fuse-17.2.7-1645.g7061ea6c.el9.x86_64.rpm 432 kB/s | 82 kB 00:00 2024-09-18T11:28:50.574 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:50.574 INFO:teuthology.orchestra.run.smithi148.stdout:Total 429 kB/s | 82 kB 00:00 2024-09-18T11:28:50.574 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-18T11:28:50.580 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-18T11:28:50.580 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-18T11:28:50.643 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-18T11:28:50.643 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-18T11:28:50.804 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-18T11:28:50.847 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:41 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:28:50.895 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:50.985 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:28:50.985 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:50.986 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repo Size 2024-09-18T11:28:50.986 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:50.986 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-09-18T11:28:50.986 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 2.9 M 2024-09-18T11:28:50.986 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:50.986 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-09-18T11:28:50.986 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:50.986 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-09-18T11:28:50.986 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:50.986 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 2.9 M 2024-09-18T11:28:50.986 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 12 M 2024-09-18T11:28:50.987 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-09-18T11:28:51.277 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:51.333 INFO:teuthology.orchestra.run.smithi119.stdout:rbd-mirror-17.2.7-1645.g7061ea6c.el9.x86_64.rpm 8.5 MB/s | 2.9 MB 00:00 2024-09-18T11:28:51.333 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:51.334 INFO:teuthology.orchestra.run.smithi119.stdout:Total 8.5 MB/s | 2.9 MB 00:00 2024-09-18T11:28:51.334 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-09-18T11:28:51.342 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-09-18T11:28:51.342 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-09-18T11:28:51.426 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-09-18T11:28:51.426 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-09-18T11:28:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:51.537 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-18T11:28:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:28:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:51.538 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:51.800 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install rbd-mirror 2024-09-18T11:28:51.862 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-09-18T11:28:51.933 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-mirror-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:51.959 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:51.959 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T11:28:51.960 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-18T11:28:51.960 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T11:28:51.960 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T11:28:51.960 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:52.412 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:42 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:52.551 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:52.552 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:52.552 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-09-18T11:28:52.552 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:52.552 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-18T11:28:52.553 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 2.9 M 2024-09-18T11:28:52.553 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:52.553 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-18T11:28:52.553 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:52.553 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-18T11:28:52.553 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:52.554 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 2.9 M 2024-09-18T11:28:52.554 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 12 M 2024-09-18T11:28:52.554 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-18T11:28:52.868 INFO:teuthology.orchestra.run.smithi148.stdout:rbd-mirror-17.2.7-1645.g7061ea6c.el9.x86_64.rpm 9.3 MB/s | 2.9 MB 00:00 2024-09-18T11:28:52.868 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:52.869 INFO:teuthology.orchestra.run.smithi148.stdout:Total 9.3 MB/s | 2.9 MB 00:00 2024-09-18T11:28:52.869 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-18T11:28:52.878 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-18T11:28:52.878 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-18T11:28:52.958 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-18T11:28:52.958 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-18T11:28:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-mirror-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:53.118 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-09-18T11:28:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:28:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:53.118 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:28:53.371 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-18T11:28:53.394 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install rbd-nbd 2024-09-18T11:28:53.461 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-mirror-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:53.487 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:53.487 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T11:28:53.487 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-18T11:28:53.487 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T11:28:53.487 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T11:28:53.487 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:53.988 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:44 ago on Wed 18 Sep 2024 11:27:09 AM UTC. 2024-09-18T11:28:54.121 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-18T11:28:54.121 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:54.121 INFO:teuthology.orchestra.run.smithi119.stdout: Package Architecture Version Repository Size 2024-09-18T11:28:54.121 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:54.122 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-09-18T11:28:54.122 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 165 k 2024-09-18T11:28:54.122 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:54.122 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-09-18T11:28:54.122 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-09-18T11:28:54.122 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-09-18T11:28:54.122 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:54.122 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 165 k 2024-09-18T11:28:54.122 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 477 k 2024-09-18T11:28:54.122 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-09-18T11:28:54.336 INFO:teuthology.orchestra.run.smithi119.stdout:rbd-nbd-17.2.7-1645.g7061ea6c.el9.x86_64.rpm 772 kB/s | 165 kB 00:00 2024-09-18T11:28:54.337 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:54.337 INFO:teuthology.orchestra.run.smithi119.stdout:Total 766 kB/s | 165 kB 00:00 2024-09-18T11:28:54.337 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-09-18T11:28:54.345 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-09-18T11:28:54.345 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-09-18T11:28:54.407 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-09-18T11:28:54.408 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-09-18T11:28:54.568 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-mirror-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:54.568 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:54.568 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-18T11:28:54.568 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:28:54.568 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:54.568 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:54.569 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-09-18T11:28:54.658 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-nbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:54.832 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install rbd-nbd 2024-09-18T11:28:55.065 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:55.354 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-nbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:55.354 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:55.354 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-09-18T11:28:55.354 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:28:55.354 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:28:55.354 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-18T11:28:55.430 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:45 ago on Wed 18 Sep 2024 11:27:10 AM UTC. 2024-09-18T11:28:55.562 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-18T11:28:55.563 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:55.563 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-09-18T11:28:55.563 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:55.563 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-18T11:28:55.563 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd x86_64 2:17.2.7-1645.g7061ea6c.el9 ceph 165 k 2024-09-18T11:28:55.563 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:55.563 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-18T11:28:55.563 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-18T11:28:55.564 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-09-18T11:28:55.564 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:55.564 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 165 k 2024-09-18T11:28:55.564 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 477 k 2024-09-18T11:28:55.564 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-18T11:28:55.598 DEBUG:teuthology.parallel:result is None 2024-09-18T11:28:55.763 INFO:teuthology.orchestra.run.smithi148.stdout:rbd-nbd-17.2.7-1645.g7061ea6c.el9.x86_64.rpm 830 kB/s | 165 kB 00:00 2024-09-18T11:28:55.763 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-18T11:28:55.764 INFO:teuthology.orchestra.run.smithi148.stdout:Total 825 kB/s | 165 kB 00:00 2024-09-18T11:28:55.764 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-18T11:28:55.771 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-18T11:28:55.771 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-18T11:28:55.834 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-18T11:28:55.834 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-18T11:28:55.984 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-18T11:28:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-nbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:56.506 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:56.771 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-nbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 1/1 2024-09-18T11:28:56.771 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:56.771 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-18T11:28:56.771 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd-2:17.2.7-1645.g7061ea6c.el9.x86_64 2024-09-18T11:28:56.771 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:28:56.771 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-18T11:28:57.017 DEBUG:teuthology.parallel:result is None 2024-09-18T11:28:57.017 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T11:28:57.157 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-18T11:28:57.183 INFO:teuthology.orchestra.run.smithi119.stdout:17.2.7-1645.g7061ea6c.el9 2024-09-18T11:28:57.183 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1645.g7061ea6c.el9 2024-09-18T11:28:57.183 INFO:teuthology.task.install:The correct ceph version 17.2.7-1645.g7061ea6c is installed. 2024-09-18T11:28:57.185 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T11:28:57.326 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-18T11:28:57.349 INFO:teuthology.orchestra.run.smithi148.stdout:17.2.7-1645.g7061ea6c.el9 2024-09-18T11:28:57.349 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1645.g7061ea6c.el9 2024-09-18T11:28:57.349 INFO:teuthology.task.install:The correct ceph version 17.2.7-1645.g7061ea6c is installed. 2024-09-18T11:28:57.351 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-18T11:28:57.351 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:28:57.352 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T11:28:57.382 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-18T11:28:57.382 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T11:28:57.416 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-18T11:28:57.416 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:28:57.416 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T11:28:57.451 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T11:28:57.519 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-18T11:28:57.520 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T11:28:57.546 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T11:28:57.612 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-18T11:28:57.613 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:28:57.613 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T11:28:57.643 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T11:28:57.711 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-18T11:28:57.711 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T11:28:57.775 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T11:28:57.800 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-18T11:28:57.801 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:28:57.801 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T11:28:57.828 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T11:28:57.895 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-18T11:28:57.895 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T11:28:57.920 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T11:28:57.985 INFO:teuthology.run_tasks:Running task ceph... 2024-09-18T11:28:58.065 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-18T11:28:58.065 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /var/log/ceph 2024-09-18T11:28:58.068 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /var/log/ceph 2024-09-18T11:28:58.092 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-18T11:28:58.093 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T11:28:58.135 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T11:28:58.160 INFO:tasks.ceph:Creating extra log directories... 2024-09-18T11:28:58.161 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T11:28:58.202 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T11:28:58.239 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-18T11:28:58.239 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd inject bad map crc probability': 0.1, 'osd map cache size': 1, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub during recovery': False, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'osd_map_cache_size', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum', '\\(POOL_APP_NOT_ENABLED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-18T11:28:58.239 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910958', 'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy', 'description': 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/dedup-io-snaps}', 'email': 'skanta@redhat.com', 'first_in_suite': False, 'job_id': '7910958', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd inject bad map crc probability': 0.1, 'osd map cache size': 1, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub during recovery': False, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'osd_map_cache_size', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}}, 'owner': 'scheduled_skanta@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 4631, 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-bharath11-testing-2024-09-18-1115-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_7061ea6ca50908d104e9f3979dc615e118d9e153/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153', 'targets': {'smithi119.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGeluLGInTfcczQVTIBdUwFHyIAxfqoH5hSiTxx2kZGTqjQyUB7X/7WCg37rkGRGbYbe4gbjP7Ht/cGkDfU3hUY=', 'smithi148.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPEvG5PCDgOEEu1FawCRpv/8Hrtl/9DnDENp5gwRmc/UV6DTBdL1at90zbJx627NMpbATe7k4wAJQnApNGlsTlk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd inject bad map crc probability': 0.1, 'osd map cache size': 1, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub during recovery': False, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'osd_map_cache_size', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153', 'cluster': 'ceph'}}, {'exec': {'mon.a': ['while ! ceph balancer status ; do sleep 1 ; done', 'ceph balancer mode crush-compat', 'ceph balancer on']}}, {'thrashosds': {'chance_pgnum_grow': 0.25, 'chance_pgnum_shrink': 0.25, 'chance_pgpnum_fix': 0.25, 'chance_test_map_discontinuity': 2, 'timeout': 1800}}, {'exec': {'client.0': ['sudo ceph osd pool create low_tier 4']}}, {'rados': {'clients': ['client.0'], 'dedup_chunk_algo': 'fastcdc', 'dedup_chunk_size': '131072', 'enable_dedup': True, 'low_tier_pool': 'low_tier', 'objects': 50, 'op_weights': {'read': 100, 'rollback': 10, 'set_chunk': 30, 'snap_create': 10, 'snap_remove': 10, 'tier_evict': 10, 'tier_flush': 5, 'tier_promote': 10, 'write': 50}, 'ops': 1500, 'set_chunk': True}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-18_09:13:00', 'tube': 'smithi', 'user': 'skanta', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-18T11:28:58.239 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T11:28:58.270 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T11:28:58.284 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T11:28:58.326 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T11:28:58.351 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:28:58.351 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/scratch_devs of=/dev/stdout 2024-09-18T11:28:58.406 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-18T11:28:58.407 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_1 2024-09-18T11:28:58.463 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-18T11:28:58.463 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T11:28:58.463 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-18T11:28:58.463 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T11:28:58.463 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T11:28:58.463 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-09-18 11:28:52.214872913 +0000 2024-09-18T11:28:58.463 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-09-18 11:26:29.625280180 +0000 2024-09-18T11:28:58.464 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-09-18 11:26:29.625280180 +0000 2024-09-18T11:28:58.464 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-09-18 11:26:29.625280180 +0000 2024-09-18T11:28:58.464 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-18T11:28:58.527 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-09-18T11:28:58.527 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-09-18T11:28:58.527 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000207935 s, 2.5 MB/s 2024-09-18T11:28:58.528 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-18T11:28:58.584 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_2 2024-09-18T11:28:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-18T11:28:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T11:28:58.640 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-18T11:28:58.640 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T11:28:58.640 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T11:28:58.640 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-09-18 11:28:52.214872913 +0000 2024-09-18T11:28:58.640 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-09-18 11:26:29.950272098 +0000 2024-09-18T11:28:58.641 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-09-18 11:26:29.950272098 +0000 2024-09-18T11:28:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-09-18 11:26:29.950272098 +0000 2024-09-18T11:28:58.641 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-18T11:28:58.703 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-09-18T11:28:58.703 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-09-18T11:28:58.703 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000189308 s, 2.7 MB/s 2024-09-18T11:28:58.704 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-18T11:28:58.760 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_3 2024-09-18T11:28:58.815 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-18T11:28:58.815 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T11:28:58.815 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-18T11:28:58.815 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T11:28:58.815 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T11:28:58.815 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-09-18 11:28:52.215872888 +0000 2024-09-18T11:28:58.816 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-09-18 11:26:30.276263992 +0000 2024-09-18T11:28:58.816 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-09-18 11:26:30.276263992 +0000 2024-09-18T11:28:58.816 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-09-18 11:26:30.276263992 +0000 2024-09-18T11:28:58.816 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-18T11:28:58.878 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-09-18T11:28:58.878 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-09-18T11:28:58.878 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000180847 s, 2.8 MB/s 2024-09-18T11:28:58.879 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-18T11:28:58.936 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_4 2024-09-18T11:28:58.991 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-18T11:28:58.991 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T11:28:58.991 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-18T11:28:58.992 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T11:28:58.992 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T11:28:58.992 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-09-18 11:28:52.215872888 +0000 2024-09-18T11:28:58.992 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-09-18 11:26:30.599255960 +0000 2024-09-18T11:28:58.992 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-09-18 11:26:30.599255960 +0000 2024-09-18T11:28:58.992 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-09-18 11:26:30.599255960 +0000 2024-09-18T11:28:58.992 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-18T11:28:59.055 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-09-18T11:28:59.055 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-09-18T11:28:59.055 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000191795 s, 2.7 MB/s 2024-09-18T11:28:59.056 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-18T11:28:59.112 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T11:28:59.112 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-18T11:28:59.112 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2024-09-18T11:28:59.131 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-18T11:28:59.131 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2024-09-18T11:28:59.189 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-18T11:28:59.189 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T11:28:59.189 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-09-18T11:28:59.189 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T11:28:59.189 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T11:28:59.190 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-18 11:28:53.761342881 +0000 2024-09-18T11:28:59.190 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-18 11:26:29.634127111 +0000 2024-09-18T11:28:59.190 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-18 11:26:29.634127111 +0000 2024-09-18T11:28:59.190 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-18 11:26:29.634127111 +0000 2024-09-18T11:28:59.190 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-18T11:28:59.255 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-18T11:28:59.255 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-18T11:28:59.256 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000220119 s, 2.3 MB/s 2024-09-18T11:28:59.257 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-18T11:28:59.315 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2024-09-18T11:28:59.372 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-18T11:28:59.372 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T11:28:59.372 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-09-18T11:28:59.372 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T11:28:59.373 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T11:28:59.373 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-18 11:28:53.761342881 +0000 2024-09-18T11:28:59.373 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-18 11:26:29.959119367 +0000 2024-09-18T11:28:59.373 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-18 11:26:29.959119367 +0000 2024-09-18T11:28:59.373 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-18 11:26:29.959119367 +0000 2024-09-18T11:28:59.373 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-18T11:28:59.437 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-18T11:28:59.437 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-18T11:28:59.437 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000239886 s, 2.1 MB/s 2024-09-18T11:28:59.439 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-18T11:28:59.496 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2024-09-18T11:28:59.552 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-18T11:28:59.552 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T11:28:59.552 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 707 Links: 1 2024-09-18T11:28:59.552 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T11:28:59.553 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T11:28:59.553 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-18 11:28:53.762342858 +0000 2024-09-18T11:28:59.553 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-18 11:26:30.292111433 +0000 2024-09-18T11:28:59.553 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-18 11:26:30.292111433 +0000 2024-09-18T11:28:59.553 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-18 11:26:30.292111433 +0000 2024-09-18T11:28:59.553 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-18T11:28:59.616 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-18T11:28:59.616 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-18T11:28:59.616 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000228605 s, 2.2 MB/s 2024-09-18T11:28:59.618 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-18T11:28:59.675 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2024-09-18T11:28:59.731 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-18T11:28:59.731 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T11:28:59.731 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 727 Links: 1 2024-09-18T11:28:59.731 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T11:28:59.731 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T11:28:59.732 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-18 11:28:53.762342858 +0000 2024-09-18T11:28:59.732 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-18 11:26:30.611103832 +0000 2024-09-18T11:28:59.732 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-18 11:26:30.611103832 +0000 2024-09-18T11:28:59.732 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-18 11:26:30.611103832 +0000 2024-09-18T11:28:59.732 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-18T11:28:59.794 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-18T11:28:59.794 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-18T11:28:59.794 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000210408 s, 2.4 MB/s 2024-09-18T11:28:59.796 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-18T11:28:59.852 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T11:28:59.852 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi148.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-18T11:28:59.853 INFO:tasks.ceph:Generating config... 2024-09-18T11:28:59.854 INFO:tasks.ceph:[osd] debug monc = 20 2024-09-18T11:28:59.854 INFO:tasks.ceph:[osd] bluefs allocator = hybrid 2024-09-18T11:28:59.854 INFO:tasks.ceph:[osd] bluestore allocator = hybrid 2024-09-18T11:28:59.854 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-18T11:28:59.854 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-18T11:28:59.854 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-18T11:28:59.855 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-18T11:28:59.855 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-18T11:28:59.855 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-18T11:28:59.855 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-18T11:28:59.855 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-18T11:28:59.855 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-18T11:28:59.855 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-18T11:28:59.855 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-18T11:28:59.855 INFO:tasks.ceph:[osd] osd backoff on degraded = True 2024-09-18T11:28:59.855 INFO:tasks.ceph:[osd] osd backoff on peering = True 2024-09-18T11:28:59.855 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-18T11:28:59.855 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-18T11:28:59.855 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-18T11:28:59.856 INFO:tasks.ceph:[osd] osd heartbeat use min delay socket = True 2024-09-18T11:28:59.856 INFO:tasks.ceph:[osd] osd inject bad map crc probability = 0.1 2024-09-18T11:28:59.856 INFO:tasks.ceph:[osd] osd map cache size = 1 2024-09-18T11:28:59.856 INFO:tasks.ceph:[osd] osd max backfills = 6 2024-09-18T11:28:59.856 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-18T11:28:59.856 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-18T11:28:59.856 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-18T11:28:59.856 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-18T11:28:59.856 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-18T11:28:59.856 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-18T11:28:59.856 INFO:tasks.ceph:[osd] osd scrub during recovery = False 2024-09-18T11:28:59.856 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2024-09-18T11:28:59.856 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2024-09-18T11:28:59.856 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-18T11:28:59.857 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-18T11:28:59.857 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-09-18T11:28:59.857 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-18T11:28:59.857 INFO:tasks.ceph:[global] ms type = async 2024-09-18T11:28:59.857 INFO:tasks.ceph:[global] osd_async_recovery_min_cost = 1 2024-09-18T11:28:59.857 INFO:tasks.ceph:[global] osd_object_clean_region_max_num_intervals = 1000 2024-09-18T11:28:59.857 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2024-09-18T11:28:59.857 INFO:tasks.ceph:[global] osd_pool_default_size = 2 2024-09-18T11:28:59.857 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-18T11:28:59.857 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-18T11:28:59.857 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-18T11:28:59.857 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-18T11:28:59.857 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-18T11:28:59.858 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-09-18T11:28:59.858 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-09-18T11:28:59.858 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-09-18T11:28:59.858 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-09-18T11:28:59.858 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-18T11:28:59.858 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-09-18T11:28:59.858 INFO:tasks.ceph:Setting up mon.a... 2024-09-18T11:28:59.858 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-18T11:28:59.910 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/ceph.keyring 2024-09-18T11:28:59.912 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-18T11:28:59.953 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T11:29:00.019 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.119'), ('mon.c', '[v2:172.21.15.119:3301,v1:172.21.15.119:6790]'), ('mon.b', '172.21.15.148')] 2024-09-18T11:29:00.019 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.119,[v2:172.21.15.119:3301,v1:172.21.15.119:6790],172.21.15.148', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd inject bad map crc probability': 0.1, 'osd map cache size': 1, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub during recovery': False, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-18T11:29:00.020 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:29:00.020 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-18T11:29:00.075 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.119 --addv c '[v2:172.21.15.119:3301,v1:172.21.15.119:6790]' --add b 172.21.15.148 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-18T11:29:00.158 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-18T11:29:00.158 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool: generated fsid f4cca259-b87a-45cc-8995-7e9fca891a9f 2024-09-18T11:29:00.158 INFO:teuthology.orchestra.run.smithi119.stdout:setting min_mon_release = octopus 2024-09-18T11:29:00.159 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 0 2024-09-18T11:29:00.159 INFO:teuthology.orchestra.run.smithi119.stdout:fsid f4cca259-b87a-45cc-8995-7e9fca891a9f 2024-09-18T11:29:00.159 INFO:teuthology.orchestra.run.smithi119.stdout:last_changed 2024-09-18T11:29:00.157969+0000 2024-09-18T11:29:00.159 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-09-18T11:29:00.157969+0000 2024-09-18T11:29:00.159 INFO:teuthology.orchestra.run.smithi119.stdout:min_mon_release 15 (octopus) 2024-09-18T11:29:00.159 INFO:teuthology.orchestra.run.smithi119.stdout:election_strategy: 1 2024-09-18T11:29:00.159 INFO:teuthology.orchestra.run.smithi119.stdout:0: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.a 2024-09-18T11:29:00.159 INFO:teuthology.orchestra.run.smithi119.stdout:1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-09-18T11:29:00.159 INFO:teuthology.orchestra.run.smithi119.stdout:2: [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] mon.c 2024-09-18T11:29:00.159 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-18T11:29:00.160 DEBUG:teuthology.orchestra.run.smithi119:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-18T11:29:00.216 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f4cca259-b87a-45cc-8995-7e9fca891a9f... 2024-09-18T11:29:00.217 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T11:29:00.258 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T11:29:00.299 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2024-09-18T11:29:00.299 INFO:teuthology.orchestra.run.smithi119.stdout: chdir = "" 2024-09-18T11:29:00.299 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T11:29:00.299 INFO:teuthology.orchestra.run.smithi119.stdout: auth supported = cephx 2024-09-18T11:29:00.299 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.299 INFO:teuthology.orchestra.run.smithi119.stdout: filestore xattr use omap = true 2024-09-18T11:29:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: mon clock drift allowed = 1.000 2024-09-18T11:29:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: osd crush chooseleaf type = 0 2024-09-18T11:29:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: auth debug = true 2024-09-18T11:29:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on old message = true 2024-09-18T11:29:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on bug = true 2024-09-18T11:29:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T11:29:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: mon pg warn max object skew = 0 2024-09-18T11:29:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T11:29:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T11:29:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default size = 2 2024-09-18T11:29:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow primary affinity = true 2024-09-18T11:29:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow pg remap = true 2024-09-18T11:29:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on legacy crush tunables = false 2024-09-18T11:29:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on crush straw calc version zero = false 2024-09-18T11:29:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on no sortbitwise = false 2024-09-18T11:29:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on osd down out interval zero = false 2024-09-18T11:29:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on too few osds = false 2024-09-18T11:29:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T11:29:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T11:29:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: mon_allow_pool_size_one = true 2024-09-18T11:29:00.301 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T11:29:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: osd default data pool replay window = 5 2024-09-18T11:29:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: mon allow pool delete = true 2024-09-18T11:29:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: mon cluster log file level = debug 2024-09-18T11:29:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: debug asserts on shutdown = true 2024-09-18T11:29:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: mon health detail to clog = false 2024-09-18T11:29:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: mon host = "172.21.15.119,[v2:172.21.15.119:3301,v1:172.21.15.119:6790],172.21.15.148" 2024-09-18T11:29:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: mon client directed command retry = 5 2024-09-18T11:29:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: mon election default strategy = 1 2024-09-18T11:29:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: ms inject socket failures = 5000 2024-09-18T11:29:00.302 INFO:teuthology.orchestra.run.smithi119.stdout: ms type = async 2024-09-18T11:29:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: osd_async_recovery_min_cost = 1 2024-09-18T11:29:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-09-18T11:29:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_min_size = 2 2024-09-18T11:29:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_size = 2 2024-09-18T11:29:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = f4cca259-b87a-45cc-8995-7e9fca891a9f 2024-09-18T11:29:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.303 INFO:teuthology.orchestra.run.smithi119.stdout:[osd] 2024-09-18T11:29:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: osd journal size = 100 2024-09-18T11:29:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub load threshold = 5.0 2024-09-18T11:29:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub max interval = 120 2024-09-18T11:29:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: osd mclock profile = high_recovery_ops 2024-09-18T11:29:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: osd recover clone overlap = true 2024-09-18T11:29:00.303 INFO:teuthology.orchestra.run.smithi119.stdout: osd recovery max chunk = 1048576 2024-09-18T11:29:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug shutdown = true 2024-09-18T11:29:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug op order = true 2024-09-18T11:29:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify stray on activate = true 2024-09-18T11:29:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: osd open classes on start = true 2024-09-18T11:29:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug pg log writeout = true 2024-09-18T11:29:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: osd deep scrub update digest min age = 30 2024-09-18T11:29:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: osd map max advance = 10 2024-09-18T11:29:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: journal zero on create = true 2024-09-18T11:29:00.304 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: filestore ondisk finisher threads = 3 2024-09-18T11:29:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: filestore apply finisher threads = 3 2024-09-18T11:29:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: bdev debug aio = true 2024-09-18T11:29:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug misdirected ops = true 2024-09-18T11:29:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: debug monc = 20 2024-09-18T11:29:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: bluefs allocator = hybrid 2024-09-18T11:29:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore allocator = hybrid 2024-09-18T11:29:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore block size = 96636764160 2024-09-18T11:29:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore fsck on mount = True 2024-09-18T11:29:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore zero block detection = True 2024-09-18T11:29:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluefs = 20 2024-09-18T11:29:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluestore = 20 2024-09-18T11:29:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-09-18T11:29:00.305 INFO:teuthology.orchestra.run.smithi119.stdout: debug osd = 20 2024-09-18T11:29:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: debug rocksdb = 10 2024-09-18T11:29:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd backfillfull_ratio = 0.85 2024-09-18T11:29:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd full ratio = 0.9 2024-09-18T11:29:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd nearfull ratio = 0.8 2024-09-18T11:29:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: osd backoff on degraded = True 2024-09-18T11:29:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: osd backoff on peering = True 2024-09-18T11:29:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify cached snaps = True 2024-09-18T11:29:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify missing on start = True 2024-09-18T11:29:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: osd failsafe full ratio = 0.95 2024-09-18T11:29:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: osd heartbeat use min delay socket = True 2024-09-18T11:29:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: osd inject bad map crc probability = 0.1 2024-09-18T11:29:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: osd map cache size = 1 2024-09-18T11:29:00.306 INFO:teuthology.orchestra.run.smithi119.stdout: osd max backfills = 6 2024-09-18T11:29:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: osd max markdown count = 1000 2024-09-18T11:29:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: osd mclock override recovery settings = True 2024-09-18T11:29:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: osd objectstore = bluestore 2024-09-18T11:29:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: osd op queue = debug_random 2024-09-18T11:29:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: osd op queue cut off = debug_random 2024-09-18T11:29:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub during recovery = False 2024-09-18T11:29:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub min interval = 60 2024-09-18T11:29:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: osd shutdown pgref assert = True 2024-09-18T11:29:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.307 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr] 2024-09-18T11:29:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-09-18T11:29:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: debug mgr = 20 2024-09-18T11:29:00.307 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-09-18T11:29:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-09-18T11:29:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-09-18T11:29:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-09-18T11:29:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: mgr/telemetry/nag = false 2024-09-18T11:29:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.308 INFO:teuthology.orchestra.run.smithi119.stdout:[mon] 2024-09-18T11:29:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-09-18T11:29:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-09-18T11:29:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: debug paxos = 20 2024-09-18T11:29:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-09-18T11:29:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: mon data avail warn = 5 2024-09-18T11:29:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: mon mgr mkfs grace = 240 2024-09-18T11:29:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-09-18T11:29:00.308 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd reporter subtree level = osd 2024-09-18T11:29:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd prime pg temp = true 2024-09-18T11:29:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-09-18T11:29:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T11:29:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T11:29:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T11:29:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T11:29:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T11:29:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T11:29:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: # 1m isn't quite enough 2024-09-18T11:29:00.309 INFO:teuthology.orchestra.run.smithi119.stdout: mon_down_mkfs_grace = 2m 2024-09-18T11:29:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_filestore_osds = false 2024-09-18T11:29:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: mon min osdmap epochs = 50 2024-09-18T11:29:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: mon osdmap full prune interval = 2 2024-09-18T11:29:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: mon osdmap full prune min = 15 2024-09-18T11:29:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: mon osdmap full prune txsize = 2 2024-09-18T11:29:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: mon scrub interval = 300 2024-09-18T11:29:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: paxos service trim min = 10 2024-09-18T11:29:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:00.310 INFO:teuthology.orchestra.run.smithi119.stdout:[client] 2024-09-18T11:29:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: rgw cache enabled = true 2024-09-18T11:29:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable ops log = true 2024-09-18T11:29:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable usage log = true 2024-09-18T11:29:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T11:29:00.311 INFO:teuthology.orchestra.run.smithi119.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T11:29:00.311 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.a] 2024-09-18T11:29:00.311 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.c] 2024-09-18T11:29:00.311 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.b] 2024-09-18T11:29:00.311 INFO:teuthology.orchestra.run.smithi148.stdout:[global] 2024-09-18T11:29:00.311 INFO:teuthology.orchestra.run.smithi148.stdout: chdir = "" 2024-09-18T11:29:00.311 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T11:29:00.312 INFO:teuthology.orchestra.run.smithi148.stdout: auth supported = cephx 2024-09-18T11:29:00.312 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.312 INFO:teuthology.orchestra.run.smithi148.stdout: filestore xattr use omap = true 2024-09-18T11:29:00.312 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.312 INFO:teuthology.orchestra.run.smithi148.stdout: mon clock drift allowed = 1.000 2024-09-18T11:29:00.312 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.312 INFO:teuthology.orchestra.run.smithi148.stdout: osd crush chooseleaf type = 0 2024-09-18T11:29:00.312 INFO:teuthology.orchestra.run.smithi148.stdout: auth debug = true 2024-09-18T11:29:00.312 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.312 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on old message = true 2024-09-18T11:29:00.312 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on bug = true 2024-09-18T11:29:00.312 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.312 INFO:teuthology.orchestra.run.smithi148.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T11:29:00.313 INFO:teuthology.orchestra.run.smithi148.stdout: mon pg warn max object skew = 0 2024-09-18T11:29:00.313 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.313 INFO:teuthology.orchestra.run.smithi148.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T11:29:00.313 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T11:29:00.313 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.313 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default size = 2 2024-09-18T11:29:00.313 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.313 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow primary affinity = true 2024-09-18T11:29:00.313 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow pg remap = true 2024-09-18T11:29:00.313 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on legacy crush tunables = false 2024-09-18T11:29:00.313 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on crush straw calc version zero = false 2024-09-18T11:29:00.313 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on no sortbitwise = false 2024-09-18T11:29:00.313 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on osd down out interval zero = false 2024-09-18T11:29:00.313 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on too few osds = false 2024-09-18T11:29:00.314 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T11:29:00.314 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T11:29:00.314 INFO:teuthology.orchestra.run.smithi148.stdout: mon_allow_pool_size_one = true 2024-09-18T11:29:00.314 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.314 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T11:29:00.314 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.314 INFO:teuthology.orchestra.run.smithi148.stdout: osd default data pool replay window = 5 2024-09-18T11:29:00.314 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.314 INFO:teuthology.orchestra.run.smithi148.stdout: mon allow pool delete = true 2024-09-18T11:29:00.314 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.314 INFO:teuthology.orchestra.run.smithi148.stdout: mon cluster log file level = debug 2024-09-18T11:29:00.314 INFO:teuthology.orchestra.run.smithi148.stdout: debug asserts on shutdown = true 2024-09-18T11:29:00.315 INFO:teuthology.orchestra.run.smithi148.stdout: mon health detail to clog = false 2024-09-18T11:29:00.315 INFO:teuthology.orchestra.run.smithi148.stdout: mon host = "172.21.15.119,[v2:172.21.15.119:3301,v1:172.21.15.119:6790],172.21.15.148" 2024-09-18T11:29:00.315 INFO:teuthology.orchestra.run.smithi148.stdout: mon client directed command retry = 5 2024-09-18T11:29:00.315 INFO:teuthology.orchestra.run.smithi148.stdout: mon election default strategy = 1 2024-09-18T11:29:00.315 INFO:teuthology.orchestra.run.smithi148.stdout: ms inject socket failures = 5000 2024-09-18T11:29:00.315 INFO:teuthology.orchestra.run.smithi148.stdout: ms type = async 2024-09-18T11:29:00.315 INFO:teuthology.orchestra.run.smithi148.stdout: osd_async_recovery_min_cost = 1 2024-09-18T11:29:00.315 INFO:teuthology.orchestra.run.smithi148.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-09-18T11:29:00.315 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_min_size = 2 2024-09-18T11:29:00.315 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_size = 2 2024-09-18T11:29:00.315 INFO:teuthology.orchestra.run.smithi148.stdout: fsid = f4cca259-b87a-45cc-8995-7e9fca891a9f 2024-09-18T11:29:00.315 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.315 INFO:teuthology.orchestra.run.smithi148.stdout:[osd] 2024-09-18T11:29:00.315 INFO:teuthology.orchestra.run.smithi148.stdout: osd journal size = 100 2024-09-18T11:29:00.316 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.316 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub load threshold = 5.0 2024-09-18T11:29:00.316 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub max interval = 120 2024-09-18T11:29:00.316 INFO:teuthology.orchestra.run.smithi148.stdout: osd mclock profile = high_recovery_ops 2024-09-18T11:29:00.316 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.316 INFO:teuthology.orchestra.run.smithi148.stdout: osd recover clone overlap = true 2024-09-18T11:29:00.316 INFO:teuthology.orchestra.run.smithi148.stdout: osd recovery max chunk = 1048576 2024-09-18T11:29:00.316 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.316 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug shutdown = true 2024-09-18T11:29:00.316 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug op order = true 2024-09-18T11:29:00.316 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug verify stray on activate = true 2024-09-18T11:29:00.316 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.316 INFO:teuthology.orchestra.run.smithi148.stdout: osd open classes on start = true 2024-09-18T11:29:00.317 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug pg log writeout = true 2024-09-18T11:29:00.317 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.317 INFO:teuthology.orchestra.run.smithi148.stdout: osd deep scrub update digest min age = 30 2024-09-18T11:29:00.317 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.317 INFO:teuthology.orchestra.run.smithi148.stdout: osd map max advance = 10 2024-09-18T11:29:00.317 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.317 INFO:teuthology.orchestra.run.smithi148.stdout: journal zero on create = true 2024-09-18T11:29:00.317 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.317 INFO:teuthology.orchestra.run.smithi148.stdout: filestore ondisk finisher threads = 3 2024-09-18T11:29:00.317 INFO:teuthology.orchestra.run.smithi148.stdout: filestore apply finisher threads = 3 2024-09-18T11:29:00.317 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.317 INFO:teuthology.orchestra.run.smithi148.stdout: bdev debug aio = true 2024-09-18T11:29:00.317 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug misdirected ops = true 2024-09-18T11:29:00.317 INFO:teuthology.orchestra.run.smithi148.stdout: debug monc = 20 2024-09-18T11:29:00.318 INFO:teuthology.orchestra.run.smithi148.stdout: bluefs allocator = hybrid 2024-09-18T11:29:00.318 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore allocator = hybrid 2024-09-18T11:29:00.318 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore block size = 96636764160 2024-09-18T11:29:00.318 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore fsck on mount = True 2024-09-18T11:29:00.318 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore zero block detection = True 2024-09-18T11:29:00.318 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluefs = 20 2024-09-18T11:29:00.318 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluestore = 20 2024-09-18T11:29:00.318 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-18T11:29:00.318 INFO:teuthology.orchestra.run.smithi148.stdout: debug osd = 20 2024-09-18T11:29:00.318 INFO:teuthology.orchestra.run.smithi148.stdout: debug rocksdb = 10 2024-09-18T11:29:00.318 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd backfillfull_ratio = 0.85 2024-09-18T11:29:00.318 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd full ratio = 0.9 2024-09-18T11:29:00.318 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd nearfull ratio = 0.8 2024-09-18T11:29:00.319 INFO:teuthology.orchestra.run.smithi148.stdout: osd backoff on degraded = True 2024-09-18T11:29:00.319 INFO:teuthology.orchestra.run.smithi148.stdout: osd backoff on peering = True 2024-09-18T11:29:00.319 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug verify cached snaps = True 2024-09-18T11:29:00.319 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug verify missing on start = True 2024-09-18T11:29:00.319 INFO:teuthology.orchestra.run.smithi148.stdout: osd failsafe full ratio = 0.95 2024-09-18T11:29:00.319 INFO:teuthology.orchestra.run.smithi148.stdout: osd heartbeat use min delay socket = True 2024-09-18T11:29:00.319 INFO:teuthology.orchestra.run.smithi148.stdout: osd inject bad map crc probability = 0.1 2024-09-18T11:29:00.319 INFO:teuthology.orchestra.run.smithi148.stdout: osd map cache size = 1 2024-09-18T11:29:00.319 INFO:teuthology.orchestra.run.smithi148.stdout: osd max backfills = 6 2024-09-18T11:29:00.319 INFO:teuthology.orchestra.run.smithi148.stdout: osd max markdown count = 1000 2024-09-18T11:29:00.319 INFO:teuthology.orchestra.run.smithi148.stdout: osd mclock override recovery settings = True 2024-09-18T11:29:00.319 INFO:teuthology.orchestra.run.smithi148.stdout: osd objectstore = bluestore 2024-09-18T11:29:00.319 INFO:teuthology.orchestra.run.smithi148.stdout: osd op queue = debug_random 2024-09-18T11:29:00.319 INFO:teuthology.orchestra.run.smithi148.stdout: osd op queue cut off = debug_random 2024-09-18T11:29:00.320 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub during recovery = False 2024-09-18T11:29:00.320 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub min interval = 60 2024-09-18T11:29:00.320 INFO:teuthology.orchestra.run.smithi148.stdout: osd shutdown pgref assert = True 2024-09-18T11:29:00.320 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.320 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr] 2024-09-18T11:29:00.320 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-18T11:29:00.320 INFO:teuthology.orchestra.run.smithi148.stdout: debug mgr = 20 2024-09-18T11:29:00.320 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-09-18T11:29:00.320 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-09-18T11:29:00.320 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-09-18T11:29:00.320 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-09-18T11:29:00.320 INFO:teuthology.orchestra.run.smithi148.stdout: mgr/telemetry/nag = false 2024-09-18T11:29:00.320 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.320 INFO:teuthology.orchestra.run.smithi148.stdout:[mon] 2024-09-18T11:29:00.321 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-18T11:29:00.321 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-09-18T11:29:00.321 INFO:teuthology.orchestra.run.smithi148.stdout: debug paxos = 20 2024-09-18T11:29:00.321 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-09-18T11:29:00.321 INFO:teuthology.orchestra.run.smithi148.stdout: mon data avail warn = 5 2024-09-18T11:29:00.321 INFO:teuthology.orchestra.run.smithi148.stdout: mon mgr mkfs grace = 240 2024-09-18T11:29:00.321 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-09-18T11:29:00.321 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd reporter subtree level = osd 2024-09-18T11:29:00.321 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd prime pg temp = true 2024-09-18T11:29:00.321 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-09-18T11:29:00.321 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.321 INFO:teuthology.orchestra.run.smithi148.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T11:29:00.321 INFO:teuthology.orchestra.run.smithi148.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T11:29:00.321 INFO:teuthology.orchestra.run.smithi148.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T11:29:00.322 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.322 INFO:teuthology.orchestra.run.smithi148.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T11:29:00.322 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T11:29:00.322 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T11:29:00.322 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.322 INFO:teuthology.orchestra.run.smithi148.stdout: # 1m isn't quite enough 2024-09-18T11:29:00.322 INFO:teuthology.orchestra.run.smithi148.stdout: mon_down_mkfs_grace = 2m 2024-09-18T11:29:00.322 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.322 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_filestore_osds = false 2024-09-18T11:29:00.322 INFO:teuthology.orchestra.run.smithi148.stdout: mon min osdmap epochs = 50 2024-09-18T11:29:00.322 INFO:teuthology.orchestra.run.smithi148.stdout: mon osdmap full prune interval = 2 2024-09-18T11:29:00.322 INFO:teuthology.orchestra.run.smithi148.stdout: mon osdmap full prune min = 15 2024-09-18T11:29:00.322 INFO:teuthology.orchestra.run.smithi148.stdout: mon osdmap full prune txsize = 2 2024-09-18T11:29:00.322 INFO:teuthology.orchestra.run.smithi148.stdout: mon scrub interval = 300 2024-09-18T11:29:00.323 INFO:teuthology.orchestra.run.smithi148.stdout: paxos service trim min = 10 2024-09-18T11:29:00.323 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-18T11:29:00.323 INFO:teuthology.orchestra.run.smithi148.stdout:[client] 2024-09-18T11:29:00.323 INFO:teuthology.orchestra.run.smithi148.stdout: rgw cache enabled = true 2024-09-18T11:29:00.323 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable ops log = true 2024-09-18T11:29:00.323 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable usage log = true 2024-09-18T11:29:00.323 INFO:teuthology.orchestra.run.smithi148.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T11:29:00.323 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T11:29:00.323 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.a] 2024-09-18T11:29:00.323 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.c] 2024-09-18T11:29:00.323 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.b] 2024-09-18T11:29:00.324 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-18T11:29:00.325 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-18T11:29:00.414 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-18T11:29:00.414 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:29:00.414 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-18T11:29:00.429 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:29:00.430 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-18T11:29:00.485 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi119.front.sepia.ceph.com 2024-09-18T11:29:00.485 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:29:00.485 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T11:29:00.485 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T11:29:00.559 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:29:00.559 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T11:29:00.615 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi148.front.sepia.ceph.com 2024-09-18T11:29:00.615 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-18T11:29:00.615 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T11:29:00.616 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T11:29:00.652 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-18T11:29:00.652 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T11:29:00.707 INFO:tasks.ceph:Setting up mon nodes... 2024-09-18T11:29:00.708 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-18T11:29:00.708 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-18T11:29:00.755 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-18T11:29:00.758 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-18T11:29:00.819 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-18T11:29:00.821 INFO:tasks.ceph:Setting up mds nodes... 2024-09-18T11:29:00.821 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-18T11:29:00.822 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-18T11:29:00.858 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-18T11:29:00.871 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-18T11:29:00.907 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-18T11:29:00.920 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-18T11:29:00.920 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi148.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-18T11:29:00.920 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-18T11:29:00.945 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T11:29:00.945 INFO:tasks.ceph:role: osd.0 2024-09-18T11:29:00.945 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-18T11:29:00.948 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-18T11:29:01.017 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T11:29:01.017 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T11:29:01.017 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T11:29:01.017 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T11:29:01.017 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T11:29:01.017 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-09-18T11:29:01.017 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T11:29:01.017 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T11:29:01.017 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T11:29:01.017 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T11:29:01.530 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-09-18T11:29:01.532 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-09-18T11:29:01.533 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-18T11:29:01.679 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-18T11:29:01.716 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-18T11:29:01.785 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T11:29:01.785 INFO:tasks.ceph:role: osd.1 2024-09-18T11:29:01.785 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-18T11:29:01.785 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-18T11:29:01.859 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T11:29:01.859 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T11:29:01.859 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T11:29:01.859 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T11:29:01.859 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T11:29:01.859 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-09-18T11:29:01.859 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T11:29:01.859 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T11:29:01.859 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T11:29:01.859 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T11:29:02.306 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-09-18T11:29:02.308 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-09-18T11:29:02.309 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-18T11:29:02.405 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-18T11:29:02.434 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-18T11:29:02.499 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T11:29:02.499 INFO:tasks.ceph:role: osd.2 2024-09-18T11:29:02.499 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-18T11:29:02.499 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-18T11:29:02.573 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T11:29:02.573 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T11:29:02.573 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T11:29:02.573 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T11:29:02.573 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T11:29:02.574 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-09-18T11:29:02.574 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T11:29:02.574 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T11:29:02.574 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T11:29:02.574 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T11:29:03.067 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-09-18T11:29:03.069 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-09-18T11:29:03.069 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-18T11:29:03.229 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-18T11:29:03.262 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-18T11:29:03.328 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T11:29:03.328 INFO:tasks.ceph:role: osd.3 2024-09-18T11:29:03.328 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-18T11:29:03.328 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-18T11:29:03.402 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T11:29:03.402 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T11:29:03.402 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T11:29:03.402 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T11:29:03.402 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T11:29:03.402 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-09-18T11:29:03.402 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T11:29:03.403 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T11:29:03.403 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T11:29:03.403 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T11:29:03.911 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-09-18T11:29:03.913 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-09-18T11:29:03.914 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-18T11:29:04.076 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-18T11:29:04.111 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T11:29:04.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:04.242+0000 7f8988522440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-18T11:29:04.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:04.242+0000 7f8988522440 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-18T11:29:04.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:04.242+0000 7f8988522440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-18T11:29:04.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:04.242+0000 7f8988522440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-18T11:29:04.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:04.242+0000 7f8988522440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-18T11:29:04.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:04.243+0000 7f8988522440 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-18T11:29:04.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:04.772+0000 7f8988522440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:05.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:05.541+0000 7f8988522440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:06.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:06.303+0000 7f8988522440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:07.104 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-18T11:29:07.133 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T11:29:07.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:07.218+0000 7f07ade24440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-18T11:29:07.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:07.219+0000 7f07ade24440 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-18T11:29:07.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:07.219+0000 7f07ade24440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-18T11:29:07.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:07.219+0000 7f07ade24440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-18T11:29:07.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:07.219+0000 7f07ade24440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-18T11:29:07.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:07.219+0000 7f07ade24440 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-18T11:29:07.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:07.747+0000 7f07ade24440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:08.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:08.509+0000 7f07ade24440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:09.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:09.275+0000 7f07ade24440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:10.076 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-18T11:29:10.106 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T11:29:10.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:10.190+0000 7f14eaa0d440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-18T11:29:10.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:10.191+0000 7f14eaa0d440 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-18T11:29:10.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:10.191+0000 7f14eaa0d440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-18T11:29:10.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:10.191+0000 7f14eaa0d440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-18T11:29:10.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:10.191+0000 7f14eaa0d440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-18T11:29:10.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:10.191+0000 7f14eaa0d440 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-18T11:29:10.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:10.717+0000 7f14eaa0d440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:11.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:11.481+0000 7f14eaa0d440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:12.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:12.245+0000 7f14eaa0d440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:13.045 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-18T11:29:13.070 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T11:29:13.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:13.151+0000 7fd877922440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-18T11:29:13.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:13.151+0000 7fd877922440 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-18T11:29:13.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:13.151+0000 7fd877922440 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-18T11:29:13.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:13.151+0000 7fd877922440 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-18T11:29:13.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:13.151+0000 7fd877922440 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-18T11:29:13.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:13.152+0000 7fd877922440 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-18T11:29:13.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:13.681+0000 7fd877922440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:14.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:14.445+0000 7fd877922440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:15.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:29:15.207+0000 7fd877922440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:16.008 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-18T11:29:16.076 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-18T11:29:16.107 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T11:29:16.107 INFO:tasks.ceph:role: osd.4 2024-09-18T11:29:16.108 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi148.front.sepia.ceph.com 2024-09-18T11:29:16.108 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-18T11:29:16.184 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T11:29:16.184 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T11:29:16.184 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T11:29:16.185 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T11:29:16.185 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T11:29:16.185 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-09-18T11:29:16.185 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T11:29:16.185 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T11:29:16.185 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T11:29:16.185 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T11:29:16.681 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-09-18T11:29:16.684 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-09-18T11:29:16.684 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-18T11:29:16.830 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-18T11:29:16.863 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-18T11:29:16.930 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T11:29:16.931 INFO:tasks.ceph:role: osd.5 2024-09-18T11:29:16.931 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi148.front.sepia.ceph.com 2024-09-18T11:29:16.931 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-18T11:29:17.005 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T11:29:17.005 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T11:29:17.005 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T11:29:17.006 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T11:29:17.006 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T11:29:17.006 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-09-18T11:29:17.006 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T11:29:17.006 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T11:29:17.006 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T11:29:17.006 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T11:29:17.458 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-09-18T11:29:17.460 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-09-18T11:29:17.460 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-18T11:29:17.553 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-18T11:29:17.582 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-18T11:29:17.647 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T11:29:17.647 INFO:tasks.ceph:role: osd.6 2024-09-18T11:29:17.647 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi148.front.sepia.ceph.com 2024-09-18T11:29:17.647 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-18T11:29:17.719 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T11:29:17.719 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T11:29:17.719 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T11:29:17.720 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T11:29:17.720 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T11:29:17.720 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-09-18T11:29:17.720 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T11:29:17.720 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T11:29:17.720 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T11:29:17.720 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T11:29:18.231 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-09-18T11:29:18.233 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-09-18T11:29:18.233 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-18T11:29:18.356 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-18T11:29:18.387 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-18T11:29:18.453 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T11:29:18.453 INFO:tasks.ceph:role: osd.7 2024-09-18T11:29:18.453 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi148.front.sepia.ceph.com 2024-09-18T11:29:18.453 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-18T11:29:18.525 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T11:29:18.525 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T11:29:18.525 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T11:29:18.525 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T11:29:18.526 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T11:29:18.526 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-09-18T11:29:18.526 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T11:29:18.526 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T11:29:18.526 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T11:29:18.526 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T11:29:19.023 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-09-18T11:29:19.025 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-09-18T11:29:19.025 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-18T11:29:19.143 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-18T11:29:19.173 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T11:29:19.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:19.268+0000 7f04a491c440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-18T11:29:19.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:19.269+0000 7f04a491c440 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-18T11:29:19.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:19.269+0000 7f04a491c440 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-18T11:29:19.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:19.269+0000 7f04a491c440 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-18T11:29:19.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:19.269+0000 7f04a491c440 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-18T11:29:19.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:19.270+0000 7f04a491c440 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-18T11:29:19.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:19.798+0000 7f04a491c440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:20.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:20.567+0000 7f04a491c440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:21.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:21.332+0000 7f04a491c440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:22.129 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-18T11:29:22.159 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T11:29:22.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:22.244+0000 7fc6bae8d440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-18T11:29:22.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:22.244+0000 7fc6bae8d440 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-18T11:29:22.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:22.244+0000 7fc6bae8d440 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-18T11:29:22.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:22.244+0000 7fc6bae8d440 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-18T11:29:22.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:22.244+0000 7fc6bae8d440 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-18T11:29:22.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:22.244+0000 7fc6bae8d440 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-18T11:29:22.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:22.768+0000 7fc6bae8d440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:23.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:23.532+0000 7fc6bae8d440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:24.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:24.297+0000 7fc6bae8d440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:25.090 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-18T11:29:25.119 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T11:29:25.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:25.204+0000 7f1b80430440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-18T11:29:25.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:25.204+0000 7f1b80430440 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-18T11:29:25.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:25.204+0000 7f1b80430440 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-18T11:29:25.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:25.204+0000 7f1b80430440 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-18T11:29:25.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:25.204+0000 7f1b80430440 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-18T11:29:25.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:25.205+0000 7f1b80430440 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-18T11:29:25.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:25.731+0000 7f1b80430440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:26.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:26.494+0000 7f1b80430440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:27.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:27.256+0000 7f1b80430440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:28.056 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-18T11:29:28.083 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T11:29:28.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:28.166+0000 7f85b7366440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-18T11:29:28.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:28.167+0000 7f85b7366440 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-18T11:29:28.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:28.167+0000 7f85b7366440 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-18T11:29:28.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:28.167+0000 7f85b7366440 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-18T11:29:28.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:28.167+0000 7f85b7366440 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-18T11:29:28.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:28.167+0000 7f85b7366440 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-18T11:29:28.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:28.690+0000 7f85b7366440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:29.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:29.459+0000 7f85b7366440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:30.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-18T11:29:30.222+0000 7f85b7366440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T11:29:31.018 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-18T11:29:31.047 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-18T11:29:31.048 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:29:31.048 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-18T11:29:31.078 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:29:31.078 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-18T11:29:31.146 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:29:31.146 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-18T11:29:31.214 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:29:31.214 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-18T11:29:31.281 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:29:31.281 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-18T11:29:31.348 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-18T11:29:31.348 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-18T11:29:31.378 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-18T11:29:31.378 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-18T11:29:31.446 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-18T11:29:31.446 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-18T11:29:31.514 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-18T11:29:31.514 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-18T11:29:31.581 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-18T11:29:31.581 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-18T11:29:31.647 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:29:31.647 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-18T11:29:31.665 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-18T11:29:31.665 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-18T11:29:31.703 INFO:tasks.ceph:Adding keys to all mons... 2024-09-18T11:29:31.703 DEBUG:teuthology.orchestra.run.smithi119:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-18T11:29:31.707 DEBUG:teuthology.orchestra.run.smithi148:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-18T11:29:31.746 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.y] 2024-09-18T11:29:31.746 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQB8uepmJKH3LBAA+S61uxLJzyO1U6q4vVGOsA== 2024-09-18T11:29:31.747 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.0] 2024-09-18T11:29:31.747 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCAuepmb3Z9DhAAmUofRfNekdceVgN7CLFsDw== 2024-09-18T11:29:31.747 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.1] 2024-09-18T11:29:31.747 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCDuepmFh8WDRAAp7WZm6silbeJIHxEn4dYnw== 2024-09-18T11:29:31.747 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.2] 2024-09-18T11:29:31.747 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCGuepmVDdoCxAAD4gZiW5PltqhpOdpyPukBA== 2024-09-18T11:29:31.747 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.3] 2024-09-18T11:29:31.747 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCJuepmDM0NCRAA5FJM2bQqfrzyuWq2DSFdaw== 2024-09-18T11:29:31.747 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.x] 2024-09-18T11:29:31.747 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQB8uepmuazOMBAACZn9lMREGd13svWRtVh6eg== 2024-09-18T11:29:31.747 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.4] 2024-09-18T11:29:31.747 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCPuepmxlAMEBAAK00bRL44GUXnBZjg/LOokA== 2024-09-18T11:29:31.747 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.5] 2024-09-18T11:29:31.748 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCSuepmSyqZDhAASRTNQz842eFaSmdlOKYMQQ== 2024-09-18T11:29:31.748 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.6] 2024-09-18T11:29:31.748 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCVuepmPAY5DBAAPuSxsusQdo0KoEzkJAVgeA== 2024-09-18T11:29:31.748 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.7] 2024-09-18T11:29:31.748 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQCYuepmr5T9CRAAq9Ogx3+/Ndf/HmMCzCR+tA== 2024-09-18T11:29:31.748 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2024-09-18T11:29:31.748 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQB8uepmNtopMxAAWSNqur/qA5f2hyPnNEALHw== 2024-09-18T11:29:31.748 INFO:teuthology.orchestra.run.smithi119.stdout:[client.1] 2024-09-18T11:29:31.748 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQB8uepmUMUUNhAA913BpiLubyh90E38e3UMCg== 2024-09-18T11:29:31.767 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr.y] 2024-09-18T11:29:31.767 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQB8uepmJKH3LBAA+S61uxLJzyO1U6q4vVGOsA== 2024-09-18T11:29:31.767 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.0] 2024-09-18T11:29:31.767 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCAuepmb3Z9DhAAmUofRfNekdceVgN7CLFsDw== 2024-09-18T11:29:31.767 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.1] 2024-09-18T11:29:31.767 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCDuepmFh8WDRAAp7WZm6silbeJIHxEn4dYnw== 2024-09-18T11:29:31.767 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.2] 2024-09-18T11:29:31.767 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCGuepmVDdoCxAAD4gZiW5PltqhpOdpyPukBA== 2024-09-18T11:29:31.767 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.3] 2024-09-18T11:29:31.768 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCJuepmDM0NCRAA5FJM2bQqfrzyuWq2DSFdaw== 2024-09-18T11:29:31.768 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr.x] 2024-09-18T11:29:31.768 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQB8uepmuazOMBAACZn9lMREGd13svWRtVh6eg== 2024-09-18T11:29:31.768 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.4] 2024-09-18T11:29:31.768 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCPuepmxlAMEBAAK00bRL44GUXnBZjg/LOokA== 2024-09-18T11:29:31.768 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.5] 2024-09-18T11:29:31.768 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCSuepmSyqZDhAASRTNQz842eFaSmdlOKYMQQ== 2024-09-18T11:29:31.768 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.6] 2024-09-18T11:29:31.768 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCVuepmPAY5DBAAPuSxsusQdo0KoEzkJAVgeA== 2024-09-18T11:29:31.768 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.7] 2024-09-18T11:29:31.769 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCYuepmr5T9CRAAq9Ogx3+/Ndf/HmMCzCR+tA== 2024-09-18T11:29:31.769 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-09-18T11:29:31.769 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQB8uepmNtopMxAAWSNqur/qA5f2hyPnNEALHw== 2024-09-18T11:29:31.769 INFO:teuthology.orchestra.run.smithi148.stdout:[client.1] 2024-09-18T11:29:31.769 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQB8uepmUMUUNhAA913BpiLubyh90E38e3UMCg== 2024-09-18T11:29:31.770 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T11:29:31.791 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T11:29:31.856 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T11:29:31.858 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T11:29:31.904 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T11:29:31.906 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T11:29:31.952 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T11:29:31.954 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T11:29:32.000 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T11:29:32.003 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T11:29:32.049 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T11:29:32.051 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T11:29:32.097 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T11:29:32.099 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T11:29:32.145 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T11:29:32.147 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T11:29:32.193 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T11:29:32.195 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T11:29:32.240 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T11:29:32.243 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T11:29:32.289 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T11:29:32.291 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T11:29:32.339 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T11:29:32.341 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T11:29:32.390 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-18T11:29:32.390 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-18T11:29:32.414 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T11:29:32.506 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-18T11:29:32.531 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-18T11:29:32.595 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T11:29:32.686 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-18T11:29:32.710 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-18T11:29:32.736 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T11:29:32.826 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-18T11:29:32.851 DEBUG:teuthology.orchestra.run.smithi119:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-18T11:29:32.853 DEBUG:teuthology.orchestra.run.smithi148:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-18T11:29:32.906 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-18T11:29:32.907 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-18T11:29:32.907 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-18T11:29:32.909 INFO:tasks.ceph.mon.a:Started 2024-09-18T11:29:32.909 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-18T11:29:32.909 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-18T11:29:32.911 INFO:tasks.ceph.mon.c:Started 2024-09-18T11:29:32.912 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-18T11:29:32.912 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-18T11:29:32.949 INFO:tasks.ceph.mon.b:Started 2024-09-18T11:29:32.949 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-18T11:29:32.949 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-18T11:29:32.949 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-18T11:29:32.951 INFO:tasks.ceph.mgr.y:Started 2024-09-18T11:29:32.952 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-18T11:29:32.952 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-18T11:29:32.954 INFO:tasks.ceph.mgr.x:Started 2024-09-18T11:29:32.954 INFO:tasks.ceph:Setting crush tunables to default 2024-09-18T11:29:32.954 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd crush tunables default 2024-09-18T11:29:38.302 INFO:teuthology.orchestra.run.smithi119.stderr:adjusted tunables profile to default 2024-09-18T11:29:38.318 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-18T11:29:38.319 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:29:38.319 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-18T11:29:38.335 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:38.333+0000 7fd49e02cec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-18T11:29:38.346 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:29:38.346 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-18T11:29:38.376 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:38.374+0000 7fd49e02cec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-18T11:29:38.401 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:29:38.401 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-18T11:29:38.428 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:38.426+0000 7fd49e02cec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-18T11:29:38.452 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-18T11:29:38.452 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-18T11:29:38.517 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-18T11:29:38.517 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-18T11:29:38.544 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:38.542+0000 7f0ecedd9ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-18T11:29:38.545 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-18T11:29:38.545 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-18T11:29:38.585 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:38.584+0000 7f0ecedd9ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-18T11:29:38.591 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:38.589+0000 7fd49e02cec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-18T11:29:38.610 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-18T11:29:38.610 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-18T11:29:38.638 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:38.636+0000 7f0ecedd9ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-18T11:29:38.662 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-18T11:29:38.662 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-18T11:29:38.697 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:38.695+0000 7fd49e02cec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-18T11:29:38.728 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 12faced2-7888-4265-ad12-815efe36466c 0 2024-09-18T11:29:38.783 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:38.782+0000 7f0ecedd9ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-18T11:29:38.896 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:38.894+0000 7f0ecedd9ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-18T11:29:38.942 INFO:teuthology.orchestra.run.smithi148.stdout:0 2024-09-18T11:29:38.953 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 20fee29d-b047-46ae-aa60-88e897fbb0fb 1 2024-09-18T11:29:38.978 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:38.976+0000 7fd49e02cec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-18T11:29:39.025 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:39.023+0000 7fd49e02cec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-18T11:29:39.154 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:39.152+0000 7fd49e02cec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-18T11:29:39.171 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-09-18T11:29:39.182 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 21a296f3-b259-4841-9a17-4cd21b302b36 2 2024-09-18T11:29:39.197 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:39.196+0000 7f0ecedd9ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-18T11:29:39.255 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:39.254+0000 7f0ecedd9ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-18T11:29:39.359 INFO:teuthology.orchestra.run.smithi148.stdout:2 2024-09-18T11:29:39.370 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 59044f30-32e1-4942-9a8e-202c1def221a 3 2024-09-18T11:29:39.393 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:39.392+0000 7f0ecedd9ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-18T11:29:39.569 INFO:teuthology.orchestra.run.smithi148.stdout:3 2024-09-18T11:29:39.580 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 7d1ae3d0-8d69-4d3c-9e51-24fdd77a22a9 4 2024-09-18T11:29:39.754 INFO:teuthology.orchestra.run.smithi148.stdout:4 2024-09-18T11:29:39.765 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new b0dd3d30-b273-428c-a4a9-4c238abf45fc 5 2024-09-18T11:29:39.939 INFO:teuthology.orchestra.run.smithi148.stdout:5 2024-09-18T11:29:39.949 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 40290cb3-2334-4c68-8e6c-59d0d5455171 6 2024-09-18T11:29:40.082 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:40.080+0000 7fd49e02cec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-18T11:29:40.122 INFO:teuthology.orchestra.run.smithi148.stdout:6 2024-09-18T11:29:40.133 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 54802abe-ea1a-448e-85af-d1f14a45be27 7 2024-09-18T11:29:40.134 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:40.131+0000 7fd49e02cec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-18T11:29:40.307 INFO:teuthology.orchestra.run.smithi148.stdout:7 2024-09-18T11:29:40.318 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-18T11:29:40.319 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-18T11:29:40.321 INFO:tasks.ceph.osd.0:Started 2024-09-18T11:29:40.321 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-18T11:29:40.322 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-18T11:29:40.324 INFO:tasks.ceph.osd.1:Started 2024-09-18T11:29:40.324 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-18T11:29:40.324 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-18T11:29:40.327 INFO:tasks.ceph.osd.2:Started 2024-09-18T11:29:40.327 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-18T11:29:40.327 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-18T11:29:40.330 INFO:tasks.ceph.osd.3:Started 2024-09-18T11:29:40.330 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-18T11:29:40.330 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-18T11:29:40.333 INFO:tasks.ceph.osd.4:Started 2024-09-18T11:29:40.334 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-18T11:29:40.334 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-18T11:29:40.336 INFO:tasks.ceph.osd.5:Started 2024-09-18T11:29:40.336 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-18T11:29:40.337 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-18T11:29:40.339 INFO:tasks.ceph.osd.6:Started 2024-09-18T11:29:40.340 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-18T11:29:40.340 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-18T11:29:40.342 INFO:tasks.ceph.osd.7:Started 2024-09-18T11:29:40.343 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T11:29:40.380 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:40.379+0000 7f0ecedd9ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-18T11:29:40.438 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:40.437+0000 7f0ecedd9ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-18T11:29:40.521 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:40.521 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":10,"fsid":"f4cca259-b87a-45cc-8995-7e9fca891a9f","created":"2024-09-18T11:29:38.205487+0000","modified":"2024-09-18T11:29:40.301770+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T11:29:40.301770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"12faced2-7888-4265-ad12-815efe36466c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"20fee29d-b047-46ae-aa60-88e897fbb0fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"21a296f3-b259-4841-9a17-4cd21b302b36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"59044f30-32e1-4942-9a8e-202c1def221a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7d1ae3d0-8d69-4d3c-9e51-24fdd77a22a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b0dd3d30-b273-428c-a4a9-4c238abf45fc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"40290cb3-2334-4c68-8e6c-59d0d5455171","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"54802abe-ea1a-448e-85af-d1f14a45be27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T11:29:40.531 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-18T11:29:40.532 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-18T11:29:40.533 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:40.531+0000 7fd49e02cec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-18T11:29:40.615 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:40.612+0000 7fd49e02cec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-18T11:29:40.703 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:40.701+0000 7fd49e02cec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-18T11:29:40.748 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:40.745+0000 7fd49e02cec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-18T11:29:40.800 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:40.799+0000 7f0ecedd9ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-18T11:29:40.840 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:40.837+0000 7fd49e02cec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-18T11:29:40.880 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:40.879+0000 7f0ecedd9ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-18T11:29:40.891 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:40.889+0000 7fd49e02cec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-18T11:29:40.927 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:29:40.925+0000 7fa478210440 -1 Falling back to public interface 2024-09-18T11:29:40.929 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:29:40.927+0000 7f6c0317a440 -1 Falling back to public interface 2024-09-18T11:29:40.931 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:29:40.930+0000 7f3165f23440 -1 Falling back to public interface 2024-09-18T11:29:40.934 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:29:40.932+0000 7fb6a3419440 -1 Falling back to public interface 2024-09-18T11:29:40.935 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:29:40.933+0000 7f698af23440 -1 Falling back to public interface 2024-09-18T11:29:40.935 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:40.933+0000 7fd49e02cec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-18T11:29:40.939 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:29:40.937+0000 7f2d52b21440 -1 Falling back to public interface 2024-09-18T11:29:40.949 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:29:40.948+0000 7fd28602f440 -1 Falling back to public interface 2024-09-18T11:29:40.953 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:29:40.951+0000 7fcb7660b440 -1 Falling back to public interface 2024-09-18T11:29:40.970 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:40.969+0000 7f0ecedd9ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-18T11:29:41.014 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:41.013+0000 7f0ecedd9ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-18T11:29:41.030 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:41.027+0000 7fd49e02cec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-18T11:29:41.078 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:41.076+0000 7fd49e02cec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-18T11:29:41.106 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:41.105+0000 7f0ecedd9ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-18T11:29:41.127 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:41.125+0000 7fd49e02cec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-18T11:29:41.157 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:41.156+0000 7f0ecedd9ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-18T11:29:41.193 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:41.191+0000 7fd49e02cec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-18T11:29:41.200 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:41.199+0000 7f0ecedd9ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-18T11:29:41.238 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:41.236+0000 7fd49e02cec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-18T11:29:41.281 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:41.278+0000 7fd49e02cec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-18T11:29:41.294 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:41.293+0000 7f0ecedd9ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-18T11:29:41.337 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T11:29:41.342 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:41.341+0000 7f0ecedd9ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-18T11:29:41.391 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:41.390+0000 7f0ecedd9ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-18T11:29:41.456 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:41.455+0000 7f0ecedd9ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-18T11:29:41.500 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:41.499+0000 7f0ecedd9ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-18T11:29:41.501 INFO:teuthology.misc.health.smithi119.stdout: 2024-09-18T11:29:41.501 INFO:teuthology.misc.health.smithi119.stdout:{"epoch":10,"fsid":"f4cca259-b87a-45cc-8995-7e9fca891a9f","created":"2024-09-18T11:29:38.205487+0000","modified":"2024-09-18T11:29:40.301770+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T11:29:40.301770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"12faced2-7888-4265-ad12-815efe36466c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"20fee29d-b047-46ae-aa60-88e897fbb0fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"21a296f3-b259-4841-9a17-4cd21b302b36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"59044f30-32e1-4942-9a8e-202c1def221a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7d1ae3d0-8d69-4d3c-9e51-24fdd77a22a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b0dd3d30-b273-428c-a4a9-4c238abf45fc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"40290cb3-2334-4c68-8e6c-59d0d5455171","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"54802abe-ea1a-448e-85af-d1f14a45be27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T11:29:41.511 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-18T11:29:41.543 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:41.542+0000 7f0ecedd9ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-18T11:29:41.912 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:41.910+0000 7fd49e02cec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-18T11:29:42.158 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-18T11:29:42.157+0000 7f0ecedd9ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-18T11:29:42.925 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:42.923+0000 7fd482ca8640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-18T11:29:42.928 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:29:42.926+0000 7fd482ca8640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-18T11:29:44.362 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:29:44.360+0000 7fa478210440 -1 osd.0 0 log_to_monitors true 2024-09-18T11:29:44.414 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:29:44.413+0000 7fb6a3419440 -1 osd.2 0 log_to_monitors true 2024-09-18T11:29:44.427 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:29:44.426+0000 7f3165f23440 -1 osd.4 0 log_to_monitors true 2024-09-18T11:29:44.442 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:29:44.441+0000 7f6c0317a440 -1 osd.5 0 log_to_monitors true 2024-09-18T11:29:44.456 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:29:44.455+0000 7f698af23440 -1 osd.3 0 log_to_monitors true 2024-09-18T11:29:44.467 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:29:44.466+0000 7fcb7660b440 -1 osd.7 0 log_to_monitors true 2024-09-18T11:29:44.474 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:29:44.473+0000 7f2d52b21440 -1 osd.1 0 log_to_monitors true 2024-09-18T11:29:44.475 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:29:44.474+0000 7fd28602f440 -1 osd.6 0 log_to_monitors true 2024-09-18T11:29:45.936 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:29:45.934+0000 7f31609f6640 -1 osd.4 0 waiting for initial osdmap 2024-09-18T11:29:45.936 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:29:45.934+0000 7fa4729f6640 -1 osd.0 0 waiting for initial osdmap 2024-09-18T11:29:45.936 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:29:45.934+0000 7f6bfdbf6640 -1 osd.5 0 waiting for initial osdmap 2024-09-18T11:29:45.936 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:29:45.934+0000 7fcb70df6640 -1 osd.7 0 waiting for initial osdmap 2024-09-18T11:29:45.937 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:29:45.935+0000 7fd2811f6640 -1 osd.6 0 waiting for initial osdmap 2024-09-18T11:29:45.937 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:29:45.935+0000 7fb69dbf6640 -1 osd.2 0 waiting for initial osdmap 2024-09-18T11:29:45.938 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:29:45.935+0000 7f2d4d5f6640 -1 osd.1 0 waiting for initial osdmap 2024-09-18T11:29:45.938 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:29:45.935+0000 7f69871f9640 -1 osd.3 0 waiting for initial osdmap 2024-09-18T11:29:45.940 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:29:45.939+0000 7f315c9ee640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T11:29:45.944 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:29:45.942+0000 7fd27d08a640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T11:29:45.944 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:29:45.943+0000 7fa46e1ed640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T11:29:45.944 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:29:45.943+0000 7fb699bee640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T11:29:45.945 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:29:45.943+0000 7f69819ee640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T11:29:45.946 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:29:45.945+0000 7f2d495ee640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T11:29:45.947 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:29:45.945+0000 7f6bf9bee640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T11:29:45.947 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:29:45.946+0000 7fcb6cdee640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T11:29:46.938 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:29:46.937+0000 7f3157e90640 -1 osd.4 12 handle_osd_map injecting map crc failure 2024-09-18T11:29:46.938 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:29:46.937+0000 7f6bf5056640 -1 osd.5 12 handle_osd_map injecting map crc failure 2024-09-18T11:29:46.939 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:29:46.937+0000 7f697ceb1640 -1 osd.3 12 handle_osd_map injecting map crc failure 2024-09-18T11:29:48.316 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T11:29:48.580 INFO:teuthology.misc.health.smithi119.stdout: 2024-09-18T11:29:48.580 INFO:teuthology.misc.health.smithi119.stdout:{"epoch":14,"fsid":"f4cca259-b87a-45cc-8995-7e9fca891a9f","created":"2024-09-18T11:29:38.205487+0000","modified":"2024-09-18T11:29:47.951437+0000","last_up_change":"2024-09-18T11:29:46.934503+0000","last_in_change":"2024-09-18T11:29:40.301770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T11:29:47.951262+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"12faced2-7888-4265-ad12-815efe36466c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":33347},{"type":"v1","addr":"172.21.15.119:6801","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":33347},{"type":"v1","addr":"172.21.15.119:6803","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":33347},{"type":"v1","addr":"172.21.15.119:6807","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":33347},{"type":"v1","addr":"172.21.15.119:6805","nonce":33347}]},"public_addr":"172.21.15.119:6801/33347","cluster_addr":"172.21.15.119:6803/33347","heartbeat_back_addr":"172.21.15.119:6807/33347","heartbeat_front_addr":"172.21.15.119:6805/33347","state":["exists","up"]},{"osd":1,"uuid":"20fee29d-b047-46ae-aa60-88e897fbb0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":33350},{"type":"v1","addr":"172.21.15.119:6825","nonce":33350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":33350},{"type":"v1","addr":"172.21.15.119:6827","nonce":33350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":33350},{"type":"v1","addr":"172.21.15.119:6831","nonce":33350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":33350},{"type":"v1","addr":"172.21.15.119:6829","nonce":33350}]},"public_addr":"172.21.15.119:6825/33350","cluster_addr":"172.21.15.119:6827/33350","heartbeat_back_addr":"172.21.15.119:6831/33350","heartbeat_front_addr":"172.21.15.119:6829/33350","state":["exists","up"]},{"osd":2,"uuid":"21a296f3-b259-4841-9a17-4cd21b302b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":33351},{"type":"v1","addr":"172.21.15.119:6809","nonce":33351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":33351},{"type":"v1","addr":"172.21.15.119:6813","nonce":33351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6819","nonce":33351},{"type":"v1","addr":"172.21.15.119:6821","nonce":33351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":33351},{"type":"v1","addr":"172.21.15.119:6817","nonce":33351}]},"public_addr":"172.21.15.119:6809/33351","cluster_addr":"172.21.15.119:6813/33351","heartbeat_back_addr":"172.21.15.119:6821/33351","heartbeat_front_addr":"172.21.15.119:6817/33351","state":["exists","up"]},{"osd":3,"uuid":"59044f30-32e1-4942-9a8e-202c1def221a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":33349},{"type":"v1","addr":"172.21.15.119:6812","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":33349},{"type":"v1","addr":"172.21.15.119:6816","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":33349},{"type":"v1","addr":"172.21.15.119:6823","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":33349},{"type":"v1","addr":"172.21.15.119:6820","nonce":33349}]},"public_addr":"172.21.15.119:6812/33349","cluster_addr":"172.21.15.119:6816/33349","heartbeat_back_addr":"172.21.15.119:6823/33349","heartbeat_front_addr":"172.21.15.119:6820/33349","state":["exists","up"]},{"osd":4,"uuid":"7d1ae3d0-8d69-4d3c-9e51-24fdd77a22a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":33088},{"type":"v1","addr":"172.21.15.148:6809","nonce":33088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":33088},{"type":"v1","addr":"172.21.15.148:6811","nonce":33088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":33088},{"type":"v1","addr":"172.21.15.148:6815","nonce":33088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":33088},{"type":"v1","addr":"172.21.15.148:6813","nonce":33088}]},"public_addr":"172.21.15.148:6809/33088","cluster_addr":"172.21.15.148:6811/33088","heartbeat_back_addr":"172.21.15.148:6815/33088","heartbeat_front_addr":"172.21.15.148:6813/33088","state":["exists","up"]},{"osd":5,"uuid":"b0dd3d30-b273-428c-a4a9-4c238abf45fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":33089},{"type":"v1","addr":"172.21.15.148:6801","nonce":33089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":33089},{"type":"v1","addr":"172.21.15.148:6803","nonce":33089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":33089},{"type":"v1","addr":"172.21.15.148:6807","nonce":33089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":33089},{"type":"v1","addr":"172.21.15.148:6805","nonce":33089}]},"public_addr":"172.21.15.148:6801/33089","cluster_addr":"172.21.15.148:6803/33089","heartbeat_back_addr":"172.21.15.148:6807/33089","heartbeat_front_addr":"172.21.15.148:6805/33089","state":["exists","up"]},{"osd":6,"uuid":"40290cb3-2334-4c68-8e6c-59d0d5455171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":33107},{"type":"v1","addr":"172.21.15.148:6817","nonce":33107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":33107},{"type":"v1","addr":"172.21.15.148:6819","nonce":33107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":33107},{"type":"v1","addr":"172.21.15.148:6823","nonce":33107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":33107},{"type":"v1","addr":"172.21.15.148:6821","nonce":33107}]},"public_addr":"172.21.15.148:6817/33107","cluster_addr":"172.21.15.148:6819/33107","heartbeat_back_addr":"172.21.15.148:6823/33107","heartbeat_front_addr":"172.21.15.148:6821/33107","state":["exists","up"]},{"osd":7,"uuid":"54802abe-ea1a-448e-85af-d1f14a45be27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":33119},{"type":"v1","addr":"172.21.15.148:6825","nonce":33119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":33119},{"type":"v1","addr":"172.21.15.148:6827","nonce":33119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":33119},{"type":"v1","addr":"172.21.15.148:6831","nonce":33119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":33119},{"type":"v1","addr":"172.21.15.148:6829","nonce":33119}]},"public_addr":"172.21.15.148:6825/33119","cluster_addr":"172.21.15.148:6827/33119","heartbeat_back_addr":"172.21.15.148:6831/33119","heartbeat_front_addr":"172.21.15.148:6829/33119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.374987+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.452341+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.393769+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.506373+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.463411+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.467809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.522054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.497313+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T11:29:48.591 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-18T11:29:48.591 INFO:tasks.ceph:Creating RBD pool 2024-09-18T11:29:48.591 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-18T11:29:48.955 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:29:48.954+0000 7f6bf5056640 -1 osd.5 14 handle_osd_map injecting map crc failure 2024-09-18T11:29:48.961 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:29:48.960+0000 7fcb682b1640 -1 osd.7 14 handle_osd_map injecting map crc failure 2024-09-18T11:29:48.962 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'rbd' created 2024-09-18T11:29:48.976 DEBUG:teuthology.orchestra.run.smithi119:> rbd --cluster ceph pool init rbd 2024-09-18T11:29:49.077 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:29:49.075+0000 7fd27659f640 -1 osd.6 14 handle_osd_map injecting map crc failure 2024-09-18T11:29:50.177 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:29:50.176+0000 7f697aead640 -1 osd.3 15 handle_osd_map injecting map crc failure 2024-09-18T11:29:51.037 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:29:51.036+0000 7fb6950ff640 -1 osd.2 16 handle_osd_map injecting map crc failure 2024-09-18T11:29:52.056 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-18T11:29:52.057 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-18T11:29:52.057 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-18T11:29:52.057 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-18T11:29:52.057 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-18T11:29:52.367 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:52.377 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":33122},{"type":"v1","addr":"172.21.15.119:6833","nonce":33122}]},"active_addr":"172.21.15.119:6833/33122","active_change":"2024-09-18T11:29:41.913822+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4099,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":3028357390}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":3584115318}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":490761693}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":2339462580}]}]} 2024-09-18T11:29:52.381 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-18T11:29:52.381 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-18T11:29:52.382 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T11:29:52.418 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:29:52.416+0000 7fcb682b1640 -1 osd.7 17 handle_osd_map injecting map crc failure 2024-09-18T11:29:52.464 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:29:52.462+0000 7f3155e8c640 -1 osd.4 17 handle_osd_map injecting map crc failure 2024-09-18T11:29:52.642 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:52.643 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":18,"fsid":"f4cca259-b87a-45cc-8995-7e9fca891a9f","created":"2024-09-18T11:29:38.205487+0000","modified":"2024-09-18T11:29:52.033600+0000","last_up_change":"2024-09-18T11:29:46.934503+0000","last_in_change":"2024-09-18T11:29:40.301770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T11:29:47.951262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T11:29:48.866238+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"12faced2-7888-4265-ad12-815efe36466c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":33347},{"type":"v1","addr":"172.21.15.119:6801","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":33347},{"type":"v1","addr":"172.21.15.119:6803","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":33347},{"type":"v1","addr":"172.21.15.119:6807","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":33347},{"type":"v1","addr":"172.21.15.119:6805","nonce":33347}]},"public_addr":"172.21.15.119:6801/33347","cluster_addr":"172.21.15.119:6803/33347","heartbeat_back_addr":"172.21.15.119:6807/33347","heartbeat_front_addr":"172.21.15.119:6805/33347","state":["exists","up"]},{"osd":1,"uuid":"20fee29d-b047-46ae-aa60-88e897fbb0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":33350},{"type":"v1","addr":"172.21.15.119:6825","nonce":33350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":33350},{"type":"v1","addr":"172.21.15.119:6827","nonce":33350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":33350},{"type":"v1","addr":"172.21.15.119:6831","nonce":33350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":33350},{"type":"v1","addr":"172.21.15.119:6829","nonce":33350}]},"public_addr":"172.21.15.119:6825/33350","cluster_addr":"172.21.15.119:6827/33350","heartbeat_back_addr":"172.21.15.119:6831/33350","heartbeat_front_addr":"172.21.15.119:6829/33350","state":["exists","up"]},{"osd":2,"uuid":"21a296f3-b259-4841-9a17-4cd21b302b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":33351},{"type":"v1","addr":"172.21.15.119:6809","nonce":33351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":33351},{"type":"v1","addr":"172.21.15.119:6813","nonce":33351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6819","nonce":33351},{"type":"v1","addr":"172.21.15.119:6821","nonce":33351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":33351},{"type":"v1","addr":"172.21.15.119:6817","nonce":33351}]},"public_addr":"172.21.15.119:6809/33351","cluster_addr":"172.21.15.119:6813/33351","heartbeat_back_addr":"172.21.15.119:6821/33351","heartbeat_front_addr":"172.21.15.119:6817/33351","state":["exists","up"]},{"osd":3,"uuid":"59044f30-32e1-4942-9a8e-202c1def221a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":33349},{"type":"v1","addr":"172.21.15.119:6812","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":33349},{"type":"v1","addr":"172.21.15.119:6816","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":33349},{"type":"v1","addr":"172.21.15.119:6823","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":33349},{"type":"v1","addr":"172.21.15.119:6820","nonce":33349}]},"public_addr":"172.21.15.119:6812/33349","cluster_addr":"172.21.15.119:6816/33349","heartbeat_back_addr":"172.21.15.119:6823/33349","heartbeat_front_addr":"172.21.15.119:6820/33349","state":["exists","up"]},{"osd":4,"uuid":"7d1ae3d0-8d69-4d3c-9e51-24fdd77a22a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":33088},{"type":"v1","addr":"172.21.15.148:6809","nonce":33088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":33088},{"type":"v1","addr":"172.21.15.148:6811","nonce":33088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":33088},{"type":"v1","addr":"172.21.15.148:6815","nonce":33088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":33088},{"type":"v1","addr":"172.21.15.148:6813","nonce":33088}]},"public_addr":"172.21.15.148:6809/33088","cluster_addr":"172.21.15.148:6811/33088","heartbeat_back_addr":"172.21.15.148:6815/33088","heartbeat_front_addr":"172.21.15.148:6813/33088","state":["exists","up"]},{"osd":5,"uuid":"b0dd3d30-b273-428c-a4a9-4c238abf45fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":33089},{"type":"v1","addr":"172.21.15.148:6801","nonce":33089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":33089},{"type":"v1","addr":"172.21.15.148:6803","nonce":33089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":33089},{"type":"v1","addr":"172.21.15.148:6807","nonce":33089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":33089},{"type":"v1","addr":"172.21.15.148:6805","nonce":33089}]},"public_addr":"172.21.15.148:6801/33089","cluster_addr":"172.21.15.148:6803/33089","heartbeat_back_addr":"172.21.15.148:6807/33089","heartbeat_front_addr":"172.21.15.148:6805/33089","state":["exists","up"]},{"osd":6,"uuid":"40290cb3-2334-4c68-8e6c-59d0d5455171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":33107},{"type":"v1","addr":"172.21.15.148:6817","nonce":33107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":33107},{"type":"v1","addr":"172.21.15.148:6819","nonce":33107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":33107},{"type":"v1","addr":"172.21.15.148:6823","nonce":33107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":33107},{"type":"v1","addr":"172.21.15.148:6821","nonce":33107}]},"public_addr":"172.21.15.148:6817/33107","cluster_addr":"172.21.15.148:6819/33107","heartbeat_back_addr":"172.21.15.148:6823/33107","heartbeat_front_addr":"172.21.15.148:6821/33107","state":["exists","up"]},{"osd":7,"uuid":"54802abe-ea1a-448e-85af-d1f14a45be27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":33119},{"type":"v1","addr":"172.21.15.148:6825","nonce":33119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":33119},{"type":"v1","addr":"172.21.15.148:6827","nonce":33119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":33119},{"type":"v1","addr":"172.21.15.148:6831","nonce":33119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":33119},{"type":"v1","addr":"172.21.15.148:6829","nonce":33119}]},"public_addr":"172.21.15.148:6825/33119","cluster_addr":"172.21.15.148:6827/33119","heartbeat_back_addr":"172.21.15.148:6831/33119","heartbeat_front_addr":"172.21.15.148:6829/33119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.374987+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.452341+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.393769+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.506373+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.463411+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.467809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.522054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.497313+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T11:29:52.654 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-18T11:29:52.654 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T11:29:52.914 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:52.914 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":18,"fsid":"f4cca259-b87a-45cc-8995-7e9fca891a9f","created":"2024-09-18T11:29:38.205487+0000","modified":"2024-09-18T11:29:52.033600+0000","last_up_change":"2024-09-18T11:29:46.934503+0000","last_in_change":"2024-09-18T11:29:40.301770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T11:29:47.951262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T11:29:48.866238+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"12faced2-7888-4265-ad12-815efe36466c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":33347},{"type":"v1","addr":"172.21.15.119:6801","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":33347},{"type":"v1","addr":"172.21.15.119:6803","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":33347},{"type":"v1","addr":"172.21.15.119:6807","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":33347},{"type":"v1","addr":"172.21.15.119:6805","nonce":33347}]},"public_addr":"172.21.15.119:6801/33347","cluster_addr":"172.21.15.119:6803/33347","heartbeat_back_addr":"172.21.15.119:6807/33347","heartbeat_front_addr":"172.21.15.119:6805/33347","state":["exists","up"]},{"osd":1,"uuid":"20fee29d-b047-46ae-aa60-88e897fbb0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":33350},{"type":"v1","addr":"172.21.15.119:6825","nonce":33350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":33350},{"type":"v1","addr":"172.21.15.119:6827","nonce":33350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":33350},{"type":"v1","addr":"172.21.15.119:6831","nonce":33350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":33350},{"type":"v1","addr":"172.21.15.119:6829","nonce":33350}]},"public_addr":"172.21.15.119:6825/33350","cluster_addr":"172.21.15.119:6827/33350","heartbeat_back_addr":"172.21.15.119:6831/33350","heartbeat_front_addr":"172.21.15.119:6829/33350","state":["exists","up"]},{"osd":2,"uuid":"21a296f3-b259-4841-9a17-4cd21b302b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":33351},{"type":"v1","addr":"172.21.15.119:6809","nonce":33351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":33351},{"type":"v1","addr":"172.21.15.119:6813","nonce":33351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6819","nonce":33351},{"type":"v1","addr":"172.21.15.119:6821","nonce":33351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":33351},{"type":"v1","addr":"172.21.15.119:6817","nonce":33351}]},"public_addr":"172.21.15.119:6809/33351","cluster_addr":"172.21.15.119:6813/33351","heartbeat_back_addr":"172.21.15.119:6821/33351","heartbeat_front_addr":"172.21.15.119:6817/33351","state":["exists","up"]},{"osd":3,"uuid":"59044f30-32e1-4942-9a8e-202c1def221a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":33349},{"type":"v1","addr":"172.21.15.119:6812","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":33349},{"type":"v1","addr":"172.21.15.119:6816","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":33349},{"type":"v1","addr":"172.21.15.119:6823","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":33349},{"type":"v1","addr":"172.21.15.119:6820","nonce":33349}]},"public_addr":"172.21.15.119:6812/33349","cluster_addr":"172.21.15.119:6816/33349","heartbeat_back_addr":"172.21.15.119:6823/33349","heartbeat_front_addr":"172.21.15.119:6820/33349","state":["exists","up"]},{"osd":4,"uuid":"7d1ae3d0-8d69-4d3c-9e51-24fdd77a22a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":33088},{"type":"v1","addr":"172.21.15.148:6809","nonce":33088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":33088},{"type":"v1","addr":"172.21.15.148:6811","nonce":33088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":33088},{"type":"v1","addr":"172.21.15.148:6815","nonce":33088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":33088},{"type":"v1","addr":"172.21.15.148:6813","nonce":33088}]},"public_addr":"172.21.15.148:6809/33088","cluster_addr":"172.21.15.148:6811/33088","heartbeat_back_addr":"172.21.15.148:6815/33088","heartbeat_front_addr":"172.21.15.148:6813/33088","state":["exists","up"]},{"osd":5,"uuid":"b0dd3d30-b273-428c-a4a9-4c238abf45fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":33089},{"type":"v1","addr":"172.21.15.148:6801","nonce":33089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":33089},{"type":"v1","addr":"172.21.15.148:6803","nonce":33089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":33089},{"type":"v1","addr":"172.21.15.148:6807","nonce":33089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":33089},{"type":"v1","addr":"172.21.15.148:6805","nonce":33089}]},"public_addr":"172.21.15.148:6801/33089","cluster_addr":"172.21.15.148:6803/33089","heartbeat_back_addr":"172.21.15.148:6807/33089","heartbeat_front_addr":"172.21.15.148:6805/33089","state":["exists","up"]},{"osd":6,"uuid":"40290cb3-2334-4c68-8e6c-59d0d5455171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":33107},{"type":"v1","addr":"172.21.15.148:6817","nonce":33107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":33107},{"type":"v1","addr":"172.21.15.148:6819","nonce":33107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":33107},{"type":"v1","addr":"172.21.15.148:6823","nonce":33107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":33107},{"type":"v1","addr":"172.21.15.148:6821","nonce":33107}]},"public_addr":"172.21.15.148:6817/33107","cluster_addr":"172.21.15.148:6819/33107","heartbeat_back_addr":"172.21.15.148:6823/33107","heartbeat_front_addr":"172.21.15.148:6821/33107","state":["exists","up"]},{"osd":7,"uuid":"54802abe-ea1a-448e-85af-d1f14a45be27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":33119},{"type":"v1","addr":"172.21.15.148:6825","nonce":33119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":33119},{"type":"v1","addr":"172.21.15.148:6827","nonce":33119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":33119},{"type":"v1","addr":"172.21.15.148:6831","nonce":33119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":33119},{"type":"v1","addr":"172.21.15.148:6829","nonce":33119}]},"public_addr":"172.21.15.148:6825/33119","cluster_addr":"172.21.15.148:6827/33119","heartbeat_back_addr":"172.21.15.148:6831/33119","heartbeat_front_addr":"172.21.15.148:6829/33119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.374987+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.452341+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.393769+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.506373+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.463411+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.467809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.522054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.497313+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T11:29:52.926 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-18T11:29:52.927 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-18T11:29:52.927 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-18T11:29:52.928 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-18T11:29:52.928 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-18T11:29:52.929 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-18T11:29:52.929 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-18T11:29:52.930 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-18T11:29:53.137 INFO:teuthology.orchestra.run.smithi119.stdout:55834574851 2024-09-18T11:29:53.137 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T11:29:53.157 INFO:teuthology.orchestra.run.smithi119.stdout:55834574852 2024-09-18T11:29:53.158 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T11:29:53.158 INFO:teuthology.orchestra.run.smithi119.stdout:55834574851 2024-09-18T11:29:53.159 INFO:teuthology.orchestra.run.smithi119.stdout:55834574851 2024-09-18T11:29:53.159 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T11:29:53.159 INFO:teuthology.orchestra.run.smithi119.stdout:55834574851 2024-09-18T11:29:53.160 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T11:29:53.161 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T11:29:53.163 INFO:teuthology.orchestra.run.smithi119.stdout:55834574853 2024-09-18T11:29:53.163 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T11:29:53.164 INFO:teuthology.orchestra.run.smithi119.stdout:55834574852 2024-09-18T11:29:53.164 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T11:29:53.351 INFO:teuthology.orchestra.run.smithi119.stdout:55834574851 2024-09-18T11:29:53.351 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T11:29:53.603 INFO:teuthology.orchestra.run.smithi119.stdout:55834574849 2024-09-18T11:29:53.621 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574849 for osd.7 2024-09-18T11:29:53.705 INFO:teuthology.orchestra.run.smithi119.stdout:55834574849 2024-09-18T11:29:53.715 INFO:teuthology.orchestra.run.smithi119.stdout:55834574850 2024-09-18T11:29:53.719 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574849 for osd.6 2024-09-18T11:29:53.726 INFO:teuthology.orchestra.run.smithi119.stdout:55834574850 2024-09-18T11:29:53.733 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574850 for osd.5 2024-09-18T11:29:53.742 INFO:teuthology.orchestra.run.smithi119.stdout:55834574849 2024-09-18T11:29:53.744 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574850 for osd.1 2024-09-18T11:29:53.747 INFO:teuthology.orchestra.run.smithi119.stdout:55834574851 2024-09-18T11:29:53.757 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574849 for osd.2 2024-09-18T11:29:53.764 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574851 for osd.3 2024-09-18T11:29:53.768 INFO:teuthology.orchestra.run.smithi119.stdout:55834574849 2024-09-18T11:29:53.781 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574849 for osd.4 2024-09-18T11:29:53.804 INFO:teuthology.orchestra.run.smithi119.stdout:55834574849 2024-09-18T11:29:53.816 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574849 for osd.0 2024-09-18T11:29:54.623 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T11:29:54.720 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T11:29:54.734 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T11:29:54.744 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T11:29:54.758 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T11:29:54.765 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T11:29:54.781 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T11:29:54.817 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T11:29:55.037 INFO:teuthology.orchestra.run.smithi119.stdout:55834574851 2024-09-18T11:29:55.057 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.7 2024-09-18T11:29:55.057 DEBUG:teuthology.parallel:result is None 2024-09-18T11:29:55.165 INFO:teuthology.orchestra.run.smithi119.stdout:55834574852 2024-09-18T11:29:55.182 INFO:teuthology.orchestra.run.smithi119.stdout:55834574851 2024-09-18T11:29:55.185 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.1 2024-09-18T11:29:55.185 DEBUG:teuthology.parallel:result is None 2024-09-18T11:29:55.196 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.6 2024-09-18T11:29:55.196 DEBUG:teuthology.parallel:result is None 2024-09-18T11:29:55.254 INFO:teuthology.orchestra.run.smithi119.stdout:55834574852 2024-09-18T11:29:55.263 INFO:teuthology.orchestra.run.smithi119.stdout:55834574851 2024-09-18T11:29:55.271 INFO:teuthology.orchestra.run.smithi119.stdout:55834574853 2024-09-18T11:29:55.272 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.5 2024-09-18T11:29:55.272 DEBUG:teuthology.parallel:result is None 2024-09-18T11:29:55.276 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.2 2024-09-18T11:29:55.276 DEBUG:teuthology.parallel:result is None 2024-09-18T11:29:55.283 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-09-18T11:29:55.283 DEBUG:teuthology.parallel:result is None 2024-09-18T11:29:55.322 INFO:teuthology.orchestra.run.smithi119.stdout:55834574851 2024-09-18T11:29:55.334 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.4 2024-09-18T11:29:55.334 DEBUG:teuthology.parallel:result is None 2024-09-18T11:29:55.345 INFO:teuthology.orchestra.run.smithi119.stdout:55834574851 2024-09-18T11:29:55.357 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.0 2024-09-18T11:29:55.357 DEBUG:teuthology.parallel:result is None 2024-09-18T11:29:55.358 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T11:29:55.358 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:29:55.658 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:55.658 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:29:55.669 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-09-18T11:29:53.921496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2381384,"kb_used_data":2388,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752593336,"statfs":{"total":773094113280,"available":770655576064,"internally_reserved":0,"allocated":2445312,"data_stored":1496804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.042259+0000","last_change":"2024-09-18T11:29:52.042474+0000","last_active":"2024-09-18T11:29:52.042259+0000","last_peered":"2024-09-18T11:29:52.042259+0000","last_clean":"2024-09-18T11:29:52.042259+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:29:52.042259+0000","last_undegraded":"2024-09-18T11:29:52.042259+0000","last_fullsized":"2024-09-18T11:29:52.042259+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:56.288858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.044579+0000","last_change":"2024-09-18T11:29:52.044763+0000","last_active":"2024-09-18T11:29:52.044579+0000","last_peered":"2024-09-18T11:29:52.044579+0000","last_clean":"2024-09-18T11:29:52.044579+0000","last_became_active":"2024-09-18T11:29:50.028674+0000","last_became_peered":"2024-09-18T11:29:50.028674+0000","last_unstale":"2024-09-18T11:29:52.044579+0000","last_undegraded":"2024-09-18T11:29:52.044579+0000","last_fullsized":"2024-09-18T11:29:52.044579+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:53.878233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031621200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.420364+0000","last_change":"2024-09-18T11:29:52.420604+0000","last_active":"2024-09-18T11:29:52.420364+0000","last_peered":"2024-09-18T11:29:52.420364+0000","last_clean":"2024-09-18T11:29:52.420364+0000","last_became_active":"2024-09-18T11:29:50.034939+0000","last_became_peered":"2024-09-18T11:29:50.034939+0000","last_unstale":"2024-09-18T11:29:52.420364+0000","last_undegraded":"2024-09-18T11:29:52.420364+0000","last_fullsized":"2024-09-18T11:29:52.420364+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:16.259587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.044544+0000","last_change":"2024-09-18T11:29:52.044757+0000","last_active":"2024-09-18T11:29:52.044544+0000","last_peered":"2024-09-18T11:29:52.044544+0000","last_clean":"2024-09-18T11:29:52.044544+0000","last_became_active":"2024-09-18T11:29:50.034886+0000","last_became_peered":"2024-09-18T11:29:50.034886+0000","last_unstale":"2024-09-18T11:29:52.044544+0000","last_undegraded":"2024-09-18T11:29:52.044544+0000","last_fullsized":"2024-09-18T11:29:52.044544+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:18.177634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.048389+0000","last_change":"2024-09-18T11:29:52.048389+0000","last_active":"2024-09-18T11:29:52.048389+0000","last_peered":"2024-09-18T11:29:52.048389+0000","last_clean":"2024-09-18T11:29:52.048389+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:29:52.048389+0000","last_undegraded":"2024-09-18T11:29:52.048389+0000","last_fullsized":"2024-09-18T11:29:52.048389+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:13.358402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.348004+0000","last_change":"2024-09-18T11:29:52.348137+0000","last_active":"2024-09-18T11:29:52.348004+0000","last_peered":"2024-09-18T11:29:52.348004+0000","last_clean":"2024-09-18T11:29:52.348004+0000","last_became_active":"2024-09-18T11:29:50.180029+0000","last_became_peered":"2024-09-18T11:29:50.180029+0000","last_unstale":"2024-09-18T11:29:52.348004+0000","last_undegraded":"2024-09-18T11:29:52.348004+0000","last_fullsized":"2024-09-18T11:29:52.348004+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.269389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023063399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.420312+0000","last_change":"2024-09-18T11:29:52.420488+0000","last_active":"2024-09-18T11:29:52.420312+0000","last_peered":"2024-09-18T11:29:52.420312+0000","last_clean":"2024-09-18T11:29:52.420312+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:29:52.420312+0000","last_undegraded":"2024-09-18T11:29:52.420312+0000","last_fullsized":"2024-09-18T11:29:52.420312+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:02.759929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.047586+0000","last_change":"2024-09-18T11:29:52.047782+0000","last_active":"2024-09-18T11:29:52.047586+0000","last_peered":"2024-09-18T11:29:52.047586+0000","last_clean":"2024-09-18T11:29:52.047586+0000","last_became_active":"2024-09-18T11:29:50.028407+0000","last_became_peered":"2024-09-18T11:29:50.028407+0000","last_unstale":"2024-09-18T11:29:52.047586+0000","last_undegraded":"2024-09-18T11:29:52.047586+0000","last_fullsized":"2024-09-18T11:29:52.047586+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.851663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'72","reported_seq":101,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T11:29:51.931635+0000","last_change":"2024-09-18T11:29:48.974790+0000","last_active":"2024-09-18T11:29:51.931635+0000","last_peered":"2024-09-18T11:29:51.931635+0000","last_clean":"2024-09-18T11:29:51.931635+0000","last_became_active":"2024-09-18T11:29:48.974043+0000","last_became_peered":"2024-09-18T11:29:48.974043+0000","last_unstale":"2024-09-18T11:29:51.931635+0000","last_undegraded":"2024-09-18T11:29:51.931635+0000","last_fullsized":"2024-09-18T11:29:51.931635+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_clean_scrub_stamp":"2024-09-18T11:29:47.951437+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:13.784759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298164,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073676,"statfs":{"total":96636764160,"available":96331444224,"internally_reserved":0,"allocated":696320,"data_stored":580678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":176128,"data_stored":57570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":176128,"data_stored":54361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297520,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":167936,"data_stored":52706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":172032,"data_stored":54314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":163840,"data_stored":52132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":184320,"data_stored":58545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298052,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073788,"statfs":{"total":96636764160,"available":96331558912,"internally_reserved":0,"allocated":708608,"data_stored":586498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:29:55.671 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:29:55.929 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:55.930 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:29:55.941 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-09-18T11:29:55.921777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2381384,"kb_used_data":2388,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752593336,"statfs":{"total":773094113280,"available":770655576064,"internally_reserved":0,"allocated":2445312,"data_stored":1496804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000280"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.042259+0000","last_change":"2024-09-18T11:29:52.042474+0000","last_active":"2024-09-18T11:29:52.042259+0000","last_peered":"2024-09-18T11:29:52.042259+0000","last_clean":"2024-09-18T11:29:52.042259+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:29:52.042259+0000","last_undegraded":"2024-09-18T11:29:52.042259+0000","last_fullsized":"2024-09-18T11:29:52.042259+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:56.288858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.044579+0000","last_change":"2024-09-18T11:29:52.044763+0000","last_active":"2024-09-18T11:29:52.044579+0000","last_peered":"2024-09-18T11:29:52.044579+0000","last_clean":"2024-09-18T11:29:52.044579+0000","last_became_active":"2024-09-18T11:29:50.028674+0000","last_became_peered":"2024-09-18T11:29:50.028674+0000","last_unstale":"2024-09-18T11:29:52.044579+0000","last_undegraded":"2024-09-18T11:29:52.044579+0000","last_fullsized":"2024-09-18T11:29:52.044579+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:53.878233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031621200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.420364+0000","last_change":"2024-09-18T11:29:52.420604+0000","last_active":"2024-09-18T11:29:52.420364+0000","last_peered":"2024-09-18T11:29:52.420364+0000","last_clean":"2024-09-18T11:29:52.420364+0000","last_became_active":"2024-09-18T11:29:50.034939+0000","last_became_peered":"2024-09-18T11:29:50.034939+0000","last_unstale":"2024-09-18T11:29:52.420364+0000","last_undegraded":"2024-09-18T11:29:52.420364+0000","last_fullsized":"2024-09-18T11:29:52.420364+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:16.259587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.044544+0000","last_change":"2024-09-18T11:29:52.044757+0000","last_active":"2024-09-18T11:29:52.044544+0000","last_peered":"2024-09-18T11:29:52.044544+0000","last_clean":"2024-09-18T11:29:52.044544+0000","last_became_active":"2024-09-18T11:29:50.034886+0000","last_became_peered":"2024-09-18T11:29:50.034886+0000","last_unstale":"2024-09-18T11:29:52.044544+0000","last_undegraded":"2024-09-18T11:29:52.044544+0000","last_fullsized":"2024-09-18T11:29:52.044544+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:18.177634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.048389+0000","last_change":"2024-09-18T11:29:52.048389+0000","last_active":"2024-09-18T11:29:52.048389+0000","last_peered":"2024-09-18T11:29:52.048389+0000","last_clean":"2024-09-18T11:29:52.048389+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:29:52.048389+0000","last_undegraded":"2024-09-18T11:29:52.048389+0000","last_fullsized":"2024-09-18T11:29:52.048389+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:13.358402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.348004+0000","last_change":"2024-09-18T11:29:52.348137+0000","last_active":"2024-09-18T11:29:52.348004+0000","last_peered":"2024-09-18T11:29:52.348004+0000","last_clean":"2024-09-18T11:29:52.348004+0000","last_became_active":"2024-09-18T11:29:50.180029+0000","last_became_peered":"2024-09-18T11:29:50.180029+0000","last_unstale":"2024-09-18T11:29:52.348004+0000","last_undegraded":"2024-09-18T11:29:52.348004+0000","last_fullsized":"2024-09-18T11:29:52.348004+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.269389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023063399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.420312+0000","last_change":"2024-09-18T11:29:52.420488+0000","last_active":"2024-09-18T11:29:52.420312+0000","last_peered":"2024-09-18T11:29:52.420312+0000","last_clean":"2024-09-18T11:29:52.420312+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:29:52.420312+0000","last_undegraded":"2024-09-18T11:29:52.420312+0000","last_fullsized":"2024-09-18T11:29:52.420312+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:02.759929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.047586+0000","last_change":"2024-09-18T11:29:52.047782+0000","last_active":"2024-09-18T11:29:52.047586+0000","last_peered":"2024-09-18T11:29:52.047586+0000","last_clean":"2024-09-18T11:29:52.047586+0000","last_became_active":"2024-09-18T11:29:50.028407+0000","last_became_peered":"2024-09-18T11:29:50.028407+0000","last_unstale":"2024-09-18T11:29:52.047586+0000","last_undegraded":"2024-09-18T11:29:52.047586+0000","last_fullsized":"2024-09-18T11:29:52.047586+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.851663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'72","reported_seq":101,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T11:29:51.931635+0000","last_change":"2024-09-18T11:29:48.974790+0000","last_active":"2024-09-18T11:29:51.931635+0000","last_peered":"2024-09-18T11:29:51.931635+0000","last_clean":"2024-09-18T11:29:51.931635+0000","last_became_active":"2024-09-18T11:29:48.974043+0000","last_became_peered":"2024-09-18T11:29:48.974043+0000","last_unstale":"2024-09-18T11:29:51.931635+0000","last_undegraded":"2024-09-18T11:29:51.931635+0000","last_fullsized":"2024-09-18T11:29:51.931635+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_clean_scrub_stamp":"2024-09-18T11:29:47.951437+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:13.784759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298164,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073676,"statfs":{"total":96636764160,"available":96331444224,"internally_reserved":0,"allocated":696320,"data_stored":580678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":176128,"data_stored":57570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":176128,"data_stored":54361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297520,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":167936,"data_stored":52706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":172032,"data_stored":54314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":163840,"data_stored":52132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":184320,"data_stored":58545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298052,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073788,"statfs":{"total":96636764160,"available":96331558912,"internally_reserved":0,"allocated":708608,"data_stored":586498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:29:55.943 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T11:29:55.943 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-18T11:29:55.943 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-18T11:29:55.943 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-18T11:29:56.232 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:56.233 INFO:teuthology.orchestra.run.smithi119.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-18T11:29:56.245 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-18T11:29:56.245 INFO:teuthology.run_tasks:Running task exec... 2024-09-18T11:29:56.255 INFO:teuthology.task.exec:Executing custom commands... 2024-09-18T11:29:56.255 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi119.front.sepia.ceph.com 2024-09-18T11:29:56.255 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'while ! ceph balancer status ; do sleep 1 ; done' 2024-09-18T11:29:56.512 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:29:56.512 INFO:teuthology.orchestra.run.smithi119.stdout: "active": true, 2024-09-18T11:29:56.512 INFO:teuthology.orchestra.run.smithi119.stdout: "last_optimize_duration": "0:00:00.000667", 2024-09-18T11:29:56.512 INFO:teuthology.orchestra.run.smithi119.stdout: "last_optimize_started": "Wed Sep 18 11:29:41 2024", 2024-09-18T11:29:56.512 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": "upmap", 2024-09-18T11:29:56.512 INFO:teuthology.orchestra.run.smithi119.stdout: "no_optimization_needed": false, 2024-09-18T11:29:56.513 INFO:teuthology.orchestra.run.smithi119.stdout: "optimize_result": "No pools available", 2024-09-18T11:29:56.513 INFO:teuthology.orchestra.run.smithi119.stdout: "plans": [] 2024-09-18T11:29:56.513 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:29:56.524 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph balancer mode crush-compat' 2024-09-18T11:29:57.076 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph balancer on' 2024-09-18T11:29:57.347 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-18T11:29:57.357 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 0.25, 'chance_pgnum_shrink': 0.25, 'chance_pgpnum_fix': 0.25, 'chance_test_map_discontinuity': 2, 'timeout': 1800, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-18T11:29:57.357 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd inject bad map crc probability': 0.1, 'osd map cache size': 1, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub during recovery': False, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'osd_map_cache_size', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}} 2024-09-18T11:29:57.357 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 0.25, 'chance_pgnum_shrink': 0.25, 'chance_pgpnum_fix': 0.25, 'chance_test_map_discontinuity': 2, 'timeout': 1800, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-18T11:29:57.358 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-18T11:29:57.635 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-18T11:29:57.636 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T11:29:57.636 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:29:57.894 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:57.894 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:29:57.905 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-18T11:29:57.055786+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2381384,"kb_used_data":2388,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752593336,"statfs":{"total":773094113280,"available":770655576064,"internally_reserved":0,"allocated":2445312,"data_stored":1496804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.134289"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.042259+0000","last_change":"2024-09-18T11:29:52.042474+0000","last_active":"2024-09-18T11:29:52.042259+0000","last_peered":"2024-09-18T11:29:52.042259+0000","last_clean":"2024-09-18T11:29:52.042259+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:29:52.042259+0000","last_undegraded":"2024-09-18T11:29:52.042259+0000","last_fullsized":"2024-09-18T11:29:52.042259+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:56.288858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.044579+0000","last_change":"2024-09-18T11:29:52.044763+0000","last_active":"2024-09-18T11:29:52.044579+0000","last_peered":"2024-09-18T11:29:52.044579+0000","last_clean":"2024-09-18T11:29:52.044579+0000","last_became_active":"2024-09-18T11:29:50.028674+0000","last_became_peered":"2024-09-18T11:29:50.028674+0000","last_unstale":"2024-09-18T11:29:52.044579+0000","last_undegraded":"2024-09-18T11:29:52.044579+0000","last_fullsized":"2024-09-18T11:29:52.044579+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:53.878233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031621200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.420364+0000","last_change":"2024-09-18T11:29:52.420604+0000","last_active":"2024-09-18T11:29:52.420364+0000","last_peered":"2024-09-18T11:29:52.420364+0000","last_clean":"2024-09-18T11:29:52.420364+0000","last_became_active":"2024-09-18T11:29:50.034939+0000","last_became_peered":"2024-09-18T11:29:50.034939+0000","last_unstale":"2024-09-18T11:29:52.420364+0000","last_undegraded":"2024-09-18T11:29:52.420364+0000","last_fullsized":"2024-09-18T11:29:52.420364+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:16.259587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.044544+0000","last_change":"2024-09-18T11:29:52.044757+0000","last_active":"2024-09-18T11:29:52.044544+0000","last_peered":"2024-09-18T11:29:52.044544+0000","last_clean":"2024-09-18T11:29:52.044544+0000","last_became_active":"2024-09-18T11:29:50.034886+0000","last_became_peered":"2024-09-18T11:29:50.034886+0000","last_unstale":"2024-09-18T11:29:52.044544+0000","last_undegraded":"2024-09-18T11:29:52.044544+0000","last_fullsized":"2024-09-18T11:29:52.044544+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:18.177634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.048389+0000","last_change":"2024-09-18T11:29:52.048389+0000","last_active":"2024-09-18T11:29:52.048389+0000","last_peered":"2024-09-18T11:29:52.048389+0000","last_clean":"2024-09-18T11:29:52.048389+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:29:52.048389+0000","last_undegraded":"2024-09-18T11:29:52.048389+0000","last_fullsized":"2024-09-18T11:29:52.048389+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:13.358402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.348004+0000","last_change":"2024-09-18T11:29:52.348137+0000","last_active":"2024-09-18T11:29:52.348004+0000","last_peered":"2024-09-18T11:29:52.348004+0000","last_clean":"2024-09-18T11:29:52.348004+0000","last_became_active":"2024-09-18T11:29:50.180029+0000","last_became_peered":"2024-09-18T11:29:50.180029+0000","last_unstale":"2024-09-18T11:29:52.348004+0000","last_undegraded":"2024-09-18T11:29:52.348004+0000","last_fullsized":"2024-09-18T11:29:52.348004+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.269389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023063399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.420312+0000","last_change":"2024-09-18T11:29:52.420488+0000","last_active":"2024-09-18T11:29:52.420312+0000","last_peered":"2024-09-18T11:29:52.420312+0000","last_clean":"2024-09-18T11:29:52.420312+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:29:52.420312+0000","last_undegraded":"2024-09-18T11:29:52.420312+0000","last_fullsized":"2024-09-18T11:29:52.420312+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:02.759929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.047586+0000","last_change":"2024-09-18T11:29:52.047782+0000","last_active":"2024-09-18T11:29:52.047586+0000","last_peered":"2024-09-18T11:29:52.047586+0000","last_clean":"2024-09-18T11:29:52.047586+0000","last_became_active":"2024-09-18T11:29:50.028407+0000","last_became_peered":"2024-09-18T11:29:50.028407+0000","last_unstale":"2024-09-18T11:29:52.047586+0000","last_undegraded":"2024-09-18T11:29:52.047586+0000","last_fullsized":"2024-09-18T11:29:52.047586+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.851663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'72","reported_seq":101,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T11:29:51.931635+0000","last_change":"2024-09-18T11:29:48.974790+0000","last_active":"2024-09-18T11:29:51.931635+0000","last_peered":"2024-09-18T11:29:51.931635+0000","last_clean":"2024-09-18T11:29:51.931635+0000","last_became_active":"2024-09-18T11:29:48.974043+0000","last_became_peered":"2024-09-18T11:29:48.974043+0000","last_unstale":"2024-09-18T11:29:51.931635+0000","last_undegraded":"2024-09-18T11:29:51.931635+0000","last_fullsized":"2024-09-18T11:29:51.931635+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_clean_scrub_stamp":"2024-09-18T11:29:47.951437+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:13.784759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298164,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073676,"statfs":{"total":96636764160,"available":96331444224,"internally_reserved":0,"allocated":696320,"data_stored":580678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":176128,"data_stored":57570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":176128,"data_stored":54361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297520,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":167936,"data_stored":52706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":172032,"data_stored":54314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":163840,"data_stored":52132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":184320,"data_stored":58545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298052,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073788,"statfs":{"total":96636764160,"available":96331558912,"internally_reserved":0,"allocated":708608,"data_stored":586498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:29:57.906 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:29:58.170 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:29:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:29:58.182 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-18T11:29:57.990285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2381328,"kb_used_data":2464,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752593392,"statfs":{"total":773094113280,"available":770655633408,"internally_reserved":0,"allocated":2523136,"data_stored":1540020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.068788"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.042474+0000","last_change":"2024-09-18T11:29:52.042474+0000","last_active":"2024-09-18T11:29:52.042474+0000","last_peered":"2024-09-18T11:29:52.042474+0000","last_clean":"2024-09-18T11:29:52.042474+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:29:52.042474+0000","last_undegraded":"2024-09-18T11:29:52.042474+0000","last_fullsized":"2024-09-18T11:29:52.042474+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:56.288858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.044579+0000","last_change":"2024-09-18T11:29:52.044763+0000","last_active":"2024-09-18T11:29:52.044579+0000","last_peered":"2024-09-18T11:29:52.044579+0000","last_clean":"2024-09-18T11:29:52.044579+0000","last_became_active":"2024-09-18T11:29:50.028674+0000","last_became_peered":"2024-09-18T11:29:50.028674+0000","last_unstale":"2024-09-18T11:29:52.044579+0000","last_undegraded":"2024-09-18T11:29:52.044579+0000","last_fullsized":"2024-09-18T11:29:52.044579+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:53.878233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031621200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.420604+0000","last_change":"2024-09-18T11:29:52.420604+0000","last_active":"2024-09-18T11:29:52.420604+0000","last_peered":"2024-09-18T11:29:52.420604+0000","last_clean":"2024-09-18T11:29:52.420604+0000","last_became_active":"2024-09-18T11:29:50.034939+0000","last_became_peered":"2024-09-18T11:29:50.034939+0000","last_unstale":"2024-09-18T11:29:52.420604+0000","last_undegraded":"2024-09-18T11:29:52.420604+0000","last_fullsized":"2024-09-18T11:29:52.420604+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:16.259587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.044544+0000","last_change":"2024-09-18T11:29:52.044757+0000","last_active":"2024-09-18T11:29:52.044544+0000","last_peered":"2024-09-18T11:29:52.044544+0000","last_clean":"2024-09-18T11:29:52.044544+0000","last_became_active":"2024-09-18T11:29:50.034886+0000","last_became_peered":"2024-09-18T11:29:50.034886+0000","last_unstale":"2024-09-18T11:29:52.044544+0000","last_undegraded":"2024-09-18T11:29:52.044544+0000","last_fullsized":"2024-09-18T11:29:52.044544+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:18.177634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.048389+0000","last_change":"2024-09-18T11:29:52.048389+0000","last_active":"2024-09-18T11:29:52.048389+0000","last_peered":"2024-09-18T11:29:52.048389+0000","last_clean":"2024-09-18T11:29:52.048389+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:29:52.048389+0000","last_undegraded":"2024-09-18T11:29:52.048389+0000","last_fullsized":"2024-09-18T11:29:52.048389+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:13.358402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.348137+0000","last_change":"2024-09-18T11:29:52.348137+0000","last_active":"2024-09-18T11:29:52.348137+0000","last_peered":"2024-09-18T11:29:52.348137+0000","last_clean":"2024-09-18T11:29:52.348137+0000","last_became_active":"2024-09-18T11:29:50.180029+0000","last_became_peered":"2024-09-18T11:29:50.180029+0000","last_unstale":"2024-09-18T11:29:52.348137+0000","last_undegraded":"2024-09-18T11:29:52.348137+0000","last_fullsized":"2024-09-18T11:29:52.348137+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.269389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023063399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.420488+0000","last_change":"2024-09-18T11:29:52.420488+0000","last_active":"2024-09-18T11:29:52.420488+0000","last_peered":"2024-09-18T11:29:52.420488+0000","last_clean":"2024-09-18T11:29:52.420488+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:29:52.420488+0000","last_undegraded":"2024-09-18T11:29:52.420488+0000","last_fullsized":"2024-09-18T11:29:52.420488+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:02.759929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.047586+0000","last_change":"2024-09-18T11:29:52.047782+0000","last_active":"2024-09-18T11:29:52.047586+0000","last_peered":"2024-09-18T11:29:52.047586+0000","last_clean":"2024-09-18T11:29:52.047586+0000","last_became_active":"2024-09-18T11:29:50.028407+0000","last_became_peered":"2024-09-18T11:29:50.028407+0000","last_unstale":"2024-09-18T11:29:52.047586+0000","last_undegraded":"2024-09-18T11:29:52.047586+0000","last_fullsized":"2024-09-18T11:29:52.047586+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.851663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'72","reported_seq":102,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T11:29:52.420133+0000","last_change":"2024-09-18T11:29:48.974790+0000","last_active":"2024-09-18T11:29:52.420133+0000","last_peered":"2024-09-18T11:29:52.420133+0000","last_clean":"2024-09-18T11:29:52.420133+0000","last_became_active":"2024-09-18T11:29:48.974043+0000","last_became_peered":"2024-09-18T11:29:48.974043+0000","last_unstale":"2024-09-18T11:29:52.420133+0000","last_undegraded":"2024-09-18T11:29:52.420133+0000","last_fullsized":"2024-09-18T11:29:52.420133+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_clean_scrub_stamp":"2024-09-18T11:29:47.951437+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:13.784759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298056,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073784,"statfs":{"total":96636764160,"available":96331554816,"internally_reserved":0,"allocated":716800,"data_stored":592636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":188416,"data_stored":64310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":176128,"data_stored":54361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297540,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":188416,"data_stored":64664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":172032,"data_stored":54314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":188416,"data_stored":64692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297540,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":184320,"data_stored":58545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298052,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073788,"statfs":{"total":96636764160,"available":96331558912,"internally_reserved":0,"allocated":708608,"data_stored":586498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:29:58.183 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T11:29:58.183 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-18T11:29:58.444 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 20 2024-09-18T11:29:58.444 INFO:teuthology.orchestra.run.smithi119.stdout:fsid f4cca259-b87a-45cc-8995-7e9fca891a9f 2024-09-18T11:29:58.444 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-09-18T11:29:38.205487+0000 2024-09-18T11:29:58.444 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2024-09-18T11:29:57.987401+0000 2024-09-18T11:29:58.444 INFO:teuthology.orchestra.run.smithi119.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-18T11:29:58.444 INFO:teuthology.orchestra.run.smithi119.stdout:crush_version 5 2024-09-18T11:29:58.445 INFO:teuthology.orchestra.run.smithi119.stdout:full_ratio 0.95 2024-09-18T11:29:58.445 INFO:teuthology.orchestra.run.smithi119.stdout:backfillfull_ratio 0.9 2024-09-18T11:29:58.445 INFO:teuthology.orchestra.run.smithi119.stdout:nearfull_ratio 0.85 2024-09-18T11:29:58.445 INFO:teuthology.orchestra.run.smithi119.stdout:require_min_compat_client luminous 2024-09-18T11:29:58.445 INFO:teuthology.orchestra.run.smithi119.stdout:min_compat_client jewel 2024-09-18T11:29:58.445 INFO:teuthology.orchestra.run.smithi119.stdout:require_osd_release quincy 2024-09-18T11:29:58.445 INFO:teuthology.orchestra.run.smithi119.stdout:stretch_mode_enabled false 2024-09-18T11:29:58.445 INFO:teuthology.orchestra.run.smithi119.stdout:pool 1 '.mgr' replicated size 2 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2024-09-18T11:29:58.445 INFO:teuthology.orchestra.run.smithi119.stdout:pool 2 'rbd' replicated size 2 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 18 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd 2024-09-18T11:29:58.445 INFO:teuthology.orchestra.run.smithi119.stdout:max_osd 8 2024-09-18T11:29:58.445 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6800/33347,v1:172.21.15.119:6801/33347] [v2:172.21.15.119:6802/33347,v1:172.21.15.119:6803/33347] exists,up 12faced2-7888-4265-ad12-815efe36466c 2024-09-18T11:29:58.445 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6824/33350,v1:172.21.15.119:6825/33350] [v2:172.21.15.119:6826/33350,v1:172.21.15.119:6827/33350] exists,up 20fee29d-b047-46ae-aa60-88e897fbb0fb 2024-09-18T11:29:58.446 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6808/33351,v1:172.21.15.119:6809/33351] [v2:172.21.15.119:6811/33351,v1:172.21.15.119:6813/33351] exists,up 21a296f3-b259-4841-9a17-4cd21b302b36 2024-09-18T11:29:58.446 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6810/33349,v1:172.21.15.119:6812/33349] [v2:172.21.15.119:6814/33349,v1:172.21.15.119:6816/33349] exists,up 59044f30-32e1-4942-9a8e-202c1def221a 2024-09-18T11:29:58.446 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6808/33088,v1:172.21.15.148:6809/33088] [v2:172.21.15.148:6810/33088,v1:172.21.15.148:6811/33088] exists,up 7d1ae3d0-8d69-4d3c-9e51-24fdd77a22a9 2024-09-18T11:29:58.446 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6800/33089,v1:172.21.15.148:6801/33089] [v2:172.21.15.148:6802/33089,v1:172.21.15.148:6803/33089] exists,up b0dd3d30-b273-428c-a4a9-4c238abf45fc 2024-09-18T11:29:58.446 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6816/33107,v1:172.21.15.148:6817/33107] [v2:172.21.15.148:6818/33107,v1:172.21.15.148:6819/33107] exists,up 40290cb3-2334-4c68-8e6c-59d0d5455171 2024-09-18T11:29:58.446 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6824/33119,v1:172.21.15.148:6825/33119] [v2:172.21.15.148:6826/33119,v1:172.21.15.148:6827/33119] exists,up 54802abe-ea1a-448e-85af-d1f14a45be27 2024-09-18T11:29:58.456 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6800/33347,v1:172.21.15.119:6801/33347] [v2:172.21.15.119:6802/33347,v1:172.21.15.119:6803/33347] exists,up 12faced2-7888-4265-ad12-815efe36466c', 'osd.1 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6824/33350,v1:172.21.15.119:6825/33350] [v2:172.21.15.119:6826/33350,v1:172.21.15.119:6827/33350] exists,up 20fee29d-b047-46ae-aa60-88e897fbb0fb', 'osd.2 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6808/33351,v1:172.21.15.119:6809/33351] [v2:172.21.15.119:6811/33351,v1:172.21.15.119:6813/33351] exists,up 21a296f3-b259-4841-9a17-4cd21b302b36', 'osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6810/33349,v1:172.21.15.119:6812/33349] [v2:172.21.15.119:6814/33349,v1:172.21.15.119:6816/33349] exists,up 59044f30-32e1-4942-9a8e-202c1def221a', 'osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6808/33088,v1:172.21.15.148:6809/33088] [v2:172.21.15.148:6810/33088,v1:172.21.15.148:6811/33088] exists,up 7d1ae3d0-8d69-4d3c-9e51-24fdd77a22a9', 'osd.5 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6800/33089,v1:172.21.15.148:6801/33089] [v2:172.21.15.148:6802/33089,v1:172.21.15.148:6803/33089] exists,up b0dd3d30-b273-428c-a4a9-4c238abf45fc', 'osd.6 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6816/33107,v1:172.21.15.148:6817/33107] [v2:172.21.15.148:6818/33107,v1:172.21.15.148:6819/33107] exists,up 40290cb3-2334-4c68-8e6c-59d0d5455171', 'osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6824/33119,v1:172.21.15.148:6825/33119] [v2:172.21.15.148:6826/33119,v1:172.21.15.148:6827/33119] exists,up 54802abe-ea1a-448e-85af-d1f14a45be27'] 2024-09-18T11:29:58.457 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-18T11:29:58.549 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:29:58.550 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mon.a", 2024-09-18T11:29:58.550 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster": "ceph", 2024-09-18T11:29:58.550 INFO:teuthology.orchestra.run.smithi119.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-18T11:29:58.550 INFO:teuthology.orchestra.run.smithi119.stdout: "admin_socket_mode": "", 2024-09-18T11:29:58.550 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-18T11:29:58.550 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_client_required": "cephx, none", 2024-09-18T11:29:58.550 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_cluster_required": "cephx", 2024-09-18T11:29:58.550 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_debug": "true", 2024-09-18T11:29:58.550 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-18T11:29:58.550 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-18T11:29:58.551 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_service_required": "cephx", 2024-09-18T11:29:58.551 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-18T11:29:58.551 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_supported": "cephx", 2024-09-18T11:29:58.551 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_aio": "true", 2024-09-18T11:29:58.551 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-18T11:29:58.551 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_aio_poll_ms": "250", 2024-09-18T11:29:58.551 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_aio_reap_max": "16", 2024-09-18T11:29:58.551 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_async_discard": "false", 2024-09-18T11:29:58.551 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_block_size": "4096", 2024-09-18T11:29:58.551 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_debug_aio": "false", 2024-09-18T11:29:58.551 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-18T11:29:58.551 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-18T11:29:58.551 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_debug_inflight_ios": "false", 2024-09-18T11:29:58.552 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_enable_discard": "false", 2024-09-18T11:29:58.552 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_flock_retry": "3", 2024-09-18T11:29:58.552 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-18T11:29:58.552 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_inject_crash": "0", 2024-09-18T11:29:58.552 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-18T11:29:58.552 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_ioring": "false", 2024-09-18T11:29:58.552 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_ioring_hipri": "false", 2024-09-18T11:29:58.552 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-18T11:29:58.552 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-18T11:29:58.552 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-18T11:29:58.552 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-18T11:29:58.552 INFO:teuthology.orchestra.run.smithi119.stdout: "bdev_type": "", 2024-09-18T11:29:58.552 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_alloc_size": "1048576", 2024-09-18T11:29:58.553 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_allocator": "hybrid", 2024-09-18T11:29:58.553 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_buffered_io": "true", 2024-09-18T11:29:58.553 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_check_for_zeros": "false", 2024-09-18T11:29:58.553 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-18T11:29:58.553 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-18T11:29:58.553 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_compact_log_sync": "false", 2024-09-18T11:29:58.553 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-18T11:29:58.553 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-18T11:29:58.554 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-18T11:29:58.554 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-18T11:29:58.554 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_max_log_runway": "4194304", 2024-09-18T11:29:58.554 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_max_prefetch": "1048576", 2024-09-18T11:29:58.554 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_min_flush_size": "524288", 2024-09-18T11:29:58.554 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_min_log_runway": "1048576", 2024-09-18T11:29:58.554 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_replay_recovery": "false", 2024-09-18T11:29:58.554 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-18T11:29:58.554 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-18T11:29:58.554 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_sync_write": "false", 2024-09-18T11:29:58.554 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-18T11:29:58.554 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-18T11:29:58.554 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-18T11:29:58.555 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_allocation_from_file": "true", 2024-09-18T11:29:58.555 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_allocator": "hybrid", 2024-09-18T11:29:58.555 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_async_db_compaction": "true", 2024-09-18T11:29:58.555 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-18T11:29:58.555 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-18T11:29:58.555 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-18T11:29:58.555 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-18T11:29:58.555 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-18T11:29:58.555 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-18T11:29:58.555 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-18T11:29:58.555 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_block_create": "true", 2024-09-18T11:29:58.555 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_block_db_create": "false", 2024-09-18T11:29:58.555 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_block_db_path": "", 2024-09-18T11:29:58.555 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_block_db_size": "0", 2024-09-18T11:29:58.556 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_block_path": "", 2024-09-18T11:29:58.556 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_block_preallocate_file": "false", 2024-09-18T11:29:58.556 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_block_size": "107374182400", 2024-09-18T11:29:58.556 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_block_wal_create": "false", 2024-09-18T11:29:58.556 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_block_wal_path": "", 2024-09-18T11:29:58.556 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_block_wal_size": "100663296", 2024-09-18T11:29:58.556 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_bluefs": "true", 2024-09-18T11:29:58.556 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-18T11:29:58.556 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-18T11:29:58.556 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-18T11:29:58.556 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-18T11:29:58.556 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-18T11:29:58.556 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-18T11:29:58.557 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-18T11:29:58.557 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-18T11:29:58.557 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_autotune": "true", 2024-09-18T11:29:58.557 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-18T11:29:58.557 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-18T11:29:58.557 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-18T11:29:58.557 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-18T11:29:58.557 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_size": "0", 2024-09-18T11:29:58.557 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-18T11:29:58.557 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-18T11:29:58.557 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-18T11:29:58.557 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-18T11:29:58.557 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_type": "2q", 2024-09-18T11:29:58.557 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-18T11:29:58.558 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_clone_cow": "true", 2024-09-18T11:29:58.558 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-18T11:29:58.558 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-18T11:29:58.558 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-18T11:29:58.558 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-18T11:29:58.558 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-18T11:29:58.558 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-18T11:29:58.558 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-18T11:29:58.558 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_compression_mode": "none", 2024-09-18T11:29:58.558 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-18T11:29:58.558 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_csum_type": "crc32c", 2024-09-18T11:29:58.558 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-18T11:29:58.559 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_debug_freelist": "false", 2024-09-18T11:29:58.559 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-18T11:29:58.559 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-18T11:29:58.559 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-18T11:29:58.559 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-18T11:29:58.559 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-18T11:29:58.559 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-18T11:29:58.559 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-18T11:29:58.559 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-18T11:29:58.559 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-18T11:29:58.559 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-18T11:29:58.559 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-18T11:29:58.559 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-18T11:29:58.560 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-18T11:29:58.560 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-18T11:29:58.560 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_debug_small_allocations": "0", 2024-09-18T11:29:58.560 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-18T11:29:58.560 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_default_buffered_read": "true", 2024-09-18T11:29:58.560 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_default_buffered_write": "false", 2024-09-18T11:29:58.560 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-18T11:29:58.560 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-18T11:29:58.560 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-18T11:29:58.560 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-18T11:29:58.560 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-18T11:29:58.560 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-18T11:29:58.560 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-18T11:29:58.561 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-18T11:29:58.561 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fail_eio": "false", 2024-09-18T11:29:58.561 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-18T11:29:58.561 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-18T11:29:58.561 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-18T11:29:58.561 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-18T11:29:58.561 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-18T11:29:58.561 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-18T11:29:58.561 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_on_mount": "false", 2024-09-18T11:29:58.561 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-18T11:29:58.561 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_on_umount": "false", 2024-09-18T11:29:58.561 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-18T11:29:58.561 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-18T11:29:58.562 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-18T11:29:58.562 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-18T11:29:58.562 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-18T11:29:58.562 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-18T11:29:58.562 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-18T11:29:58.562 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-18T11:29:58.562 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_ignore_data_csum": "false", 2024-09-18T11:29:58.562 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-18T11:29:58.562 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-18T11:29:58.562 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-18T11:29:58.562 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-18T11:29:58.562 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_log_op_age": "5.000000", 2024-09-18T11:29:58.562 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_max_alloc_size": "0", 2024-09-18T11:29:58.562 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_max_blob_size": "0", 2024-09-18T11:29:58.563 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-18T11:29:58.563 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-18T11:29:58.563 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-18T11:29:58.563 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_max_deferred_txc": "32", 2024-09-18T11:29:58.563 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_min_alloc_size": "0", 2024-09-18T11:29:58.563 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-18T11:29:58.563 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-18T11:29:58.563 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_nid_prealloc": "1024", 2024-09-18T11:29:58.563 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-18T11:29:58.563 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-18T11:29:58.563 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-18T11:29:58.563 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-18T11:29:58.563 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_retry_disk_reads": "3", 2024-09-18T11:29:58.563 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_rocksdb_cf": "true", 2024-09-18T11:29:58.564 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2024-09-18T11:29:58.564 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2024-09-18T11:29:58.564 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-18T11:29:58.564 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-18T11:29:58.564 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-18T11:29:58.564 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-18T11:29:58.564 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_spdk_mem": "512", 2024-09-18T11:29:58.564 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-18T11:29:58.564 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-18T11:29:58.564 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-18T11:29:58.564 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-18T11:29:58.564 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-18T11:29:58.565 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-18T11:29:58.565 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-18T11:29:58.565 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_tracing": "false", 2024-09-18T11:29:58.565 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-18T11:29:58.565 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-18T11:29:58.565 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-18T11:29:58.565 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-18T11:29:58.565 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-18T11:29:58.565 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-18T11:29:58.565 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-18T11:29:58.565 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-18T11:29:58.565 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-18T11:29:58.565 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_zero_block_detection": "false", 2024-09-18T11:29:58.566 INFO:teuthology.orchestra.run.smithi119.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-18T11:29:58.566 INFO:teuthology.orchestra.run.smithi119.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-18T11:29:58.566 INFO:teuthology.orchestra.run.smithi119.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-18T11:29:58.566 INFO:teuthology.orchestra.run.smithi119.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-18T11:29:58.566 INFO:teuthology.orchestra.run.smithi119.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-18T11:29:58.566 INFO:teuthology.orchestra.run.smithi119.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-18T11:29:58.566 INFO:teuthology.orchestra.run.smithi119.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-18T11:29:58.566 INFO:teuthology.orchestra.run.smithi119.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-18T11:29:58.566 INFO:teuthology.orchestra.run.smithi119.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-18T11:29:58.566 INFO:teuthology.orchestra.run.smithi119.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-18T11:29:58.566 INFO:teuthology.orchestra.run.smithi119.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-18T11:29:58.566 INFO:teuthology.orchestra.run.smithi119.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-18T11:29:58.566 INFO:teuthology.orchestra.run.smithi119.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-18T11:29:58.567 INFO:teuthology.orchestra.run.smithi119.stdout: "cephx_cluster_require_signatures": "false", 2024-09-18T11:29:58.567 INFO:teuthology.orchestra.run.smithi119.stdout: "cephx_cluster_require_version": "2", 2024-09-18T11:29:58.567 INFO:teuthology.orchestra.run.smithi119.stdout: "cephx_require_signatures": "false", 2024-09-18T11:29:58.567 INFO:teuthology.orchestra.run.smithi119.stdout: "cephx_require_version": "2", 2024-09-18T11:29:58.567 INFO:teuthology.orchestra.run.smithi119.stdout: "cephx_service_require_signatures": "false", 2024-09-18T11:29:58.567 INFO:teuthology.orchestra.run.smithi119.stdout: "cephx_service_require_version": "2", 2024-09-18T11:29:58.567 INFO:teuthology.orchestra.run.smithi119.stdout: "cephx_sign_messages": "true", 2024-09-18T11:29:58.567 INFO:teuthology.orchestra.run.smithi119.stdout: "chdir": "", 2024-09-18T11:29:58.567 INFO:teuthology.orchestra.run.smithi119.stdout: "client_acl_type": "", 2024-09-18T11:29:58.567 INFO:teuthology.orchestra.run.smithi119.stdout: "client_asio_thread_count": "2", 2024-09-18T11:29:58.567 INFO:teuthology.orchestra.run.smithi119.stdout: "client_cache_mid": "0.750000", 2024-09-18T11:29:58.567 INFO:teuthology.orchestra.run.smithi119.stdout: "client_cache_size": "16384", 2024-09-18T11:29:58.567 INFO:teuthology.orchestra.run.smithi119.stdout: "client_caps_release_delay": "5", 2024-09-18T11:29:58.567 INFO:teuthology.orchestra.run.smithi119.stdout: "client_check_pool_perm": "true", 2024-09-18T11:29:58.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-18T11:29:58.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_debug_force_sync_read": "false", 2024-09-18T11:29:58.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_debug_getattr_caps": "false", 2024-09-18T11:29:58.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_debug_inject_features": "", 2024-09-18T11:29:58.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_debug_inject_tick_delay": "0", 2024-09-18T11:29:58.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-18T11:29:58.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_die_on_failed_remount": "false", 2024-09-18T11:29:58.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_dirsize_rbytes": "true", 2024-09-18T11:29:58.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_force_lazyio": "false", 2024-09-18T11:29:58.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_fs": "", 2024-09-18T11:29:58.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-18T11:29:58.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_inject_release_failure": "false", 2024-09-18T11:29:58.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_max_inline_size": "4096", 2024-09-18T11:29:58.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-18T11:29:58.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client_mds_namespace": "", 2024-09-18T11:29:58.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client_metadata": "", 2024-09-18T11:29:58.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client_mount_gid": "-1", 2024-09-18T11:29:58.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client_mount_timeout": "300", 2024-09-18T11:29:58.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client_mount_uid": "-1", 2024-09-18T11:29:58.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client_mountpoint": "/", 2024-09-18T11:29:58.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client_notify_timeout": "10", 2024-09-18T11:29:58.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client_oc": "true", 2024-09-18T11:29:58.570 INFO:teuthology.orchestra.run.smithi119.stdout: "client_oc_max_dirty": "104857600", 2024-09-18T11:29:58.570 INFO:teuthology.orchestra.run.smithi119.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-18T11:29:58.570 INFO:teuthology.orchestra.run.smithi119.stdout: "client_oc_max_objects": "1000", 2024-09-18T11:29:58.570 INFO:teuthology.orchestra.run.smithi119.stdout: "client_oc_size": "209715200", 2024-09-18T11:29:58.570 INFO:teuthology.orchestra.run.smithi119.stdout: "client_oc_target_dirty": "8388608", 2024-09-18T11:29:58.570 INFO:teuthology.orchestra.run.smithi119.stdout: "client_permissions": "true", 2024-09-18T11:29:58.570 INFO:teuthology.orchestra.run.smithi119.stdout: "client_quota": "true", 2024-09-18T11:29:58.570 INFO:teuthology.orchestra.run.smithi119.stdout: "client_quota_df": "true", 2024-09-18T11:29:58.570 INFO:teuthology.orchestra.run.smithi119.stdout: "client_readahead_max_bytes": "0", 2024-09-18T11:29:58.570 INFO:teuthology.orchestra.run.smithi119.stdout: "client_readahead_max_periods": "4", 2024-09-18T11:29:58.570 INFO:teuthology.orchestra.run.smithi119.stdout: "client_readahead_min": "131072", 2024-09-18T11:29:58.570 INFO:teuthology.orchestra.run.smithi119.stdout: "client_reconnect_stale": "false", 2024-09-18T11:29:58.570 INFO:teuthology.orchestra.run.smithi119.stdout: "client_shutdown_timeout": "30", 2024-09-18T11:29:58.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client_snapdir": ".snap", 2024-09-18T11:29:58.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client_tick_interval": "1", 2024-09-18T11:29:58.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client_trace": "", 2024-09-18T11:29:58.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client_try_dentry_invalidate": "false", 2024-09-18T11:29:58.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client_use_faked_inos": "false", 2024-09-18T11:29:58.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client_use_random_mds": "false", 2024-09-18T11:29:58.571 INFO:teuthology.orchestra.run.smithi119.stdout: "clog_to_graylog": "false", 2024-09-18T11:29:58.571 INFO:teuthology.orchestra.run.smithi119.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-18T11:29:58.571 INFO:teuthology.orchestra.run.smithi119.stdout: "clog_to_graylog_port": "12201", 2024-09-18T11:29:58.571 INFO:teuthology.orchestra.run.smithi119.stdout: "clog_to_monitors": "default=true", 2024-09-18T11:29:58.571 INFO:teuthology.orchestra.run.smithi119.stdout: "clog_to_syslog": "false", 2024-09-18T11:29:58.571 INFO:teuthology.orchestra.run.smithi119.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-18T11:29:58.572 INFO:teuthology.orchestra.run.smithi119.stdout: "clog_to_syslog_level": "info", 2024-09-18T11:29:58.572 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addr": "-", 2024-09-18T11:29:58.572 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_network": "", 2024-09-18T11:29:58.572 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_network_interface": "", 2024-09-18T11:29:58.572 INFO:teuthology.orchestra.run.smithi119.stdout: "compressor_zlib_isal": "false", 2024-09-18T11:29:58.572 INFO:teuthology.orchestra.run.smithi119.stdout: "compressor_zlib_level": "5", 2024-09-18T11:29:58.572 INFO:teuthology.orchestra.run.smithi119.stdout: "compressor_zlib_winsize": "-15", 2024-09-18T11:29:58.572 INFO:teuthology.orchestra.run.smithi119.stdout: "compressor_zstd_level": "1", 2024-09-18T11:29:58.572 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-18T11:29:58.572 INFO:teuthology.orchestra.run.smithi119.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-18T11:29:58.572 INFO:teuthology.orchestra.run.smithi119.stdout: "crimson_alien_op_num_threads": "6", 2024-09-18T11:29:58.572 INFO:teuthology.orchestra.run.smithi119.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-18T11:29:58.572 INFO:teuthology.orchestra.run.smithi119.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-18T11:29:58.573 INFO:teuthology.orchestra.run.smithi119.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-18T11:29:58.573 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_location": "", 2024-09-18T11:29:58.573 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_location_hook": "", 2024-09-18T11:29:58.573 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_location_hook_timeout": "10", 2024-09-18T11:29:58.573 INFO:teuthology.orchestra.run.smithi119.stdout: "daemonize": "false", 2024-09-18T11:29:58.573 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_alienstore": "0/5", 2024-09-18T11:29:58.573 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_allow_any_pool_priority": "false", 2024-09-18T11:29:58.573 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_asok": "1/5", 2024-09-18T11:29:58.573 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_asok_assert_abort": "false", 2024-09-18T11:29:58.573 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_asserts_on_shutdown": "true", 2024-09-18T11:29:58.573 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_auth": "20/20", 2024-09-18T11:29:58.573 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_bdev": "1/3", 2024-09-18T11:29:58.573 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_bluefs": "1/5", 2024-09-18T11:29:58.574 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_bluestore": "1/5", 2024-09-18T11:29:58.574 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_buffer": "0/1", 2024-09-18T11:29:58.574 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_ceph_exporter": "1/5", 2024-09-18T11:29:58.574 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_cephfs_mirror": "0/5", 2024-09-18T11:29:58.574 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_cephsqlite": "0/5", 2024-09-18T11:29:58.574 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_civetweb": "1/10", 2024-09-18T11:29:58.574 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_client": "0/5", 2024-09-18T11:29:58.574 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_compressor": "1/5", 2024-09-18T11:29:58.574 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_context": "0/1", 2024-09-18T11:29:58.574 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_crush": "1/1", 2024-09-18T11:29:58.574 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_crypto": "1/5", 2024-09-18T11:29:58.574 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_deliberately_leak_memory": "false", 2024-09-18T11:29:58.574 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_disable_randomized_ping": "false", 2024-09-18T11:29:58.575 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_dpdk": "1/5", 2024-09-18T11:29:58.575 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_eventtrace": "1/5", 2024-09-18T11:29:58.575 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_filer": "0/1", 2024-09-18T11:29:58.575 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_filestore": "1/3", 2024-09-18T11:29:58.575 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_finisher": "1/1", 2024-09-18T11:29:58.575 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_fuse": "1/5", 2024-09-18T11:29:58.575 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_heartbeat_testing_span": "0", 2024-09-18T11:29:58.575 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_heartbeatmap": "1/5", 2024-09-18T11:29:58.575 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-18T11:29:58.575 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_javaclient": "1/5", 2024-09-18T11:29:58.575 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_journal": "1/3", 2024-09-18T11:29:58.575 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_journaler": "0/5", 2024-09-18T11:29:58.575 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_kstore": "1/5", 2024-09-18T11:29:58.575 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_leveldb": "4/5", 2024-09-18T11:29:58.576 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_lockdep": "0/1", 2024-09-18T11:29:58.576 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_mclock": "1/5", 2024-09-18T11:29:58.576 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_mds": "1/5", 2024-09-18T11:29:58.576 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_mds_balancer": "1/5", 2024-09-18T11:29:58.576 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_mds_locker": "1/5", 2024-09-18T11:29:58.576 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_mds_log": "1/5", 2024-09-18T11:29:58.576 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_mds_log_expire": "1/5", 2024-09-18T11:29:58.576 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_mds_migrator": "1/5", 2024-09-18T11:29:58.576 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_memdb": "4/5", 2024-09-18T11:29:58.576 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_mgr": "2/5", 2024-09-18T11:29:58.576 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_mgrc": "1/5", 2024-09-18T11:29:58.576 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_mon": "20/20", 2024-09-18T11:29:58.577 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_monc": "0/10", 2024-09-18T11:29:58.577 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_ms": "1/1", 2024-09-18T11:29:58.577 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_none": "0/5", 2024-09-18T11:29:58.577 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_objclass": "0/5", 2024-09-18T11:29:58.577 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_objectcacher": "0/5", 2024-09-18T11:29:58.577 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_objecter": "0/1", 2024-09-18T11:29:58.577 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_optracker": "0/5", 2024-09-18T11:29:58.577 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_osd": "1/5", 2024-09-18T11:29:58.577 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_paxos": "20/20", 2024-09-18T11:29:58.577 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_perfcounter": "1/5", 2024-09-18T11:29:58.577 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_prioritycache": "1/5", 2024-09-18T11:29:58.577 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_rados": "0/5", 2024-09-18T11:29:58.577 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_rbd": "0/5", 2024-09-18T11:29:58.577 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_rbd_mirror": "0/5", 2024-09-18T11:29:58.578 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_rbd_pwl": "0/5", 2024-09-18T11:29:58.578 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_rbd_replay": "0/5", 2024-09-18T11:29:58.578 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_refs": "0/0", 2024-09-18T11:29:58.578 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_reserver": "1/1", 2024-09-18T11:29:58.578 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_rgw": "1/5", 2024-09-18T11:29:58.578 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_rgw_datacache": "1/5", 2024-09-18T11:29:58.578 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_rgw_sync": "1/5", 2024-09-18T11:29:58.578 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_rocksdb": "4/5", 2024-09-18T11:29:58.578 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_seastore": "0/5", 2024-09-18T11:29:58.578 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_seastore_cache": "0/5", 2024-09-18T11:29:58.578 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_seastore_cleaner": "0/5", 2024-09-18T11:29:58.578 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_seastore_device": "0/5", 2024-09-18T11:29:58.579 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_seastore_journal": "0/5", 2024-09-18T11:29:58.579 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_seastore_lba": "0/5", 2024-09-18T11:29:58.579 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_seastore_odata": "0/5", 2024-09-18T11:29:58.579 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_seastore_omap": "0/5", 2024-09-18T11:29:58.579 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_seastore_onode": "0/5", 2024-09-18T11:29:58.579 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_seastore_tm": "0/5", 2024-09-18T11:29:58.579 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_striper": "0/1", 2024-09-18T11:29:58.579 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_test": "0/5", 2024-09-18T11:29:58.579 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_throttle": "1/1", 2024-09-18T11:29:58.579 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_timer": "0/1", 2024-09-18T11:29:58.579 INFO:teuthology.orchestra.run.smithi119.stdout: "debug_tp": "0/5", 2024-09-18T11:29:58.579 INFO:teuthology.orchestra.run.smithi119.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-18T11:29:58.579 INFO:teuthology.orchestra.run.smithi119.stdout: "device_failure_prediction_mode": "none", 2024-09-18T11:29:58.579 INFO:teuthology.orchestra.run.smithi119.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-18T11:29:58.580 INFO:teuthology.orchestra.run.smithi119.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-18T11:29:58.580 INFO:teuthology.orchestra.run.smithi119.stdout: "err_to_graylog": "false", 2024-09-18T11:29:58.580 INFO:teuthology.orchestra.run.smithi119.stdout: "err_to_journald": "false", 2024-09-18T11:29:58.580 INFO:teuthology.orchestra.run.smithi119.stdout: "err_to_stderr": "true", 2024-09-18T11:29:58.580 INFO:teuthology.orchestra.run.smithi119.stdout: "err_to_syslog": "false", 2024-09-18T11:29:58.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event_tracing": "false", 2024-09-18T11:29:58.580 INFO:teuthology.orchestra.run.smithi119.stdout: "exporter_addr": "0.0.0.0", 2024-09-18T11:29:58.580 INFO:teuthology.orchestra.run.smithi119.stdout: "exporter_http_port": "9926", 2024-09-18T11:29:58.580 INFO:teuthology.orchestra.run.smithi119.stdout: "exporter_prio_limit": "5", 2024-09-18T11:29:58.580 INFO:teuthology.orchestra.run.smithi119.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-18T11:29:58.580 INFO:teuthology.orchestra.run.smithi119.stdout: "exporter_sort_metrics": "true", 2024-09-18T11:29:58.580 INFO:teuthology.orchestra.run.smithi119.stdout: "exporter_stats_period": "5", 2024-09-18T11:29:58.580 INFO:teuthology.orchestra.run.smithi119.stdout: "fake_statfs_for_testing": "0", 2024-09-18T11:29:58.581 INFO:teuthology.orchestra.run.smithi119.stdout: "fatal_signal_handlers": "true", 2024-09-18T11:29:58.581 INFO:teuthology.orchestra.run.smithi119.stdout: "filer_max_purge_ops": "10", 2024-09-18T11:29:58.581 INFO:teuthology.orchestra.run.smithi119.stdout: "filer_max_truncate_ops": "128", 2024-09-18T11:29:58.581 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_apply_finisher_threads": "1", 2024-09-18T11:29:58.581 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_blackhole": "false", 2024-09-18T11:29:58.581 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_btrfs_clone_range": "true", 2024-09-18T11:29:58.581 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_btrfs_snap": "true", 2024-09-18T11:29:58.581 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_caller_concurrency": "10", 2024-09-18T11:29:58.581 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_collect_device_partition_information": "true", 2024-09-18T11:29:58.581 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_commit_timeout": "600.000000", 2024-09-18T11:29:58.581 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_debug_inject_read_err": "false", 2024-09-18T11:29:58.581 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_debug_omap_check": "false", 2024-09-18T11:29:58.582 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-18T11:29:58.582 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_debug_verify_split": "false", 2024-09-18T11:29:58.582 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_dump_file": "", 2024-09-18T11:29:58.582 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-18T11:29:58.582 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-18T11:29:58.582 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_fadvise": "true", 2024-09-18T11:29:58.582 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_fail_eio": "true", 2024-09-18T11:29:58.582 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_fd_cache_shards": "16", 2024-09-18T11:29:58.582 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_fd_cache_size": "128", 2024-09-18T11:29:58.582 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_fiemap": "false", 2024-09-18T11:29:58.582 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_fiemap_threshold": "4096", 2024-09-18T11:29:58.582 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-18T11:29:58.582 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-18T11:29:58.583 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_inject_stall": "0", 2024-09-18T11:29:58.583 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_journal_parallel": "false", 2024-09-18T11:29:58.583 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_journal_trailing": "false", 2024-09-18T11:29:58.583 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_journal_writeahead": "false", 2024-09-18T11:29:58.583 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_kill_at": "0", 2024-09-18T11:29:58.583 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-18T11:29:58.583 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-18T11:29:58.583 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-18T11:29:58.583 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-18T11:29:58.583 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-18T11:29:58.583 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_max_inline_xattrs": "0", 2024-09-18T11:29:58.583 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-18T11:29:58.583 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-18T11:29:58.584 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-18T11:29:58.584 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-18T11:29:58.584 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_max_xattr_value_size": "0", 2024-09-18T11:29:58.584 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-18T11:29:58.584 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-18T11:29:58.584 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-18T11:29:58.584 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_merge_threshold": "-10", 2024-09-18T11:29:58.584 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-18T11:29:58.584 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_odsync_write": "false", 2024-09-18T11:29:58.584 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_omap_backend": "rocksdb", 2024-09-18T11:29:58.584 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_omap_backend_path": "", 2024-09-18T11:29:58.584 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-18T11:29:58.584 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-18T11:29:58.585 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-18T11:29:58.585 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_op_thread_timeout": "60", 2024-09-18T11:29:58.585 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_op_threads": "2", 2024-09-18T11:29:58.585 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_punch_hole": "false", 2024-09-18T11:29:58.585 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-18T11:29:58.585 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-18T11:29:58.585 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-18T11:29:58.585 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-18T11:29:58.585 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-18T11:29:58.585 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-18T11:29:58.585 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-18T11:29:58.585 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-18T11:29:58.585 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-18T11:29:58.586 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_queue_max_ops": "50", 2024-09-18T11:29:58.586 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-18T11:29:58.586 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_seek_data_hole": "false", 2024-09-18T11:29:58.586 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_sloppy_crc": "false", 2024-09-18T11:29:58.586 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-18T11:29:58.586 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_splice": "false", 2024-09-18T11:29:58.586 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_split_multiple": "2", 2024-09-18T11:29:58.586 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_split_rand_factor": "20", 2024-09-18T11:29:58.586 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_update_to": "1000", 2024-09-18T11:29:58.586 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-18T11:29:58.586 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-18T11:29:58.586 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-18T11:29:58.586 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-18T11:29:58.586 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-18T11:29:58.587 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-18T11:29:58.587 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_wbthrottle_enable": "true", 2024-09-18T11:29:58.587 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-18T11:29:58.587 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-18T11:29:58.587 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-18T11:29:58.587 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-18T11:29:58.587 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-18T11:29:58.587 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-18T11:29:58.587 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_xfs_extsize": "false", 2024-09-18T11:29:58.587 INFO:teuthology.orchestra.run.smithi119.stdout: "filestore_zfs_snap": "false", 2024-09-18T11:29:58.587 INFO:teuthology.orchestra.run.smithi119.stdout: "fio_dir": "/tmp/fio", 2024-09-18T11:29:58.587 INFO:teuthology.orchestra.run.smithi119.stdout: "fsid": "f4cca259-b87a-45cc-8995-7e9fca891a9f", 2024-09-18T11:29:58.587 INFO:teuthology.orchestra.run.smithi119.stdout: "fuse_allow_other": "true", 2024-09-18T11:29:58.588 INFO:teuthology.orchestra.run.smithi119.stdout: "fuse_atomic_o_trunc": "true", 2024-09-18T11:29:58.588 INFO:teuthology.orchestra.run.smithi119.stdout: "fuse_big_writes": "true", 2024-09-18T11:29:58.588 INFO:teuthology.orchestra.run.smithi119.stdout: "fuse_debug": "false", 2024-09-18T11:29:58.588 INFO:teuthology.orchestra.run.smithi119.stdout: "fuse_default_permissions": "false", 2024-09-18T11:29:58.588 INFO:teuthology.orchestra.run.smithi119.stdout: "fuse_disable_pagecache": "false", 2024-09-18T11:29:58.588 INFO:teuthology.orchestra.run.smithi119.stdout: "fuse_max_write": "0", 2024-09-18T11:29:58.588 INFO:teuthology.orchestra.run.smithi119.stdout: "fuse_multithreaded": "true", 2024-09-18T11:29:58.588 INFO:teuthology.orchestra.run.smithi119.stdout: "fuse_require_active_mds": "true", 2024-09-18T11:29:58.588 INFO:teuthology.orchestra.run.smithi119.stdout: "fuse_set_user_groups": "true", 2024-09-18T11:29:58.588 INFO:teuthology.orchestra.run.smithi119.stdout: "fuse_splice_move": "true", 2024-09-18T11:29:58.588 INFO:teuthology.orchestra.run.smithi119.stdout: "fuse_splice_read": "true", 2024-09-18T11:29:58.588 INFO:teuthology.orchestra.run.smithi119.stdout: "fuse_splice_write": "true", 2024-09-18T11:29:58.588 INFO:teuthology.orchestra.run.smithi119.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-18T11:29:58.589 INFO:teuthology.orchestra.run.smithi119.stdout: "fuse_use_invalidate_cb": "true", 2024-09-18T11:29:58.589 INFO:teuthology.orchestra.run.smithi119.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-18T11:29:58.589 INFO:teuthology.orchestra.run.smithi119.stdout: "gss_target_name": "ceph", 2024-09-18T11:29:58.589 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_file": "", 2024-09-18T11:29:58.589 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_inject_failure": "0", 2024-09-18T11:29:58.589 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_interval": "5", 2024-09-18T11:29:58.589 INFO:teuthology.orchestra.run.smithi119.stdout: "host": "smithi119", 2024-09-18T11:29:58.589 INFO:teuthology.orchestra.run.smithi119.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-18T11:29:58.589 INFO:teuthology.orchestra.run.smithi119.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-18T11:29:58.589 INFO:teuthology.orchestra.run.smithi119.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-18T11:29:58.589 INFO:teuthology.orchestra.run.smithi119.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-18T11:29:58.589 INFO:teuthology.orchestra.run.smithi119.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-18T11:29:58.589 INFO:teuthology.orchestra.run.smithi119.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-18T11:29:58.590 INFO:teuthology.orchestra.run.smithi119.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-18T11:29:58.590 INFO:teuthology.orchestra.run.smithi119.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-18T11:29:58.590 INFO:teuthology.orchestra.run.smithi119.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-18T11:29:58.590 INFO:teuthology.orchestra.run.smithi119.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-18T11:29:58.590 INFO:teuthology.orchestra.run.smithi119.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-18T11:29:58.590 INFO:teuthology.orchestra.run.smithi119.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-18T11:29:58.590 INFO:teuthology.orchestra.run.smithi119.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-18T11:29:58.590 INFO:teuthology.orchestra.run.smithi119.stdout: "inject_early_sigterm": "false", 2024-09-18T11:29:58.590 INFO:teuthology.orchestra.run.smithi119.stdout: "jaeger_tracing_enable": "false", 2024-09-18T11:29:58.590 INFO:teuthology.orchestra.run.smithi119.stdout: "journal_aio": "true", 2024-09-18T11:29:58.590 INFO:teuthology.orchestra.run.smithi119.stdout: "journal_align_min_size": "65536", 2024-09-18T11:29:58.590 INFO:teuthology.orchestra.run.smithi119.stdout: "journal_block_align": "true", 2024-09-18T11:29:58.591 INFO:teuthology.orchestra.run.smithi119.stdout: "journal_block_size": "4096", 2024-09-18T11:29:58.591 INFO:teuthology.orchestra.run.smithi119.stdout: "journal_dio": "true", 2024-09-18T11:29:58.591 INFO:teuthology.orchestra.run.smithi119.stdout: "journal_discard": "false", 2024-09-18T11:29:58.591 INFO:teuthology.orchestra.run.smithi119.stdout: "journal_force_aio": "false", 2024-09-18T11:29:58.591 INFO:teuthology.orchestra.run.smithi119.stdout: "journal_ignore_corruption": "false", 2024-09-18T11:29:58.591 INFO:teuthology.orchestra.run.smithi119.stdout: "journal_max_write_bytes": "10485760", 2024-09-18T11:29:58.591 INFO:teuthology.orchestra.run.smithi119.stdout: "journal_max_write_entries": "100", 2024-09-18T11:29:58.591 INFO:teuthology.orchestra.run.smithi119.stdout: "journal_replay_from": "0", 2024-09-18T11:29:58.591 INFO:teuthology.orchestra.run.smithi119.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-18T11:29:58.591 INFO:teuthology.orchestra.run.smithi119.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-18T11:29:58.591 INFO:teuthology.orchestra.run.smithi119.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-18T11:29:58.591 INFO:teuthology.orchestra.run.smithi119.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-18T11:29:58.591 INFO:teuthology.orchestra.run.smithi119.stdout: "journal_write_header_frequency": "0", 2024-09-18T11:29:58.591 INFO:teuthology.orchestra.run.smithi119.stdout: "journal_zero_on_create": "false", 2024-09-18T11:29:58.592 INFO:teuthology.orchestra.run.smithi119.stdout: "journaler_prefetch_periods": "10", 2024-09-18T11:29:58.592 INFO:teuthology.orchestra.run.smithi119.stdout: "journaler_prezero_periods": "5", 2024-09-18T11:29:58.592 INFO:teuthology.orchestra.run.smithi119.stdout: "journaler_write_head_interval": "15", 2024-09-18T11:29:58.592 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "", 2024-09-18T11:29:58.592 INFO:teuthology.orchestra.run.smithi119.stdout: "keyfile": "", 2024-09-18T11:29:58.592 INFO:teuthology.orchestra.run.smithi119.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-18T11:29:58.592 INFO:teuthology.orchestra.run.smithi119.stdout: "kstore_backend": "rocksdb", 2024-09-18T11:29:58.592 INFO:teuthology.orchestra.run.smithi119.stdout: "kstore_default_stripe_size": "65536", 2024-09-18T11:29:58.592 INFO:teuthology.orchestra.run.smithi119.stdout: "kstore_fsck_on_mount": "false", 2024-09-18T11:29:58.592 INFO:teuthology.orchestra.run.smithi119.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-18T11:29:58.592 INFO:teuthology.orchestra.run.smithi119.stdout: "kstore_max_bytes": "67108864", 2024-09-18T11:29:58.592 INFO:teuthology.orchestra.run.smithi119.stdout: "kstore_max_ops": "512", 2024-09-18T11:29:58.593 INFO:teuthology.orchestra.run.smithi119.stdout: "kstore_nid_prealloc": "1024", 2024-09-18T11:29:58.593 INFO:teuthology.orchestra.run.smithi119.stdout: "kstore_onode_map_size": "1024", 2024-09-18T11:29:58.593 INFO:teuthology.orchestra.run.smithi119.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-18T11:29:58.593 INFO:teuthology.orchestra.run.smithi119.stdout: "kstore_sync_submit_transaction": "false", 2024-09-18T11:29:58.593 INFO:teuthology.orchestra.run.smithi119.stdout: "kstore_sync_transaction": "false", 2024-09-18T11:29:58.593 INFO:teuthology.orchestra.run.smithi119.stdout: "leveldb_block_size": "65536", 2024-09-18T11:29:58.593 INFO:teuthology.orchestra.run.smithi119.stdout: "leveldb_bloom_size": "0", 2024-09-18T11:29:58.593 INFO:teuthology.orchestra.run.smithi119.stdout: "leveldb_cache_size": "536870912", 2024-09-18T11:29:58.593 INFO:teuthology.orchestra.run.smithi119.stdout: "leveldb_compact_on_mount": "false", 2024-09-18T11:29:58.593 INFO:teuthology.orchestra.run.smithi119.stdout: "leveldb_compression": "false", 2024-09-18T11:29:58.593 INFO:teuthology.orchestra.run.smithi119.stdout: "leveldb_log": "", 2024-09-18T11:29:58.593 INFO:teuthology.orchestra.run.smithi119.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-18T11:29:58.593 INFO:teuthology.orchestra.run.smithi119.stdout: "leveldb_max_open_files": "0", 2024-09-18T11:29:58.594 INFO:teuthology.orchestra.run.smithi119.stdout: "leveldb_paranoid": "false", 2024-09-18T11:29:58.594 INFO:teuthology.orchestra.run.smithi119.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-18T11:29:58.594 INFO:teuthology.orchestra.run.smithi119.stdout: "librados_thread_count": "2", 2024-09-18T11:29:58.594 INFO:teuthology.orchestra.run.smithi119.stdout: "lockdep": "false", 2024-09-18T11:29:58.594 INFO:teuthology.orchestra.run.smithi119.stdout: "lockdep_force_backtrace": "false", 2024-09-18T11:29:58.594 INFO:teuthology.orchestra.run.smithi119.stdout: "log_coarse_timestamps": "true", 2024-09-18T11:29:58.594 INFO:teuthology.orchestra.run.smithi119.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-18T11:29:58.594 INFO:teuthology.orchestra.run.smithi119.stdout: "log_flush_on_exit": "false", 2024-09-18T11:29:58.594 INFO:teuthology.orchestra.run.smithi119.stdout: "log_graylog_host": "127.0.0.1", 2024-09-18T11:29:58.594 INFO:teuthology.orchestra.run.smithi119.stdout: "log_graylog_port": "12201", 2024-09-18T11:29:58.594 INFO:teuthology.orchestra.run.smithi119.stdout: "log_max_new": "1000", 2024-09-18T11:29:58.594 INFO:teuthology.orchestra.run.smithi119.stdout: "log_max_recent": "10000", 2024-09-18T11:29:58.594 INFO:teuthology.orchestra.run.smithi119.stdout: "log_stderr_prefix": "", 2024-09-18T11:29:58.594 INFO:teuthology.orchestra.run.smithi119.stdout: "log_stop_at_utilization": "0.970000", 2024-09-18T11:29:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": "true", 2024-09-18T11:29:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_graylog": "false", 2024-09-18T11:29:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_journald": "false", 2024-09-18T11:29:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_stderr": "false", 2024-09-18T11:29:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_syslog": "false", 2024-09-18T11:29:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "max_rotating_auth_attempts": "10", 2024-09-18T11:29:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-18T11:29:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_action_on_write_error": "1", 2024-09-18T11:29:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_alternate_name_max": "8192", 2024-09-18T11:29:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_asio_thread_count": "2", 2024-09-18T11:29:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_export_pin": "true", 2024-09-18T11:29:58.595 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_fragment_dirs": "true", 2024-09-18T11:29:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-18T11:29:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_fragment_interval": "5", 2024-09-18T11:29:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-18T11:29:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-18T11:29:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_interval": "10", 2024-09-18T11:29:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_max": "-1", 2024-09-18T11:29:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_max_until": "-1", 2024-09-18T11:29:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_merge_size": "50", 2024-09-18T11:29:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_midchunk": "0.300000", 2024-09-18T11:29:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-18T11:29:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_min_start": "0.200000", 2024-09-18T11:29:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_minchunk": "0.001000", 2024-09-18T11:29:58.596 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_mode": "0", 2024-09-18T11:29:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_need_max": "1.200000", 2024-09-18T11:29:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_need_min": "0.800000", 2024-09-18T11:29:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-18T11:29:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-18T11:29:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_split_bits": "3", 2024-09-18T11:29:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-18T11:29:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_split_size": "10000", 2024-09-18T11:29:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-18T11:29:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_target_decay": "10.000000", 2024-09-18T11:29:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-18T11:29:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_beacon_grace": "15.000000", 2024-09-18T11:29:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_beacon_interval": "4.000000", 2024-09-18T11:29:58.597 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-18T11:29:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-18T11:29:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache_mid": "0.700000", 2024-09-18T11:29:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache_release_free_interval": "10", 2024-09-18T11:29:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache_reservation": "0.050000", 2024-09-18T11:29:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-18T11:29:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache_trim_interval": "1", 2024-09-18T11:29:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache_trim_threshold": "262144", 2024-09-18T11:29:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-18T11:29:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-18T11:29:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-18T11:29:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_client_prealloc_inos": "1000", 2024-09-18T11:29:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-18T11:29:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_connect_bootstrapping": "false", 2024-09-18T11:29:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_damage_table_max_entries": "10000", 2024-09-18T11:29:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-18T11:29:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_debug_auth_pins": "false", 2024-09-18T11:29:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_debug_frag": "false", 2024-09-18T11:29:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_debug_scatterstat": "false", 2024-09-18T11:29:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_debug_subtrees": "false", 2024-09-18T11:29:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_decay_halflife": "5.000000", 2024-09-18T11:29:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_default_dir_hash": "2", 2024-09-18T11:29:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_defer_session_stale": "true", 2024-09-18T11:29:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_deny_all_reconnect": "false", 2024-09-18T11:29:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_dir_keys_per_op": "16384", 2024-09-18T11:29:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_dir_max_commit_size": "10", 2024-09-18T11:29:58.599 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_dir_max_entries": "0", 2024-09-18T11:29:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-18T11:29:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-18T11:29:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_dump_cache_on_map": "false", 2024-09-18T11:29:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-18T11:29:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-18T11:29:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_early_reply": "true", 2024-09-18T11:29:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_enable_op_tracker": "true", 2024-09-18T11:29:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_enforce_unique_name": "true", 2024-09-18T11:29:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-18T11:29:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-18T11:29:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_export_ephemeral_random": "true", 2024-09-18T11:29:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-18T11:29:58.600 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-18T11:29:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-18T11:29:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-18T11:29:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-18T11:29:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-18T11:29:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_health_summarize_threshold": "10", 2024-09-18T11:29:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-18T11:29:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-18T11:29:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_inject_health_dummy": "false", 2024-09-18T11:29:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-18T11:29:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_inject_migrator_session_race": "false", 2024-09-18T11:29:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-18T11:29:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-18T11:29:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-18T11:29:58.601 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_join_fs": "", 2024-09-18T11:29:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_journal_format": "1", 2024-09-18T11:29:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_kill_create_at": "0", 2024-09-18T11:29:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_kill_export_at": "0", 2024-09-18T11:29:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_kill_import_at": "0", 2024-09-18T11:29:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_kill_journal_at": "0", 2024-09-18T11:29:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_kill_journal_expire_at": "0", 2024-09-18T11:29:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_kill_journal_replay_at": "0", 2024-09-18T11:29:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_kill_link_at": "0", 2024-09-18T11:29:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_kill_mdstable_at": "0", 2024-09-18T11:29:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_kill_openc_at": "0", 2024-09-18T11:29:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_kill_rename_at": "0", 2024-09-18T11:29:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-09-18T11:29:58.602 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log_events_per_segment": "1024", 2024-09-18T11:29:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log_max_events": "-1", 2024-09-18T11:29:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log_max_segments": "128", 2024-09-18T11:29:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log_pause": "false", 2024-09-18T11:29:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log_segment_size": "0", 2024-09-18T11:29:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-18T11:29:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log_warn_factor": "2.000000", 2024-09-18T11:29:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_max_caps_per_client": "1048576", 2024-09-18T11:29:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_max_completed_flushes": "100000", 2024-09-18T11:29:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_max_completed_requests": "100000", 2024-09-18T11:29:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_max_export_size": "20971520", 2024-09-18T11:29:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_max_file_recover": "32", 2024-09-18T11:29:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_max_purge_files": "64", 2024-09-18T11:29:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_max_purge_ops": "8192", 2024-09-18T11:29:58.603 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-18T11:29:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-18T11:29:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_max_snaps_per_dir": "100", 2024-09-18T11:29:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_max_xattr_pairs_size": "65536", 2024-09-18T11:29:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_metrics_update_interval": "2", 2024-09-18T11:29:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_min_caps_per_client": "100", 2024-09-18T11:29:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_min_caps_working_set": "10000", 2024-09-18T11:29:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-18T11:29:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_numa_node": "-1", 2024-09-18T11:29:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-18T11:29:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_op_complaint_time": "30.000000", 2024-09-18T11:29:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_op_history_duration": "600", 2024-09-18T11:29:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_op_history_size": "20", 2024-09-18T11:29:58.604 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_op_history_slow_op_size": "20", 2024-09-18T11:29:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-18T11:29:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_op_log_threshold": "5", 2024-09-18T11:29:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_ping_grace": "15", 2024-09-18T11:29:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_ping_interval": "5", 2024-09-18T11:29:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-18T11:29:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-18T11:29:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_recall_max_caps": "30000", 2024-09-18T11:29:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-18T11:29:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-18T11:29:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-18T11:29:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_recall_warning_threshold": "262144", 2024-09-18T11:29:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-18T11:29:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_replay_interval": "1.000000", 2024-09-18T11:29:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-18T11:29:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-18T11:29:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_root_ino_gid": "0", 2024-09-18T11:29:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_root_ino_uid": "0", 2024-09-18T11:29:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-18T11:29:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-18T11:29:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-18T11:29:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-18T11:29:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-18T11:29:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-18T11:29:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-18T11:29:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-18T11:29:58.606 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-18T11:29:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-18T11:29:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_shutdown_check": "0", 2024-09-18T11:29:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_skip_ino": "0", 2024-09-18T11:29:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-18T11:29:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_snap_max_uid": "4294967294", 2024-09-18T11:29:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_snap_min_uid": "0", 2024-09-18T11:29:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_snap_rstat": "false", 2024-09-18T11:29:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_standby_replay_damaged": "false", 2024-09-18T11:29:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_symlink_recovery": "true", 2024-09-18T11:29:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-18T11:29:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_thrash_exports": "0", 2024-09-18T11:29:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_thrash_fragments": "0", 2024-09-18T11:29:58.607 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_tick_interval": "5.000000", 2024-09-18T11:29:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_valgrind_exit": "false", 2024-09-18T11:29:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_verify_backtrace": "1", 2024-09-18T11:29:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_verify_scatter": "false", 2024-09-18T11:29:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-18T11:29:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_wipe_sessions": "false", 2024-09-18T11:29:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool_debug": "false", 2024-09-18T11:29:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-18T11:29:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: "memstore_device_bytes": "1073741824", 2024-09-18T11:29:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: "memstore_page_set": "false", 2024-09-18T11:29:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: "memstore_page_size": "65536", 2024-09-18T11:29:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_client_bytes": "134217728", 2024-09-18T11:29:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_client_messages": "512", 2024-09-18T11:29:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-18T11:29:58.608 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-18T11:29:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-18T11:29:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-18T11:29:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_disabled_modules": "diskprediction_local", 2024-09-18T11:29:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-18T11:29:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_max_pg_num_change": "128", 2024-09-18T11:29:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_mds_bytes": "134217728", 2024-09-18T11:29:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_mds_messages": "128", 2024-09-18T11:29:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-18T11:29:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_mon_bytes": "134217728", 2024-09-18T11:29:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_mon_messages": "128", 2024-09-18T11:29:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_osd_bytes": "536870912", 2024-09-18T11:29:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_osd_messages": "8192", 2024-09-18T11:29:58.609 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_pool": "true", 2024-09-18T11:29:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-18T11:29:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_standby_modules": "true", 2024-09-18T11:29:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_stats_period": "5", 2024-09-18T11:29:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_stats_threshold": "5", 2024-09-18T11:29:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_tick_period": "2", 2024-09-18T11:29:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-18T11:29:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-18T11:29:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_allow_pool_delete": "true", 2024-09-18T11:29:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_allow_pool_size_one": "true", 2024-09-18T11:29:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_auth_validate_all_caps": "true", 2024-09-18T11:29:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-18T11:29:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-18T11:29:58.610 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_client_bytes": "104857600", 2024-09-18T11:29:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_client_directed_command_retry": "5", 2024-09-18T11:29:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-18T11:29:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-18T11:29:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-18T11:29:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-18T11:29:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_client_hunt_parallel": "3", 2024-09-18T11:29:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_client_log_interval": "1.000000", 2024-09-18T11:29:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-18T11:29:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_client_ping_interval": "10.000000", 2024-09-18T11:29:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-18T11:29:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-18T11:29:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-18T11:29:58.611 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-18T11:29:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_cluster_log_level": "debug", 2024-09-18T11:29:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_cluster_log_to_file": "true", 2024-09-18T11:29:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-18T11:29:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-18T11:29:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-18T11:29:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_cluster_log_to_journald": "false", 2024-09-18T11:29:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-18T11:29:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-18T11:29:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-18T11:29:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_compact_on_bootstrap": "false", 2024-09-18T11:29:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_compact_on_start": "false", 2024-09-18T11:29:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_compact_on_trim": "true", 2024-09-18T11:29:58.612 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-18T11:29:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-18T11:29:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-18T11:29:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_cpu_threads": "4", 2024-09-18T11:29:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_crush_min_required_version": "hammer", 2024-09-18T11:29:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_daemon_bytes": "419430400", 2024-09-18T11:29:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-18T11:29:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_data_avail_crit": "5", 2024-09-18T11:29:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_data_avail_warn": "5", 2024-09-18T11:29:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_data_size_warn": "16106127360", 2024-09-18T11:29:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-18T11:29:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-18T11:29:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_debug_dump_json": "false", 2024-09-18T11:29:58.613 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-18T11:29:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_debug_dump_transactions": "false", 2024-09-18T11:29:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_debug_extra_checks": "false", 2024-09-18T11:29:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-18T11:29:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-18T11:29:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_debug_no_require_pacific": "false", 2024-09-18T11:29:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_debug_no_require_quincy": "false", 2024-09-18T11:29:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-18T11:29:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-18T11:29:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-18T11:29:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_down_mkfs_grace": "120", 2024-09-18T11:29:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_election_default_strategy": "1", 2024-09-18T11:29:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_election_timeout": "5.000000", 2024-09-18T11:29:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-18T11:29:58.614 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_elector_ping_divisor": "2", 2024-09-18T11:29:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-18T11:29:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_enable_op_tracker": "true", 2024-09-18T11:29:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_fake_pool_delete": "false", 2024-09-18T11:29:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_force_quorum_join": "false", 2024-09-18T11:29:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_globalid_prealloc": "10000", 2024-09-18T11:29:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_health_detail_to_clog": "false", 2024-09-18T11:29:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_health_log_update_period": "5", 2024-09-18T11:29:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_health_max_detail": "50", 2024-09-18T11:29:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_health_to_clog": "true", 2024-09-18T11:29:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_health_to_clog_interval": "600", 2024-09-18T11:29:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-18T11:29:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_host": "172.21.15.119,[v2:172.21.15.119:3301,v1:172.21.15.119:6790],172.21.15.148", 2024-09-18T11:29:58.615 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_host_override": "", 2024-09-18T11:29:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_initial_members": "", 2024-09-18T11:29:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-18T11:29:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-18T11:29:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-18T11:29:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-18T11:29:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-18T11:29:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_lease": "5.000000", 2024-09-18T11:29:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-18T11:29:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-18T11:29:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_log_full_interval": "50", 2024-09-18T11:29:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_log_max": "10000", 2024-09-18T11:29:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_log_max_summary": "50", 2024-09-18T11:29:58.616 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-18T11:29:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_max_log_epochs": "500", 2024-09-18T11:29:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-18T11:29:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-18T11:29:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_max_osd": "10000", 2024-09-18T11:29:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_max_pg_per_osd": "10000", 2024-09-18T11:29:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_max_pool_pg_num": "65536", 2024-09-18T11:29:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-18T11:29:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-18T11:29:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_mds_force_trim_to": "0", 2024-09-18T11:29:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_mds_skip_sanity": "false", 2024-09-18T11:29:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_memory_autotune": "true", 2024-09-18T11:29:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_memory_target": "2147483648", 2024-09-18T11:29:58.617 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_mgr_beacon_grace": "30", 2024-09-18T11:29:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-18T11:29:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_mgr_digest_period": "5", 2024-09-18T11:29:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_mgr_inactive_grace": "60", 2024-09-18T11:29:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-18T11:29:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-18T11:29:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_min_osdmap_epochs": "50", 2024-09-18T11:29:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_op_complaint_time": "30", 2024-09-18T11:29:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_op_history_duration": "600", 2024-09-18T11:29:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_op_history_size": "20", 2024-09-18T11:29:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_op_history_slow_op_size": "20", 2024-09-18T11:29:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-18T11:29:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_op_log_threshold": "5", 2024-09-18T11:29:58.618 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-18T11:29:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-18T11:29:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-18T11:29:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_auto_mark_in": "false", 2024-09-18T11:29:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-18T11:29:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-18T11:29:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-18T11:29:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_cache_size": "500", 2024-09-18T11:29:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-18T11:29:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-18T11:29:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-18T11:29:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_down_out_interval": "600", 2024-09-18T11:29:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-18T11:29:58.619 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-18T11:29:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_force_trim_to": "0", 2024-09-18T11:29:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-18T11:29:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-18T11:29:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-18T11:29:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-18T11:29:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-18T11:29:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-18T11:29:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-18T11:29:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-18T11:29:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_min_down_reporters": "2", 2024-09-18T11:29:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-18T11:29:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-18T11:29:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-18T11:29:58.620 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-18T11:29:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-18T11:29:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-18T11:29:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_report_timeout": "900", 2024-09-18T11:29:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-18T11:29:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-18T11:29:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-18T11:29:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-18T11:29:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-18T11:29:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osdmap_full_prune_interval": "2", 2024-09-18T11:29:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osdmap_full_prune_min": "15", 2024-09-18T11:29:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_osdmap_full_prune_txsize": "2", 2024-09-18T11:29:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-18T11:29:58.621 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_pg_stuck_threshold": "60", 2024-09-18T11:29:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-18T11:29:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-18T11:29:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-18T11:29:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-18T11:29:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-18T11:29:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-18T11:29:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_probe_timeout": "2.000000", 2024-09-18T11:29:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_reweight_max_change": "0.050000", 2024-09-18T11:29:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_reweight_max_osds": "4", 2024-09-18T11:29:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-18T11:29:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-18T11:29:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-18T11:29:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-18T11:29:58.622 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-18T11:29:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_scrub_interval": "300", 2024-09-18T11:29:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_scrub_max_keys": "100", 2024-09-18T11:29:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_scrub_timeout": "300", 2024-09-18T11:29:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_session_timeout": "300", 2024-09-18T11:29:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_smart_report_timeout": "5", 2024-09-18T11:29:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_stat_smooth_intervals": "6", 2024-09-18T11:29:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-18T11:29:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_stretch_pool_min_size": "2", 2024-09-18T11:29:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_stretch_pool_size": "4", 2024-09-18T11:29:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-18T11:29:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-18T11:29:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_sync_debug": "false", 2024-09-18T11:29:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-18T11:29:58.623 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-18T11:29:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_sync_provider_kill_at": "0", 2024-09-18T11:29:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_sync_requester_kill_at": "0", 2024-09-18T11:29:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_sync_timeout": "60.000000", 2024-09-18T11:29:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_target_pg_per_osd": "100", 2024-09-18T11:29:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_tick_interval": "5", 2024-09-18T11:29:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_timecheck_interval": "300.000000", 2024-09-18T11:29:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-18T11:29:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_warn_older_version_delay": "604800", 2024-09-18T11:29:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-18T11:29:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-18T11:29:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-18T11:29:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-18T11:29:58.624 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-18T11:29:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-18T11:29:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-18T11:29:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_warn_on_misplaced": "false", 2024-09-18T11:29:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-18T11:29:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_warn_on_older_version": "true", 2024-09-18T11:29:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-18T11:29:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-18T11:29:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-18T11:29:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-18T11:29:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-18T11:29:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-18T11:29:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-18T11:29:58.625 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-18T11:29:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-18T11:29:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "monmap": "", 2024-09-18T11:29:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_async_op_threads": "3", 2024-09-18T11:29:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-18T11:29:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_async_rdma_cm": "false", 2024-09-18T11:29:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_async_rdma_device_name": "", 2024-09-18T11:29:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_async_rdma_dscp": "96", 2024-09-18T11:29:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-18T11:29:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-18T11:29:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_async_rdma_local_gid": "", 2024-09-18T11:29:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-18T11:29:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_async_rdma_port_num": "1", 2024-09-18T11:29:58.626 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-18T11:29:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-18T11:29:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-18T11:29:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-18T11:29:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_async_rdma_sl": "3", 2024-09-18T11:29:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_async_rdma_support_srq": "true", 2024-09-18T11:29:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_async_rdma_type": "ib", 2024-09-18T11:29:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_async_reap_threshold": "5", 2024-09-18T11:29:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_bind_before_connect": "false", 2024-09-18T11:29:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_bind_ipv4": "true", 2024-09-18T11:29:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_bind_ipv6": "false", 2024-09-18T11:29:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_bind_msgr1": "true", 2024-09-18T11:29:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_bind_msgr2": "true", 2024-09-18T11:29:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_bind_port_max": "7568", 2024-09-18T11:29:58.627 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_bind_port_min": "6800", 2024-09-18T11:29:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-18T11:29:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_bind_retry_count": "3", 2024-09-18T11:29:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_bind_retry_delay": "5", 2024-09-18T11:29:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_blackhole_client": "false", 2024-09-18T11:29:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_blackhole_mds": "false", 2024-09-18T11:29:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_blackhole_mgr": "false", 2024-09-18T11:29:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_blackhole_mon": "false", 2024-09-18T11:29:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_blackhole_osd": "false", 2024-09-18T11:29:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_client_mode": "crc secure", 2024-09-18T11:29:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_cluster_mode": "crc secure", 2024-09-18T11:29:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_cluster_type": "", 2024-09-18T11:29:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_compress_secure": "false", 2024-09-18T11:29:58.628 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_connection_idle_timeout": "900", 2024-09-18T11:29:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_connection_ready_timeout": "10", 2024-09-18T11:29:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_crc_data": "true", 2024-09-18T11:29:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_crc_header": "true", 2024-09-18T11:29:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_die_on_bad_msg": "false", 2024-09-18T11:29:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_die_on_bug": "true", 2024-09-18T11:29:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_die_on_old_message": "true", 2024-09-18T11:29:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_die_on_skipped_message": "false", 2024-09-18T11:29:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-18T11:29:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-18T11:29:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_dpdk_coremask": "0xF", 2024-09-18T11:29:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-18T11:29:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-18T11:29:58.629 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_dpdk_enable_tso": "true", 2024-09-18T11:29:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-18T11:29:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-18T11:29:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_dpdk_hugepages": "", 2024-09-18T11:29:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-18T11:29:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-18T11:29:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_dpdk_lro": "true", 2024-09-18T11:29:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_dpdk_memory_channel": "4", 2024-09-18T11:29:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-18T11:29:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_dpdk_pmd": "", 2024-09-18T11:29:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_dpdk_port_id": "0", 2024-09-18T11:29:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-18T11:29:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-18T11:29:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_dump_on_send": "false", 2024-09-18T11:29:58.630 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_initial_backoff": "0.200000", 2024-09-18T11:29:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_inject_delay_max": "1.000000", 2024-09-18T11:29:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-18T11:29:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_inject_delay_type": "", 2024-09-18T11:29:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-18T11:29:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_inject_socket_failures": "5000", 2024-09-18T11:29:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_learn_addr_from_peer": "true", 2024-09-18T11:29:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_max_accept_failures": "4", 2024-09-18T11:29:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_max_backoff": "15.000000", 2024-09-18T11:29:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_mon_client_mode": "secure crc", 2024-09-18T11:29:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-18T11:29:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_mon_service_mode": "secure crc", 2024-09-18T11:29:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_osd_compress_min_size": "1024", 2024-09-18T11:29:58.631 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_osd_compress_mode": "none", 2024-09-18T11:29:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_osd_compression_algorithm": "snappy zlib zstd lz4", 2024-09-18T11:29:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-18T11:29:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_pq_min_cost": "65536", 2024-09-18T11:29:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_public_type": "", 2024-09-18T11:29:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_service_mode": "crc secure", 2024-09-18T11:29:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_tcp_listen_backlog": "512", 2024-09-18T11:29:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_tcp_nodelay": "true", 2024-09-18T11:29:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-18T11:29:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_tcp_rcvbuf": "0", 2024-09-18T11:29:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "ms_type": "async", 2024-09-18T11:29:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "no_config_file": "false", 2024-09-18T11:29:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter_completion_locks_per_session": "32", 2024-09-18T11:29:58.632 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-18T11:29:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-18T11:29:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter_inflight_ops": "1024", 2024-09-18T11:29:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-18T11:29:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-18T11:29:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter_tick_interval": "5.000000", 2024-09-18T11:29:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter_timeout": "10.000000", 2024-09-18T11:29:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: "objectstore_blackhole": "false", 2024-09-18T11:29:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: "openssl_engine_opts": "", 2024-09-18T11:29:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_agent_delay_time": "5.000000", 2024-09-18T11:29:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_agent_hist_halflife": "1000", 2024-09-18T11:29:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_agent_max_low_ops": "2", 2024-09-18T11:29:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_agent_max_ops": "4", 2024-09-18T11:29:58.633 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-18T11:29:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-18T11:29:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_agent_slop": "0.020000", 2024-09-18T11:29:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-18T11:29:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-18T11:29:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_asio_thread_count": "2", 2024-09-18T11:29:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_async_recovery_min_cost": "1", 2024-09-18T11:29:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-18T11:29:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-18T11:29:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_backfill_scan_max": "512", 2024-09-18T11:29:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_backfill_scan_min": "64", 2024-09-18T11:29:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_backoff_on_degraded": "false", 2024-09-18T11:29:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_backoff_on_peering": "false", 2024-09-18T11:29:58.634 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_backoff_on_unfound": "true", 2024-09-18T11:29:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_beacon_report_interval": "300", 2024-09-18T11:29:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bench_duration": "30", 2024-09-18T11:29:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-18T11:29:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bench_max_block_size": "67108864", 2024-09-18T11:29:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-18T11:29:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_blkin_trace_all": "false", 2024-09-18T11:29:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-18T11:29:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-18T11:29:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_check_for_log_corruption": "false", 2024-09-18T11:29:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-18T11:29:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-18T11:29:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-18T11:29:58.635 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-18T11:29:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_class_update_on_start": "true", 2024-09-18T11:29:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_client_message_cap": "256", 2024-09-18T11:29:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_client_message_size_cap": "524288000", 2024-09-18T11:29:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_client_op_priority": "63", 2024-09-18T11:29:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_client_watch_timeout": "30", 2024-09-18T11:29:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_command_max_records": "256", 2024-09-18T11:29:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_compact_on_start": "false", 2024-09-18T11:29:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-18T11:29:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-18T11:29:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-18T11:29:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_crush_update_on_start": "true", 2024-09-18T11:29:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_crush_update_weight_set": "true", 2024-09-18T11:29:58.636 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-18T11:29:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-18T11:29:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-18T11:29:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-18T11:29:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-18T11:29:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_feed_pullee": "-1", 2024-09-18T11:29:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-18T11:29:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-18T11:29:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-18T11:29:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_misdirected_ops": "false", 2024-09-18T11:29:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_no_acting_change": "false", 2024-09-18T11:29:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_no_purge_strays": "false", 2024-09-18T11:29:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_op_order": "false", 2024-09-18T11:29:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-18T11:29:58.637 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-18T11:29:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-18T11:29:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-18T11:29:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_shutdown": "false", 2024-09-18T11:29:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-18T11:29:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-18T11:29:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-18T11:29:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-18T11:29:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_verify_snaps": "false", 2024-09-18T11:29:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-18T11:29:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-18T11:29:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_deep_scrub_keys": "1024", 2024-09-18T11:29:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-18T11:29:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-18T11:29:58.638 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-18T11:29:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_deep_scrub_stride": "524288", 2024-09-18T11:29:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-18T11:29:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-18T11:29:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_default_notify_timeout": "30", 2024-09-18T11:29:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_delete_sleep": "0.000000", 2024-09-18T11:29:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-18T11:29:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-18T11:29:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-18T11:29:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_discard_disconnected_ops": "true", 2024-09-18T11:29:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_enable_op_tracker": "true", 2024-09-18T11:29:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-18T11:29:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-18T11:29:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-18T11:29:58.639 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_fast_info": "true", 2024-09-18T11:29:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_fast_shutdown": "true", 2024-09-18T11:29:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-18T11:29:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-18T11:29:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-18T11:29:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-18T11:29:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-18T11:29:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_function_tracing": "false", 2024-09-18T11:29:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_heartbeat_grace": "20", 2024-09-18T11:29:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_heartbeat_interval": "6", 2024-09-18T11:29:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-18T11:29:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_heartbeat_min_peers": "10", 2024-09-18T11:29:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_heartbeat_min_size": "2000", 2024-09-18T11:29:58.640 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_heartbeat_stale": "600", 2024-09-18T11:29:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-18T11:29:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_hit_set_max_size": "100000", 2024-09-18T11:29:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_hit_set_min_size": "1000", 2024-09-18T11:29:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-18T11:29:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-18T11:29:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-18T11:29:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-18T11:29:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-18T11:29:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-18T11:29:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_journal_size": "5120", 2024-09-18T11:29:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_kill_backfill_at": "0", 2024-09-18T11:29:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-18T11:29:58.641 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_map_cache_size": "50", 2024-09-18T11:29:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_map_dedup": "true", 2024-09-18T11:29:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_map_message_max": "40", 2024-09-18T11:29:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-18T11:29:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_map_share_max_epochs": "40", 2024-09-18T11:29:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_max_attr_name_len": "100", 2024-09-18T11:29:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_max_attr_size": "0", 2024-09-18T11:29:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_max_backfills": "1", 2024-09-18T11:29:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_max_markdown_count": "5", 2024-09-18T11:29:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_max_markdown_period": "600", 2024-09-18T11:29:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_max_object_name_len": "2048", 2024-09-18T11:29:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_max_object_namespace_len": "256", 2024-09-18T11:29:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_max_object_size": "134217728", 2024-09-18T11:29:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-18T11:29:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-18T11:29:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_max_pg_blocked_by": "16", 2024-09-18T11:29:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_max_pg_log_entries": "10000", 2024-09-18T11:29:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-18T11:29:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_max_pgls": "1024", 2024-09-18T11:29:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_max_push_cost": "8388608", 2024-09-18T11:29:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_max_push_objects": "10", 2024-09-18T11:29:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_max_scrubs": "3", 2024-09-18T11:29:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-18T11:29:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_max_trimming_pgs": "2", 2024-09-18T11:29:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_max_write_op_reply_len": "64", 2024-09-18T11:29:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_max_write_size": "90", 2024-09-18T11:29:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-18T11:29:58.643 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-18T11:29:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-18T11:29:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-18T11:29:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-18T11:29:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-18T11:29:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-18T11:29:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-18T11:29:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mclock_profile": "balanced", 2024-09-18T11:29:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-18T11:29:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-18T11:29:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-18T11:29:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-18T11:29:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-18T11:29:58.644 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-18T11:29:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-18T11:29:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-18T11:29:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-18T11:29:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-18T11:29:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-18T11:29:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_memory_base": "805306368", 2024-09-18T11:29:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_memory_cache_min": "134217728", 2024-09-18T11:29:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-18T11:29:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-18T11:29:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_memory_target": "4294967296", 2024-09-18T11:29:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_memory_target_autotune": "false", 2024-09-18T11:29:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-18T11:29:58.645 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_min_pg_log_entries": "250", 2024-09-18T11:29:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_min_recovery_priority": "0", 2024-09-18T11:29:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-18T11:29:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-18T11:29:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mon_report_interval": "5", 2024-09-18T11:29:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-18T11:29:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-18T11:29:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_num_cache_shards": "32", 2024-09-18T11:29:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_num_op_tracker_shard": "32", 2024-09-18T11:29:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_numa_auto_affinity": "true", 2024-09-18T11:29:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_numa_node": "-1", 2024-09-18T11:29:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_numa_prefer_iface": "true", 2024-09-18T11:29:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_object_clean_region_max_num_intervals": "1000", 2024-09-18T11:29:58.646 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_objecter_finishers": "1", 2024-09-18T11:29:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_objectstore": "bluestore", 2024-09-18T11:29:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_objectstore_fuse": "false", 2024-09-18T11:29:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_objectstore_tracing": "false", 2024-09-18T11:29:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_op_complaint_time": "30.000000", 2024-09-18T11:29:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_op_history_duration": "600", 2024-09-18T11:29:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_op_history_size": "20", 2024-09-18T11:29:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_op_history_slow_op_size": "20", 2024-09-18T11:29:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-18T11:29:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_op_log_threshold": "5", 2024-09-18T11:29:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_op_num_shards": "0", 2024-09-18T11:29:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_op_num_shards_hdd": "5", 2024-09-18T11:29:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_op_num_shards_ssd": "8", 2024-09-18T11:29:58.647 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-18T11:29:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-18T11:29:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-18T11:29:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-18T11:29:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_op_pq_min_cost": "65536", 2024-09-18T11:29:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-18T11:29:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_op_queue_cut_off": "high", 2024-09-18T11:29:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-18T11:29:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_op_thread_timeout": "15", 2024-09-18T11:29:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_open_classes_on_start": "true", 2024-09-18T11:29:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_os_flags": "0", 2024-09-18T11:29:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_peering_op_priority": "255", 2024-09-18T11:29:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pg_delete_cost": "1048576", 2024-09-18T11:29:58.648 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pg_delete_priority": "5", 2024-09-18T11:29:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-18T11:29:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-18T11:29:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-18T11:29:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pg_log_trim_max": "10000", 2024-09-18T11:29:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pg_log_trim_min": "100", 2024-09-18T11:29:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-18T11:29:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-18T11:29:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-18T11:29:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-18T11:29:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-18T11:29:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-18T11:29:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-18T11:29:58.649 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-18T11:29:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-18T11:29:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-18T11:29:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-18T11:29:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-18T11:29:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-18T11:29:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-18T11:29:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-18T11:29:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-18T11:29:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-18T11:29:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_flags": "0", 2024-09-18T11:29:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-18T11:29:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_min_size": "2", 2024-09-18T11:29:58.650 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-18T11:29:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_pg_num": "32", 2024-09-18T11:29:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_pgp_num": "0", 2024-09-18T11:29:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-18T11:29:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_size": "2", 2024-09-18T11:29:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_default_type": "replicated", 2024-09-18T11:29:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-18T11:29:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-18T11:29:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_push_per_object_cost": "1000", 2024-09-18T11:29:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-18T11:29:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_recover_clone_overlap": "true", 2024-09-18T11:29:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-18T11:29:58.651 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_recovery_cost": "20971520", 2024-09-18T11:29:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-18T11:29:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_recovery_max_active": "0", 2024-09-18T11:29:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-18T11:29:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-18T11:29:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-18T11:29:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-18T11:29:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_recovery_max_single_start": "1", 2024-09-18T11:29:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_recovery_op_priority": "3", 2024-09-18T11:29:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-18T11:29:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_recovery_priority": "5", 2024-09-18T11:29:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-18T11:29:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_recovery_sleep": "0.000000", 2024-09-18T11:29:58.652 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-18T11:29:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-18T11:29:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-18T11:29:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_repair_during_recovery": "false", 2024-09-18T11:29:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_requested_scrub_priority": "120", 2024-09-18T11:29:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-18T11:29:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-18T11:29:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_scrub_auto_repair": "false", 2024-09-18T11:29:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-18T11:29:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-18T11:29:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_scrub_begin_hour": "0", 2024-09-18T11:29:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_scrub_begin_week_day": "0", 2024-09-18T11:29:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_scrub_chunk_max": "25", 2024-09-18T11:29:58.653 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_scrub_chunk_min": "5", 2024-09-18T11:29:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_scrub_cost": "52428800", 2024-09-18T11:29:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_scrub_during_recovery": "false", 2024-09-18T11:29:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_scrub_end_hour": "0", 2024-09-18T11:29:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_scrub_end_week_day": "0", 2024-09-18T11:29:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_scrub_event_cost": "4096", 2024-09-18T11:29:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-18T11:29:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-18T11:29:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_scrub_invalid_stats": "true", 2024-09-18T11:29:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-18T11:29:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-18T11:29:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_scrub_max_preemptions": "5", 2024-09-18T11:29:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-18T11:29:58.654 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_scrub_priority": "5", 2024-09-18T11:29:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_scrub_sleep": "0.000000", 2024-09-18T11:29:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-18T11:29:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_skip_data_digest": "false", 2024-09-18T11:29:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_smart_report_timeout": "5", 2024-09-18T11:29:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_snap_trim_cost": "1048576", 2024-09-18T11:29:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_snap_trim_priority": "5", 2024-09-18T11:29:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-18T11:29:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-18T11:29:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-18T11:29:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-18T11:29:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-18T11:29:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_target_transaction_size": "30", 2024-09-18T11:29:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-18T11:29:58.655 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-18T11:29:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-18T11:29:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-18T11:29:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-18T11:29:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-18T11:29:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-18T11:29:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-18T11:29:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-18T11:29:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-18T11:29:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_tracing": "false", 2024-09-18T11:29:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_use_stale_snap": "false", 2024-09-18T11:29:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-18T11:29:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "osdc_blkin_trace_all": "false", 2024-09-18T11:29:58.656 INFO:teuthology.orchestra.run.smithi119.stdout: "paxos_kill_at": "0", 2024-09-18T11:29:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "paxos_max_join_drift": "10", 2024-09-18T11:29:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "paxos_min": "500", 2024-09-18T11:29:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "paxos_min_wait": "0.050000", 2024-09-18T11:29:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "paxos_propose_interval": "1.000000", 2024-09-18T11:29:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "paxos_service_trim_max": "500", 2024-09-18T11:29:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-18T11:29:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "paxos_service_trim_min": "10", 2024-09-18T11:29:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "paxos_stash_full_interval": "25", 2024-09-18T11:29:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "paxos_trim_max": "500", 2024-09-18T11:29:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "paxos_trim_min": "250", 2024-09-18T11:29:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "perf": "true", 2024-09-18T11:29:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-18T11:29:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-18T11:29:58.657 INFO:teuthology.orchestra.run.smithi119.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-18T11:29:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addr": "-", 2024-09-18T11:29:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addrv": "", 2024-09-18T11:29:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "public_bind_addr": "-", 2024-09-18T11:29:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "public_network": "", 2024-09-18T11:29:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "public_network_interface": "", 2024-09-18T11:29:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "qat_compressor_enabled": "false", 2024-09-18T11:29:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "rados_mon_op_timeout": "0", 2024-09-18T11:29:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "rados_osd_op_timeout": "0", 2024-09-18T11:29:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "rados_tracing": "false", 2024-09-18T11:29:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_atime_update_interval": "60", 2024-09-18T11:29:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-18T11:29:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_balance_parent_reads": "false", 2024-09-18T11:29:58.658 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_balance_snap_reads": "false", 2024-09-18T11:29:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_blkin_trace_all": "false", 2024-09-18T11:29:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-18T11:29:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-18T11:29:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_cache": "true", 2024-09-18T11:29:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-18T11:29:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-18T11:29:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-18T11:29:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-18T11:29:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_cache_policy": "writearound", 2024-09-18T11:29:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_cache_size": "33554432", 2024-09-18T11:29:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-18T11:29:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-18T11:29:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_clone_copy_on_read": "false", 2024-09-18T11:29:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_compression_hint": "none", 2024-09-18T11:29:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_concurrent_management_ops": "10", 2024-09-18T11:29:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-18T11:29:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_default_clone_format": "auto", 2024-09-18T11:29:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_default_data_pool": "", 2024-09-18T11:29:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_default_features": "61", 2024-09-18T11:29:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_default_format": "2", 2024-09-18T11:29:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_default_map_options": "", 2024-09-18T11:29:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_default_order": "22", 2024-09-18T11:29:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_default_pool": "rbd", 2024-09-18T11:29:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-18T11:29:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_default_stripe_count": "0", 2024-09-18T11:29:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_default_stripe_unit": "0", 2024-09-18T11:29:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-18T11:29:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-18T11:29:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-18T11:29:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_enable_alloc_hint": "true", 2024-09-18T11:29:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-18T11:29:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_io_scheduler": "simple", 2024-09-18T11:29:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-18T11:29:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-18T11:29:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-18T11:29:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-18T11:29:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-18T11:29:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-18T11:29:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-18T11:29:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-18T11:29:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-18T11:29:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_journal_order": "24", 2024-09-18T11:29:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_journal_pool": "", 2024-09-18T11:29:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_journal_splay_width": "4", 2024-09-18T11:29:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_localize_parent_reads": "false", 2024-09-18T11:29:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_localize_snap_reads": "false", 2024-09-18T11:29:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-18T11:29:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-18T11:29:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-18T11:29:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-18T11:29:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-18T11:29:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-18T11:29:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-18T11:29:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-18T11:29:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-18T11:29:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-18T11:29:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-18T11:29:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-18T11:29:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-18T11:29:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-18T11:29:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-18T11:29:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-18T11:29:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-18T11:29:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-18T11:29:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-18T11:29:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-18T11:29:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-18T11:29:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-18T11:29:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-18T11:29:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-18T11:29:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-18T11:29:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-18T11:29:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-18T11:29:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-18T11:29:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-18T11:29:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-18T11:29:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-18T11:29:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_mtime_update_interval": "60", 2024-09-18T11:29:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_non_blocking_aio": "true", 2024-09-18T11:29:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_op_thread_timeout": "60", 2024-09-18T11:29:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_op_threads": "1", 2024-09-18T11:29:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_parent_cache_enabled": "false", 2024-09-18T11:29:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-18T11:29:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-18T11:29:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-18T11:29:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_plugins": "", 2024-09-18T11:29:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_qos_bps_burst": "0", 2024-09-18T11:29:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-18T11:29:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_qos_bps_limit": "0", 2024-09-18T11:29:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_qos_exclude_ops": "0", 2024-09-18T11:29:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_qos_iops_burst": "0", 2024-09-18T11:29:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-18T11:29:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_qos_iops_limit": "0", 2024-09-18T11:29:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-18T11:29:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-18T11:29:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-18T11:29:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-18T11:29:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-18T11:29:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-18T11:29:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-18T11:29:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-18T11:29:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-18T11:29:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-18T11:29:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-18T11:29:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-18T11:29:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-18T11:29:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-18T11:29:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_read_from_replica_policy": "default", 2024-09-18T11:29:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-18T11:29:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-18T11:29:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-18T11:29:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-18T11:29:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_skip_partial_discard": "true", 2024-09-18T11:29:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-18T11:29:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_tracing": "false", 2024-09-18T11:29:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_validate_names": "true", 2024-09-18T11:29:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_validate_pool": "true", 2024-09-18T11:29:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "restapi_base_url": "", 2024-09-18T11:29:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "restapi_log_level": "", 2024-09-18T11:29:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_acl_grants_max_num": "100", 2024-09-18T11:29:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_admin_entry": "admin", 2024-09-18T11:29:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_backend_store": "rados", 2024-09-18T11:29:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_barbican_url": "", 2024-09-18T11:29:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_beast_enable_async": "true", 2024-09-18T11:29:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-18T11:29:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-18T11:29:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-18T11:29:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-18T11:29:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-18T11:29:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_cache_enabled": "true", 2024-09-18T11:29:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_cache_expiry_interval": "900", 2024-09-18T11:29:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_cache_lru_size": "10000", 2024-09-18T11:29:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_content_length_compat": "false", 2024-09-18T11:29:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_copy_obj_progress": "true", 2024-09-18T11:29:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-18T11:29:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_cors_rules_max_num": "100", 2024-09-18T11:29:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_cross_domain_policy": "", 2024-09-18T11:29:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-18T11:29:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_kmip_addr": "", 2024-09-18T11:29:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-18T11:29:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-18T11:29:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-18T11:29:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-18T11:29:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_kmip_password": "", 2024-09-18T11:29:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-18T11:29:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_kmip_username": "", 2024-09-18T11:29:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_require_ssl": "true", 2024-09-18T11:29:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-18T11:29:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-18T11:29:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-18T11:29:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-18T11:29:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-18T11:29:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-18T11:29:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-18T11:29:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-18T11:29:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-18T11:29:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-18T11:29:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-18T11:29:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-18T11:29:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-18T11:29:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-18T11:29:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-18T11:29:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_vault_addr": "", 2024-09-18T11:29:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_vault_auth": "token", 2024-09-18T11:29:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_vault_namespace": "", 2024-09-18T11:29:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_vault_prefix": "", 2024-09-18T11:29:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-18T11:29:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-18T11:29:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-18T11:29:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-18T11:29:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_vault_token_file": "", 2024-09-18T11:29:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-18T11:29:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_curl_buffersize": "524288", 2024-09-18T11:29:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-18T11:29:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_curl_low_speed_time": "300", 2024-09-18T11:29:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-18T11:29:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-18T11:29:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-18T11:29:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-18T11:29:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-18T11:29:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-18T11:29:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-18T11:29:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-18T11:29:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-18T11:29:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-18T11:29:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_data_log_changes_size": "1000", 2024-09-18T11:29:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_data_log_num_shards": "128", 2024-09-18T11:29:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-18T11:29:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_data_log_window": "30", 2024-09-18T11:29:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_data_notify_interval_msec": "200", 2024-09-18T11:29:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-18T11:29:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-18T11:29:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-18T11:29:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-18T11:29:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-18T11:29:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-18T11:29:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-18T11:29:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-18T11:29:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-18T11:29:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-18T11:29:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-18T11:29:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-18T11:29:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-18T11:29:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-18T11:29:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-18T11:29:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-18T11:29:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-18T11:29:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-18T11:29:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-18T11:29:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-18T11:29:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-18T11:29:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-18T11:29:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_dns_name": "", 2024-09-18T11:29:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_dns_s3website_name": "", 2024-09-18T11:29:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_dynamic_resharding": "true", 2024-09-18T11:29:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-18T11:29:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_enable_gc_threads": "true", 2024-09-18T11:29:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_enable_lc_threads": "true", 2024-09-18T11:29:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_enable_ops_log": "false", 2024-09-18T11:29:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_enable_quota_threads": "true", 2024-09-18T11:29:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_enable_static_website": "false", 2024-09-18T11:29:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_enable_usage_log": "false", 2024-09-18T11:29:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_enforce_swift_acls": "true", 2024-09-18T11:29:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_exit_timeout_secs": "120", 2024-09-18T11:29:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_expose_bucket": "false", 2024-09-18T11:29:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_extended_http_attrs": "", 2024-09-18T11:29:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-18T11:29:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_frontends": "beast port=7480", 2024-09-18T11:29:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-18T11:29:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_gc_max_deferred": "50", 2024-09-18T11:29:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-18T11:29:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_gc_max_objs": "32", 2024-09-18T11:29:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-18T11:29:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-18T11:29:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-18T11:29:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-18T11:29:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_gc_processor_period": "3600", 2024-09-18T11:29:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-18T11:29:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-18T11:29:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-18T11:29:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-18T11:29:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_init_timeout": "300", 2024-09-18T11:29:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-18T11:29:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_kafka_connection_idle": "300", 2024-09-18T11:29:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-18T11:29:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-18T11:29:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-18T11:29:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_admin_domain": "", 2024-09-18T11:29:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_admin_password": "", 2024-09-18T11:29:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_admin_password_path": "", 2024-09-18T11:29:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_admin_project": "", 2024-09-18T11:29:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_admin_tenant": "", 2024-09-18T11:29:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_admin_token": "", 2024-09-18T11:29:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_admin_token_path": "", 2024-09-18T11:29:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_admin_user": "", 2024-09-18T11:29:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_api_version": "2", 2024-09-18T11:29:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_barbican_domain": "", 2024-09-18T11:29:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_barbican_password": "", 2024-09-18T11:29:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_barbican_project": "", 2024-09-18T11:29:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-18T11:29:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_barbican_user": "", 2024-09-18T11:29:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-18T11:29:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-18T11:29:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-18T11:29:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-18T11:29:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-18T11:29:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_url": "", 2024-09-18T11:29:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-18T11:29:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_lc_debug_interval": "-1", 2024-09-18T11:29:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_lc_lock_max_time": "90", 2024-09-18T11:29:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_lc_max_objs": "32", 2024-09-18T11:29:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_lc_max_rules": "1000", 2024-09-18T11:29:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_lc_max_worker": "3", 2024-09-18T11:29:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-18T11:29:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_lc_thread_delay": "0", 2024-09-18T11:29:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-18T11:29:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_ldap_dnattr": "uid", 2024-09-18T11:29:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-18T11:29:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_ldap_searchfilter": "", 2024-09-18T11:29:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-18T11:29:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_ldap_uri": "", 2024-09-18T11:29:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-18T11:29:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-18T11:29:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-18T11:29:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_log_http_headers": "", 2024-09-18T11:29:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-18T11:29:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-18T11:29:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_log_object_name_utc": "false", 2024-09-18T11:29:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-18T11:29:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_max_attr_name_len": "0", 2024-09-18T11:29:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_max_attr_size": "0", 2024-09-18T11:29:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-18T11:29:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_max_chunk_size": "4194304", 2024-09-18T11:29:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-18T11:29:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-18T11:29:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_max_listing_results": "1000", 2024-09-18T11:29:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_max_notify_retries": "10", 2024-09-18T11:29:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-18T11:29:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_max_put_param_size": "1048576", 2024-09-18T11:29:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_max_put_size": "5368709120", 2024-09-18T11:29:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_max_slo_entries": "1000", 2024-09-18T11:29:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_md_log_max_shards": "64", 2024-09-18T11:29:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-18T11:29:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-18T11:29:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_mp_lock_max_time": "600", 2024-09-18T11:29:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-18T11:29:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-18T11:29:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-18T11:29:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-18T11:29:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-18T11:29:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-18T11:29:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-18T11:29:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_nfs_max_gc": "300", 2024-09-18T11:29:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-18T11:29:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-18T11:29:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-18T11:29:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-18T11:29:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-18T11:29:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-18T11:29:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-18T11:29:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_num_async_rados_threads": "32", 2024-09-18T11:29:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_num_control_oids": "8", 2024-09-18T11:29:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_numa_node": "-1", 2024-09-18T11:29:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-18T11:29:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-18T11:29:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_objexp_chunk_size": "100", 2024-09-18T11:29:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_objexp_gc_interval": "600", 2024-09-18T11:29:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-18T11:29:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-18T11:29:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-18T11:29:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_op_thread_timeout": "600", 2024-09-18T11:29:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_op_tracing": "false", 2024-09-18T11:29:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_opa_token": "", 2024-09-18T11:29:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_opa_url": "", 2024-09-18T11:29:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_opa_verify_ssl": "true", 2024-09-18T11:29:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-18T11:29:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_ops_log_file_path": "", 2024-09-18T11:29:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_ops_log_rados": "true", 2024-09-18T11:29:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_ops_log_socket_path": "", 2024-09-18T11:29:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-18T11:29:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-18T11:29:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_period_push_interval": "2.000000", 2024-09-18T11:29:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-18T11:29:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-18T11:29:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_print_continue": "true", 2024-09-18T11:29:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-18T11:29:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-18T11:29:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-18T11:29:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-18T11:29:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-18T11:29:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_rados_tracing": "false", 2024-09-18T11:29:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_realm": "", 2024-09-18T11:29:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_realm_id": "", 2024-09-18T11:29:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-18T11:29:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_region": "", 2024-09-18T11:29:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-18T11:29:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-18T11:29:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-18T11:29:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-18T11:29:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_request_uri": "", 2024-09-18T11:29:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_reshard_batch_size": "64", 2024-09-18T11:29:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-18T11:29:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_reshard_max_aio": "128", 2024-09-18T11:29:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_reshard_num_logs": "16", 2024-09-18T11:29:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_reshard_thread_interval": "600", 2024-09-18T11:29:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_resolve_cname": "false", 2024-09-18T11:29:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-18T11:29:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_run_sync_thread": "true", 2024-09-18T11:29:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-18T11:29:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-18T11:29:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-18T11:29:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-18T11:29:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-18T11:29:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-18T11:29:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-18T11:29:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-18T11:29:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-18T11:29:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_scheduler_type": "throttler", 2024-09-18T11:29:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_script_uri": "", 2024-09-18T11:29:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_service_provider_name": "", 2024-09-18T11:29:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-18T11:29:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_sts_client_id": "", 2024-09-18T11:29:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_sts_client_secret": "", 2024-09-18T11:29:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_sts_entry": "sts", 2024-09-18T11:29:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_sts_key": "sts", 2024-09-18T11:29:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-18T11:29:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_sts_min_session_duration": "900", 2024-09-18T11:29:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_sts_token_introspection_url": "", 2024-09-18T11:29:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_swift_account_in_url": "false", 2024-09-18T11:29:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_swift_auth_entry": "auth", 2024-09-18T11:29:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_swift_auth_url": "", 2024-09-18T11:29:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_swift_custom_header": "", 2024-09-18T11:29:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-18T11:29:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_swift_need_stats": "true", 2024-09-18T11:29:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_swift_tenant_name": "", 2024-09-18T11:29:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_swift_token_expiration": "86400", 2024-09-18T11:29:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_swift_url": "", 2024-09-18T11:29:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_swift_url_prefix": "swift", 2024-09-18T11:29:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-18T11:29:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-18T11:29:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_sync_lease_period": "120", 2024-09-18T11:29:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-18T11:29:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-18T11:29:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-18T11:29:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-18T11:29:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-18T11:29:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-18T11:29:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-18T11:29:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-18T11:29:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-18T11:29:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_thread_pool_size": "512", 2024-09-18T11:29:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_torrent_comment": "", 2024-09-18T11:29:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_torrent_createby": "", 2024-09-18T11:29:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_torrent_encoding": "", 2024-09-18T11:29:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_torrent_flag": "false", 2024-09-18T11:29:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_torrent_origin": "", 2024-09-18T11:29:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-18T11:29:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_torrent_tracker": "", 2024-09-18T11:29:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_trust_forwarded_https": "false", 2024-09-18T11:29:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-18T11:29:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-18T11:29:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_usage_max_shards": "32", 2024-09-18T11:29:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_usage_max_user_shards": "1", 2024-09-18T11:29:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_use_opa_authz": "false", 2024-09-18T11:29:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-18T11:29:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-18T11:29:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_user_max_buckets": "1000", 2024-09-18T11:29:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-18T11:29:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-18T11:29:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-18T11:29:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-18T11:29:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_user_unique_email": "true", 2024-09-18T11:29:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_verify_ssl": "true", 2024-09-18T11:29:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-18T11:29:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_zone": "", 2024-09-18T11:29:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_zone_id": "", 2024-09-18T11:29:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-18T11:29:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_zonegroup": "", 2024-09-18T11:29:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_zonegroup_id": "", 2024-09-18T11:29:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-18T11:29:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "rocksdb_block_size": "4096", 2024-09-18T11:29:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-18T11:29:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-18T11:29:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-18T11:29:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-18T11:29:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-18T11:29:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "rocksdb_cache_size": "536870912", 2024-09-18T11:29:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-18T11:29:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-18T11:29:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-18T11:29:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-18T11:29:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-18T11:29:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-18T11:29:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-18T11:29:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-18T11:29:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: "rocksdb_index_type": "binary_search", 2024-09-18T11:29:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-18T11:29:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-18T11:29:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: "rocksdb_partition_filters": "false", 2024-09-18T11:29:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: "rocksdb_perf": "false", 2024-09-18T11:29:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-18T11:29:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-18T11:29:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-18T11:29:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "run_dir": "/var/run/ceph", 2024-09-18T11:29:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "seastore_block_create": "true", 2024-09-18T11:29:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "seastore_cache_lru_size": "67108864", 2024-09-18T11:29:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "seastore_default_max_object_size": "16777216", 2024-09-18T11:29:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-18T11:29:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "seastore_device_size": "107374182400", 2024-09-18T11:29:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "seastore_init_rewrite_segments_num_per_device": "6", 2024-09-18T11:29:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "seastore_journal_batch_capacity": "16", 2024-09-18T11:29:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-18T11:29:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-18T11:29:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-18T11:29:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "seastore_segment_size": "67108864", 2024-09-18T11:29:58.698 INFO:teuthology.orchestra.run.smithi119.stdout: "setgroup": "", 2024-09-18T11:29:58.698 INFO:teuthology.orchestra.run.smithi119.stdout: "setuser": "", 2024-09-18T11:29:58.698 INFO:teuthology.orchestra.run.smithi119.stdout: "setuser_match_path": "", 2024-09-18T11:29:58.698 INFO:teuthology.orchestra.run.smithi119.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-18T11:29:58.698 INFO:teuthology.orchestra.run.smithi119.stdout: "thp": "false", 2024-09-18T11:29:58.698 INFO:teuthology.orchestra.run.smithi119.stdout: "threadpool_default_timeout": "60", 2024-09-18T11:29:58.698 INFO:teuthology.orchestra.run.smithi119.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-18T11:29:58.698 INFO:teuthology.orchestra.run.smithi119.stdout: "throttler_perf_counter": "true" 2024-09-18T11:29:58.698 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:29:58.700 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-18T11:29:58.807 INFO:teuthology.orchestra.run.smithi119.stdout:mon.a: {} 2024-09-18T11:29:58.808 INFO:teuthology.orchestra.run.smithi119.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T11:29:58.817 INFO:teuthology.orchestra.run.smithi119.stdout:mon.b: {} 2024-09-18T11:29:58.817 INFO:teuthology.orchestra.run.smithi119.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T11:29:58.825 INFO:teuthology.orchestra.run.smithi119.stdout:mon.c: {} 2024-09-18T11:29:58.825 INFO:teuthology.orchestra.run.smithi119.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T11:29:58.835 DEBUG:teuthology.orchestra.run.smithi148:> type ceph-objectstore-tool 2024-09-18T11:29:58.854 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T11:29:58.854 DEBUG:teuthology.orchestra.run.smithi119:> type ceph-objectstore-tool 2024-09-18T11:29:58.868 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T11:29:58.869 INFO:teuthology.run_tasks:Running task exec... 2024-09-18T11:29:58.877 INFO:teuthology.task.exec:Executing custom commands... 2024-09-18T11:29:58.880 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi119.front.sepia.ceph.com 2024-09-18T11:29:58.880 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool create low_tier 4' 2024-09-18T11:29:58.880 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-18T11:29:58.881 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T11:29:58.881 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T11:29:58.881 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-18T11:29:58.881 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-18T11:29:58.882 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:29:58.882 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-18T11:29:58.882 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:29:58.882 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-18T11:29:58.883 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T11:29:58.911 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:29:58.909+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:29:58.983 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:29:58.981+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:29:59.041 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:29:59.041 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:29:59.057 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:29:59.058 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:29:59.071 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:29:59.072 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:29:59.072 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:29:59.089 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:29:59.089 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:29:59.090 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:29:59.088+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:29:59.105 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:29:59.105 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:29:59.121 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:29:59.121 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:29:59.137 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:29:59.137 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:29:59.154 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:29:59.154 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:29:59.170 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:29:59.170 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:29:59.174 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:29:59.174 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:29:59.183 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:29:59.224 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:29:59.223+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:29:59.290 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:29:59.289+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:29:59.322 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:29:59.322 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:29:59.332 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:29:59.391 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:29:59.390+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:29:59.429 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:29:59.429 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:29:59.437 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:29:59.522 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:29:59.521+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:29:59.576 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:29:59.576 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:29:59.584 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:29:59.593 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:29:59.592+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:29:59.697 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:29:59.696+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:29:59.721 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:29:59.722 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:29:59.730 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:29:59.798 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:29:59.797+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:29:59.869 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:29:59.869 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:29:59.877 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:29:59.901 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:29:59.899+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:29:59.975 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:29:59.975 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:29:59.983 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:30:00.002 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:00.000+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:00.069 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is set 2024-09-18T11:30:00.069 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T11:30:00.069 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'low_tier' created 2024-09-18T11:30:00.085 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T11:30:00.090 INFO:teuthology.run_tasks:Running task rados... 2024-09-18T11:30:00.104 INFO:tasks.rados:Beginning rados... 2024-09-18T11:30:00.104 INFO:tasks.rados:config is {'clients': ['client.0'], 'dedup_chunk_algo': 'fastcdc', 'dedup_chunk_size': '131072', 'enable_dedup': True, 'low_tier_pool': 'low_tier', 'objects': 50, 'op_weights': {'read': 100, 'rollback': 10, 'set_chunk': 30, 'snap_create': 10, 'snap_remove': 10, 'tier_evict': 10, 'tier_flush': 5, 'tier_promote': 10, 'write': 50}, 'ops': 1500, 'set_chunk': True} 2024-09-18T11:30:00.105 INFO:tasks.rados:overrides is {'admin_socket': {'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd inject bad map crc probability': 0.1, 'osd map cache size': 1, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub during recovery': False, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'osd_map_cache_size', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}} 2024-09-18T11:30:00.105 INFO:tasks.rados:config is {'clients': ['client.0'], 'dedup_chunk_algo': 'fastcdc', 'dedup_chunk_size': '131072', 'enable_dedup': True, 'low_tier_pool': 'low_tier', 'objects': 50, 'op_weights': {'read': 100, 'rollback': 10, 'set_chunk': 30, 'snap_create': 10, 'snap_remove': 10, 'tier_evict': 10, 'tier_flush': 5, 'tier_promote': 10, 'write': 50}, 'ops': 1500, 'set_chunk': True} 2024-09-18T11:30:00.105 DEBUG:teuthology.run_tasks:Unwinding manager rados 2024-09-18T11:30:00.178 INFO:tasks.rados:joining rados 2024-09-18T11:30:00.178 INFO:tasks.rados:clients are ['client.0', 'client.1'] 2024-09-18T11:30:00.178 INFO:tasks.rados:starting run 0 out of 1 2024-09-18T11:30:00.178 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-18T11:30:00.178 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 2024-09-18T11:30:00.180 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:00.180 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:00.181 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:30:00.181 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:00.178+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:00.245 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:00.243+0000 7f6bf3052640 -1 osd.5 20 handle_osd_map injecting map crc failure 2024-09-18T11:30:00.281 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:00.279+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:00.321 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:00.321 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:00.331 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:30:00.381 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:00.380+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:00.431 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:00.431 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:00.439 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:30:00.482 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:00.480+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:00.578 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:00.578 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:00.582 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:00.581+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:00.586 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:30:00.683 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:00.682+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:00.699 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:00.699 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:00.707 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:30:00.784 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:00.783+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:00.806 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:00.806 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:00.814 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:30:00.885 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:00.884+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:00.912 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:00.912 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:00.921 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:30:00.987 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:00.986+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:01.020 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:01.021 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:01.029 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:30:01.071 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'unique_pool_0' created 2024-09-18T11:30:01.071 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T11:30:01.088 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:01.086+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:01.089 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T11:30:01.090 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-18T11:30:01.131 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:01.131 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:01.138 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:30:01.182 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:30:01.188 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:01.186+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:01.236 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:01.236 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:01.245 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:30:01.303 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:30:01.303 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:30:01.319 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:01.319 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:01.333 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:01.331+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:01.335 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:01.335 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:01.350 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:01.350 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:01.367 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:01.367 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:01.382 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:01.386 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:01.386 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:01.389 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:01.387+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:01.395 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:30:01.397 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:01.397 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:01.413 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:01.413 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:01.491 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:01.490+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:01.532 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:01.533 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:01.533 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:01.533 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:01.533 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:01.541 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:30:01.592 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:01.591+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:01.638 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:01.639 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:01.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:01.639 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [] 2024-09-18T11:30:01.639 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:01.647 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:30:01.693 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:01.692+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:01.785 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:01.785 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:01.785 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:01.785 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:01.794 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:30:01.794 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:01.792+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:01.891 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:01.891 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:01.891 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:01.891 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:01.891 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:01.895 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:01.893+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:01.899 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:30:01.995 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:01.994+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:02.040 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:02.040 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:02.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:02.040 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:02.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.040 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:41 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:02.041 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157303+0000", 2024-09-18T11:30:02.041 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.879959080999999, 2024-09-18T11:30:02.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054616040000000001, 2024-09-18T11:30:02.041 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:02.041 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.041 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:02.041 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:02.041 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:02.041 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 41 2024-09-18T11:30:02.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.041 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:02.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:02.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157303+0000", 2024-09-18T11:30:02.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:02.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157303+0000", 2024-09-18T11:30:02.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.043 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:02.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157303+0000", 2024-09-18T11:30:02.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.043 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:02.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157388+0000", 2024-09-18T11:30:02.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4938999999999994e-05 2024-09-18T11:30:02.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.044 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:02.044 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157395+0000", 2024-09-18T11:30:02.044 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1130000000000003e-06 2024-09-18T11:30:02.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.044 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.045 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157405+0000", 2024-09-18T11:30:02.045 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7510000000000003e-06 2024-09-18T11:30:02.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.045 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:02.045 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158213+0000", 2024-09-18T11:30:02.045 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00080873000000000002 2024-09-18T11:30:02.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.045 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:02.045 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158267+0000", 2024-09-18T11:30:02.045 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3709999999999999e-05 2024-09-18T11:30:02.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.046 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:02.046 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158352+0000", 2024-09-18T11:30:02.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4782000000000002e-05 2024-09-18T11:30:02.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.047 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:02.047 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160262+0000", 2024-09-18T11:30:02.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001910068 2024-09-18T11:30:02.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.047 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:02.047 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162672+0000", 2024-09-18T11:30:02.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024101159999999999 2024-09-18T11:30:02.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.048 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:02.048 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162724+0000", 2024-09-18T11:30:02.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1637e-05 2024-09-18T11:30:02.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.048 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:02.048 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162765+0000", 2024-09-18T11:30:02.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1282000000000002e-05 2024-09-18T11:30:02.049 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.049 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:02.049 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.050 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:02.051 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:02.051 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.879849368, 2024-09-18T11:30:02.051 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054864400000000004, 2024-09-18T11:30:02.051 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:02.051 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.051 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:02.052 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:02.052 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:02.052 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 42 2024-09-18T11:30:02.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.052 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:02.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.052 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:02.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:02.052 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.053 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:02.053 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:02.053 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.054 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:02.054 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:02.054 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.054 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:02.054 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157416+0000", 2024-09-18T11:30:02.054 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.721e-06 2024-09-18T11:30:02.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.054 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:02.054 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157423+0000", 2024-09-18T11:30:02.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4049999999999997e-06 2024-09-18T11:30:02.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.055 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157425+0000", 2024-09-18T11:30:02.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.481e-06 2024-09-18T11:30:02.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:02.055 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158649+0000", 2024-09-18T11:30:02.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001223453 2024-09-18T11:30:02.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:02.056 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158729+0000", 2024-09-18T11:30:02.056 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0205999999999997e-05 2024-09-18T11:30:02.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.056 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:02.056 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158875+0000", 2024-09-18T11:30:02.056 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014628800000000001 2024-09-18T11:30:02.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.056 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:02.056 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161518+0000", 2024-09-18T11:30:02.056 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026427949999999999 2024-09-18T11:30:02.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.057 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:02.057 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162821+0000", 2024-09-18T11:30:02.057 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013028099999999999 2024-09-18T11:30:02.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.057 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:02.057 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162864+0000", 2024-09-18T11:30:02.057 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2839000000000001e-05 2024-09-18T11:30:02.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.058 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:02.058 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162900+0000", 2024-09-18T11:30:02.058 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5948999999999998e-05 2024-09-18T11:30:02.058 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.058 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:02.058 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.058 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.058 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:02.059 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:02.059 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.879763004000001, 2024-09-18T11:30:02.059 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055400850000000001, 2024-09-18T11:30:02.059 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:02.059 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.059 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:02.059 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:02.059 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:02.059 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 43 2024-09-18T11:30:02.059 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.059 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:02.060 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.060 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:02.060 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:02.060 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.060 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.060 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.060 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:02.060 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:02.060 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.060 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.060 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.060 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:02.060 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:02.061 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:02.061 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.061 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.061 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:02.061 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157504+0000", 2024-09-18T11:30:02.061 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4519999999999996e-06 2024-09-18T11:30:02.061 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.061 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.061 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:02.061 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157534+0000", 2024-09-18T11:30:02.061 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.97e-05 2024-09-18T11:30:02.061 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.061 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.062 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.062 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157539+0000", 2024-09-18T11:30:02.062 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4360000000000001e-06 2024-09-18T11:30:02.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.062 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.062 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:02.062 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159428+0000", 2024-09-18T11:30:02.062 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018888220000000001 2024-09-18T11:30:02.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.062 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.062 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:02.062 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159479+0000", 2024-09-18T11:30:02.062 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0608000000000001e-05 2024-09-18T11:30:02.063 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.063 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.063 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:02.063 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159559+0000", 2024-09-18T11:30:02.063 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0667000000000005e-05 2024-09-18T11:30:02.063 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.063 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.063 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:02.063 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162779+0000", 2024-09-18T11:30:02.063 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032197459999999999 2024-09-18T11:30:02.063 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.063 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.063 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:02.063 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162950+0000", 2024-09-18T11:30:02.064 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017059599999999999 2024-09-18T11:30:02.064 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.064 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:02.064 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163001+0000", 2024-09-18T11:30:02.064 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0968999999999999e-05 2024-09-18T11:30:02.064 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.064 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:02.064 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163040+0000", 2024-09-18T11:30:02.064 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8843000000000001e-05 2024-09-18T11:30:02.064 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.064 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:02.065 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.065 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.065 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:02.065 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:02.065 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.87971185, 2024-09-18T11:30:02.065 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:30:02.065 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:02.065 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.065 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:02.065 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:02.065 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:02.066 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:30:02.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.066 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:02.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.066 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:02.066 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:02.066 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.066 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:02.066 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:02.066 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.067 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:02.067 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:30:02.067 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.067 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:02.067 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:30:02.067 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:30:02.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.067 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:02.067 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:30:02.067 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:30:02.068 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.068 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.068 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.068 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:30:02.068 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:30:02.068 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.068 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.068 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:02.068 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:30:02.069 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:30:02.069 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.069 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.069 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:02.069 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:30:02.069 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:30:02.069 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.069 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.069 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:02.069 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:30:02.069 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:30:02.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.070 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:02.070 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:30:02.070 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:30:02.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.070 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:02.070 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:30:02.070 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:30:02.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.071 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:02.071 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:30:02.071 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:30:02.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.071 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.071 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:02.072 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:30:02.072 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:30:02.072 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.072 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:02.072 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.072 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.072 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.072 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:02.072 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:02.072 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.879585587999999, 2024-09-18T11:30:02.073 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055717040000000002, 2024-09-18T11:30:02.073 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:02.073 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.073 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:02.073 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:02.073 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:02.073 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 45 2024-09-18T11:30:02.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.073 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:02.073 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.073 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:02.073 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:02.073 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.074 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:02.074 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:02.074 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.074 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:02.074 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157676+0000", 2024-09-18T11:30:02.074 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.074 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:02.075 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157694+0000", 2024-09-18T11:30:02.075 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7422999999999999e-05 2024-09-18T11:30:02.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.075 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:02.075 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157704+0000", 2024-09-18T11:30:02.075 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9000000000000001e-06 2024-09-18T11:30:02.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.075 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.075 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157718+0000", 2024-09-18T11:30:02.075 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4788e-05 2024-09-18T11:30:02.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.076 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:02.076 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160937+0000", 2024-09-18T11:30:02.076 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00321886 2024-09-18T11:30:02.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.076 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:02.076 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161023+0000", 2024-09-18T11:30:02.076 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6063999999999995e-05 2024-09-18T11:30:02.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.076 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:02.076 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161202+0000", 2024-09-18T11:30:02.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017912599999999999 2024-09-18T11:30:02.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.077 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:02.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163052+0000", 2024-09-18T11:30:02.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001849554 2024-09-18T11:30:02.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.077 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:02.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163204+0000", 2024-09-18T11:30:02.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015186900000000001 2024-09-18T11:30:02.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:02.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163229+0000", 2024-09-18T11:30:02.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:30:02.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:02.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163249+0000", 2024-09-18T11:30:02.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9992000000000001e-05 2024-09-18T11:30:02.078 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.079 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:02.079 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.079 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:02.079 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:02.079 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.879536475, 2024-09-18T11:30:02.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:30:02.079 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:02.079 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.079 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:02.079 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:02.079 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:02.079 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:30:02.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.080 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:02.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:02.080 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:02.080 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:02.080 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:02.080 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:02.081 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:02.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.081 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:02.081 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:30:02.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:30:02.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.081 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:02.081 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:30:02.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:30:02.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.082 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.082 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:30:02.082 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:30:02.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.082 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:02.082 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:30:02.083 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:30:02.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.083 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:02.083 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:30:02.083 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:30:02.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.084 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:02.084 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:30:02.084 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:30:02.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.084 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:02.084 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:30:02.084 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:30:02.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.084 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:02.084 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:30:02.084 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:30:02.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:02.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:30:02.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:30:02.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:02.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:30:02.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:30:02.085 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.085 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:02.085 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.086 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:02.086 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:02.086 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.879493944, 2024-09-18T11:30:02.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:30:02.086 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:02.086 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.086 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:02.086 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:02.086 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:02.086 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:30:02.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.086 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:02.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.087 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:02.087 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:02.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.087 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:02.087 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:02.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.087 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:02.087 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:30:02.088 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.088 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:02.088 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:30:02.088 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:30:02.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.088 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:02.088 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:30:02.088 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:30:02.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.088 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.089 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:30:02.089 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:30:02.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.089 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:02.089 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:30:02.089 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:30:02.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:02.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:30:02.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:30:02.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:02.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:30:02.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:30:02.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:02.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:30:02.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:30:02.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:02.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:30:02.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:30:02.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.092 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:02.092 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:30:02.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:30:02.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.092 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:02.092 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:30:02.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:30:02.092 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.092 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:02.092 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.093 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:02.093 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:02.093 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.879439014000001, 2024-09-18T11:30:02.093 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:30:02.093 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:02.093 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.093 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:02.093 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:02.093 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:02.093 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:30:02.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.094 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:02.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.094 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:02.094 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:02.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.094 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:02.094 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:02.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.095 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:02.095 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:02.095 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.095 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:02.095 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:30:02.095 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:30:02.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.095 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:02.095 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:30:02.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:30:02.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.096 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:30:02.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:30:02.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:02.096 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:30:02.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:30:02.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:02.097 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:30:02.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:30:02.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:02.097 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:30:02.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:30:02.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:02.097 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:30:02.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:30:02.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.098 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:02.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:30:02.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:30:02.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.098 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:02.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:30:02.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:30:02.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.098 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:02.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:30:02.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:30:02.099 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.099 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:02.099 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.099 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:02.099 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:02.099 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.879429033999999, 2024-09-18T11:30:02.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:30:02.099 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:02.099 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.099 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:02.100 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:02.100 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:02.100 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:30:02.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.100 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:02.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:02.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:02.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:02.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:02.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.101 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:02.101 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:02.101 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:02.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.101 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:02.101 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:30:02.101 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:30:02.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.101 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:02.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:30:02.102 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:30:02.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.102 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:30:02.102 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:30:02.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.102 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:02.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:30:02.102 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:30:02.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.103 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:02.103 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:30:02.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:30:02.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.103 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:02.103 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:30:02.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:30:02.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.103 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:02.103 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:30:02.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:30:02.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:02.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:30:02.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:30:02.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:02.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:30:02.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:30:02.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:02.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:30:02.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:30:02.105 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.105 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:02.105 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.105 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:02.105 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:02.105 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.879417435000001, 2024-09-18T11:30:02.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:30:02.105 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:02.105 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.105 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:02.106 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:02.106 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:02.106 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:30:02.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.106 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:02.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:02.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:02.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:02.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:02.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:02.107 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:02.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:02.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:02.107 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:30:02.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:30:02.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:30:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:30:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:30:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:30:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:30:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:30:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:30:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:30:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:30:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:30:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:30:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:30:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:30:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:30:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:30:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:30:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:30:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:30:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.879360953999999, 2024-09-18T11:30:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:30:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:30:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:30:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:30:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:30:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:30:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:30:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:30:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:30:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:30:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:30:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:30:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:30:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:30:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:30:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:30:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:30:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:30:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:30:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:30:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:30:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:30:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:30:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:02.118 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.118 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:02.118 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:02.118 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.879347129999999, 2024-09-18T11:30:02.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:30:02.118 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:02.118 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.118 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:02.118 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:02.118 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:02.118 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:30:02.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.119 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:02.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.119 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:02.119 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:02.119 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.119 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:02.119 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:02.119 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.119 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:02.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:02.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:02.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:02.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:30:02.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:30:02.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:02.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:30:02.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:30:02.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:30:02.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:30:02.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:02.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:30:02.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:30:02.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:02.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:30:02.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:30:02.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.122 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:02.122 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:30:02.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:30:02.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.122 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:02.122 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:30:02.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:30:02.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.122 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:02.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:30:02.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:30:02.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:02.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:30:02.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:30:02.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:02.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:30:02.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:30:02.123 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.123 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:02.124 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.124 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:02.124 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:02.124 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.463168223, 2024-09-18T11:30:02.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:30:02.124 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:02.124 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.125 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:02.125 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:02.125 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:02.125 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:30:02.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.125 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:02.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.125 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:02.125 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:02.125 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.125 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:02.125 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:02.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.126 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:02.126 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:02.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:02.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.126 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:02.126 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:30:02.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:30:02.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.127 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:02.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:30:02.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:30:02.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.127 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:30:02.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:30:02.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.127 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:02.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:30:02.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:30:02.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:02.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:30:02.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:30:02.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:02.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:30:02.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:30:02.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:02.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:30:02.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:30:02.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:02.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:30:02.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:30:02.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:02.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:30:02.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:30:02.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.130 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:02.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:30:02.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:30:02.130 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.130 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:02.130 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.130 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:02.130 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:02.131 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.463144923, 2024-09-18T11:30:02.131 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:30:02.131 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:02.131 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.131 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:02.131 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:02.131 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:02.131 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:30:02.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.131 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:02.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.131 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:02.132 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:02.132 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.132 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:02.132 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:02.132 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.132 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:02.132 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:30:02.132 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.133 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.133 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:02.133 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:30:02.133 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:30:02.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.133 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.133 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:02.133 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:30:02.133 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:30:02.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.133 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.133 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.133 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:30:02.133 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:30:02.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:02.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:30:02.134 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:30:02.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:02.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:30:02.134 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:30:02.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:02.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:30:02.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:30:02.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.135 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:02.135 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:30:02.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:30:02.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.135 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:02.135 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:30:02.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:30:02.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.136 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:02.136 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:30:02.136 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:30:02.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.136 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:02.136 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:30:02.136 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:30:02.136 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.136 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:02.136 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.136 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:02.137 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:02.137 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.463059314000001, 2024-09-18T11:30:02.137 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:30:02.137 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:02.137 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.137 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:02.137 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:02.137 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:02.137 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:30:02.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.137 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:02.137 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.137 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:02.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:02.138 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.138 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:02.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:02.138 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.138 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:02.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:02.138 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.139 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:02.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:30:02.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:30:02.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.139 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:02.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:30:02.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:30:02.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.139 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.140 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:30:02.140 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:30:02.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.140 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:02.140 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:30:02.140 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:30:02.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.140 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:02.140 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:30:02.140 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:30:02.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.141 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:02.141 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:30:02.141 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:30:02.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.141 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:02.141 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:30:02.141 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:30:02.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.141 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:02.141 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:30:02.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:30:02.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.142 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:02.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:30:02.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:30:02.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.143 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:02.143 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:30:02.143 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:30:02.143 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.143 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:02.143 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.143 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:02.143 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:02.143 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.463048348999999, 2024-09-18T11:30:02.144 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:30:02.144 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:02.144 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.144 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:02.144 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:02.144 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:02.144 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:30:02.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.144 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:02.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.144 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:02.144 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:02.145 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.145 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:02.145 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:02.145 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.145 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:02.145 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:02.145 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:02.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.146 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.146 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:02.146 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:30:02.146 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:30:02.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.146 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.146 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:02.146 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:30:02.146 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:30:02.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.146 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.146 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.146 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:30:02.146 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:30:02.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.147 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:02.147 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:30:02.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:30:02.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.147 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:02.147 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:30:02.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:30:02.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.147 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:02.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:30:02.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:30:02.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:02.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:30:02.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:30:02.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:02.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:30:02.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:30:02.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.149 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:02.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:30:02.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:30:02.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.149 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:02.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:30:02.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:30:02.152 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.152 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:02.152 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.153 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:02.153 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:02.153 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.463037181000001, 2024-09-18T11:30:02.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:30:02.153 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:02.153 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.153 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:02.153 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:02.153 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:02.153 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:30:02.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.153 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:02.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:02.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:02.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:02.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:02.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:02.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:02.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:02.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.155 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:02.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:30:02.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:30:02.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.155 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:02.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:30:02.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:30:02.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.156 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:30:02.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:30:02.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.156 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:02.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:30:02.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:30:02.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.156 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:02.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:30:02.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:30:02.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.157 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:02.157 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:30:02.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:30:02.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.157 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:02.157 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:30:02.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:30:02.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.157 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:02.158 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:30:02.158 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:30:02.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.158 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:02.158 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:30:02.158 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:30:02.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.158 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:02.158 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:30:02.158 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:30:02.158 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.159 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:02.159 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.159 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:02.159 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:02.159 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.462971852000001, 2024-09-18T11:30:02.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:30:02.159 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:02.159 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.159 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:02.159 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:02.159 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:02.160 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:30:02.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.160 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:02.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.160 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:02.160 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:02.160 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.160 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:02.160 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:02.160 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:02.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:30:02.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:02.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:30:02.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:30:02.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:02.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:30:02.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:30:02.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:30:02.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:30:02.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:02.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:30:02.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:30:02.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.163 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:02.163 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:30:02.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:30:02.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.163 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:02.163 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:30:02.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:30:02.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.163 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:02.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:30:02.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:30:02.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.164 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:02.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:30:02.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:30:02.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.164 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:02.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:30:02.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:30:02.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:02.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:30:02.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:30:02.165 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.165 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:02.165 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.165 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:02.165 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:02.165 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.452809874, 2024-09-18T11:30:02.166 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:30:02.166 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:02.166 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.166 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:02.166 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:02.166 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:02.166 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:30:02.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.166 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:02.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:02.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:02.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:02.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:02.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:02.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:30:02.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:02.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:30:02.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:30:02.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:02.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:30:02.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:30:02.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:30:02.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:30:02.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:02.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:30:02.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:30:02.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:02.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:30:02.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:30:02.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:02.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:30:02.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:30:02.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:02.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:30:02.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:30:02.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:02.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:30:02.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:30:02.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.172 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:02.172 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:30:02.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:30:02.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.172 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:02.172 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:30:02.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:30:02.172 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.172 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:02.172 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.173 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:02.173 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:02.173 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.452717395000001, 2024-09-18T11:30:02.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:30:02.173 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:02.173 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.173 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:02.173 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:02.173 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:02.173 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:30:02.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.174 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:02.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.174 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:02.174 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:02.174 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.174 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:02.174 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:02.174 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:02.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.175 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:02.175 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:02.175 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.175 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:02.175 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:30:02.175 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:30:02.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:02.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:30:02.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:30:02.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:30:02.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:30:02.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:02.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:30:02.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:30:02.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:02.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:30:02.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:30:02.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:02.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:30:02.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:30:02.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:02.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:30:02.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:30:02.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:02.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:30:02.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:30:02.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:02.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:30:02.179 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:30:02.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:02.179 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:02.179 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:02.179 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:30:02.179 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:30:02.179 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.179 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:02.179 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.179 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:02.179 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:02.179 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:02.180 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T11:30:02.181 INFO:teuthology.orchestra.run.smithi119.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-18T11:30:02.181 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T11:30:02.182 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:30:02.184 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T11:30:02.184 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:02.180+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:02.282 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:02.281+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:02.333 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:02.333 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:02.333 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:02.333 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:02.346 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:30:02.389 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:02.388+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:02.484 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:02.483+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:02.487 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:02.487 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:02.487 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:02.487 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:02.487 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:02.496 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:30:02.585 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:02.584+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:02.597 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:02.597 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:02.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:02.597 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:02.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.597 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:41 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'19)", 2024-09-18T11:30:02.597 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.158446+0000", 2024-09-18T11:30:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.436326727999999, 2024-09-18T11:30:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028837609999999999, 2024-09-18T11:30:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158446+0000", 2024-09-18T11:30:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158446+0000", 2024-09-18T11:30:02.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158446+0000", 2024-09-18T11:30:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158513+0000", 2024-09-18T11:30:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7379999999999995e-05 2024-09-18T11:30:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158520+0000", 2024-09-18T11:30:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0149999999999998e-06 2024-09-18T11:30:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158524+0000", 2024-09-18T11:30:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4400000000000001e-06 2024-09-18T11:30:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158551+0000", 2024-09-18T11:30:02.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7331e-05 2024-09-18T11:30:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158562+0000", 2024-09-18T11:30:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0951999999999999e-05 2024-09-18T11:30:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.161297+0000", 2024-09-18T11:30:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027348810000000002 2024-09-18T11:30:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.161330+0000", 2024-09-18T11:30:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3104e-05 2024-09-18T11:30:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:42 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'24, mlcod=15'19)", 2024-09-18T11:30:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.158950+0000", 2024-09-18T11:30:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.435823056, 2024-09-18T11:30:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026570640000000002, 2024-09-18T11:30:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:02.602 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158950+0000", 2024-09-18T11:30:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158950+0000", 2024-09-18T11:30:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158950+0000", 2024-09-18T11:30:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158953+0000", 2024-09-18T11:30:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1370000000000002e-06 2024-09-18T11:30:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:02.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158960+0000", 2024-09-18T11:30:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4630000000000004e-06 2024-09-18T11:30:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158964+0000", 2024-09-18T11:30:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0960000000000003e-06 2024-09-18T11:30:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158990+0000", 2024-09-18T11:30:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5332000000000001e-05 2024-09-18T11:30:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.159002+0000", 2024-09-18T11:30:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2242000000000001e-05 2024-09-18T11:30:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.161583+0000", 2024-09-18T11:30:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002580695 2024-09-18T11:30:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.161607+0000", 2024-09-18T11:30:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4474e-05 2024-09-18T11:30:02.606 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:43 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'25, mlcod=15'21)", 2024-09-18T11:30:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.159677+0000", 2024-09-18T11:30:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.435096099999999, 2024-09-18T11:30:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001947603, 2024-09-18T11:30:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.159677+0000", 2024-09-18T11:30:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.159677+0000", 2024-09-18T11:30:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.159676+0000", 2024-09-18T11:30:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.159681+0000", 2024-09-18T11:30:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3760000000000001e-06 2024-09-18T11:30:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.159689+0000", 2024-09-18T11:30:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7819999999999994e-06 2024-09-18T11:30:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.159694+0000", 2024-09-18T11:30:02.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4090000000000002e-06 2024-09-18T11:30:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.159723+0000", 2024-09-18T11:30:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8955e-05 2024-09-18T11:30:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.159734+0000", 2024-09-18T11:30:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1201e-05 2024-09-18T11:30:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.161609+0000", 2024-09-18T11:30:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018749680000000001 2024-09-18T11:30:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.161625+0000", 2024-09-18T11:30:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5449000000000001e-05 2024-09-18T11:30:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.611 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:66 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'44, mlcod=15'40)", 2024-09-18T11:30:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.574964+0000", 2024-09-18T11:30:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.019808849, 2024-09-18T11:30:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014251929999999999, 2024-09-18T11:30:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.574964+0000", 2024-09-18T11:30:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.574964+0000", 2024-09-18T11:30:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.574964+0000", 2024-09-18T11:30:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.575026+0000", 2024-09-18T11:30:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1817999999999997e-05 2024-09-18T11:30:02.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.575032+0000", 2024-09-18T11:30:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3840000000000002e-06 2024-09-18T11:30:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.575037+0000", 2024-09-18T11:30:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7219999999999999e-06 2024-09-18T11:30:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:02.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.575063+0000", 2024-09-18T11:30:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6041e-05 2024-09-18T11:30:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.575074+0000", 2024-09-18T11:30:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1325000000000001e-05 2024-09-18T11:30:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.576358+0000", 2024-09-18T11:30:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012834299999999999 2024-09-18T11:30:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.576389+0000", 2024-09-18T11:30:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1822000000000001e-05 2024-09-18T11:30:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:71 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'49, mlcod=15'46)", 2024-09-18T11:30:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.578925+0000", 2024-09-18T11:30:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.015847931, 2024-09-18T11:30:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001431035, 2024-09-18T11:30:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:02.616 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.578925+0000", 2024-09-18T11:30:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.578925+0000", 2024-09-18T11:30:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.578925+0000", 2024-09-18T11:30:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.578932+0000", 2024-09-18T11:30:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9079999999999998e-06 2024-09-18T11:30:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.578939+0000", 2024-09-18T11:30:02.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8040000000000002e-06 2024-09-18T11:30:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.578943+0000", 2024-09-18T11:30:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.304e-06 2024-09-18T11:30:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.578960+0000", 2024-09-18T11:30:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7506000000000001e-05 2024-09-18T11:30:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:02.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.578971+0000", 2024-09-18T11:30:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1087000000000001e-05 2024-09-18T11:30:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580314+0000", 2024-09-18T11:30:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013431809999999999 2024-09-18T11:30:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580356+0000", 2024-09-18T11:30:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1727999999999998e-05 2024-09-18T11:30:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.620 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:73 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'51, mlcod=15'47)", 2024-09-18T11:30:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.580104+0000", 2024-09-18T11:30:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.014668564000001, 2024-09-18T11:30:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019781070000000002, 2024-09-18T11:30:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:02.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580104+0000", 2024-09-18T11:30:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580104+0000", 2024-09-18T11:30:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580104+0000", 2024-09-18T11:30:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580109+0000", 2024-09-18T11:30:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8550000000000001e-06 2024-09-18T11:30:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580117+0000", 2024-09-18T11:30:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9389999999999997e-06 2024-09-18T11:30:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580124+0000", 2024-09-18T11:30:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.221e-06 2024-09-18T11:30:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580373+0000", 2024-09-18T11:30:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00024910600000000002 2024-09-18T11:30:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580386+0000", 2024-09-18T11:30:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2843999999999999e-05 2024-09-18T11:30:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.582062+0000", 2024-09-18T11:30:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016761059999999999 2024-09-18T11:30:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.582083+0000", 2024-09-18T11:30:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0647999999999999e-05 2024-09-18T11:30:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:74 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'52, mlcod=15'47)", 2024-09-18T11:30:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.580465+0000", 2024-09-18T11:30:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.014307758999999, 2024-09-18T11:30:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019207899999999999, 2024-09-18T11:30:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580465+0000", 2024-09-18T11:30:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580465+0000", 2024-09-18T11:30:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580465+0000", 2024-09-18T11:30:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:02.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580469+0000", 2024-09-18T11:30:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1450000000000001e-06 2024-09-18T11:30:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580483+0000", 2024-09-18T11:30:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3722e-05 2024-09-18T11:30:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580487+0000", 2024-09-18T11:30:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.549e-06 2024-09-18T11:30:02.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580555+0000", 2024-09-18T11:30:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7729000000000002e-05 2024-09-18T11:30:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580567+0000", 2024-09-18T11:30:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1707e-05 2024-09-18T11:30:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:02.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.582368+0000", 2024-09-18T11:30:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001801162 2024-09-18T11:30:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.582386+0000", 2024-09-18T11:30:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8304999999999998e-05 2024-09-18T11:30:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:75 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'53, mlcod=15'47)", 2024-09-18T11:30:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.580904+0000", 2024-09-18T11:30:02.630 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.013869373, 2024-09-18T11:30:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015037749999999999, 2024-09-18T11:30:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580904+0000", 2024-09-18T11:30:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580904+0000", 2024-09-18T11:30:02.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580903+0000", 2024-09-18T11:30:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580966+0000", 2024-09-18T11:30:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2935999999999995e-05 2024-09-18T11:30:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580973+0000", 2024-09-18T11:30:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3780000000000003e-06 2024-09-18T11:30:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.580977+0000", 2024-09-18T11:30:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3079999999999999e-06 2024-09-18T11:30:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.581003+0000", 2024-09-18T11:30:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5831000000000001e-05 2024-09-18T11:30:02.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.581014+0000", 2024-09-18T11:30:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0994e-05 2024-09-18T11:30:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.582388+0000", 2024-09-18T11:30:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001374197 2024-09-18T11:30:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.582407+0000", 2024-09-18T11:30:02.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9636e-05 2024-09-18T11:30:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:77 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'55, mlcod=15'53)", 2024-09-18T11:30:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.584563+0000", 2024-09-18T11:30:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.010210196999999, 2024-09-18T11:30:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037297200000000002, 2024-09-18T11:30:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:02.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584563+0000", 2024-09-18T11:30:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584563+0000", 2024-09-18T11:30:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584562+0000", 2024-09-18T11:30:02.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584613+0000", 2024-09-18T11:30:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0892e-05 2024-09-18T11:30:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584648+0000", 2024-09-18T11:30:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4533999999999998e-05 2024-09-18T11:30:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584653+0000", 2024-09-18T11:30:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9749999999999998e-06 2024-09-18T11:30:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584687+0000", 2024-09-18T11:30:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4453999999999999e-05 2024-09-18T11:30:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584699+0000", 2024-09-18T11:30:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1605e-05 2024-09-18T11:30:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.588269+0000", 2024-09-18T11:30:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035706650000000002 2024-09-18T11:30:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.588293+0000", 2024-09-18T11:30:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3226000000000001e-05 2024-09-18T11:30:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.639 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:78 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'56, mlcod=15'53)", 2024-09-18T11:30:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.585042+0000", 2024-09-18T11:30:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.009731126, 2024-09-18T11:30:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032615389999999999, 2024-09-18T11:30:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585042+0000", 2024-09-18T11:30:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585042+0000", 2024-09-18T11:30:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585041+0000", 2024-09-18T11:30:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585129+0000", 2024-09-18T11:30:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7905000000000006e-05 2024-09-18T11:30:02.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585138+0000", 2024-09-18T11:30:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9069999999999996e-06 2024-09-18T11:30:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585143+0000", 2024-09-18T11:30:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5329999999999996e-06 2024-09-18T11:30:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:02.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585166+0000", 2024-09-18T11:30:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3308000000000001e-05 2024-09-18T11:30:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585177+0000", 2024-09-18T11:30:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1255e-05 2024-09-18T11:30:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.588294+0000", 2024-09-18T11:30:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031165989999999998 2024-09-18T11:30:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:02.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.588303+0000", 2024-09-18T11:30:02.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5279999999999992e-06 2024-09-18T11:30:02.644 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.644 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:02.644 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.644 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:79 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'57, mlcod=15'53)", 2024-09-18T11:30:02.644 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.586176+0000", 2024-09-18T11:30:02.644 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.008596817999999, 2024-09-18T11:30:02.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043147059999999997, 2024-09-18T11:30:02.644 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586176+0000", 2024-09-18T11:30:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586176+0000", 2024-09-18T11:30:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586170+0000", 2024-09-18T11:30:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999938 2024-09-18T11:30:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586282+0000", 2024-09-18T11:30:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112254 2024-09-18T11:30:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586307+0000", 2024-09-18T11:30:02.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5296e-05 2024-09-18T11:30:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586321+0000", 2024-09-18T11:30:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3723000000000001e-05 2024-09-18T11:30:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.587754+0000", 2024-09-18T11:30:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014330829999999999 2024-09-18T11:30:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:02.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.587785+0000", 2024-09-18T11:30:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1176000000000001e-05 2024-09-18T11:30:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.590467+0000", 2024-09-18T11:30:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026815089999999999 2024-09-18T11:30:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.590491+0000", 2024-09-18T11:30:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3941e-05 2024-09-18T11:30:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.648 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:80 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'58, mlcod=15'53)", 2024-09-18T11:30:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.586532+0000", 2024-09-18T11:30:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.008241100999999, 2024-09-18T11:30:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041534989999999997, 2024-09-18T11:30:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:02.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586532+0000", 2024-09-18T11:30:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586532+0000", 2024-09-18T11:30:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586531+0000", 2024-09-18T11:30:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586591+0000", 2024-09-18T11:30:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9472000000000003e-05 2024-09-18T11:30:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586598+0000", 2024-09-18T11:30:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0890000000000002e-06 2024-09-18T11:30:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586602+0000", 2024-09-18T11:30:02.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0090000000000001e-06 2024-09-18T11:30:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.588075+0000", 2024-09-18T11:30:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014728429999999999 2024-09-18T11:30:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.588086+0000", 2024-09-18T11:30:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1452e-05 2024-09-18T11:30:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.590672+0000", 2024-09-18T11:30:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025858629999999999 2024-09-18T11:30:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.590685+0000", 2024-09-18T11:30:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3254999999999999e-05 2024-09-18T11:30:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:82 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'60, mlcod=15'58)", 2024-09-18T11:30:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.592335+0000", 2024-09-18T11:30:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.002438516, 2024-09-18T11:30:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020482019999999998, 2024-09-18T11:30:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.592335+0000", 2024-09-18T11:30:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.592335+0000", 2024-09-18T11:30:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.592334+0000", 2024-09-18T11:30:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.592340+0000", 2024-09-18T11:30:02.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6840000000000001e-06 2024-09-18T11:30:02.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:02.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.592348+0000", 2024-09-18T11:30:02.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5210000000000002e-06 2024-09-18T11:30:02.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.592353+0000", 2024-09-18T11:30:02.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.741e-06 2024-09-18T11:30:02.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.592377+0000", 2024-09-18T11:30:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3714e-05 2024-09-18T11:30:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.592387+0000", 2024-09-18T11:30:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0489000000000001e-05 2024-09-18T11:30:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:02.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.594357+0000", 2024-09-18T11:30:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019698710000000002 2024-09-18T11:30:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.594383+0000", 2024-09-18T11:30:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5723000000000001e-05 2024-09-18T11:30:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.658 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:88 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'62, mlcod=15'60)", 2024-09-18T11:30:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:50.031675+0000", 2024-09-18T11:30:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.563097537999999, 2024-09-18T11:30:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035383889999999999, 2024-09-18T11:30:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031675+0000", 2024-09-18T11:30:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031675+0000", 2024-09-18T11:30:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031675+0000", 2024-09-18T11:30:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031682+0000", 2024-09-18T11:30:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.542e-06 2024-09-18T11:30:02.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031692+0000", 2024-09-18T11:30:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7119999999999995e-06 2024-09-18T11:30:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031700+0000", 2024-09-18T11:30:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7819999999999994e-06 2024-09-18T11:30:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031738+0000", 2024-09-18T11:30:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8538000000000003e-05 2024-09-18T11:30:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031757+0000", 2024-09-18T11:30:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8258999999999999e-05 2024-09-18T11:30:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035185+0000", 2024-09-18T11:30:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034287219999999999 2024-09-18T11:30:02.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035214+0000", 2024-09-18T11:30:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8524e-05 2024-09-18T11:30:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:89 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'63, mlcod=15'60)", 2024-09-18T11:30:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:50.032044+0000", 2024-09-18T11:30:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.562728804000001, 2024-09-18T11:30:02.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032268829999999998, 2024-09-18T11:30:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032044+0000", 2024-09-18T11:30:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032044+0000", 2024-09-18T11:30:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032044+0000", 2024-09-18T11:30:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032049+0000", 2024-09-18T11:30:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.78e-06 2024-09-18T11:30:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032057+0000", 2024-09-18T11:30:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5879999999999998e-06 2024-09-18T11:30:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032061+0000", 2024-09-18T11:30:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0820000000000001e-06 2024-09-18T11:30:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032083+0000", 2024-09-18T11:30:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1359999999999999e-05 2024-09-18T11:30:02.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032093+0000", 2024-09-18T11:30:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0662999999999999e-05 2024-09-18T11:30:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035250+0000", 2024-09-18T11:30:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031568429999999999 2024-09-18T11:30:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035271+0000", 2024-09-18T11:30:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1053999999999999e-05 2024-09-18T11:30:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:90 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'64, mlcod=15'60)", 2024-09-18T11:30:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:50.032410+0000", 2024-09-18T11:30:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.562363153, 2024-09-18T11:30:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029831089999999998, 2024-09-18T11:30:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:02.668 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.669 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:02.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:02.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032410+0000", 2024-09-18T11:30:02.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:02.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032410+0000", 2024-09-18T11:30:02.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:02.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032410+0000", 2024-09-18T11:30:02.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:02.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032534+0000", 2024-09-18T11:30:02.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012397600000000001 2024-09-18T11:30:02.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:02.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032540+0000", 2024-09-18T11:30:02.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1500000000000004e-06 2024-09-18T11:30:02.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032544+0000", 2024-09-18T11:30:02.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2910000000000001e-06 2024-09-18T11:30:02.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:02.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032576+0000", 2024-09-18T11:30:02.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2228999999999999e-05 2024-09-18T11:30:02.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:02.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032597+0000", 2024-09-18T11:30:02.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0579000000000001e-05 2024-09-18T11:30:02.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:02.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035344+0000", 2024-09-18T11:30:02.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002747527 2024-09-18T11:30:02.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:02.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035393+0000", 2024-09-18T11:30:02.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8677999999999997e-05 2024-09-18T11:30:02.673 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.673 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:02.673 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.673 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:91 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'65, mlcod=15'60)", 2024-09-18T11:30:02.673 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:50.032758+0000", 2024-09-18T11:30:02.673 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.562015435999999, 2024-09-18T11:30:02.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026551370000000001, 2024-09-18T11:30:02.673 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:02.673 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.673 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:02.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:02.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032758+0000", 2024-09-18T11:30:02.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:02.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032758+0000", 2024-09-18T11:30:02.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:02.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032757+0000", 2024-09-18T11:30:02.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:02.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032761+0000", 2024-09-18T11:30:02.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9570000000000004e-06 2024-09-18T11:30:02.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:02.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032766+0000", 2024-09-18T11:30:02.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7770000000000002e-06 2024-09-18T11:30:02.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032768+0000", 2024-09-18T11:30:02.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4849999999999999e-06 2024-09-18T11:30:02.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:02.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032880+0000", 2024-09-18T11:30:02.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000111789 2024-09-18T11:30:02.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:02.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032897+0000", 2024-09-18T11:30:02.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6467999999999999e-05 2024-09-18T11:30:02.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:02.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035395+0000", 2024-09-18T11:30:02.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024979310000000001 2024-09-18T11:30:02.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:02.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035413+0000", 2024-09-18T11:30:02.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8111e-05 2024-09-18T11:30:02.677 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.677 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:02.678 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.678 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:92 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'66, mlcod=15'60)", 2024-09-18T11:30:02.678 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:50.033090+0000", 2024-09-18T11:30:02.678 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.561682776, 2024-09-18T11:30:02.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002340745, 2024-09-18T11:30:02.678 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:02.678 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.679 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:02.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:02.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.033090+0000", 2024-09-18T11:30:02.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:02.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.033090+0000", 2024-09-18T11:30:02.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.033090+0000", 2024-09-18T11:30:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.033093+0000", 2024-09-18T11:30:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2059999999999999e-06 2024-09-18T11:30:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.033100+0000", 2024-09-18T11:30:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6649999999999997e-06 2024-09-18T11:30:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.033104+0000", 2024-09-18T11:30:02.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9489999999999998e-06 2024-09-18T11:30:02.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:02.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.033132+0000", 2024-09-18T11:30:02.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8388e-05 2024-09-18T11:30:02.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:02.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.033149+0000", 2024-09-18T11:30:02.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6901e-05 2024-09-18T11:30:02.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:02.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035414+0000", 2024-09-18T11:30:02.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002264617 2024-09-18T11:30:02.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:02.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035431+0000", 2024-09-18T11:30:02.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7317999999999999e-05 2024-09-18T11:30:02.682 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.682 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:02.682 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.683 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:93 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'67, mlcod=15'60)", 2024-09-18T11:30:02.683 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:50.033404+0000", 2024-09-18T11:30:02.683 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.561369352, 2024-09-18T11:30:02.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020439999999999998, 2024-09-18T11:30:02.683 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:02.683 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.683 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:02.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:02.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.033404+0000", 2024-09-18T11:30:02.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:02.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.033404+0000", 2024-09-18T11:30:02.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:02.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.033403+0000", 2024-09-18T11:30:02.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:02.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.033463+0000", 2024-09-18T11:30:02.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9543999999999998e-05 2024-09-18T11:30:02.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:02.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.033470+0000", 2024-09-18T11:30:02.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.109e-06 2024-09-18T11:30:02.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.033472+0000", 2024-09-18T11:30:02.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3309999999999998e-06 2024-09-18T11:30:02.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:02.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.033492+0000", 2024-09-18T11:30:02.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9463999999999999e-05 2024-09-18T11:30:02.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:02.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.033502+0000", 2024-09-18T11:30:02.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0219e-05 2024-09-18T11:30:02.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:02.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035432+0000", 2024-09-18T11:30:02.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019298060000000001 2024-09-18T11:30:02.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:02.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035448+0000", 2024-09-18T11:30:02.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5812e-05 2024-09-18T11:30:02.687 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.687 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:02.687 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.687 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:97 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'71, mlcod=16'69)", 2024-09-18T11:30:02.687 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:50.038349+0000", 2024-09-18T11:30:02.688 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.556424133, 2024-09-18T11:30:02.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002504236, 2024-09-18T11:30:02.688 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:02.688 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:02.688 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:02.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:02.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.038349+0000", 2024-09-18T11:30:02.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:02.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.038349+0000", 2024-09-18T11:30:02.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:02.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:02.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.038348+0000", 2024-09-18T11:30:02.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:02.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:02.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.038353+0000", 2024-09-18T11:30:02.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4279999999999998e-06 2024-09-18T11:30:02.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:02.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.038360+0000", 2024-09-18T11:30:02.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8619999999999997e-06 2024-09-18T11:30:02.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:02.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.038364+0000", 2024-09-18T11:30:02.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3780000000000003e-06 2024-09-18T11:30:02.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:02.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.038392+0000", 2024-09-18T11:30:02.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7987000000000002e-05 2024-09-18T11:30:02.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:02.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.038404+0000", 2024-09-18T11:30:02.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1972e-05 2024-09-18T11:30:02.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:02.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.040832+0000", 2024-09-18T11:30:02.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024282430000000001 2024-09-18T11:30:02.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:02.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:02.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:02.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.040853+0000", 2024-09-18T11:30:02.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0911e-05 2024-09-18T11:30:02.692 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.692 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:02.692 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.692 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:02.692 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:02.692 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:02.693 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:30:02.694 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:02.692+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:02.794 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:02.793+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:02.832 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:02.832 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:02.833 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:02.841 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:30:02.895 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:02.894+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:02.939 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:02.940 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:02.940 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:02.940 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:02.940 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:02.948 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:30:02.996 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:02.994+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:03.071 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:03.070+0000 7fb6950ff640 -1 osd.2 23 handle_osd_map injecting map crc failure 2024-09-18T11:30:03.071 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:03.070+0000 7fcb682b1640 -1 osd.7 23 handle_osd_map injecting map crc failure 2024-09-18T11:30:03.073 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T11:30:03.073 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is set 2024-09-18T11:30:03.092 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T11:30:03.096 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4266.0:5 2.2 e17/15 2:5cea7035:::rbd_info:head v 17'1, mlcod=0'0)", 2024-09-18T11:30:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:51.040980+0000", 2024-09-18T11:30:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.054561926, 2024-09-18T11:30:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00070072500000000002, 2024-09-18T11:30:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:51.040980+0000", 2024-09-18T11:30:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:51.040980+0000", 2024-09-18T11:30:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:51.040980+0000", 2024-09-18T11:30:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:30:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:51.040987+0000", 2024-09-18T11:30:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.277e-06 2024-09-18T11:30:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:51.041008+0000", 2024-09-18T11:30:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0608e-05 2024-09-18T11:30:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:51.041016+0000", 2024-09-18T11:30:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9619999999999993e-06 2024-09-18T11:30:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:51.041058+0000", 2024-09-18T11:30:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2181000000000002e-05 2024-09-18T11:30:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:51.041081+0000", 2024-09-18T11:30:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3047000000000001e-05 2024-09-18T11:30:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:51.041633+0000", 2024-09-18T11:30:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00055207900000000002 2024-09-18T11:30:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:51.041681+0000", 2024-09-18T11:30:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7790000000000002e-05 2024-09-18T11:30:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4266.0:7 2.2 e18/15 2:5cea7035:::rbd_info:head v 18'2, mlcod=0'0)", 2024-09-18T11:30:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:52.048021+0000", 2024-09-18T11:30:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.047521472, 2024-09-18T11:30:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0030933409999999999, 2024-09-18T11:30:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048021+0000", 2024-09-18T11:30:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048021+0000", 2024-09-18T11:30:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048020+0000", 2024-09-18T11:30:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:30:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048025+0000", 2024-09-18T11:30:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1949999999999996e-06 2024-09-18T11:30:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048035+0000", 2024-09-18T11:30:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9639999999999998e-06 2024-09-18T11:30:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048041+0000", 2024-09-18T11:30:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6150000000000002e-06 2024-09-18T11:30:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048082+0000", 2024-09-18T11:30:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0453e-05 2024-09-18T11:30:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048100+0000", 2024-09-18T11:30:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8556999999999999e-05 2024-09-18T11:30:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.051080+0000", 2024-09-18T11:30:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029795440000000002 2024-09-18T11:30:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.051114+0000", 2024-09-18T11:30:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4218999999999999e-05 2024-09-18T11:30:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:03.106 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:03.107 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:03.095+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:03.107 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:30:03.184 DEBUG:teuthology.orchestra.run.smithi119:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --set_chunk --enable_dedup --low_tier_pool low_tier --dedup_chunk_size 131072 --dedup_chunk_algo fastcdc --max-ops 1500 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 25 --op delete 10 --op snap_create 10 --op snap_remove 10 --op rollback 10 --op set_chunk 30 --op tier_promote 10 --op tier_evict 10 --op tier_flush 5 --op write_excl 25 --pool unique_pool_0 2024-09-18T11:30:03.197 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:03.196+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:03.219 INFO:tasks.rados.rados.0.smithi119.stdout:adding op weight read -> 100 2024-09-18T11:30:03.219 INFO:tasks.rados.rados.0.smithi119.stdout:adding op weight write -> 25 2024-09-18T11:30:03.219 INFO:tasks.rados.rados.0.smithi119.stdout:adding op weight delete -> 10 2024-09-18T11:30:03.219 INFO:tasks.rados.rados.0.smithi119.stdout:adding op weight snap_create -> 10 2024-09-18T11:30:03.219 INFO:tasks.rados.rados.0.smithi119.stdout:adding op weight snap_remove -> 10 2024-09-18T11:30:03.220 INFO:tasks.rados.rados.0.smithi119.stdout:adding op weight rollback -> 10 2024-09-18T11:30:03.220 INFO:tasks.rados.rados.0.smithi119.stdout:adding op weight set_chunk -> 30 2024-09-18T11:30:03.220 INFO:tasks.rados.rados.0.smithi119.stdout:adding op weight tier_promote -> 10 2024-09-18T11:30:03.220 INFO:tasks.rados.rados.0.smithi119.stdout:adding op weight tier_evict -> 10 2024-09-18T11:30:03.220 INFO:tasks.rados.rados.0.smithi119.stdout:adding op weight tier_flush -> 5 2024-09-18T11:30:03.220 INFO:tasks.rados.rados.0.smithi119.stdout:adding op weight write_excl -> 25 2024-09-18T11:30:03.220 INFO:tasks.rados.rados.0.smithi119.stdout:ceph version 17.2.7-1645-g7061ea6c (7061ea6ca50908d104e9f3979dc615e118d9e153) quincy (stable) 2024-09-18T11:30:03.220 INFO:tasks.rados.rados.0.smithi119.stdout:Configuration: 2024-09-18T11:30:03.220 INFO:tasks.rados.rados.0.smithi119.stdout: Number of operations: 1500 2024-09-18T11:30:03.220 INFO:tasks.rados.rados.0.smithi119.stdout: Number of objects: 50 2024-09-18T11:30:03.220 INFO:tasks.rados.rados.0.smithi119.stdout: Max in flight operations: 16 2024-09-18T11:30:03.220 INFO:tasks.rados.rados.0.smithi119.stdout: Object size (in bytes): 4000000 2024-09-18T11:30:03.220 INFO:tasks.rados.rados.0.smithi119.stdout: Write stride min: 400000 2024-09-18T11:30:03.220 INFO:tasks.rados.rados.0.smithi119.stdout: Write stride max: 800000 2024-09-18T11:30:03.252 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:03.252 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:03.252 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:03.252 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:03.261 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:30:03.299 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:03.297+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:03.399 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:03.398+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:03.400 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:03.400 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:03.400 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:03.400 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:03.400 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:03.409 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:30:03.424 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:30:03.500 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:03.499+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:03.550 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4266.0:1 2.3 e16/15 2:c4c92e5a:::rbd_trash:head v 16'1, mlcod=0'0)", 2024-09-18T11:30:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:50.028963+0000", 2024-09-18T11:30:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.520476553, 2024-09-18T11:30:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001253021, 2024-09-18T11:30:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.028963+0000", 2024-09-18T11:30:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.028963+0000", 2024-09-18T11:30:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.028963+0000", 2024-09-18T11:30:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.028967+0000", 2024-09-18T11:30:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2119999999999997e-06 2024-09-18T11:30:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.029011+0000", 2024-09-18T11:30:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4011000000000003e-05 2024-09-18T11:30:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.029019+0000", 2024-09-18T11:30:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.729e-06 2024-09-18T11:30:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.029185+0000", 2024-09-18T11:30:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016637600000000001 2024-09-18T11:30:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.029211+0000", 2024-09-18T11:30:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6211e-05 2024-09-18T11:30:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.030175+0000", 2024-09-18T11:30:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096347000000000002 2024-09-18T11:30:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.030216+0000", 2024-09-18T11:30:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.138e-05 2024-09-18T11:30:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:03.557 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:03.559 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:30:03.565 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:30:03.565 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:30:03.582 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:03.582 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:03.597 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:03.598 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:03.601 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:03.600+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:03.612 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:03.613 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:03.629 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:03.629 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:03.645 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:03.645 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:03.659 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:03.660 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:03.661 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:03.661 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:03.669 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:30:03.677 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:03.678 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:03.702 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:03.700+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:03.775 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:03.775 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:03.783 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:30:03.827 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:03.826+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:03.882 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:03.882 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:03.890 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:30:03.936 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:03.935+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:04.001 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:04.001 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:04.004 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:04.002+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:04.010 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:30:04.090 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T11:30:04.103 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-18T11:30:04.144 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:04.143+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:04.153 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:04.153 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:04.161 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:30:04.206 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:04.205+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:04.259 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:04.309 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:04.311 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:30:04.311 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:04.309+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:04.411 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:04.410+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:04.451 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:04.451 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:04.459 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:30:04.512 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:04.511+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:04.596 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:04.597 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:04.605 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:30:04.648 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:04.647+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:04.702 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:04.702 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:04.710 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:30:04.751 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:04.750+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:04.817 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:04.816+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:04.849 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:04.849 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:04.857 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:30:04.919 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:04.918+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:04.955 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:04.955 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:04.963 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:30:05.020 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:05.019+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:05.061 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:05.062 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:05.070 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:30:05.089 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T11:30:05.089 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.5 to 1 (10000) 2024-09-18T11:30:05.105 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-18T11:30:05.146 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:05.145+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:05.169 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:05.169 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:05.177 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:30:05.222 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:05.221+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:05.275 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:05.276 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:05.284 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:30:05.323 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:05.322+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:05.379 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:05.377+0000 7f6bf3052640 -1 osd.5 25 handle_osd_map injecting map crc failure 2024-09-18T11:30:05.383 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:05.383 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:05.392 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:30:05.437 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:05.435+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:05.490 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:05.491 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:05.499 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:30:05.525 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:05.524+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:05.627 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:05.625+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:05.640 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:05.641 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:05.646 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:30:05.690 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:30:05.735 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:05.734+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:05.747 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:05.747 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:05.756 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:30:05.807 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:30:05.808 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:30:05.824 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:05.824 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:05.828 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:05.827+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:05.839 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:05.839 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:05.854 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:05.854 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:05.869 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:05.870 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:05.885 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:05.886 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:05.896 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:05.896 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:05.896 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:05.896 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:05.901 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:05.902 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:05.904 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:30:05.917 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:05.918 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:05.929 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:05.928+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:06.014 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:06.014 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:06.014 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:06.014 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:06.014 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:06.023 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:30:06.067 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:06.066+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:06.091 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is unset 2024-09-18T11:30:06.091 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.6 to 1 (10000) 2024-09-18T11:30:06.111 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-18T11:30:06.130 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:06.129+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:06.131 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:06.131 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:06.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:06.131 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:06.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.131 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4266.0:5 2.2 e17/15 2:5cea7035:::rbd_info:head v 17'1, mlcod=0'0)", 2024-09-18T11:30:06.132 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:51.040980+0000", 2024-09-18T11:30:06.132 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.089706609, 2024-09-18T11:30:06.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00070072500000000002, 2024-09-18T11:30:06.132 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:06.132 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:06.132 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:06.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:06.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:51.040980+0000", 2024-09-18T11:30:06.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:06.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:06.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:51.040980+0000", 2024-09-18T11:30:06.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:06.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:06.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:51.040980+0000", 2024-09-18T11:30:06.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:30:06.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:06.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:51.040987+0000", 2024-09-18T11:30:06.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.277e-06 2024-09-18T11:30:06.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:06.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:51.041008+0000", 2024-09-18T11:30:06.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0608e-05 2024-09-18T11:30:06.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:06.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:51.041016+0000", 2024-09-18T11:30:06.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9619999999999993e-06 2024-09-18T11:30:06.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:06.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:51.041058+0000", 2024-09-18T11:30:06.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2181000000000002e-05 2024-09-18T11:30:06.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:06.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:51.041081+0000", 2024-09-18T11:30:06.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3047000000000001e-05 2024-09-18T11:30:06.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:06.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:51.041633+0000", 2024-09-18T11:30:06.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00055207900000000002 2024-09-18T11:30:06.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:06.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:51.041681+0000", 2024-09-18T11:30:06.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7790000000000002e-05 2024-09-18T11:30:06.136 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:06.136 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:06.136 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:06.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.136 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4266.0:7 2.2 e18/15 2:5cea7035:::rbd_info:head v 18'2, mlcod=0'0)", 2024-09-18T11:30:06.137 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:52.048021+0000", 2024-09-18T11:30:06.137 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.082666155, 2024-09-18T11:30:06.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0030933409999999999, 2024-09-18T11:30:06.137 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:06.137 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:06.137 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:06.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:06.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048021+0000", 2024-09-18T11:30:06.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:06.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:06.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048021+0000", 2024-09-18T11:30:06.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:06.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:06.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048020+0000", 2024-09-18T11:30:06.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:30:06.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:06.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048025+0000", 2024-09-18T11:30:06.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1949999999999996e-06 2024-09-18T11:30:06.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:06.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048035+0000", 2024-09-18T11:30:06.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9639999999999998e-06 2024-09-18T11:30:06.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:06.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048041+0000", 2024-09-18T11:30:06.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6150000000000002e-06 2024-09-18T11:30:06.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:06.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048082+0000", 2024-09-18T11:30:06.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0453e-05 2024-09-18T11:30:06.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:06.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048100+0000", 2024-09-18T11:30:06.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8556999999999999e-05 2024-09-18T11:30:06.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:06.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.051080+0000", 2024-09-18T11:30:06.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029795440000000002 2024-09-18T11:30:06.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:06.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.051114+0000", 2024-09-18T11:30:06.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4218999999999999e-05 2024-09-18T11:30:06.141 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:06.141 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:06.141 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:06.141 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:06.141 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:06.141 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:06.142 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:30:06.231 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:06.230+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:06.285 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:06.286 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:06.294 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:30:06.332 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:06.331+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:06.393 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:06.393 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:06.394 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:06.394 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:06.394 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:06.402 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:30:06.434 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:06.432+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:06.501 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:06.501 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:06.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:06.501 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:06.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.501 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4266.0:1 2.3 e16/15 2:c4c92e5a:::rbd_trash:head v 16'1, mlcod=0'0)", 2024-09-18T11:30:06.501 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:50.028963+0000", 2024-09-18T11:30:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 16.471365392999999, 2024-09-18T11:30:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001253021, 2024-09-18T11:30:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.028963+0000", 2024-09-18T11:30:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.028963+0000", 2024-09-18T11:30:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.028963+0000", 2024-09-18T11:30:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:06.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.028967+0000", 2024-09-18T11:30:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2119999999999997e-06 2024-09-18T11:30:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.029011+0000", 2024-09-18T11:30:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4011000000000003e-05 2024-09-18T11:30:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.029019+0000", 2024-09-18T11:30:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.729e-06 2024-09-18T11:30:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.029185+0000", 2024-09-18T11:30:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016637600000000001 2024-09-18T11:30:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:06.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.029211+0000", 2024-09-18T11:30:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6211e-05 2024-09-18T11:30:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.030175+0000", 2024-09-18T11:30:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096347000000000002 2024-09-18T11:30:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.030216+0000", 2024-09-18T11:30:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.138e-05 2024-09-18T11:30:06.506 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:06.507 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:06.507 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:06.507 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:06.507 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:06.507 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:06.509 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:30:06.535 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:06.533+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:06.606 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:06.606 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:06.606 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:06.607 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:06.615 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:30:06.658 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:06.657+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:06.712 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:06.712 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:06.712 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:06.712 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:06.712 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:06.720 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:30:06.761 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:06.760+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:06.837 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:06.836+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:06.858 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:06.858 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:06.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:06.859 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [] 2024-09-18T11:30:06.859 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:06.869 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:30:06.938 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:06.937+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:06.979 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:06.980 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:06.980 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:06.980 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:06.988 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:30:07.072 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:07.071+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:07.090 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:07.089+0000 7f3157e90640 -1 osd.4 26 handle_osd_map injecting map crc failure 2024-09-18T11:30:07.091 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:07.090+0000 7f2d449e8640 -1 osd.1 27 handle_osd_map injecting map crc failure 2024-09-18T11:30:07.092 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.7 to 1 (10000) 2024-09-18T11:30:07.107 INFO:tasks.thrashosds.thrasher:test_map_discontinuity 2024-09-18T11:30:07.107 INFO:tasks.thrashosds.thrasher:Waiting for recovery 2024-09-18T11:30:07.107 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-18T11:30:07.107 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T11:30:07.129 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:07.129 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:07.129 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:07.129 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:07.129 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:07.138 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:30:07.151 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:07.150+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:07.241 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:07.240+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:07.246 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:07.245+0000 7fd27659f640 -1 osd.6 26 handle_osd_map injecting map crc failure 2024-09-18T11:30:07.278 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:07.278 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:07.278 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:07.278 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [] 2024-09-18T11:30:07.278 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:07.287 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:30:07.369 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:30:07.370 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":28,"fsid":"f4cca259-b87a-45cc-8995-7e9fca891a9f","created":"2024-09-18T11:29:38.205487+0000","modified":"2024-09-18T11:30:07.087792+0000","last_up_change":"2024-09-18T11:29:46.934503+0000","last_in_change":"2024-09-18T11:29:40.301770+0000","flags":"nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5804032,"flags_set":["nodeep-scrub","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T11:29:47.951262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T11:29:48.866238+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"low_tier","create_time":"2024-09-18T11:29:59.343964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}},{"pool":4,"pool_name":"unique_pool_0","create_time":"2024-09-18T11:30:00.523818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"dedup_chunk_algorithm":"fastcdc","dedup_tier":3,"fingerprint_algorithm":"sha256"},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"12faced2-7888-4265-ad12-815efe36466c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":33347},{"type":"v1","addr":"172.21.15.119:6801","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":33347},{"type":"v1","addr":"172.21.15.119:6803","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":33347},{"type":"v1","addr":"172.21.15.119:6807","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":33347},{"type":"v1","addr":"172.21.15.119:6805","nonce":33347}]},"public_addr":"172.21.15.119:6801/33347","cluster_addr":"172.21.15.119:6803/33347","heartbeat_back_addr":"172.21.15.119:6807/33347","heartbeat_front_addr":"172.21.15.119:6805/33347","state":["exists","up"]},{"osd":1,"uuid":"20fee29d-b047-46ae-aa60-88e897fbb0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":33350},{"type":"v1","addr":"172.21.15.119:6825","nonce":33350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":33350},{"type":"v1","addr":"172.21.15.119:6827","nonce":33350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":33350},{"type":"v1","addr":"172.21.15.119:6831","nonce":33350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":33350},{"type":"v1","addr":"172.21.15.119:6829","nonce":33350}]},"public_addr":"172.21.15.119:6825/33350","cluster_addr":"172.21.15.119:6827/33350","heartbeat_back_addr":"172.21.15.119:6831/33350","heartbeat_front_addr":"172.21.15.119:6829/33350","state":["exists","up"]},{"osd":2,"uuid":"21a296f3-b259-4841-9a17-4cd21b302b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":33351},{"type":"v1","addr":"172.21.15.119:6809","nonce":33351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":33351},{"type":"v1","addr":"172.21.15.119:6813","nonce":33351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6819","nonce":33351},{"type":"v1","addr":"172.21.15.119:6821","nonce":33351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":33351},{"type":"v1","addr":"172.21.15.119:6817","nonce":33351}]},"public_addr":"172.21.15.119:6809/33351","cluster_addr":"172.21.15.119:6813/33351","heartbeat_back_addr":"172.21.15.119:6821/33351","heartbeat_front_addr":"172.21.15.119:6817/33351","state":["exists","up"]},{"osd":3,"uuid":"59044f30-32e1-4942-9a8e-202c1def221a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":33349},{"type":"v1","addr":"172.21.15.119:6812","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":33349},{"type":"v1","addr":"172.21.15.119:6816","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":33349},{"type":"v1","addr":"172.21.15.119:6823","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":33349},{"type":"v1","addr":"172.21.15.119:6820","nonce":33349}]},"public_addr":"172.21.15.119:6812/33349","cluster_addr":"172.21.15.119:6816/33349","heartbeat_back_addr":"172.21.15.119:6823/33349","heartbeat_front_addr":"172.21.15.119:6820/33349","state":["exists","up"]},{"osd":4,"uuid":"7d1ae3d0-8d69-4d3c-9e51-24fdd77a22a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":33088},{"type":"v1","addr":"172.21.15.148:6809","nonce":33088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":33088},{"type":"v1","addr":"172.21.15.148:6811","nonce":33088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":33088},{"type":"v1","addr":"172.21.15.148:6815","nonce":33088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":33088},{"type":"v1","addr":"172.21.15.148:6813","nonce":33088}]},"public_addr":"172.21.15.148:6809/33088","cluster_addr":"172.21.15.148:6811/33088","heartbeat_back_addr":"172.21.15.148:6815/33088","heartbeat_front_addr":"172.21.15.148:6813/33088","state":["exists","up"]},{"osd":5,"uuid":"b0dd3d30-b273-428c-a4a9-4c238abf45fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":33089},{"type":"v1","addr":"172.21.15.148:6801","nonce":33089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":33089},{"type":"v1","addr":"172.21.15.148:6803","nonce":33089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":33089},{"type":"v1","addr":"172.21.15.148:6807","nonce":33089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":33089},{"type":"v1","addr":"172.21.15.148:6805","nonce":33089}]},"public_addr":"172.21.15.148:6801/33089","cluster_addr":"172.21.15.148:6803/33089","heartbeat_back_addr":"172.21.15.148:6807/33089","heartbeat_front_addr":"172.21.15.148:6805/33089","state":["exists","up"]},{"osd":6,"uuid":"40290cb3-2334-4c68-8e6c-59d0d5455171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":33107},{"type":"v1","addr":"172.21.15.148:6817","nonce":33107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":33107},{"type":"v1","addr":"172.21.15.148:6819","nonce":33107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":33107},{"type":"v1","addr":"172.21.15.148:6823","nonce":33107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":33107},{"type":"v1","addr":"172.21.15.148:6821","nonce":33107}]},"public_addr":"172.21.15.148:6817/33107","cluster_addr":"172.21.15.148:6819/33107","heartbeat_back_addr":"172.21.15.148:6823/33107","heartbeat_front_addr":"172.21.15.148:6821/33107","state":["exists","up"]},{"osd":7,"uuid":"54802abe-ea1a-448e-85af-d1f14a45be27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":33119},{"type":"v1","addr":"172.21.15.148:6825","nonce":33119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":33119},{"type":"v1","addr":"172.21.15.148:6827","nonce":33119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":33119},{"type":"v1","addr":"172.21.15.148:6831","nonce":33119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":33119},{"type":"v1","addr":"172.21.15.148:6829","nonce":33119}]},"public_addr":"172.21.15.148:6825/33119","cluster_addr":"172.21.15.148:6827/33119","heartbeat_back_addr":"172.21.15.148:6831/33119","heartbeat_front_addr":"172.21.15.148:6829/33119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.374987+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.452341+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.393769+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.506373+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.463411+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.467809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.522054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.497313+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T11:30:07.371 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:07.370+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:07.381 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-18T11:30:07.425 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:07.426 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:07.426 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:07.426 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:07.434 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:30:07.443 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:07.442+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:07.543 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:07.542+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:07.574 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:07.574 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:07.574 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:07.574 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:07.574 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:07.583 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:30:07.644 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:07.643+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:07.721 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:07.721 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:07.721 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:30:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:30:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.693342235999999, 2024-09-18T11:30:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073601550000000002, 2024-09-18T11:30:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1 2024-09-18T11:30:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:30:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:30:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026524+0000", 2024-09-18T11:30:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999905 2024-09-18T11:30:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026541+0000", 2024-09-18T11:30:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7070000000000001e-05 2024-09-18T11:30:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026554+0000", 2024-09-18T11:30:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.288e-05 2024-09-18T11:30:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026564+0000", 2024-09-18T11:30:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0197000000000001e-05 2024-09-18T11:30:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026826+0000", 2024-09-18T11:30:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026212400000000003 2024-09-18T11:30:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:30:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026839+0000", 2024-09-18T11:30:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2831e-05 2024-09-18T11:30:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028605+0000", 2024-09-18T11:30:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001765794 2024-09-18T11:30:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028709+0000", 2024-09-18T11:30:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104348 2024-09-18T11:30:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028824+0000", 2024-09-18T11:30:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011527399999999999 2024-09-18T11:30:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030319+0000", 2024-09-18T11:30:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001494439 2024-09-18T11:30:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033805+0000", 2024-09-18T11:30:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034857820000000002 2024-09-18T11:30:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033856+0000", 2024-09-18T11:30:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1687999999999997e-05 2024-09-18T11:30:07.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.729 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:07.729 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033894+0000", 2024-09-18T11:30:07.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7481000000000002e-05 2024-09-18T11:30:07.729 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:07.729 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:07.729 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(client.4266.0:1 2.3 e16/15 ondisk, result = 0)", 2024-09-18T11:30:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.030256+0000", 2024-09-18T11:30:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.689619659000002, 2024-09-18T11:30:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4136999999999999e-05, 2024-09-18T11:30:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030256+0000", 2024-09-18T11:30:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030256+0000", 2024-09-18T11:30:07.732 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:07.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.732 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:07.732 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030256+0000", 2024-09-18T11:30:07.732 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:07.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030260+0000", 2024-09-18T11:30:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3100000000000002e-06 2024-09-18T11:30:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030270+0000", 2024-09-18T11:30:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0276e-05 2024-09-18T11:30:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030278+0000", 2024-09-18T11:30:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8250000000000005e-06 2024-09-18T11:30:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030310+0000", 2024-09-18T11:30:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1797000000000001e-05 2024-09-18T11:30:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030315+0000", 2024-09-18T11:30:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6909999999999996e-06 2024-09-18T11:30:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:07.735 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030320+0000", 2024-09-18T11:30:07.735 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7899999999999996e-06 2024-09-18T11:30:07.735 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:07.735 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:07.735 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:07.735 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.735 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.735 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:30:07.736 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.036715+0000", 2024-09-18T11:30:07.736 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.683161321, 2024-09-18T11:30:07.736 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016893799999999999, 2024-09-18T11:30:07.736 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:07.736 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "reached pg", 2024-09-18T11:30:07.736 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:07.736 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:07.736 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:07.736 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2 2024-09-18T11:30:07.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.737 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:07.737 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:07.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.036715+0000", 2024-09-18T11:30:07.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:07.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:07.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.036715+0000", 2024-09-18T11:30:07.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:07.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:07.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.036713+0000", 2024-09-18T11:30:07.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:07.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.739 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:07.739 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.036720+0000", 2024-09-18T11:30:07.739 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9600000000000003e-06 2024-09-18T11:30:07.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.739 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.739 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:07.739 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.036731+0000", 2024-09-18T11:30:07.739 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1685000000000001e-05 2024-09-18T11:30:07.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.739 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.739 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:07.739 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.036741+0000", 2024-09-18T11:30:07.739 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2469999999999997e-06 2024-09-18T11:30:07.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.740 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:07.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.036788+0000", 2024-09-18T11:30:07.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6823000000000002e-05 2024-09-18T11:30:07.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.740 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:07.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.036883+0000", 2024-09-18T11:30:07.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5942999999999999e-05 2024-09-18T11:30:07.740 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:07.740 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:07.741 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:07.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.741 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:30:07.741 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.037023+0000", 2024-09-18T11:30:07.741 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.682852635, 2024-09-18T11:30:07.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013824700000000001, 2024-09-18T11:30:07.741 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:07.741 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "reached pg", 2024-09-18T11:30:07.741 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:07.742 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:07.742 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:07.742 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3 2024-09-18T11:30:07.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.742 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:07.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:07.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.037023+0000", 2024-09-18T11:30:07.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:07.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:07.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.037023+0000", 2024-09-18T11:30:07.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:07.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:07.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.037023+0000", 2024-09-18T11:30:07.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:07.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:07.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.037027+0000", 2024-09-18T11:30:07.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5519999999999998e-06 2024-09-18T11:30:07.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.744 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:07.744 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.037036+0000", 2024-09-18T11:30:07.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.653e-06 2024-09-18T11:30:07.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.744 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:07.744 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.037041+0000", 2024-09-18T11:30:07.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9690000000000002e-06 2024-09-18T11:30:07.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:07.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.037074+0000", 2024-09-18T11:30:07.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3031999999999998e-05 2024-09-18T11:30:07.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:07.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.037161+0000", 2024-09-18T11:30:07.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7664000000000006e-05 2024-09-18T11:30:07.745 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:07.745 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:07.745 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:07.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.746 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T11:30:07.746 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:51.038968+0000", 2024-09-18T11:30:07.746 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 16.680908041999999, 2024-09-18T11:30:07.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028722909999999999, 2024-09-18T11:30:07.746 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:07.746 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:07.746 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:07.746 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:07.746 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:07.746 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 5 2024-09-18T11:30:07.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.746 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:07.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.746 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:07.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.038968+0000", 2024-09-18T11:30:07.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:07.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:07.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.038968+0000", 2024-09-18T11:30:07.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:07.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:07.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.038966+0000", 2024-09-18T11:30:07.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:07.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:07.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.038983+0000", 2024-09-18T11:30:07.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7099e-05 2024-09-18T11:30:07.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:07.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.039002+0000", 2024-09-18T11:30:07.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8938000000000001e-05 2024-09-18T11:30:07.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:07.749 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.039012+0000", 2024-09-18T11:30:07.749 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9529999999999994e-06 2024-09-18T11:30:07.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.749 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:07.749 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.039068+0000", 2024-09-18T11:30:07.749 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5893999999999997e-05 2024-09-18T11:30:07.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.749 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:07.749 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.039074+0000", 2024-09-18T11:30:07.749 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8510000000000001e-06 2024-09-18T11:30:07.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:07.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.040586+0000", 2024-09-18T11:30:07.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015119809999999999 2024-09-18T11:30:07.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:07.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.040705+0000", 2024-09-18T11:30:07.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000119054 2024-09-18T11:30:07.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:07.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.040812+0000", 2024-09-18T11:30:07.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010652700000000001 2024-09-18T11:30:07.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.751 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:07.751 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041405+0000", 2024-09-18T11:30:07.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059319099999999997 2024-09-18T11:30:07.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:07.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041787+0000", 2024-09-18T11:30:07.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00038150800000000002 2024-09-18T11:30:07.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:07.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041817+0000", 2024-09-18T11:30:07.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0091e-05 2024-09-18T11:30:07.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.753 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:07.753 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041840+0000", 2024-09-18T11:30:07.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3385000000000001e-05 2024-09-18T11:30:07.753 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:07.753 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:07.753 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:07.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.753 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(client.4266.0:5 2.2 e17/15 ondisk, result = 0)", 2024-09-18T11:30:07.753 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:51.041723+0000", 2024-09-18T11:30:07.754 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 16.678152568000002, 2024-09-18T11:30:07.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123433, 2024-09-18T11:30:07.754 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:07.754 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:07.754 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:07.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.754 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:07.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041723+0000", 2024-09-18T11:30:07.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:07.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.754 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:07.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041723+0000", 2024-09-18T11:30:07.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:07.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:07.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041722+0000", 2024-09-18T11:30:07.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:07.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:07.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041728+0000", 2024-09-18T11:30:07.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6160000000000001e-06 2024-09-18T11:30:07.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:07.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041740+0000", 2024-09-18T11:30:07.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2602999999999999e-05 2024-09-18T11:30:07.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:07.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041752+0000", 2024-09-18T11:30:07.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1630999999999999e-05 2024-09-18T11:30:07.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:07.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041778+0000", 2024-09-18T11:30:07.757 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6435e-05 2024-09-18T11:30:07.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:07.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041783+0000", 2024-09-18T11:30:07.757 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8130000000000003e-06 2024-09-18T11:30:07.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:07.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041847+0000", 2024-09-18T11:30:07.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3689000000000006e-05 2024-09-18T11:30:07.758 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:07.758 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:07.758 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:07.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.758 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T11:30:07.758 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:30:07.758 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 15.674063804999999, 2024-09-18T11:30:07.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055107869999999996, 2024-09-18T11:30:07.759 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:07.759 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:07.759 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:07.759 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:07.759 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:07.759 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 7 2024-09-18T11:30:07.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.759 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:07.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:07.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:30:07.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:07.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.760 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.760 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:07.760 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:30:07.760 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:07.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.760 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.760 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:07.760 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045810+0000", 2024-09-18T11:30:07.760 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:07.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.760 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.760 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:07.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045820+0000", 2024-09-18T11:30:07.761 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0215e-05 2024-09-18T11:30:07.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.761 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:07.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045838+0000", 2024-09-18T11:30:07.761 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7849000000000001e-05 2024-09-18T11:30:07.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.761 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:07.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045845+0000", 2024-09-18T11:30:07.761 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4229999999999999e-06 2024-09-18T11:30:07.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:07.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045898+0000", 2024-09-18T11:30:07.762 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2537999999999997e-05 2024-09-18T11:30:07.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:07.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045904+0000", 2024-09-18T11:30:07.762 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4019999999999999e-06 2024-09-18T11:30:07.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.763 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:07.763 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047663+0000", 2024-09-18T11:30:07.763 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017584079999999999 2024-09-18T11:30:07.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.763 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:07.763 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047736+0000", 2024-09-18T11:30:07.763 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3509000000000004e-05 2024-09-18T11:30:07.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:07.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047857+0000", 2024-09-18T11:30:07.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012037800000000001 2024-09-18T11:30:07.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:07.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.050868+0000", 2024-09-18T11:30:07.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030111420000000001 2024-09-18T11:30:07.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.765 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:07.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051239+0000", 2024-09-18T11:30:07.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037141000000000002 2024-09-18T11:30:07.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.765 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:07.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051288+0000", 2024-09-18T11:30:07.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9051000000000002e-05 2024-09-18T11:30:07.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.766 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:07.766 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051323+0000", 2024-09-18T11:30:07.766 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4545000000000002e-05 2024-09-18T11:30:07.766 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:07.766 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:07.767 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:07.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.767 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.767 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(client.4266.0:7 2.2 e18/15 ondisk, result = 0)", 2024-09-18T11:30:07.767 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:52.051168+0000", 2024-09-18T11:30:07.767 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 15.668708045000001, 2024-09-18T11:30:07.768 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000162237, 2024-09-18T11:30:07.768 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:07.768 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:07.768 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:07.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.769 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:07.769 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051168+0000", 2024-09-18T11:30:07.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:07.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.769 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.769 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:07.769 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051168+0000", 2024-09-18T11:30:07.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:07.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.770 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:07.770 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051167+0000", 2024-09-18T11:30:07.770 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:07.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.770 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:07.770 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051172+0000", 2024-09-18T11:30:07.770 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5609999999999998e-06 2024-09-18T11:30:07.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.770 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:07.770 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051184+0000", 2024-09-18T11:30:07.770 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1731e-05 2024-09-18T11:30:07.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:07.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051190+0000", 2024-09-18T11:30:07.771 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2199999999999997e-06 2024-09-18T11:30:07.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:07.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051224+0000", 2024-09-18T11:30:07.771 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3899999999999997e-05 2024-09-18T11:30:07.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.772 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:07.772 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051233+0000", 2024-09-18T11:30:07.772 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9560000000000003e-06 2024-09-18T11:30:07.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:07.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:07.772 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:07.772 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051330+0000", 2024-09-18T11:30:07.772 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.6807999999999999e-05 2024-09-18T11:30:07.772 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:07.772 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:07.772 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:07.772 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:07.772 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:07.772 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:07.774 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:30:07.775 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:07.773+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:07.873 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:07.873 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:07.873 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:07.873 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:07.874 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:07.873+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:07.881 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:30:07.930 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:30:07.975 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:07.974+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:07.979 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:07.979 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:07.979 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:07.979 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:07.979 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:07.988 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:30:08.047 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:30:08.047 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:30:08.062 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:08.062 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:08.076 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:08.075+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:08.077 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:08.077 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:08.092 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:08.092 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:08.093 INFO:tasks.rados.rados.0.smithi119.stdout:1: write initial oid 1 2024-09-18T11:30:08.097 INFO:tasks.rados.rados.0.smithi119.stdout:1: seq_num 0 ranges {416807=475249,1542129=474678} 2024-09-18T11:30:08.098 INFO:tasks.rados.rados.0.smithi119.stdout:1: writing smithi11939506-1 from 416807 to 892056 tid 1 2024-09-18T11:30:08.105 INFO:tasks.rados.rados.0.smithi119.stdout:1: writing smithi11939506-1 from 1542129 to 2016807 tid 2 2024-09-18T11:30:08.105 INFO:tasks.rados.rados.0.smithi119.stdout:2: write initial oid 2 2024-09-18T11:30:08.105 INFO:tasks.rados.rados.0.smithi119.stdout:2: seq_num 1 ranges {416807=475249,1542129=474678} 2024-09-18T11:30:08.106 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T11:30:08.109 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:08.110 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:08.110 INFO:tasks.rados.rados.0.smithi119.stdout:2: writing smithi11939506-2 from 416807 to 892056 tid 1 2024-09-18T11:30:08.116 INFO:tasks.rados.rados.0.smithi119.stdout:2: writing smithi11939506-2 from 1542129 to 2016807 tid 2 2024-09-18T11:30:08.116 INFO:tasks.rados.rados.0.smithi119.stdout:3: write initial oid 3 2024-09-18T11:30:08.117 INFO:tasks.rados.rados.0.smithi119.stdout:3: seq_num 2 ranges {433614=550498,1600611=433003} 2024-09-18T11:30:08.122 INFO:tasks.rados.rados.0.smithi119.stdout:3: writing smithi11939506-3 from 433614 to 984112 tid 1 2024-09-18T11:30:08.128 INFO:tasks.rados.rados.0.smithi119.stdout:3: writing smithi11939506-3 from 1600611 to 2033614 tid 2 2024-09-18T11:30:08.128 INFO:tasks.rados.rados.0.smithi119.stdout:4: write initial oid 4 2024-09-18T11:30:08.128 INFO:tasks.rados.rados.0.smithi119.stdout:4: seq_num 3 ranges {450421=625747,1659093=391328} 2024-09-18T11:30:08.128 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:08.128 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:08.129 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:08.129 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:08.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:08.129 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [] 2024-09-18T11:30:08.129 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:08.134 INFO:tasks.rados.rados.0.smithi119.stdout:4: writing smithi11939506-4 from 450421 to 1076168 tid 1 2024-09-18T11:30:08.138 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:30:08.139 INFO:tasks.rados.rados.0.smithi119.stdout:4: writing smithi11939506-4 from 1659093 to 2050421 tid 2 2024-09-18T11:30:08.140 INFO:tasks.rados.rados.0.smithi119.stdout:1: finishing write tid 1 to smithi11939506-1 2024-09-18T11:30:08.140 INFO:tasks.rados.rados.0.smithi119.stdout:1: finishing write tid 2 to smithi11939506-1 2024-09-18T11:30:08.140 INFO:tasks.rados.rados.0.smithi119.stdout:1: finishing write tid 3 to smithi11939506-1 2024-09-18T11:30:08.140 INFO:tasks.rados.rados.0.smithi119.stdout:1: finishing write tid 4 to smithi11939506-1 2024-09-18T11:30:08.140 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-18T11:30:08.140 INFO:tasks.rados.rados.0.smithi119.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2024-09-18T11:30:08.140 INFO:tasks.rados.rados.0.smithi119.stdout:2: finishing write tid 1 to smithi11939506-2 2024-09-18T11:30:08.140 INFO:tasks.rados.rados.0.smithi119.stdout:2: finishing write tid 2 to smithi11939506-2 2024-09-18T11:30:08.140 INFO:tasks.rados.rados.0.smithi119.stdout:2: finishing write tid 3 to smithi11939506-2 2024-09-18T11:30:08.140 INFO:tasks.rados.rados.0.smithi119.stdout:2: finishing write tid 4 to smithi11939506-2 2024-09-18T11:30:08.140 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 3 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-18T11:30:08.140 INFO:tasks.rados.rados.0.smithi119.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2024-09-18T11:30:08.141 INFO:tasks.rados.rados.0.smithi119.stdout:3: finishing write tid 1 to smithi11939506-3 2024-09-18T11:30:08.141 INFO:tasks.rados.rados.0.smithi119.stdout:3: finishing write tid 2 to smithi11939506-3 2024-09-18T11:30:08.141 INFO:tasks.rados.rados.0.smithi119.stdout:3: finishing write tid 3 to smithi11939506-3 2024-09-18T11:30:08.141 INFO:tasks.rados.rados.0.smithi119.stdout:3: finishing write tid 4 to smithi11939506-3 2024-09-18T11:30:08.141 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 3 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-18T11:30:08.141 INFO:tasks.rados.rados.0.smithi119.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2024-09-18T11:30:08.141 INFO:tasks.rados.rados.0.smithi119.stdout:1: done (3 left) 2024-09-18T11:30:08.141 INFO:tasks.rados.rados.0.smithi119.stdout:2: done (2 left) 2024-09-18T11:30:08.141 INFO:tasks.rados.rados.0.smithi119.stdout:3: done (1 left) 2024-09-18T11:30:08.141 INFO:tasks.rados.rados.0.smithi119.stdout:5: write initial oid 5 2024-09-18T11:30:08.141 INFO:tasks.rados.rados.0.smithi119.stdout:5: seq_num 4 ranges {467228=700996,1717575=349653} 2024-09-18T11:30:08.146 INFO:tasks.rados.rados.0.smithi119.stdout:5: writing smithi11939506-5 from 467228 to 1168224 tid 1 2024-09-18T11:30:08.146 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:08.146 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:08.151 INFO:tasks.rados.rados.0.smithi119.stdout:5: writing smithi11939506-5 from 1717575 to 2067228 tid 2 2024-09-18T11:30:08.151 INFO:tasks.rados.rados.0.smithi119.stdout:6: write initial oid 6 2024-09-18T11:30:08.151 INFO:tasks.rados.rados.0.smithi119.stdout:6: seq_num 5 ranges {484035=776245,2059704=24331} 2024-09-18T11:30:08.158 INFO:tasks.rados.rados.0.smithi119.stdout:6: writing smithi11939506-6 from 484035 to 1260280 tid 1 2024-09-18T11:30:08.162 INFO:tasks.rados.rados.0.smithi119.stdout:6: writing smithi11939506-6 from 2059704 to 2084035 tid 2 2024-09-18T11:30:08.163 INFO:tasks.rados.rados.0.smithi119.stdout:7: write initial oid 7 2024-09-18T11:30:08.163 INFO:tasks.rados.rados.0.smithi119.stdout:7: seq_num 6 ranges {500842=451494,1718186=382656} 2024-09-18T11:30:08.164 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:08.164 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:08.167 INFO:tasks.rados.rados.0.smithi119.stdout:7: writing smithi11939506-7 from 500842 to 952336 tid 1 2024-09-18T11:30:08.174 INFO:tasks.rados.rados.0.smithi119.stdout:7: writing smithi11939506-7 from 1718186 to 2100842 tid 2 2024-09-18T11:30:08.174 INFO:tasks.rados.rados.0.smithi119.stdout:8: write initial oid 8 2024-09-18T11:30:08.174 INFO:tasks.rados.rados.0.smithi119.stdout:8: seq_num 7 ranges {517649=526743,1776668=340981} 2024-09-18T11:30:08.179 INFO:tasks.rados.rados.0.smithi119.stdout:8: writing smithi11939506-8 from 517649 to 1044392 tid 1 2024-09-18T11:30:08.183 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:08.182+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:08.185 INFO:tasks.rados.rados.0.smithi119.stdout:8: writing smithi11939506-8 from 1776668 to 2117649 tid 2 2024-09-18T11:30:08.186 INFO:tasks.rados.rados.0.smithi119.stdout:9: write initial oid 9 2024-09-18T11:30:08.186 INFO:tasks.rados.rados.0.smithi119.stdout:9: seq_num 8 ranges {534456=718345,1951503=182953} 2024-09-18T11:30:08.192 INFO:tasks.rados.rados.0.smithi119.stdout:9: writing smithi11939506-9 from 534456 to 1252801 tid 1 2024-09-18T11:30:08.197 INFO:tasks.rados.rados.0.smithi119.stdout:9: writing smithi11939506-9 from 1951503 to 2134456 tid 2 2024-09-18T11:30:08.198 INFO:tasks.rados.rados.0.smithi119.stdout:10: write initial oid 10 2024-09-18T11:30:08.198 INFO:tasks.rados.rados.0.smithi119.stdout:10: seq_num 9 ranges {551263=793594,1893632=257631} 2024-09-18T11:30:08.205 INFO:tasks.rados.rados.0.smithi119.stdout:10: writing smithi11939506-10 from 551263 to 1344857 tid 1 2024-09-18T11:30:08.209 INFO:tasks.rados.rados.0.smithi119.stdout:10: writing smithi11939506-10 from 1893632 to 2151263 tid 2 2024-09-18T11:30:08.209 INFO:tasks.rados.rados.0.smithi119.stdout:11: write initial oid 11 2024-09-18T11:30:08.210 INFO:tasks.rados.rados.0.smithi119.stdout:11: seq_num 10 ranges {568070=468843,1552114=615956} 2024-09-18T11:30:08.215 INFO:tasks.rados.rados.0.smithi119.stdout:11: writing smithi11939506-11 from 568070 to 1036913 tid 1 2024-09-18T11:30:08.222 INFO:tasks.rados.rados.0.smithi119.stdout:11: writing smithi11939506-11 from 1552114 to 2168070 tid 2 2024-09-18T11:30:08.222 INFO:tasks.rados.rados.0.smithi119.stdout:12: write initial oid 12 2024-09-18T11:30:08.222 INFO:tasks.rados.rados.0.smithi119.stdout:12: seq_num 11 ranges {584877=544092,1610596=562003} 2024-09-18T11:30:08.229 INFO:tasks.rados.rados.0.smithi119.stdout:12: writing smithi11939506-12 from 584877 to 1128969 tid 1 2024-09-18T11:30:08.237 INFO:tasks.rados.rados.0.smithi119.stdout:12: writing smithi11939506-12 from 1610596 to 2172599 tid 2 2024-09-18T11:30:08.237 INFO:tasks.rados.rados.0.smithi119.stdout:13: write initial oid 13 2024-09-18T11:30:08.237 INFO:tasks.rados.rados.0.smithi119.stdout:13: seq_num 12 ranges {601684=619341,1669078=532606} 2024-09-18T11:30:08.244 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:08.244 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:08.244 INFO:tasks.rados.rados.0.smithi119.stdout:13: writing smithi11939506-13 from 601684 to 1221025 tid 1 2024-09-18T11:30:08.251 INFO:tasks.rados.rados.0.smithi119.stdout:13: writing smithi11939506-13 from 1669078 to 2201684 tid 2 2024-09-18T11:30:08.251 INFO:tasks.rados.rados.0.smithi119.stdout:14: write initial oid 14 2024-09-18T11:30:08.252 INFO:tasks.rados.rados.0.smithi119.stdout:14: seq_num 13 ranges {618491=694590,2011207=207284} 2024-09-18T11:30:08.253 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:30:08.260 INFO:tasks.rados.rados.0.smithi119.stdout:14: writing smithi11939506-14 from 618491 to 1313081 tid 1 2024-09-18T11:30:08.266 INFO:tasks.rados.rados.0.smithi119.stdout:14: writing smithi11939506-14 from 2011207 to 2218491 tid 2 2024-09-18T11:30:08.266 INFO:tasks.rados.rados.0.smithi119.stdout:15: write initial oid 15 2024-09-18T11:30:08.266 INFO:tasks.rados.rados.0.smithi119.stdout:15: seq_num 14 ranges {635298=769839,2069689=165609} 2024-09-18T11:30:08.275 INFO:tasks.rados.rados.0.smithi119.stdout:15: writing smithi11939506-15 from 635298 to 1405137 tid 1 2024-09-18T11:30:08.281 INFO:tasks.rados.rados.0.smithi119.stdout:15: writing smithi11939506-15 from 2069689 to 2235298 tid 2 2024-09-18T11:30:08.281 INFO:tasks.rados.rados.0.smithi119.stdout:16: write initial oid 16 2024-09-18T11:30:08.281 INFO:tasks.rados.rados.0.smithi119.stdout:16: seq_num 15 ranges {652105=445088,1728171=452988} 2024-09-18T11:30:08.286 INFO:tasks.rados.rados.0.smithi119.stdout:16: writing smithi11939506-16 from 652105 to 1097193 tid 1 2024-09-18T11:30:08.292 INFO:tasks.rados.rados.0.smithi119.stdout:16: writing smithi11939506-16 from 1728171 to 2181159 tid 2 2024-09-18T11:30:08.292 INFO:tasks.rados.rados.0.smithi119.stdout:17: write initial oid 17 2024-09-18T11:30:08.292 INFO:tasks.rados.rados.0.smithi119.stdout:17: seq_num 16 ranges {668912=636690,1903006=365906} 2024-09-18T11:30:08.294 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:08.293+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:08.299 INFO:tasks.rados.rados.0.smithi119.stdout:17: writing smithi11939506-17 from 668912 to 1305602 tid 1 2024-09-18T11:30:08.304 INFO:tasks.rados.rados.0.smithi119.stdout:17: writing smithi11939506-17 from 1903006 to 2268912 tid 2 2024-09-18T11:30:08.304 INFO:tasks.rados.rados.0.smithi119.stdout:18: write initial oid 18 2024-09-18T11:30:08.305 INFO:tasks.rados.rados.0.smithi119.stdout:18: seq_num 17 ranges {685719=711939,1845135=440584} 2024-09-18T11:30:08.312 INFO:tasks.rados.rados.0.smithi119.stdout:18: writing smithi11939506-18 from 685719 to 1397658 tid 1 2024-09-18T11:30:08.317 INFO:tasks.rados.rados.0.smithi119.stdout:18: writing smithi11939506-18 from 1845135 to 2285719 tid 2 2024-09-18T11:30:08.317 INFO:tasks.rados.rados.0.smithi119.stdout:19: write initial oid 19 2024-09-18T11:30:08.318 INFO:tasks.rados.rados.0.smithi119.stdout:19: seq_num 18 ranges {702526=787188,1903617=398909} 2024-09-18T11:30:08.325 INFO:tasks.rados.rados.0.smithi119.stdout:19: writing smithi11939506-19 from 702526 to 1489714 tid 1 2024-09-18T11:30:08.330 INFO:tasks.rados.rados.0.smithi119.stdout:19: writing smithi11939506-19 from 1903617 to 2302526 tid 2 2024-09-18T11:30:08.331 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.331 INFO:tasks.rados.rados.0.smithi119.stdout:4: finishing write tid 1 to smithi11939506-4 2024-09-18T11:30:08.331 INFO:tasks.rados.rados.0.smithi119.stdout:4: finishing write tid 2 to smithi11939506-4 2024-09-18T11:30:08.331 INFO:tasks.rados.rados.0.smithi119.stdout:4: finishing write tid 3 to smithi11939506-4 2024-09-18T11:30:08.331 INFO:tasks.rados.rados.0.smithi119.stdout:4: finishing write tid 4 to smithi11939506-4 2024-09-18T11:30:08.331 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 4 v 3 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-18T11:30:08.331 INFO:tasks.rados.rados.0.smithi119.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2024-09-18T11:30:08.331 INFO:tasks.rados.rados.0.smithi119.stdout:5: finishing write tid 1 to smithi11939506-5 2024-09-18T11:30:08.331 INFO:tasks.rados.rados.0.smithi119.stdout:5: finishing write tid 2 to smithi11939506-5 2024-09-18T11:30:08.331 INFO:tasks.rados.rados.0.smithi119.stdout:5: finishing write tid 3 to smithi11939506-5 2024-09-18T11:30:08.331 INFO:tasks.rados.rados.0.smithi119.stdout:5: finishing write tid 4 to smithi11939506-5 2024-09-18T11:30:08.331 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 6 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-18T11:30:08.331 INFO:tasks.rados.rados.0.smithi119.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2024-09-18T11:30:08.332 INFO:tasks.rados.rados.0.smithi119.stdout:6: finishing write tid 1 to smithi11939506-6 2024-09-18T11:30:08.332 INFO:tasks.rados.rados.0.smithi119.stdout:4: done (15 left) 2024-09-18T11:30:08.332 INFO:tasks.rados.rados.0.smithi119.stdout:5: done (14 left) 2024-09-18T11:30:08.332 INFO:tasks.rados.rados.0.smithi119.stdout:20: write initial oid 20 2024-09-18T11:30:08.332 INFO:tasks.rados.rados.0.smithi119.stdout:20: seq_num 19 ranges {719333=462437,1962099=357234} 2024-09-18T11:30:08.336 INFO:tasks.rados.rados.0.smithi119.stdout:20: writing smithi11939506-20 from 719333 to 1181770 tid 1 2024-09-18T11:30:08.343 INFO:tasks.rados.rados.0.smithi119.stdout:20: writing smithi11939506-20 from 1962099 to 2319333 tid 2 2024-09-18T11:30:08.343 INFO:tasks.rados.rados.0.smithi119.stdout:21: write initial oid 21 2024-09-18T11:30:08.343 INFO:tasks.rados.rados.0.smithi119.stdout:21: seq_num 20 ranges {736140=537686,2020581=315559} 2024-09-18T11:30:08.349 INFO:tasks.rados.rados.0.smithi119.stdout:21: writing smithi11939506-21 from 736140 to 1273826 tid 1 2024-09-18T11:30:08.356 INFO:tasks.rados.rados.0.smithi119.stdout:21: writing smithi11939506-21 from 2020581 to 2336140 tid 2 2024-09-18T11:30:08.356 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.356 INFO:tasks.rados.rados.0.smithi119.stdout:6: finishing write tid 2 to smithi11939506-6 2024-09-18T11:30:08.356 INFO:tasks.rados.rados.0.smithi119.stdout:6: finishing write tid 3 to smithi11939506-6 2024-09-18T11:30:08.356 INFO:tasks.rados.rados.0.smithi119.stdout:6: finishing write tid 4 to smithi11939506-6 2024-09-18T11:30:08.356 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 6 v 3 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-18T11:30:08.356 INFO:tasks.rados.rados.0.smithi119.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2024-09-18T11:30:08.356 INFO:tasks.rados.rados.0.smithi119.stdout:7: finishing write tid 1 to smithi11939506-7 2024-09-18T11:30:08.357 INFO:tasks.rados.rados.0.smithi119.stdout:7: finishing write tid 2 to smithi11939506-7 2024-09-18T11:30:08.357 INFO:tasks.rados.rados.0.smithi119.stdout:7: finishing write tid 3 to smithi11939506-7 2024-09-18T11:30:08.357 INFO:tasks.rados.rados.0.smithi119.stdout:7: finishing write tid 4 to smithi11939506-7 2024-09-18T11:30:08.357 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 7 v 6 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-18T11:30:08.357 INFO:tasks.rados.rados.0.smithi119.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2024-09-18T11:30:08.357 INFO:tasks.rados.rados.0.smithi119.stdout:8: finishing write tid 1 to smithi11939506-8 2024-09-18T11:30:08.357 INFO:tasks.rados.rados.0.smithi119.stdout:8: finishing write tid 2 to smithi11939506-8 2024-09-18T11:30:08.357 INFO:tasks.rados.rados.0.smithi119.stdout:8: finishing write tid 3 to smithi11939506-8 2024-09-18T11:30:08.357 INFO:tasks.rados.rados.0.smithi119.stdout:6: done (15 left) 2024-09-18T11:30:08.357 INFO:tasks.rados.rados.0.smithi119.stdout:7: done (14 left) 2024-09-18T11:30:08.357 INFO:tasks.rados.rados.0.smithi119.stdout:22: write initial oid 22 2024-09-18T11:30:08.357 INFO:tasks.rados.rados.0.smithi119.stdout:22: seq_num 21 ranges {752947=612935,1962710=390237} 2024-09-18T11:30:08.363 INFO:tasks.rados.rados.0.smithi119.stdout:22: writing smithi11939506-22 from 752947 to 1365882 tid 1 2024-09-18T11:30:08.368 INFO:tasks.rados.rados.0.smithi119.stdout:22: writing smithi11939506-22 from 1962710 to 2352947 tid 2 2024-09-18T11:30:08.369 INFO:tasks.rados.rados.0.smithi119.stdout:23: write initial oid 23 2024-09-18T11:30:08.369 INFO:tasks.rados.rados.0.smithi119.stdout:23: seq_num 22 ranges {769754=688184,2021192=348562} 2024-09-18T11:30:08.376 INFO:tasks.rados.rados.0.smithi119.stdout:23: writing smithi11939506-23 from 769754 to 1457938 tid 1 2024-09-18T11:30:08.380 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:08.378+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:08.382 INFO:tasks.rados.rados.0.smithi119.stdout:23: writing smithi11939506-23 from 2021192 to 2369754 tid 2 2024-09-18T11:30:08.382 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.382 INFO:tasks.rados.rados.0.smithi119.stdout:8: finishing write tid 4 to smithi11939506-8 2024-09-18T11:30:08.382 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 8 v 6 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-18T11:30:08.382 INFO:tasks.rados.rados.0.smithi119.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2024-09-18T11:30:08.382 INFO:tasks.rados.rados.0.smithi119.stdout:9: finishing write tid 1 to smithi11939506-9 2024-09-18T11:30:08.382 INFO:tasks.rados.rados.0.smithi119.stdout:9: finishing write tid 2 to smithi11939506-9 2024-09-18T11:30:08.382 INFO:tasks.rados.rados.0.smithi119.stdout:8: done (15 left) 2024-09-18T11:30:08.382 INFO:tasks.rados.rados.0.smithi119.stdout:24: write initial oid 24 2024-09-18T11:30:08.382 INFO:tasks.rados.rados.0.smithi119.stdout:24: seq_num 23 ranges {786561=479786,1796027=467664} 2024-09-18T11:30:08.388 INFO:tasks.rados.rados.0.smithi119.stdout:24: writing smithi11939506-24 from 786561 to 1266347 tid 1 2024-09-18T11:30:08.394 INFO:tasks.rados.rados.0.smithi119.stdout:24: writing smithi11939506-24 from 1796027 to 2263691 tid 2 2024-09-18T11:30:08.394 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.394 INFO:tasks.rados.rados.0.smithi119.stdout:9: finishing write tid 3 to smithi11939506-9 2024-09-18T11:30:08.394 INFO:tasks.rados.rados.0.smithi119.stdout:9: finishing write tid 4 to smithi11939506-9 2024-09-18T11:30:08.394 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 6 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T11:30:08.395 INFO:tasks.rados.rados.0.smithi119.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2024-09-18T11:30:08.395 INFO:tasks.rados.rados.0.smithi119.stdout:10: finishing write tid 1 to smithi11939506-10 2024-09-18T11:30:08.395 INFO:tasks.rados.rados.0.smithi119.stdout:10: finishing write tid 2 to smithi11939506-10 2024-09-18T11:30:08.395 INFO:tasks.rados.rados.0.smithi119.stdout:10: finishing write tid 3 to smithi11939506-10 2024-09-18T11:30:08.395 INFO:tasks.rados.rados.0.smithi119.stdout:9: done (15 left) 2024-09-18T11:30:08.395 INFO:tasks.rados.rados.0.smithi119.stdout:25: write initial oid 25 2024-09-18T11:30:08.395 INFO:tasks.rados.rados.0.smithi119.stdout:25: seq_num 24 ranges {403368=555035,1454509=727675} 2024-09-18T11:30:08.395 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:08.395 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:08.399 INFO:tasks.rados.rados.0.smithi119.stdout:25: writing smithi11939506-25 from 403368 to 958403 tid 1 2024-09-18T11:30:08.405 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:30:08.406 INFO:tasks.rados.rados.0.smithi119.stdout:25: writing smithi11939506-25 from 1454509 to 2182184 tid 2 2024-09-18T11:30:08.406 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.406 INFO:tasks.rados.rados.0.smithi119.stdout:10: finishing write tid 4 to smithi11939506-10 2024-09-18T11:30:08.406 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 10 v 9 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-18T11:30:08.407 INFO:tasks.rados.rados.0.smithi119.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2024-09-18T11:30:08.407 INFO:tasks.rados.rados.0.smithi119.stdout:11: finishing write tid 1 to smithi11939506-11 2024-09-18T11:30:08.407 INFO:tasks.rados.rados.0.smithi119.stdout:10: done (15 left) 2024-09-18T11:30:08.407 INFO:tasks.rados.rados.0.smithi119.stdout:26: write initial oid 26 2024-09-18T11:30:08.407 INFO:tasks.rados.rados.0.smithi119.stdout:26: seq_num 25 ranges {420175=630284,1796638=471333} 2024-09-18T11:30:08.411 INFO:tasks.rados.rados.0.smithi119.stdout:26: writing smithi11939506-26 from 420175 to 1050459 tid 1 2024-09-18T11:30:08.419 INFO:tasks.rados.rados.0.smithi119.stdout:26: writing smithi11939506-26 from 1796638 to 2267971 tid 2 2024-09-18T11:30:08.419 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.419 INFO:tasks.rados.rados.0.smithi119.stdout:11: finishing write tid 2 to smithi11939506-11 2024-09-18T11:30:08.419 INFO:tasks.rados.rados.0.smithi119.stdout:11: finishing write tid 3 to smithi11939506-11 2024-09-18T11:30:08.419 INFO:tasks.rados.rados.0.smithi119.stdout:11: finishing write tid 4 to smithi11939506-11 2024-09-18T11:30:08.419 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 11 v 3 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-18T11:30:08.419 INFO:tasks.rados.rados.0.smithi119.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2024-09-18T11:30:08.419 INFO:tasks.rados.rados.0.smithi119.stdout:12: finishing write tid 1 to smithi11939506-12 2024-09-18T11:30:08.419 INFO:tasks.rados.rados.0.smithi119.stdout:12: finishing write tid 2 to smithi11939506-12 2024-09-18T11:30:08.419 INFO:tasks.rados.rados.0.smithi119.stdout:12: finishing write tid 3 to smithi11939506-12 2024-09-18T11:30:08.419 INFO:tasks.rados.rados.0.smithi119.stdout:12: finishing write tid 4 to smithi11939506-12 2024-09-18T11:30:08.419 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 12 v 6 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-18T11:30:08.420 INFO:tasks.rados.rados.0.smithi119.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2024-09-18T11:30:08.420 INFO:tasks.rados.rados.0.smithi119.stdout:13: finishing write tid 1 to smithi11939506-13 2024-09-18T11:30:08.420 INFO:tasks.rados.rados.0.smithi119.stdout:13: finishing write tid 2 to smithi11939506-13 2024-09-18T11:30:08.420 INFO:tasks.rados.rados.0.smithi119.stdout:11: done (15 left) 2024-09-18T11:30:08.420 INFO:tasks.rados.rados.0.smithi119.stdout:12: done (14 left) 2024-09-18T11:30:08.420 INFO:tasks.rados.rados.0.smithi119.stdout:27: write initial oid 27 2024-09-18T11:30:08.420 INFO:tasks.rados.rados.0.smithi119.stdout:27: seq_num 26 ranges {436982=705533,1855120=581862} 2024-09-18T11:30:08.424 INFO:tasks.rados.rados.0.smithi119.stdout:27: writing smithi11939506-27 from 436982 to 1142515 tid 1 2024-09-18T11:30:08.432 INFO:tasks.rados.rados.0.smithi119.stdout:27: writing smithi11939506-27 from 1855120 to 2436982 tid 2 2024-09-18T11:30:08.432 INFO:tasks.rados.rados.0.smithi119.stdout:28: write initial oid 28 2024-09-18T11:30:08.432 INFO:tasks.rados.rados.0.smithi119.stdout:28: seq_num 27 ranges {453789=780782,1913602=475002} 2024-09-18T11:30:08.439 INFO:tasks.rados.rados.0.smithi119.stdout:28: writing smithi11939506-28 from 453789 to 1234571 tid 1 2024-09-18T11:30:08.447 INFO:tasks.rados.rados.0.smithi119.stdout:28: writing smithi11939506-28 from 1913602 to 2388604 tid 2 2024-09-18T11:30:08.447 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.447 INFO:tasks.rados.rados.0.smithi119.stdout:13: finishing write tid 3 to smithi11939506-13 2024-09-18T11:30:08.447 INFO:tasks.rados.rados.0.smithi119.stdout:13: finishing write tid 4 to smithi11939506-13 2024-09-18T11:30:08.447 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 13 v 9 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-18T11:30:08.447 INFO:tasks.rados.rados.0.smithi119.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2024-09-18T11:30:08.447 INFO:tasks.rados.rados.0.smithi119.stdout:14: finishing write tid 1 to smithi11939506-14 2024-09-18T11:30:08.447 INFO:tasks.rados.rados.0.smithi119.stdout:14: finishing write tid 2 to smithi11939506-14 2024-09-18T11:30:08.447 INFO:tasks.rados.rados.0.smithi119.stdout:14: finishing write tid 3 to smithi11939506-14 2024-09-18T11:30:08.447 INFO:tasks.rados.rados.0.smithi119.stdout:14: finishing write tid 4 to smithi11939506-14 2024-09-18T11:30:08.447 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 3 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-18T11:30:08.447 INFO:tasks.rados.rados.0.smithi119.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2024-09-18T11:30:08.448 INFO:tasks.rados.rados.0.smithi119.stdout:15: finishing write tid 1 to smithi11939506-15 2024-09-18T11:30:08.448 INFO:tasks.rados.rados.0.smithi119.stdout:15: finishing write tid 2 to smithi11939506-15 2024-09-18T11:30:08.448 INFO:tasks.rados.rados.0.smithi119.stdout:15: finishing write tid 3 to smithi11939506-15 2024-09-18T11:30:08.448 INFO:tasks.rados.rados.0.smithi119.stdout:15: finishing write tid 4 to smithi11939506-15 2024-09-18T11:30:08.448 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 3 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-18T11:30:08.448 INFO:tasks.rados.rados.0.smithi119.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2024-09-18T11:30:08.448 INFO:tasks.rados.rados.0.smithi119.stdout:16: finishing write tid 1 to smithi11939506-16 2024-09-18T11:30:08.448 INFO:tasks.rados.rados.0.smithi119.stdout:16: finishing write tid 2 to smithi11939506-16 2024-09-18T11:30:08.448 INFO:tasks.rados.rados.0.smithi119.stdout:16: finishing write tid 3 to smithi11939506-16 2024-09-18T11:30:08.448 INFO:tasks.rados.rados.0.smithi119.stdout:16: finishing write tid 4 to smithi11939506-16 2024-09-18T11:30:08.448 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 3 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-18T11:30:08.448 INFO:tasks.rados.rados.0.smithi119.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2024-09-18T11:30:08.448 INFO:tasks.rados.rados.0.smithi119.stdout:17: finishing write tid 1 to smithi11939506-17 2024-09-18T11:30:08.448 INFO:tasks.rados.rados.0.smithi119.stdout:17: finishing write tid 2 to smithi11939506-17 2024-09-18T11:30:08.449 INFO:tasks.rados.rados.0.smithi119.stdout:17: finishing write tid 3 to smithi11939506-17 2024-09-18T11:30:08.449 INFO:tasks.rados.rados.0.smithi119.stdout:17: finishing write tid 4 to smithi11939506-17 2024-09-18T11:30:08.449 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 12 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-18T11:30:08.449 INFO:tasks.rados.rados.0.smithi119.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2024-09-18T11:30:08.449 INFO:tasks.rados.rados.0.smithi119.stdout:13: done (15 left) 2024-09-18T11:30:08.449 INFO:tasks.rados.rados.0.smithi119.stdout:14: done (14 left) 2024-09-18T11:30:08.449 INFO:tasks.rados.rados.0.smithi119.stdout:15: done (13 left) 2024-09-18T11:30:08.449 INFO:tasks.rados.rados.0.smithi119.stdout:16: done (12 left) 2024-09-18T11:30:08.449 INFO:tasks.rados.rados.0.smithi119.stdout:17: done (11 left) 2024-09-18T11:30:08.449 INFO:tasks.rados.rados.0.smithi119.stdout:29: write initial oid 29 2024-09-18T11:30:08.449 INFO:tasks.rados.rados.0.smithi119.stdout:29: seq_num 28 ranges {470596=456031,1572084=618660} 2024-09-18T11:30:08.450 INFO:tasks.rados.rados.0.smithi119.stdout:29: writing smithi11939506-29 from 470596 to 926627 tid 1 2024-09-18T11:30:08.457 INFO:tasks.rados.rados.0.smithi119.stdout:29: writing smithi11939506-29 from 1572084 to 2190744 tid 2 2024-09-18T11:30:08.458 INFO:tasks.rados.rados.0.smithi119.stdout:30: write initial oid 30 2024-09-18T11:30:08.458 INFO:tasks.rados.rados.0.smithi119.stdout:30: seq_num 29 ranges {487403=531280,1514213=478671} 2024-09-18T11:30:08.463 INFO:tasks.rados.rados.0.smithi119.stdout:30: writing smithi11939506-30 from 487403 to 1018683 tid 1 2024-09-18T11:30:08.469 INFO:tasks.rados.rados.0.smithi119.stdout:30: writing smithi11939506-30 from 1514213 to 1992884 tid 2 2024-09-18T11:30:08.469 INFO:tasks.rados.rados.0.smithi119.stdout:31: write initial oid 31 2024-09-18T11:30:08.469 INFO:tasks.rados.rados.0.smithi119.stdout:31: seq_num 30 ranges {504210=606529,1572695=622329} 2024-09-18T11:30:08.475 INFO:tasks.rados.rados.0.smithi119.stdout:31: writing smithi11939506-31 from 504210 to 1110739 tid 1 2024-09-18T11:30:08.481 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:08.479+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:08.482 INFO:tasks.rados.rados.0.smithi119.stdout:31: writing smithi11939506-31 from 1572695 to 2195024 tid 2 2024-09-18T11:30:08.482 INFO:tasks.rados.rados.0.smithi119.stdout:32: write initial oid 32 2024-09-18T11:30:08.482 INFO:tasks.rados.rados.0.smithi119.stdout:32: seq_num 31 ranges {521017=798131,1747530=482340} 2024-09-18T11:30:08.488 INFO:tasks.rados.rados.0.smithi119.stdout:32: writing smithi11939506-32 from 521017 to 1319148 tid 1 2024-09-18T11:30:08.493 INFO:tasks.rados.rados.0.smithi119.stdout:32: writing smithi11939506-32 from 1747530 to 2229870 tid 2 2024-09-18T11:30:08.494 INFO:tasks.rados.rados.0.smithi119.stdout:33: write initial oid 33 2024-09-18T11:30:08.494 INFO:tasks.rados.rados.0.smithi119.stdout:33: seq_num 32 ranges {537824=473380,1806012=625998} 2024-09-18T11:30:08.499 INFO:tasks.rados.rados.0.smithi119.stdout:33: writing smithi11939506-33 from 537824 to 1011204 tid 1 2024-09-18T11:30:08.507 INFO:tasks.rados.rados.0.smithi119.stdout:33: writing smithi11939506-33 from 1806012 to 2432010 tid 2 2024-09-18T11:30:08.507 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.508 INFO:tasks.rados.rados.0.smithi119.stdout:18: finishing write tid 1 to smithi11939506-18 2024-09-18T11:30:08.508 INFO:tasks.rados.rados.0.smithi119.stdout:18: finishing write tid 2 to smithi11939506-18 2024-09-18T11:30:08.508 INFO:tasks.rados.rados.0.smithi119.stdout:18: finishing write tid 3 to smithi11939506-18 2024-09-18T11:30:08.508 INFO:tasks.rados.rados.0.smithi119.stdout:18: finishing write tid 4 to smithi11939506-18 2024-09-18T11:30:08.508 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 3 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-18T11:30:08.508 INFO:tasks.rados.rados.0.smithi119.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2024-09-18T11:30:08.508 INFO:tasks.rados.rados.0.smithi119.stdout:19: finishing write tid 1 to smithi11939506-19 2024-09-18T11:30:08.508 INFO:tasks.rados.rados.0.smithi119.stdout:19: finishing write tid 2 to smithi11939506-19 2024-09-18T11:30:08.508 INFO:tasks.rados.rados.0.smithi119.stdout:19: finishing write tid 3 to smithi11939506-19 2024-09-18T11:30:08.508 INFO:tasks.rados.rados.0.smithi119.stdout:19: finishing write tid 4 to smithi11939506-19 2024-09-18T11:30:08.508 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 12 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-18T11:30:08.509 INFO:tasks.rados.rados.0.smithi119.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2024-09-18T11:30:08.509 INFO:tasks.rados.rados.0.smithi119.stdout:20: finishing write tid 1 to smithi11939506-20 2024-09-18T11:30:08.509 INFO:tasks.rados.rados.0.smithi119.stdout:20: finishing write tid 2 to smithi11939506-20 2024-09-18T11:30:08.509 INFO:tasks.rados.rados.0.smithi119.stdout:20: finishing write tid 3 to smithi11939506-20 2024-09-18T11:30:08.509 INFO:tasks.rados.rados.0.smithi119.stdout:18: done (15 left) 2024-09-18T11:30:08.509 INFO:tasks.rados.rados.0.smithi119.stdout:19: done (14 left) 2024-09-18T11:30:08.509 INFO:tasks.rados.rados.0.smithi119.stdout:34: write initial oid 34 2024-09-18T11:30:08.509 INFO:tasks.rados.rados.0.smithi119.stdout:34: seq_num 33 ranges {554631=548629,1748141=486009} 2024-09-18T11:30:08.510 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:08.510 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:08.513 INFO:tasks.rados.rados.0.smithi119.stdout:34: writing smithi11939506-34 from 554631 to 1103260 tid 1 2024-09-18T11:30:08.517 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:30:08.520 INFO:tasks.rados.rados.0.smithi119.stdout:34: writing smithi11939506-34 from 1748141 to 2234150 tid 2 2024-09-18T11:30:08.520 INFO:tasks.rados.rados.0.smithi119.stdout:35: write initial oid 35 2024-09-18T11:30:08.520 INFO:tasks.rados.rados.0.smithi119.stdout:35: seq_num 34 ranges {571438=623878,1806623=629667} 2024-09-18T11:30:08.526 INFO:tasks.rados.rados.0.smithi119.stdout:35: writing smithi11939506-35 from 571438 to 1195316 tid 1 2024-09-18T11:30:08.533 INFO:tasks.rados.rados.0.smithi119.stdout:35: writing smithi11939506-35 from 1806623 to 2436290 tid 2 2024-09-18T11:30:08.534 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.534 INFO:tasks.rados.rados.0.smithi119.stdout:20: finishing write tid 4 to smithi11939506-20 2024-09-18T11:30:08.534 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 20 v 3 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-18T11:30:08.534 INFO:tasks.rados.rados.0.smithi119.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2024-09-18T11:30:08.534 INFO:tasks.rados.rados.0.smithi119.stdout:21: finishing write tid 1 to smithi11939506-21 2024-09-18T11:30:08.534 INFO:tasks.rados.rados.0.smithi119.stdout:21: finishing write tid 2 to smithi11939506-21 2024-09-18T11:30:08.534 INFO:tasks.rados.rados.0.smithi119.stdout:21: finishing write tid 3 to smithi11939506-21 2024-09-18T11:30:08.534 INFO:tasks.rados.rados.0.smithi119.stdout:21: finishing write tid 4 to smithi11939506-21 2024-09-18T11:30:08.534 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 6 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-18T11:30:08.534 INFO:tasks.rados.rados.0.smithi119.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2024-09-18T11:30:08.534 INFO:tasks.rados.rados.0.smithi119.stdout:22: finishing write tid 1 to smithi11939506-22 2024-09-18T11:30:08.534 INFO:tasks.rados.rados.0.smithi119.stdout:22: finishing write tid 2 to smithi11939506-22 2024-09-18T11:30:08.535 INFO:tasks.rados.rados.0.smithi119.stdout:22: finishing write tid 3 to smithi11939506-22 2024-09-18T11:30:08.535 INFO:tasks.rados.rados.0.smithi119.stdout:22: finishing write tid 4 to smithi11939506-22 2024-09-18T11:30:08.535 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 15 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-18T11:30:08.535 INFO:tasks.rados.rados.0.smithi119.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2024-09-18T11:30:08.535 INFO:tasks.rados.rados.0.smithi119.stdout:23: finishing write tid 1 to smithi11939506-23 2024-09-18T11:30:08.535 INFO:tasks.rados.rados.0.smithi119.stdout:23: finishing write tid 2 to smithi11939506-23 2024-09-18T11:30:08.535 INFO:tasks.rados.rados.0.smithi119.stdout:23: finishing write tid 3 to smithi11939506-23 2024-09-18T11:30:08.535 INFO:tasks.rados.rados.0.smithi119.stdout:23: finishing write tid 4 to smithi11939506-23 2024-09-18T11:30:08.535 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 6 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-18T11:30:08.535 INFO:tasks.rados.rados.0.smithi119.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2024-09-18T11:30:08.535 INFO:tasks.rados.rados.0.smithi119.stdout:24: finishing write tid 1 to smithi11939506-24 2024-09-18T11:30:08.536 INFO:tasks.rados.rados.0.smithi119.stdout:24: finishing write tid 2 to smithi11939506-24 2024-09-18T11:30:08.536 INFO:tasks.rados.rados.0.smithi119.stdout:24: finishing write tid 3 to smithi11939506-24 2024-09-18T11:30:08.536 INFO:tasks.rados.rados.0.smithi119.stdout:24: finishing write tid 4 to smithi11939506-24 2024-09-18T11:30:08.536 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 9 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-18T11:30:08.536 INFO:tasks.rados.rados.0.smithi119.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2024-09-18T11:30:08.536 INFO:tasks.rados.rados.0.smithi119.stdout:25: finishing write tid 1 to smithi11939506-25 2024-09-18T11:30:08.536 INFO:tasks.rados.rados.0.smithi119.stdout:25: finishing write tid 2 to smithi11939506-25 2024-09-18T11:30:08.536 INFO:tasks.rados.rados.0.smithi119.stdout:25: finishing write tid 3 to smithi11939506-25 2024-09-18T11:30:08.536 INFO:tasks.rados.rados.0.smithi119.stdout:25: finishing write tid 4 to smithi11939506-25 2024-09-18T11:30:08.536 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 3 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-18T11:30:08.536 INFO:tasks.rados.rados.0.smithi119.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2024-09-18T11:30:08.536 INFO:tasks.rados.rados.0.smithi119.stdout:26: finishing write tid 1 to smithi11939506-26 2024-09-18T11:30:08.536 INFO:tasks.rados.rados.0.smithi119.stdout:26: finishing write tid 2 to smithi11939506-26 2024-09-18T11:30:08.537 INFO:tasks.rados.rados.0.smithi119.stdout:26: finishing write tid 3 to smithi11939506-26 2024-09-18T11:30:08.537 INFO:tasks.rados.rados.0.smithi119.stdout:26: finishing write tid 4 to smithi11939506-26 2024-09-18T11:30:08.537 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 9 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-18T11:30:08.537 INFO:tasks.rados.rados.0.smithi119.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2024-09-18T11:30:08.537 INFO:tasks.rados.rados.0.smithi119.stdout:27: finishing write tid 1 to smithi11939506-27 2024-09-18T11:30:08.537 INFO:tasks.rados.rados.0.smithi119.stdout:27: finishing write tid 2 to smithi11939506-27 2024-09-18T11:30:08.537 INFO:tasks.rados.rados.0.smithi119.stdout:20: done (15 left) 2024-09-18T11:30:08.537 INFO:tasks.rados.rados.0.smithi119.stdout:21: done (14 left) 2024-09-18T11:30:08.537 INFO:tasks.rados.rados.0.smithi119.stdout:22: done (13 left) 2024-09-18T11:30:08.537 INFO:tasks.rados.rados.0.smithi119.stdout:23: done (12 left) 2024-09-18T11:30:08.538 INFO:tasks.rados.rados.0.smithi119.stdout:24: done (11 left) 2024-09-18T11:30:08.538 INFO:tasks.rados.rados.0.smithi119.stdout:25: done (10 left) 2024-09-18T11:30:08.538 INFO:tasks.rados.rados.0.smithi119.stdout:26: done (9 left) 2024-09-18T11:30:08.538 INFO:tasks.rados.rados.0.smithi119.stdout:36: write initial oid 36 2024-09-18T11:30:08.538 INFO:tasks.rados.rados.0.smithi119.stdout:36: seq_num 35 ranges {588245=699127,1865105=489678} 2024-09-18T11:30:08.540 INFO:tasks.rados.rados.0.smithi119.stdout:36: writing smithi11939506-36 from 588245 to 1287372 tid 1 2024-09-18T11:30:08.546 INFO:tasks.rados.rados.0.smithi119.stdout:36: writing smithi11939506-36 from 1865105 to 2354783 tid 2 2024-09-18T11:30:08.546 INFO:tasks.rados.rados.0.smithi119.stdout:37: write initial oid 37 2024-09-18T11:30:08.546 INFO:tasks.rados.rados.0.smithi119.stdout:37: seq_num 36 ranges {605052=774376,1923587=633336} 2024-09-18T11:30:08.553 INFO:tasks.rados.rados.0.smithi119.stdout:37: writing smithi11939506-37 from 605052 to 1379428 tid 1 2024-09-18T11:30:08.560 INFO:tasks.rados.rados.0.smithi119.stdout:37: writing smithi11939506-37 from 1923587 to 2556923 tid 2 2024-09-18T11:30:08.560 INFO:tasks.rados.rados.0.smithi119.stdout:38: write initial oid 38 2024-09-18T11:30:08.560 INFO:tasks.rados.rados.0.smithi119.stdout:38: seq_num 37 ranges {621859=449625,1865716=756143} 2024-09-18T11:30:08.565 INFO:tasks.rados.rados.0.smithi119.stdout:38: writing smithi11939506-38 from 621859 to 1071484 tid 1 2024-09-18T11:30:08.574 INFO:tasks.rados.rados.0.smithi119.stdout:38: writing smithi11939506-38 from 1865716 to 2621859 tid 2 2024-09-18T11:30:08.574 INFO:tasks.rados.rados.0.smithi119.stdout:39: write initial oid 39 2024-09-18T11:30:08.574 INFO:tasks.rados.rados.0.smithi119.stdout:39: seq_num 38 ranges {638666=524874,1924198=637005} 2024-09-18T11:30:08.581 INFO:tasks.rados.rados.0.smithi119.stdout:39: writing smithi11939506-39 from 638666 to 1163540 tid 1 2024-09-18T11:30:08.581 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:08.579+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:08.588 INFO:tasks.rados.rados.0.smithi119.stdout:39: writing smithi11939506-39 from 1924198 to 2561203 tid 2 2024-09-18T11:30:08.589 INFO:tasks.rados.rados.0.smithi119.stdout:40: write initial oid 40 2024-09-18T11:30:08.589 INFO:tasks.rados.rados.0.smithi119.stdout:40: seq_num 39 ranges {655473=716476,2099033=556440} 2024-09-18T11:30:08.596 INFO:tasks.rados.rados.0.smithi119.stdout:40: writing smithi11939506-40 from 655473 to 1371949 tid 1 2024-09-18T11:30:08.603 INFO:tasks.rados.rados.0.smithi119.stdout:40: writing smithi11939506-40 from 2099033 to 2655473 tid 2 2024-09-18T11:30:08.603 INFO:tasks.rados.rados.0.smithi119.stdout:41: write initial oid 41 2024-09-18T11:30:08.603 INFO:tasks.rados.rados.0.smithi119.stdout:41: seq_num 40 ranges {672280=791725,2157515=514765} 2024-09-18T11:30:08.610 INFO:tasks.rados.rados.0.smithi119.stdout:41: writing smithi11939506-41 from 672280 to 1464005 tid 1 2024-09-18T11:30:08.617 INFO:tasks.rados.rados.0.smithi119.stdout:41: writing smithi11939506-41 from 2157515 to 2672280 tid 2 2024-09-18T11:30:08.618 INFO:tasks.rados.rados.0.smithi119.stdout:42: write initial oid 42 2024-09-18T11:30:08.618 INFO:tasks.rados.rados.0.smithi119.stdout:27: finishing write tid 3 to smithi11939506-27 2024-09-18T11:30:08.618 INFO:tasks.rados.rados.0.smithi119.stdout:27: finishing write tid 4 to smithi11939506-27 2024-09-18T11:30:08.618 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 6 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T11:30:08.618 INFO:tasks.rados.rados.0.smithi119.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2024-09-18T11:30:08.618 INFO:tasks.rados.rados.0.smithi119.stdout:28: finishing write tid 1 to smithi11939506-28 2024-09-18T11:30:08.618 INFO:tasks.rados.rados.0.smithi119.stdout:42: seq_num 41 ranges {689087=466974,1699644=784332} 2024-09-18T11:30:08.622 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:08.623 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:08.624 INFO:tasks.rados.rados.0.smithi119.stdout:42: writing smithi11939506-42 from 689087 to 1156061 tid 1 2024-09-18T11:30:08.631 INFO:tasks.rados.rados.0.smithi119.stdout:42: writing smithi11939506-42 from 1699644 to 2483976 tid 2 2024-09-18T11:30:08.631 INFO:tasks.rados.rados.0.smithi119.stdout:27: done (15 left) 2024-09-18T11:30:08.631 INFO:tasks.rados.rados.0.smithi119.stdout:43: write initial oid 43 2024-09-18T11:30:08.631 INFO:tasks.rados.rados.0.smithi119.stdout:28: finishing write tid 2 to smithi11939506-28 2024-09-18T11:30:08.632 INFO:tasks.rados.rados.0.smithi119.stdout:28: finishing write tid 3 to smithi11939506-28 2024-09-18T11:30:08.632 INFO:tasks.rados.rados.0.smithi119.stdout:28: finishing write tid 4 to smithi11939506-28 2024-09-18T11:30:08.632 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 28 v 9 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-18T11:30:08.632 INFO:tasks.rados.rados.0.smithi119.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2024-09-18T11:30:08.632 INFO:tasks.rados.rados.0.smithi119.stdout:29: finishing write tid 1 to smithi11939506-29 2024-09-18T11:30:08.632 INFO:tasks.rados.rados.0.smithi119.stdout:43: seq_num 42 ranges {705894=542223,1758126=644343} 2024-09-18T11:30:08.633 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:30:08.638 INFO:tasks.rados.rados.0.smithi119.stdout:43: writing smithi11939506-43 from 705894 to 1248117 tid 1 2024-09-18T11:30:08.644 INFO:tasks.rados.rados.0.smithi119.stdout:43: writing smithi11939506-43 from 1758126 to 2402469 tid 2 2024-09-18T11:30:08.644 INFO:tasks.rados.rados.0.smithi119.stdout:28: done (15 left) 2024-09-18T11:30:08.644 INFO:tasks.rados.rados.0.smithi119.stdout:44: write initial oid 44 2024-09-18T11:30:08.645 INFO:tasks.rados.rados.0.smithi119.stdout:44: seq_num 43 ranges {722701=617472,1816608=788001} 2024-09-18T11:30:08.651 INFO:tasks.rados.rados.0.smithi119.stdout:44: writing smithi11939506-44 from 722701 to 1340173 tid 1 2024-09-18T11:30:08.659 INFO:tasks.rados.rados.0.smithi119.stdout:44: writing smithi11939506-44 from 1816608 to 2604609 tid 2 2024-09-18T11:30:08.659 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.659 INFO:tasks.rados.rados.0.smithi119.stdout:29: finishing write tid 2 to smithi11939506-29 2024-09-18T11:30:08.659 INFO:tasks.rados.rados.0.smithi119.stdout:29: finishing write tid 3 to smithi11939506-29 2024-09-18T11:30:08.659 INFO:tasks.rados.rados.0.smithi119.stdout:29: finishing write tid 4 to smithi11939506-29 2024-09-18T11:30:08.659 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 3 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-18T11:30:08.659 INFO:tasks.rados.rados.0.smithi119.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2024-09-18T11:30:08.659 INFO:tasks.rados.rados.0.smithi119.stdout:30: finishing write tid 1 to smithi11939506-30 2024-09-18T11:30:08.660 INFO:tasks.rados.rados.0.smithi119.stdout:30: finishing write tid 2 to smithi11939506-30 2024-09-18T11:30:08.660 INFO:tasks.rados.rados.0.smithi119.stdout:30: finishing write tid 3 to smithi11939506-30 2024-09-18T11:30:08.660 INFO:tasks.rados.rados.0.smithi119.stdout:29: done (15 left) 2024-09-18T11:30:08.660 INFO:tasks.rados.rados.0.smithi119.stdout:45: write initial oid 45 2024-09-18T11:30:08.660 INFO:tasks.rados.rados.0.smithi119.stdout:45: seq_num 44 ranges {739508=692721,1875090=648012} 2024-09-18T11:30:08.666 INFO:tasks.rados.rados.0.smithi119.stdout:45: writing smithi11939506-45 from 739508 to 1432229 tid 1 2024-09-18T11:30:08.673 INFO:tasks.rados.rados.0.smithi119.stdout:45: writing smithi11939506-45 from 1875090 to 2523102 tid 2 2024-09-18T11:30:08.673 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.673 INFO:tasks.rados.rados.0.smithi119.stdout:30: finishing write tid 4 to smithi11939506-30 2024-09-18T11:30:08.673 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 30 v 6 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-18T11:30:08.673 INFO:tasks.rados.rados.0.smithi119.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2024-09-18T11:30:08.673 INFO:tasks.rados.rados.0.smithi119.stdout:31: finishing write tid 1 to smithi11939506-31 2024-09-18T11:30:08.673 INFO:tasks.rados.rados.0.smithi119.stdout:31: finishing write tid 2 to smithi11939506-31 2024-09-18T11:30:08.673 INFO:tasks.rados.rados.0.smithi119.stdout:30: done (15 left) 2024-09-18T11:30:08.673 INFO:tasks.rados.rados.0.smithi119.stdout:46: write initial oid 46 2024-09-18T11:30:08.673 INFO:tasks.rados.rados.0.smithi119.stdout:46: seq_num 45 ranges {756315=767970,1933572=791670} 2024-09-18T11:30:08.681 INFO:tasks.rados.rados.0.smithi119.stdout:46: writing smithi11939506-46 from 756315 to 1524285 tid 1 2024-09-18T11:30:08.682 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:08.681+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:08.688 INFO:tasks.rados.rados.0.smithi119.stdout:46: writing smithi11939506-46 from 1933572 to 2725242 tid 2 2024-09-18T11:30:08.688 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.688 INFO:tasks.rados.rados.0.smithi119.stdout:31: finishing write tid 3 to smithi11939506-31 2024-09-18T11:30:08.688 INFO:tasks.rados.rados.0.smithi119.stdout:31: finishing write tid 4 to smithi11939506-31 2024-09-18T11:30:08.688 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 12 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-18T11:30:08.688 INFO:tasks.rados.rados.0.smithi119.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2024-09-18T11:30:08.689 INFO:tasks.rados.rados.0.smithi119.stdout:32: finishing write tid 1 to smithi11939506-32 2024-09-18T11:30:08.689 INFO:tasks.rados.rados.0.smithi119.stdout:32: finishing write tid 2 to smithi11939506-32 2024-09-18T11:30:08.689 INFO:tasks.rados.rados.0.smithi119.stdout:32: finishing write tid 3 to smithi11939506-32 2024-09-18T11:30:08.689 INFO:tasks.rados.rados.0.smithi119.stdout:32: finishing write tid 4 to smithi11939506-32 2024-09-18T11:30:08.689 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 3 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-18T11:30:08.689 INFO:tasks.rados.rados.0.smithi119.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2024-09-18T11:30:08.689 INFO:tasks.rados.rados.0.smithi119.stdout:33: finishing write tid 1 to smithi11939506-33 2024-09-18T11:30:08.689 INFO:tasks.rados.rados.0.smithi119.stdout:31: done (15 left) 2024-09-18T11:30:08.689 INFO:tasks.rados.rados.0.smithi119.stdout:32: done (14 left) 2024-09-18T11:30:08.689 INFO:tasks.rados.rados.0.smithi119.stdout:47: write initial oid 47 2024-09-18T11:30:08.689 INFO:tasks.rados.rados.0.smithi119.stdout:47: seq_num 46 ranges {773122=559572,1992054=651681} 2024-09-18T11:30:08.695 INFO:tasks.rados.rados.0.smithi119.stdout:47: writing smithi11939506-47 from 773122 to 1332694 tid 1 2024-09-18T11:30:08.702 INFO:tasks.rados.rados.0.smithi119.stdout:47: writing smithi11939506-47 from 1992054 to 2643735 tid 2 2024-09-18T11:30:08.702 INFO:tasks.rados.rados.0.smithi119.stdout:48: write initial oid 48 2024-09-18T11:30:08.702 INFO:tasks.rados.rados.0.smithi119.stdout:48: seq_num 47 ranges {789929=634821,2050536=739393} 2024-09-18T11:30:08.710 INFO:tasks.rados.rados.0.smithi119.stdout:48: writing smithi11939506-48 from 789929 to 1424750 tid 1 2024-09-18T11:30:08.718 INFO:tasks.rados.rados.0.smithi119.stdout:48: writing smithi11939506-48 from 2050536 to 2789929 tid 2 2024-09-18T11:30:08.718 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.718 INFO:tasks.rados.rados.0.smithi119.stdout:33: finishing write tid 2 to smithi11939506-33 2024-09-18T11:30:08.718 INFO:tasks.rados.rados.0.smithi119.stdout:33: finishing write tid 3 to smithi11939506-33 2024-09-18T11:30:08.718 INFO:tasks.rados.rados.0.smithi119.stdout:33: finishing write tid 4 to smithi11939506-33 2024-09-18T11:30:08.718 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 33 v 3 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-18T11:30:08.718 INFO:tasks.rados.rados.0.smithi119.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2024-09-18T11:30:08.718 INFO:tasks.rados.rados.0.smithi119.stdout:34: finishing write tid 1 to smithi11939506-34 2024-09-18T11:30:08.718 INFO:tasks.rados.rados.0.smithi119.stdout:34: finishing write tid 2 to smithi11939506-34 2024-09-18T11:30:08.718 INFO:tasks.rados.rados.0.smithi119.stdout:34: finishing write tid 3 to smithi11939506-34 2024-09-18T11:30:08.718 INFO:tasks.rados.rados.0.smithi119.stdout:34: finishing write tid 4 to smithi11939506-34 2024-09-18T11:30:08.718 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 9 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T11:30:08.718 INFO:tasks.rados.rados.0.smithi119.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2024-09-18T11:30:08.718 INFO:tasks.rados.rados.0.smithi119.stdout:35: finishing write tid 1 to smithi11939506-35 2024-09-18T11:30:08.719 INFO:tasks.rados.rados.0.smithi119.stdout:33: done (15 left) 2024-09-18T11:30:08.719 INFO:tasks.rados.rados.0.smithi119.stdout:34: done (14 left) 2024-09-18T11:30:08.719 INFO:tasks.rados.rados.0.smithi119.stdout:49: write initial oid 49 2024-09-18T11:30:08.719 INFO:tasks.rados.rados.0.smithi119.stdout:49: seq_num 48 ranges {406736=710070,1709018=655350} 2024-09-18T11:30:08.724 INFO:tasks.rados.rados.0.smithi119.stdout:49: writing smithi11939506-49 from 406736 to 1116806 tid 1 2024-09-18T11:30:08.731 INFO:tasks.rados.rados.0.smithi119.stdout:49: writing smithi11939506-49 from 1709018 to 2364368 tid 2 2024-09-18T11:30:08.731 INFO:tasks.rados.rados.0.smithi119.stdout:50: write initial oid 50 2024-09-18T11:30:08.731 INFO:tasks.rados.rados.0.smithi119.stdout:50: seq_num 49 ranges {423543=785319,1767500=799008} 2024-09-18T11:30:08.737 INFO:tasks.rados.rados.0.smithi119.stdout:50: writing smithi11939506-50 from 423543 to 1208862 tid 1 2024-09-18T11:30:08.741 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:08.741 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:08.747 INFO:tasks.rados.rados.0.smithi119.stdout:50: writing smithi11939506-50 from 1767500 to 2566508 tid 2 2024-09-18T11:30:08.748 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.748 INFO:tasks.rados.rados.0.smithi119.stdout:35: finishing write tid 2 to smithi11939506-35 2024-09-18T11:30:08.748 INFO:tasks.rados.rados.0.smithi119.stdout:35: finishing write tid 3 to smithi11939506-35 2024-09-18T11:30:08.748 INFO:tasks.rados.rados.0.smithi119.stdout:35: finishing write tid 4 to smithi11939506-35 2024-09-18T11:30:08.748 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 6 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-18T11:30:08.748 INFO:tasks.rados.rados.0.smithi119.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2024-09-18T11:30:08.748 INFO:tasks.rados.rados.0.smithi119.stdout:36: finishing write tid 1 to smithi11939506-36 2024-09-18T11:30:08.748 INFO:tasks.rados.rados.0.smithi119.stdout:36: finishing write tid 2 to smithi11939506-36 2024-09-18T11:30:08.748 INFO:tasks.rados.rados.0.smithi119.stdout:35: done (15 left) 2024-09-18T11:30:08.748 INFO:tasks.rados.rados.0.smithi119.stdout:51: copy oid 2 target oid 2 low_tier 2024-09-18T11:30:08.748 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 2 to 2 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.748 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.748 INFO:tasks.rados.rados.0.smithi119.stdout:36: finishing write tid 3 to smithi11939506-36 2024-09-18T11:30:08.748 INFO:tasks.rados.rados.0.smithi119.stdout:36: finishing write tid 4 to smithi11939506-36 2024-09-18T11:30:08.749 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 36 v 6 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-18T11:30:08.749 INFO:tasks.rados.rados.0.smithi119.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2024-09-18T11:30:08.749 INFO:tasks.rados.rados.0.smithi119.stdout:37: finishing write tid 1 to smithi11939506-37 2024-09-18T11:30:08.749 INFO:tasks.rados.rados.0.smithi119.stdout:37: finishing write tid 2 to smithi11939506-37 2024-09-18T11:30:08.749 INFO:tasks.rados.rados.0.smithi119.stdout:37: finishing write tid 3 to smithi11939506-37 2024-09-18T11:30:08.749 INFO:tasks.rados.rados.0.smithi119.stdout:36: done (15 left) 2024-09-18T11:30:08.749 INFO:tasks.rados.rados.0.smithi119.stdout:52: copy oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.749 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 3 to 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.749 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.749 INFO:tasks.rados.rados.0.smithi119.stdout:37: finishing write tid 4 to smithi11939506-37 2024-09-18T11:30:08.749 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 6 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-18T11:30:08.749 INFO:tasks.rados.rados.0.smithi119.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2024-09-18T11:30:08.750 INFO:tasks.rados.rados.0.smithi119.stdout:38: finishing write tid 1 to smithi11939506-38 2024-09-18T11:30:08.750 INFO:tasks.rados.rados.0.smithi119.stdout:38: finishing write tid 2 to smithi11939506-38 2024-09-18T11:30:08.750 INFO:tasks.rados.rados.0.smithi119.stdout:38: finishing write tid 3 to smithi11939506-38 2024-09-18T11:30:08.750 INFO:tasks.rados.rados.0.smithi119.stdout:38: finishing write tid 4 to smithi11939506-38 2024-09-18T11:30:08.750 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 9 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-18T11:30:08.750 INFO:tasks.rados.rados.0.smithi119.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2024-09-18T11:30:08.750 INFO:tasks.rados.rados.0.smithi119.stdout:39: finishing write tid 1 to smithi11939506-39 2024-09-18T11:30:08.750 INFO:tasks.rados.rados.0.smithi119.stdout:37: done (15 left) 2024-09-18T11:30:08.750 INFO:tasks.rados.rados.0.smithi119.stdout:38: done (14 left) 2024-09-18T11:30:08.750 INFO:tasks.rados.rados.0.smithi119.stdout:53: copy oid 4 target oid 4 low_tier 2024-09-18T11:30:08.750 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 4 to 4 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.750 INFO:tasks.rados.rados.0.smithi119.stdout:54: copy oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.750 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 5 to 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.751 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.751 INFO:tasks.rados.rados.0.smithi119.stdout:39: finishing write tid 2 to smithi11939506-39 2024-09-18T11:30:08.751 INFO:tasks.rados.rados.0.smithi119.stdout:39: finishing write tid 3 to smithi11939506-39 2024-09-18T11:30:08.751 INFO:tasks.rados.rados.0.smithi119.stdout:39: finishing write tid 4 to smithi11939506-39 2024-09-18T11:30:08.751 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 39 v 12 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-18T11:30:08.751 INFO:tasks.rados.rados.0.smithi119.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2024-09-18T11:30:08.751 INFO:tasks.rados.rados.0.smithi119.stdout:40: finishing write tid 1 to smithi11939506-40 2024-09-18T11:30:08.751 INFO:tasks.rados.rados.0.smithi119.stdout:40: finishing write tid 2 to smithi11939506-40 2024-09-18T11:30:08.751 INFO:tasks.rados.rados.0.smithi119.stdout:40: finishing write tid 3 to smithi11939506-40 2024-09-18T11:30:08.751 INFO:tasks.rados.rados.0.smithi119.stdout:40: finishing write tid 4 to smithi11939506-40 2024-09-18T11:30:08.751 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 6 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-18T11:30:08.751 INFO:tasks.rados.rados.0.smithi119.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2024-09-18T11:30:08.751 INFO:tasks.rados.rados.0.smithi119.stdout:41: finishing write tid 1 to smithi11939506-41 2024-09-18T11:30:08.751 INFO:tasks.rados.rados.0.smithi119.stdout:41: finishing write tid 2 to smithi11939506-41 2024-09-18T11:30:08.752 INFO:tasks.rados.rados.0.smithi119.stdout:41: finishing write tid 3 to smithi11939506-41 2024-09-18T11:30:08.752 INFO:tasks.rados.rados.0.smithi119.stdout:41: finishing write tid 4 to smithi11939506-41 2024-09-18T11:30:08.752 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 41 v 15 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-18T11:30:08.752 INFO:tasks.rados.rados.0.smithi119.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2024-09-18T11:30:08.752 INFO:tasks.rados.rados.0.smithi119.stdout:42: finishing write tid 1 to smithi11939506-42 2024-09-18T11:30:08.752 INFO:tasks.rados.rados.0.smithi119.stdout:42: finishing write tid 2 to smithi11939506-42 2024-09-18T11:30:08.752 INFO:tasks.rados.rados.0.smithi119.stdout:42: finishing write tid 3 to smithi11939506-42 2024-09-18T11:30:08.752 INFO:tasks.rados.rados.0.smithi119.stdout:42: finishing write tid 4 to smithi11939506-42 2024-09-18T11:30:08.752 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 12 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T11:30:08.752 INFO:tasks.rados.rados.0.smithi119.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2024-09-18T11:30:08.752 INFO:tasks.rados.rados.0.smithi119.stdout:43: finishing write tid 1 to smithi11939506-43 2024-09-18T11:30:08.752 INFO:tasks.rados.rados.0.smithi119.stdout:39: done (15 left) 2024-09-18T11:30:08.752 INFO:tasks.rados.rados.0.smithi119.stdout:40: done (14 left) 2024-09-18T11:30:08.753 INFO:tasks.rados.rados.0.smithi119.stdout:41: done (13 left) 2024-09-18T11:30:08.753 INFO:tasks.rados.rados.0.smithi119.stdout:42: done (12 left) 2024-09-18T11:30:08.753 INFO:tasks.rados.rados.0.smithi119.stdout:55: copy oid 6 target oid 6 low_tier 2024-09-18T11:30:08.753 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 6 to 6 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.753 INFO:tasks.rados.rados.0.smithi119.stdout:56: copy oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.753 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 7 to 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.753 INFO:tasks.rados.rados.0.smithi119.stdout:57: copy oid 8 target oid 8 low_tier 2024-09-18T11:30:08.753 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 8 to 8 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.753 INFO:tasks.rados.rados.0.smithi119.stdout:58: copy oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.753 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 9 to 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.753 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.753 INFO:tasks.rados.rados.0.smithi119.stdout:43: finishing write tid 2 to smithi11939506-43 2024-09-18T11:30:08.754 INFO:tasks.rados.rados.0.smithi119.stdout:43: finishing write tid 3 to smithi11939506-43 2024-09-18T11:30:08.754 INFO:tasks.rados.rados.0.smithi119.stdout:43: finishing write tid 4 to smithi11939506-43 2024-09-18T11:30:08.754 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 43 v 15 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-18T11:30:08.754 INFO:tasks.rados.rados.0.smithi119.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:30:08.754 INFO:tasks.rados.rados.0.smithi119.stdout:44: finishing write tid 1 to smithi11939506-44 2024-09-18T11:30:08.754 INFO:tasks.rados.rados.0.smithi119.stdout:44: finishing write tid 2 to smithi11939506-44 2024-09-18T11:30:08.754 INFO:tasks.rados.rados.0.smithi119.stdout:44: finishing write tid 3 to smithi11939506-44 2024-09-18T11:30:08.754 INFO:tasks.rados.rados.0.smithi119.stdout:44: finishing write tid 4 to smithi11939506-44 2024-09-18T11:30:08.754 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 44 v 9 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-18T11:30:08.754 INFO:tasks.rados.rados.0.smithi119.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2024-09-18T11:30:08.754 INFO:tasks.rados.rados.0.smithi119.stdout:45: finishing write tid 1 to smithi11939506-45 2024-09-18T11:30:08.755 INFO:tasks.rados.rados.0.smithi119.stdout:45: finishing write tid 2 to smithi11939506-45 2024-09-18T11:30:08.755 INFO:tasks.rados.rados.0.smithi119.stdout:45: finishing write tid 3 to smithi11939506-45 2024-09-18T11:30:08.755 INFO:tasks.rados.rados.0.smithi119.stdout:45: finishing write tid 4 to smithi11939506-45 2024-09-18T11:30:08.755 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 3 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-18T11:30:08.755 INFO:tasks.rados.rados.0.smithi119.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2024-09-18T11:30:08.755 INFO:tasks.rados.rados.0.smithi119.stdout:46: finishing write tid 1 to smithi11939506-46 2024-09-18T11:30:08.755 INFO:tasks.rados.rados.0.smithi119.stdout:46: finishing write tid 2 to smithi11939506-46 2024-09-18T11:30:08.755 INFO:tasks.rados.rados.0.smithi119.stdout:46: finishing write tid 3 to smithi11939506-46 2024-09-18T11:30:08.755 INFO:tasks.rados.rados.0.smithi119.stdout:46: finishing write tid 4 to smithi11939506-46 2024-09-18T11:30:08.755 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 9 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-18T11:30:08.755 INFO:tasks.rados.rados.0.smithi119.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2024-09-18T11:30:08.755 INFO:tasks.rados.rados.0.smithi119.stdout:47: finishing write tid 1 to smithi11939506-47 2024-09-18T11:30:08.755 INFO:tasks.rados.rados.0.smithi119.stdout:47: finishing write tid 2 to smithi11939506-47 2024-09-18T11:30:08.755 INFO:tasks.rados.rados.0.smithi119.stdout:47: finishing write tid 3 to smithi11939506-47 2024-09-18T11:30:08.756 INFO:tasks.rados.rados.0.smithi119.stdout:47: finishing write tid 4 to smithi11939506-47 2024-09-18T11:30:08.756 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 9 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-18T11:30:08.756 INFO:tasks.rados.rados.0.smithi119.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2024-09-18T11:30:08.756 INFO:tasks.rados.rados.0.smithi119.stdout:48: finishing write tid 1 to smithi11939506-48 2024-09-18T11:30:08.756 INFO:tasks.rados.rados.0.smithi119.stdout:48: finishing write tid 2 to smithi11939506-48 2024-09-18T11:30:08.756 INFO:tasks.rados.rados.0.smithi119.stdout:48: finishing write tid 3 to smithi11939506-48 2024-09-18T11:30:08.756 INFO:tasks.rados.rados.0.smithi119.stdout:48: finishing write tid 4 to smithi11939506-48 2024-09-18T11:30:08.756 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 48 v 12 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-18T11:30:08.756 INFO:tasks.rados.rados.0.smithi119.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2024-09-18T11:30:08.756 INFO:tasks.rados.rados.0.smithi119.stdout:49: finishing write tid 1 to smithi11939506-49 2024-09-18T11:30:08.756 INFO:tasks.rados.rados.0.smithi119.stdout:49: finishing write tid 2 to smithi11939506-49 2024-09-18T11:30:08.756 INFO:tasks.rados.rados.0.smithi119.stdout:49: finishing write tid 3 to smithi11939506-49 2024-09-18T11:30:08.756 INFO:tasks.rados.rados.0.smithi119.stdout:49: finishing write tid 4 to smithi11939506-49 2024-09-18T11:30:08.757 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 18 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-18T11:30:08.757 INFO:tasks.rados.rados.0.smithi119.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2024-09-18T11:30:08.757 INFO:tasks.rados.rados.0.smithi119.stdout:50: finishing write tid 1 to smithi11939506-50 2024-09-18T11:30:08.757 INFO:tasks.rados.rados.0.smithi119.stdout:43: done (15 left) 2024-09-18T11:30:08.757 INFO:tasks.rados.rados.0.smithi119.stdout:44: done (14 left) 2024-09-18T11:30:08.757 INFO:tasks.rados.rados.0.smithi119.stdout:45: done (13 left) 2024-09-18T11:30:08.757 INFO:tasks.rados.rados.0.smithi119.stdout:46: done (12 left) 2024-09-18T11:30:08.757 INFO:tasks.rados.rados.0.smithi119.stdout:47: done (11 left) 2024-09-18T11:30:08.757 INFO:tasks.rados.rados.0.smithi119.stdout:48: done (10 left) 2024-09-18T11:30:08.757 INFO:tasks.rados.rados.0.smithi119.stdout:49: done (9 left) 2024-09-18T11:30:08.757 INFO:tasks.rados.rados.0.smithi119.stdout:59: copy oid 10 target oid 10 low_tier 2024-09-18T11:30:08.757 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 10 to 10 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.757 INFO:tasks.rados.rados.0.smithi119.stdout:60: copy oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.758 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 11 to 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.758 INFO:tasks.rados.rados.0.smithi119.stdout:61: copy oid 12 target oid 12 low_tier 2024-09-18T11:30:08.758 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 12 to 12 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.758 INFO:tasks.rados.rados.0.smithi119.stdout:62: copy oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.758 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 13 to 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.758 INFO:tasks.rados.rados.0.smithi119.stdout:63: copy oid 14 target oid 14 low_tier 2024-09-18T11:30:08.758 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 14 to 14 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.758 INFO:tasks.rados.rados.0.smithi119.stdout:64: copy oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.758 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 15 to 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.758 INFO:tasks.rados.rados.0.smithi119.stdout:65: copy oid 16 target oid 16 low_tier 2024-09-18T11:30:08.758 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 16 to 16 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.758 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.759 INFO:tasks.rados.rados.0.smithi119.stdout:50: finishing write tid 2 to smithi11939506-50 2024-09-18T11:30:08.759 INFO:tasks.rados.rados.0.smithi119.stdout:50: finishing write tid 3 to smithi11939506-50 2024-09-18T11:30:08.759 INFO:tasks.rados.rados.0.smithi119.stdout:50: finishing write tid 4 to smithi11939506-50 2024-09-18T11:30:08.759 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 9 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-18T11:30:08.759 INFO:tasks.rados.rados.0.smithi119.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2024-09-18T11:30:08.761 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:30:08.761 INFO:tasks.rados.rados.0.smithi119.stdout:50: done (15 left) 2024-09-18T11:30:08.761 INFO:tasks.rados.rados.0.smithi119.stdout:66: copy oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.761 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 17 to 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.761 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.764 INFO:tasks.rados.rados.0.smithi119.stdout:56: finishing copy op to oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.764 INFO:tasks.rados.rados.0.smithi119.stdout:56: done (15 left) 2024-09-18T11:30:08.764 INFO:tasks.rados.rados.0.smithi119.stdout:67: copy oid 18 target oid 18 low_tier 2024-09-18T11:30:08.764 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 18 to 18 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.764 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.767 INFO:tasks.rados.rados.0.smithi119.stdout:51: finishing copy op to oid 2 low_tier 2024-09-18T11:30:08.767 INFO:tasks.rados.rados.0.smithi119.stdout:51: done (15 left) 2024-09-18T11:30:08.767 INFO:tasks.rados.rados.0.smithi119.stdout:68: copy oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.767 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 19 to 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.768 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.768 INFO:tasks.rados.rados.0.smithi119.stdout:53: finishing copy op to oid 4 low_tier 2024-09-18T11:30:08.768 INFO:tasks.rados.rados.0.smithi119.stdout:53: done (15 left) 2024-09-18T11:30:08.768 INFO:tasks.rados.rados.0.smithi119.stdout:69: copy oid 20 target oid 20 low_tier 2024-09-18T11:30:08.768 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 20 to 20 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.768 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.773 INFO:tasks.rados.rados.0.smithi119.stdout:57: finishing copy op to oid 8 low_tier 2024-09-18T11:30:08.773 INFO:tasks.rados.rados.0.smithi119.stdout:57: done (15 left) 2024-09-18T11:30:08.774 INFO:tasks.rados.rados.0.smithi119.stdout:70: copy oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.774 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 21 to 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.774 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.775 INFO:tasks.rados.rados.0.smithi119.stdout:64: finishing copy op to oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.775 INFO:tasks.rados.rados.0.smithi119.stdout:64: done (15 left) 2024-09-18T11:30:08.775 INFO:tasks.rados.rados.0.smithi119.stdout:71: copy oid 22 target oid 22 low_tier 2024-09-18T11:30:08.775 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 22 to 22 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.775 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.777 INFO:tasks.rados.rados.0.smithi119.stdout:60: finishing copy op to oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.777 INFO:tasks.rados.rados.0.smithi119.stdout:60: done (15 left) 2024-09-18T11:30:08.777 INFO:tasks.rados.rados.0.smithi119.stdout:72: copy oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.777 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 23 to 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.777 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.777 INFO:tasks.rados.rados.0.smithi119.stdout:58: finishing copy op to oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.777 INFO:tasks.rados.rados.0.smithi119.stdout:58: done (15 left) 2024-09-18T11:30:08.777 INFO:tasks.rados.rados.0.smithi119.stdout:73: copy oid 24 target oid 24 low_tier 2024-09-18T11:30:08.777 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 24 to 24 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.778 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.778 INFO:tasks.rados.rados.0.smithi119.stdout:55: finishing copy op to oid 6 low_tier 2024-09-18T11:30:08.778 INFO:tasks.rados.rados.0.smithi119.stdout:55: done (15 left) 2024-09-18T11:30:08.778 INFO:tasks.rados.rados.0.smithi119.stdout:74: copy oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.778 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 25 to 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.779 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.779 INFO:tasks.rados.rados.0.smithi119.stdout:61: finishing copy op to oid 12 low_tier 2024-09-18T11:30:08.779 INFO:tasks.rados.rados.0.smithi119.stdout:61: done (15 left) 2024-09-18T11:30:08.779 INFO:tasks.rados.rados.0.smithi119.stdout:75: copy oid 26 target oid 26 low_tier 2024-09-18T11:30:08.779 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 26 to 26 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.779 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.783 INFO:tasks.rados.rados.0.smithi119.stdout:52: finishing copy op to oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.783 INFO:tasks.rados.rados.0.smithi119.stdout:52: done (15 left) 2024-09-18T11:30:08.783 INFO:tasks.rados.rados.0.smithi119.stdout:76: copy oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.784 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 27 to 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.784 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.784 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:08.782+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:08.785 INFO:tasks.rados.rados.0.smithi119.stdout:65: finishing copy op to oid 16 low_tier 2024-09-18T11:30:08.785 INFO:tasks.rados.rados.0.smithi119.stdout:65: done (15 left) 2024-09-18T11:30:08.785 INFO:tasks.rados.rados.0.smithi119.stdout:77: copy oid 28 target oid 28 low_tier 2024-09-18T11:30:08.785 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 28 to 28 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.785 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.789 INFO:tasks.rados.rados.0.smithi119.stdout:71: finishing copy op to oid 22 low_tier 2024-09-18T11:30:08.789 INFO:tasks.rados.rados.0.smithi119.stdout:71: done (15 left) 2024-09-18T11:30:08.789 INFO:tasks.rados.rados.0.smithi119.stdout:78: copy oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.789 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 29 to 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.789 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.789 INFO:tasks.rados.rados.0.smithi119.stdout:62: finishing copy op to oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.789 INFO:tasks.rados.rados.0.smithi119.stdout:62: done (15 left) 2024-09-18T11:30:08.789 INFO:tasks.rados.rados.0.smithi119.stdout:79: copy oid 30 target oid 30 low_tier 2024-09-18T11:30:08.790 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 30 to 30 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.790 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.790 INFO:tasks.rados.rados.0.smithi119.stdout:59: finishing copy op to oid 10 low_tier 2024-09-18T11:30:08.790 INFO:tasks.rados.rados.0.smithi119.stdout:59: done (15 left) 2024-09-18T11:30:08.790 INFO:tasks.rados.rados.0.smithi119.stdout:80: copy oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.790 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 31 to 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.790 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.790 INFO:tasks.rados.rados.0.smithi119.stdout:63: finishing copy op to oid 14 low_tier 2024-09-18T11:30:08.790 INFO:tasks.rados.rados.0.smithi119.stdout:63: done (15 left) 2024-09-18T11:30:08.790 INFO:tasks.rados.rados.0.smithi119.stdout:81: copy oid 32 target oid 32 low_tier 2024-09-18T11:30:08.790 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 32 to 32 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.790 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.792 INFO:tasks.rados.rados.0.smithi119.stdout:68: finishing copy op to oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.792 INFO:tasks.rados.rados.0.smithi119.stdout:68: done (15 left) 2024-09-18T11:30:08.792 INFO:tasks.rados.rados.0.smithi119.stdout:82: copy oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.792 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 33 to 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.792 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.796 INFO:tasks.rados.rados.0.smithi119.stdout:70: finishing copy op to oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.796 INFO:tasks.rados.rados.0.smithi119.stdout:70: done (15 left) 2024-09-18T11:30:08.796 INFO:tasks.rados.rados.0.smithi119.stdout:83: copy oid 34 target oid 34 low_tier 2024-09-18T11:30:08.796 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 34 to 34 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.796 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.798 INFO:tasks.rados.rados.0.smithi119.stdout:69: finishing copy op to oid 20 low_tier 2024-09-18T11:30:08.798 INFO:tasks.rados.rados.0.smithi119.stdout:69: done (15 left) 2024-09-18T11:30:08.798 INFO:tasks.rados.rados.0.smithi119.stdout:84: copy oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.798 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 35 to 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.798 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.799 INFO:tasks.rados.rados.0.smithi119.stdout:74: finishing copy op to oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.799 INFO:tasks.rados.rados.0.smithi119.stdout:74: done (15 left) 2024-09-18T11:30:08.799 INFO:tasks.rados.rados.0.smithi119.stdout:85: copy oid 36 target oid 36 low_tier 2024-09-18T11:30:08.799 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 36 to 36 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.799 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.806 INFO:tasks.rados.rados.0.smithi119.stdout:67: finishing copy op to oid 18 low_tier 2024-09-18T11:30:08.806 INFO:tasks.rados.rados.0.smithi119.stdout:54: finishing copy op to oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.806 INFO:tasks.rados.rados.0.smithi119.stdout:54: done (15 left) 2024-09-18T11:30:08.807 INFO:tasks.rados.rados.0.smithi119.stdout:67: done (14 left) 2024-09-18T11:30:08.807 INFO:tasks.rados.rados.0.smithi119.stdout:86: copy oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.807 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 37 to 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.807 INFO:tasks.rados.rados.0.smithi119.stdout:87: copy oid 38 target oid 38 low_tier 2024-09-18T11:30:08.807 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 38 to 38 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.807 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.811 INFO:tasks.rados.rados.0.smithi119.stdout:79: finishing copy op to oid 30 low_tier 2024-09-18T11:30:08.811 INFO:tasks.rados.rados.0.smithi119.stdout:79: done (15 left) 2024-09-18T11:30:08.811 INFO:tasks.rados.rados.0.smithi119.stdout:88: copy oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.812 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 39 to 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.812 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.814 INFO:tasks.rados.rados.0.smithi119.stdout:66: finishing copy op to oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.814 INFO:tasks.rados.rados.0.smithi119.stdout:66: done (15 left) 2024-09-18T11:30:08.814 INFO:tasks.rados.rados.0.smithi119.stdout:89: copy oid 40 target oid 40 low_tier 2024-09-18T11:30:08.814 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 40 to 40 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.814 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.815 INFO:tasks.rados.rados.0.smithi119.stdout:73: finishing copy op to oid 24 low_tier 2024-09-18T11:30:08.815 INFO:tasks.rados.rados.0.smithi119.stdout:73: done (15 left) 2024-09-18T11:30:08.815 INFO:tasks.rados.rados.0.smithi119.stdout:90: copy oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.815 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 41 to 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.815 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.816 INFO:tasks.rados.rados.0.smithi119.stdout:84: finishing copy op to oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.817 INFO:tasks.rados.rados.0.smithi119.stdout:84: done (15 left) 2024-09-18T11:30:08.817 INFO:tasks.rados.rados.0.smithi119.stdout:91: copy oid 42 target oid 42 low_tier 2024-09-18T11:30:08.817 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 42 to 42 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.817 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.821 INFO:tasks.rados.rados.0.smithi119.stdout:85: finishing copy op to oid 36 low_tier 2024-09-18T11:30:08.821 INFO:tasks.rados.rados.0.smithi119.stdout:85: done (15 left) 2024-09-18T11:30:08.821 INFO:tasks.rados.rados.0.smithi119.stdout:92: copy oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.821 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 43 to 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.821 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.822 INFO:tasks.rados.rados.0.smithi119.stdout:77: finishing copy op to oid 28 low_tier 2024-09-18T11:30:08.822 INFO:tasks.rados.rados.0.smithi119.stdout:77: done (15 left) 2024-09-18T11:30:08.822 INFO:tasks.rados.rados.0.smithi119.stdout:93: copy oid 44 target oid 44 low_tier 2024-09-18T11:30:08.822 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 44 to 44 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.822 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.823 INFO:tasks.rados.rados.0.smithi119.stdout:76: finishing copy op to oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.823 INFO:tasks.rados.rados.0.smithi119.stdout:80: finishing copy op to oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.823 INFO:tasks.rados.rados.0.smithi119.stdout:81: finishing copy op to oid 32 low_tier 2024-09-18T11:30:08.824 INFO:tasks.rados.rados.0.smithi119.stdout:76: done (15 left) 2024-09-18T11:30:08.824 INFO:tasks.rados.rados.0.smithi119.stdout:80: done (14 left) 2024-09-18T11:30:08.824 INFO:tasks.rados.rados.0.smithi119.stdout:81: done (13 left) 2024-09-18T11:30:08.824 INFO:tasks.rados.rados.0.smithi119.stdout:94: copy oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.824 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 45 to 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.824 INFO:tasks.rados.rados.0.smithi119.stdout:95: copy oid 46 target oid 46 low_tier 2024-09-18T11:30:08.824 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 46 to 46 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.824 INFO:tasks.rados.rados.0.smithi119.stdout:96: copy oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.824 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 47 to 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.824 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.825 INFO:tasks.rados.rados.0.smithi119.stdout:72: finishing copy op to oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.825 INFO:tasks.rados.rados.0.smithi119.stdout:75: finishing copy op to oid 26 low_tier 2024-09-18T11:30:08.825 INFO:tasks.rados.rados.0.smithi119.stdout:72: done (15 left) 2024-09-18T11:30:08.825 INFO:tasks.rados.rados.0.smithi119.stdout:75: done (14 left) 2024-09-18T11:30:08.825 INFO:tasks.rados.rados.0.smithi119.stdout:97: copy oid 48 target oid 48 low_tier 2024-09-18T11:30:08.825 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 48 to 48 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.825 INFO:tasks.rados.rados.0.smithi119.stdout:98: copy oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.825 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 49 to 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.826 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.829 INFO:tasks.rados.rados.0.smithi119.stdout:86: finishing copy op to oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.829 INFO:tasks.rados.rados.0.smithi119.stdout:86: done (15 left) 2024-09-18T11:30:08.829 INFO:tasks.rados.rados.0.smithi119.stdout:99: copy oid 50 target oid 50 low_tier 2024-09-18T11:30:08.829 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 50 to 50 low_tier tgt_pool_name low_tier 2024-09-18T11:30:08.829 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.832 INFO:tasks.rados.rados.0.smithi119.stdout:78: finishing copy op to oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.832 INFO:tasks.rados.rados.0.smithi119.stdout:78: done (15 left) 2024-09-18T11:30:08.832 INFO:tasks.rados.rados.0.smithi119.stdout:100: copy oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.832 INFO:tasks.rados.rados.0.smithi119.stdout:copy op oid 1 to 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-18T11:30:08.832 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.834 INFO:tasks.rados.rados.0.smithi119.stdout:91: finishing copy op to oid 42 low_tier 2024-09-18T11:30:08.835 INFO:tasks.rados.rados.0.smithi119.stdout:91: done (15 left) 2024-09-18T11:30:08.835 INFO:tasks.rados.rados.0.smithi119.stdout:101: set_chunk oid 2 target oid 2 low_tier 2024-09-18T11:30:08.835 INFO:tasks.rados.rados.0.smithi119.stdout:101: set_chunk oid 2 offset: 1029120 length: 820224 target oid smithi11939506-2 low_tier offset: 1029120 snap -1 2024-09-18T11:30:08.835 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.836 INFO:tasks.rados.rados.0.smithi119.stdout:82: finishing copy op to oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.836 INFO:tasks.rados.rados.0.smithi119.stdout:82: done (15 left) 2024-09-18T11:30:08.836 INFO:tasks.rados.rados.0.smithi119.stdout:102: set_chunk oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.836 INFO:tasks.rados.rados.0.smithi119.stdout:102: set_chunk oid 3 offset: 936960 length: 818688 target oid smithi11939506-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 936960 snap -1 2024-09-18T11:30:08.836 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.836 INFO:tasks.rados.rados.0.smithi119.stdout:83: finishing copy op to oid 34 low_tier 2024-09-18T11:30:08.836 INFO:tasks.rados.rados.0.smithi119.stdout:83: done (15 left) 2024-09-18T11:30:08.836 INFO:tasks.rados.rados.0.smithi119.stdout:103: set_chunk oid 4 target oid 4 low_tier 2024-09-18T11:30:08.836 INFO:tasks.rados.rados.0.smithi119.stdout:103: set_chunk oid 4 offset: 1169920 length: 229888 target oid smithi11939506-4 low_tier offset: 1169920 snap -1 2024-09-18T11:30:08.837 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.837 INFO:tasks.rados.rados.0.smithi119.stdout:90: finishing copy op to oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.837 INFO:tasks.rados.rados.0.smithi119.stdout:90: done (15 left) 2024-09-18T11:30:08.837 INFO:tasks.rados.rados.0.smithi119.stdout:104: set_chunk oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.837 INFO:tasks.rados.rados.0.smithi119.stdout:104: set_chunk oid 5 offset: 1008640 length: 758272 target oid smithi11939506-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1008640 snap -1 2024-09-18T11:30:08.837 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.850 INFO:tasks.rados.rados.0.smithi119.stdout:93: finishing copy op to oid 44 low_tier 2024-09-18T11:30:08.850 INFO:tasks.rados.rados.0.smithi119.stdout:93: done (15 left) 2024-09-18T11:30:08.850 INFO:tasks.rados.rados.0.smithi119.stdout:105: set_chunk oid 6 target oid 6 low_tier 2024-09-18T11:30:08.850 INFO:tasks.rados.rados.0.smithi119.stdout:105: set_chunk oid 6 offset: 1276416 length: 60928 target oid smithi11939506-6 low_tier offset: 1276416 snap -1 2024-09-18T11:30:08.850 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.852 INFO:tasks.rados.rados.0.smithi119.stdout:99: finishing copy op to oid 50 low_tier 2024-09-18T11:30:08.852 INFO:tasks.rados.rados.0.smithi119.stdout:99: done (15 left) 2024-09-18T11:30:08.852 INFO:tasks.rados.rados.0.smithi119.stdout:106: set_chunk oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.852 INFO:tasks.rados.rados.0.smithi119.stdout:106: set_chunk oid 7 offset: 823296 length: 735232 target oid smithi11939506-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 823296 snap -1 2024-09-18T11:30:08.852 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.853 INFO:tasks.rados.rados.0.smithi119.stdout:88: finishing copy op to oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.853 INFO:tasks.rados.rados.0.smithi119.stdout:88: done (15 left) 2024-09-18T11:30:08.853 INFO:tasks.rados.rados.0.smithi119.stdout:107: set_chunk oid 8 target oid 8 low_tier 2024-09-18T11:30:08.853 INFO:tasks.rados.rados.0.smithi119.stdout:107: set_chunk oid 8 offset: 529920 length: 869888 target oid smithi11939506-8 low_tier offset: 529920 snap -1 2024-09-18T11:30:08.853 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.855 INFO:tasks.rados.rados.0.smithi119.stdout:87: finishing copy op to oid 38 low_tier 2024-09-18T11:30:08.855 INFO:tasks.rados.rados.0.smithi119.stdout:87: done (15 left) 2024-09-18T11:30:08.855 INFO:tasks.rados.rados.0.smithi119.stdout:108: set_chunk oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.855 INFO:tasks.rados.rados.0.smithi119.stdout:108: set_chunk oid 9 offset: 958464 length: 208384 target oid smithi11939506-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 958464 snap -1 2024-09-18T11:30:08.855 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.856 INFO:tasks.rados.rados.0.smithi119.stdout:89: finishing copy op to oid 40 low_tier 2024-09-18T11:30:08.856 INFO:tasks.rados.rados.0.smithi119.stdout:89: done (15 left) 2024-09-18T11:30:08.856 INFO:tasks.rados.rados.0.smithi119.stdout:109: set_chunk oid 10 target oid 10 low_tier 2024-09-18T11:30:08.856 INFO:tasks.rados.rados.0.smithi119.stdout:109: set_chunk oid 10 offset: 780800 length: 539648 target oid smithi11939506-10 low_tier offset: 780800 snap -1 2024-09-18T11:30:08.856 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.856 INFO:tasks.rados.rados.0.smithi119.stdout:94: finishing copy op to oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.856 INFO:tasks.rados.rados.0.smithi119.stdout:94: done (15 left) 2024-09-18T11:30:08.857 INFO:tasks.rados.rados.0.smithi119.stdout:110: set_chunk oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.857 INFO:tasks.rados.rados.0.smithi119.stdout:110: set_chunk oid 11 offset: 540672 length: 1238016 target oid smithi11939506-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 540672 snap -1 2024-09-18T11:30:08.857 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.858 INFO:tasks.rados.rados.0.smithi119.stdout:105: finishing set_chunk to oid 6 2024-09-18T11:30:08.859 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 6 v 3 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-18T11:30:08.859 INFO:tasks.rados.rados.0.smithi119.stdout:105: done (15 left) 2024-09-18T11:30:08.859 INFO:tasks.rados.rados.0.smithi119.stdout:111: set_chunk oid 12 target oid 12 low_tier 2024-09-18T11:30:08.859 INFO:tasks.rados.rados.0.smithi119.stdout:111: set_chunk oid 12 offset: 240128 length: 332288 target oid smithi11939506-12 low_tier offset: 240128 snap -1 2024-09-18T11:30:08.859 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.859 INFO:tasks.rados.rados.0.smithi119.stdout:100: finishing copy op to oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.859 INFO:tasks.rados.rados.0.smithi119.stdout:100: done (15 left) 2024-09-18T11:30:08.859 INFO:tasks.rados.rados.0.smithi119.stdout:112: set_chunk oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.859 INFO:tasks.rados.rados.0.smithi119.stdout:112: set_chunk oid 13 offset: 72704 length: 1255424 target oid smithi11939506-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 72704 snap -1 2024-09-18T11:30:08.859 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.859 INFO:tasks.rados.rados.0.smithi119.stdout:103: finishing set_chunk to oid 4 2024-09-18T11:30:08.859 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 4 v 3 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-18T11:30:08.859 INFO:tasks.rados.rados.0.smithi119.stdout:103: done (15 left) 2024-09-18T11:30:08.860 INFO:tasks.rados.rados.0.smithi119.stdout:113: set_chunk oid 14 target oid 14 low_tier 2024-09-18T11:30:08.860 INFO:tasks.rados.rados.0.smithi119.stdout:113: set_chunk oid 14 offset: 1027072 length: 599552 target oid smithi11939506-14 low_tier offset: 1027072 snap -1 2024-09-18T11:30:08.860 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.860 INFO:tasks.rados.rados.0.smithi119.stdout:96: finishing copy op to oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.861 INFO:tasks.rados.rados.0.smithi119.stdout:95: finishing copy op to oid 46 low_tier 2024-09-18T11:30:08.861 INFO:tasks.rados.rados.0.smithi119.stdout:95: done (15 left) 2024-09-18T11:30:08.861 INFO:tasks.rados.rados.0.smithi119.stdout:96: done (14 left) 2024-09-18T11:30:08.861 INFO:tasks.rados.rados.0.smithi119.stdout:114: set_chunk oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.861 INFO:tasks.rados.rados.0.smithi119.stdout:114: set_chunk oid 15 offset: 205312 length: 1156096 target oid smithi11939506-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 205312 snap -1 2024-09-18T11:30:08.861 INFO:tasks.rados.rados.0.smithi119.stdout:115: set_chunk oid 16 target oid 16 low_tier 2024-09-18T11:30:08.861 INFO:tasks.rados.rados.0.smithi119.stdout:115: set_chunk oid 16 offset: 5632 length: 1084928 target oid smithi11939506-16 low_tier offset: 5632 snap -1 2024-09-18T11:30:08.861 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.861 INFO:tasks.rados.rados.0.smithi119.stdout:92: finishing copy op to oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.861 INFO:tasks.rados.rados.0.smithi119.stdout:92: done (15 left) 2024-09-18T11:30:08.861 INFO:tasks.rados.rados.0.smithi119.stdout:116: set_chunk oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.861 INFO:tasks.rados.rados.0.smithi119.stdout:116: set_chunk oid 17 offset: 285696 length: 769536 target oid smithi11939506-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 285696 snap -1 2024-09-18T11:30:08.862 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.864 INFO:tasks.rados.rados.0.smithi119.stdout:102: finishing set_chunk to oid 3 2024-09-18T11:30:08.864 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 3 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-18T11:30:08.865 INFO:tasks.rados.rados.0.smithi119.stdout:102: done (15 left) 2024-09-18T11:30:08.865 INFO:tasks.rados.rados.0.smithi119.stdout:117: set_chunk oid 18 target oid 18 low_tier 2024-09-18T11:30:08.865 INFO:tasks.rados.rados.0.smithi119.stdout:117: set_chunk oid 18 offset: 221696 length: 1707520 target oid smithi11939506-18 low_tier offset: 221696 snap -1 2024-09-18T11:30:08.865 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.865 INFO:tasks.rados.rados.0.smithi119.stdout:97: finishing copy op to oid 48 low_tier 2024-09-18T11:30:08.865 INFO:tasks.rados.rados.0.smithi119.stdout:97: done (15 left) 2024-09-18T11:30:08.865 INFO:tasks.rados.rados.0.smithi119.stdout:118: set_chunk oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.865 INFO:tasks.rados.rados.0.smithi119.stdout:118: set_chunk oid 19 offset: 1221632 length: 318976 target oid smithi11939506-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1221632 snap -1 2024-09-18T11:30:08.865 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.865 INFO:tasks.rados.rados.0.smithi119.stdout:98: finishing copy op to oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.865 INFO:tasks.rados.rados.0.smithi119.stdout:98: done (15 left) 2024-09-18T11:30:08.865 INFO:tasks.rados.rados.0.smithi119.stdout:119: set_chunk oid 20 target oid 20 low_tier 2024-09-18T11:30:08.865 INFO:tasks.rados.rados.0.smithi119.stdout:119: set_chunk oid 20 offset: 600576 length: 192512 target oid smithi11939506-20 low_tier offset: 600576 snap -1 2024-09-18T11:30:08.866 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.866 INFO:tasks.rados.rados.0.smithi119.stdout:101: finishing set_chunk to oid 2 2024-09-18T11:30:08.866 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 3 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-18T11:30:08.866 INFO:tasks.rados.rados.0.smithi119.stdout:101: done (15 left) 2024-09-18T11:30:08.866 INFO:tasks.rados.rados.0.smithi119.stdout:120: set_chunk oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.866 INFO:tasks.rados.rados.0.smithi119.stdout:120: set_chunk oid 21 offset: 833536 length: 49664 target oid smithi11939506-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 833536 snap -1 2024-09-18T11:30:08.866 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.866 INFO:tasks.rados.rados.0.smithi119.stdout:108: finishing set_chunk to oid 9 2024-09-18T11:30:08.866 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 6 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T11:30:08.866 INFO:tasks.rados.rados.0.smithi119.stdout:104: finishing set_chunk to oid 5 2024-09-18T11:30:08.866 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 6 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-18T11:30:08.866 INFO:tasks.rados.rados.0.smithi119.stdout:104: done (15 left) 2024-09-18T11:30:08.866 INFO:tasks.rados.rados.0.smithi119.stdout:108: done (14 left) 2024-09-18T11:30:08.867 INFO:tasks.rados.rados.0.smithi119.stdout:121: set_chunk oid 22 target oid 22 low_tier 2024-09-18T11:30:08.867 INFO:tasks.rados.rados.0.smithi119.stdout:121: set_chunk oid 22 offset: 54784 length: 1348096 target oid smithi11939506-22 low_tier offset: 54784 snap -1 2024-09-18T11:30:08.867 INFO:tasks.rados.rados.0.smithi119.stdout:122: set_chunk oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.867 INFO:tasks.rados.rados.0.smithi119.stdout:122: set_chunk oid 23 offset: 1157632 length: 1017856 target oid smithi11939506-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1157632 snap -1 2024-09-18T11:30:08.867 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.867 INFO:tasks.rados.rados.0.smithi119.stdout:107: finishing set_chunk to oid 8 2024-09-18T11:30:08.867 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 8 v 6 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-18T11:30:08.867 INFO:tasks.rados.rados.0.smithi119.stdout:107: done (15 left) 2024-09-18T11:30:08.867 INFO:tasks.rados.rados.0.smithi119.stdout:123: set_chunk oid 24 target oid 24 low_tier 2024-09-18T11:30:08.867 INFO:tasks.rados.rados.0.smithi119.stdout:123: set_chunk oid 24 offset: 1648128 length: 147968 target oid smithi11939506-24 low_tier offset: 1648128 snap -1 2024-09-18T11:30:08.867 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.867 INFO:tasks.rados.rados.0.smithi119.stdout:110: finishing set_chunk to oid 11 2024-09-18T11:30:08.867 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 11 v 3 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-18T11:30:08.868 INFO:tasks.rados.rados.0.smithi119.stdout:110: done (15 left) 2024-09-18T11:30:08.868 INFO:tasks.rados.rados.0.smithi119.stdout:124: set_chunk oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.868 INFO:tasks.rados.rados.0.smithi119.stdout:124: set_chunk oid 25 offset: 1197056 length: 867328 target oid smithi11939506-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1197056 snap -1 2024-09-18T11:30:08.868 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.868 INFO:tasks.rados.rados.0.smithi119.stdout:109: finishing set_chunk to oid 10 2024-09-18T11:30:08.868 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 10 v 9 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-18T11:30:08.868 INFO:tasks.rados.rados.0.smithi119.stdout:109: done (15 left) 2024-09-18T11:30:08.868 INFO:tasks.rados.rados.0.smithi119.stdout:125: set_chunk oid 26 target oid 26 low_tier 2024-09-18T11:30:08.868 INFO:tasks.rados.rados.0.smithi119.stdout:125: set_chunk oid 26 offset: 1281024 length: 299008 target oid smithi11939506-26 low_tier offset: 1281024 snap -1 2024-09-18T11:30:08.868 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.870 INFO:tasks.rados.rados.0.smithi119.stdout:112: finishing set_chunk to oid 13 2024-09-18T11:30:08.871 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 13 v 9 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-18T11:30:08.871 INFO:tasks.rados.rados.0.smithi119.stdout:112: done (15 left) 2024-09-18T11:30:08.871 INFO:tasks.rados.rados.0.smithi119.stdout:126: set_chunk oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.871 INFO:tasks.rados.rados.0.smithi119.stdout:126: set_chunk oid 27 offset: 748544 length: 1571328 target oid smithi11939506-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 748544 snap -1 2024-09-18T11:30:08.871 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.871 INFO:tasks.rados.rados.0.smithi119.stdout:115: finishing set_chunk to oid 16 2024-09-18T11:30:08.871 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 3 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-18T11:30:08.871 INFO:tasks.rados.rados.0.smithi119.stdout:115: done (15 left) 2024-09-18T11:30:08.871 INFO:tasks.rados.rados.0.smithi119.stdout:127: set_chunk oid 28 target oid 28 low_tier 2024-09-18T11:30:08.871 INFO:tasks.rados.rados.0.smithi119.stdout:127: set_chunk oid 28 offset: 1176576 length: 934400 target oid smithi11939506-28 low_tier offset: 1176576 snap -1 2024-09-18T11:30:08.871 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.872 INFO:tasks.rados.rados.0.smithi119.stdout:106: finishing set_chunk to oid 7 2024-09-18T11:30:08.872 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 7 v 6 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-18T11:30:08.872 INFO:tasks.rados.rados.0.smithi119.stdout:111: finishing set_chunk to oid 12 2024-09-18T11:30:08.872 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 12 v 6 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-18T11:30:08.872 INFO:tasks.rados.rados.0.smithi119.stdout:106: done (15 left) 2024-09-18T11:30:08.872 INFO:tasks.rados.rados.0.smithi119.stdout:111: done (14 left) 2024-09-18T11:30:08.872 INFO:tasks.rados.rados.0.smithi119.stdout:128: set_chunk oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.872 INFO:tasks.rados.rados.0.smithi119.stdout:128: set_chunk oid 29 offset: 2334208 length: 88576 target oid smithi11939506-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 2334208 snap -1 2024-09-18T11:30:08.872 INFO:tasks.rados.rados.0.smithi119.stdout:129: set_chunk oid 30 target oid 30 low_tier 2024-09-18T11:30:08.872 INFO:tasks.rados.rados.0.smithi119.stdout:129: set_chunk oid 30 offset: 404480 length: 701440 target oid smithi11939506-30 low_tier offset: 404480 snap -1 2024-09-18T11:30:08.872 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.872 INFO:tasks.rados.rados.0.smithi119.stdout:117: finishing set_chunk to oid 18 2024-09-18T11:30:08.872 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 3 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-18T11:30:08.873 INFO:tasks.rados.rados.0.smithi119.stdout:117: done (15 left) 2024-09-18T11:30:08.873 INFO:tasks.rados.rados.0.smithi119.stdout:130: set_chunk oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.873 INFO:tasks.rados.rados.0.smithi119.stdout:130: set_chunk oid 31 offset: 33792 length: 1595392 target oid smithi11939506-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 33792 snap -1 2024-09-18T11:30:08.873 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.873 INFO:tasks.rados.rados.0.smithi119.stdout:121: finishing set_chunk to oid 22 2024-09-18T11:30:08.873 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 15 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-18T11:30:08.873 INFO:tasks.rados.rados.0.smithi119.stdout:121: done (15 left) 2024-09-18T11:30:08.873 INFO:tasks.rados.rados.0.smithi119.stdout:131: set_chunk oid 32 target oid 32 low_tier 2024-09-18T11:30:08.873 INFO:tasks.rados.rados.0.smithi119.stdout:131: set_chunk oid 32 offset: 1563136 length: 434176 target oid smithi11939506-32 low_tier offset: 1563136 snap -1 2024-09-18T11:30:08.873 INFO:tasks.rados.rados.0.smithi119.stdout:113: finishing set_chunk to oid 14 2024-09-18T11:30:08.873 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 3 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-18T11:30:08.873 INFO:tasks.rados.rados.0.smithi119.stdout:113: done (15 left) 2024-09-18T11:30:08.873 INFO:tasks.rados.rados.0.smithi119.stdout:132: set_chunk oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.874 INFO:tasks.rados.rados.0.smithi119.stdout:132: set_chunk oid 33 offset: 1959936 length: 556544 target oid smithi11939506-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1959936 snap -1 2024-09-18T11:30:08.874 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.874 INFO:tasks.rados.rados.0.smithi119.stdout:118: finishing set_chunk to oid 19 2024-09-18T11:30:08.874 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 12 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-18T11:30:08.874 INFO:tasks.rados.rados.0.smithi119.stdout:118: done (15 left) 2024-09-18T11:30:08.874 INFO:tasks.rados.rados.0.smithi119.stdout:133: set_chunk oid 34 target oid 34 low_tier 2024-09-18T11:30:08.874 INFO:tasks.rados.rados.0.smithi119.stdout:133: set_chunk oid 34 offset: 533504 length: 420352 target oid smithi11939506-34 low_tier offset: 533504 snap -1 2024-09-18T11:30:08.874 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.880 INFO:tasks.rados.rados.0.smithi119.stdout:116: finishing set_chunk to oid 17 2024-09-18T11:30:08.881 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 12 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-18T11:30:08.881 INFO:tasks.rados.rados.0.smithi119.stdout:114: finishing set_chunk to oid 15 2024-09-18T11:30:08.881 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 3 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-18T11:30:08.881 INFO:tasks.rados.rados.0.smithi119.stdout:114: done (15 left) 2024-09-18T11:30:08.881 INFO:tasks.rados.rados.0.smithi119.stdout:116: done (14 left) 2024-09-18T11:30:08.881 INFO:tasks.rados.rados.0.smithi119.stdout:134: set_chunk oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.881 INFO:tasks.rados.rados.0.smithi119.stdout:134: set_chunk oid 35 offset: 928256 length: 1592832 target oid smithi11939506-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 928256 snap -1 2024-09-18T11:30:08.881 INFO:tasks.rados.rados.0.smithi119.stdout:135: set_chunk oid 36 target oid 36 low_tier 2024-09-18T11:30:08.881 INFO:tasks.rados.rados.0.smithi119.stdout:135: set_chunk oid 36 offset: 2460672 length: 120832 target oid smithi11939506-36 low_tier offset: 2460672 snap -1 2024-09-18T11:30:08.881 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.881 INFO:tasks.rados.rados.0.smithi119.stdout:120: finishing set_chunk to oid 21 2024-09-18T11:30:08.881 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 6 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-18T11:30:08.882 INFO:tasks.rados.rados.0.smithi119.stdout:120: done (15 left) 2024-09-18T11:30:08.882 INFO:tasks.rados.rados.0.smithi119.stdout:136: set_chunk oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.882 INFO:tasks.rados.rados.0.smithi119.stdout:136: set_chunk oid 37 offset: 1391616 length: 862208 target oid smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1391616 snap -1 2024-09-18T11:30:08.882 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.882 INFO:tasks.rados.rados.0.smithi119.stdout:123: finishing set_chunk to oid 24 2024-09-18T11:30:08.882 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 9 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-18T11:30:08.882 INFO:tasks.rados.rados.0.smithi119.stdout:123: done (15 left) 2024-09-18T11:30:08.882 INFO:tasks.rados.rados.0.smithi119.stdout:137: set_chunk oid 38 target oid 38 low_tier 2024-09-18T11:30:08.882 INFO:tasks.rados.rados.0.smithi119.stdout:137: set_chunk oid 38 offset: 1235456 length: 1340416 target oid smithi11939506-38 low_tier offset: 1235456 snap -1 2024-09-18T11:30:08.882 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.882 INFO:tasks.rados.rados.0.smithi119.stdout:119: finishing set_chunk to oid 20 2024-09-18T11:30:08.882 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 20 v 3 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-18T11:30:08.883 INFO:tasks.rados.rados.0.smithi119.stdout:119: done (15 left) 2024-09-18T11:30:08.883 INFO:tasks.rados.rados.0.smithi119.stdout:138: set_chunk oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.883 INFO:tasks.rados.rados.0.smithi119.stdout:138: set_chunk oid 39 offset: 671232 length: 1901056 target oid smithi11939506-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 671232 snap -1 2024-09-18T11:30:08.883 INFO:tasks.rados.rados.0.smithi119.stdout:124: finishing set_chunk to oid 25 2024-09-18T11:30:08.883 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 3 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-18T11:30:08.883 INFO:tasks.rados.rados.0.smithi119.stdout:124: done (15 left) 2024-09-18T11:30:08.883 INFO:tasks.rados.rados.0.smithi119.stdout:139: set_chunk oid 40 target oid 40 low_tier 2024-09-18T11:30:08.883 INFO:tasks.rados.rados.0.smithi119.stdout:139: set_chunk oid 40 offset: 704000 length: 650240 target oid smithi11939506-40 low_tier offset: 704000 snap -1 2024-09-18T11:30:08.883 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.883 INFO:tasks.rados.rados.0.smithi119.stdout:122: finishing set_chunk to oid 23 2024-09-18T11:30:08.883 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 6 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-18T11:30:08.883 INFO:tasks.rados.rados.0.smithi119.stdout:125: finishing set_chunk to oid 26 2024-09-18T11:30:08.883 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 9 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-18T11:30:08.883 INFO:tasks.rados.rados.0.smithi119.stdout:122: done (15 left) 2024-09-18T11:30:08.884 INFO:tasks.rados.rados.0.smithi119.stdout:125: done (14 left) 2024-09-18T11:30:08.884 INFO:tasks.rados.rados.0.smithi119.stdout:140: set_chunk oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.884 INFO:tasks.rados.rados.0.smithi119.stdout:140: set_chunk oid 41 offset: 30208 length: 2558976 target oid smithi11939506-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 30208 snap -1 2024-09-18T11:30:08.884 INFO:tasks.rados.rados.0.smithi119.stdout:141: set_chunk oid 42 target oid 42 low_tier 2024-09-18T11:30:08.884 INFO:tasks.rados.rados.0.smithi119.stdout:141: set_chunk oid 42 offset: 90624 length: 1204224 target oid smithi11939506-42 low_tier offset: 90624 snap -1 2024-09-18T11:30:08.884 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.884 INFO:tasks.rados.rados.0.smithi119.stdout:131: finishing set_chunk to oid 32 2024-09-18T11:30:08.884 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 3 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-18T11:30:08.884 INFO:tasks.rados.rados.0.smithi119.stdout:131: done (15 left) 2024-09-18T11:30:08.884 INFO:tasks.rados.rados.0.smithi119.stdout:142: set_chunk oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.884 INFO:tasks.rados.rados.0.smithi119.stdout:142: set_chunk oid 43 offset: 936960 length: 872448 target oid smithi11939506-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 936960 snap -1 2024-09-18T11:30:08.884 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.885 INFO:tasks.rados.rados.0.smithi119.stdout:128: finishing set_chunk to oid 29 2024-09-18T11:30:08.885 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 3 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-18T11:30:08.885 INFO:tasks.rados.rados.0.smithi119.stdout:128: done (15 left) 2024-09-18T11:30:08.885 INFO:tasks.rados.rados.0.smithi119.stdout:143: set_chunk oid 44 target oid 44 low_tier 2024-09-18T11:30:08.885 INFO:tasks.rados.rados.0.smithi119.stdout:143: set_chunk oid 44 offset: 172544 length: 1645568 target oid smithi11939506-44 low_tier offset: 172544 snap -1 2024-09-18T11:30:08.885 INFO:tasks.rados.rados.0.smithi119.stdout:130: finishing set_chunk to oid 31 2024-09-18T11:30:08.885 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 12 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-18T11:30:08.885 INFO:tasks.rados.rados.0.smithi119.stdout:126: finishing set_chunk to oid 27 2024-09-18T11:30:08.885 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 6 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T11:30:08.885 INFO:tasks.rados.rados.0.smithi119.stdout:127: finishing set_chunk to oid 28 2024-09-18T11:30:08.885 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 28 v 9 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-18T11:30:08.885 INFO:tasks.rados.rados.0.smithi119.stdout:126: done (15 left) 2024-09-18T11:30:08.885 INFO:tasks.rados.rados.0.smithi119.stdout:127: done (14 left) 2024-09-18T11:30:08.886 INFO:tasks.rados.rados.0.smithi119.stdout:130: done (13 left) 2024-09-18T11:30:08.886 INFO:tasks.rados.rados.0.smithi119.stdout:144: set_chunk oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.886 INFO:tasks.rados.rados.0.smithi119.stdout:144: set_chunk oid 45 offset: 907264 length: 1523712 target oid smithi11939506-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 907264 snap -1 2024-09-18T11:30:08.886 INFO:tasks.rados.rados.0.smithi119.stdout:145: set_chunk oid 46 target oid 46 low_tier 2024-09-18T11:30:08.886 INFO:tasks.rados.rados.0.smithi119.stdout:145: set_chunk oid 46 offset: 2311680 length: 395776 target oid smithi11939506-46 low_tier offset: 2311680 snap -1 2024-09-18T11:30:08.886 INFO:tasks.rados.rados.0.smithi119.stdout:146: set_chunk oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.886 INFO:tasks.rados.rados.0.smithi119.stdout:146: set_chunk oid 47 offset: 2022912 length: 53760 target oid smithi11939506-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 2022912 snap -1 2024-09-18T11:30:08.886 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.886 INFO:tasks.rados.rados.0.smithi119.stdout:129: finishing set_chunk to oid 30 2024-09-18T11:30:08.886 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 30 v 6 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-18T11:30:08.886 INFO:tasks.rados.rados.0.smithi119.stdout:129: done (15 left) 2024-09-18T11:30:08.886 INFO:tasks.rados.rados.0.smithi119.stdout:147: set_chunk oid 48 target oid 48 low_tier 2024-09-18T11:30:08.886 INFO:tasks.rados.rados.0.smithi119.stdout:147: set_chunk oid 48 offset: 399872 length: 798720 target oid smithi11939506-48 low_tier offset: 399872 snap -1 2024-09-18T11:30:08.887 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.887 INFO:tasks.rados.rados.0.smithi119.stdout:136: finishing set_chunk to oid 37 2024-09-18T11:30:08.887 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 6 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-18T11:30:08.887 INFO:tasks.rados.rados.0.smithi119.stdout:136: done (15 left) 2024-09-18T11:30:08.887 INFO:tasks.rados.rados.0.smithi119.stdout:148: set_chunk oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.887 INFO:tasks.rados.rados.0.smithi119.stdout:148: set_chunk oid 49 offset: 1097728 length: 671232 target oid smithi11939506-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1097728 snap -1 2024-09-18T11:30:08.887 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.887 INFO:tasks.rados.rados.0.smithi119.stdout:138: finishing set_chunk to oid 39 2024-09-18T11:30:08.887 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 39 v 12 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-18T11:30:08.887 INFO:tasks.rados.rados.0.smithi119.stdout:138: done (15 left) 2024-09-18T11:30:08.887 INFO:tasks.rados.rados.0.smithi119.stdout:149: set_chunk oid 50 target oid 50 low_tier 2024-09-18T11:30:08.887 INFO:tasks.rados.rados.0.smithi119.stdout:149: set_chunk oid 50 offset: 1017344 length: 836096 target oid smithi11939506-50 low_tier offset: 1017344 snap -1 2024-09-18T11:30:08.887 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.888 INFO:tasks.rados.rados.0.smithi119.stdout:133: finishing set_chunk to oid 34 2024-09-18T11:30:08.888 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 9 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T11:30:08.888 INFO:tasks.rados.rados.0.smithi119.stdout:133: done (15 left) 2024-09-18T11:30:08.888 INFO:tasks.rados.rados.0.smithi119.stdout:150: set_chunk oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.888 INFO:tasks.rados.rados.0.smithi119.stdout:150: set_chunk oid 1 offset: 927744 length: 432640 target oid smithi11939506-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 927744 snap -1 2024-09-18T11:30:08.888 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.888 INFO:tasks.rados.rados.0.smithi119.stdout:141: finishing set_chunk to oid 42 2024-09-18T11:30:08.888 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 12 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T11:30:08.888 INFO:tasks.rados.rados.0.smithi119.stdout:141: done (15 left) 2024-09-18T11:30:08.888 INFO:tasks.rados.rados.0.smithi119.stdout:151: set_chunk oid 2 target oid 2 low_tier 2024-09-18T11:30:08.888 INFO:tasks.rados.rados.0.smithi119.stdout:151: set_chunk oid 2 offset: 171008 length: 80896 target oid smithi11939506-2 low_tier offset: 171008 snap -1 2024-09-18T11:30:08.888 INFO:tasks.rados.rados.0.smithi119.stdout:139: finishing set_chunk to oid 40 2024-09-18T11:30:08.888 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 6 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-18T11:30:08.889 INFO:tasks.rados.rados.0.smithi119.stdout:135: finishing set_chunk to oid 36 2024-09-18T11:30:08.889 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 36 v 6 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-18T11:30:08.889 INFO:tasks.rados.rados.0.smithi119.stdout:135: done (15 left) 2024-09-18T11:30:08.889 INFO:tasks.rados.rados.0.smithi119.stdout:139: done (14 left) 2024-09-18T11:30:08.889 INFO:tasks.rados.rados.0.smithi119.stdout:152: set_chunk oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.889 INFO:tasks.rados.rados.0.smithi119.stdout:152: set_chunk oid 3 offset: 534016 length: 332800 target oid smithi11939506-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 534016 snap -1 2024-09-18T11:30:08.889 INFO:tasks.rados.rados.0.smithi119.stdout:153: set_chunk oid 4 target oid 4 low_tier 2024-09-18T11:30:08.889 INFO:tasks.rados.rados.0.smithi119.stdout:153: set_chunk oid 4 offset: 1669632 length: 49152 target oid smithi11939506-4 low_tier offset: 1669632 snap -1 2024-09-18T11:30:08.889 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.889 INFO:tasks.rados.rados.0.smithi119.stdout:132: finishing set_chunk to oid 33 2024-09-18T11:30:08.889 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 33 v 3 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-18T11:30:08.889 INFO:tasks.rados.rados.0.smithi119.stdout:132: done (15 left) 2024-09-18T11:30:08.889 INFO:tasks.rados.rados.0.smithi119.stdout:154: set_chunk oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.890 INFO:tasks.rados.rados.0.smithi119.stdout:154: set_chunk oid 5 offset: 1420288 length: 158720 target oid smithi11939506-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1420288 snap -1 2024-09-18T11:30:08.890 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.890 INFO:tasks.rados.rados.0.smithi119.stdout:154: finishing set_chunk to oid 5 2024-09-18T11:30:08.890 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 5 set_chunk 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1420288 length: 158720 2024-09-18T11:30:08.890 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 0 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-18T11:30:08.890 INFO:tasks.rados.rados.0.smithi119.stdout:154: done (15 left) 2024-09-18T11:30:08.890 INFO:tasks.rados.rados.0.smithi119.stdout:155: set_chunk oid 6 target oid 6 low_tier 2024-09-18T11:30:08.890 INFO:tasks.rados.rados.0.smithi119.stdout:155: set_chunk oid 6 offset: 1247232 length: 6144 target oid smithi11939506-6 low_tier offset: 1247232 snap -1 2024-09-18T11:30:08.890 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.898 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:08.890+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:08.927 INFO:tasks.rados.rados.0.smithi119.stdout:140: finishing set_chunk to oid 41 2024-09-18T11:30:08.927 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 41 v 15 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-18T11:30:08.927 INFO:tasks.rados.rados.0.smithi119.stdout:134: finishing set_chunk to oid 35 2024-09-18T11:30:08.927 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 6 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-18T11:30:08.927 INFO:tasks.rados.rados.0.smithi119.stdout:134: done (15 left) 2024-09-18T11:30:08.927 INFO:tasks.rados.rados.0.smithi119.stdout:140: done (14 left) 2024-09-18T11:30:08.927 INFO:tasks.rados.rados.0.smithi119.stdout:156: set_chunk oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.927 INFO:tasks.rados.rados.0.smithi119.stdout:156: set_chunk oid 7 offset: 1251328 length: 711168 target oid smithi11939506-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1251328 snap -1 2024-09-18T11:30:08.927 INFO:tasks.rados.rados.0.smithi119.stdout:157: set_chunk oid 8 target oid 8 low_tier 2024-09-18T11:30:08.927 INFO:tasks.rados.rados.0.smithi119.stdout:157: set_chunk oid 8 offset: 921600 length: 1021952 target oid smithi11939506-8 low_tier offset: 921600 snap -1 2024-09-18T11:30:08.927 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.928 INFO:tasks.rados.rados.0.smithi119.stdout:157: finishing set_chunk to oid 8 2024-09-18T11:30:08.928 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 8 set_chunk 8 low_tier returned error code -95 offset: 921600 length: 1021952 2024-09-18T11:30:08.928 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 8 v 0 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-18T11:30:08.928 INFO:tasks.rados.rados.0.smithi119.stdout:157: done (15 left) 2024-09-18T11:30:08.928 INFO:tasks.rados.rados.0.smithi119.stdout:158: set_chunk oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.928 INFO:tasks.rados.rados.0.smithi119.stdout:158: set_chunk oid 9 offset: 952320 length: 857600 target oid smithi11939506-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 952320 snap -1 2024-09-18T11:30:08.928 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.928 INFO:tasks.rados.rados.0.smithi119.stdout:156: finishing set_chunk to oid 7 2024-09-18T11:30:08.928 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 7 set_chunk 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1251328 length: 711168 2024-09-18T11:30:08.928 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 7 v 0 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-18T11:30:08.928 INFO:tasks.rados.rados.0.smithi119.stdout:156: done (15 left) 2024-09-18T11:30:08.929 INFO:tasks.rados.rados.0.smithi119.stdout:159: set_chunk oid 10 target oid 10 low_tier 2024-09-18T11:30:08.929 INFO:tasks.rados.rados.0.smithi119.stdout:159: set_chunk oid 10 offset: 108544 length: 577536 target oid smithi11939506-10 low_tier offset: 108544 snap -1 2024-09-18T11:30:08.929 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.929 INFO:tasks.rados.rados.0.smithi119.stdout:158: finishing set_chunk to oid 9 2024-09-18T11:30:08.929 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 9 set_chunk 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 952320 length: 857600 2024-09-18T11:30:08.929 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 0 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T11:30:08.929 INFO:tasks.rados.rados.0.smithi119.stdout:158: done (15 left) 2024-09-18T11:30:08.929 INFO:tasks.rados.rados.0.smithi119.stdout:160: set_chunk oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.929 INFO:tasks.rados.rados.0.smithi119.stdout:160: set_chunk oid 11 offset: 712704 length: 126464 target oid smithi11939506-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 712704 snap -1 2024-09-18T11:30:08.929 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.929 INFO:tasks.rados.rados.0.smithi119.stdout:160: finishing set_chunk to oid 11 2024-09-18T11:30:08.929 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 11 set_chunk 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 712704 length: 126464 2024-09-18T11:30:08.930 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 11 v 0 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-18T11:30:08.930 INFO:tasks.rados.rados.0.smithi119.stdout:160: done (15 left) 2024-09-18T11:30:08.930 INFO:tasks.rados.rados.0.smithi119.stdout:161: set_chunk oid 12 target oid 12 low_tier 2024-09-18T11:30:08.930 INFO:tasks.rados.rados.0.smithi119.stdout:161: set_chunk oid 12 offset: 1125376 length: 65536 target oid smithi11939506-12 low_tier offset: 1125376 snap -1 2024-09-18T11:30:08.930 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.930 INFO:tasks.rados.rados.0.smithi119.stdout:146: finishing set_chunk to oid 47 2024-09-18T11:30:08.930 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 9 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-18T11:30:08.930 INFO:tasks.rados.rados.0.smithi119.stdout:146: done (15 left) 2024-09-18T11:30:08.930 INFO:tasks.rados.rados.0.smithi119.stdout:162: set_chunk oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.930 INFO:tasks.rados.rados.0.smithi119.stdout:162: set_chunk oid 13 offset: 885760 length: 257024 target oid smithi11939506-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 885760 snap -1 2024-09-18T11:30:08.930 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.930 INFO:tasks.rados.rados.0.smithi119.stdout:162: finishing set_chunk to oid 13 2024-09-18T11:30:08.931 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 13 set_chunk 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 885760 length: 257024 2024-09-18T11:30:08.931 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 13 v 0 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-18T11:30:08.931 INFO:tasks.rados.rados.0.smithi119.stdout:162: done (15 left) 2024-09-18T11:30:08.931 INFO:tasks.rados.rados.0.smithi119.stdout:163: set_chunk oid 14 target oid 14 low_tier 2024-09-18T11:30:08.931 INFO:tasks.rados.rados.0.smithi119.stdout:163: set_chunk oid 14 offset: 873984 length: 1040384 target oid smithi11939506-14 low_tier offset: 873984 snap -1 2024-09-18T11:30:08.931 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.931 INFO:tasks.rados.rados.0.smithi119.stdout:137: finishing set_chunk to oid 38 2024-09-18T11:30:08.931 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 9 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-18T11:30:08.931 INFO:tasks.rados.rados.0.smithi119.stdout:137: done (15 left) 2024-09-18T11:30:08.931 INFO:tasks.rados.rados.0.smithi119.stdout:164: set_chunk oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.931 INFO:tasks.rados.rados.0.smithi119.stdout:163: finishing set_chunk to oid 14 2024-09-18T11:30:08.931 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 14 set_chunk 14 low_tier returned error code -95 offset: 873984 length: 1040384 2024-09-18T11:30:08.931 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 0 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-18T11:30:08.932 INFO:tasks.rados.rados.0.smithi119.stdout:164: set_chunk oid 15 offset: 703488 length: 577536 target oid smithi11939506-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 703488 snap -1 2024-09-18T11:30:08.932 INFO:tasks.rados.rados.0.smithi119.stdout:142: finishing set_chunk to oid 43 2024-09-18T11:30:08.932 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 43 v 15 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-18T11:30:08.932 INFO:tasks.rados.rados.0.smithi119.stdout:142: done (15 left) 2024-09-18T11:30:08.932 INFO:tasks.rados.rados.0.smithi119.stdout:163: done (14 left) 2024-09-18T11:30:08.932 INFO:tasks.rados.rados.0.smithi119.stdout:165: set_chunk oid 16 target oid 16 low_tier 2024-09-18T11:30:08.932 INFO:tasks.rados.rados.0.smithi119.stdout:165: set_chunk oid 16 offset: 894976 length: 637440 target oid smithi11939506-16 low_tier offset: 894976 snap -1 2024-09-18T11:30:08.932 INFO:tasks.rados.rados.0.smithi119.stdout:166: set_chunk oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.932 INFO:tasks.rados.rados.0.smithi119.stdout:166: set_chunk oid 17 offset: 655872 length: 494080 target oid smithi11939506-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 655872 snap -1 2024-09-18T11:30:08.932 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.932 INFO:tasks.rados.rados.0.smithi119.stdout:164: finishing set_chunk to oid 15 2024-09-18T11:30:08.932 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 15 set_chunk 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 703488 length: 577536 2024-09-18T11:30:08.933 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 0 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-18T11:30:08.933 INFO:tasks.rados.rados.0.smithi119.stdout:164: done (15 left) 2024-09-18T11:30:08.933 INFO:tasks.rados.rados.0.smithi119.stdout:167: set_chunk oid 18 target oid 18 low_tier 2024-09-18T11:30:08.933 INFO:tasks.rados.rados.0.smithi119.stdout:167: set_chunk oid 18 offset: 889856 length: 931328 target oid smithi11939506-18 low_tier offset: 889856 snap -1 2024-09-18T11:30:08.933 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.933 INFO:tasks.rados.rados.0.smithi119.stdout:150: finishing set_chunk to oid 1 2024-09-18T11:30:08.933 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-18T11:30:08.933 INFO:tasks.rados.rados.0.smithi119.stdout:150: done (15 left) 2024-09-18T11:30:08.933 INFO:tasks.rados.rados.0.smithi119.stdout:168: set_chunk oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.933 INFO:tasks.rados.rados.0.smithi119.stdout:168: set_chunk oid 19 offset: 1028096 length: 1049088 target oid smithi11939506-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1028096 snap -1 2024-09-18T11:30:08.933 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.934 INFO:tasks.rados.rados.0.smithi119.stdout:165: finishing set_chunk to oid 16 2024-09-18T11:30:08.934 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 16 set_chunk 16 low_tier returned error code -95 offset: 894976 length: 637440 2024-09-18T11:30:08.934 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 0 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-18T11:30:08.934 INFO:tasks.rados.rados.0.smithi119.stdout:167: finishing set_chunk to oid 18 2024-09-18T11:30:08.934 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 18 set_chunk 18 low_tier returned error code -95 offset: 889856 length: 931328 2024-09-18T11:30:08.934 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 0 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-18T11:30:08.934 INFO:tasks.rados.rados.0.smithi119.stdout:144: finishing set_chunk to oid 45 2024-09-18T11:30:08.934 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 3 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-18T11:30:08.934 INFO:tasks.rados.rados.0.smithi119.stdout:144: done (15 left) 2024-09-18T11:30:08.934 INFO:tasks.rados.rados.0.smithi119.stdout:165: done (14 left) 2024-09-18T11:30:08.934 INFO:tasks.rados.rados.0.smithi119.stdout:167: done (13 left) 2024-09-18T11:30:08.934 INFO:tasks.rados.rados.0.smithi119.stdout:169: set_chunk oid 20 target oid 20 low_tier 2024-09-18T11:30:08.934 INFO:tasks.rados.rados.0.smithi119.stdout:169: set_chunk oid 20 offset: 202240 length: 1391616 target oid smithi11939506-20 low_tier offset: 202240 snap -1 2024-09-18T11:30:08.935 INFO:tasks.rados.rados.0.smithi119.stdout:170: set_chunk oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.935 INFO:tasks.rados.rados.0.smithi119.stdout:170: set_chunk oid 21 offset: 847872 length: 87040 target oid smithi11939506-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 847872 snap -1 2024-09-18T11:30:08.935 INFO:tasks.rados.rados.0.smithi119.stdout:171: set_chunk oid 22 target oid 22 low_tier 2024-09-18T11:30:08.935 INFO:tasks.rados.rados.0.smithi119.stdout:171: set_chunk oid 22 offset: 239616 length: 523776 target oid smithi11939506-22 low_tier offset: 239616 snap -1 2024-09-18T11:30:08.935 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.935 INFO:tasks.rados.rados.0.smithi119.stdout:168: finishing set_chunk to oid 19 2024-09-18T11:30:08.935 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 19 set_chunk 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1028096 length: 1049088 2024-09-18T11:30:08.935 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 0 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-18T11:30:08.935 INFO:tasks.rados.rados.0.smithi119.stdout:168: done (15 left) 2024-09-18T11:30:08.935 INFO:tasks.rados.rados.0.smithi119.stdout:172: set_chunk oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.935 INFO:tasks.rados.rados.0.smithi119.stdout:172: set_chunk oid 23 offset: 215552 length: 1645056 target oid smithi11939506-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 215552 snap -1 2024-09-18T11:30:08.935 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.936 INFO:tasks.rados.rados.0.smithi119.stdout:169: finishing set_chunk to oid 20 2024-09-18T11:30:08.936 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 20 set_chunk 20 low_tier returned error code -95 offset: 202240 length: 1391616 2024-09-18T11:30:08.936 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 20 v 0 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-18T11:30:08.936 INFO:tasks.rados.rados.0.smithi119.stdout:169: done (15 left) 2024-09-18T11:30:08.936 INFO:tasks.rados.rados.0.smithi119.stdout:173: set_chunk oid 24 target oid 24 low_tier 2024-09-18T11:30:08.936 INFO:tasks.rados.rados.0.smithi119.stdout:173: set_chunk oid 24 offset: 375296 length: 134656 target oid smithi11939506-24 low_tier offset: 375296 snap -1 2024-09-18T11:30:08.936 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.936 INFO:tasks.rados.rados.0.smithi119.stdout:166: finishing set_chunk to oid 17 2024-09-18T11:30:08.936 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 17 set_chunk 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 655872 length: 494080 2024-09-18T11:30:08.936 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 0 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-18T11:30:08.936 INFO:tasks.rados.rados.0.smithi119.stdout:172: finishing set_chunk to oid 23 2024-09-18T11:30:08.937 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 23 set_chunk 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 215552 length: 1645056 2024-09-18T11:30:08.937 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 0 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-18T11:30:08.937 INFO:tasks.rados.rados.0.smithi119.stdout:166: done (15 left) 2024-09-18T11:30:08.937 INFO:tasks.rados.rados.0.smithi119.stdout:172: done (14 left) 2024-09-18T11:30:08.937 INFO:tasks.rados.rados.0.smithi119.stdout:174: set_chunk oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.937 INFO:tasks.rados.rados.0.smithi119.stdout:174: set_chunk oid 25 offset: 477184 length: 1776640 target oid smithi11939506-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 477184 snap -1 2024-09-18T11:30:08.937 INFO:tasks.rados.rados.0.smithi119.stdout:175: set_chunk oid 26 target oid 26 low_tier 2024-09-18T11:30:08.937 INFO:tasks.rados.rados.0.smithi119.stdout:170: finishing set_chunk to oid 21 2024-09-18T11:30:08.937 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 21 set_chunk 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 847872 length: 87040 2024-09-18T11:30:08.937 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 0 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-18T11:30:08.937 INFO:tasks.rados.rados.0.smithi119.stdout:175: set_chunk oid 26 offset: 354816 length: 949760 target oid smithi11939506-26 low_tier offset: 354816 snap -1 2024-09-18T11:30:08.937 INFO:tasks.rados.rados.0.smithi119.stdout:171: finishing set_chunk to oid 22 2024-09-18T11:30:08.938 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 22 set_chunk 22 low_tier returned error code -95 offset: 239616 length: 523776 2024-09-18T11:30:08.938 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 0 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-18T11:30:08.938 INFO:tasks.rados.rados.0.smithi119.stdout:170: done (15 left) 2024-09-18T11:30:08.938 INFO:tasks.rados.rados.0.smithi119.stdout:171: done (14 left) 2024-09-18T11:30:08.938 INFO:tasks.rados.rados.0.smithi119.stdout:176: set_chunk oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.938 INFO:tasks.rados.rados.0.smithi119.stdout:176: set_chunk oid 27 offset: 1404416 length: 796160 target oid smithi11939506-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1404416 snap -1 2024-09-18T11:30:08.938 INFO:tasks.rados.rados.0.smithi119.stdout:177: set_chunk oid 28 target oid 28 low_tier 2024-09-18T11:30:08.938 INFO:tasks.rados.rados.0.smithi119.stdout:177: set_chunk oid 28 offset: 1733120 length: 189952 target oid smithi11939506-28 low_tier offset: 1733120 snap -1 2024-09-18T11:30:08.938 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.938 INFO:tasks.rados.rados.0.smithi119.stdout:174: finishing set_chunk to oid 25 2024-09-18T11:30:08.939 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 25 set_chunk 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 477184 length: 1776640 2024-09-18T11:30:08.939 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 0 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-18T11:30:08.939 INFO:tasks.rados.rados.0.smithi119.stdout:174: done (15 left) 2024-09-18T11:30:08.939 INFO:tasks.rados.rados.0.smithi119.stdout:178: set_chunk oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.939 INFO:tasks.rados.rados.0.smithi119.stdout:178: set_chunk oid 29 offset: 564224 length: 1053696 target oid smithi11939506-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 564224 snap -1 2024-09-18T11:30:08.939 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.939 INFO:tasks.rados.rados.0.smithi119.stdout:175: finishing set_chunk to oid 26 2024-09-18T11:30:08.939 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 26 set_chunk 26 low_tier returned error code -95 offset: 354816 length: 949760 2024-09-18T11:30:08.939 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 0 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-18T11:30:08.939 INFO:tasks.rados.rados.0.smithi119.stdout:176: finishing set_chunk to oid 27 2024-09-18T11:30:08.939 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 27 set_chunk 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1404416 length: 796160 2024-09-18T11:30:08.940 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 0 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T11:30:08.940 INFO:tasks.rados.rados.0.smithi119.stdout:175: done (15 left) 2024-09-18T11:30:08.940 INFO:tasks.rados.rados.0.smithi119.stdout:176: done (14 left) 2024-09-18T11:30:08.940 INFO:tasks.rados.rados.0.smithi119.stdout:179: set_chunk oid 30 target oid 30 low_tier 2024-09-18T11:30:08.940 INFO:tasks.rados.rados.0.smithi119.stdout:177: finishing set_chunk to oid 28 2024-09-18T11:30:08.940 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 28 set_chunk 28 low_tier returned error code -95 offset: 1733120 length: 189952 2024-09-18T11:30:08.940 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 28 v 0 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-18T11:30:08.940 INFO:tasks.rados.rados.0.smithi119.stdout:179: set_chunk oid 30 offset: 289280 length: 732672 target oid smithi11939506-30 low_tier offset: 289280 snap -1 2024-09-18T11:30:08.940 INFO:tasks.rados.rados.0.smithi119.stdout:177: done (14 left) 2024-09-18T11:30:08.940 INFO:tasks.rados.rados.0.smithi119.stdout:180: set_chunk oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.941 INFO:tasks.rados.rados.0.smithi119.stdout:180: set_chunk oid 31 offset: 244736 length: 233472 target oid smithi11939506-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 244736 snap -1 2024-09-18T11:30:08.941 INFO:tasks.rados.rados.0.smithi119.stdout:181: set_chunk oid 32 target oid 32 low_tier 2024-09-18T11:30:08.941 INFO:tasks.rados.rados.0.smithi119.stdout:181: set_chunk oid 32 offset: 1251840 length: 675328 target oid smithi11939506-32 low_tier offset: 1251840 snap -1 2024-09-18T11:30:08.941 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.941 INFO:tasks.rados.rados.0.smithi119.stdout:145: finishing set_chunk to oid 46 2024-09-18T11:30:08.941 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 9 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-18T11:30:08.941 INFO:tasks.rados.rados.0.smithi119.stdout:147: finishing set_chunk to oid 48 2024-09-18T11:30:08.941 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 48 v 12 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-18T11:30:08.941 INFO:tasks.rados.rados.0.smithi119.stdout:179: finishing set_chunk to oid 30 2024-09-18T11:30:08.941 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 30 set_chunk 30 low_tier returned error code -95 offset: 289280 length: 732672 2024-09-18T11:30:08.941 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 30 v 0 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-18T11:30:08.941 INFO:tasks.rados.rados.0.smithi119.stdout:180: finishing set_chunk to oid 31 2024-09-18T11:30:08.942 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 31 set_chunk 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 244736 length: 233472 2024-09-18T11:30:08.942 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 0 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-18T11:30:08.942 INFO:tasks.rados.rados.0.smithi119.stdout:181: finishing set_chunk to oid 32 2024-09-18T11:30:08.942 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 32 set_chunk 32 low_tier returned error code -95 offset: 1251840 length: 675328 2024-09-18T11:30:08.942 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 0 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-18T11:30:08.942 INFO:tasks.rados.rados.0.smithi119.stdout:145: done (15 left) 2024-09-18T11:30:08.942 INFO:tasks.rados.rados.0.smithi119.stdout:147: done (14 left) 2024-09-18T11:30:08.942 INFO:tasks.rados.rados.0.smithi119.stdout:179: done (13 left) 2024-09-18T11:30:08.942 INFO:tasks.rados.rados.0.smithi119.stdout:180: done (12 left) 2024-09-18T11:30:08.942 INFO:tasks.rados.rados.0.smithi119.stdout:181: done (11 left) 2024-09-18T11:30:08.942 INFO:tasks.rados.rados.0.smithi119.stdout:182: set_chunk oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.943 INFO:tasks.rados.rados.0.smithi119.stdout:182: set_chunk oid 33 offset: 463872 length: 726016 target oid smithi11939506-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 463872 snap -1 2024-09-18T11:30:08.943 INFO:tasks.rados.rados.0.smithi119.stdout:183: set_chunk oid 34 target oid 34 low_tier 2024-09-18T11:30:08.943 INFO:tasks.rados.rados.0.smithi119.stdout:183: set_chunk oid 34 offset: 288256 length: 629248 target oid smithi11939506-34 low_tier offset: 288256 snap -1 2024-09-18T11:30:08.943 INFO:tasks.rados.rados.0.smithi119.stdout:184: set_chunk oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.943 INFO:tasks.rados.rados.0.smithi119.stdout:184: set_chunk oid 35 offset: 7680 length: 247296 target oid smithi11939506-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 7680 snap -1 2024-09-18T11:30:08.943 INFO:tasks.rados.rados.0.smithi119.stdout:185: set_chunk oid 36 target oid 36 low_tier 2024-09-18T11:30:08.943 INFO:tasks.rados.rados.0.smithi119.stdout:185: set_chunk oid 36 offset: 639488 length: 237056 target oid smithi11939506-36 low_tier offset: 639488 snap -1 2024-09-18T11:30:08.943 INFO:tasks.rados.rados.0.smithi119.stdout:186: set_chunk oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.943 INFO:tasks.rados.rados.0.smithi119.stdout:186: set_chunk oid 37 offset: 1008128 length: 534528 target oid smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1008128 snap -1 2024-09-18T11:30:08.943 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.943 INFO:tasks.rados.rados.0.smithi119.stdout:149: finishing set_chunk to oid 50 2024-09-18T11:30:08.944 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 9 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-18T11:30:08.944 INFO:tasks.rados.rados.0.smithi119.stdout:149: done (15 left) 2024-09-18T11:30:08.944 INFO:tasks.rados.rados.0.smithi119.stdout:187: set_chunk oid 38 target oid 38 low_tier 2024-09-18T11:30:08.944 INFO:tasks.rados.rados.0.smithi119.stdout:187: set_chunk oid 38 offset: 2370560 length: 46080 target oid smithi11939506-38 low_tier offset: 2370560 snap -1 2024-09-18T11:30:08.944 INFO:tasks.rados.rados.0.smithi119.stdout:183: finishing set_chunk to oid 34 2024-09-18T11:30:08.944 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 34 set_chunk 34 low_tier returned error code -95 offset: 288256 length: 629248 2024-09-18T11:30:08.944 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 0 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T11:30:08.944 INFO:tasks.rados.rados.0.smithi119.stdout:186: finishing set_chunk to oid 37 2024-09-18T11:30:08.944 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 37 set_chunk 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1008128 length: 534528 2024-09-18T11:30:08.944 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 0 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-18T11:30:08.944 INFO:tasks.rados.rados.0.smithi119.stdout:183: done (15 left) 2024-09-18T11:30:08.945 INFO:tasks.rados.rados.0.smithi119.stdout:186: done (14 left) 2024-09-18T11:30:08.945 INFO:tasks.rados.rados.0.smithi119.stdout:188: set_chunk oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.945 INFO:tasks.rados.rados.0.smithi119.stdout:188: set_chunk oid 39 offset: 847360 length: 1520640 target oid smithi11939506-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 847360 snap -1 2024-09-18T11:30:08.945 INFO:tasks.rados.rados.0.smithi119.stdout:189: set_chunk oid 40 target oid 40 low_tier 2024-09-18T11:30:08.945 INFO:tasks.rados.rados.0.smithi119.stdout:189: set_chunk oid 40 offset: 795136 length: 583680 target oid smithi11939506-40 low_tier offset: 795136 snap -1 2024-09-18T11:30:08.945 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.945 INFO:tasks.rados.rados.0.smithi119.stdout:188: finishing set_chunk to oid 39 2024-09-18T11:30:08.945 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 39 set_chunk 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 847360 length: 1520640 2024-09-18T11:30:08.945 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 39 v 0 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-18T11:30:08.945 INFO:tasks.rados.rados.0.smithi119.stdout:187: finishing set_chunk to oid 38 2024-09-18T11:30:08.945 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 38 set_chunk 38 low_tier returned error code -95 offset: 2370560 length: 46080 2024-09-18T11:30:08.946 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 0 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-18T11:30:08.946 INFO:tasks.rados.rados.0.smithi119.stdout:189: finishing set_chunk to oid 40 2024-09-18T11:30:08.946 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 40 set_chunk 40 low_tier returned error code -95 offset: 795136 length: 583680 2024-09-18T11:30:08.946 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 0 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-18T11:30:08.946 INFO:tasks.rados.rados.0.smithi119.stdout:187: done (15 left) 2024-09-18T11:30:08.946 INFO:tasks.rados.rados.0.smithi119.stdout:188: done (14 left) 2024-09-18T11:30:08.946 INFO:tasks.rados.rados.0.smithi119.stdout:189: done (13 left) 2024-09-18T11:30:08.946 INFO:tasks.rados.rados.0.smithi119.stdout:190: set_chunk oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.946 INFO:tasks.rados.rados.0.smithi119.stdout:190: set_chunk oid 41 offset: 2044416 length: 442880 target oid smithi11939506-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 2044416 snap -1 2024-09-18T11:30:08.946 INFO:tasks.rados.rados.0.smithi119.stdout:191: set_chunk oid 42 target oid 42 low_tier 2024-09-18T11:30:08.946 INFO:tasks.rados.rados.0.smithi119.stdout:191: set_chunk oid 42 offset: 997888 length: 1324544 target oid smithi11939506-42 low_tier offset: 997888 snap -1 2024-09-18T11:30:08.946 INFO:tasks.rados.rados.0.smithi119.stdout:192: set_chunk oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.947 INFO:tasks.rados.rados.0.smithi119.stdout:192: set_chunk oid 43 offset: 867328 length: 133120 target oid smithi11939506-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 867328 snap -1 2024-09-18T11:30:08.947 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.947 INFO:tasks.rados.rados.0.smithi119.stdout:152: finishing set_chunk to oid 3 2024-09-18T11:30:08.947 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 3 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-18T11:30:08.947 INFO:tasks.rados.rados.0.smithi119.stdout:152: done (15 left) 2024-09-18T11:30:08.947 INFO:tasks.rados.rados.0.smithi119.stdout:192 retry set_chunk ! 2024-09-18T11:30:08.947 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 15 2024-09-18T11:30:08.947 INFO:tasks.rados.rados.0.smithi119.stdout:191: finishing set_chunk to oid 42 2024-09-18T11:30:08.947 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 42 set_chunk 42 low_tier returned error code -95 offset: 997888 length: 1324544 2024-09-18T11:30:08.947 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 0 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T11:30:08.947 INFO:tasks.rados.rados.0.smithi119.stdout:190: finishing set_chunk to oid 41 2024-09-18T11:30:08.947 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 41 set_chunk 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 2044416 length: 442880 2024-09-18T11:30:08.947 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 41 v 0 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-18T11:30:08.948 INFO:tasks.rados.rados.0.smithi119.stdout:190: done (14 left) 2024-09-18T11:30:08.948 INFO:tasks.rados.rados.0.smithi119.stdout:191: done (13 left) 2024-09-18T11:30:08.948 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 13 2024-09-18T11:30:08.948 INFO:tasks.rados.rados.0.smithi119.stdout:192: finishing set_chunk to oid 43 2024-09-18T11:30:08.948 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 43 set_chunk 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 867328 length: 133120 2024-09-18T11:30:08.948 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-18T11:30:08.948 INFO:tasks.rados.rados.0.smithi119.stdout:192: done (12 left) 2024-09-18T11:30:08.948 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 12 2024-09-18T11:30:08.948 INFO:tasks.rados.rados.0.smithi119.stdout:143: finishing set_chunk to oid 44 2024-09-18T11:30:08.948 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 44 v 9 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-18T11:30:08.948 INFO:tasks.rados.rados.0.smithi119.stdout:143: done (11 left) 2024-09-18T11:30:08.948 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 11 2024-09-18T11:30:08.949 INFO:tasks.rados.rados.0.smithi119.stdout:155: finishing set_chunk to oid 6 2024-09-18T11:30:08.949 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 6 v 3 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-18T11:30:08.949 INFO:tasks.rados.rados.0.smithi119.stdout:155: done (10 left) 2024-09-18T11:30:08.949 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 10 2024-09-18T11:30:08.949 INFO:tasks.rados.rados.0.smithi119.stdout:153: finishing set_chunk to oid 4 2024-09-18T11:30:08.949 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 4 v 3 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-18T11:30:08.949 INFO:tasks.rados.rados.0.smithi119.stdout:153: done (9 left) 2024-09-18T11:30:08.949 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 9 2024-09-18T11:30:08.949 INFO:tasks.rados.rados.0.smithi119.stdout:159: finishing set_chunk to oid 10 2024-09-18T11:30:08.949 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 10 v 9 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-18T11:30:08.949 INFO:tasks.rados.rados.0.smithi119.stdout:159: done (8 left) 2024-09-18T11:30:08.949 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 8 2024-09-18T11:30:08.949 INFO:tasks.rados.rados.0.smithi119.stdout:161: finishing set_chunk to oid 12 2024-09-18T11:30:08.950 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 12 v 6 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-18T11:30:08.950 INFO:tasks.rados.rados.0.smithi119.stdout:161: done (7 left) 2024-09-18T11:30:08.950 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 7 2024-09-18T11:30:08.950 INFO:tasks.rados.rados.0.smithi119.stdout:151: finishing set_chunk to oid 2 2024-09-18T11:30:08.950 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 3 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-18T11:30:08.950 INFO:tasks.rados.rados.0.smithi119.stdout:148: finishing set_chunk to oid 49 2024-09-18T11:30:08.950 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 18 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-18T11:30:08.950 INFO:tasks.rados.rados.0.smithi119.stdout:148: done (6 left) 2024-09-18T11:30:08.950 INFO:tasks.rados.rados.0.smithi119.stdout:151: done (5 left) 2024-09-18T11:30:08.950 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 5 2024-09-18T11:30:08.950 INFO:tasks.rados.rados.0.smithi119.stdout:184: finishing set_chunk to oid 35 2024-09-18T11:30:08.950 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 6 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-18T11:30:08.950 INFO:tasks.rados.rados.0.smithi119.stdout:184: done (4 left) 2024-09-18T11:30:08.951 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 4 2024-09-18T11:30:08.951 INFO:tasks.rados.rados.0.smithi119.stdout:173: finishing set_chunk to oid 24 2024-09-18T11:30:08.951 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 9 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-18T11:30:08.951 INFO:tasks.rados.rados.0.smithi119.stdout:173: done (3 left) 2024-09-18T11:30:08.951 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 3 2024-09-18T11:30:08.951 INFO:tasks.rados.rados.0.smithi119.stdout:185: finishing set_chunk to oid 36 2024-09-18T11:30:08.951 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 36 v 6 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-18T11:30:08.951 INFO:tasks.rados.rados.0.smithi119.stdout:185: done (2 left) 2024-09-18T11:30:08.951 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 2 2024-09-18T11:30:08.951 INFO:tasks.rados.rados.0.smithi119.stdout:178: finishing set_chunk to oid 29 2024-09-18T11:30:08.951 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 3 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-18T11:30:08.951 INFO:tasks.rados.rados.0.smithi119.stdout:178: done (1 left) 2024-09-18T11:30:08.952 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 1 2024-09-18T11:30:08.952 INFO:tasks.rados.rados.0.smithi119.stdout:182: finishing set_chunk to oid 33 2024-09-18T11:30:08.952 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 33 v 3 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-18T11:30:08.952 INFO:tasks.rados.rados.0.smithi119.stdout:182: done (0 left) 2024-09-18T11:30:08.952 INFO:tasks.rados.rados.0.smithi119.stdout:193: set_chunk oid 44 target oid 44 low_tier 2024-09-18T11:30:08.952 INFO:tasks.rados.rados.0.smithi119.stdout:193: set_chunk oid 44 offset: 2027520 length: 541184 target oid smithi11939506-44 low_tier offset: 2027520 snap -1 2024-09-18T11:30:08.952 INFO:tasks.rados.rados.0.smithi119.stdout:194: set_chunk oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.952 INFO:tasks.rados.rados.0.smithi119.stdout:194: set_chunk oid 45 offset: 867840 length: 601088 target oid smithi11939506-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 867840 snap -1 2024-09-18T11:30:08.952 INFO:tasks.rados.rados.0.smithi119.stdout:195: set_chunk oid 46 target oid 46 low_tier 2024-09-18T11:30:08.952 INFO:tasks.rados.rados.0.smithi119.stdout:195: set_chunk oid 46 offset: 655360 length: 191488 target oid smithi11939506-46 low_tier offset: 655360 snap -1 2024-09-18T11:30:08.952 INFO:tasks.rados.rados.0.smithi119.stdout:196: set_chunk oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.952 INFO:tasks.rados.rados.0.smithi119.stdout:196: set_chunk oid 47 offset: 169472 length: 493568 target oid smithi11939506-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 169472 snap -1 2024-09-18T11:30:08.953 INFO:tasks.rados.rados.0.smithi119.stdout:197: set_chunk oid 48 target oid 48 low_tier 2024-09-18T11:30:08.953 INFO:tasks.rados.rados.0.smithi119.stdout:197: set_chunk oid 48 offset: 474624 length: 1957888 target oid smithi11939506-48 low_tier offset: 474624 snap -1 2024-09-18T11:30:08.953 INFO:tasks.rados.rados.0.smithi119.stdout:198: set_chunk oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.953 INFO:tasks.rados.rados.0.smithi119.stdout:198: set_chunk oid 49 offset: 454656 length: 377344 target oid smithi11939506-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 454656 snap -1 2024-09-18T11:30:08.953 INFO:tasks.rados.rados.0.smithi119.stdout:199: set_chunk oid 50 target oid 50 low_tier 2024-09-18T11:30:08.953 INFO:tasks.rados.rados.0.smithi119.stdout:199: set_chunk oid 50 offset: 2354688 length: 158720 target oid smithi11939506-50 low_tier offset: 2354688 snap -1 2024-09-18T11:30:08.953 INFO:tasks.rados.rados.0.smithi119.stdout:200: set_chunk oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.953 INFO:tasks.rados.rados.0.smithi119.stdout:200: set_chunk oid 1 offset: 1677312 length: 272384 target oid smithi11939506-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1677312 snap -1 2024-09-18T11:30:08.953 INFO:tasks.rados.rados.0.smithi119.stdout:201: set_chunk oid 2 target oid 2 low_tier 2024-09-18T11:30:08.953 INFO:tasks.rados.rados.0.smithi119.stdout:201: set_chunk oid 2 offset: 60928 length: 758784 target oid smithi11939506-2 low_tier offset: 60928 snap -1 2024-09-18T11:30:08.953 INFO:tasks.rados.rados.0.smithi119.stdout:202: set_chunk oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.954 INFO:tasks.rados.rados.0.smithi119.stdout:202: set_chunk oid 3 offset: 26624 length: 590336 target oid smithi11939506-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 26624 snap -1 2024-09-18T11:30:08.954 INFO:tasks.rados.rados.0.smithi119.stdout:203: set_chunk oid 4 target oid 4 low_tier 2024-09-18T11:30:08.954 INFO:tasks.rados.rados.0.smithi119.stdout:203: set_chunk oid 4 offset: 1461248 length: 309760 target oid smithi11939506-4 low_tier offset: 1461248 snap -1 2024-09-18T11:30:08.954 INFO:tasks.rados.rados.0.smithi119.stdout:204: set_chunk oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.954 INFO:tasks.rados.rados.0.smithi119.stdout:204: set_chunk oid 5 offset: 1335808 length: 476160 target oid smithi11939506-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1335808 snap -1 2024-09-18T11:30:08.954 INFO:tasks.rados.rados.0.smithi119.stdout:205: set_chunk oid 6 target oid 6 low_tier 2024-09-18T11:30:08.954 INFO:tasks.rados.rados.0.smithi119.stdout:205: set_chunk oid 6 offset: 227840 length: 1695232 target oid smithi11939506-6 low_tier offset: 227840 snap -1 2024-09-18T11:30:08.954 INFO:tasks.rados.rados.0.smithi119.stdout:206: set_chunk oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.954 INFO:tasks.rados.rados.0.smithi119.stdout:206: set_chunk oid 7 offset: 1163776 length: 592896 target oid smithi11939506-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1163776 snap -1 2024-09-18T11:30:08.954 INFO:tasks.rados.rados.0.smithi119.stdout:207: set_chunk oid 8 target oid 8 low_tier 2024-09-18T11:30:08.954 INFO:tasks.rados.rados.0.smithi119.stdout:207: set_chunk oid 8 offset: 289280 length: 1492992 target oid smithi11939506-8 low_tier offset: 289280 snap -1 2024-09-18T11:30:08.954 INFO:tasks.rados.rados.0.smithi119.stdout:208: set_chunk oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.955 INFO:tasks.rados.rados.0.smithi119.stdout:208: set_chunk oid 9 offset: 441856 length: 494080 target oid smithi11939506-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 441856 snap -1 2024-09-18T11:30:08.955 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.955 INFO:tasks.rados.rados.0.smithi119.stdout:194: finishing set_chunk to oid 45 2024-09-18T11:30:08.955 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 45 set_chunk 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 867840 length: 601088 2024-09-18T11:30:08.955 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 0 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-18T11:30:08.955 INFO:tasks.rados.rados.0.smithi119.stdout:203: finishing set_chunk to oid 4 2024-09-18T11:30:08.955 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 4 set_chunk 4 low_tier returned error code -95 offset: 1461248 length: 309760 2024-09-18T11:30:08.955 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 4 v 0 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-18T11:30:08.955 INFO:tasks.rados.rados.0.smithi119.stdout:194: done (15 left) 2024-09-18T11:30:08.955 INFO:tasks.rados.rados.0.smithi119.stdout:203: done (14 left) 2024-09-18T11:30:08.955 INFO:tasks.rados.rados.0.smithi119.stdout:209: set_chunk oid 10 target oid 10 low_tier 2024-09-18T11:30:08.955 INFO:tasks.rados.rados.0.smithi119.stdout:209: set_chunk oid 10 offset: 490496 length: 1450496 target oid smithi11939506-10 low_tier offset: 490496 snap -1 2024-09-18T11:30:08.955 INFO:tasks.rados.rados.0.smithi119.stdout:210: set_chunk oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.956 INFO:tasks.rados.rados.0.smithi119.stdout:210: set_chunk oid 11 offset: 377344 length: 735744 target oid smithi11939506-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 377344 snap -1 2024-09-18T11:30:08.956 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.956 INFO:tasks.rados.rados.0.smithi119.stdout:197: finishing set_chunk to oid 48 2024-09-18T11:30:08.956 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 48 set_chunk 48 low_tier returned error code -95 offset: 474624 length: 1957888 2024-09-18T11:30:08.956 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 48 v 0 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-18T11:30:08.956 INFO:tasks.rados.rados.0.smithi119.stdout:205: finishing set_chunk to oid 6 2024-09-18T11:30:08.956 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 6 set_chunk 6 low_tier returned error code -95 offset: 227840 length: 1695232 2024-09-18T11:30:08.956 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 6 v 0 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-18T11:30:08.956 INFO:tasks.rados.rados.0.smithi119.stdout:197: done (15 left) 2024-09-18T11:30:08.956 INFO:tasks.rados.rados.0.smithi119.stdout:205: done (14 left) 2024-09-18T11:30:08.956 INFO:tasks.rados.rados.0.smithi119.stdout:211: set_chunk oid 12 target oid 12 low_tier 2024-09-18T11:30:08.956 INFO:tasks.rados.rados.0.smithi119.stdout:211: set_chunk oid 12 offset: 1019392 length: 776192 target oid smithi11939506-12 low_tier offset: 1019392 snap -1 2024-09-18T11:30:08.957 INFO:tasks.rados.rados.0.smithi119.stdout:212: set_chunk oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.957 INFO:tasks.rados.rados.0.smithi119.stdout:212: set_chunk oid 13 offset: 87552 length: 1277440 target oid smithi11939506-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 87552 snap -1 2024-09-18T11:30:08.957 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.957 INFO:tasks.rados.rados.0.smithi119.stdout:206: finishing set_chunk to oid 7 2024-09-18T11:30:08.957 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 7 set_chunk 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1163776 length: 592896 2024-09-18T11:30:08.957 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 7 v 0 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-18T11:30:08.957 INFO:tasks.rados.rados.0.smithi119.stdout:206: done (15 left) 2024-09-18T11:30:08.957 INFO:tasks.rados.rados.0.smithi119.stdout:213: set_chunk oid 14 target oid 14 low_tier 2024-09-18T11:30:08.957 INFO:tasks.rados.rados.0.smithi119.stdout:213: set_chunk oid 14 offset: 1124864 length: 1042944 target oid smithi11939506-14 low_tier offset: 1124864 snap -1 2024-09-18T11:30:08.957 INFO:tasks.rados.rados.0.smithi119.stdout:210: finishing set_chunk to oid 11 2024-09-18T11:30:08.957 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 11 set_chunk 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 377344 length: 735744 2024-09-18T11:30:08.957 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 11 v 0 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-18T11:30:08.957 INFO:tasks.rados.rados.0.smithi119.stdout:210: done (15 left) 2024-09-18T11:30:08.958 INFO:tasks.rados.rados.0.smithi119.stdout:214: set_chunk oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.958 INFO:tasks.rados.rados.0.smithi119.stdout:214: set_chunk oid 15 offset: 1279488 length: 400384 target oid smithi11939506-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1279488 snap -1 2024-09-18T11:30:08.958 INFO:tasks.rados.rados.0.smithi119.stdout:202: finishing set_chunk to oid 3 2024-09-18T11:30:08.958 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 3 set_chunk 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 26624 length: 590336 2024-09-18T11:30:08.958 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 0 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-18T11:30:08.958 INFO:tasks.rados.rados.0.smithi119.stdout:202: done (15 left) 2024-09-18T11:30:08.958 INFO:tasks.rados.rados.0.smithi119.stdout:215: set_chunk oid 16 target oid 16 low_tier 2024-09-18T11:30:08.958 INFO:tasks.rados.rados.0.smithi119.stdout:215: set_chunk oid 16 offset: 274432 length: 681472 target oid smithi11939506-16 low_tier offset: 274432 snap -1 2024-09-18T11:30:08.958 INFO:tasks.rados.rados.0.smithi119.stdout:204: finishing set_chunk to oid 5 2024-09-18T11:30:08.958 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 5 set_chunk 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1335808 length: 476160 2024-09-18T11:30:08.958 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 0 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-18T11:30:08.958 INFO:tasks.rados.rados.0.smithi119.stdout:207: finishing set_chunk to oid 8 2024-09-18T11:30:08.958 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 8 set_chunk 8 low_tier returned error code -95 offset: 289280 length: 1492992 2024-09-18T11:30:08.959 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 8 v 0 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-18T11:30:08.959 INFO:tasks.rados.rados.0.smithi119.stdout:209: finishing set_chunk to oid 10 2024-09-18T11:30:08.959 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 10 set_chunk 10 low_tier returned error code -95 offset: 490496 length: 1450496 2024-09-18T11:30:08.959 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 10 v 0 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-18T11:30:08.959 INFO:tasks.rados.rados.0.smithi119.stdout:204: done (15 left) 2024-09-18T11:30:08.959 INFO:tasks.rados.rados.0.smithi119.stdout:207: done (14 left) 2024-09-18T11:30:08.959 INFO:tasks.rados.rados.0.smithi119.stdout:209: done (13 left) 2024-09-18T11:30:08.959 INFO:tasks.rados.rados.0.smithi119.stdout:216: set_chunk oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.959 INFO:tasks.rados.rados.0.smithi119.stdout:216: set_chunk oid 17 offset: 738816 length: 964096 target oid smithi11939506-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 738816 snap -1 2024-09-18T11:30:08.959 INFO:tasks.rados.rados.0.smithi119.stdout:217: set_chunk oid 18 target oid 18 low_tier 2024-09-18T11:30:08.959 INFO:tasks.rados.rados.0.smithi119.stdout:217: set_chunk oid 18 offset: 17920 length: 1212416 target oid smithi11939506-18 low_tier offset: 17920 snap -1 2024-09-18T11:30:08.959 INFO:tasks.rados.rados.0.smithi119.stdout:218: set_chunk oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.959 INFO:tasks.rados.rados.0.smithi119.stdout:218: set_chunk oid 19 offset: 389120 length: 1477632 target oid smithi11939506-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 389120 snap -1 2024-09-18T11:30:08.959 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.959 INFO:tasks.rados.rados.0.smithi119.stdout:211: finishing set_chunk to oid 12 2024-09-18T11:30:08.960 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 12 set_chunk 12 low_tier returned error code -95 offset: 1019392 length: 776192 2024-09-18T11:30:08.960 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 12 v 0 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-18T11:30:08.960 INFO:tasks.rados.rados.0.smithi119.stdout:213: finishing set_chunk to oid 14 2024-09-18T11:30:08.960 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 14 set_chunk 14 low_tier returned error code -95 offset: 1124864 length: 1042944 2024-09-18T11:30:08.960 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 0 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-18T11:30:08.960 INFO:tasks.rados.rados.0.smithi119.stdout:215: finishing set_chunk to oid 16 2024-09-18T11:30:08.960 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 16 set_chunk 16 low_tier returned error code -95 offset: 274432 length: 681472 2024-09-18T11:30:08.960 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 0 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-18T11:30:08.960 INFO:tasks.rados.rados.0.smithi119.stdout:211: done (15 left) 2024-09-18T11:30:08.960 INFO:tasks.rados.rados.0.smithi119.stdout:213: done (14 left) 2024-09-18T11:30:08.960 INFO:tasks.rados.rados.0.smithi119.stdout:215: done (13 left) 2024-09-18T11:30:08.960 INFO:tasks.rados.rados.0.smithi119.stdout:219: set_chunk oid 20 target oid 20 low_tier 2024-09-18T11:30:08.960 INFO:tasks.rados.rados.0.smithi119.stdout:219: set_chunk oid 20 offset: 1054208 length: 1043456 target oid smithi11939506-20 low_tier offset: 1054208 snap -1 2024-09-18T11:30:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:220: set_chunk oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:220: set_chunk oid 21 offset: 1160192 length: 745472 target oid smithi11939506-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1160192 snap -1 2024-09-18T11:30:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:221: set_chunk oid 22 target oid 22 low_tier 2024-09-18T11:30:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:221: set_chunk oid 22 offset: 292352 length: 968192 target oid smithi11939506-22 low_tier offset: 292352 snap -1 2024-09-18T11:30:08.961 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:217: finishing set_chunk to oid 18 2024-09-18T11:30:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 18 set_chunk 18 low_tier returned error code -95 offset: 17920 length: 1212416 2024-09-18T11:30:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 0 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-18T11:30:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:218: finishing set_chunk to oid 19 2024-09-18T11:30:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 19 set_chunk 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 389120 length: 1477632 2024-09-18T11:30:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 0 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-18T11:30:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:217: done (15 left) 2024-09-18T11:30:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:218: done (14 left) 2024-09-18T11:30:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:222: set_chunk oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.962 INFO:tasks.rados.rados.0.smithi119.stdout:222: set_chunk oid 23 offset: 614400 length: 1725952 target oid smithi11939506-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 614400 snap -1 2024-09-18T11:30:08.962 INFO:tasks.rados.rados.0.smithi119.stdout:223: set_chunk oid 24 target oid 24 low_tier 2024-09-18T11:30:08.962 INFO:tasks.rados.rados.0.smithi119.stdout:223: set_chunk oid 24 offset: 381952 length: 1350144 target oid smithi11939506-24 low_tier offset: 381952 snap -1 2024-09-18T11:30:08.962 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.962 INFO:tasks.rados.rados.0.smithi119.stdout:222: finishing set_chunk to oid 23 2024-09-18T11:30:08.962 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 23 set_chunk 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 614400 length: 1725952 2024-09-18T11:30:08.962 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 0 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-18T11:30:08.962 INFO:tasks.rados.rados.0.smithi119.stdout:222: done (15 left) 2024-09-18T11:30:08.962 INFO:tasks.rados.rados.0.smithi119.stdout:224: set_chunk oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.962 INFO:tasks.rados.rados.0.smithi119.stdout:224: set_chunk oid 25 offset: 480768 length: 380928 target oid smithi11939506-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 480768 snap -1 2024-09-18T11:30:08.962 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.962 INFO:tasks.rados.rados.0.smithi119.stdout:221: finishing set_chunk to oid 22 2024-09-18T11:30:08.962 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 22 set_chunk 22 low_tier returned error code -95 offset: 292352 length: 968192 2024-09-18T11:30:08.962 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 0 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-18T11:30:08.963 INFO:tasks.rados.rados.0.smithi119.stdout:223: finishing set_chunk to oid 24 2024-09-18T11:30:08.963 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 24 set_chunk 24 low_tier returned error code -95 offset: 381952 length: 1350144 2024-09-18T11:30:08.963 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 0 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-18T11:30:08.963 INFO:tasks.rados.rados.0.smithi119.stdout:201: finishing set_chunk to oid 2 2024-09-18T11:30:08.963 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 2 set_chunk 2 low_tier returned error code -95 offset: 60928 length: 758784 2024-09-18T11:30:08.963 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 0 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-18T11:30:08.963 INFO:tasks.rados.rados.0.smithi119.stdout:212: finishing set_chunk to oid 13 2024-09-18T11:30:08.963 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 13 set_chunk 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 87552 length: 1277440 2024-09-18T11:30:08.963 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 13 v 0 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-18T11:30:08.963 INFO:tasks.rados.rados.0.smithi119.stdout:214: finishing set_chunk to oid 15 2024-09-18T11:30:08.963 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 15 set_chunk 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1279488 length: 400384 2024-09-18T11:30:08.963 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 0 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-18T11:30:08.963 INFO:tasks.rados.rados.0.smithi119.stdout:201: done (15 left) 2024-09-18T11:30:08.964 INFO:tasks.rados.rados.0.smithi119.stdout:212: done (14 left) 2024-09-18T11:30:08.964 INFO:tasks.rados.rados.0.smithi119.stdout:214: done (13 left) 2024-09-18T11:30:08.964 INFO:tasks.rados.rados.0.smithi119.stdout:221: done (12 left) 2024-09-18T11:30:08.964 INFO:tasks.rados.rados.0.smithi119.stdout:223: done (11 left) 2024-09-18T11:30:08.964 INFO:tasks.rados.rados.0.smithi119.stdout:225: set_chunk oid 26 target oid 26 low_tier 2024-09-18T11:30:08.964 INFO:tasks.rados.rados.0.smithi119.stdout:225: set_chunk oid 26 offset: 374784 length: 1387520 target oid smithi11939506-26 low_tier offset: 374784 snap -1 2024-09-18T11:30:08.964 INFO:tasks.rados.rados.0.smithi119.stdout:226: set_chunk oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.964 INFO:tasks.rados.rados.0.smithi119.stdout:226: set_chunk oid 27 offset: 964096 length: 409600 target oid smithi11939506-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 964096 snap -1 2024-09-18T11:30:08.964 INFO:tasks.rados.rados.0.smithi119.stdout:227: set_chunk oid 28 target oid 28 low_tier 2024-09-18T11:30:08.964 INFO:tasks.rados.rados.0.smithi119.stdout:227: set_chunk oid 28 offset: 1574400 length: 226816 target oid smithi11939506-28 low_tier offset: 1574400 snap -1 2024-09-18T11:30:08.964 INFO:tasks.rados.rados.0.smithi119.stdout:228: set_chunk oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.964 INFO:tasks.rados.rados.0.smithi119.stdout:228: set_chunk oid 29 offset: 2123776 length: 64512 target oid smithi11939506-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 2123776 snap -1 2024-09-18T11:30:08.964 INFO:tasks.rados.rados.0.smithi119.stdout:229: set_chunk oid 30 target oid 30 low_tier 2024-09-18T11:30:08.964 INFO:tasks.rados.rados.0.smithi119.stdout:229: set_chunk oid 30 offset: 1509376 length: 176640 target oid smithi11939506-30 low_tier offset: 1509376 snap -1 2024-09-18T11:30:08.965 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.965 INFO:tasks.rados.rados.0.smithi119.stdout:216: finishing set_chunk to oid 17 2024-09-18T11:30:08.965 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 17 set_chunk 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 738816 length: 964096 2024-09-18T11:30:08.965 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 0 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-18T11:30:08.965 INFO:tasks.rados.rados.0.smithi119.stdout:216: done (15 left) 2024-09-18T11:30:08.965 INFO:tasks.rados.rados.0.smithi119.stdout:230: set_chunk oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.965 INFO:tasks.rados.rados.0.smithi119.stdout:230: set_chunk oid 31 offset: 1466368 length: 329728 target oid smithi11939506-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1466368 snap -1 2024-09-18T11:30:08.965 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.965 INFO:tasks.rados.rados.0.smithi119.stdout:225: finishing set_chunk to oid 26 2024-09-18T11:30:08.965 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 26 set_chunk 26 low_tier returned error code -95 offset: 374784 length: 1387520 2024-09-18T11:30:08.965 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 0 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-18T11:30:08.965 INFO:tasks.rados.rados.0.smithi119.stdout:226: finishing set_chunk to oid 27 2024-09-18T11:30:08.965 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 27 set_chunk 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 964096 length: 409600 2024-09-18T11:30:08.965 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 0 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T11:30:08.966 INFO:tasks.rados.rados.0.smithi119.stdout:225: done (15 left) 2024-09-18T11:30:08.966 INFO:tasks.rados.rados.0.smithi119.stdout:226: done (14 left) 2024-09-18T11:30:08.966 INFO:tasks.rados.rados.0.smithi119.stdout:231: set_chunk oid 32 target oid 32 low_tier 2024-09-18T11:30:08.966 INFO:tasks.rados.rados.0.smithi119.stdout:231: set_chunk oid 32 offset: 601088 length: 1489920 target oid smithi11939506-32 low_tier offset: 601088 snap -1 2024-09-18T11:30:08.966 INFO:tasks.rados.rados.0.smithi119.stdout:232: set_chunk oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.966 INFO:tasks.rados.rados.0.smithi119.stdout:232: set_chunk oid 33 offset: 197632 length: 958464 target oid smithi11939506-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 197632 snap -1 2024-09-18T11:30:08.966 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.966 INFO:tasks.rados.rados.0.smithi119.stdout:231: finishing set_chunk to oid 32 2024-09-18T11:30:08.966 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 32 set_chunk 32 low_tier returned error code -95 offset: 601088 length: 1489920 2024-09-18T11:30:08.966 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 0 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-18T11:30:08.966 INFO:tasks.rados.rados.0.smithi119.stdout:227: finishing set_chunk to oid 28 2024-09-18T11:30:08.966 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 28 set_chunk 28 low_tier returned error code -95 offset: 1574400 length: 226816 2024-09-18T11:30:08.966 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 28 v 0 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-18T11:30:08.967 INFO:tasks.rados.rados.0.smithi119.stdout:227: done (15 left) 2024-09-18T11:30:08.967 INFO:tasks.rados.rados.0.smithi119.stdout:231: done (14 left) 2024-09-18T11:30:08.967 INFO:tasks.rados.rados.0.smithi119.stdout:233: set_chunk oid 34 target oid 34 low_tier 2024-09-18T11:30:08.967 INFO:tasks.rados.rados.0.smithi119.stdout:233: set_chunk oid 34 offset: 1001984 length: 793088 target oid smithi11939506-34 low_tier offset: 1001984 snap -1 2024-09-18T11:30:08.967 INFO:tasks.rados.rados.0.smithi119.stdout:234: set_chunk oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.967 INFO:tasks.rados.rados.0.smithi119.stdout:234: set_chunk oid 35 offset: 1007104 length: 545280 target oid smithi11939506-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1007104 snap -1 2024-09-18T11:30:08.967 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.967 INFO:tasks.rados.rados.0.smithi119.stdout:230: finishing set_chunk to oid 31 2024-09-18T11:30:08.967 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 31 set_chunk 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1466368 length: 329728 2024-09-18T11:30:08.967 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 0 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-18T11:30:08.967 INFO:tasks.rados.rados.0.smithi119.stdout:230: done (15 left) 2024-09-18T11:30:08.967 INFO:tasks.rados.rados.0.smithi119.stdout:235: set_chunk oid 36 target oid 36 low_tier 2024-09-18T11:30:08.967 INFO:tasks.rados.rados.0.smithi119.stdout:235: set_chunk oid 36 offset: 1300480 length: 910336 target oid smithi11939506-36 low_tier offset: 1300480 snap -1 2024-09-18T11:30:08.967 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.968 INFO:tasks.rados.rados.0.smithi119.stdout:232: finishing set_chunk to oid 33 2024-09-18T11:30:08.968 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 33 set_chunk 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 197632 length: 958464 2024-09-18T11:30:08.968 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 33 v 0 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-18T11:30:08.968 INFO:tasks.rados.rados.0.smithi119.stdout:234: finishing set_chunk to oid 35 2024-09-18T11:30:08.968 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 35 set_chunk 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1007104 length: 545280 2024-09-18T11:30:08.968 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 0 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-18T11:30:08.968 INFO:tasks.rados.rados.0.smithi119.stdout:232: done (15 left) 2024-09-18T11:30:08.968 INFO:tasks.rados.rados.0.smithi119.stdout:234: done (14 left) 2024-09-18T11:30:08.968 INFO:tasks.rados.rados.0.smithi119.stdout:236: set_chunk oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.968 INFO:tasks.rados.rados.0.smithi119.stdout:236: set_chunk oid 37 offset: 1644032 length: 113152 target oid smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1644032 snap -1 2024-09-18T11:30:08.968 INFO:tasks.rados.rados.0.smithi119.stdout:237: set_chunk oid 38 target oid 38 low_tier 2024-09-18T11:30:08.968 INFO:tasks.rados.rados.0.smithi119.stdout:237: set_chunk oid 38 offset: 705536 length: 632832 target oid smithi11939506-38 low_tier offset: 705536 snap -1 2024-09-18T11:30:08.968 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.969 INFO:tasks.rados.rados.0.smithi119.stdout:236: finishing set_chunk to oid 37 2024-09-18T11:30:08.969 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 37 set_chunk 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1644032 length: 113152 2024-09-18T11:30:08.969 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 0 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-18T11:30:08.969 INFO:tasks.rados.rados.0.smithi119.stdout:236: done (15 left) 2024-09-18T11:30:08.969 INFO:tasks.rados.rados.0.smithi119.stdout:238: set_chunk oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.969 INFO:tasks.rados.rados.0.smithi119.stdout:237: finishing set_chunk to oid 38 2024-09-18T11:30:08.969 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 38 set_chunk 38 low_tier returned error code -95 offset: 705536 length: 632832 2024-09-18T11:30:08.969 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 0 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-18T11:30:08.969 INFO:tasks.rados.rados.0.smithi119.stdout:238: set_chunk oid 39 offset: 1381376 length: 443392 target oid smithi11939506-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1381376 snap -1 2024-09-18T11:30:08.969 INFO:tasks.rados.rados.0.smithi119.stdout:237: done (15 left) 2024-09-18T11:30:08.969 INFO:tasks.rados.rados.0.smithi119.stdout:239: set_chunk oid 40 target oid 40 low_tier 2024-09-18T11:30:08.969 INFO:tasks.rados.rados.0.smithi119.stdout:239: set_chunk oid 40 offset: 2018304 length: 397824 target oid smithi11939506-40 low_tier offset: 2018304 snap -1 2024-09-18T11:30:08.969 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.970 INFO:tasks.rados.rados.0.smithi119.stdout:238: finishing set_chunk to oid 39 2024-09-18T11:30:08.970 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 39 set_chunk 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1381376 length: 443392 2024-09-18T11:30:08.970 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 39 v 0 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-18T11:30:08.970 INFO:tasks.rados.rados.0.smithi119.stdout:238: done (15 left) 2024-09-18T11:30:08.970 INFO:tasks.rados.rados.0.smithi119.stdout:240: set_chunk oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.970 INFO:tasks.rados.rados.0.smithi119.stdout:240: set_chunk oid 41 offset: 503296 length: 2000896 target oid smithi11939506-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 503296 snap -1 2024-09-18T11:30:08.970 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.970 INFO:tasks.rados.rados.0.smithi119.stdout:196: finishing set_chunk to oid 47 2024-09-18T11:30:08.970 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 9 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-18T11:30:08.970 INFO:tasks.rados.rados.0.smithi119.stdout:196: done (15 left) 2024-09-18T11:30:08.970 INFO:tasks.rados.rados.0.smithi119.stdout:241: set_chunk oid 42 target oid 42 low_tier 2024-09-18T11:30:08.970 INFO:tasks.rados.rados.0.smithi119.stdout:241: set_chunk oid 42 offset: 1221632 length: 1083904 target oid smithi11939506-42 low_tier offset: 1221632 snap -1 2024-09-18T11:30:08.970 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.970 INFO:tasks.rados.rados.0.smithi119.stdout:200: finishing set_chunk to oid 1 2024-09-18T11:30:08.970 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-18T11:30:08.971 INFO:tasks.rados.rados.0.smithi119.stdout:200: done (15 left) 2024-09-18T11:30:08.971 INFO:tasks.rados.rados.0.smithi119.stdout:242: set_chunk oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.971 INFO:tasks.rados.rados.0.smithi119.stdout:242: set_chunk oid 43 offset: 1434112 length: 1156608 target oid smithi11939506-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1434112 snap -1 2024-09-18T11:30:08.971 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.971 INFO:tasks.rados.rados.0.smithi119.stdout:241: finishing set_chunk to oid 42 2024-09-18T11:30:08.971 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 42 set_chunk 42 low_tier returned error code -95 offset: 1221632 length: 1083904 2024-09-18T11:30:08.971 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 0 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T11:30:08.971 INFO:tasks.rados.rados.0.smithi119.stdout:240: finishing set_chunk to oid 41 2024-09-18T11:30:08.971 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 41 set_chunk 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 503296 length: 2000896 2024-09-18T11:30:08.971 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 41 v 0 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-18T11:30:08.971 INFO:tasks.rados.rados.0.smithi119.stdout:240: done (15 left) 2024-09-18T11:30:08.971 INFO:tasks.rados.rados.0.smithi119.stdout:241: done (14 left) 2024-09-18T11:30:08.971 INFO:tasks.rados.rados.0.smithi119.stdout:242 retry set_chunk ! 2024-09-18T11:30:08.971 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 14 2024-09-18T11:30:08.971 INFO:tasks.rados.rados.0.smithi119.stdout:242: finishing set_chunk to oid 43 2024-09-18T11:30:08.972 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 43 set_chunk 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1434112 length: 1156608 2024-09-18T11:30:08.972 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-18T11:30:08.972 INFO:tasks.rados.rados.0.smithi119.stdout:242: done (13 left) 2024-09-18T11:30:08.972 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 13 2024-09-18T11:30:08.972 INFO:tasks.rados.rados.0.smithi119.stdout:199: finishing set_chunk to oid 50 2024-09-18T11:30:08.972 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 9 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-18T11:30:08.972 INFO:tasks.rados.rados.0.smithi119.stdout:199: done (12 left) 2024-09-18T11:30:08.972 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 12 2024-09-18T11:30:08.972 INFO:tasks.rados.rados.0.smithi119.stdout:193: finishing set_chunk to oid 44 2024-09-18T11:30:08.972 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 44 v 9 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-18T11:30:08.972 INFO:tasks.rados.rados.0.smithi119.stdout:193: done (11 left) 2024-09-18T11:30:08.972 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 11 2024-09-18T11:30:08.972 INFO:tasks.rados.rados.0.smithi119.stdout:219: finishing set_chunk to oid 20 2024-09-18T11:30:08.972 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 20 v 3 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-18T11:30:08.973 INFO:tasks.rados.rados.0.smithi119.stdout:219: done (10 left) 2024-09-18T11:30:08.973 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 10 2024-09-18T11:30:08.973 INFO:tasks.rados.rados.0.smithi119.stdout:195: finishing set_chunk to oid 46 2024-09-18T11:30:08.973 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 9 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-18T11:30:08.973 INFO:tasks.rados.rados.0.smithi119.stdout:195: done (9 left) 2024-09-18T11:30:08.973 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 9 2024-09-18T11:30:08.973 INFO:tasks.rados.rados.0.smithi119.stdout:208: finishing set_chunk to oid 9 2024-09-18T11:30:08.973 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 6 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T11:30:08.973 INFO:tasks.rados.rados.0.smithi119.stdout:208: done (8 left) 2024-09-18T11:30:08.973 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 8 2024-09-18T11:30:08.973 INFO:tasks.rados.rados.0.smithi119.stdout:198: finishing set_chunk to oid 49 2024-09-18T11:30:08.973 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 18 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-18T11:30:08.973 INFO:tasks.rados.rados.0.smithi119.stdout:198: done (7 left) 2024-09-18T11:30:08.973 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 7 2024-09-18T11:30:08.973 INFO:tasks.rados.rados.0.smithi119.stdout:233: finishing set_chunk to oid 34 2024-09-18T11:30:08.974 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 9 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T11:30:08.974 INFO:tasks.rados.rados.0.smithi119.stdout:233: done (6 left) 2024-09-18T11:30:08.974 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 6 2024-09-18T11:30:08.974 INFO:tasks.rados.rados.0.smithi119.stdout:224: finishing set_chunk to oid 25 2024-09-18T11:30:08.974 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 3 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-18T11:30:08.974 INFO:tasks.rados.rados.0.smithi119.stdout:224: done (5 left) 2024-09-18T11:30:08.974 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 5 2024-09-18T11:30:08.974 INFO:tasks.rados.rados.0.smithi119.stdout:228: finishing set_chunk to oid 29 2024-09-18T11:30:08.974 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 3 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-18T11:30:08.974 INFO:tasks.rados.rados.0.smithi119.stdout:239: finishing set_chunk to oid 40 2024-09-18T11:30:08.974 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 6 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-18T11:30:08.974 INFO:tasks.rados.rados.0.smithi119.stdout:220: finishing set_chunk to oid 21 2024-09-18T11:30:08.974 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 6 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-18T11:30:08.974 INFO:tasks.rados.rados.0.smithi119.stdout:220: done (4 left) 2024-09-18T11:30:08.974 INFO:tasks.rados.rados.0.smithi119.stdout:228: done (3 left) 2024-09-18T11:30:08.975 INFO:tasks.rados.rados.0.smithi119.stdout:239: done (2 left) 2024-09-18T11:30:08.975 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 2 2024-09-18T11:30:08.975 INFO:tasks.rados.rados.0.smithi119.stdout:229: finishing set_chunk to oid 30 2024-09-18T11:30:08.975 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 30 v 6 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-18T11:30:08.975 INFO:tasks.rados.rados.0.smithi119.stdout:229: done (1 left) 2024-09-18T11:30:08.975 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 1 2024-09-18T11:30:08.975 INFO:tasks.rados.rados.0.smithi119.stdout:235: finishing set_chunk to oid 36 2024-09-18T11:30:08.975 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 36 v 6 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-18T11:30:08.975 INFO:tasks.rados.rados.0.smithi119.stdout:235: done (0 left) 2024-09-18T11:30:08.975 INFO:tasks.rados.rados.0.smithi119.stdout:243: set_chunk oid 44 target oid 44 low_tier 2024-09-18T11:30:08.975 INFO:tasks.rados.rados.0.smithi119.stdout:243: set_chunk oid 44 offset: 141312 length: 914944 target oid smithi11939506-44 low_tier offset: 141312 snap -1 2024-09-18T11:30:08.975 INFO:tasks.rados.rados.0.smithi119.stdout:244: set_chunk oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.975 INFO:tasks.rados.rados.0.smithi119.stdout:244: set_chunk oid 45 offset: 538624 length: 336896 target oid smithi11939506-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 538624 snap -1 2024-09-18T11:30:08.975 INFO:tasks.rados.rados.0.smithi119.stdout:245: set_chunk oid 46 target oid 46 low_tier 2024-09-18T11:30:08.975 INFO:tasks.rados.rados.0.smithi119.stdout:245: set_chunk oid 46 offset: 452608 length: 16384 target oid smithi11939506-46 low_tier offset: 452608 snap -1 2024-09-18T11:30:08.976 INFO:tasks.rados.rados.0.smithi119.stdout:246: set_chunk oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.976 INFO:tasks.rados.rados.0.smithi119.stdout:246: set_chunk oid 47 offset: 150016 length: 1290240 target oid smithi11939506-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 150016 snap -1 2024-09-18T11:30:08.976 INFO:tasks.rados.rados.0.smithi119.stdout:247: set_chunk oid 48 target oid 48 low_tier 2024-09-18T11:30:08.976 INFO:tasks.rados.rados.0.smithi119.stdout:247: set_chunk oid 48 offset: 650240 length: 129536 target oid smithi11939506-48 low_tier offset: 650240 snap -1 2024-09-18T11:30:08.976 INFO:tasks.rados.rados.0.smithi119.stdout:248: set_chunk oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.976 INFO:tasks.rados.rados.0.smithi119.stdout:248: set_chunk oid 49 offset: 69632 length: 146944 target oid smithi11939506-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 69632 snap -1 2024-09-18T11:30:08.976 INFO:tasks.rados.rados.0.smithi119.stdout:249: set_chunk oid 50 target oid 50 low_tier 2024-09-18T11:30:08.976 INFO:tasks.rados.rados.0.smithi119.stdout:249: set_chunk oid 50 offset: 178176 length: 211968 target oid smithi11939506-50 low_tier offset: 178176 snap -1 2024-09-18T11:30:08.976 INFO:tasks.rados.rados.0.smithi119.stdout:250: set_chunk oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.976 INFO:tasks.rados.rados.0.smithi119.stdout:250: set_chunk oid 1 offset: 388608 length: 1506304 target oid smithi11939506-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 388608 snap -1 2024-09-18T11:30:08.976 INFO:tasks.rados.rados.0.smithi119.stdout:251: set_chunk oid 2 target oid 2 low_tier 2024-09-18T11:30:08.976 INFO:tasks.rados.rados.0.smithi119.stdout:251: set_chunk oid 2 offset: 114688 length: 374272 target oid smithi11939506-2 low_tier offset: 114688 snap -1 2024-09-18T11:30:08.976 INFO:tasks.rados.rados.0.smithi119.stdout:252: set_chunk oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.977 INFO:tasks.rados.rados.0.smithi119.stdout:252: set_chunk oid 3 offset: 1058304 length: 596480 target oid smithi11939506-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1058304 snap -1 2024-09-18T11:30:08.977 INFO:tasks.rados.rados.0.smithi119.stdout:253: set_chunk oid 4 target oid 4 low_tier 2024-09-18T11:30:08.977 INFO:tasks.rados.rados.0.smithi119.stdout:253: set_chunk oid 4 offset: 23552 length: 777216 target oid smithi11939506-4 low_tier offset: 23552 snap -1 2024-09-18T11:30:08.977 INFO:tasks.rados.rados.0.smithi119.stdout:254: set_chunk oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.977 INFO:tasks.rados.rados.0.smithi119.stdout:254: set_chunk oid 5 offset: 1954304 length: 42496 target oid smithi11939506-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1954304 snap -1 2024-09-18T11:30:08.977 INFO:tasks.rados.rados.0.smithi119.stdout:255: set_chunk oid 6 target oid 6 low_tier 2024-09-18T11:30:08.977 INFO:tasks.rados.rados.0.smithi119.stdout:255: set_chunk oid 6 offset: 617984 length: 24064 target oid smithi11939506-6 low_tier offset: 617984 snap -1 2024-09-18T11:30:08.977 INFO:tasks.rados.rados.0.smithi119.stdout:256: set_chunk oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.977 INFO:tasks.rados.rados.0.smithi119.stdout:256: set_chunk oid 7 offset: 93184 length: 1929216 target oid smithi11939506-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 93184 snap -1 2024-09-18T11:30:08.977 INFO:tasks.rados.rados.0.smithi119.stdout:257: set_chunk oid 8 target oid 8 low_tier 2024-09-18T11:30:08.977 INFO:tasks.rados.rados.0.smithi119.stdout:257: set_chunk oid 8 offset: 273408 length: 29184 target oid smithi11939506-8 low_tier offset: 273408 snap -1 2024-09-18T11:30:08.977 INFO:tasks.rados.rados.0.smithi119.stdout:258: set_chunk oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.977 INFO:tasks.rados.rados.0.smithi119.stdout:258: set_chunk oid 9 offset: 859136 length: 478720 target oid smithi11939506-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 859136 snap -1 2024-09-18T11:30:08.978 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.978 INFO:tasks.rados.rados.0.smithi119.stdout:243: finishing set_chunk to oid 44 2024-09-18T11:30:08.978 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 44 set_chunk 44 low_tier returned error code -95 offset: 141312 length: 914944 2024-09-18T11:30:08.978 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 44 v 0 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-18T11:30:08.978 INFO:tasks.rados.rados.0.smithi119.stdout:246: finishing set_chunk to oid 47 2024-09-18T11:30:08.978 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 47 set_chunk 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 150016 length: 1290240 2024-09-18T11:30:08.978 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 0 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-18T11:30:08.978 INFO:tasks.rados.rados.0.smithi119.stdout:243: done (15 left) 2024-09-18T11:30:08.978 INFO:tasks.rados.rados.0.smithi119.stdout:246: done (14 left) 2024-09-18T11:30:08.978 INFO:tasks.rados.rados.0.smithi119.stdout:259: set_chunk oid 10 target oid 10 low_tier 2024-09-18T11:30:08.978 INFO:tasks.rados.rados.0.smithi119.stdout:259: set_chunk oid 10 offset: 829952 length: 1104896 target oid smithi11939506-10 low_tier offset: 829952 snap -1 2024-09-18T11:30:08.978 INFO:tasks.rados.rados.0.smithi119.stdout:260: set_chunk oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.978 INFO:tasks.rados.rados.0.smithi119.stdout:260: set_chunk oid 11 offset: 1285632 length: 725504 target oid smithi11939506-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1285632 snap -1 2024-09-18T11:30:08.979 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.979 INFO:tasks.rados.rados.0.smithi119.stdout:247: finishing set_chunk to oid 48 2024-09-18T11:30:08.979 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 48 set_chunk 48 low_tier returned error code -95 offset: 650240 length: 129536 2024-09-18T11:30:08.979 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 48 v 0 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-18T11:30:08.979 INFO:tasks.rados.rados.0.smithi119.stdout:247: done (15 left) 2024-09-18T11:30:08.979 INFO:tasks.rados.rados.0.smithi119.stdout:261: set_chunk oid 12 target oid 12 low_tier 2024-09-18T11:30:08.979 INFO:tasks.rados.rados.0.smithi119.stdout:261: set_chunk oid 12 offset: 313344 length: 499712 target oid smithi11939506-12 low_tier offset: 313344 snap -1 2024-09-18T11:30:08.979 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.979 INFO:tasks.rados.rados.0.smithi119.stdout:250: finishing set_chunk to oid 1 2024-09-18T11:30:08.979 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 1 set_chunk 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 388608 length: 1506304 2024-09-18T11:30:08.979 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 0 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-18T11:30:08.979 INFO:tasks.rados.rados.0.smithi119.stdout:250: done (15 left) 2024-09-18T11:30:08.979 INFO:tasks.rados.rados.0.smithi119.stdout:262: set_chunk oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.979 INFO:tasks.rados.rados.0.smithi119.stdout:262: set_chunk oid 13 offset: 178688 length: 931328 target oid smithi11939506-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 178688 snap -1 2024-09-18T11:30:08.979 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.980 INFO:tasks.rados.rados.0.smithi119.stdout:256: finishing set_chunk to oid 7 2024-09-18T11:30:08.980 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 7 set_chunk 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 93184 length: 1929216 2024-09-18T11:30:08.980 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 7 v 0 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-18T11:30:08.980 INFO:tasks.rados.rados.0.smithi119.stdout:256: done (15 left) 2024-09-18T11:30:08.980 INFO:tasks.rados.rados.0.smithi119.stdout:263: set_chunk oid 14 target oid 14 low_tier 2024-09-18T11:30:08.980 INFO:tasks.rados.rados.0.smithi119.stdout:263: set_chunk oid 14 offset: 2184704 length: 2560 target oid smithi11939506-14 low_tier offset: 2184704 snap -1 2024-09-18T11:30:08.980 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.980 INFO:tasks.rados.rados.0.smithi119.stdout:261: finishing set_chunk to oid 12 2024-09-18T11:30:08.980 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 12 set_chunk 12 low_tier returned error code -95 offset: 313344 length: 499712 2024-09-18T11:30:08.980 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 12 v 0 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-18T11:30:08.980 INFO:tasks.rados.rados.0.smithi119.stdout:260: finishing set_chunk to oid 11 2024-09-18T11:30:08.980 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 11 set_chunk 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1285632 length: 725504 2024-09-18T11:30:08.980 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 11 v 0 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-18T11:30:08.980 INFO:tasks.rados.rados.0.smithi119.stdout:260: done (15 left) 2024-09-18T11:30:08.981 INFO:tasks.rados.rados.0.smithi119.stdout:261: done (14 left) 2024-09-18T11:30:08.981 INFO:tasks.rados.rados.0.smithi119.stdout:264: set_chunk oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.981 INFO:tasks.rados.rados.0.smithi119.stdout:264: set_chunk oid 15 offset: 1392640 length: 787968 target oid smithi11939506-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1392640 snap -1 2024-09-18T11:30:08.981 INFO:tasks.rados.rados.0.smithi119.stdout:265: set_chunk oid 16 target oid 16 low_tier 2024-09-18T11:30:08.981 INFO:tasks.rados.rados.0.smithi119.stdout:265: set_chunk oid 16 offset: 271872 length: 950272 target oid smithi11939506-16 low_tier offset: 271872 snap -1 2024-09-18T11:30:08.981 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.981 INFO:tasks.rados.rados.0.smithi119.stdout:252: finishing set_chunk to oid 3 2024-09-18T11:30:08.981 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 3 set_chunk 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1058304 length: 596480 2024-09-18T11:30:08.981 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 0 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-18T11:30:08.981 INFO:tasks.rados.rados.0.smithi119.stdout:252: done (15 left) 2024-09-18T11:30:08.981 INFO:tasks.rados.rados.0.smithi119.stdout:266: set_chunk oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.981 INFO:tasks.rados.rados.0.smithi119.stdout:266: set_chunk oid 17 offset: 485888 length: 1564160 target oid smithi11939506-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 485888 snap -1 2024-09-18T11:30:08.981 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.982 INFO:tasks.rados.rados.0.smithi119.stdout:251: finishing set_chunk to oid 2 2024-09-18T11:30:08.982 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 2 set_chunk 2 low_tier returned error code -95 offset: 114688 length: 374272 2024-09-18T11:30:08.982 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 0 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-18T11:30:08.982 INFO:tasks.rados.rados.0.smithi119.stdout:258: finishing set_chunk to oid 9 2024-09-18T11:30:08.982 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 9 set_chunk 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 859136 length: 478720 2024-09-18T11:30:08.982 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 0 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T11:30:08.982 INFO:tasks.rados.rados.0.smithi119.stdout:262: finishing set_chunk to oid 13 2024-09-18T11:30:08.982 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 13 set_chunk 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 178688 length: 931328 2024-09-18T11:30:08.982 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 13 v 0 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-18T11:30:08.982 INFO:tasks.rados.rados.0.smithi119.stdout:251: done (15 left) 2024-09-18T11:30:08.982 INFO:tasks.rados.rados.0.smithi119.stdout:258: done (14 left) 2024-09-18T11:30:08.982 INFO:tasks.rados.rados.0.smithi119.stdout:262: done (13 left) 2024-09-18T11:30:08.983 INFO:tasks.rados.rados.0.smithi119.stdout:267: set_chunk oid 18 target oid 18 low_tier 2024-09-18T11:30:08.983 INFO:tasks.rados.rados.0.smithi119.stdout:267: set_chunk oid 18 offset: 271360 length: 522752 target oid smithi11939506-18 low_tier offset: 271360 snap -1 2024-09-18T11:30:08.983 INFO:tasks.rados.rados.0.smithi119.stdout:268: set_chunk oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.983 INFO:tasks.rados.rados.0.smithi119.stdout:268: set_chunk oid 19 offset: 51712 length: 1988096 target oid smithi11939506-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 51712 snap -1 2024-09-18T11:30:08.983 INFO:tasks.rados.rados.0.smithi119.stdout:269: set_chunk oid 20 target oid 20 low_tier 2024-09-18T11:30:08.983 INFO:tasks.rados.rados.0.smithi119.stdout:269: set_chunk oid 20 offset: 1185792 length: 1111040 target oid smithi11939506-20 low_tier offset: 1185792 snap -1 2024-09-18T11:30:08.983 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.983 INFO:tasks.rados.rados.0.smithi119.stdout:259: finishing set_chunk to oid 10 2024-09-18T11:30:08.983 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 10 set_chunk 10 low_tier returned error code -95 offset: 829952 length: 1104896 2024-09-18T11:30:08.983 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 10 v 0 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-18T11:30:08.983 INFO:tasks.rados.rados.0.smithi119.stdout:265: finishing set_chunk to oid 16 2024-09-18T11:30:08.983 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 16 set_chunk 16 low_tier returned error code -95 offset: 271872 length: 950272 2024-09-18T11:30:08.983 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 0 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-18T11:30:08.983 INFO:tasks.rados.rados.0.smithi119.stdout:259: done (15 left) 2024-09-18T11:30:08.983 INFO:tasks.rados.rados.0.smithi119.stdout:265: done (14 left) 2024-09-18T11:30:08.984 INFO:tasks.rados.rados.0.smithi119.stdout:270: set_chunk oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.984 INFO:tasks.rados.rados.0.smithi119.stdout:270: set_chunk oid 21 offset: 353280 length: 784384 target oid smithi11939506-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 353280 snap -1 2024-09-18T11:30:08.984 INFO:tasks.rados.rados.0.smithi119.stdout:271: set_chunk oid 22 target oid 22 low_tier 2024-09-18T11:30:08.984 INFO:tasks.rados.rados.0.smithi119.stdout:271: set_chunk oid 22 offset: 1044992 length: 830976 target oid smithi11939506-22 low_tier offset: 1044992 snap -1 2024-09-18T11:30:08.984 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.984 INFO:tasks.rados.rados.0.smithi119.stdout:266: finishing set_chunk to oid 17 2024-09-18T11:30:08.984 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 17 set_chunk 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 485888 length: 1564160 2024-09-18T11:30:08.984 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 0 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-18T11:30:08.984 INFO:tasks.rados.rados.0.smithi119.stdout:266: done (15 left) 2024-09-18T11:30:08.984 INFO:tasks.rados.rados.0.smithi119.stdout:272: set_chunk oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.984 INFO:tasks.rados.rados.0.smithi119.stdout:272: set_chunk oid 23 offset: 1848832 length: 519168 target oid smithi11939506-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1848832 snap -1 2024-09-18T11:30:08.985 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.985 INFO:tasks.rados.rados.0.smithi119.stdout:269: finishing set_chunk to oid 20 2024-09-18T11:30:08.985 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 20 set_chunk 20 low_tier returned error code -95 offset: 1185792 length: 1111040 2024-09-18T11:30:08.985 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 20 v 0 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-18T11:30:08.985 INFO:tasks.rados.rados.0.smithi119.stdout:272: finishing set_chunk to oid 23 2024-09-18T11:30:08.985 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 23 set_chunk 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1848832 length: 519168 2024-09-18T11:30:08.985 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 0 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-18T11:30:08.985 INFO:tasks.rados.rados.0.smithi119.stdout:267: finishing set_chunk to oid 18 2024-09-18T11:30:08.985 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 18 set_chunk 18 low_tier returned error code -95 offset: 271360 length: 522752 2024-09-18T11:30:08.985 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 0 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-18T11:30:08.985 INFO:tasks.rados.rados.0.smithi119.stdout:268: finishing set_chunk to oid 19 2024-09-18T11:30:08.985 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 19 set_chunk 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 51712 length: 1988096 2024-09-18T11:30:08.986 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 0 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-18T11:30:08.986 INFO:tasks.rados.rados.0.smithi119.stdout:270: finishing set_chunk to oid 21 2024-09-18T11:30:08.986 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 21 set_chunk 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 353280 length: 784384 2024-09-18T11:30:08.986 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 0 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-18T11:30:08.986 INFO:tasks.rados.rados.0.smithi119.stdout:271: finishing set_chunk to oid 22 2024-09-18T11:30:08.986 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 22 set_chunk 22 low_tier returned error code -95 offset: 1044992 length: 830976 2024-09-18T11:30:08.986 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 0 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-18T11:30:08.986 INFO:tasks.rados.rados.0.smithi119.stdout:267: done (15 left) 2024-09-18T11:30:08.986 INFO:tasks.rados.rados.0.smithi119.stdout:268: done (14 left) 2024-09-18T11:30:08.986 INFO:tasks.rados.rados.0.smithi119.stdout:269: done (13 left) 2024-09-18T11:30:08.986 INFO:tasks.rados.rados.0.smithi119.stdout:270: done (12 left) 2024-09-18T11:30:08.986 INFO:tasks.rados.rados.0.smithi119.stdout:271: done (11 left) 2024-09-18T11:30:08.987 INFO:tasks.rados.rados.0.smithi119.stdout:272: done (10 left) 2024-09-18T11:30:08.987 INFO:tasks.rados.rados.0.smithi119.stdout:273: set_chunk oid 24 target oid 24 low_tier 2024-09-18T11:30:08.987 INFO:tasks.rados.rados.0.smithi119.stdout:273: set_chunk oid 24 offset: 1104384 length: 573440 target oid smithi11939506-24 low_tier offset: 1104384 snap -1 2024-09-18T11:30:08.987 INFO:tasks.rados.rados.0.smithi119.stdout:274: set_chunk oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.987 INFO:tasks.rados.rados.0.smithi119.stdout:274: set_chunk oid 25 offset: 471552 length: 276992 target oid smithi11939506-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 471552 snap -1 2024-09-18T11:30:08.987 INFO:tasks.rados.rados.0.smithi119.stdout:275: set_chunk oid 26 target oid 26 low_tier 2024-09-18T11:30:08.987 INFO:tasks.rados.rados.0.smithi119.stdout:275: set_chunk oid 26 offset: 622080 length: 284160 target oid smithi11939506-26 low_tier offset: 622080 snap -1 2024-09-18T11:30:08.987 INFO:tasks.rados.rados.0.smithi119.stdout:276: set_chunk oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.987 INFO:tasks.rados.rados.0.smithi119.stdout:276: set_chunk oid 27 offset: 1795584 length: 116224 target oid smithi11939506-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1795584 snap -1 2024-09-18T11:30:08.987 INFO:tasks.rados.rados.0.smithi119.stdout:277: set_chunk oid 28 target oid 28 low_tier 2024-09-18T11:30:08.987 INFO:tasks.rados.rados.0.smithi119.stdout:277: set_chunk oid 28 offset: 208896 length: 1858048 target oid smithi11939506-28 low_tier offset: 208896 snap -1 2024-09-18T11:30:08.987 INFO:tasks.rados.rados.0.smithi119.stdout:278: set_chunk oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.987 INFO:tasks.rados.rados.0.smithi119.stdout:278: set_chunk oid 29 offset: 1006080 length: 598016 target oid smithi11939506-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1006080 snap -1 2024-09-18T11:30:08.988 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.988 INFO:tasks.rados.rados.0.smithi119.stdout:274: finishing set_chunk to oid 25 2024-09-18T11:30:08.988 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 25 set_chunk 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 471552 length: 276992 2024-09-18T11:30:08.988 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 0 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-18T11:30:08.988 INFO:tasks.rados.rados.0.smithi119.stdout:274: done (15 left) 2024-09-18T11:30:08.988 INFO:tasks.rados.rados.0.smithi119.stdout:279: set_chunk oid 30 target oid 30 low_tier 2024-09-18T11:30:08.988 INFO:tasks.rados.rados.0.smithi119.stdout:279: set_chunk oid 30 offset: 145920 length: 703488 target oid smithi11939506-30 low_tier offset: 145920 snap -1 2024-09-18T11:30:08.988 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.988 INFO:tasks.rados.rados.0.smithi119.stdout:273: finishing set_chunk to oid 24 2024-09-18T11:30:08.988 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 24 set_chunk 24 low_tier returned error code -95 offset: 1104384 length: 573440 2024-09-18T11:30:08.988 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 0 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-18T11:30:08.988 INFO:tasks.rados.rados.0.smithi119.stdout:277: finishing set_chunk to oid 28 2024-09-18T11:30:08.989 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 28 set_chunk 28 low_tier returned error code -95 offset: 208896 length: 1858048 2024-09-18T11:30:08.989 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 28 v 0 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-18T11:30:08.989 INFO:tasks.rados.rados.0.smithi119.stdout:278: finishing set_chunk to oid 29 2024-09-18T11:30:08.989 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 29 set_chunk 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1006080 length: 598016 2024-09-18T11:30:08.989 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 0 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-18T11:30:08.989 INFO:tasks.rados.rados.0.smithi119.stdout:276: finishing set_chunk to oid 27 2024-09-18T11:30:08.989 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 27 set_chunk 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1795584 length: 116224 2024-09-18T11:30:08.989 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 0 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T11:30:08.989 INFO:tasks.rados.rados.0.smithi119.stdout:273: done (15 left) 2024-09-18T11:30:08.989 INFO:tasks.rados.rados.0.smithi119.stdout:276: done (14 left) 2024-09-18T11:30:08.989 INFO:tasks.rados.rados.0.smithi119.stdout:277: done (13 left) 2024-09-18T11:30:08.989 INFO:tasks.rados.rados.0.smithi119.stdout:278: done (12 left) 2024-09-18T11:30:08.989 INFO:tasks.rados.rados.0.smithi119.stdout:280: set_chunk oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.990 INFO:tasks.rados.rados.0.smithi119.stdout:280: set_chunk oid 31 offset: 753664 length: 1348096 target oid smithi11939506-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 753664 snap -1 2024-09-18T11:30:08.990 INFO:tasks.rados.rados.0.smithi119.stdout:281: set_chunk oid 32 target oid 32 low_tier 2024-09-18T11:30:08.990 INFO:tasks.rados.rados.0.smithi119.stdout:281: set_chunk oid 32 offset: 728064 length: 103936 target oid smithi11939506-32 low_tier offset: 728064 snap -1 2024-09-18T11:30:08.990 INFO:tasks.rados.rados.0.smithi119.stdout:282: set_chunk oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.990 INFO:tasks.rados.rados.0.smithi119.stdout:282: set_chunk oid 33 offset: 19968 length: 2275840 target oid smithi11939506-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 19968 snap -1 2024-09-18T11:30:08.990 INFO:tasks.rados.rados.0.smithi119.stdout:283: set_chunk oid 34 target oid 34 low_tier 2024-09-18T11:30:08.990 INFO:tasks.rados.rados.0.smithi119.stdout:283: set_chunk oid 34 offset: 1849344 length: 474624 target oid smithi11939506-34 low_tier offset: 1849344 snap -1 2024-09-18T11:30:08.990 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.990 INFO:tasks.rados.rados.0.smithi119.stdout:279: finishing set_chunk to oid 30 2024-09-18T11:30:08.990 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 30 set_chunk 30 low_tier returned error code -95 offset: 145920 length: 703488 2024-09-18T11:30:08.990 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 30 v 0 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-18T11:30:08.990 INFO:tasks.rados.rados.0.smithi119.stdout:279: done (15 left) 2024-09-18T11:30:08.990 INFO:tasks.rados.rados.0.smithi119.stdout:284: set_chunk oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.990 INFO:tasks.rados.rados.0.smithi119.stdout:284: set_chunk oid 35 offset: 859648 length: 1067008 target oid smithi11939506-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 859648 snap -1 2024-09-18T11:30:08.991 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.991 INFO:tasks.rados.rados.0.smithi119.stdout:282: finishing set_chunk to oid 33 2024-09-18T11:30:08.991 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 33 set_chunk 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 19968 length: 2275840 2024-09-18T11:30:08.991 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 33 v 0 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-18T11:30:08.991 INFO:tasks.rados.rados.0.smithi119.stdout:280: finishing set_chunk to oid 31 2024-09-18T11:30:08.991 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 31 set_chunk 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 753664 length: 1348096 2024-09-18T11:30:08.991 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 0 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-18T11:30:08.991 INFO:tasks.rados.rados.0.smithi119.stdout:280: done (15 left) 2024-09-18T11:30:08.991 INFO:tasks.rados.rados.0.smithi119.stdout:282: done (14 left) 2024-09-18T11:30:08.991 INFO:tasks.rados.rados.0.smithi119.stdout:285: set_chunk oid 36 target oid 36 low_tier 2024-09-18T11:30:08.991 INFO:tasks.rados.rados.0.smithi119.stdout:285: set_chunk oid 36 offset: 1075200 length: 760832 target oid smithi11939506-36 low_tier offset: 1075200 snap -1 2024-09-18T11:30:08.991 INFO:tasks.rados.rados.0.smithi119.stdout:286: set_chunk oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.991 INFO:tasks.rados.rados.0.smithi119.stdout:286: set_chunk oid 37 offset: 161792 length: 1345536 target oid smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 161792 snap -1 2024-09-18T11:30:08.991 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.992 INFO:tasks.rados.rados.0.smithi119.stdout:284: finishing set_chunk to oid 35 2024-09-18T11:30:08.992 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 35 set_chunk 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 859648 length: 1067008 2024-09-18T11:30:08.992 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 0 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-18T11:30:08.992 INFO:tasks.rados.rados.0.smithi119.stdout:284: done (15 left) 2024-09-18T11:30:08.992 INFO:tasks.rados.rados.0.smithi119.stdout:287: set_chunk oid 38 target oid 38 low_tier 2024-09-18T11:30:08.992 INFO:tasks.rados.rados.0.smithi119.stdout:287: set_chunk oid 38 offset: 225792 length: 1913344 target oid smithi11939506-38 low_tier offset: 225792 snap -1 2024-09-18T11:30:08.992 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.992 INFO:tasks.rados.rados.0.smithi119.stdout:285: finishing set_chunk to oid 36 2024-09-18T11:30:08.992 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 36 set_chunk 36 low_tier returned error code -95 offset: 1075200 length: 760832 2024-09-18T11:30:08.992 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 36 v 0 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-18T11:30:08.992 INFO:tasks.rados.rados.0.smithi119.stdout:286: finishing set_chunk to oid 37 2024-09-18T11:30:08.992 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 37 set_chunk 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 161792 length: 1345536 2024-09-18T11:30:08.992 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 0 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-18T11:30:08.992 INFO:tasks.rados.rados.0.smithi119.stdout:285: done (15 left) 2024-09-18T11:30:08.993 INFO:tasks.rados.rados.0.smithi119.stdout:286: done (14 left) 2024-09-18T11:30:08.993 INFO:tasks.rados.rados.0.smithi119.stdout:288: set_chunk oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.993 INFO:tasks.rados.rados.0.smithi119.stdout:288: set_chunk oid 39 offset: 1370112 length: 102400 target oid smithi11939506-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1370112 snap -1 2024-09-18T11:30:08.993 INFO:tasks.rados.rados.0.smithi119.stdout:289: set_chunk oid 40 target oid 40 low_tier 2024-09-18T11:30:08.993 INFO:tasks.rados.rados.0.smithi119.stdout:289: set_chunk oid 40 offset: 2438656 length: 185344 target oid smithi11939506-40 low_tier offset: 2438656 snap -1 2024-09-18T11:30:08.993 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.993 INFO:tasks.rados.rados.0.smithi119.stdout:287: finishing set_chunk to oid 38 2024-09-18T11:30:08.993 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 38 set_chunk 38 low_tier returned error code -95 offset: 225792 length: 1913344 2024-09-18T11:30:08.993 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 0 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-18T11:30:08.993 INFO:tasks.rados.rados.0.smithi119.stdout:287: done (15 left) 2024-09-18T11:30:08.993 INFO:tasks.rados.rados.0.smithi119.stdout:290: set_chunk oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.993 INFO:tasks.rados.rados.0.smithi119.stdout:290: set_chunk oid 41 offset: 710144 length: 407552 target oid smithi11939506-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 710144 snap -1 2024-09-18T11:30:08.993 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.993 INFO:tasks.rados.rados.0.smithi119.stdout:288: finishing set_chunk to oid 39 2024-09-18T11:30:08.994 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 39 set_chunk 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1370112 length: 102400 2024-09-18T11:30:08.994 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 39 v 0 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-18T11:30:08.994 INFO:tasks.rados.rados.0.smithi119.stdout:288: done (15 left) 2024-09-18T11:30:08.994 INFO:tasks.rados.rados.0.smithi119.stdout:291: set_chunk oid 42 target oid 42 low_tier 2024-09-18T11:30:08.994 INFO:tasks.rados.rados.0.smithi119.stdout:291: set_chunk oid 42 offset: 2082816 length: 444928 target oid smithi11939506-42 low_tier offset: 2082816 snap -1 2024-09-18T11:30:08.994 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.994 INFO:tasks.rados.rados.0.smithi119.stdout:245: finishing set_chunk to oid 46 2024-09-18T11:30:08.994 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 9 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-18T11:30:08.994 INFO:tasks.rados.rados.0.smithi119.stdout:245: done (15 left) 2024-09-18T11:30:08.994 INFO:tasks.rados.rados.0.smithi119.stdout:292: set_chunk oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:08.994 INFO:tasks.rados.rados.0.smithi119.stdout:292: set_chunk oid 43 offset: 123392 length: 1060864 target oid smithi11939506-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 123392 snap -1 2024-09-18T11:30:08.994 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.994 INFO:tasks.rados.rados.0.smithi119.stdout:290: finishing set_chunk to oid 41 2024-09-18T11:30:08.994 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 41 set_chunk 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 710144 length: 407552 2024-09-18T11:30:08.994 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 41 v 0 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-18T11:30:08.995 INFO:tasks.rados.rados.0.smithi119.stdout:290: done (15 left) 2024-09-18T11:30:08.995 INFO:tasks.rados.rados.0.smithi119.stdout:293: set_chunk oid 44 target oid 44 low_tier 2024-09-18T11:30:08.995 INFO:tasks.rados.rados.0.smithi119.stdout:293: set_chunk oid 44 offset: 529920 length: 2157056 target oid smithi11939506-44 low_tier offset: 529920 snap -1 2024-09-18T11:30:08.995 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:08.995 INFO:tasks.rados.rados.0.smithi119.stdout:292: finishing set_chunk to oid 43 2024-09-18T11:30:08.995 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 43 set_chunk 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 123392 length: 1060864 2024-09-18T11:30:08.995 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-18T11:30:08.995 INFO:tasks.rados.rados.0.smithi119.stdout:292: done (15 left) 2024-09-18T11:30:08.995 INFO:tasks.rados.rados.0.smithi119.stdout:293 retry set_chunk ! 2024-09-18T11:30:08.995 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 15 2024-09-18T11:30:08.995 INFO:tasks.rados.rados.0.smithi119.stdout:293: finishing set_chunk to oid 44 2024-09-18T11:30:08.995 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 44 set_chunk 44 low_tier returned error code -95 offset: 529920 length: 2157056 2024-09-18T11:30:08.995 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 44 v 0 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-18T11:30:08.995 INFO:tasks.rados.rados.0.smithi119.stdout:293: done (14 left) 2024-09-18T11:30:08.995 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 14 2024-09-18T11:30:08.995 INFO:tasks.rados.rados.0.smithi119.stdout:244: finishing set_chunk to oid 45 2024-09-18T11:30:08.996 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 3 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-18T11:30:08.996 INFO:tasks.rados.rados.0.smithi119.stdout:244: done (13 left) 2024-09-18T11:30:08.996 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 13 2024-09-18T11:30:08.996 INFO:tasks.rados.rados.0.smithi119.stdout:257: finishing set_chunk to oid 8 2024-09-18T11:30:08.996 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 8 v 6 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-18T11:30:08.996 INFO:tasks.rados.rados.0.smithi119.stdout:249: finishing set_chunk to oid 50 2024-09-18T11:30:08.996 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 9 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-18T11:30:08.996 INFO:tasks.rados.rados.0.smithi119.stdout:254: finishing set_chunk to oid 5 2024-09-18T11:30:08.996 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 6 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-18T11:30:08.996 INFO:tasks.rados.rados.0.smithi119.stdout:249: done (12 left) 2024-09-18T11:30:08.996 INFO:tasks.rados.rados.0.smithi119.stdout:254: done (11 left) 2024-09-18T11:30:08.996 INFO:tasks.rados.rados.0.smithi119.stdout:257: done (10 left) 2024-09-18T11:30:08.996 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 10 2024-09-18T11:30:08.996 INFO:tasks.rados.rados.0.smithi119.stdout:248: finishing set_chunk to oid 49 2024-09-18T11:30:08.997 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 18 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-18T11:30:08.997 INFO:tasks.rados.rados.0.smithi119.stdout:248: done (9 left) 2024-09-18T11:30:08.997 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 9 2024-09-18T11:30:08.997 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:08.997 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:09.005 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:08.997+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:09.006 INFO:tasks.rados.rados.0.smithi119.stdout:263: finishing set_chunk to oid 14 2024-09-18T11:30:09.006 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 3 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-18T11:30:09.007 INFO:tasks.rados.rados.0.smithi119.stdout:263: done (8 left) 2024-09-18T11:30:09.007 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 8 2024-09-18T11:30:09.007 INFO:tasks.rados.rados.0.smithi119.stdout:255: finishing set_chunk to oid 6 2024-09-18T11:30:09.007 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 6 v 3 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-18T11:30:09.007 INFO:tasks.rados.rados.0.smithi119.stdout:255: done (7 left) 2024-09-18T11:30:09.007 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 7 2024-09-18T11:30:09.007 INFO:tasks.rados.rados.0.smithi119.stdout:283: finishing set_chunk to oid 34 2024-09-18T11:30:09.007 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 9 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T11:30:09.007 INFO:tasks.rados.rados.0.smithi119.stdout:283: done (6 left) 2024-09-18T11:30:09.007 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 6 2024-09-18T11:30:09.007 INFO:tasks.rados.rados.0.smithi119.stdout:281: finishing set_chunk to oid 32 2024-09-18T11:30:09.007 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 3 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-18T11:30:09.008 INFO:tasks.rados.rados.0.smithi119.stdout:264: finishing set_chunk to oid 15 2024-09-18T11:30:09.008 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 3 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-18T11:30:09.008 INFO:tasks.rados.rados.0.smithi119.stdout:264: done (5 left) 2024-09-18T11:30:09.008 INFO:tasks.rados.rados.0.smithi119.stdout:281: done (4 left) 2024-09-18T11:30:09.008 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 4 2024-09-18T11:30:09.008 INFO:tasks.rados.rados.0.smithi119.stdout:253: finishing set_chunk to oid 4 2024-09-18T11:30:09.008 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 4 v 3 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-18T11:30:09.008 INFO:tasks.rados.rados.0.smithi119.stdout:253: done (3 left) 2024-09-18T11:30:09.008 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 3 2024-09-18T11:30:09.008 INFO:tasks.rados.rados.0.smithi119.stdout:275: finishing set_chunk to oid 26 2024-09-18T11:30:09.008 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 9 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-18T11:30:09.008 INFO:tasks.rados.rados.0.smithi119.stdout:275: done (2 left) 2024-09-18T11:30:09.009 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 2 2024-09-18T11:30:09.009 INFO:tasks.rados.rados.0.smithi119.stdout:289: finishing set_chunk to oid 40 2024-09-18T11:30:09.009 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 6 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-18T11:30:09.009 INFO:tasks.rados.rados.0.smithi119.stdout:289: done (1 left) 2024-09-18T11:30:09.009 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 1 2024-09-18T11:30:09.009 INFO:tasks.rados.rados.0.smithi119.stdout:291: finishing set_chunk to oid 42 2024-09-18T11:30:09.009 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 12 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T11:30:09.009 INFO:tasks.rados.rados.0.smithi119.stdout:291: done (0 left) 2024-09-18T11:30:09.009 INFO:tasks.rados.rados.0.smithi119.stdout:294: set_chunk oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.009 INFO:tasks.rados.rados.0.smithi119.stdout:294: set_chunk oid 45 offset: 1297408 length: 798208 target oid smithi11939506-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1297408 snap -1 2024-09-18T11:30:09.009 INFO:tasks.rados.rados.0.smithi119.stdout:295: set_chunk oid 46 target oid 46 low_tier 2024-09-18T11:30:09.009 INFO:tasks.rados.rados.0.smithi119.stdout:295: set_chunk oid 46 offset: 2416640 length: 142336 target oid smithi11939506-46 low_tier offset: 2416640 snap -1 2024-09-18T11:30:09.009 INFO:tasks.rados.rados.0.smithi119.stdout:296: set_chunk oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.010 INFO:tasks.rados.rados.0.smithi119.stdout:296: set_chunk oid 47 offset: 975360 length: 1432064 target oid smithi11939506-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 975360 snap -1 2024-09-18T11:30:09.010 INFO:tasks.rados.rados.0.smithi119.stdout:297: set_chunk oid 48 target oid 48 low_tier 2024-09-18T11:30:09.010 INFO:tasks.rados.rados.0.smithi119.stdout:297: set_chunk oid 48 offset: 424448 length: 290304 target oid smithi11939506-48 low_tier offset: 424448 snap -1 2024-09-18T11:30:09.010 INFO:tasks.rados.rados.0.smithi119.stdout:298: set_chunk oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.010 INFO:tasks.rados.rados.0.smithi119.stdout:298: set_chunk oid 49 offset: 1002496 length: 941568 target oid smithi11939506-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1002496 snap -1 2024-09-18T11:30:09.010 INFO:tasks.rados.rados.0.smithi119.stdout:299: set_chunk oid 50 target oid 50 low_tier 2024-09-18T11:30:09.010 INFO:tasks.rados.rados.0.smithi119.stdout:299: set_chunk oid 50 offset: 141312 length: 934912 target oid smithi11939506-50 low_tier offset: 141312 snap -1 2024-09-18T11:30:09.010 INFO:tasks.rados.rados.0.smithi119.stdout:300: set_chunk oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.010 INFO:tasks.rados.rados.0.smithi119.stdout:300: set_chunk oid 1 offset: 845312 length: 39424 target oid smithi11939506-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 845312 snap -1 2024-09-18T11:30:09.010 INFO:tasks.rados.rados.0.smithi119.stdout:301: set_chunk oid 2 target oid 2 low_tier 2024-09-18T11:30:09.010 INFO:tasks.rados.rados.0.smithi119.stdout:301: set_chunk oid 2 offset: 1490944 length: 424448 target oid smithi11939506-2 low_tier offset: 1490944 snap -1 2024-09-18T11:30:09.010 INFO:tasks.rados.rados.0.smithi119.stdout:302: set_chunk oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.010 INFO:tasks.rados.rados.0.smithi119.stdout:302: set_chunk oid 3 offset: 181248 length: 186880 target oid smithi11939506-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 181248 snap -1 2024-09-18T11:30:09.011 INFO:tasks.rados.rados.0.smithi119.stdout:303: set_chunk oid 4 target oid 4 low_tier 2024-09-18T11:30:09.011 INFO:tasks.rados.rados.0.smithi119.stdout:303: set_chunk oid 4 offset: 442880 length: 1283072 target oid smithi11939506-4 low_tier offset: 442880 snap -1 2024-09-18T11:30:09.011 INFO:tasks.rados.rados.0.smithi119.stdout:304: set_chunk oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.011 INFO:tasks.rados.rados.0.smithi119.stdout:304: set_chunk oid 5 offset: 1275392 length: 158208 target oid smithi11939506-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1275392 snap -1 2024-09-18T11:30:09.011 INFO:tasks.rados.rados.0.smithi119.stdout:305: set_chunk oid 6 target oid 6 low_tier 2024-09-18T11:30:09.011 INFO:tasks.rados.rados.0.smithi119.stdout:305: set_chunk oid 6 offset: 78336 length: 259584 target oid smithi11939506-6 low_tier offset: 78336 snap -1 2024-09-18T11:30:09.011 INFO:tasks.rados.rados.0.smithi119.stdout:306: set_chunk oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.011 INFO:tasks.rados.rados.0.smithi119.stdout:306: set_chunk oid 7 offset: 1092608 length: 436736 target oid smithi11939506-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1092608 snap -1 2024-09-18T11:30:09.011 INFO:tasks.rados.rados.0.smithi119.stdout:307: set_chunk oid 8 target oid 8 low_tier 2024-09-18T11:30:09.011 INFO:tasks.rados.rados.0.smithi119.stdout:307: set_chunk oid 8 offset: 1276416 length: 5632 target oid smithi11939506-8 low_tier offset: 1276416 snap -1 2024-09-18T11:30:09.011 INFO:tasks.rados.rados.0.smithi119.stdout:308: set_chunk oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.011 INFO:tasks.rados.rados.0.smithi119.stdout:308: set_chunk oid 9 offset: 1172992 length: 134144 target oid smithi11939506-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1172992 snap -1 2024-09-18T11:30:09.012 INFO:tasks.rados.rados.0.smithi119.stdout:309: set_chunk oid 10 target oid 10 low_tier 2024-09-18T11:30:09.012 INFO:tasks.rados.rados.0.smithi119.stdout:309: set_chunk oid 10 offset: 170496 length: 106496 target oid smithi11939506-10 low_tier offset: 170496 snap -1 2024-09-18T11:30:09.012 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.012 INFO:tasks.rados.rados.0.smithi119.stdout:294: finishing set_chunk to oid 45 2024-09-18T11:30:09.012 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 45 set_chunk 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1297408 length: 798208 2024-09-18T11:30:09.012 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 0 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-18T11:30:09.012 INFO:tasks.rados.rados.0.smithi119.stdout:294: done (15 left) 2024-09-18T11:30:09.012 INFO:tasks.rados.rados.0.smithi119.stdout:310: set_chunk oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.012 INFO:tasks.rados.rados.0.smithi119.stdout:310: set_chunk oid 11 offset: 577024 length: 652288 target oid smithi11939506-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 577024 snap -1 2024-09-18T11:30:09.012 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.012 INFO:tasks.rados.rados.0.smithi119.stdout:295: finishing set_chunk to oid 46 2024-09-18T11:30:09.012 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 46 set_chunk 46 low_tier returned error code -95 offset: 2416640 length: 142336 2024-09-18T11:30:09.013 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 0 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-18T11:30:09.013 INFO:tasks.rados.rados.0.smithi119.stdout:296: finishing set_chunk to oid 47 2024-09-18T11:30:09.013 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 47 set_chunk 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 975360 length: 1432064 2024-09-18T11:30:09.013 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 0 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-18T11:30:09.013 INFO:tasks.rados.rados.0.smithi119.stdout:299: finishing set_chunk to oid 50 2024-09-18T11:30:09.013 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 50 set_chunk 50 low_tier returned error code -95 offset: 141312 length: 934912 2024-09-18T11:30:09.013 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 0 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-18T11:30:09.013 INFO:tasks.rados.rados.0.smithi119.stdout:295: done (15 left) 2024-09-18T11:30:09.013 INFO:tasks.rados.rados.0.smithi119.stdout:296: done (14 left) 2024-09-18T11:30:09.013 INFO:tasks.rados.rados.0.smithi119.stdout:299: done (13 left) 2024-09-18T11:30:09.013 INFO:tasks.rados.rados.0.smithi119.stdout:311: set_chunk oid 12 target oid 12 low_tier 2024-09-18T11:30:09.013 INFO:tasks.rados.rados.0.smithi119.stdout:311: set_chunk oid 12 offset: 1017856 length: 202752 target oid smithi11939506-12 low_tier offset: 1017856 snap -1 2024-09-18T11:30:09.013 INFO:tasks.rados.rados.0.smithi119.stdout:312: set_chunk oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.014 INFO:tasks.rados.rados.0.smithi119.stdout:312: set_chunk oid 13 offset: 1383424 length: 471552 target oid smithi11939506-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1383424 snap -1 2024-09-18T11:30:09.014 INFO:tasks.rados.rados.0.smithi119.stdout:313: set_chunk oid 14 target oid 14 low_tier 2024-09-18T11:30:09.014 INFO:tasks.rados.rados.0.smithi119.stdout:313: set_chunk oid 14 offset: 1603584 length: 396800 target oid smithi11939506-14 low_tier offset: 1603584 snap -1 2024-09-18T11:30:09.014 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.014 INFO:tasks.rados.rados.0.smithi119.stdout:297: finishing set_chunk to oid 48 2024-09-18T11:30:09.014 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 48 set_chunk 48 low_tier returned error code -95 offset: 424448 length: 290304 2024-09-18T11:30:09.014 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 48 v 0 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-18T11:30:09.014 INFO:tasks.rados.rados.0.smithi119.stdout:297: done (15 left) 2024-09-18T11:30:09.014 INFO:tasks.rados.rados.0.smithi119.stdout:314: set_chunk oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.014 INFO:tasks.rados.rados.0.smithi119.stdout:314: set_chunk oid 15 offset: 963072 length: 134144 target oid smithi11939506-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 963072 snap -1 2024-09-18T11:30:09.015 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.015 INFO:tasks.rados.rados.0.smithi119.stdout:303: finishing set_chunk to oid 4 2024-09-18T11:30:09.015 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 4 set_chunk 4 low_tier returned error code -95 offset: 442880 length: 1283072 2024-09-18T11:30:09.015 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 4 v 0 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-18T11:30:09.015 INFO:tasks.rados.rados.0.smithi119.stdout:298: finishing set_chunk to oid 49 2024-09-18T11:30:09.015 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 49 set_chunk 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1002496 length: 941568 2024-09-18T11:30:09.015 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 0 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-18T11:30:09.015 INFO:tasks.rados.rados.0.smithi119.stdout:298: done (15 left) 2024-09-18T11:30:09.015 INFO:tasks.rados.rados.0.smithi119.stdout:303: done (14 left) 2024-09-18T11:30:09.015 INFO:tasks.rados.rados.0.smithi119.stdout:315: set_chunk oid 16 target oid 16 low_tier 2024-09-18T11:30:09.015 INFO:tasks.rados.rados.0.smithi119.stdout:310: finishing set_chunk to oid 11 2024-09-18T11:30:09.015 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 11 set_chunk 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 577024 length: 652288 2024-09-18T11:30:09.015 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 11 v 0 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-18T11:30:09.016 INFO:tasks.rados.rados.0.smithi119.stdout:306: finishing set_chunk to oid 7 2024-09-18T11:30:09.016 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 7 set_chunk 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1092608 length: 436736 2024-09-18T11:30:09.016 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 7 v 0 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-18T11:30:09.016 INFO:tasks.rados.rados.0.smithi119.stdout:313: finishing set_chunk to oid 14 2024-09-18T11:30:09.016 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 14 set_chunk 14 low_tier returned error code -95 offset: 1603584 length: 396800 2024-09-18T11:30:09.016 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 0 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-18T11:30:09.016 INFO:tasks.rados.rados.0.smithi119.stdout:311: finishing set_chunk to oid 12 2024-09-18T11:30:09.016 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 12 set_chunk 12 low_tier returned error code -95 offset: 1017856 length: 202752 2024-09-18T11:30:09.016 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 12 v 0 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-18T11:30:09.016 INFO:tasks.rados.rados.0.smithi119.stdout:315: set_chunk oid 16 offset: 28160 length: 1814528 target oid smithi11939506-16 low_tier offset: 28160 snap -1 2024-09-18T11:30:09.016 INFO:tasks.rados.rados.0.smithi119.stdout:306: done (14 left) 2024-09-18T11:30:09.016 INFO:tasks.rados.rados.0.smithi119.stdout:310: done (13 left) 2024-09-18T11:30:09.016 INFO:tasks.rados.rados.0.smithi119.stdout:311: done (12 left) 2024-09-18T11:30:09.016 INFO:tasks.rados.rados.0.smithi119.stdout:313: done (11 left) 2024-09-18T11:30:09.017 INFO:tasks.rados.rados.0.smithi119.stdout:316: set_chunk oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.017 INFO:tasks.rados.rados.0.smithi119.stdout:316: set_chunk oid 17 offset: 286208 length: 1779200 target oid smithi11939506-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 286208 snap -1 2024-09-18T11:30:09.017 INFO:tasks.rados.rados.0.smithi119.stdout:317: set_chunk oid 18 target oid 18 low_tier 2024-09-18T11:30:09.017 INFO:tasks.rados.rados.0.smithi119.stdout:317: set_chunk oid 18 offset: 31232 length: 1988096 target oid smithi11939506-18 low_tier offset: 31232 snap -1 2024-09-18T11:30:09.017 INFO:tasks.rados.rados.0.smithi119.stdout:318: set_chunk oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.017 INFO:tasks.rados.rados.0.smithi119.stdout:318: set_chunk oid 19 offset: 182784 length: 1445888 target oid smithi11939506-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 182784 snap -1 2024-09-18T11:30:09.017 INFO:tasks.rados.rados.0.smithi119.stdout:319: set_chunk oid 20 target oid 20 low_tier 2024-09-18T11:30:09.017 INFO:tasks.rados.rados.0.smithi119.stdout:319: set_chunk oid 20 offset: 399872 length: 1297408 target oid smithi11939506-20 low_tier offset: 399872 snap -1 2024-09-18T11:30:09.017 INFO:tasks.rados.rados.0.smithi119.stdout:320: set_chunk oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.017 INFO:tasks.rados.rados.0.smithi119.stdout:320: set_chunk oid 21 offset: 101888 length: 634880 target oid smithi11939506-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 101888 snap -1 2024-09-18T11:30:09.017 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.017 INFO:tasks.rados.rados.0.smithi119.stdout:304: finishing set_chunk to oid 5 2024-09-18T11:30:09.018 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 5 set_chunk 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1275392 length: 158208 2024-09-18T11:30:09.018 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 0 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-18T11:30:09.018 INFO:tasks.rados.rados.0.smithi119.stdout:307: finishing set_chunk to oid 8 2024-09-18T11:30:09.018 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 8 set_chunk 8 low_tier returned error code -95 offset: 1276416 length: 5632 2024-09-18T11:30:09.018 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 8 v 0 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-18T11:30:09.018 INFO:tasks.rados.rados.0.smithi119.stdout:309: finishing set_chunk to oid 10 2024-09-18T11:30:09.018 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 10 set_chunk 10 low_tier returned error code -95 offset: 170496 length: 106496 2024-09-18T11:30:09.018 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 10 v 0 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-18T11:30:09.018 INFO:tasks.rados.rados.0.smithi119.stdout:304: done (15 left) 2024-09-18T11:30:09.018 INFO:tasks.rados.rados.0.smithi119.stdout:307: done (14 left) 2024-09-18T11:30:09.018 INFO:tasks.rados.rados.0.smithi119.stdout:309: done (13 left) 2024-09-18T11:30:09.018 INFO:tasks.rados.rados.0.smithi119.stdout:321: set_chunk oid 22 target oid 22 low_tier 2024-09-18T11:30:09.018 INFO:tasks.rados.rados.0.smithi119.stdout:321: set_chunk oid 22 offset: 892416 length: 1223680 target oid smithi11939506-22 low_tier offset: 892416 snap -1 2024-09-18T11:30:09.019 INFO:tasks.rados.rados.0.smithi119.stdout:301: finishing set_chunk to oid 2 2024-09-18T11:30:09.019 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 2 set_chunk 2 low_tier returned error code -95 offset: 1490944 length: 424448 2024-09-18T11:30:09.019 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 0 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-18T11:30:09.019 INFO:tasks.rados.rados.0.smithi119.stdout:301: done (13 left) 2024-09-18T11:30:09.019 INFO:tasks.rados.rados.0.smithi119.stdout:322: set_chunk oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.019 INFO:tasks.rados.rados.0.smithi119.stdout:322: set_chunk oid 23 offset: 741376 length: 942592 target oid smithi11939506-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 741376 snap -1 2024-09-18T11:30:09.019 INFO:tasks.rados.rados.0.smithi119.stdout:323: set_chunk oid 24 target oid 24 low_tier 2024-09-18T11:30:09.019 INFO:tasks.rados.rados.0.smithi119.stdout:323: set_chunk oid 24 offset: 1005056 length: 321024 target oid smithi11939506-24 low_tier offset: 1005056 snap -1 2024-09-18T11:30:09.019 INFO:tasks.rados.rados.0.smithi119.stdout:324: set_chunk oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.019 INFO:tasks.rados.rados.0.smithi119.stdout:319: finishing set_chunk to oid 20 2024-09-18T11:30:09.019 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 20 set_chunk 20 low_tier returned error code -95 offset: 399872 length: 1297408 2024-09-18T11:30:09.019 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 20 v 0 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-18T11:30:09.019 INFO:tasks.rados.rados.0.smithi119.stdout:317: finishing set_chunk to oid 18 2024-09-18T11:30:09.019 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 18 set_chunk 18 low_tier returned error code -95 offset: 31232 length: 1988096 2024-09-18T11:30:09.019 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 0 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-18T11:30:09.020 INFO:tasks.rados.rados.0.smithi119.stdout:324: set_chunk oid 25 offset: 267776 length: 623104 target oid smithi11939506-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 267776 snap -1 2024-09-18T11:30:09.020 INFO:tasks.rados.rados.0.smithi119.stdout:317: done (15 left) 2024-09-18T11:30:09.020 INFO:tasks.rados.rados.0.smithi119.stdout:319: done (14 left) 2024-09-18T11:30:09.020 INFO:tasks.rados.rados.0.smithi119.stdout:325: set_chunk oid 26 target oid 26 low_tier 2024-09-18T11:30:09.020 INFO:tasks.rados.rados.0.smithi119.stdout:325: set_chunk oid 26 offset: 994304 length: 121856 target oid smithi11939506-26 low_tier offset: 994304 snap -1 2024-09-18T11:30:09.020 INFO:tasks.rados.rados.0.smithi119.stdout:326: set_chunk oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.020 INFO:tasks.rados.rados.0.smithi119.stdout:326: set_chunk oid 27 offset: 1118720 length: 365056 target oid smithi11939506-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1118720 snap -1 2024-09-18T11:30:09.020 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.020 INFO:tasks.rados.rados.0.smithi119.stdout:322: finishing set_chunk to oid 23 2024-09-18T11:30:09.020 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 23 set_chunk 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 741376 length: 942592 2024-09-18T11:30:09.020 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 0 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-18T11:30:09.020 INFO:tasks.rados.rados.0.smithi119.stdout:314: finishing set_chunk to oid 15 2024-09-18T11:30:09.020 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 15 set_chunk 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 963072 length: 134144 2024-09-18T11:30:09.020 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 0 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-18T11:30:09.021 INFO:tasks.rados.rados.0.smithi119.stdout:316: finishing set_chunk to oid 17 2024-09-18T11:30:09.021 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 17 set_chunk 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 286208 length: 1779200 2024-09-18T11:30:09.021 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 0 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-18T11:30:09.021 INFO:tasks.rados.rados.0.smithi119.stdout:324: finishing set_chunk to oid 25 2024-09-18T11:30:09.021 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 25 set_chunk 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 267776 length: 623104 2024-09-18T11:30:09.021 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 0 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-18T11:30:09.021 INFO:tasks.rados.rados.0.smithi119.stdout:326: finishing set_chunk to oid 27 2024-09-18T11:30:09.021 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 27 set_chunk 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1118720 length: 365056 2024-09-18T11:30:09.021 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 0 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T11:30:09.021 INFO:tasks.rados.rados.0.smithi119.stdout:314: done (15 left) 2024-09-18T11:30:09.021 INFO:tasks.rados.rados.0.smithi119.stdout:316: done (14 left) 2024-09-18T11:30:09.021 INFO:tasks.rados.rados.0.smithi119.stdout:322: done (13 left) 2024-09-18T11:30:09.021 INFO:tasks.rados.rados.0.smithi119.stdout:324: done (12 left) 2024-09-18T11:30:09.021 INFO:tasks.rados.rados.0.smithi119.stdout:326: done (11 left) 2024-09-18T11:30:09.021 INFO:tasks.rados.rados.0.smithi119.stdout:327: set_chunk oid 28 target oid 28 low_tier 2024-09-18T11:30:09.022 INFO:tasks.rados.rados.0.smithi119.stdout:327: set_chunk oid 28 offset: 1380352 length: 1025024 target oid smithi11939506-28 low_tier offset: 1380352 snap -1 2024-09-18T11:30:09.022 INFO:tasks.rados.rados.0.smithi119.stdout:328: set_chunk oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.022 INFO:tasks.rados.rados.0.smithi119.stdout:328: set_chunk oid 29 offset: 1008128 length: 813568 target oid smithi11939506-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1008128 snap -1 2024-09-18T11:30:09.022 INFO:tasks.rados.rados.0.smithi119.stdout:329: set_chunk oid 30 target oid 30 low_tier 2024-09-18T11:30:09.022 INFO:tasks.rados.rados.0.smithi119.stdout:329: set_chunk oid 30 offset: 481792 length: 418816 target oid smithi11939506-30 low_tier offset: 481792 snap -1 2024-09-18T11:30:09.022 INFO:tasks.rados.rados.0.smithi119.stdout:330: set_chunk oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.022 INFO:tasks.rados.rados.0.smithi119.stdout:330: set_chunk oid 31 offset: 1479168 length: 2560 target oid smithi11939506-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1479168 snap -1 2024-09-18T11:30:09.022 INFO:tasks.rados.rados.0.smithi119.stdout:331: set_chunk oid 32 target oid 32 low_tier 2024-09-18T11:30:09.022 INFO:tasks.rados.rados.0.smithi119.stdout:331: set_chunk oid 32 offset: 325120 length: 1108480 target oid smithi11939506-32 low_tier offset: 325120 snap -1 2024-09-18T11:30:09.022 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.022 INFO:tasks.rados.rados.0.smithi119.stdout:315: finishing set_chunk to oid 16 2024-09-18T11:30:09.022 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 16 set_chunk 16 low_tier returned error code -95 offset: 28160 length: 1814528 2024-09-18T11:30:09.022 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 0 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-18T11:30:09.023 INFO:tasks.rados.rados.0.smithi119.stdout:318: finishing set_chunk to oid 19 2024-09-18T11:30:09.023 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 19 set_chunk 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 182784 length: 1445888 2024-09-18T11:30:09.023 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 0 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-18T11:30:09.023 INFO:tasks.rados.rados.0.smithi119.stdout:315: done (15 left) 2024-09-18T11:30:09.023 INFO:tasks.rados.rados.0.smithi119.stdout:318: done (14 left) 2024-09-18T11:30:09.023 INFO:tasks.rados.rados.0.smithi119.stdout:332: set_chunk oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.023 INFO:tasks.rados.rados.0.smithi119.stdout:332: set_chunk oid 33 offset: 277504 length: 969728 target oid smithi11939506-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 277504 snap -1 2024-09-18T11:30:09.023 INFO:tasks.rados.rados.0.smithi119.stdout:333: set_chunk oid 34 target oid 34 low_tier 2024-09-18T11:30:09.023 INFO:tasks.rados.rados.0.smithi119.stdout:333: set_chunk oid 34 offset: 764928 length: 828928 target oid smithi11939506-34 low_tier offset: 764928 snap -1 2024-09-18T11:30:09.023 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.023 INFO:tasks.rados.rados.0.smithi119.stdout:331: finishing set_chunk to oid 32 2024-09-18T11:30:09.023 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 32 set_chunk 32 low_tier returned error code -95 offset: 325120 length: 1108480 2024-09-18T11:30:09.023 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 0 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-18T11:30:09.023 INFO:tasks.rados.rados.0.smithi119.stdout:331: done (15 left) 2024-09-18T11:30:09.024 INFO:tasks.rados.rados.0.smithi119.stdout:334: set_chunk oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.024 INFO:tasks.rados.rados.0.smithi119.stdout:334: set_chunk oid 35 offset: 1892864 length: 515072 target oid smithi11939506-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1892864 snap -1 2024-09-18T11:30:09.024 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.024 INFO:tasks.rados.rados.0.smithi119.stdout:321: finishing set_chunk to oid 22 2024-09-18T11:30:09.024 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 22 set_chunk 22 low_tier returned error code -95 offset: 892416 length: 1223680 2024-09-18T11:30:09.024 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 0 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-18T11:30:09.024 INFO:tasks.rados.rados.0.smithi119.stdout:327: finishing set_chunk to oid 28 2024-09-18T11:30:09.024 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 28 set_chunk 28 low_tier returned error code -95 offset: 1380352 length: 1025024 2024-09-18T11:30:09.024 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 28 v 0 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-18T11:30:09.024 INFO:tasks.rados.rados.0.smithi119.stdout:321: done (15 left) 2024-09-18T11:30:09.024 INFO:tasks.rados.rados.0.smithi119.stdout:327: done (14 left) 2024-09-18T11:30:09.024 INFO:tasks.rados.rados.0.smithi119.stdout:335: set_chunk oid 36 target oid 36 low_tier 2024-09-18T11:30:09.025 INFO:tasks.rados.rados.0.smithi119.stdout:335: set_chunk oid 36 offset: 1193472 length: 413184 target oid smithi11939506-36 low_tier offset: 1193472 snap -1 2024-09-18T11:30:09.025 INFO:tasks.rados.rados.0.smithi119.stdout:336: set_chunk oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.025 INFO:tasks.rados.rados.0.smithi119.stdout:336: set_chunk oid 37 offset: 812544 length: 292864 target oid smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 812544 snap -1 2024-09-18T11:30:09.025 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.025 INFO:tasks.rados.rados.0.smithi119.stdout:333: finishing set_chunk to oid 34 2024-09-18T11:30:09.025 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 34 set_chunk 34 low_tier returned error code -95 offset: 764928 length: 828928 2024-09-18T11:30:09.025 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 0 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T11:30:09.025 INFO:tasks.rados.rados.0.smithi119.stdout:333: done (15 left) 2024-09-18T11:30:09.025 INFO:tasks.rados.rados.0.smithi119.stdout:337: set_chunk oid 38 target oid 38 low_tier 2024-09-18T11:30:09.025 INFO:tasks.rados.rados.0.smithi119.stdout:337: set_chunk oid 38 offset: 336384 length: 173568 target oid smithi11939506-38 low_tier offset: 336384 snap -1 2024-09-18T11:30:09.025 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.025 INFO:tasks.rados.rados.0.smithi119.stdout:329: finishing set_chunk to oid 30 2024-09-18T11:30:09.025 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 30 set_chunk 30 low_tier returned error code -95 offset: 481792 length: 418816 2024-09-18T11:30:09.025 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 30 v 0 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-18T11:30:09.026 INFO:tasks.rados.rados.0.smithi119.stdout:329: done (15 left) 2024-09-18T11:30:09.026 INFO:tasks.rados.rados.0.smithi119.stdout:338: set_chunk oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.026 INFO:tasks.rados.rados.0.smithi119.stdout:338: set_chunk oid 39 offset: 45056 length: 1915904 target oid smithi11939506-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 45056 snap -1 2024-09-18T11:30:09.026 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.026 INFO:tasks.rados.rados.0.smithi119.stdout:332: finishing set_chunk to oid 33 2024-09-18T11:30:09.026 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 33 set_chunk 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 277504 length: 969728 2024-09-18T11:30:09.026 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 33 v 0 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-18T11:30:09.026 INFO:tasks.rados.rados.0.smithi119.stdout:332: done (15 left) 2024-09-18T11:30:09.026 INFO:tasks.rados.rados.0.smithi119.stdout:339: set_chunk oid 40 target oid 40 low_tier 2024-09-18T11:30:09.026 INFO:tasks.rados.rados.0.smithi119.stdout:339: set_chunk oid 40 offset: 508416 length: 1698816 target oid smithi11939506-40 low_tier offset: 508416 snap -1 2024-09-18T11:30:09.026 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.026 INFO:tasks.rados.rados.0.smithi119.stdout:328: finishing set_chunk to oid 29 2024-09-18T11:30:09.026 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 29 set_chunk 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1008128 length: 813568 2024-09-18T11:30:09.027 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 0 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-18T11:30:09.027 INFO:tasks.rados.rados.0.smithi119.stdout:330: finishing set_chunk to oid 31 2024-09-18T11:30:09.027 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 31 set_chunk 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1479168 length: 2560 2024-09-18T11:30:09.027 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 0 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-18T11:30:09.027 INFO:tasks.rados.rados.0.smithi119.stdout:335: finishing set_chunk to oid 36 2024-09-18T11:30:09.027 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 36 set_chunk 36 low_tier returned error code -95 offset: 1193472 length: 413184 2024-09-18T11:30:09.027 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 36 v 0 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-18T11:30:09.027 INFO:tasks.rados.rados.0.smithi119.stdout:328: done (15 left) 2024-09-18T11:30:09.027 INFO:tasks.rados.rados.0.smithi119.stdout:330: done (14 left) 2024-09-18T11:30:09.027 INFO:tasks.rados.rados.0.smithi119.stdout:335: done (13 left) 2024-09-18T11:30:09.027 INFO:tasks.rados.rados.0.smithi119.stdout:340: set_chunk oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.027 INFO:tasks.rados.rados.0.smithi119.stdout:340: set_chunk oid 41 offset: 975360 length: 1114624 target oid smithi11939506-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 975360 snap -1 2024-09-18T11:30:09.027 INFO:tasks.rados.rados.0.smithi119.stdout:341: set_chunk oid 42 target oid 42 low_tier 2024-09-18T11:30:09.028 INFO:tasks.rados.rados.0.smithi119.stdout:341: set_chunk oid 42 offset: 2285056 length: 362496 target oid smithi11939506-42 low_tier offset: 2285056 snap -1 2024-09-18T11:30:09.028 INFO:tasks.rados.rados.0.smithi119.stdout:342: set_chunk oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.028 INFO:tasks.rados.rados.0.smithi119.stdout:342: set_chunk oid 43 offset: 259584 length: 1233920 target oid smithi11939506-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 259584 snap -1 2024-09-18T11:30:09.028 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.028 INFO:tasks.rados.rados.0.smithi119.stdout:339: finishing set_chunk to oid 40 2024-09-18T11:30:09.028 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 40 set_chunk 40 low_tier returned error code -95 offset: 508416 length: 1698816 2024-09-18T11:30:09.028 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 0 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-18T11:30:09.028 INFO:tasks.rados.rados.0.smithi119.stdout:338: finishing set_chunk to oid 39 2024-09-18T11:30:09.028 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 39 set_chunk 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 45056 length: 1915904 2024-09-18T11:30:09.028 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 39 v 0 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-18T11:30:09.028 INFO:tasks.rados.rados.0.smithi119.stdout:338: done (15 left) 2024-09-18T11:30:09.028 INFO:tasks.rados.rados.0.smithi119.stdout:339: done (14 left) 2024-09-18T11:30:09.029 INFO:tasks.rados.rados.0.smithi119.stdout:343: set_chunk oid 44 target oid 44 low_tier 2024-09-18T11:30:09.029 INFO:tasks.rados.rados.0.smithi119.stdout:343: set_chunk oid 44 offset: 927744 length: 1207296 target oid smithi11939506-44 low_tier offset: 927744 snap -1 2024-09-18T11:30:09.029 INFO:tasks.rados.rados.0.smithi119.stdout:344: set_chunk oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.029 INFO:tasks.rados.rados.0.smithi119.stdout:344: set_chunk oid 45 offset: 983552 length: 711680 target oid smithi11939506-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 983552 snap -1 2024-09-18T11:30:09.029 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.029 INFO:tasks.rados.rados.0.smithi119.stdout:340: finishing set_chunk to oid 41 2024-09-18T11:30:09.029 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 41 set_chunk 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 975360 length: 1114624 2024-09-18T11:30:09.029 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 41 v 0 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-18T11:30:09.029 INFO:tasks.rados.rados.0.smithi119.stdout:341: finishing set_chunk to oid 42 2024-09-18T11:30:09.029 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 42 set_chunk 42 low_tier returned error code -95 offset: 2285056 length: 362496 2024-09-18T11:30:09.029 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 0 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T11:30:09.029 INFO:tasks.rados.rados.0.smithi119.stdout:340: done (15 left) 2024-09-18T11:30:09.030 INFO:tasks.rados.rados.0.smithi119.stdout:341: done (14 left) 2024-09-18T11:30:09.030 INFO:tasks.rados.rados.0.smithi119.stdout:345: set_chunk oid 46 target oid 46 low_tier 2024-09-18T11:30:09.030 INFO:tasks.rados.rados.0.smithi119.stdout:345: set_chunk oid 46 offset: 104960 length: 2566656 target oid smithi11939506-46 low_tier offset: 104960 snap -1 2024-09-18T11:30:09.030 INFO:tasks.rados.rados.0.smithi119.stdout:346: set_chunk oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.030 INFO:tasks.rados.rados.0.smithi119.stdout:346: set_chunk oid 47 offset: 746496 length: 1649152 target oid smithi11939506-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 746496 snap -1 2024-09-18T11:30:09.030 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.030 INFO:tasks.rados.rados.0.smithi119.stdout:334: finishing set_chunk to oid 35 2024-09-18T11:30:09.030 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 35 set_chunk 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1892864 length: 515072 2024-09-18T11:30:09.030 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 0 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-18T11:30:09.030 INFO:tasks.rados.rados.0.smithi119.stdout:342: finishing set_chunk to oid 43 2024-09-18T11:30:09.030 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 43 set_chunk 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 259584 length: 1233920 2024-09-18T11:30:09.030 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-18T11:30:09.031 INFO:tasks.rados.rados.0.smithi119.stdout:344: finishing set_chunk to oid 45 2024-09-18T11:30:09.031 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 45 set_chunk 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 983552 length: 711680 2024-09-18T11:30:09.031 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 0 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-18T11:30:09.031 INFO:tasks.rados.rados.0.smithi119.stdout:334: done (15 left) 2024-09-18T11:30:09.031 INFO:tasks.rados.rados.0.smithi119.stdout:342: done (14 left) 2024-09-18T11:30:09.031 INFO:tasks.rados.rados.0.smithi119.stdout:344: done (13 left) 2024-09-18T11:30:09.031 INFO:tasks.rados.rados.0.smithi119.stdout:347: set_chunk oid 48 target oid 48 low_tier 2024-09-18T11:30:09.031 INFO:tasks.rados.rados.0.smithi119.stdout:347: set_chunk oid 48 offset: 377856 length: 2238464 target oid smithi11939506-48 low_tier offset: 377856 snap -1 2024-09-18T11:30:09.031 INFO:tasks.rados.rados.0.smithi119.stdout:348: set_chunk oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.031 INFO:tasks.rados.rados.0.smithi119.stdout:348: set_chunk oid 49 offset: 286208 length: 1575424 target oid smithi11939506-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 286208 snap -1 2024-09-18T11:30:09.031 INFO:tasks.rados.rados.0.smithi119.stdout:349: set_chunk oid 50 target oid 50 low_tier 2024-09-18T11:30:09.032 INFO:tasks.rados.rados.0.smithi119.stdout:349: set_chunk oid 50 offset: 483328 length: 103424 target oid smithi11939506-50 low_tier offset: 483328 snap -1 2024-09-18T11:30:09.032 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.032 INFO:tasks.rados.rados.0.smithi119.stdout:343: finishing set_chunk to oid 44 2024-09-18T11:30:09.032 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 44 set_chunk 44 low_tier returned error code -95 offset: 927744 length: 1207296 2024-09-18T11:30:09.032 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 44 v 0 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-18T11:30:09.032 INFO:tasks.rados.rados.0.smithi119.stdout:346: finishing set_chunk to oid 47 2024-09-18T11:30:09.032 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 47 set_chunk 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 746496 length: 1649152 2024-09-18T11:30:09.032 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 0 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-18T11:30:09.032 INFO:tasks.rados.rados.0.smithi119.stdout:345: finishing set_chunk to oid 46 2024-09-18T11:30:09.032 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 46 set_chunk 46 low_tier returned error code -95 offset: 104960 length: 2566656 2024-09-18T11:30:09.032 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 0 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-18T11:30:09.033 INFO:tasks.rados.rados.0.smithi119.stdout:347: finishing set_chunk to oid 48 2024-09-18T11:30:09.033 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 48 set_chunk 48 low_tier returned error code -95 offset: 377856 length: 2238464 2024-09-18T11:30:09.033 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 48 v 0 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-18T11:30:09.033 INFO:tasks.rados.rados.0.smithi119.stdout:343: done (15 left) 2024-09-18T11:30:09.033 INFO:tasks.rados.rados.0.smithi119.stdout:345: done (14 left) 2024-09-18T11:30:09.033 INFO:tasks.rados.rados.0.smithi119.stdout:346: done (13 left) 2024-09-18T11:30:09.033 INFO:tasks.rados.rados.0.smithi119.stdout:347: done (12 left) 2024-09-18T11:30:09.033 INFO:tasks.rados.rados.0.smithi119.stdout:349 retry set_chunk ! 2024-09-18T11:30:09.033 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 12 2024-09-18T11:30:09.033 INFO:tasks.rados.rados.0.smithi119.stdout:348: finishing set_chunk to oid 49 2024-09-18T11:30:09.033 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 49 set_chunk 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 286208 length: 1575424 2024-09-18T11:30:09.033 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 0 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-18T11:30:09.034 INFO:tasks.rados.rados.0.smithi119.stdout:348: done (11 left) 2024-09-18T11:30:09.034 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 11 2024-09-18T11:30:09.034 INFO:tasks.rados.rados.0.smithi119.stdout:305: finishing set_chunk to oid 6 2024-09-18T11:30:09.034 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 6 v 3 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-18T11:30:09.034 INFO:tasks.rados.rados.0.smithi119.stdout:305: done (10 left) 2024-09-18T11:30:09.034 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 10 2024-09-18T11:30:09.035 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:30:09.086 INFO:tasks.rados.rados.0.smithi119.stdout:325: finishing set_chunk to oid 26 2024-09-18T11:30:09.086 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 9 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-18T11:30:09.086 INFO:tasks.rados.rados.0.smithi119.stdout:325: done (9 left) 2024-09-18T11:30:09.086 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 9 2024-09-18T11:30:09.086 INFO:tasks.rados.rados.0.smithi119.stdout:300: finishing set_chunk to oid 1 2024-09-18T11:30:09.086 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-18T11:30:09.086 INFO:tasks.rados.rados.0.smithi119.stdout:300: done (8 left) 2024-09-18T11:30:09.086 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 8 2024-09-18T11:30:09.086 INFO:tasks.rados.rados.0.smithi119.stdout:302: finishing set_chunk to oid 3 2024-09-18T11:30:09.086 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 3 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-18T11:30:09.087 INFO:tasks.rados.rados.0.smithi119.stdout:302: done (7 left) 2024-09-18T11:30:09.087 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 7 2024-09-18T11:30:09.087 INFO:tasks.rados.rados.0.smithi119.stdout:308: finishing set_chunk to oid 9 2024-09-18T11:30:09.087 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 6 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T11:30:09.087 INFO:tasks.rados.rados.0.smithi119.stdout:308: done (6 left) 2024-09-18T11:30:09.087 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 6 2024-09-18T11:30:09.087 INFO:tasks.rados.rados.0.smithi119.stdout:312: finishing set_chunk to oid 13 2024-09-18T11:30:09.087 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 13 v 9 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-18T11:30:09.087 INFO:tasks.rados.rados.0.smithi119.stdout:336: finishing set_chunk to oid 37 2024-09-18T11:30:09.087 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 6 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-18T11:30:09.087 INFO:tasks.rados.rados.0.smithi119.stdout:312: done (5 left) 2024-09-18T11:30:09.087 INFO:tasks.rados.rados.0.smithi119.stdout:336: done (4 left) 2024-09-18T11:30:09.087 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 4 2024-09-18T11:30:09.088 INFO:tasks.rados.rados.0.smithi119.stdout:323: finishing set_chunk to oid 24 2024-09-18T11:30:09.088 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 9 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-18T11:30:09.088 INFO:tasks.rados.rados.0.smithi119.stdout:323: done (3 left) 2024-09-18T11:30:09.088 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 3 2024-09-18T11:30:09.088 INFO:tasks.rados.rados.0.smithi119.stdout:320: finishing set_chunk to oid 21 2024-09-18T11:30:09.088 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 6 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-18T11:30:09.088 INFO:tasks.rados.rados.0.smithi119.stdout:337: finishing set_chunk to oid 38 2024-09-18T11:30:09.088 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 9 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-18T11:30:09.088 INFO:tasks.rados.rados.0.smithi119.stdout:320: done (2 left) 2024-09-18T11:30:09.088 INFO:tasks.rados.rados.0.smithi119.stdout:337: done (1 left) 2024-09-18T11:30:09.088 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 1 2024-09-18T11:30:09.088 INFO:tasks.rados.rados.0.smithi119.stdout:349: finishing set_chunk to oid 50 2024-09-18T11:30:09.089 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 9 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-18T11:30:09.089 INFO:tasks.rados.rados.0.smithi119.stdout:349: done (0 left) 2024-09-18T11:30:09.089 INFO:tasks.rados.rados.0.smithi119.stdout:350: set_chunk oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.089 INFO:tasks.rados.rados.0.smithi119.stdout:350: set_chunk oid 1 offset: 761344 length: 1075200 target oid smithi11939506-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 761344 snap -1 2024-09-18T11:30:09.089 INFO:tasks.rados.rados.0.smithi119.stdout:351: set_chunk oid 2 target oid 2 low_tier 2024-09-18T11:30:09.089 INFO:tasks.rados.rados.0.smithi119.stdout:351: set_chunk oid 2 offset: 171520 length: 1762816 target oid smithi11939506-2 low_tier offset: 171520 snap -1 2024-09-18T11:30:09.089 INFO:tasks.rados.rados.0.smithi119.stdout:352: set_chunk oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.089 INFO:tasks.rados.rados.0.smithi119.stdout:352: set_chunk oid 3 offset: 476672 length: 74752 target oid smithi11939506-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 476672 snap -1 2024-09-18T11:30:09.089 INFO:tasks.rados.rados.0.smithi119.stdout:353: set_chunk oid 4 target oid 4 low_tier 2024-09-18T11:30:09.089 INFO:tasks.rados.rados.0.smithi119.stdout:353: set_chunk oid 4 offset: 545792 length: 733696 target oid smithi11939506-4 low_tier offset: 545792 snap -1 2024-09-18T11:30:09.089 INFO:tasks.rados.rados.0.smithi119.stdout:354: set_chunk oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.089 INFO:tasks.rados.rados.0.smithi119.stdout:354: set_chunk oid 5 offset: 273408 length: 1561088 target oid smithi11939506-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 273408 snap -1 2024-09-18T11:30:09.089 INFO:tasks.rados.rados.0.smithi119.stdout:355: set_chunk oid 6 target oid 6 low_tier 2024-09-18T11:30:09.090 INFO:tasks.rados.rados.0.smithi119.stdout:355: set_chunk oid 6 offset: 340480 length: 1515520 target oid smithi11939506-6 low_tier offset: 340480 snap -1 2024-09-18T11:30:09.090 INFO:tasks.rados.rados.0.smithi119.stdout:356: set_chunk oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.090 INFO:tasks.rados.rados.0.smithi119.stdout:356: set_chunk oid 7 offset: 1143296 length: 577536 target oid smithi11939506-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1143296 snap -1 2024-09-18T11:30:09.090 INFO:tasks.rados.rados.0.smithi119.stdout:357: set_chunk oid 8 target oid 8 low_tier 2024-09-18T11:30:09.090 INFO:tasks.rados.rados.0.smithi119.stdout:357: set_chunk oid 8 offset: 320000 length: 803840 target oid smithi11939506-8 low_tier offset: 320000 snap -1 2024-09-18T11:30:09.090 INFO:tasks.rados.rados.0.smithi119.stdout:358: set_chunk oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.090 INFO:tasks.rados.rados.0.smithi119.stdout:358: set_chunk oid 9 offset: 669696 length: 915968 target oid smithi11939506-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 669696 snap -1 2024-09-18T11:30:09.090 INFO:tasks.rados.rados.0.smithi119.stdout:359: set_chunk oid 10 target oid 10 low_tier 2024-09-18T11:30:09.090 INFO:tasks.rados.rados.0.smithi119.stdout:359: set_chunk oid 10 offset: 522752 length: 1521152 target oid smithi11939506-10 low_tier offset: 522752 snap -1 2024-09-18T11:30:09.090 INFO:tasks.rados.rados.0.smithi119.stdout:360: set_chunk oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.090 INFO:tasks.rados.rados.0.smithi119.stdout:360: set_chunk oid 11 offset: 955904 length: 256512 target oid smithi11939506-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 955904 snap -1 2024-09-18T11:30:09.090 INFO:tasks.rados.rados.0.smithi119.stdout:361: set_chunk oid 12 target oid 12 low_tier 2024-09-18T11:30:09.091 INFO:tasks.rados.rados.0.smithi119.stdout:361: set_chunk oid 12 offset: 829440 length: 810496 target oid smithi11939506-12 low_tier offset: 829440 snap -1 2024-09-18T11:30:09.091 INFO:tasks.rados.rados.0.smithi119.stdout:362: set_chunk oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.091 INFO:tasks.rados.rados.0.smithi119.stdout:362: set_chunk oid 13 offset: 1340928 length: 696832 target oid smithi11939506-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1340928 snap -1 2024-09-18T11:30:09.091 INFO:tasks.rados.rados.0.smithi119.stdout:363: set_chunk oid 14 target oid 14 low_tier 2024-09-18T11:30:09.091 INFO:tasks.rados.rados.0.smithi119.stdout:363: set_chunk oid 14 offset: 1400320 length: 643072 target oid smithi11939506-14 low_tier offset: 1400320 snap -1 2024-09-18T11:30:09.091 INFO:tasks.rados.rados.0.smithi119.stdout:364: set_chunk oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.091 INFO:tasks.rados.rados.0.smithi119.stdout:364: set_chunk oid 15 offset: 506368 length: 1585152 target oid smithi11939506-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 506368 snap -1 2024-09-18T11:30:09.091 INFO:tasks.rados.rados.0.smithi119.stdout:365: set_chunk oid 16 target oid 16 low_tier 2024-09-18T11:30:09.091 INFO:tasks.rados.rados.0.smithi119.stdout:365: set_chunk oid 16 offset: 360960 length: 1687552 target oid smithi11939506-16 low_tier offset: 360960 snap -1 2024-09-18T11:30:09.091 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.091 INFO:tasks.rados.rados.0.smithi119.stdout:351: finishing set_chunk to oid 2 2024-09-18T11:30:09.091 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 2 set_chunk 2 low_tier returned error code -95 offset: 171520 length: 1762816 2024-09-18T11:30:09.092 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 0 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-18T11:30:09.092 INFO:tasks.rados.rados.0.smithi119.stdout:350: finishing set_chunk to oid 1 2024-09-18T11:30:09.092 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 1 set_chunk 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 761344 length: 1075200 2024-09-18T11:30:09.092 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 0 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-18T11:30:09.092 INFO:tasks.rados.rados.0.smithi119.stdout:352: finishing set_chunk to oid 3 2024-09-18T11:30:09.092 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 3 set_chunk 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 476672 length: 74752 2024-09-18T11:30:09.092 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 0 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-18T11:30:09.092 INFO:tasks.rados.rados.0.smithi119.stdout:353: finishing set_chunk to oid 4 2024-09-18T11:30:09.092 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 4 set_chunk 4 low_tier returned error code -95 offset: 545792 length: 733696 2024-09-18T11:30:09.092 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 4 v 0 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-18T11:30:09.092 INFO:tasks.rados.rados.0.smithi119.stdout:350: done (15 left) 2024-09-18T11:30:09.093 INFO:tasks.rados.rados.0.smithi119.stdout:351: done (14 left) 2024-09-18T11:30:09.093 INFO:tasks.rados.rados.0.smithi119.stdout:352: done (13 left) 2024-09-18T11:30:09.093 INFO:tasks.rados.rados.0.smithi119.stdout:353: done (12 left) 2024-09-18T11:30:09.093 INFO:tasks.rados.rados.0.smithi119.stdout:366: set_chunk oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.093 INFO:tasks.rados.rados.0.smithi119.stdout:366: set_chunk oid 17 offset: 1418752 length: 405504 target oid smithi11939506-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1418752 snap -1 2024-09-18T11:30:09.093 INFO:tasks.rados.rados.0.smithi119.stdout:367: set_chunk oid 18 target oid 18 low_tier 2024-09-18T11:30:09.093 INFO:tasks.rados.rados.0.smithi119.stdout:367: set_chunk oid 18 offset: 1634304 length: 167424 target oid smithi11939506-18 low_tier offset: 1634304 snap -1 2024-09-18T11:30:09.093 INFO:tasks.rados.rados.0.smithi119.stdout:368: set_chunk oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.093 INFO:tasks.rados.rados.0.smithi119.stdout:368: set_chunk oid 19 offset: 1457664 length: 471552 target oid smithi11939506-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1457664 snap -1 2024-09-18T11:30:09.093 INFO:tasks.rados.rados.0.smithi119.stdout:369: set_chunk oid 20 target oid 20 low_tier 2024-09-18T11:30:09.093 INFO:tasks.rados.rados.0.smithi119.stdout:369: set_chunk oid 20 offset: 112640 length: 1752064 target oid smithi11939506-20 low_tier offset: 112640 snap -1 2024-09-18T11:30:09.093 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.093 INFO:tasks.rados.rados.0.smithi119.stdout:355: finishing set_chunk to oid 6 2024-09-18T11:30:09.093 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 6 set_chunk 6 low_tier returned error code -95 offset: 340480 length: 1515520 2024-09-18T11:30:09.094 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 6 v 0 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-18T11:30:09.094 INFO:tasks.rados.rados.0.smithi119.stdout:360: finishing set_chunk to oid 11 2024-09-18T11:30:09.094 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 11 set_chunk 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 955904 length: 256512 2024-09-18T11:30:09.094 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 11 v 0 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-18T11:30:09.094 INFO:tasks.rados.rados.0.smithi119.stdout:355: done (15 left) 2024-09-18T11:30:09.094 INFO:tasks.rados.rados.0.smithi119.stdout:360: done (14 left) 2024-09-18T11:30:09.094 INFO:tasks.rados.rados.0.smithi119.stdout:370: set_chunk oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.094 INFO:tasks.rados.rados.0.smithi119.stdout:370: set_chunk oid 21 offset: 1205760 length: 184320 target oid smithi11939506-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1205760 snap -1 2024-09-18T11:30:09.094 INFO:tasks.rados.rados.0.smithi119.stdout:371: set_chunk oid 22 target oid 22 low_tier 2024-09-18T11:30:09.094 INFO:tasks.rados.rados.0.smithi119.stdout:371: set_chunk oid 22 offset: 1279488 length: 669696 target oid smithi11939506-22 low_tier offset: 1279488 snap -1 2024-09-18T11:30:09.094 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.094 INFO:tasks.rados.rados.0.smithi119.stdout:357: finishing set_chunk to oid 8 2024-09-18T11:30:09.094 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 8 set_chunk 8 low_tier returned error code -95 offset: 320000 length: 803840 2024-09-18T11:30:09.094 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 8 v 0 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-18T11:30:09.095 INFO:tasks.rados.rados.0.smithi119.stdout:354: finishing set_chunk to oid 5 2024-09-18T11:30:09.095 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 5 set_chunk 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 273408 length: 1561088 2024-09-18T11:30:09.095 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 0 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-18T11:30:09.095 INFO:tasks.rados.rados.0.smithi119.stdout:354: done (15 left) 2024-09-18T11:30:09.095 INFO:tasks.rados.rados.0.smithi119.stdout:357: done (14 left) 2024-09-18T11:30:09.095 INFO:tasks.rados.rados.0.smithi119.stdout:372: set_chunk oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.095 INFO:tasks.rados.rados.0.smithi119.stdout:372: set_chunk oid 23 offset: 403968 length: 159744 target oid smithi11939506-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 403968 snap -1 2024-09-18T11:30:09.095 INFO:tasks.rados.rados.0.smithi119.stdout:373: set_chunk oid 24 target oid 24 low_tier 2024-09-18T11:30:09.095 INFO:tasks.rados.rados.0.smithi119.stdout:373: set_chunk oid 24 offset: 1769472 length: 2048 target oid smithi11939506-24 low_tier offset: 1769472 snap -1 2024-09-18T11:30:09.095 INFO:tasks.rados.rados.0.smithi119.stdout:358: finishing set_chunk to oid 9 2024-09-18T11:30:09.095 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 9 set_chunk 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 669696 length: 915968 2024-09-18T11:30:09.095 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 0 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T11:30:09.095 INFO:tasks.rados.rados.0.smithi119.stdout:361: finishing set_chunk to oid 12 2024-09-18T11:30:09.095 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 12 set_chunk 12 low_tier returned error code -95 offset: 829440 length: 810496 2024-09-18T11:30:09.095 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 12 v 0 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-18T11:30:09.096 INFO:tasks.rados.rados.0.smithi119.stdout:364: finishing set_chunk to oid 15 2024-09-18T11:30:09.096 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 15 set_chunk 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 506368 length: 1585152 2024-09-18T11:30:09.096 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 0 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-18T11:30:09.096 INFO:tasks.rados.rados.0.smithi119.stdout:363: finishing set_chunk to oid 14 2024-09-18T11:30:09.096 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 14 set_chunk 14 low_tier returned error code -95 offset: 1400320 length: 643072 2024-09-18T11:30:09.096 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 0 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-18T11:30:09.096 INFO:tasks.rados.rados.0.smithi119.stdout:358: done (15 left) 2024-09-18T11:30:09.096 INFO:tasks.rados.rados.0.smithi119.stdout:361: done (14 left) 2024-09-18T11:30:09.096 INFO:tasks.rados.rados.0.smithi119.stdout:363: done (13 left) 2024-09-18T11:30:09.096 INFO:tasks.rados.rados.0.smithi119.stdout:364: done (12 left) 2024-09-18T11:30:09.096 INFO:tasks.rados.rados.0.smithi119.stdout:374: set_chunk oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.096 INFO:tasks.rados.rados.0.smithi119.stdout:374: set_chunk oid 25 offset: 102912 length: 996352 target oid smithi11939506-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 102912 snap -1 2024-09-18T11:30:09.096 INFO:tasks.rados.rados.0.smithi119.stdout:375: set_chunk oid 26 target oid 26 low_tier 2024-09-18T11:30:09.097 INFO:tasks.rados.rados.0.smithi119.stdout:375: set_chunk oid 26 offset: 734720 length: 1079296 target oid smithi11939506-26 low_tier offset: 734720 snap -1 2024-09-18T11:30:09.097 INFO:tasks.rados.rados.0.smithi119.stdout:376: set_chunk oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.097 INFO:tasks.rados.rados.0.smithi119.stdout:376: set_chunk oid 27 offset: 546304 length: 830464 target oid smithi11939506-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 546304 snap -1 2024-09-18T11:30:09.097 INFO:tasks.rados.rados.0.smithi119.stdout:377: set_chunk oid 28 target oid 28 low_tier 2024-09-18T11:30:09.097 INFO:tasks.rados.rados.0.smithi119.stdout:377: set_chunk oid 28 offset: 677376 length: 1342976 target oid smithi11939506-28 low_tier offset: 677376 snap -1 2024-09-18T11:30:09.097 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.097 INFO:tasks.rados.rados.0.smithi119.stdout:356: finishing set_chunk to oid 7 2024-09-18T11:30:09.097 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 7 set_chunk 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1143296 length: 577536 2024-09-18T11:30:09.097 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 7 v 0 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-18T11:30:09.097 INFO:tasks.rados.rados.0.smithi119.stdout:369: finishing set_chunk to oid 20 2024-09-18T11:30:09.097 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 20 set_chunk 20 low_tier returned error code -95 offset: 112640 length: 1752064 2024-09-18T11:30:09.097 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 20 v 0 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-18T11:30:09.097 INFO:tasks.rados.rados.0.smithi119.stdout:356: done (15 left) 2024-09-18T11:30:09.098 INFO:tasks.rados.rados.0.smithi119.stdout:369: done (14 left) 2024-09-18T11:30:09.098 INFO:tasks.rados.rados.0.smithi119.stdout:378: set_chunk oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.098 INFO:tasks.rados.rados.0.smithi119.stdout:367: finishing set_chunk to oid 18 2024-09-18T11:30:09.098 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 18 set_chunk 18 low_tier returned error code -95 offset: 1634304 length: 167424 2024-09-18T11:30:09.098 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 0 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-18T11:30:09.098 INFO:tasks.rados.rados.0.smithi119.stdout:378: set_chunk oid 29 offset: 1154048 length: 239104 target oid smithi11939506-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1154048 snap -1 2024-09-18T11:30:09.098 INFO:tasks.rados.rados.0.smithi119.stdout:367: done (14 left) 2024-09-18T11:30:09.098 INFO:tasks.rados.rados.0.smithi119.stdout:379: set_chunk oid 30 target oid 30 low_tier 2024-09-18T11:30:09.098 INFO:tasks.rados.rados.0.smithi119.stdout:379: set_chunk oid 30 offset: 920576 length: 407552 target oid smithi11939506-30 low_tier offset: 920576 snap -1 2024-09-18T11:30:09.098 INFO:tasks.rados.rados.0.smithi119.stdout:380: set_chunk oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.098 INFO:tasks.rados.rados.0.smithi119.stdout:380: set_chunk oid 31 offset: 1275904 length: 617472 target oid smithi11939506-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1275904 snap -1 2024-09-18T11:30:09.098 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.098 INFO:tasks.rados.rados.0.smithi119.stdout:362: finishing set_chunk to oid 13 2024-09-18T11:30:09.099 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 13 set_chunk 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1340928 length: 696832 2024-09-18T11:30:09.099 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 13 v 0 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-18T11:30:09.099 INFO:tasks.rados.rados.0.smithi119.stdout:359: finishing set_chunk to oid 10 2024-09-18T11:30:09.099 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 10 set_chunk 10 low_tier returned error code -95 offset: 522752 length: 1521152 2024-09-18T11:30:09.099 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 10 v 0 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-18T11:30:09.099 INFO:tasks.rados.rados.0.smithi119.stdout:365: finishing set_chunk to oid 16 2024-09-18T11:30:09.099 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 16 set_chunk 16 low_tier returned error code -95 offset: 360960 length: 1687552 2024-09-18T11:30:09.099 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 0 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-18T11:30:09.099 INFO:tasks.rados.rados.0.smithi119.stdout:368: finishing set_chunk to oid 19 2024-09-18T11:30:09.099 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 19 set_chunk 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1457664 length: 471552 2024-09-18T11:30:09.099 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 0 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-18T11:30:09.099 INFO:tasks.rados.rados.0.smithi119.stdout:359: done (15 left) 2024-09-18T11:30:09.099 INFO:tasks.rados.rados.0.smithi119.stdout:362: done (14 left) 2024-09-18T11:30:09.100 INFO:tasks.rados.rados.0.smithi119.stdout:365: done (13 left) 2024-09-18T11:30:09.100 INFO:tasks.rados.rados.0.smithi119.stdout:368: done (12 left) 2024-09-18T11:30:09.100 INFO:tasks.rados.rados.0.smithi119.stdout:381: set_chunk oid 32 target oid 32 low_tier 2024-09-18T11:30:09.100 INFO:tasks.rados.rados.0.smithi119.stdout:381: set_chunk oid 32 offset: 823296 length: 270336 target oid smithi11939506-32 low_tier offset: 823296 snap -1 2024-09-18T11:30:09.100 INFO:tasks.rados.rados.0.smithi119.stdout:382: set_chunk oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.100 INFO:tasks.rados.rados.0.smithi119.stdout:382: set_chunk oid 33 offset: 19456 length: 1858560 target oid smithi11939506-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 19456 snap -1 2024-09-18T11:30:09.100 INFO:tasks.rados.rados.0.smithi119.stdout:383: set_chunk oid 34 target oid 34 low_tier 2024-09-18T11:30:09.100 INFO:tasks.rados.rados.0.smithi119.stdout:383: set_chunk oid 34 offset: 749568 length: 659456 target oid smithi11939506-34 low_tier offset: 749568 snap -1 2024-09-18T11:30:09.100 INFO:tasks.rados.rados.0.smithi119.stdout:384: set_chunk oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.100 INFO:tasks.rados.rados.0.smithi119.stdout:384: set_chunk oid 35 offset: 2010624 length: 332288 target oid smithi11939506-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 2010624 snap -1 2024-09-18T11:30:09.100 INFO:tasks.rados.rados.0.smithi119.stdout:376: finishing set_chunk to oid 27 2024-09-18T11:30:09.101 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 27 set_chunk 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 546304 length: 830464 2024-09-18T11:30:09.101 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 0 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T11:30:09.101 INFO:tasks.rados.rados.0.smithi119.stdout:376: done (15 left) 2024-09-18T11:30:09.101 INFO:tasks.rados.rados.0.smithi119.stdout:385: set_chunk oid 36 target oid 36 low_tier 2024-09-18T11:30:09.101 INFO:tasks.rados.rados.0.smithi119.stdout:375: finishing set_chunk to oid 26 2024-09-18T11:30:09.101 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 26 set_chunk 26 low_tier returned error code -95 offset: 734720 length: 1079296 2024-09-18T11:30:09.101 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 0 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-18T11:30:09.101 INFO:tasks.rados.rados.0.smithi119.stdout:374: finishing set_chunk to oid 25 2024-09-18T11:30:09.101 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 25 set_chunk 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 102912 length: 996352 2024-09-18T11:30:09.101 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 0 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-18T11:30:09.101 INFO:tasks.rados.rados.0.smithi119.stdout:385: set_chunk oid 36 offset: 1566720 length: 775168 target oid smithi11939506-36 low_tier offset: 1566720 snap -1 2024-09-18T11:30:09.101 INFO:tasks.rados.rados.0.smithi119.stdout:374: done (15 left) 2024-09-18T11:30:09.102 INFO:tasks.rados.rados.0.smithi119.stdout:375: done (14 left) 2024-09-18T11:30:09.102 INFO:tasks.rados.rados.0.smithi119.stdout:386: set_chunk oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.102 INFO:tasks.rados.rados.0.smithi119.stdout:386: set_chunk oid 37 offset: 1122304 length: 456192 target oid smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1122304 snap -1 2024-09-18T11:30:09.102 INFO:tasks.rados.rados.0.smithi119.stdout:387: set_chunk oid 38 target oid 38 low_tier 2024-09-18T11:30:09.102 INFO:tasks.rados.rados.0.smithi119.stdout:387: set_chunk oid 38 offset: 249856 length: 424960 target oid smithi11939506-38 low_tier offset: 249856 snap -1 2024-09-18T11:30:09.102 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.102 INFO:tasks.rados.rados.0.smithi119.stdout:379: finishing set_chunk to oid 30 2024-09-18T11:30:09.102 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 30 set_chunk 30 low_tier returned error code -95 offset: 920576 length: 407552 2024-09-18T11:30:09.102 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 30 v 0 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-18T11:30:09.102 INFO:tasks.rados.rados.0.smithi119.stdout:379: done (15 left) 2024-09-18T11:30:09.102 INFO:tasks.rados.rados.0.smithi119.stdout:388: set_chunk oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.102 INFO:tasks.rados.rados.0.smithi119.stdout:388: set_chunk oid 39 offset: 326144 length: 738816 target oid smithi11939506-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 326144 snap -1 2024-09-18T11:30:09.102 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.103 INFO:tasks.rados.rados.0.smithi119.stdout:370: finishing set_chunk to oid 21 2024-09-18T11:30:09.103 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 21 set_chunk 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1205760 length: 184320 2024-09-18T11:30:09.103 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 0 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-18T11:30:09.103 INFO:tasks.rados.rados.0.smithi119.stdout:371: finishing set_chunk to oid 22 2024-09-18T11:30:09.103 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 22 set_chunk 22 low_tier returned error code -95 offset: 1279488 length: 669696 2024-09-18T11:30:09.103 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 0 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-18T11:30:09.103 INFO:tasks.rados.rados.0.smithi119.stdout:373: finishing set_chunk to oid 24 2024-09-18T11:30:09.103 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 24 set_chunk 24 low_tier returned error code -95 offset: 1769472 length: 2048 2024-09-18T11:30:09.103 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 0 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-18T11:30:09.103 INFO:tasks.rados.rados.0.smithi119.stdout:377: finishing set_chunk to oid 28 2024-09-18T11:30:09.103 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 28 set_chunk 28 low_tier returned error code -95 offset: 677376 length: 1342976 2024-09-18T11:30:09.103 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 28 v 0 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-18T11:30:09.103 INFO:tasks.rados.rados.0.smithi119.stdout:370: done (15 left) 2024-09-18T11:30:09.103 INFO:tasks.rados.rados.0.smithi119.stdout:371: done (14 left) 2024-09-18T11:30:09.104 INFO:tasks.rados.rados.0.smithi119.stdout:373: done (13 left) 2024-09-18T11:30:09.104 INFO:tasks.rados.rados.0.smithi119.stdout:377: done (12 left) 2024-09-18T11:30:09.104 INFO:tasks.rados.rados.0.smithi119.stdout:389: set_chunk oid 40 target oid 40 low_tier 2024-09-18T11:30:09.104 INFO:tasks.rados.rados.0.smithi119.stdout:389: set_chunk oid 40 offset: 1122816 length: 942080 target oid smithi11939506-40 low_tier offset: 1122816 snap -1 2024-09-18T11:30:09.104 INFO:tasks.rados.rados.0.smithi119.stdout:390: set_chunk oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.104 INFO:tasks.rados.rados.0.smithi119.stdout:390: set_chunk oid 41 offset: 172032 length: 288768 target oid smithi11939506-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 172032 snap -1 2024-09-18T11:30:09.104 INFO:tasks.rados.rados.0.smithi119.stdout:382: finishing set_chunk to oid 33 2024-09-18T11:30:09.104 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 33 set_chunk 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 19456 length: 1858560 2024-09-18T11:30:09.104 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 33 v 0 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-18T11:30:09.104 INFO:tasks.rados.rados.0.smithi119.stdout:381: finishing set_chunk to oid 32 2024-09-18T11:30:09.104 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 32 set_chunk 32 low_tier returned error code -95 offset: 823296 length: 270336 2024-09-18T11:30:09.104 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 0 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-18T11:30:09.104 INFO:tasks.rados.rados.0.smithi119.stdout:383: finishing set_chunk to oid 34 2024-09-18T11:30:09.105 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 34 set_chunk 34 low_tier returned error code -95 offset: 749568 length: 659456 2024-09-18T11:30:09.105 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 0 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T11:30:09.105 INFO:tasks.rados.rados.0.smithi119.stdout:378: finishing set_chunk to oid 29 2024-09-18T11:30:09.105 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 29 set_chunk 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1154048 length: 239104 2024-09-18T11:30:09.105 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 0 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-18T11:30:09.105 INFO:tasks.rados.rados.0.smithi119.stdout:380: finishing set_chunk to oid 31 2024-09-18T11:30:09.105 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 31 set_chunk 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1275904 length: 617472 2024-09-18T11:30:09.105 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 0 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-18T11:30:09.105 INFO:tasks.rados.rados.0.smithi119.stdout:385: finishing set_chunk to oid 36 2024-09-18T11:30:09.105 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 36 set_chunk 36 low_tier returned error code -95 offset: 1566720 length: 775168 2024-09-18T11:30:09.105 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 36 v 0 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-18T11:30:09.106 INFO:tasks.rados.rados.0.smithi119.stdout:378: done (13 left) 2024-09-18T11:30:09.106 INFO:tasks.rados.rados.0.smithi119.stdout:380: done (12 left) 2024-09-18T11:30:09.106 INFO:tasks.rados.rados.0.smithi119.stdout:381: done (11 left) 2024-09-18T11:30:09.106 INFO:tasks.rados.rados.0.smithi119.stdout:382: done (10 left) 2024-09-18T11:30:09.106 INFO:tasks.rados.rados.0.smithi119.stdout:383: done (9 left) 2024-09-18T11:30:09.106 INFO:tasks.rados.rados.0.smithi119.stdout:385: done (8 left) 2024-09-18T11:30:09.106 INFO:tasks.rados.rados.0.smithi119.stdout:391: set_chunk oid 42 target oid 42 low_tier 2024-09-18T11:30:09.106 INFO:tasks.rados.rados.0.smithi119.stdout:391: set_chunk oid 42 offset: 1332224 length: 150016 target oid smithi11939506-42 low_tier offset: 1332224 snap -1 2024-09-18T11:30:09.106 INFO:tasks.rados.rados.0.smithi119.stdout:384: finishing set_chunk to oid 35 2024-09-18T11:30:09.106 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 35 set_chunk 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 2010624 length: 332288 2024-09-18T11:30:09.106 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 0 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-18T11:30:09.106 INFO:tasks.rados.rados.0.smithi119.stdout:388: finishing set_chunk to oid 39 2024-09-18T11:30:09.107 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 39 set_chunk 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 326144 length: 738816 2024-09-18T11:30:09.107 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 39 v 0 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-18T11:30:09.107 INFO:tasks.rados.rados.0.smithi119.stdout:384: done (8 left) 2024-09-18T11:30:09.107 INFO:tasks.rados.rados.0.smithi119.stdout:388: done (7 left) 2024-09-18T11:30:09.107 INFO:tasks.rados.rados.0.smithi119.stdout:392: set_chunk oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.107 INFO:tasks.rados.rados.0.smithi119.stdout:392: set_chunk oid 43 offset: 1156608 length: 149504 target oid smithi11939506-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1156608 snap -1 2024-09-18T11:30:09.107 INFO:tasks.rados.rados.0.smithi119.stdout:393: set_chunk oid 44 target oid 44 low_tier 2024-09-18T11:30:09.107 INFO:tasks.rados.rados.0.smithi119.stdout:393: set_chunk oid 44 offset: 1385984 length: 458240 target oid smithi11939506-44 low_tier offset: 1385984 snap -1 2024-09-18T11:30:09.107 INFO:tasks.rados.rados.0.smithi119.stdout:394: set_chunk oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.107 INFO:tasks.rados.rados.0.smithi119.stdout:394: set_chunk oid 45 offset: 444416 length: 313856 target oid smithi11939506-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 444416 snap -1 2024-09-18T11:30:09.107 INFO:tasks.rados.rados.0.smithi119.stdout:390: finishing set_chunk to oid 41 2024-09-18T11:30:09.107 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 41 set_chunk 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 172032 length: 288768 2024-09-18T11:30:09.107 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 41 v 0 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-18T11:30:09.108 INFO:tasks.rados.rados.0.smithi119.stdout:387: finishing set_chunk to oid 38 2024-09-18T11:30:09.108 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 38 set_chunk 38 low_tier returned error code -95 offset: 249856 length: 424960 2024-09-18T11:30:09.108 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 0 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-18T11:30:09.108 INFO:tasks.rados.rados.0.smithi119.stdout:386: finishing set_chunk to oid 37 2024-09-18T11:30:09.108 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 37 set_chunk 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1122304 length: 456192 2024-09-18T11:30:09.108 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 0 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-18T11:30:09.108 INFO:tasks.rados.rados.0.smithi119.stdout:386: done (9 left) 2024-09-18T11:30:09.108 INFO:tasks.rados.rados.0.smithi119.stdout:387: done (8 left) 2024-09-18T11:30:09.108 INFO:tasks.rados.rados.0.smithi119.stdout:390: done (7 left) 2024-09-18T11:30:09.108 INFO:tasks.rados.rados.0.smithi119.stdout:395: set_chunk oid 46 target oid 46 low_tier 2024-09-18T11:30:09.108 INFO:tasks.rados.rados.0.smithi119.stdout:395: set_chunk oid 46 offset: 1123840 length: 1356800 target oid smithi11939506-46 low_tier offset: 1123840 snap -1 2024-09-18T11:30:09.109 INFO:tasks.rados.rados.0.smithi119.stdout:396: set_chunk oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.109 INFO:tasks.rados.rados.0.smithi119.stdout:396: set_chunk oid 47 offset: 818688 length: 1169920 target oid smithi11939506-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 818688 snap -1 2024-09-18T11:30:09.109 INFO:tasks.rados.rados.0.smithi119.stdout:397: set_chunk oid 48 target oid 48 low_tier 2024-09-18T11:30:09.109 INFO:tasks.rados.rados.0.smithi119.stdout:397: set_chunk oid 48 offset: 983040 length: 855040 target oid smithi11939506-48 low_tier offset: 983040 snap -1 2024-09-18T11:30:09.109 INFO:tasks.rados.rados.0.smithi119.stdout:398: set_chunk oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.109 INFO:tasks.rados.rados.0.smithi119.stdout:398: set_chunk oid 49 offset: 626688 length: 94208 target oid smithi11939506-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 626688 snap -1 2024-09-18T11:30:09.109 INFO:tasks.rados.rados.0.smithi119.stdout:399: set_chunk oid 50 target oid 50 low_tier 2024-09-18T11:30:09.109 INFO:tasks.rados.rados.0.smithi119.stdout:399: set_chunk oid 50 offset: 1697792 length: 1032192 target oid smithi11939506-50 low_tier offset: 1697792 snap -1 2024-09-18T11:30:09.109 INFO:tasks.rados.rados.0.smithi119.stdout:400: set_chunk oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.109 INFO:tasks.rados.rados.0.smithi119.stdout:400: set_chunk oid 1 offset: 971264 length: 255488 target oid smithi11939506-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 971264 snap -1 2024-09-18T11:30:09.109 INFO:tasks.rados.rados.0.smithi119.stdout:401: set_chunk oid 2 target oid 2 low_tier 2024-09-18T11:30:09.109 INFO:tasks.rados.rados.0.smithi119.stdout:401: set_chunk oid 2 offset: 1294336 length: 91648 target oid smithi11939506-2 low_tier offset: 1294336 snap -1 2024-09-18T11:30:09.110 INFO:tasks.rados.rados.0.smithi119.stdout:402: set_chunk oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.110 INFO:tasks.rados.rados.0.smithi119.stdout:402: set_chunk oid 3 offset: 935424 length: 268800 target oid smithi11939506-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 935424 snap -1 2024-09-18T11:30:09.110 INFO:tasks.rados.rados.0.smithi119.stdout:403: set_chunk oid 4 target oid 4 low_tier 2024-09-18T11:30:09.110 INFO:tasks.rados.rados.0.smithi119.stdout:403: set_chunk oid 4 offset: 723456 length: 1087488 target oid smithi11939506-4 low_tier offset: 723456 snap -1 2024-09-18T11:30:09.110 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.110 INFO:tasks.rados.rados.0.smithi119.stdout:389: finishing set_chunk to oid 40 2024-09-18T11:30:09.110 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 40 set_chunk 40 low_tier returned error code -95 offset: 1122816 length: 942080 2024-09-18T11:30:09.110 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 0 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-18T11:30:09.110 INFO:tasks.rados.rados.0.smithi119.stdout:394: finishing set_chunk to oid 45 2024-09-18T11:30:09.110 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 45 set_chunk 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 444416 length: 313856 2024-09-18T11:30:09.110 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 0 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-18T11:30:09.110 INFO:tasks.rados.rados.0.smithi119.stdout:389: done (15 left) 2024-09-18T11:30:09.110 INFO:tasks.rados.rados.0.smithi119.stdout:394: done (14 left) 2024-09-18T11:30:09.111 INFO:tasks.rados.rados.0.smithi119.stdout:404: set_chunk oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.111 INFO:tasks.rados.rados.0.smithi119.stdout:404: set_chunk oid 5 offset: 54272 length: 136192 target oid smithi11939506-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 54272 snap -1 2024-09-18T11:30:09.111 INFO:tasks.rados.rados.0.smithi119.stdout:405: set_chunk oid 6 target oid 6 low_tier 2024-09-18T11:30:09.111 INFO:tasks.rados.rados.0.smithi119.stdout:405: set_chunk oid 6 offset: 416256 length: 1511424 target oid smithi11939506-6 low_tier offset: 416256 snap -1 2024-09-18T11:30:09.111 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.111 INFO:tasks.rados.rados.0.smithi119.stdout:393: finishing set_chunk to oid 44 2024-09-18T11:30:09.111 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 44 set_chunk 44 low_tier returned error code -95 offset: 1385984 length: 458240 2024-09-18T11:30:09.111 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 44 v 0 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-18T11:30:09.111 INFO:tasks.rados.rados.0.smithi119.stdout:393: done (15 left) 2024-09-18T11:30:09.111 INFO:tasks.rados.rados.0.smithi119.stdout:406: set_chunk oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.111 INFO:tasks.rados.rados.0.smithi119.stdout:406: set_chunk oid 7 offset: 1093632 length: 934400 target oid smithi11939506-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1093632 snap -1 2024-09-18T11:30:09.111 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.111 INFO:tasks.rados.rados.0.smithi119.stdout:395: finishing set_chunk to oid 46 2024-09-18T11:30:09.111 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 46 set_chunk 46 low_tier returned error code -95 offset: 1123840 length: 1356800 2024-09-18T11:30:09.112 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 0 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-18T11:30:09.112 INFO:tasks.rados.rados.0.smithi119.stdout:392: finishing set_chunk to oid 43 2024-09-18T11:30:09.112 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 43 set_chunk 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1156608 length: 149504 2024-09-18T11:30:09.112 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-18T11:30:09.112 INFO:tasks.rados.rados.0.smithi119.stdout:403: finishing set_chunk to oid 4 2024-09-18T11:30:09.112 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 4 set_chunk 4 low_tier returned error code -95 offset: 723456 length: 1087488 2024-09-18T11:30:09.112 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 4 v 0 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-18T11:30:09.112 INFO:tasks.rados.rados.0.smithi119.stdout:392: done (15 left) 2024-09-18T11:30:09.112 INFO:tasks.rados.rados.0.smithi119.stdout:395: done (14 left) 2024-09-18T11:30:09.112 INFO:tasks.rados.rados.0.smithi119.stdout:403: done (13 left) 2024-09-18T11:30:09.112 INFO:tasks.rados.rados.0.smithi119.stdout:407: set_chunk oid 8 target oid 8 low_tier 2024-09-18T11:30:09.112 INFO:tasks.rados.rados.0.smithi119.stdout:407: set_chunk oid 8 offset: 1040896 length: 18432 target oid smithi11939506-8 low_tier offset: 1040896 snap -1 2024-09-18T11:30:09.112 INFO:tasks.rados.rados.0.smithi119.stdout:408: set_chunk oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.113 INFO:tasks.rados.rados.0.smithi119.stdout:408: set_chunk oid 9 offset: 1430528 length: 397824 target oid smithi11939506-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1430528 snap -1 2024-09-18T11:30:09.113 INFO:tasks.rados.rados.0.smithi119.stdout:409: set_chunk oid 10 target oid 10 low_tier 2024-09-18T11:30:09.113 INFO:tasks.rados.rados.0.smithi119.stdout:409: set_chunk oid 10 offset: 725504 length: 697856 target oid smithi11939506-10 low_tier offset: 725504 snap -1 2024-09-18T11:30:09.113 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.113 INFO:tasks.rados.rados.0.smithi119.stdout:397: finishing set_chunk to oid 48 2024-09-18T11:30:09.113 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 48 set_chunk 48 low_tier returned error code -95 offset: 983040 length: 855040 2024-09-18T11:30:09.113 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 48 v 0 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-18T11:30:09.113 INFO:tasks.rados.rados.0.smithi119.stdout:397: done (15 left) 2024-09-18T11:30:09.113 INFO:tasks.rados.rados.0.smithi119.stdout:410: set_chunk oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.113 INFO:tasks.rados.rados.0.smithi119.stdout:405: finishing set_chunk to oid 6 2024-09-18T11:30:09.113 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 6 set_chunk 6 low_tier returned error code -95 offset: 416256 length: 1511424 2024-09-18T11:30:09.113 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 6 v 0 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-18T11:30:09.113 INFO:tasks.rados.rados.0.smithi119.stdout:410: set_chunk oid 11 offset: 449536 length: 1213952 target oid smithi11939506-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 449536 snap -1 2024-09-18T11:30:09.114 INFO:tasks.rados.rados.0.smithi119.stdout:405: done (15 left) 2024-09-18T11:30:09.114 INFO:tasks.rados.rados.0.smithi119.stdout:411: set_chunk oid 12 target oid 12 low_tier 2024-09-18T11:30:09.114 INFO:tasks.rados.rados.0.smithi119.stdout:411: set_chunk oid 12 offset: 237568 length: 265728 target oid smithi11939506-12 low_tier offset: 237568 snap -1 2024-09-18T11:30:09.114 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.114 INFO:tasks.rados.rados.0.smithi119.stdout:400: finishing set_chunk to oid 1 2024-09-18T11:30:09.114 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 1 set_chunk 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 971264 length: 255488 2024-09-18T11:30:09.114 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 0 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-18T11:30:09.114 INFO:tasks.rados.rados.0.smithi119.stdout:402: finishing set_chunk to oid 3 2024-09-18T11:30:09.114 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 3 set_chunk 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 935424 length: 268800 2024-09-18T11:30:09.114 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 0 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-18T11:30:09.114 INFO:tasks.rados.rados.0.smithi119.stdout:399: finishing set_chunk to oid 50 2024-09-18T11:30:09.114 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 50 set_chunk 50 low_tier returned error code -95 offset: 1697792 length: 1032192 2024-09-18T11:30:09.114 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 0 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-18T11:30:09.115 INFO:tasks.rados.rados.0.smithi119.stdout:406: finishing set_chunk to oid 7 2024-09-18T11:30:09.115 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 7 set_chunk 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1093632 length: 934400 2024-09-18T11:30:09.115 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 7 v 0 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-18T11:30:09.115 INFO:tasks.rados.rados.0.smithi119.stdout:399: done (15 left) 2024-09-18T11:30:09.115 INFO:tasks.rados.rados.0.smithi119.stdout:400: done (14 left) 2024-09-18T11:30:09.115 INFO:tasks.rados.rados.0.smithi119.stdout:402: done (13 left) 2024-09-18T11:30:09.115 INFO:tasks.rados.rados.0.smithi119.stdout:406: done (12 left) 2024-09-18T11:30:09.115 INFO:tasks.rados.rados.0.smithi119.stdout:412: set_chunk oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.115 INFO:tasks.rados.rados.0.smithi119.stdout:412: set_chunk oid 13 offset: 774656 length: 98816 target oid smithi11939506-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 774656 snap -1 2024-09-18T11:30:09.115 INFO:tasks.rados.rados.0.smithi119.stdout:413: set_chunk oid 14 target oid 14 low_tier 2024-09-18T11:30:09.115 INFO:tasks.rados.rados.0.smithi119.stdout:413: set_chunk oid 14 offset: 552448 length: 91648 target oid smithi11939506-14 low_tier offset: 552448 snap -1 2024-09-18T11:30:09.115 INFO:tasks.rados.rados.0.smithi119.stdout:414: set_chunk oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.115 INFO:tasks.rados.rados.0.smithi119.stdout:414: set_chunk oid 15 offset: 1714688 length: 18944 target oid smithi11939506-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1714688 snap -1 2024-09-18T11:30:09.115 INFO:tasks.rados.rados.0.smithi119.stdout:415: set_chunk oid 16 target oid 16 low_tier 2024-09-18T11:30:09.116 INFO:tasks.rados.rados.0.smithi119.stdout:415: set_chunk oid 16 offset: 1665024 length: 9728 target oid smithi11939506-16 low_tier offset: 1665024 snap -1 2024-09-18T11:30:09.116 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.116 INFO:tasks.rados.rados.0.smithi119.stdout:398: finishing set_chunk to oid 49 2024-09-18T11:30:09.116 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 49 set_chunk 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 626688 length: 94208 2024-09-18T11:30:09.116 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 0 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-18T11:30:09.116 INFO:tasks.rados.rados.0.smithi119.stdout:398: done (15 left) 2024-09-18T11:30:09.116 INFO:tasks.rados.rados.0.smithi119.stdout:415 retry set_chunk ! 2024-09-18T11:30:09.116 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 15 2024-09-18T11:30:09.116 INFO:tasks.rados.rados.0.smithi119.stdout:410: finishing set_chunk to oid 11 2024-09-18T11:30:09.116 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 11 set_chunk 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 449536 length: 1213952 2024-09-18T11:30:09.116 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 11 v 0 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-18T11:30:09.116 INFO:tasks.rados.rados.0.smithi119.stdout:410: done (14 left) 2024-09-18T11:30:09.116 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 14 2024-09-18T11:30:09.117 INFO:tasks.rados.rados.0.smithi119.stdout:411: finishing set_chunk to oid 12 2024-09-18T11:30:09.117 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 12 set_chunk 12 low_tier returned error code -95 offset: 237568 length: 265728 2024-09-18T11:30:09.117 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 12 v 0 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-18T11:30:09.117 INFO:tasks.rados.rados.0.smithi119.stdout:411: done (13 left) 2024-09-18T11:30:09.117 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 13 2024-09-18T11:30:09.117 INFO:tasks.rados.rados.0.smithi119.stdout:401: finishing set_chunk to oid 2 2024-09-18T11:30:09.117 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 2 set_chunk 2 low_tier returned error code -95 offset: 1294336 length: 91648 2024-09-18T11:30:09.117 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 0 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-18T11:30:09.117 INFO:tasks.rados.rados.0.smithi119.stdout:414: finishing set_chunk to oid 15 2024-09-18T11:30:09.117 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 15 set_chunk 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1714688 length: 18944 2024-09-18T11:30:09.117 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 0 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-18T11:30:09.117 INFO:tasks.rados.rados.0.smithi119.stdout:401: done (12 left) 2024-09-18T11:30:09.117 INFO:tasks.rados.rados.0.smithi119.stdout:414: done (11 left) 2024-09-18T11:30:09.118 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 11 2024-09-18T11:30:09.118 INFO:tasks.rados.rados.0.smithi119.stdout:407: finishing set_chunk to oid 8 2024-09-18T11:30:09.118 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 8 set_chunk 8 low_tier returned error code -95 offset: 1040896 length: 18432 2024-09-18T11:30:09.118 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 8 v 0 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-18T11:30:09.118 INFO:tasks.rados.rados.0.smithi119.stdout:407: done (10 left) 2024-09-18T11:30:09.118 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 10 2024-09-18T11:30:09.118 INFO:tasks.rados.rados.0.smithi119.stdout:412: finishing set_chunk to oid 13 2024-09-18T11:30:09.118 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 13 set_chunk 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 774656 length: 98816 2024-09-18T11:30:09.118 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 13 v 0 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-18T11:30:09.118 INFO:tasks.rados.rados.0.smithi119.stdout:412: done (9 left) 2024-09-18T11:30:09.118 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 9 2024-09-18T11:30:09.118 INFO:tasks.rados.rados.0.smithi119.stdout:409: finishing set_chunk to oid 10 2024-09-18T11:30:09.118 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 10 set_chunk 10 low_tier returned error code -95 offset: 725504 length: 697856 2024-09-18T11:30:09.118 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 10 v 0 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-18T11:30:09.119 INFO:tasks.rados.rados.0.smithi119.stdout:409: done (8 left) 2024-09-18T11:30:09.119 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 8 2024-09-18T11:30:09.119 INFO:tasks.rados.rados.0.smithi119.stdout:372: finishing set_chunk to oid 23 2024-09-18T11:30:09.119 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 6 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-18T11:30:09.119 INFO:tasks.rados.rados.0.smithi119.stdout:372: done (7 left) 2024-09-18T11:30:09.119 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 7 2024-09-18T11:30:09.119 INFO:tasks.rados.rados.0.smithi119.stdout:391: finishing set_chunk to oid 42 2024-09-18T11:30:09.119 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 12 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T11:30:09.119 INFO:tasks.rados.rados.0.smithi119.stdout:396: finishing set_chunk to oid 47 2024-09-18T11:30:09.119 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 9 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-18T11:30:09.119 INFO:tasks.rados.rados.0.smithi119.stdout:391: done (6 left) 2024-09-18T11:30:09.119 INFO:tasks.rados.rados.0.smithi119.stdout:396: done (5 left) 2024-09-18T11:30:09.119 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 5 2024-09-18T11:30:09.119 INFO:tasks.rados.rados.0.smithi119.stdout:366: finishing set_chunk to oid 17 2024-09-18T11:30:09.120 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 12 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-18T11:30:09.120 INFO:tasks.rados.rados.0.smithi119.stdout:366: done (4 left) 2024-09-18T11:30:09.120 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 4 2024-09-18T11:30:09.120 INFO:tasks.rados.rados.0.smithi119.stdout:404: finishing set_chunk to oid 5 2024-09-18T11:30:09.120 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 6 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-18T11:30:09.120 INFO:tasks.rados.rados.0.smithi119.stdout:404: done (3 left) 2024-09-18T11:30:09.120 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 3 2024-09-18T11:30:09.120 INFO:tasks.rados.rados.0.smithi119.stdout:408: finishing set_chunk to oid 9 2024-09-18T11:30:09.120 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 6 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T11:30:09.120 INFO:tasks.rados.rados.0.smithi119.stdout:408: done (2 left) 2024-09-18T11:30:09.120 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 2 2024-09-18T11:30:09.120 INFO:tasks.rados.rados.0.smithi119.stdout:413: finishing set_chunk to oid 14 2024-09-18T11:30:09.121 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 3 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-18T11:30:09.121 INFO:tasks.rados.rados.0.smithi119.stdout:413: done (1 left) 2024-09-18T11:30:09.121 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 1 2024-09-18T11:30:09.121 INFO:tasks.rados.rados.0.smithi119.stdout:415: finishing set_chunk to oid 16 2024-09-18T11:30:09.121 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 3 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-18T11:30:09.121 INFO:tasks.rados.rados.0.smithi119.stdout:415: done (0 left) 2024-09-18T11:30:09.121 INFO:tasks.rados.rados.0.smithi119.stdout:416: set_chunk oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.121 INFO:tasks.rados.rados.0.smithi119.stdout:416: set_chunk oid 17 offset: 1072640 length: 1041408 target oid smithi11939506-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1072640 snap -1 2024-09-18T11:30:09.121 INFO:tasks.rados.rados.0.smithi119.stdout:417: set_chunk oid 18 target oid 18 low_tier 2024-09-18T11:30:09.121 INFO:tasks.rados.rados.0.smithi119.stdout:417: set_chunk oid 18 offset: 285184 length: 1868800 target oid smithi11939506-18 low_tier offset: 285184 snap -1 2024-09-18T11:30:09.121 INFO:tasks.rados.rados.0.smithi119.stdout:418: set_chunk oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.121 INFO:tasks.rados.rados.0.smithi119.stdout:418: set_chunk oid 19 offset: 638976 length: 1193472 target oid smithi11939506-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 638976 snap -1 2024-09-18T11:30:09.121 INFO:tasks.rados.rados.0.smithi119.stdout:419: set_chunk oid 20 target oid 20 low_tier 2024-09-18T11:30:09.122 INFO:tasks.rados.rados.0.smithi119.stdout:419: set_chunk oid 20 offset: 291840 length: 613888 target oid smithi11939506-20 low_tier offset: 291840 snap -1 2024-09-18T11:30:09.122 INFO:tasks.rados.rados.0.smithi119.stdout:420: set_chunk oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.122 INFO:tasks.rados.rados.0.smithi119.stdout:420: set_chunk oid 21 offset: 341504 length: 172032 target oid smithi11939506-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 341504 snap -1 2024-09-18T11:30:09.122 INFO:tasks.rados.rados.0.smithi119.stdout:421: set_chunk oid 22 target oid 22 low_tier 2024-09-18T11:30:09.122 INFO:tasks.rados.rados.0.smithi119.stdout:421: set_chunk oid 22 offset: 155136 length: 480768 target oid smithi11939506-22 low_tier offset: 155136 snap -1 2024-09-18T11:30:09.122 INFO:tasks.rados.rados.0.smithi119.stdout:422: set_chunk oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.122 INFO:tasks.rados.rados.0.smithi119.stdout:422: set_chunk oid 23 offset: 383488 length: 1105920 target oid smithi11939506-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 383488 snap -1 2024-09-18T11:30:09.122 INFO:tasks.rados.rados.0.smithi119.stdout:423: set_chunk oid 24 target oid 24 low_tier 2024-09-18T11:30:09.122 INFO:tasks.rados.rados.0.smithi119.stdout:423: set_chunk oid 24 offset: 1211904 length: 256000 target oid smithi11939506-24 low_tier offset: 1211904 snap -1 2024-09-18T11:30:09.122 INFO:tasks.rados.rados.0.smithi119.stdout:424: set_chunk oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.122 INFO:tasks.rados.rados.0.smithi119.stdout:424: set_chunk oid 25 offset: 1490944 length: 559616 target oid smithi11939506-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1490944 snap -1 2024-09-18T11:30:09.122 INFO:tasks.rados.rados.0.smithi119.stdout:425: set_chunk oid 26 target oid 26 low_tier 2024-09-18T11:30:09.123 INFO:tasks.rados.rados.0.smithi119.stdout:425: set_chunk oid 26 offset: 538112 length: 1760256 target oid smithi11939506-26 low_tier offset: 538112 snap -1 2024-09-18T11:30:09.123 INFO:tasks.rados.rados.0.smithi119.stdout:426: set_chunk oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.123 INFO:tasks.rados.rados.0.smithi119.stdout:426: set_chunk oid 27 offset: 840704 length: 829952 target oid smithi11939506-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 840704 snap -1 2024-09-18T11:30:09.123 INFO:tasks.rados.rados.0.smithi119.stdout:427: set_chunk oid 28 target oid 28 low_tier 2024-09-18T11:30:09.123 INFO:tasks.rados.rados.0.smithi119.stdout:427: set_chunk oid 28 offset: 6144 length: 616448 target oid smithi11939506-28 low_tier offset: 6144 snap -1 2024-09-18T11:30:09.123 INFO:tasks.rados.rados.0.smithi119.stdout:428: set_chunk oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.123 INFO:tasks.rados.rados.0.smithi119.stdout:428: set_chunk oid 29 offset: 1674240 length: 634880 target oid smithi11939506-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1674240 snap -1 2024-09-18T11:30:09.123 INFO:tasks.rados.rados.0.smithi119.stdout:429: set_chunk oid 30 target oid 30 low_tier 2024-09-18T11:30:09.123 INFO:tasks.rados.rados.0.smithi119.stdout:429: set_chunk oid 30 offset: 1382400 length: 760832 target oid smithi11939506-30 low_tier offset: 1382400 snap -1 2024-09-18T11:30:09.123 INFO:tasks.rados.rados.0.smithi119.stdout:430: set_chunk oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.124 INFO:tasks.rados.rados.0.smithi119.stdout:430: set_chunk oid 31 offset: 295424 length: 1909760 target oid smithi11939506-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 295424 snap -1 2024-09-18T11:30:09.124 INFO:tasks.rados.rados.0.smithi119.stdout:431: set_chunk oid 32 target oid 32 low_tier 2024-09-18T11:30:09.124 INFO:tasks.rados.rados.0.smithi119.stdout:431: set_chunk oid 32 offset: 150016 length: 1190912 target oid smithi11939506-32 low_tier offset: 150016 snap -1 2024-09-18T11:30:09.124 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.124 INFO:tasks.rados.rados.0.smithi119.stdout:419: finishing set_chunk to oid 20 2024-09-18T11:30:09.124 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 20 set_chunk 20 low_tier returned error code -95 offset: 291840 length: 613888 2024-09-18T11:30:09.124 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 20 v 0 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-18T11:30:09.124 INFO:tasks.rados.rados.0.smithi119.stdout:419: done (15 left) 2024-09-18T11:30:09.124 INFO:tasks.rados.rados.0.smithi119.stdout:432: set_chunk oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.124 INFO:tasks.rados.rados.0.smithi119.stdout:432: set_chunk oid 33 offset: 1564160 length: 684544 target oid smithi11939506-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1564160 snap -1 2024-09-18T11:30:09.124 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.124 INFO:tasks.rados.rados.0.smithi119.stdout:417: finishing set_chunk to oid 18 2024-09-18T11:30:09.125 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 18 set_chunk 18 low_tier returned error code -95 offset: 285184 length: 1868800 2024-09-18T11:30:09.125 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 0 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-18T11:30:09.125 INFO:tasks.rados.rados.0.smithi119.stdout:422: finishing set_chunk to oid 23 2024-09-18T11:30:09.125 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 23 set_chunk 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 383488 length: 1105920 2024-09-18T11:30:09.125 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 0 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-18T11:30:09.125 INFO:tasks.rados.rados.0.smithi119.stdout:424: finishing set_chunk to oid 25 2024-09-18T11:30:09.125 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 25 set_chunk 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1490944 length: 559616 2024-09-18T11:30:09.125 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 0 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-18T11:30:09.125 INFO:tasks.rados.rados.0.smithi119.stdout:417: done (15 left) 2024-09-18T11:30:09.125 INFO:tasks.rados.rados.0.smithi119.stdout:422: done (14 left) 2024-09-18T11:30:09.125 INFO:tasks.rados.rados.0.smithi119.stdout:424: done (13 left) 2024-09-18T11:30:09.125 INFO:tasks.rados.rados.0.smithi119.stdout:433: set_chunk oid 34 target oid 34 low_tier 2024-09-18T11:30:09.126 INFO:tasks.rados.rados.0.smithi119.stdout:433: set_chunk oid 34 offset: 1720832 length: 294400 target oid smithi11939506-34 low_tier offset: 1720832 snap -1 2024-09-18T11:30:09.126 INFO:tasks.rados.rados.0.smithi119.stdout:434: set_chunk oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.126 INFO:tasks.rados.rados.0.smithi119.stdout:434: set_chunk oid 35 offset: 1318912 length: 207360 target oid smithi11939506-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1318912 snap -1 2024-09-18T11:30:09.126 INFO:tasks.rados.rados.0.smithi119.stdout:435: set_chunk oid 36 target oid 36 low_tier 2024-09-18T11:30:09.126 INFO:tasks.rados.rados.0.smithi119.stdout:435: set_chunk oid 36 offset: 320512 length: 1148416 target oid smithi11939506-36 low_tier offset: 320512 snap -1 2024-09-18T11:30:09.126 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.126 INFO:tasks.rados.rados.0.smithi119.stdout:425: finishing set_chunk to oid 26 2024-09-18T11:30:09.126 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 26 set_chunk 26 low_tier returned error code -95 offset: 538112 length: 1760256 2024-09-18T11:30:09.126 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 0 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-18T11:30:09.126 INFO:tasks.rados.rados.0.smithi119.stdout:418: finishing set_chunk to oid 19 2024-09-18T11:30:09.126 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 19 set_chunk 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 638976 length: 1193472 2024-09-18T11:30:09.126 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 0 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-18T11:30:09.127 INFO:tasks.rados.rados.0.smithi119.stdout:420: finishing set_chunk to oid 21 2024-09-18T11:30:09.127 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 21 set_chunk 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 341504 length: 172032 2024-09-18T11:30:09.127 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 0 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-18T11:30:09.127 INFO:tasks.rados.rados.0.smithi119.stdout:421: finishing set_chunk to oid 22 2024-09-18T11:30:09.127 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 22 set_chunk 22 low_tier returned error code -95 offset: 155136 length: 480768 2024-09-18T11:30:09.127 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 0 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-18T11:30:09.127 INFO:tasks.rados.rados.0.smithi119.stdout:416: finishing set_chunk to oid 17 2024-09-18T11:30:09.127 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 17 set_chunk 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1072640 length: 1041408 2024-09-18T11:30:09.127 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 0 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-18T11:30:09.127 INFO:tasks.rados.rados.0.smithi119.stdout:428: finishing set_chunk to oid 29 2024-09-18T11:30:09.127 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 29 set_chunk 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1674240 length: 634880 2024-09-18T11:30:09.127 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 0 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-18T11:30:09.128 INFO:tasks.rados.rados.0.smithi119.stdout:423: finishing set_chunk to oid 24 2024-09-18T11:30:09.128 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 24 set_chunk 24 low_tier returned error code -95 offset: 1211904 length: 256000 2024-09-18T11:30:09.128 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 0 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-18T11:30:09.128 INFO:tasks.rados.rados.0.smithi119.stdout:429: finishing set_chunk to oid 30 2024-09-18T11:30:09.128 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 30 set_chunk 30 low_tier returned error code -95 offset: 1382400 length: 760832 2024-09-18T11:30:09.128 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 30 v 0 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-18T11:30:09.128 INFO:tasks.rados.rados.0.smithi119.stdout:426: finishing set_chunk to oid 27 2024-09-18T11:30:09.128 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 27 set_chunk 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 840704 length: 829952 2024-09-18T11:30:09.128 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 0 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T11:30:09.128 INFO:tasks.rados.rados.0.smithi119.stdout:432: finishing set_chunk to oid 33 2024-09-18T11:30:09.128 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 33 set_chunk 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1564160 length: 684544 2024-09-18T11:30:09.129 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 33 v 0 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-18T11:30:09.129 INFO:tasks.rados.rados.0.smithi119.stdout:430: finishing set_chunk to oid 31 2024-09-18T11:30:09.129 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 31 set_chunk 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 295424 length: 1909760 2024-09-18T11:30:09.129 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 0 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-18T11:30:09.129 INFO:tasks.rados.rados.0.smithi119.stdout:431: finishing set_chunk to oid 32 2024-09-18T11:30:09.129 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 32 set_chunk 32 low_tier returned error code -95 offset: 150016 length: 1190912 2024-09-18T11:30:09.129 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 0 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-18T11:30:09.129 INFO:tasks.rados.rados.0.smithi119.stdout:416: done (15 left) 2024-09-18T11:30:09.129 INFO:tasks.rados.rados.0.smithi119.stdout:418: done (14 left) 2024-09-18T11:30:09.129 INFO:tasks.rados.rados.0.smithi119.stdout:420: done (13 left) 2024-09-18T11:30:09.129 INFO:tasks.rados.rados.0.smithi119.stdout:421: done (12 left) 2024-09-18T11:30:09.129 INFO:tasks.rados.rados.0.smithi119.stdout:423: done (11 left) 2024-09-18T11:30:09.129 INFO:tasks.rados.rados.0.smithi119.stdout:425: done (10 left) 2024-09-18T11:30:09.129 INFO:tasks.rados.rados.0.smithi119.stdout:426: done (9 left) 2024-09-18T11:30:09.130 INFO:tasks.rados.rados.0.smithi119.stdout:428: done (8 left) 2024-09-18T11:30:09.130 INFO:tasks.rados.rados.0.smithi119.stdout:429: done (7 left) 2024-09-18T11:30:09.130 INFO:tasks.rados.rados.0.smithi119.stdout:430: done (6 left) 2024-09-18T11:30:09.130 INFO:tasks.rados.rados.0.smithi119.stdout:431: done (5 left) 2024-09-18T11:30:09.130 INFO:tasks.rados.rados.0.smithi119.stdout:432: done (4 left) 2024-09-18T11:30:09.130 INFO:tasks.rados.rados.0.smithi119.stdout:436: set_chunk oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.130 INFO:tasks.rados.rados.0.smithi119.stdout:436: set_chunk oid 37 offset: 392192 length: 199168 target oid smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 392192 snap -1 2024-09-18T11:30:09.130 INFO:tasks.rados.rados.0.smithi119.stdout:437: set_chunk oid 38 target oid 38 low_tier 2024-09-18T11:30:09.130 INFO:tasks.rados.rados.0.smithi119.stdout:437: set_chunk oid 38 offset: 1861120 length: 212992 target oid smithi11939506-38 low_tier offset: 1861120 snap -1 2024-09-18T11:30:09.130 INFO:tasks.rados.rados.0.smithi119.stdout:438: set_chunk oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.130 INFO:tasks.rados.rados.0.smithi119.stdout:438: set_chunk oid 39 offset: 50688 length: 2190848 target oid smithi11939506-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 50688 snap -1 2024-09-18T11:30:09.130 INFO:tasks.rados.rados.0.smithi119.stdout:439: set_chunk oid 40 target oid 40 low_tier 2024-09-18T11:30:09.130 INFO:tasks.rados.rados.0.smithi119.stdout:439: set_chunk oid 40 offset: 524800 length: 1402368 target oid smithi11939506-40 low_tier offset: 524800 snap -1 2024-09-18T11:30:09.131 INFO:tasks.rados.rados.0.smithi119.stdout:440: set_chunk oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.131 INFO:tasks.rados.rados.0.smithi119.stdout:440: set_chunk oid 41 offset: 816640 length: 974848 target oid smithi11939506-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 816640 snap -1 2024-09-18T11:30:09.131 INFO:tasks.rados.rados.0.smithi119.stdout:441: set_chunk oid 42 target oid 42 low_tier 2024-09-18T11:30:09.131 INFO:tasks.rados.rados.0.smithi119.stdout:441: set_chunk oid 42 offset: 1627136 length: 663040 target oid smithi11939506-42 low_tier offset: 1627136 snap -1 2024-09-18T11:30:09.131 INFO:tasks.rados.rados.0.smithi119.stdout:442: set_chunk oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.131 INFO:tasks.rados.rados.0.smithi119.stdout:442: set_chunk oid 43 offset: 1754112 length: 87040 target oid smithi11939506-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1754112 snap -1 2024-09-18T11:30:09.131 INFO:tasks.rados.rados.0.smithi119.stdout:443: set_chunk oid 44 target oid 44 low_tier 2024-09-18T11:30:09.131 INFO:tasks.rados.rados.0.smithi119.stdout:443: set_chunk oid 44 offset: 1218048 length: 1068032 target oid smithi11939506-44 low_tier offset: 1218048 snap -1 2024-09-18T11:30:09.131 INFO:tasks.rados.rados.0.smithi119.stdout:444: set_chunk oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.131 INFO:tasks.rados.rados.0.smithi119.stdout:444: set_chunk oid 45 offset: 835072 length: 1749504 target oid smithi11939506-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 835072 snap -1 2024-09-18T11:30:09.131 INFO:tasks.rados.rados.0.smithi119.stdout:433: finishing set_chunk to oid 34 2024-09-18T11:30:09.131 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 34 set_chunk 34 low_tier returned error code -95 offset: 1720832 length: 294400 2024-09-18T11:30:09.131 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 0 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T11:30:09.132 INFO:tasks.rados.rados.0.smithi119.stdout:435: finishing set_chunk to oid 36 2024-09-18T11:30:09.132 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 36 set_chunk 36 low_tier returned error code -95 offset: 320512 length: 1148416 2024-09-18T11:30:09.132 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 36 v 0 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-18T11:30:09.132 INFO:tasks.rados.rados.0.smithi119.stdout:433: done (12 left) 2024-09-18T11:30:09.132 INFO:tasks.rados.rados.0.smithi119.stdout:435: done (11 left) 2024-09-18T11:30:09.132 INFO:tasks.rados.rados.0.smithi119.stdout:445: set_chunk oid 46 target oid 46 low_tier 2024-09-18T11:30:09.132 INFO:tasks.rados.rados.0.smithi119.stdout:445: set_chunk oid 46 offset: 377344 length: 2358784 target oid smithi11939506-46 low_tier offset: 377344 snap -1 2024-09-18T11:30:09.132 INFO:tasks.rados.rados.0.smithi119.stdout:446: set_chunk oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.132 INFO:tasks.rados.rados.0.smithi119.stdout:446: set_chunk oid 47 offset: 1020416 length: 980480 target oid smithi11939506-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1020416 snap -1 2024-09-18T11:30:09.132 INFO:tasks.rados.rados.0.smithi119.stdout:447: set_chunk oid 48 target oid 48 low_tier 2024-09-18T11:30:09.132 INFO:tasks.rados.rados.0.smithi119.stdout:447: set_chunk oid 48 offset: 781312 length: 18944 target oid smithi11939506-48 low_tier offset: 781312 snap -1 2024-09-18T11:30:09.133 INFO:tasks.rados.rados.0.smithi119.stdout:448: set_chunk oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.133 INFO:tasks.rados.rados.0.smithi119.stdout:434: finishing set_chunk to oid 35 2024-09-18T11:30:09.133 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 35 set_chunk 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1318912 length: 207360 2024-09-18T11:30:09.133 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 0 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-18T11:30:09.133 INFO:tasks.rados.rados.0.smithi119.stdout:448: set_chunk oid 49 offset: 1158144 length: 412160 target oid smithi11939506-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1158144 snap -1 2024-09-18T11:30:09.133 INFO:tasks.rados.rados.0.smithi119.stdout:434: done (14 left) 2024-09-18T11:30:09.133 INFO:tasks.rados.rados.0.smithi119.stdout:449: set_chunk oid 50 target oid 50 low_tier 2024-09-18T11:30:09.133 INFO:tasks.rados.rados.0.smithi119.stdout:449: set_chunk oid 50 offset: 1445376 length: 1148416 target oid smithi11939506-50 low_tier offset: 1445376 snap -1 2024-09-18T11:30:09.133 INFO:tasks.rados.rados.0.smithi119.stdout:450: set_chunk oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.133 INFO:tasks.rados.rados.0.smithi119.stdout:450: set_chunk oid 1 offset: 192512 length: 1338368 target oid smithi11939506-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 192512 snap -1 2024-09-18T11:30:09.133 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.133 INFO:tasks.rados.rados.0.smithi119.stdout:437: finishing set_chunk to oid 38 2024-09-18T11:30:09.134 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 38 set_chunk 38 low_tier returned error code -95 offset: 1861120 length: 212992 2024-09-18T11:30:09.134 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 0 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-18T11:30:09.134 INFO:tasks.rados.rados.0.smithi119.stdout:438: finishing set_chunk to oid 39 2024-09-18T11:30:09.134 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 39 set_chunk 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 50688 length: 2190848 2024-09-18T11:30:09.134 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 39 v 0 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-18T11:30:09.134 INFO:tasks.rados.rados.0.smithi119.stdout:440: finishing set_chunk to oid 41 2024-09-18T11:30:09.134 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 41 set_chunk 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 816640 length: 974848 2024-09-18T11:30:09.134 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 41 v 0 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-18T11:30:09.134 INFO:tasks.rados.rados.0.smithi119.stdout:443: finishing set_chunk to oid 44 2024-09-18T11:30:09.134 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 44 set_chunk 44 low_tier returned error code -95 offset: 1218048 length: 1068032 2024-09-18T11:30:09.134 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 44 v 0 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-18T11:30:09.134 INFO:tasks.rados.rados.0.smithi119.stdout:441: finishing set_chunk to oid 42 2024-09-18T11:30:09.134 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 42 set_chunk 42 low_tier returned error code -95 offset: 1627136 length: 663040 2024-09-18T11:30:09.134 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 0 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T11:30:09.135 INFO:tasks.rados.rados.0.smithi119.stdout:439: finishing set_chunk to oid 40 2024-09-18T11:30:09.135 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 40 set_chunk 40 low_tier returned error code -95 offset: 524800 length: 1402368 2024-09-18T11:30:09.135 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 0 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-18T11:30:09.135 INFO:tasks.rados.rados.0.smithi119.stdout:446: finishing set_chunk to oid 47 2024-09-18T11:30:09.135 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 47 set_chunk 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1020416 length: 980480 2024-09-18T11:30:09.135 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 0 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-18T11:30:09.135 INFO:tasks.rados.rados.0.smithi119.stdout:437: done (15 left) 2024-09-18T11:30:09.135 INFO:tasks.rados.rados.0.smithi119.stdout:438: done (14 left) 2024-09-18T11:30:09.135 INFO:tasks.rados.rados.0.smithi119.stdout:439: done (13 left) 2024-09-18T11:30:09.135 INFO:tasks.rados.rados.0.smithi119.stdout:440: done (12 left) 2024-09-18T11:30:09.135 INFO:tasks.rados.rados.0.smithi119.stdout:441: done (11 left) 2024-09-18T11:30:09.135 INFO:tasks.rados.rados.0.smithi119.stdout:443: done (10 left) 2024-09-18T11:30:09.135 INFO:tasks.rados.rados.0.smithi119.stdout:446: done (9 left) 2024-09-18T11:30:09.135 INFO:tasks.rados.rados.0.smithi119.stdout:451: set_chunk oid 2 target oid 2 low_tier 2024-09-18T11:30:09.136 INFO:tasks.rados.rados.0.smithi119.stdout:451: set_chunk oid 2 offset: 849920 length: 194048 target oid smithi11939506-2 low_tier offset: 849920 snap -1 2024-09-18T11:30:09.136 INFO:tasks.rados.rados.0.smithi119.stdout:452: set_chunk oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.136 INFO:tasks.rados.rados.0.smithi119.stdout:452: set_chunk oid 3 offset: 707584 length: 124928 target oid smithi11939506-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 707584 snap -1 2024-09-18T11:30:09.136 INFO:tasks.rados.rados.0.smithi119.stdout:453: set_chunk oid 4 target oid 4 low_tier 2024-09-18T11:30:09.136 INFO:tasks.rados.rados.0.smithi119.stdout:453: set_chunk oid 4 offset: 289792 length: 890880 target oid smithi11939506-4 low_tier offset: 289792 snap -1 2024-09-18T11:30:09.136 INFO:tasks.rados.rados.0.smithi119.stdout:454: set_chunk oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.136 INFO:tasks.rados.rados.0.smithi119.stdout:445: finishing set_chunk to oid 46 2024-09-18T11:30:09.136 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 46 set_chunk 46 low_tier returned error code -95 offset: 377344 length: 2358784 2024-09-18T11:30:09.136 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 0 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-18T11:30:09.136 INFO:tasks.rados.rados.0.smithi119.stdout:444: finishing set_chunk to oid 45 2024-09-18T11:30:09.136 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 45 set_chunk 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 835072 length: 1749504 2024-09-18T11:30:09.136 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 0 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-18T11:30:09.136 INFO:tasks.rados.rados.0.smithi119.stdout:450: finishing set_chunk to oid 1 2024-09-18T11:30:09.137 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 1 set_chunk 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 192512 length: 1338368 2024-09-18T11:30:09.137 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 0 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-18T11:30:09.137 INFO:tasks.rados.rados.0.smithi119.stdout:442: finishing set_chunk to oid 43 2024-09-18T11:30:09.137 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 43 set_chunk 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1754112 length: 87040 2024-09-18T11:30:09.137 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-18T11:30:09.137 INFO:tasks.rados.rados.0.smithi119.stdout:454: set_chunk oid 5 offset: 1492480 length: 235008 target oid smithi11939506-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1492480 snap -1 2024-09-18T11:30:09.137 INFO:tasks.rados.rados.0.smithi119.stdout:442: done (12 left) 2024-09-18T11:30:09.137 INFO:tasks.rados.rados.0.smithi119.stdout:444: done (11 left) 2024-09-18T11:30:09.137 INFO:tasks.rados.rados.0.smithi119.stdout:445: done (10 left) 2024-09-18T11:30:09.137 INFO:tasks.rados.rados.0.smithi119.stdout:450: done (9 left) 2024-09-18T11:30:09.137 INFO:tasks.rados.rados.0.smithi119.stdout:455: set_chunk oid 6 target oid 6 low_tier 2024-09-18T11:30:09.137 INFO:tasks.rados.rados.0.smithi119.stdout:455: set_chunk oid 6 offset: 325632 length: 1092608 target oid smithi11939506-6 low_tier offset: 325632 snap -1 2024-09-18T11:30:09.137 INFO:tasks.rados.rados.0.smithi119.stdout:447: finishing set_chunk to oid 48 2024-09-18T11:30:09.137 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 48 set_chunk 48 low_tier returned error code -95 offset: 781312 length: 18944 2024-09-18T11:30:09.138 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 48 v 0 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-18T11:30:09.138 INFO:tasks.rados.rados.0.smithi119.stdout:449: finishing set_chunk to oid 50 2024-09-18T11:30:09.138 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 50 set_chunk 50 low_tier returned error code -95 offset: 1445376 length: 1148416 2024-09-18T11:30:09.138 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 0 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-18T11:30:09.138 INFO:tasks.rados.rados.0.smithi119.stdout:447: done (9 left) 2024-09-18T11:30:09.138 INFO:tasks.rados.rados.0.smithi119.stdout:449: done (8 left) 2024-09-18T11:30:09.138 INFO:tasks.rados.rados.0.smithi119.stdout:456: set_chunk oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.138 INFO:tasks.rados.rados.0.smithi119.stdout:456: set_chunk oid 7 offset: 1068544 length: 391680 target oid smithi11939506-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1068544 snap -1 2024-09-18T11:30:09.138 INFO:tasks.rados.rados.0.smithi119.stdout:457: set_chunk oid 8 target oid 8 low_tier 2024-09-18T11:30:09.138 INFO:tasks.rados.rados.0.smithi119.stdout:457: set_chunk oid 8 offset: 783360 length: 1028096 target oid smithi11939506-8 low_tier offset: 783360 snap -1 2024-09-18T11:30:09.138 INFO:tasks.rados.rados.0.smithi119.stdout:458: set_chunk oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.138 INFO:tasks.rados.rados.0.smithi119.stdout:448: finishing set_chunk to oid 49 2024-09-18T11:30:09.138 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 49 set_chunk 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1158144 length: 412160 2024-09-18T11:30:09.139 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 0 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-18T11:30:09.139 INFO:tasks.rados.rados.0.smithi119.stdout:458: set_chunk oid 9 offset: 674304 length: 543232 target oid smithi11939506-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 674304 snap -1 2024-09-18T11:30:09.139 INFO:tasks.rados.rados.0.smithi119.stdout:448: done (10 left) 2024-09-18T11:30:09.139 INFO:tasks.rados.rados.0.smithi119.stdout:459: set_chunk oid 10 target oid 10 low_tier 2024-09-18T11:30:09.139 INFO:tasks.rados.rados.0.smithi119.stdout:459: set_chunk oid 10 offset: 497664 length: 1257472 target oid smithi11939506-10 low_tier offset: 497664 snap -1 2024-09-18T11:30:09.139 INFO:tasks.rados.rados.0.smithi119.stdout:460: set_chunk oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.139 INFO:tasks.rados.rados.0.smithi119.stdout:460: set_chunk oid 11 offset: 276992 length: 1611776 target oid smithi11939506-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 276992 snap -1 2024-09-18T11:30:09.139 INFO:tasks.rados.rados.0.smithi119.stdout:461: set_chunk oid 12 target oid 12 low_tier 2024-09-18T11:30:09.139 INFO:tasks.rados.rados.0.smithi119.stdout:461: set_chunk oid 12 offset: 429056 length: 882688 target oid smithi11939506-12 low_tier offset: 429056 snap -1 2024-09-18T11:30:09.139 INFO:tasks.rados.rados.0.smithi119.stdout:462: set_chunk oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.139 INFO:tasks.rados.rados.0.smithi119.stdout:462: set_chunk oid 13 offset: 329728 length: 535552 target oid smithi11939506-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 329728 snap -1 2024-09-18T11:30:09.139 INFO:tasks.rados.rados.0.smithi119.stdout:463: set_chunk oid 14 target oid 14 low_tier 2024-09-18T11:30:09.139 INFO:tasks.rados.rados.0.smithi119.stdout:463: set_chunk oid 14 offset: 47616 length: 1719808 target oid smithi11939506-14 low_tier offset: 47616 snap -1 2024-09-18T11:30:09.140 INFO:tasks.rados.rados.0.smithi119.stdout:464: set_chunk oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.140 INFO:tasks.rados.rados.0.smithi119.stdout:464: set_chunk oid 15 offset: 4608 length: 314368 target oid smithi11939506-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 4608 snap -1 2024-09-18T11:30:09.140 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.140 INFO:tasks.rados.rados.0.smithi119.stdout:453: finishing set_chunk to oid 4 2024-09-18T11:30:09.140 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 4 set_chunk 4 low_tier returned error code -95 offset: 289792 length: 890880 2024-09-18T11:30:09.140 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 4 v 0 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-18T11:30:09.140 INFO:tasks.rados.rados.0.smithi119.stdout:455: finishing set_chunk to oid 6 2024-09-18T11:30:09.140 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 6 set_chunk 6 low_tier returned error code -95 offset: 325632 length: 1092608 2024-09-18T11:30:09.140 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 6 v 0 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-18T11:30:09.140 INFO:tasks.rados.rados.0.smithi119.stdout:451: finishing set_chunk to oid 2 2024-09-18T11:30:09.140 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 2 set_chunk 2 low_tier returned error code -95 offset: 849920 length: 194048 2024-09-18T11:30:09.140 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 0 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-18T11:30:09.140 INFO:tasks.rados.rados.0.smithi119.stdout:451: done (15 left) 2024-09-18T11:30:09.141 INFO:tasks.rados.rados.0.smithi119.stdout:453: done (14 left) 2024-09-18T11:30:09.141 INFO:tasks.rados.rados.0.smithi119.stdout:455: done (13 left) 2024-09-18T11:30:09.141 INFO:tasks.rados.rados.0.smithi119.stdout:465: set_chunk oid 16 target oid 16 low_tier 2024-09-18T11:30:09.141 INFO:tasks.rados.rados.0.smithi119.stdout:465: set_chunk oid 16 offset: 919552 length: 1112576 target oid smithi11939506-16 low_tier offset: 919552 snap -1 2024-09-18T11:30:09.141 INFO:tasks.rados.rados.0.smithi119.stdout:466: set_chunk oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.141 INFO:tasks.rados.rados.0.smithi119.stdout:466: set_chunk oid 17 offset: 770048 length: 1181184 target oid smithi11939506-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 770048 snap -1 2024-09-18T11:30:09.141 INFO:tasks.rados.rados.0.smithi119.stdout:467: set_chunk oid 18 target oid 18 low_tier 2024-09-18T11:30:09.141 INFO:tasks.rados.rados.0.smithi119.stdout:467: set_chunk oid 18 offset: 1001472 length: 896512 target oid smithi11939506-18 low_tier offset: 1001472 snap -1 2024-09-18T11:30:09.141 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.141 INFO:tasks.rados.rados.0.smithi119.stdout:452: finishing set_chunk to oid 3 2024-09-18T11:30:09.141 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 3 set_chunk 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 707584 length: 124928 2024-09-18T11:30:09.141 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 0 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-18T11:30:09.141 INFO:tasks.rados.rados.0.smithi119.stdout:454: finishing set_chunk to oid 5 2024-09-18T11:30:09.141 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 5 set_chunk 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1492480 length: 235008 2024-09-18T11:30:09.142 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 0 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-18T11:30:09.142 INFO:tasks.rados.rados.0.smithi119.stdout:457: finishing set_chunk to oid 8 2024-09-18T11:30:09.142 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 8 set_chunk 8 low_tier returned error code -95 offset: 783360 length: 1028096 2024-09-18T11:30:09.142 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 8 v 0 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-18T11:30:09.142 INFO:tasks.rados.rados.0.smithi119.stdout:456: finishing set_chunk to oid 7 2024-09-18T11:30:09.142 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 7 set_chunk 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1068544 length: 391680 2024-09-18T11:30:09.142 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 7 v 0 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-18T11:30:09.142 INFO:tasks.rados.rados.0.smithi119.stdout:452: done (15 left) 2024-09-18T11:30:09.142 INFO:tasks.rados.rados.0.smithi119.stdout:454: done (14 left) 2024-09-18T11:30:09.142 INFO:tasks.rados.rados.0.smithi119.stdout:456: done (13 left) 2024-09-18T11:30:09.142 INFO:tasks.rados.rados.0.smithi119.stdout:457: done (12 left) 2024-09-18T11:30:09.142 INFO:tasks.rados.rados.0.smithi119.stdout:468: set_chunk oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.142 INFO:tasks.rados.rados.0.smithi119.stdout:468: set_chunk oid 19 offset: 1544192 length: 674304 target oid smithi11939506-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1544192 snap -1 2024-09-18T11:30:09.142 INFO:tasks.rados.rados.0.smithi119.stdout:461: finishing set_chunk to oid 12 2024-09-18T11:30:09.143 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 12 set_chunk 12 low_tier returned error code -95 offset: 429056 length: 882688 2024-09-18T11:30:09.143 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 12 v 0 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-18T11:30:09.143 INFO:tasks.rados.rados.0.smithi119.stdout:463: finishing set_chunk to oid 14 2024-09-18T11:30:09.143 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 14 set_chunk 14 low_tier returned error code -95 offset: 47616 length: 1719808 2024-09-18T11:30:09.143 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 0 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-18T11:30:09.143 INFO:tasks.rados.rados.0.smithi119.stdout:461: done (12 left) 2024-09-18T11:30:09.143 INFO:tasks.rados.rados.0.smithi119.stdout:463: done (11 left) 2024-09-18T11:30:09.143 INFO:tasks.rados.rados.0.smithi119.stdout:469: set_chunk oid 20 target oid 20 low_tier 2024-09-18T11:30:09.143 INFO:tasks.rados.rados.0.smithi119.stdout:469: set_chunk oid 20 offset: 1904128 length: 229376 target oid smithi11939506-20 low_tier offset: 1904128 snap -1 2024-09-18T11:30:09.143 INFO:tasks.rados.rados.0.smithi119.stdout:470: set_chunk oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.143 INFO:tasks.rados.rados.0.smithi119.stdout:470: set_chunk oid 21 offset: 1591808 length: 269824 target oid smithi11939506-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1591808 snap -1 2024-09-18T11:30:09.143 INFO:tasks.rados.rados.0.smithi119.stdout:471: set_chunk oid 22 target oid 22 low_tier 2024-09-18T11:30:09.143 INFO:tasks.rados.rados.0.smithi119.stdout:471: set_chunk oid 22 offset: 275456 length: 1189376 target oid smithi11939506-22 low_tier offset: 275456 snap -1 2024-09-18T11:30:09.143 INFO:tasks.rados.rados.0.smithi119.stdout:472: set_chunk oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.143 INFO:tasks.rados.rados.0.smithi119.stdout:472: set_chunk oid 23 offset: 351232 length: 1848832 target oid smithi11939506-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 351232 snap -1 2024-09-18T11:30:09.143 INFO:tasks.rados.rados.0.smithi119.stdout:473: set_chunk oid 24 target oid 24 low_tier 2024-09-18T11:30:09.144 INFO:tasks.rados.rados.0.smithi119.stdout:473: set_chunk oid 24 offset: 524800 length: 38400 target oid smithi11939506-24 low_tier offset: 524800 snap -1 2024-09-18T11:30:09.144 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.144 INFO:tasks.rados.rados.0.smithi119.stdout:467: finishing set_chunk to oid 18 2024-09-18T11:30:09.144 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 18 set_chunk 18 low_tier returned error code -95 offset: 1001472 length: 896512 2024-09-18T11:30:09.144 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 0 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-18T11:30:09.144 INFO:tasks.rados.rados.0.smithi119.stdout:460: finishing set_chunk to oid 11 2024-09-18T11:30:09.144 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 11 set_chunk 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 276992 length: 1611776 2024-09-18T11:30:09.144 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 11 v 0 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-18T11:30:09.144 INFO:tasks.rados.rados.0.smithi119.stdout:460: done (15 left) 2024-09-18T11:30:09.144 INFO:tasks.rados.rados.0.smithi119.stdout:467: done (14 left) 2024-09-18T11:30:09.144 INFO:tasks.rados.rados.0.smithi119.stdout:474: set_chunk oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.144 INFO:tasks.rados.rados.0.smithi119.stdout:474: set_chunk oid 25 offset: 1600000 length: 48128 target oid smithi11939506-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1600000 snap -1 2024-09-18T11:30:09.144 INFO:tasks.rados.rados.0.smithi119.stdout:475: set_chunk oid 26 target oid 26 low_tier 2024-09-18T11:30:09.144 INFO:tasks.rados.rados.0.smithi119.stdout:475: set_chunk oid 26 offset: 596480 length: 1755648 target oid smithi11939506-26 low_tier offset: 596480 snap -1 2024-09-18T11:30:09.145 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.145 INFO:tasks.rados.rados.0.smithi119.stdout:458: finishing set_chunk to oid 9 2024-09-18T11:30:09.145 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 9 set_chunk 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 674304 length: 543232 2024-09-18T11:30:09.145 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 0 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T11:30:09.145 INFO:tasks.rados.rados.0.smithi119.stdout:458: done (15 left) 2024-09-18T11:30:09.145 INFO:tasks.rados.rados.0.smithi119.stdout:476: set_chunk oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.145 INFO:tasks.rados.rados.0.smithi119.stdout:476: set_chunk oid 27 offset: 448512 length: 304640 target oid smithi11939506-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 448512 snap -1 2024-09-18T11:30:09.145 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.145 INFO:tasks.rados.rados.0.smithi119.stdout:469: finishing set_chunk to oid 20 2024-09-18T11:30:09.145 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 20 set_chunk 20 low_tier returned error code -95 offset: 1904128 length: 229376 2024-09-18T11:30:09.145 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 20 v 0 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-18T11:30:09.145 INFO:tasks.rados.rados.0.smithi119.stdout:472: finishing set_chunk to oid 23 2024-09-18T11:30:09.146 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 23 set_chunk 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 351232 length: 1848832 2024-09-18T11:30:09.146 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 0 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-18T11:30:09.146 INFO:tasks.rados.rados.0.smithi119.stdout:469: done (15 left) 2024-09-18T11:30:09.146 INFO:tasks.rados.rados.0.smithi119.stdout:472: done (14 left) 2024-09-18T11:30:09.146 INFO:tasks.rados.rados.0.smithi119.stdout:476 retry set_chunk ! 2024-09-18T11:30:09.146 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 14 2024-09-18T11:30:09.146 INFO:tasks.rados.rados.0.smithi119.stdout:459: finishing set_chunk to oid 10 2024-09-18T11:30:09.146 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 10 set_chunk 10 low_tier returned error code -95 offset: 497664 length: 1257472 2024-09-18T11:30:09.146 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 10 v 0 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-18T11:30:09.146 INFO:tasks.rados.rados.0.smithi119.stdout:459: done (13 left) 2024-09-18T11:30:09.146 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 13 2024-09-18T11:30:09.146 INFO:tasks.rados.rados.0.smithi119.stdout:474: finishing set_chunk to oid 25 2024-09-18T11:30:09.146 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 25 set_chunk 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1600000 length: 48128 2024-09-18T11:30:09.146 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 0 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-18T11:30:09.146 INFO:tasks.rados.rados.0.smithi119.stdout:474: done (12 left) 2024-09-18T11:30:09.147 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 12 2024-09-18T11:30:09.147 INFO:tasks.rados.rados.0.smithi119.stdout:464: finishing set_chunk to oid 15 2024-09-18T11:30:09.147 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 15 set_chunk 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 4608 length: 314368 2024-09-18T11:30:09.147 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 0 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-18T11:30:09.147 INFO:tasks.rados.rados.0.smithi119.stdout:465: finishing set_chunk to oid 16 2024-09-18T11:30:09.147 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 16 set_chunk 16 low_tier returned error code -95 offset: 919552 length: 1112576 2024-09-18T11:30:09.147 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 0 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-18T11:30:09.147 INFO:tasks.rados.rados.0.smithi119.stdout:464: done (11 left) 2024-09-18T11:30:09.147 INFO:tasks.rados.rados.0.smithi119.stdout:465: done (10 left) 2024-09-18T11:30:09.147 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 10 2024-09-18T11:30:09.147 INFO:tasks.rados.rados.0.smithi119.stdout:475: finishing set_chunk to oid 26 2024-09-18T11:30:09.147 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 26 set_chunk 26 low_tier returned error code -95 offset: 596480 length: 1755648 2024-09-18T11:30:09.147 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 0 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-18T11:30:09.147 INFO:tasks.rados.rados.0.smithi119.stdout:462: finishing set_chunk to oid 13 2024-09-18T11:30:09.148 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 13 set_chunk 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 329728 length: 535552 2024-09-18T11:30:09.148 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 13 v 0 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-18T11:30:09.148 INFO:tasks.rados.rados.0.smithi119.stdout:476: finishing set_chunk to oid 27 2024-09-18T11:30:09.148 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 27 set_chunk 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 448512 length: 304640 2024-09-18T11:30:09.148 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 0 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T11:30:09.148 INFO:tasks.rados.rados.0.smithi119.stdout:462: done (9 left) 2024-09-18T11:30:09.148 INFO:tasks.rados.rados.0.smithi119.stdout:475: done (8 left) 2024-09-18T11:30:09.148 INFO:tasks.rados.rados.0.smithi119.stdout:476: done (7 left) 2024-09-18T11:30:09.148 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 7 2024-09-18T11:30:09.148 INFO:tasks.rados.rados.0.smithi119.stdout:466: finishing set_chunk to oid 17 2024-09-18T11:30:09.148 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 17 set_chunk 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 770048 length: 1181184 2024-09-18T11:30:09.148 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 0 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-18T11:30:09.148 INFO:tasks.rados.rados.0.smithi119.stdout:466: done (6 left) 2024-09-18T11:30:09.148 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 6 2024-09-18T11:30:09.149 INFO:tasks.rados.rados.0.smithi119.stdout:470: finishing set_chunk to oid 21 2024-09-18T11:30:09.149 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 21 set_chunk 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1591808 length: 269824 2024-09-18T11:30:09.149 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 0 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-18T11:30:09.149 INFO:tasks.rados.rados.0.smithi119.stdout:471: finishing set_chunk to oid 22 2024-09-18T11:30:09.149 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 22 set_chunk 22 low_tier returned error code -95 offset: 275456 length: 1189376 2024-09-18T11:30:09.149 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 0 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-18T11:30:09.149 INFO:tasks.rados.rados.0.smithi119.stdout:470: done (5 left) 2024-09-18T11:30:09.149 INFO:tasks.rados.rados.0.smithi119.stdout:471: done (4 left) 2024-09-18T11:30:09.149 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 4 2024-09-18T11:30:09.149 INFO:tasks.rados.rados.0.smithi119.stdout:427: finishing set_chunk to oid 28 2024-09-18T11:30:09.149 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 28 v 9 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-18T11:30:09.149 INFO:tasks.rados.rados.0.smithi119.stdout:427: done (3 left) 2024-09-18T11:30:09.149 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 3 2024-09-18T11:30:09.150 INFO:tasks.rados.rados.0.smithi119.stdout:436: finishing set_chunk to oid 37 2024-09-18T11:30:09.150 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 6 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-18T11:30:09.150 INFO:tasks.rados.rados.0.smithi119.stdout:436: done (2 left) 2024-09-18T11:30:09.150 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 2 2024-09-18T11:30:09.150 INFO:tasks.rados.rados.0.smithi119.stdout:468: finishing set_chunk to oid 19 2024-09-18T11:30:09.150 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 12 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-18T11:30:09.150 INFO:tasks.rados.rados.0.smithi119.stdout:468: done (1 left) 2024-09-18T11:30:09.150 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 1 2024-09-18T11:30:09.150 INFO:tasks.rados.rados.0.smithi119.stdout:473: finishing set_chunk to oid 24 2024-09-18T11:30:09.150 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 9 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-18T11:30:09.150 INFO:tasks.rados.rados.0.smithi119.stdout:473: done (0 left) 2024-09-18T11:30:09.150 INFO:tasks.rados.rados.0.smithi119.stdout:477: set_chunk oid 28 target oid 28 low_tier 2024-09-18T11:30:09.150 INFO:tasks.rados.rados.0.smithi119.stdout:477: set_chunk oid 28 offset: 1400832 length: 603648 target oid smithi11939506-28 low_tier offset: 1400832 snap -1 2024-09-18T11:30:09.150 INFO:tasks.rados.rados.0.smithi119.stdout:478: set_chunk oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.151 INFO:tasks.rados.rados.0.smithi119.stdout:478: set_chunk oid 29 offset: 289792 length: 458240 target oid smithi11939506-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 289792 snap -1 2024-09-18T11:30:09.151 INFO:tasks.rados.rados.0.smithi119.stdout:479: set_chunk oid 30 target oid 30 low_tier 2024-09-18T11:30:09.151 INFO:tasks.rados.rados.0.smithi119.stdout:479: set_chunk oid 30 offset: 1852416 length: 608768 target oid smithi11939506-30 low_tier offset: 1852416 snap -1 2024-09-18T11:30:09.151 INFO:tasks.rados.rados.0.smithi119.stdout:480: set_chunk oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.151 INFO:tasks.rados.rados.0.smithi119.stdout:480: set_chunk oid 31 offset: 1432064 length: 733696 target oid smithi11939506-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1432064 snap -1 2024-09-18T11:30:09.151 INFO:tasks.rados.rados.0.smithi119.stdout:481: set_chunk oid 32 target oid 32 low_tier 2024-09-18T11:30:09.151 INFO:tasks.rados.rados.0.smithi119.stdout:481: set_chunk oid 32 offset: 550912 length: 1549312 target oid smithi11939506-32 low_tier offset: 550912 snap -1 2024-09-18T11:30:09.151 INFO:tasks.rados.rados.0.smithi119.stdout:482: set_chunk oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.151 INFO:tasks.rados.rados.0.smithi119.stdout:482: set_chunk oid 33 offset: 699904 length: 495616 target oid smithi11939506-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 699904 snap -1 2024-09-18T11:30:09.151 INFO:tasks.rados.rados.0.smithi119.stdout:483: set_chunk oid 34 target oid 34 low_tier 2024-09-18T11:30:09.151 INFO:tasks.rados.rados.0.smithi119.stdout:483: set_chunk oid 34 offset: 124928 length: 544256 target oid smithi11939506-34 low_tier offset: 124928 snap -1 2024-09-18T11:30:09.151 INFO:tasks.rados.rados.0.smithi119.stdout:484: set_chunk oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.151 INFO:tasks.rados.rados.0.smithi119.stdout:484: set_chunk oid 35 offset: 395264 length: 507904 target oid smithi11939506-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 395264 snap -1 2024-09-18T11:30:09.151 INFO:tasks.rados.rados.0.smithi119.stdout:485: set_chunk oid 36 target oid 36 low_tier 2024-09-18T11:30:09.152 INFO:tasks.rados.rados.0.smithi119.stdout:485: set_chunk oid 36 offset: 634880 length: 1616384 target oid smithi11939506-36 low_tier offset: 634880 snap -1 2024-09-18T11:30:09.152 INFO:tasks.rados.rados.0.smithi119.stdout:486: set_chunk oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.152 INFO:tasks.rados.rados.0.smithi119.stdout:486: set_chunk oid 37 offset: 1097728 length: 1131008 target oid smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1097728 snap -1 2024-09-18T11:30:09.152 INFO:tasks.rados.rados.0.smithi119.stdout:487: set_chunk oid 38 target oid 38 low_tier 2024-09-18T11:30:09.152 INFO:tasks.rados.rados.0.smithi119.stdout:487: set_chunk oid 38 offset: 1137664 length: 1207296 target oid smithi11939506-38 low_tier offset: 1137664 snap -1 2024-09-18T11:30:09.152 INFO:tasks.rados.rados.0.smithi119.stdout:488: set_chunk oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.152 INFO:tasks.rados.rados.0.smithi119.stdout:488: set_chunk oid 39 offset: 115712 length: 1434112 target oid smithi11939506-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 115712 snap -1 2024-09-18T11:30:09.152 INFO:tasks.rados.rados.0.smithi119.stdout:489: set_chunk oid 40 target oid 40 low_tier 2024-09-18T11:30:09.152 INFO:tasks.rados.rados.0.smithi119.stdout:489: set_chunk oid 40 offset: 411648 length: 956416 target oid smithi11939506-40 low_tier offset: 411648 snap -1 2024-09-18T11:30:09.152 INFO:tasks.rados.rados.0.smithi119.stdout:490: set_chunk oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.152 INFO:tasks.rados.rados.0.smithi119.stdout:490: set_chunk oid 41 offset: 460288 length: 161792 target oid smithi11939506-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 460288 snap -1 2024-09-18T11:30:09.152 INFO:tasks.rados.rados.0.smithi119.stdout:491: set_chunk oid 42 target oid 42 low_tier 2024-09-18T11:30:09.152 INFO:tasks.rados.rados.0.smithi119.stdout:491: set_chunk oid 42 offset: 202240 length: 799232 target oid smithi11939506-42 low_tier offset: 202240 snap -1 2024-09-18T11:30:09.153 INFO:tasks.rados.rados.0.smithi119.stdout:492: set_chunk oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.153 INFO:tasks.rados.rados.0.smithi119.stdout:492: set_chunk oid 43 offset: 260608 length: 1590272 target oid smithi11939506-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 260608 snap -1 2024-09-18T11:30:09.153 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.153 INFO:tasks.rados.rados.0.smithi119.stdout:481: finishing set_chunk to oid 32 2024-09-18T11:30:09.153 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 32 set_chunk 32 low_tier returned error code -95 offset: 550912 length: 1549312 2024-09-18T11:30:09.153 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 0 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-18T11:30:09.153 INFO:tasks.rados.rados.0.smithi119.stdout:481: done (15 left) 2024-09-18T11:30:09.153 INFO:tasks.rados.rados.0.smithi119.stdout:493: set_chunk oid 44 target oid 44 low_tier 2024-09-18T11:30:09.153 INFO:tasks.rados.rados.0.smithi119.stdout:493: set_chunk oid 44 offset: 1349120 length: 669184 target oid smithi11939506-44 low_tier offset: 1349120 snap -1 2024-09-18T11:30:09.153 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.153 INFO:tasks.rados.rados.0.smithi119.stdout:477: finishing set_chunk to oid 28 2024-09-18T11:30:09.153 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 28 set_chunk 28 low_tier returned error code -95 offset: 1400832 length: 603648 2024-09-18T11:30:09.153 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 28 v 0 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-18T11:30:09.153 INFO:tasks.rados.rados.0.smithi119.stdout:480: finishing set_chunk to oid 31 2024-09-18T11:30:09.154 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 31 set_chunk 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1432064 length: 733696 2024-09-18T11:30:09.154 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 0 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-18T11:30:09.154 INFO:tasks.rados.rados.0.smithi119.stdout:478: finishing set_chunk to oid 29 2024-09-18T11:30:09.154 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 29 set_chunk 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 289792 length: 458240 2024-09-18T11:30:09.154 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 0 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-18T11:30:09.154 INFO:tasks.rados.rados.0.smithi119.stdout:477: done (15 left) 2024-09-18T11:30:09.154 INFO:tasks.rados.rados.0.smithi119.stdout:478: done (14 left) 2024-09-18T11:30:09.154 INFO:tasks.rados.rados.0.smithi119.stdout:480: done (13 left) 2024-09-18T11:30:09.154 INFO:tasks.rados.rados.0.smithi119.stdout:494: set_chunk oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.154 INFO:tasks.rados.rados.0.smithi119.stdout:494: set_chunk oid 45 offset: 544768 length: 1479168 target oid smithi11939506-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 544768 snap -1 2024-09-18T11:30:09.154 INFO:tasks.rados.rados.0.smithi119.stdout:495: set_chunk oid 46 target oid 46 low_tier 2024-09-18T11:30:09.154 INFO:tasks.rados.rados.0.smithi119.stdout:495: set_chunk oid 46 offset: 515072 length: 949248 target oid smithi11939506-46 low_tier offset: 515072 snap -1 2024-09-18T11:30:09.154 INFO:tasks.rados.rados.0.smithi119.stdout:496: set_chunk oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.154 INFO:tasks.rados.rados.0.smithi119.stdout:496: set_chunk oid 47 offset: 438272 length: 1701888 target oid smithi11939506-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 438272 snap -1 2024-09-18T11:30:09.154 INFO:tasks.rados.rados.0.smithi119.stdout:483: finishing set_chunk to oid 34 2024-09-18T11:30:09.155 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 34 set_chunk 34 low_tier returned error code -95 offset: 124928 length: 544256 2024-09-18T11:30:09.155 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 0 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T11:30:09.155 INFO:tasks.rados.rados.0.smithi119.stdout:488: finishing set_chunk to oid 39 2024-09-18T11:30:09.155 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 39 set_chunk 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 115712 length: 1434112 2024-09-18T11:30:09.155 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 39 v 0 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-18T11:30:09.155 INFO:tasks.rados.rados.0.smithi119.stdout:483: done (15 left) 2024-09-18T11:30:09.155 INFO:tasks.rados.rados.0.smithi119.stdout:488: done (14 left) 2024-09-18T11:30:09.155 INFO:tasks.rados.rados.0.smithi119.stdout:497: set_chunk oid 48 target oid 48 low_tier 2024-09-18T11:30:09.155 INFO:tasks.rados.rados.0.smithi119.stdout:497: set_chunk oid 48 offset: 1415168 length: 1220608 target oid smithi11939506-48 low_tier offset: 1415168 snap -1 2024-09-18T11:30:09.155 INFO:tasks.rados.rados.0.smithi119.stdout:498: set_chunk oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.155 INFO:tasks.rados.rados.0.smithi119.stdout:498: set_chunk oid 49 offset: 1753088 length: 160768 target oid smithi11939506-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1753088 snap -1 2024-09-18T11:30:09.155 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.155 INFO:tasks.rados.rados.0.smithi119.stdout:489: finishing set_chunk to oid 40 2024-09-18T11:30:09.155 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 40 set_chunk 40 low_tier returned error code -95 offset: 411648 length: 956416 2024-09-18T11:30:09.156 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 0 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-18T11:30:09.156 INFO:tasks.rados.rados.0.smithi119.stdout:482: finishing set_chunk to oid 33 2024-09-18T11:30:09.156 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 33 set_chunk 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 699904 length: 495616 2024-09-18T11:30:09.156 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 33 v 0 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-18T11:30:09.156 INFO:tasks.rados.rados.0.smithi119.stdout:482: done (15 left) 2024-09-18T11:30:09.156 INFO:tasks.rados.rados.0.smithi119.stdout:489: done (14 left) 2024-09-18T11:30:09.156 INFO:tasks.rados.rados.0.smithi119.stdout:499: set_chunk oid 50 target oid 50 low_tier 2024-09-18T11:30:09.156 INFO:tasks.rados.rados.0.smithi119.stdout:499: set_chunk oid 50 offset: 7168 length: 588800 target oid smithi11939506-50 low_tier offset: 7168 snap -1 2024-09-18T11:30:09.156 INFO:tasks.rados.rados.0.smithi119.stdout:491: finishing set_chunk to oid 42 2024-09-18T11:30:09.156 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 42 set_chunk 42 low_tier returned error code -95 offset: 202240 length: 799232 2024-09-18T11:30:09.156 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 0 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T11:30:09.156 INFO:tasks.rados.rados.0.smithi119.stdout:485: finishing set_chunk to oid 36 2024-09-18T11:30:09.156 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 36 set_chunk 36 low_tier returned error code -95 offset: 634880 length: 1616384 2024-09-18T11:30:09.156 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 36 v 0 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-18T11:30:09.157 INFO:tasks.rados.rados.0.smithi119.stdout:486: finishing set_chunk to oid 37 2024-09-18T11:30:09.157 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 37 set_chunk 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1097728 length: 1131008 2024-09-18T11:30:09.157 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 0 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-18T11:30:09.157 INFO:tasks.rados.rados.0.smithi119.stdout:490: finishing set_chunk to oid 41 2024-09-18T11:30:09.157 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 41 set_chunk 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 460288 length: 161792 2024-09-18T11:30:09.157 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 41 v 0 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-18T11:30:09.157 INFO:tasks.rados.rados.0.smithi119.stdout:485: done (14 left) 2024-09-18T11:30:09.157 INFO:tasks.rados.rados.0.smithi119.stdout:486: done (13 left) 2024-09-18T11:30:09.157 INFO:tasks.rados.rados.0.smithi119.stdout:490: done (12 left) 2024-09-18T11:30:09.157 INFO:tasks.rados.rados.0.smithi119.stdout:491: done (11 left) 2024-09-18T11:30:09.157 INFO:tasks.rados.rados.0.smithi119.stdout:500: set_chunk oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.157 INFO:tasks.rados.rados.0.smithi119.stdout:500: set_chunk oid 1 offset: 109568 length: 1137664 target oid smithi11939506-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 109568 snap -1 2024-09-18T11:30:09.157 INFO:tasks.rados.rados.0.smithi119.stdout:501: set_chunk oid 2 target oid 2 low_tier 2024-09-18T11:30:09.158 INFO:tasks.rados.rados.0.smithi119.stdout:501: set_chunk oid 2 offset: 1197056 length: 666112 target oid smithi11939506-2 low_tier offset: 1197056 snap -1 2024-09-18T11:30:09.158 INFO:tasks.rados.rados.0.smithi119.stdout:502: set_chunk oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.158 INFO:tasks.rados.rados.0.smithi119.stdout:502: set_chunk oid 3 offset: 696320 length: 905216 target oid smithi11939506-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 696320 snap -1 2024-09-18T11:30:09.158 INFO:tasks.rados.rados.0.smithi119.stdout:503: set_chunk oid 4 target oid 4 low_tier 2024-09-18T11:30:09.158 INFO:tasks.rados.rados.0.smithi119.stdout:503: set_chunk oid 4 offset: 24064 length: 1745408 target oid smithi11939506-4 low_tier offset: 24064 snap -1 2024-09-18T11:30:09.158 INFO:tasks.rados.rados.0.smithi119.stdout:504: set_chunk oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.158 INFO:tasks.rados.rados.0.smithi119.stdout:504: set_chunk oid 5 offset: 517120 length: 673792 target oid smithi11939506-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 517120 snap -1 2024-09-18T11:30:09.158 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.158 INFO:tasks.rados.rados.0.smithi119.stdout:492: finishing set_chunk to oid 43 2024-09-18T11:30:09.158 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 43 set_chunk 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 260608 length: 1590272 2024-09-18T11:30:09.158 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-18T11:30:09.158 INFO:tasks.rados.rados.0.smithi119.stdout:487: finishing set_chunk to oid 38 2024-09-18T11:30:09.158 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 38 set_chunk 38 low_tier returned error code -95 offset: 1137664 length: 1207296 2024-09-18T11:30:09.159 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 0 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-18T11:30:09.159 INFO:tasks.rados.rados.0.smithi119.stdout:495: finishing set_chunk to oid 46 2024-09-18T11:30:09.159 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 46 set_chunk 46 low_tier returned error code -95 offset: 515072 length: 949248 2024-09-18T11:30:09.159 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 0 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-18T11:30:09.159 INFO:tasks.rados.rados.0.smithi119.stdout:496: finishing set_chunk to oid 47 2024-09-18T11:30:09.159 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 47 set_chunk 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 438272 length: 1701888 2024-09-18T11:30:09.159 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 0 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-18T11:30:09.159 INFO:tasks.rados.rados.0.smithi119.stdout:487: done (15 left) 2024-09-18T11:30:09.159 INFO:tasks.rados.rados.0.smithi119.stdout:492: done (14 left) 2024-09-18T11:30:09.159 INFO:tasks.rados.rados.0.smithi119.stdout:495: done (13 left) 2024-09-18T11:30:09.159 INFO:tasks.rados.rados.0.smithi119.stdout:496: done (12 left) 2024-09-18T11:30:09.159 INFO:tasks.rados.rados.0.smithi119.stdout:505: set_chunk oid 6 target oid 6 low_tier 2024-09-18T11:30:09.159 INFO:tasks.rados.rados.0.smithi119.stdout:494: finishing set_chunk to oid 45 2024-09-18T11:30:09.160 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 45 set_chunk 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 544768 length: 1479168 2024-09-18T11:30:09.160 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 0 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-18T11:30:09.160 INFO:tasks.rados.rados.0.smithi119.stdout:505: set_chunk oid 6 offset: 497152 length: 169472 target oid smithi11939506-6 low_tier offset: 497152 snap -1 2024-09-18T11:30:09.160 INFO:tasks.rados.rados.0.smithi119.stdout:494: done (12 left) 2024-09-18T11:30:09.160 INFO:tasks.rados.rados.0.smithi119.stdout:506: set_chunk oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.160 INFO:tasks.rados.rados.0.smithi119.stdout:506: set_chunk oid 7 offset: 346624 length: 793600 target oid smithi11939506-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 346624 snap -1 2024-09-18T11:30:09.160 INFO:tasks.rados.rados.0.smithi119.stdout:507: set_chunk oid 8 target oid 8 low_tier 2024-09-18T11:30:09.160 INFO:tasks.rados.rados.0.smithi119.stdout:507: set_chunk oid 8 offset: 26624 length: 1303040 target oid smithi11939506-8 low_tier offset: 26624 snap -1 2024-09-18T11:30:09.160 INFO:tasks.rados.rados.0.smithi119.stdout:508: set_chunk oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.160 INFO:tasks.rados.rados.0.smithi119.stdout:508: set_chunk oid 9 offset: 854528 length: 676864 target oid smithi11939506-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 854528 snap -1 2024-09-18T11:30:09.160 INFO:tasks.rados.rados.0.smithi119.stdout:509: set_chunk oid 10 target oid 10 low_tier 2024-09-18T11:30:09.160 INFO:tasks.rados.rados.0.smithi119.stdout:509: set_chunk oid 10 offset: 455680 length: 1380864 target oid smithi11939506-10 low_tier offset: 455680 snap -1 2024-09-18T11:30:09.161 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.161 INFO:tasks.rados.rados.0.smithi119.stdout:499: finishing set_chunk to oid 50 2024-09-18T11:30:09.161 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 50 set_chunk 50 low_tier returned error code -95 offset: 7168 length: 588800 2024-09-18T11:30:09.161 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 0 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-18T11:30:09.161 INFO:tasks.rados.rados.0.smithi119.stdout:500: finishing set_chunk to oid 1 2024-09-18T11:30:09.161 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 1 set_chunk 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 109568 length: 1137664 2024-09-18T11:30:09.161 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 0 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-18T11:30:09.161 INFO:tasks.rados.rados.0.smithi119.stdout:499: done (15 left) 2024-09-18T11:30:09.161 INFO:tasks.rados.rados.0.smithi119.stdout:500: done (14 left) 2024-09-18T11:30:09.161 INFO:tasks.rados.rados.0.smithi119.stdout:510: set_chunk oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.161 INFO:tasks.rados.rados.0.smithi119.stdout:510: set_chunk oid 11 offset: 475136 length: 378880 target oid smithi11939506-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 475136 snap -1 2024-09-18T11:30:09.161 INFO:tasks.rados.rados.0.smithi119.stdout:511: set_chunk oid 12 target oid 12 low_tier 2024-09-18T11:30:09.161 INFO:tasks.rados.rados.0.smithi119.stdout:511: set_chunk oid 12 offset: 383488 length: 1016832 target oid smithi11939506-12 low_tier offset: 383488 snap -1 2024-09-18T11:30:09.161 INFO:tasks.rados.rados.0.smithi119.stdout:502: finishing set_chunk to oid 3 2024-09-18T11:30:09.162 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 3 set_chunk 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 696320 length: 905216 2024-09-18T11:30:09.162 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 0 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-18T11:30:09.162 INFO:tasks.rados.rados.0.smithi119.stdout:503: finishing set_chunk to oid 4 2024-09-18T11:30:09.162 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 4 set_chunk 4 low_tier returned error code -95 offset: 24064 length: 1745408 2024-09-18T11:30:09.162 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 4 v 0 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-18T11:30:09.162 INFO:tasks.rados.rados.0.smithi119.stdout:502: done (15 left) 2024-09-18T11:30:09.162 INFO:tasks.rados.rados.0.smithi119.stdout:503: done (14 left) 2024-09-18T11:30:09.162 INFO:tasks.rados.rados.0.smithi119.stdout:512: set_chunk oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.162 INFO:tasks.rados.rados.0.smithi119.stdout:512: set_chunk oid 13 offset: 244224 length: 389632 target oid smithi11939506-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 244224 snap -1 2024-09-18T11:30:09.162 INFO:tasks.rados.rados.0.smithi119.stdout:513: set_chunk oid 14 target oid 14 low_tier 2024-09-18T11:30:09.162 INFO:tasks.rados.rados.0.smithi119.stdout:513: set_chunk oid 14 offset: 14848 length: 1690112 target oid smithi11939506-14 low_tier offset: 14848 snap -1 2024-09-18T11:30:09.162 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.162 INFO:tasks.rados.rados.0.smithi119.stdout:505: finishing set_chunk to oid 6 2024-09-18T11:30:09.163 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 6 set_chunk 6 low_tier returned error code -95 offset: 497152 length: 169472 2024-09-18T11:30:09.163 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 6 v 0 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-18T11:30:09.163 INFO:tasks.rados.rados.0.smithi119.stdout:505: done (15 left) 2024-09-18T11:30:09.163 INFO:tasks.rados.rados.0.smithi119.stdout:514: set_chunk oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.163 INFO:tasks.rados.rados.0.smithi119.stdout:514: set_chunk oid 15 offset: 262656 length: 1455104 target oid smithi11939506-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 262656 snap -1 2024-09-18T11:30:09.163 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.163 INFO:tasks.rados.rados.0.smithi119.stdout:506: finishing set_chunk to oid 7 2024-09-18T11:30:09.163 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 7 set_chunk 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 346624 length: 793600 2024-09-18T11:30:09.163 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 7 v 0 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-18T11:30:09.163 INFO:tasks.rados.rados.0.smithi119.stdout:506: done (15 left) 2024-09-18T11:30:09.163 INFO:tasks.rados.rados.0.smithi119.stdout:515: set_chunk oid 16 target oid 16 low_tier 2024-09-18T11:30:09.163 INFO:tasks.rados.rados.0.smithi119.stdout:515: set_chunk oid 16 offset: 182784 length: 1222656 target oid smithi11939506-16 low_tier offset: 182784 snap -1 2024-09-18T11:30:09.164 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.164 INFO:tasks.rados.rados.0.smithi119.stdout:511: finishing set_chunk to oid 12 2024-09-18T11:30:09.164 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 12 set_chunk 12 low_tier returned error code -95 offset: 383488 length: 1016832 2024-09-18T11:30:09.164 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 12 v 0 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-18T11:30:09.164 INFO:tasks.rados.rados.0.smithi119.stdout:498: finishing set_chunk to oid 49 2024-09-18T11:30:09.164 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 49 set_chunk 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1753088 length: 160768 2024-09-18T11:30:09.164 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 0 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-18T11:30:09.164 INFO:tasks.rados.rados.0.smithi119.stdout:513: finishing set_chunk to oid 14 2024-09-18T11:30:09.164 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 14 set_chunk 14 low_tier returned error code -95 offset: 14848 length: 1690112 2024-09-18T11:30:09.164 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 0 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-18T11:30:09.164 INFO:tasks.rados.rados.0.smithi119.stdout:493: finishing set_chunk to oid 44 2024-09-18T11:30:09.164 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 44 set_chunk 44 low_tier returned error code -95 offset: 1349120 length: 669184 2024-09-18T11:30:09.164 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 44 v 0 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-18T11:30:09.165 INFO:tasks.rados.rados.0.smithi119.stdout:501: finishing set_chunk to oid 2 2024-09-18T11:30:09.165 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 2 set_chunk 2 low_tier returned error code -95 offset: 1197056 length: 666112 2024-09-18T11:30:09.165 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 0 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-18T11:30:09.165 INFO:tasks.rados.rados.0.smithi119.stdout:510: finishing set_chunk to oid 11 2024-09-18T11:30:09.165 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 11 set_chunk 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 475136 length: 378880 2024-09-18T11:30:09.165 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 11 v 0 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-18T11:30:09.165 INFO:tasks.rados.rados.0.smithi119.stdout:493: done (15 left) 2024-09-18T11:30:09.165 INFO:tasks.rados.rados.0.smithi119.stdout:498: done (14 left) 2024-09-18T11:30:09.165 INFO:tasks.rados.rados.0.smithi119.stdout:501: done (13 left) 2024-09-18T11:30:09.165 INFO:tasks.rados.rados.0.smithi119.stdout:510: done (12 left) 2024-09-18T11:30:09.165 INFO:tasks.rados.rados.0.smithi119.stdout:511: done (11 left) 2024-09-18T11:30:09.165 INFO:tasks.rados.rados.0.smithi119.stdout:513: done (10 left) 2024-09-18T11:30:09.166 INFO:tasks.rados.rados.0.smithi119.stdout:516: set_chunk oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.166 INFO:tasks.rados.rados.0.smithi119.stdout:504: finishing set_chunk to oid 5 2024-09-18T11:30:09.166 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 5 set_chunk 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 517120 length: 673792 2024-09-18T11:30:09.166 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 0 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-18T11:30:09.166 INFO:tasks.rados.rados.0.smithi119.stdout:507: finishing set_chunk to oid 8 2024-09-18T11:30:09.166 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 8 set_chunk 8 low_tier returned error code -95 offset: 26624 length: 1303040 2024-09-18T11:30:09.166 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 8 v 0 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-18T11:30:09.166 INFO:tasks.rados.rados.0.smithi119.stdout:509: finishing set_chunk to oid 10 2024-09-18T11:30:09.166 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 10 set_chunk 10 low_tier returned error code -95 offset: 455680 length: 1380864 2024-09-18T11:30:09.166 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 10 v 0 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-18T11:30:09.166 INFO:tasks.rados.rados.0.smithi119.stdout:516: set_chunk oid 17 offset: 434176 length: 455680 target oid smithi11939506-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 434176 snap -1 2024-09-18T11:30:09.166 INFO:tasks.rados.rados.0.smithi119.stdout:504: done (10 left) 2024-09-18T11:30:09.166 INFO:tasks.rados.rados.0.smithi119.stdout:507: done (9 left) 2024-09-18T11:30:09.167 INFO:tasks.rados.rados.0.smithi119.stdout:509: done (8 left) 2024-09-18T11:30:09.167 INFO:tasks.rados.rados.0.smithi119.stdout:517: set_chunk oid 18 target oid 18 low_tier 2024-09-18T11:30:09.167 INFO:tasks.rados.rados.0.smithi119.stdout:517: set_chunk oid 18 offset: 1744896 length: 39936 target oid smithi11939506-18 low_tier offset: 1744896 snap -1 2024-09-18T11:30:09.167 INFO:tasks.rados.rados.0.smithi119.stdout:518: set_chunk oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.167 INFO:tasks.rados.rados.0.smithi119.stdout:518: set_chunk oid 19 offset: 252928 length: 353792 target oid smithi11939506-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 252928 snap -1 2024-09-18T11:30:09.167 INFO:tasks.rados.rados.0.smithi119.stdout:519: set_chunk oid 20 target oid 20 low_tier 2024-09-18T11:30:09.167 INFO:tasks.rados.rados.0.smithi119.stdout:519: set_chunk oid 20 offset: 310272 length: 1134592 target oid smithi11939506-20 low_tier offset: 310272 snap -1 2024-09-18T11:30:09.167 INFO:tasks.rados.rados.0.smithi119.stdout:520: set_chunk oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.167 INFO:tasks.rados.rados.0.smithi119.stdout:520: set_chunk oid 21 offset: 52736 length: 1880576 target oid smithi11939506-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 52736 snap -1 2024-09-18T11:30:09.167 INFO:tasks.rados.rados.0.smithi119.stdout:521: set_chunk oid 22 target oid 22 low_tier 2024-09-18T11:30:09.167 INFO:tasks.rados.rados.0.smithi119.stdout:521: set_chunk oid 22 offset: 16384 length: 318976 target oid smithi11939506-22 low_tier offset: 16384 snap -1 2024-09-18T11:30:09.167 INFO:tasks.rados.rados.0.smithi119.stdout:522: set_chunk oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.168 INFO:tasks.rados.rados.0.smithi119.stdout:522: set_chunk oid 23 offset: 454656 length: 192512 target oid smithi11939506-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 454656 snap -1 2024-09-18T11:30:09.168 INFO:tasks.rados.rados.0.smithi119.stdout:523: set_chunk oid 24 target oid 24 low_tier 2024-09-18T11:30:09.168 INFO:tasks.rados.rados.0.smithi119.stdout:523: set_chunk oid 24 offset: 587264 length: 98816 target oid smithi11939506-24 low_tier offset: 587264 snap -1 2024-09-18T11:30:09.168 INFO:tasks.rados.rados.0.smithi119.stdout:524: set_chunk oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.168 INFO:tasks.rados.rados.0.smithi119.stdout:524: set_chunk oid 25 offset: 554496 length: 855040 target oid smithi11939506-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 554496 snap -1 2024-09-18T11:30:09.168 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.168 INFO:tasks.rados.rados.0.smithi119.stdout:508: finishing set_chunk to oid 9 2024-09-18T11:30:09.168 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 9 set_chunk 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 854528 length: 676864 2024-09-18T11:30:09.168 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 0 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T11:30:09.168 INFO:tasks.rados.rados.0.smithi119.stdout:512: finishing set_chunk to oid 13 2024-09-18T11:30:09.168 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 13 set_chunk 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 244224 length: 389632 2024-09-18T11:30:09.169 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 13 v 0 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-18T11:30:09.169 INFO:tasks.rados.rados.0.smithi119.stdout:514: finishing set_chunk to oid 15 2024-09-18T11:30:09.169 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 15 set_chunk 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 262656 length: 1455104 2024-09-18T11:30:09.169 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 0 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-18T11:30:09.169 INFO:tasks.rados.rados.0.smithi119.stdout:515: finishing set_chunk to oid 16 2024-09-18T11:30:09.169 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 16 set_chunk 16 low_tier returned error code -95 offset: 182784 length: 1222656 2024-09-18T11:30:09.169 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 0 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-18T11:30:09.169 INFO:tasks.rados.rados.0.smithi119.stdout:508: done (15 left) 2024-09-18T11:30:09.169 INFO:tasks.rados.rados.0.smithi119.stdout:512: done (14 left) 2024-09-18T11:30:09.169 INFO:tasks.rados.rados.0.smithi119.stdout:514: done (13 left) 2024-09-18T11:30:09.169 INFO:tasks.rados.rados.0.smithi119.stdout:515: done (12 left) 2024-09-18T11:30:09.169 INFO:tasks.rados.rados.0.smithi119.stdout:525: set_chunk oid 26 target oid 26 low_tier 2024-09-18T11:30:09.170 INFO:tasks.rados.rados.0.smithi119.stdout:525: set_chunk oid 26 offset: 463360 length: 1686016 target oid smithi11939506-26 low_tier offset: 463360 snap -1 2024-09-18T11:30:09.170 INFO:tasks.rados.rados.0.smithi119.stdout:526: set_chunk oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.170 INFO:tasks.rados.rados.0.smithi119.stdout:526: set_chunk oid 27 offset: 531968 length: 190464 target oid smithi11939506-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 531968 snap -1 2024-09-18T11:30:09.170 INFO:tasks.rados.rados.0.smithi119.stdout:527: set_chunk oid 28 target oid 28 low_tier 2024-09-18T11:30:09.170 INFO:tasks.rados.rados.0.smithi119.stdout:527: set_chunk oid 28 offset: 1439744 length: 196608 target oid smithi11939506-28 low_tier offset: 1439744 snap -1 2024-09-18T11:30:09.170 INFO:tasks.rados.rados.0.smithi119.stdout:528: set_chunk oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.170 INFO:tasks.rados.rados.0.smithi119.stdout:528: set_chunk oid 29 offset: 440320 length: 278528 target oid smithi11939506-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 440320 snap -1 2024-09-18T11:30:09.170 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.170 INFO:tasks.rados.rados.0.smithi119.stdout:524: finishing set_chunk to oid 25 2024-09-18T11:30:09.170 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 25 set_chunk 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 554496 length: 855040 2024-09-18T11:30:09.170 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 0 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-18T11:30:09.170 INFO:tasks.rados.rados.0.smithi119.stdout:517: finishing set_chunk to oid 18 2024-09-18T11:30:09.170 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 18 set_chunk 18 low_tier returned error code -95 offset: 1744896 length: 39936 2024-09-18T11:30:09.171 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 0 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-18T11:30:09.171 INFO:tasks.rados.rados.0.smithi119.stdout:517: done (15 left) 2024-09-18T11:30:09.171 INFO:tasks.rados.rados.0.smithi119.stdout:524: done (14 left) 2024-09-18T11:30:09.171 INFO:tasks.rados.rados.0.smithi119.stdout:528 retry set_chunk ! 2024-09-18T11:30:09.171 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 14 2024-09-18T11:30:09.171 INFO:tasks.rados.rados.0.smithi119.stdout:519: finishing set_chunk to oid 20 2024-09-18T11:30:09.171 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 20 set_chunk 20 low_tier returned error code -95 offset: 310272 length: 1134592 2024-09-18T11:30:09.171 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 20 v 0 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-18T11:30:09.171 INFO:tasks.rados.rados.0.smithi119.stdout:522: finishing set_chunk to oid 23 2024-09-18T11:30:09.171 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 23 set_chunk 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 454656 length: 192512 2024-09-18T11:30:09.171 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 0 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-18T11:30:09.171 INFO:tasks.rados.rados.0.smithi119.stdout:520: finishing set_chunk to oid 21 2024-09-18T11:30:09.171 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 21 set_chunk 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 52736 length: 1880576 2024-09-18T11:30:09.171 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 0 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-18T11:30:09.172 INFO:tasks.rados.rados.0.smithi119.stdout:519: done (13 left) 2024-09-18T11:30:09.172 INFO:tasks.rados.rados.0.smithi119.stdout:520: done (12 left) 2024-09-18T11:30:09.172 INFO:tasks.rados.rados.0.smithi119.stdout:522: done (11 left) 2024-09-18T11:30:09.172 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 11 2024-09-18T11:30:09.172 INFO:tasks.rados.rados.0.smithi119.stdout:525: finishing set_chunk to oid 26 2024-09-18T11:30:09.172 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 26 set_chunk 26 low_tier returned error code -95 offset: 463360 length: 1686016 2024-09-18T11:30:09.172 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 0 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-18T11:30:09.172 INFO:tasks.rados.rados.0.smithi119.stdout:525: done (10 left) 2024-09-18T11:30:09.172 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 10 2024-09-18T11:30:09.172 INFO:tasks.rados.rados.0.smithi119.stdout:516: finishing set_chunk to oid 17 2024-09-18T11:30:09.172 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 17 set_chunk 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 434176 length: 455680 2024-09-18T11:30:09.172 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 0 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-18T11:30:09.172 INFO:tasks.rados.rados.0.smithi119.stdout:516: done (9 left) 2024-09-18T11:30:09.172 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 9 2024-09-18T11:30:09.173 INFO:tasks.rados.rados.0.smithi119.stdout:527: finishing set_chunk to oid 28 2024-09-18T11:30:09.173 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 28 set_chunk 28 low_tier returned error code -95 offset: 1439744 length: 196608 2024-09-18T11:30:09.173 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 28 v 0 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-18T11:30:09.173 INFO:tasks.rados.rados.0.smithi119.stdout:528: finishing set_chunk to oid 29 2024-09-18T11:30:09.173 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 29 set_chunk 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 440320 length: 278528 2024-09-18T11:30:09.173 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 0 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-18T11:30:09.173 INFO:tasks.rados.rados.0.smithi119.stdout:527: done (8 left) 2024-09-18T11:30:09.173 INFO:tasks.rados.rados.0.smithi119.stdout:528: done (7 left) 2024-09-18T11:30:09.173 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 7 2024-09-18T11:30:09.173 INFO:tasks.rados.rados.0.smithi119.stdout:521: finishing set_chunk to oid 22 2024-09-18T11:30:09.173 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 22 set_chunk 22 low_tier returned error code -95 offset: 16384 length: 318976 2024-09-18T11:30:09.173 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 0 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-18T11:30:09.173 INFO:tasks.rados.rados.0.smithi119.stdout:521: done (6 left) 2024-09-18T11:30:09.173 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 6 2024-09-18T11:30:09.174 INFO:tasks.rados.rados.0.smithi119.stdout:497: finishing set_chunk to oid 48 2024-09-18T11:30:09.174 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 48 v 12 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-18T11:30:09.174 INFO:tasks.rados.rados.0.smithi119.stdout:497: done (5 left) 2024-09-18T11:30:09.174 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 5 2024-09-18T11:30:09.174 INFO:tasks.rados.rados.0.smithi119.stdout:479: finishing set_chunk to oid 30 2024-09-18T11:30:09.174 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 30 v 6 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-18T11:30:09.174 INFO:tasks.rados.rados.0.smithi119.stdout:479: done (4 left) 2024-09-18T11:30:09.174 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 4 2024-09-18T11:30:09.174 INFO:tasks.rados.rados.0.smithi119.stdout:484: finishing set_chunk to oid 35 2024-09-18T11:30:09.174 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 6 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-18T11:30:09.174 INFO:tasks.rados.rados.0.smithi119.stdout:484: done (3 left) 2024-09-18T11:30:09.174 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 3 2024-09-18T11:30:09.174 INFO:tasks.rados.rados.0.smithi119.stdout:523: finishing set_chunk to oid 24 2024-09-18T11:30:09.174 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 9 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-18T11:30:09.174 INFO:tasks.rados.rados.0.smithi119.stdout:523: done (2 left) 2024-09-18T11:30:09.175 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 2 2024-09-18T11:30:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:526: finishing set_chunk to oid 27 2024-09-18T11:30:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 6 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T11:30:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:526: done (1 left) 2024-09-18T11:30:09.175 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 1 2024-09-18T11:30:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:518: finishing set_chunk to oid 19 2024-09-18T11:30:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 12 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-18T11:30:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:518: done (0 left) 2024-09-18T11:30:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:529: set_chunk oid 30 target oid 30 low_tier 2024-09-18T11:30:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:529: set_chunk oid 30 offset: 1133568 length: 153600 target oid smithi11939506-30 low_tier offset: 1133568 snap -1 2024-09-18T11:30:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:530: set_chunk oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:530: set_chunk oid 31 offset: 416768 length: 491008 target oid smithi11939506-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 416768 snap -1 2024-09-18T11:30:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:531: set_chunk oid 32 target oid 32 low_tier 2024-09-18T11:30:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:531: set_chunk oid 32 offset: 431616 length: 1110016 target oid smithi11939506-32 low_tier offset: 431616 snap -1 2024-09-18T11:30:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:532: set_chunk oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:532: set_chunk oid 33 offset: 87552 length: 2305536 target oid smithi11939506-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 87552 snap -1 2024-09-18T11:30:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:533: set_chunk oid 34 target oid 34 low_tier 2024-09-18T11:30:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:533: set_chunk oid 34 offset: 1789440 length: 17920 target oid smithi11939506-34 low_tier offset: 1789440 snap -1 2024-09-18T11:30:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:534: set_chunk oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:534: set_chunk oid 35 offset: 1181184 length: 659456 target oid smithi11939506-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1181184 snap -1 2024-09-18T11:30:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:535: set_chunk oid 36 target oid 36 low_tier 2024-09-18T11:30:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:535: set_chunk oid 36 offset: 724480 length: 686592 target oid smithi11939506-36 low_tier offset: 724480 snap -1 2024-09-18T11:30:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:536: set_chunk oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:536: set_chunk oid 37 offset: 198144 length: 2333696 target oid smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 198144 snap -1 2024-09-18T11:30:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:537: set_chunk oid 38 target oid 38 low_tier 2024-09-18T11:30:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:537: set_chunk oid 38 offset: 1311744 length: 1121792 target oid smithi11939506-38 low_tier offset: 1311744 snap -1 2024-09-18T11:30:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:538: set_chunk oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:538: set_chunk oid 39 offset: 952320 length: 389632 target oid smithi11939506-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 952320 snap -1 2024-09-18T11:30:09.177 INFO:tasks.rados.rados.0.smithi119.stdout:539: set_chunk oid 40 target oid 40 low_tier 2024-09-18T11:30:09.177 INFO:tasks.rados.rados.0.smithi119.stdout:539: set_chunk oid 40 offset: 2427904 length: 56320 target oid smithi11939506-40 low_tier offset: 2427904 snap -1 2024-09-18T11:30:09.177 INFO:tasks.rados.rados.0.smithi119.stdout:540: set_chunk oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.177 INFO:tasks.rados.rados.0.smithi119.stdout:540: set_chunk oid 41 offset: 597504 length: 352256 target oid smithi11939506-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 597504 snap -1 2024-09-18T11:30:09.177 INFO:tasks.rados.rados.0.smithi119.stdout:541: set_chunk oid 42 target oid 42 low_tier 2024-09-18T11:30:09.177 INFO:tasks.rados.rados.0.smithi119.stdout:541: set_chunk oid 42 offset: 1145344 length: 1184768 target oid smithi11939506-42 low_tier offset: 1145344 snap -1 2024-09-18T11:30:09.177 INFO:tasks.rados.rados.0.smithi119.stdout:542: set_chunk oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.177 INFO:tasks.rados.rados.0.smithi119.stdout:542: set_chunk oid 43 offset: 865280 length: 573952 target oid smithi11939506-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 865280 snap -1 2024-09-18T11:30:09.177 INFO:tasks.rados.rados.0.smithi119.stdout:543: set_chunk oid 44 target oid 44 low_tier 2024-09-18T11:30:09.177 INFO:tasks.rados.rados.0.smithi119.stdout:543: set_chunk oid 44 offset: 569344 length: 1516032 target oid smithi11939506-44 low_tier offset: 569344 snap -1 2024-09-18T11:30:09.177 INFO:tasks.rados.rados.0.smithi119.stdout:544: set_chunk oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.177 INFO:tasks.rados.rados.0.smithi119.stdout:544: set_chunk oid 45 offset: 1245184 length: 136192 target oid smithi11939506-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1245184 snap -1 2024-09-18T11:30:09.178 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.178 INFO:tasks.rados.rados.0.smithi119.stdout:530: finishing set_chunk to oid 31 2024-09-18T11:30:09.178 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 31 set_chunk 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 416768 length: 491008 2024-09-18T11:30:09.178 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 0 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-18T11:30:09.178 INFO:tasks.rados.rados.0.smithi119.stdout:530: done (15 left) 2024-09-18T11:30:09.178 INFO:tasks.rados.rados.0.smithi119.stdout:545: set_chunk oid 46 target oid 46 low_tier 2024-09-18T11:30:09.178 INFO:tasks.rados.rados.0.smithi119.stdout:545: set_chunk oid 46 offset: 258560 length: 1298944 target oid smithi11939506-46 low_tier offset: 258560 snap -1 2024-09-18T11:30:09.178 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.178 INFO:tasks.rados.rados.0.smithi119.stdout:533: finishing set_chunk to oid 34 2024-09-18T11:30:09.178 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 34 set_chunk 34 low_tier returned error code -95 offset: 1789440 length: 17920 2024-09-18T11:30:09.178 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 0 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T11:30:09.178 INFO:tasks.rados.rados.0.smithi119.stdout:533: done (15 left) 2024-09-18T11:30:09.179 INFO:tasks.rados.rados.0.smithi119.stdout:546: set_chunk oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.179 INFO:tasks.rados.rados.0.smithi119.stdout:546: set_chunk oid 47 offset: 1737728 length: 463360 target oid smithi11939506-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1737728 snap -1 2024-09-18T11:30:09.179 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.179 INFO:tasks.rados.rados.0.smithi119.stdout:535: finishing set_chunk to oid 36 2024-09-18T11:30:09.179 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 36 set_chunk 36 low_tier returned error code -95 offset: 724480 length: 686592 2024-09-18T11:30:09.179 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 36 v 0 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-18T11:30:09.179 INFO:tasks.rados.rados.0.smithi119.stdout:535: done (15 left) 2024-09-18T11:30:09.179 INFO:tasks.rados.rados.0.smithi119.stdout:547: set_chunk oid 48 target oid 48 low_tier 2024-09-18T11:30:09.179 INFO:tasks.rados.rados.0.smithi119.stdout:547: set_chunk oid 48 offset: 1718784 length: 371200 target oid smithi11939506-48 low_tier offset: 1718784 snap -1 2024-09-18T11:30:09.179 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.179 INFO:tasks.rados.rados.0.smithi119.stdout:531: finishing set_chunk to oid 32 2024-09-18T11:30:09.179 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 32 set_chunk 32 low_tier returned error code -95 offset: 431616 length: 1110016 2024-09-18T11:30:09.179 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 0 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-18T11:30:09.179 INFO:tasks.rados.rados.0.smithi119.stdout:531: done (15 left) 2024-09-18T11:30:09.179 INFO:tasks.rados.rados.0.smithi119.stdout:548: set_chunk oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.180 INFO:tasks.rados.rados.0.smithi119.stdout:548: set_chunk oid 49 offset: 152064 length: 1104384 target oid smithi11939506-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 152064 snap -1 2024-09-18T11:30:09.180 INFO:tasks.rados.rados.0.smithi119.stdout:539: finishing set_chunk to oid 40 2024-09-18T11:30:09.180 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 40 set_chunk 40 low_tier returned error code -95 offset: 2427904 length: 56320 2024-09-18T11:30:09.180 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 0 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-18T11:30:09.180 INFO:tasks.rados.rados.0.smithi119.stdout:532: finishing set_chunk to oid 33 2024-09-18T11:30:09.180 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 33 set_chunk 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 87552 length: 2305536 2024-09-18T11:30:09.180 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 33 v 0 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-18T11:30:09.180 INFO:tasks.rados.rados.0.smithi119.stdout:536: finishing set_chunk to oid 37 2024-09-18T11:30:09.180 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 37 set_chunk 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 198144 length: 2333696 2024-09-18T11:30:09.180 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 0 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-18T11:30:09.180 INFO:tasks.rados.rados.0.smithi119.stdout:532: done (15 left) 2024-09-18T11:30:09.180 INFO:tasks.rados.rados.0.smithi119.stdout:536: done (14 left) 2024-09-18T11:30:09.180 INFO:tasks.rados.rados.0.smithi119.stdout:539: done (13 left) 2024-09-18T11:30:09.180 INFO:tasks.rados.rados.0.smithi119.stdout:549: set_chunk oid 50 target oid 50 low_tier 2024-09-18T11:30:09.181 INFO:tasks.rados.rados.0.smithi119.stdout:549: set_chunk oid 50 offset: 1172480 length: 1461760 target oid smithi11939506-50 low_tier offset: 1172480 snap -1 2024-09-18T11:30:09.181 INFO:tasks.rados.rados.0.smithi119.stdout:537: finishing set_chunk to oid 38 2024-09-18T11:30:09.181 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 38 set_chunk 38 low_tier returned error code -95 offset: 1311744 length: 1121792 2024-09-18T11:30:09.181 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 0 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-18T11:30:09.181 INFO:tasks.rados.rados.0.smithi119.stdout:537: done (13 left) 2024-09-18T11:30:09.181 INFO:tasks.rados.rados.0.smithi119.stdout:550: set_chunk oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.181 INFO:tasks.rados.rados.0.smithi119.stdout:550: set_chunk oid 1 offset: 444928 length: 948736 target oid smithi11939506-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 444928 snap -1 2024-09-18T11:30:09.181 INFO:tasks.rados.rados.0.smithi119.stdout:551: set_chunk oid 2 target oid 2 low_tier 2024-09-18T11:30:09.181 INFO:tasks.rados.rados.0.smithi119.stdout:551: set_chunk oid 2 offset: 177152 length: 685056 target oid smithi11939506-2 low_tier offset: 177152 snap -1 2024-09-18T11:30:09.181 INFO:tasks.rados.rados.0.smithi119.stdout:552: set_chunk oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.181 INFO:tasks.rados.rados.0.smithi119.stdout:552: set_chunk oid 3 offset: 1190912 length: 310272 target oid smithi11939506-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1190912 snap -1 2024-09-18T11:30:09.181 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.181 INFO:tasks.rados.rados.0.smithi119.stdout:538: finishing set_chunk to oid 39 2024-09-18T11:30:09.181 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 39 set_chunk 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 952320 length: 389632 2024-09-18T11:30:09.181 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 39 v 0 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-18T11:30:09.182 INFO:tasks.rados.rados.0.smithi119.stdout:541: finishing set_chunk to oid 42 2024-09-18T11:30:09.182 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 42 set_chunk 42 low_tier returned error code -95 offset: 1145344 length: 1184768 2024-09-18T11:30:09.182 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 0 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T11:30:09.182 INFO:tasks.rados.rados.0.smithi119.stdout:544: finishing set_chunk to oid 45 2024-09-18T11:30:09.182 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 45 set_chunk 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1245184 length: 136192 2024-09-18T11:30:09.182 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 0 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-18T11:30:09.182 INFO:tasks.rados.rados.0.smithi119.stdout:546: finishing set_chunk to oid 47 2024-09-18T11:30:09.182 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 47 set_chunk 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1737728 length: 463360 2024-09-18T11:30:09.182 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 0 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-18T11:30:09.182 INFO:tasks.rados.rados.0.smithi119.stdout:542: finishing set_chunk to oid 43 2024-09-18T11:30:09.182 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 43 set_chunk 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 865280 length: 573952 2024-09-18T11:30:09.182 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-18T11:30:09.182 INFO:tasks.rados.rados.0.smithi119.stdout:545: finishing set_chunk to oid 46 2024-09-18T11:30:09.182 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 46 set_chunk 46 low_tier returned error code -95 offset: 258560 length: 1298944 2024-09-18T11:30:09.182 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 0 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-18T11:30:09.183 INFO:tasks.rados.rados.0.smithi119.stdout:538: done (15 left) 2024-09-18T11:30:09.183 INFO:tasks.rados.rados.0.smithi119.stdout:541: done (14 left) 2024-09-18T11:30:09.183 INFO:tasks.rados.rados.0.smithi119.stdout:542: done (13 left) 2024-09-18T11:30:09.183 INFO:tasks.rados.rados.0.smithi119.stdout:544: done (12 left) 2024-09-18T11:30:09.183 INFO:tasks.rados.rados.0.smithi119.stdout:545: done (11 left) 2024-09-18T11:30:09.183 INFO:tasks.rados.rados.0.smithi119.stdout:546: done (10 left) 2024-09-18T11:30:09.183 INFO:tasks.rados.rados.0.smithi119.stdout:553: set_chunk oid 4 target oid 4 low_tier 2024-09-18T11:30:09.183 INFO:tasks.rados.rados.0.smithi119.stdout:553: set_chunk oid 4 offset: 835584 length: 20480 target oid smithi11939506-4 low_tier offset: 835584 snap -1 2024-09-18T11:30:09.183 INFO:tasks.rados.rados.0.smithi119.stdout:554: set_chunk oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.183 INFO:tasks.rados.rados.0.smithi119.stdout:547: finishing set_chunk to oid 48 2024-09-18T11:30:09.183 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 48 set_chunk 48 low_tier returned error code -95 offset: 1718784 length: 371200 2024-09-18T11:30:09.183 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 48 v 0 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-18T11:30:09.183 INFO:tasks.rados.rados.0.smithi119.stdout:554: set_chunk oid 5 offset: 162304 length: 1017344 target oid smithi11939506-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 162304 snap -1 2024-09-18T11:30:09.183 INFO:tasks.rados.rados.0.smithi119.stdout:547: done (11 left) 2024-09-18T11:30:09.183 INFO:tasks.rados.rados.0.smithi119.stdout:555: set_chunk oid 6 target oid 6 low_tier 2024-09-18T11:30:09.184 INFO:tasks.rados.rados.0.smithi119.stdout:555: set_chunk oid 6 offset: 591360 length: 942080 target oid smithi11939506-6 low_tier offset: 591360 snap -1 2024-09-18T11:30:09.184 INFO:tasks.rados.rados.0.smithi119.stdout:556: set_chunk oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.184 INFO:tasks.rados.rados.0.smithi119.stdout:556: set_chunk oid 7 offset: 596480 length: 593920 target oid smithi11939506-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 596480 snap -1 2024-09-18T11:30:09.184 INFO:tasks.rados.rados.0.smithi119.stdout:557: set_chunk oid 8 target oid 8 low_tier 2024-09-18T11:30:09.184 INFO:tasks.rados.rados.0.smithi119.stdout:557: set_chunk oid 8 offset: 547328 length: 1000448 target oid smithi11939506-8 low_tier offset: 547328 snap -1 2024-09-18T11:30:09.184 INFO:tasks.rados.rados.0.smithi119.stdout:558: set_chunk oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.184 INFO:tasks.rados.rados.0.smithi119.stdout:558: set_chunk oid 9 offset: 566272 length: 1503232 target oid smithi11939506-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 566272 snap -1 2024-09-18T11:30:09.184 INFO:tasks.rados.rados.0.smithi119.stdout:559: set_chunk oid 10 target oid 10 low_tier 2024-09-18T11:30:09.184 INFO:tasks.rados.rados.0.smithi119.stdout:559: set_chunk oid 10 offset: 81920 length: 688640 target oid smithi11939506-10 low_tier offset: 81920 snap -1 2024-09-18T11:30:09.184 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.184 INFO:tasks.rados.rados.0.smithi119.stdout:549: finishing set_chunk to oid 50 2024-09-18T11:30:09.184 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 50 set_chunk 50 low_tier returned error code -95 offset: 1172480 length: 1461760 2024-09-18T11:30:09.184 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 0 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-18T11:30:09.185 INFO:tasks.rados.rados.0.smithi119.stdout:540: finishing set_chunk to oid 41 2024-09-18T11:30:09.185 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 41 set_chunk 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 597504 length: 352256 2024-09-18T11:30:09.185 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 41 v 0 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-18T11:30:09.185 INFO:tasks.rados.rados.0.smithi119.stdout:550: finishing set_chunk to oid 1 2024-09-18T11:30:09.185 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 1 set_chunk 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 444928 length: 948736 2024-09-18T11:30:09.185 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 0 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-18T11:30:09.185 INFO:tasks.rados.rados.0.smithi119.stdout:534: finishing set_chunk to oid 35 2024-09-18T11:30:09.185 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 35 set_chunk 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1181184 length: 659456 2024-09-18T11:30:09.185 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 0 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-18T11:30:09.185 INFO:tasks.rados.rados.0.smithi119.stdout:548: finishing set_chunk to oid 49 2024-09-18T11:30:09.185 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 49 set_chunk 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 152064 length: 1104384 2024-09-18T11:30:09.185 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 0 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-18T11:30:09.185 INFO:tasks.rados.rados.0.smithi119.stdout:543: finishing set_chunk to oid 44 2024-09-18T11:30:09.186 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 44 set_chunk 44 low_tier returned error code -95 offset: 569344 length: 1516032 2024-09-18T11:30:09.186 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 44 v 0 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-18T11:30:09.186 INFO:tasks.rados.rados.0.smithi119.stdout:552: finishing set_chunk to oid 3 2024-09-18T11:30:09.186 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 3 set_chunk 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1190912 length: 310272 2024-09-18T11:30:09.186 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 0 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-18T11:30:09.186 INFO:tasks.rados.rados.0.smithi119.stdout:534: done (15 left) 2024-09-18T11:30:09.186 INFO:tasks.rados.rados.0.smithi119.stdout:540: done (14 left) 2024-09-18T11:30:09.186 INFO:tasks.rados.rados.0.smithi119.stdout:543: done (13 left) 2024-09-18T11:30:09.186 INFO:tasks.rados.rados.0.smithi119.stdout:548: done (12 left) 2024-09-18T11:30:09.186 INFO:tasks.rados.rados.0.smithi119.stdout:549: done (11 left) 2024-09-18T11:30:09.186 INFO:tasks.rados.rados.0.smithi119.stdout:550: done (10 left) 2024-09-18T11:30:09.186 INFO:tasks.rados.rados.0.smithi119.stdout:552: done (9 left) 2024-09-18T11:30:09.186 INFO:tasks.rados.rados.0.smithi119.stdout:560: set_chunk oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.186 INFO:tasks.rados.rados.0.smithi119.stdout:560: set_chunk oid 11 offset: 1935360 length: 124416 target oid smithi11939506-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1935360 snap -1 2024-09-18T11:30:09.187 INFO:tasks.rados.rados.0.smithi119.stdout:561: set_chunk oid 12 target oid 12 low_tier 2024-09-18T11:30:09.187 INFO:tasks.rados.rados.0.smithi119.stdout:561: set_chunk oid 12 offset: 238080 length: 1519104 target oid smithi11939506-12 low_tier offset: 238080 snap -1 2024-09-18T11:30:09.187 INFO:tasks.rados.rados.0.smithi119.stdout:562: set_chunk oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.187 INFO:tasks.rados.rados.0.smithi119.stdout:562: set_chunk oid 13 offset: 103424 length: 1723904 target oid smithi11939506-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 103424 snap -1 2024-09-18T11:30:09.187 INFO:tasks.rados.rados.0.smithi119.stdout:563: set_chunk oid 14 target oid 14 low_tier 2024-09-18T11:30:09.187 INFO:tasks.rados.rados.0.smithi119.stdout:563: set_chunk oid 14 offset: 91136 length: 216064 target oid smithi11939506-14 low_tier offset: 91136 snap -1 2024-09-18T11:30:09.187 INFO:tasks.rados.rados.0.smithi119.stdout:564: set_chunk oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.187 INFO:tasks.rados.rados.0.smithi119.stdout:564: set_chunk oid 15 offset: 1439232 length: 363008 target oid smithi11939506-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1439232 snap -1 2024-09-18T11:30:09.187 INFO:tasks.rados.rados.0.smithi119.stdout:565: set_chunk oid 16 target oid 16 low_tier 2024-09-18T11:30:09.187 INFO:tasks.rados.rados.0.smithi119.stdout:565: set_chunk oid 16 offset: 1587200 length: 470016 target oid smithi11939506-16 low_tier offset: 1587200 snap -1 2024-09-18T11:30:09.187 INFO:tasks.rados.rados.0.smithi119.stdout:566: set_chunk oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.187 INFO:tasks.rados.rados.0.smithi119.stdout:566: set_chunk oid 17 offset: 415744 length: 1352192 target oid smithi11939506-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 415744 snap -1 2024-09-18T11:30:09.187 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.187 INFO:tasks.rados.rados.0.smithi119.stdout:557: finishing set_chunk to oid 8 2024-09-18T11:30:09.188 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 8 set_chunk 8 low_tier returned error code -95 offset: 547328 length: 1000448 2024-09-18T11:30:09.188 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 8 v 0 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-18T11:30:09.188 INFO:tasks.rados.rados.0.smithi119.stdout:554: finishing set_chunk to oid 5 2024-09-18T11:30:09.188 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 5 set_chunk 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 162304 length: 1017344 2024-09-18T11:30:09.188 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 0 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-18T11:30:09.188 INFO:tasks.rados.rados.0.smithi119.stdout:554: done (15 left) 2024-09-18T11:30:09.188 INFO:tasks.rados.rados.0.smithi119.stdout:557: done (14 left) 2024-09-18T11:30:09.188 INFO:tasks.rados.rados.0.smithi119.stdout:567: set_chunk oid 18 target oid 18 low_tier 2024-09-18T11:30:09.188 INFO:tasks.rados.rados.0.smithi119.stdout:567: set_chunk oid 18 offset: 555520 length: 197120 target oid smithi11939506-18 low_tier offset: 555520 snap -1 2024-09-18T11:30:09.188 INFO:tasks.rados.rados.0.smithi119.stdout:568: set_chunk oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.188 INFO:tasks.rados.rados.0.smithi119.stdout:568: set_chunk oid 19 offset: 1303040 length: 752640 target oid smithi11939506-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1303040 snap -1 2024-09-18T11:30:09.188 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.188 INFO:tasks.rados.rados.0.smithi119.stdout:555: finishing set_chunk to oid 6 2024-09-18T11:30:09.188 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 6 set_chunk 6 low_tier returned error code -95 offset: 591360 length: 942080 2024-09-18T11:30:09.189 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 6 v 0 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-18T11:30:09.189 INFO:tasks.rados.rados.0.smithi119.stdout:556: finishing set_chunk to oid 7 2024-09-18T11:30:09.189 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 7 set_chunk 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 596480 length: 593920 2024-09-18T11:30:09.189 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 7 v 0 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-18T11:30:09.189 INFO:tasks.rados.rados.0.smithi119.stdout:561: finishing set_chunk to oid 12 2024-09-18T11:30:09.189 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 12 set_chunk 12 low_tier returned error code -95 offset: 238080 length: 1519104 2024-09-18T11:30:09.189 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 12 v 0 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-18T11:30:09.189 INFO:tasks.rados.rados.0.smithi119.stdout:555: done (15 left) 2024-09-18T11:30:09.189 INFO:tasks.rados.rados.0.smithi119.stdout:556: done (14 left) 2024-09-18T11:30:09.189 INFO:tasks.rados.rados.0.smithi119.stdout:561: done (13 left) 2024-09-18T11:30:09.189 INFO:tasks.rados.rados.0.smithi119.stdout:569: set_chunk oid 20 target oid 20 low_tier 2024-09-18T11:30:09.189 INFO:tasks.rados.rados.0.smithi119.stdout:569: set_chunk oid 20 offset: 13824 length: 2231296 target oid smithi11939506-20 low_tier offset: 13824 snap -1 2024-09-18T11:30:09.189 INFO:tasks.rados.rados.0.smithi119.stdout:570: set_chunk oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.190 INFO:tasks.rados.rados.0.smithi119.stdout:570: set_chunk oid 21 offset: 1313792 length: 970752 target oid smithi11939506-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1313792 snap -1 2024-09-18T11:30:09.190 INFO:tasks.rados.rados.0.smithi119.stdout:571: set_chunk oid 22 target oid 22 low_tier 2024-09-18T11:30:09.190 INFO:tasks.rados.rados.0.smithi119.stdout:571: set_chunk oid 22 offset: 1230336 length: 248832 target oid smithi11939506-22 low_tier offset: 1230336 snap -1 2024-09-18T11:30:09.190 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.190 INFO:tasks.rados.rados.0.smithi119.stdout:551: finishing set_chunk to oid 2 2024-09-18T11:30:09.190 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 2 set_chunk 2 low_tier returned error code -95 offset: 177152 length: 685056 2024-09-18T11:30:09.190 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 0 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-18T11:30:09.190 INFO:tasks.rados.rados.0.smithi119.stdout:558: finishing set_chunk to oid 9 2024-09-18T11:30:09.190 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 9 set_chunk 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 566272 length: 1503232 2024-09-18T11:30:09.190 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 0 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T11:30:09.190 INFO:tasks.rados.rados.0.smithi119.stdout:564: finishing set_chunk to oid 15 2024-09-18T11:30:09.191 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 15 set_chunk 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1439232 length: 363008 2024-09-18T11:30:09.191 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 0 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-18T11:30:09.191 INFO:tasks.rados.rados.0.smithi119.stdout:562: finishing set_chunk to oid 13 2024-09-18T11:30:09.191 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 13 set_chunk 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 103424 length: 1723904 2024-09-18T11:30:09.191 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 13 v 0 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-18T11:30:09.191 INFO:tasks.rados.rados.0.smithi119.stdout:566: finishing set_chunk to oid 17 2024-09-18T11:30:09.191 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 17 set_chunk 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 415744 length: 1352192 2024-09-18T11:30:09.191 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 0 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-18T11:30:09.191 INFO:tasks.rados.rados.0.smithi119.stdout:559: finishing set_chunk to oid 10 2024-09-18T11:30:09.191 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 10 set_chunk 10 low_tier returned error code -95 offset: 81920 length: 688640 2024-09-18T11:30:09.191 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 10 v 0 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-18T11:30:09.192 INFO:tasks.rados.rados.0.smithi119.stdout:567: finishing set_chunk to oid 18 2024-09-18T11:30:09.192 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 18 set_chunk 18 low_tier returned error code -95 offset: 555520 length: 197120 2024-09-18T11:30:09.192 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 0 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-18T11:30:09.192 INFO:tasks.rados.rados.0.smithi119.stdout:551: done (15 left) 2024-09-18T11:30:09.192 INFO:tasks.rados.rados.0.smithi119.stdout:558: done (14 left) 2024-09-18T11:30:09.192 INFO:tasks.rados.rados.0.smithi119.stdout:559: done (13 left) 2024-09-18T11:30:09.192 INFO:tasks.rados.rados.0.smithi119.stdout:562: done (12 left) 2024-09-18T11:30:09.192 INFO:tasks.rados.rados.0.smithi119.stdout:564: done (11 left) 2024-09-18T11:30:09.192 INFO:tasks.rados.rados.0.smithi119.stdout:566: done (10 left) 2024-09-18T11:30:09.192 INFO:tasks.rados.rados.0.smithi119.stdout:567: done (9 left) 2024-09-18T11:30:09.192 INFO:tasks.rados.rados.0.smithi119.stdout:572: set_chunk oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.192 INFO:tasks.rados.rados.0.smithi119.stdout:572: set_chunk oid 23 offset: 295936 length: 637440 target oid smithi11939506-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 295936 snap -1 2024-09-18T11:30:09.192 INFO:tasks.rados.rados.0.smithi119.stdout:573: set_chunk oid 24 target oid 24 low_tier 2024-09-18T11:30:09.192 INFO:tasks.rados.rados.0.smithi119.stdout:573: set_chunk oid 24 offset: 927744 length: 1366016 target oid smithi11939506-24 low_tier offset: 927744 snap -1 2024-09-18T11:30:09.193 INFO:tasks.rados.rados.0.smithi119.stdout:569: finishing set_chunk to oid 20 2024-09-18T11:30:09.193 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 20 set_chunk 20 low_tier returned error code -95 offset: 13824 length: 2231296 2024-09-18T11:30:09.193 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 20 v 0 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-18T11:30:09.193 INFO:tasks.rados.rados.0.smithi119.stdout:569: done (10 left) 2024-09-18T11:30:09.193 INFO:tasks.rados.rados.0.smithi119.stdout:574: set_chunk oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.193 INFO:tasks.rados.rados.0.smithi119.stdout:574: set_chunk oid 25 offset: 451584 length: 249856 target oid smithi11939506-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 451584 snap -1 2024-09-18T11:30:09.193 INFO:tasks.rados.rados.0.smithi119.stdout:575: set_chunk oid 26 target oid 26 low_tier 2024-09-18T11:30:09.193 INFO:tasks.rados.rados.0.smithi119.stdout:575: set_chunk oid 26 offset: 1584640 length: 88576 target oid smithi11939506-26 low_tier offset: 1584640 snap -1 2024-09-18T11:30:09.193 INFO:tasks.rados.rados.0.smithi119.stdout:576: set_chunk oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.193 INFO:tasks.rados.rados.0.smithi119.stdout:568: finishing set_chunk to oid 19 2024-09-18T11:30:09.193 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 19 set_chunk 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1303040 length: 752640 2024-09-18T11:30:09.193 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 0 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-18T11:30:09.193 INFO:tasks.rados.rados.0.smithi119.stdout:565: finishing set_chunk to oid 16 2024-09-18T11:30:09.193 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 16 set_chunk 16 low_tier returned error code -95 offset: 1587200 length: 470016 2024-09-18T11:30:09.194 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 0 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-18T11:30:09.194 INFO:tasks.rados.rados.0.smithi119.stdout:576: set_chunk oid 27 offset: 86016 length: 268800 target oid smithi11939506-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 86016 snap -1 2024-09-18T11:30:09.194 INFO:tasks.rados.rados.0.smithi119.stdout:565: done (12 left) 2024-09-18T11:30:09.194 INFO:tasks.rados.rados.0.smithi119.stdout:568: done (11 left) 2024-09-18T11:30:09.194 INFO:tasks.rados.rados.0.smithi119.stdout:577: set_chunk oid 28 target oid 28 low_tier 2024-09-18T11:30:09.194 INFO:tasks.rados.rados.0.smithi119.stdout:577: set_chunk oid 28 offset: 518144 length: 1857536 target oid smithi11939506-28 low_tier offset: 518144 snap -1 2024-09-18T11:30:09.194 INFO:tasks.rados.rados.0.smithi119.stdout:578: set_chunk oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.194 INFO:tasks.rados.rados.0.smithi119.stdout:578: set_chunk oid 29 offset: 779776 length: 1400832 target oid smithi11939506-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 779776 snap -1 2024-09-18T11:30:09.194 INFO:tasks.rados.rados.0.smithi119.stdout:578 retry set_chunk ! 2024-09-18T11:30:09.194 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 13 2024-09-18T11:30:09.194 INFO:tasks.rados.rados.0.smithi119.stdout:572: finishing set_chunk to oid 23 2024-09-18T11:30:09.194 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 23 set_chunk 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 295936 length: 637440 2024-09-18T11:30:09.194 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 0 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-18T11:30:09.194 INFO:tasks.rados.rados.0.smithi119.stdout:572: done (12 left) 2024-09-18T11:30:09.195 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 12 2024-09-18T11:30:09.195 INFO:tasks.rados.rados.0.smithi119.stdout:570: finishing set_chunk to oid 21 2024-09-18T11:30:09.195 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 21 set_chunk 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1313792 length: 970752 2024-09-18T11:30:09.195 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 0 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-18T11:30:09.195 INFO:tasks.rados.rados.0.smithi119.stdout:571: finishing set_chunk to oid 22 2024-09-18T11:30:09.195 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 22 set_chunk 22 low_tier returned error code -95 offset: 1230336 length: 248832 2024-09-18T11:30:09.195 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 0 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-18T11:30:09.195 INFO:tasks.rados.rados.0.smithi119.stdout:577: finishing set_chunk to oid 28 2024-09-18T11:30:09.195 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 28 set_chunk 28 low_tier returned error code -95 offset: 518144 length: 1857536 2024-09-18T11:30:09.195 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 28 v 0 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-18T11:30:09.195 INFO:tasks.rados.rados.0.smithi119.stdout:573: finishing set_chunk to oid 24 2024-09-18T11:30:09.195 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 24 set_chunk 24 low_tier returned error code -95 offset: 927744 length: 1366016 2024-09-18T11:30:09.195 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 0 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-18T11:30:09.195 INFO:tasks.rados.rados.0.smithi119.stdout:570: done (11 left) 2024-09-18T11:30:09.196 INFO:tasks.rados.rados.0.smithi119.stdout:571: done (10 left) 2024-09-18T11:30:09.196 INFO:tasks.rados.rados.0.smithi119.stdout:573: done (9 left) 2024-09-18T11:30:09.196 INFO:tasks.rados.rados.0.smithi119.stdout:577: done (8 left) 2024-09-18T11:30:09.196 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 8 2024-09-18T11:30:09.196 INFO:tasks.rados.rados.0.smithi119.stdout:574: finishing set_chunk to oid 25 2024-09-18T11:30:09.196 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 25 set_chunk 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 451584 length: 249856 2024-09-18T11:30:09.196 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 0 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-18T11:30:09.196 INFO:tasks.rados.rados.0.smithi119.stdout:574: done (7 left) 2024-09-18T11:30:09.196 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 7 2024-09-18T11:30:09.196 INFO:tasks.rados.rados.0.smithi119.stdout:578: finishing set_chunk to oid 29 2024-09-18T11:30:09.196 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 29 set_chunk 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 779776 length: 1400832 2024-09-18T11:30:09.196 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 0 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-18T11:30:09.197 INFO:tasks.rados.rados.0.smithi119.stdout:578: done (6 left) 2024-09-18T11:30:09.197 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 6 2024-09-18T11:30:09.197 INFO:tasks.rados.rados.0.smithi119.stdout:529: finishing set_chunk to oid 30 2024-09-18T11:30:09.197 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 30 v 6 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-18T11:30:09.197 INFO:tasks.rados.rados.0.smithi119.stdout:529: done (5 left) 2024-09-18T11:30:09.197 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 5 2024-09-18T11:30:09.197 INFO:tasks.rados.rados.0.smithi119.stdout:553: finishing set_chunk to oid 4 2024-09-18T11:30:09.197 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 4 v 3 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-18T11:30:09.197 INFO:tasks.rados.rados.0.smithi119.stdout:553: done (4 left) 2024-09-18T11:30:09.197 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 4 2024-09-18T11:30:09.197 INFO:tasks.rados.rados.0.smithi119.stdout:563: finishing set_chunk to oid 14 2024-09-18T11:30:09.197 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 3 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-18T11:30:09.197 INFO:tasks.rados.rados.0.smithi119.stdout:575: finishing set_chunk to oid 26 2024-09-18T11:30:09.197 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 9 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-18T11:30:09.198 INFO:tasks.rados.rados.0.smithi119.stdout:560: finishing set_chunk to oid 11 2024-09-18T11:30:09.198 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 11 v 3 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-18T11:30:09.198 INFO:tasks.rados.rados.0.smithi119.stdout:560: done (3 left) 2024-09-18T11:30:09.198 INFO:tasks.rados.rados.0.smithi119.stdout:563: done (2 left) 2024-09-18T11:30:09.198 INFO:tasks.rados.rados.0.smithi119.stdout:575: done (1 left) 2024-09-18T11:30:09.198 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 1 2024-09-18T11:30:09.198 INFO:tasks.rados.rados.0.smithi119.stdout:576: finishing set_chunk to oid 27 2024-09-18T11:30:09.198 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 6 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T11:30:09.198 INFO:tasks.rados.rados.0.smithi119.stdout:576: done (0 left) 2024-09-18T11:30:09.198 INFO:tasks.rados.rados.0.smithi119.stdout:579: set_chunk oid 30 target oid 30 low_tier 2024-09-18T11:30:09.198 INFO:tasks.rados.rados.0.smithi119.stdout:579: set_chunk oid 30 offset: 1288192 length: 228352 target oid smithi11939506-30 low_tier offset: 1288192 snap -1 2024-09-18T11:30:09.198 INFO:tasks.rados.rados.0.smithi119.stdout:580: set_chunk oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.199 INFO:tasks.rados.rados.0.smithi119.stdout:580: set_chunk oid 31 offset: 1613824 length: 333824 target oid smithi11939506-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1613824 snap -1 2024-09-18T11:30:09.199 INFO:tasks.rados.rados.0.smithi119.stdout:581: set_chunk oid 32 target oid 32 low_tier 2024-09-18T11:30:09.199 INFO:tasks.rados.rados.0.smithi119.stdout:581: set_chunk oid 32 offset: 746496 length: 1757184 target oid smithi11939506-32 low_tier offset: 746496 snap -1 2024-09-18T11:30:09.199 INFO:tasks.rados.rados.0.smithi119.stdout:582: set_chunk oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.199 INFO:tasks.rados.rados.0.smithi119.stdout:582: set_chunk oid 33 offset: 717824 length: 798720 target oid smithi11939506-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 717824 snap -1 2024-09-18T11:30:09.199 INFO:tasks.rados.rados.0.smithi119.stdout:583: set_chunk oid 34 target oid 34 low_tier 2024-09-18T11:30:09.199 INFO:tasks.rados.rados.0.smithi119.stdout:583: set_chunk oid 34 offset: 540160 length: 625152 target oid smithi11939506-34 low_tier offset: 540160 snap -1 2024-09-18T11:30:09.199 INFO:tasks.rados.rados.0.smithi119.stdout:584: set_chunk oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.199 INFO:tasks.rados.rados.0.smithi119.stdout:584: set_chunk oid 35 offset: 1019392 length: 486400 target oid smithi11939506-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1019392 snap -1 2024-09-18T11:30:09.199 INFO:tasks.rados.rados.0.smithi119.stdout:585: set_chunk oid 36 target oid 36 low_tier 2024-09-18T11:30:09.199 INFO:tasks.rados.rados.0.smithi119.stdout:585: set_chunk oid 36 offset: 905216 length: 1292288 target oid smithi11939506-36 low_tier offset: 905216 snap -1 2024-09-18T11:30:09.199 INFO:tasks.rados.rados.0.smithi119.stdout:586: set_chunk oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.199 INFO:tasks.rados.rados.0.smithi119.stdout:586: set_chunk oid 37 offset: 1908224 length: 166912 target oid smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1908224 snap -1 2024-09-18T11:30:09.200 INFO:tasks.rados.rados.0.smithi119.stdout:587: set_chunk oid 38 target oid 38 low_tier 2024-09-18T11:30:09.200 INFO:tasks.rados.rados.0.smithi119.stdout:587: set_chunk oid 38 offset: 1780736 length: 133632 target oid smithi11939506-38 low_tier offset: 1780736 snap -1 2024-09-18T11:30:09.200 INFO:tasks.rados.rados.0.smithi119.stdout:588: set_chunk oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.200 INFO:tasks.rados.rados.0.smithi119.stdout:588: set_chunk oid 39 offset: 484864 length: 708608 target oid smithi11939506-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 484864 snap -1 2024-09-18T11:30:09.200 INFO:tasks.rados.rados.0.smithi119.stdout:589: set_chunk oid 40 target oid 40 low_tier 2024-09-18T11:30:09.200 INFO:tasks.rados.rados.0.smithi119.stdout:589: set_chunk oid 40 offset: 990208 length: 1536000 target oid smithi11939506-40 low_tier offset: 990208 snap -1 2024-09-18T11:30:09.200 INFO:tasks.rados.rados.0.smithi119.stdout:590: set_chunk oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.200 INFO:tasks.rados.rados.0.smithi119.stdout:590: set_chunk oid 41 offset: 2103808 length: 187904 target oid smithi11939506-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 2103808 snap -1 2024-09-18T11:30:09.200 INFO:tasks.rados.rados.0.smithi119.stdout:591: set_chunk oid 42 target oid 42 low_tier 2024-09-18T11:30:09.200 INFO:tasks.rados.rados.0.smithi119.stdout:591: set_chunk oid 42 offset: 574976 length: 1043456 target oid smithi11939506-42 low_tier offset: 574976 snap -1 2024-09-18T11:30:09.200 INFO:tasks.rados.rados.0.smithi119.stdout:592: set_chunk oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.200 INFO:tasks.rados.rados.0.smithi119.stdout:592: set_chunk oid 43 offset: 1097728 length: 1299968 target oid smithi11939506-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1097728 snap -1 2024-09-18T11:30:09.201 INFO:tasks.rados.rados.0.smithi119.stdout:593: set_chunk oid 44 target oid 44 low_tier 2024-09-18T11:30:09.201 INFO:tasks.rados.rados.0.smithi119.stdout:593: set_chunk oid 44 offset: 1380864 length: 713728 target oid smithi11939506-44 low_tier offset: 1380864 snap -1 2024-09-18T11:30:09.201 INFO:tasks.rados.rados.0.smithi119.stdout:594: set_chunk oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.201 INFO:tasks.rados.rados.0.smithi119.stdout:594: set_chunk oid 45 offset: 651776 length: 1400832 target oid smithi11939506-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 651776 snap -1 2024-09-18T11:30:09.201 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.201 INFO:tasks.rados.rados.0.smithi119.stdout:583: finishing set_chunk to oid 34 2024-09-18T11:30:09.201 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 34 set_chunk 34 low_tier returned error code -95 offset: 540160 length: 625152 2024-09-18T11:30:09.201 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 0 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-18T11:30:09.201 INFO:tasks.rados.rados.0.smithi119.stdout:583: done (15 left) 2024-09-18T11:30:09.201 INFO:tasks.rados.rados.0.smithi119.stdout:595: set_chunk oid 46 target oid 46 low_tier 2024-09-18T11:30:09.201 INFO:tasks.rados.rados.0.smithi119.stdout:595: set_chunk oid 46 offset: 259072 length: 281088 target oid smithi11939506-46 low_tier offset: 259072 snap -1 2024-09-18T11:30:09.202 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.202 INFO:tasks.rados.rados.0.smithi119.stdout:580: finishing set_chunk to oid 31 2024-09-18T11:30:09.202 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 31 set_chunk 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1613824 length: 333824 2024-09-18T11:30:09.202 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 0 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-18T11:30:09.202 INFO:tasks.rados.rados.0.smithi119.stdout:588: finishing set_chunk to oid 39 2024-09-18T11:30:09.202 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 39 set_chunk 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 484864 length: 708608 2024-09-18T11:30:09.202 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 39 v 0 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-18T11:30:09.202 INFO:tasks.rados.rados.0.smithi119.stdout:579: finishing set_chunk to oid 30 2024-09-18T11:30:09.202 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 30 set_chunk 30 low_tier returned error code -95 offset: 1288192 length: 228352 2024-09-18T11:30:09.202 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 30 v 0 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-18T11:30:09.202 INFO:tasks.rados.rados.0.smithi119.stdout:582: finishing set_chunk to oid 33 2024-09-18T11:30:09.202 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 33 set_chunk 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 717824 length: 798720 2024-09-18T11:30:09.202 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 33 v 0 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-18T11:30:09.202 INFO:tasks.rados.rados.0.smithi119.stdout:585: finishing set_chunk to oid 36 2024-09-18T11:30:09.203 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 36 set_chunk 36 low_tier returned error code -95 offset: 905216 length: 1292288 2024-09-18T11:30:09.203 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 36 v 0 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-18T11:30:09.203 INFO:tasks.rados.rados.0.smithi119.stdout:586: finishing set_chunk to oid 37 2024-09-18T11:30:09.203 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 37 set_chunk 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1908224 length: 166912 2024-09-18T11:30:09.203 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 0 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-18T11:30:09.203 INFO:tasks.rados.rados.0.smithi119.stdout:579: done (15 left) 2024-09-18T11:30:09.203 INFO:tasks.rados.rados.0.smithi119.stdout:580: done (14 left) 2024-09-18T11:30:09.203 INFO:tasks.rados.rados.0.smithi119.stdout:582: done (13 left) 2024-09-18T11:30:09.203 INFO:tasks.rados.rados.0.smithi119.stdout:585: done (12 left) 2024-09-18T11:30:09.203 INFO:tasks.rados.rados.0.smithi119.stdout:586: done (11 left) 2024-09-18T11:30:09.203 INFO:tasks.rados.rados.0.smithi119.stdout:588: done (10 left) 2024-09-18T11:30:09.203 INFO:tasks.rados.rados.0.smithi119.stdout:596: set_chunk oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.203 INFO:tasks.rados.rados.0.smithi119.stdout:581: finishing set_chunk to oid 32 2024-09-18T11:30:09.203 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 32 set_chunk 32 low_tier returned error code -95 offset: 746496 length: 1757184 2024-09-18T11:30:09.204 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 0 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-18T11:30:09.204 INFO:tasks.rados.rados.0.smithi119.stdout:589: finishing set_chunk to oid 40 2024-09-18T11:30:09.204 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 40 set_chunk 40 low_tier returned error code -95 offset: 990208 length: 1536000 2024-09-18T11:30:09.204 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 0 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-18T11:30:09.204 INFO:tasks.rados.rados.0.smithi119.stdout:596: set_chunk oid 47 offset: 1167872 length: 1071616 target oid smithi11939506-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1167872 snap -1 2024-09-18T11:30:09.204 INFO:tasks.rados.rados.0.smithi119.stdout:581: done (10 left) 2024-09-18T11:30:09.204 INFO:tasks.rados.rados.0.smithi119.stdout:589: done (9 left) 2024-09-18T11:30:09.204 INFO:tasks.rados.rados.0.smithi119.stdout:597: set_chunk oid 48 target oid 48 low_tier 2024-09-18T11:30:09.204 INFO:tasks.rados.rados.0.smithi119.stdout:597: set_chunk oid 48 offset: 175616 length: 1518592 target oid smithi11939506-48 low_tier offset: 175616 snap -1 2024-09-18T11:30:09.204 INFO:tasks.rados.rados.0.smithi119.stdout:598: set_chunk oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.204 INFO:tasks.rados.rados.0.smithi119.stdout:598: set_chunk oid 49 offset: 2463744 length: 323584 target oid smithi11939506-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 2463744 snap -1 2024-09-18T11:30:09.204 INFO:tasks.rados.rados.0.smithi119.stdout:599: set_chunk oid 50 target oid 50 low_tier 2024-09-18T11:30:09.204 INFO:tasks.rados.rados.0.smithi119.stdout:599: set_chunk oid 50 offset: 771072 length: 1901056 target oid smithi11939506-50 low_tier offset: 771072 snap -1 2024-09-18T11:30:09.204 INFO:tasks.rados.rados.0.smithi119.stdout:600: set_chunk oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.205 INFO:tasks.rados.rados.0.smithi119.stdout:600: set_chunk oid 1 offset: 447488 length: 1398272 target oid smithi11939506-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 447488 snap -1 2024-09-18T11:30:09.205 INFO:tasks.rados.rados.0.smithi119.stdout:601 oid_not_in_use 37 oid_redirect_not_in_use 0 2024-09-18T11:30:09.205 INFO:tasks.rados.rados.0.smithi119.stdout:600 wait for manifest initialization 2024-09-18T11:30:09.205 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 13 2024-09-18T11:30:09.205 INFO:tasks.rados.rados.0.smithi119.stdout:584: finishing set_chunk to oid 35 2024-09-18T11:30:09.205 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 35 set_chunk 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1019392 length: 486400 2024-09-18T11:30:09.205 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 0 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-18T11:30:09.205 INFO:tasks.rados.rados.0.smithi119.stdout:587: finishing set_chunk to oid 38 2024-09-18T11:30:09.205 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 38 set_chunk 38 low_tier returned error code -95 offset: 1780736 length: 133632 2024-09-18T11:30:09.205 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 0 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-18T11:30:09.205 INFO:tasks.rados.rados.0.smithi119.stdout:594: finishing set_chunk to oid 45 2024-09-18T11:30:09.205 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 45 set_chunk 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 651776 length: 1400832 2024-09-18T11:30:09.205 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 0 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-18T11:30:09.205 INFO:tasks.rados.rados.0.smithi119.stdout:591: finishing set_chunk to oid 42 2024-09-18T11:30:09.206 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 42 set_chunk 42 low_tier returned error code -95 offset: 574976 length: 1043456 2024-09-18T11:30:09.206 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 0 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-18T11:30:09.206 INFO:tasks.rados.rados.0.smithi119.stdout:590: finishing set_chunk to oid 41 2024-09-18T11:30:09.206 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 41 set_chunk 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 2103808 length: 187904 2024-09-18T11:30:09.206 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 41 v 0 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-18T11:30:09.206 INFO:tasks.rados.rados.0.smithi119.stdout:593: finishing set_chunk to oid 44 2024-09-18T11:30:09.206 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 44 set_chunk 44 low_tier returned error code -95 offset: 1380864 length: 713728 2024-09-18T11:30:09.206 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 44 v 0 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-18T11:30:09.206 INFO:tasks.rados.rados.0.smithi119.stdout:595: finishing set_chunk to oid 46 2024-09-18T11:30:09.206 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 46 set_chunk 46 low_tier returned error code -95 offset: 259072 length: 281088 2024-09-18T11:30:09.206 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 0 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-18T11:30:09.206 INFO:tasks.rados.rados.0.smithi119.stdout:592: finishing set_chunk to oid 43 2024-09-18T11:30:09.206 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 43 set_chunk 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1097728 length: 1299968 2024-09-18T11:30:09.207 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-18T11:30:09.207 INFO:tasks.rados.rados.0.smithi119.stdout:584: done (12 left) 2024-09-18T11:30:09.207 INFO:tasks.rados.rados.0.smithi119.stdout:587: done (11 left) 2024-09-18T11:30:09.207 INFO:tasks.rados.rados.0.smithi119.stdout:590: done (10 left) 2024-09-18T11:30:09.207 INFO:tasks.rados.rados.0.smithi119.stdout:591: done (9 left) 2024-09-18T11:30:09.207 INFO:tasks.rados.rados.0.smithi119.stdout:592: done (8 left) 2024-09-18T11:30:09.207 INFO:tasks.rados.rados.0.smithi119.stdout:593: done (7 left) 2024-09-18T11:30:09.207 INFO:tasks.rados.rados.0.smithi119.stdout:594: done (6 left) 2024-09-18T11:30:09.207 INFO:tasks.rados.rados.0.smithi119.stdout:595: done (5 left) 2024-09-18T11:30:09.207 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 5 2024-09-18T11:30:09.207 INFO:tasks.rados.rados.0.smithi119.stdout:596: finishing set_chunk to oid 47 2024-09-18T11:30:09.207 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 47 set_chunk 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1167872 length: 1071616 2024-09-18T11:30:09.207 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 0 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-18T11:30:09.207 INFO:tasks.rados.rados.0.smithi119.stdout:597: finishing set_chunk to oid 48 2024-09-18T11:30:09.208 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 48 set_chunk 48 low_tier returned error code -95 offset: 175616 length: 1518592 2024-09-18T11:30:09.208 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 48 v 0 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-18T11:30:09.208 INFO:tasks.rados.rados.0.smithi119.stdout:596: done (4 left) 2024-09-18T11:30:09.208 INFO:tasks.rados.rados.0.smithi119.stdout:597: done (3 left) 2024-09-18T11:30:09.208 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 3 2024-09-18T11:30:09.208 INFO:tasks.rados.rados.0.smithi119.stdout:599: finishing set_chunk to oid 50 2024-09-18T11:30:09.208 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 50 set_chunk 50 low_tier returned error code -95 offset: 771072 length: 1901056 2024-09-18T11:30:09.208 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 0 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-18T11:30:09.208 INFO:tasks.rados.rados.0.smithi119.stdout:599: done (2 left) 2024-09-18T11:30:09.208 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 2 2024-09-18T11:30:09.208 INFO:tasks.rados.rados.0.smithi119.stdout:600: finishing set_chunk to oid 1 2024-09-18T11:30:09.208 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 1 set_chunk 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 447488 length: 1398272 2024-09-18T11:30:09.209 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 0 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-18T11:30:09.209 INFO:tasks.rados.rados.0.smithi119.stdout:600: done (1 left) 2024-09-18T11:30:09.209 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 1 2024-09-18T11:30:09.209 INFO:tasks.rados.rados.0.smithi119.stdout:598: finishing set_chunk to oid 49 2024-09-18T11:30:09.209 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 18 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-18T11:30:09.209 INFO:tasks.rados.rados.0.smithi119.stdout:598: done (0 left) 2024-09-18T11:30:09.209 INFO:tasks.rados.rados.0.smithi119.stdout:601 oid_not_in_use 50 oid_redirect_not_in_use 0 2024-09-18T11:30:09.209 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 51 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.209 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 52 low_tier 2024-09-18T11:30:09.209 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 53 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.209 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 54 low_tier 2024-09-18T11:30:09.209 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 55 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.209 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 56 low_tier 2024-09-18T11:30:09.209 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 57 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.210 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 58 low_tier 2024-09-18T11:30:09.210 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 59 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.210 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 60 low_tier 2024-09-18T11:30:09.210 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 61 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.210 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 62 low_tier 2024-09-18T11:30:09.210 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 63 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.210 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 64 low_tier 2024-09-18T11:30:09.210 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 65 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.210 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 66 low_tier 2024-09-18T11:30:09.210 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 67 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.210 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 68 low_tier 2024-09-18T11:30:09.211 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 69 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.211 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 70 low_tier 2024-09-18T11:30:09.211 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 71 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.211 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 72 low_tier 2024-09-18T11:30:09.211 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 73 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.211 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 74 low_tier 2024-09-18T11:30:09.211 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 75 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.211 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 76 low_tier 2024-09-18T11:30:09.211 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 77 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.211 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 78 low_tier 2024-09-18T11:30:09.211 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 79 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.211 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 80 low_tier 2024-09-18T11:30:09.211 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 81 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.211 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 82 low_tier 2024-09-18T11:30:09.212 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 83 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.212 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 84 low_tier 2024-09-18T11:30:09.212 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 85 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.212 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 86 low_tier 2024-09-18T11:30:09.212 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 87 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.212 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 88 low_tier 2024-09-18T11:30:09.212 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 89 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.212 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 90 low_tier 2024-09-18T11:30:09.212 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 91 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.212 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 92 low_tier 2024-09-18T11:30:09.212 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 93 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.212 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 94 low_tier 2024-09-18T11:30:09.213 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 95 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.213 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 96 low_tier 2024-09-18T11:30:09.213 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 97 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.213 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 98 low_tier 2024-09-18T11:30:09.213 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 99 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-18T11:30:09.213 INFO:tasks.rados.rados.0.smithi119.stdout: redirect_not_in_use: 100 low_tier 2024-09-18T11:30:09.213 INFO:tasks.rados.rados.0.smithi119.stdout:601: read oid 24 snap -1 2024-09-18T11:30:09.213 INFO:tasks.rados.rados.0.smithi119.stdout:601: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-18T11:30:09.213 INFO:tasks.rados.rados.0.smithi119.stdout:602: read oid 2 snap -1 2024-09-18T11:30:09.213 INFO:tasks.rados.rados.0.smithi119.stdout:602: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-18T11:30:09.213 INFO:tasks.rados.rados.0.smithi119.stdout:603: write (excl) oid 7 current snap is 0 2024-09-18T11:30:09.214 INFO:tasks.rados.rados.0.smithi119.stdout:603: seq_num 50 ranges {440350=460568,1309629=542666,2323973=516377} 2024-09-18T11:30:09.214 INFO:tasks.rados.rados.0.smithi119.stdout:603: writing smithi11939506-7 from 440350 to 900918 tid 1 2024-09-18T11:30:09.214 INFO:tasks.rados.rados.0.smithi119.stdout:603: writing smithi11939506-7 from 1309629 to 1852295 tid 2 2024-09-18T11:30:09.214 INFO:tasks.rados.rados.0.smithi119.stdout:603: writing smithi11939506-7 from 2323973 to 2840350 tid 3 2024-09-18T11:30:09.214 INFO:tasks.rados.rados.0.smithi119.stdout:604: set_chunk oid 31 target oid 2024-09-18T11:30:09.214 INFO:tasks.rados.rados.0.smithi119.stdout:604: set_chunk oid 31 offset: 667648 length: 826880 target oid bf499bb08bbd89b59f6043ee2beb2dab8ffbc5028008dc8bd0ff94d336fb3ea1 offset: 0 snap -1 2024-09-18T11:30:09.214 INFO:tasks.rados.rados.0.smithi119.stdout:605: delete oid 10 current snap is 0 2024-09-18T11:30:09.214 INFO:tasks.rados.rados.0.smithi119.stdout:602: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-18T11:30:09.214 INFO:tasks.rados.rados.0.smithi119.stdout:601: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-18T11:30:09.216 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is unset 2024-09-18T11:30:09.217 INFO:tasks.rados.rados.0.smithi119.stdout:603: finishing write tid 1 to smithi11939506-7 2024-09-18T11:30:09.217 INFO:tasks.rados.rados.0.smithi119.stdout:603: finishing write tid 2 to smithi11939506-7 2024-09-18T11:30:09.217 INFO:tasks.rados.rados.0.smithi119.stdout:603: finishing write tid 3 to smithi11939506-7 2024-09-18T11:30:09.217 INFO:tasks.rados.rados.0.smithi119.stdout:601: done (4 left) 2024-09-18T11:30:09.217 INFO:tasks.rados.rados.0.smithi119.stdout:602: done (3 left) 2024-09-18T11:30:09.217 INFO:tasks.rados.rados.0.smithi119.stdout:605: done (2 left) 2024-09-18T11:30:09.217 INFO:tasks.rados.rados.0.smithi119.stdout:606: set_chunk oid 40 target oid 2024-09-18T11:30:09.218 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:09.147+0000 7f697ceb1640 -1 osd.3 29 handle_osd_map injecting map crc failure 2024-09-18T11:30:09.219 INFO:tasks.rados.rados.0.smithi119.stdout:606: set_chunk oid 40 offset: 1753088 length: 883200 target oid afb6892989df42297567669ec2e3bab02dbeab06bcda445972b98caf007d64b8 offset: 0 snap -1 2024-09-18T11:30:09.220 INFO:tasks.rados.rados.0.smithi119.stdout:607: write (excl) oid 13 current snap is 0 2024-09-18T11:30:09.220 INFO:tasks.rados.rados.0.smithi119.stdout:603: finishing write tid 4 to smithi11939506-7 2024-09-18T11:30:09.220 INFO:tasks.rados.rados.0.smithi119.stdout:603: finishing write tid 5 to smithi11939506-7 2024-09-18T11:30:09.220 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 7 v 21 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-09-18T11:30:09.220 INFO:tasks.rados.rados.0.smithi119.stdout:603: left oid 7 (ObjNum 50 snap 0 seq_num 50) 2024-09-18T11:30:09.220 INFO:tasks.rados.rados.0.smithi119.stdout:607: seq_num 52 ranges {473964=611066,1826593=546335,2778819=95145} 2024-09-18T11:30:09.220 INFO:tasks.rados.rados.0.smithi119.stdout:607: writing smithi11939506-13 from 473964 to 1085030 tid 1 2024-09-18T11:30:09.220 INFO:tasks.rados.rados.0.smithi119.stdout:607: writing smithi11939506-13 from 1826593 to 2372928 tid 2 2024-09-18T11:30:09.220 INFO:tasks.rados.rados.0.smithi119.stdout:607: writing smithi11939506-13 from 2778819 to 2873964 tid 3 2024-09-18T11:30:09.220 INFO:tasks.rados.rados.0.smithi119.stdout:603: done (3 left) 2024-09-18T11:30:09.220 INFO:tasks.rados.rados.0.smithi119.stdout:608: write oid 8 current snap is 0 2024-09-18T11:30:09.220 INFO:tasks.rados.rados.0.smithi119.stdout:604: finishing set_chunk to oid 31 2024-09-18T11:30:09.221 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 31 set_chunk returned error code -95 offset: 667648 length: 826880 2024-09-18T11:30:09.221 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 0 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-18T11:30:09.221 INFO:tasks.rados.rados.0.smithi119.stdout:608: seq_num 53 ranges {490771=686315,1885075=406346} 2024-09-18T11:30:09.221 INFO:tasks.rados.rados.0.smithi119.stdout:608: writing smithi11939506-8 from 490771 to 1177086 tid 1 2024-09-18T11:30:09.221 INFO:tasks.rados.rados.0.smithi119.stdout:608: writing smithi11939506-8 from 1885075 to 2291421 tid 2 2024-09-18T11:30:09.221 INFO:tasks.rados.rados.0.smithi119.stdout:604: done (3 left) 2024-09-18T11:30:09.221 INFO:tasks.rados.rados.0.smithi119.stdout:609: write oid 14 current snap is 0 2024-09-18T11:30:09.221 INFO:tasks.rados.rados.0.smithi119.stdout:606: finishing set_chunk to oid 40 2024-09-18T11:30:09.221 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 40 set_chunk returned error code -95 offset: 1753088 length: 883200 2024-09-18T11:30:09.221 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 0 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-18T11:30:09.221 INFO:tasks.rados.rados.0.smithi119.stdout:607: finishing write tid 1 to smithi11939506-13 2024-09-18T11:30:09.221 INFO:tasks.rados.rados.0.smithi119.stdout:607: finishing write tid 2 to smithi11939506-13 2024-09-18T11:30:09.221 INFO:tasks.rados.rados.0.smithi119.stdout:607: finishing write tid 3 to smithi11939506-13 2024-09-18T11:30:09.221 INFO:tasks.rados.rados.0.smithi119.stdout:607: finishing write tid 4 to smithi11939506-13 2024-09-18T11:30:09.222 INFO:tasks.rados.rados.0.smithi119.stdout:607: finishing write tid 5 to smithi11939506-13 2024-09-18T11:30:09.222 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 13 v 37 (ObjNum 52 snap 0 seq_num 52) dirty exists 2024-09-18T11:30:09.222 INFO:tasks.rados.rados.0.smithi119.stdout:607: left oid 13 (ObjNum 52 snap 0 seq_num 52) 2024-09-18T11:30:09.222 INFO:tasks.rados.rados.0.smithi119.stdout:608: finishing write tid 1 to smithi11939506-8 2024-09-18T11:30:09.222 INFO:tasks.rados.rados.0.smithi119.stdout:609: seq_num 54 ranges {507578=477917,1543557=550004,2833665=73913} 2024-09-18T11:30:09.222 INFO:tasks.rados.rados.0.smithi119.stdout:609: writing smithi11939506-14 from 507578 to 985495 tid 1 2024-09-18T11:30:09.222 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:09.222 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:09.222 INFO:tasks.rados.rados.0.smithi119.stdout:609: writing smithi11939506-14 from 1543557 to 2093561 tid 2 2024-09-18T11:30:09.223 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:30:09.225 INFO:tasks.rados.rados.0.smithi119.stdout:609: writing smithi11939506-14 from 2833665 to 2907578 tid 3 2024-09-18T11:30:09.225 INFO:tasks.rados.rados.0.smithi119.stdout:606: done (3 left) 2024-09-18T11:30:09.225 INFO:tasks.rados.rados.0.smithi119.stdout:607: done (2 left) 2024-09-18T11:30:09.225 INFO:tasks.rados.rados.0.smithi119.stdout:610: delete oid 34 current snap is 0 2024-09-18T11:30:09.225 INFO:tasks.rados.rados.0.smithi119.stdout:608: finishing write tid 2 to smithi11939506-8 2024-09-18T11:30:09.225 INFO:tasks.rados.rados.0.smithi119.stdout:608: finishing write tid 3 to smithi11939506-8 2024-09-18T11:30:09.225 INFO:tasks.rados.rados.0.smithi119.stdout:608: finishing write tid 4 to smithi11939506-8 2024-09-18T11:30:09.225 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 8 v 30 (ObjNum 53 snap 0 seq_num 53) dirty exists 2024-09-18T11:30:09.225 INFO:tasks.rados.rados.0.smithi119.stdout:608: left oid 8 (ObjNum 53 snap 0 seq_num 53) 2024-09-18T11:30:09.225 INFO:tasks.rados.rados.0.smithi119.stdout:609: finishing write tid 1 to smithi11939506-14 2024-09-18T11:30:09.226 INFO:tasks.rados.rados.0.smithi119.stdout:609: finishing write tid 2 to smithi11939506-14 2024-09-18T11:30:09.226 INFO:tasks.rados.rados.0.smithi119.stdout:609: finishing write tid 3 to smithi11939506-14 2024-09-18T11:30:09.226 INFO:tasks.rados.rados.0.smithi119.stdout:608: done (2 left) 2024-09-18T11:30:09.226 INFO:tasks.rados.rados.0.smithi119.stdout:610: done (1 left) 2024-09-18T11:30:09.226 INFO:tasks.rados.rados.0.smithi119.stdout:611: write (excl) oid 11 current snap is 0 2024-09-18T11:30:09.226 INFO:tasks.rados.rados.0.smithi119.stdout:609: finishing write tid 4 to smithi11939506-14 2024-09-18T11:30:09.226 INFO:tasks.rados.rados.0.smithi119.stdout:611: seq_num 56 ranges {541192=628415,1660521=553673,2888511=52681} 2024-09-18T11:30:09.226 INFO:tasks.rados.rados.0.smithi119.stdout:611: writing smithi11939506-11 from 541192 to 1169607 tid 1 2024-09-18T11:30:09.226 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:09.213+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:09.227 INFO:tasks.rados.rados.0.smithi119.stdout:611: writing smithi11939506-11 from 1660521 to 2214194 tid 2 2024-09-18T11:30:09.227 INFO:tasks.rados.rados.0.smithi119.stdout:611: writing smithi11939506-11 from 2888511 to 2941192 tid 3 2024-09-18T11:30:09.227 INFO:tasks.rados.rados.0.smithi119.stdout:612: write (excl) oid 33 current snap is 0 2024-09-18T11:30:09.228 INFO:tasks.rados.rados.0.smithi119.stdout:609: finishing write tid 5 to smithi11939506-14 2024-09-18T11:30:09.228 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 27 (ObjNum 54 snap 0 seq_num 54) dirty exists 2024-09-18T11:30:09.228 INFO:tasks.rados.rados.0.smithi119.stdout:609: left oid 14 (ObjNum 54 snap 0 seq_num 54) 2024-09-18T11:30:09.228 INFO:tasks.rados.rados.0.smithi119.stdout:611: finishing write tid 1 to smithi11939506-11 2024-09-18T11:30:09.228 INFO:tasks.rados.rados.0.smithi119.stdout:612: seq_num 57 ranges {557999=703664,1719003=413684,2632287=325712} 2024-09-18T11:30:09.228 INFO:tasks.rados.rados.0.smithi119.stdout:612: writing smithi11939506-33 from 557999 to 1261663 tid 1 2024-09-18T11:30:09.232 INFO:tasks.rados.rados.0.smithi119.stdout:612: writing smithi11939506-33 from 1719003 to 2132687 tid 2 2024-09-18T11:30:09.237 INFO:tasks.rados.rados.0.smithi119.stdout:612: writing smithi11939506-33 from 2632287 to 2957999 tid 3 2024-09-18T11:30:09.237 INFO:tasks.rados.rados.0.smithi119.stdout:609: done (2 left) 2024-09-18T11:30:09.237 INFO:tasks.rados.rados.0.smithi119.stdout:611: finishing write tid 2 to smithi11939506-11 2024-09-18T11:30:09.237 INFO:tasks.rados.rados.0.smithi119.stdout:613: read oid 9 snap -1 2024-09-18T11:30:09.237 INFO:tasks.rados.rados.0.smithi119.stdout:613: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T11:30:09.237 INFO:tasks.rados.rados.0.smithi119.stdout:614: write oid 50 current snap is 0 2024-09-18T11:30:09.238 INFO:tasks.rados.rados.0.smithi119.stdout:611: finishing write tid 3 to smithi11939506-11 2024-09-18T11:30:09.238 INFO:tasks.rados.rados.0.smithi119.stdout:611: finishing write tid 4 to smithi11939506-11 2024-09-18T11:30:09.238 INFO:tasks.rados.rados.0.smithi119.stdout:614: seq_num 58 ranges {574806=778913,2061132=557342} 2024-09-18T11:30:09.244 INFO:tasks.rados.rados.0.smithi119.stdout:614: writing smithi11939506-50 from 574806 to 1353719 tid 1 2024-09-18T11:30:09.252 INFO:tasks.rados.rados.0.smithi119.stdout:614: writing smithi11939506-50 from 2061132 to 2618474 tid 2 2024-09-18T11:30:09.252 INFO:tasks.rados.rados.0.smithi119.stdout:615: tier_evict oid 25 2024-09-18T11:30:09.252 INFO:tasks.rados.rados.0.smithi119.stdout:615: tier_evict oid 25 snap -1 2024-09-18T11:30:09.252 INFO:tasks.rados.rados.0.smithi119.stdout:611: finishing write tid 5 to smithi11939506-11 2024-09-18T11:30:09.252 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 11 v 26 (ObjNum 56 snap 0 seq_num 56) dirty exists 2024-09-18T11:30:09.252 INFO:tasks.rados.rados.0.smithi119.stdout:611: left oid 11 (ObjNum 56 snap 0 seq_num 56) 2024-09-18T11:30:09.252 INFO:tasks.rados.rados.0.smithi119.stdout:612: finishing write tid 1 to smithi11939506-33 2024-09-18T11:30:09.252 INFO:tasks.rados.rados.0.smithi119.stdout:612: finishing write tid 2 to smithi11939506-33 2024-09-18T11:30:09.253 INFO:tasks.rados.rados.0.smithi119.stdout:611: done (4 left) 2024-09-18T11:30:09.253 INFO:tasks.rados.rados.0.smithi119.stdout:616: write oid 3 current snap is 0 2024-09-18T11:30:09.253 INFO:tasks.rados.rados.0.smithi119.stdout:616: seq_num 59 ranges {591613=454162,1719614=417353,2570780=420833} 2024-09-18T11:30:09.257 INFO:tasks.rados.rados.0.smithi119.stdout:616: writing smithi11939506-3 from 591613 to 1045775 tid 1 2024-09-18T11:30:09.263 INFO:tasks.rados.rados.0.smithi119.stdout:616: writing smithi11939506-3 from 1719614 to 2136967 tid 2 2024-09-18T11:30:09.268 INFO:tasks.rados.rados.0.smithi119.stdout:616: writing smithi11939506-3 from 2570780 to 2991613 tid 3 2024-09-18T11:30:09.268 INFO:tasks.rados.rados.0.smithi119.stdout:617: tier_promote oid 21 2024-09-18T11:30:09.268 INFO:tasks.rados.rados.0.smithi119.stdout:618: set_chunk oid 14 target oid 2024-09-18T11:30:09.273 INFO:tasks.rados.rados.0.smithi119.stdout:618: set_chunk oid 14 offset: 858624 length: 3072 target oid 210530627915c952a07cf0e3f8de51c8b2ff6dd70e8af2a9590a297feefa623d offset: 0 snap -1 2024-09-18T11:30:09.273 INFO:tasks.rados.rados.0.smithi119.stdout:619: read oid 38 snap -1 2024-09-18T11:30:09.273 INFO:tasks.rados.rados.0.smithi119.stdout:619: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-18T11:30:09.273 INFO:tasks.rados.rados.0.smithi119.stdout:620: write oid 18 current snap is 0 2024-09-18T11:30:09.273 INFO:tasks.rados.rados.0.smithi119.stdout:620: seq_num 60 ranges {608420=529411,1778096=561011,2881850=126570} 2024-09-18T11:30:09.279 INFO:tasks.rados.rados.0.smithi119.stdout:620: writing smithi11939506-18 from 608420 to 1137831 tid 1 2024-09-18T11:30:09.286 INFO:tasks.rados.rados.0.smithi119.stdout:620: writing smithi11939506-18 from 1778096 to 2339107 tid 2 2024-09-18T11:30:09.290 INFO:tasks.rados.rados.0.smithi119.stdout:620: writing smithi11939506-18 from 2881850 to 3008420 tid 3 2024-09-18T11:30:09.290 INFO:tasks.rados.rados.0.smithi119.stdout:621: read oid 42 snap -1 2024-09-18T11:30:09.290 INFO:tasks.rados.rados.0.smithi119.stdout:621: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-18T11:30:09.290 INFO:tasks.rados.rados.0.smithi119.stdout:622: write oid 16 current snap is 0 2024-09-18T11:30:09.290 INFO:tasks.rados.rados.0.smithi119.stdout:622: seq_num 61 ranges {625227=721013,1952931=704669} 2024-09-18T11:30:09.297 INFO:tasks.rados.rados.0.smithi119.stdout:622: writing smithi11939506-16 from 625227 to 1346240 tid 1 2024-09-18T11:30:09.304 INFO:tasks.rados.rados.0.smithi119.stdout:622: writing smithi11939506-16 from 1952931 to 2657600 tid 2 2024-09-18T11:30:09.304 INFO:tasks.rados.rados.0.smithi119.stdout:623: read oid 4 snap -1 2024-09-18T11:30:09.305 INFO:tasks.rados.rados.0.smithi119.stdout:623: expect (ObjNum 3 snap 0 seq_num 3) 2024-09-18T11:30:09.305 INFO:tasks.rados.rados.0.smithi119.stdout:624: write (excl) oid 29 current snap is 0 2024-09-18T11:30:09.305 INFO:tasks.rados.rados.0.smithi119.stdout:613: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T11:30:09.316 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:09.315+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:09.340 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:09.340 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:09.349 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:30:09.377 INFO:tasks.rados.rados.0.smithi119.stdout:612: finishing write tid 3 to smithi11939506-33 2024-09-18T11:30:09.377 INFO:tasks.rados.rados.0.smithi119.stdout:624: seq_num 62 ranges {642034=796262,1895060=564680} 2024-09-18T11:30:09.384 INFO:tasks.rados.rados.0.smithi119.stdout:624: writing smithi11939506-29 from 642034 to 1438296 tid 1 2024-09-18T11:30:09.391 INFO:tasks.rados.rados.0.smithi119.stdout:624: writing smithi11939506-29 from 1895060 to 2459740 tid 2 2024-09-18T11:30:09.391 INFO:tasks.rados.rados.0.smithi119.stdout:613: done (12 left) 2024-09-18T11:30:09.391 INFO:tasks.rados.rados.0.smithi119.stdout:625: write (excl) oid 31 current snap is 0 2024-09-18T11:30:09.391 INFO:tasks.rados.rados.0.smithi119.stdout:612: finishing write tid 4 to smithi11939506-33 2024-09-18T11:30:09.391 INFO:tasks.rados.rados.0.smithi119.stdout:612: finishing write tid 5 to smithi11939506-33 2024-09-18T11:30:09.391 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 33 v 20 (ObjNum 57 snap 0 seq_num 57) dirty exists 2024-09-18T11:30:09.391 INFO:tasks.rados.rados.0.smithi119.stdout:612: left oid 33 (ObjNum 57 snap 0 seq_num 57) 2024-09-18T11:30:09.391 INFO:tasks.rados.rados.0.smithi119.stdout:614: finishing write tid 1 to smithi11939506-50 2024-09-18T11:30:09.391 INFO:tasks.rados.rados.0.smithi119.stdout:625: seq_num 63 ranges {658841=471511,1553542=708338,2964119=94722} 2024-09-18T11:30:09.397 INFO:tasks.rados.rados.0.smithi119.stdout:625: writing smithi11939506-31 from 658841 to 1130352 tid 1 2024-09-18T11:30:09.403 INFO:tasks.rados.rados.0.smithi119.stdout:625: writing smithi11939506-31 from 1553542 to 2261880 tid 2 2024-09-18T11:30:09.408 INFO:tasks.rados.rados.0.smithi119.stdout:625: writing smithi11939506-31 from 2964119 to 3058841 tid 3 2024-09-18T11:30:09.408 INFO:tasks.rados.rados.0.smithi119.stdout:612: done (12 left) 2024-09-18T11:30:09.408 INFO:tasks.rados.rados.0.smithi119.stdout:626: read oid 40 snap -1 2024-09-18T11:30:09.408 INFO:tasks.rados.rados.0.smithi119.stdout:626: expect (ObjNum 39 snap 0 seq_num 39) 2024-09-18T11:30:09.408 INFO:tasks.rados.rados.0.smithi119.stdout:627: tier_flush oid 35 2024-09-18T11:30:09.408 INFO:tasks.rados.rados.0.smithi119.stdout:627: tier_flush oid 35 snap -1 2024-09-18T11:30:09.408 INFO:tasks.rados.rados.0.smithi119.stdout:615: got (22) Invalid argument 2024-09-18T11:30:09.408 INFO:tasks.rados.rados.0.smithi119.stdout:614: finishing write tid 2 to smithi11939506-50 2024-09-18T11:30:09.408 INFO:tasks.rados.rados.0.smithi119.stdout:614: finishing write tid 3 to smithi11939506-50 2024-09-18T11:30:09.409 INFO:tasks.rados.rados.0.smithi119.stdout:615: done (13 left) 2024-09-18T11:30:09.409 INFO:tasks.rados.rados.0.smithi119.stdout:628: set_chunk oid 20 target oid 2024-09-18T11:30:09.417 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:09.415+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:09.419 INFO:tasks.rados.rados.0.smithi119.stdout:628: set_chunk oid 20 offset: 477696 length: 940032 target oid 688c88ad6e0d5f2f59920d9ca419b7405e788e1845d59ddfe083dcf76259039d offset: 0 snap -1 2024-09-18T11:30:09.419 INFO:tasks.rados.rados.0.smithi119.stdout:629: read oid 22 snap -1 2024-09-18T11:30:09.419 INFO:tasks.rados.rados.0.smithi119.stdout:629: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-18T11:30:09.419 INFO:tasks.rados.rados.0.smithi119.stdout:630: rollback oid 30 current snap is 0 2024-09-18T11:30:09.419 INFO:tasks.rados.rados.0.smithi119.stdout:630: done (15 left) 2024-09-18T11:30:09.419 INFO:tasks.rados.rados.0.smithi119.stdout:631: set_chunk oid 11 target oid 2024-09-18T11:30:09.433 INFO:tasks.rados.rados.0.smithi119.stdout:631: set_chunk oid 11 offset: 1222144 length: 1692160 target oid 9c42d0d158eef3c3bcb28f13d112900d64b713ec154963fc43105b41c02c72c2 offset: 0 snap -1 2024-09-18T11:30:09.433 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:09.433 INFO:tasks.rados.rados.0.smithi119.stdout:614: finishing write tid 4 to smithi11939506-50 2024-09-18T11:30:09.434 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 21 (ObjNum 58 snap 0 seq_num 58) dirty exists 2024-09-18T11:30:09.434 INFO:tasks.rados.rados.0.smithi119.stdout:614: left oid 50 (ObjNum 58 snap 0 seq_num 58) 2024-09-18T11:30:09.434 INFO:tasks.rados.rados.0.smithi119.stdout:616: finishing write tid 1 to smithi11939506-3 2024-09-18T11:30:09.434 INFO:tasks.rados.rados.0.smithi119.stdout:616: finishing write tid 2 to smithi11939506-3 2024-09-18T11:30:09.434 INFO:tasks.rados.rados.0.smithi119.stdout:616: finishing write tid 3 to smithi11939506-3 2024-09-18T11:30:09.434 INFO:tasks.rados.rados.0.smithi119.stdout:616: finishing write tid 4 to smithi11939506-3 2024-09-18T11:30:09.434 INFO:tasks.rados.rados.0.smithi119.stdout:616: finishing write tid 5 to smithi11939506-3 2024-09-18T11:30:09.434 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 21 (ObjNum 59 snap 0 seq_num 59) dirty exists 2024-09-18T11:30:09.434 INFO:tasks.rados.rados.0.smithi119.stdout:616: left oid 3 (ObjNum 59 snap 0 seq_num 59) 2024-09-18T11:30:09.434 INFO:tasks.rados.rados.0.smithi119.stdout:618: finishing set_chunk to oid 14 2024-09-18T11:30:09.434 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 27 (ObjNum 54 snap 0 seq_num 54) dirty exists 2024-09-18T11:30:09.434 INFO:tasks.rados.rados.0.smithi119.stdout:619: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-18T11:30:09.486 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:09.486 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:09.495 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:30:09.519 INFO:tasks.rados.rados.0.smithi119.stdout:617: got (0) Success 2024-09-18T11:30:09.519 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 6 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-18T11:30:09.519 INFO:tasks.rados.rados.0.smithi119.stdout:620: finishing write tid 1 to smithi11939506-18 2024-09-18T11:30:09.520 INFO:tasks.rados.rados.0.smithi119.stdout:620: finishing write tid 2 to smithi11939506-18 2024-09-18T11:30:09.520 INFO:tasks.rados.rados.0.smithi119.stdout:620: finishing write tid 3 to smithi11939506-18 2024-09-18T11:30:09.520 INFO:tasks.rados.rados.0.smithi119.stdout:614: done (15 left) 2024-09-18T11:30:09.520 INFO:tasks.rados.rados.0.smithi119.stdout:616: done (14 left) 2024-09-18T11:30:09.520 INFO:tasks.rados.rados.0.smithi119.stdout:617: done (13 left) 2024-09-18T11:30:09.520 INFO:tasks.rados.rados.0.smithi119.stdout:618: done (12 left) 2024-09-18T11:30:09.520 INFO:tasks.rados.rados.0.smithi119.stdout:619: done (11 left) 2024-09-18T11:30:09.520 INFO:tasks.rados.rados.0.smithi119.stdout:632: write (excl) oid 32 current snap is 0 2024-09-18T11:30:09.520 INFO:tasks.rados.rados.0.smithi119.stdout:632: seq_num 64 ranges {675648=546760,2012024=568349} 2024-09-18T11:30:09.521 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:09.517+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:09.525 INFO:tasks.rados.rados.0.smithi119.stdout:632: writing smithi11939506-32 from 675648 to 1222408 tid 1 2024-09-18T11:30:09.533 INFO:tasks.rados.rados.0.smithi119.stdout:632: writing smithi11939506-32 from 2012024 to 2580373 tid 2 2024-09-18T11:30:09.533 INFO:tasks.rados.rados.0.smithi119.stdout:633: tier_flush oid 41 2024-09-18T11:30:09.533 INFO:tasks.rados.rados.0.smithi119.stdout:633: tier_flush oid 41 snap -1 2024-09-18T11:30:09.533 INFO:tasks.rados.rados.0.smithi119.stdout:620: finishing write tid 4 to smithi11939506-18 2024-09-18T11:30:09.534 INFO:tasks.rados.rados.0.smithi119.stdout:620: finishing write tid 5 to smithi11939506-18 2024-09-18T11:30:09.534 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 19 (ObjNum 60 snap 0 seq_num 60) dirty exists 2024-09-18T11:30:09.534 INFO:tasks.rados.rados.0.smithi119.stdout:620: left oid 18 (ObjNum 60 snap 0 seq_num 60) 2024-09-18T11:30:09.534 INFO:tasks.rados.rados.0.smithi119.stdout:621: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-18T11:30:09.618 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:09.616+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:09.623 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:09.623 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:09.625 INFO:tasks.rados.rados.0.smithi119.stdout:622: finishing write tid 1 to smithi11939506-16 2024-09-18T11:30:09.625 INFO:tasks.rados.rados.0.smithi119.stdout:620: done (12 left) 2024-09-18T11:30:09.625 INFO:tasks.rados.rados.0.smithi119.stdout:621: done (11 left) 2024-09-18T11:30:09.625 INFO:tasks.rados.rados.0.smithi119.stdout:634: tier_evict oid 48 2024-09-18T11:30:09.625 INFO:tasks.rados.rados.0.smithi119.stdout:634: tier_evict oid 48 snap -1 2024-09-18T11:30:09.625 INFO:tasks.rados.rados.0.smithi119.stdout:622: finishing write tid 2 to smithi11939506-16 2024-09-18T11:30:09.625 INFO:tasks.rados.rados.0.smithi119.stdout:622: finishing write tid 3 to smithi11939506-16 2024-09-18T11:30:09.625 INFO:tasks.rados.rados.0.smithi119.stdout:622: finishing write tid 4 to smithi11939506-16 2024-09-18T11:30:09.625 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 28 (ObjNum 61 snap 0 seq_num 61) dirty exists 2024-09-18T11:30:09.626 INFO:tasks.rados.rados.0.smithi119.stdout:622: left oid 16 (ObjNum 61 snap 0 seq_num 61) 2024-09-18T11:30:09.626 INFO:tasks.rados.rados.0.smithi119.stdout:623: expect (ObjNum 3 snap 0 seq_num 3) 2024-09-18T11:30:09.632 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:30:09.692 INFO:tasks.rados.rados.0.smithi119.stdout:624: finishing write tid 1 to smithi11939506-29 2024-09-18T11:30:09.692 INFO:tasks.rados.rados.0.smithi119.stdout:622: done (11 left) 2024-09-18T11:30:09.692 INFO:tasks.rados.rados.0.smithi119.stdout:623: done (10 left) 2024-09-18T11:30:09.692 INFO:tasks.rados.rados.0.smithi119.stdout:635: rollback oid 9 current snap is 0 2024-09-18T11:30:09.692 INFO:tasks.rados.rados.0.smithi119.stdout:635: done (10 left) 2024-09-18T11:30:09.692 INFO:tasks.rados.rados.0.smithi119.stdout:636: write oid 45 current snap is 0 2024-09-18T11:30:09.692 INFO:tasks.rados.rados.0.smithi119.stdout:624: finishing write tid 2 to smithi11939506-29 2024-09-18T11:30:09.692 INFO:tasks.rados.rados.0.smithi119.stdout:624: finishing write tid 3 to smithi11939506-29 2024-09-18T11:30:09.692 INFO:tasks.rados.rados.0.smithi119.stdout:624: finishing write tid 4 to smithi11939506-29 2024-09-18T11:30:09.693 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 9 (ObjNum 62 snap 0 seq_num 62) dirty exists 2024-09-18T11:30:09.693 INFO:tasks.rados.rados.0.smithi119.stdout:624: left oid 29 (ObjNum 62 snap 0 seq_num 62) 2024-09-18T11:30:09.693 INFO:tasks.rados.rados.0.smithi119.stdout:625: finishing write tid 1 to smithi11939506-31 2024-09-18T11:30:09.693 INFO:tasks.rados.rados.0.smithi119.stdout:636: seq_num 65 ranges {692455=622009,2070506=712007} 2024-09-18T11:30:09.698 INFO:tasks.rados.rados.0.smithi119.stdout:636: writing smithi11939506-45 from 692455 to 1314464 tid 1 2024-09-18T11:30:09.707 INFO:tasks.rados.rados.0.smithi119.stdout:636: writing smithi11939506-45 from 2070506 to 2782513 tid 2 2024-09-18T11:30:09.708 INFO:tasks.rados.rados.0.smithi119.stdout:624: done (10 left) 2024-09-18T11:30:09.708 INFO:tasks.rados.rados.0.smithi119.stdout:637: read oid 37 snap -1 2024-09-18T11:30:09.708 INFO:tasks.rados.rados.0.smithi119.stdout:637: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-18T11:30:09.708 INFO:tasks.rados.rados.0.smithi119.stdout:638: write (excl) oid 14 current snap is 0 2024-09-18T11:30:09.708 INFO:tasks.rados.rados.0.smithi119.stdout:625: finishing write tid 2 to smithi11939506-31 2024-09-18T11:30:09.708 INFO:tasks.rados.rados.0.smithi119.stdout:625: finishing write tid 3 to smithi11939506-31 2024-09-18T11:30:09.708 INFO:tasks.rados.rados.0.smithi119.stdout:625: finishing write tid 4 to smithi11939506-31 2024-09-18T11:30:09.708 INFO:tasks.rados.rados.0.smithi119.stdout:625: finishing write tid 5 to smithi11939506-31 2024-09-18T11:30:09.708 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 32 (ObjNum 63 snap 0 seq_num 63) dirty exists 2024-09-18T11:30:09.708 INFO:tasks.rados.rados.0.smithi119.stdout:625: left oid 31 (ObjNum 63 snap 0 seq_num 63) 2024-09-18T11:30:09.708 INFO:tasks.rados.rados.0.smithi119.stdout:626: expect (ObjNum 39 snap 0 seq_num 39) 2024-09-18T11:30:09.719 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:09.717+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:09.772 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:09.772 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:09.781 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:30:09.795 INFO:tasks.rados.rados.0.smithi119.stdout:628: finishing set_chunk to oid 20 2024-09-18T11:30:09.795 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 20 set_chunk returned error code -95 offset: 477696 length: 940032 2024-09-18T11:30:09.795 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 20 v 0 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-18T11:30:09.795 INFO:tasks.rados.rados.0.smithi119.stdout:629: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-18T11:30:09.819 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:09.818+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:09.873 INFO:tasks.rados.rados.0.smithi119.stdout:627: got (0) Success 2024-09-18T11:30:09.873 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 0 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-18T11:30:09.874 INFO:tasks.rados.rados.0.smithi119.stdout:631: finishing set_chunk to oid 11 2024-09-18T11:30:09.874 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 11 v 26 (ObjNum 56 snap 0 seq_num 56) dirty exists 2024-09-18T11:30:09.874 INFO:tasks.rados.rados.0.smithi119.stdout:632: finishing write tid 1 to smithi11939506-32 2024-09-18T11:30:09.874 INFO:tasks.rados.rados.0.smithi119.stdout:632: finishing write tid 2 to smithi11939506-32 2024-09-18T11:30:09.874 INFO:tasks.rados.rados.0.smithi119.stdout:632: finishing write tid 3 to smithi11939506-32 2024-09-18T11:30:09.874 INFO:tasks.rados.rados.0.smithi119.stdout:632: finishing write tid 4 to smithi11939506-32 2024-09-18T11:30:09.874 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 24 (ObjNum 64 snap 0 seq_num 64) dirty exists 2024-09-18T11:30:09.874 INFO:tasks.rados.rados.0.smithi119.stdout:632: left oid 32 (ObjNum 64 snap 0 seq_num 64) 2024-09-18T11:30:09.874 INFO:tasks.rados.rados.0.smithi119.stdout:633: got (0) Success 2024-09-18T11:30:09.874 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 41 v 0 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-18T11:30:09.874 INFO:tasks.rados.rados.0.smithi119.stdout:634: got (22) Invalid argument 2024-09-18T11:30:09.874 INFO:tasks.rados.rados.0.smithi119.stdout:638: seq_num 66 ranges {709262=697258,2012635=572018,3046388=62874} 2024-09-18T11:30:09.881 INFO:tasks.rados.rados.0.smithi119.stdout:638: writing smithi11939506-14 from 709262 to 1406520 tid 1 2024-09-18T11:30:09.887 INFO:tasks.rados.rados.0.smithi119.stdout:638: writing smithi11939506-14 from 2012635 to 2584653 tid 2 2024-09-18T11:30:09.890 INFO:tasks.rados.rados.0.smithi119.stdout:638: writing smithi11939506-14 from 3046388 to 3109262 tid 3 2024-09-18T11:30:09.891 INFO:tasks.rados.rados.0.smithi119.stdout:625: done (11 left) 2024-09-18T11:30:09.891 INFO:tasks.rados.rados.0.smithi119.stdout:626: done (10 left) 2024-09-18T11:30:09.891 INFO:tasks.rados.rados.0.smithi119.stdout:627: done (9 left) 2024-09-18T11:30:09.891 INFO:tasks.rados.rados.0.smithi119.stdout:628: done (8 left) 2024-09-18T11:30:09.891 INFO:tasks.rados.rados.0.smithi119.stdout:629: done (7 left) 2024-09-18T11:30:09.891 INFO:tasks.rados.rados.0.smithi119.stdout:631: done (6 left) 2024-09-18T11:30:09.891 INFO:tasks.rados.rados.0.smithi119.stdout:632: done (5 left) 2024-09-18T11:30:09.891 INFO:tasks.rados.rados.0.smithi119.stdout:633: done (4 left) 2024-09-18T11:30:09.891 INFO:tasks.rados.rados.0.smithi119.stdout:634: done (3 left) 2024-09-18T11:30:09.891 INFO:tasks.rados.rados.0.smithi119.stdout:639: read oid 43 snap -1 2024-09-18T11:30:09.891 INFO:tasks.rados.rados.0.smithi119.stdout:639: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:30:09.891 INFO:tasks.rados.rados.0.smithi119.stdout:640: rollback oid 29 current snap is 0 2024-09-18T11:30:09.892 INFO:tasks.rados.rados.0.smithi119.stdout:640: done (4 left) 2024-09-18T11:30:09.892 INFO:tasks.rados.rados.0.smithi119.stdout:641: read oid 22 snap -1 2024-09-18T11:30:09.892 INFO:tasks.rados.rados.0.smithi119.stdout:641: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-18T11:30:09.892 INFO:tasks.rados.rados.0.smithi119.stdout:642: delete oid 49 current snap is 0 2024-09-18T11:30:09.892 INFO:tasks.rados.rados.0.smithi119.stdout:636: finishing write tid 1 to smithi11939506-45 2024-09-18T11:30:09.892 INFO:tasks.rados.rados.0.smithi119.stdout:636: finishing write tid 2 to smithi11939506-45 2024-09-18T11:30:09.892 INFO:tasks.rados.rados.0.smithi119.stdout:636: finishing write tid 3 to smithi11939506-45 2024-09-18T11:30:09.892 INFO:tasks.rados.rados.0.smithi119.stdout:636: finishing write tid 4 to smithi11939506-45 2024-09-18T11:30:09.892 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 8 (ObjNum 65 snap 0 seq_num 65) dirty exists 2024-09-18T11:30:09.892 INFO:tasks.rados.rados.0.smithi119.stdout:636: left oid 45 (ObjNum 65 snap 0 seq_num 65) 2024-09-18T11:30:09.892 INFO:tasks.rados.rados.0.smithi119.stdout:637: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-18T11:30:09.895 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:09.895 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:09.904 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:30:09.949 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:09.947+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:09.976 INFO:tasks.rados.rados.0.smithi119.stdout:638: finishing write tid 1 to smithi11939506-14 2024-09-18T11:30:09.976 INFO:tasks.rados.rados.0.smithi119.stdout:638: finishing write tid 2 to smithi11939506-14 2024-09-18T11:30:09.976 INFO:tasks.rados.rados.0.smithi119.stdout:639: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:30:10.004 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:10.004 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:10.012 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:30:10.022 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:10.021+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:10.066 INFO:tasks.rados.rados.0.smithi119.stdout:638: finishing write tid 3 to smithi11939506-14 2024-09-18T11:30:10.066 INFO:tasks.rados.rados.0.smithi119.stdout:638: finishing write tid 4 to smithi11939506-14 2024-09-18T11:30:10.066 INFO:tasks.rados.rados.0.smithi119.stdout:638: finishing write tid 5 to smithi11939506-14 2024-09-18T11:30:10.067 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 32 (ObjNum 66 snap 0 seq_num 66) dirty exists 2024-09-18T11:30:10.067 INFO:tasks.rados.rados.0.smithi119.stdout:638: left oid 14 (ObjNum 66 snap 0 seq_num 66) 2024-09-18T11:30:10.067 INFO:tasks.rados.rados.0.smithi119.stdout:641: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-18T11:30:10.123 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:10.122+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:10.147 INFO:tasks.rados.rados.0.smithi119.stdout:636: done (5 left) 2024-09-18T11:30:10.147 INFO:tasks.rados.rados.0.smithi119.stdout:637: done (4 left) 2024-09-18T11:30:10.147 INFO:tasks.rados.rados.0.smithi119.stdout:638: done (3 left) 2024-09-18T11:30:10.147 INFO:tasks.rados.rados.0.smithi119.stdout:639: done (2 left) 2024-09-18T11:30:10.147 INFO:tasks.rados.rados.0.smithi119.stdout:641: done (1 left) 2024-09-18T11:30:10.147 INFO:tasks.rados.rados.0.smithi119.stdout:642: done (0 left) 2024-09-18T11:30:10.148 INFO:tasks.rados.rados.0.smithi119.stdout:643: snap_create 2024-09-18T11:30:10.152 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:10.152 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:10.161 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:30:10.176 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:30:10.245 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:10.243+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:10.304 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:10.304 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:10.313 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:30:10.318 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:30:10.319 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:30:10.334 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:10.334 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:10.349 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:10.349 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:10.357 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:10.356+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:10.364 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:10.364 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:10.379 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:10.379 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:10.396 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:10.396 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:10.410 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:10.411 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:10.416 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:10.416 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:10.416 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:10.416 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:10.425 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:30:10.426 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:10.426 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:10.466 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:10.465+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:10.527 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:10.526+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:10.563 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:10.563 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:10.563 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:10.563 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:10.563 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:10.571 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:30:10.656 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:10.655+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:10.711 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:10.711 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:10.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:10.711 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [] 2024-09-18T11:30:10.711 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:10.719 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:30:10.760 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:10.759+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:10.830 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:10.829+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:10.857 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:10.857 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:10.857 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:10.858 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:10.866 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:30:10.951 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:10.950+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:11.005 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:11.005 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:11.005 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:11.005 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:11.005 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:11.013 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:30:11.054 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:11.053+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:11.117 INFO:tasks.rados.rados.0.smithi119.stdout:643: done (0 left) 2024-09-18T11:30:11.117 INFO:tasks.rados.rados.0.smithi119.stdout:644: read oid 47 snap 0 2024-09-18T11:30:11.117 INFO:tasks.rados.rados.0.smithi119.stdout:644: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T11:30:11.117 INFO:tasks.rados.rados.0.smithi119.stdout:645: read oid 34 snap 0 2024-09-18T11:30:11.117 INFO:tasks.rados.rados.0.smithi119.stdout:645: expect deleted 2024-09-18T11:30:11.117 INFO:tasks.rados.rados.0.smithi119.stdout:646: tier_promote oid 38 2024-09-18T11:30:11.117 INFO:tasks.rados.rados.0.smithi119.stdout:647: read oid 43 snap 0 2024-09-18T11:30:11.118 INFO:tasks.rados.rados.0.smithi119.stdout:647: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:30:11.118 INFO:tasks.rados.rados.0.smithi119.stdout:648: read oid 22 snap -1 2024-09-18T11:30:11.118 INFO:tasks.rados.rados.0.smithi119.stdout:648: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-18T11:30:11.118 INFO:tasks.rados.rados.0.smithi119.stdout:649: set_chunk oid 4 target oid 2024-09-18T11:30:11.119 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:11.117+0000 7fa4696f1640 -1 osd.0 30 handle_osd_map injecting map crc failure 2024-09-18T11:30:11.132 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:11.131+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:11.133 INFO:tasks.rados.rados.0.smithi119.stdout:649: set_chunk oid 4 offset: 951808 length: 276992 target oid af1dd004311afeb9a2c923c847c83541b2f70de017741e00e9757a0c3b62b102 offset: 0 snap -1 2024-09-18T11:30:11.133 INFO:tasks.rados.rados.0.smithi119.stdout:650: read oid 28 snap -1 2024-09-18T11:30:11.134 INFO:tasks.rados.rados.0.smithi119.stdout:650: expect (ObjNum 27 snap 0 seq_num 27) 2024-09-18T11:30:11.134 INFO:tasks.rados.rados.0.smithi119.stdout:651: read oid 11 snap 0 2024-09-18T11:30:11.134 INFO:tasks.rados.rados.0.smithi119.stdout:651: expect (ObjNum 56 snap 0 seq_num 56) 2024-09-18T11:30:11.134 INFO:tasks.rados.rados.0.smithi119.stdout:652: read oid 50 snap -1 2024-09-18T11:30:11.134 INFO:tasks.rados.rados.0.smithi119.stdout:652: expect (ObjNum 58 snap 0 seq_num 58) 2024-09-18T11:30:11.134 INFO:tasks.rados.rados.0.smithi119.stdout:653: read oid 20 snap 0 2024-09-18T11:30:11.134 INFO:tasks.rados.rados.0.smithi119.stdout:653: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-18T11:30:11.134 INFO:tasks.rados.rados.0.smithi119.stdout:654: write (excl) oid 1 current snap is 1 2024-09-18T11:30:11.134 INFO:tasks.rados.rados.0.smithi119.stdout:654: seq_num 68 ranges {742876=447756,1729599=575687,3101234=41642} 2024-09-18T11:30:11.140 INFO:tasks.rados.rados.0.smithi119.stdout:654: writing smithi11939506-1 from 742876 to 1190632 tid 1 2024-09-18T11:30:11.142 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:11.140+0000 7f6bf3052640 -1 osd.5 30 handle_osd_map injecting map crc failure 2024-09-18T11:30:11.146 INFO:tasks.rados.rados.0.smithi119.stdout:654: writing smithi11939506-1 from 1729599 to 2305286 tid 2 2024-09-18T11:30:11.152 INFO:tasks.rados.rados.0.smithi119.stdout:654: writing smithi11939506-1 from 3101234 to 3142876 tid 3 2024-09-18T11:30:11.152 INFO:tasks.rados.rados.0.smithi119.stdout:655: write (excl) oid 33 current snap is 1 2024-09-18T11:30:11.152 INFO:tasks.rados.rados.0.smithi119.stdout:655: seq_num 69 ranges {759683=639358,1904434=719345,3128657=31026} 2024-09-18T11:30:11.159 INFO:tasks.rados.rados.0.smithi119.stdout:655: writing smithi11939506-33 from 759683 to 1399041 tid 1 2024-09-18T11:30:11.163 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:11.163 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:11.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:11.163 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:11.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.163 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:30:11.164 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:30:11.164 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.135254890999999, 2024-09-18T11:30:11.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073601550000000002, 2024-09-18T11:30:11.164 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:11.164 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:11.164 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:11.164 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:11.164 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:11.164 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1 2024-09-18T11:30:11.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.164 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:11.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.164 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:11.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:30:11.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:11.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:30:11.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:11.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026524+0000", 2024-09-18T11:30:11.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999905 2024-09-18T11:30:11.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:11.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026541+0000", 2024-09-18T11:30:11.166 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7070000000000001e-05 2024-09-18T11:30:11.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:11.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026554+0000", 2024-09-18T11:30:11.166 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.288e-05 2024-09-18T11:30:11.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:11.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026564+0000", 2024-09-18T11:30:11.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0197000000000001e-05 2024-09-18T11:30:11.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026826+0000", 2024-09-18T11:30:11.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026212400000000003 2024-09-18T11:30:11.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:30:11.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026839+0000", 2024-09-18T11:30:11.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2831e-05 2024-09-18T11:30:11.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028605+0000", 2024-09-18T11:30:11.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001765794 2024-09-18T11:30:11.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:11.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028709+0000", 2024-09-18T11:30:11.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104348 2024-09-18T11:30:11.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:11.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028824+0000", 2024-09-18T11:30:11.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011527399999999999 2024-09-18T11:30:11.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:11.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030319+0000", 2024-09-18T11:30:11.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001494439 2024-09-18T11:30:11.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:11.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033805+0000", 2024-09-18T11:30:11.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034857820000000002 2024-09-18T11:30:11.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:11.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033856+0000", 2024-09-18T11:30:11.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1687999999999997e-05 2024-09-18T11:30:11.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:11.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033894+0000", 2024-09-18T11:30:11.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7481000000000002e-05 2024-09-18T11:30:11.170 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.170 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.170 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.170 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(client.4266.0:1 2.3 e16/15 ondisk, result = 0)", 2024-09-18T11:30:11.170 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.030256+0000", 2024-09-18T11:30:11.170 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.131532314000001, 2024-09-18T11:30:11.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4136999999999999e-05, 2024-09-18T11:30:11.171 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:11.171 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:11.171 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:11.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:11.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030256+0000", 2024-09-18T11:30:11.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:11.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030256+0000", 2024-09-18T11:30:11.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.172 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:11.172 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030256+0000", 2024-09-18T11:30:11.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:11.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.172 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:11.172 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030260+0000", 2024-09-18T11:30:11.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3100000000000002e-06 2024-09-18T11:30:11.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.172 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:11.172 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030270+0000", 2024-09-18T11:30:11.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0276e-05 2024-09-18T11:30:11.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.173 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:11.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030278+0000", 2024-09-18T11:30:11.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8250000000000005e-06 2024-09-18T11:30:11.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.173 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030310+0000", 2024-09-18T11:30:11.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1797000000000001e-05 2024-09-18T11:30:11.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.173 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:11.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030315+0000", 2024-09-18T11:30:11.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6909999999999996e-06 2024-09-18T11:30:11.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.174 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:11.174 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030320+0000", 2024-09-18T11:30:11.174 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7899999999999996e-06 2024-09-18T11:30:11.174 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.174 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.174 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.174 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:30:11.174 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.036715+0000", 2024-09-18T11:30:11.174 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.125073975999999, 2024-09-18T11:30:11.175 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016893799999999999, 2024-09-18T11:30:11.175 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:11.175 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "reached pg", 2024-09-18T11:30:11.175 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:11.175 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:11.175 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:11.175 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2 2024-09-18T11:30:11.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.175 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:11.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.175 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:11.175 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.036715+0000", 2024-09-18T11:30:11.175 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:11.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.036715+0000", 2024-09-18T11:30:11.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:11.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.036713+0000", 2024-09-18T11:30:11.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:11.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:11.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.036720+0000", 2024-09-18T11:30:11.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9600000000000003e-06 2024-09-18T11:30:11.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:11.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.036731+0000", 2024-09-18T11:30:11.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1685000000000001e-05 2024-09-18T11:30:11.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:11.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.036741+0000", 2024-09-18T11:30:11.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2469999999999997e-06 2024-09-18T11:30:11.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.036788+0000", 2024-09-18T11:30:11.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6823000000000002e-05 2024-09-18T11:30:11.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:11.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.036883+0000", 2024-09-18T11:30:11.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5942999999999999e-05 2024-09-18T11:30:11.178 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.178 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.178 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.178 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:30:11.178 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.037023+0000", 2024-09-18T11:30:11.179 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.124765289999999, 2024-09-18T11:30:11.179 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013824700000000001, 2024-09-18T11:30:11.179 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:11.179 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "reached pg", 2024-09-18T11:30:11.179 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:11.179 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:11.179 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:11.179 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3 2024-09-18T11:30:11.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.179 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:11.179 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.179 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:11.179 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.037023+0000", 2024-09-18T11:30:11.180 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.180 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.180 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.180 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:11.180 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.037023+0000", 2024-09-18T11:30:11.180 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.180 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.180 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.180 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:11.180 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.037023+0000", 2024-09-18T11:30:11.180 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:11.180 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.180 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.180 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:11.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.037027+0000", 2024-09-18T11:30:11.181 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5519999999999998e-06 2024-09-18T11:30:11.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.181 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:11.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.037036+0000", 2024-09-18T11:30:11.181 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.653e-06 2024-09-18T11:30:11.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.181 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:11.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.037041+0000", 2024-09-18T11:30:11.181 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9690000000000002e-06 2024-09-18T11:30:11.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.181 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.182 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.037074+0000", 2024-09-18T11:30:11.182 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3031999999999998e-05 2024-09-18T11:30:11.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.182 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.182 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:11.182 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.037161+0000", 2024-09-18T11:30:11.182 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7664000000000006e-05 2024-09-18T11:30:11.182 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.182 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.182 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.182 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.182 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T11:30:11.182 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:51.038968+0000", 2024-09-18T11:30:11.182 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 20.122820697000002, 2024-09-18T11:30:11.183 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028722909999999999, 2024-09-18T11:30:11.183 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:11.183 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:11.183 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:11.183 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:11.183 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:11.183 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 5 2024-09-18T11:30:11.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.183 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:11.183 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.183 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:11.183 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.038968+0000", 2024-09-18T11:30:11.183 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.184 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.184 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:11.184 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.038968+0000", 2024-09-18T11:30:11.184 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.184 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.184 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:11.184 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.038966+0000", 2024-09-18T11:30:11.184 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:11.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.184 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.184 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:11.184 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.038983+0000", 2024-09-18T11:30:11.184 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7099e-05 2024-09-18T11:30:11.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.185 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.185 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:11.185 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.039002+0000", 2024-09-18T11:30:11.185 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8938000000000001e-05 2024-09-18T11:30:11.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.185 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.185 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:11.185 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.039012+0000", 2024-09-18T11:30:11.185 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9529999999999994e-06 2024-09-18T11:30:11.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.185 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.185 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.185 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.039068+0000", 2024-09-18T11:30:11.186 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5893999999999997e-05 2024-09-18T11:30:11.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.186 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:11.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.039074+0000", 2024-09-18T11:30:11.186 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8510000000000001e-06 2024-09-18T11:30:11.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.186 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.040586+0000", 2024-09-18T11:30:11.186 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015119809999999999 2024-09-18T11:30:11.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.186 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:11.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.040705+0000", 2024-09-18T11:30:11.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000119054 2024-09-18T11:30:11.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:11.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.040812+0000", 2024-09-18T11:30:11.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010652700000000001 2024-09-18T11:30:11.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:11.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041405+0000", 2024-09-18T11:30:11.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059319099999999997 2024-09-18T11:30:11.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:11.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041787+0000", 2024-09-18T11:30:11.188 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00038150800000000002 2024-09-18T11:30:11.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.188 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.188 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:11.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041817+0000", 2024-09-18T11:30:11.188 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0091e-05 2024-09-18T11:30:11.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.188 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.188 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:11.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041840+0000", 2024-09-18T11:30:11.188 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3385000000000001e-05 2024-09-18T11:30:11.188 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.188 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.188 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.189 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.189 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(client.4266.0:5 2.2 e17/15 ondisk, result = 0)", 2024-09-18T11:30:11.189 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:51.041723+0000", 2024-09-18T11:30:11.189 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 20.120065223000001, 2024-09-18T11:30:11.189 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123433, 2024-09-18T11:30:11.189 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:11.189 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:11.189 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:11.189 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.189 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:11.189 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041723+0000", 2024-09-18T11:30:11.189 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.190 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.190 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:11.190 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041723+0000", 2024-09-18T11:30:11.190 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.190 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.190 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:11.190 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041722+0000", 2024-09-18T11:30:11.190 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:11.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.190 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.190 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:11.190 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041728+0000", 2024-09-18T11:30:11.190 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6160000000000001e-06 2024-09-18T11:30:11.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.191 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.191 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:11.191 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041740+0000", 2024-09-18T11:30:11.191 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2602999999999999e-05 2024-09-18T11:30:11.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.191 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.191 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:11.191 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041752+0000", 2024-09-18T11:30:11.191 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1630999999999999e-05 2024-09-18T11:30:11.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.191 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.191 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.191 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041778+0000", 2024-09-18T11:30:11.192 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6435e-05 2024-09-18T11:30:11.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.192 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.192 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:11.192 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041783+0000", 2024-09-18T11:30:11.192 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8130000000000003e-06 2024-09-18T11:30:11.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.192 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.192 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:11.192 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041847+0000", 2024-09-18T11:30:11.192 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3689000000000006e-05 2024-09-18T11:30:11.192 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.192 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.193 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.193 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.193 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T11:30:11.193 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:30:11.193 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.115976459999999, 2024-09-18T11:30:11.193 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055107869999999996, 2024-09-18T11:30:11.193 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:11.193 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:11.193 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:11.193 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:11.193 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:11.193 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 7 2024-09-18T11:30:11.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.194 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:11.194 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.194 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:11.194 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:30:11.194 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.194 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.194 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.194 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:11.194 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:30:11.194 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.194 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.194 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.194 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:11.195 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045810+0000", 2024-09-18T11:30:11.195 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:11.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.195 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.195 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:11.195 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045820+0000", 2024-09-18T11:30:11.195 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0215e-05 2024-09-18T11:30:11.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.195 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.195 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:11.195 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045838+0000", 2024-09-18T11:30:11.195 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7849000000000001e-05 2024-09-18T11:30:11.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.195 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.196 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:11.196 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045845+0000", 2024-09-18T11:30:11.196 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4229999999999999e-06 2024-09-18T11:30:11.196 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.196 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.196 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.196 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045898+0000", 2024-09-18T11:30:11.196 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2537999999999997e-05 2024-09-18T11:30:11.196 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.196 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.196 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:11.196 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045904+0000", 2024-09-18T11:30:11.196 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4019999999999999e-06 2024-09-18T11:30:11.197 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.197 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.197 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.197 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047663+0000", 2024-09-18T11:30:11.197 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017584079999999999 2024-09-18T11:30:11.197 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.197 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.197 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:11.197 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047736+0000", 2024-09-18T11:30:11.197 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3509000000000004e-05 2024-09-18T11:30:11.197 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.197 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.197 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:11.198 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047857+0000", 2024-09-18T11:30:11.198 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012037800000000001 2024-09-18T11:30:11.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.198 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.198 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:11.198 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.050868+0000", 2024-09-18T11:30:11.198 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030111420000000001 2024-09-18T11:30:11.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.198 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.198 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:11.198 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051239+0000", 2024-09-18T11:30:11.198 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037141000000000002 2024-09-18T11:30:11.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.199 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.199 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:11.199 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051288+0000", 2024-09-18T11:30:11.199 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9051000000000002e-05 2024-09-18T11:30:11.199 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.199 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.199 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:11.199 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051323+0000", 2024-09-18T11:30:11.199 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4545000000000002e-05 2024-09-18T11:30:11.199 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.199 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.199 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.199 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.200 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.200 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(client.4266.0:7 2.2 e18/15 ondisk, result = 0)", 2024-09-18T11:30:11.200 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:52.051168+0000", 2024-09-18T11:30:11.200 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.110620699999998, 2024-09-18T11:30:11.200 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000162237, 2024-09-18T11:30:11.200 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:11.200 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:11.200 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:11.200 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.200 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:11.200 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051168+0000", 2024-09-18T11:30:11.200 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.200 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.201 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.201 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:11.201 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051168+0000", 2024-09-18T11:30:11.201 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.201 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.201 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:11.201 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051167+0000", 2024-09-18T11:30:11.201 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:11.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.201 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.201 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:11.201 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051172+0000", 2024-09-18T11:30:11.201 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5609999999999998e-06 2024-09-18T11:30:11.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.202 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.202 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:11.202 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051184+0000", 2024-09-18T11:30:11.202 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1731e-05 2024-09-18T11:30:11.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.202 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.202 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:11.202 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051190+0000", 2024-09-18T11:30:11.202 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2199999999999997e-06 2024-09-18T11:30:11.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.202 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.202 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.202 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051224+0000", 2024-09-18T11:30:11.202 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3899999999999997e-05 2024-09-18T11:30:11.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.203 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.203 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:11.203 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051233+0000", 2024-09-18T11:30:11.203 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9560000000000003e-06 2024-09-18T11:30:11.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.203 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.203 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:11.203 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051330+0000", 2024-09-18T11:30:11.203 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.6807999999999999e-05 2024-09-18T11:30:11.203 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.203 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.203 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.203 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.204 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1 4.7 e28/22 4:e73ba68f:::smithi11939506-1:head v 28'1, mlcod=0'0)", 2024-09-18T11:30:11.204 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:08.099940+0000", 2024-09-18T11:30:11.204 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.0618482340000002, 2024-09-18T11:30:11.204 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024513, 2024-09-18T11:30:11.204 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:11.204 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:11.204 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:11.204 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.204 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:11.204 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.099940+0000", 2024-09-18T11:30:11.204 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.204 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.205 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:11.205 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.099940+0000", 2024-09-18T11:30:11.205 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.205 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.205 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.205 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:11.205 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.099938+0000", 2024-09-18T11:30:11.205 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:11.205 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.205 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.205 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:11.205 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.100510+0000", 2024-09-18T11:30:11.205 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00057168799999999999 2024-09-18T11:30:11.205 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.206 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.206 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:11.206 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.100526+0000", 2024-09-18T11:30:11.206 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5753999999999999e-05 2024-09-18T11:30:11.206 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.206 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.206 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:11.206 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.100533+0000", 2024-09-18T11:30:11.206 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9450000000000004e-06 2024-09-18T11:30:11.206 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.206 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.206 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.206 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.100578+0000", 2024-09-18T11:30:11.207 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5247999999999998e-05 2024-09-18T11:30:11.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.207 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.207 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:11.207 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.100602+0000", 2024-09-18T11:30:11.207 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4275999999999999e-05 2024-09-18T11:30:11.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.207 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.207 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:11.207 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.102350+0000", 2024-09-18T11:30:11.207 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017479699999999999 2024-09-18T11:30:11.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.207 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.207 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:11.207 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.102392+0000", 2024-09-18T11:30:11.208 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1544000000000001e-05 2024-09-18T11:30:11.208 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.208 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.208 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.208 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.208 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.208 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2 4.7 e28/22 4:e73ba68f:::smithi11939506-1:head v 28'2, mlcod=0'0)", 2024-09-18T11:30:11.208 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:08.105343+0000", 2024-09-18T11:30:11.208 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.0564456849999999, 2024-09-18T11:30:11.208 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040860050000000002, 2024-09-18T11:30:11.208 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:11.208 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:11.208 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:11.208 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.209 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:11.209 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105343+0000", 2024-09-18T11:30:11.209 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.209 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.209 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:11.209 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105343+0000", 2024-09-18T11:30:11.209 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.209 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.209 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:11.209 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105342+0000", 2024-09-18T11:30:11.209 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:11.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.210 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.210 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:11.210 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105801+0000", 2024-09-18T11:30:11.210 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00045840099999999999 2024-09-18T11:30:11.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.210 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.210 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:11.210 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105813+0000", 2024-09-18T11:30:11.210 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2461e-05 2024-09-18T11:30:11.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.210 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.210 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:11.210 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105818+0000", 2024-09-18T11:30:11.210 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.989e-06 2024-09-18T11:30:11.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.211 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.211 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.211 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105856+0000", 2024-09-18T11:30:11.211 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7839999999999997e-05 2024-09-18T11:30:11.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.211 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.211 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:11.211 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105872+0000", 2024-09-18T11:30:11.211 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6582000000000001e-05 2024-09-18T11:30:11.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.211 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.211 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:11.211 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.109389+0000", 2024-09-18T11:30:11.212 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003516211 2024-09-18T11:30:11.212 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.212 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.212 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:11.212 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.109429+0000", 2024-09-18T11:30:11.212 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0261e-05 2024-09-18T11:30:11.212 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.212 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.212 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.212 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.212 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.212 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3 4.7 e28/22 4:e73ba68f:::smithi11939506-1:head v 28'3, mlcod=0'0)", 2024-09-18T11:30:11.212 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:08.106522+0000", 2024-09-18T11:30:11.212 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.0552666689999999, 2024-09-18T11:30:11.213 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031398509999999999, 2024-09-18T11:30:11.213 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:11.213 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:11.213 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:11.213 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.213 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:11.213 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106522+0000", 2024-09-18T11:30:11.213 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.213 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.213 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.213 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:11.213 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106522+0000", 2024-09-18T11:30:11.213 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.213 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.214 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.214 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:11.214 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106521+0000", 2024-09-18T11:30:11.214 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:11.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.214 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.214 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:11.214 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106526+0000", 2024-09-18T11:30:11.214 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9799999999999998e-06 2024-09-18T11:30:11.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.214 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.214 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:11.214 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106533+0000", 2024-09-18T11:30:11.214 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1229999999999997e-06 2024-09-18T11:30:11.215 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.215 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.215 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:11.215 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106536+0000", 2024-09-18T11:30:11.215 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.055e-06 2024-09-18T11:30:11.215 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.215 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.215 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.215 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106577+0000", 2024-09-18T11:30:11.215 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0932e-05 2024-09-18T11:30:11.215 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.215 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.216 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:11.216 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106588+0000", 2024-09-18T11:30:11.216 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1486e-05 2024-09-18T11:30:11.216 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.216 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.216 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:11.216 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.109630+0000", 2024-09-18T11:30:11.216 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030422159999999999 2024-09-18T11:30:11.216 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.216 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.216 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:11.216 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.109662+0000", 2024-09-18T11:30:11.216 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1498999999999998e-05 2024-09-18T11:30:11.216 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.217 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.217 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.217 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.217 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.217 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:11.217 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T11:30:11.219 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:30:11.220 INFO:tasks.rados.rados.0.smithi119.stdout:655: writing smithi11939506-33 from 1904434 to 2623779 tid 2 2024-09-18T11:30:11.220 INFO:tasks.rados.rados.0.smithi119.stdout:655: writing smithi11939506-33 from 3128657 to 3159683 tid 3 2024-09-18T11:30:11.220 INFO:tasks.rados.rados.0.smithi119.stdout:656: read oid 16 snap 0 2024-09-18T11:30:11.220 INFO:tasks.rados.rados.0.smithi119.stdout:656: expect (ObjNum 61 snap 0 seq_num 61) 2024-09-18T11:30:11.220 INFO:tasks.rados.rados.0.smithi119.stdout:657: read oid 36 snap -1 2024-09-18T11:30:11.220 INFO:tasks.rados.rados.0.smithi119.stdout:657: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-18T11:30:11.220 INFO:tasks.rados.rados.0.smithi119.stdout:658: tier_promote oid 29 2024-09-18T11:30:11.220 INFO:tasks.rados.rados.0.smithi119.stdout:659: read oid 13 snap -1 2024-09-18T11:30:11.220 INFO:tasks.rados.rados.0.smithi119.stdout:659: expect (ObjNum 52 snap 0 seq_num 52) 2024-09-18T11:30:11.220 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:11.221 INFO:tasks.rados.rados.0.smithi119.stdout:645: done (15 left) 2024-09-18T11:30:11.221 INFO:tasks.rados.rados.0.smithi119.stdout:660: tier_promote oid 40 2024-09-18T11:30:11.221 INFO:tasks.rados.rados.0.smithi119.stdout:648: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-18T11:30:11.250 INFO:tasks.rados.rados.0.smithi119.stdout:647: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:30:11.260 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:11.259+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:11.333 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:11.331+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:11.340 INFO:tasks.rados.rados.0.smithi119.stdout:644: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T11:30:11.359 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:11.359 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:11.359 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:11.359 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:11.367 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:30:11.431 INFO:tasks.rados.rados.0.smithi119.stdout:649: finishing set_chunk to oid 4 2024-09-18T11:30:11.431 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 4 set_chunk returned error code -95 offset: 951808 length: 276992 2024-09-18T11:30:11.431 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 4 v 0 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-18T11:30:11.431 INFO:tasks.rados.rados.0.smithi119.stdout:644: done (15 left) 2024-09-18T11:30:11.432 INFO:tasks.rados.rados.0.smithi119.stdout:647: done (14 left) 2024-09-18T11:30:11.432 INFO:tasks.rados.rados.0.smithi119.stdout:648: done (13 left) 2024-09-18T11:30:11.432 INFO:tasks.rados.rados.0.smithi119.stdout:649: done (12 left) 2024-09-18T11:30:11.432 INFO:tasks.rados.rados.0.smithi119.stdout:661: tier_evict oid 41 2024-09-18T11:30:11.432 INFO:tasks.rados.rados.0.smithi119.stdout:661: tier_evict oid 41 snap -1 2024-09-18T11:30:11.432 INFO:tasks.rados.rados.0.smithi119.stdout:650: expect (ObjNum 27 snap 0 seq_num 27) 2024-09-18T11:30:11.434 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:11.433+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:11.465 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:11.465 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:11.465 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:11.466 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:11.466 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:11.474 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:30:11.508 INFO:tasks.rados.rados.0.smithi119.stdout:653: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-18T11:30:11.535 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:11.533+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:11.585 INFO:tasks.rados.rados.0.smithi119.stdout:652: expect (ObjNum 58 snap 0 seq_num 58) 2024-09-18T11:30:11.613 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:11.613 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:11.613 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:11.614 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:11.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.614 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:11.614 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:11.614 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.49594179700000002, 2024-09-18T11:30:11.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:30:11.614 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:11.614 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:11.614 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:11.614 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:11.614 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:11.614 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:30:11.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.615 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:11.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.615 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:11.615 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:11.615 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.615 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:11.615 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:11.615 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.615 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:11.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:30:11.616 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:11.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.616 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:11.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:30:11.616 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:30:11.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.616 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:11.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:30:11.616 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:30:11.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.617 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:11.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:30:11.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:30:11.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.617 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:30:11.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:30:11.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.617 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:11.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:30:11.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:30:11.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:30:11.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:30:11.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:11.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:30:11.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:30:11.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:11.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:30:11.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:30:11.619 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.619 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.619 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.619 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:896 4.5 4:aa575018:::smithi11939506-11:2 [read 0~2941192,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:11.619 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.132995+0000", 2024-09-18T11:30:11.619 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.479266156, 2024-09-18T11:30:11.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047874989999999997, 2024-09-18T11:30:11.619 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:11.619 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:11.619 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:11.620 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:11.620 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:11.620 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 896 2024-09-18T11:30:11.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.620 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:11.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.620 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:11.620 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.132995+0000", 2024-09-18T11:30:11.620 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.620 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:11.620 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.132995+0000", 2024-09-18T11:30:11.620 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:11.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.132993+0000", 2024-09-18T11:30:11.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:11.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:11.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133000+0000", 2024-09-18T11:30:11.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5509999999999996e-06 2024-09-18T11:30:11.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:11.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133010+0000", 2024-09-18T11:30:11.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0254e-05 2024-09-18T11:30:11.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:11.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133023+0000", 2024-09-18T11:30:11.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3458999999999999e-05 2024-09-18T11:30:11.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133069+0000", 2024-09-18T11:30:11.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5324000000000002e-05 2024-09-18T11:30:11.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.623 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:11.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133161+0000", 2024-09-18T11:30:11.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2773000000000001e-05 2024-09-18T11:30:11.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.623 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:11.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.137782+0000", 2024-09-18T11:30:11.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046208680000000002 2024-09-18T11:30:11.623 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.623 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.624 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.624 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:897 4.5 4:aa575018:::smithi11939506-11:2 [sparse-read 0~1343176] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:11.624 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.133034+0000", 2024-09-18T11:30:11.624 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.47922679699999998, 2024-09-18T11:30:11.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051617950000000003, 2024-09-18T11:30:11.624 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:11.624 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:11.624 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:11.624 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:11.624 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:11.624 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 897 2024-09-18T11:30:11.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.625 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:11.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.625 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:11.625 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133034+0000", 2024-09-18T11:30:11.625 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.625 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:11.625 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133034+0000", 2024-09-18T11:30:11.625 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.625 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:11.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133033+0000", 2024-09-18T11:30:11.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:11.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:11.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133037+0000", 2024-09-18T11:30:11.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.118e-06 2024-09-18T11:30:11.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:11.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133045+0000", 2024-09-18T11:30:11.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1780000000000003e-06 2024-09-18T11:30:11.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:11.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133049+0000", 2024-09-18T11:30:11.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7730000000000001e-06 2024-09-18T11:30:11.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.137838+0000", 2024-09-18T11:30:11.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00478912 2024-09-18T11:30:11.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:11.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.137936+0000", 2024-09-18T11:30:11.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.823e-05 2024-09-18T11:30:11.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.628 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:11.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138196+0000", 2024-09-18T11:30:11.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00025993799999999998 2024-09-18T11:30:11.628 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.628 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.628 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.628 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:898 4.5 4:aa575018:::smithi11939506-11:2 [sparse-read 0~1343176] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:11.628 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:11.628 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.479197759, 2024-09-18T11:30:11.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055588349999999998, 2024-09-18T11:30:11.629 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:11.629 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:11.629 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:11.629 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:11.629 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:11.629 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 898 2024-09-18T11:30:11.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.629 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:11.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:11.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:11.629 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:11.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:11.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:11.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:11.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:11.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:11.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133066+0000", 2024-09-18T11:30:11.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2729999999999998e-06 2024-09-18T11:30:11.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:11.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133072+0000", 2024-09-18T11:30:11.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0680000000000002e-06 2024-09-18T11:30:11.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:11.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133077+0000", 2024-09-18T11:30:11.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3569999999999997e-06 2024-09-18T11:30:11.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138226+0000", 2024-09-18T11:30:11.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051480980000000003 2024-09-18T11:30:11.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.632 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:11.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138324+0000", 2024-09-18T11:30:11.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8639000000000002e-05 2024-09-18T11:30:11.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.632 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:11.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138622+0000", 2024-09-18T11:30:11.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029774100000000002 2024-09-18T11:30:11.632 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.632 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.632 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.633 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.6.0:46 4.a e31/22 ondisk, result = 0)", 2024-09-18T11:30:11.633 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.138507+0000", 2024-09-18T11:30:11.633 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.47375345899999999, 2024-09-18T11:30:11.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000151705, 2024-09-18T11:30:11.633 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:11.633 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:11.633 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:11.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:11.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138507+0000", 2024-09-18T11:30:11.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:11.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138507+0000", 2024-09-18T11:30:11.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:11.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138505+0000", 2024-09-18T11:30:11.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:11.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:11.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138517+0000", 2024-09-18T11:30:11.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.172e-05 2024-09-18T11:30:11.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.635 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:11.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138531+0000", 2024-09-18T11:30:11.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3516000000000001e-05 2024-09-18T11:30:11.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.635 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:11.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138544+0000", 2024-09-18T11:30:11.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3362e-05 2024-09-18T11:30:11.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.635 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138587+0000", 2024-09-18T11:30:11.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3056999999999998e-05 2024-09-18T11:30:11.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.636 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:11.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138595+0000", 2024-09-18T11:30:11.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1089999999999998e-06 2024-09-18T11:30:11.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.636 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:11.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138659+0000", 2024-09-18T11:30:11.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4032e-05 2024-09-18T11:30:11.636 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.636 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.636 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.637 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "pg_update_log_missing_reply(4.a epoch 31/22 rep_tid 47 lcod 31'21)", 2024-09-18T11:30:11.637 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.138561+0000", 2024-09-18T11:30:11.637 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.47369962100000002, 2024-09-18T11:30:11.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017231700000000001, 2024-09-18T11:30:11.637 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:11.637 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "reached pg", 2024-09-18T11:30:11.637 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:11.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:11.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138561+0000", 2024-09-18T11:30:11.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:11.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138561+0000", 2024-09-18T11:30:11.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:11.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138561+0000", 2024-09-18T11:30:11.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:11.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:11.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138565+0000", 2024-09-18T11:30:11.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1019999999999999e-06 2024-09-18T11:30:11.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.639 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:11.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138572+0000", 2024-09-18T11:30:11.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7220000000000002e-06 2024-09-18T11:30:11.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.639 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:11.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138575+0000", 2024-09-18T11:30:11.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8249999999999998e-06 2024-09-18T11:30:11.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.639 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138679+0000", 2024-09-18T11:30:11.640 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103088 2024-09-18T11:30:11.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.640 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:11.640 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138734+0000", 2024-09-18T11:30:11.640 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4954999999999998e-05 2024-09-18T11:30:11.640 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.640 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.640 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.640 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:918 4.a 4:5975a5ec:::smithi11939506-36:head [sparse-read 0~1188805,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:11.640 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.169320+0000", 2024-09-18T11:30:11.640 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.44294046199999998, 2024-09-18T11:30:11.640 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000387907, 2024-09-18T11:30:11.641 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:11.641 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:11.641 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:11.641 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:11.641 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:11.641 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 918 2024-09-18T11:30:11.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.641 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:11.641 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.641 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:11.641 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169320+0000", 2024-09-18T11:30:11.641 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.641 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:11.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169320+0000", 2024-09-18T11:30:11.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:11.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169319+0000", 2024-09-18T11:30:11.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:11.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:11.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169328+0000", 2024-09-18T11:30:11.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2289999999999992e-06 2024-09-18T11:30:11.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:11.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169345+0000", 2024-09-18T11:30:11.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7597000000000001e-05 2024-09-18T11:30:11.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:11.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169355+0000", 2024-09-18T11:30:11.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.132e-06 2024-09-18T11:30:11.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169393+0000", 2024-09-18T11:30:11.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8600000000000003e-05 2024-09-18T11:30:11.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.644 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:11.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169464+0000", 2024-09-18T11:30:11.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0704999999999995e-05 2024-09-18T11:30:11.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.644 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:11.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169708+0000", 2024-09-18T11:30:11.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00024441399999999998 2024-09-18T11:30:11.644 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.644 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.644 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.645 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:919 4.a 4:5975a5ec:::smithi11939506-36:head [sparse-read 0~1188805] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:11.645 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.169362+0000", 2024-09-18T11:30:11.645 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.442898348, 2024-09-18T11:30:11.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000637215, 2024-09-18T11:30:11.645 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:11.645 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:11.645 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:11.645 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:11.645 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:11.645 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 919 2024-09-18T11:30:11.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.645 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:11.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.646 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:11.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169362+0000", 2024-09-18T11:30:11.646 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.646 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:11.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169362+0000", 2024-09-18T11:30:11.646 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.646 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:11.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169362+0000", 2024-09-18T11:30:11.646 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:11.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:11.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169364+0000", 2024-09-18T11:30:11.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9439999999999999e-06 2024-09-18T11:30:11.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:11.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169368+0000", 2024-09-18T11:30:11.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5779999999999999e-06 2024-09-18T11:30:11.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:11.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169370+0000", 2024-09-18T11:30:11.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0679999999999999e-06 2024-09-18T11:30:11.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169738+0000", 2024-09-18T11:30:11.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036796099999999998 2024-09-18T11:30:11.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:11.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169797+0000", 2024-09-18T11:30:11.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9592999999999997e-05 2024-09-18T11:30:11.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.649 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:11.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.170000+0000", 2024-09-18T11:30:11.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020237200000000001 2024-09-18T11:30:11.649 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.649 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.649 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.649 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:920 4.a 4:5975a5ec:::smithi11939506-36:head [read 0~2588245,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:11.649 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.169388+0000", 2024-09-18T11:30:11.649 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.44287238800000001, 2024-09-18T11:30:11.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012905620000000001, 2024-09-18T11:30:11.649 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:11.650 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:11.650 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:11.650 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:11.650 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:11.650 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 920 2024-09-18T11:30:11.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.650 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:11.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.650 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:11.650 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169388+0000", 2024-09-18T11:30:11.650 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.650 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:11.651 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169388+0000", 2024-09-18T11:30:11.651 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:11.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.651 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:11.651 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169388+0000", 2024-09-18T11:30:11.651 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:11.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.651 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:11.651 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169391+0000", 2024-09-18T11:30:11.651 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5129999999999999e-06 2024-09-18T11:30:11.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.651 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:11.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169395+0000", 2024-09-18T11:30:11.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3460000000000004e-06 2024-09-18T11:30:11.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:11.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169398+0000", 2024-09-18T11:30:11.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.553e-06 2024-09-18T11:30:11.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:11.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.170021+0000", 2024-09-18T11:30:11.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00062326999999999999 2024-09-18T11:30:11.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:11.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.170091+0000", 2024-09-18T11:30:11.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9836e-05 2024-09-18T11:30:11.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:11.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:11.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:11.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.170679+0000", 2024-09-18T11:30:11.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00058834600000000001 2024-09-18T11:30:11.653 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.653 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.653 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.653 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:11.653 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:11.653 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:11.655 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:30:11.655 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:11.653+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:11.717 INFO:tasks.rados.rados.0.smithi119.stdout:651: expect (ObjNum 56 snap 0 seq_num 56) 2024-09-18T11:30:11.755 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:11.753+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:11.793 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:11.793 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:11.793 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:11.794 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:11.802 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:30:11.847 INFO:tasks.rados.rados.0.smithi119.stdout:646: got (0) Success 2024-09-18T11:30:11.847 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 9 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-18T11:30:11.847 INFO:tasks.rados.rados.0.smithi119.stdout:654: finishing write tid 1 to smithi11939506-1 2024-09-18T11:30:11.847 INFO:tasks.rados.rados.0.smithi119.stdout:654: finishing write tid 2 to smithi11939506-1 2024-09-18T11:30:11.847 INFO:tasks.rados.rados.0.smithi119.stdout:654: finishing write tid 3 to smithi11939506-1 2024-09-18T11:30:11.847 INFO:tasks.rados.rados.0.smithi119.stdout:654: finishing write tid 4 to smithi11939506-1 2024-09-18T11:30:11.847 INFO:tasks.rados.rados.0.smithi119.stdout:654: finishing write tid 5 to smithi11939506-1 2024-09-18T11:30:11.847 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 35 (ObjNum 68 snap 1 seq_num 68) dirty exists 2024-09-18T11:30:11.847 INFO:tasks.rados.rados.0.smithi119.stdout:654: left oid 1 (ObjNum 68 snap 1 seq_num 68) 2024-09-18T11:30:11.847 INFO:tasks.rados.rados.0.smithi119.stdout:655: finishing write tid 1 to smithi11939506-33 2024-09-18T11:30:11.847 INFO:tasks.rados.rados.0.smithi119.stdout:646: done (12 left) 2024-09-18T11:30:11.848 INFO:tasks.rados.rados.0.smithi119.stdout:650: done (11 left) 2024-09-18T11:30:11.848 INFO:tasks.rados.rados.0.smithi119.stdout:651: done (10 left) 2024-09-18T11:30:11.848 INFO:tasks.rados.rados.0.smithi119.stdout:652: done (9 left) 2024-09-18T11:30:11.848 INFO:tasks.rados.rados.0.smithi119.stdout:653: done (8 left) 2024-09-18T11:30:11.848 INFO:tasks.rados.rados.0.smithi119.stdout:654: done (7 left) 2024-09-18T11:30:11.848 INFO:tasks.rados.rados.0.smithi119.stdout:662: set_chunk oid 49 target oid 2024-09-18T11:30:11.848 INFO:tasks.rados.rados.0.smithi119.stdout:662: set_chunk oid 49 offset: 0 length: 4000000 target oid smithi11939506- offset: 0 snap -1 2024-09-18T11:30:11.848 INFO:tasks.rados.rados.0.smithi119.stdout:663: read oid 20 snap -1 2024-09-18T11:30:11.848 INFO:tasks.rados.rados.0.smithi119.stdout:663: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-18T11:30:11.848 INFO:tasks.rados.rados.0.smithi119.stdout:664: snap_create 2024-09-18T11:30:11.848 INFO:tasks.rados.rados.0.smithi119.stdout:658: got (0) Success 2024-09-18T11:30:11.848 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 9 (ObjNum 62 snap 0 seq_num 62) dirty exists 2024-09-18T11:30:11.848 INFO:tasks.rados.rados.0.smithi119.stdout:655: finishing write tid 2 to smithi11939506-33 2024-09-18T11:30:11.849 INFO:tasks.rados.rados.0.smithi119.stdout:655: finishing write tid 3 to smithi11939506-33 2024-09-18T11:30:11.849 INFO:tasks.rados.rados.0.smithi119.stdout:655: finishing write tid 4 to smithi11939506-33 2024-09-18T11:30:11.849 INFO:tasks.rados.rados.0.smithi119.stdout:655: finishing write tid 5 to smithi11939506-33 2024-09-18T11:30:11.849 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 33 v 26 (ObjNum 69 snap 1 seq_num 69) dirty exists 2024-09-18T11:30:11.849 INFO:tasks.rados.rados.0.smithi119.stdout:655: left oid 33 (ObjNum 69 snap 1 seq_num 69) 2024-09-18T11:30:11.849 INFO:tasks.rados.rados.0.smithi119.stdout:659: expect (ObjNum 52 snap 0 seq_num 52) 2024-09-18T11:30:11.856 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:11.854+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:11.901 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:11.902 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:11.902 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:11.902 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:11.902 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:11.910 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:30:11.957 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:11.956+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:11.973 INFO:tasks.rados.rados.0.smithi119.stdout:656: expect (ObjNum 61 snap 0 seq_num 61) 2024-09-18T11:30:12.010 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:12.010 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:12.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:12.010 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:12.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.010 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:41 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:12.010 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157303+0000", 2024-09-18T11:30:12.011 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.850129717000002, 2024-09-18T11:30:12.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054616040000000001, 2024-09-18T11:30:12.011 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:12.011 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.011 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:12.011 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:12.011 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:12.011 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 41 2024-09-18T11:30:12.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.011 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:12.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:12.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157303+0000", 2024-09-18T11:30:12.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:12.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157303+0000", 2024-09-18T11:30:12.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:12.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157303+0000", 2024-09-18T11:30:12.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:12.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157388+0000", 2024-09-18T11:30:12.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4938999999999994e-05 2024-09-18T11:30:12.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:12.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157395+0000", 2024-09-18T11:30:12.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1130000000000003e-06 2024-09-18T11:30:12.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157405+0000", 2024-09-18T11:30:12.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7510000000000003e-06 2024-09-18T11:30:12.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:12.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158213+0000", 2024-09-18T11:30:12.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00080873000000000002 2024-09-18T11:30:12.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:12.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158267+0000", 2024-09-18T11:30:12.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3709999999999999e-05 2024-09-18T11:30:12.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:12.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158352+0000", 2024-09-18T11:30:12.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4782000000000002e-05 2024-09-18T11:30:12.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:12.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160262+0000", 2024-09-18T11:30:12.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001910068 2024-09-18T11:30:12.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162672+0000", 2024-09-18T11:30:12.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024101159999999999 2024-09-18T11:30:12.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:12.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162724+0000", 2024-09-18T11:30:12.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1637e-05 2024-09-18T11:30:12.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:12.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162765+0000", 2024-09-18T11:30:12.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1282000000000002e-05 2024-09-18T11:30:12.016 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.016 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:12.016 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.016 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:12.016 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:12.016 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.850020004000001, 2024-09-18T11:30:12.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054864400000000004, 2024-09-18T11:30:12.017 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:12.017 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.017 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:12.017 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:12.017 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:12.017 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 42 2024-09-18T11:30:12.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.017 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:12.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:12.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:12.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:12.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:12.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:12.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:12.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:12.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157416+0000", 2024-09-18T11:30:12.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.721e-06 2024-09-18T11:30:12.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:12.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157423+0000", 2024-09-18T11:30:12.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4049999999999997e-06 2024-09-18T11:30:12.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157425+0000", 2024-09-18T11:30:12.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.481e-06 2024-09-18T11:30:12.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:12.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158649+0000", 2024-09-18T11:30:12.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001223453 2024-09-18T11:30:12.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:12.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158729+0000", 2024-09-18T11:30:12.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0205999999999997e-05 2024-09-18T11:30:12.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:12.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158875+0000", 2024-09-18T11:30:12.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014628800000000001 2024-09-18T11:30:12.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:12.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161518+0000", 2024-09-18T11:30:12.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026427949999999999 2024-09-18T11:30:12.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162821+0000", 2024-09-18T11:30:12.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013028099999999999 2024-09-18T11:30:12.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:12.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162864+0000", 2024-09-18T11:30:12.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2839000000000001e-05 2024-09-18T11:30:12.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:12.022 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162900+0000", 2024-09-18T11:30:12.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5948999999999998e-05 2024-09-18T11:30:12.022 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.022 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:12.022 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.022 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:12.022 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:12.022 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.84993364, 2024-09-18T11:30:12.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055400850000000001, 2024-09-18T11:30:12.022 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:12.022 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.022 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:12.023 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:12.023 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:12.023 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 43 2024-09-18T11:30:12.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.023 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:12.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:12.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:12.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:12.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:12.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:12.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:12.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:12.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:12.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157504+0000", 2024-09-18T11:30:12.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4519999999999996e-06 2024-09-18T11:30:12.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:12.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157534+0000", 2024-09-18T11:30:12.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.97e-05 2024-09-18T11:30:12.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157539+0000", 2024-09-18T11:30:12.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4360000000000001e-06 2024-09-18T11:30:12.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:12.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159428+0000", 2024-09-18T11:30:12.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018888220000000001 2024-09-18T11:30:12.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:12.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159479+0000", 2024-09-18T11:30:12.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0608000000000001e-05 2024-09-18T11:30:12.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:12.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159559+0000", 2024-09-18T11:30:12.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0667000000000005e-05 2024-09-18T11:30:12.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.027 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:12.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162779+0000", 2024-09-18T11:30:12.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032197459999999999 2024-09-18T11:30:12.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.027 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162950+0000", 2024-09-18T11:30:12.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017059599999999999 2024-09-18T11:30:12.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.027 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:12.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163001+0000", 2024-09-18T11:30:12.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0968999999999999e-05 2024-09-18T11:30:12.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:12.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163040+0000", 2024-09-18T11:30:12.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8843000000000001e-05 2024-09-18T11:30:12.028 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.028 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:12.028 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.028 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:12.028 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:12.029 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.849882485999998, 2024-09-18T11:30:12.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:30:12.029 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:12.029 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.029 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:12.029 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:12.029 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:12.029 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:30:12.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.029 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:12.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:12.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:12.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:12.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:12.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:12.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:30:12.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:12.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:30:12.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:30:12.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:12.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:30:12.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:30:12.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:30:12.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:30:12.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.032 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:12.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:30:12.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:30:12.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.032 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:12.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:30:12.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:30:12.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:12.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:30:12.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:30:12.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:12.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:30:12.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:30:12.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:30:12.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:30:12.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:12.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:30:12.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:30:12.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:12.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:30:12.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:30:12.034 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.034 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:12.034 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.035 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:12.035 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:12.035 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.849756224, 2024-09-18T11:30:12.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055717040000000002, 2024-09-18T11:30:12.035 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:12.035 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.035 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:12.035 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:12.035 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:12.035 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 45 2024-09-18T11:30:12.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.035 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:12.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.036 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:12.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:12.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.036 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:12.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:12.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.036 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:12.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157676+0000", 2024-09-18T11:30:12.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:12.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157694+0000", 2024-09-18T11:30:12.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7422999999999999e-05 2024-09-18T11:30:12.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:12.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157704+0000", 2024-09-18T11:30:12.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9000000000000001e-06 2024-09-18T11:30:12.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157718+0000", 2024-09-18T11:30:12.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4788e-05 2024-09-18T11:30:12.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:12.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160937+0000", 2024-09-18T11:30:12.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00321886 2024-09-18T11:30:12.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:12.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161023+0000", 2024-09-18T11:30:12.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6063999999999995e-05 2024-09-18T11:30:12.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:12.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161202+0000", 2024-09-18T11:30:12.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017912599999999999 2024-09-18T11:30:12.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:12.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163052+0000", 2024-09-18T11:30:12.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001849554 2024-09-18T11:30:12.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163204+0000", 2024-09-18T11:30:12.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015186900000000001 2024-09-18T11:30:12.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:12.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163229+0000", 2024-09-18T11:30:12.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:30:12.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:12.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163249+0000", 2024-09-18T11:30:12.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9992000000000001e-05 2024-09-18T11:30:12.040 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.040 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:12.040 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.040 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:12.040 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:12.040 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.849707111000001, 2024-09-18T11:30:12.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:30:12.041 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:12.041 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.041 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:12.041 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:12.041 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:12.041 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:30:12.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.041 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:12.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:12.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:12.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:12.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:12.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:12.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:12.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:12.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:30:12.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:30:12.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.043 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:12.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:30:12.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:30:12.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.043 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:30:12.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:30:12.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.043 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:12.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:30:12.044 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:30:12.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.044 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:12.044 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:30:12.044 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:30:12.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.044 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:12.044 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:30:12.044 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:30:12.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.045 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:12.045 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:30:12.045 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:30:12.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.045 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.045 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:30:12.045 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:30:12.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.046 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:12.046 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:30:12.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:30:12.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.046 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:12.046 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:30:12.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:30:12.046 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.046 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:12.047 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.047 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:12.047 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:12.047 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.849664579999999, 2024-09-18T11:30:12.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:30:12.047 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:12.047 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.047 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:12.047 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:12.047 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:12.047 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:30:12.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.048 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:12.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.048 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:12.048 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:12.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.048 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:12.048 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:12.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.049 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:12.049 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:30:12.049 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.050 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:12.050 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:30:12.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:30:12.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.050 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:12.050 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:30:12.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:30:12.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.050 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.050 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:30:12.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:30:12.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.051 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:12.051 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:30:12.051 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:30:12.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.051 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:12.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:30:12.052 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:30:12.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.052 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:12.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:30:12.052 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:30:12.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.052 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:30:12.052 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:30:12.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.053 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:12.053 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:30:12.053 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:30:12.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.053 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:12.053 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:30:12.053 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:30:12.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.053 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:12.053 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:30:12.053 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:30:12.054 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.054 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:12.054 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.054 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:12.054 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:12.054 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.849609650000001, 2024-09-18T11:30:12.054 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:30:12.054 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:12.054 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.054 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:12.054 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:12.055 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:12.055 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:30:12.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.055 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:12.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:12.055 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:12.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:12.055 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:12.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.056 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:12.056 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:12.056 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.056 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:12.056 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:30:12.056 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:30:12.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.056 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:12.056 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:30:12.057 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:30:12.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.057 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.057 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:30:12.057 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:30:12.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.057 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:12.057 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:30:12.057 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:30:12.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.057 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:12.057 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:30:12.058 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:30:12.058 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.058 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:12.058 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:30:12.058 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:30:12.058 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.058 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:12.058 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:30:12.058 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:30:12.058 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.059 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.059 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:30:12.059 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:30:12.059 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.059 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.059 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:12.059 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:30:12.059 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:30:12.059 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.059 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.059 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:12.059 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:30:12.059 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:30:12.059 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.060 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:12.060 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.060 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.060 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.060 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:12.060 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:12.060 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.84959967, 2024-09-18T11:30:12.060 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:30:12.060 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:12.060 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.060 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:12.060 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:12.060 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:12.060 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:30:12.061 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.061 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:12.061 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.061 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:12.061 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:12.061 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.061 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.061 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.061 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:12.061 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:12.061 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.061 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.061 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.061 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:30:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:30:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:30:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:30:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:30:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:30:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:30:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:30:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:30:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:30:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:30:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:30:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:30:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:30:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:30:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:30:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:30:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:30:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:30:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:30:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.849588070999999, 2024-09-18T11:30:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:30:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:30:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.068 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:12.068 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:30:12.068 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:30:12.068 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.068 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.068 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:12.068 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:30:12.068 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:30:12.068 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.068 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.068 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.068 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:30:12.068 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:30:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:30:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:30:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:30:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:30:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:30:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:30:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:30:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:30:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:30:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:30:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:30:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:30:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:30:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:30:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.072 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:12.072 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:12.072 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.849531590000002, 2024-09-18T11:30:12.072 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:30:12.072 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:12.072 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.072 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:12.072 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:12.072 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:12.072 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:30:12.072 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.072 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:12.072 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.072 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:30:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:30:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:30:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:30:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:30:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:30:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:30:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:30:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:30:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:30:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:30:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:30:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:30:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:30:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:30:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:30:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:30:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:30:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:30:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:30:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:30:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.849517765999998, 2024-09-18T11:30:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:30:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:30:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:30:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:30:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:30:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:30:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:30:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:30:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:30:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:30:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:30:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:30:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:30:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:30:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:30:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:30:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:30:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:30:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:30:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:30:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:30:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:30:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.433338858999999, 2024-09-18T11:30:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:30:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:30:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:30:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:30:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:30:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:30:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:30:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:30:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:30:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:30:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:30:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:30:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:30:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:30:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:30:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:30:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:30:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:30:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:30:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:30:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:30:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:30:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.433315559, 2024-09-18T11:30:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:30:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:30:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:30:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:30:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:30:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:30:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:30:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:30:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:30:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:30:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:30:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:30:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:30:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:30:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:30:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:30:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:30:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:30:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:30:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:30:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:30:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:30:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:30:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.433229950000001, 2024-09-18T11:30:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:30:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:30:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:30:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:30:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.098 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:12.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:30:12.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:30:12.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.098 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:30:12.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:30:12.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.098 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:12.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:30:12.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:30:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:30:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:30:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:30:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:30:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:12.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:30:12.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:30:12.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:30:12.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:30:12.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:12.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:30:12.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:30:12.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:12.101 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:30:12.101 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:30:12.101 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.101 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:12.101 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.101 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:12.101 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:12.101 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.433218985, 2024-09-18T11:30:12.101 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:30:12.101 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:12.101 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:30:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:30:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:30:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:30:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:30:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:30:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:30:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:30:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:30:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:12.105 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:30:12.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:30:12.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.105 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:12.105 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:30:12.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:30:12.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.105 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:12.105 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:30:12.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:30:12.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:30:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:30:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:30:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:30:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:30:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:30:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.433207817, 2024-09-18T11:30:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:30:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:30:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.109 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:12.109 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:12.109 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:12.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.109 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:12.109 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:30:12.109 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:30:12.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.109 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:12.109 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:30:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:30:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:30:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:30:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:30:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:30:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:30:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:30:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:30:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:30:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:30:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:30:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.112 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:12.112 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:30:12.112 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:30:12.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.112 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:12.112 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:30:12.112 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:30:12.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.112 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:12.112 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:30:12.112 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:30:12.113 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.113 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:12.113 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.113 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:12.113 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:12.113 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.433142488000001, 2024-09-18T11:30:12.113 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:30:12.113 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:12.113 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.113 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:12.113 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:12.113 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:12.114 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:30:12.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.114 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:12.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.114 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:12.114 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:12.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.114 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:12.114 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:12.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:30:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:30:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:30:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:30:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:30:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:30:12.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:30:12.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:12.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:30:12.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:30:12.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:12.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:30:12.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:30:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:30:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:30:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:30:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:30:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:30:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:30:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:30:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:30:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:30:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:30:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.422980509999999, 2024-09-18T11:30:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:30:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:30:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:30:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:30:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:30:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:30:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:30:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:30:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:30:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:30:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:30:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:30:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:30:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:30:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:30:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:30:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:30:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:30:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:30:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:30:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:30:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:30:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:30:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.422888030999999, 2024-09-18T11:30:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:30:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:30:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:30:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:30:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:30:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:30:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:30:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:30:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:30:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:30:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:30:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:30:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:30:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:30:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:30:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:30:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:30:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:30:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:30:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:30:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:30:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:30:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:12.130 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:12.131 INFO:tasks.rados.rados.0.smithi119.stdout:657: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-18T11:30:12.132 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is set 2024-09-18T11:30:12.132 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:12.129+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:12.132 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:30:12.196 INFO:tasks.rados.rados.0.smithi119.stdout:660: got (0) Success 2024-09-18T11:30:12.196 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 6 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-18T11:30:12.196 INFO:tasks.rados.rados.0.smithi119.stdout:661: got (22) Invalid argument 2024-09-18T11:30:12.196 INFO:tasks.rados.rados.0.smithi119.stdout:662: finishing set_chunk to oid 49 2024-09-18T11:30:12.196 INFO:tasks.rados.rados.0.smithi119.stdout:662: got expected ENOENT (src dne) 2024-09-18T11:30:12.196 INFO:tasks.rados.rados.0.smithi119.stdout:663: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-18T11:30:12.233 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:12.231+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:12.234 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:12.234 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:12.234 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:12.234 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:12.243 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:30:12.278 INFO:tasks.rados.rados.0.smithi119.stdout:655: done (9 left) 2024-09-18T11:30:12.278 INFO:tasks.rados.rados.0.smithi119.stdout:656: done (8 left) 2024-09-18T11:30:12.278 INFO:tasks.rados.rados.0.smithi119.stdout:657: done (7 left) 2024-09-18T11:30:12.278 INFO:tasks.rados.rados.0.smithi119.stdout:658: done (6 left) 2024-09-18T11:30:12.278 INFO:tasks.rados.rados.0.smithi119.stdout:659: done (5 left) 2024-09-18T11:30:12.278 INFO:tasks.rados.rados.0.smithi119.stdout:660: done (4 left) 2024-09-18T11:30:12.278 INFO:tasks.rados.rados.0.smithi119.stdout:661: done (3 left) 2024-09-18T11:30:12.278 INFO:tasks.rados.rados.0.smithi119.stdout:662: done (2 left) 2024-09-18T11:30:12.279 INFO:tasks.rados.rados.0.smithi119.stdout:663: done (1 left) 2024-09-18T11:30:12.279 INFO:tasks.rados.rados.0.smithi119.stdout:664: done (0 left) 2024-09-18T11:30:12.279 INFO:tasks.rados.rados.0.smithi119.stdout:665: read oid 35 snap -1 2024-09-18T11:30:12.279 INFO:tasks.rados.rados.0.smithi119.stdout:665: expect (ObjNum 34 snap 0 seq_num 34) 2024-09-18T11:30:12.279 INFO:tasks.rados.rados.0.smithi119.stdout:666: snap_remove snap 1 2024-09-18T11:30:12.333 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:12.332+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:12.342 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:12.342 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:12.343 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:12.343 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:12.343 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:12.351 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:30:12.435 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:12.433+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:12.437 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:30:12.493 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:12.493 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:12.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:12.493 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:12.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.494 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:41 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'19)", 2024-09-18T11:30:12.494 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.158446+0000", 2024-09-18T11:30:12.494 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.332650278999999, 2024-09-18T11:30:12.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028837609999999999, 2024-09-18T11:30:12.494 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:12.494 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.494 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:12.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:12.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158446+0000", 2024-09-18T11:30:12.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:12.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158446+0000", 2024-09-18T11:30:12.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:12.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158446+0000", 2024-09-18T11:30:12.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:12.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158513+0000", 2024-09-18T11:30:12.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7379999999999995e-05 2024-09-18T11:30:12.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:12.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158520+0000", 2024-09-18T11:30:12.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0149999999999998e-06 2024-09-18T11:30:12.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158524+0000", 2024-09-18T11:30:12.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4400000000000001e-06 2024-09-18T11:30:12.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158551+0000", 2024-09-18T11:30:12.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7331e-05 2024-09-18T11:30:12.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:12.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158562+0000", 2024-09-18T11:30:12.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0951999999999999e-05 2024-09-18T11:30:12.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:12.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.161297+0000", 2024-09-18T11:30:12.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027348810000000002 2024-09-18T11:30:12.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:12.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.161330+0000", 2024-09-18T11:30:12.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3104e-05 2024-09-18T11:30:12.497 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.498 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:12.498 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.498 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:42 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'24, mlcod=15'19)", 2024-09-18T11:30:12.498 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.158950+0000", 2024-09-18T11:30:12.498 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.332146606999999, 2024-09-18T11:30:12.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026570640000000002, 2024-09-18T11:30:12.498 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:12.498 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.498 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:12.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:12.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158950+0000", 2024-09-18T11:30:12.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:12.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158950+0000", 2024-09-18T11:30:12.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:12.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158950+0000", 2024-09-18T11:30:12.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:12.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158953+0000", 2024-09-18T11:30:12.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1370000000000002e-06 2024-09-18T11:30:12.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:12.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158960+0000", 2024-09-18T11:30:12.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4630000000000004e-06 2024-09-18T11:30:12.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158964+0000", 2024-09-18T11:30:12.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0960000000000003e-06 2024-09-18T11:30:12.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158990+0000", 2024-09-18T11:30:12.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5332000000000001e-05 2024-09-18T11:30:12.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:12.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.159002+0000", 2024-09-18T11:30:12.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2242000000000001e-05 2024-09-18T11:30:12.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:12.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.161583+0000", 2024-09-18T11:30:12.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002580695 2024-09-18T11:30:12.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:12.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.161607+0000", 2024-09-18T11:30:12.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4474e-05 2024-09-18T11:30:12.502 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.502 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:12.502 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.502 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:77 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'55, mlcod=15'53)", 2024-09-18T11:30:12.502 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.584563+0000", 2024-09-18T11:30:12.502 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.906533748000001, 2024-09-18T11:30:12.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037297200000000002, 2024-09-18T11:30:12.502 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:12.503 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.503 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:12.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:12.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584563+0000", 2024-09-18T11:30:12.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:12.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584563+0000", 2024-09-18T11:30:12.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:12.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584562+0000", 2024-09-18T11:30:12.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:12.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584613+0000", 2024-09-18T11:30:12.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0892e-05 2024-09-18T11:30:12.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:12.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584648+0000", 2024-09-18T11:30:12.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4533999999999998e-05 2024-09-18T11:30:12.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584653+0000", 2024-09-18T11:30:12.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9749999999999998e-06 2024-09-18T11:30:12.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584687+0000", 2024-09-18T11:30:12.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4453999999999999e-05 2024-09-18T11:30:12.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:12.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584699+0000", 2024-09-18T11:30:12.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1605e-05 2024-09-18T11:30:12.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:12.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.588269+0000", 2024-09-18T11:30:12.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035706650000000002 2024-09-18T11:30:12.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:12.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.588293+0000", 2024-09-18T11:30:12.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3226000000000001e-05 2024-09-18T11:30:12.506 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.506 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:12.506 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.507 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:78 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'56, mlcod=15'53)", 2024-09-18T11:30:12.507 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.585042+0000", 2024-09-18T11:30:12.507 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.906054677, 2024-09-18T11:30:12.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032615389999999999, 2024-09-18T11:30:12.507 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:12.507 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.507 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:12.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:12.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585042+0000", 2024-09-18T11:30:12.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:12.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585042+0000", 2024-09-18T11:30:12.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:12.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585041+0000", 2024-09-18T11:30:12.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:12.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585129+0000", 2024-09-18T11:30:12.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7905000000000006e-05 2024-09-18T11:30:12.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:12.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585138+0000", 2024-09-18T11:30:12.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9069999999999996e-06 2024-09-18T11:30:12.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585143+0000", 2024-09-18T11:30:12.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5329999999999996e-06 2024-09-18T11:30:12.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585166+0000", 2024-09-18T11:30:12.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3308000000000001e-05 2024-09-18T11:30:12.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:12.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585177+0000", 2024-09-18T11:30:12.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1255e-05 2024-09-18T11:30:12.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:12.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.588294+0000", 2024-09-18T11:30:12.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031165989999999998 2024-09-18T11:30:12.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:12.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.588303+0000", 2024-09-18T11:30:12.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5279999999999992e-06 2024-09-18T11:30:12.510 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.511 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:12.511 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.511 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:79 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'57, mlcod=15'53)", 2024-09-18T11:30:12.511 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.586176+0000", 2024-09-18T11:30:12.511 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.904920368999999, 2024-09-18T11:30:12.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043147059999999997, 2024-09-18T11:30:12.511 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:12.511 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.511 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:12.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:12.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586176+0000", 2024-09-18T11:30:12.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:12.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586176+0000", 2024-09-18T11:30:12.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:12.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586170+0000", 2024-09-18T11:30:12.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999938 2024-09-18T11:30:12.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:12.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586282+0000", 2024-09-18T11:30:12.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112254 2024-09-18T11:30:12.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:12.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586307+0000", 2024-09-18T11:30:12.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5296e-05 2024-09-18T11:30:12.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586321+0000", 2024-09-18T11:30:12.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3723000000000001e-05 2024-09-18T11:30:12.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.587754+0000", 2024-09-18T11:30:12.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014330829999999999 2024-09-18T11:30:12.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:12.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.587785+0000", 2024-09-18T11:30:12.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1176000000000001e-05 2024-09-18T11:30:12.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:12.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.590467+0000", 2024-09-18T11:30:12.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026815089999999999 2024-09-18T11:30:12.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:12.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.590491+0000", 2024-09-18T11:30:12.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3941e-05 2024-09-18T11:30:12.515 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.515 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:12.516 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.516 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:80 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'58, mlcod=15'53)", 2024-09-18T11:30:12.516 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.586532+0000", 2024-09-18T11:30:12.516 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.904564652000001, 2024-09-18T11:30:12.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041534989999999997, 2024-09-18T11:30:12.516 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:12.516 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.516 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:12.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:12.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586532+0000", 2024-09-18T11:30:12.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:12.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586532+0000", 2024-09-18T11:30:12.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:12.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586531+0000", 2024-09-18T11:30:12.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:12.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586591+0000", 2024-09-18T11:30:12.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9472000000000003e-05 2024-09-18T11:30:12.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:12.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586598+0000", 2024-09-18T11:30:12.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0890000000000002e-06 2024-09-18T11:30:12.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586602+0000", 2024-09-18T11:30:12.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0090000000000001e-06 2024-09-18T11:30:12.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.588075+0000", 2024-09-18T11:30:12.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014728429999999999 2024-09-18T11:30:12.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:12.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.588086+0000", 2024-09-18T11:30:12.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1452e-05 2024-09-18T11:30:12.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:12.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.590672+0000", 2024-09-18T11:30:12.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025858629999999999 2024-09-18T11:30:12.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:12.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.590685+0000", 2024-09-18T11:30:12.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3254999999999999e-05 2024-09-18T11:30:12.520 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.520 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:12.520 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.521 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:88 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'62, mlcod=15'60)", 2024-09-18T11:30:12.521 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:50.031675+0000", 2024-09-18T11:30:12.521 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.459421088999999, 2024-09-18T11:30:12.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035383889999999999, 2024-09-18T11:30:12.521 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:12.521 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.521 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:12.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:12.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031675+0000", 2024-09-18T11:30:12.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:12.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031675+0000", 2024-09-18T11:30:12.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:12.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031675+0000", 2024-09-18T11:30:12.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:12.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031682+0000", 2024-09-18T11:30:12.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.542e-06 2024-09-18T11:30:12.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:12.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031692+0000", 2024-09-18T11:30:12.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7119999999999995e-06 2024-09-18T11:30:12.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031700+0000", 2024-09-18T11:30:12.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7819999999999994e-06 2024-09-18T11:30:12.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031738+0000", 2024-09-18T11:30:12.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8538000000000003e-05 2024-09-18T11:30:12.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:12.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031757+0000", 2024-09-18T11:30:12.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8258999999999999e-05 2024-09-18T11:30:12.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:12.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035185+0000", 2024-09-18T11:30:12.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034287219999999999 2024-09-18T11:30:12.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:12.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035214+0000", 2024-09-18T11:30:12.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8524e-05 2024-09-18T11:30:12.525 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.525 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:12.525 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.525 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:89 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'63, mlcod=15'60)", 2024-09-18T11:30:12.525 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:50.032044+0000", 2024-09-18T11:30:12.525 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.459052355000001, 2024-09-18T11:30:12.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032268829999999998, 2024-09-18T11:30:12.525 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:12.526 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.526 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:12.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:12.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032044+0000", 2024-09-18T11:30:12.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:12.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032044+0000", 2024-09-18T11:30:12.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:12.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032044+0000", 2024-09-18T11:30:12.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:12.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032049+0000", 2024-09-18T11:30:12.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.78e-06 2024-09-18T11:30:12.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:12.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032057+0000", 2024-09-18T11:30:12.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5879999999999998e-06 2024-09-18T11:30:12.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032061+0000", 2024-09-18T11:30:12.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0820000000000001e-06 2024-09-18T11:30:12.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032083+0000", 2024-09-18T11:30:12.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1359999999999999e-05 2024-09-18T11:30:12.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:12.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032093+0000", 2024-09-18T11:30:12.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0662999999999999e-05 2024-09-18T11:30:12.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:12.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035250+0000", 2024-09-18T11:30:12.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031568429999999999 2024-09-18T11:30:12.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:12.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035271+0000", 2024-09-18T11:30:12.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1053999999999999e-05 2024-09-18T11:30:12.529 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.529 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:12.530 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.530 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:90 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'64, mlcod=15'60)", 2024-09-18T11:30:12.530 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:50.032410+0000", 2024-09-18T11:30:12.530 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.458686704000002, 2024-09-18T11:30:12.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029831089999999998, 2024-09-18T11:30:12.530 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:12.530 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.530 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:12.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:12.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032410+0000", 2024-09-18T11:30:12.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:12.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032410+0000", 2024-09-18T11:30:12.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:12.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032410+0000", 2024-09-18T11:30:12.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:12.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032534+0000", 2024-09-18T11:30:12.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012397600000000001 2024-09-18T11:30:12.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:12.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032540+0000", 2024-09-18T11:30:12.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1500000000000004e-06 2024-09-18T11:30:12.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032544+0000", 2024-09-18T11:30:12.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2910000000000001e-06 2024-09-18T11:30:12.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032576+0000", 2024-09-18T11:30:12.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2228999999999999e-05 2024-09-18T11:30:12.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:12.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032597+0000", 2024-09-18T11:30:12.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0579000000000001e-05 2024-09-18T11:30:12.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:12.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035344+0000", 2024-09-18T11:30:12.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002747527 2024-09-18T11:30:12.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:12.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035393+0000", 2024-09-18T11:30:12.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8677999999999997e-05 2024-09-18T11:30:12.534 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.534 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:12.534 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.534 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:91 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'65, mlcod=15'60)", 2024-09-18T11:30:12.534 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:50.032758+0000", 2024-09-18T11:30:12.534 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.458338987000001, 2024-09-18T11:30:12.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026551370000000001, 2024-09-18T11:30:12.534 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:12.535 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.535 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:12.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:12.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032758+0000", 2024-09-18T11:30:12.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:12.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032758+0000", 2024-09-18T11:30:12.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:12.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032757+0000", 2024-09-18T11:30:12.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:12.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032761+0000", 2024-09-18T11:30:12.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9570000000000004e-06 2024-09-18T11:30:12.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:12.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032766+0000", 2024-09-18T11:30:12.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7770000000000002e-06 2024-09-18T11:30:12.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032768+0000", 2024-09-18T11:30:12.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4849999999999999e-06 2024-09-18T11:30:12.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032880+0000", 2024-09-18T11:30:12.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000111789 2024-09-18T11:30:12.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:12.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032897+0000", 2024-09-18T11:30:12.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6467999999999999e-05 2024-09-18T11:30:12.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:12.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035395+0000", 2024-09-18T11:30:12.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024979310000000001 2024-09-18T11:30:12.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:12.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035413+0000", 2024-09-18T11:30:12.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8111e-05 2024-09-18T11:30:12.538 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.538 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:12.538 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.539 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:21 3.1 3:9f1cb8cf:::smithi11939506-38 low_tier:head [read 336384~173568 [fadvise_sequential]] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:12.539 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.118534+0000", 2024-09-18T11:30:12.539 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.3725621779999999, 2024-09-18T11:30:12.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006633842, 2024-09-18T11:30:12.539 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:12.539 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:12.539 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:12.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:12.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118534+0000", 2024-09-18T11:30:12.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:12.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118534+0000", 2024-09-18T11:30:12.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:12.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118532+0000", 2024-09-18T11:30:12.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:12.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:12.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118542+0000", 2024-09-18T11:30:12.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0151e-05 2024-09-18T11:30:12.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:12.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118563+0000", 2024-09-18T11:30:12.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1089000000000001e-05 2024-09-18T11:30:12.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118572+0000", 2024-09-18T11:30:12.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1090000000000004e-06 2024-09-18T11:30:12.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118633+0000", 2024-09-18T11:30:12.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0940000000000003e-05 2024-09-18T11:30:12.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:12.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118640+0000", 2024-09-18T11:30:12.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3690000000000003e-06 2024-09-18T11:30:12.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124910+0000", 2024-09-18T11:30:12.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062697569999999999 2024-09-18T11:30:12.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:12.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124980+0000", 2024-09-18T11:30:12.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0846000000000006e-05 2024-09-18T11:30:12.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:12.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.125168+0000", 2024-09-18T11:30:12.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018786200000000001 2024-09-18T11:30:12.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.543 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:12.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.543 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:22 3.1 3:9f1cb8cf:::smithi11939506-38 low_tier:head [read 1235456~1340416 [fadvise_sequential]] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:12.544 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:12.544 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.3725094499999999, 2024-09-18T11:30:12.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0090773199999999998, 2024-09-18T11:30:12.544 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:12.544 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:12.544 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:12.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:12.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:12.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:12.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:12.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:12.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:12.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:12.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118594+0000", 2024-09-18T11:30:12.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1590000000000004e-06 2024-09-18T11:30:12.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:12.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118601+0000", 2024-09-18T11:30:12.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3900000000000004e-06 2024-09-18T11:30:12.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118605+0000", 2024-09-18T11:30:12.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2479999999999998e-06 2024-09-18T11:30:12.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118697+0000", 2024-09-18T11:30:12.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1447999999999996e-05 2024-09-18T11:30:12.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:12.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118702+0000", 2024-09-18T11:30:12.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6670000000000001e-06 2024-09-18T11:30:12.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.125285+0000", 2024-09-18T11:30:12.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006582124 2024-09-18T11:30:12.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:12.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.125349+0000", 2024-09-18T11:30:12.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4777000000000007e-05 2024-09-18T11:30:12.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:12.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.127664+0000", 2024-09-18T11:30:12.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023151069999999998 2024-09-18T11:30:12.548 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.548 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:12.548 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.548 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:910 4.3 4:cc46532a:::smithi11939506-33:head [write 759683~639358,stat] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:12.549 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.159388+0000", 2024-09-18T11:30:12.549 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.3317090659999999, 2024-09-18T11:30:12.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003341344, 2024-09-18T11:30:12.549 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:12.549 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.549 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:12.549 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:12.549 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:12.549 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 910 2024-09-18T11:30:12.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.549 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:12.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:12.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.159388+0000", 2024-09-18T11:30:12.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:12.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.159388+0000", 2024-09-18T11:30:12.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:12.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.159385+0000", 2024-09-18T11:30:12.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:12.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:12.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.159676+0000", 2024-09-18T11:30:12.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029050000000000001 2024-09-18T11:30:12.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:12.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.159693+0000", 2024-09-18T11:30:12.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7450999999999999e-05 2024-09-18T11:30:12.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.159699+0000", 2024-09-18T11:30:12.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8749999999999997e-06 2024-09-18T11:30:12.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.159742+0000", 2024-09-18T11:30:12.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2855000000000002e-05 2024-09-18T11:30:12.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:12.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.159796+0000", 2024-09-18T11:30:12.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4517e-05 2024-09-18T11:30:12.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:12.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.159910+0000", 2024-09-18T11:30:12.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011401899999999999 2024-09-18T11:30:12.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:12.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.162152+0000", 2024-09-18T11:30:12.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022414230000000002 2024-09-18T11:30:12.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.162674+0000", 2024-09-18T11:30:12.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00052174100000000005 2024-09-18T11:30:12.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:12.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.162704+0000", 2024-09-18T11:30:12.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0555999999999997e-05 2024-09-18T11:30:12.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:12.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.162729+0000", 2024-09-18T11:30:12.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4672999999999999e-05 2024-09-18T11:30:12.554 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.554 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:12.554 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.554 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:911 4.3 4:cc46532a:::smithi11939506-33:head [write 1904434~719345] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:12.554 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.166081+0000", 2024-09-18T11:30:12.554 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.3250153840000001, 2024-09-18T11:30:12.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081214100000000008, 2024-09-18T11:30:12.555 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:12.555 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.555 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:12.555 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:12.555 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:12.555 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 911 2024-09-18T11:30:12.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.555 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:12.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:12.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166081+0000", 2024-09-18T11:30:12.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:12.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166081+0000", 2024-09-18T11:30:12.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:12.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166080+0000", 2024-09-18T11:30:12.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:12.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:12.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166386+0000", 2024-09-18T11:30:12.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030589499999999998 2024-09-18T11:30:12.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:12.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166398+0000", 2024-09-18T11:30:12.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1678000000000001e-05 2024-09-18T11:30:12.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166402+0000", 2024-09-18T11:30:12.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6059999999999996e-06 2024-09-18T11:30:12.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166443+0000", 2024-09-18T11:30:12.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1044000000000002e-05 2024-09-18T11:30:12.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:12.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166526+0000", 2024-09-18T11:30:12.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2281999999999996e-05 2024-09-18T11:30:12.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:12.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166661+0000", 2024-09-18T11:30:12.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013516299999999999 2024-09-18T11:30:12.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:12.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.168437+0000", 2024-09-18T11:30:12.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001775794 2024-09-18T11:30:12.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174126+0000", 2024-09-18T11:30:12.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056894859999999997 2024-09-18T11:30:12.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:12.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174166+0000", 2024-09-18T11:30:12.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9787999999999999e-05 2024-09-18T11:30:12.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:12.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174203+0000", 2024-09-18T11:30:12.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6644999999999999e-05 2024-09-18T11:30:12.560 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.560 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:12.560 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.560 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:912 4.3 4:cc46532a:::smithi11939506-33:head [write 3128657~31026] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:12.560 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169139+0000", 2024-09-18T11:30:12.560 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.32195771, 2024-09-18T11:30:12.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051669280000000003, 2024-09-18T11:30:12.561 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:12.561 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.561 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:12.561 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:12.561 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:12.561 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 912 2024-09-18T11:30:12.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.561 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:12.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:12.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169139+0000", 2024-09-18T11:30:12.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:12.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169139+0000", 2024-09-18T11:30:12.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:12.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169138+0000", 2024-09-18T11:30:12.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:12.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:12.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169176+0000", 2024-09-18T11:30:12.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7679000000000003e-05 2024-09-18T11:30:12.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:12.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169187+0000", 2024-09-18T11:30:12.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1353e-05 2024-09-18T11:30:12.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169193+0000", 2024-09-18T11:30:12.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0390000000000001e-06 2024-09-18T11:30:12.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169230+0000", 2024-09-18T11:30:12.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6397999999999999e-05 2024-09-18T11:30:12.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:12.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169302+0000", 2024-09-18T11:30:12.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2922000000000004e-05 2024-09-18T11:30:12.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:12.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169426+0000", 2024-09-18T11:30:12.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012302600000000001 2024-09-18T11:30:12.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:12.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174078+0000", 2024-09-18T11:30:12.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046524330000000001 2024-09-18T11:30:12.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174240+0000", 2024-09-18T11:30:12.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016188899999999999 2024-09-18T11:30:12.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:12.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174275+0000", 2024-09-18T11:30:12.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5556000000000003e-05 2024-09-18T11:30:12.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:12.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174306+0000", 2024-09-18T11:30:12.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0408000000000001e-05 2024-09-18T11:30:12.566 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.566 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:12.566 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.566 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:913 4.3 4:cc46532a:::smithi11939506-33:head [setxattr _header (56) in=7b,truncate 3159683] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:12.566 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169203+0000", 2024-09-18T11:30:12.566 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.321893945, 2024-09-18T11:30:12.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052201790000000001, 2024-09-18T11:30:12.567 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:12.567 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:12.567 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:12.567 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:12.567 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:12.567 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 913 2024-09-18T11:30:12.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.567 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:12.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:12.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169203+0000", 2024-09-18T11:30:12.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:12.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169203+0000", 2024-09-18T11:30:12.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:12.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169202+0000", 2024-09-18T11:30:12.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:12.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169206+0000", 2024-09-18T11:30:12.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8160000000000004e-06 2024-09-18T11:30:12.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:12.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169212+0000", 2024-09-18T11:30:12.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3770000000000001e-06 2024-09-18T11:30:12.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169216+0000", 2024-09-18T11:30:12.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0060000000000003e-06 2024-09-18T11:30:12.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169888+0000", 2024-09-18T11:30:12.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00067139700000000001 2024-09-18T11:30:12.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:12.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169963+0000", 2024-09-18T11:30:12.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4999000000000005e-05 2024-09-18T11:30:12.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:12.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.170077+0000", 2024-09-18T11:30:12.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011401899999999999 2024-09-18T11:30:12.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:12.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174084+0000", 2024-09-18T11:30:12.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040071550000000001 2024-09-18T11:30:12.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:12.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174336+0000", 2024-09-18T11:30:12.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00025225300000000003 2024-09-18T11:30:12.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:12.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174371+0000", 2024-09-18T11:30:12.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5361000000000002e-05 2024-09-18T11:30:12.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:12.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174423+0000", 2024-09-18T11:30:12.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1347999999999997e-05 2024-09-18T11:30:12.572 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.572 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:12.572 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.572 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:914 4.3 4:cc46532a:::smithi11939506-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:12.572 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169236+0000", 2024-09-18T11:30:12.572 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.3218604250000001, 2024-09-18T11:30:12.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005369965, 2024-09-18T11:30:12.572 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:12.573 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:12.573 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:12.573 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:12.573 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:12.573 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 914 2024-09-18T11:30:12.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.573 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:12.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:12.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169236+0000", 2024-09-18T11:30:12.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:12.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169236+0000", 2024-09-18T11:30:12.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:12.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169236+0000", 2024-09-18T11:30:12.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:12.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169241+0000", 2024-09-18T11:30:12.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8239999999999996e-06 2024-09-18T11:30:12.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:12.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169249+0000", 2024-09-18T11:30:12.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9319999999999996e-06 2024-09-18T11:30:12.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169254+0000", 2024-09-18T11:30:12.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3220000000000003e-06 2024-09-18T11:30:12.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.170304+0000", 2024-09-18T11:30:12.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001050184 2024-09-18T11:30:12.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:12.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.170390+0000", 2024-09-18T11:30:12.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5839000000000003e-05 2024-09-18T11:30:12.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174455+0000", 2024-09-18T11:30:12.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040656119999999997 2024-09-18T11:30:12.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:12.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174525+0000", 2024-09-18T11:30:12.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9151000000000003e-05 2024-09-18T11:30:12.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:12.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174606+0000", 2024-09-18T11:30:12.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1588000000000006e-05 2024-09-18T11:30:12.577 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.577 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:12.577 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.577 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:922 4.b 4:db5b7794:::smithi11939506-13:head [read 0~2873964,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:12.577 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169355+0000", 2024-09-18T11:30:12.578 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.321741879, 2024-09-18T11:30:12.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032492969999999999, 2024-09-18T11:30:12.578 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:12.578 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:12.578 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:12.578 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:12.578 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:12.578 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 922 2024-09-18T11:30:12.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.578 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:12.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:12.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169355+0000", 2024-09-18T11:30:12.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:12.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169355+0000", 2024-09-18T11:30:12.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:12.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169354+0000", 2024-09-18T11:30:12.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:12.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:12.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169361+0000", 2024-09-18T11:30:12.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.328e-06 2024-09-18T11:30:12.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:12.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169371+0000", 2024-09-18T11:30:12.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3870000000000001e-06 2024-09-18T11:30:12.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169375+0000", 2024-09-18T11:30:12.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6229999999999996e-06 2024-09-18T11:30:12.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.170253+0000", 2024-09-18T11:30:12.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00087807 2024-09-18T11:30:12.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:12.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.170327+0000", 2024-09-18T11:30:12.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3423999999999994e-05 2024-09-18T11:30:12.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:12.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.172604+0000", 2024-09-18T11:30:12.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022772220000000002 2024-09-18T11:30:12.581 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.581 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:12.581 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.582 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:923 4.b 4:db5b7794:::smithi11939506-13:head [read 0~2873964,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:12.582 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169380+0000", 2024-09-18T11:30:12.582 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.3217163009999999, 2024-09-18T11:30:12.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047044239999999996, 2024-09-18T11:30:12.582 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:12.582 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:12.582 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:12.582 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:12.582 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:12.582 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 923 2024-09-18T11:30:12.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.582 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:12.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:12.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169380+0000", 2024-09-18T11:30:12.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:12.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169380+0000", 2024-09-18T11:30:12.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:12.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169380+0000", 2024-09-18T11:30:12.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:12.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169383+0000", 2024-09-18T11:30:12.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1429999999999998e-06 2024-09-18T11:30:12.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:12.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169389+0000", 2024-09-18T11:30:12.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4579999999999998e-06 2024-09-18T11:30:12.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169393+0000", 2024-09-18T11:30:12.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7840000000000001e-06 2024-09-18T11:30:12.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.172647+0000", 2024-09-18T11:30:12.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032532849999999999 2024-09-18T11:30:12.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:12.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.172724+0000", 2024-09-18T11:30:12.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7836999999999994e-05 2024-09-18T11:30:12.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:12.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174085+0000", 2024-09-18T11:30:12.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013603580000000001 2024-09-18T11:30:12.586 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.586 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:12.586 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.586 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:924 4.b 4:db5b7794:::smithi11939506-13:head [sparse-read 0~1546056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:12.586 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169399+0000", 2024-09-18T11:30:12.586 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.321697957, 2024-09-18T11:30:12.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054498719999999997, 2024-09-18T11:30:12.586 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:12.587 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:12.587 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:12.587 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:12.587 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:12.587 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 924 2024-09-18T11:30:12.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.587 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:12.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:12.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169399+0000", 2024-09-18T11:30:12.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:12.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169399+0000", 2024-09-18T11:30:12.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:12.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:12.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169398+0000", 2024-09-18T11:30:12.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:12.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:12.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169401+0000", 2024-09-18T11:30:12.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4490000000000002e-06 2024-09-18T11:30:12.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:12.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169407+0000", 2024-09-18T11:30:12.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9490000000000001e-06 2024-09-18T11:30:12.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:12.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169411+0000", 2024-09-18T11:30:12.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.924e-06 2024-09-18T11:30:12.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:12.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174189+0000", 2024-09-18T11:30:12.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047787039999999999 2024-09-18T11:30:12.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:12.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174263+0000", 2024-09-18T11:30:12.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3301999999999997e-05 2024-09-18T11:30:12.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:12.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:12.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:12.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174849+0000", 2024-09-18T11:30:12.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00058598500000000002 2024-09-18T11:30:12.590 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.590 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:12.590 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.591 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:12.591 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:12.591 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:12.592 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:30:12.593 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:30:12.593 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:30:12.594 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:12.594 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:12.594 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:12.595 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:12.595 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:12.591+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:12.599 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:12.599 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:12.614 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:12.615 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:12.630 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:12.631 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:12.646 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:12.646 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:12.661 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:12.662 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:12.696 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:12.696 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:12.696 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:12.694+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:12.706 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:30:12.794 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:12.793+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:12.845 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:12.845 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:12.853 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:30:12.895 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:12.894+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:12.951 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:12.951 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:12.960 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:30:13.003 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:13.001+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:13.057 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:13.058 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:13.066 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:30:13.097 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:13.096+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:13.134 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:13.132+0000 7f2d449e8640 -1 osd.1 32 handle_osd_map injecting map crc failure 2024-09-18T11:30:13.136 INFO:tasks.rados.rados.0.smithi119.stdout:666: done (1 left) 2024-09-18T11:30:13.136 INFO:tasks.rados.rados.0.smithi119.stdout:667: set_chunk oid 9 target oid 2024-09-18T11:30:13.145 INFO:tasks.rados.rados.0.smithi119.stdout:667: set_chunk oid 9 offset: 1455616 length: 451584 target oid b71d4ced2797a1c29f5ea886ab72f02ac8e4be1b2f2da685126b5add072dc1cb offset: 0 snap -1 2024-09-18T11:30:13.145 INFO:tasks.rados.rados.0.smithi119.stdout:668: read oid 14 snap 0 2024-09-18T11:30:13.145 INFO:tasks.rados.rados.0.smithi119.stdout:668: expect (ObjNum 66 snap 0 seq_num 66) 2024-09-18T11:30:13.145 INFO:tasks.rados.rados.0.smithi119.stdout:669: read oid 36 snap -1 2024-09-18T11:30:13.145 INFO:tasks.rados.rados.0.smithi119.stdout:669: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-18T11:30:13.145 INFO:tasks.rados.rados.0.smithi119.stdout:670: write oid 12 current snap is 2 2024-09-18T11:30:13.145 INFO:tasks.rados.rados.0.smithi119.stdout:670: seq_num 70 ranges {776490=714607,2246563=579356} 2024-09-18T11:30:13.154 INFO:tasks.rados.rados.0.smithi119.stdout:670: writing smithi11939506-12 from 776490 to 1491097 tid 1 2024-09-18T11:30:13.163 INFO:tasks.rados.rados.0.smithi119.stdout:670: writing smithi11939506-12 from 2246563 to 2825919 tid 2 2024-09-18T11:30:13.163 INFO:tasks.rados.rados.0.smithi119.stdout:671: read oid 29 snap -1 2024-09-18T11:30:13.163 INFO:tasks.rados.rados.0.smithi119.stdout:671: expect (ObjNum 62 snap 0 seq_num 62) 2024-09-18T11:30:13.163 INFO:tasks.rados.rados.0.smithi119.stdout:672: set_chunk oid 23 target oid 2024-09-18T11:30:13.176 INFO:tasks.rados.rados.0.smithi119.stdout:672: set_chunk oid 23 offset: 859136 length: 1175040 target oid 3f8435ee4f4497c7248ac55b3962ee05f72f35c915dc28d2cd3e0734af7cb550 offset: 0 snap -1 2024-09-18T11:30:13.176 INFO:tasks.rados.rados.0.smithi119.stdout:673: read oid 34 snap -1 2024-09-18T11:30:13.176 INFO:tasks.rados.rados.0.smithi119.stdout:673: expect deleted 2024-09-18T11:30:13.176 INFO:tasks.rados.rados.0.smithi119.stdout:674: tier_evict oid 49 2024-09-18T11:30:13.176 INFO:tasks.rados.rados.0.smithi119.stdout:674: tier_evict oid 49 snap -1 2024-09-18T11:30:13.177 INFO:tasks.rados.rados.0.smithi119.stdout:665: expect (ObjNum 34 snap 0 seq_num 34) 2024-09-18T11:30:13.198 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:13.197+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:13.220 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:13.221 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:13.229 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:30:13.261 INFO:tasks.rados.rados.0.smithi119.stdout:667: finishing set_chunk to oid 9 2024-09-18T11:30:13.261 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 9 set_chunk returned error code -95 offset: 1455616 length: 451584 2024-09-18T11:30:13.261 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 0 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-18T11:30:13.262 INFO:tasks.rados.rados.0.smithi119.stdout:669: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-18T11:30:13.305 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:13.303+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:13.347 INFO:tasks.rados.rados.0.smithi119.stdout:668: expect (ObjNum 66 snap 0 seq_num 66) 2024-09-18T11:30:13.351 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:13.349+0000 7fb6950ff640 -1 osd.2 32 handle_osd_map injecting map crc failure 2024-09-18T11:30:13.360 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:13.360 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:13.369 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:30:13.399 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:13.398+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:13.500 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:13.498+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:13.508 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:13.508 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:13.517 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:30:13.531 INFO:tasks.rados.rados.0.smithi119.stdout:670: finishing write tid 1 to smithi11939506-12 2024-09-18T11:30:13.531 INFO:tasks.rados.rados.0.smithi119.stdout:671: expect (ObjNum 62 snap 0 seq_num 62) 2024-09-18T11:30:13.601 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:13.600+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:13.629 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:13.629 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:13.637 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:30:13.668 INFO:tasks.rados.rados.0.smithi119.stdout:670: finishing write tid 2 to smithi11939506-12 2024-09-18T11:30:13.668 INFO:tasks.rados.rados.0.smithi119.stdout:670: finishing write tid 3 to smithi11939506-12 2024-09-18T11:30:13.668 INFO:tasks.rados.rados.0.smithi119.stdout:670: finishing write tid 4 to smithi11939506-12 2024-09-18T11:30:13.668 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 12 v 16 (ObjNum 70 snap 2 seq_num 70) dirty exists 2024-09-18T11:30:13.668 INFO:tasks.rados.rados.0.smithi119.stdout:670: left oid 12 (ObjNum 70 snap 2 seq_num 70) 2024-09-18T11:30:13.668 INFO:tasks.rados.rados.0.smithi119.stdout:665: done (8 left) 2024-09-18T11:30:13.668 INFO:tasks.rados.rados.0.smithi119.stdout:667: done (7 left) 2024-09-18T11:30:13.668 INFO:tasks.rados.rados.0.smithi119.stdout:668: done (6 left) 2024-09-18T11:30:13.668 INFO:tasks.rados.rados.0.smithi119.stdout:669: done (5 left) 2024-09-18T11:30:13.668 INFO:tasks.rados.rados.0.smithi119.stdout:670: done (4 left) 2024-09-18T11:30:13.668 INFO:tasks.rados.rados.0.smithi119.stdout:671: done (3 left) 2024-09-18T11:30:13.669 INFO:tasks.rados.rados.0.smithi119.stdout:675: read oid 22 snap -1 2024-09-18T11:30:13.669 INFO:tasks.rados.rados.0.smithi119.stdout:675: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-18T11:30:13.669 INFO:tasks.rados.rados.0.smithi119.stdout:676: read oid 15 snap -1 2024-09-18T11:30:13.669 INFO:tasks.rados.rados.0.smithi119.stdout:676: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-18T11:30:13.669 INFO:tasks.rados.rados.0.smithi119.stdout:677: read oid 6 snap -1 2024-09-18T11:30:13.669 INFO:tasks.rados.rados.0.smithi119.stdout:677: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-18T11:30:13.669 INFO:tasks.rados.rados.0.smithi119.stdout:672: finishing set_chunk to oid 23 2024-09-18T11:30:13.669 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 23 set_chunk returned error code -95 offset: 859136 length: 1175040 2024-09-18T11:30:13.669 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 0 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-18T11:30:13.669 INFO:tasks.rados.rados.0.smithi119.stdout:674: got (22) Invalid argument 2024-09-18T11:30:13.669 INFO:tasks.rados.rados.0.smithi119.stdout:678: read oid 35 snap -1 2024-09-18T11:30:13.669 INFO:tasks.rados.rados.0.smithi119.stdout:678: expect (ObjNum 34 snap 0 seq_num 34) 2024-09-18T11:30:13.669 INFO:tasks.rados.rados.0.smithi119.stdout:672: done (6 left) 2024-09-18T11:30:13.669 INFO:tasks.rados.rados.0.smithi119.stdout:673: done (5 left) 2024-09-18T11:30:13.670 INFO:tasks.rados.rados.0.smithi119.stdout:674: done (4 left) 2024-09-18T11:30:13.670 INFO:tasks.rados.rados.0.smithi119.stdout:679: snap_remove snap 0 2024-09-18T11:30:13.702 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:13.701+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:13.736 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:13.736 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:13.744 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:30:13.803 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:13.802+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:13.855 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:13.856 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:13.864 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:30:13.905 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:13.904+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:14.003 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:14.003 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:14.004 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:14.002+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:14.011 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:30:14.105 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:14.104+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:14.109 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:14.110 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:14.118 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:30:14.146 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T11:30:14.149 INFO:tasks.rados.rados.0.smithi119.stdout:679: done (4 left) 2024-09-18T11:30:14.149 INFO:tasks.rados.rados.0.smithi119.stdout:680: set_chunk oid 19 target oid 2024-09-18T11:30:14.155 INFO:tasks.rados.rados.0.smithi119.stdout:680: set_chunk oid 19 offset: 1140224 length: 47104 target oid 58b9f1b3306aaad4bbac1f01593fea176f754b6806effd08b8003d275d5a27db offset: 0 snap -1 2024-09-18T11:30:14.155 INFO:tasks.rados.rados.0.smithi119.stdout:681: read oid 39 snap -1 2024-09-18T11:30:14.155 INFO:tasks.rados.rados.0.smithi119.stdout:681: expect (ObjNum 38 snap 0 seq_num 38) 2024-09-18T11:30:14.155 INFO:tasks.rados.rados.0.smithi119.stdout:682: read oid 29 snap -1 2024-09-18T11:30:14.155 INFO:tasks.rados.rados.0.smithi119.stdout:682: expect (ObjNum 62 snap 0 seq_num 62) 2024-09-18T11:30:14.155 INFO:tasks.rados.rados.0.smithi119.stdout:683: write oid 34 current snap is 2 2024-09-18T11:30:14.155 INFO:tasks.rados.rados.0.smithi119.stdout:683: seq_num 71 ranges {793297=789856,2305045=723014} 2024-09-18T11:30:14.163 INFO:tasks.rados.rados.0.smithi119.stdout:683: writing smithi11939506-34 from 793297 to 1583153 tid 1 2024-09-18T11:30:14.171 INFO:tasks.rados.rados.0.smithi119.stdout:683: writing smithi11939506-34 from 2305045 to 3028059 tid 2 2024-09-18T11:30:14.171 INFO:tasks.rados.rados.0.smithi119.stdout:684: snap_create 2024-09-18T11:30:14.171 INFO:tasks.rados.rados.0.smithi119.stdout:675: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-18T11:30:14.207 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:14.204+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:14.251 INFO:tasks.rados.rados.0.smithi119.stdout:677: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-18T11:30:14.259 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:14.259 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:14.268 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:30:14.311 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:14.310+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:14.322 INFO:tasks.rados.rados.0.smithi119.stdout:678: expect (ObjNum 34 snap 0 seq_num 34) 2024-09-18T11:30:14.366 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:14.366 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:14.375 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:30:14.404 INFO:tasks.rados.rados.0.smithi119.stdout:676: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-18T11:30:14.408 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:14.406+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:14.477 INFO:tasks.rados.rados.0.smithi119.stdout:682: expect (ObjNum 62 snap 0 seq_num 62) 2024-09-18T11:30:14.509 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:14.508+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:14.513 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:14.513 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:14.522 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:30:14.606 INFO:tasks.rados.rados.0.smithi119.stdout:680: finishing set_chunk to oid 19 2024-09-18T11:30:14.606 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 12 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-18T11:30:14.606 INFO:tasks.rados.rados.0.smithi119.stdout:681: expect (ObjNum 38 snap 0 seq_num 38) 2024-09-18T11:30:14.609 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:14.607+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:14.660 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:14.660 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:14.668 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:30:14.674 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:30:14.692 INFO:tasks.rados.rados.0.smithi119.stdout:683: finishing write tid 1 to smithi11939506-34 2024-09-18T11:30:14.692 INFO:tasks.rados.rados.0.smithi119.stdout:683: finishing write tid 2 to smithi11939506-34 2024-09-18T11:30:14.692 INFO:tasks.rados.rados.0.smithi119.stdout:683: finishing write tid 3 to smithi11939506-34 2024-09-18T11:30:14.692 INFO:tasks.rados.rados.0.smithi119.stdout:683: finishing write tid 4 to smithi11939506-34 2024-09-18T11:30:14.692 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 31 (ObjNum 71 snap 2 seq_num 71) dirty exists 2024-09-18T11:30:14.692 INFO:tasks.rados.rados.0.smithi119.stdout:683: left oid 34 (ObjNum 71 snap 2 seq_num 71) 2024-09-18T11:30:14.710 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:14.708+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:14.765 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:14.766 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:14.774 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:30:14.787 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:30:14.788 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:30:14.803 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:14.803 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:14.811 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:14.809+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:14.817 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:14.817 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:14.832 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:14.832 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:14.847 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:14.847 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:14.862 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:14.862 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:14.874 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:14.874 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:14.877 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:14.877 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:14.883 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:30:14.892 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:14.892 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:14.927 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:14.926+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:14.981 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:14.981 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:14.981 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:14.981 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:14.989 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:30:15.012 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:15.011+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:15.113 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:15.112+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:15.127 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:15.127 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:15.127 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:15.127 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:15.127 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:15.135 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:30:15.145 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:15.143+0000 7fb6950ff640 -1 osd.2 34 handle_osd_map injecting map crc failure 2024-09-18T11:30:15.147 INFO:tasks.rados.rados.0.smithi119.stdout:675: done (8 left) 2024-09-18T11:30:15.147 INFO:tasks.rados.rados.0.smithi119.stdout:676: done (7 left) 2024-09-18T11:30:15.147 INFO:tasks.rados.rados.0.smithi119.stdout:677: done (6 left) 2024-09-18T11:30:15.147 INFO:tasks.rados.rados.0.smithi119.stdout:678: done (5 left) 2024-09-18T11:30:15.147 INFO:tasks.rados.rados.0.smithi119.stdout:680: done (4 left) 2024-09-18T11:30:15.147 INFO:tasks.rados.rados.0.smithi119.stdout:681: done (3 left) 2024-09-18T11:30:15.147 INFO:tasks.rados.rados.0.smithi119.stdout:682: done (2 left) 2024-09-18T11:30:15.147 INFO:tasks.rados.rados.0.smithi119.stdout:683: done (1 left) 2024-09-18T11:30:15.148 INFO:tasks.rados.rados.0.smithi119.stdout:684: done (0 left) 2024-09-18T11:30:15.148 INFO:tasks.rados.rados.0.smithi119.stdout:685: read oid 27 snap -1 2024-09-18T11:30:15.148 INFO:tasks.rados.rados.0.smithi119.stdout:685: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-18T11:30:15.148 INFO:tasks.rados.rados.0.smithi119.stdout:686: write (excl) oid 44 current snap is 3 2024-09-18T11:30:15.148 INFO:tasks.rados.rados.0.smithi119.stdout:686: seq_num 72 ranges {410104=465105,1563527=583025,2810926=399178} 2024-09-18T11:30:15.148 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is set 2024-09-18T11:30:15.151 INFO:tasks.rados.rados.0.smithi119.stdout:686: writing smithi11939506-44 from 410104 to 875209 tid 1 2024-09-18T11:30:15.158 INFO:tasks.rados.rados.0.smithi119.stdout:686: writing smithi11939506-44 from 1563527 to 2146552 tid 2 2024-09-18T11:30:15.165 INFO:tasks.rados.rados.0.smithi119.stdout:686: writing smithi11939506-44 from 2810926 to 3210104 tid 3 2024-09-18T11:30:15.165 INFO:tasks.rados.rados.0.smithi119.stdout:687: read oid 24 snap -1 2024-09-18T11:30:15.165 INFO:tasks.rados.rados.0.smithi119.stdout:687: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-18T11:30:15.165 INFO:tasks.rados.rados.0.smithi119.stdout:688: read oid 22 snap 2 2024-09-18T11:30:15.166 INFO:tasks.rados.rados.0.smithi119.stdout:688: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-18T11:30:15.166 INFO:tasks.rados.rados.0.smithi119.stdout:689: read oid 40 snap 2 2024-09-18T11:30:15.166 INFO:tasks.rados.rados.0.smithi119.stdout:689: expect (ObjNum 39 snap 0 seq_num 39) 2024-09-18T11:30:15.166 INFO:tasks.rados.rados.0.smithi119.stdout:690: read oid 32 snap 2 2024-09-18T11:30:15.166 INFO:tasks.rados.rados.0.smithi119.stdout:690: expect (ObjNum 64 snap 0 seq_num 64) 2024-09-18T11:30:15.166 INFO:tasks.rados.rados.0.smithi119.stdout:691: tier_evict oid 34 2024-09-18T11:30:15.166 INFO:tasks.rados.rados.0.smithi119.stdout:691: tier_evict oid 34 snap -1 2024-09-18T11:30:15.166 INFO:tasks.rados.rados.0.smithi119.stdout:685: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-18T11:30:15.214 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:15.213+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:15.237 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:15.238 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:15.238 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:15.238 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:15.238 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.238 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:41 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:15.238 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157303+0000", 2024-09-18T11:30:15.238 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.077828530000001, 2024-09-18T11:30:15.238 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054616040000000001, 2024-09-18T11:30:15.238 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:15.238 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.238 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:15.239 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:15.239 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:15.239 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 41 2024-09-18T11:30:15.239 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.239 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:15.239 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.239 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:15.239 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157303+0000", 2024-09-18T11:30:15.239 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.239 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.239 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.240 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:15.240 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157303+0000", 2024-09-18T11:30:15.240 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.240 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.240 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.240 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:15.240 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157303+0000", 2024-09-18T11:30:15.240 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.240 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.240 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.240 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:15.240 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157388+0000", 2024-09-18T11:30:15.241 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4938999999999994e-05 2024-09-18T11:30:15.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.241 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.241 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:15.241 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157395+0000", 2024-09-18T11:30:15.241 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1130000000000003e-06 2024-09-18T11:30:15.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.241 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.241 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.241 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157405+0000", 2024-09-18T11:30:15.241 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7510000000000003e-06 2024-09-18T11:30:15.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.241 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.242 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:15.242 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158213+0000", 2024-09-18T11:30:15.242 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00080873000000000002 2024-09-18T11:30:15.242 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.242 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.242 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:15.242 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158267+0000", 2024-09-18T11:30:15.242 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3709999999999999e-05 2024-09-18T11:30:15.242 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.242 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.242 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:15.242 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158352+0000", 2024-09-18T11:30:15.242 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4782000000000002e-05 2024-09-18T11:30:15.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.243 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.243 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:15.243 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160262+0000", 2024-09-18T11:30:15.243 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001910068 2024-09-18T11:30:15.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.243 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.243 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.243 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162672+0000", 2024-09-18T11:30:15.243 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024101159999999999 2024-09-18T11:30:15.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.243 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.243 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:15.243 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162724+0000", 2024-09-18T11:30:15.244 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1637e-05 2024-09-18T11:30:15.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.244 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.244 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:15.244 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162765+0000", 2024-09-18T11:30:15.244 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1282000000000002e-05 2024-09-18T11:30:15.244 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.244 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:15.244 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.244 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.244 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:15.244 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:15.245 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.077718817000001, 2024-09-18T11:30:15.245 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054864400000000004, 2024-09-18T11:30:15.245 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:15.245 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.245 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:15.245 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:15.245 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:15.245 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 42 2024-09-18T11:30:15.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.245 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:15.245 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.245 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:15.245 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:15.245 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.246 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.246 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:15.246 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:15.246 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.246 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.246 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:15.246 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:15.246 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.246 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.246 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:15.246 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157416+0000", 2024-09-18T11:30:15.247 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.721e-06 2024-09-18T11:30:15.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.247 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.247 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:15.247 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157423+0000", 2024-09-18T11:30:15.247 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4049999999999997e-06 2024-09-18T11:30:15.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.247 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.247 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.247 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157425+0000", 2024-09-18T11:30:15.247 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.481e-06 2024-09-18T11:30:15.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.247 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.247 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:15.248 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158649+0000", 2024-09-18T11:30:15.248 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001223453 2024-09-18T11:30:15.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.248 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.248 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:15.248 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158729+0000", 2024-09-18T11:30:15.248 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0205999999999997e-05 2024-09-18T11:30:15.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.248 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.248 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:15.248 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158875+0000", 2024-09-18T11:30:15.248 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014628800000000001 2024-09-18T11:30:15.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.249 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.249 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:15.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161518+0000", 2024-09-18T11:30:15.249 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026427949999999999 2024-09-18T11:30:15.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.249 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.249 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162821+0000", 2024-09-18T11:30:15.249 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013028099999999999 2024-09-18T11:30:15.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.249 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.249 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:15.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162864+0000", 2024-09-18T11:30:15.249 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2839000000000001e-05 2024-09-18T11:30:15.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.250 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.250 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:15.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162900+0000", 2024-09-18T11:30:15.250 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5948999999999998e-05 2024-09-18T11:30:15.250 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.250 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:15.250 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.250 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.250 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:15.250 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:15.250 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.077632453, 2024-09-18T11:30:15.251 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055400850000000001, 2024-09-18T11:30:15.251 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:15.251 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.251 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:15.251 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:15.251 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:15.251 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 43 2024-09-18T11:30:15.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.251 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:15.251 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.251 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:15.251 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:15.251 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.252 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.252 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:15.252 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:15.252 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.252 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.252 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:15.252 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:15.252 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:15.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.252 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.252 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:15.252 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157504+0000", 2024-09-18T11:30:15.253 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4519999999999996e-06 2024-09-18T11:30:15.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.253 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.253 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:15.253 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157534+0000", 2024-09-18T11:30:15.253 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.97e-05 2024-09-18T11:30:15.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.253 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.253 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.253 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157539+0000", 2024-09-18T11:30:15.253 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4360000000000001e-06 2024-09-18T11:30:15.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.253 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.254 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:15.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159428+0000", 2024-09-18T11:30:15.254 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018888220000000001 2024-09-18T11:30:15.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.254 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.254 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:15.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159479+0000", 2024-09-18T11:30:15.254 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0608000000000001e-05 2024-09-18T11:30:15.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.254 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.254 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:15.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159559+0000", 2024-09-18T11:30:15.254 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0667000000000005e-05 2024-09-18T11:30:15.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.255 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.255 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:15.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162779+0000", 2024-09-18T11:30:15.255 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032197459999999999 2024-09-18T11:30:15.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.255 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.255 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162950+0000", 2024-09-18T11:30:15.255 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017059599999999999 2024-09-18T11:30:15.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.255 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.255 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:15.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163001+0000", 2024-09-18T11:30:15.255 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0968999999999999e-05 2024-09-18T11:30:15.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.256 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.256 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:15.256 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163040+0000", 2024-09-18T11:30:15.256 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8843000000000001e-05 2024-09-18T11:30:15.256 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.256 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:15.256 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.256 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.256 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:15.256 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:15.257 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.077581298999998, 2024-09-18T11:30:15.257 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:30:15.257 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:15.257 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.257 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:15.257 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:15.257 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:15.257 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:30:15.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.257 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:15.257 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.257 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:15.257 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:15.258 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.258 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.258 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:15.258 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:15.258 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.258 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.258 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:15.258 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:30:15.258 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.258 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.259 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:15.259 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:30:15.259 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:30:15.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.259 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.259 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:15.259 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:30:15.259 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:30:15.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.259 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.259 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.259 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:30:15.259 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:30:15.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.260 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.260 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:15.260 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:30:15.260 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:30:15.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.260 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.260 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:15.260 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:30:15.260 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:30:15.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.260 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.260 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:15.260 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:30:15.261 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:30:15.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.261 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.261 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:15.261 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:30:15.261 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:30:15.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.261 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.261 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.261 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:30:15.261 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:30:15.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.261 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.262 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:15.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:30:15.262 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:30:15.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.262 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.262 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:15.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:30:15.262 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:30:15.262 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.262 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:15.262 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.262 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.262 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:15.263 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:15.263 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.077455037, 2024-09-18T11:30:15.263 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055717040000000002, 2024-09-18T11:30:15.263 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:15.263 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.263 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:15.263 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:15.263 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:15.263 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 45 2024-09-18T11:30:15.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.263 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:15.263 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.263 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:15.264 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:15.264 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.264 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.264 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:15.264 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:15.264 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.264 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.264 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:15.264 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157676+0000", 2024-09-18T11:30:15.264 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.265 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.265 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:15.265 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157694+0000", 2024-09-18T11:30:15.265 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7422999999999999e-05 2024-09-18T11:30:15.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.265 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.265 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:15.265 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157704+0000", 2024-09-18T11:30:15.265 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9000000000000001e-06 2024-09-18T11:30:15.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.265 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.265 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.265 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157718+0000", 2024-09-18T11:30:15.266 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4788e-05 2024-09-18T11:30:15.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.266 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.266 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:15.266 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160937+0000", 2024-09-18T11:30:15.266 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00321886 2024-09-18T11:30:15.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.266 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.266 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:15.266 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161023+0000", 2024-09-18T11:30:15.266 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6063999999999995e-05 2024-09-18T11:30:15.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.266 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.267 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:15.267 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161202+0000", 2024-09-18T11:30:15.267 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017912599999999999 2024-09-18T11:30:15.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.267 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.267 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:15.267 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163052+0000", 2024-09-18T11:30:15.267 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001849554 2024-09-18T11:30:15.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.267 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.267 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.267 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163204+0000", 2024-09-18T11:30:15.267 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015186900000000001 2024-09-18T11:30:15.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.268 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.268 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:15.268 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163229+0000", 2024-09-18T11:30:15.268 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:30:15.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.268 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.268 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:15.268 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163249+0000", 2024-09-18T11:30:15.268 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9992000000000001e-05 2024-09-18T11:30:15.268 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.268 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:15.268 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.269 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.269 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:15.269 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:15.269 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.077405924000001, 2024-09-18T11:30:15.269 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:30:15.269 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:15.269 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.269 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:15.269 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:15.269 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:15.269 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:30:15.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.269 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:15.269 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.270 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:15.270 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:15.270 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.270 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.270 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:15.270 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:15.270 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.270 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.270 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:15.270 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:15.270 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.271 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.271 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:15.271 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:30:15.271 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:30:15.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.271 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.271 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:15.271 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:30:15.271 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:30:15.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.271 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.271 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.271 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:30:15.272 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:30:15.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.272 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.272 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:15.272 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:30:15.272 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:30:15.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.272 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.272 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:15.272 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:30:15.272 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:30:15.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.272 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.273 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:15.273 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:30:15.273 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:30:15.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.273 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.273 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:15.273 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:30:15.273 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:30:15.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.273 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.273 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.273 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:30:15.273 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:30:15.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.274 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.274 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:15.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:30:15.274 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:30:15.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.274 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.274 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:15.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:30:15.274 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:30:15.274 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.274 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:15.274 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.274 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.275 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:15.275 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:15.275 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.077363392999999, 2024-09-18T11:30:15.275 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:30:15.275 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:15.275 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.275 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:15.275 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:15.275 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:15.275 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:30:15.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.275 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:15.275 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.276 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:15.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:15.276 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.276 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.276 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:15.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:15.276 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.276 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.276 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:15.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:30:15.276 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.277 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:15.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:30:15.277 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:30:15.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.277 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:15.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:30:15.277 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:30:15.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.277 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:30:15.277 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:30:15.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.278 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.278 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:15.278 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:30:15.278 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:30:15.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.278 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.278 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:15.278 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:30:15.278 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:30:15.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.278 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.278 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:15.278 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:30:15.279 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:30:15.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.279 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.279 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.279 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:30:15.279 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:30:15.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.279 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.279 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:15.279 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:30:15.280 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:30:15.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.280 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:15.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:30:15.280 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:30:15.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.280 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:15.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:30:15.280 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:30:15.280 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.280 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:15.280 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.281 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:15.281 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:15.281 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.077308463000001, 2024-09-18T11:30:15.281 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:30:15.281 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:15.281 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.281 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:15.281 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:15.281 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:15.281 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:30:15.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.282 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:15.282 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.282 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:15.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:15.282 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.282 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.282 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:15.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:15.282 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.282 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.282 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:15.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:15.283 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.283 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:15.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:30:15.283 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:30:15.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.283 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:15.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:30:15.283 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:30:15.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.284 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.284 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.284 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:30:15.284 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:30:15.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.284 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.284 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:15.284 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:30:15.284 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:30:15.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.284 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.284 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:15.285 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:30:15.285 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:30:15.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.285 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:15.285 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:30:15.285 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:30:15.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.285 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:15.285 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:30:15.285 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:30:15.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.286 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.286 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:30:15.286 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:30:15.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.286 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.286 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:15.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:30:15.286 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:30:15.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.286 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.286 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:15.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:30:15.287 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:30:15.287 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.287 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:15.287 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.287 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:15.287 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:15.287 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.077298483, 2024-09-18T11:30:15.287 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:30:15.287 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:15.287 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.287 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:15.288 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:15.288 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:15.288 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:30:15.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.288 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:15.288 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.288 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:15.288 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:15.288 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.288 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.288 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:15.289 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:15.289 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.289 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.289 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:15.289 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:15.289 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:15.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.289 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.289 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:15.289 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:30:15.289 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:30:15.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.289 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.290 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:15.290 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:30:15.290 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:30:15.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.290 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.290 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.290 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:30:15.290 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:30:15.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.290 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.290 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:15.290 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:30:15.290 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:30:15.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.291 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:15.291 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:30:15.291 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:30:15.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.291 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:15.291 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:30:15.291 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:30:15.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.291 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.291 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:30:15.291 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:30:15.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.292 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.292 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:15.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:30:15.292 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:30:15.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.292 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.292 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:15.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:30:15.292 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:30:15.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.292 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.292 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:15.293 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:30:15.293 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:30:15.293 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.293 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:15.293 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.293 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:15.293 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:15.293 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.077286883999999, 2024-09-18T11:30:15.293 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:30:15.293 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:15.293 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.294 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:15.294 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:15.294 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:15.294 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:30:15.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.294 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:15.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.294 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:15.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:15.294 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.294 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:15.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:15.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.295 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:15.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:15.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:15.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.295 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:15.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:30:15.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:30:15.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.296 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:15.296 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:30:15.296 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:30:15.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.296 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.296 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:30:15.296 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:30:15.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.296 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:15.296 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:30:15.296 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:30:15.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.297 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:15.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:30:15.297 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:30:15.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.297 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:15.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:30:15.297 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:30:15.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.297 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:15.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:30:15.298 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:30:15.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.298 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.298 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:30:15.298 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:30:15.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.298 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:15.298 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:30:15.298 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:30:15.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.299 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:15.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:30:15.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:30:15.299 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.299 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:15.299 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.299 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:15.299 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:15.299 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.077230403000002, 2024-09-18T11:30:15.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:30:15.299 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:15.300 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.300 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:15.300 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:15.300 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:15.300 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:30:15.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.300 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:15.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:15.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:15.300 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.301 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:15.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:15.301 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.301 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:15.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:30:15.301 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.301 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:15.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:30:15.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:30:15.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.302 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:15.302 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:30:15.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:30:15.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.302 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.302 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:30:15.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:30:15.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.303 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:15.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:30:15.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:30:15.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.303 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:15.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:30:15.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:30:15.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.303 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:15.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:30:15.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:30:15.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.304 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:30:15.304 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:30:15.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.304 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:15.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:30:15.304 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:30:15.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.304 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:15.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:30:15.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:30:15.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:15.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:30:15.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:30:15.305 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.305 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:15.305 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.306 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:15.306 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:15.306 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.077216579000002, 2024-09-18T11:30:15.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:30:15.306 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:15.306 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.306 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:15.306 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:15.306 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:15.306 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:30:15.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.306 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:15.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:15.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:15.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:15.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:15.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:15.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:15.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:15.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.308 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.308 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:15.308 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:30:15.308 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:30:15.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.308 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.308 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:15.308 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:30:15.308 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:30:15.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.308 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.308 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.308 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:30:15.308 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:30:15.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.309 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:15.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:30:15.309 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:30:15.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.309 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:15.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:30:15.309 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:30:15.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.309 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:15.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:30:15.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:30:15.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:30:15.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:30:15.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:15.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:30:15.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:30:15.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:15.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:30:15.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:30:15.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:15.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:30:15.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:30:15.311 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.311 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:15.311 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.312 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:15.312 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:15.312 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.661037671999999, 2024-09-18T11:30:15.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:30:15.312 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:15.312 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.312 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:15.312 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:15.312 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:15.312 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:30:15.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.312 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:15.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.313 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:15.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:15.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.313 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:15.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:15.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.313 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:15.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:15.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:15.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.314 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:15.314 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:30:15.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:30:15.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.314 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:15.314 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:30:15.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:30:15.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.314 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:30:15.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:30:15.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:15.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:30:15.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:30:15.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:15.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:30:15.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:30:15.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:15.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:30:15.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:30:15.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:15.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:30:15.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:30:15.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:30:15.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:30:15.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:15.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:30:15.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:30:15.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:15.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:30:15.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:30:15.317 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.317 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:15.317 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.318 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:15.318 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:15.318 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.661014372, 2024-09-18T11:30:15.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:30:15.318 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:15.318 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.318 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:15.318 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:15.318 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:15.318 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:30:15.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.318 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:15.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:15.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:15.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:15.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:15.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:15.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:30:15.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:15.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:30:15.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:30:15.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:15.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:30:15.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:30:15.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:30:15.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:30:15.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:15.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:30:15.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:30:15.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:15.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:30:15.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:30:15.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:15.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:30:15.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:30:15.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:15.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:30:15.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:30:15.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:30:15.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:30:15.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:15.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:30:15.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:30:15.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:15.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:30:15.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:30:15.323 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.323 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:15.323 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.324 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:15.324 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:15.324 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.660928763000001, 2024-09-18T11:30:15.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:30:15.324 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:15.324 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.324 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:15.324 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:15.324 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:15.324 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:30:15.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.324 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:15.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:15.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:15.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:15.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:15.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:15.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:15.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:15.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:30:15.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:30:15.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:15.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:30:15.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:30:15.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:30:15.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:30:15.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:15.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:30:15.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:30:15.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:15.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:30:15.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:30:15.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:15.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:30:15.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:30:15.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.328 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:15.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:30:15.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:30:15.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.328 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:30:15.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:30:15.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:15.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:30:15.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:30:15.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:15.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:30:15.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:30:15.329 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.329 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:15.329 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.330 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:15.330 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:15.330 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.660917798, 2024-09-18T11:30:15.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:30:15.330 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:15.330 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.330 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:15.330 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:15.330 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:15.330 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:30:15.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.330 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:15.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:15.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:15.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:15.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:15.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:15.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:15.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:15.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:15.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:30:15.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:30:15.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:15.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:30:15.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:30:15.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:30:15.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:30:15.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:15.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:30:15.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:30:15.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:15.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:30:15.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:30:15.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:15.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:30:15.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:30:15.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:15.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:30:15.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:30:15.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:30:15.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:30:15.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:15.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:30:15.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:30:15.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:15.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:30:15.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:30:15.335 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.335 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:15.335 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.336 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:15.336 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:15.336 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.660906629999999, 2024-09-18T11:30:15.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:30:15.336 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:15.336 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.336 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:15.336 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:15.336 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:15.336 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:30:15.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.336 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:15.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:15.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:15.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:15.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:15.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:15.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:15.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:15.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:15.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:30:15.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:30:15.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:15.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:30:15.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:30:15.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:30:15.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:30:15.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:15.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:30:15.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:30:15.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:15.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:30:15.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:30:15.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:15.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:30:15.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:30:15.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:30:15.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:30:15.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:15.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:30:15.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:30:15.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:15.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:30:15.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:30:15.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:15.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:30:15.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:30:15.341 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.341 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:15.341 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.342 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:15.342 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:15.342 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.660841301000001, 2024-09-18T11:30:15.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:30:15.342 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:15.342 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.342 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:15.342 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:15.342 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:15.342 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:30:15.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.342 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:15.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:15.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:15.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:15.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:15.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:15.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:30:15.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:15.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:30:15.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:30:15.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:15.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:30:15.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:30:15.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:30:15.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:30:15.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:15.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:30:15.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:30:15.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:15.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:30:15.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:30:15.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:15.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:30:15.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:30:15.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:30:15.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:30:15.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:15.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:30:15.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:30:15.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:15.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:30:15.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:30:15.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:15.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:30:15.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:30:15.347 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.347 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:15.347 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.348 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:15.348 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:15.348 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.650679322999999, 2024-09-18T11:30:15.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:30:15.348 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:15.348 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.348 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:15.348 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:15.348 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:15.348 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:30:15.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.348 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:15.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:15.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:15.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:15.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:15.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:15.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:30:15.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:15.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:30:15.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:30:15.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:15.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:30:15.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:30:15.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:30:15.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:30:15.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:15.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:30:15.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:30:15.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:15.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:30:15.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:30:15.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:15.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:30:15.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:30:15.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:15.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:30:15.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:30:15.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:30:15.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:30:15.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:15.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:30:15.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:30:15.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:15.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:30:15.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:30:15.353 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.353 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:15.353 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.354 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:15.354 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:15.354 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.650586843999999, 2024-09-18T11:30:15.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:30:15.354 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:15.354 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.354 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:15.354 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:15.354 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:15.354 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:30:15.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.354 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:15.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:15.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:15.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:15.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:15.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:15.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:15.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:15.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:15.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:30:15.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:30:15.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:15.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:30:15.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:30:15.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:30:15.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:30:15.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:15.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:30:15.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:30:15.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:15.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:30:15.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:30:15.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:15.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:30:15.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:30:15.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:15.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:30:15.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:30:15.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:30:15.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:30:15.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:15.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:30:15.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:30:15.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:15.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:15.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:15.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:30:15.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:30:15.359 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.359 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:15.359 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.359 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:15.360 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:15.360 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:15.362 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:15.360+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:15.362 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:30:15.363 INFO:tasks.rados.rados.0.smithi119.stdout:686: finishing write tid 1 to smithi11939506-44 2024-09-18T11:30:15.363 INFO:tasks.rados.rados.0.smithi119.stdout:686: finishing write tid 2 to smithi11939506-44 2024-09-18T11:30:15.363 INFO:tasks.rados.rados.0.smithi119.stdout:685: done (6 left) 2024-09-18T11:30:15.363 INFO:tasks.rados.rados.0.smithi119.stdout:692: write (excl) oid 29 current snap is 3 2024-09-18T11:30:15.363 INFO:tasks.rados.rados.0.smithi119.stdout:691: got (22) Invalid argument 2024-09-18T11:30:15.363 INFO:tasks.rados.rados.0.smithi119.stdout:686: finishing write tid 3 to smithi11939506-44 2024-09-18T11:30:15.363 INFO:tasks.rados.rados.0.smithi119.stdout:692: seq_num 73 ranges {426911=540354,1622009=726683,3121996=104915} 2024-09-18T11:30:15.363 INFO:tasks.rados.rados.0.smithi119.stdout:692: writing smithi11939506-29 from 426911 to 967265 tid 1 2024-09-18T11:30:15.363 INFO:tasks.rados.rados.0.smithi119.stdout:692: writing smithi11939506-29 from 1622009 to 2348692 tid 2 2024-09-18T11:30:15.363 INFO:tasks.rados.rados.0.smithi119.stdout:692: writing smithi11939506-29 from 3121996 to 3226911 tid 3 2024-09-18T11:30:15.363 INFO:tasks.rados.rados.0.smithi119.stdout:691: done (6 left) 2024-09-18T11:30:15.363 INFO:tasks.rados.rados.0.smithi119.stdout:693: write oid 13 current snap is 3 2024-09-18T11:30:15.364 INFO:tasks.rados.rados.0.smithi119.stdout:686: finishing write tid 4 to smithi11939506-44 2024-09-18T11:30:15.364 INFO:tasks.rados.rados.0.smithi119.stdout:686: finishing write tid 5 to smithi11939506-44 2024-09-18T11:30:15.364 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 44 v 33 (ObjNum 72 snap 3 seq_num 72) dirty exists 2024-09-18T11:30:15.364 INFO:tasks.rados.rados.0.smithi119.stdout:686: left oid 44 (ObjNum 72 snap 3 seq_num 72) 2024-09-18T11:30:15.364 INFO:tasks.rados.rados.0.smithi119.stdout:688: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-18T11:30:15.364 INFO:tasks.rados.rados.0.smithi119.stdout:687: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-18T11:30:15.420 INFO:tasks.rados.rados.0.smithi119.stdout:689: expect (ObjNum 39 snap 0 seq_num 39) 2024-09-18T11:30:15.461 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:15.459+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:15.463 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:15.463 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:15.472 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:30:15.507 INFO:tasks.rados.rados.0.smithi119.stdout:690: expect (ObjNum 64 snap 0 seq_num 64) 2024-09-18T11:30:15.562 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:15.561+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:15.605 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:15.605 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:15.605 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:15.606 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:15.606 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:15.615 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:30:15.650 INFO:tasks.rados.rados.0.smithi119.stdout:692: finishing write tid 1 to smithi11939506-29 2024-09-18T11:30:15.650 INFO:tasks.rados.rados.0.smithi119.stdout:692: finishing write tid 2 to smithi11939506-29 2024-09-18T11:30:15.650 INFO:tasks.rados.rados.0.smithi119.stdout:693: seq_num 74 ranges {443718=615603,1564138=470341,2633066=610652} 2024-09-18T11:30:15.655 INFO:tasks.rados.rados.0.smithi119.stdout:693: writing smithi11939506-13 from 443718 to 1059321 tid 1 2024-09-18T11:30:15.661 INFO:tasks.rados.rados.0.smithi119.stdout:693: writing smithi11939506-13 from 1564138 to 2034479 tid 2 2024-09-18T11:30:15.663 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:15.661+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:15.668 INFO:tasks.rados.rados.0.smithi119.stdout:693: writing smithi11939506-13 from 2633066 to 3243718 tid 3 2024-09-18T11:30:15.668 INFO:tasks.rados.rados.0.smithi119.stdout:686: done (6 left) 2024-09-18T11:30:15.668 INFO:tasks.rados.rados.0.smithi119.stdout:687: done (5 left) 2024-09-18T11:30:15.668 INFO:tasks.rados.rados.0.smithi119.stdout:688: done (4 left) 2024-09-18T11:30:15.669 INFO:tasks.rados.rados.0.smithi119.stdout:689: done (3 left) 2024-09-18T11:30:15.669 INFO:tasks.rados.rados.0.smithi119.stdout:690: done (2 left) 2024-09-18T11:30:15.669 INFO:tasks.rados.rados.0.smithi119.stdout:694: write (excl) oid 50 current snap is 3 2024-09-18T11:30:15.669 INFO:tasks.rados.rados.0.smithi119.stdout:692: finishing write tid 3 to smithi11939506-29 2024-09-18T11:30:15.669 INFO:tasks.rados.rados.0.smithi119.stdout:692: finishing write tid 4 to smithi11939506-29 2024-09-18T11:30:15.669 INFO:tasks.rados.rados.0.smithi119.stdout:694: seq_num 75 ranges {460525=690852,1622620=730352,3060489=200036} 2024-09-18T11:30:15.674 INFO:tasks.rados.rados.0.smithi119.stdout:694: writing smithi11939506-50 from 460525 to 1151377 tid 1 2024-09-18T11:30:15.681 INFO:tasks.rados.rados.0.smithi119.stdout:694: writing smithi11939506-50 from 1622620 to 2352972 tid 2 2024-09-18T11:30:15.686 INFO:tasks.rados.rados.0.smithi119.stdout:694: writing smithi11939506-50 from 3060489 to 3260525 tid 3 2024-09-18T11:30:15.686 INFO:tasks.rados.rados.0.smithi119.stdout:695: read oid 25 snap 2 2024-09-18T11:30:15.686 INFO:tasks.rados.rados.0.smithi119.stdout:695: expect (ObjNum 24 snap 0 seq_num 24) 2024-09-18T11:30:15.686 INFO:tasks.rados.rados.0.smithi119.stdout:696: read oid 12 snap -1 2024-09-18T11:30:15.686 INFO:tasks.rados.rados.0.smithi119.stdout:696: expect (ObjNum 70 snap 2 seq_num 70) 2024-09-18T11:30:15.687 INFO:tasks.rados.rados.0.smithi119.stdout:697: set_chunk oid 38 target oid 2024-09-18T11:30:15.699 INFO:tasks.rados.rados.0.smithi119.stdout:697: set_chunk oid 38 offset: 2076160 length: 356864 target oid 8da2985a35cccabf5edc5a8900d2327bcfe6ed768fa38d466fa9a390ce713320 offset: 0 snap -1 2024-09-18T11:30:15.699 INFO:tasks.rados.rados.0.smithi119.stdout:698: set_chunk oid 17 target oid 2024-09-18T11:30:15.701 INFO:tasks.rados.rados.0.smithi119.stdout:698: set_chunk oid 17 offset: 453120 length: 38400 target oid 5c9b37300155c1f3338bc88f17d69c51fb248b0de0155630962ca6504d789bba offset: 0 snap 2 2024-09-18T11:30:15.701 INFO:tasks.rados.rados.0.smithi119.stdout:699: read oid 18 snap -1 2024-09-18T11:30:15.701 INFO:tasks.rados.rados.0.smithi119.stdout:699: expect (ObjNum 60 snap 0 seq_num 60) 2024-09-18T11:30:15.701 INFO:tasks.rados.rados.0.smithi119.stdout:700: write oid 40 current snap is 3 2024-09-18T11:30:15.702 INFO:tasks.rados.rados.0.smithi119.stdout:692: finishing write tid 5 to smithi11939506-29 2024-09-18T11:30:15.702 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 15 (ObjNum 73 snap 3 seq_num 73) dirty exists 2024-09-18T11:30:15.702 INFO:tasks.rados.rados.0.smithi119.stdout:692: left oid 29 (ObjNum 73 snap 3 seq_num 73) 2024-09-18T11:30:15.702 INFO:tasks.rados.rados.0.smithi119.stdout:693: finishing write tid 1 to smithi11939506-13 2024-09-18T11:30:15.702 INFO:tasks.rados.rados.0.smithi119.stdout:693: finishing write tid 2 to smithi11939506-13 2024-09-18T11:30:15.702 INFO:tasks.rados.rados.0.smithi119.stdout:700: seq_num 76 ranges {477332=766101,1681102=474010,2687912=589420} 2024-09-18T11:30:15.707 INFO:tasks.rados.rados.0.smithi119.stdout:700: writing smithi11939506-40 from 477332 to 1243433 tid 1 2024-09-18T11:30:15.713 INFO:tasks.rados.rados.0.smithi119.stdout:700: writing smithi11939506-40 from 1681102 to 2155112 tid 2 2024-09-18T11:30:15.719 INFO:tasks.rados.rados.0.smithi119.stdout:700: writing smithi11939506-40 from 2687912 to 3277332 tid 3 2024-09-18T11:30:15.719 INFO:tasks.rados.rados.0.smithi119.stdout:692: done (8 left) 2024-09-18T11:30:15.720 INFO:tasks.rados.rados.0.smithi119.stdout:701: read oid 30 snap 2 2024-09-18T11:30:15.720 INFO:tasks.rados.rados.0.smithi119.stdout:701: expect (ObjNum 29 snap 0 seq_num 29) 2024-09-18T11:30:15.720 INFO:tasks.rados.rados.0.smithi119.stdout:702: write (excl) oid 3 current snap is 3 2024-09-18T11:30:15.720 INFO:tasks.rados.rados.0.smithi119.stdout:693: finishing write tid 3 to smithi11939506-13 2024-09-18T11:30:15.720 INFO:tasks.rados.rados.0.smithi119.stdout:693: finishing write tid 4 to smithi11939506-13 2024-09-18T11:30:15.720 INFO:tasks.rados.rados.0.smithi119.stdout:693: finishing write tid 5 to smithi11939506-13 2024-09-18T11:30:15.720 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 13 v 44 (ObjNum 74 snap 3 seq_num 74) dirty exists 2024-09-18T11:30:15.720 INFO:tasks.rados.rados.0.smithi119.stdout:693: left oid 13 (ObjNum 74 snap 3 seq_num 74) 2024-09-18T11:30:15.720 INFO:tasks.rados.rados.0.smithi119.stdout:694: finishing write tid 1 to smithi11939506-50 2024-09-18T11:30:15.720 INFO:tasks.rados.rados.0.smithi119.stdout:694: finishing write tid 2 to smithi11939506-50 2024-09-18T11:30:15.720 INFO:tasks.rados.rados.0.smithi119.stdout:694: finishing write tid 3 to smithi11939506-50 2024-09-18T11:30:15.720 INFO:tasks.rados.rados.0.smithi119.stdout:694: finishing write tid 4 to smithi11939506-50 2024-09-18T11:30:15.720 INFO:tasks.rados.rados.0.smithi119.stdout:694: finishing write tid 5 to smithi11939506-50 2024-09-18T11:30:15.721 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 31 (ObjNum 75 snap 3 seq_num 75) dirty exists 2024-09-18T11:30:15.721 INFO:tasks.rados.rados.0.smithi119.stdout:694: left oid 50 (ObjNum 75 snap 3 seq_num 75) 2024-09-18T11:30:15.721 INFO:tasks.rados.rados.0.smithi119.stdout:695: expect (ObjNum 24 snap 0 seq_num 24) 2024-09-18T11:30:15.726 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:41 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'19)", 2024-09-18T11:30:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.158446+0000", 2024-09-18T11:30:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.56608048, 2024-09-18T11:30:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028837609999999999, 2024-09-18T11:30:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:15.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158446+0000", 2024-09-18T11:30:15.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:15.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158446+0000", 2024-09-18T11:30:15.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:15.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158446+0000", 2024-09-18T11:30:15.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158513+0000", 2024-09-18T11:30:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7379999999999995e-05 2024-09-18T11:30:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158520+0000", 2024-09-18T11:30:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0149999999999998e-06 2024-09-18T11:30:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158524+0000", 2024-09-18T11:30:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4400000000000001e-06 2024-09-18T11:30:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158551+0000", 2024-09-18T11:30:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7331e-05 2024-09-18T11:30:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.158562+0000", 2024-09-18T11:30:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0951999999999999e-05 2024-09-18T11:30:15.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.161297+0000", 2024-09-18T11:30:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027348810000000002 2024-09-18T11:30:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.161330+0000", 2024-09-18T11:30:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3104e-05 2024-09-18T11:30:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:77 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'55, mlcod=15'53)", 2024-09-18T11:30:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.584563+0000", 2024-09-18T11:30:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.139963948999998, 2024-09-18T11:30:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037297200000000002, 2024-09-18T11:30:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584563+0000", 2024-09-18T11:30:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584563+0000", 2024-09-18T11:30:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584562+0000", 2024-09-18T11:30:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584613+0000", 2024-09-18T11:30:15.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0892e-05 2024-09-18T11:30:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584648+0000", 2024-09-18T11:30:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4533999999999998e-05 2024-09-18T11:30:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584653+0000", 2024-09-18T11:30:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9749999999999998e-06 2024-09-18T11:30:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584687+0000", 2024-09-18T11:30:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4453999999999999e-05 2024-09-18T11:30:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.584699+0000", 2024-09-18T11:30:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1605e-05 2024-09-18T11:30:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.588269+0000", 2024-09-18T11:30:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035706650000000002 2024-09-18T11:30:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.588293+0000", 2024-09-18T11:30:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3226000000000001e-05 2024-09-18T11:30:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:78 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'56, mlcod=15'53)", 2024-09-18T11:30:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.585042+0000", 2024-09-18T11:30:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.139484878000001, 2024-09-18T11:30:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032615389999999999, 2024-09-18T11:30:15.736 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585042+0000", 2024-09-18T11:30:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585042+0000", 2024-09-18T11:30:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:15.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585041+0000", 2024-09-18T11:30:15.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:15.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585129+0000", 2024-09-18T11:30:15.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7905000000000006e-05 2024-09-18T11:30:15.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:15.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585138+0000", 2024-09-18T11:30:15.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9069999999999996e-06 2024-09-18T11:30:15.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585143+0000", 2024-09-18T11:30:15.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5329999999999996e-06 2024-09-18T11:30:15.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585166+0000", 2024-09-18T11:30:15.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3308000000000001e-05 2024-09-18T11:30:15.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:15.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.585177+0000", 2024-09-18T11:30:15.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1255e-05 2024-09-18T11:30:15.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:15.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.588294+0000", 2024-09-18T11:30:15.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031165989999999998 2024-09-18T11:30:15.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:15.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.588303+0000", 2024-09-18T11:30:15.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5279999999999992e-06 2024-09-18T11:30:15.741 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.741 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:15.741 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.741 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:79 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'57, mlcod=15'53)", 2024-09-18T11:30:15.741 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.586176+0000", 2024-09-18T11:30:15.741 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.13835057, 2024-09-18T11:30:15.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043147059999999997, 2024-09-18T11:30:15.741 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:15.741 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.741 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:15.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:15.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586176+0000", 2024-09-18T11:30:15.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:15.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586176+0000", 2024-09-18T11:30:15.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:15.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586170+0000", 2024-09-18T11:30:15.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999938 2024-09-18T11:30:15.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:15.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586282+0000", 2024-09-18T11:30:15.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112254 2024-09-18T11:30:15.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:15.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586307+0000", 2024-09-18T11:30:15.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5296e-05 2024-09-18T11:30:15.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586321+0000", 2024-09-18T11:30:15.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3723000000000001e-05 2024-09-18T11:30:15.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.587754+0000", 2024-09-18T11:30:15.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014330829999999999 2024-09-18T11:30:15.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:15.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.587785+0000", 2024-09-18T11:30:15.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1176000000000001e-05 2024-09-18T11:30:15.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:15.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.590467+0000", 2024-09-18T11:30:15.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026815089999999999 2024-09-18T11:30:15.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:15.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.590491+0000", 2024-09-18T11:30:15.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3941e-05 2024-09-18T11:30:15.745 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.745 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:15.745 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.746 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:80 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'58, mlcod=15'53)", 2024-09-18T11:30:15.746 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.586532+0000", 2024-09-18T11:30:15.746 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.137994852999999, 2024-09-18T11:30:15.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041534989999999997, 2024-09-18T11:30:15.746 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:15.746 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.746 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:15.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:15.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586532+0000", 2024-09-18T11:30:15.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:15.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586532+0000", 2024-09-18T11:30:15.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:15.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586531+0000", 2024-09-18T11:30:15.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:15.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586591+0000", 2024-09-18T11:30:15.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9472000000000003e-05 2024-09-18T11:30:15.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:15.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586598+0000", 2024-09-18T11:30:15.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0890000000000002e-06 2024-09-18T11:30:15.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586602+0000", 2024-09-18T11:30:15.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0090000000000001e-06 2024-09-18T11:30:15.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.588075+0000", 2024-09-18T11:30:15.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014728429999999999 2024-09-18T11:30:15.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:15.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.588086+0000", 2024-09-18T11:30:15.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1452e-05 2024-09-18T11:30:15.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:15.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.590672+0000", 2024-09-18T11:30:15.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025858629999999999 2024-09-18T11:30:15.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:15.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.590685+0000", 2024-09-18T11:30:15.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3254999999999999e-05 2024-09-18T11:30:15.750 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.750 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:15.750 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.750 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:88 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'62, mlcod=15'60)", 2024-09-18T11:30:15.750 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:50.031675+0000", 2024-09-18T11:30:15.750 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 25.69285129, 2024-09-18T11:30:15.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035383889999999999, 2024-09-18T11:30:15.751 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:15.751 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.751 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:15.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:15.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031675+0000", 2024-09-18T11:30:15.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:15.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031675+0000", 2024-09-18T11:30:15.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:15.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031675+0000", 2024-09-18T11:30:15.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:15.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031682+0000", 2024-09-18T11:30:15.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.542e-06 2024-09-18T11:30:15.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:15.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031692+0000", 2024-09-18T11:30:15.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7119999999999995e-06 2024-09-18T11:30:15.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031700+0000", 2024-09-18T11:30:15.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7819999999999994e-06 2024-09-18T11:30:15.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031738+0000", 2024-09-18T11:30:15.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8538000000000003e-05 2024-09-18T11:30:15.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:15.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.031757+0000", 2024-09-18T11:30:15.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8258999999999999e-05 2024-09-18T11:30:15.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:15.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035185+0000", 2024-09-18T11:30:15.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034287219999999999 2024-09-18T11:30:15.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035214+0000", 2024-09-18T11:30:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8524e-05 2024-09-18T11:30:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:89 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'63, mlcod=15'60)", 2024-09-18T11:30:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:50.032044+0000", 2024-09-18T11:30:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 25.692482556000002, 2024-09-18T11:30:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032268829999999998, 2024-09-18T11:30:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:15.755 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.756 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:15.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:15.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032044+0000", 2024-09-18T11:30:15.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:15.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032044+0000", 2024-09-18T11:30:15.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:15.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032044+0000", 2024-09-18T11:30:15.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:15.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032049+0000", 2024-09-18T11:30:15.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.78e-06 2024-09-18T11:30:15.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:15.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032057+0000", 2024-09-18T11:30:15.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5879999999999998e-06 2024-09-18T11:30:15.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032061+0000", 2024-09-18T11:30:15.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0820000000000001e-06 2024-09-18T11:30:15.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032083+0000", 2024-09-18T11:30:15.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1359999999999999e-05 2024-09-18T11:30:15.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:15.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032093+0000", 2024-09-18T11:30:15.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0662999999999999e-05 2024-09-18T11:30:15.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:15.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035250+0000", 2024-09-18T11:30:15.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031568429999999999 2024-09-18T11:30:15.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:15.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035271+0000", 2024-09-18T11:30:15.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1053999999999999e-05 2024-09-18T11:30:15.760 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.760 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:15.760 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.760 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:90 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'64, mlcod=15'60)", 2024-09-18T11:30:15.760 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:50.032410+0000", 2024-09-18T11:30:15.760 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 25.692116904999999, 2024-09-18T11:30:15.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029831089999999998, 2024-09-18T11:30:15.760 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:15.760 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.760 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:15.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:15.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032410+0000", 2024-09-18T11:30:15.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:15.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032410+0000", 2024-09-18T11:30:15.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:15.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032410+0000", 2024-09-18T11:30:15.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:15.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032534+0000", 2024-09-18T11:30:15.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012397600000000001 2024-09-18T11:30:15.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:15.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032540+0000", 2024-09-18T11:30:15.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1500000000000004e-06 2024-09-18T11:30:15.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032544+0000", 2024-09-18T11:30:15.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2910000000000001e-06 2024-09-18T11:30:15.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032576+0000", 2024-09-18T11:30:15.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2228999999999999e-05 2024-09-18T11:30:15.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:15.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.032597+0000", 2024-09-18T11:30:15.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0579000000000001e-05 2024-09-18T11:30:15.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:15.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035344+0000", 2024-09-18T11:30:15.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002747527 2024-09-18T11:30:15.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:15.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.035393+0000", 2024-09-18T11:30:15.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8677999999999997e-05 2024-09-18T11:30:15.764 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.764 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:15.764 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.765 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:21 3.1 3:9f1cb8cf:::smithi11939506-38 low_tier:head [read 336384~173568 [fadvise_sequential]] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:15.765 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.118534+0000", 2024-09-18T11:30:15.765 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.6059923789999999, 2024-09-18T11:30:15.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006633842, 2024-09-18T11:30:15.765 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:15.765 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:15.765 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:15.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:15.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118534+0000", 2024-09-18T11:30:15.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:15.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118534+0000", 2024-09-18T11:30:15.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:15.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118532+0000", 2024-09-18T11:30:15.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:15.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:15.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118542+0000", 2024-09-18T11:30:15.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0151e-05 2024-09-18T11:30:15.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:15.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118563+0000", 2024-09-18T11:30:15.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1089000000000001e-05 2024-09-18T11:30:15.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118572+0000", 2024-09-18T11:30:15.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1090000000000004e-06 2024-09-18T11:30:15.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118633+0000", 2024-09-18T11:30:15.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0940000000000003e-05 2024-09-18T11:30:15.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:15.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118640+0000", 2024-09-18T11:30:15.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3690000000000003e-06 2024-09-18T11:30:15.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124910+0000", 2024-09-18T11:30:15.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062697569999999999 2024-09-18T11:30:15.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:15.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124980+0000", 2024-09-18T11:30:15.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0846000000000006e-05 2024-09-18T11:30:15.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:15.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.125168+0000", 2024-09-18T11:30:15.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018786200000000001 2024-09-18T11:30:15.769 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.769 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:15.769 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.769 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:22 3.1 3:9f1cb8cf:::smithi11939506-38 low_tier:head [read 1235456~1340416 [fadvise_sequential]] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:15.769 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:15.770 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.6059396509999999, 2024-09-18T11:30:15.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0090773199999999998, 2024-09-18T11:30:15.770 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:15.770 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:15.770 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:15.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:15.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:15.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:15.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:15.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:15.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:15.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:15.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118594+0000", 2024-09-18T11:30:15.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1590000000000004e-06 2024-09-18T11:30:15.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:15.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118601+0000", 2024-09-18T11:30:15.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3900000000000004e-06 2024-09-18T11:30:15.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118605+0000", 2024-09-18T11:30:15.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2479999999999998e-06 2024-09-18T11:30:15.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118697+0000", 2024-09-18T11:30:15.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1447999999999996e-05 2024-09-18T11:30:15.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:15.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118702+0000", 2024-09-18T11:30:15.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6670000000000001e-06 2024-09-18T11:30:15.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.125285+0000", 2024-09-18T11:30:15.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006582124 2024-09-18T11:30:15.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:15.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.125349+0000", 2024-09-18T11:30:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4777000000000007e-05 2024-09-18T11:30:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.127664+0000", 2024-09-18T11:30:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023151069999999998 2024-09-18T11:30:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:910 4.3 4:cc46532a:::smithi11939506-33:head [write 759683~639358,stat] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.159388+0000", 2024-09-18T11:30:15.774 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5651392670000002, 2024-09-18T11:30:15.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003341344, 2024-09-18T11:30:15.775 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:15.775 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.775 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:15.775 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:15.775 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:15.775 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 910 2024-09-18T11:30:15.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.775 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:15.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:15.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.159388+0000", 2024-09-18T11:30:15.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:15.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.159388+0000", 2024-09-18T11:30:15.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:15.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.159385+0000", 2024-09-18T11:30:15.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:15.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.159676+0000", 2024-09-18T11:30:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029050000000000001 2024-09-18T11:30:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.159693+0000", 2024-09-18T11:30:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7450999999999999e-05 2024-09-18T11:30:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.159699+0000", 2024-09-18T11:30:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8749999999999997e-06 2024-09-18T11:30:15.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.159742+0000", 2024-09-18T11:30:15.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2855000000000002e-05 2024-09-18T11:30:15.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:15.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.159796+0000", 2024-09-18T11:30:15.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4517e-05 2024-09-18T11:30:15.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:15.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.159910+0000", 2024-09-18T11:30:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011401899999999999 2024-09-18T11:30:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.162152+0000", 2024-09-18T11:30:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022414230000000002 2024-09-18T11:30:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.162674+0000", 2024-09-18T11:30:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00052174100000000005 2024-09-18T11:30:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.162704+0000", 2024-09-18T11:30:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0555999999999997e-05 2024-09-18T11:30:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.162729+0000", 2024-09-18T11:30:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4672999999999999e-05 2024-09-18T11:30:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.780 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:911 4.3 4:cc46532a:::smithi11939506-33:head [write 1904434~719345] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.166081+0000", 2024-09-18T11:30:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5584455850000003, 2024-09-18T11:30:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081214100000000008, 2024-09-18T11:30:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 911 2024-09-18T11:30:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:15.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166081+0000", 2024-09-18T11:30:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166081+0000", 2024-09-18T11:30:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166080+0000", 2024-09-18T11:30:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166386+0000", 2024-09-18T11:30:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030589499999999998 2024-09-18T11:30:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166398+0000", 2024-09-18T11:30:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1678000000000001e-05 2024-09-18T11:30:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166402+0000", 2024-09-18T11:30:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6059999999999996e-06 2024-09-18T11:30:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166443+0000", 2024-09-18T11:30:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1044000000000002e-05 2024-09-18T11:30:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166526+0000", 2024-09-18T11:30:15.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2281999999999996e-05 2024-09-18T11:30:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166661+0000", 2024-09-18T11:30:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013516299999999999 2024-09-18T11:30:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.168437+0000", 2024-09-18T11:30:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001775794 2024-09-18T11:30:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174126+0000", 2024-09-18T11:30:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056894859999999997 2024-09-18T11:30:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174166+0000", 2024-09-18T11:30:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9787999999999999e-05 2024-09-18T11:30:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174203+0000", 2024-09-18T11:30:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6644999999999999e-05 2024-09-18T11:30:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.786 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:912 4.3 4:cc46532a:::smithi11939506-33:head [write 3128657~31026] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169139+0000", 2024-09-18T11:30:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5553879110000004, 2024-09-18T11:30:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051669280000000003, 2024-09-18T11:30:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:15.787 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 912 2024-09-18T11:30:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169139+0000", 2024-09-18T11:30:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169139+0000", 2024-09-18T11:30:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169138+0000", 2024-09-18T11:30:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169176+0000", 2024-09-18T11:30:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7679000000000003e-05 2024-09-18T11:30:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169187+0000", 2024-09-18T11:30:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1353e-05 2024-09-18T11:30:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169193+0000", 2024-09-18T11:30:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0390000000000001e-06 2024-09-18T11:30:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169230+0000", 2024-09-18T11:30:15.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6397999999999999e-05 2024-09-18T11:30:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169302+0000", 2024-09-18T11:30:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2922000000000004e-05 2024-09-18T11:30:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169426+0000", 2024-09-18T11:30:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012302600000000001 2024-09-18T11:30:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:15.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174078+0000", 2024-09-18T11:30:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046524330000000001 2024-09-18T11:30:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174240+0000", 2024-09-18T11:30:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016188899999999999 2024-09-18T11:30:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174275+0000", 2024-09-18T11:30:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5556000000000003e-05 2024-09-18T11:30:15.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174306+0000", 2024-09-18T11:30:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0408000000000001e-05 2024-09-18T11:30:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:913 4.3 4:cc46532a:::smithi11939506-33:head [setxattr _header (56) in=7b,truncate 3159683] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:15.793 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169203+0000", 2024-09-18T11:30:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5553241460000002, 2024-09-18T11:30:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052201790000000001, 2024-09-18T11:30:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 913 2024-09-18T11:30:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169203+0000", 2024-09-18T11:30:15.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169203+0000", 2024-09-18T11:30:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169202+0000", 2024-09-18T11:30:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169206+0000", 2024-09-18T11:30:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8160000000000004e-06 2024-09-18T11:30:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169212+0000", 2024-09-18T11:30:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3770000000000001e-06 2024-09-18T11:30:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169216+0000", 2024-09-18T11:30:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0060000000000003e-06 2024-09-18T11:30:15.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169888+0000", 2024-09-18T11:30:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00067139700000000001 2024-09-18T11:30:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169963+0000", 2024-09-18T11:30:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4999000000000005e-05 2024-09-18T11:30:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.170077+0000", 2024-09-18T11:30:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011401899999999999 2024-09-18T11:30:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174084+0000", 2024-09-18T11:30:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040071550000000001 2024-09-18T11:30:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174336+0000", 2024-09-18T11:30:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00025225300000000003 2024-09-18T11:30:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174371+0000", 2024-09-18T11:30:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5361000000000002e-05 2024-09-18T11:30:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174423+0000", 2024-09-18T11:30:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1347999999999997e-05 2024-09-18T11:30:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:914 4.3 4:cc46532a:::smithi11939506-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:15.799 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169236+0000", 2024-09-18T11:30:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5552906259999997, 2024-09-18T11:30:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005369965, 2024-09-18T11:30:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 914 2024-09-18T11:30:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:15.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169236+0000", 2024-09-18T11:30:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169236+0000", 2024-09-18T11:30:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169236+0000", 2024-09-18T11:30:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169241+0000", 2024-09-18T11:30:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8239999999999996e-06 2024-09-18T11:30:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169249+0000", 2024-09-18T11:30:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9319999999999996e-06 2024-09-18T11:30:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169254+0000", 2024-09-18T11:30:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3220000000000003e-06 2024-09-18T11:30:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.170304+0000", 2024-09-18T11:30:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001050184 2024-09-18T11:30:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.170390+0000", 2024-09-18T11:30:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5839000000000003e-05 2024-09-18T11:30:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174455+0000", 2024-09-18T11:30:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040656119999999997 2024-09-18T11:30:15.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174525+0000", 2024-09-18T11:30:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9151000000000003e-05 2024-09-18T11:30:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174606+0000", 2024-09-18T11:30:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1588000000000006e-05 2024-09-18T11:30:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:922 4.b 4:db5b7794:::smithi11939506-13:head [read 0~2873964,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169355+0000", 2024-09-18T11:30:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5551720800000002, 2024-09-18T11:30:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032492969999999999, 2024-09-18T11:30:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 922 2024-09-18T11:30:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:15.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169355+0000", 2024-09-18T11:30:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169355+0000", 2024-09-18T11:30:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169354+0000", 2024-09-18T11:30:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:15.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169361+0000", 2024-09-18T11:30:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.328e-06 2024-09-18T11:30:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169371+0000", 2024-09-18T11:30:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3870000000000001e-06 2024-09-18T11:30:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169375+0000", 2024-09-18T11:30:15.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6229999999999996e-06 2024-09-18T11:30:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.170253+0000", 2024-09-18T11:30:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00087807 2024-09-18T11:30:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.170327+0000", 2024-09-18T11:30:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3423999999999994e-05 2024-09-18T11:30:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:15.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.172604+0000", 2024-09-18T11:30:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022772220000000002 2024-09-18T11:30:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:923 4.b 4:db5b7794:::smithi11939506-13:head [read 0~2873964,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169380+0000", 2024-09-18T11:30:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5551465020000004, 2024-09-18T11:30:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047044239999999996, 2024-09-18T11:30:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:15.809 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 923 2024-09-18T11:30:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169380+0000", 2024-09-18T11:30:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169380+0000", 2024-09-18T11:30:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169380+0000", 2024-09-18T11:30:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:15.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169383+0000", 2024-09-18T11:30:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1429999999999998e-06 2024-09-18T11:30:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169389+0000", 2024-09-18T11:30:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4579999999999998e-06 2024-09-18T11:30:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169393+0000", 2024-09-18T11:30:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7840000000000001e-06 2024-09-18T11:30:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.172647+0000", 2024-09-18T11:30:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032532849999999999 2024-09-18T11:30:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.172724+0000", 2024-09-18T11:30:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7836999999999994e-05 2024-09-18T11:30:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174085+0000", 2024-09-18T11:30:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013603580000000001 2024-09-18T11:30:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:15.813 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:924 4.b 4:db5b7794:::smithi11939506-13:head [sparse-read 0~1546056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169399+0000", 2024-09-18T11:30:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5551281580000005, 2024-09-18T11:30:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054498719999999997, 2024-09-18T11:30:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 924 2024-09-18T11:30:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169399+0000", 2024-09-18T11:30:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169399+0000", 2024-09-18T11:30:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169398+0000", 2024-09-18T11:30:15.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169401+0000", 2024-09-18T11:30:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4490000000000002e-06 2024-09-18T11:30:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169407+0000", 2024-09-18T11:30:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9490000000000001e-06 2024-09-18T11:30:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169411+0000", 2024-09-18T11:30:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.924e-06 2024-09-18T11:30:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174189+0000", 2024-09-18T11:30:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047787039999999999 2024-09-18T11:30:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174263+0000", 2024-09-18T11:30:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3301999999999997e-05 2024-09-18T11:30:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174849+0000", 2024-09-18T11:30:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00058598500000000002 2024-09-18T11:30:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:989 4.3 4:ca2b2309:::smithi11939506-44:head [write 410104~465105,stat] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.151379+0000", 2024-09-18T11:30:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.57314762100000005, 2024-09-18T11:30:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062831420000000002, 2024-09-18T11:30:15.818 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 989 2024-09-18T11:30:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151379+0000", 2024-09-18T11:30:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151379+0000", 2024-09-18T11:30:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151376+0000", 2024-09-18T11:30:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151666+0000", 2024-09-18T11:30:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028977899999999999 2024-09-18T11:30:15.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151697+0000", 2024-09-18T11:30:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0516000000000001e-05 2024-09-18T11:30:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151707+0000", 2024-09-18T11:30:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0322999999999999e-05 2024-09-18T11:30:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151770+0000", 2024-09-18T11:30:15.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879000000000006e-05 2024-09-18T11:30:15.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:15.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151783+0000", 2024-09-18T11:30:15.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3154e-05 2024-09-18T11:30:15.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.153878+0000", 2024-09-18T11:30:15.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002095155 2024-09-18T11:30:15.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:15.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.153957+0000", 2024-09-18T11:30:15.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8877999999999999e-05 2024-09-18T11:30:15.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:15.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.154073+0000", 2024-09-18T11:30:15.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116045 2024-09-18T11:30:15.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:15.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.156031+0000", 2024-09-18T11:30:15.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019581099999999999 2024-09-18T11:30:15.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.157569+0000", 2024-09-18T11:30:15.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015383199999999999 2024-09-18T11:30:15.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:15.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.157606+0000", 2024-09-18T11:30:15.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6278000000000001e-05 2024-09-18T11:30:15.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:15.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.157662+0000", 2024-09-18T11:30:15.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6589000000000003e-05 2024-09-18T11:30:15.824 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.824 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:15.824 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.825 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:990 4.3 4:ca2b2309:::smithi11939506-44:head [write 1563527~583025] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:15.825 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.158301+0000", 2024-09-18T11:30:15.825 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.56622611099999998, 2024-09-18T11:30:15.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049105670000000002, 2024-09-18T11:30:15.825 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:15.825 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:15.825 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:15.825 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:15.825 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:15.825 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 990 2024-09-18T11:30:15.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.825 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:15.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:15.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158301+0000", 2024-09-18T11:30:15.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:15.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158301+0000", 2024-09-18T11:30:15.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:15.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:15.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158299+0000", 2024-09-18T11:30:15.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:15.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:15.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158457+0000", 2024-09-18T11:30:15.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000157374 2024-09-18T11:30:15.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:15.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158469+0000", 2024-09-18T11:30:15.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1969e-05 2024-09-18T11:30:15.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:15.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158475+0000", 2024-09-18T11:30:15.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4450000000000001e-06 2024-09-18T11:30:15.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:15.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158504+0000", 2024-09-18T11:30:15.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8924999999999998e-05 2024-09-18T11:30:15.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:15.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158568+0000", 2024-09-18T11:30:15.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3781000000000005e-05 2024-09-18T11:30:15.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:15.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158677+0000", 2024-09-18T11:30:15.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010893899999999999 2024-09-18T11:30:15.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:15.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.160238+0000", 2024-09-18T11:30:15.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015608340000000001 2024-09-18T11:30:15.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:15.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.163135+0000", 2024-09-18T11:30:15.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028975400000000001 2024-09-18T11:30:15.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:15.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.163166+0000", 2024-09-18T11:30:15.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1164999999999997e-05 2024-09-18T11:30:15.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:15.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:15.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:15.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.163211+0000", 2024-09-18T11:30:15.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4836000000000002e-05 2024-09-18T11:30:15.830 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.830 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:15.830 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.830 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:15.830 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:15.830 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:15.832 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:30:15.832 INFO:tasks.rados.rados.0.smithi119.stdout:696: expect (ObjNum 70 snap 2 seq_num 70) 2024-09-18T11:30:15.833 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:15.830+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:15.933 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:15.932+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:15.939 INFO:tasks.rados.rados.0.smithi119.stdout:697: finishing set_chunk to oid 38 2024-09-18T11:30:15.939 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 38 set_chunk returned error code -95 offset: 2076160 length: 356864 2024-09-18T11:30:15.939 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 0 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-18T11:30:15.939 INFO:tasks.rados.rados.0.smithi119.stdout:698: finishing set_chunk to oid 17 2024-09-18T11:30:15.940 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 17 set_chunk returned error code -95 offset: 453120 length: 38400 2024-09-18T11:30:15.940 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 0 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-18T11:30:15.940 INFO:tasks.rados.rados.0.smithi119.stdout:699: expect (ObjNum 60 snap 0 seq_num 60) 2024-09-18T11:30:15.973 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:15.973 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:15.973 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:15.973 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:15.981 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:30:16.034 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:16.032+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:16.076 INFO:tasks.rados.rados.0.smithi119.stdout:700: finishing write tid 1 to smithi11939506-40 2024-09-18T11:30:16.076 INFO:tasks.rados.rados.0.smithi119.stdout:700: finishing write tid 2 to smithi11939506-40 2024-09-18T11:30:16.076 INFO:tasks.rados.rados.0.smithi119.stdout:700: finishing write tid 3 to smithi11939506-40 2024-09-18T11:30:16.077 INFO:tasks.rados.rados.0.smithi119.stdout:701: expect (ObjNum 29 snap 0 seq_num 29) 2024-09-18T11:30:16.080 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:16.081 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:16.081 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:16.081 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:16.081 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:16.088 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:30:16.135 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:16.134+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:16.161 INFO:tasks.rados.rados.0.smithi119.stdout:700: finishing write tid 4 to smithi11939506-40 2024-09-18T11:30:16.161 INFO:tasks.rados.rados.0.smithi119.stdout:700: finishing write tid 5 to smithi11939506-40 2024-09-18T11:30:16.161 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 36 (ObjNum 76 snap 3 seq_num 76) dirty exists 2024-09-18T11:30:16.161 INFO:tasks.rados.rados.0.smithi119.stdout:700: left oid 40 (ObjNum 76 snap 3 seq_num 76) 2024-09-18T11:30:16.162 INFO:tasks.rados.rados.0.smithi119.stdout:702: seq_num 77 ranges {494139=557703,1455937=734021,2948041=346098} 2024-09-18T11:30:16.166 INFO:tasks.rados.rados.0.smithi119.stdout:702: writing smithi11939506-3 from 494139 to 1051842 tid 1 2024-09-18T11:30:16.173 INFO:tasks.rados.rados.0.smithi119.stdout:702: writing smithi11939506-3 from 1455937 to 2189958 tid 2 2024-09-18T11:30:16.180 INFO:tasks.rados.rados.0.smithi119.stdout:702: writing smithi11939506-3 from 2948041 to 3294139 tid 3 2024-09-18T11:30:16.180 INFO:tasks.rados.rados.0.smithi119.stdout:693: done (9 left) 2024-09-18T11:30:16.180 INFO:tasks.rados.rados.0.smithi119.stdout:694: done (8 left) 2024-09-18T11:30:16.180 INFO:tasks.rados.rados.0.smithi119.stdout:695: done (7 left) 2024-09-18T11:30:16.180 INFO:tasks.rados.rados.0.smithi119.stdout:696: done (6 left) 2024-09-18T11:30:16.180 INFO:tasks.rados.rados.0.smithi119.stdout:697: done (5 left) 2024-09-18T11:30:16.180 INFO:tasks.rados.rados.0.smithi119.stdout:698: done (4 left) 2024-09-18T11:30:16.181 INFO:tasks.rados.rados.0.smithi119.stdout:699: done (3 left) 2024-09-18T11:30:16.181 INFO:tasks.rados.rados.0.smithi119.stdout:700: done (2 left) 2024-09-18T11:30:16.181 INFO:tasks.rados.rados.0.smithi119.stdout:701: done (1 left) 2024-09-18T11:30:16.181 INFO:tasks.rados.rados.0.smithi119.stdout:703: write oid 50 current snap is 3 2024-09-18T11:30:16.181 INFO:tasks.rados.rados.0.smithi119.stdout:702: finishing write tid 1 to smithi11939506-3 2024-09-18T11:30:16.181 INFO:tasks.rados.rados.0.smithi119.stdout:702: finishing write tid 2 to smithi11939506-3 2024-09-18T11:30:16.181 INFO:tasks.rados.rados.0.smithi119.stdout:703: seq_num 78 ranges {510946=632952,1798066=477679,2742758=457217} 2024-09-18T11:30:16.185 INFO:tasks.rados.rados.0.smithi119.stdout:703: writing smithi11939506-50 from 510946 to 1143898 tid 1 2024-09-18T11:30:16.192 INFO:tasks.rados.rados.0.smithi119.stdout:703: writing smithi11939506-50 from 1798066 to 2275745 tid 2 2024-09-18T11:30:16.197 INFO:tasks.rados.rados.0.smithi119.stdout:703: writing smithi11939506-50 from 2742758 to 3199975 tid 3 2024-09-18T11:30:16.197 INFO:tasks.rados.rados.0.smithi119.stdout:704: snap_create 2024-09-18T11:30:16.198 INFO:tasks.rados.rados.0.smithi119.stdout:702: finishing write tid 3 to smithi11939506-3 2024-09-18T11:30:16.198 INFO:tasks.rados.rados.0.smithi119.stdout:702: finishing write tid 4 to smithi11939506-3 2024-09-18T11:30:16.198 INFO:tasks.rados.rados.0.smithi119.stdout:702: finishing write tid 5 to smithi11939506-3 2024-09-18T11:30:16.198 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 26 (ObjNum 77 snap 3 seq_num 77) dirty exists 2024-09-18T11:30:16.198 INFO:tasks.rados.rados.0.smithi119.stdout:702: left oid 3 (ObjNum 77 snap 3 seq_num 77) 2024-09-18T11:30:16.198 INFO:tasks.rados.rados.0.smithi119.stdout:703: finishing write tid 1 to smithi11939506-50 2024-09-18T11:30:16.198 INFO:tasks.rados.rados.0.smithi119.stdout:703: finishing write tid 2 to smithi11939506-50 2024-09-18T11:30:16.200 INFO:tasks.rados.rados.0.smithi119.stdout:703: finishing write tid 3 to smithi11939506-50 2024-09-18T11:30:16.200 INFO:tasks.rados.rados.0.smithi119.stdout:703: finishing write tid 4 to smithi11939506-50 2024-09-18T11:30:16.200 INFO:tasks.rados.rados.0.smithi119.stdout:703: finishing write tid 5 to smithi11939506-50 2024-09-18T11:30:16.200 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 40 (ObjNum 78 snap 3 seq_num 78) dirty exists 2024-09-18T11:30:16.200 INFO:tasks.rados.rados.0.smithi119.stdout:703: left oid 50 (ObjNum 78 snap 3 seq_num 78) 2024-09-18T11:30:16.234 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:16.234 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:16.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:16.234 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:16.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.234 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4266.0:7 2.2 e18/15 2:5cea7035:::rbd_info:head v 18'2, mlcod=0'0)", 2024-09-18T11:30:16.235 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:52.048021+0000", 2024-09-18T11:30:16.235 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.183814430000002, 2024-09-18T11:30:16.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0030933409999999999, 2024-09-18T11:30:16.235 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.235 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:16.235 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048021+0000", 2024-09-18T11:30:16.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048021+0000", 2024-09-18T11:30:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048020+0000", 2024-09-18T11:30:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:30:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048025+0000", 2024-09-18T11:30:16.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1949999999999996e-06 2024-09-18T11:30:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048035+0000", 2024-09-18T11:30:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9639999999999998e-06 2024-09-18T11:30:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048041+0000", 2024-09-18T11:30:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6150000000000002e-06 2024-09-18T11:30:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048082+0000", 2024-09-18T11:30:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0453e-05 2024-09-18T11:30:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.048100+0000", 2024-09-18T11:30:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8556999999999999e-05 2024-09-18T11:30:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.051080+0000", 2024-09-18T11:30:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029795440000000002 2024-09-18T11:30:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:52.051114+0000", 2024-09-18T11:30:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4218999999999999e-05 2024-09-18T11:30:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:887 4.7 4:e0d95e10:::smithi11939506-22:head [read 0~2352947,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:30:16.239 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.115468141, 2024-09-18T11:30:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079115169999999999, 2024-09-18T11:30:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 887 2024-09-18T11:30:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:30:16.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:30:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116365+0000", 2024-09-18T11:30:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116376+0000", 2024-09-18T11:30:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1471000000000001e-05 2024-09-18T11:30:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116394+0000", 2024-09-18T11:30:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8088e-05 2024-09-18T11:30:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116402+0000", 2024-09-18T11:30:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2320000000000004e-06 2024-09-18T11:30:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116458+0000", 2024-09-18T11:30:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6536000000000002e-05 2024-09-18T11:30:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116466+0000", 2024-09-18T11:30:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0819999999999999e-06 2024-09-18T11:30:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.122509+0000", 2024-09-18T11:30:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060431879999999997 2024-09-18T11:30:16.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.122577+0000", 2024-09-18T11:30:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7646999999999998e-05 2024-09-18T11:30:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124278+0000", 2024-09-18T11:30:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017013200000000001 2024-09-18T11:30:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:888 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.1154146779999996, 2024-09-18T11:30:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082282120000000004, 2024-09-18T11:30:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 888 2024-09-18T11:30:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:16.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116423+0000", 2024-09-18T11:30:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1e-06 2024-09-18T11:30:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:30:16.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5510000000000001e-06 2024-09-18T11:30:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116431+0000", 2024-09-18T11:30:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6109999999999998e-06 2024-09-18T11:30:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116527+0000", 2024-09-18T11:30:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6212999999999995e-05 2024-09-18T11:30:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:16.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116532+0000", 2024-09-18T11:30:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6140000000000002e-06 2024-09-18T11:30:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124314+0000", 2024-09-18T11:30:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077818929999999998 2024-09-18T11:30:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124411+0000", 2024-09-18T11:30:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6992999999999998e-05 2024-09-18T11:30:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124649+0000", 2024-09-18T11:30:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237729 2024-09-18T11:30:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:889 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.1153900280000002, 2024-09-18T11:30:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085383349999999993, 2024-09-18T11:30:16.250 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 889 2024-09-18T11:30:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116447+0000", 2024-09-18T11:30:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.854e-06 2024-09-18T11:30:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116452+0000", 2024-09-18T11:30:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0250000000000002e-06 2024-09-18T11:30:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116455+0000", 2024-09-18T11:30:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8739999999999999e-06 2024-09-18T11:30:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116565+0000", 2024-09-18T11:30:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110741 2024-09-18T11:30:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116571+0000", 2024-09-18T11:30:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4990000000000002e-06 2024-09-18T11:30:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124691+0000", 2024-09-18T11:30:16.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081200339999999999 2024-09-18T11:30:16.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124784+0000", 2024-09-18T11:30:16.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2886999999999993e-05 2024-09-18T11:30:16.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124983+0000", 2024-09-18T11:30:16.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019969000000000001 2024-09-18T11:30:16.255 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.255 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.255 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.256 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:46 4.a e31/22 4:552523b6:::smithi11939506-38:head v 31'20, mlcod=30'18)", 2024-09-18T11:30:16.256 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:30:16.256 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.1014383959999998, 2024-09-18T11:30:16.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080208420000000002, 2024-09-18T11:30:16.256 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.256 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:16.256 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:30:16.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:30:16.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130396+0000", 2024-09-18T11:30:16.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:16.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133245+0000", 2024-09-18T11:30:16.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028486929999999998 2024-09-18T11:30:16.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133253+0000", 2024-09-18T11:30:16.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1540000000000002e-06 2024-09-18T11:30:16.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133261+0000", 2024-09-18T11:30:16.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3620000000000006e-06 2024-09-18T11:30:16.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133319+0000", 2024-09-18T11:30:16.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7719000000000001e-05 2024-09-18T11:30:16.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133341+0000", 2024-09-18T11:30:16.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1946999999999998e-05 2024-09-18T11:30:16.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:16.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138368+0000", 2024-09-18T11:30:16.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005026712 2024-09-18T11:30:16.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138418+0000", 2024-09-18T11:30:16.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9864000000000002e-05 2024-09-18T11:30:16.260 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.260 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.260 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.260 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:895 4.4 4:2c93196f:::smithi11939506-28:head [read 0~2453789,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:16.260 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.133045+0000", 2024-09-18T11:30:16.260 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.0987899490000004, 2024-09-18T11:30:16.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036999950000000002, 2024-09-18T11:30:16.261 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.261 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:16.261 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:16.261 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:16.261 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:16.261 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 895 2024-09-18T11:30:16.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.261 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133045+0000", 2024-09-18T11:30:16.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133045+0000", 2024-09-18T11:30:16.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133045+0000", 2024-09-18T11:30:16.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:16.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133048+0000", 2024-09-18T11:30:16.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8079999999999998e-06 2024-09-18T11:30:16.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133055+0000", 2024-09-18T11:30:16.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1680000000000001e-06 2024-09-18T11:30:16.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133059+0000", 2024-09-18T11:30:16.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3000000000000003e-06 2024-09-18T11:30:16.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.135048+0000", 2024-09-18T11:30:16.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019892780000000001 2024-09-18T11:30:16.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.135155+0000", 2024-09-18T11:30:16.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010662000000000001 2024-09-18T11:30:16.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.136745+0000", 2024-09-18T11:30:16.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015903009999999999 2024-09-18T11:30:16.264 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.265 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.265 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.265 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "pg_update_log_missing(4.a epoch 31/22 rep_tid 47 entries 31'21 (0'0) error 4:552523b6:::smithi11939506-38:head by client.4590.0:883 0.000000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 30'18)", 2024-09-18T11:30:16.265 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.133275+0000", 2024-09-18T11:30:16.265 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.098560075, 2024-09-18T11:30:16.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051654209999999999, 2024-09-18T11:30:16.265 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.265 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:30:16.265 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133275+0000", 2024-09-18T11:30:16.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133275+0000", 2024-09-18T11:30:16.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133275+0000", 2024-09-18T11:30:16.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:16.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133283+0000", 2024-09-18T11:30:16.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3210000000000005e-06 2024-09-18T11:30:16.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133294+0000", 2024-09-18T11:30:16.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0849000000000001e-05 2024-09-18T11:30:16.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133300+0000", 2024-09-18T11:30:16.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3110000000000002e-06 2024-09-18T11:30:16.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.134908+0000", 2024-09-18T11:30:16.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016080339999999999 2024-09-18T11:30:16.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138440+0000", 2024-09-18T11:30:16.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035323059999999998 2024-09-18T11:30:16.268 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.268 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.268 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.269 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:905 4.7 4:e73ba68f:::smithi11939506-1:head [write 742876~447756,stat] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:16.269 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.140411+0000", 2024-09-18T11:30:16.269 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.0914236989999999, 2024-09-18T11:30:16.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060845960000000003, 2024-09-18T11:30:16.269 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.269 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:16.269 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:16.269 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:16.269 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:16.269 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 905 2024-09-18T11:30:16.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.270 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140411+0000", 2024-09-18T11:30:16.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140411+0000", 2024-09-18T11:30:16.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140408+0000", 2024-09-18T11:30:16.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:30:16.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140676+0000", 2024-09-18T11:30:16.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026841300000000001 2024-09-18T11:30:16.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140708+0000", 2024-09-18T11:30:16.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1952000000000002e-05 2024-09-18T11:30:16.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140721+0000", 2024-09-18T11:30:16.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3005e-05 2024-09-18T11:30:16.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140788+0000", 2024-09-18T11:30:16.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6839000000000002e-05 2024-09-18T11:30:16.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140923+0000", 2024-09-18T11:30:16.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000135258 2024-09-18T11:30:16.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:16.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.141159+0000", 2024-09-18T11:30:16.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023572900000000001 2024-09-18T11:30:16.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:16.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.143764+0000", 2024-09-18T11:30:16.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026044919999999999 2024-09-18T11:30:16.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:16.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.146411+0000", 2024-09-18T11:30:16.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026475130000000002 2024-09-18T11:30:16.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:16.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.146448+0000", 2024-09-18T11:30:16.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6650999999999999e-05 2024-09-18T11:30:16.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.146496+0000", 2024-09-18T11:30:16.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8177000000000003e-05 2024-09-18T11:30:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:915 4.f 4:f5d7df41:::smithi11939506-16:2 [read 0~3025227,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169278+0000", 2024-09-18T11:30:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.0625569600000002, 2024-09-18T11:30:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029259630000000002, 2024-09-18T11:30:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:16.276 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 915 2024-09-18T11:30:16.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.276 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169278+0000", 2024-09-18T11:30:16.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169278+0000", 2024-09-18T11:30:16.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169276+0000", 2024-09-18T11:30:16.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:16.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169289+0000", 2024-09-18T11:30:16.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3020000000000001e-05 2024-09-18T11:30:16.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169309+0000", 2024-09-18T11:30:16.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9919e-05 2024-09-18T11:30:16.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169316+0000", 2024-09-18T11:30:16.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3819999999999998e-06 2024-09-18T11:30:16.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169394+0000", 2024-09-18T11:30:16.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7595000000000005e-05 2024-09-18T11:30:16.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169512+0000", 2024-09-18T11:30:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011842599999999999 2024-09-18T11:30:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.172204+0000", 2024-09-18T11:30:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026920780000000001 2024-09-18T11:30:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:916 4.f 4:f5d7df41:::smithi11939506-16:2 [read 0~3025227,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169328+0000", 2024-09-18T11:30:16.279 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.0625067929999998, 2024-09-18T11:30:16.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045152350000000003, 2024-09-18T11:30:16.280 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.280 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:16.280 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:16.280 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:16.280 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:16.280 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 916 2024-09-18T11:30:16.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.280 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169328+0000", 2024-09-18T11:30:16.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169328+0000", 2024-09-18T11:30:16.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169328+0000", 2024-09-18T11:30:16.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:16.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169332+0000", 2024-09-18T11:30:16.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9790000000000004e-06 2024-09-18T11:30:16.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169339+0000", 2024-09-18T11:30:16.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0589999999999997e-06 2024-09-18T11:30:16.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169343+0000", 2024-09-18T11:30:16.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1810000000000003e-06 2024-09-18T11:30:16.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.172235+0000", 2024-09-18T11:30:16.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028918559999999999 2024-09-18T11:30:16.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.172321+0000", 2024-09-18T11:30:16.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6122e-05 2024-09-18T11:30:16.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.173844+0000", 2024-09-18T11:30:16.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015225449999999999 2024-09-18T11:30:16.285 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.285 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.285 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.285 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:917 4.f 4:f5d7df41:::smithi11939506-16:2 [sparse-read 0~1650442] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:16.285 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169348+0000", 2024-09-18T11:30:16.286 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.0624873739999998, 2024-09-18T11:30:16.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050103350000000003, 2024-09-18T11:30:16.286 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.286 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:16.286 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:16.286 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:16.286 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:16.286 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 917 2024-09-18T11:30:16.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.286 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169348+0000", 2024-09-18T11:30:16.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169348+0000", 2024-09-18T11:30:16.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169347+0000", 2024-09-18T11:30:16.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:16.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169350+0000", 2024-09-18T11:30:16.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.537e-06 2024-09-18T11:30:16.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169356+0000", 2024-09-18T11:30:16.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1129999999999997e-06 2024-09-18T11:30:16.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169359+0000", 2024-09-18T11:30:16.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.444e-06 2024-09-18T11:30:16.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.173900+0000", 2024-09-18T11:30:16.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045409079999999998 2024-09-18T11:30:16.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174024+0000", 2024-09-18T11:30:16.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012400399999999999 2024-09-18T11:30:16.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174358+0000", 2024-09-18T11:30:16.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00033376199999999998 2024-09-18T11:30:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:921 4.2 4:425afe9c:::smithi11939506-29:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169430+0000", 2024-09-18T11:30:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.0624049820000003, 2024-09-18T11:30:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038118980000000002, 2024-09-18T11:30:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:16.290 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 921 2024-09-18T11:30:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169430+0000", 2024-09-18T11:30:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169430+0000", 2024-09-18T11:30:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169429+0000", 2024-09-18T11:30:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169433+0000", 2024-09-18T11:30:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9380000000000002e-06 2024-09-18T11:30:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169443+0000", 2024-09-18T11:30:16.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0149999999999999e-05 2024-09-18T11:30:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:30:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4327e-05 2024-09-18T11:30:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169508+0000", 2024-09-18T11:30:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9743999999999997e-05 2024-09-18T11:30:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169582+0000", 2024-09-18T11:30:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4635999999999995e-05 2024-09-18T11:30:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.173242+0000", 2024-09-18T11:30:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036598220000000001 2024-09-18T11:30:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.294 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:994 4.f 4:f269e3d0:::smithi11939506-24:head [read 0~2386561,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164432+0000", 2024-09-18T11:30:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.0674027180000001, 2024-09-18T11:30:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042192219999999999, 2024-09-18T11:30:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 994 2024-09-18T11:30:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164432+0000", 2024-09-18T11:30:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164432+0000", 2024-09-18T11:30:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164431+0000", 2024-09-18T11:30:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164441+0000", 2024-09-18T11:30:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0288e-05 2024-09-18T11:30:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164459+0000", 2024-09-18T11:30:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8331000000000001e-05 2024-09-18T11:30:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164468+0000", 2024-09-18T11:30:16.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5990000000000002e-06 2024-09-18T11:30:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164520+0000", 2024-09-18T11:30:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1869000000000001e-05 2024-09-18T11:30:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164526+0000", 2024-09-18T11:30:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1160000000000004e-06 2024-09-18T11:30:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.166786+0000", 2024-09-18T11:30:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022604069999999999 2024-09-18T11:30:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.166880+0000", 2024-09-18T11:30:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3968999999999994e-05 2024-09-18T11:30:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.168652+0000", 2024-09-18T11:30:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017714300000000001 2024-09-18T11:30:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.299 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:995 4.f 4:f269e3d0:::smithi11939506-24:head [read 0~2386561,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164480+0000", 2024-09-18T11:30:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.0673553689999999, 2024-09-18T11:30:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057271170000000003, 2024-09-18T11:30:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:16.300 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 995 2024-09-18T11:30:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164480+0000", 2024-09-18T11:30:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164480+0000", 2024-09-18T11:30:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164479+0000", 2024-09-18T11:30:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164483+0000", 2024-09-18T11:30:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9859999999999996e-06 2024-09-18T11:30:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164497+0000", 2024-09-18T11:30:16.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3559e-05 2024-09-18T11:30:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164506+0000", 2024-09-18T11:30:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3959999999999995e-06 2024-09-18T11:30:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164573+0000", 2024-09-18T11:30:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7123000000000001e-05 2024-09-18T11:30:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:16.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164577+0000", 2024-09-18T11:30:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9999999999999998e-06 2024-09-18T11:30:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.168696+0000", 2024-09-18T11:30:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041188379999999997 2024-09-18T11:30:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.168824+0000", 2024-09-18T11:30:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012762099999999999 2024-09-18T11:30:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170207+0000", 2024-09-18T11:30:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001383076 2024-09-18T11:30:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:996 4.f 4:f269e3d0:::smithi11939506-24:head [read 0~2386561,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164511+0000", 2024-09-18T11:30:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.0673237980000001, 2024-09-18T11:30:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069965679999999999, 2024-09-18T11:30:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:16.305 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 996 2024-09-18T11:30:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164511+0000", 2024-09-18T11:30:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164511+0000", 2024-09-18T11:30:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164511+0000", 2024-09-18T11:30:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164514+0000", 2024-09-18T11:30:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3210000000000001e-06 2024-09-18T11:30:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164521+0000", 2024-09-18T11:30:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5169999999999996e-06 2024-09-18T11:30:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164524+0000", 2024-09-18T11:30:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:30:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164674+0000", 2024-09-18T11:30:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014943199999999999 2024-09-18T11:30:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:30:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8579999999999996e-06 2024-09-18T11:30:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170262+0000", 2024-09-18T11:30:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055827719999999997 2024-09-18T11:30:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170405+0000", 2024-09-18T11:30:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000143115 2024-09-18T11:30:16.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.171508+0000", 2024-09-18T11:30:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001102448 2024-09-18T11:30:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:997 4.7 4:e0d95e10:::smithi11939506-22:6 [sparse-read 0~1003172,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164528+0000", 2024-09-18T11:30:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.067306651, 2024-09-18T11:30:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028885180000000001, 2024-09-18T11:30:16.310 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 997 2024-09-18T11:30:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164528+0000", 2024-09-18T11:30:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164528+0000", 2024-09-18T11:30:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164528+0000", 2024-09-18T11:30:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164531+0000", 2024-09-18T11:30:16.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1269999999999999e-06 2024-09-18T11:30:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164537+0000", 2024-09-18T11:30:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6110000000000001e-06 2024-09-18T11:30:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164540+0000", 2024-09-18T11:30:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4910000000000001e-06 2024-09-18T11:30:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164600+0000", 2024-09-18T11:30:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0096000000000002e-05 2024-09-18T11:30:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164605+0000", 2024-09-18T11:30:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0780000000000003e-06 2024-09-18T11:30:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.167176+0000", 2024-09-18T11:30:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025707630000000002 2024-09-18T11:30:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.167253+0000", 2024-09-18T11:30:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6658999999999999e-05 2024-09-18T11:30:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.167417+0000", 2024-09-18T11:30:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000164038 2024-09-18T11:30:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:998 4.7 4:e0d95e10:::smithi11939506-22:6 [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164545+0000", 2024-09-18T11:30:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.0672905079999999, 2024-09-18T11:30:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0030479959999999999, 2024-09-18T11:30:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 998 2024-09-18T11:30:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.316 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164545+0000", 2024-09-18T11:30:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164545+0000", 2024-09-18T11:30:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164544+0000", 2024-09-18T11:30:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164546+0000", 2024-09-18T11:30:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3159999999999999e-06 2024-09-18T11:30:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164553+0000", 2024-09-18T11:30:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1160000000000004e-06 2024-09-18T11:30:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164556+0000", 2024-09-18T11:30:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5319999999999998e-06 2024-09-18T11:30:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164635+0000", 2024-09-18T11:30:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8926999999999998e-05 2024-09-18T11:30:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164640+0000", 2024-09-18T11:30:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5299999999999998e-06 2024-09-18T11:30:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.167433+0000", 2024-09-18T11:30:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002793327 2024-09-18T11:30:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.167471+0000", 2024-09-18T11:30:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8587000000000002e-05 2024-09-18T11:30:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.167593+0000", 2024-09-18T11:30:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121066 2024-09-18T11:30:16.320 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:999 4.7 4:e0d95e10:::smithi11939506-22:6 [read 0~2352947,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.0672377609999999, 2024-09-18T11:30:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042304250000000003, 2024-09-18T11:30:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:16.321 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 999 2024-09-18T11:30:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:30:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.783e-06 2024-09-18T11:30:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164609+0000", 2024-09-18T11:30:16.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4130000000000006e-06 2024-09-18T11:30:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164614+0000", 2024-09-18T11:30:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6469999999999997e-06 2024-09-18T11:30:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164656+0000", 2024-09-18T11:30:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2360000000000001e-05 2024-09-18T11:30:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:16.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164660+0000", 2024-09-18T11:30:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3900000000000003e-06 2024-09-18T11:30:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.167605+0000", 2024-09-18T11:30:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029447549999999999 2024-09-18T11:30:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.167653+0000", 2024-09-18T11:30:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8004999999999999e-05 2024-09-18T11:30:16.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.168828+0000", 2024-09-18T11:30:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001174558 2024-09-18T11:30:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1007 4.2 4:425afe9c:::smithi11939506-29:head [write 426911~540354,stat] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.250099+0000", 2024-09-18T11:30:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.98173654399999999, 2024-09-18T11:30:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003652944, 2024-09-18T11:30:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.326 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1007 2024-09-18T11:30:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.250099+0000", 2024-09-18T11:30:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.250099+0000", 2024-09-18T11:30:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.250097+0000", 2024-09-18T11:30:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.250215+0000", 2024-09-18T11:30:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117705 2024-09-18T11:30:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.250232+0000", 2024-09-18T11:30:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7042e-05 2024-09-18T11:30:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.250239+0000", 2024-09-18T11:30:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1550000000000001e-06 2024-09-18T11:30:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.250280+0000", 2024-09-18T11:30:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1325000000000002e-05 2024-09-18T11:30:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.250346+0000", 2024-09-18T11:30:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6101999999999998e-05 2024-09-18T11:30:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.250471+0000", 2024-09-18T11:30:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012430699999999999 2024-09-18T11:30:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.252352+0000", 2024-09-18T11:30:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001881438 2024-09-18T11:30:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.253678+0000", 2024-09-18T11:30:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013258110000000001 2024-09-18T11:30:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.253709+0000", 2024-09-18T11:30:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1140999999999999e-05 2024-09-18T11:30:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.253751+0000", 2024-09-18T11:30:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2567000000000001e-05 2024-09-18T11:30:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.331 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1008 4.2 4:425afe9c:::smithi11939506-29:head [write 1622009~726683] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.257535+0000", 2024-09-18T11:30:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.97430045600000004, 2024-09-18T11:30:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043772960000000001, 2024-09-18T11:30:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:16.332 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1008 2024-09-18T11:30:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.257535+0000", 2024-09-18T11:30:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.257535+0000", 2024-09-18T11:30:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.257534+0000", 2024-09-18T11:30:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.257658+0000", 2024-09-18T11:30:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012445499999999999 2024-09-18T11:30:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.257670+0000", 2024-09-18T11:30:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1751e-05 2024-09-18T11:30:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.257674+0000", 2024-09-18T11:30:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4410000000000003e-06 2024-09-18T11:30:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.257706+0000", 2024-09-18T11:30:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2079000000000002e-05 2024-09-18T11:30:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.257760+0000", 2024-09-18T11:30:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3387000000000002e-05 2024-09-18T11:30:16.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.257839+0000", 2024-09-18T11:30:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8978000000000002e-05 2024-09-18T11:30:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.259997+0000", 2024-09-18T11:30:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002158564 2024-09-18T11:30:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.261859+0000", 2024-09-18T11:30:16.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001862013 2024-09-18T11:30:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.261890+0000", 2024-09-18T11:30:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0477999999999999e-05 2024-09-18T11:30:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.261912+0000", 2024-09-18T11:30:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2022999999999999e-05 2024-09-18T11:30:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.337 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.338 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.338 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.338 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:16.339 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:30:16.340 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:16.337+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:16.440 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:16.439+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:16.478 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:16.479 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:16.488 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:30:16.541 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:16.539+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:16.586 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:16.586 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:16.586 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:16.586 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:16.586 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:16.594 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:30:16.678 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:16.676+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:16.732 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:16.732 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:16.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:16.732 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:16.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.732 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4266.0:1 2.3 e16/15 2:c4c92e5a:::rbd_trash:head v 16'1, mlcod=0'0)", 2024-09-18T11:30:16.732 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:50.028963+0000", 2024-09-18T11:30:16.732 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.702182808, 2024-09-18T11:30:16.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001253021, 2024-09-18T11:30:16.733 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.733 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:16.733 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.028963+0000", 2024-09-18T11:30:16.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.028963+0000", 2024-09-18T11:30:16.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.028963+0000", 2024-09-18T11:30:16.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:16.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.028967+0000", 2024-09-18T11:30:16.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2119999999999997e-06 2024-09-18T11:30:16.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.029011+0000", 2024-09-18T11:30:16.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4011000000000003e-05 2024-09-18T11:30:16.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.029019+0000", 2024-09-18T11:30:16.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.729e-06 2024-09-18T11:30:16.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.029185+0000", 2024-09-18T11:30:16.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016637600000000001 2024-09-18T11:30:16.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.029211+0000", 2024-09-18T11:30:16.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6211e-05 2024-09-18T11:30:16.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:16.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.030175+0000", 2024-09-18T11:30:16.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096347000000000002 2024-09-18T11:30:16.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.030216+0000", 2024-09-18T11:30:16.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.138e-05 2024-09-18T11:30:16.736 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.736 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.736 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.737 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1045 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'23, mlcod=30'20)", 2024-09-18T11:30:16.737 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.167583+0000", 2024-09-18T11:30:16.737 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.563562385, 2024-09-18T11:30:16.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031033380000000002, 2024-09-18T11:30:16.737 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.737 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:16.737 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167583+0000", 2024-09-18T11:30:16.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167583+0000", 2024-09-18T11:30:16.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167582+0000", 2024-09-18T11:30:16.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:16.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167934+0000", 2024-09-18T11:30:16.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000352262 2024-09-18T11:30:16.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167952+0000", 2024-09-18T11:30:16.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7859999999999998e-05 2024-09-18T11:30:16.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167960+0000", 2024-09-18T11:30:16.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9729999999999997e-06 2024-09-18T11:30:16.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.168008+0000", 2024-09-18T11:30:16.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7531000000000003e-05 2024-09-18T11:30:16.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.168032+0000", 2024-09-18T11:30:16.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4443e-05 2024-09-18T11:30:16.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:16.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.170646+0000", 2024-09-18T11:30:16.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026136739999999999 2024-09-18T11:30:16.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.170687+0000", 2024-09-18T11:30:16.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0859000000000003e-05 2024-09-18T11:30:16.741 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.742 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.742 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.742 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1046 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'24, mlcod=30'21)", 2024-09-18T11:30:16.742 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:30:16.742 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.55736260500000001, 2024-09-18T11:30:16.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042519100000000002, 2024-09-18T11:30:16.742 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.742 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:16.742 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:30:16.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:30:16.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173782+0000", 2024-09-18T11:30:16.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:16.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174207+0000", 2024-09-18T11:30:16.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00042479799999999999 2024-09-18T11:30:16.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174224+0000", 2024-09-18T11:30:16.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7513999999999999e-05 2024-09-18T11:30:16.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174232+0000", 2024-09-18T11:30:16.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0840000000000003e-06 2024-09-18T11:30:16.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174297+0000", 2024-09-18T11:30:16.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5215000000000005e-05 2024-09-18T11:30:16.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174336+0000", 2024-09-18T11:30:16.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9403000000000002e-05 2024-09-18T11:30:16.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:16.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.177990+0000", 2024-09-18T11:30:16.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036533709999999999 2024-09-18T11:30:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.178035+0000", 2024-09-18T11:30:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5495999999999999e-05 2024-09-18T11:30:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1047 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'25, mlcod=35'23)", 2024-09-18T11:30:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:30:16.746 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.55139040699999997, 2024-09-18T11:30:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038129129999999998, 2024-09-18T11:30:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:30:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:30:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179754+0000", 2024-09-18T11:30:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180088+0000", 2024-09-18T11:30:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00033395000000000001 2024-09-18T11:30:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180105+0000", 2024-09-18T11:30:16.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7039000000000001e-05 2024-09-18T11:30:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180112+0000", 2024-09-18T11:30:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3710000000000002e-06 2024-09-18T11:30:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180176+0000", 2024-09-18T11:30:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3178999999999999e-05 2024-09-18T11:30:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180197+0000", 2024-09-18T11:30:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1687000000000001e-05 2024-09-18T11:30:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183542+0000", 2024-09-18T11:30:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003344877 2024-09-18T11:30:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183568+0000", 2024-09-18T11:30:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6146e-05 2024-09-18T11:30:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.750 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1048 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'26, mlcod=35'23)", 2024-09-18T11:30:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.180697+0000", 2024-09-18T11:30:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.55044828099999998, 2024-09-18T11:30:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031436290000000002, 2024-09-18T11:30:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180697+0000", 2024-09-18T11:30:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180697+0000", 2024-09-18T11:30:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180697+0000", 2024-09-18T11:30:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180704+0000", 2024-09-18T11:30:16.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8360000000000003e-06 2024-09-18T11:30:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180716+0000", 2024-09-18T11:30:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2308e-05 2024-09-18T11:30:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180724+0000", 2024-09-18T11:30:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6499999999999996e-06 2024-09-18T11:30:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:16.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180884+0000", 2024-09-18T11:30:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016041699999999999 2024-09-18T11:30:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180905+0000", 2024-09-18T11:30:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1103000000000001e-05 2024-09-18T11:30:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183819+0000", 2024-09-18T11:30:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029141800000000002 2024-09-18T11:30:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:16.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183841+0000", 2024-09-18T11:30:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1869e-05 2024-09-18T11:30:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:16.755 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:16.755 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:16.756 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:30:16.757 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:16.754+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:16.857 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:16.856+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:16.894 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:16.894 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:16.902 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:30:16.904 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:30:16.958 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:16.956+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:17.002 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:17.003 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:17.003 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:17.003 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:17.003 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:17.011 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:30:17.017 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:30:17.018 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:30:17.033 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:17.033 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:17.048 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:17.049 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:17.059 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:17.058+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:17.063 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:17.063 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:17.079 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:17.079 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:17.095 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:17.096 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:17.112 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:17.112 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:17.116 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:17.117 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:17.125 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:30:17.127 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:17.127 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:17.158 INFO:tasks.rados.rados.0.smithi119.stdout:702: done (2 left) 2024-09-18T11:30:17.158 INFO:tasks.rados.rados.0.smithi119.stdout:703: done (1 left) 2024-09-18T11:30:17.158 INFO:tasks.rados.rados.0.smithi119.stdout:704: done (0 left) 2024-09-18T11:30:17.158 INFO:tasks.rados.rados.0.smithi119.stdout:705: write (excl) oid 23 current snap is 4 2024-09-18T11:30:17.158 INFO:tasks.rados.rados.0.smithi119.stdout:705: seq_num 79 ranges {527753=708201,1856548=737690,3286534=41219} 2024-09-18T11:30:17.164 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T11:30:17.165 INFO:tasks.rados.rados.0.smithi119.stdout:705: writing smithi11939506-23 from 527753 to 1235954 tid 1 2024-09-18T11:30:17.169 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:17.168+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:17.172 INFO:tasks.rados.rados.0.smithi119.stdout:705: writing smithi11939506-23 from 1856548 to 2594238 tid 2 2024-09-18T11:30:17.176 INFO:tasks.rados.rados.0.smithi119.stdout:705: writing smithi11939506-23 from 3286534 to 3327753 tid 3 2024-09-18T11:30:17.176 INFO:tasks.rados.rados.0.smithi119.stdout:706: snap_remove snap 3 2024-09-18T11:30:17.238 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:17.238 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:17.247 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:30:17.260 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:17.258+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:17.361 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:17.360+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:17.387 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:17.387 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:17.396 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:30:17.462 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:17.460+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:17.534 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:17.534 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:17.542 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:30:17.562 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:17.560+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:17.641 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:17.642 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:17.650 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:30:17.694 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:17.693+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:17.747 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:17.747 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:17.756 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:30:17.764 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:17.763+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:17.865 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:17.863+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:17.894 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:17.894 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:17.902 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:30:17.966 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:17.964+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:18.001 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:18.001 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:18.010 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:30:18.094 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:18.093+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:18.150 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:18.150 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:18.159 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:30:18.162 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:18.160+0000 7f6bf3052640 -1 osd.5 36 handle_osd_map injecting map crc failure 2024-09-18T11:30:18.163 INFO:tasks.rados.rados.0.smithi119.stdout:706: done (1 left) 2024-09-18T11:30:18.163 INFO:tasks.rados.rados.0.smithi119.stdout:707: read oid 5 snap -1 2024-09-18T11:30:18.163 INFO:tasks.rados.rados.0.smithi119.stdout:707: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-18T11:30:18.163 INFO:tasks.rados.rados.0.smithi119.stdout:708: read oid 38 snap -1 2024-09-18T11:30:18.163 INFO:tasks.rados.rados.0.smithi119.stdout:708: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-18T11:30:18.163 INFO:tasks.rados.rados.0.smithi119.stdout:709: tier_evict oid 30 2024-09-18T11:30:18.163 INFO:tasks.rados.rados.0.smithi119.stdout:709: tier_evict oid 30 snap -1 2024-09-18T11:30:18.163 INFO:tasks.rados.rados.0.smithi119.stdout:705: finishing write tid 1 to smithi11939506-23 2024-09-18T11:30:18.163 INFO:tasks.rados.rados.0.smithi119.stdout:705: finishing write tid 2 to smithi11939506-23 2024-09-18T11:30:18.164 INFO:tasks.rados.rados.0.smithi119.stdout:705: finishing write tid 3 to smithi11939506-23 2024-09-18T11:30:18.164 INFO:tasks.rados.rados.0.smithi119.stdout:710: read oid 2 snap -1 2024-09-18T11:30:18.164 INFO:tasks.rados.rados.0.smithi119.stdout:710: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-18T11:30:18.164 INFO:tasks.rados.rados.0.smithi119.stdout:711: snap_remove snap 2 2024-09-18T11:30:18.164 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is unset 2024-09-18T11:30:18.168 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:18.166+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:18.261 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:18.262 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:18.269 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:18.267+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:18.270 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:30:18.369 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:18.369 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:18.369 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:18.368+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:18.377 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:30:18.470 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:18.469+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:18.515 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:18.515 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:18.523 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:30:18.607 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:18.606+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:18.662 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:18.662 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:18.670 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:30:18.711 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:18.710+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:18.772 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:18.770+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:18.779 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:18.779 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:18.788 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:30:18.873 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:18.871+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:18.928 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:18.928 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:18.937 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:30:18.978 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:18.976+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:19.075 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:19.075 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:19.075 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:19.074+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:19.083 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:30:19.139 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:30:19.160 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:19.158+0000 7fb6950ff640 -1 osd.2 37 handle_osd_map injecting map crc failure 2024-09-18T11:30:19.161 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:19.159+0000 7f6bf5056640 -1 osd.5 37 handle_osd_map injecting map crc failure 2024-09-18T11:30:19.163 INFO:tasks.rados.rados.0.smithi119.stdout:711: done (5 left) 2024-09-18T11:30:19.163 INFO:tasks.rados.rados.0.smithi119.stdout:712: read oid 13 snap -1 2024-09-18T11:30:19.163 INFO:tasks.rados.rados.0.smithi119.stdout:712: expect (ObjNum 74 snap 3 seq_num 74) 2024-09-18T11:30:19.163 INFO:tasks.rados.rados.0.smithi119.stdout:713: read oid 24 snap -1 2024-09-18T11:30:19.163 INFO:tasks.rados.rados.0.smithi119.stdout:713: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-18T11:30:19.163 INFO:tasks.rados.rados.0.smithi119.stdout:714: read oid 37 snap -1 2024-09-18T11:30:19.164 INFO:tasks.rados.rados.0.smithi119.stdout:714: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-18T11:30:19.164 INFO:tasks.rados.rados.0.smithi119.stdout:715: tier_promote oid 49 2024-09-18T11:30:19.164 INFO:tasks.rados.rados.0.smithi119.stdout:716: read oid 15 snap -1 2024-09-18T11:30:19.164 INFO:tasks.rados.rados.0.smithi119.stdout:716: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-18T11:30:19.164 INFO:tasks.rados.rados.0.smithi119.stdout:717: read oid 42 snap -1 2024-09-18T11:30:19.164 INFO:tasks.rados.rados.0.smithi119.stdout:717: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-18T11:30:19.164 INFO:tasks.rados.rados.0.smithi119.stdout:718: write oid 20 current snap is 4 2024-09-18T11:30:19.164 INFO:tasks.rados.rados.0.smithi119.stdout:718: seq_num 80 ranges {544560=783450,1915030=481348,2797604=479761} 2024-09-18T11:30:19.167 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:19.164+0000 7f2d449e8640 -1 osd.1 37 handle_osd_map injecting map crc failure 2024-09-18T11:30:19.173 INFO:tasks.rados.rados.0.smithi119.stdout:718: writing smithi11939506-20 from 544560 to 1328010 tid 1 2024-09-18T11:30:19.177 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:19.174+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:19.181 INFO:tasks.rados.rados.0.smithi119.stdout:718: writing smithi11939506-20 from 1915030 to 2396378 tid 2 2024-09-18T11:30:19.188 INFO:tasks.rados.rados.0.smithi119.stdout:718: writing smithi11939506-20 from 2797604 to 3277365 tid 3 2024-09-18T11:30:19.188 INFO:tasks.rados.rados.0.smithi119.stdout:719: read oid 3 snap -1 2024-09-18T11:30:19.188 INFO:tasks.rados.rados.0.smithi119.stdout:719: expect (ObjNum 77 snap 3 seq_num 77) 2024-09-18T11:30:19.188 INFO:tasks.rados.rados.0.smithi119.stdout:720: tier_evict oid 41 2024-09-18T11:30:19.188 INFO:tasks.rados.rados.0.smithi119.stdout:720: tier_evict oid 41 snap -1 2024-09-18T11:30:19.188 INFO:tasks.rados.rados.0.smithi119.stdout:705: finishing write tid 4 to smithi11939506-23 2024-09-18T11:30:19.188 INFO:tasks.rados.rados.0.smithi119.stdout:705: finishing write tid 5 to smithi11939506-23 2024-09-18T11:30:19.188 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 37 (ObjNum 79 snap 4 seq_num 79) dirty exists 2024-09-18T11:30:19.189 INFO:tasks.rados.rados.0.smithi119.stdout:705: left oid 23 (ObjNum 79 snap 4 seq_num 79) 2024-09-18T11:30:19.189 INFO:tasks.rados.rados.0.smithi119.stdout:705: done (13 left) 2024-09-18T11:30:19.189 INFO:tasks.rados.rados.0.smithi119.stdout:721: rollback oid 8 current snap is 4 2024-09-18T11:30:19.189 INFO:tasks.rados.rados.0.smithi119.stdout:721: done (13 left) 2024-09-18T11:30:19.189 INFO:tasks.rados.rados.0.smithi119.stdout:722: write oid 11 current snap is 4 2024-09-18T11:30:19.189 INFO:tasks.rados.rados.0.smithi119.stdout:722: seq_num 81 ranges {561367=458699,1573512=741359,2941380=419987} 2024-09-18T11:30:19.194 INFO:tasks.rados.rados.0.smithi119.stdout:722: writing smithi11939506-11 from 561367 to 1020066 tid 1 2024-09-18T11:30:19.200 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:19.200 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:19.201 INFO:tasks.rados.rados.0.smithi119.stdout:722: writing smithi11939506-11 from 1573512 to 2314871 tid 2 2024-09-18T11:30:19.207 INFO:tasks.rados.rados.0.smithi119.stdout:722: writing smithi11939506-11 from 2941380 to 3361367 tid 3 2024-09-18T11:30:19.207 INFO:tasks.rados.rados.0.smithi119.stdout:723: tier_evict oid 44 2024-09-18T11:30:19.208 INFO:tasks.rados.rados.0.smithi119.stdout:723: tier_evict oid 44 snap -1 2024-09-18T11:30:19.208 INFO:tasks.rados.rados.0.smithi119.stdout:709: got (22) Invalid argument 2024-09-18T11:30:19.208 INFO:tasks.rados.rados.0.smithi119.stdout:707: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-18T11:30:19.210 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:30:19.270 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:30:19.270 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:30:19.276 INFO:tasks.rados.rados.0.smithi119.stdout:710: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-18T11:30:19.277 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:19.274+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:19.286 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:19.286 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:19.301 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:19.301 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:19.316 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:19.316 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:19.332 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:19.332 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:19.343 INFO:tasks.rados.rados.0.smithi119.stdout:708: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-18T11:30:19.347 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:19.347 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:19.350 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:19.351 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:19.351 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:19.351 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:19.351 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:19.360 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:30:19.363 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:19.363 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:19.379 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:19.379 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:19.379 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:19.377+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:19.432 INFO:tasks.rados.rados.0.smithi119.stdout:716: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-18T11:30:19.465 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:887 4.7 4:e0d95e10:::smithi11939506-22:head [read 0~2352947,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:30:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3461782230000008, 2024-09-18T11:30:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079115169999999999, 2024-09-18T11:30:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 887 2024-09-18T11:30:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:30:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:30:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116365+0000", 2024-09-18T11:30:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116376+0000", 2024-09-18T11:30:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1471000000000001e-05 2024-09-18T11:30:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116394+0000", 2024-09-18T11:30:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8088e-05 2024-09-18T11:30:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116402+0000", 2024-09-18T11:30:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2320000000000004e-06 2024-09-18T11:30:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116458+0000", 2024-09-18T11:30:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6536000000000002e-05 2024-09-18T11:30:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116466+0000", 2024-09-18T11:30:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0819999999999999e-06 2024-09-18T11:30:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.122509+0000", 2024-09-18T11:30:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060431879999999997 2024-09-18T11:30:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.122577+0000", 2024-09-18T11:30:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7646999999999998e-05 2024-09-18T11:30:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124278+0000", 2024-09-18T11:30:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017013200000000001 2024-09-18T11:30:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:888 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3461247600000004, 2024-09-18T11:30:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082282120000000004, 2024-09-18T11:30:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 888 2024-09-18T11:30:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116423+0000", 2024-09-18T11:30:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1e-06 2024-09-18T11:30:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:30:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5510000000000001e-06 2024-09-18T11:30:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116431+0000", 2024-09-18T11:30:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6109999999999998e-06 2024-09-18T11:30:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116527+0000", 2024-09-18T11:30:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6212999999999995e-05 2024-09-18T11:30:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116532+0000", 2024-09-18T11:30:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6140000000000002e-06 2024-09-18T11:30:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124314+0000", 2024-09-18T11:30:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077818929999999998 2024-09-18T11:30:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124411+0000", 2024-09-18T11:30:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6992999999999998e-05 2024-09-18T11:30:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124649+0000", 2024-09-18T11:30:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237729 2024-09-18T11:30:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:889 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3461001100000001, 2024-09-18T11:30:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085383349999999993, 2024-09-18T11:30:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 889 2024-09-18T11:30:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116447+0000", 2024-09-18T11:30:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.854e-06 2024-09-18T11:30:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116452+0000", 2024-09-18T11:30:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0250000000000002e-06 2024-09-18T11:30:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116455+0000", 2024-09-18T11:30:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8739999999999999e-06 2024-09-18T11:30:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116565+0000", 2024-09-18T11:30:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110741 2024-09-18T11:30:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116571+0000", 2024-09-18T11:30:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4990000000000002e-06 2024-09-18T11:30:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124691+0000", 2024-09-18T11:30:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081200339999999999 2024-09-18T11:30:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124784+0000", 2024-09-18T11:30:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2886999999999993e-05 2024-09-18T11:30:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124983+0000", 2024-09-18T11:30:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019969000000000001 2024-09-18T11:30:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:46 4.a e31/22 4:552523b6:::smithi11939506-38:head v 31'20, mlcod=30'18)", 2024-09-18T11:30:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:30:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3321484780000006, 2024-09-18T11:30:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080208420000000002, 2024-09-18T11:30:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:30:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:30:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130396+0000", 2024-09-18T11:30:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133245+0000", 2024-09-18T11:30:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028486929999999998 2024-09-18T11:30:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133253+0000", 2024-09-18T11:30:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1540000000000002e-06 2024-09-18T11:30:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133261+0000", 2024-09-18T11:30:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3620000000000006e-06 2024-09-18T11:30:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133319+0000", 2024-09-18T11:30:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7719000000000001e-05 2024-09-18T11:30:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133341+0000", 2024-09-18T11:30:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1946999999999998e-05 2024-09-18T11:30:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138368+0000", 2024-09-18T11:30:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005026712 2024-09-18T11:30:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138418+0000", 2024-09-18T11:30:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9864000000000002e-05 2024-09-18T11:30:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:895 4.4 4:2c93196f:::smithi11939506-28:head [read 0~2453789,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.133045+0000", 2024-09-18T11:30:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3295000310000002, 2024-09-18T11:30:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036999950000000002, 2024-09-18T11:30:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 895 2024-09-18T11:30:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133045+0000", 2024-09-18T11:30:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133045+0000", 2024-09-18T11:30:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133045+0000", 2024-09-18T11:30:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133048+0000", 2024-09-18T11:30:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8079999999999998e-06 2024-09-18T11:30:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133055+0000", 2024-09-18T11:30:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1680000000000001e-06 2024-09-18T11:30:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133059+0000", 2024-09-18T11:30:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3000000000000003e-06 2024-09-18T11:30:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.135048+0000", 2024-09-18T11:30:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019892780000000001 2024-09-18T11:30:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.135155+0000", 2024-09-18T11:30:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010662000000000001 2024-09-18T11:30:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.136745+0000", 2024-09-18T11:30:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015903009999999999 2024-09-18T11:30:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "pg_update_log_missing(4.a epoch 31/22 rep_tid 47 entries 31'21 (0'0) error 4:552523b6:::smithi11939506-38:head by client.4590.0:883 0.000000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 30'18)", 2024-09-18T11:30:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.133275+0000", 2024-09-18T11:30:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3292701569999998, 2024-09-18T11:30:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051654209999999999, 2024-09-18T11:30:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:30:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133275+0000", 2024-09-18T11:30:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133275+0000", 2024-09-18T11:30:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133275+0000", 2024-09-18T11:30:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133283+0000", 2024-09-18T11:30:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3210000000000005e-06 2024-09-18T11:30:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133294+0000", 2024-09-18T11:30:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0849000000000001e-05 2024-09-18T11:30:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133300+0000", 2024-09-18T11:30:19.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3110000000000002e-06 2024-09-18T11:30:19.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.134908+0000", 2024-09-18T11:30:19.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016080339999999999 2024-09-18T11:30:19.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138440+0000", 2024-09-18T11:30:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035323059999999998 2024-09-18T11:30:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:905 4.7 4:e73ba68f:::smithi11939506-1:head [write 742876~447756,stat] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.140411+0000", 2024-09-18T11:30:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3221337809999998, 2024-09-18T11:30:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060845960000000003, 2024-09-18T11:30:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 905 2024-09-18T11:30:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140411+0000", 2024-09-18T11:30:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140411+0000", 2024-09-18T11:30:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140408+0000", 2024-09-18T11:30:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:30:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140676+0000", 2024-09-18T11:30:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026841300000000001 2024-09-18T11:30:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140708+0000", 2024-09-18T11:30:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1952000000000002e-05 2024-09-18T11:30:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140721+0000", 2024-09-18T11:30:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3005e-05 2024-09-18T11:30:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140788+0000", 2024-09-18T11:30:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6839000000000002e-05 2024-09-18T11:30:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140923+0000", 2024-09-18T11:30:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000135258 2024-09-18T11:30:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.141159+0000", 2024-09-18T11:30:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023572900000000001 2024-09-18T11:30:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.143764+0000", 2024-09-18T11:30:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026044919999999999 2024-09-18T11:30:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.146411+0000", 2024-09-18T11:30:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026475130000000002 2024-09-18T11:30:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.146448+0000", 2024-09-18T11:30:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6650999999999999e-05 2024-09-18T11:30:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.146496+0000", 2024-09-18T11:30:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8177000000000003e-05 2024-09-18T11:30:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:916 4.f 4:f5d7df41:::smithi11939506-16:2 [read 0~3025227,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169328+0000", 2024-09-18T11:30:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.2932168750000006, 2024-09-18T11:30:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045152350000000003, 2024-09-18T11:30:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:19.502 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:19.502 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:19.502 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:19.502 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 916 2024-09-18T11:30:19.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.502 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169328+0000", 2024-09-18T11:30:19.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169328+0000", 2024-09-18T11:30:19.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169328+0000", 2024-09-18T11:30:19.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:19.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169332+0000", 2024-09-18T11:30:19.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9790000000000004e-06 2024-09-18T11:30:19.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169339+0000", 2024-09-18T11:30:19.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0589999999999997e-06 2024-09-18T11:30:19.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169343+0000", 2024-09-18T11:30:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1810000000000003e-06 2024-09-18T11:30:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.172235+0000", 2024-09-18T11:30:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028918559999999999 2024-09-18T11:30:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.172321+0000", 2024-09-18T11:30:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6122e-05 2024-09-18T11:30:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.173844+0000", 2024-09-18T11:30:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015225449999999999 2024-09-18T11:30:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:917 4.f 4:f5d7df41:::smithi11939506-16:2 [sparse-read 0~1650442] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169348+0000", 2024-09-18T11:30:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.2931974559999997, 2024-09-18T11:30:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050103350000000003, 2024-09-18T11:30:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 917 2024-09-18T11:30:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169348+0000", 2024-09-18T11:30:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169348+0000", 2024-09-18T11:30:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169347+0000", 2024-09-18T11:30:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169350+0000", 2024-09-18T11:30:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.537e-06 2024-09-18T11:30:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169356+0000", 2024-09-18T11:30:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1129999999999997e-06 2024-09-18T11:30:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169359+0000", 2024-09-18T11:30:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.444e-06 2024-09-18T11:30:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.173900+0000", 2024-09-18T11:30:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045409079999999998 2024-09-18T11:30:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174024+0000", 2024-09-18T11:30:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012400399999999999 2024-09-18T11:30:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174358+0000", 2024-09-18T11:30:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00033376199999999998 2024-09-18T11:30:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:921 4.2 4:425afe9c:::smithi11939506-29:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169430+0000", 2024-09-18T11:30:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.2931150640000002, 2024-09-18T11:30:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038118980000000002, 2024-09-18T11:30:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 921 2024-09-18T11:30:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169430+0000", 2024-09-18T11:30:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169430+0000", 2024-09-18T11:30:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169429+0000", 2024-09-18T11:30:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169433+0000", 2024-09-18T11:30:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9380000000000002e-06 2024-09-18T11:30:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169443+0000", 2024-09-18T11:30:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0149999999999999e-05 2024-09-18T11:30:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:30:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4327e-05 2024-09-18T11:30:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169508+0000", 2024-09-18T11:30:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9743999999999997e-05 2024-09-18T11:30:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169582+0000", 2024-09-18T11:30:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4635999999999995e-05 2024-09-18T11:30:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.173242+0000", 2024-09-18T11:30:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036598220000000001 2024-09-18T11:30:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:994 4.f 4:f269e3d0:::smithi11939506-24:head [read 0~2386561,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164432+0000", 2024-09-18T11:30:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.2981128000000002, 2024-09-18T11:30:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042192219999999999, 2024-09-18T11:30:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:19.516 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 994 2024-09-18T11:30:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164432+0000", 2024-09-18T11:30:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164432+0000", 2024-09-18T11:30:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164431+0000", 2024-09-18T11:30:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164441+0000", 2024-09-18T11:30:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0288e-05 2024-09-18T11:30:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164459+0000", 2024-09-18T11:30:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8331000000000001e-05 2024-09-18T11:30:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164468+0000", 2024-09-18T11:30:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5990000000000002e-06 2024-09-18T11:30:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164520+0000", 2024-09-18T11:30:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1869000000000001e-05 2024-09-18T11:30:19.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164526+0000", 2024-09-18T11:30:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1160000000000004e-06 2024-09-18T11:30:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.166786+0000", 2024-09-18T11:30:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022604069999999999 2024-09-18T11:30:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.166880+0000", 2024-09-18T11:30:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3968999999999994e-05 2024-09-18T11:30:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.168652+0000", 2024-09-18T11:30:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017714300000000001 2024-09-18T11:30:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:995 4.f 4:f269e3d0:::smithi11939506-24:head [read 0~2386561,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:19.521 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164480+0000", 2024-09-18T11:30:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.2980654510000003, 2024-09-18T11:30:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057271170000000003, 2024-09-18T11:30:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 995 2024-09-18T11:30:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164480+0000", 2024-09-18T11:30:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164480+0000", 2024-09-18T11:30:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164479+0000", 2024-09-18T11:30:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164483+0000", 2024-09-18T11:30:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9859999999999996e-06 2024-09-18T11:30:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164497+0000", 2024-09-18T11:30:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3559e-05 2024-09-18T11:30:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164506+0000", 2024-09-18T11:30:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3959999999999995e-06 2024-09-18T11:30:19.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164573+0000", 2024-09-18T11:30:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7123000000000001e-05 2024-09-18T11:30:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164577+0000", 2024-09-18T11:30:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9999999999999998e-06 2024-09-18T11:30:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.168696+0000", 2024-09-18T11:30:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041188379999999997 2024-09-18T11:30:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.168824+0000", 2024-09-18T11:30:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012762099999999999 2024-09-18T11:30:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170207+0000", 2024-09-18T11:30:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001383076 2024-09-18T11:30:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.526 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:996 4.f 4:f269e3d0:::smithi11939506-24:head [read 0~2386561,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164511+0000", 2024-09-18T11:30:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.2980338800000002, 2024-09-18T11:30:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069965679999999999, 2024-09-18T11:30:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:19.527 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 996 2024-09-18T11:30:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164511+0000", 2024-09-18T11:30:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164511+0000", 2024-09-18T11:30:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164511+0000", 2024-09-18T11:30:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164514+0000", 2024-09-18T11:30:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3210000000000001e-06 2024-09-18T11:30:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164521+0000", 2024-09-18T11:30:19.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5169999999999996e-06 2024-09-18T11:30:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164524+0000", 2024-09-18T11:30:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:30:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164674+0000", 2024-09-18T11:30:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014943199999999999 2024-09-18T11:30:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:30:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8579999999999996e-06 2024-09-18T11:30:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170262+0000", 2024-09-18T11:30:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055827719999999997 2024-09-18T11:30:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170405+0000", 2024-09-18T11:30:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000143115 2024-09-18T11:30:19.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.171508+0000", 2024-09-18T11:30:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001102448 2024-09-18T11:30:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:999 4.7 4:e0d95e10:::smithi11939506-22:6 [read 0~2352947,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.2979478430000002, 2024-09-18T11:30:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042304250000000003, 2024-09-18T11:30:19.532 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 999 2024-09-18T11:30:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:30:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.783e-06 2024-09-18T11:30:19.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164609+0000", 2024-09-18T11:30:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4130000000000006e-06 2024-09-18T11:30:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164614+0000", 2024-09-18T11:30:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6469999999999997e-06 2024-09-18T11:30:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164656+0000", 2024-09-18T11:30:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2360000000000001e-05 2024-09-18T11:30:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164660+0000", 2024-09-18T11:30:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3900000000000003e-06 2024-09-18T11:30:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.167605+0000", 2024-09-18T11:30:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029447549999999999 2024-09-18T11:30:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.167653+0000", 2024-09-18T11:30:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8004999999999999e-05 2024-09-18T11:30:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.168828+0000", 2024-09-18T11:30:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001174558 2024-09-18T11:30:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1008 4.2 4:425afe9c:::smithi11939506-29:head [write 1622009~726683] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:19.537 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.257535+0000", 2024-09-18T11:30:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.2050105379999998, 2024-09-18T11:30:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043772960000000001, 2024-09-18T11:30:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1008 2024-09-18T11:30:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.257535+0000", 2024-09-18T11:30:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.257535+0000", 2024-09-18T11:30:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.257534+0000", 2024-09-18T11:30:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.257658+0000", 2024-09-18T11:30:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012445499999999999 2024-09-18T11:30:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.257670+0000", 2024-09-18T11:30:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1751e-05 2024-09-18T11:30:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.257674+0000", 2024-09-18T11:30:19.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4410000000000003e-06 2024-09-18T11:30:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.257706+0000", 2024-09-18T11:30:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2079000000000002e-05 2024-09-18T11:30:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.257760+0000", 2024-09-18T11:30:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3387000000000002e-05 2024-09-18T11:30:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.257839+0000", 2024-09-18T11:30:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8978000000000002e-05 2024-09-18T11:30:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.259997+0000", 2024-09-18T11:30:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002158564 2024-09-18T11:30:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.261859+0000", 2024-09-18T11:30:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001862013 2024-09-18T11:30:19.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.261890+0000", 2024-09-18T11:30:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0477999999999999e-05 2024-09-18T11:30:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.261912+0000", 2024-09-18T11:30:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2022999999999999e-05 2024-09-18T11:30:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1045 4.4 4:2cb3e6a5:::smithi11939506-3:head [write 494139~557703,stat] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.166747+0000", 2024-09-18T11:30:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.2957978269999999, 2024-09-18T11:30:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041058759999999996, 2024-09-18T11:30:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1045 2024-09-18T11:30:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.166747+0000", 2024-09-18T11:30:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.166747+0000", 2024-09-18T11:30:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.166745+0000", 2024-09-18T11:30:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167004+0000", 2024-09-18T11:30:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00025888099999999998 2024-09-18T11:30:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167026+0000", 2024-09-18T11:30:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1764999999999999e-05 2024-09-18T11:30:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167035+0000", 2024-09-18T11:30:19.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3030000000000005e-06 2024-09-18T11:30:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167103+0000", 2024-09-18T11:30:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8419999999999999e-05 2024-09-18T11:30:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167223+0000", 2024-09-18T11:30:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000119391 2024-09-18T11:30:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167457+0000", 2024-09-18T11:30:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000233971 2024-09-18T11:30:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.170754+0000", 2024-09-18T11:30:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032969119999999999 2024-09-18T11:30:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.170789+0000", 2024-09-18T11:30:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5692000000000003e-05 2024-09-18T11:30:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.170823+0000", 2024-09-18T11:30:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3606000000000003e-05 2024-09-18T11:30:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.170853+0000", 2024-09-18T11:30:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.023e-05 2024-09-18T11:30:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1046 4.4 4:2cb3e6a5:::smithi11939506-3:head [write 1455937~734021] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.173120+0000", 2024-09-18T11:30:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.289425493, 2024-09-18T11:30:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051191429999999996, 2024-09-18T11:30:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1046 2024-09-18T11:30:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.550 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173120+0000", 2024-09-18T11:30:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173120+0000", 2024-09-18T11:30:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173119+0000", 2024-09-18T11:30:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173464+0000", 2024-09-18T11:30:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00034509200000000002 2024-09-18T11:30:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173475+0000", 2024-09-18T11:30:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1625e-05 2024-09-18T11:30:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173481+0000", 2024-09-18T11:30:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0639999999999999e-06 2024-09-18T11:30:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173517+0000", 2024-09-18T11:30:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5540000000000002e-05 2024-09-18T11:30:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173581+0000", 2024-09-18T11:30:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3862000000000006e-05 2024-09-18T11:30:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173692+0000", 2024-09-18T11:30:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000111487 2024-09-18T11:30:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.177793+0000", 2024-09-18T11:30:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041005879999999996 2024-09-18T11:30:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.178146+0000", 2024-09-18T11:30:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00035280100000000003 2024-09-18T11:30:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.178196+0000", 2024-09-18T11:30:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0132e-05 2024-09-18T11:30:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.178239+0000", 2024-09-18T11:30:19.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3024e-05 2024-09-18T11:30:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1047 4.4 4:2cb3e6a5:::smithi11939506-3:head [write 2948041~346098] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.179209+0000", 2024-09-18T11:30:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.2833362639999999, 2024-09-18T11:30:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045258060000000003, 2024-09-18T11:30:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:19.556 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1047 2024-09-18T11:30:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179209+0000", 2024-09-18T11:30:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179209+0000", 2024-09-18T11:30:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179208+0000", 2024-09-18T11:30:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179367+0000", 2024-09-18T11:30:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000159087 2024-09-18T11:30:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179378+0000", 2024-09-18T11:30:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1241e-05 2024-09-18T11:30:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179382+0000", 2024-09-18T11:30:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1679999999999997e-06 2024-09-18T11:30:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179420+0000", 2024-09-18T11:30:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8010999999999999e-05 2024-09-18T11:30:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179514+0000", 2024-09-18T11:30:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3820999999999995e-05 2024-09-18T11:30:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179686+0000", 2024-09-18T11:30:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017240800000000001 2024-09-18T11:30:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.181004+0000", 2024-09-18T11:30:19.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013178949999999999 2024-09-18T11:30:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183672+0000", 2024-09-18T11:30:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026672050000000002 2024-09-18T11:30:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183704+0000", 2024-09-18T11:30:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2601999999999997e-05 2024-09-18T11:30:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183735+0000", 2024-09-18T11:30:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0511e-05 2024-09-18T11:30:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1048 4.4 4:2cb3e6a5:::smithi11939506-3:head [setxattr _header (55) in=7b,truncate 3294139] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.179388+0000", 2024-09-18T11:30:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.2831575800000001, 2024-09-18T11:30:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045880979999999997, 2024-09-18T11:30:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1048 2024-09-18T11:30:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179388+0000", 2024-09-18T11:30:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179388+0000", 2024-09-18T11:30:19.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179387+0000", 2024-09-18T11:30:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179390+0000", 2024-09-18T11:30:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6479999999999999e-06 2024-09-18T11:30:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179393+0000", 2024-09-18T11:30:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.331e-06 2024-09-18T11:30:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179395+0000", 2024-09-18T11:30:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2170000000000001e-06 2024-09-18T11:30:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180405+0000", 2024-09-18T11:30:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001009649 2024-09-18T11:30:19.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180478+0000", 2024-09-18T11:30:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3399999999999995e-05 2024-09-18T11:30:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180630+0000", 2024-09-18T11:30:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015126300000000001 2024-09-18T11:30:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183496+0000", 2024-09-18T11:30:19.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002866314 2024-09-18T11:30:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183922+0000", 2024-09-18T11:30:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000425773 2024-09-18T11:30:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183949+0000", 2024-09-18T11:30:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.673e-05 2024-09-18T11:30:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183976+0000", 2024-09-18T11:30:19.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7160000000000001e-05 2024-09-18T11:30:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1049 4.4 4:2cb3e6a5:::smithi11939506-3:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.179416+0000", 2024-09-18T11:30:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.2831290960000001, 2024-09-18T11:30:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046886159999999996, 2024-09-18T11:30:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:19.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1049 2024-09-18T11:30:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179416+0000", 2024-09-18T11:30:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179416+0000", 2024-09-18T11:30:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179416+0000", 2024-09-18T11:30:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179418+0000", 2024-09-18T11:30:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.934e-06 2024-09-18T11:30:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179423+0000", 2024-09-18T11:30:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3909999999999997e-06 2024-09-18T11:30:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179425+0000", 2024-09-18T11:30:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3350000000000001e-06 2024-09-18T11:30:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180816+0000", 2024-09-18T11:30:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013903909999999999 2024-09-18T11:30:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180893+0000", 2024-09-18T11:30:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6893999999999999e-05 2024-09-18T11:30:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.184002+0000", 2024-09-18T11:30:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031096409999999998 2024-09-18T11:30:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.184049+0000", 2024-09-18T11:30:19.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6279000000000002e-05 2024-09-18T11:30:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.184105+0000", 2024-09-18T11:30:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6110000000000003e-05 2024-09-18T11:30:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.573 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.573 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:19.575 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:30:19.575 INFO:tasks.rados.rados.0.smithi119.stdout:717: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-18T11:30:19.575 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:19.572+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:19.593 INFO:tasks.rados.rados.0.smithi119.stdout:712: expect (ObjNum 74 snap 3 seq_num 74) 2024-09-18T11:30:19.676 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:19.674+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:19.691 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:19.691 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:19.691 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:19.691 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:19.700 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:30:19.769 INFO:tasks.rados.rados.0.smithi119.stdout:715: got (2) No such file or directory 2024-09-18T11:30:19.769 INFO:tasks.rados.rados.0.smithi119.stdout:715: got expected ENOENT (src dne) 2024-09-18T11:30:19.769 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 44 (ObjNum 97 snap 0 seq_num 1) clean dne 2024-09-18T11:30:19.769 INFO:tasks.rados.rados.0.smithi119.stdout:718: finishing write tid 1 to smithi11939506-20 2024-09-18T11:30:19.769 INFO:tasks.rados.rados.0.smithi119.stdout:718: finishing write tid 2 to smithi11939506-20 2024-09-18T11:30:19.769 INFO:tasks.rados.rados.0.smithi119.stdout:713: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-18T11:30:19.777 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:19.776+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:19.839 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:19.839 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:19.839 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:19.839 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:19.839 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:19.848 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:30:19.850 INFO:tasks.rados.rados.0.smithi119.stdout:707: done (14 left) 2024-09-18T11:30:19.850 INFO:tasks.rados.rados.0.smithi119.stdout:708: done (13 left) 2024-09-18T11:30:19.850 INFO:tasks.rados.rados.0.smithi119.stdout:709: done (12 left) 2024-09-18T11:30:19.850 INFO:tasks.rados.rados.0.smithi119.stdout:710: done (11 left) 2024-09-18T11:30:19.850 INFO:tasks.rados.rados.0.smithi119.stdout:712: done (10 left) 2024-09-18T11:30:19.850 INFO:tasks.rados.rados.0.smithi119.stdout:713: done (9 left) 2024-09-18T11:30:19.850 INFO:tasks.rados.rados.0.smithi119.stdout:715: done (8 left) 2024-09-18T11:30:19.850 INFO:tasks.rados.rados.0.smithi119.stdout:716: done (7 left) 2024-09-18T11:30:19.850 INFO:tasks.rados.rados.0.smithi119.stdout:717: done (6 left) 2024-09-18T11:30:19.850 INFO:tasks.rados.rados.0.smithi119.stdout:724: set_chunk oid 21 target oid 2024-09-18T11:30:19.852 INFO:tasks.rados.rados.0.smithi119.stdout:724: set_chunk oid 21 offset: 1527808 length: 28672 target oid 18619b678a5c207a971a0aa931604f48162e307c57ecdec450d5f095fe9f32c7 offset: 0 snap -1 2024-09-18T11:30:19.852 INFO:tasks.rados.rados.0.smithi119.stdout:725: read oid 38 snap -1 2024-09-18T11:30:19.852 INFO:tasks.rados.rados.0.smithi119.stdout:725: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-18T11:30:19.852 INFO:tasks.rados.rados.0.smithi119.stdout:726: set_chunk oid 10 target oid 2024-09-18T11:30:19.852 INFO:tasks.rados.rados.0.smithi119.stdout:726: set_chunk oid 10 offset: 0 length: 4000000 target oid smithi11939506- offset: 0 snap -1 2024-09-18T11:30:19.852 INFO:tasks.rados.rados.0.smithi119.stdout:727: read oid 46 snap -1 2024-09-18T11:30:19.852 INFO:tasks.rados.rados.0.smithi119.stdout:727: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-18T11:30:19.853 INFO:tasks.rados.rados.0.smithi119.stdout:714: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-18T11:30:19.879 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:19.877+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:19.936 INFO:tasks.rados.rados.0.smithi119.stdout:720: got (22) Invalid argument 2024-09-18T11:30:19.937 INFO:tasks.rados.rados.0.smithi119.stdout:718: finishing write tid 3 to smithi11939506-20 2024-09-18T11:30:19.937 INFO:tasks.rados.rados.0.smithi119.stdout:718: finishing write tid 4 to smithi11939506-20 2024-09-18T11:30:19.937 INFO:tasks.rados.rados.0.smithi119.stdout:718: finishing write tid 5 to smithi11939506-20 2024-09-18T11:30:19.937 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 20 v 35 (ObjNum 80 snap 4 seq_num 80) dirty exists 2024-09-18T11:30:19.937 INFO:tasks.rados.rados.0.smithi119.stdout:718: left oid 20 (ObjNum 80 snap 4 seq_num 80) 2024-09-18T11:30:19.937 INFO:tasks.rados.rados.0.smithi119.stdout:728: read oid 14 snap -1 2024-09-18T11:30:19.937 INFO:tasks.rados.rados.0.smithi119.stdout:728: expect (ObjNum 66 snap 0 seq_num 66) 2024-09-18T11:30:19.937 INFO:tasks.rados.rados.0.smithi119.stdout:714: done (10 left) 2024-09-18T11:30:19.937 INFO:tasks.rados.rados.0.smithi119.stdout:718: done (9 left) 2024-09-18T11:30:19.937 INFO:tasks.rados.rados.0.smithi119.stdout:720: done (8 left) 2024-09-18T11:30:19.937 INFO:tasks.rados.rados.0.smithi119.stdout:729: set_chunk oid 47 target oid 2024-09-18T11:30:19.950 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:19.950 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:19.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:19.950 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:19.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.951 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4266.0:1 2.3 e16/15 2:c4c92e5a:::rbd_trash:head v 16'1, mlcod=0'0)", 2024-09-18T11:30:19.951 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:50.028963+0000", 2024-09-18T11:30:19.951 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 29.920508699999999, 2024-09-18T11:30:19.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001253021, 2024-09-18T11:30:19.951 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.951 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:19.951 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.028963+0000", 2024-09-18T11:30:19.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.028963+0000", 2024-09-18T11:30:19.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.028963+0000", 2024-09-18T11:30:19.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:19.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.028967+0000", 2024-09-18T11:30:19.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2119999999999997e-06 2024-09-18T11:30:19.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.029011+0000", 2024-09-18T11:30:19.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4011000000000003e-05 2024-09-18T11:30:19.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.029019+0000", 2024-09-18T11:30:19.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.729e-06 2024-09-18T11:30:19.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.029185+0000", 2024-09-18T11:30:19.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016637600000000001 2024-09-18T11:30:19.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.029211+0000", 2024-09-18T11:30:19.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6211e-05 2024-09-18T11:30:19.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:19.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.030175+0000", 2024-09-18T11:30:19.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096347000000000002 2024-09-18T11:30:19.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.030216+0000", 2024-09-18T11:30:19.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.138e-05 2024-09-18T11:30:19.955 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.955 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.955 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.956 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1045 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'23, mlcod=30'20)", 2024-09-18T11:30:19.956 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.167583+0000", 2024-09-18T11:30:19.956 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.7818882770000002, 2024-09-18T11:30:19.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031033380000000002, 2024-09-18T11:30:19.956 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.956 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:19.956 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167583+0000", 2024-09-18T11:30:19.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167583+0000", 2024-09-18T11:30:19.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167582+0000", 2024-09-18T11:30:19.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:19.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167934+0000", 2024-09-18T11:30:19.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000352262 2024-09-18T11:30:19.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167952+0000", 2024-09-18T11:30:19.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7859999999999998e-05 2024-09-18T11:30:19.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167960+0000", 2024-09-18T11:30:19.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9729999999999997e-06 2024-09-18T11:30:19.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.168008+0000", 2024-09-18T11:30:19.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7531000000000003e-05 2024-09-18T11:30:19.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.168032+0000", 2024-09-18T11:30:19.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4443e-05 2024-09-18T11:30:19.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:19.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.170646+0000", 2024-09-18T11:30:19.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026136739999999999 2024-09-18T11:30:19.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.170687+0000", 2024-09-18T11:30:19.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0859000000000003e-05 2024-09-18T11:30:19.960 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.960 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.960 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.960 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1046 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'24, mlcod=30'21)", 2024-09-18T11:30:19.961 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:30:19.961 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.775688497, 2024-09-18T11:30:19.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042519100000000002, 2024-09-18T11:30:19.961 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.961 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:19.961 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:30:19.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:30:19.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173782+0000", 2024-09-18T11:30:19.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:19.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174207+0000", 2024-09-18T11:30:19.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00042479799999999999 2024-09-18T11:30:19.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174224+0000", 2024-09-18T11:30:19.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7513999999999999e-05 2024-09-18T11:30:19.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174232+0000", 2024-09-18T11:30:19.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0840000000000003e-06 2024-09-18T11:30:19.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174297+0000", 2024-09-18T11:30:19.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5215000000000005e-05 2024-09-18T11:30:19.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174336+0000", 2024-09-18T11:30:19.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9403000000000002e-05 2024-09-18T11:30:19.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:19.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.177990+0000", 2024-09-18T11:30:19.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036533709999999999 2024-09-18T11:30:19.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.178035+0000", 2024-09-18T11:30:19.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5495999999999999e-05 2024-09-18T11:30:19.965 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.965 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.965 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.965 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1047 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'25, mlcod=35'23)", 2024-09-18T11:30:19.965 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:30:19.965 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.7697162990000002, 2024-09-18T11:30:19.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038129129999999998, 2024-09-18T11:30:19.965 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.965 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:19.966 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:30:19.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:30:19.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179754+0000", 2024-09-18T11:30:19.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:19.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180088+0000", 2024-09-18T11:30:19.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00033395000000000001 2024-09-18T11:30:19.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180105+0000", 2024-09-18T11:30:19.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7039000000000001e-05 2024-09-18T11:30:19.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180112+0000", 2024-09-18T11:30:19.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3710000000000002e-06 2024-09-18T11:30:19.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180176+0000", 2024-09-18T11:30:19.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3178999999999999e-05 2024-09-18T11:30:19.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180197+0000", 2024-09-18T11:30:19.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1687000000000001e-05 2024-09-18T11:30:19.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:19.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183542+0000", 2024-09-18T11:30:19.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003344877 2024-09-18T11:30:19.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183568+0000", 2024-09-18T11:30:19.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6146e-05 2024-09-18T11:30:19.971 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.971 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.971 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.971 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1048 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'26, mlcod=35'23)", 2024-09-18T11:30:19.971 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.180697+0000", 2024-09-18T11:30:19.971 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.7687741729999997, 2024-09-18T11:30:19.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031436290000000002, 2024-09-18T11:30:19.972 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.972 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:19.972 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180697+0000", 2024-09-18T11:30:19.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180697+0000", 2024-09-18T11:30:19.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180697+0000", 2024-09-18T11:30:19.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:19.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180704+0000", 2024-09-18T11:30:19.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8360000000000003e-06 2024-09-18T11:30:19.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180716+0000", 2024-09-18T11:30:19.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2308e-05 2024-09-18T11:30:19.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180724+0000", 2024-09-18T11:30:19.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6499999999999996e-06 2024-09-18T11:30:19.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180884+0000", 2024-09-18T11:30:19.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016041699999999999 2024-09-18T11:30:19.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180905+0000", 2024-09-18T11:30:19.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1103000000000001e-05 2024-09-18T11:30:19.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:19.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183819+0000", 2024-09-18T11:30:19.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029141800000000002 2024-09-18T11:30:19.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183841+0000", 2024-09-18T11:30:19.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1869e-05 2024-09-18T11:30:19.975 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.975 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.976 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.976 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1105 3.0 e38/21 3:0846b5d5:::18619b678a5c207a971a0aa931604f48162e307c57ecdec450d5f095fe9f32c7:head v 38'55, mlcod=38'53)", 2024-09-18T11:30:19.976 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.850634+0000", 2024-09-18T11:30:19.976 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.098837288999999995, 2024-09-18T11:30:19.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00057272200000000001, 2024-09-18T11:30:19.976 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:19.976 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:19.976 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:19.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:19.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.850634+0000", 2024-09-18T11:30:19.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:19.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.850634+0000", 2024-09-18T11:30:19.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:19.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:19.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.850633+0000", 2024-09-18T11:30:19.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:19.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:19.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.850661+0000", 2024-09-18T11:30:19.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8177000000000001e-05 2024-09-18T11:30:19.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:19.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.850676+0000", 2024-09-18T11:30:19.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4973e-05 2024-09-18T11:30:19.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:19.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.850684+0000", 2024-09-18T11:30:19.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4939999999999997e-06 2024-09-18T11:30:19.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:19.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.850722+0000", 2024-09-18T11:30:19.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8509e-05 2024-09-18T11:30:19.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:19.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.850739+0000", 2024-09-18T11:30:19.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6795000000000001e-05 2024-09-18T11:30:19.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:19.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.851171+0000", 2024-09-18T11:30:19.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00043166100000000001 2024-09-18T11:30:19.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:19.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:19.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:19.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.851207+0000", 2024-09-18T11:30:19.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6304999999999999e-05 2024-09-18T11:30:19.980 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.980 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.980 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.980 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:19.980 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:19.980 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:19.981 INFO:tasks.rados.rados.0.smithi119.stdout:729: set_chunk oid 47 offset: 306176 length: 1835520 target oid afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b offset: 0 snap -1 2024-09-18T11:30:19.981 INFO:tasks.rados.rados.0.smithi119.stdout:730: write oid 16 current snap is 4 2024-09-18T11:30:19.982 INFO:tasks.rados.rados.0.smithi119.stdout:730: seq_num 82 ranges {578174=533948,1515641=485017,2736097=502305} 2024-09-18T11:30:19.983 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:30:19.983 INFO:tasks.rados.rados.0.smithi119.stdout:730: writing smithi11939506-16 from 578174 to 1112122 tid 1 2024-09-18T11:30:19.983 INFO:tasks.rados.rados.0.smithi119.stdout:730: writing smithi11939506-16 from 1515641 to 2000658 tid 2 2024-09-18T11:30:19.983 INFO:tasks.rados.rados.0.smithi119.stdout:730: writing smithi11939506-16 from 2736097 to 3238402 tid 3 2024-09-18T11:30:19.984 INFO:tasks.rados.rados.0.smithi119.stdout:731: set_chunk oid 50 target oid 2024-09-18T11:30:19.984 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:19.979+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:19.991 INFO:tasks.rados.rados.0.smithi119.stdout:731: set_chunk oid 50 offset: 696832 length: 2520064 target oid 95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40 offset: 0 snap -1 2024-09-18T11:30:19.991 INFO:tasks.rados.rados.0.smithi119.stdout:732: snap_create 2024-09-18T11:30:19.991 INFO:tasks.rados.rados.0.smithi119.stdout:719: expect (ObjNum 77 snap 3 seq_num 77) 2024-09-18T11:30:20.082 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:20.080+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:20.089 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:20.089 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:20.089 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:20.089 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:20.097 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:30:20.179 INFO:tasks.rados.rados.0.smithi119.stdout:722: finishing write tid 1 to smithi11939506-11 2024-09-18T11:30:20.180 INFO:tasks.rados.rados.0.smithi119.stdout:722: finishing write tid 2 to smithi11939506-11 2024-09-18T11:30:20.180 INFO:tasks.rados.rados.0.smithi119.stdout:723: got (22) Invalid argument 2024-09-18T11:30:20.180 INFO:tasks.rados.rados.0.smithi119.stdout:722: finishing write tid 3 to smithi11939506-11 2024-09-18T11:30:20.180 INFO:tasks.rados.rados.0.smithi119.stdout:719: done (11 left) 2024-09-18T11:30:20.180 INFO:tasks.rados.rados.0.smithi119.stdout:723: done (10 left) 2024-09-18T11:30:20.180 INFO:tasks.rados.rados.0.smithi119.stdout:732: done (9 left) 2024-09-18T11:30:20.180 INFO:tasks.rados.rados.0.smithi119.stdout:733: tier_promote oid 49 2024-09-18T11:30:20.180 INFO:tasks.rados.rados.0.smithi119.stdout:722: finishing write tid 4 to smithi11939506-11 2024-09-18T11:30:20.180 INFO:tasks.rados.rados.0.smithi119.stdout:722: finishing write tid 5 to smithi11939506-11 2024-09-18T11:30:20.180 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 11 v 31 (ObjNum 81 snap 4 seq_num 81) dirty exists 2024-09-18T11:30:20.180 INFO:tasks.rados.rados.0.smithi119.stdout:722: left oid 11 (ObjNum 81 snap 4 seq_num 81) 2024-09-18T11:30:20.180 INFO:tasks.rados.rados.0.smithi119.stdout:724: finishing set_chunk to oid 21 2024-09-18T11:30:20.181 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 21 set_chunk returned error code -95 offset: 1527808 length: 28672 2024-09-18T11:30:20.181 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 0 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-18T11:30:20.181 INFO:tasks.rados.rados.0.smithi119.stdout:726: finishing set_chunk to oid 10 2024-09-18T11:30:20.181 INFO:tasks.rados.rados.0.smithi119.stdout:726: got expected ENOENT (src dne) 2024-09-18T11:30:20.181 INFO:tasks.rados.rados.0.smithi119.stdout:722: done (9 left) 2024-09-18T11:30:20.181 INFO:tasks.rados.rados.0.smithi119.stdout:724: done (8 left) 2024-09-18T11:30:20.181 INFO:tasks.rados.rados.0.smithi119.stdout:726: done (7 left) 2024-09-18T11:30:20.181 INFO:tasks.rados.rados.0.smithi119.stdout:734: read oid 37 snap -1 2024-09-18T11:30:20.181 INFO:tasks.rados.rados.0.smithi119.stdout:734: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-18T11:30:20.181 INFO:tasks.rados.rados.0.smithi119.stdout:735: read oid 23 snap 4 2024-09-18T11:30:20.181 INFO:tasks.rados.rados.0.smithi119.stdout:735: expect (ObjNum 79 snap 4 seq_num 79) 2024-09-18T11:30:20.181 INFO:tasks.rados.rados.0.smithi119.stdout:736: read oid 8 snap -1 2024-09-18T11:30:20.181 INFO:tasks.rados.rados.0.smithi119.stdout:736: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-18T11:30:20.182 INFO:tasks.rados.rados.0.smithi119.stdout:737: write oid 29 current snap is 5 2024-09-18T11:30:20.182 INFO:tasks.rados.rados.0.smithi119.stdout:737: seq_num 83 ranges {594981=609197,1974123=745028,3279873=115108} 2024-09-18T11:30:20.182 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T11:30:20.183 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:20.181+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:20.186 INFO:tasks.rados.rados.0.smithi119.stdout:737: writing smithi11939506-29 from 594981 to 1204178 tid 1 2024-09-18T11:30:20.194 INFO:tasks.rados.rados.0.smithi119.stdout:737: writing smithi11939506-29 from 1974123 to 2719151 tid 2 2024-09-18T11:30:20.198 INFO:tasks.rados.rados.0.smithi119.stdout:737: writing smithi11939506-29 from 3279873 to 3394981 tid 3 2024-09-18T11:30:20.198 INFO:tasks.rados.rados.0.smithi119.stdout:738: write oid 5 current snap is 5 2024-09-18T11:30:20.198 INFO:tasks.rados.rados.0.smithi119.stdout:738: seq_num 84 ranges {611788=400799,1748958=488686,2907296=504492} 2024-09-18T11:30:20.203 INFO:tasks.rados.rados.0.smithi119.stdout:738: writing smithi11939506-5 from 611788 to 1012587 tid 1 2024-09-18T11:30:20.210 INFO:tasks.rados.rados.0.smithi119.stdout:738: writing smithi11939506-5 from 1748958 to 2237644 tid 2 2024-09-18T11:30:20.217 INFO:tasks.rados.rados.0.smithi119.stdout:738: writing smithi11939506-5 from 2907296 to 3411788 tid 3 2024-09-18T11:30:20.217 INFO:tasks.rados.rados.0.smithi119.stdout:739: write oid 41 current snap is 5 2024-09-18T11:30:20.217 INFO:tasks.rados.rados.0.smithi119.stdout:727: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-18T11:30:20.250 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:20.250 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:20.251 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:20.251 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:20.251 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:20.259 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:30:20.284 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:20.283+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:20.326 INFO:tasks.rados.rados.0.smithi119.stdout:725: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-18T11:30:20.385 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:20.383+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:20.402 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.402 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:899 4.0 4:0aec5e1c:::smithi11939506-50:head [read 0~2974806,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.133065+0000", 2024-09-18T11:30:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.2648860259999992, 2024-09-18T11:30:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044382470000000002, 2024-09-18T11:30:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 899 2024-09-18T11:30:20.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133065+0000", 2024-09-18T11:30:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133065+0000", 2024-09-18T11:30:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133075+0000", 2024-09-18T11:30:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1684e-05 2024-09-18T11:30:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133100+0000", 2024-09-18T11:30:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5840999999999999e-05 2024-09-18T11:30:20.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133109+0000", 2024-09-18T11:30:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6989999999999992e-06 2024-09-18T11:30:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133168+0000", 2024-09-18T11:30:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9268000000000002e-05 2024-09-18T11:30:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133192+0000", 2024-09-18T11:30:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3373000000000001e-05 2024-09-18T11:30:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.135246+0000", 2024-09-18T11:30:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020545310000000001 2024-09-18T11:30:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.135348+0000", 2024-09-18T11:30:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010129899999999999 2024-09-18T11:30:20.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.137503+0000", 2024-09-18T11:30:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0021554690000000001 2024-09-18T11:30:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:900 4.0 4:0aec5e1c:::smithi11939506-50:head [read 0~2974806,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.133226+0000", 2024-09-18T11:30:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.2647246400000007, 2024-09-18T11:30:20.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059578069999999999, 2024-09-18T11:30:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 900 2024-09-18T11:30:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133226+0000", 2024-09-18T11:30:20.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133226+0000", 2024-09-18T11:30:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133225+0000", 2024-09-18T11:30:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133234+0000", 2024-09-18T11:30:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5799999999999992e-06 2024-09-18T11:30:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133242+0000", 2024-09-18T11:30:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4039999999999995e-06 2024-09-18T11:30:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133256+0000", 2024-09-18T11:30:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3278000000000001e-05 2024-09-18T11:30:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133293+0000", 2024-09-18T11:30:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7799000000000002e-05 2024-09-18T11:30:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133298+0000", 2024-09-18T11:30:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9110000000000001e-06 2024-09-18T11:30:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.137554+0000", 2024-09-18T11:30:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042556099999999999 2024-09-18T11:30:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.137713+0000", 2024-09-18T11:30:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015937199999999999 2024-09-18T11:30:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:20.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139184+0000", 2024-09-18T11:30:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001470694 2024-09-18T11:30:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:901 4.0 4:0aec5e1c:::smithi11939506-50:head [sparse-read 0~1781150] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.2646854300000001, 2024-09-18T11:30:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063828720000000004, 2024-09-18T11:30:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:20.415 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 901 2024-09-18T11:30:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:20.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133268+0000", 2024-09-18T11:30:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9610000000000001e-06 2024-09-18T11:30:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:20.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133280+0000", 2024-09-18T11:30:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2481e-05 2024-09-18T11:30:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133285+0000", 2024-09-18T11:30:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5469999999999998e-06 2024-09-18T11:30:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133317+0000", 2024-09-18T11:30:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1838999999999998e-05 2024-09-18T11:30:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133320+0000", 2024-09-18T11:30:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3009999999999998e-06 2024-09-18T11:30:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139225+0000", 2024-09-18T11:30:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059047919999999999 2024-09-18T11:30:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139330+0000", 2024-09-18T11:30:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104924 2024-09-18T11:30:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139648+0000", 2024-09-18T11:30:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00031854799999999999 2024-09-18T11:30:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:30:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.2285313900000006, 2024-09-18T11:30:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:30:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:30:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.421 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:30:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:30:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:20.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:30:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:30:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:30:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:30:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:30:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:30:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:30:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:30:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:30:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:30:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:30:20.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:30:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:30:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1000 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164518+0000", 2024-09-18T11:30:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2334332840000002, 2024-09-18T11:30:20.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005734093, 2024-09-18T11:30:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1000 2024-09-18T11:30:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164518+0000", 2024-09-18T11:30:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164518+0000", 2024-09-18T11:30:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164515+0000", 2024-09-18T11:30:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164525+0000", 2024-09-18T11:30:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5480000000000007e-06 2024-09-18T11:30:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164542+0000", 2024-09-18T11:30:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7643e-05 2024-09-18T11:30:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164553+0000", 2024-09-18T11:30:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0581000000000001e-05 2024-09-18T11:30:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164637+0000", 2024-09-18T11:30:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3825999999999993e-05 2024-09-18T11:30:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164644+0000", 2024-09-18T11:30:20.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5499999999999997e-06 2024-09-18T11:30:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.166446+0000", 2024-09-18T11:30:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018018789999999999 2024-09-18T11:30:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.166537+0000", 2024-09-18T11:30:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0325e-05 2024-09-18T11:30:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:20.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170252+0000", 2024-09-18T11:30:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037150820000000002 2024-09-18T11:30:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1001 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:30:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2333801119999999, 2024-09-18T11:30:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007461082, 2024-09-18T11:30:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:20.431 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1001 2024-09-18T11:30:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:30:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:30:20.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164570+0000", 2024-09-18T11:30:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164574+0000", 2024-09-18T11:30:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.822e-06 2024-09-18T11:30:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:20.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164582+0000", 2024-09-18T11:30:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3410000000000003e-06 2024-09-18T11:30:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164587+0000", 2024-09-18T11:30:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769999999999999e-06 2024-09-18T11:30:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164702+0000", 2024-09-18T11:30:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115705 2024-09-18T11:30:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164709+0000", 2024-09-18T11:30:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-09-18T11:30:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170296+0000", 2024-09-18T11:30:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055876390000000001 2024-09-18T11:30:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170418+0000", 2024-09-18T11:30:20.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121549 2024-09-18T11:30:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172032+0000", 2024-09-18T11:30:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00161407 2024-09-18T11:30:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.436 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.233353396, 2024-09-18T11:30:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:30:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:20.437 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:30:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:20.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:30:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:30:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:30:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:30:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:30:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:30:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:30:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:30:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:30:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:30:20.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:30:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:30:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:30:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:30:20.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:30:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:30:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:30:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2333351070000003, 2024-09-18T11:30:20.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:30:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:30:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:30:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:30:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:30:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:30:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:30:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:20.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:30:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:30:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:30:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:30:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:30:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:30:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:30:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:30:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:30:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:30:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:30:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:30:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:30:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:30:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:20.450 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:30:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2332743380000002, 2024-09-18T11:30:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:30:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:30:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.451 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:20.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:20.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:30:20.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:30:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:30:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:30:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:30:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:30:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:30:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:30:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:30:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:30:20.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:30:20.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:20.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:30:20.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:30:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:30:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:30:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:30:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:30:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:30:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:30:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:30:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2332520169999999, 2024-09-18T11:30:20.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:30:20.458 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:20.458 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:20.458 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:20.458 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:20.458 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.458 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:30:20.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.458 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:20.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:20.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:30:20.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:20.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:30:20.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:20.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:30:20.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:20.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:20.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:30:20.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:30:20.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:20.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:30:20.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:30:20.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:30:20.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:30:20.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:30:20.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:30:20.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:20.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:30:20.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:30:20.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:30:20.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:30:20.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:20.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:30:20.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:30:20.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:20.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:30:20.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:30:20.464 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.464 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:20.464 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.464 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1017 4.0 4:0aec5e1c:::smithi11939506-50:head [write 460525~690852,stat] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:20.464 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.674349+0000", 2024-09-18T11:30:20.464 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.7236015670000002, 2024-09-18T11:30:20.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039298309999999999, 2024-09-18T11:30:20.464 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:20.464 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:20.465 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:20.465 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:20.465 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.465 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1017 2024-09-18T11:30:20.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.465 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:20.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:20.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.674349+0000", 2024-09-18T11:30:20.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:20.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.674349+0000", 2024-09-18T11:30:20.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:20.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.674347+0000", 2024-09-18T11:30:20.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:20.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:20.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.674752+0000", 2024-09-18T11:30:20.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040463499999999998 2024-09-18T11:30:20.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:20.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.674774+0000", 2024-09-18T11:30:20.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1710000000000001e-05 2024-09-18T11:30:20.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.674782+0000", 2024-09-18T11:30:20.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6119999999999999e-06 2024-09-18T11:30:20.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.674827+0000", 2024-09-18T11:30:20.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4512999999999999e-05 2024-09-18T11:30:20.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:20.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.674889+0000", 2024-09-18T11:30:20.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2273999999999994e-05 2024-09-18T11:30:20.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:20.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.675012+0000", 2024-09-18T11:30:20.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012338 2024-09-18T11:30:20.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:20.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.677555+0000", 2024-09-18T11:30:20.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025423809999999998 2024-09-18T11:30:20.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:20.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.678177+0000", 2024-09-18T11:30:20.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00062258400000000003 2024-09-18T11:30:20.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:20.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.678232+0000", 2024-09-18T11:30:20.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4778000000000003e-05 2024-09-18T11:30:20.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:20.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.678279+0000", 2024-09-18T11:30:20.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7068999999999999e-05 2024-09-18T11:30:20.471 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.471 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:20.471 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.471 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1018 4.0 4:0aec5e1c:::smithi11939506-50:head [write 1622620~730352] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:20.471 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:30:20.471 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.7170174610000002, 2024-09-18T11:30:20.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064405160000000003, 2024-09-18T11:30:20.472 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:20.472 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:20.472 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:20.472 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:20.472 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.472 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1018 2024-09-18T11:30:20.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.472 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:20.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:20.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:30:20.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:20.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:30:20.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:20.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680932+0000", 2024-09-18T11:30:20.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:20.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:20.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681334+0000", 2024-09-18T11:30:20.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000401803 2024-09-18T11:30:20.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:20.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681350+0000", 2024-09-18T11:30:20.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6218999999999998e-05 2024-09-18T11:30:20.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681374+0000", 2024-09-18T11:30:20.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4088e-05 2024-09-18T11:30:20.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681415+0000", 2024-09-18T11:30:20.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1307000000000003e-05 2024-09-18T11:30:20.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:20.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681507+0000", 2024-09-18T11:30:20.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1867999999999995e-05 2024-09-18T11:30:20.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:20.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681671+0000", 2024-09-18T11:30:20.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016406 2024-09-18T11:30:20.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:20.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.683856+0000", 2024-09-18T11:30:20.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002184624 2024-09-18T11:30:20.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:20.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687266+0000", 2024-09-18T11:30:20.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034105490000000001 2024-09-18T11:30:20.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:20.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687321+0000", 2024-09-18T11:30:20.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4960999999999998e-05 2024-09-18T11:30:20.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:20.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687374+0000", 2024-09-18T11:30:20.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2553000000000003e-05 2024-09-18T11:30:20.477 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.477 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:20.477 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.477 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1037 4.0 4:019b6e09:::smithi11939506-40:head [write 477332~766101] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:20.477 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.707508+0000", 2024-09-18T11:30:20.478 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.6904427349999995, 2024-09-18T11:30:20.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050976160000000001, 2024-09-18T11:30:20.478 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:20.478 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:20.478 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:20.478 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:20.478 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.478 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1037 2024-09-18T11:30:20.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.478 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:20.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:20.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.707508+0000", 2024-09-18T11:30:20.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:20.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.707508+0000", 2024-09-18T11:30:20.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:20.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.707506+0000", 2024-09-18T11:30:20.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:20.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:20.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708363+0000", 2024-09-18T11:30:20.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00085727299999999998 2024-09-18T11:30:20.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:20.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708388+0000", 2024-09-18T11:30:20.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5031000000000001e-05 2024-09-18T11:30:20.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708396+0000", 2024-09-18T11:30:20.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0870000000000007e-06 2024-09-18T11:30:20.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708452+0000", 2024-09-18T11:30:20.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.533e-05 2024-09-18T11:30:20.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:20.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708563+0000", 2024-09-18T11:30:20.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011086600000000001 2024-09-18T11:30:20.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:20.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708761+0000", 2024-09-18T11:30:20.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019803 2024-09-18T11:30:20.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:20.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.711546+0000", 2024-09-18T11:30:20.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002785689 2024-09-18T11:30:20.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:20.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712507+0000", 2024-09-18T11:30:20.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096094300000000002 2024-09-18T11:30:20.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:20.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712557+0000", 2024-09-18T11:30:20.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9701999999999999e-05 2024-09-18T11:30:20.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:20.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712606+0000", 2024-09-18T11:30:20.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.888e-05 2024-09-18T11:30:20.484 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.484 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:20.484 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.484 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1038 4.0 4:019b6e09:::smithi11939506-40:head [write 1681102~474010] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:20.484 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.712705+0000", 2024-09-18T11:30:20.485 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.685245396, 2024-09-18T11:30:20.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045213800000000002, 2024-09-18T11:30:20.485 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:20.485 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:20.485 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:20.485 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:20.485 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.485 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1038 2024-09-18T11:30:20.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.485 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:20.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:20.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712705+0000", 2024-09-18T11:30:20.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:20.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712705+0000", 2024-09-18T11:30:20.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:20.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712704+0000", 2024-09-18T11:30:20.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:20.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:20.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712926+0000", 2024-09-18T11:30:20.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000221874 2024-09-18T11:30:20.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:20.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712936+0000", 2024-09-18T11:30:20.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0077e-05 2024-09-18T11:30:20.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712942+0000", 2024-09-18T11:30:20.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4299999999999997e-06 2024-09-18T11:30:20.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712976+0000", 2024-09-18T11:30:20.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4279999999999997e-05 2024-09-18T11:30:20.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:20.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.713065+0000", 2024-09-18T11:30:20.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9007999999999999e-05 2024-09-18T11:30:20.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:20.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.713220+0000", 2024-09-18T11:30:20.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015529500000000001 2024-09-18T11:30:20.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:20.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.714970+0000", 2024-09-18T11:30:20.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001749682 2024-09-18T11:30:20.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:20.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.717130+0000", 2024-09-18T11:30:20.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0021598450000000001 2024-09-18T11:30:20.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:20.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.717183+0000", 2024-09-18T11:30:20.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2592000000000001e-05 2024-09-18T11:30:20.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:20.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.717227+0000", 2024-09-18T11:30:20.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4277000000000003e-05 2024-09-18T11:30:20.491 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.491 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:20.491 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.491 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1040 4.0 4:019b6e09:::smithi11939506-40:head [setxattr _header (56) in=7b,truncate 3277332] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:20.491 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.719140+0000", 2024-09-18T11:30:20.491 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.6788107620000003, 2024-09-18T11:30:20.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049512000000000002, 2024-09-18T11:30:20.491 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:20.492 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:20.492 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:20.492 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:20.492 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.492 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1040 2024-09-18T11:30:20.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.492 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:20.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:20.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719140+0000", 2024-09-18T11:30:20.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:20.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719140+0000", 2024-09-18T11:30:20.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:20.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719140+0000", 2024-09-18T11:30:20.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:20.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:20.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719143+0000", 2024-09-18T11:30:20.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5389999999999999e-06 2024-09-18T11:30:20.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:20.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719149+0000", 2024-09-18T11:30:20.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0249999999999999e-06 2024-09-18T11:30:20.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719153+0000", 2024-09-18T11:30:20.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4910000000000001e-06 2024-09-18T11:30:20.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.720365+0000", 2024-09-18T11:30:20.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001212346 2024-09-18T11:30:20.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:20.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.720476+0000", 2024-09-18T11:30:20.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011156500000000001 2024-09-18T11:30:20.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:20.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.720662+0000", 2024-09-18T11:30:20.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018538199999999999 2024-09-18T11:30:20.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:20.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.722050+0000", 2024-09-18T11:30:20.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001388291 2024-09-18T11:30:20.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:20.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.724024+0000", 2024-09-18T11:30:20.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019738860000000002 2024-09-18T11:30:20.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:20.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.724055+0000", 2024-09-18T11:30:20.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1288000000000003e-05 2024-09-18T11:30:20.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:20.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.724091+0000", 2024-09-18T11:30:20.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5960999999999997e-05 2024-09-18T11:30:20.497 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.497 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:20.498 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.498 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1041 4.0 4:019b6e09:::smithi11939506-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:20.498 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.719166+0000", 2024-09-18T11:30:20.498 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.6787851180000004, 2024-09-18T11:30:20.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050746640000000004, 2024-09-18T11:30:20.498 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:20.498 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:20.498 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:20.498 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:20.499 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.499 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1041 2024-09-18T11:30:20.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.499 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:20.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:20.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719166+0000", 2024-09-18T11:30:20.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:20.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719166+0000", 2024-09-18T11:30:20.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:20.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719165+0000", 2024-09-18T11:30:20.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:20.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:20.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719168+0000", 2024-09-18T11:30:20.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6220000000000001e-06 2024-09-18T11:30:20.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:20.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719174+0000", 2024-09-18T11:30:20.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2020000000000001e-06 2024-09-18T11:30:20.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719178+0000", 2024-09-18T11:30:20.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6890000000000002e-06 2024-09-18T11:30:20.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.720928+0000", 2024-09-18T11:30:20.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017504490000000001 2024-09-18T11:30:20.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:20.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.721050+0000", 2024-09-18T11:30:20.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121477 2024-09-18T11:30:20.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.724120+0000", 2024-09-18T11:30:20.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0030705569999999998 2024-09-18T11:30:20.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:20.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.724176+0000", 2024-09-18T11:30:20.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5327000000000001e-05 2024-09-18T11:30:20.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:20.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.724240+0000", 2024-09-18T11:30:20.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4806000000000003e-05 2024-09-18T11:30:20.503 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.503 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:20.503 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.503 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1050 4.0 4:0aec5e1c:::smithi11939506-50:head [write 510946~632952] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:20.504 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.185718+0000", 2024-09-18T11:30:20.504 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.2122324889999998, 2024-09-18T11:30:20.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055213279999999998, 2024-09-18T11:30:20.504 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:20.504 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:20.504 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:20.504 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:20.504 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.504 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1050 2024-09-18T11:30:20.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.504 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:20.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:20.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185718+0000", 2024-09-18T11:30:20.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:20.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185718+0000", 2024-09-18T11:30:20.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:20.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185717+0000", 2024-09-18T11:30:20.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:20.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:20.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185850+0000", 2024-09-18T11:30:20.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000133616 2024-09-18T11:30:20.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:20.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185867+0000", 2024-09-18T11:30:20.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7047000000000001e-05 2024-09-18T11:30:20.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185874+0000", 2024-09-18T11:30:20.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4250000000000003e-06 2024-09-18T11:30:20.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185926+0000", 2024-09-18T11:30:20.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1867999999999999e-05 2024-09-18T11:30:20.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:20.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185985+0000", 2024-09-18T11:30:20.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9769000000000003e-05 2024-09-18T11:30:20.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:20.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.186079+0000", 2024-09-18T11:30:20.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3109000000000006e-05 2024-09-18T11:30:20.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:20.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.187878+0000", 2024-09-18T11:30:20.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001799557 2024-09-18T11:30:20.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:20.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191165+0000", 2024-09-18T11:30:20.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003286892 2024-09-18T11:30:20.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:20.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191196+0000", 2024-09-18T11:30:20.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0607e-05 2024-09-18T11:30:20.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:20.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191240+0000", 2024-09-18T11:30:20.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4078e-05 2024-09-18T11:30:20.510 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.510 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:20.510 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.511 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1051 4.0 4:0aec5e1c:::smithi11939506-50:head [write 1798066~477679] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:20.511 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.191955+0000", 2024-09-18T11:30:20.511 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.2059960319999998, 2024-09-18T11:30:20.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004674246, 2024-09-18T11:30:20.511 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:20.511 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:20.511 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:20.511 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:20.512 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.512 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1051 2024-09-18T11:30:20.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.512 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:20.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:20.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191955+0000", 2024-09-18T11:30:20.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:20.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191955+0000", 2024-09-18T11:30:20.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:20.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191954+0000", 2024-09-18T11:30:20.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:20.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:20.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.192068+0000", 2024-09-18T11:30:20.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113999 2024-09-18T11:30:20.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:20.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.192079+0000", 2024-09-18T11:30:20.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1477999999999999e-05 2024-09-18T11:30:20.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.192083+0000", 2024-09-18T11:30:20.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4170000000000002e-06 2024-09-18T11:30:20.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.192115+0000", 2024-09-18T11:30:20.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1378999999999999e-05 2024-09-18T11:30:20.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:20.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.192170+0000", 2024-09-18T11:30:20.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5631000000000003e-05 2024-09-18T11:30:20.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:20.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.192261+0000", 2024-09-18T11:30:20.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0748000000000006e-05 2024-09-18T11:30:20.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:20.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.193630+0000", 2024-09-18T11:30:20.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013684999999999999 2024-09-18T11:30:20.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:20.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.196567+0000", 2024-09-18T11:30:20.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029371279999999998 2024-09-18T11:30:20.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:20.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.196597+0000", 2024-09-18T11:30:20.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0480999999999998e-05 2024-09-18T11:30:20.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:20.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.196629+0000", 2024-09-18T11:30:20.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1767999999999998e-05 2024-09-18T11:30:20.518 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.518 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:20.518 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.518 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:20.518 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:30:20.518 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.45212713399999999, 2024-09-18T11:30:20.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:30:20.518 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:20.518 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:20.518 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:20.518 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:20.519 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.519 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:30:20.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.519 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:20.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:20.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:30:20.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:20.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:30:20.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:20.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:30:20.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:30:20.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:20.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:30:20.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:30:20.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:20.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:30:20.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:30:20.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:30:20.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:30:20.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:30:20.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:30:20.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:20.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:30:20.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:30:20.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:20.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:30:20.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:30:20.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:20.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:30:20.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:30:20.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:20.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:30:20.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:30:20.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:20.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:30:20.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:30:20.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:20.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:30:20.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:30:20.524 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.524 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:20.524 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.524 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1121 4.f e38/22 4:f5d7df41:::smithi11939506-16:head v 38'34, mlcod=30'32)", 2024-09-18T11:30:20.524 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:30:20.525 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.43213987500000001, 2024-09-18T11:30:20.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060953680000000003, 2024-09-18T11:30:20.525 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:20.525 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:20.525 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:20.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:20.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:30:20.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:20.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:30:20.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:20.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:20.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965810+0000", 2024-09-18T11:30:20.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:20.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:20.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966142+0000", 2024-09-18T11:30:20.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000331591 2024-09-18T11:30:20.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:20.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966154+0000", 2024-09-18T11:30:20.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.253e-05 2024-09-18T11:30:20.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966160+0000", 2024-09-18T11:30:20.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7130000000000002e-06 2024-09-18T11:30:20.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:20.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966196+0000", 2024-09-18T11:30:20.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6276999999999998e-05 2024-09-18T11:30:20.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:20.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966217+0000", 2024-09-18T11:30:20.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0335000000000001e-05 2024-09-18T11:30:20.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:20.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.971872+0000", 2024-09-18T11:30:20.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056549290000000004 2024-09-18T11:30:20.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:20.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:20.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:20.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.971906+0000", 2024-09-18T11:30:20.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4702000000000001e-05 2024-09-18T11:30:20.528 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.529 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:20.529 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.529 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:20.529 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:20.529 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:20.530 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:30:20.531 INFO:tasks.rados.rados.0.smithi119.stdout:728: expect (ObjNum 66 snap 0 seq_num 66) 2024-09-18T11:30:20.531 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:20.529+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:20.608 INFO:tasks.rados.rados.0.smithi119.stdout:729: finishing set_chunk to oid 47 2024-09-18T11:30:20.608 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 47 set_chunk returned error code -95 offset: 306176 length: 1835520 2024-09-18T11:30:20.608 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 0 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-18T11:30:20.609 INFO:tasks.rados.rados.0.smithi119.stdout:730: finishing write tid 1 to smithi11939506-16 2024-09-18T11:30:20.609 INFO:tasks.rados.rados.0.smithi119.stdout:730: finishing write tid 2 to smithi11939506-16 2024-09-18T11:30:20.609 INFO:tasks.rados.rados.0.smithi119.stdout:739: seq_num 85 ranges {628595=476048,1807440=632344,2934719=452474} 2024-09-18T11:30:20.613 INFO:tasks.rados.rados.0.smithi119.stdout:739: writing smithi11939506-41 from 628595 to 1104643 tid 1 2024-09-18T11:30:20.621 INFO:tasks.rados.rados.0.smithi119.stdout:739: writing smithi11939506-41 from 1807440 to 2439784 tid 2 2024-09-18T11:30:20.626 INFO:tasks.rados.rados.0.smithi119.stdout:739: writing smithi11939506-41 from 2934719 to 3387193 tid 3 2024-09-18T11:30:20.626 INFO:tasks.rados.rados.0.smithi119.stdout:725: done (12 left) 2024-09-18T11:30:20.626 INFO:tasks.rados.rados.0.smithi119.stdout:727: done (11 left) 2024-09-18T11:30:20.626 INFO:tasks.rados.rados.0.smithi119.stdout:728: done (10 left) 2024-09-18T11:30:20.626 INFO:tasks.rados.rados.0.smithi119.stdout:729: done (9 left) 2024-09-18T11:30:20.626 INFO:tasks.rados.rados.0.smithi119.stdout:740: set_chunk oid 17 target oid 2024-09-18T11:30:20.632 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:20.630+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:20.632 INFO:tasks.rados.rados.0.smithi119.stdout:740: set_chunk oid 17 offset: 1210880 length: 552448 target oid a156d0210ef9b8565e5530bfcf4ff055bb9dfa5d6799c70a49e41cf6f0d7203f offset: 0 snap -1 2024-09-18T11:30:20.633 INFO:tasks.rados.rados.0.smithi119.stdout:741: write oid 40 current snap is 5 2024-09-18T11:30:20.633 INFO:tasks.rados.rados.0.smithi119.stdout:730: finishing write tid 3 to smithi11939506-16 2024-09-18T11:30:20.633 INFO:tasks.rados.rados.0.smithi119.stdout:741: seq_num 86 ranges {645402=551297,1749569=492355,2845789=599613} 2024-09-18T11:30:20.638 INFO:tasks.rados.rados.0.smithi119.stdout:741: writing smithi11939506-40 from 645402 to 1196699 tid 1 2024-09-18T11:30:20.638 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:20.638 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:20.638 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:20.638 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:20.643 INFO:tasks.rados.rados.0.smithi119.stdout:741: writing smithi11939506-40 from 1749569 to 2241924 tid 2 2024-09-18T11:30:20.648 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:30:20.650 INFO:tasks.rados.rados.0.smithi119.stdout:741: writing smithi11939506-40 from 2845789 to 3445402 tid 3 2024-09-18T11:30:20.651 INFO:tasks.rados.rados.0.smithi119.stdout:742: read oid 47 snap -1 2024-09-18T11:30:20.651 INFO:tasks.rados.rados.0.smithi119.stdout:742: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T11:30:20.651 INFO:tasks.rados.rados.0.smithi119.stdout:743: write oid 1 current snap is 5 2024-09-18T11:30:20.651 INFO:tasks.rados.rados.0.smithi119.stdout:730: finishing write tid 4 to smithi11939506-16 2024-09-18T11:30:20.651 INFO:tasks.rados.rados.0.smithi119.stdout:730: finishing write tid 5 to smithi11939506-16 2024-09-18T11:30:20.651 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 36 (ObjNum 82 snap 4 seq_num 82) dirty exists 2024-09-18T11:30:20.651 INFO:tasks.rados.rados.0.smithi119.stdout:730: left oid 16 (ObjNum 82 snap 4 seq_num 82) 2024-09-18T11:30:20.651 INFO:tasks.rados.rados.0.smithi119.stdout:731: finishing set_chunk to oid 50 2024-09-18T11:30:20.651 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 40 (ObjNum 78 snap 3 seq_num 78) dirty exists 2024-09-18T11:30:20.651 INFO:tasks.rados.rados.0.smithi119.stdout:743: seq_num 87 ranges {662209=626546,1808051=636013,2873212=588997} 2024-09-18T11:30:20.657 INFO:tasks.rados.rados.0.smithi119.stdout:743: writing smithi11939506-1 from 662209 to 1288755 tid 1 2024-09-18T11:30:20.663 INFO:tasks.rados.rados.0.smithi119.stdout:743: writing smithi11939506-1 from 1808051 to 2444064 tid 2 2024-09-18T11:30:20.669 INFO:tasks.rados.rados.0.smithi119.stdout:743: writing smithi11939506-1 from 2873212 to 3462209 tid 3 2024-09-18T11:30:20.670 INFO:tasks.rados.rados.0.smithi119.stdout:730: done (12 left) 2024-09-18T11:30:20.670 INFO:tasks.rados.rados.0.smithi119.stdout:731: done (11 left) 2024-09-18T11:30:20.670 INFO:tasks.rados.rados.0.smithi119.stdout:744: tier_flush oid 36 2024-09-18T11:30:20.670 INFO:tasks.rados.rados.0.smithi119.stdout:744: tier_flush oid 36 snap -1 2024-09-18T11:30:20.670 INFO:tasks.rados.rados.0.smithi119.stdout:733: got (2) No such file or directory 2024-09-18T11:30:20.670 INFO:tasks.rados.rados.0.smithi119.stdout:733: got expected ENOENT (src dne) 2024-09-18T11:30:20.670 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 44 (ObjNum 97 snap 0 seq_num 1) clean dne 2024-09-18T11:30:20.670 INFO:tasks.rados.rados.0.smithi119.stdout:736: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-18T11:30:20.733 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:20.732+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:20.786 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:20.786 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:20.786 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:20.787 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:20.787 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:20.794 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:30:20.801 INFO:tasks.rados.rados.0.smithi119.stdout:737: finishing write tid 1 to smithi11939506-29 2024-09-18T11:30:20.801 INFO:tasks.rados.rados.0.smithi119.stdout:734: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-18T11:30:20.835 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:20.834+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:20.881 INFO:tasks.rados.rados.0.smithi119.stdout:735: expect (ObjNum 79 snap 4 seq_num 79) 2024-09-18T11:30:20.933 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:20.933 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:20.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:20.934 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:20.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.934 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:878 4.e 4:7410c0e0:::smithi11939506-47:2 [read 0~2773122,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:20.934 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116276+0000", 2024-09-18T11:30:20.934 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.8149345029999999, 2024-09-18T11:30:20.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041562079999999998, 2024-09-18T11:30:20.934 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:20.934 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:20.934 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:20.934 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:20.934 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.935 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 878 2024-09-18T11:30:20.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.935 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:20.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:20.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116276+0000", 2024-09-18T11:30:20.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:20.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116276+0000", 2024-09-18T11:30:20.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:20.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116276+0000", 2024-09-18T11:30:20.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:20.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:20.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116279+0000", 2024-09-18T11:30:20.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.704e-06 2024-09-18T11:30:20.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:20.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116289+0000", 2024-09-18T11:30:20.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7210000000000006e-06 2024-09-18T11:30:20.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116293+0000", 2024-09-18T11:30:20.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8739999999999997e-06 2024-09-18T11:30:20.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:20.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118772+0000", 2024-09-18T11:30:20.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024794560000000001 2024-09-18T11:30:20.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:20.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118852+0000", 2024-09-18T11:30:20.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9338999999999994e-05 2024-09-18T11:30:20.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:20.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.120432+0000", 2024-09-18T11:30:20.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015806749999999999 2024-09-18T11:30:20.938 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.938 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:20.938 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.939 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:879 4.e 4:7410c0e0:::smithi11939506-47:2 [sparse-read 0~1211253] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:20.939 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116299+0000", 2024-09-18T11:30:20.939 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.8149115089999999, 2024-09-18T11:30:20.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043627500000000003, 2024-09-18T11:30:20.939 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:20.939 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:20.940 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:20.940 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:20.940 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.940 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 879 2024-09-18T11:30:20.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.940 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:20.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:20.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116299+0000", 2024-09-18T11:30:20.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:20.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116299+0000", 2024-09-18T11:30:20.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:20.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116299+0000", 2024-09-18T11:30:20.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:20.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:20.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116301+0000", 2024-09-18T11:30:20.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6860000000000002e-06 2024-09-18T11:30:20.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:20.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116309+0000", 2024-09-18T11:30:20.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1300000000000003e-06 2024-09-18T11:30:20.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116315+0000", 2024-09-18T11:30:20.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3219999999999998e-06 2024-09-18T11:30:20.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:20.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.120452+0000", 2024-09-18T11:30:20.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041371539999999997 2024-09-18T11:30:20.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:20.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.120521+0000", 2024-09-18T11:30:20.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8674e-05 2024-09-18T11:30:20.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:20.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.120662+0000", 2024-09-18T11:30:20.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000141225 2024-09-18T11:30:20.944 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.944 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:20.944 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.944 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:885 4.d 4:b30010d9:::smithi11939506-43:2 [read 0~2705894,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:20.944 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116520+0000", 2024-09-18T11:30:20.944 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.8146909660000006, 2024-09-18T11:30:20.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029373759999999998, 2024-09-18T11:30:20.944 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:20.944 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:20.944 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:20.944 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:20.945 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.945 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 885 2024-09-18T11:30:20.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.945 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:20.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:20.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116520+0000", 2024-09-18T11:30:20.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:20.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116520+0000", 2024-09-18T11:30:20.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:20.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116519+0000", 2024-09-18T11:30:20.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:20.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:20.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116523+0000", 2024-09-18T11:30:20.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5429999999999998e-06 2024-09-18T11:30:20.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:20.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116530+0000", 2024-09-18T11:30:20.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0639999999999996e-06 2024-09-18T11:30:20.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116534+0000", 2024-09-18T11:30:20.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7000000000000002e-06 2024-09-18T11:30:20.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:20.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118604+0000", 2024-09-18T11:30:20.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020704709999999999 2024-09-18T11:30:20.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:20.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118679+0000", 2024-09-18T11:30:20.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4655999999999998e-05 2024-09-18T11:30:20.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:20.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.119457+0000", 2024-09-18T11:30:20.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00077840400000000001 2024-09-18T11:30:20.948 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.948 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:20.948 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.949 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:886 4.d 4:b30010d9:::smithi11939506-43:2 [sparse-read 0~1186566] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:20.949 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116561+0000", 2024-09-18T11:30:20.949 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.8146498809999994, 2024-09-18T11:30:20.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032690699999999998, 2024-09-18T11:30:20.949 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:20.949 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:20.949 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:20.949 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:20.949 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.949 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 886 2024-09-18T11:30:20.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.949 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:20.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:20.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116561+0000", 2024-09-18T11:30:20.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:20.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116561+0000", 2024-09-18T11:30:20.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:20.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116560+0000", 2024-09-18T11:30:20.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:20.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:20.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116563+0000", 2024-09-18T11:30:20.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.994e-06 2024-09-18T11:30:20.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:20.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116570+0000", 2024-09-18T11:30:20.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7150000000000001e-06 2024-09-18T11:30:20.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116574+0000", 2024-09-18T11:30:20.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5990000000000002e-06 2024-09-18T11:30:20.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:20.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.119485+0000", 2024-09-18T11:30:20.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002911004 2024-09-18T11:30:20.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:20.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.119579+0000", 2024-09-18T11:30:20.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4775e-05 2024-09-18T11:30:20.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:20.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.119830+0000", 2024-09-18T11:30:20.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00025051700000000001 2024-09-18T11:30:20.953 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.953 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:20.953 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.953 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:891 3.3 3:ef319324:::af1dd004311afeb9a2c923c847c83541b2f70de017741e00e9757a0c3b62b102:head [write 0~276992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:20.953 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.118070+0000", 2024-09-18T11:30:20.953 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.8131404599999996, 2024-09-18T11:30:20.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038997620000000002, 2024-09-18T11:30:20.954 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:20.954 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:20.954 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:20.954 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:20.954 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.954 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 891 2024-09-18T11:30:20.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.954 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:20.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:20.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118070+0000", 2024-09-18T11:30:20.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:20.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118070+0000", 2024-09-18T11:30:20.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:20.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118069+0000", 2024-09-18T11:30:20.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:20.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:20.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118576+0000", 2024-09-18T11:30:20.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00050754299999999995 2024-09-18T11:30:20.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:20.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118592+0000", 2024-09-18T11:30:20.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5551000000000001e-05 2024-09-18T11:30:20.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118600+0000", 2024-09-18T11:30:20.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2949999999999994e-06 2024-09-18T11:30:20.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:20.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118638+0000", 2024-09-18T11:30:20.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8229000000000003e-05 2024-09-18T11:30:20.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:20.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118759+0000", 2024-09-18T11:30:20.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012116999999999999 2024-09-18T11:30:20.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:20.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118904+0000", 2024-09-18T11:30:20.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000144816 2024-09-18T11:30:20.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:20.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.120275+0000", 2024-09-18T11:30:20.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00137031 2024-09-18T11:30:20.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:20.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.121915+0000", 2024-09-18T11:30:20.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016402509999999999 2024-09-18T11:30:20.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:20.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.121947+0000", 2024-09-18T11:30:20.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2218999999999998e-05 2024-09-18T11:30:20.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:20.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.121970+0000", 2024-09-18T11:30:20.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2983e-05 2024-09-18T11:30:20.959 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.959 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:20.959 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.960 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.3.13:29 3.3 3:e93cf80d:::smithi11939506-40 low_tier:head [read 2438656~185344 [fadvise_sequential]] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:20.960 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.169975+0000", 2024-09-18T11:30:20.960 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.7612357599999999, 2024-09-18T11:30:20.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003234799, 2024-09-18T11:30:20.960 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:20.960 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:20.960 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:20.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:20.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169975+0000", 2024-09-18T11:30:20.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:20.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169975+0000", 2024-09-18T11:30:20.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:20.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169975+0000", 2024-09-18T11:30:20.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:20.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:20.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169977+0000", 2024-09-18T11:30:20.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.768e-06 2024-09-18T11:30:20.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:20.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169983+0000", 2024-09-18T11:30:20.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0619999999999997e-06 2024-09-18T11:30:20.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169987+0000", 2024-09-18T11:30:20.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6100000000000002e-06 2024-09-18T11:30:20.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:20.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.172692+0000", 2024-09-18T11:30:20.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00270491 2024-09-18T11:30:20.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:20.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.172736+0000", 2024-09-18T11:30:20.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4005000000000003e-05 2024-09-18T11:30:20.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:20.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.173210+0000", 2024-09-18T11:30:20.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00047385100000000001 2024-09-18T11:30:20.963 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.963 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:20.964 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.964 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:988 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~2436982,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:20.964 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.146537+0000", 2024-09-18T11:30:20.964 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.784673293, 2024-09-18T11:30:20.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036166480000000001, 2024-09-18T11:30:20.964 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:20.964 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:20.964 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:20.964 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:20.964 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.964 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 988 2024-09-18T11:30:20.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.965 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:20.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:20.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.146537+0000", 2024-09-18T11:30:20.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:20.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.146537+0000", 2024-09-18T11:30:20.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:20.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.146537+0000", 2024-09-18T11:30:20.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:20.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:20.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.146539+0000", 2024-09-18T11:30:20.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1639999999999999e-06 2024-09-18T11:30:20.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:20.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.146543+0000", 2024-09-18T11:30:20.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.163e-06 2024-09-18T11:30:20.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.146544+0000", 2024-09-18T11:30:20.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9860000000000001e-06 2024-09-18T11:30:20.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:20.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.146637+0000", 2024-09-18T11:30:20.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2134999999999998e-05 2024-09-18T11:30:20.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:20.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.146643+0000", 2024-09-18T11:30:20.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7429999999999996e-06 2024-09-18T11:30:20.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:20.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.149466+0000", 2024-09-18T11:30:20.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028227870000000002 2024-09-18T11:30:20.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:20.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.149567+0000", 2024-09-18T11:30:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101257 2024-09-18T11:30:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.150154+0000", 2024-09-18T11:30:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000586679 2024-09-18T11:30:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1025 4.c 4:323f30dd:::smithi11939506-12:head [sparse-read 0~2047579,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.686094+0000", 2024-09-18T11:30:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.2451171109999999, 2024-09-18T11:30:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034385359999999998, 2024-09-18T11:30:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1025 2024-09-18T11:30:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686094+0000", 2024-09-18T11:30:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686094+0000", 2024-09-18T11:30:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686092+0000", 2024-09-18T11:30:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686101+0000", 2024-09-18T11:30:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.816e-06 2024-09-18T11:30:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686116+0000", 2024-09-18T11:30:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5709000000000002e-05 2024-09-18T11:30:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686125+0000", 2024-09-18T11:30:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2660000000000001e-06 2024-09-18T11:30:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686167+0000", 2024-09-18T11:30:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1916999999999999e-05 2024-09-18T11:30:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686253+0000", 2024-09-18T11:30:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6546999999999997e-05 2024-09-18T11:30:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689532+0000", 2024-09-18T11:30:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032790190000000002 2024-09-18T11:30:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1026 4.c 4:323f30dd:::smithi11939506-12:head [read 0~3176490,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.686142+0000", 2024-09-18T11:30:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.2450687949999999, 2024-09-18T11:30:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048199560000000002, 2024-09-18T11:30:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1026 2024-09-18T11:30:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686142+0000", 2024-09-18T11:30:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686142+0000", 2024-09-18T11:30:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686141+0000", 2024-09-18T11:30:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686145+0000", 2024-09-18T11:30:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0999999999999997e-06 2024-09-18T11:30:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686153+0000", 2024-09-18T11:30:20.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0250000000000002e-06 2024-09-18T11:30:20.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686158+0000", 2024-09-18T11:30:20.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1389999999999997e-06 2024-09-18T11:30:20.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:20.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689573+0000", 2024-09-18T11:30:20.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034151369999999999 2024-09-18T11:30:20.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689630+0000", 2024-09-18T11:30:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6830000000000003e-05 2024-09-18T11:30:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690962+0000", 2024-09-18T11:30:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013323090000000001 2024-09-18T11:30:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1027 4.c 4:323f30dd:::smithi11939506-12:head [sparse-read 0~2047579] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.686162+0000", 2024-09-18T11:30:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.2450487050000003, 2024-09-18T11:30:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050932319999999996, 2024-09-18T11:30:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1027 2024-09-18T11:30:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686162+0000", 2024-09-18T11:30:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686162+0000", 2024-09-18T11:30:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686162+0000", 2024-09-18T11:30:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686164+0000", 2024-09-18T11:30:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3489999999999999e-06 2024-09-18T11:30:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686169+0000", 2024-09-18T11:30:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1039999999999998e-06 2024-09-18T11:30:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686172+0000", 2024-09-18T11:30:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3680000000000002e-06 2024-09-18T11:30:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690990+0000", 2024-09-18T11:30:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048174460000000004 2024-09-18T11:30:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.691056+0000", 2024-09-18T11:30:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6192000000000006e-05 2024-09-18T11:30:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:20.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.691255+0000", 2024-09-18T11:30:20.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000199142 2024-09-18T11:30:20.983 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.983 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:20.983 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.983 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1029 3.3 3:df268f6c:::8da2985a35cccabf5edc5a8900d2327bcfe6ed768fa38d466fa9a390ce713320:head [write 0~356864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:20.983 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.689033+0000", 2024-09-18T11:30:20.983 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.2421775400000001, 2024-09-18T11:30:20.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053345750000000003, 2024-09-18T11:30:20.984 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:20.984 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:20.984 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:20.984 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:20.984 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.984 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1029 2024-09-18T11:30:20.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.984 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:20.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:20.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689033+0000", 2024-09-18T11:30:20.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:20.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689033+0000", 2024-09-18T11:30:20.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:20.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689031+0000", 2024-09-18T11:30:20.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:20.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:20.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690898+0000", 2024-09-18T11:30:20.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001867032 2024-09-18T11:30:20.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:20.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690914+0000", 2024-09-18T11:30:20.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5806000000000001e-05 2024-09-18T11:30:20.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690921+0000", 2024-09-18T11:30:20.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8859999999999998e-06 2024-09-18T11:30:20.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:20.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690977+0000", 2024-09-18T11:30:20.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5612999999999998e-05 2024-09-18T11:30:20.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:20.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.691142+0000", 2024-09-18T11:30:20.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000165112 2024-09-18T11:30:20.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:20.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.691351+0000", 2024-09-18T11:30:20.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020942700000000001 2024-09-18T11:30:20.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:20.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.692616+0000", 2024-09-18T11:30:20.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012648200000000001 2024-09-18T11:30:20.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:20.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.694303+0000", 2024-09-18T11:30:20.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016875320000000001 2024-09-18T11:30:20.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:20.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.694335+0000", 2024-09-18T11:30:20.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1971000000000003e-05 2024-09-18T11:30:20.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:20.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.694368+0000", 2024-09-18T11:30:20.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2403000000000001e-05 2024-09-18T11:30:20.995 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.996 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:20.996 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:20.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.996 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1116 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3109262,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:20.996 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.936633+0000", 2024-09-18T11:30:20.996 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.99457778100000005, 2024-09-18T11:30:20.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003857287, 2024-09-18T11:30:20.996 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:20.996 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:20.996 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:20.997 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:20.997 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:20.997 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1116 2024-09-18T11:30:20.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.997 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:20.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:20.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.936633+0000", 2024-09-18T11:30:20.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:20.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.936633+0000", 2024-09-18T11:30:20.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:20.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:20.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.936633+0000", 2024-09-18T11:30:20.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:20.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:20.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.936636+0000", 2024-09-18T11:30:20.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6959999999999998e-06 2024-09-18T11:30:20.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:20.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.936642+0000", 2024-09-18T11:30:20.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7189999999999998e-06 2024-09-18T11:30:20.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:20.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.936645+0000", 2024-09-18T11:30:20.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4520000000000002e-06 2024-09-18T11:30:20.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:20.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:20.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:20.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.938926+0000", 2024-09-18T11:30:21.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022801409999999999 2024-09-18T11:30:21.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.939055+0000", 2024-09-18T11:30:21.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000129386 2024-09-18T11:30:21.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.940490+0000", 2024-09-18T11:30:21.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014353289999999999 2024-09-18T11:30:21.001 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.001 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.001 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.001 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1133 4.1 4:8b9a985d:::smithi11939506-23:a [read 0~3327753,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:21.001 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.178999+0000", 2024-09-18T11:30:21.001 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.752211568, 2024-09-18T11:30:21.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059312619999999996, 2024-09-18T11:30:21.002 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.002 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:21.002 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:21.002 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:21.002 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:21.002 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1133 2024-09-18T11:30:21.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.002 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178999+0000", 2024-09-18T11:30:21.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178999+0000", 2024-09-18T11:30:21.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178997+0000", 2024-09-18T11:30:21.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:21.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179012+0000", 2024-09-18T11:30:21.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5166e-05 2024-09-18T11:30:21.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179043+0000", 2024-09-18T11:30:21.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0892999999999997e-05 2024-09-18T11:30:21.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179057+0000", 2024-09-18T11:30:21.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4212e-05 2024-09-18T11:30:21.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179217+0000", 2024-09-18T11:30:21.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016031699999999999 2024-09-18T11:30:21.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:21.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179235+0000", 2024-09-18T11:30:21.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7903000000000001e-05 2024-09-18T11:30:21.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.181282+0000", 2024-09-18T11:30:21.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002046614 2024-09-18T11:30:21.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.181350+0000", 2024-09-18T11:30:21.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8190999999999998e-05 2024-09-18T11:30:21.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184930+0000", 2024-09-18T11:30:21.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035805960000000001 2024-09-18T11:30:21.006 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.006 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.006 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.006 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1134 4.1 4:8b9a985d:::smithi11939506-23:a [sparse-read 0~1709094] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:21.006 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:21.007 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.75211778699999998, 2024-09-18T11:30:21.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061866400000000002, 2024-09-18T11:30:21.007 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.007 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:21.007 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:21.007 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:21.007 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:21.007 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1134 2024-09-18T11:30:21.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.007 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:21.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:21.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179092+0000", 2024-09-18T11:30:21.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:21.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179104+0000", 2024-09-18T11:30:21.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2346e-05 2024-09-18T11:30:21.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179128+0000", 2024-09-18T11:30:21.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4046999999999999e-05 2024-09-18T11:30:21.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179141+0000", 2024-09-18T11:30:21.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3149000000000001e-05 2024-09-18T11:30:21.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179406+0000", 2024-09-18T11:30:21.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026496100000000003 2024-09-18T11:30:21.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:21.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179431+0000", 2024-09-18T11:30:21.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5242999999999999e-05 2024-09-18T11:30:21.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184981+0000", 2024-09-18T11:30:21.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055495969999999999 2024-09-18T11:30:21.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185055+0000", 2024-09-18T11:30:21.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4375999999999994e-05 2024-09-18T11:30:21.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185280+0000", 2024-09-18T11:30:21.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022433800000000001 2024-09-18T11:30:21.011 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.011 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.011 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.012 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1135 4.1 4:8b9a985d:::smithi11939506-23:a [sparse-read 0~1709094] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:21.012 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179201+0000", 2024-09-18T11:30:21.012 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.75200933999999997, 2024-09-18T11:30:21.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063123149999999998, 2024-09-18T11:30:21.012 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.012 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:21.012 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:21.012 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:21.012 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:21.012 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1135 2024-09-18T11:30:21.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.012 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179201+0000", 2024-09-18T11:30:21.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179201+0000", 2024-09-18T11:30:21.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179200+0000", 2024-09-18T11:30:21.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:21.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179207+0000", 2024-09-18T11:30:21.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1960000000000002e-06 2024-09-18T11:30:21.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179229+0000", 2024-09-18T11:30:21.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1322999999999999e-05 2024-09-18T11:30:21.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179240+0000", 2024-09-18T11:30:21.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1793e-05 2024-09-18T11:30:21.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179460+0000", 2024-09-18T11:30:21.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000219407 2024-09-18T11:30:21.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:21.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179466+0000", 2024-09-18T11:30:21.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7490000000000004e-06 2024-09-18T11:30:21.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185300+0000", 2024-09-18T11:30:21.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058347490000000002 2024-09-18T11:30:21.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185345+0000", 2024-09-18T11:30:21.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4316000000000001e-05 2024-09-18T11:30:21.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185514+0000", 2024-09-18T11:30:21.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016905800000000001 2024-09-18T11:30:21.017 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.017 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.017 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.017 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1163 4.e 4:7410c0e0:::smithi11939506-47:head [read 0~2773122,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:21.017 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.650381+0000", 2024-09-18T11:30:21.017 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.28082959499999999, 2024-09-18T11:30:21.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030701830000000002, 2024-09-18T11:30:21.017 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.017 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:21.017 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:21.018 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:21.018 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:21.018 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1163 2024-09-18T11:30:21.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.018 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.650381+0000", 2024-09-18T11:30:21.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.650381+0000", 2024-09-18T11:30:21.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.650381+0000", 2024-09-18T11:30:21.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:21.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.650383+0000", 2024-09-18T11:30:21.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4700000000000001e-06 2024-09-18T11:30:21.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.650387+0000", 2024-09-18T11:30:21.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.551e-06 2024-09-18T11:30:21.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.650409+0000", 2024-09-18T11:30:21.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2340999999999998e-05 2024-09-18T11:30:21.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.652231+0000", 2024-09-18T11:30:21.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018218310000000001 2024-09-18T11:30:21.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.652345+0000", 2024-09-18T11:30:21.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011347499999999999 2024-09-18T11:30:21.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.653451+0000", 2024-09-18T11:30:21.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011067869999999999 2024-09-18T11:30:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1164 4.e 4:7410c0e0:::smithi11939506-47:head [sparse-read 0~1211253] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.650412+0000", 2024-09-18T11:30:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.28079840900000003, 2024-09-18T11:30:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032716659999999999, 2024-09-18T11:30:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1164 2024-09-18T11:30:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.650412+0000", 2024-09-18T11:30:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.650412+0000", 2024-09-18T11:30:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.650412+0000", 2024-09-18T11:30:21.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:21.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.650414+0000", 2024-09-18T11:30:21.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5060000000000001e-06 2024-09-18T11:30:21.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.650417+0000", 2024-09-18T11:30:21.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9170000000000002e-06 2024-09-18T11:30:21.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.650419+0000", 2024-09-18T11:30:21.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0729999999999999e-06 2024-09-18T11:30:21.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.653472+0000", 2024-09-18T11:30:21.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003053896 2024-09-18T11:30:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.653544+0000", 2024-09-18T11:30:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1408000000000005e-05 2024-09-18T11:30:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.653684+0000", 2024-09-18T11:30:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014011199999999999 2024-09-18T11:30:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.6.13:26 3.3 3:f159e761:::e59e93d7c4f7efebda4afc5896e5fcf5614bf8edbf483060bccde004b8ac774d:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.678107+0000", 2024-09-18T11:30:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.25310386400000001, 2024-09-18T11:30:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0044766939999999998, 2024-09-18T11:30:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678107+0000", 2024-09-18T11:30:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678107+0000", 2024-09-18T11:30:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678105+0000", 2024-09-18T11:30:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678351+0000", 2024-09-18T11:30:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000245751 2024-09-18T11:30:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678391+0000", 2024-09-18T11:30:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0695000000000003e-05 2024-09-18T11:30:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678405+0000", 2024-09-18T11:30:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.309e-05 2024-09-18T11:30:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678476+0000", 2024-09-18T11:30:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0938000000000004e-05 2024-09-18T11:30:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678573+0000", 2024-09-18T11:30:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7331999999999996e-05 2024-09-18T11:30:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678751+0000", 2024-09-18T11:30:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017789100000000001 2024-09-18T11:30:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.680035+0000", 2024-09-18T11:30:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012847469999999999 2024-09-18T11:30:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682495+0000", 2024-09-18T11:30:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024598599999999999 2024-09-18T11:30:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682535+0000", 2024-09-18T11:30:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9187999999999998e-05 2024-09-18T11:30:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682584+0000", 2024-09-18T11:30:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9048000000000002e-05 2024-09-18T11:30:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.13:31 3.1 e39/21 3:88ab294d:::972cde73ee90bbfa4e1e8219b21f333678dc7dbfb3d5501c6973feb14420b94c:head v 39'68, mlcod=30'66)", 2024-09-18T11:30:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.679882+0000", 2024-09-18T11:30:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.25132855700000001, 2024-09-18T11:30:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002961843, 2024-09-18T11:30:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.679882+0000", 2024-09-18T11:30:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.679882+0000", 2024-09-18T11:30:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.679882+0000", 2024-09-18T11:30:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681303+0000", 2024-09-18T11:30:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001420927 2024-09-18T11:30:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681312+0000", 2024-09-18T11:30:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2410000000000001e-06 2024-09-18T11:30:21.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681316+0000", 2024-09-18T11:30:21.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3839999999999999e-06 2024-09-18T11:30:21.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681353+0000", 2024-09-18T11:30:21.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6910999999999999e-05 2024-09-18T11:30:21.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681366+0000", 2024-09-18T11:30:21.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3138e-05 2024-09-18T11:30:21.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.036 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682812+0000", 2024-09-18T11:30:21.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014455990000000001 2024-09-18T11:30:21.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.036 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682844+0000", 2024-09-18T11:30:21.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1993999999999999e-05 2024-09-18T11:30:21.036 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.036 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.037 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.037 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.13:32 3.1 e39/21 3:94708c4c:::888f815178b2a748104fffa559dcb609f143f519d8737c6aa160c9cce13a6441:head v 39'69, mlcod=30'66)", 2024-09-18T11:30:21.037 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.681793+0000", 2024-09-18T11:30:21.037 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.24941744199999999, 2024-09-18T11:30:21.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037217130000000002, 2024-09-18T11:30:21.037 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.037 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.037 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681793+0000", 2024-09-18T11:30:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681793+0000", 2024-09-18T11:30:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681788+0000", 2024-09-18T11:30:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999952 2024-09-18T11:30:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682280+0000", 2024-09-18T11:30:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00049141099999999997 2024-09-18T11:30:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682289+0000", 2024-09-18T11:30:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8859999999999993e-06 2024-09-18T11:30:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682292+0000", 2024-09-18T11:30:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6839999999999998e-06 2024-09-18T11:30:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682327+0000", 2024-09-18T11:30:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4289999999999999e-05 2024-09-18T11:30:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682340+0000", 2024-09-18T11:30:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3739e-05 2024-09-18T11:30:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.685493+0000", 2024-09-18T11:30:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031523850000000002 2024-09-18T11:30:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.685515+0000", 2024-09-18T11:30:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2279000000000001e-05 2024-09-18T11:30:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.041 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:21.043 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:30:21.044 INFO:tasks.rados.rados.0.smithi119.stdout:737: finishing write tid 2 to smithi11939506-29 2024-09-18T11:30:21.044 INFO:tasks.rados.rados.0.smithi119.stdout:737: finishing write tid 3 to smithi11939506-29 2024-09-18T11:30:21.044 INFO:tasks.rados.rados.0.smithi119.stdout:737: finishing write tid 4 to smithi11939506-29 2024-09-18T11:30:21.044 INFO:tasks.rados.rados.0.smithi119.stdout:737: finishing write tid 5 to smithi11939506-29 2024-09-18T11:30:21.044 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 22 (ObjNum 83 snap 5 seq_num 83) dirty exists 2024-09-18T11:30:21.045 INFO:tasks.rados.rados.0.smithi119.stdout:737: left oid 29 (ObjNum 83 snap 5 seq_num 83) 2024-09-18T11:30:21.045 INFO:tasks.rados.rados.0.smithi119.stdout:733: done (11 left) 2024-09-18T11:30:21.045 INFO:tasks.rados.rados.0.smithi119.stdout:734: done (10 left) 2024-09-18T11:30:21.045 INFO:tasks.rados.rados.0.smithi119.stdout:735: done (9 left) 2024-09-18T11:30:21.045 INFO:tasks.rados.rados.0.smithi119.stdout:736: done (8 left) 2024-09-18T11:30:21.045 INFO:tasks.rados.rados.0.smithi119.stdout:737: done (7 left) 2024-09-18T11:30:21.045 INFO:tasks.rados.rados.0.smithi119.stdout:745: set_chunk oid 46 target oid 2024-09-18T11:30:21.045 INFO:tasks.rados.rados.0.smithi119.stdout:745: set_chunk oid 46 offset: 273408 length: 62464 target oid 5a2e8ef2723c8f5d1d297f883c14f845fd89b9e4d6ed4dcb2cc1b0214d9bc578 offset: 0 snap -1 2024-09-18T11:30:21.045 INFO:tasks.rados.rados.0.smithi119.stdout:746: write (excl) oid 22 current snap is 5 2024-09-18T11:30:21.045 INFO:tasks.rados.rados.0.smithi119.stdout:738: finishing write tid 1 to smithi11939506-5 2024-09-18T11:30:21.045 INFO:tasks.rados.rados.0.smithi119.stdout:738: finishing write tid 2 to smithi11939506-5 2024-09-18T11:30:21.045 INFO:tasks.rados.rados.0.smithi119.stdout:738: finishing write tid 3 to smithi11939506-5 2024-09-18T11:30:21.045 INFO:tasks.rados.rados.0.smithi119.stdout:738: finishing write tid 4 to smithi11939506-5 2024-09-18T11:30:21.045 INFO:tasks.rados.rados.0.smithi119.stdout:738: finishing write tid 5 to smithi11939506-5 2024-09-18T11:30:21.046 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 33 (ObjNum 84 snap 5 seq_num 84) dirty exists 2024-09-18T11:30:21.046 INFO:tasks.rados.rados.0.smithi119.stdout:738: left oid 5 (ObjNum 84 snap 5 seq_num 84) 2024-09-18T11:30:21.046 INFO:tasks.rados.rados.0.smithi119.stdout:739: finishing write tid 1 to smithi11939506-41 2024-09-18T11:30:21.046 INFO:tasks.rados.rados.0.smithi119.stdout:739: finishing write tid 2 to smithi11939506-41 2024-09-18T11:30:21.046 INFO:tasks.rados.rados.0.smithi119.stdout:739: finishing write tid 3 to smithi11939506-41 2024-09-18T11:30:21.046 INFO:tasks.rados.rados.0.smithi119.stdout:739: finishing write tid 4 to smithi11939506-41 2024-09-18T11:30:21.046 INFO:tasks.rados.rados.0.smithi119.stdout:746: seq_num 88 ranges {679016=701795,1866533=496024,2900635=402643} 2024-09-18T11:30:21.046 INFO:tasks.rados.rados.0.smithi119.stdout:746: writing smithi11939506-22 from 679016 to 1380811 tid 1 2024-09-18T11:30:21.046 INFO:tasks.rados.rados.0.smithi119.stdout:746: writing smithi11939506-22 from 1866533 to 2362557 tid 2 2024-09-18T11:30:21.046 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:21.040+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:21.047 INFO:tasks.rados.rados.0.smithi119.stdout:746: writing smithi11939506-22 from 2900635 to 3303278 tid 3 2024-09-18T11:30:21.048 INFO:tasks.rados.rados.0.smithi119.stdout:738: done (8 left) 2024-09-18T11:30:21.048 INFO:tasks.rados.rados.0.smithi119.stdout:747: delete oid 39 current snap is 5 2024-09-18T11:30:21.048 INFO:tasks.rados.rados.0.smithi119.stdout:739: finishing write tid 5 to smithi11939506-41 2024-09-18T11:30:21.048 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 41 v 53 (ObjNum 85 snap 5 seq_num 85) dirty exists 2024-09-18T11:30:21.048 INFO:tasks.rados.rados.0.smithi119.stdout:739: left oid 41 (ObjNum 85 snap 5 seq_num 85) 2024-09-18T11:30:21.048 INFO:tasks.rados.rados.0.smithi119.stdout:740: finishing set_chunk to oid 17 2024-09-18T11:30:21.048 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 17 set_chunk returned error code -95 offset: 1210880 length: 552448 2024-09-18T11:30:21.048 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 0 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-18T11:30:21.048 INFO:tasks.rados.rados.0.smithi119.stdout:741: finishing write tid 1 to smithi11939506-40 2024-09-18T11:30:21.049 INFO:tasks.rados.rados.0.smithi119.stdout:741: finishing write tid 2 to smithi11939506-40 2024-09-18T11:30:21.049 INFO:tasks.rados.rados.0.smithi119.stdout:741: finishing write tid 3 to smithi11939506-40 2024-09-18T11:30:21.049 INFO:tasks.rados.rados.0.smithi119.stdout:741: finishing write tid 4 to smithi11939506-40 2024-09-18T11:30:21.049 INFO:tasks.rados.rados.0.smithi119.stdout:741: finishing write tid 5 to smithi11939506-40 2024-09-18T11:30:21.049 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 50 (ObjNum 86 snap 5 seq_num 86) dirty exists 2024-09-18T11:30:21.049 INFO:tasks.rados.rados.0.smithi119.stdout:741: left oid 40 (ObjNum 86 snap 5 seq_num 86) 2024-09-18T11:30:21.049 INFO:tasks.rados.rados.0.smithi119.stdout:742: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T11:30:21.134 INFO:tasks.rados.rados.0.smithi119.stdout:743: finishing write tid 1 to smithi11939506-1 2024-09-18T11:30:21.134 INFO:tasks.rados.rados.0.smithi119.stdout:743: finishing write tid 2 to smithi11939506-1 2024-09-18T11:30:21.134 INFO:tasks.rados.rados.0.smithi119.stdout:743: finishing write tid 3 to smithi11939506-1 2024-09-18T11:30:21.135 INFO:tasks.rados.rados.0.smithi119.stdout:739: done (8 left) 2024-09-18T11:30:21.135 INFO:tasks.rados.rados.0.smithi119.stdout:740: done (7 left) 2024-09-18T11:30:21.135 INFO:tasks.rados.rados.0.smithi119.stdout:741: done (6 left) 2024-09-18T11:30:21.135 INFO:tasks.rados.rados.0.smithi119.stdout:742: done (5 left) 2024-09-18T11:30:21.135 INFO:tasks.rados.rados.0.smithi119.stdout:747: done (4 left) 2024-09-18T11:30:21.135 INFO:tasks.rados.rados.0.smithi119.stdout:748: write (excl) oid 23 current snap is 5 2024-09-18T11:30:21.135 INFO:tasks.rados.rados.0.smithi119.stdout:743: finishing write tid 4 to smithi11939506-1 2024-09-18T11:30:21.135 INFO:tasks.rados.rados.0.smithi119.stdout:743: finishing write tid 5 to smithi11939506-1 2024-09-18T11:30:21.135 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 43 (ObjNum 87 snap 5 seq_num 87) dirty exists 2024-09-18T11:30:21.135 INFO:tasks.rados.rados.0.smithi119.stdout:743: left oid 1 (ObjNum 87 snap 5 seq_num 87) 2024-09-18T11:30:21.135 INFO:tasks.rados.rados.0.smithi119.stdout:744: got (0) Success 2024-09-18T11:30:21.135 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 36 v 0 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-18T11:30:21.135 INFO:tasks.rados.rados.0.smithi119.stdout:748: seq_num 90 ranges {712630=452293,1583497=499693,2555481=425187} 2024-09-18T11:30:21.140 INFO:tasks.rados.rados.0.smithi119.stdout:748: writing smithi11939506-23 from 712630 to 1164923 tid 1 2024-09-18T11:30:21.143 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:21.142+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:21.145 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:21.145 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:21.145 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:21.145 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:21.145 INFO:tasks.rados.rados.0.smithi119.stdout:748: writing smithi11939506-23 from 1583497 to 2083190 tid 2 2024-09-18T11:30:21.150 INFO:tasks.rados.rados.0.smithi119.stdout:748: writing smithi11939506-23 from 2555481 to 2980668 tid 3 2024-09-18T11:30:21.150 INFO:tasks.rados.rados.0.smithi119.stdout:743: done (4 left) 2024-09-18T11:30:21.150 INFO:tasks.rados.rados.0.smithi119.stdout:744: done (3 left) 2024-09-18T11:30:21.150 INFO:tasks.rados.rados.0.smithi119.stdout:749: write oid 37 current snap is 5 2024-09-18T11:30:21.150 INFO:tasks.rados.rados.0.smithi119.stdout:745: finishing set_chunk to oid 46 2024-09-18T11:30:21.150 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 9 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-18T11:30:21.150 INFO:tasks.rados.rados.0.smithi119.stdout:746: finishing write tid 1 to smithi11939506-22 2024-09-18T11:30:21.150 INFO:tasks.rados.rados.0.smithi119.stdout:746: finishing write tid 2 to smithi11939506-22 2024-09-18T11:30:21.151 INFO:tasks.rados.rados.0.smithi119.stdout:749: seq_num 91 ranges {729437=527542,1925626=643351,3266551=262886} 2024-09-18T11:30:21.155 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:30:21.156 INFO:tasks.rados.rados.0.smithi119.stdout:749: writing smithi11939506-37 from 729437 to 1256979 tid 1 2024-09-18T11:30:21.163 INFO:tasks.rados.rados.0.smithi119.stdout:749: writing smithi11939506-37 from 1925626 to 2568977 tid 2 2024-09-18T11:30:21.168 INFO:tasks.rados.rados.0.smithi119.stdout:749: writing smithi11939506-37 from 3266551 to 3529437 tid 3 2024-09-18T11:30:21.169 INFO:tasks.rados.rados.0.smithi119.stdout:745: done (3 left) 2024-09-18T11:30:21.169 INFO:tasks.rados.rados.0.smithi119.stdout:750: write (excl) oid 29 current snap is 5 2024-09-18T11:30:21.169 INFO:tasks.rados.rados.0.smithi119.stdout:750: seq_num 92 ranges {746244=719144,2100461=503362,3126680=419564} 2024-09-18T11:30:21.176 INFO:tasks.rados.rados.0.smithi119.stdout:750: writing smithi11939506-29 from 746244 to 1465388 tid 1 2024-09-18T11:30:21.179 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is unset 2024-09-18T11:30:21.183 INFO:tasks.rados.rados.0.smithi119.stdout:750: writing smithi11939506-29 from 2100461 to 2603823 tid 2 2024-09-18T11:30:21.189 INFO:tasks.rados.rados.0.smithi119.stdout:750: writing smithi11939506-29 from 3126680 to 3546244 tid 3 2024-09-18T11:30:21.189 INFO:tasks.rados.rados.0.smithi119.stdout:751: read oid 39 snap 4 2024-09-18T11:30:21.189 INFO:tasks.rados.rados.0.smithi119.stdout:751: expect (ObjNum 38 snap 0 seq_num 38) 2024-09-18T11:30:21.189 INFO:tasks.rados.rados.0.smithi119.stdout:752: tier_promote oid 24 2024-09-18T11:30:21.189 INFO:tasks.rados.rados.0.smithi119.stdout:746: finishing write tid 3 to smithi11939506-22 2024-09-18T11:30:21.189 INFO:tasks.rados.rados.0.smithi119.stdout:746: finishing write tid 4 to smithi11939506-22 2024-09-18T11:30:21.189 INFO:tasks.rados.rados.0.smithi119.stdout:746: finishing write tid 5 to smithi11939506-22 2024-09-18T11:30:21.189 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 48 (ObjNum 88 snap 5 seq_num 88) dirty exists 2024-09-18T11:30:21.190 INFO:tasks.rados.rados.0.smithi119.stdout:746: left oid 22 (ObjNum 88 snap 5 seq_num 88) 2024-09-18T11:30:21.190 INFO:tasks.rados.rados.0.smithi119.stdout:748: finishing write tid 1 to smithi11939506-23 2024-09-18T11:30:21.190 INFO:tasks.rados.rados.0.smithi119.stdout:748: finishing write tid 2 to smithi11939506-23 2024-09-18T11:30:21.190 INFO:tasks.rados.rados.0.smithi119.stdout:748: finishing write tid 3 to smithi11939506-23 2024-09-18T11:30:21.190 INFO:tasks.rados.rados.0.smithi119.stdout:746: done (5 left) 2024-09-18T11:30:21.190 INFO:tasks.rados.rados.0.smithi119.stdout:753: write oid 50 current snap is 5 2024-09-18T11:30:21.190 INFO:tasks.rados.rados.0.smithi119.stdout:748: finishing write tid 4 to smithi11939506-23 2024-09-18T11:30:21.190 INFO:tasks.rados.rados.0.smithi119.stdout:748: finishing write tid 5 to smithi11939506-23 2024-09-18T11:30:21.190 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 48 (ObjNum 90 snap 5 seq_num 90) dirty exists 2024-09-18T11:30:21.190 INFO:tasks.rados.rados.0.smithi119.stdout:748: left oid 23 (ObjNum 90 snap 5 seq_num 90) 2024-09-18T11:30:21.190 INFO:tasks.rados.rados.0.smithi119.stdout:749: finishing write tid 1 to smithi11939506-37 2024-09-18T11:30:21.190 INFO:tasks.rados.rados.0.smithi119.stdout:749: finishing write tid 2 to smithi11939506-37 2024-09-18T11:30:21.190 INFO:tasks.rados.rados.0.smithi119.stdout:749: finishing write tid 3 to smithi11939506-37 2024-09-18T11:30:21.190 INFO:tasks.rados.rados.0.smithi119.stdout:749: finishing write tid 4 to smithi11939506-37 2024-09-18T11:30:21.191 INFO:tasks.rados.rados.0.smithi119.stdout:749: finishing write tid 5 to smithi11939506-37 2024-09-18T11:30:21.191 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 37 (ObjNum 91 snap 5 seq_num 91) dirty exists 2024-09-18T11:30:21.191 INFO:tasks.rados.rados.0.smithi119.stdout:749: left oid 37 (ObjNum 91 snap 5 seq_num 91) 2024-09-18T11:30:21.191 INFO:tasks.rados.rados.0.smithi119.stdout:753: seq_num 93 ranges {763051=794393,2158943=647020,3437750=125301} 2024-09-18T11:30:21.197 INFO:tasks.rados.rados.0.smithi119.stdout:753: writing smithi11939506-50 from 763051 to 1557444 tid 1 2024-09-18T11:30:21.204 INFO:tasks.rados.rados.0.smithi119.stdout:753: writing smithi11939506-50 from 2158943 to 2805963 tid 2 2024-09-18T11:30:21.210 INFO:tasks.rados.rados.0.smithi119.stdout:753: writing smithi11939506-50 from 3437750 to 3563051 tid 3 2024-09-18T11:30:21.210 INFO:tasks.rados.rados.0.smithi119.stdout:748: done (5 left) 2024-09-18T11:30:21.210 INFO:tasks.rados.rados.0.smithi119.stdout:749: done (4 left) 2024-09-18T11:30:21.210 INFO:tasks.rados.rados.0.smithi119.stdout:754: write (excl) oid 15 current snap is 5 2024-09-18T11:30:21.210 INFO:tasks.rados.rados.0.smithi119.stdout:754: seq_num 94 ranges {779858=469642,1817425=507031,2781526=586628} 2024-09-18T11:30:21.215 INFO:tasks.rados.rados.0.smithi119.stdout:754: writing smithi11939506-15 from 779858 to 1249500 tid 1 2024-09-18T11:30:21.221 INFO:tasks.rados.rados.0.smithi119.stdout:754: writing smithi11939506-15 from 1817425 to 2324456 tid 2 2024-09-18T11:30:21.227 INFO:tasks.rados.rados.0.smithi119.stdout:754: writing smithi11939506-15 from 2781526 to 3368154 tid 3 2024-09-18T11:30:21.227 INFO:tasks.rados.rados.0.smithi119.stdout:755: delete oid 33 current snap is 5 2024-09-18T11:30:21.227 INFO:tasks.rados.rados.0.smithi119.stdout:750: finishing write tid 1 to smithi11939506-29 2024-09-18T11:30:21.227 INFO:tasks.rados.rados.0.smithi119.stdout:750: finishing write tid 2 to smithi11939506-29 2024-09-18T11:30:21.227 INFO:tasks.rados.rados.0.smithi119.stdout:750: finishing write tid 3 to smithi11939506-29 2024-09-18T11:30:21.227 INFO:tasks.rados.rados.0.smithi119.stdout:750: finishing write tid 4 to smithi11939506-29 2024-09-18T11:30:21.227 INFO:tasks.rados.rados.0.smithi119.stdout:751: expect (ObjNum 38 snap 0 seq_num 38) 2024-09-18T11:30:21.244 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:21.242+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:21.295 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:21.295 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:21.295 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:21.295 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:21.295 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:21.304 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:30:21.314 INFO:tasks.rados.rados.0.smithi119.stdout:750: finishing write tid 5 to smithi11939506-29 2024-09-18T11:30:21.314 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 26 (ObjNum 92 snap 5 seq_num 92) dirty exists 2024-09-18T11:30:21.314 INFO:tasks.rados.rados.0.smithi119.stdout:750: left oid 29 (ObjNum 92 snap 5 seq_num 92) 2024-09-18T11:30:21.314 INFO:tasks.rados.rados.0.smithi119.stdout:750: done (5 left) 2024-09-18T11:30:21.314 INFO:tasks.rados.rados.0.smithi119.stdout:751: done (4 left) 2024-09-18T11:30:21.315 INFO:tasks.rados.rados.0.smithi119.stdout:755: done (3 left) 2024-09-18T11:30:21.315 INFO:tasks.rados.rados.0.smithi119.stdout:756: tier_evict oid 19 2024-09-18T11:30:21.315 INFO:tasks.rados.rados.0.smithi119.stdout:756: tier_evict oid 19 snap -1 2024-09-18T11:30:21.315 INFO:tasks.rados.rados.0.smithi119.stdout:752: got (0) Success 2024-09-18T11:30:21.315 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 9 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-18T11:30:21.315 INFO:tasks.rados.rados.0.smithi119.stdout:753: finishing write tid 1 to smithi11939506-50 2024-09-18T11:30:21.315 INFO:tasks.rados.rados.0.smithi119.stdout:753: finishing write tid 2 to smithi11939506-50 2024-09-18T11:30:21.315 INFO:tasks.rados.rados.0.smithi119.stdout:753: finishing write tid 3 to smithi11939506-50 2024-09-18T11:30:21.315 INFO:tasks.rados.rados.0.smithi119.stdout:753: finishing write tid 4 to smithi11939506-50 2024-09-18T11:30:21.315 INFO:tasks.rados.rados.0.smithi119.stdout:753: finishing write tid 5 to smithi11939506-50 2024-09-18T11:30:21.315 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 55 (ObjNum 93 snap 5 seq_num 93) dirty exists 2024-09-18T11:30:21.315 INFO:tasks.rados.rados.0.smithi119.stdout:753: left oid 50 (ObjNum 93 snap 5 seq_num 93) 2024-09-18T11:30:21.315 INFO:tasks.rados.rados.0.smithi119.stdout:754: finishing write tid 1 to smithi11939506-15 2024-09-18T11:30:21.316 INFO:tasks.rados.rados.0.smithi119.stdout:754: finishing write tid 2 to smithi11939506-15 2024-09-18T11:30:21.316 INFO:tasks.rados.rados.0.smithi119.stdout:752: done (3 left) 2024-09-18T11:30:21.316 INFO:tasks.rados.rados.0.smithi119.stdout:753: done (2 left) 2024-09-18T11:30:21.316 INFO:tasks.rados.rados.0.smithi119.stdout:757: read oid 49 snap 4 2024-09-18T11:30:21.316 INFO:tasks.rados.rados.0.smithi119.stdout:757: expect deleted 2024-09-18T11:30:21.316 INFO:tasks.rados.rados.0.smithi119.stdout:758: snap_create 2024-09-18T11:30:21.316 INFO:tasks.rados.rados.0.smithi119.stdout:754: finishing write tid 3 to smithi11939506-15 2024-09-18T11:30:21.316 INFO:tasks.rados.rados.0.smithi119.stdout:754: finishing write tid 4 to smithi11939506-15 2024-09-18T11:30:21.316 INFO:tasks.rados.rados.0.smithi119.stdout:754: finishing write tid 5 to smithi11939506-15 2024-09-18T11:30:21.316 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 17 (ObjNum 94 snap 5 seq_num 94) dirty exists 2024-09-18T11:30:21.316 INFO:tasks.rados.rados.0.smithi119.stdout:754: left oid 15 (ObjNum 94 snap 5 seq_num 94) 2024-09-18T11:30:21.316 INFO:tasks.rados.rados.0.smithi119.stdout:756: got (22) Invalid argument 2024-09-18T11:30:21.347 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:21.346+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:21.390 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:30:21.403 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:21.404 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:21.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:21.404 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:21.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.404 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:30:21.404 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:30:21.404 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.375176001, 2024-09-18T11:30:21.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073601550000000002, 2024-09-18T11:30:21.404 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.404 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1 2024-09-18T11:30:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:30:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:30:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026524+0000", 2024-09-18T11:30:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999905 2024-09-18T11:30:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026541+0000", 2024-09-18T11:30:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7070000000000001e-05 2024-09-18T11:30:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026554+0000", 2024-09-18T11:30:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.288e-05 2024-09-18T11:30:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026564+0000", 2024-09-18T11:30:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0197000000000001e-05 2024-09-18T11:30:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026826+0000", 2024-09-18T11:30:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026212400000000003 2024-09-18T11:30:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:30:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026839+0000", 2024-09-18T11:30:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2831e-05 2024-09-18T11:30:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028605+0000", 2024-09-18T11:30:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001765794 2024-09-18T11:30:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028709+0000", 2024-09-18T11:30:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104348 2024-09-18T11:30:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028824+0000", 2024-09-18T11:30:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011527399999999999 2024-09-18T11:30:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030319+0000", 2024-09-18T11:30:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001494439 2024-09-18T11:30:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033805+0000", 2024-09-18T11:30:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034857820000000002 2024-09-18T11:30:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033856+0000", 2024-09-18T11:30:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1687999999999997e-05 2024-09-18T11:30:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033894+0000", 2024-09-18T11:30:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7481000000000002e-05 2024-09-18T11:30:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T11:30:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:51.038968+0000", 2024-09-18T11:30:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 30.362741806999999, 2024-09-18T11:30:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028722909999999999, 2024-09-18T11:30:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 5 2024-09-18T11:30:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.038968+0000", 2024-09-18T11:30:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.038968+0000", 2024-09-18T11:30:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.038966+0000", 2024-09-18T11:30:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.038983+0000", 2024-09-18T11:30:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7099e-05 2024-09-18T11:30:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.039002+0000", 2024-09-18T11:30:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8938000000000001e-05 2024-09-18T11:30:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.039012+0000", 2024-09-18T11:30:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9529999999999994e-06 2024-09-18T11:30:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.039068+0000", 2024-09-18T11:30:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5893999999999997e-05 2024-09-18T11:30:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.039074+0000", 2024-09-18T11:30:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8510000000000001e-06 2024-09-18T11:30:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.040586+0000", 2024-09-18T11:30:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015119809999999999 2024-09-18T11:30:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.040705+0000", 2024-09-18T11:30:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000119054 2024-09-18T11:30:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.040812+0000", 2024-09-18T11:30:21.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010652700000000001 2024-09-18T11:30:21.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:21.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041405+0000", 2024-09-18T11:30:21.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059319099999999997 2024-09-18T11:30:21.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041787+0000", 2024-09-18T11:30:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00038150800000000002 2024-09-18T11:30:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041817+0000", 2024-09-18T11:30:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0091e-05 2024-09-18T11:30:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041840+0000", 2024-09-18T11:30:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3385000000000001e-05 2024-09-18T11:30:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T11:30:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:30:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 29.35589757, 2024-09-18T11:30:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055107869999999996, 2024-09-18T11:30:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 7 2024-09-18T11:30:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:30:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:30:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045810+0000", 2024-09-18T11:30:21.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:21.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045820+0000", 2024-09-18T11:30:21.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0215e-05 2024-09-18T11:30:21.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045838+0000", 2024-09-18T11:30:21.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7849000000000001e-05 2024-09-18T11:30:21.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045845+0000", 2024-09-18T11:30:21.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4229999999999999e-06 2024-09-18T11:30:21.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045898+0000", 2024-09-18T11:30:21.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2537999999999997e-05 2024-09-18T11:30:21.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:21.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045904+0000", 2024-09-18T11:30:21.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4019999999999999e-06 2024-09-18T11:30:21.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047663+0000", 2024-09-18T11:30:21.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017584079999999999 2024-09-18T11:30:21.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047736+0000", 2024-09-18T11:30:21.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3509000000000004e-05 2024-09-18T11:30:21.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:21.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047857+0000", 2024-09-18T11:30:21.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012037800000000001 2024-09-18T11:30:21.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:21.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.050868+0000", 2024-09-18T11:30:21.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030111420000000001 2024-09-18T11:30:21.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:21.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051239+0000", 2024-09-18T11:30:21.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037141000000000002 2024-09-18T11:30:21.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051288+0000", 2024-09-18T11:30:21.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9051000000000002e-05 2024-09-18T11:30:21.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051323+0000", 2024-09-18T11:30:21.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4545000000000002e-05 2024-09-18T11:30:21.426 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.426 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.426 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.427 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2 4.7 e28/22 4:e73ba68f:::smithi11939506-1:head v 28'2, mlcod=0'0)", 2024-09-18T11:30:21.427 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:08.105343+0000", 2024-09-18T11:30:21.427 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.296366795000001, 2024-09-18T11:30:21.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040860050000000002, 2024-09-18T11:30:21.427 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.427 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.427 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105343+0000", 2024-09-18T11:30:21.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105343+0000", 2024-09-18T11:30:21.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105342+0000", 2024-09-18T11:30:21.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:21.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105801+0000", 2024-09-18T11:30:21.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00045840099999999999 2024-09-18T11:30:21.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105813+0000", 2024-09-18T11:30:21.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2461e-05 2024-09-18T11:30:21.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105818+0000", 2024-09-18T11:30:21.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.989e-06 2024-09-18T11:30:21.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105856+0000", 2024-09-18T11:30:21.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7839999999999997e-05 2024-09-18T11:30:21.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105872+0000", 2024-09-18T11:30:21.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6582000000000001e-05 2024-09-18T11:30:21.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.109389+0000", 2024-09-18T11:30:21.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003516211 2024-09-18T11:30:21.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.109429+0000", 2024-09-18T11:30:21.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0261e-05 2024-09-18T11:30:21.431 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.431 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.431 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.432 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3 4.7 e28/22 4:e73ba68f:::smithi11939506-1:head v 28'3, mlcod=0'0)", 2024-09-18T11:30:21.432 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:08.106522+0000", 2024-09-18T11:30:21.432 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.295187779000001, 2024-09-18T11:30:21.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031398509999999999, 2024-09-18T11:30:21.432 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.432 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.432 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106522+0000", 2024-09-18T11:30:21.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106522+0000", 2024-09-18T11:30:21.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106521+0000", 2024-09-18T11:30:21.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:21.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106526+0000", 2024-09-18T11:30:21.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9799999999999998e-06 2024-09-18T11:30:21.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106533+0000", 2024-09-18T11:30:21.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1229999999999997e-06 2024-09-18T11:30:21.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106536+0000", 2024-09-18T11:30:21.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.055e-06 2024-09-18T11:30:21.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106577+0000", 2024-09-18T11:30:21.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0932e-05 2024-09-18T11:30:21.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106588+0000", 2024-09-18T11:30:21.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1486e-05 2024-09-18T11:30:21.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.109630+0000", 2024-09-18T11:30:21.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030422159999999999 2024-09-18T11:30:21.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.109662+0000", 2024-09-18T11:30:21.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1498999999999998e-05 2024-09-18T11:30:21.436 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.436 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.436 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.437 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:905 4.7 e31/22 4:e73ba68f:::smithi11939506-1:head v 31'32, mlcod=30'29)", 2024-09-18T11:30:21.438 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:30:21.438 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 10.260241359, 2024-09-18T11:30:21.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048351990000000001, 2024-09-18T11:30:21.438 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.438 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.438 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:30:21.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:30:21.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:30:21.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:21.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144912+0000", 2024-09-18T11:30:21.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003444 2024-09-18T11:30:21.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144921+0000", 2024-09-18T11:30:21.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1900000000000001e-06 2024-09-18T11:30:21.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144927+0000", 2024-09-18T11:30:21.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4940000000000003e-06 2024-09-18T11:30:21.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144952+0000", 2024-09-18T11:30:21.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5845000000000001e-05 2024-09-18T11:30:21.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144966+0000", 2024-09-18T11:30:21.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.396e-05 2024-09-18T11:30:21.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.146274+0000", 2024-09-18T11:30:21.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013073550000000001 2024-09-18T11:30:21.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.146303+0000", 2024-09-18T11:30:21.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9754e-05 2024-09-18T11:30:21.442 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.442 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.442 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.442 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:910 4.3 e31/22 4:cc46532a:::smithi11939506-33:head v 31'23, mlcod=30'20)", 2024-09-18T11:30:21.442 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.160095+0000", 2024-09-18T11:30:21.443 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 10.241614307000001, 2024-09-18T11:30:21.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002481731, 2024-09-18T11:30:21.443 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.443 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.443 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.160095+0000", 2024-09-18T11:30:21.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.160095+0000", 2024-09-18T11:30:21.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.160095+0000", 2024-09-18T11:30:21.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:21.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.160806+0000", 2024-09-18T11:30:21.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00071076799999999997 2024-09-18T11:30:21.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.160816+0000", 2024-09-18T11:30:21.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0295999999999999e-05 2024-09-18T11:30:21.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.160820+0000", 2024-09-18T11:30:21.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0849999999999999e-06 2024-09-18T11:30:21.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.160858+0000", 2024-09-18T11:30:21.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7261999999999997e-05 2024-09-18T11:30:21.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.160871+0000", 2024-09-18T11:30:21.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3665999999999999e-05 2024-09-18T11:30:21.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.162552+0000", 2024-09-18T11:30:21.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016811020000000001 2024-09-18T11:30:21.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.162577+0000", 2024-09-18T11:30:21.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4711000000000001e-05 2024-09-18T11:30:21.447 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.447 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.447 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.447 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:911 4.3 e31/22 4:cc46532a:::smithi11939506-33:head v 31'24, mlcod=30'21)", 2024-09-18T11:30:21.447 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.166801+0000", 2024-09-18T11:30:21.447 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 10.234909032999999, 2024-09-18T11:30:21.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048144349999999997, 2024-09-18T11:30:21.448 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.448 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.448 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.166801+0000", 2024-09-18T11:30:21.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.166801+0000", 2024-09-18T11:30:21.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.166799+0000", 2024-09-18T11:30:21.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:21.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167413+0000", 2024-09-18T11:30:21.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000613259 2024-09-18T11:30:21.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167431+0000", 2024-09-18T11:30:21.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7906000000000001e-05 2024-09-18T11:30:21.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167438+0000", 2024-09-18T11:30:21.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.883e-06 2024-09-18T11:30:21.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167468+0000", 2024-09-18T11:30:21.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0168e-05 2024-09-18T11:30:21.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167483+0000", 2024-09-18T11:30:21.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4997999999999999e-05 2024-09-18T11:30:21.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.171574+0000", 2024-09-18T11:30:21.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040909589999999999 2024-09-18T11:30:21.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.171615+0000", 2024-09-18T11:30:21.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1371000000000001e-05 2024-09-18T11:30:21.452 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.452 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.452 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.452 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:989 4.3 e35/22 4:ca2b2309:::smithi11939506-44:head v 35'30, mlcod=31'26)", 2024-09-18T11:30:21.452 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.154272+0000", 2024-09-18T11:30:21.452 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 6.2474378799999997, 2024-09-18T11:30:21.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031808090000000002, 2024-09-18T11:30:21.453 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.453 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.453 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.154272+0000", 2024-09-18T11:30:21.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.154272+0000", 2024-09-18T11:30:21.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.154272+0000", 2024-09-18T11:30:21.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:21.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.155085+0000", 2024-09-18T11:30:21.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000813921 2024-09-18T11:30:21.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.155097+0000", 2024-09-18T11:30:21.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1881e-05 2024-09-18T11:30:21.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.155103+0000", 2024-09-18T11:30:21.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6250000000000004e-06 2024-09-18T11:30:21.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.155161+0000", 2024-09-18T11:30:21.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8514999999999998e-05 2024-09-18T11:30:21.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.155185+0000", 2024-09-18T11:30:21.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3841000000000001e-05 2024-09-18T11:30:21.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.157411+0000", 2024-09-18T11:30:21.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022258719999999998 2024-09-18T11:30:21.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.157453+0000", 2024-09-18T11:30:21.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1387999999999997e-05 2024-09-18T11:30:21.457 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.457 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.457 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.457 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:990 4.3 e35/22 4:ca2b2309:::smithi11939506-44:head v 35'31, mlcod=34'28)", 2024-09-18T11:30:21.458 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.158794+0000", 2024-09-18T11:30:21.458 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 6.2429159849999998, 2024-09-18T11:30:21.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042163180000000001, 2024-09-18T11:30:21.458 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.458 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.458 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.158794+0000", 2024-09-18T11:30:21.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.158794+0000", 2024-09-18T11:30:21.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.158793+0000", 2024-09-18T11:30:21.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:21.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159307+0000", 2024-09-18T11:30:21.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00051453999999999996 2024-09-18T11:30:21.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159320+0000", 2024-09-18T11:30:21.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2639e-05 2024-09-18T11:30:21.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159327+0000", 2024-09-18T11:30:21.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3200000000000002e-06 2024-09-18T11:30:21.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159359+0000", 2024-09-18T11:30:21.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2147999999999998e-05 2024-09-18T11:30:21.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159380+0000", 2024-09-18T11:30:21.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0625999999999999e-05 2024-09-18T11:30:21.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.162980+0000", 2024-09-18T11:30:21.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003600125 2024-09-18T11:30:21.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.163010+0000", 2024-09-18T11:30:21.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0020999999999999e-05 2024-09-18T11:30:21.462 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.462 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.462 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.463 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1007 4.2 e35/22 4:425afe9c:::smithi11939506-29:head v 35'12, mlcod=30'9)", 2024-09-18T11:30:21.463 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.250686+0000", 2024-09-18T11:30:21.463 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 6.1510234950000005, 2024-09-18T11:30:21.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002857474, 2024-09-18T11:30:21.463 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.463 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.463 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.250686+0000", 2024-09-18T11:30:21.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.250686+0000", 2024-09-18T11:30:21.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.250684+0000", 2024-09-18T11:30:21.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:21.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.251174+0000", 2024-09-18T11:30:21.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048958300000000003 2024-09-18T11:30:21.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.251192+0000", 2024-09-18T11:30:21.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8227999999999999e-05 2024-09-18T11:30:21.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.251199+0000", 2024-09-18T11:30:21.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7830000000000001e-06 2024-09-18T11:30:21.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.251243+0000", 2024-09-18T11:30:21.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4302999999999999e-05 2024-09-18T11:30:21.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.251261+0000", 2024-09-18T11:30:21.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7782e-05 2024-09-18T11:30:21.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.253514+0000", 2024-09-18T11:30:21.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022532479999999998 2024-09-18T11:30:21.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.253544+0000", 2024-09-18T11:30:21.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9631000000000001e-05 2024-09-18T11:30:21.467 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.467 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.467 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.467 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1118 3.2 e38/21 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head v 38'76, mlcod=38'74)", 2024-09-18T11:30:21.467 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:30:21.467 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.4537658490000001, 2024-09-18T11:30:21.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006048688, 2024-09-18T11:30:21.467 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.467 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.467 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:30:21.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:30:21.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947941+0000", 2024-09-18T11:30:21.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:21.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949868+0000", 2024-09-18T11:30:21.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019267659999999999 2024-09-18T11:30:21.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949893+0000", 2024-09-18T11:30:21.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5239999999999999e-05 2024-09-18T11:30:21.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949901+0000", 2024-09-18T11:30:21.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8420000000000005e-06 2024-09-18T11:30:21.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949954+0000", 2024-09-18T11:30:21.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3464999999999997e-05 2024-09-18T11:30:21.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949974+0000", 2024-09-18T11:30:21.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9731000000000001e-05 2024-09-18T11:30:21.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953948+0000", 2024-09-18T11:30:21.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039735489999999998 2024-09-18T11:30:21.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953992+0000", 2024-09-18T11:30:21.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4700999999999998e-05 2024-09-18T11:30:21.471 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.471 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.471 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.472 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1139 4.2 e39/22 4:425afe9c:::smithi11939506-29:head v 39'19, mlcod=35'15)", 2024-09-18T11:30:21.472 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.187224+0000", 2024-09-18T11:30:21.472 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.2144855720000001, 2024-09-18T11:30:21.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036968130000000002, 2024-09-18T11:30:21.472 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.472 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.472 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.187224+0000", 2024-09-18T11:30:21.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.187224+0000", 2024-09-18T11:30:21.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.187224+0000", 2024-09-18T11:30:21.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:21.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.188633+0000", 2024-09-18T11:30:21.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001409158 2024-09-18T11:30:21.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.188646+0000", 2024-09-18T11:30:21.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3458999999999999e-05 2024-09-18T11:30:21.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.188653+0000", 2024-09-18T11:30:21.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8569999999999997e-06 2024-09-18T11:30:21.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.188700+0000", 2024-09-18T11:30:21.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7176999999999999e-05 2024-09-18T11:30:21.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.188720+0000", 2024-09-18T11:30:21.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0043000000000001e-05 2024-09-18T11:30:21.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.190874+0000", 2024-09-18T11:30:21.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021537739999999998 2024-09-18T11:30:21.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.190921+0000", 2024-09-18T11:30:21.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6864999999999999e-05 2024-09-18T11:30:21.476 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.476 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.476 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.476 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1165 4.7 e39/22 4:e73ba68f:::smithi11939506-1:head v 39'40, mlcod=34'37)", 2024-09-18T11:30:21.476 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.657838+0000", 2024-09-18T11:30:21.476 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.74387118699999999, 2024-09-18T11:30:21.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042136450000000002, 2024-09-18T11:30:21.477 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.477 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.477 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.657838+0000", 2024-09-18T11:30:21.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.657838+0000", 2024-09-18T11:30:21.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.657837+0000", 2024-09-18T11:30:21.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:21.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659431+0000", 2024-09-18T11:30:21.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015942479999999999 2024-09-18T11:30:21.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659450+0000", 2024-09-18T11:30:21.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8909000000000001e-05 2024-09-18T11:30:21.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659457+0000", 2024-09-18T11:30:21.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8390000000000001e-06 2024-09-18T11:30:21.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659500+0000", 2024-09-18T11:30:21.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3101e-05 2024-09-18T11:30:21.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659518+0000", 2024-09-18T11:30:21.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8020999999999999e-05 2024-09-18T11:30:21.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.662024+0000", 2024-09-18T11:30:21.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025059399999999999 2024-09-18T11:30:21.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.662052+0000", 2024-09-18T11:30:21.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8206999999999999e-05 2024-09-18T11:30:21.481 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.482 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.482 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.482 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.13:29 3.2 e39/21 3:5fa8fcfa:::ec59520759824f0488c7a3ca135dacb324de2330e16b7f54053de5719c4e96b6:head v 39'81, mlcod=38'76)", 2024-09-18T11:30:21.482 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:30:21.482 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.72006537299999995, 2024-09-18T11:30:21.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004755858, 2024-09-18T11:30:21.482 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.482 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.482 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:30:21.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:30:21.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:30:21.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:21.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681961+0000", 2024-09-18T11:30:21.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00031696799999999999 2024-09-18T11:30:21.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681969+0000", 2024-09-18T11:30:21.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4800000000000001e-06 2024-09-18T11:30:21.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681973+0000", 2024-09-18T11:30:21.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4819999999999999e-06 2024-09-18T11:30:21.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682007+0000", 2024-09-18T11:30:21.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3961999999999998e-05 2024-09-18T11:30:21.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682033+0000", 2024-09-18T11:30:21.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6103999999999999e-05 2024-09-18T11:30:21.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686375+0000", 2024-09-18T11:30:21.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043422019999999999 2024-09-18T11:30:21.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686400+0000", 2024-09-18T11:30:21.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5265e-05 2024-09-18T11:30:21.487 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.488 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.488 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.488 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.13:30 3.2 e39/21 3:49074e56:::7fc21ffee1ba17f2a9197c61ff60ef0b03804c121c8aeb83ea424b7ac77081e9:head v 39'82, mlcod=39'77)", 2024-09-18T11:30:21.488 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.682532+0000", 2024-09-18T11:30:21.488 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.71917731100000004, 2024-09-18T11:30:21.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004072694, 2024-09-18T11:30:21.488 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.488 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.488 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682532+0000", 2024-09-18T11:30:21.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682532+0000", 2024-09-18T11:30:21.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682532+0000", 2024-09-18T11:30:21.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:21.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682681+0000", 2024-09-18T11:30:21.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014942500000000001 2024-09-18T11:30:21.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682691+0000", 2024-09-18T11:30:21.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0125e-05 2024-09-18T11:30:21.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682695+0000", 2024-09-18T11:30:21.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9339999999999999e-06 2024-09-18T11:30:21.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682724+0000", 2024-09-18T11:30:21.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8918e-05 2024-09-18T11:30:21.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682738+0000", 2024-09-18T11:30:21.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3793e-05 2024-09-18T11:30:21.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686590+0000", 2024-09-18T11:30:21.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038524380000000001 2024-09-18T11:30:21.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686605+0000", 2024-09-18T11:30:21.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4827999999999999e-05 2024-09-18T11:30:21.492 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.493 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.493 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.493 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1174 4.7 e39/22 4:e0d95e10:::smithi11939506-22:head v 39'45, mlcod=39'42)", 2024-09-18T11:30:21.493 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.035250+0000", 2024-09-18T11:30:21.493 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.36645981999999999, 2024-09-18T11:30:21.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028686929999999999, 2024-09-18T11:30:21.493 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.493 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.493 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035250+0000", 2024-09-18T11:30:21.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035250+0000", 2024-09-18T11:30:21.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035249+0000", 2024-09-18T11:30:21.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:21.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035852+0000", 2024-09-18T11:30:21.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060368100000000001 2024-09-18T11:30:21.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035866+0000", 2024-09-18T11:30:21.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3448999999999999e-05 2024-09-18T11:30:21.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035873+0000", 2024-09-18T11:30:21.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5739999999999998e-06 2024-09-18T11:30:21.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035918+0000", 2024-09-18T11:30:21.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4648999999999999e-05 2024-09-18T11:30:21.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035940+0000", 2024-09-18T11:30:21.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2166000000000001e-05 2024-09-18T11:30:21.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.038085+0000", 2024-09-18T11:30:21.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021446210000000002 2024-09-18T11:30:21.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.038119+0000", 2024-09-18T11:30:21.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3800000000000002e-05 2024-09-18T11:30:21.498 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.498 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.498 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.499 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1179 4.1 e39/22 4:86d80ae6:::smithi11939506-39:head v 39'43, mlcod=37'39)", 2024-09-18T11:30:21.499 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.045567+0000", 2024-09-18T11:30:21.499 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.35614215500000002, 2024-09-18T11:30:21.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025000840000000001, 2024-09-18T11:30:21.499 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.500 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.500 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.045567+0000", 2024-09-18T11:30:21.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.045567+0000", 2024-09-18T11:30:21.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.045567+0000", 2024-09-18T11:30:21.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:21.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.045574+0000", 2024-09-18T11:30:21.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6380000000000004e-06 2024-09-18T11:30:21.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.045603+0000", 2024-09-18T11:30:21.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9056000000000001e-05 2024-09-18T11:30:21.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.504 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.045627+0000", 2024-09-18T11:30:21.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3181e-05 2024-09-18T11:30:21.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.045669+0000", 2024-09-18T11:30:21.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2378e-05 2024-09-18T11:30:21.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.045694+0000", 2024-09-18T11:30:21.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4873000000000001e-05 2024-09-18T11:30:21.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.048044+0000", 2024-09-18T11:30:21.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023499879999999999 2024-09-18T11:30:21.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.048068+0000", 2024-09-18T11:30:21.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3764000000000002e-05 2024-09-18T11:30:21.506 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.506 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.506 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.507 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1190 4.2 e39/22 4:425afe9c:::smithi11939506-29:head v 39'23, mlcod=39'21)", 2024-09-18T11:30:21.507 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.177245+0000", 2024-09-18T11:30:21.507 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.224464257, 2024-09-18T11:30:21.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027170720000000001, 2024-09-18T11:30:21.507 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.507 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.507 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.177245+0000", 2024-09-18T11:30:21.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.177245+0000", 2024-09-18T11:30:21.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.177244+0000", 2024-09-18T11:30:21.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:21.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.177883+0000", 2024-09-18T11:30:21.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00063948499999999997 2024-09-18T11:30:21.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.177901+0000", 2024-09-18T11:30:21.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7442999999999999e-05 2024-09-18T11:30:21.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.177907+0000", 2024-09-18T11:30:21.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.336e-06 2024-09-18T11:30:21.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.177952+0000", 2024-09-18T11:30:21.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4672000000000002e-05 2024-09-18T11:30:21.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.177969+0000", 2024-09-18T11:30:21.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6968000000000001e-05 2024-09-18T11:30:21.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.179922+0000", 2024-09-18T11:30:21.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019529630000000001 2024-09-18T11:30:21.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.179962+0000", 2024-09-18T11:30:21.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0921999999999999e-05 2024-09-18T11:30:21.511 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.511 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.511 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.511 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1193 4.2 e39/22 4:425afe9c:::smithi11939506-29:head v 39'26, mlcod=39'23)", 2024-09-18T11:30:21.511 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.189951+0000", 2024-09-18T11:30:21.511 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.21175909400000001, 2024-09-18T11:30:21.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025525370000000001, 2024-09-18T11:30:21.511 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:21.512 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:21.512 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:21.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:21.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.189951+0000", 2024-09-18T11:30:21.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:21.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.189951+0000", 2024-09-18T11:30:21.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:21.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:21.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.189950+0000", 2024-09-18T11:30:21.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:21.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:21.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.189958+0000", 2024-09-18T11:30:21.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1790000000000008e-06 2024-09-18T11:30:21.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:21.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.189966+0000", 2024-09-18T11:30:21.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7109999999999995e-06 2024-09-18T11:30:21.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:21.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.189970+0000", 2024-09-18T11:30:21.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.048e-06 2024-09-18T11:30:21.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:21.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.190007+0000", 2024-09-18T11:30:21.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6955000000000001e-05 2024-09-18T11:30:21.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:21.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.190023+0000", 2024-09-18T11:30:21.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6566e-05 2024-09-18T11:30:21.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:21.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.192473+0000", 2024-09-18T11:30:21.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002449447 2024-09-18T11:30:21.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:21.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:21.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:21.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.192503+0000", 2024-09-18T11:30:21.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0549000000000002e-05 2024-09-18T11:30:21.516 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.516 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.516 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.516 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:21.516 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:21.516 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:21.517 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:30:21.518 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:30:21.518 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:21.515+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:21.519 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:30:21.520 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:21.520 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:21.534 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:21.534 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:21.548 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:21.549 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:21.564 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:21.564 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:21.579 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:21.579 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:21.594 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:21.594 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:21.609 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:21.610 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:21.619 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:21.618+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:21.658 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:21.658 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:21.666 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:30:21.750 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:21.749+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:21.808 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:21.808 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:21.815 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:30:21.857 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:21.855+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:21.922 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:21.920+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:21.954 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:21.954 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:21.962 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:30:22.023 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:22.022+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:22.061 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:22.061 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:22.070 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:30:22.124 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:22.123+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:22.167 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:22.167 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:22.176 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:30:22.181 INFO:tasks.rados.rados.0.smithi119.stdout:754: done (3 left) 2024-09-18T11:30:22.182 INFO:tasks.rados.rados.0.smithi119.stdout:756: done (2 left) 2024-09-18T11:30:22.182 INFO:tasks.rados.rados.0.smithi119.stdout:757: done (1 left) 2024-09-18T11:30:22.182 INFO:tasks.rados.rados.0.smithi119.stdout:758: done (0 left) 2024-09-18T11:30:22.182 INFO:tasks.rados.rados.0.smithi119.stdout:759: write (excl) oid 5 current snap is 6 2024-09-18T11:30:22.182 INFO:tasks.rados.rados.0.smithi119.stdout:759: seq_num 96 ranges {413472=620140,1818036=510700,3120019=493453} 2024-09-18T11:30:22.187 INFO:tasks.rados.rados.0.smithi119.stdout:759: writing smithi11939506-5 from 413472 to 1033612 tid 1 2024-09-18T11:30:22.194 INFO:tasks.rados.rados.0.smithi119.stdout:759: writing smithi11939506-5 from 1818036 to 2328736 tid 2 2024-09-18T11:30:22.202 INFO:tasks.rados.rados.0.smithi119.stdout:759: writing smithi11939506-5 from 3120019 to 3613472 tid 3 2024-09-18T11:30:22.202 INFO:tasks.rados.rados.0.smithi119.stdout:760: set_chunk oid 24 target oid 2024-09-18T11:30:22.207 INFO:tasks.rados.rados.0.smithi119.stdout:760: set_chunk oid 24 offset: 152576 length: 726528 target oid 16c9af8752ce95120a507ec000947859dae1395d7d6d118e597b9892dcd8465e offset: 0 snap 4 2024-09-18T11:30:22.207 INFO:tasks.rados.rados.0.smithi119.stdout:761: set_chunk oid 32 target oid 2024-09-18T11:30:22.219 INFO:tasks.rados.rados.0.smithi119.stdout:761: set_chunk oid 32 offset: 735232 length: 1800704 target oid 45274de591d4198ebdf7025d46b87dfab949a0114f7ca760f95c9bdc5f783864 offset: 0 snap -1 2024-09-18T11:30:22.219 INFO:tasks.rados.rados.0.smithi119.stdout:762: read oid 13 snap 5 2024-09-18T11:30:22.219 INFO:tasks.rados.rados.0.smithi119.stdout:762: expect (ObjNum 74 snap 3 seq_num 74) 2024-09-18T11:30:22.219 INFO:tasks.rados.rados.0.smithi119.stdout:763: read oid 2 snap 5 2024-09-18T11:30:22.220 INFO:tasks.rados.rados.0.smithi119.stdout:763: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-18T11:30:22.220 INFO:tasks.rados.rados.0.smithi119.stdout:764: set_chunk oid 45 target oid 2024-09-18T11:30:22.224 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:22.222+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:22.227 INFO:tasks.rados.rados.0.smithi119.stdout:764: set_chunk oid 45 offset: 1669632 length: 547840 target oid 6d8940f597e647b732fd14e194ed26b851b96b3752dc13ab5e4f039b180f80e9 offset: 0 snap 5 2024-09-18T11:30:22.227 INFO:tasks.rados.rados.0.smithi119.stdout:765: write oid 34 current snap is 6 2024-09-18T11:30:22.227 INFO:tasks.rados.rados.0.smithi119.stdout:759: finishing write tid 1 to smithi11939506-5 2024-09-18T11:30:22.227 INFO:tasks.rados.rados.0.smithi119.stdout:765: seq_num 97 ranges {430279=695389,1876518=654358,3031089=536797} 2024-09-18T11:30:22.233 INFO:tasks.rados.rados.0.smithi119.stdout:765: writing smithi11939506-34 from 430279 to 1125668 tid 1 2024-09-18T11:30:22.240 INFO:tasks.rados.rados.0.smithi119.stdout:765: writing smithi11939506-34 from 1876518 to 2530876 tid 2 2024-09-18T11:30:22.246 INFO:tasks.rados.rados.0.smithi119.stdout:765: writing smithi11939506-34 from 3031089 to 3567886 tid 3 2024-09-18T11:30:22.246 INFO:tasks.rados.rados.0.smithi119.stdout:766: set_chunk oid 46 target oid 2024-09-18T11:30:22.254 INFO:tasks.rados.rados.0.smithi119.stdout:766: set_chunk oid 46 offset: 1496576 length: 1068544 target oid cce7b6371944790d4d8fee8f2e74ccabc133c8293d2fc4fdd36de0faec0dece8 offset: 0 snap 5 2024-09-18T11:30:22.254 INFO:tasks.rados.rados.0.smithi119.stdout:767: set_chunk oid 18 target oid 2024-09-18T11:30:22.264 INFO:tasks.rados.rados.0.smithi119.stdout:767: set_chunk oid 18 offset: 376320 length: 1503744 target oid bf4193846831f55aaf6aeab728a0bc8518d007e88305903ccd0b90db37e1e172 offset: 0 snap 5 2024-09-18T11:30:22.264 INFO:tasks.rados.rados.0.smithi119.stdout:768: read oid 35 snap -1 2024-09-18T11:30:22.264 INFO:tasks.rados.rados.0.smithi119.stdout:768: expect (ObjNum 34 snap 0 seq_num 34) 2024-09-18T11:30:22.264 INFO:tasks.rados.rados.0.smithi119.stdout:769: write (excl) oid 26 current snap is 6 2024-09-18T11:30:22.264 INFO:tasks.rados.rados.0.smithi119.stdout:759: finishing write tid 2 to smithi11939506-5 2024-09-18T11:30:22.264 INFO:tasks.rados.rados.0.smithi119.stdout:759: finishing write tid 3 to smithi11939506-5 2024-09-18T11:30:22.265 INFO:tasks.rados.rados.0.smithi119.stdout:759: finishing write tid 4 to smithi11939506-5 2024-09-18T11:30:22.265 INFO:tasks.rados.rados.0.smithi119.stdout:759: finishing write tid 5 to smithi11939506-5 2024-09-18T11:30:22.265 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 38 (ObjNum 96 snap 6 seq_num 96) dirty exists 2024-09-18T11:30:22.265 INFO:tasks.rados.rados.0.smithi119.stdout:759: left oid 5 (ObjNum 96 snap 6 seq_num 96) 2024-09-18T11:30:22.265 INFO:tasks.rados.rados.0.smithi119.stdout:760: finishing set_chunk to oid 24 2024-09-18T11:30:22.265 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 24 set_chunk returned error code -95 offset: 152576 length: 726528 2024-09-18T11:30:22.265 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 0 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-18T11:30:22.265 INFO:tasks.rados.rados.0.smithi119.stdout:761: finishing set_chunk to oid 32 2024-09-18T11:30:22.265 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 32 set_chunk returned error code -95 offset: 735232 length: 1800704 2024-09-18T11:30:22.265 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 0 (ObjNum 64 snap 0 seq_num 64) dirty exists 2024-09-18T11:30:22.265 INFO:tasks.rados.rados.0.smithi119.stdout:762: expect (ObjNum 74 snap 3 seq_num 74) 2024-09-18T11:30:22.319 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:22.320 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:22.325 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:22.323+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:22.328 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:30:22.426 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:22.425+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:22.427 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:22.427 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:22.436 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:30:22.439 INFO:tasks.rados.rados.0.smithi119.stdout:763: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-18T11:30:22.506 INFO:tasks.rados.rados.0.smithi119.stdout:764: finishing set_chunk to oid 45 2024-09-18T11:30:22.507 INFO:tasks.rados.rados.0.smithi119.stdout:765: finishing write tid 1 to smithi11939506-34 2024-09-18T11:30:22.507 INFO:tasks.rados.rados.0.smithi119.stdout:765: finishing write tid 2 to smithi11939506-34 2024-09-18T11:30:22.507 INFO:tasks.rados.rados.0.smithi119.stdout:769: seq_num 98 ranges {447086=770638,1935000=798016,3458512=188574} 2024-09-18T11:30:22.513 INFO:tasks.rados.rados.0.smithi119.stdout:769: writing smithi11939506-26 from 447086 to 1217724 tid 1 2024-09-18T11:30:22.521 INFO:tasks.rados.rados.0.smithi119.stdout:769: writing smithi11939506-26 from 1935000 to 2733016 tid 2 2024-09-18T11:30:22.526 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:22.525+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:22.527 INFO:tasks.rados.rados.0.smithi119.stdout:769: writing smithi11939506-26 from 3458512 to 3647086 tid 3 2024-09-18T11:30:22.527 INFO:tasks.rados.rados.0.smithi119.stdout:759: done (10 left) 2024-09-18T11:30:22.528 INFO:tasks.rados.rados.0.smithi119.stdout:760: done (9 left) 2024-09-18T11:30:22.528 INFO:tasks.rados.rados.0.smithi119.stdout:761: done (8 left) 2024-09-18T11:30:22.528 INFO:tasks.rados.rados.0.smithi119.stdout:762: done (7 left) 2024-09-18T11:30:22.528 INFO:tasks.rados.rados.0.smithi119.stdout:763: done (6 left) 2024-09-18T11:30:22.528 INFO:tasks.rados.rados.0.smithi119.stdout:764: done (5 left) 2024-09-18T11:30:22.528 INFO:tasks.rados.rados.0.smithi119.stdout:770: read oid 11 snap -1 2024-09-18T11:30:22.528 INFO:tasks.rados.rados.0.smithi119.stdout:770: expect (ObjNum 81 snap 4 seq_num 81) 2024-09-18T11:30:22.528 INFO:tasks.rados.rados.0.smithi119.stdout:771: read oid 47 snap 4 2024-09-18T11:30:22.528 INFO:tasks.rados.rados.0.smithi119.stdout:771: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T11:30:22.528 INFO:tasks.rados.rados.0.smithi119.stdout:772: write (excl) oid 42 current snap is 6 2024-09-18T11:30:22.528 INFO:tasks.rados.rados.0.smithi119.stdout:765: finishing write tid 3 to smithi11939506-34 2024-09-18T11:30:22.528 INFO:tasks.rados.rados.0.smithi119.stdout:765: finishing write tid 4 to smithi11939506-34 2024-09-18T11:30:22.528 INFO:tasks.rados.rados.0.smithi119.stdout:765: finishing write tid 5 to smithi11939506-34 2024-09-18T11:30:22.529 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 40 (ObjNum 97 snap 6 seq_num 97) dirty exists 2024-09-18T11:30:22.529 INFO:tasks.rados.rados.0.smithi119.stdout:765: left oid 34 (ObjNum 97 snap 6 seq_num 97) 2024-09-18T11:30:22.529 INFO:tasks.rados.rados.0.smithi119.stdout:766: finishing set_chunk to oid 46 2024-09-18T11:30:22.529 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 46 set_chunk returned error code -95 offset: 1496576 length: 1068544 2024-09-18T11:30:22.529 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 0 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-18T11:30:22.529 INFO:tasks.rados.rados.0.smithi119.stdout:767: finishing set_chunk to oid 18 2024-09-18T11:30:22.529 INFO:tasks.rados.rados.0.smithi119.stdout:768: expect (ObjNum 34 snap 0 seq_num 34) 2024-09-18T11:30:22.542 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:22.542 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:22.553 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:30:22.621 INFO:tasks.rados.rados.0.smithi119.stdout:769: finishing write tid 1 to smithi11939506-26 2024-09-18T11:30:22.622 INFO:tasks.rados.rados.0.smithi119.stdout:769: finishing write tid 2 to smithi11939506-26 2024-09-18T11:30:22.622 INFO:tasks.rados.rados.0.smithi119.stdout:771: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T11:30:22.628 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:22.625+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:22.693 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:22.693 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:22.702 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:30:22.714 INFO:tasks.rados.rados.0.smithi119.stdout:769: finishing write tid 3 to smithi11939506-26 2024-09-18T11:30:22.714 INFO:tasks.rados.rados.0.smithi119.stdout:769: finishing write tid 4 to smithi11939506-26 2024-09-18T11:30:22.714 INFO:tasks.rados.rados.0.smithi119.stdout:769: finishing write tid 5 to smithi11939506-26 2024-09-18T11:30:22.714 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 53 (ObjNum 98 snap 6 seq_num 98) dirty exists 2024-09-18T11:30:22.714 INFO:tasks.rados.rados.0.smithi119.stdout:769: left oid 26 (ObjNum 98 snap 6 seq_num 98) 2024-09-18T11:30:22.715 INFO:tasks.rados.rados.0.smithi119.stdout:770: expect (ObjNum 81 snap 4 seq_num 81) 2024-09-18T11:30:22.758 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:22.756+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:22.813 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:22.813 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:22.821 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:30:22.862 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:22.860+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:22.907 INFO:tasks.rados.rados.0.smithi119.stdout:772: seq_num 99 ranges {463893=562240,1593482=658027,2685935=559341} 2024-09-18T11:30:22.912 INFO:tasks.rados.rados.0.smithi119.stdout:772: writing smithi11939506-42 from 463893 to 1026133 tid 1 2024-09-18T11:30:22.919 INFO:tasks.rados.rados.0.smithi119.stdout:772: writing smithi11939506-42 from 1593482 to 2251509 tid 2 2024-09-18T11:30:22.925 INFO:tasks.rados.rados.0.smithi119.stdout:772: writing smithi11939506-42 from 2685935 to 3245276 tid 3 2024-09-18T11:30:22.925 INFO:tasks.rados.rados.0.smithi119.stdout:765: done (7 left) 2024-09-18T11:30:22.925 INFO:tasks.rados.rados.0.smithi119.stdout:766: done (6 left) 2024-09-18T11:30:22.925 INFO:tasks.rados.rados.0.smithi119.stdout:767: done (5 left) 2024-09-18T11:30:22.925 INFO:tasks.rados.rados.0.smithi119.stdout:768: done (4 left) 2024-09-18T11:30:22.925 INFO:tasks.rados.rados.0.smithi119.stdout:769: done (3 left) 2024-09-18T11:30:22.925 INFO:tasks.rados.rados.0.smithi119.stdout:770: done (2 left) 2024-09-18T11:30:22.925 INFO:tasks.rados.rados.0.smithi119.stdout:771: done (1 left) 2024-09-18T11:30:22.925 INFO:tasks.rados.rados.0.smithi119.stdout:773: read oid 31 snap -1 2024-09-18T11:30:22.925 INFO:tasks.rados.rados.0.smithi119.stdout:773: expect (ObjNum 63 snap 0 seq_num 63) 2024-09-18T11:30:22.925 INFO:tasks.rados.rados.0.smithi119.stdout:774: write (excl) oid 36 current snap is 6 2024-09-18T11:30:22.925 INFO:tasks.rados.rados.0.smithi119.stdout:772: finishing write tid 1 to smithi11939506-42 2024-09-18T11:30:22.926 INFO:tasks.rados.rados.0.smithi119.stdout:772: finishing write tid 2 to smithi11939506-42 2024-09-18T11:30:22.926 INFO:tasks.rados.rados.0.smithi119.stdout:774: seq_num 100 ranges {480700=637489,1651964=401685,2713358=770613} 2024-09-18T11:30:22.931 INFO:tasks.rados.rados.0.smithi119.stdout:774: writing smithi11939506-36 from 480700 to 1118189 tid 1 2024-09-18T11:30:22.932 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:22.929+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:22.936 INFO:tasks.rados.rados.0.smithi119.stdout:774: writing smithi11939506-36 from 1651964 to 2053649 tid 2 2024-09-18T11:30:22.944 INFO:tasks.rados.rados.0.smithi119.stdout:774: writing smithi11939506-36 from 2713358 to 3483971 tid 3 2024-09-18T11:30:22.944 INFO:tasks.rados.rados.0.smithi119.stdout:775: read oid 15 snap 4 2024-09-18T11:30:22.944 INFO:tasks.rados.rados.0.smithi119.stdout:775: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-18T11:30:22.944 INFO:tasks.rados.rados.0.smithi119.stdout:776: set_chunk oid 2 target oid 2024-09-18T11:30:22.951 INFO:tasks.rados.rados.0.smithi119.stdout:776: set_chunk oid 2 offset: 672256 length: 466944 target oid 19e83c652319703aa7c65e9006dbd1a097dceda43b4629e71d7708bedfe2cbba offset: 0 snap -1 2024-09-18T11:30:22.951 INFO:tasks.rados.rados.0.smithi119.stdout:777: read oid 33 snap 5 2024-09-18T11:30:22.951 INFO:tasks.rados.rados.0.smithi119.stdout:777: expect deleted 2024-09-18T11:30:22.951 INFO:tasks.rados.rados.0.smithi119.stdout:778: write (excl) oid 22 current snap is 6 2024-09-18T11:30:22.951 INFO:tasks.rados.rados.0.smithi119.stdout:772: finishing write tid 3 to smithi11939506-42 2024-09-18T11:30:22.951 INFO:tasks.rados.rados.0.smithi119.stdout:772: finishing write tid 4 to smithi11939506-42 2024-09-18T11:30:22.951 INFO:tasks.rados.rados.0.smithi119.stdout:772: finishing write tid 5 to smithi11939506-42 2024-09-18T11:30:22.951 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 45 (ObjNum 99 snap 6 seq_num 99) dirty exists 2024-09-18T11:30:22.951 INFO:tasks.rados.rados.0.smithi119.stdout:772: left oid 42 (ObjNum 99 snap 6 seq_num 99) 2024-09-18T11:30:22.951 INFO:tasks.rados.rados.0.smithi119.stdout:773: expect (ObjNum 63 snap 0 seq_num 63) 2024-09-18T11:30:22.965 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:22.965 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:22.974 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:30:22.994 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:22.993+0000 7f3157e90640 -1 osd.4 41 handle_osd_map injecting map crc failure 2024-09-18T11:30:23.038 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:23.037+0000 7fd2785a3640 -1 osd.6 41 handle_osd_map injecting map crc failure 2024-09-18T11:30:23.058 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:23.056+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:23.093 INFO:tasks.rados.rados.0.smithi119.stdout:774: finishing write tid 1 to smithi11939506-36 2024-09-18T11:30:23.093 INFO:tasks.rados.rados.0.smithi119.stdout:774: finishing write tid 2 to smithi11939506-36 2024-09-18T11:30:23.093 INFO:tasks.rados.rados.0.smithi119.stdout:778: seq_num 101 ranges {497507=712738,1710446=661696,3140781=556726} 2024-09-18T11:30:23.099 INFO:tasks.rados.rados.0.smithi119.stdout:778: writing smithi11939506-22 from 497507 to 1210245 tid 1 2024-09-18T11:30:23.105 INFO:tasks.rados.rados.0.smithi119.stdout:778: writing smithi11939506-22 from 1710446 to 2372142 tid 2 2024-09-18T11:30:23.113 INFO:tasks.rados.rados.0.smithi119.stdout:778: writing smithi11939506-22 from 3140781 to 3697507 tid 3 2024-09-18T11:30:23.113 INFO:tasks.rados.rados.0.smithi119.stdout:772: done (6 left) 2024-09-18T11:30:23.113 INFO:tasks.rados.rados.0.smithi119.stdout:773: done (5 left) 2024-09-18T11:30:23.113 INFO:tasks.rados.rados.0.smithi119.stdout:779: delete oid 32 current snap is 6 2024-09-18T11:30:23.113 INFO:tasks.rados.rados.0.smithi119.stdout:775: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-18T11:30:23.114 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:23.114 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:23.124 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:30:23.133 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:23.132+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:23.191 INFO:tasks.rados.rados.0.smithi119.stdout:774: finishing write tid 3 to smithi11939506-36 2024-09-18T11:30:23.192 INFO:tasks.rados.rados.0.smithi119.stdout:774: finishing write tid 4 to smithi11939506-36 2024-09-18T11:30:23.192 INFO:tasks.rados.rados.0.smithi119.stdout:774: finishing write tid 5 to smithi11939506-36 2024-09-18T11:30:23.192 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 36 v 28 (ObjNum 100 snap 6 seq_num 100) dirty exists 2024-09-18T11:30:23.192 INFO:tasks.rados.rados.0.smithi119.stdout:774: left oid 36 (ObjNum 100 snap 6 seq_num 100) 2024-09-18T11:30:23.192 INFO:tasks.rados.rados.0.smithi119.stdout:774: done (5 left) 2024-09-18T11:30:23.192 INFO:tasks.rados.rados.0.smithi119.stdout:775: done (4 left) 2024-09-18T11:30:23.192 INFO:tasks.rados.rados.0.smithi119.stdout:779: done (3 left) 2024-09-18T11:30:23.192 INFO:tasks.rados.rados.0.smithi119.stdout:780: rollback oid 15 current snap is 6 2024-09-18T11:30:23.192 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 15 to 4 2024-09-18T11:30:23.192 INFO:tasks.rados.rados.0.smithi119.stdout:776: finishing set_chunk to oid 2 2024-09-18T11:30:23.192 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 2 set_chunk returned error code -95 offset: 672256 length: 466944 2024-09-18T11:30:23.193 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 0 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-18T11:30:23.193 INFO:tasks.rados.rados.0.smithi119.stdout:778: finishing write tid 1 to smithi11939506-22 2024-09-18T11:30:23.193 INFO:tasks.rados.rados.0.smithi119.stdout:778: finishing write tid 2 to smithi11939506-22 2024-09-18T11:30:23.193 INFO:tasks.rados.rados.0.smithi119.stdout:778: finishing write tid 3 to smithi11939506-22 2024-09-18T11:30:23.193 INFO:tasks.rados.rados.0.smithi119.stdout:778: finishing write tid 4 to smithi11939506-22 2024-09-18T11:30:23.193 INFO:tasks.rados.rados.0.smithi119.stdout:778: finishing write tid 5 to smithi11939506-22 2024-09-18T11:30:23.193 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 53 (ObjNum 101 snap 6 seq_num 101) dirty exists 2024-09-18T11:30:23.193 INFO:tasks.rados.rados.0.smithi119.stdout:778: left oid 22 (ObjNum 101 snap 6 seq_num 101) 2024-09-18T11:30:23.193 INFO:tasks.rados.rados.0.smithi119.stdout:776: done (3 left) 2024-09-18T11:30:23.193 INFO:tasks.rados.rados.0.smithi119.stdout:777: done (2 left) 2024-09-18T11:30:23.193 INFO:tasks.rados.rados.0.smithi119.stdout:778: done (1 left) 2024-09-18T11:30:23.193 INFO:tasks.rados.rados.0.smithi119.stdout:781: tier_promote oid 38 2024-09-18T11:30:23.193 INFO:tasks.rados.rados.0.smithi119.stdout:782: read oid 24 snap -1 2024-09-18T11:30:23.194 INFO:tasks.rados.rados.0.smithi119.stdout:782: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-18T11:30:23.194 INFO:tasks.rados.rados.0.smithi119.stdout:783: read oid 7 snap -1 2024-09-18T11:30:23.194 INFO:tasks.rados.rados.0.smithi119.stdout:783: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-18T11:30:23.194 INFO:tasks.rados.rados.0.smithi119.stdout:784: read oid 10 snap 4 2024-09-18T11:30:23.194 INFO:tasks.rados.rados.0.smithi119.stdout:784: expect deleted 2024-09-18T11:30:23.194 INFO:tasks.rados.rados.0.smithi119.stdout:785: tier_promote oid 42 2024-09-18T11:30:23.194 INFO:tasks.rados.rados.0.smithi119.stdout:786: write (excl) oid 2 current snap is 6 2024-09-18T11:30:23.194 INFO:tasks.rados.rados.0.smithi119.stdout:786: seq_num 103 ranges {531121=463236,1711057=665365,3079274=651847} 2024-09-18T11:30:23.196 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T11:30:23.197 INFO:tasks.rados.rados.0.smithi119.stdout:786: writing smithi11939506-2 from 531121 to 994357 tid 1 2024-09-18T11:30:23.205 INFO:tasks.rados.rados.0.smithi119.stdout:786: writing smithi11939506-2 from 1711057 to 2376422 tid 2 2024-09-18T11:30:23.213 INFO:tasks.rados.rados.0.smithi119.stdout:786: writing smithi11939506-2 from 3079274 to 3731121 tid 3 2024-09-18T11:30:23.213 INFO:tasks.rados.rados.0.smithi119.stdout:787: read oid 29 snap 5 2024-09-18T11:30:23.213 INFO:tasks.rados.rados.0.smithi119.stdout:787: expect (ObjNum 92 snap 5 seq_num 92) 2024-09-18T11:30:23.213 INFO:tasks.rados.rados.0.smithi119.stdout:788: set_chunk oid 37 target oid 2024-09-18T11:30:23.234 INFO:tasks.rados.rados.0.smithi119.stdout:788: set_chunk oid 37 offset: 128512 length: 2651648 target oid a9b7a59152df9279eb382918f27c49082be0ff24b025a10b12970658dd16b35a offset: 0 snap -1 2024-09-18T11:30:23.234 INFO:tasks.rados.rados.0.smithi119.stdout:789: read oid 12 snap 5 2024-09-18T11:30:23.234 INFO:tasks.rados.rados.0.smithi119.stdout:789: expect (ObjNum 70 snap 2 seq_num 70) 2024-09-18T11:30:23.234 INFO:tasks.rados.rados.0.smithi119.stdout:790: read oid 28 snap -1 2024-09-18T11:30:23.234 INFO:tasks.rados.rados.0.smithi119.stdout:790: expect (ObjNum 27 snap 0 seq_num 27) 2024-09-18T11:30:23.234 INFO:tasks.rados.rados.0.smithi119.stdout:791: read oid 4 snap -1 2024-09-18T11:30:23.234 INFO:tasks.rados.rados.0.smithi119.stdout:791: expect (ObjNum 3 snap 0 seq_num 3) 2024-09-18T11:30:23.234 INFO:tasks.rados.rados.0.smithi119.stdout:792: read oid 8 snap 4 2024-09-18T11:30:23.234 INFO:tasks.rados.rados.0.smithi119.stdout:792: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-18T11:30:23.235 INFO:tasks.rados.rados.0.smithi119.stdout:793: tier_evict oid 34 2024-09-18T11:30:23.235 INFO:tasks.rados.rados.0.smithi119.stdout:793: tier_evict oid 34 snap -1 2024-09-18T11:30:23.235 INFO:tasks.rados.rados.0.smithi119.stdout:794: read oid 47 snap -1 2024-09-18T11:30:23.235 INFO:tasks.rados.rados.0.smithi119.stdout:794: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T11:30:23.235 INFO:tasks.rados.rados.0.smithi119.stdout:795: write (excl) oid 41 current snap is 6 2024-09-18T11:30:23.235 INFO:tasks.rados.rados.0.smithi119.stdout:795: seq_num 104 ranges {547928=538485,1769539=409023,2706697=532054} 2024-09-18T11:30:23.236 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:23.235+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:23.240 INFO:tasks.rados.rados.0.smithi119.stdout:795: writing smithi11939506-41 from 547928 to 1086413 tid 1 2024-09-18T11:30:23.248 INFO:tasks.rados.rados.0.smithi119.stdout:795: writing smithi11939506-41 from 1769539 to 2178562 tid 2 2024-09-18T11:30:23.254 INFO:tasks.rados.rados.0.smithi119.stdout:795: writing smithi11939506-41 from 2706697 to 3238751 tid 3 2024-09-18T11:30:23.254 INFO:tasks.rados.rados.0.smithi119.stdout:784: done (15 left) 2024-09-18T11:30:23.254 INFO:tasks.rados.rados.0.smithi119.stdout:796: write (excl) oid 5 current snap is 6 2024-09-18T11:30:23.254 INFO:tasks.rados.rados.0.smithi119.stdout:782: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-18T11:30:23.289 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:23.289 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:23.298 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:30:23.337 INFO:tasks.rados.rados.0.smithi119.stdout:780: finishing rollback tid 0 to smithi11939506-15 2024-09-18T11:30:23.337 INFO:tasks.rados.rados.0.smithi119.stdout:783: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-18T11:30:23.337 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:23.336+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:23.436 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:23.436 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:23.437 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:23.436+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:23.444 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:30:23.459 INFO:tasks.rados.rados.0.smithi119.stdout:785: got (0) Success 2024-09-18T11:30:23.459 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 45 (ObjNum 99 snap 6 seq_num 99) dirty exists 2024-09-18T11:30:23.459 INFO:tasks.rados.rados.0.smithi119.stdout:786: finishing write tid 1 to smithi11939506-2 2024-09-18T11:30:23.459 INFO:tasks.rados.rados.0.smithi119.stdout:781: got (0) Success 2024-09-18T11:30:23.459 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 9 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-18T11:30:23.459 INFO:tasks.rados.rados.0.smithi119.stdout:780: finishing rollback tid 1 to smithi11939506-15 2024-09-18T11:30:23.459 INFO:tasks.rados.rados.0.smithi119.stdout:780: finishing rollback tid 2 to smithi11939506-15 2024-09-18T11:30:23.459 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 21 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-18T11:30:23.460 INFO:tasks.rados.rados.0.smithi119.stdout:786: finishing write tid 2 to smithi11939506-2 2024-09-18T11:30:23.460 INFO:tasks.rados.rados.0.smithi119.stdout:796: seq_num 105 ranges {564735=613734,1828021=669034,3134120=630615} 2024-09-18T11:30:23.465 INFO:tasks.rados.rados.0.smithi119.stdout:796: writing smithi11939506-5 from 564735 to 1178469 tid 1 2024-09-18T11:30:23.472 INFO:tasks.rados.rados.0.smithi119.stdout:796: writing smithi11939506-5 from 1828021 to 2497055 tid 2 2024-09-18T11:30:23.480 INFO:tasks.rados.rados.0.smithi119.stdout:796: writing smithi11939506-5 from 3134120 to 3764735 tid 3 2024-09-18T11:30:23.480 INFO:tasks.rados.rados.0.smithi119.stdout:780: done (15 left) 2024-09-18T11:30:23.480 INFO:tasks.rados.rados.0.smithi119.stdout:781: done (14 left) 2024-09-18T11:30:23.480 INFO:tasks.rados.rados.0.smithi119.stdout:782: done (13 left) 2024-09-18T11:30:23.480 INFO:tasks.rados.rados.0.smithi119.stdout:783: done (12 left) 2024-09-18T11:30:23.480 INFO:tasks.rados.rados.0.smithi119.stdout:785: done (11 left) 2024-09-18T11:30:23.480 INFO:tasks.rados.rados.0.smithi119.stdout:797: read oid 15 snap -1 2024-09-18T11:30:23.481 INFO:tasks.rados.rados.0.smithi119.stdout:797: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-18T11:30:23.481 INFO:tasks.rados.rados.0.smithi119.stdout:798: read oid 13 snap 5 2024-09-18T11:30:23.481 INFO:tasks.rados.rados.0.smithi119.stdout:798: expect (ObjNum 74 snap 3 seq_num 74) 2024-09-18T11:30:23.481 INFO:tasks.rados.rados.0.smithi119.stdout:799: read oid 33 snap -1 2024-09-18T11:30:23.481 INFO:tasks.rados.rados.0.smithi119.stdout:799: expect deleted 2024-09-18T11:30:23.481 INFO:tasks.rados.rados.0.smithi119.stdout:800: write (excl) oid 16 current snap is 6 2024-09-18T11:30:23.481 INFO:tasks.rados.rados.0.smithi119.stdout:786: finishing write tid 3 to smithi11939506-2 2024-09-18T11:30:23.481 INFO:tasks.rados.rados.0.smithi119.stdout:786: finishing write tid 4 to smithi11939506-2 2024-09-18T11:30:23.481 INFO:tasks.rados.rados.0.smithi119.stdout:786: finishing write tid 5 to smithi11939506-2 2024-09-18T11:30:23.481 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 58 (ObjNum 103 snap 6 seq_num 103) dirty exists 2024-09-18T11:30:23.481 INFO:tasks.rados.rados.0.smithi119.stdout:786: left oid 2 (ObjNum 103 snap 6 seq_num 103) 2024-09-18T11:30:23.481 INFO:tasks.rados.rados.0.smithi119.stdout:787: expect (ObjNum 92 snap 5 seq_num 92) 2024-09-18T11:30:23.538 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:23.537+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:23.547 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:23.547 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:23.556 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:30:23.621 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:30:23.639 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:23.638+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:23.698 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:23.698 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:23.706 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:30:23.750 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:23.748+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:23.757 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:30:23.757 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:30:23.773 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:23.773 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:23.775 INFO:tasks.rados.rados.0.smithi119.stdout:788: finishing set_chunk to oid 37 2024-09-18T11:30:23.775 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 37 set_chunk returned error code -95 offset: 128512 length: 2651648 2024-09-18T11:30:23.775 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 0 (ObjNum 91 snap 5 seq_num 91) dirty exists 2024-09-18T11:30:23.775 INFO:tasks.rados.rados.0.smithi119.stdout:793: got (22) Invalid argument 2024-09-18T11:30:23.775 INFO:tasks.rados.rados.0.smithi119.stdout:800: seq_num 106 ranges {581542=688983,1886503=412692,2761543=670951} 2024-09-18T11:30:23.782 INFO:tasks.rados.rados.0.smithi119.stdout:800: writing smithi11939506-16 from 581542 to 1270525 tid 1 2024-09-18T11:30:23.788 INFO:tasks.rados.rados.0.smithi119.stdout:800: writing smithi11939506-16 from 1886503 to 2299195 tid 2 2024-09-18T11:30:23.788 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:23.789 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:23.794 INFO:tasks.rados.rados.0.smithi119.stdout:800: writing smithi11939506-16 from 2761543 to 3432494 tid 3 2024-09-18T11:30:23.795 INFO:tasks.rados.rados.0.smithi119.stdout:786: done (14 left) 2024-09-18T11:30:23.795 INFO:tasks.rados.rados.0.smithi119.stdout:787: done (13 left) 2024-09-18T11:30:23.795 INFO:tasks.rados.rados.0.smithi119.stdout:788: done (12 left) 2024-09-18T11:30:23.795 INFO:tasks.rados.rados.0.smithi119.stdout:793: done (11 left) 2024-09-18T11:30:23.795 INFO:tasks.rados.rados.0.smithi119.stdout:801: read oid 35 snap -1 2024-09-18T11:30:23.795 INFO:tasks.rados.rados.0.smithi119.stdout:801: expect (ObjNum 34 snap 0 seq_num 34) 2024-09-18T11:30:23.795 INFO:tasks.rados.rados.0.smithi119.stdout:802: read oid 49 snap 5 2024-09-18T11:30:23.795 INFO:tasks.rados.rados.0.smithi119.stdout:802: expect deleted 2024-09-18T11:30:23.795 INFO:tasks.rados.rados.0.smithi119.stdout:803: tier_evict oid 24 2024-09-18T11:30:23.795 INFO:tasks.rados.rados.0.smithi119.stdout:803: tier_evict oid 24 snap -1 2024-09-18T11:30:23.795 INFO:tasks.rados.rados.0.smithi119.stdout:804: tier_promote oid 46 2024-09-18T11:30:23.795 INFO:tasks.rados.rados.0.smithi119.stdout:790: expect (ObjNum 27 snap 0 seq_num 27) 2024-09-18T11:30:23.807 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:23.807 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:23.821 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:23.821 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:23.821 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:23.822 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:23.822 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:23.823 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:23.823 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:23.831 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:30:23.840 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:23.840 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:23.840 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:23.839+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:23.856 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:23.857 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:23.873 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:23.873 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:23.887 INFO:tasks.rados.rados.0.smithi119.stdout:792: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-18T11:30:23.941 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:23.940+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:23.942 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:23.942 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:23.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:23.942 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:23.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.942 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:900 4.0 4:0aec5e1c:::smithi11939506-50:head [read 0~2974806,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:23.943 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.133226+0000", 2024-09-18T11:30:23.943 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.806558172000001, 2024-09-18T11:30:23.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059578069999999999, 2024-09-18T11:30:23.943 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:23.943 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:23.943 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:23.943 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:23.943 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:23.943 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 900 2024-09-18T11:30:23.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.943 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:23.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:23.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133226+0000", 2024-09-18T11:30:23.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:23.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133226+0000", 2024-09-18T11:30:23.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:23.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133225+0000", 2024-09-18T11:30:23.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:23.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:23.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133234+0000", 2024-09-18T11:30:23.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5799999999999992e-06 2024-09-18T11:30:23.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:23.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133242+0000", 2024-09-18T11:30:23.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4039999999999995e-06 2024-09-18T11:30:23.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:23.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133256+0000", 2024-09-18T11:30:23.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3278000000000001e-05 2024-09-18T11:30:23.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:23.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133293+0000", 2024-09-18T11:30:23.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7799000000000002e-05 2024-09-18T11:30:23.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:23.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133298+0000", 2024-09-18T11:30:23.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9110000000000001e-06 2024-09-18T11:30:23.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:23.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.137554+0000", 2024-09-18T11:30:23.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042556099999999999 2024-09-18T11:30:23.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:23.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.137713+0000", 2024-09-18T11:30:23.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015937199999999999 2024-09-18T11:30:23.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:23.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139184+0000", 2024-09-18T11:30:23.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001470694 2024-09-18T11:30:23.948 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:23.948 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:23.948 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:23.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.948 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:901 4.0 4:0aec5e1c:::smithi11939506-50:head [sparse-read 0~1781150] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:23.949 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:23.949 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.806518962, 2024-09-18T11:30:23.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063828720000000004, 2024-09-18T11:30:23.949 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:23.949 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:23.949 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:23.949 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:23.949 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:23.949 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 901 2024-09-18T11:30:23.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.949 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:23.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:23.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:23.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:23.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:23.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:23.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:23.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:23.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:23.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133268+0000", 2024-09-18T11:30:23.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9610000000000001e-06 2024-09-18T11:30:23.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:23.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133280+0000", 2024-09-18T11:30:23.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2481e-05 2024-09-18T11:30:23.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:23.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133285+0000", 2024-09-18T11:30:23.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5469999999999998e-06 2024-09-18T11:30:23.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:23.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133317+0000", 2024-09-18T11:30:23.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1838999999999998e-05 2024-09-18T11:30:23.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:23.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133320+0000", 2024-09-18T11:30:23.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3009999999999998e-06 2024-09-18T11:30:23.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:23.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139225+0000", 2024-09-18T11:30:23.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059047919999999999 2024-09-18T11:30:23.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:23.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139330+0000", 2024-09-18T11:30:23.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104924 2024-09-18T11:30:23.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:23.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139648+0000", 2024-09-18T11:30:23.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00031854799999999999 2024-09-18T11:30:23.953 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:23.954 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:23.954 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:23.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.954 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:23.954 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:30:23.954 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.770364922000001, 2024-09-18T11:30:23.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:30:23.954 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:23.954 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:23.954 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:23.954 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:23.954 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:30:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:30:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:30:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:30:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:30:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:30:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:23.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:30:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:30:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:30:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:30:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:30:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:30:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:30:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:30:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:30:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:30:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1000 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164518+0000", 2024-09-18T11:30:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.7752668160000002, 2024-09-18T11:30:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005734093, 2024-09-18T11:30:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1000 2024-09-18T11:30:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:23.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164518+0000", 2024-09-18T11:30:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164518+0000", 2024-09-18T11:30:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:23.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164515+0000", 2024-09-18T11:30:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164525+0000", 2024-09-18T11:30:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5480000000000007e-06 2024-09-18T11:30:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164542+0000", 2024-09-18T11:30:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7643e-05 2024-09-18T11:30:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164553+0000", 2024-09-18T11:30:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0581000000000001e-05 2024-09-18T11:30:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164637+0000", 2024-09-18T11:30:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3825999999999993e-05 2024-09-18T11:30:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164644+0000", 2024-09-18T11:30:23.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5499999999999997e-06 2024-09-18T11:30:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.166446+0000", 2024-09-18T11:30:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018018789999999999 2024-09-18T11:30:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.166537+0000", 2024-09-18T11:30:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0325e-05 2024-09-18T11:30:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170252+0000", 2024-09-18T11:30:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037150820000000002 2024-09-18T11:30:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1001 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:30:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.7752136440000008, 2024-09-18T11:30:23.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007461082, 2024-09-18T11:30:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1001 2024-09-18T11:30:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:30:23.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:30:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164570+0000", 2024-09-18T11:30:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164574+0000", 2024-09-18T11:30:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.822e-06 2024-09-18T11:30:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164582+0000", 2024-09-18T11:30:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3410000000000003e-06 2024-09-18T11:30:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164587+0000", 2024-09-18T11:30:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769999999999999e-06 2024-09-18T11:30:23.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164702+0000", 2024-09-18T11:30:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115705 2024-09-18T11:30:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164709+0000", 2024-09-18T11:30:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-09-18T11:30:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170296+0000", 2024-09-18T11:30:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055876390000000001 2024-09-18T11:30:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170418+0000", 2024-09-18T11:30:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121549 2024-09-18T11:30:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:23.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172032+0000", 2024-09-18T11:30:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00161407 2024-09-18T11:30:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.7751869280000001, 2024-09-18T11:30:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:30:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:23.970 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:30:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:23.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:23.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:23.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:23.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:23.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:30:23.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:30:23.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:30:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:30:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:30:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:30:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:30:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:30:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:30:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:30:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:30:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:30:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:30:23.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:30:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:30:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:30:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:30:23.975 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.7751686390000003, 2024-09-18T11:30:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:30:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:30:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:30:23.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:23.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:30:23.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:23.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:30:23.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:23.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:30:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:30:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:30:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:30:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:30:23.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:30:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:30:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:30:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:30:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:30:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:30:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:30:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:30:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:30:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:30:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:30:23.980 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:30:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.7751078699999994, 2024-09-18T11:30:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:30:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:23.981 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:30:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:30:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:23.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:30:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:30:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:30:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:30:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:30:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:30:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:30:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:30:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:30:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:30:23.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:30:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:30:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:30:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:30:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:30:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:30:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:30:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:30:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.986 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:30:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.7750855489999999, 2024-09-18T11:30:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:30:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:30:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:30:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:30:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:30:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:23.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:30:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:30:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:30:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:30:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:23.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:30:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:30:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:30:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:30:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:30:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:30:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:30:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:30:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:30:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:30:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:30:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:30:23.991 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1018 4.0 4:0aec5e1c:::smithi11939506-50:head [write 1622620~730352] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:30:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.2588509929999994, 2024-09-18T11:30:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064405160000000003, 2024-09-18T11:30:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:23.992 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1018 2024-09-18T11:30:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:30:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:30:23.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680932+0000", 2024-09-18T11:30:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681334+0000", 2024-09-18T11:30:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000401803 2024-09-18T11:30:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:23.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681350+0000", 2024-09-18T11:30:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6218999999999998e-05 2024-09-18T11:30:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681374+0000", 2024-09-18T11:30:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4088e-05 2024-09-18T11:30:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681415+0000", 2024-09-18T11:30:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1307000000000003e-05 2024-09-18T11:30:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681507+0000", 2024-09-18T11:30:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1867999999999995e-05 2024-09-18T11:30:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681671+0000", 2024-09-18T11:30:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016406 2024-09-18T11:30:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.683856+0000", 2024-09-18T11:30:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002184624 2024-09-18T11:30:23.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687266+0000", 2024-09-18T11:30:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034105490000000001 2024-09-18T11:30:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687321+0000", 2024-09-18T11:30:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4960999999999998e-05 2024-09-18T11:30:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687374+0000", 2024-09-18T11:30:23.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2553000000000003e-05 2024-09-18T11:30:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1037 4.0 4:019b6e09:::smithi11939506-40:head [write 477332~766101] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.707508+0000", 2024-09-18T11:30:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.2322762669999996, 2024-09-18T11:30:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050976160000000001, 2024-09-18T11:30:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:23.998 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1037 2024-09-18T11:30:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.707508+0000", 2024-09-18T11:30:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.707508+0000", 2024-09-18T11:30:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:23.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.707506+0000", 2024-09-18T11:30:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708363+0000", 2024-09-18T11:30:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00085727299999999998 2024-09-18T11:30:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708388+0000", 2024-09-18T11:30:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5031000000000001e-05 2024-09-18T11:30:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708396+0000", 2024-09-18T11:30:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0870000000000007e-06 2024-09-18T11:30:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708452+0000", 2024-09-18T11:30:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.533e-05 2024-09-18T11:30:24.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:24.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708563+0000", 2024-09-18T11:30:24.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011086600000000001 2024-09-18T11:30:24.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:24.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708761+0000", 2024-09-18T11:30:24.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019803 2024-09-18T11:30:24.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:24.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.711546+0000", 2024-09-18T11:30:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002785689 2024-09-18T11:30:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712507+0000", 2024-09-18T11:30:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096094300000000002 2024-09-18T11:30:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712557+0000", 2024-09-18T11:30:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9701999999999999e-05 2024-09-18T11:30:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712606+0000", 2024-09-18T11:30:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.888e-05 2024-09-18T11:30:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1040 4.0 4:019b6e09:::smithi11939506-40:head [setxattr _header (56) in=7b,truncate 3277332] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.719140+0000", 2024-09-18T11:30:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.2206442939999995, 2024-09-18T11:30:24.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049512000000000002, 2024-09-18T11:30:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1040 2024-09-18T11:30:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719140+0000", 2024-09-18T11:30:24.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719140+0000", 2024-09-18T11:30:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719140+0000", 2024-09-18T11:30:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719143+0000", 2024-09-18T11:30:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5389999999999999e-06 2024-09-18T11:30:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719149+0000", 2024-09-18T11:30:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0249999999999999e-06 2024-09-18T11:30:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719153+0000", 2024-09-18T11:30:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4910000000000001e-06 2024-09-18T11:30:24.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.720365+0000", 2024-09-18T11:30:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001212346 2024-09-18T11:30:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.720476+0000", 2024-09-18T11:30:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011156500000000001 2024-09-18T11:30:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.720662+0000", 2024-09-18T11:30:24.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018538199999999999 2024-09-18T11:30:24.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:24.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.722050+0000", 2024-09-18T11:30:24.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001388291 2024-09-18T11:30:24.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:24.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.724024+0000", 2024-09-18T11:30:24.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019738860000000002 2024-09-18T11:30:24.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.724055+0000", 2024-09-18T11:30:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1288000000000003e-05 2024-09-18T11:30:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.724091+0000", 2024-09-18T11:30:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5960999999999997e-05 2024-09-18T11:30:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.010 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1041 4.0 4:019b6e09:::smithi11939506-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.719166+0000", 2024-09-18T11:30:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.2206186500000005, 2024-09-18T11:30:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050746640000000004, 2024-09-18T11:30:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1041 2024-09-18T11:30:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719166+0000", 2024-09-18T11:30:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719166+0000", 2024-09-18T11:30:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719165+0000", 2024-09-18T11:30:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719168+0000", 2024-09-18T11:30:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6220000000000001e-06 2024-09-18T11:30:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719174+0000", 2024-09-18T11:30:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2020000000000001e-06 2024-09-18T11:30:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719178+0000", 2024-09-18T11:30:24.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6890000000000002e-06 2024-09-18T11:30:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.720928+0000", 2024-09-18T11:30:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017504490000000001 2024-09-18T11:30:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.721050+0000", 2024-09-18T11:30:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121477 2024-09-18T11:30:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:24.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.724120+0000", 2024-09-18T11:30:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0030705569999999998 2024-09-18T11:30:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.724176+0000", 2024-09-18T11:30:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5327000000000001e-05 2024-09-18T11:30:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.724240+0000", 2024-09-18T11:30:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4806000000000003e-05 2024-09-18T11:30:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:24.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1050 4.0 4:0aec5e1c:::smithi11939506-50:head [write 510946~632952] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.185718+0000", 2024-09-18T11:30:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.7540660209999999, 2024-09-18T11:30:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055213279999999998, 2024-09-18T11:30:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1050 2024-09-18T11:30:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185718+0000", 2024-09-18T11:30:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185718+0000", 2024-09-18T11:30:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185717+0000", 2024-09-18T11:30:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:24.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185850+0000", 2024-09-18T11:30:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000133616 2024-09-18T11:30:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185867+0000", 2024-09-18T11:30:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7047000000000001e-05 2024-09-18T11:30:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185874+0000", 2024-09-18T11:30:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4250000000000003e-06 2024-09-18T11:30:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185926+0000", 2024-09-18T11:30:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1867999999999999e-05 2024-09-18T11:30:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185985+0000", 2024-09-18T11:30:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9769000000000003e-05 2024-09-18T11:30:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.186079+0000", 2024-09-18T11:30:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3109000000000006e-05 2024-09-18T11:30:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.187878+0000", 2024-09-18T11:30:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001799557 2024-09-18T11:30:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191165+0000", 2024-09-18T11:30:24.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003286892 2024-09-18T11:30:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191196+0000", 2024-09-18T11:30:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0607e-05 2024-09-18T11:30:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191240+0000", 2024-09-18T11:30:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4078e-05 2024-09-18T11:30:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:24.021 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1051 4.0 4:0aec5e1c:::smithi11939506-50:head [write 1798066~477679] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.191955+0000", 2024-09-18T11:30:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.7478295639999999, 2024-09-18T11:30:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004674246, 2024-09-18T11:30:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1051 2024-09-18T11:30:24.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191955+0000", 2024-09-18T11:30:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191955+0000", 2024-09-18T11:30:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191954+0000", 2024-09-18T11:30:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.192068+0000", 2024-09-18T11:30:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113999 2024-09-18T11:30:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.192079+0000", 2024-09-18T11:30:24.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1477999999999999e-05 2024-09-18T11:30:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.192083+0000", 2024-09-18T11:30:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4170000000000002e-06 2024-09-18T11:30:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.192115+0000", 2024-09-18T11:30:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1378999999999999e-05 2024-09-18T11:30:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:24.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.192170+0000", 2024-09-18T11:30:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5631000000000003e-05 2024-09-18T11:30:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.192261+0000", 2024-09-18T11:30:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0748000000000006e-05 2024-09-18T11:30:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.193630+0000", 2024-09-18T11:30:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013684999999999999 2024-09-18T11:30:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.196567+0000", 2024-09-18T11:30:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029371279999999998 2024-09-18T11:30:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.196597+0000", 2024-09-18T11:30:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0480999999999998e-05 2024-09-18T11:30:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.196629+0000", 2024-09-18T11:30:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1767999999999998e-05 2024-09-18T11:30:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:24.027 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:30:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.993960666, 2024-09-18T11:30:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:30:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:24.028 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:30:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:30:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:30:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:30:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:30:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:30:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:30:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:30:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:30:24.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:30:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:30:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:30:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:30:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:30:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:30:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:30:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:30:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:30:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:30:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:30:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:30:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:30:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:30:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:30:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:30:24.033 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1121 4.f e38/22 4:f5d7df41:::smithi11939506-16:head v 38'34, mlcod=30'32)", 2024-09-18T11:30:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:30:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.9739734069999999, 2024-09-18T11:30:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060953680000000003, 2024-09-18T11:30:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:30:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:30:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965810+0000", 2024-09-18T11:30:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966142+0000", 2024-09-18T11:30:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000331591 2024-09-18T11:30:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966154+0000", 2024-09-18T11:30:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.253e-05 2024-09-18T11:30:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966160+0000", 2024-09-18T11:30:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7130000000000002e-06 2024-09-18T11:30:24.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966196+0000", 2024-09-18T11:30:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6276999999999998e-05 2024-09-18T11:30:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966217+0000", 2024-09-18T11:30:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0335000000000001e-05 2024-09-18T11:30:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:24.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.971872+0000", 2024-09-18T11:30:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056549290000000004 2024-09-18T11:30:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.971906+0000", 2024-09-18T11:30:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4702000000000001e-05 2024-09-18T11:30:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:29 3.2 3:5fa8fcfa:::ec59520759824f0488c7a3ca135dacb324de2330e16b7f54053de5719c4e96b6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:24.038 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:30:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.25960275, 2024-09-18T11:30:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064126499999999998, 2024-09-18T11:30:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:30:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:30:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680181+0000", 2024-09-18T11:30:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680654+0000", 2024-09-18T11:30:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047334900000000002 2024-09-18T11:30:24.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680666+0000", 2024-09-18T11:30:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2232000000000001e-05 2024-09-18T11:30:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680672+0000", 2024-09-18T11:30:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:30:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:24.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681222+0000", 2024-09-18T11:30:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00054928699999999997 2024-09-18T11:30:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681332+0000", 2024-09-18T11:30:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110634 2024-09-18T11:30:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681552+0000", 2024-09-18T11:30:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021926899999999999 2024-09-18T11:30:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682950+0000", 2024-09-18T11:30:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013982669999999999 2024-09-18T11:30:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686503+0000", 2024-09-18T11:30:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035536130000000002 2024-09-18T11:30:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686553+0000", 2024-09-18T11:30:24.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9801e-05 2024-09-18T11:30:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686594+0000", 2024-09-18T11:30:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1022000000000001e-05 2024-09-18T11:30:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:30 3.2 3:49074e56:::7fc21ffee1ba17f2a9197c61ff60ef0b03804c121c8aeb83ea424b7ac77081e9:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.680698+0000", 2024-09-18T11:30:24.044 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.2590864320000001, 2024-09-18T11:30:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060349130000000003, 2024-09-18T11:30:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680698+0000", 2024-09-18T11:30:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680698+0000", 2024-09-18T11:30:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:24.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680697+0000", 2024-09-18T11:30:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680750+0000", 2024-09-18T11:30:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3170000000000001e-05 2024-09-18T11:30:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680760+0000", 2024-09-18T11:30:24.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3379999999999994e-06 2024-09-18T11:30:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680766+0000", 2024-09-18T11:30:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8249999999999999e-06 2024-09-18T11:30:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682201+0000", 2024-09-18T11:30:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001434868 2024-09-18T11:30:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:24.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682294+0000", 2024-09-18T11:30:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2965000000000002e-05 2024-09-18T11:30:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682445+0000", 2024-09-18T11:30:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015103600000000001 2024-09-18T11:30:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.683524+0000", 2024-09-18T11:30:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0010789980000000001 2024-09-18T11:30:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686687+0000", 2024-09-18T11:30:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003163052 2024-09-18T11:30:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686714+0000", 2024-09-18T11:30:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6369000000000001e-05 2024-09-18T11:30:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686733+0000", 2024-09-18T11:30:24.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9117999999999999e-05 2024-09-18T11:30:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1199 4.0 4:0aec5e1c:::smithi11939506-50:head [write 763051~794393] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-18T11:30:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:30:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.7427886259999998, 2024-09-18T11:30:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067768530000000002, 2024-09-18T11:30:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:24.050 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1199 2024-09-18T11:30:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:30:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:24.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:30:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196994+0000", 2024-09-18T11:30:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197355+0000", 2024-09-18T11:30:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036035199999999998 2024-09-18T11:30:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197366+0000", 2024-09-18T11:30:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.17e-05 2024-09-18T11:30:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197371+0000", 2024-09-18T11:30:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6099999999999999e-06 2024-09-18T11:30:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197416+0000", 2024-09-18T11:30:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5099000000000003e-05 2024-09-18T11:30:24.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197420+0000", 2024-09-18T11:30:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3189999999999996e-06 2024-09-18T11:30:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199392+0000", 2024-09-18T11:30:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019720139999999998 2024-09-18T11:30:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:24.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199452+0000", 2024-09-18T11:30:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9929999999999997e-05 2024-09-18T11:30:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199565+0000", 2024-09-18T11:30:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112811 2024-09-18T11:30:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.202108+0000", 2024-09-18T11:30:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025429060000000002 2024-09-18T11:30:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:24.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203714+0000", 2024-09-18T11:30:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001606335 2024-09-18T11:30:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203745+0000", 2024-09-18T11:30:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0837999999999998e-05 2024-09-18T11:30:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203773+0000", 2024-09-18T11:30:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7562000000000001e-05 2024-09-18T11:30:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:24.056 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:24.057 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:24.057 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:24.057 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:24.057 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:24.058 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:30:24.059 INFO:tasks.rados.rados.0.smithi119.stdout:795: finishing write tid 1 to smithi11939506-41 2024-09-18T11:30:24.059 INFO:tasks.rados.rados.0.smithi119.stdout:794: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T11:30:24.059 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:24.057+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:24.109 INFO:tasks.rados.rados.0.smithi119.stdout:789: expect (ObjNum 70 snap 2 seq_num 70) 2024-09-18T11:30:24.158 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:24.157+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:24.168 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:24.169 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:24.169 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:24.169 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:24.178 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:30:24.188 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is set 2024-09-18T11:30:24.252 INFO:tasks.rados.rados.0.smithi119.stdout:795: finishing write tid 2 to smithi11939506-41 2024-09-18T11:30:24.252 INFO:tasks.rados.rados.0.smithi119.stdout:791: expect (ObjNum 3 snap 0 seq_num 3) 2024-09-18T11:30:24.259 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:24.258+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:24.277 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:24.277 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:24.277 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:24.277 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:24.278 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:24.285 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:30:24.317 INFO:tasks.rados.rados.0.smithi119.stdout:795: finishing write tid 3 to smithi11939506-41 2024-09-18T11:30:24.317 INFO:tasks.rados.rados.0.smithi119.stdout:795: finishing write tid 4 to smithi11939506-41 2024-09-18T11:30:24.318 INFO:tasks.rados.rados.0.smithi119.stdout:789: done (14 left) 2024-09-18T11:30:24.318 INFO:tasks.rados.rados.0.smithi119.stdout:790: done (13 left) 2024-09-18T11:30:24.318 INFO:tasks.rados.rados.0.smithi119.stdout:791: done (12 left) 2024-09-18T11:30:24.318 INFO:tasks.rados.rados.0.smithi119.stdout:792: done (11 left) 2024-09-18T11:30:24.318 INFO:tasks.rados.rados.0.smithi119.stdout:794: done (10 left) 2024-09-18T11:30:24.318 INFO:tasks.rados.rados.0.smithi119.stdout:805: read oid 31 snap -1 2024-09-18T11:30:24.318 INFO:tasks.rados.rados.0.smithi119.stdout:805: expect (ObjNum 63 snap 0 seq_num 63) 2024-09-18T11:30:24.318 INFO:tasks.rados.rados.0.smithi119.stdout:806: read oid 39 snap -1 2024-09-18T11:30:24.318 INFO:tasks.rados.rados.0.smithi119.stdout:806: expect deleted 2024-09-18T11:30:24.318 INFO:tasks.rados.rados.0.smithi119.stdout:807: read oid 17 snap -1 2024-09-18T11:30:24.318 INFO:tasks.rados.rados.0.smithi119.stdout:807: expect (ObjNum 16 snap 0 seq_num 16) 2024-09-18T11:30:24.318 INFO:tasks.rados.rados.0.smithi119.stdout:808: read oid 18 snap -1 2024-09-18T11:30:24.318 INFO:tasks.rados.rados.0.smithi119.stdout:808: expect (ObjNum 60 snap 0 seq_num 60) 2024-09-18T11:30:24.318 INFO:tasks.rados.rados.0.smithi119.stdout:809: delete oid 21 current snap is 6 2024-09-18T11:30:24.319 INFO:tasks.rados.rados.0.smithi119.stdout:795: finishing write tid 5 to smithi11939506-41 2024-09-18T11:30:24.319 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 41 v 63 (ObjNum 104 snap 6 seq_num 104) dirty exists 2024-09-18T11:30:24.319 INFO:tasks.rados.rados.0.smithi119.stdout:795: left oid 41 (ObjNum 104 snap 6 seq_num 104) 2024-09-18T11:30:24.319 INFO:tasks.rados.rados.0.smithi119.stdout:796: finishing write tid 1 to smithi11939506-5 2024-09-18T11:30:24.319 INFO:tasks.rados.rados.0.smithi119.stdout:796: finishing write tid 2 to smithi11939506-5 2024-09-18T11:30:24.319 INFO:tasks.rados.rados.0.smithi119.stdout:798: expect (ObjNum 74 snap 3 seq_num 74) 2024-09-18T11:30:24.360 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:24.359+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:24.426 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:24.426 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:24.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:24.426 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:24.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.427 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:878 4.e 4:7410c0e0:::smithi11939506-47:2 [read 0~2773122,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:24.427 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116276+0000", 2024-09-18T11:30:24.427 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.307401974999999, 2024-09-18T11:30:24.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041562079999999998, 2024-09-18T11:30:24.427 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.427 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:24.427 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:24.427 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:24.427 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:24.427 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 878 2024-09-18T11:30:24.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.427 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116276+0000", 2024-09-18T11:30:24.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116276+0000", 2024-09-18T11:30:24.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116276+0000", 2024-09-18T11:30:24.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:24.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116279+0000", 2024-09-18T11:30:24.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.704e-06 2024-09-18T11:30:24.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116289+0000", 2024-09-18T11:30:24.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7210000000000006e-06 2024-09-18T11:30:24.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116293+0000", 2024-09-18T11:30:24.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8739999999999997e-06 2024-09-18T11:30:24.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118772+0000", 2024-09-18T11:30:24.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024794560000000001 2024-09-18T11:30:24.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118852+0000", 2024-09-18T11:30:24.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9338999999999994e-05 2024-09-18T11:30:24.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.120432+0000", 2024-09-18T11:30:24.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015806749999999999 2024-09-18T11:30:24.431 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.431 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.431 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.432 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:879 4.e 4:7410c0e0:::smithi11939506-47:2 [sparse-read 0~1211253] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:24.432 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116299+0000", 2024-09-18T11:30:24.432 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.307378980999999, 2024-09-18T11:30:24.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043627500000000003, 2024-09-18T11:30:24.432 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.432 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:24.432 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:24.432 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:24.432 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:24.432 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 879 2024-09-18T11:30:24.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.432 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116299+0000", 2024-09-18T11:30:24.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116299+0000", 2024-09-18T11:30:24.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116299+0000", 2024-09-18T11:30:24.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:24.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116301+0000", 2024-09-18T11:30:24.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6860000000000002e-06 2024-09-18T11:30:24.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116309+0000", 2024-09-18T11:30:24.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1300000000000003e-06 2024-09-18T11:30:24.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116315+0000", 2024-09-18T11:30:24.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3219999999999998e-06 2024-09-18T11:30:24.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.120452+0000", 2024-09-18T11:30:24.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041371539999999997 2024-09-18T11:30:24.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.120521+0000", 2024-09-18T11:30:24.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8674e-05 2024-09-18T11:30:24.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.120662+0000", 2024-09-18T11:30:24.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000141225 2024-09-18T11:30:24.436 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.436 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.436 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.436 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:886 4.d 4:b30010d9:::smithi11939506-43:2 [sparse-read 0~1186566] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:24.436 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116561+0000", 2024-09-18T11:30:24.437 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.307117353000001, 2024-09-18T11:30:24.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032690699999999998, 2024-09-18T11:30:24.437 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.437 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:24.437 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:24.437 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:24.437 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:24.437 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 886 2024-09-18T11:30:24.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.437 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116561+0000", 2024-09-18T11:30:24.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116561+0000", 2024-09-18T11:30:24.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116560+0000", 2024-09-18T11:30:24.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:24.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116563+0000", 2024-09-18T11:30:24.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.994e-06 2024-09-18T11:30:24.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116570+0000", 2024-09-18T11:30:24.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7150000000000001e-06 2024-09-18T11:30:24.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116574+0000", 2024-09-18T11:30:24.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5990000000000002e-06 2024-09-18T11:30:24.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.119485+0000", 2024-09-18T11:30:24.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002911004 2024-09-18T11:30:24.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.119579+0000", 2024-09-18T11:30:24.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4775e-05 2024-09-18T11:30:24.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.119830+0000", 2024-09-18T11:30:24.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00025051700000000001 2024-09-18T11:30:24.441 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.441 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.441 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.442 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:891 3.3 3:ef319324:::af1dd004311afeb9a2c923c847c83541b2f70de017741e00e9757a0c3b62b102:head [write 0~276992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:24.442 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.118070+0000", 2024-09-18T11:30:24.442 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.305607932000001, 2024-09-18T11:30:24.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038997620000000002, 2024-09-18T11:30:24.442 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.442 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.442 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:24.442 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:24.443 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:24.443 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 891 2024-09-18T11:30:24.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.443 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118070+0000", 2024-09-18T11:30:24.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118070+0000", 2024-09-18T11:30:24.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118069+0000", 2024-09-18T11:30:24.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:24.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118576+0000", 2024-09-18T11:30:24.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00050754299999999995 2024-09-18T11:30:24.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118592+0000", 2024-09-18T11:30:24.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5551000000000001e-05 2024-09-18T11:30:24.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118600+0000", 2024-09-18T11:30:24.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2949999999999994e-06 2024-09-18T11:30:24.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118638+0000", 2024-09-18T11:30:24.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8229000000000003e-05 2024-09-18T11:30:24.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118759+0000", 2024-09-18T11:30:24.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012116999999999999 2024-09-18T11:30:24.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:24.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118904+0000", 2024-09-18T11:30:24.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000144816 2024-09-18T11:30:24.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:24.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.120275+0000", 2024-09-18T11:30:24.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00137031 2024-09-18T11:30:24.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:24.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.121915+0000", 2024-09-18T11:30:24.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016402509999999999 2024-09-18T11:30:24.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:24.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.121947+0000", 2024-09-18T11:30:24.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2218999999999998e-05 2024-09-18T11:30:24.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.121970+0000", 2024-09-18T11:30:24.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2983e-05 2024-09-18T11:30:24.448 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.448 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.448 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.448 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.3.13:29 3.3 3:e93cf80d:::smithi11939506-40 low_tier:head [read 2438656~185344 [fadvise_sequential]] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:24.448 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.169975+0000", 2024-09-18T11:30:24.449 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.253703231999999, 2024-09-18T11:30:24.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003234799, 2024-09-18T11:30:24.449 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.449 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:24.449 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169975+0000", 2024-09-18T11:30:24.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169975+0000", 2024-09-18T11:30:24.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169975+0000", 2024-09-18T11:30:24.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:24.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169977+0000", 2024-09-18T11:30:24.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.768e-06 2024-09-18T11:30:24.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169983+0000", 2024-09-18T11:30:24.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0619999999999997e-06 2024-09-18T11:30:24.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.169987+0000", 2024-09-18T11:30:24.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6100000000000002e-06 2024-09-18T11:30:24.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.172692+0000", 2024-09-18T11:30:24.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00270491 2024-09-18T11:30:24.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.172736+0000", 2024-09-18T11:30:24.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4005000000000003e-05 2024-09-18T11:30:24.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.173210+0000", 2024-09-18T11:30:24.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00047385100000000001 2024-09-18T11:30:24.454 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.454 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.454 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.455 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:988 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~2436982,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:24.455 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.146537+0000", 2024-09-18T11:30:24.455 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.2771407650000004, 2024-09-18T11:30:24.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036166480000000001, 2024-09-18T11:30:24.455 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.455 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:24.455 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:24.455 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:24.455 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:24.455 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 988 2024-09-18T11:30:24.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.455 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.146537+0000", 2024-09-18T11:30:24.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.146537+0000", 2024-09-18T11:30:24.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.146537+0000", 2024-09-18T11:30:24.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:24.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.146539+0000", 2024-09-18T11:30:24.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1639999999999999e-06 2024-09-18T11:30:24.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.146543+0000", 2024-09-18T11:30:24.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.163e-06 2024-09-18T11:30:24.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.146544+0000", 2024-09-18T11:30:24.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9860000000000001e-06 2024-09-18T11:30:24.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.146637+0000", 2024-09-18T11:30:24.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2134999999999998e-05 2024-09-18T11:30:24.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:24.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.146643+0000", 2024-09-18T11:30:24.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7429999999999996e-06 2024-09-18T11:30:24.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.149466+0000", 2024-09-18T11:30:24.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028227870000000002 2024-09-18T11:30:24.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.149567+0000", 2024-09-18T11:30:24.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101257 2024-09-18T11:30:24.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.150154+0000", 2024-09-18T11:30:24.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000586679 2024-09-18T11:30:24.460 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.460 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.460 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.460 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1025 4.c 4:323f30dd:::smithi11939506-12:head [sparse-read 0~2047579,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:24.460 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.686094+0000", 2024-09-18T11:30:24.460 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.7375845830000003, 2024-09-18T11:30:24.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034385359999999998, 2024-09-18T11:30:24.461 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.461 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:24.461 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:24.461 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:24.461 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:24.461 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1025 2024-09-18T11:30:24.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.461 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686094+0000", 2024-09-18T11:30:24.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686094+0000", 2024-09-18T11:30:24.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686092+0000", 2024-09-18T11:30:24.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:24.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686101+0000", 2024-09-18T11:30:24.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.816e-06 2024-09-18T11:30:24.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686116+0000", 2024-09-18T11:30:24.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5709000000000002e-05 2024-09-18T11:30:24.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686125+0000", 2024-09-18T11:30:24.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2660000000000001e-06 2024-09-18T11:30:24.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686167+0000", 2024-09-18T11:30:24.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1916999999999999e-05 2024-09-18T11:30:24.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686253+0000", 2024-09-18T11:30:24.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6546999999999997e-05 2024-09-18T11:30:24.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689532+0000", 2024-09-18T11:30:24.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032790190000000002 2024-09-18T11:30:24.464 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.464 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.465 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.465 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1026 4.c 4:323f30dd:::smithi11939506-12:head [read 0~3176490,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:24.465 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.686142+0000", 2024-09-18T11:30:24.465 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.7375362669999994, 2024-09-18T11:30:24.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048199560000000002, 2024-09-18T11:30:24.465 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.465 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:24.465 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:24.465 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:24.465 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:24.465 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1026 2024-09-18T11:30:24.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.466 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686142+0000", 2024-09-18T11:30:24.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686142+0000", 2024-09-18T11:30:24.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686141+0000", 2024-09-18T11:30:24.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:24.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686145+0000", 2024-09-18T11:30:24.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0999999999999997e-06 2024-09-18T11:30:24.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686153+0000", 2024-09-18T11:30:24.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0250000000000002e-06 2024-09-18T11:30:24.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686158+0000", 2024-09-18T11:30:24.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1389999999999997e-06 2024-09-18T11:30:24.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689573+0000", 2024-09-18T11:30:24.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034151369999999999 2024-09-18T11:30:24.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689630+0000", 2024-09-18T11:30:24.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6830000000000003e-05 2024-09-18T11:30:24.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690962+0000", 2024-09-18T11:30:24.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013323090000000001 2024-09-18T11:30:24.469 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.469 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.469 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.470 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1027 4.c 4:323f30dd:::smithi11939506-12:head [sparse-read 0~2047579] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:24.470 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.686162+0000", 2024-09-18T11:30:24.470 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.7375161769999998, 2024-09-18T11:30:24.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050932319999999996, 2024-09-18T11:30:24.470 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.470 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:24.470 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:24.471 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:24.471 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:24.471 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1027 2024-09-18T11:30:24.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.471 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686162+0000", 2024-09-18T11:30:24.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686162+0000", 2024-09-18T11:30:24.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686162+0000", 2024-09-18T11:30:24.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:24.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686164+0000", 2024-09-18T11:30:24.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3489999999999999e-06 2024-09-18T11:30:24.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686169+0000", 2024-09-18T11:30:24.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1039999999999998e-06 2024-09-18T11:30:24.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686172+0000", 2024-09-18T11:30:24.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3680000000000002e-06 2024-09-18T11:30:24.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690990+0000", 2024-09-18T11:30:24.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048174460000000004 2024-09-18T11:30:24.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.691056+0000", 2024-09-18T11:30:24.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6192000000000006e-05 2024-09-18T11:30:24.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.691255+0000", 2024-09-18T11:30:24.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000199142 2024-09-18T11:30:24.474 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.474 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.474 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.475 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1029 3.3 3:df268f6c:::8da2985a35cccabf5edc5a8900d2327bcfe6ed768fa38d466fa9a390ce713320:head [write 0~356864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:24.475 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.689033+0000", 2024-09-18T11:30:24.475 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.7346450119999997, 2024-09-18T11:30:24.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053345750000000003, 2024-09-18T11:30:24.475 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.475 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.475 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:24.475 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:24.475 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:24.475 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1029 2024-09-18T11:30:24.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.475 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689033+0000", 2024-09-18T11:30:24.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689033+0000", 2024-09-18T11:30:24.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689031+0000", 2024-09-18T11:30:24.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:24.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690898+0000", 2024-09-18T11:30:24.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001867032 2024-09-18T11:30:24.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690914+0000", 2024-09-18T11:30:24.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5806000000000001e-05 2024-09-18T11:30:24.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690921+0000", 2024-09-18T11:30:24.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8859999999999998e-06 2024-09-18T11:30:24.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690977+0000", 2024-09-18T11:30:24.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5612999999999998e-05 2024-09-18T11:30:24.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.691142+0000", 2024-09-18T11:30:24.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000165112 2024-09-18T11:30:24.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:24.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.691351+0000", 2024-09-18T11:30:24.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020942700000000001 2024-09-18T11:30:24.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:24.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.692616+0000", 2024-09-18T11:30:24.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012648200000000001 2024-09-18T11:30:24.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:24.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.694303+0000", 2024-09-18T11:30:24.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016875320000000001 2024-09-18T11:30:24.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:24.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.694335+0000", 2024-09-18T11:30:24.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1971000000000003e-05 2024-09-18T11:30:24.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.694368+0000", 2024-09-18T11:30:24.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2403000000000001e-05 2024-09-18T11:30:24.480 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.480 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.480 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.481 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1116 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3109262,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:24.481 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.936633+0000", 2024-09-18T11:30:24.481 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.4870452529999998, 2024-09-18T11:30:24.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003857287, 2024-09-18T11:30:24.481 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.481 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:24.481 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:24.481 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:24.481 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:24.481 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1116 2024-09-18T11:30:24.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.482 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.936633+0000", 2024-09-18T11:30:24.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.936633+0000", 2024-09-18T11:30:24.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.936633+0000", 2024-09-18T11:30:24.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:24.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.936636+0000", 2024-09-18T11:30:24.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6959999999999998e-06 2024-09-18T11:30:24.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.936642+0000", 2024-09-18T11:30:24.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7189999999999998e-06 2024-09-18T11:30:24.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.936645+0000", 2024-09-18T11:30:24.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4520000000000002e-06 2024-09-18T11:30:24.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.938926+0000", 2024-09-18T11:30:24.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022801409999999999 2024-09-18T11:30:24.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.939055+0000", 2024-09-18T11:30:24.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000129386 2024-09-18T11:30:24.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.940490+0000", 2024-09-18T11:30:24.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014353289999999999 2024-09-18T11:30:24.485 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.485 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.485 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.485 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1133 4.1 4:8b9a985d:::smithi11939506-23:a [read 0~3327753,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:24.485 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.178999+0000", 2024-09-18T11:30:24.486 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.2446790400000003, 2024-09-18T11:30:24.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059312619999999996, 2024-09-18T11:30:24.486 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.486 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:24.486 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:24.486 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:24.486 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:24.486 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1133 2024-09-18T11:30:24.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.486 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178999+0000", 2024-09-18T11:30:24.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178999+0000", 2024-09-18T11:30:24.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178997+0000", 2024-09-18T11:30:24.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:24.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179012+0000", 2024-09-18T11:30:24.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5166e-05 2024-09-18T11:30:24.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179043+0000", 2024-09-18T11:30:24.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0892999999999997e-05 2024-09-18T11:30:24.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179057+0000", 2024-09-18T11:30:24.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4212e-05 2024-09-18T11:30:24.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179217+0000", 2024-09-18T11:30:24.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016031699999999999 2024-09-18T11:30:24.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:24.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179235+0000", 2024-09-18T11:30:24.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7903000000000001e-05 2024-09-18T11:30:24.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.181282+0000", 2024-09-18T11:30:24.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002046614 2024-09-18T11:30:24.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.181350+0000", 2024-09-18T11:30:24.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8190999999999998e-05 2024-09-18T11:30:24.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184930+0000", 2024-09-18T11:30:24.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035805960000000001 2024-09-18T11:30:24.490 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.490 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.490 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.491 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1134 4.1 4:8b9a985d:::smithi11939506-23:a [sparse-read 0~1709094] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:24.491 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:24.491 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.2445852589999999, 2024-09-18T11:30:24.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061866400000000002, 2024-09-18T11:30:24.491 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.491 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:24.491 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:24.491 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:24.491 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:24.491 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1134 2024-09-18T11:30:24.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.492 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:24.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:24.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.493 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179092+0000", 2024-09-18T11:30:24.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:24.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.493 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179104+0000", 2024-09-18T11:30:24.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2346e-05 2024-09-18T11:30:24.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.493 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179128+0000", 2024-09-18T11:30:24.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4046999999999999e-05 2024-09-18T11:30:24.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179141+0000", 2024-09-18T11:30:24.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3149000000000001e-05 2024-09-18T11:30:24.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179406+0000", 2024-09-18T11:30:24.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026496100000000003 2024-09-18T11:30:24.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:24.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179431+0000", 2024-09-18T11:30:24.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5242999999999999e-05 2024-09-18T11:30:24.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184981+0000", 2024-09-18T11:30:24.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055495969999999999 2024-09-18T11:30:24.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185055+0000", 2024-09-18T11:30:24.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4375999999999994e-05 2024-09-18T11:30:24.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185280+0000", 2024-09-18T11:30:24.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022433800000000001 2024-09-18T11:30:24.496 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.496 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.496 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.496 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1135 4.1 4:8b9a985d:::smithi11939506-23:a [sparse-read 0~1709094] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:24.496 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179201+0000", 2024-09-18T11:30:24.497 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.2444768120000003, 2024-09-18T11:30:24.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063123149999999998, 2024-09-18T11:30:24.497 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.497 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:24.497 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:24.497 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:24.497 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:24.497 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1135 2024-09-18T11:30:24.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.497 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179201+0000", 2024-09-18T11:30:24.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179201+0000", 2024-09-18T11:30:24.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179200+0000", 2024-09-18T11:30:24.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:24.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179207+0000", 2024-09-18T11:30:24.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1960000000000002e-06 2024-09-18T11:30:24.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179229+0000", 2024-09-18T11:30:24.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1322999999999999e-05 2024-09-18T11:30:24.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179240+0000", 2024-09-18T11:30:24.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1793e-05 2024-09-18T11:30:24.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179460+0000", 2024-09-18T11:30:24.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000219407 2024-09-18T11:30:24.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:24.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179466+0000", 2024-09-18T11:30:24.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7490000000000004e-06 2024-09-18T11:30:24.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185300+0000", 2024-09-18T11:30:24.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058347490000000002 2024-09-18T11:30:24.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185345+0000", 2024-09-18T11:30:24.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4316000000000001e-05 2024-09-18T11:30:24.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185514+0000", 2024-09-18T11:30:24.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016905800000000001 2024-09-18T11:30:24.502 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.502 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.502 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.502 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1164 4.e 4:7410c0e0:::smithi11939506-47:head [sparse-read 0~1211253] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:24.502 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.650412+0000", 2024-09-18T11:30:24.502 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.7732658809999999, 2024-09-18T11:30:24.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032716659999999999, 2024-09-18T11:30:24.502 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.502 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:24.502 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:24.502 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:24.503 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:24.503 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1164 2024-09-18T11:30:24.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.503 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.650412+0000", 2024-09-18T11:30:24.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.650412+0000", 2024-09-18T11:30:24.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.504 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.650412+0000", 2024-09-18T11:30:24.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:24.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.504 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.650414+0000", 2024-09-18T11:30:24.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5060000000000001e-06 2024-09-18T11:30:24.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.650417+0000", 2024-09-18T11:30:24.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9170000000000002e-06 2024-09-18T11:30:24.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.650419+0000", 2024-09-18T11:30:24.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0729999999999999e-06 2024-09-18T11:30:24.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.653472+0000", 2024-09-18T11:30:24.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003053896 2024-09-18T11:30:24.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.653544+0000", 2024-09-18T11:30:24.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1408000000000005e-05 2024-09-18T11:30:24.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.653684+0000", 2024-09-18T11:30:24.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014011199999999999 2024-09-18T11:30:24.509 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.509 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.509 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.509 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.6.13:26 3.3 3:f159e761:::e59e93d7c4f7efebda4afc5896e5fcf5614bf8edbf483060bccde004b8ac774d:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:24.509 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.678107+0000", 2024-09-18T11:30:24.510 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.7455713360000003, 2024-09-18T11:30:24.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0044766939999999998, 2024-09-18T11:30:24.510 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.510 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.510 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678107+0000", 2024-09-18T11:30:24.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678107+0000", 2024-09-18T11:30:24.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678105+0000", 2024-09-18T11:30:24.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:24.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678351+0000", 2024-09-18T11:30:24.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000245751 2024-09-18T11:30:24.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678391+0000", 2024-09-18T11:30:24.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0695000000000003e-05 2024-09-18T11:30:24.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678405+0000", 2024-09-18T11:30:24.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.309e-05 2024-09-18T11:30:24.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678476+0000", 2024-09-18T11:30:24.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0938000000000004e-05 2024-09-18T11:30:24.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678573+0000", 2024-09-18T11:30:24.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7331999999999996e-05 2024-09-18T11:30:24.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:24.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678751+0000", 2024-09-18T11:30:24.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017789100000000001 2024-09-18T11:30:24.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:24.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.680035+0000", 2024-09-18T11:30:24.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012847469999999999 2024-09-18T11:30:24.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:24.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682495+0000", 2024-09-18T11:30:24.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024598599999999999 2024-09-18T11:30:24.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:24.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682535+0000", 2024-09-18T11:30:24.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9187999999999998e-05 2024-09-18T11:30:24.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682584+0000", 2024-09-18T11:30:24.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9048000000000002e-05 2024-09-18T11:30:24.516 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.517 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.517 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.517 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.13:32 3.1 e39/21 3:94708c4c:::888f815178b2a748104fffa559dcb609f143f519d8737c6aa160c9cce13a6441:head v 39'69, mlcod=30'66)", 2024-09-18T11:30:24.517 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.681793+0000", 2024-09-18T11:30:24.517 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.7418849139999999, 2024-09-18T11:30:24.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037217130000000002, 2024-09-18T11:30:24.517 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.517 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.517 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681793+0000", 2024-09-18T11:30:24.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681793+0000", 2024-09-18T11:30:24.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681788+0000", 2024-09-18T11:30:24.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999952 2024-09-18T11:30:24.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.519 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682280+0000", 2024-09-18T11:30:24.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00049141099999999997 2024-09-18T11:30:24.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.519 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682289+0000", 2024-09-18T11:30:24.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8859999999999993e-06 2024-09-18T11:30:24.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.519 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682292+0000", 2024-09-18T11:30:24.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6839999999999998e-06 2024-09-18T11:30:24.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682327+0000", 2024-09-18T11:30:24.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4289999999999999e-05 2024-09-18T11:30:24.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682340+0000", 2024-09-18T11:30:24.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3739e-05 2024-09-18T11:30:24.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:24.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.685493+0000", 2024-09-18T11:30:24.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031523850000000002 2024-09-18T11:30:24.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.685515+0000", 2024-09-18T11:30:24.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2279000000000001e-05 2024-09-18T11:30:24.521 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.521 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.522 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.522 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1180 4.1 4:8b9a985d:::smithi11939506-23:head [write 712630~452293,stat] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:24.522 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.140449+0000", 2024-09-18T11:30:24.522 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.283229639, 2024-09-18T11:30:24.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034064389999999998, 2024-09-18T11:30:24.522 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.522 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.522 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:24.522 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:24.522 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:24.522 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1180 2024-09-18T11:30:24.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.523 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.140449+0000", 2024-09-18T11:30:24.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.140449+0000", 2024-09-18T11:30:24.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.140444+0000", 2024-09-18T11:30:24.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999957 2024-09-18T11:30:24.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.524 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.140842+0000", 2024-09-18T11:30:24.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00039797999999999998 2024-09-18T11:30:24.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.140861+0000", 2024-09-18T11:30:24.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.823e-05 2024-09-18T11:30:24.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.140868+0000", 2024-09-18T11:30:24.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3259999999999998e-06 2024-09-18T11:30:24.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.140924+0000", 2024-09-18T11:30:24.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5850000000000002e-05 2024-09-18T11:30:24.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.140992+0000", 2024-09-18T11:30:24.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8617999999999999e-05 2024-09-18T11:30:24.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:24.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.141118+0000", 2024-09-18T11:30:24.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012552 2024-09-18T11:30:24.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:24.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.143206+0000", 2024-09-18T11:30:24.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020878369999999999 2024-09-18T11:30:24.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:24.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.143793+0000", 2024-09-18T11:30:24.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00058742099999999997 2024-09-18T11:30:24.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:24.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.143828+0000", 2024-09-18T11:30:24.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4755000000000002e-05 2024-09-18T11:30:24.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.143855+0000", 2024-09-18T11:30:24.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7073999999999999e-05 2024-09-18T11:30:24.530 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.530 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.530 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.530 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1185 4.5 e39/22 4:af4ba75a:::smithi11939506-37:head v 39'34, mlcod=38'31)", 2024-09-18T11:30:24.530 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.157513+0000", 2024-09-18T11:30:24.530 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.2661648720000001, 2024-09-18T11:30:24.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047192279999999998, 2024-09-18T11:30:24.530 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.530 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.530 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157513+0000", 2024-09-18T11:30:24.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157513+0000", 2024-09-18T11:30:24.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157512+0000", 2024-09-18T11:30:24.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:24.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.158094+0000", 2024-09-18T11:30:24.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00058189099999999996 2024-09-18T11:30:24.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.158110+0000", 2024-09-18T11:30:24.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5345e-05 2024-09-18T11:30:24.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.158117+0000", 2024-09-18T11:30:24.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.002e-06 2024-09-18T11:30:24.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.158151+0000", 2024-09-18T11:30:24.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4335000000000002e-05 2024-09-18T11:30:24.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.158165+0000", 2024-09-18T11:30:24.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3677e-05 2024-09-18T11:30:24.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:24.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162208+0000", 2024-09-18T11:30:24.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040431829999999997 2024-09-18T11:30:24.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162233+0000", 2024-09-18T11:30:24.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4783999999999999e-05 2024-09-18T11:30:24.535 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.535 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.535 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.536 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1186 4.5 e39/22 4:af4ba75a:::smithi11939506-37:head v 39'35, mlcod=39'32)", 2024-09-18T11:30:24.536 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.164439+0000", 2024-09-18T11:30:24.536 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.2592395789999999, 2024-09-18T11:30:24.536 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004086569, 2024-09-18T11:30:24.536 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.536 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.537 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.537 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164439+0000", 2024-09-18T11:30:24.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.537 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164439+0000", 2024-09-18T11:30:24.538 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.538 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.538 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164438+0000", 2024-09-18T11:30:24.538 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:24.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.538 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.538 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164613+0000", 2024-09-18T11:30:24.538 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017544300000000001 2024-09-18T11:30:24.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164625+0000", 2024-09-18T11:30:24.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2088e-05 2024-09-18T11:30:24.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.540 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.540 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164631+0000", 2024-09-18T11:30:24.540 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7610000000000004e-06 2024-09-18T11:30:24.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.540 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164659+0000", 2024-09-18T11:30:24.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7770999999999999e-05 2024-09-18T11:30:24.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164674+0000", 2024-09-18T11:30:24.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4620000000000001e-05 2024-09-18T11:30:24.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.542 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:24.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168491+0000", 2024-09-18T11:30:24.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038169810000000001 2024-09-18T11:30:24.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168525+0000", 2024-09-18T11:30:24.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4579000000000002e-05 2024-09-18T11:30:24.543 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.543 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.543 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.543 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.544 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.544 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:24.545 INFO:tasks.rados.rados.0.smithi119.stdout:796: finishing write tid 3 to smithi11939506-5 2024-09-18T11:30:24.545 INFO:tasks.rados.rados.0.smithi119.stdout:797: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-18T11:30:24.546 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:30:24.547 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:24.544+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:24.560 INFO:tasks.rados.rados.0.smithi119.stdout:796: finishing write tid 4 to smithi11939506-5 2024-09-18T11:30:24.560 INFO:tasks.rados.rados.0.smithi119.stdout:796: finishing write tid 5 to smithi11939506-5 2024-09-18T11:30:24.560 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 42 (ObjNum 105 snap 6 seq_num 105) dirty exists 2024-09-18T11:30:24.561 INFO:tasks.rados.rados.0.smithi119.stdout:796: left oid 5 (ObjNum 105 snap 6 seq_num 105) 2024-09-18T11:30:24.561 INFO:tasks.rados.rados.0.smithi119.stdout:800: finishing write tid 1 to smithi11939506-16 2024-09-18T11:30:24.561 INFO:tasks.rados.rados.0.smithi119.stdout:800: finishing write tid 2 to smithi11939506-16 2024-09-18T11:30:24.561 INFO:tasks.rados.rados.0.smithi119.stdout:795: done (14 left) 2024-09-18T11:30:24.561 INFO:tasks.rados.rados.0.smithi119.stdout:796: done (13 left) 2024-09-18T11:30:24.561 INFO:tasks.rados.rados.0.smithi119.stdout:797: done (12 left) 2024-09-18T11:30:24.561 INFO:tasks.rados.rados.0.smithi119.stdout:798: done (11 left) 2024-09-18T11:30:24.561 INFO:tasks.rados.rados.0.smithi119.stdout:799: done (10 left) 2024-09-18T11:30:24.561 INFO:tasks.rados.rados.0.smithi119.stdout:809: done (9 left) 2024-09-18T11:30:24.561 INFO:tasks.rados.rados.0.smithi119.stdout:810: set_chunk oid 47 target oid 2024-09-18T11:30:24.567 INFO:tasks.rados.rados.0.smithi119.stdout:810: set_chunk oid 47 offset: 209408 length: 881664 target oid a705ccfdb1cdc6ef082e4405c7b5f3c792a2b27cdec018193178c38e5ec259a4 offset: 0 snap -1 2024-09-18T11:30:24.567 INFO:tasks.rados.rados.0.smithi119.stdout:811: snap_create 2024-09-18T11:30:24.567 INFO:tasks.rados.rados.0.smithi119.stdout:803: got (22) Invalid argument 2024-09-18T11:30:24.568 INFO:tasks.rados.rados.0.smithi119.stdout:800: finishing write tid 3 to smithi11939506-16 2024-09-18T11:30:24.568 INFO:tasks.rados.rados.0.smithi119.stdout:801: expect (ObjNum 34 snap 0 seq_num 34) 2024-09-18T11:30:24.646 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:24.645+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:24.648 INFO:tasks.rados.rados.0.smithi119.stdout:800: finishing write tid 4 to smithi11939506-16 2024-09-18T11:30:24.648 INFO:tasks.rados.rados.0.smithi119.stdout:800: finishing write tid 5 to smithi11939506-16 2024-09-18T11:30:24.648 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 43 (ObjNum 106 snap 6 seq_num 106) dirty exists 2024-09-18T11:30:24.648 INFO:tasks.rados.rados.0.smithi119.stdout:800: left oid 16 (ObjNum 106 snap 6 seq_num 106) 2024-09-18T11:30:24.648 INFO:tasks.rados.rados.0.smithi119.stdout:804: got (0) Success 2024-09-18T11:30:24.648 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 9 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-18T11:30:24.648 INFO:tasks.rados.rados.0.smithi119.stdout:807: expect (ObjNum 16 snap 0 seq_num 16) 2024-09-18T11:30:24.684 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:24.684 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:24.684 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:24.684 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:24.692 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:30:24.720 INFO:tasks.rados.rados.0.smithi119.stdout:805: expect (ObjNum 63 snap 0 seq_num 63) 2024-09-18T11:30:24.756 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:24.755+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:24.852 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:24.852 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:24.852 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:24.852 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:24.852 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:24.855 INFO:tasks.rados.rados.0.smithi119.stdout:808: expect (ObjNum 60 snap 0 seq_num 60) 2024-09-18T11:30:24.857 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:24.856+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:24.860 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:30:24.958 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:24.957+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:24.959 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:24.959 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:24.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:24.960 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:24.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.960 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:30:24.960 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:30:24.960 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 34.930896177999998, 2024-09-18T11:30:24.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073601550000000002, 2024-09-18T11:30:24.960 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.960 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.960 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:24.960 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:24.960 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:24.960 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1 2024-09-18T11:30:24.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.961 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:30:24.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:30:24.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026524+0000", 2024-09-18T11:30:24.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999905 2024-09-18T11:30:24.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026541+0000", 2024-09-18T11:30:24.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7070000000000001e-05 2024-09-18T11:30:24.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026554+0000", 2024-09-18T11:30:24.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.288e-05 2024-09-18T11:30:24.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026564+0000", 2024-09-18T11:30:24.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0197000000000001e-05 2024-09-18T11:30:24.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026826+0000", 2024-09-18T11:30:24.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026212400000000003 2024-09-18T11:30:24.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:30:24.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026839+0000", 2024-09-18T11:30:24.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2831e-05 2024-09-18T11:30:24.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028605+0000", 2024-09-18T11:30:24.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001765794 2024-09-18T11:30:24.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028709+0000", 2024-09-18T11:30:24.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104348 2024-09-18T11:30:24.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:24.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028824+0000", 2024-09-18T11:30:24.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011527399999999999 2024-09-18T11:30:24.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:24.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030319+0000", 2024-09-18T11:30:24.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001494439 2024-09-18T11:30:24.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:24.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033805+0000", 2024-09-18T11:30:24.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034857820000000002 2024-09-18T11:30:24.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:24.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033856+0000", 2024-09-18T11:30:24.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1687999999999997e-05 2024-09-18T11:30:24.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033894+0000", 2024-09-18T11:30:24.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7481000000000002e-05 2024-09-18T11:30:24.966 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.966 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.967 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.967 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T11:30:24.967 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:51.038968+0000", 2024-09-18T11:30:24.967 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 33.918461983999997, 2024-09-18T11:30:24.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028722909999999999, 2024-09-18T11:30:24.967 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.967 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.967 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:24.967 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:24.967 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:24.967 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 5 2024-09-18T11:30:24.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.968 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.038968+0000", 2024-09-18T11:30:24.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.038968+0000", 2024-09-18T11:30:24.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.038966+0000", 2024-09-18T11:30:24.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:24.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.038983+0000", 2024-09-18T11:30:24.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7099e-05 2024-09-18T11:30:24.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.039002+0000", 2024-09-18T11:30:24.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8938000000000001e-05 2024-09-18T11:30:24.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.039012+0000", 2024-09-18T11:30:24.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9529999999999994e-06 2024-09-18T11:30:24.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.039068+0000", 2024-09-18T11:30:24.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5893999999999997e-05 2024-09-18T11:30:24.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:24.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.039074+0000", 2024-09-18T11:30:24.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8510000000000001e-06 2024-09-18T11:30:24.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.040586+0000", 2024-09-18T11:30:24.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015119809999999999 2024-09-18T11:30:24.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.040705+0000", 2024-09-18T11:30:24.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000119054 2024-09-18T11:30:24.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:24.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.040812+0000", 2024-09-18T11:30:24.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010652700000000001 2024-09-18T11:30:24.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:24.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041405+0000", 2024-09-18T11:30:24.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059319099999999997 2024-09-18T11:30:24.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:24.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041787+0000", 2024-09-18T11:30:24.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00038150800000000002 2024-09-18T11:30:24.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:24.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041817+0000", 2024-09-18T11:30:24.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0091e-05 2024-09-18T11:30:24.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041840+0000", 2024-09-18T11:30:24.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3385000000000001e-05 2024-09-18T11:30:24.973 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.973 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.973 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.974 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T11:30:24.974 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:30:24.974 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.911617747000001, 2024-09-18T11:30:24.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055107869999999996, 2024-09-18T11:30:24.974 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.974 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.974 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:24.974 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:24.974 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:24.974 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 7 2024-09-18T11:30:24.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.974 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:30:24.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:30:24.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045810+0000", 2024-09-18T11:30:24.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:24.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045820+0000", 2024-09-18T11:30:24.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0215e-05 2024-09-18T11:30:24.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045838+0000", 2024-09-18T11:30:24.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7849000000000001e-05 2024-09-18T11:30:24.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045845+0000", 2024-09-18T11:30:24.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4229999999999999e-06 2024-09-18T11:30:24.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045898+0000", 2024-09-18T11:30:24.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2537999999999997e-05 2024-09-18T11:30:24.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:24.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045904+0000", 2024-09-18T11:30:24.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4019999999999999e-06 2024-09-18T11:30:24.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047663+0000", 2024-09-18T11:30:24.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017584079999999999 2024-09-18T11:30:24.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047736+0000", 2024-09-18T11:30:24.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3509000000000004e-05 2024-09-18T11:30:24.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:24.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047857+0000", 2024-09-18T11:30:24.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012037800000000001 2024-09-18T11:30:24.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:24.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.050868+0000", 2024-09-18T11:30:24.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030111420000000001 2024-09-18T11:30:24.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:24.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051239+0000", 2024-09-18T11:30:24.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037141000000000002 2024-09-18T11:30:24.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:24.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051288+0000", 2024-09-18T11:30:24.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9051000000000002e-05 2024-09-18T11:30:24.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051323+0000", 2024-09-18T11:30:24.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4545000000000002e-05 2024-09-18T11:30:24.980 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.980 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.980 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.981 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2 4.7 e28/22 4:e73ba68f:::smithi11939506-1:head v 28'2, mlcod=0'0)", 2024-09-18T11:30:24.981 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:08.105343+0000", 2024-09-18T11:30:24.981 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 16.852086971999999, 2024-09-18T11:30:24.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040860050000000002, 2024-09-18T11:30:24.981 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.981 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.981 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105343+0000", 2024-09-18T11:30:24.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105343+0000", 2024-09-18T11:30:24.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105342+0000", 2024-09-18T11:30:24.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:24.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105801+0000", 2024-09-18T11:30:24.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00045840099999999999 2024-09-18T11:30:24.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105813+0000", 2024-09-18T11:30:24.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2461e-05 2024-09-18T11:30:24.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105818+0000", 2024-09-18T11:30:24.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.989e-06 2024-09-18T11:30:24.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105856+0000", 2024-09-18T11:30:24.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7839999999999997e-05 2024-09-18T11:30:24.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105872+0000", 2024-09-18T11:30:24.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6582000000000001e-05 2024-09-18T11:30:24.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:24.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.109389+0000", 2024-09-18T11:30:24.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003516211 2024-09-18T11:30:24.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.109429+0000", 2024-09-18T11:30:24.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0261e-05 2024-09-18T11:30:24.985 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.985 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.985 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.985 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3 4.7 e28/22 4:e73ba68f:::smithi11939506-1:head v 28'3, mlcod=0'0)", 2024-09-18T11:30:24.985 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:08.106522+0000", 2024-09-18T11:30:24.985 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 16.850907956, 2024-09-18T11:30:24.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031398509999999999, 2024-09-18T11:30:24.985 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.985 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.986 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106522+0000", 2024-09-18T11:30:24.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106522+0000", 2024-09-18T11:30:24.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106521+0000", 2024-09-18T11:30:24.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:24.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106526+0000", 2024-09-18T11:30:24.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9799999999999998e-06 2024-09-18T11:30:24.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106533+0000", 2024-09-18T11:30:24.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1229999999999997e-06 2024-09-18T11:30:24.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106536+0000", 2024-09-18T11:30:24.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.055e-06 2024-09-18T11:30:24.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106577+0000", 2024-09-18T11:30:24.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0932e-05 2024-09-18T11:30:24.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106588+0000", 2024-09-18T11:30:24.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1486e-05 2024-09-18T11:30:24.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:24.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.109630+0000", 2024-09-18T11:30:24.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030422159999999999 2024-09-18T11:30:24.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.109662+0000", 2024-09-18T11:30:24.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1498999999999998e-05 2024-09-18T11:30:24.989 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.989 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.989 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.990 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:905 4.7 e31/22 4:e73ba68f:::smithi11939506-1:head v 31'32, mlcod=30'29)", 2024-09-18T11:30:24.990 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:30:24.990 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.815961536, 2024-09-18T11:30:24.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048351990000000001, 2024-09-18T11:30:24.990 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.990 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.990 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:30:24.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:30:24.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:30:24.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:24.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144912+0000", 2024-09-18T11:30:24.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003444 2024-09-18T11:30:24.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144921+0000", 2024-09-18T11:30:24.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1900000000000001e-06 2024-09-18T11:30:24.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144927+0000", 2024-09-18T11:30:24.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4940000000000003e-06 2024-09-18T11:30:24.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144952+0000", 2024-09-18T11:30:24.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5845000000000001e-05 2024-09-18T11:30:24.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144966+0000", 2024-09-18T11:30:24.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.396e-05 2024-09-18T11:30:24.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:24.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.146274+0000", 2024-09-18T11:30:24.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013073550000000001 2024-09-18T11:30:24.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.146303+0000", 2024-09-18T11:30:24.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9754e-05 2024-09-18T11:30:24.994 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.994 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.994 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.994 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:910 4.3 e31/22 4:cc46532a:::smithi11939506-33:head v 31'23, mlcod=30'20)", 2024-09-18T11:30:24.994 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.160095+0000", 2024-09-18T11:30:24.994 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.797334484, 2024-09-18T11:30:24.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002481731, 2024-09-18T11:30:24.995 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.995 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.995 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:24.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:24.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.160095+0000", 2024-09-18T11:30:24.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:24.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.160095+0000", 2024-09-18T11:30:24.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:24.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:24.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.160095+0000", 2024-09-18T11:30:24.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:24.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:24.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.160806+0000", 2024-09-18T11:30:24.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00071076799999999997 2024-09-18T11:30:24.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:24.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.160816+0000", 2024-09-18T11:30:24.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0295999999999999e-05 2024-09-18T11:30:24.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:24.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.160820+0000", 2024-09-18T11:30:24.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0849999999999999e-06 2024-09-18T11:30:24.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:24.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.160858+0000", 2024-09-18T11:30:24.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7261999999999997e-05 2024-09-18T11:30:24.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:24.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.160871+0000", 2024-09-18T11:30:24.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3665999999999999e-05 2024-09-18T11:30:24.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:24.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.162552+0000", 2024-09-18T11:30:24.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016811020000000001 2024-09-18T11:30:24.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:24.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.162577+0000", 2024-09-18T11:30:24.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4711000000000001e-05 2024-09-18T11:30:24.998 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.999 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:24.999 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:24.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:24.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:24.999 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:911 4.3 e31/22 4:cc46532a:::smithi11939506-33:head v 31'24, mlcod=30'21)", 2024-09-18T11:30:24.999 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.166801+0000", 2024-09-18T11:30:24.999 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.790629210000001, 2024-09-18T11:30:24.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048144349999999997, 2024-09-18T11:30:24.999 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:24.999 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:24.999 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.166801+0000", 2024-09-18T11:30:25.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.166801+0000", 2024-09-18T11:30:25.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.166799+0000", 2024-09-18T11:30:25.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:25.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167413+0000", 2024-09-18T11:30:25.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000613259 2024-09-18T11:30:25.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167431+0000", 2024-09-18T11:30:25.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7906000000000001e-05 2024-09-18T11:30:25.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167438+0000", 2024-09-18T11:30:25.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.883e-06 2024-09-18T11:30:25.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167468+0000", 2024-09-18T11:30:25.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0168e-05 2024-09-18T11:30:25.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167483+0000", 2024-09-18T11:30:25.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4997999999999999e-05 2024-09-18T11:30:25.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.171574+0000", 2024-09-18T11:30:25.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040909589999999999 2024-09-18T11:30:25.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.171615+0000", 2024-09-18T11:30:25.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1371000000000001e-05 2024-09-18T11:30:25.003 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.003 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.003 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.003 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:989 4.3 e35/22 4:ca2b2309:::smithi11939506-44:head v 35'30, mlcod=31'26)", 2024-09-18T11:30:25.004 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.154272+0000", 2024-09-18T11:30:25.004 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.8031580569999992, 2024-09-18T11:30:25.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031808090000000002, 2024-09-18T11:30:25.004 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.004 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.004 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.154272+0000", 2024-09-18T11:30:25.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.154272+0000", 2024-09-18T11:30:25.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.154272+0000", 2024-09-18T11:30:25.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:25.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.155085+0000", 2024-09-18T11:30:25.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000813921 2024-09-18T11:30:25.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.155097+0000", 2024-09-18T11:30:25.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1881e-05 2024-09-18T11:30:25.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.155103+0000", 2024-09-18T11:30:25.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6250000000000004e-06 2024-09-18T11:30:25.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.155161+0000", 2024-09-18T11:30:25.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8514999999999998e-05 2024-09-18T11:30:25.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.155185+0000", 2024-09-18T11:30:25.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3841000000000001e-05 2024-09-18T11:30:25.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.157411+0000", 2024-09-18T11:30:25.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022258719999999998 2024-09-18T11:30:25.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.157453+0000", 2024-09-18T11:30:25.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1387999999999997e-05 2024-09-18T11:30:25.008 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.008 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.008 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.008 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:990 4.3 e35/22 4:ca2b2309:::smithi11939506-44:head v 35'31, mlcod=34'28)", 2024-09-18T11:30:25.008 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.158794+0000", 2024-09-18T11:30:25.008 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.7986361619999993, 2024-09-18T11:30:25.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042163180000000001, 2024-09-18T11:30:25.008 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.009 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.009 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.158794+0000", 2024-09-18T11:30:25.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.158794+0000", 2024-09-18T11:30:25.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.158793+0000", 2024-09-18T11:30:25.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:25.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159307+0000", 2024-09-18T11:30:25.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00051453999999999996 2024-09-18T11:30:25.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159320+0000", 2024-09-18T11:30:25.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2639e-05 2024-09-18T11:30:25.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159327+0000", 2024-09-18T11:30:25.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3200000000000002e-06 2024-09-18T11:30:25.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159359+0000", 2024-09-18T11:30:25.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2147999999999998e-05 2024-09-18T11:30:25.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159380+0000", 2024-09-18T11:30:25.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0625999999999999e-05 2024-09-18T11:30:25.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.162980+0000", 2024-09-18T11:30:25.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003600125 2024-09-18T11:30:25.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.163010+0000", 2024-09-18T11:30:25.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0020999999999999e-05 2024-09-18T11:30:25.012 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.012 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.013 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.013 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1007 4.2 e35/22 4:425afe9c:::smithi11939506-29:head v 35'12, mlcod=30'9)", 2024-09-18T11:30:25.013 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.250686+0000", 2024-09-18T11:30:25.013 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.706743672, 2024-09-18T11:30:25.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002857474, 2024-09-18T11:30:25.013 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.013 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.013 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.250686+0000", 2024-09-18T11:30:25.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.250686+0000", 2024-09-18T11:30:25.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.250684+0000", 2024-09-18T11:30:25.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:25.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.251174+0000", 2024-09-18T11:30:25.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048958300000000003 2024-09-18T11:30:25.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.251192+0000", 2024-09-18T11:30:25.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8227999999999999e-05 2024-09-18T11:30:25.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.251199+0000", 2024-09-18T11:30:25.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7830000000000001e-06 2024-09-18T11:30:25.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.251243+0000", 2024-09-18T11:30:25.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4302999999999999e-05 2024-09-18T11:30:25.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.251261+0000", 2024-09-18T11:30:25.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7782e-05 2024-09-18T11:30:25.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.253514+0000", 2024-09-18T11:30:25.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022532479999999998 2024-09-18T11:30:25.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.253544+0000", 2024-09-18T11:30:25.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9631000000000001e-05 2024-09-18T11:30:25.017 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.017 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.017 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.017 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1118 3.2 e38/21 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head v 38'76, mlcod=38'74)", 2024-09-18T11:30:25.017 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:30:25.017 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.0094860260000003, 2024-09-18T11:30:25.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006048688, 2024-09-18T11:30:25.018 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.018 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.018 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:30:25.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:30:25.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947941+0000", 2024-09-18T11:30:25.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:25.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949868+0000", 2024-09-18T11:30:25.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019267659999999999 2024-09-18T11:30:25.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949893+0000", 2024-09-18T11:30:25.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5239999999999999e-05 2024-09-18T11:30:25.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949901+0000", 2024-09-18T11:30:25.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8420000000000005e-06 2024-09-18T11:30:25.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949954+0000", 2024-09-18T11:30:25.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3464999999999997e-05 2024-09-18T11:30:25.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949974+0000", 2024-09-18T11:30:25.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9731000000000001e-05 2024-09-18T11:30:25.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953948+0000", 2024-09-18T11:30:25.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039735489999999998 2024-09-18T11:30:25.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953992+0000", 2024-09-18T11:30:25.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4700999999999998e-05 2024-09-18T11:30:25.022 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.022 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.022 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.022 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1139 4.2 e39/22 4:425afe9c:::smithi11939506-29:head v 39'19, mlcod=35'15)", 2024-09-18T11:30:25.022 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.187224+0000", 2024-09-18T11:30:25.022 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.7702057490000005, 2024-09-18T11:30:25.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036968130000000002, 2024-09-18T11:30:25.022 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.022 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.022 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.187224+0000", 2024-09-18T11:30:25.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.187224+0000", 2024-09-18T11:30:25.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.187224+0000", 2024-09-18T11:30:25.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:25.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.188633+0000", 2024-09-18T11:30:25.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001409158 2024-09-18T11:30:25.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.188646+0000", 2024-09-18T11:30:25.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3458999999999999e-05 2024-09-18T11:30:25.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.188653+0000", 2024-09-18T11:30:25.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8569999999999997e-06 2024-09-18T11:30:25.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.188700+0000", 2024-09-18T11:30:25.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7176999999999999e-05 2024-09-18T11:30:25.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.188720+0000", 2024-09-18T11:30:25.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0043000000000001e-05 2024-09-18T11:30:25.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.190874+0000", 2024-09-18T11:30:25.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021537739999999998 2024-09-18T11:30:25.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.190921+0000", 2024-09-18T11:30:25.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6864999999999999e-05 2024-09-18T11:30:25.026 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.027 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.027 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.027 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1165 4.7 e39/22 4:e73ba68f:::smithi11939506-1:head v 39'40, mlcod=34'37)", 2024-09-18T11:30:25.027 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.657838+0000", 2024-09-18T11:30:25.027 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.2995913640000003, 2024-09-18T11:30:25.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042136450000000002, 2024-09-18T11:30:25.027 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.027 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.027 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.027 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.657838+0000", 2024-09-18T11:30:25.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.657838+0000", 2024-09-18T11:30:25.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.657837+0000", 2024-09-18T11:30:25.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:25.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659431+0000", 2024-09-18T11:30:25.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015942479999999999 2024-09-18T11:30:25.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659450+0000", 2024-09-18T11:30:25.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8909000000000001e-05 2024-09-18T11:30:25.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659457+0000", 2024-09-18T11:30:25.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8390000000000001e-06 2024-09-18T11:30:25.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659500+0000", 2024-09-18T11:30:25.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3101e-05 2024-09-18T11:30:25.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659518+0000", 2024-09-18T11:30:25.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8020999999999999e-05 2024-09-18T11:30:25.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.662024+0000", 2024-09-18T11:30:25.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025059399999999999 2024-09-18T11:30:25.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.662052+0000", 2024-09-18T11:30:25.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8206999999999999e-05 2024-09-18T11:30:25.031 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.031 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.031 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.031 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.13:29 3.2 e39/21 3:5fa8fcfa:::ec59520759824f0488c7a3ca135dacb324de2330e16b7f54053de5719c4e96b6:head v 39'81, mlcod=38'76)", 2024-09-18T11:30:25.031 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:30:25.032 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.2757855500000002, 2024-09-18T11:30:25.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004755858, 2024-09-18T11:30:25.032 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.032 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.032 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.032 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:30:25.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.032 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:30:25.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:30:25.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:25.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681961+0000", 2024-09-18T11:30:25.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00031696799999999999 2024-09-18T11:30:25.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681969+0000", 2024-09-18T11:30:25.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4800000000000001e-06 2024-09-18T11:30:25.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681973+0000", 2024-09-18T11:30:25.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4819999999999999e-06 2024-09-18T11:30:25.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682007+0000", 2024-09-18T11:30:25.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3961999999999998e-05 2024-09-18T11:30:25.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682033+0000", 2024-09-18T11:30:25.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6103999999999999e-05 2024-09-18T11:30:25.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686375+0000", 2024-09-18T11:30:25.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043422019999999999 2024-09-18T11:30:25.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.036 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686400+0000", 2024-09-18T11:30:25.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5265e-05 2024-09-18T11:30:25.036 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.036 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.036 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.036 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.13:30 3.2 e39/21 3:49074e56:::7fc21ffee1ba17f2a9197c61ff60ef0b03804c121c8aeb83ea424b7ac77081e9:head v 39'82, mlcod=39'77)", 2024-09-18T11:30:25.036 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.682532+0000", 2024-09-18T11:30:25.036 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.2748974879999997, 2024-09-18T11:30:25.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004072694, 2024-09-18T11:30:25.037 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.037 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.037 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682532+0000", 2024-09-18T11:30:25.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682532+0000", 2024-09-18T11:30:25.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682532+0000", 2024-09-18T11:30:25.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:25.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682681+0000", 2024-09-18T11:30:25.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014942500000000001 2024-09-18T11:30:25.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682691+0000", 2024-09-18T11:30:25.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0125e-05 2024-09-18T11:30:25.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682695+0000", 2024-09-18T11:30:25.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9339999999999999e-06 2024-09-18T11:30:25.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682724+0000", 2024-09-18T11:30:25.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8918e-05 2024-09-18T11:30:25.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682738+0000", 2024-09-18T11:30:25.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3793e-05 2024-09-18T11:30:25.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686590+0000", 2024-09-18T11:30:25.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038524380000000001 2024-09-18T11:30:25.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686605+0000", 2024-09-18T11:30:25.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4827999999999999e-05 2024-09-18T11:30:25.041 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.041 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.041 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.041 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1174 4.7 e39/22 4:e0d95e10:::smithi11939506-22:head v 39'45, mlcod=39'42)", 2024-09-18T11:30:25.041 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.035250+0000", 2024-09-18T11:30:25.041 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.9221799969999998, 2024-09-18T11:30:25.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028686929999999999, 2024-09-18T11:30:25.041 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.041 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.041 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035250+0000", 2024-09-18T11:30:25.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035250+0000", 2024-09-18T11:30:25.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035249+0000", 2024-09-18T11:30:25.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:25.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.043 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035852+0000", 2024-09-18T11:30:25.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060368100000000001 2024-09-18T11:30:25.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.043 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035866+0000", 2024-09-18T11:30:25.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3448999999999999e-05 2024-09-18T11:30:25.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.043 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.044 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035873+0000", 2024-09-18T11:30:25.044 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5739999999999998e-06 2024-09-18T11:30:25.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.044 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.044 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035918+0000", 2024-09-18T11:30:25.044 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4648999999999999e-05 2024-09-18T11:30:25.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.044 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.044 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035940+0000", 2024-09-18T11:30:25.044 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2166000000000001e-05 2024-09-18T11:30:25.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.045 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.045 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.038085+0000", 2024-09-18T11:30:25.045 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021446210000000002 2024-09-18T11:30:25.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.045 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.045 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.038119+0000", 2024-09-18T11:30:25.045 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3800000000000002e-05 2024-09-18T11:30:25.045 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.045 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.045 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.046 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1179 4.1 e39/22 4:86d80ae6:::smithi11939506-39:head v 39'43, mlcod=37'39)", 2024-09-18T11:30:25.046 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.045567+0000", 2024-09-18T11:30:25.046 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.9118623320000001, 2024-09-18T11:30:25.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025000840000000001, 2024-09-18T11:30:25.046 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.046 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.046 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.046 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.046 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.045567+0000", 2024-09-18T11:30:25.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.047 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.047 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.045567+0000", 2024-09-18T11:30:25.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.047 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.047 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.045567+0000", 2024-09-18T11:30:25.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:25.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.047 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.047 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.045574+0000", 2024-09-18T11:30:25.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6380000000000004e-06 2024-09-18T11:30:25.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.048 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.048 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.045603+0000", 2024-09-18T11:30:25.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9056000000000001e-05 2024-09-18T11:30:25.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.048 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.048 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.045627+0000", 2024-09-18T11:30:25.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3181e-05 2024-09-18T11:30:25.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.048 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.049 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.045669+0000", 2024-09-18T11:30:25.049 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2378e-05 2024-09-18T11:30:25.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.049 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.049 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.045694+0000", 2024-09-18T11:30:25.049 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4873000000000001e-05 2024-09-18T11:30:25.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.049 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.049 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.048044+0000", 2024-09-18T11:30:25.049 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023499879999999999 2024-09-18T11:30:25.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.050 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.050 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.048068+0000", 2024-09-18T11:30:25.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3764000000000002e-05 2024-09-18T11:30:25.050 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.050 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.050 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.050 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1190 4.2 e39/22 4:425afe9c:::smithi11939506-29:head v 39'23, mlcod=39'21)", 2024-09-18T11:30:25.050 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.177245+0000", 2024-09-18T11:30:25.050 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.7801844340000001, 2024-09-18T11:30:25.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027170720000000001, 2024-09-18T11:30:25.051 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.051 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.051 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.051 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.051 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.177245+0000", 2024-09-18T11:30:25.051 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.051 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.051 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.177245+0000", 2024-09-18T11:30:25.051 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.052 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.177244+0000", 2024-09-18T11:30:25.052 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:25.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.052 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.177883+0000", 2024-09-18T11:30:25.052 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00063948499999999997 2024-09-18T11:30:25.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.052 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.177901+0000", 2024-09-18T11:30:25.053 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7442999999999999e-05 2024-09-18T11:30:25.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.053 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.053 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.177907+0000", 2024-09-18T11:30:25.053 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.336e-06 2024-09-18T11:30:25.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.053 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.053 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.177952+0000", 2024-09-18T11:30:25.053 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4672000000000002e-05 2024-09-18T11:30:25.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.054 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.054 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.177969+0000", 2024-09-18T11:30:25.054 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6968000000000001e-05 2024-09-18T11:30:25.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.054 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.054 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.179922+0000", 2024-09-18T11:30:25.054 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019529630000000001 2024-09-18T11:30:25.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.054 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.054 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.179962+0000", 2024-09-18T11:30:25.054 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0921999999999999e-05 2024-09-18T11:30:25.055 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.055 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.055 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.055 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1193 4.2 e39/22 4:425afe9c:::smithi11939506-29:head v 39'26, mlcod=39'23)", 2024-09-18T11:30:25.055 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.189951+0000", 2024-09-18T11:30:25.055 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.767479271, 2024-09-18T11:30:25.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025525370000000001, 2024-09-18T11:30:25.055 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.055 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.055 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.056 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.189951+0000", 2024-09-18T11:30:25.056 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.056 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.056 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.189951+0000", 2024-09-18T11:30:25.056 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.056 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.056 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.189950+0000", 2024-09-18T11:30:25.056 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:25.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.057 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.057 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.189958+0000", 2024-09-18T11:30:25.057 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1790000000000008e-06 2024-09-18T11:30:25.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.057 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.057 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.189966+0000", 2024-09-18T11:30:25.057 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7109999999999995e-06 2024-09-18T11:30:25.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.057 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.057 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.189970+0000", 2024-09-18T11:30:25.058 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.048e-06 2024-09-18T11:30:25.058 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.058 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.058 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.190007+0000", 2024-09-18T11:30:25.058 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6955000000000001e-05 2024-09-18T11:30:25.058 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.058 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.058 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.190023+0000", 2024-09-18T11:30:25.058 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6566e-05 2024-09-18T11:30:25.058 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.058 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.059 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.192473+0000", 2024-09-18T11:30:25.059 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002449447 2024-09-18T11:30:25.059 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.059 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.059 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.059 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.192503+0000", 2024-09-18T11:30:25.059 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0549000000000002e-05 2024-09-18T11:30:25.059 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.059 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.059 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.059 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.059 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.059 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:25.060 INFO:tasks.rados.rados.0.smithi119.stdout:810: finishing set_chunk to oid 47 2024-09-18T11:30:25.060 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 47 set_chunk returned error code -95 offset: 209408 length: 881664 2024-09-18T11:30:25.061 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 0 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-18T11:30:25.061 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:30:25.062 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:25.059+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:25.161 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:25.161 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:25.161 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:25.161 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:25.162 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:25.160+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:25.169 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:30:25.195 INFO:tasks.rados.rados.0.smithi119.stdout:800: done (10 left) 2024-09-18T11:30:25.195 INFO:tasks.rados.rados.0.smithi119.stdout:801: done (9 left) 2024-09-18T11:30:25.196 INFO:tasks.rados.rados.0.smithi119.stdout:802: done (8 left) 2024-09-18T11:30:25.196 INFO:tasks.rados.rados.0.smithi119.stdout:803: done (7 left) 2024-09-18T11:30:25.196 INFO:tasks.rados.rados.0.smithi119.stdout:804: done (6 left) 2024-09-18T11:30:25.196 INFO:tasks.rados.rados.0.smithi119.stdout:805: done (5 left) 2024-09-18T11:30:25.196 INFO:tasks.rados.rados.0.smithi119.stdout:806: done (4 left) 2024-09-18T11:30:25.196 INFO:tasks.rados.rados.0.smithi119.stdout:807: done (3 left) 2024-09-18T11:30:25.196 INFO:tasks.rados.rados.0.smithi119.stdout:808: done (2 left) 2024-09-18T11:30:25.196 INFO:tasks.rados.rados.0.smithi119.stdout:810: done (1 left) 2024-09-18T11:30:25.196 INFO:tasks.rados.rados.0.smithi119.stdout:811: done (0 left) 2024-09-18T11:30:25.196 INFO:tasks.rados.rados.0.smithi119.stdout:812: write (excl) oid 34 current snap is 7 2024-09-18T11:30:25.196 INFO:tasks.rados.rados.0.smithi119.stdout:812: seq_num 108 ranges {615156=555834,1603467=416361,2816389=693495} 2024-09-18T11:30:25.201 INFO:tasks.rados.rados.0.smithi119.stdout:812: writing smithi11939506-34 from 615156 to 1170990 tid 1 2024-09-18T11:30:25.206 INFO:tasks.rados.rados.0.smithi119.stdout:812: writing smithi11939506-34 from 1603467 to 2019828 tid 2 2024-09-18T11:30:25.214 INFO:tasks.rados.rados.0.smithi119.stdout:812: writing smithi11939506-34 from 2816389 to 3509884 tid 3 2024-09-18T11:30:25.214 INFO:tasks.rados.rados.0.smithi119.stdout:813: snap_remove snap 5 2024-09-18T11:30:25.219 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:25.218+0000 7fa4696f1640 -1 osd.0 42 handle_osd_map injecting map crc failure 2024-09-18T11:30:25.248 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:25.247+0000 7fd27659f640 -1 osd.6 42 handle_osd_map injecting map crc failure 2024-09-18T11:30:25.262 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:25.261+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:25.271 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:25.271 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:25.271 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:25.271 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:25.271 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:25.280 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:30:25.363 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:25.362+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:25.379 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:25.380 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:25.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:25.380 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:25.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.380 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:25.380 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:25.380 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.26062417, 2024-09-18T11:30:25.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:30:25.381 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.381 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:25.381 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.381 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:25.381 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:25.382 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:30:25.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.382 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:25.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:25.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:30:25.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:25.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:30:25.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:30:25.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:30:25.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:30:25.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:30:25.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:30:25.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:30:25.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:30:25.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:25.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:30:25.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:30:25.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:30:25.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:30:25.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:30:25.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:30:25.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:30:25.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:30:25.391 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.391 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.391 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.391 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:896 4.5 4:aa575018:::smithi11939506-11:2 [read 0~2941192,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:25.392 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.132995+0000", 2024-09-18T11:30:25.392 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.243948529000001, 2024-09-18T11:30:25.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047874989999999997, 2024-09-18T11:30:25.392 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.392 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:25.392 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.392 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:25.392 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:25.392 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 896 2024-09-18T11:30:25.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.392 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.132995+0000", 2024-09-18T11:30:25.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.132995+0000", 2024-09-18T11:30:25.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.132993+0000", 2024-09-18T11:30:25.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:25.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133000+0000", 2024-09-18T11:30:25.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5509999999999996e-06 2024-09-18T11:30:25.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133010+0000", 2024-09-18T11:30:25.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0254e-05 2024-09-18T11:30:25.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133023+0000", 2024-09-18T11:30:25.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3458999999999999e-05 2024-09-18T11:30:25.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133069+0000", 2024-09-18T11:30:25.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5324000000000002e-05 2024-09-18T11:30:25.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133161+0000", 2024-09-18T11:30:25.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2773000000000001e-05 2024-09-18T11:30:25.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.137782+0000", 2024-09-18T11:30:25.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046208680000000002 2024-09-18T11:30:25.396 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.396 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.396 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.396 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:897 4.5 4:aa575018:::smithi11939506-11:2 [sparse-read 0~1343176] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:25.396 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.133034+0000", 2024-09-18T11:30:25.396 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.24390917, 2024-09-18T11:30:25.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051617950000000003, 2024-09-18T11:30:25.397 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.397 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:25.397 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.397 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:25.397 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:25.397 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 897 2024-09-18T11:30:25.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.397 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133034+0000", 2024-09-18T11:30:25.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133034+0000", 2024-09-18T11:30:25.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133033+0000", 2024-09-18T11:30:25.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:25.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133037+0000", 2024-09-18T11:30:25.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.118e-06 2024-09-18T11:30:25.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133045+0000", 2024-09-18T11:30:25.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1780000000000003e-06 2024-09-18T11:30:25.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133049+0000", 2024-09-18T11:30:25.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7730000000000001e-06 2024-09-18T11:30:25.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.137838+0000", 2024-09-18T11:30:25.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00478912 2024-09-18T11:30:25.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.137936+0000", 2024-09-18T11:30:25.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.823e-05 2024-09-18T11:30:25.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138196+0000", 2024-09-18T11:30:25.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00025993799999999998 2024-09-18T11:30:25.400 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.401 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.401 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.401 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:898 4.5 4:aa575018:::smithi11939506-11:2 [sparse-read 0~1343176] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:25.401 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:25.401 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.243880131999999, 2024-09-18T11:30:25.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055588349999999998, 2024-09-18T11:30:25.401 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.401 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:25.401 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.401 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:25.402 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:25.402 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 898 2024-09-18T11:30:25.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.402 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:25.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:25.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:25.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:25.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133066+0000", 2024-09-18T11:30:25.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2729999999999998e-06 2024-09-18T11:30:25.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133072+0000", 2024-09-18T11:30:25.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0680000000000002e-06 2024-09-18T11:30:25.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133077+0000", 2024-09-18T11:30:25.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3569999999999997e-06 2024-09-18T11:30:25.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138226+0000", 2024-09-18T11:30:25.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051480980000000003 2024-09-18T11:30:25.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138324+0000", 2024-09-18T11:30:25.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8639000000000002e-05 2024-09-18T11:30:25.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138622+0000", 2024-09-18T11:30:25.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029774100000000002 2024-09-18T11:30:25.407 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.407 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.407 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.407 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1034 4.a 4:588debc2:::smithi11939506-18:head [sparse-read 0~1476819,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:25.407 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.700666+0000", 2024-09-18T11:30:25.407 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.6762771619999999, 2024-09-18T11:30:25.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002576254, 2024-09-18T11:30:25.408 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.408 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:25.408 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.408 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:25.408 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:25.408 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1034 2024-09-18T11:30:25.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.408 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700666+0000", 2024-09-18T11:30:25.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700666+0000", 2024-09-18T11:30:25.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700665+0000", 2024-09-18T11:30:25.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:25.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700671+0000", 2024-09-18T11:30:25.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3839999999999996e-06 2024-09-18T11:30:25.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700683+0000", 2024-09-18T11:30:25.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1912999999999999e-05 2024-09-18T11:30:25.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700687+0000", 2024-09-18T11:30:25.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6059999999999996e-06 2024-09-18T11:30:25.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700720+0000", 2024-09-18T11:30:25.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2642999999999999e-05 2024-09-18T11:30:25.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700779+0000", 2024-09-18T11:30:25.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8992e-05 2024-09-18T11:30:25.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.703242+0000", 2024-09-18T11:30:25.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024635479999999999 2024-09-18T11:30:25.412 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.412 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.412 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.412 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1035 4.a 4:588debc2:::smithi11939506-18:head [read 0~3008420,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:25.412 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.700693+0000", 2024-09-18T11:30:25.412 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.676250542, 2024-09-18T11:30:25.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043653629999999997, 2024-09-18T11:30:25.412 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.413 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:25.413 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.413 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:25.413 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:25.413 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1035 2024-09-18T11:30:25.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.413 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700693+0000", 2024-09-18T11:30:25.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700693+0000", 2024-09-18T11:30:25.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700692+0000", 2024-09-18T11:30:25.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:25.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700695+0000", 2024-09-18T11:30:25.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3e-06 2024-09-18T11:30:25.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700698+0000", 2024-09-18T11:30:25.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6799999999999999e-06 2024-09-18T11:30:25.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700704+0000", 2024-09-18T11:30:25.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7010000000000001e-06 2024-09-18T11:30:25.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.703282+0000", 2024-09-18T11:30:25.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002577686 2024-09-18T11:30:25.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.703363+0000", 2024-09-18T11:30:25.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0962000000000002e-05 2024-09-18T11:30:25.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.705058+0000", 2024-09-18T11:30:25.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016953179999999999 2024-09-18T11:30:25.419 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.419 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.419 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.419 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1036 4.a 4:588debc2:::smithi11939506-18:head [sparse-read 0~1476819] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:25.419 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.700715+0000", 2024-09-18T11:30:25.419 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.6762281819999991, 2024-09-18T11:30:25.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0045971959999999996, 2024-09-18T11:30:25.419 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.419 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:25.419 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.419 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:25.420 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:25.420 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1036 2024-09-18T11:30:25.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.420 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700715+0000", 2024-09-18T11:30:25.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700715+0000", 2024-09-18T11:30:25.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700715+0000", 2024-09-18T11:30:25.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:25.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700717+0000", 2024-09-18T11:30:25.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9530000000000002e-06 2024-09-18T11:30:25.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700721+0000", 2024-09-18T11:30:25.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9729999999999999e-06 2024-09-18T11:30:25.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700723+0000", 2024-09-18T11:30:25.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.153e-06 2024-09-18T11:30:25.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.705107+0000", 2024-09-18T11:30:25.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043840459999999999 2024-09-18T11:30:25.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.705156+0000", 2024-09-18T11:30:25.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9378000000000001e-05 2024-09-18T11:30:25.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.705312+0000", 2024-09-18T11:30:25.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015603200000000001 2024-09-18T11:30:25.424 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.424 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.425 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.425 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1108 4.a 4:552523b6:::smithi11939506-38:head [read 0~2621859,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:25.425 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.851579+0000", 2024-09-18T11:30:25.425 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.5253643270000001, 2024-09-18T11:30:25.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0033177340000000001, 2024-09-18T11:30:25.425 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.425 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:25.425 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.425 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:25.426 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:25.426 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1108 2024-09-18T11:30:25.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.426 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851579+0000", 2024-09-18T11:30:25.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851579+0000", 2024-09-18T11:30:25.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851579+0000", 2024-09-18T11:30:25.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:25.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851581+0000", 2024-09-18T11:30:25.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.48e-06 2024-09-18T11:30:25.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851585+0000", 2024-09-18T11:30:25.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0860000000000004e-06 2024-09-18T11:30:25.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851589+0000", 2024-09-18T11:30:25.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.968e-06 2024-09-18T11:30:25.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.853620+0000", 2024-09-18T11:30:25.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020308750000000001 2024-09-18T11:30:25.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.853668+0000", 2024-09-18T11:30:25.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8544999999999997e-05 2024-09-18T11:30:25.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.854897+0000", 2024-09-18T11:30:25.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001228125 2024-09-18T11:30:25.430 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.430 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.430 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.430 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1109 4.a 4:552523b6:::smithi11939506-38:head [read 0~2621859,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:25.430 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.851616+0000", 2024-09-18T11:30:25.430 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.5253275369999999, 2024-09-18T11:30:25.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041528579999999997, 2024-09-18T11:30:25.431 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.431 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:25.431 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.431 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:25.431 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:25.431 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1109 2024-09-18T11:30:25.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.431 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851616+0000", 2024-09-18T11:30:25.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851616+0000", 2024-09-18T11:30:25.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851615+0000", 2024-09-18T11:30:25.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:25.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851617+0000", 2024-09-18T11:30:25.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0020000000000001e-06 2024-09-18T11:30:25.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851624+0000", 2024-09-18T11:30:25.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5710000000000003e-06 2024-09-18T11:30:25.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851631+0000", 2024-09-18T11:30:25.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0659999999999998e-06 2024-09-18T11:30:25.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.854937+0000", 2024-09-18T11:30:25.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003306284 2024-09-18T11:30:25.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.855040+0000", 2024-09-18T11:30:25.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010267099999999999 2024-09-18T11:30:25.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.855769+0000", 2024-09-18T11:30:25.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00072857200000000003 2024-09-18T11:30:25.435 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.435 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.435 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.435 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1113 4.5 4:a087a566:::smithi11939506-46:head [read 0~2756315,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:25.435 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.851696+0000", 2024-09-18T11:30:25.435 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.5252474129999998, 2024-09-18T11:30:25.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028874479999999999, 2024-09-18T11:30:25.436 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.436 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:25.436 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.436 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:25.436 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:25.436 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1113 2024-09-18T11:30:25.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.436 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851696+0000", 2024-09-18T11:30:25.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851696+0000", 2024-09-18T11:30:25.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851696+0000", 2024-09-18T11:30:25.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:25.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851698+0000", 2024-09-18T11:30:25.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.979e-06 2024-09-18T11:30:25.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851701+0000", 2024-09-18T11:30:25.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9319999999999998e-06 2024-09-18T11:30:25.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851706+0000", 2024-09-18T11:30:25.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3150000000000002e-06 2024-09-18T11:30:25.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.853523+0000", 2024-09-18T11:30:25.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018174350000000001 2024-09-18T11:30:25.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.853578+0000", 2024-09-18T11:30:25.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5072999999999999e-05 2024-09-18T11:30:25.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.854583+0000", 2024-09-18T11:30:25.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001004958 2024-09-18T11:30:25.440 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.440 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.440 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.440 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1130 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2605052,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:25.440 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.178984+0000", 2024-09-18T11:30:25.440 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.197959472, 2024-09-18T11:30:25.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041647380000000003, 2024-09-18T11:30:25.440 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.440 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:25.440 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.441 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:25.441 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:25.441 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1130 2024-09-18T11:30:25.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.441 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178984+0000", 2024-09-18T11:30:25.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178984+0000", 2024-09-18T11:30:25.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178981+0000", 2024-09-18T11:30:25.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:25.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178998+0000", 2024-09-18T11:30:25.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7493e-05 2024-09-18T11:30:25.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179032+0000", 2024-09-18T11:30:25.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3562000000000002e-05 2024-09-18T11:30:25.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179048+0000", 2024-09-18T11:30:25.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6552999999999999e-05 2024-09-18T11:30:25.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179175+0000", 2024-09-18T11:30:25.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012698399999999999 2024-09-18T11:30:25.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:25.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179193+0000", 2024-09-18T11:30:25.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7558999999999999e-05 2024-09-18T11:30:25.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.181969+0000", 2024-09-18T11:30:25.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027756479999999999 2024-09-18T11:30:25.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.182096+0000", 2024-09-18T11:30:25.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012708700000000001 2024-09-18T11:30:25.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.183148+0000", 2024-09-18T11:30:25.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0010528250000000001 2024-09-18T11:30:25.445 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.445 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.445 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.446 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1131 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2605052,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:25.446 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179072+0000", 2024-09-18T11:30:25.446 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.1978715109999998, 2024-09-18T11:30:25.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050097329999999997, 2024-09-18T11:30:25.446 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.446 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:25.446 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.446 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:25.446 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:25.446 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1131 2024-09-18T11:30:25.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.446 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179072+0000", 2024-09-18T11:30:25.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179072+0000", 2024-09-18T11:30:25.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179070+0000", 2024-09-18T11:30:25.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:25.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179084+0000", 2024-09-18T11:30:25.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3689999999999999e-05 2024-09-18T11:30:25.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179105+0000", 2024-09-18T11:30:25.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0605999999999999e-05 2024-09-18T11:30:25.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179115+0000", 2024-09-18T11:30:25.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0862999999999999e-05 2024-09-18T11:30:25.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179365+0000", 2024-09-18T11:30:25.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00024911399999999999 2024-09-18T11:30:25.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:25.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179382+0000", 2024-09-18T11:30:25.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7856999999999999e-05 2024-09-18T11:30:25.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.183178+0000", 2024-09-18T11:30:25.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037957780000000001 2024-09-18T11:30:25.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.183256+0000", 2024-09-18T11:30:25.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7476000000000002e-05 2024-09-18T11:30:25.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184081+0000", 2024-09-18T11:30:25.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00082578199999999995 2024-09-18T11:30:25.450 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.450 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.451 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.451 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1132 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2605052,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:25.451 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179179+0000", 2024-09-18T11:30:25.451 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.1977641659999998, 2024-09-18T11:30:25.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059816840000000001, 2024-09-18T11:30:25.451 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.451 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:25.451 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.451 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:25.451 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:25.451 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1132 2024-09-18T11:30:25.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.452 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179179+0000", 2024-09-18T11:30:25.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179179+0000", 2024-09-18T11:30:25.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179178+0000", 2024-09-18T11:30:25.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:25.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179193+0000", 2024-09-18T11:30:25.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5446000000000001e-05 2024-09-18T11:30:25.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179215+0000", 2024-09-18T11:30:25.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1253000000000001e-05 2024-09-18T11:30:25.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179227+0000", 2024-09-18T11:30:25.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2612999999999999e-05 2024-09-18T11:30:25.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179447+0000", 2024-09-18T11:30:25.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021972199999999999 2024-09-18T11:30:25.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:25.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179453+0000", 2024-09-18T11:30:25.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8830000000000003e-06 2024-09-18T11:30:25.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184113+0000", 2024-09-18T11:30:25.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046603180000000001 2024-09-18T11:30:25.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184229+0000", 2024-09-18T11:30:25.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000115516 2024-09-18T11:30:25.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185161+0000", 2024-09-18T11:30:25.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00093216000000000002 2024-09-18T11:30:25.458 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.458 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.458 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.459 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:25.459 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:30:25.459 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.7075987689999996, 2024-09-18T11:30:25.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:30:25.459 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.459 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:25.460 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.460 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:25.460 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:25.460 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:30:25.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.460 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:30:25.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:30:25.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:30:25.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:25.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:30:25.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:30:25.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:30:25.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:30:25.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:30:25.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:30:25.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:30:25.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:30:25.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:30:25.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:30:25.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:30:25.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:30:25.466 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.466 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.466 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.467 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1173 4.5 4:a087a566:::smithi11939506-46:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:25.467 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.027238+0000", 2024-09-18T11:30:25.467 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.3497051540000005, 2024-09-18T11:30:25.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038968649999999998, 2024-09-18T11:30:25.467 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.467 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.467 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.467 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:25.467 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:25.467 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1173 2024-09-18T11:30:25.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.467 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027238+0000", 2024-09-18T11:30:25.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027238+0000", 2024-09-18T11:30:25.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027237+0000", 2024-09-18T11:30:25.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:25.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027243+0000", 2024-09-18T11:30:25.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5439999999999998e-06 2024-09-18T11:30:25.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027254+0000", 2024-09-18T11:30:25.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0851e-05 2024-09-18T11:30:25.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027260+0000", 2024-09-18T11:30:25.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.212e-06 2024-09-18T11:30:25.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027289+0000", 2024-09-18T11:30:25.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9499e-05 2024-09-18T11:30:25.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027347+0000", 2024-09-18T11:30:25.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8244e-05 2024-09-18T11:30:25.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:25.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.028549+0000", 2024-09-18T11:30:25.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001201543 2024-09-18T11:30:25.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:25.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.029048+0000", 2024-09-18T11:30:25.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00049897600000000004 2024-09-18T11:30:25.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:25.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.031051+0000", 2024-09-18T11:30:25.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020031889999999998 2024-09-18T11:30:25.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.031096+0000", 2024-09-18T11:30:25.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4959000000000001e-05 2024-09-18T11:30:25.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.031135+0000", 2024-09-18T11:30:25.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8769000000000001e-05 2024-09-18T11:30:25.473 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.473 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.473 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.473 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1185 4.5 4:af4ba75a:::smithi11939506-37:head [write 729437~527542] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:25.473 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.156648+0000", 2024-09-18T11:30:25.473 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.2202948449999997, 2024-09-18T11:30:25.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057485540000000003, 2024-09-18T11:30:25.474 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.474 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.474 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.474 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:25.474 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:25.474 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1185 2024-09-18T11:30:25.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.474 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.156648+0000", 2024-09-18T11:30:25.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.156648+0000", 2024-09-18T11:30:25.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.156647+0000", 2024-09-18T11:30:25.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:25.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157166+0000", 2024-09-18T11:30:25.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00051975999999999997 2024-09-18T11:30:25.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157183+0000", 2024-09-18T11:30:25.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.611e-05 2024-09-18T11:30:25.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157188+0000", 2024-09-18T11:30:25.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.648e-06 2024-09-18T11:30:25.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157253+0000", 2024-09-18T11:30:25.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4503000000000002e-05 2024-09-18T11:30:25.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157322+0000", 2024-09-18T11:30:25.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9031999999999999e-05 2024-09-18T11:30:25.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:25.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157437+0000", 2024-09-18T11:30:25.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011543400000000001 2024-09-18T11:30:25.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:25.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162104+0000", 2024-09-18T11:30:25.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046668009999999999 2024-09-18T11:30:25.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:25.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162321+0000", 2024-09-18T11:30:25.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021706699999999999 2024-09-18T11:30:25.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162355+0000", 2024-09-18T11:30:25.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3670000000000001e-05 2024-09-18T11:30:25.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162397+0000", 2024-09-18T11:30:25.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2168e-05 2024-09-18T11:30:25.480 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.480 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.480 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.480 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1186 4.5 4:af4ba75a:::smithi11939506-37:head [write 1925626~643351] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:25.480 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.163566+0000", 2024-09-18T11:30:25.480 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.2133772589999996, 2024-09-18T11:30:25.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051394919999999998, 2024-09-18T11:30:25.480 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.480 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.480 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.481 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:25.481 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:25.481 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1186 2024-09-18T11:30:25.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.481 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.163566+0000", 2024-09-18T11:30:25.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.163566+0000", 2024-09-18T11:30:25.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.163565+0000", 2024-09-18T11:30:25.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:25.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164172+0000", 2024-09-18T11:30:25.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060731000000000001 2024-09-18T11:30:25.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164184+0000", 2024-09-18T11:30:25.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1202999999999999e-05 2024-09-18T11:30:25.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164188+0000", 2024-09-18T11:30:25.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2810000000000002e-06 2024-09-18T11:30:25.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164212+0000", 2024-09-18T11:30:25.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4227999999999999e-05 2024-09-18T11:30:25.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164275+0000", 2024-09-18T11:30:25.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3264999999999998e-05 2024-09-18T11:30:25.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:25.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164376+0000", 2024-09-18T11:30:25.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000100821 2024-09-18T11:30:25.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:25.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168331+0000", 2024-09-18T11:30:25.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039549779999999996 2024-09-18T11:30:25.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:25.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168621+0000", 2024-09-18T11:30:25.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029003900000000002 2024-09-18T11:30:25.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168662+0000", 2024-09-18T11:30:25.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1307999999999998e-05 2024-09-18T11:30:25.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168705+0000", 2024-09-18T11:30:25.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2994999999999997e-05 2024-09-18T11:30:25.486 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.486 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.486 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.486 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1188 4.5 4:af4ba75a:::smithi11939506-37:head [setxattr _header (56) in=7b,truncate 3529437] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:25.486 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.168793+0000", 2024-09-18T11:30:25.486 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.2081500250000001, 2024-09-18T11:30:25.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040182960000000002, 2024-09-18T11:30:25.487 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.487 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.487 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.487 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:25.487 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:25.487 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1188 2024-09-18T11:30:25.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.487 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168793+0000", 2024-09-18T11:30:25.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168793+0000", 2024-09-18T11:30:25.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168793+0000", 2024-09-18T11:30:25.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:25.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168797+0000", 2024-09-18T11:30:25.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5510000000000001e-06 2024-09-18T11:30:25.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168802+0000", 2024-09-18T11:30:25.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.023e-06 2024-09-18T11:30:25.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168806+0000", 2024-09-18T11:30:25.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4180000000000001e-06 2024-09-18T11:30:25.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.169485+0000", 2024-09-18T11:30:25.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00067961699999999998 2024-09-18T11:30:25.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.169580+0000", 2024-09-18T11:30:25.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4827000000000005e-05 2024-09-18T11:30:25.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:25.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.169713+0000", 2024-09-18T11:30:25.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000133285 2024-09-18T11:30:25.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:25.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.170114+0000", 2024-09-18T11:30:25.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000400381 2024-09-18T11:30:25.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:25.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.172752+0000", 2024-09-18T11:30:25.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002638793 2024-09-18T11:30:25.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.172783+0000", 2024-09-18T11:30:25.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0275999999999999e-05 2024-09-18T11:30:25.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.172811+0000", 2024-09-18T11:30:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8759999999999999e-05 2024-09-18T11:30:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1189 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.168812+0000", 2024-09-18T11:30:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.2081313739999997, 2024-09-18T11:30:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004129377, 2024-09-18T11:30:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1189 2024-09-18T11:30:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168812+0000", 2024-09-18T11:30:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168812+0000", 2024-09-18T11:30:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168812+0000", 2024-09-18T11:30:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168814+0000", 2024-09-18T11:30:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1399999999999998e-06 2024-09-18T11:30:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168820+0000", 2024-09-18T11:30:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6320000000000002e-06 2024-09-18T11:30:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168823+0000", 2024-09-18T11:30:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3599999999999999e-06 2024-09-18T11:30:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.169903+0000", 2024-09-18T11:30:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001080136 2024-09-18T11:30:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.169978+0000", 2024-09-18T11:30:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5298999999999999e-05 2024-09-18T11:30:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.172837+0000", 2024-09-18T11:30:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028588620000000002 2024-09-18T11:30:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.172884+0000", 2024-09-18T11:30:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6767999999999997e-05 2024-09-18T11:30:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.172941+0000", 2024-09-18T11:30:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7515000000000001e-05 2024-09-18T11:30:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1372 4.a 4:588debc2:::smithi11939506-18:head [read 0~3008420,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-18T11:30:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:24.316548+0000", 2024-09-18T11:30:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.0603955270000001, 2024-09-18T11:30:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023069980000000002, 2024-09-18T11:30:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1372 2024-09-18T11:30:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:24.316548+0000", 2024-09-18T11:30:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:24.316548+0000", 2024-09-18T11:30:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:24.316547+0000", 2024-09-18T11:30:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:24.316550+0000", 2024-09-18T11:30:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3579999999999999e-06 2024-09-18T11:30:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:24.316556+0000", 2024-09-18T11:30:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7810000000000002e-06 2024-09-18T11:30:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:24.316561+0000", 2024-09-18T11:30:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1769999999999997e-06 2024-09-18T11:30:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:24.318024+0000", 2024-09-18T11:30:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001463057 2024-09-18T11:30:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:24.318133+0000", 2024-09-18T11:30:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000108961 2024-09-18T11:30:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:24.318855+0000", 2024-09-18T11:30:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00072140699999999997 2024-09-18T11:30:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.503 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:25.505 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:30:25.505 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:25.504+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:25.606 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:25.605+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:25.644 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:25.644 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:25.644 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:25.644 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:25.652 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:30:25.708 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:25.705+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:25.750 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:25.750 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:25.750 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:25.750 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:25.750 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:25.758 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:30:25.842 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:25.841+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:25.885 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:30:25.898 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:25.898 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:25.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:25.898 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:25.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.899 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:41 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:25.899 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157303+0000", 2024-09-18T11:30:25.899 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.738533093999997, 2024-09-18T11:30:25.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054616040000000001, 2024-09-18T11:30:25.899 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.899 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.899 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.899 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:25.899 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:25.899 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 41 2024-09-18T11:30:25.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.900 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157303+0000", 2024-09-18T11:30:25.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157303+0000", 2024-09-18T11:30:25.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157303+0000", 2024-09-18T11:30:25.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:25.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157388+0000", 2024-09-18T11:30:25.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4938999999999994e-05 2024-09-18T11:30:25.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157395+0000", 2024-09-18T11:30:25.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1130000000000003e-06 2024-09-18T11:30:25.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157405+0000", 2024-09-18T11:30:25.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7510000000000003e-06 2024-09-18T11:30:25.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158213+0000", 2024-09-18T11:30:25.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00080873000000000002 2024-09-18T11:30:25.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158267+0000", 2024-09-18T11:30:25.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3709999999999999e-05 2024-09-18T11:30:25.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:25.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158352+0000", 2024-09-18T11:30:25.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4782000000000002e-05 2024-09-18T11:30:25.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:25.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160262+0000", 2024-09-18T11:30:25.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001910068 2024-09-18T11:30:25.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:25.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162672+0000", 2024-09-18T11:30:25.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024101159999999999 2024-09-18T11:30:25.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162724+0000", 2024-09-18T11:30:25.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1637e-05 2024-09-18T11:30:25.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162765+0000", 2024-09-18T11:30:25.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1282000000000002e-05 2024-09-18T11:30:25.905 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.905 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.905 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.905 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:25.905 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:25.905 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.738423380999997, 2024-09-18T11:30:25.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054864400000000004, 2024-09-18T11:30:25.905 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.906 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.906 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.906 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:25.906 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:25.906 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 42 2024-09-18T11:30:25.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.906 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:25.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:25.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:25.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:25.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157416+0000", 2024-09-18T11:30:25.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.721e-06 2024-09-18T11:30:25.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157423+0000", 2024-09-18T11:30:25.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4049999999999997e-06 2024-09-18T11:30:25.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157425+0000", 2024-09-18T11:30:25.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.481e-06 2024-09-18T11:30:25.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158649+0000", 2024-09-18T11:30:25.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001223453 2024-09-18T11:30:25.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158729+0000", 2024-09-18T11:30:25.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0205999999999997e-05 2024-09-18T11:30:25.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:25.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158875+0000", 2024-09-18T11:30:25.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014628800000000001 2024-09-18T11:30:25.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:25.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161518+0000", 2024-09-18T11:30:25.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026427949999999999 2024-09-18T11:30:25.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:25.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162821+0000", 2024-09-18T11:30:25.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013028099999999999 2024-09-18T11:30:25.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162864+0000", 2024-09-18T11:30:25.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2839000000000001e-05 2024-09-18T11:30:25.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162900+0000", 2024-09-18T11:30:25.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5948999999999998e-05 2024-09-18T11:30:25.911 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.911 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.911 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.911 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:25.911 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:25.911 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.738337016999999, 2024-09-18T11:30:25.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055400850000000001, 2024-09-18T11:30:25.912 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.912 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.912 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.912 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:25.912 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:25.912 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 43 2024-09-18T11:30:25.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.912 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:25.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:25.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:25.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:25.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157504+0000", 2024-09-18T11:30:25.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4519999999999996e-06 2024-09-18T11:30:25.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157534+0000", 2024-09-18T11:30:25.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.97e-05 2024-09-18T11:30:25.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157539+0000", 2024-09-18T11:30:25.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4360000000000001e-06 2024-09-18T11:30:25.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159428+0000", 2024-09-18T11:30:25.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018888220000000001 2024-09-18T11:30:25.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159479+0000", 2024-09-18T11:30:25.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0608000000000001e-05 2024-09-18T11:30:25.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:25.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159559+0000", 2024-09-18T11:30:25.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0667000000000005e-05 2024-09-18T11:30:25.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:25.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162779+0000", 2024-09-18T11:30:25.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032197459999999999 2024-09-18T11:30:25.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:25.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162950+0000", 2024-09-18T11:30:25.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017059599999999999 2024-09-18T11:30:25.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163001+0000", 2024-09-18T11:30:25.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0968999999999999e-05 2024-09-18T11:30:25.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163040+0000", 2024-09-18T11:30:25.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8843000000000001e-05 2024-09-18T11:30:25.917 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.917 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.918 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.918 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:25.918 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:25.918 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.738285863000002, 2024-09-18T11:30:25.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:30:25.918 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.918 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.918 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.918 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:25.918 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:25.918 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:30:25.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.919 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:25.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:25.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:30:25.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:25.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:30:25.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:30:25.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:30:25.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:30:25.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:30:25.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:30:25.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:30:25.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:30:25.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:30:25.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:30:25.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:25.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:30:25.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:30:25.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:25.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:30:25.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:30:25.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:25.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:30:25.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:30:25.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:30:25.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:30:25.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:30:25.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:30:25.924 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.924 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.924 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.924 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:25.924 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:25.924 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.738159601, 2024-09-18T11:30:25.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055717040000000002, 2024-09-18T11:30:25.924 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.924 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.925 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.925 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:25.925 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:25.925 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 45 2024-09-18T11:30:25.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.925 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:25.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:25.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157676+0000", 2024-09-18T11:30:25.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:25.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157694+0000", 2024-09-18T11:30:25.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7422999999999999e-05 2024-09-18T11:30:25.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157704+0000", 2024-09-18T11:30:25.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9000000000000001e-06 2024-09-18T11:30:25.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157718+0000", 2024-09-18T11:30:25.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4788e-05 2024-09-18T11:30:25.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160937+0000", 2024-09-18T11:30:25.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00321886 2024-09-18T11:30:25.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161023+0000", 2024-09-18T11:30:25.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6063999999999995e-05 2024-09-18T11:30:25.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:25.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161202+0000", 2024-09-18T11:30:25.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017912599999999999 2024-09-18T11:30:25.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:25.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163052+0000", 2024-09-18T11:30:25.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001849554 2024-09-18T11:30:25.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:25.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163204+0000", 2024-09-18T11:30:25.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015186900000000001 2024-09-18T11:30:25.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163229+0000", 2024-09-18T11:30:25.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:30:25.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163249+0000", 2024-09-18T11:30:25.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9992000000000001e-05 2024-09-18T11:30:25.930 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.930 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.930 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.931 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:25.931 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:25.931 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.738110487999997, 2024-09-18T11:30:25.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:30:25.931 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.931 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.931 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.931 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:25.931 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:25.931 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:30:25.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.931 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:25.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:25.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:25.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:25.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:30:25.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:30:25.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:30:25.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:30:25.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:30:25.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:30:25.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:30:25.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:30:25.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:30:25.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:30:25.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:25.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:30:25.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:30:25.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:25.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:30:25.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:30:25.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:25.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:30:25.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:30:25.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:30:25.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:30:25.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:30:25.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:30:25.936 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.936 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.937 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.937 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:25.937 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:25.937 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.738067956999998, 2024-09-18T11:30:25.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:30:25.937 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.937 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.937 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.938 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:25.938 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:25.938 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:30:25.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.938 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:25.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:25.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:30:25.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:25.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:30:25.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:30:25.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:30:25.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:30:25.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:30:25.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:30:25.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:30:25.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:30:25.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:30:25.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:30:25.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:25.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:30:25.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:30:25.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:25.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:30:25.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:30:25.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:25.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:30:25.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:30:25.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:30:25.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:30:25.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:30:25.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:30:25.943 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.943 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.943 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.943 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:25.943 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:25.943 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.738013027000001, 2024-09-18T11:30:25.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:30:25.943 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.944 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.944 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.944 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:25.944 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:25.944 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:30:25.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.944 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:25.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:25.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:25.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:25.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:30:25.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:30:25.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:30:25.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:30:25.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:30:25.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:30:25.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:30:25.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:30:25.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:30:25.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:30:25.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:25.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:30:25.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:30:25.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:25.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:30:25.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:30:25.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:25.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:30:25.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:30:25.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:30:25.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:30:25.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:30:25.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:30:25.949 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.949 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.949 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.949 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:25.949 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:25.949 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.738003046999999, 2024-09-18T11:30:25.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:30:25.949 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.949 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.950 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.950 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:25.950 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:25.950 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:30:25.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.950 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:25.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:25.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:25.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:25.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:30:25.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:30:25.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:30:25.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:30:25.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:30:25.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:30:25.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:30:25.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:30:25.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:30:25.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:30:25.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:25.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:30:25.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:30:25.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:25.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:30:25.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:30:25.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:25.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:30:25.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:30:25.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:30:25.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:30:25.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:30:25.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:30:25.955 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.955 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.955 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.955 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:25.956 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:25.956 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.737991448000002, 2024-09-18T11:30:25.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:30:25.956 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.956 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.956 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.956 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:25.956 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:25.956 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:30:25.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.956 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:25.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:25.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:25.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:25.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:30:25.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:30:25.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:30:25.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:30:25.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:30:25.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:30:25.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:30:25.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:30:25.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:30:25.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:30:25.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:25.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:30:25.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:30:25.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:25.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:30:25.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:30:25.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:25.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:30:25.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:30:25.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:30:25.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:30:25.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:30:25.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:30:25.961 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.961 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.961 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.961 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:25.961 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:25.962 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.737934967000001, 2024-09-18T11:30:25.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:30:25.962 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.962 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.962 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.962 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:25.962 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:25.962 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:30:25.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.962 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:25.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:25.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:30:25.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:25.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:30:25.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:30:25.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:30:25.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:30:25.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:30:25.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:30:25.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:30:25.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:30:25.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:30:25.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:30:25.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:25.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:30:25.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:30:25.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:25.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:30:25.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:30:25.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:25.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:30:25.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:30:25.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:30:25.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:30:25.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:30:25.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:30:25.967 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.967 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.967 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.967 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:25.967 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:25.968 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.737921143000001, 2024-09-18T11:30:25.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:30:25.968 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.968 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.968 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.968 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:25.968 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:25.968 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:30:25.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.968 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:25.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:25.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:25.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:25.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:30:25.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:30:25.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:30:25.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:30:25.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:30:25.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:30:25.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:30:25.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:30:25.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:30:25.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:30:25.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:25.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:30:25.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:30:25.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:25.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:30:25.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:30:25.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:25.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:30:25.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:30:25.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:30:25.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:30:25.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:30:25.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:30:25.973 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.973 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.973 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.973 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:25.974 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:25.974 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.321742235999999, 2024-09-18T11:30:25.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:30:25.974 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.974 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.974 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.974 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:25.974 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:25.974 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:30:25.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.974 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:25.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:25.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:25.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:25.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:30:25.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:30:25.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:30:25.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:30:25.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:30:25.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:30:25.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:30:25.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:30:25.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:30:25.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:30:25.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:25.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:30:25.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:30:25.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:25.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:30:25.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:30:25.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:25.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:30:25.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:30:25.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:30:25.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:30:25.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:30:25.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:30:25.986 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.986 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.986 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.986 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:25.986 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:25.986 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.321718936000003, 2024-09-18T11:30:25.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:30:25.986 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.986 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.986 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.986 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:25.986 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:25.987 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:30:25.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.987 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:25.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:25.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:30:25.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:25.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:30:25.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:30:25.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:30:25.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:30:25.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:30:25.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:30:25.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:30:25.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:30:25.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:30:25.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:30:25.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:25.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:30:25.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:30:25.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:25.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:30:25.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:30:25.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:25.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:30:25.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:30:25.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:30:25.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:30:25.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:30:25.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:30:25.991 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.992 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.992 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.992 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:25.992 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:25.992 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.321633327000001, 2024-09-18T11:30:25.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:30:25.992 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.992 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.992 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.993 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:25.993 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:25.993 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:30:25.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.993 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:25.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:25.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:25.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:25.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:25.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:25.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:25.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:30:25.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:30:25.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:25.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:30:25.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:30:25.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:25.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:30:25.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:30:25.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:25.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:30:25.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:30:25.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:25.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:30:25.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:30:25.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:25.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:30:25.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:30:25.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:25.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:30:25.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:30:25.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:25.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:30:25.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:30:25.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:25.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:30:25.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:30:25.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:25.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:30:25.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:30:25.998 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.998 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:25.998 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:25.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.998 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:25.998 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:25.998 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.321622361999999, 2024-09-18T11:30:25.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:30:25.998 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:25.999 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:25.999 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:25.999 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:25.999 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:25.999 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:30:25.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.999 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:25.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:25.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:25.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:25.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:25.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:25.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:26.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:26.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:26.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:26.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:26.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:26.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:26.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:30:26.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:30:26.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:26.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:30:26.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:30:26.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:26.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:30:26.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:30:26.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:26.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:30:26.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:30:26.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:26.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:30:26.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:30:26.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:26.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:30:26.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:30:26.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:26.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:30:26.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:30:26.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:26.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:30:26.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:30:26.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:26.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:30:26.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:30:26.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:26.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:30:26.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:30:26.004 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:26.004 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:26.004 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:26.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.004 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:26.004 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:26.004 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.321611193999999, 2024-09-18T11:30:26.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:30:26.004 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:26.004 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:26.005 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:26.005 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:26.005 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:26.005 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:30:26.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.005 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:26.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:26.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:26.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:26.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:26.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:26.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:26.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:26.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:26.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:26.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:26.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:30:26.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:30:26.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:26.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:30:26.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:30:26.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:26.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:30:26.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:30:26.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:26.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:30:26.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:30:26.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:26.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:30:26.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:30:26.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:26.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:30:26.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:30:26.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:26.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:30:26.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:30:26.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:26.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:30:26.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:30:26.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:26.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:30:26.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:30:26.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:26.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:30:26.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:30:26.010 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:26.010 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:26.010 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:26.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.010 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:26.010 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:26.010 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.321545864999997, 2024-09-18T11:30:26.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:30:26.010 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:26.011 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:26.011 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:26.011 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:26.011 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:26.011 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:30:26.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.011 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:26.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:26.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:26.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:26.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:26.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:26.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:26.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:26.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:30:26.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:26.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:26.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:30:26.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:30:26.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:26.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:30:26.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:30:26.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:26.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:30:26.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:30:26.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:26.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:30:26.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:30:26.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:26.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:30:26.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:30:26.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:26.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:30:26.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:30:26.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:26.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:30:26.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:30:26.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:26.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:30:26.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:30:26.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:26.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:30:26.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:30:26.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:26.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:30:26.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:30:26.016 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:26.016 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:26.016 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:26.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.016 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:26.016 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:26.016 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.311383886999998, 2024-09-18T11:30:26.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:30:26.016 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:26.016 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:26.017 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:26.017 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:26.017 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:26.017 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:30:26.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.017 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:26.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:26.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:26.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:26.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:26.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:26.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:26.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:26.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:30:26.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:26.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:26.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:30:26.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:30:26.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:26.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:30:26.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:30:26.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:26.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:30:26.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:30:26.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:26.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:30:26.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:30:26.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:26.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:30:26.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:30:26.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:26.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:30:26.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:30:26.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:26.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:30:26.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:30:26.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:26.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:30:26.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:30:26.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:26.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:30:26.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:30:26.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:26.022 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:30:26.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:30:26.022 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:26.022 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:26.022 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:26.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.022 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:26.022 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:26.022 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.311291408000002, 2024-09-18T11:30:26.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:30:26.022 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:26.022 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:26.022 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:26.023 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:26.023 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:26.023 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:30:26.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.023 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:26.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:26.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:26.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:26.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:26.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:26.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:26.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:26.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:26.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:26.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:26.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:30:26.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:30:26.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:26.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:30:26.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:30:26.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:26.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:30:26.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:30:26.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:26.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:30:26.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:30:26.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:26.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:30:26.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:30:26.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:26.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:30:26.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:30:26.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:26.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:30:26.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:30:26.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.027 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:26.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:30:26.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:30:26.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.027 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:26.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:30:26.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:30:26.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:26.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:26.027 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:26.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:30:26.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:30:26.028 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:26.028 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:26.028 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:26.028 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:26.028 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:26.028 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:26.029 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:30:26.029 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:30:26.030 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:26.030 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:26.030 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:26.028+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:26.031 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:30:26.042 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:26.042 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:26.057 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:26.058 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:26.073 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:26.073 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:26.089 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:26.089 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:26.104 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:26.105 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:26.120 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:26.120 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:26.131 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:26.129+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:26.132 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:26.132 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:26.141 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:30:26.201 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T11:30:26.202 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:26.200+0000 7fb6930fb640 -1 osd.2 44 handle_osd_map injecting map crc failure 2024-09-18T11:30:26.203 INFO:tasks.rados.rados.0.smithi119.stdout:813: done (1 left) 2024-09-18T11:30:26.203 INFO:tasks.rados.rados.0.smithi119.stdout:814: snap_create 2024-09-18T11:30:26.203 INFO:tasks.rados.rados.0.smithi119.stdout:812: finishing write tid 1 to smithi11939506-34 2024-09-18T11:30:26.203 INFO:tasks.rados.rados.0.smithi119.stdout:812: finishing write tid 2 to smithi11939506-34 2024-09-18T11:30:26.203 INFO:tasks.rados.rados.0.smithi119.stdout:812: finishing write tid 3 to smithi11939506-34 2024-09-18T11:30:26.203 INFO:tasks.rados.rados.0.smithi119.stdout:812: finishing write tid 4 to smithi11939506-34 2024-09-18T11:30:26.203 INFO:tasks.rados.rados.0.smithi119.stdout:812: finishing write tid 5 to smithi11939506-34 2024-09-18T11:30:26.203 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 52 (ObjNum 108 snap 7 seq_num 108) dirty exists 2024-09-18T11:30:26.203 INFO:tasks.rados.rados.0.smithi119.stdout:812: left oid 34 (ObjNum 108 snap 7 seq_num 108) 2024-09-18T11:30:26.249 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:26.248+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:26.287 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:26.287 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:26.296 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:30:26.332 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:26.331+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:26.434 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:26.432+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:26.435 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:26.435 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:26.443 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:30:26.534 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:26.533+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:26.541 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:26.541 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:26.549 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:30:26.635 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:26.634+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:26.647 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:26.647 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:26.655 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:30:26.741 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:26.740+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:26.793 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:26.793 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:26.801 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:30:26.842 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:26.841+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:26.939 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:26.938+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:26.940 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:26.940 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:26.949 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:30:27.039 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:27.038+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:27.046 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:27.046 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:27.055 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:30:27.140 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:27.140+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:27.152 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:27.152 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:27.160 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:30:27.204 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is set 2024-09-18T11:30:27.204 INFO:tasks.rados.rados.0.smithi119.stdout:812: done (1 left) 2024-09-18T11:30:27.204 INFO:tasks.rados.rados.0.smithi119.stdout:814: done (0 left) 2024-09-18T11:30:27.205 INFO:tasks.rados.rados.0.smithi119.stdout:815: set_chunk oid 45 target oid 2024-09-18T11:30:27.212 INFO:tasks.rados.rados.0.smithi119.stdout:815: set_chunk oid 45 offset: 330240 length: 620032 target oid 7e4698999a9b5393b640dd0295d8e455adc5ee01f6c80a50e9d292bf9c2dcab1 offset: 0 snap -1 2024-09-18T11:30:27.213 INFO:tasks.rados.rados.0.smithi119.stdout:816: set_chunk oid 23 target oid 2024-09-18T11:30:27.219 INFO:tasks.rados.rados.0.smithi119.stdout:816: set_chunk oid 23 offset: 2604544 length: 160256 target oid 7856b99fb91b782815b48032ec13ddc646eb0629f5a0a67daeab963f22d15680 offset: 0 snap -1 2024-09-18T11:30:27.219 INFO:tasks.rados.rados.0.smithi119.stdout:817: write (excl) oid 24 current snap is 8 2024-09-18T11:30:27.219 INFO:tasks.rados.rados.0.smithi119.stdout:815: finishing set_chunk to oid 45 2024-09-18T11:30:27.219 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 8 (ObjNum 65 snap 0 seq_num 65) dirty exists 2024-09-18T11:30:27.219 INFO:tasks.rados.rados.0.smithi119.stdout:817: seq_num 109 ranges {631963=631083,2061949=560019,3243812=504767} 2024-09-18T11:30:27.225 INFO:tasks.rados.rados.0.smithi119.stdout:817: writing smithi11939506-24 from 631963 to 1263046 tid 1 2024-09-18T11:30:27.233 INFO:tasks.rados.rados.0.smithi119.stdout:817: writing smithi11939506-24 from 2061949 to 2621968 tid 2 2024-09-18T11:30:27.239 INFO:tasks.rados.rados.0.smithi119.stdout:817: writing smithi11939506-24 from 3243812 to 3748579 tid 3 2024-09-18T11:30:27.239 INFO:tasks.rados.rados.0.smithi119.stdout:815: done (2 left) 2024-09-18T11:30:27.239 INFO:tasks.rados.rados.0.smithi119.stdout:818: read oid 6 snap -1 2024-09-18T11:30:27.239 INFO:tasks.rados.rados.0.smithi119.stdout:818: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-18T11:30:27.240 INFO:tasks.rados.rados.0.smithi119.stdout:819: set_chunk oid 20 target oid 2024-09-18T11:30:27.241 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:27.240+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:27.256 INFO:tasks.rados.rados.0.smithi119.stdout:819: set_chunk oid 20 offset: 1449472 length: 1624064 target oid be0721f3a30df04624eb43dc58fbbc00d7ea210014325fb73bbc8d75e46bcb7d offset: 0 snap -1 2024-09-18T11:30:27.256 INFO:tasks.rados.rados.0.smithi119.stdout:820: read oid 22 snap 7 2024-09-18T11:30:27.256 INFO:tasks.rados.rados.0.smithi119.stdout:820: expect (ObjNum 101 snap 6 seq_num 101) 2024-09-18T11:30:27.256 INFO:tasks.rados.rados.0.smithi119.stdout:821: read oid 26 snap -1 2024-09-18T11:30:27.256 INFO:tasks.rados.rados.0.smithi119.stdout:821: expect (ObjNum 98 snap 6 seq_num 98) 2024-09-18T11:30:27.256 INFO:tasks.rados.rados.0.smithi119.stdout:822: read oid 37 snap -1 2024-09-18T11:30:27.256 INFO:tasks.rados.rados.0.smithi119.stdout:822: expect (ObjNum 91 snap 5 seq_num 91) 2024-09-18T11:30:27.256 INFO:tasks.rados.rados.0.smithi119.stdout:823: write oid 4 current snap is 8 2024-09-18T11:30:27.256 INFO:tasks.rados.rados.0.smithi119.stdout:816: finishing set_chunk to oid 23 2024-09-18T11:30:27.256 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 48 (ObjNum 90 snap 5 seq_num 90) dirty exists 2024-09-18T11:30:27.257 INFO:tasks.rados.rados.0.smithi119.stdout:823: seq_num 110 ranges {648770=706332,2120431=420030,3271235=432392} 2024-09-18T11:30:27.263 INFO:tasks.rados.rados.0.smithi119.stdout:823: writing smithi11939506-4 from 648770 to 1355102 tid 1 2024-09-18T11:30:27.270 INFO:tasks.rados.rados.0.smithi119.stdout:823: writing smithi11939506-4 from 2120431 to 2540461 tid 2 2024-09-18T11:30:27.271 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:27.271 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:27.276 INFO:tasks.rados.rados.0.smithi119.stdout:823: writing smithi11939506-4 from 3271235 to 3703627 tid 3 2024-09-18T11:30:27.277 INFO:tasks.rados.rados.0.smithi119.stdout:816: done (7 left) 2024-09-18T11:30:27.277 INFO:tasks.rados.rados.0.smithi119.stdout:824: snap_remove snap 4 2024-09-18T11:30:27.282 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:30:27.366 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:27.365+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:27.382 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T11:30:27.383 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:30:27.421 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:27.422 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:27.430 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:30:27.472 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:27.471+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:27.544 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:27.543+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:27.571 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:27.571 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:27.580 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:30:27.645 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:27.643+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:27.650 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:30:27.651 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:30:27.662 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-09-18T11:30:27.202341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":258808774,"num_objects":150,"num_object_clones":17,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":2,"num_read":1050,"num_read_kb":684393,"num_write":865,"num_write_kb":268581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":45,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":895,"ondisk_log_size":895,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2662392,"kb_used_data":280912,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752312328,"statfs":{"total":773094113280,"available":770367823872,"internally_reserved":0,"allocated":287653888,"data_stored":296922795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.208242"},"pg_stats":[{"pgid":"4.f","version":"39'38","reported_seq":110,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.928976+0000","last_change":"2024-09-18T11:30:19.178889+0000","last_active":"2024-09-18T11:30:22.928976+0000","last_peered":"2024-09-18T11:30:22.928976+0000","last_clean":"2024-09-18T11:30:22.928976+0000","last_became_active":"2024-09-18T11:30:02.074391+0000","last_became_peered":"2024-09-18T11:30:02.074391+0000","last_unstale":"2024-09-18T11:30:22.928976+0000","last_undegraded":"2024-09-18T11:30:22.928976+0000","last_fullsized":"2024-09-18T11:30:22.928976+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:11.123483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006049072,"stat_sum":{"num_bytes":11159716,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":64,"num_read_kb":71156,"num_write":38,"num_write_kb":10090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"4.e","version":"29'21","reported_seq":83,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.527195+0000","last_change":"2024-09-18T11:30:19.167944+0000","last_active":"2024-09-18T11:30:22.527195+0000","last_peered":"2024-09-18T11:30:22.527195+0000","last_clean":"2024-09-18T11:30:22.527195+0000","last_became_active":"2024-09-18T11:30:02.073185+0000","last_became_peered":"2024-09-18T11:30:02.073185+0000","last_unstale":"2024-09-18T11:30:22.527195+0000","last_undegraded":"2024-09-18T11:30:22.527195+0000","last_fullsized":"2024-09-18T11:30:22.527195+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:12.655066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047952999999999998,"stat_sum":{"num_bytes":7697507,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":38326,"num_write":17,"num_write_kb":4267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"4.d","version":"41'45","reported_seq":121,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.927655+0000","last_change":"2024-09-18T11:30:19.171822+0000","last_active":"2024-09-18T11:30:22.927655+0000","last_peered":"2024-09-18T11:30:22.927655+0000","last_clean":"2024-09-18T11:30:22.927655+0000","last_became_active":"2024-09-18T11:30:02.334498+0000","last_became_peered":"2024-09-18T11:30:02.334498+0000","last_unstale":"2024-09-18T11:30:22.927655+0000","last_undegraded":"2024-09-18T11:30:22.927655+0000","last_fullsized":"2024-09-18T11:30:22.927655+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:10.245504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041312459999999999,"stat_sum":{"num_bytes":18516982,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":77,"num_read_kb":66091,"num_write":40,"num_write_kb":12223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"4.c","version":"34'18","reported_seq":71,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.179685+0000","last_change":"2024-09-18T11:30:19.167768+0000","last_active":"2024-09-18T11:30:21.179685+0000","last_peered":"2024-09-18T11:30:21.179685+0000","last_clean":"2024-09-18T11:30:21.179685+0000","last_became_active":"2024-09-18T11:30:02.073883+0000","last_became_peered":"2024-09-18T11:30:02.073883+0000","last_unstale":"2024-09-18T11:30:21.179685+0000","last_undegraded":"2024-09-18T11:30:21.179685+0000","last_fullsized":"2024-09-18T11:30:21.179685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:20.359490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042985799999999998,"stat_sum":{"num_bytes":5226911,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":19494,"num_write":12,"num_write_kb":3340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"4.b","version":"39'53","reported_seq":131,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.223523+0000","last_change":"2024-09-18T11:30:19.188724+0000","last_active":"2024-09-18T11:30:22.223523+0000","last_peered":"2024-09-18T11:30:22.223523+0000","last_clean":"2024-09-18T11:30:22.223523+0000","last_became_active":"2024-09-18T11:30:02.075449+0000","last_became_peered":"2024-09-18T11:30:02.075449+0000","last_unstale":"2024-09-18T11:30:22.223523+0000","last_undegraded":"2024-09-18T11:30:22.223523+0000","last_fullsized":"2024-09-18T11:30:22.223523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:16.299317+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.021340656999999999,"stat_sum":{"num_bytes":14200880,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":76964,"num_write":40,"num_write_kb":10935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":5,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"4.a","version":"41'26","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.265944+0000","last_change":"2024-09-18T11:30:19.171211+0000","last_active":"2024-09-18T11:30:22.265944+0000","last_peered":"2024-09-18T11:30:22.265944+0000","last_clean":"2024-09-18T11:30:22.265944+0000","last_became_active":"2024-09-18T11:30:02.075538+0000","last_became_peered":"2024-09-18T11:30:02.075538+0000","last_unstale":"2024-09-18T11:30:22.265944+0000","last_undegraded":"2024-09-18T11:30:22.265944+0000","last_fullsized":"2024-09-18T11:30:22.265944+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:26.428990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038727599999999998,"stat_sum":{"num_bytes":9257698,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":62,"num_read_kb":63574,"num_write":21,"num_write_kb":7153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"4.9","version":"41'9","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.221112+0000","last_change":"2024-09-18T11:30:19.167762+0000","last_active":"2024-09-18T11:30:22.221112+0000","last_peered":"2024-09-18T11:30:22.221112+0000","last_clean":"2024-09-18T11:30:22.221112+0000","last_became_active":"2024-09-18T11:30:02.074414+0000","last_became_peered":"2024-09-18T11:30:02.074414+0000","last_unstale":"2024-09-18T11:30:22.221112+0000","last_undegraded":"2024-09-18T11:30:22.221112+0000","last_fullsized":"2024-09-18T11:30:22.221112+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:02.341219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045765200000000002,"stat_sum":{"num_bytes":3092455,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":537,"num_write":8,"num_write_kb":2614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"4.8","version":"29'5","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.228840+0000","last_change":"2024-09-18T11:30:19.163869+0000","last_active":"2024-09-18T11:30:21.228840+0000","last_peered":"2024-09-18T11:30:21.228840+0000","last_clean":"2024-09-18T11:30:21.228840+0000","last_became_active":"2024-09-18T11:30:02.076172+0000","last_became_peered":"2024-09-18T11:30:02.076172+0000","last_unstale":"2024-09-18T11:30:21.228840+0000","last_undegraded":"2024-09-18T11:30:21.228840+0000","last_fullsized":"2024-09-18T11:30:21.228840+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:29.097553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025779900000000002,"stat_sum":{"num_bytes":2403368,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":7207,"num_write":4,"num_write_kb":1254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"4.5","version":"39'37","reported_seq":109,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.529172+0000","last_change":"2024-09-18T11:30:19.171380+0000","last_active":"2024-09-18T11:30:22.529172+0000","last_peered":"2024-09-18T11:30:22.529172+0000","last_clean":"2024-09-18T11:30:22.529172+0000","last_became_active":"2024-09-18T11:30:02.074697+0000","last_became_peered":"2024-09-18T11:30:02.074697+0000","last_unstale":"2024-09-18T11:30:22.529172+0000","last_undegraded":"2024-09-18T11:30:22.529172+0000","last_fullsized":"2024-09-18T11:30:22.529172+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.577548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041201270000000003,"stat_sum":{"num_bytes":13607941,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":67,"num_read_kb":67280,"num_write":31,"num_write_kb":9495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"1.0","version":"16'72","reported_seq":123,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.178541+0000","last_change":"2024-09-18T11:29:48.974790+0000","last_active":"2024-09-18T11:30:21.178541+0000","last_peered":"2024-09-18T11:30:21.178541+0000","last_clean":"2024-09-18T11:30:21.178541+0000","last_became_active":"2024-09-18T11:29:48.974043+0000","last_became_peered":"2024-09-18T11:29:48.974043+0000","last_unstale":"2024-09-18T11:30:21.178541+0000","last_undegraded":"2024-09-18T11:30:21.178541+0000","last_fullsized":"2024-09-18T11:30:21.178541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_clean_scrub_stamp":"2024-09-18T11:29:47.951437+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:13.784759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":37,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.229181+0000","last_change":"2024-09-18T11:29:52.047782+0000","last_active":"2024-09-18T11:30:21.229181+0000","last_peered":"2024-09-18T11:30:21.229181+0000","last_clean":"2024-09-18T11:30:21.229181+0000","last_became_active":"2024-09-18T11:29:50.028407+0000","last_became_peered":"2024-09-18T11:29:50.028407+0000","last_unstale":"2024-09-18T11:30:21.229181+0000","last_undegraded":"2024-09-18T11:30:21.229181+0000","last_fullsized":"2024-09-18T11:30:21.229181+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.851663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"41'90","reported_seq":119,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.263168+0000","last_change":"2024-09-18T11:30:01.077948+0000","last_active":"2024-09-18T11:30:22.263168+0000","last_peered":"2024-09-18T11:30:22.263168+0000","last_clean":"2024-09-18T11:30:22.263168+0000","last_became_active":"2024-09-18T11:30:01.077612+0000","last_became_peered":"2024-09-18T11:30:01.077612+0000","last_unstale":"2024-09-18T11:30:22.263168+0000","last_undegraded":"2024-09-18T11:30:22.263168+0000","last_fullsized":"2024-09-18T11:30:22.263168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:30:00.059457+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:11.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26266683,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":71,"num_read_kb":1434,"num_write":106,"num_write_kb":41966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"40'17","reported_seq":71,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.229931+0000","last_change":"2024-09-18T11:30:19.172754+0000","last_active":"2024-09-18T11:30:21.229931+0000","last_peered":"2024-09-18T11:30:21.229931+0000","last_clean":"2024-09-18T11:30:21.229931+0000","last_became_active":"2024-09-18T11:30:02.076812+0000","last_became_peered":"2024-09-18T11:30:02.076812+0000","last_unstale":"2024-09-18T11:30:21.229931+0000","last_undegraded":"2024-09-18T11:30:21.229931+0000","last_fullsized":"2024-09-18T11:30:21.229931+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:28.398569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053628970000000002,"stat_sum":{"num_bytes":6954776,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":33,"num_read_kb":30014,"num_write":13,"num_write_kb":3429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.178518+0000","last_change":"2024-09-18T11:29:52.420488+0000","last_active":"2024-09-18T11:30:21.178518+0000","last_peered":"2024-09-18T11:30:21.178518+0000","last_clean":"2024-09-18T11:30:21.178518+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:30:21.178518+0000","last_undegraded":"2024-09-18T11:30:21.178518+0000","last_fullsized":"2024-09-18T11:30:21.178518+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:02.759929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"41'71","reported_seq":104,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.207106+0000","last_change":"2024-09-18T11:30:01.073376+0000","last_active":"2024-09-18T11:30:22.207106+0000","last_peered":"2024-09-18T11:30:22.207106+0000","last_clean":"2024-09-18T11:30:22.207106+0000","last_became_active":"2024-09-18T11:30:01.073106+0000","last_became_peered":"2024-09-18T11:30:01.073106+0000","last_unstale":"2024-09-18T11:30:22.207106+0000","last_undegraded":"2024-09-18T11:30:22.207106+0000","last_fullsized":"2024-09-18T11:30:22.207106+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:30:00.059457+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:10.905311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30877265,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":61,"num_read_kb":2237,"num_write":78,"num_write_kb":39235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"39'48","reported_seq":120,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.218504+0000","last_change":"2024-09-18T11:30:19.179124+0000","last_active":"2024-09-18T11:30:21.218504+0000","last_peered":"2024-09-18T11:30:21.218504+0000","last_clean":"2024-09-18T11:30:21.218504+0000","last_became_active":"2024-09-18T11:30:02.076465+0000","last_became_peered":"2024-09-18T11:30:02.076465+0000","last_unstale":"2024-09-18T11:30:21.218504+0000","last_undegraded":"2024-09-18T11:30:21.218504+0000","last_fullsized":"2024-09-18T11:30:21.218504+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:15.876699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062013570000000002,"stat_sum":{"num_bytes":14854954,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":65,"num_read_kb":39994,"num_write":40,"num_write_kb":10429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"2.1","version":"0'0","reported_seq":38,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.178008+0000","last_change":"2024-09-18T11:29:52.348137+0000","last_active":"2024-09-18T11:30:21.178008+0000","last_peered":"2024-09-18T11:30:21.178008+0000","last_clean":"2024-09-18T11:30:21.178008+0000","last_became_active":"2024-09-18T11:29:50.180029+0000","last_became_peered":"2024-09-18T11:29:50.180029+0000","last_unstale":"2024-09-18T11:30:21.178008+0000","last_undegraded":"2024-09-18T11:30:21.178008+0000","last_fullsized":"2024-09-18T11:30:21.178008+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.269389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023063399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"39'58","reported_seq":85,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.218289+0000","last_change":"2024-09-18T11:30:01.071704+0000","last_active":"2024-09-18T11:30:21.218289+0000","last_peered":"2024-09-18T11:30:21.218289+0000","last_clean":"2024-09-18T11:30:21.218289+0000","last_became_active":"2024-09-18T11:30:01.071435+0000","last_became_peered":"2024-09-18T11:30:01.071435+0000","last_unstale":"2024-09-18T11:30:21.218289+0000","last_undegraded":"2024-09-18T11:30:21.218289+0000","last_fullsized":"2024-09-18T11:30:21.218289+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:30:00.059457+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:01.083180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18046455,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":44,"num_read_kb":29,"num_write":61,"num_write_kb":24921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"41'38","reported_seq":96,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.206678+0000","last_change":"2024-09-18T11:30:19.185275+0000","last_active":"2024-09-18T11:30:22.206678+0000","last_peered":"2024-09-18T11:30:22.206678+0000","last_clean":"2024-09-18T11:30:22.206678+0000","last_became_active":"2024-09-18T11:30:02.334751+0000","last_became_peered":"2024-09-18T11:30:02.334751+0000","last_unstale":"2024-09-18T11:30:22.206678+0000","last_undegraded":"2024-09-18T11:30:22.206678+0000","last_fullsized":"2024-09-18T11:30:22.206678+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:20.359490+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.014869475,"stat_sum":{"num_bytes":11503078,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":37,"num_read_kb":28810,"num_write":32,"num_write_kb":9028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":5,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"2.2","version":"18'2","reported_seq":38,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.228901+0000","last_change":"2024-09-18T11:29:52.048389+0000","last_active":"2024-09-18T11:30:21.228901+0000","last_peered":"2024-09-18T11:30:21.228901+0000","last_clean":"2024-09-18T11:30:21.228901+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:30:21.228901+0000","last_undegraded":"2024-09-18T11:30:21.228901+0000","last_fullsized":"2024-09-18T11:30:21.228901+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:13.358402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"39'75","reported_seq":105,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.179660+0000","last_change":"2024-09-18T11:30:01.073376+0000","last_active":"2024-09-18T11:30:21.179660+0000","last_peered":"2024-09-18T11:30:21.179660+0000","last_clean":"2024-09-18T11:30:21.179660+0000","last_became_active":"2024-09-18T11:30:01.073090+0000","last_became_peered":"2024-09-18T11:30:01.073090+0000","last_unstale":"2024-09-18T11:30:21.179660+0000","last_undegraded":"2024-09-18T11:30:21.179660+0000","last_fullsized":"2024-09-18T11:30:21.179660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:30:00.059457+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:13.135139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21807244,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":60,"num_read_kb":1242,"num_write":80,"num_write_kb":35256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"39'26","reported_seq":78,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.218599+0000","last_change":"2024-09-18T11:30:19.177978+0000","last_active":"2024-09-18T11:30:21.218599+0000","last_peered":"2024-09-18T11:30:21.218599+0000","last_clean":"2024-09-18T11:30:21.218599+0000","last_became_active":"2024-09-18T11:30:02.076874+0000","last_became_peered":"2024-09-18T11:30:02.076874+0000","last_unstale":"2024-09-18T11:30:21.218599+0000","last_undegraded":"2024-09-18T11:30:21.218599+0000","last_fullsized":"2024-09-18T11:30:21.218599+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:06.825500+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0077089469999999998,"stat_sum":{"num_bytes":5261910,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":19145,"num_write":23,"num_write_kb":6763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.218440+0000","last_change":"2024-09-18T11:29:52.044757+0000","last_active":"2024-09-18T11:30:21.218440+0000","last_peered":"2024-09-18T11:30:21.218440+0000","last_clean":"2024-09-18T11:30:21.218440+0000","last_became_active":"2024-09-18T11:29:50.034886+0000","last_became_peered":"2024-09-18T11:29:50.034886+0000","last_unstale":"2024-09-18T11:30:21.218440+0000","last_undegraded":"2024-09-18T11:30:21.218440+0000","last_fullsized":"2024-09-18T11:30:21.218440+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:18.177634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"40'37","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.264807+0000","last_change":"2024-09-18T11:30:19.182225+0000","last_active":"2024-09-18T11:30:22.264807+0000","last_peered":"2024-09-18T11:30:22.264807+0000","last_clean":"2024-09-18T11:30:22.264807+0000","last_became_active":"2024-09-18T11:30:02.075960+0000","last_became_peered":"2024-09-18T11:30:02.075960+0000","last_unstale":"2024-09-18T11:30:22.264807+0000","last_undegraded":"2024-09-18T11:30:22.264807+0000","last_fullsized":"2024-09-18T11:30:22.264807+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:20.654025+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.014664574999999999,"stat_sum":{"num_bytes":8941225,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":40,"num_read_kb":37636,"num_write":28,"num_write_kb":7858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.178462+0000","last_change":"2024-09-18T11:29:52.420604+0000","last_active":"2024-09-18T11:30:21.178462+0000","last_peered":"2024-09-18T11:30:21.178462+0000","last_clean":"2024-09-18T11:30:21.178462+0000","last_became_active":"2024-09-18T11:29:50.034939+0000","last_became_peered":"2024-09-18T11:29:50.034939+0000","last_unstale":"2024-09-18T11:30:21.178462+0000","last_undegraded":"2024-09-18T11:30:21.178462+0000","last_fullsized":"2024-09-18T11:30:21.178462+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:16.259587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"40'55","reported_seq":119,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.208641+0000","last_change":"2024-09-18T11:30:19.195513+0000","last_active":"2024-09-18T11:30:22.208641+0000","last_peered":"2024-09-18T11:30:22.208641+0000","last_clean":"2024-09-18T11:30:22.208641+0000","last_became_active":"2024-09-18T11:30:02.072165+0000","last_became_peered":"2024-09-18T11:30:02.072165+0000","last_unstale":"2024-09-18T11:30:22.208641+0000","last_undegraded":"2024-09-18T11:30:22.208641+0000","last_fullsized":"2024-09-18T11:30:22.208641+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:04.270484+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0064590749999999999,"stat_sum":{"num_bytes":13000709,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":53,"num_read_kb":50908,"num_write":48,"num_write_kb":15688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":5,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.219008+0000","last_change":"2024-09-18T11:29:52.044763+0000","last_active":"2024-09-18T11:30:21.219008+0000","last_peered":"2024-09-18T11:30:21.219008+0000","last_clean":"2024-09-18T11:30:21.219008+0000","last_became_active":"2024-09-18T11:29:50.028674+0000","last_became_peered":"2024-09-18T11:29:50.028674+0000","last_unstale":"2024-09-18T11:30:21.219008+0000","last_undegraded":"2024-09-18T11:30:21.219008+0000","last_fullsized":"2024-09-18T11:30:21.219008+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:53.878233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031621200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"41'53","reported_seq":121,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.530919+0000","last_change":"2024-09-18T11:30:19.170173+0000","last_active":"2024-09-18T11:30:22.530919+0000","last_peered":"2024-09-18T11:30:22.530919+0000","last_clean":"2024-09-18T11:30:22.530919+0000","last_became_active":"2024-09-18T11:30:02.075529+0000","last_became_peered":"2024-09-18T11:30:02.075529+0000","last_unstale":"2024-09-18T11:30:22.530919+0000","last_undegraded":"2024-09-18T11:30:22.530919+0000","last_fullsized":"2024-09-18T11:30:22.530919+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:21.261787+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0026356589999999998,"stat_sum":{"num_bytes":15540630,"num_objects":7,"num_object_clones":3,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":63,"num_read_kb":62260,"num_write":42,"num_write_kb":11007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":37,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.774574+0000","last_change":"2024-09-18T11:29:52.042474+0000","last_active":"2024-09-18T11:30:21.774574+0000","last_peered":"2024-09-18T11:30:21.774574+0000","last_clean":"2024-09-18T11:30:21.774574+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:30:21.774574+0000","last_undegraded":"2024-09-18T11:30:21.774574+0000","last_fullsized":"2024-09-18T11:30:21.774574+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:56.288858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":161220740,"num_objects":65,"num_object_clones":17,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":2,"num_read":748,"num_read_kb":679396,"num_write":437,"num_write_kb":125573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":45,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":190930944,"data_stored":206016468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":526,"ondisk_log_size":526,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":96997647,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":236,"num_read_kb":4942,"num_write":325,"num_write_kb":141378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":104857600,"data_stored":104640250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":294,"ondisk_log_size":294,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309168,"kb_used_data":11808,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94062672,"statfs":{"total":96636764160,"available":96320176128,"internally_reserved":0,"allocated":12091392,"data_stored":14078460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312904,"kb_used_data":15548,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94058936,"statfs":{"total":96636764160,"available":96316350464,"internally_reserved":0,"allocated":15921152,"data_stored":15825369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":340160,"kb_used_data":41752,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94031680,"statfs":{"total":96636764160,"available":96288440320,"internally_reserved":0,"allocated":42754048,"data_stored":44473385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":354168,"kb_used_data":56812,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94017672,"statfs":{"total":96636764160,"available":96274096128,"internally_reserved":0,"allocated":58175488,"data_stored":58895244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":329356,"kb_used_data":31740,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94042484,"statfs":{"total":96636764160,"available":96299503616,"internally_reserved":0,"allocated":32501760,"data_stored":34409337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.247}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322292,"kb_used_data":24932,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94049548,"statfs":{"total":96636764160,"available":96306737152,"internally_reserved":0,"allocated":25530368,"data_stored":25936579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.249}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":351328,"kb_used_data":52664,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94020512,"statfs":{"total":96636764160,"available":96277004288,"internally_reserved":0,"allocated":53927936,"data_stored":56096942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":343016,"kb_used_data":45656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94028824,"statfs":{"total":96636764160,"available":96285515776,"internally_reserved":0,"allocated":46751744,"data_stored":47207479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28450816,"data_stored":28402658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10584064,"data_stored":10565145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10584064,"data_stored":10565145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13393920,"data_stored":13352322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28450816,"data_stored":28402658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":13393920,"data_stored":13352322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17313792,"data_stored":18065286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":43937792,"data_stored":46831636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18186240,"data_stored":20145598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":18661376,"data_stored":20849847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36278272,"data_stored":38737716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28921856,"data_stored":30915857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16523264,"data_stored":17131054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":11108352,"data_stored":13339474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:30:27.665 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:30:27.690 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:27.691 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:27.699 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:30:27.745 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:27.744+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:27.839 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:27.839 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:27.847 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:27.845+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:27.847 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:30:27.924 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:30:27.924 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:30:27.936 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-09-18T11:30:27.202341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":258808774,"num_objects":150,"num_object_clones":17,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":2,"num_read":1050,"num_read_kb":684393,"num_write":865,"num_write_kb":268581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":45,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":895,"ondisk_log_size":895,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2662392,"kb_used_data":280912,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752312328,"statfs":{"total":773094113280,"available":770367823872,"internally_reserved":0,"allocated":287653888,"data_stored":296922795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.208242"},"pg_stats":[{"pgid":"4.f","version":"39'38","reported_seq":110,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.928976+0000","last_change":"2024-09-18T11:30:19.178889+0000","last_active":"2024-09-18T11:30:22.928976+0000","last_peered":"2024-09-18T11:30:22.928976+0000","last_clean":"2024-09-18T11:30:22.928976+0000","last_became_active":"2024-09-18T11:30:02.074391+0000","last_became_peered":"2024-09-18T11:30:02.074391+0000","last_unstale":"2024-09-18T11:30:22.928976+0000","last_undegraded":"2024-09-18T11:30:22.928976+0000","last_fullsized":"2024-09-18T11:30:22.928976+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:11.123483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006049072,"stat_sum":{"num_bytes":11159716,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":64,"num_read_kb":71156,"num_write":38,"num_write_kb":10090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"4.e","version":"29'21","reported_seq":83,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.527195+0000","last_change":"2024-09-18T11:30:19.167944+0000","last_active":"2024-09-18T11:30:22.527195+0000","last_peered":"2024-09-18T11:30:22.527195+0000","last_clean":"2024-09-18T11:30:22.527195+0000","last_became_active":"2024-09-18T11:30:02.073185+0000","last_became_peered":"2024-09-18T11:30:02.073185+0000","last_unstale":"2024-09-18T11:30:22.527195+0000","last_undegraded":"2024-09-18T11:30:22.527195+0000","last_fullsized":"2024-09-18T11:30:22.527195+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:12.655066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047952999999999998,"stat_sum":{"num_bytes":7697507,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":38326,"num_write":17,"num_write_kb":4267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"4.d","version":"41'45","reported_seq":121,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.927655+0000","last_change":"2024-09-18T11:30:19.171822+0000","last_active":"2024-09-18T11:30:22.927655+0000","last_peered":"2024-09-18T11:30:22.927655+0000","last_clean":"2024-09-18T11:30:22.927655+0000","last_became_active":"2024-09-18T11:30:02.334498+0000","last_became_peered":"2024-09-18T11:30:02.334498+0000","last_unstale":"2024-09-18T11:30:22.927655+0000","last_undegraded":"2024-09-18T11:30:22.927655+0000","last_fullsized":"2024-09-18T11:30:22.927655+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:10.245504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041312459999999999,"stat_sum":{"num_bytes":18516982,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":77,"num_read_kb":66091,"num_write":40,"num_write_kb":12223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"4.c","version":"34'18","reported_seq":71,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.179685+0000","last_change":"2024-09-18T11:30:19.167768+0000","last_active":"2024-09-18T11:30:21.179685+0000","last_peered":"2024-09-18T11:30:21.179685+0000","last_clean":"2024-09-18T11:30:21.179685+0000","last_became_active":"2024-09-18T11:30:02.073883+0000","last_became_peered":"2024-09-18T11:30:02.073883+0000","last_unstale":"2024-09-18T11:30:21.179685+0000","last_undegraded":"2024-09-18T11:30:21.179685+0000","last_fullsized":"2024-09-18T11:30:21.179685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:20.359490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042985799999999998,"stat_sum":{"num_bytes":5226911,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":19494,"num_write":12,"num_write_kb":3340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"4.b","version":"39'53","reported_seq":131,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.223523+0000","last_change":"2024-09-18T11:30:19.188724+0000","last_active":"2024-09-18T11:30:22.223523+0000","last_peered":"2024-09-18T11:30:22.223523+0000","last_clean":"2024-09-18T11:30:22.223523+0000","last_became_active":"2024-09-18T11:30:02.075449+0000","last_became_peered":"2024-09-18T11:30:02.075449+0000","last_unstale":"2024-09-18T11:30:22.223523+0000","last_undegraded":"2024-09-18T11:30:22.223523+0000","last_fullsized":"2024-09-18T11:30:22.223523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:16.299317+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.021340656999999999,"stat_sum":{"num_bytes":14200880,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":76964,"num_write":40,"num_write_kb":10935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":5,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"4.a","version":"41'26","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.265944+0000","last_change":"2024-09-18T11:30:19.171211+0000","last_active":"2024-09-18T11:30:22.265944+0000","last_peered":"2024-09-18T11:30:22.265944+0000","last_clean":"2024-09-18T11:30:22.265944+0000","last_became_active":"2024-09-18T11:30:02.075538+0000","last_became_peered":"2024-09-18T11:30:02.075538+0000","last_unstale":"2024-09-18T11:30:22.265944+0000","last_undegraded":"2024-09-18T11:30:22.265944+0000","last_fullsized":"2024-09-18T11:30:22.265944+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:26.428990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038727599999999998,"stat_sum":{"num_bytes":9257698,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":62,"num_read_kb":63574,"num_write":21,"num_write_kb":7153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"4.9","version":"41'9","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.221112+0000","last_change":"2024-09-18T11:30:19.167762+0000","last_active":"2024-09-18T11:30:22.221112+0000","last_peered":"2024-09-18T11:30:22.221112+0000","last_clean":"2024-09-18T11:30:22.221112+0000","last_became_active":"2024-09-18T11:30:02.074414+0000","last_became_peered":"2024-09-18T11:30:02.074414+0000","last_unstale":"2024-09-18T11:30:22.221112+0000","last_undegraded":"2024-09-18T11:30:22.221112+0000","last_fullsized":"2024-09-18T11:30:22.221112+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:02.341219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045765200000000002,"stat_sum":{"num_bytes":3092455,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":537,"num_write":8,"num_write_kb":2614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"4.8","version":"29'5","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.228840+0000","last_change":"2024-09-18T11:30:19.163869+0000","last_active":"2024-09-18T11:30:21.228840+0000","last_peered":"2024-09-18T11:30:21.228840+0000","last_clean":"2024-09-18T11:30:21.228840+0000","last_became_active":"2024-09-18T11:30:02.076172+0000","last_became_peered":"2024-09-18T11:30:02.076172+0000","last_unstale":"2024-09-18T11:30:21.228840+0000","last_undegraded":"2024-09-18T11:30:21.228840+0000","last_fullsized":"2024-09-18T11:30:21.228840+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:29.097553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025779900000000002,"stat_sum":{"num_bytes":2403368,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":7207,"num_write":4,"num_write_kb":1254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"4.5","version":"39'37","reported_seq":109,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.529172+0000","last_change":"2024-09-18T11:30:19.171380+0000","last_active":"2024-09-18T11:30:22.529172+0000","last_peered":"2024-09-18T11:30:22.529172+0000","last_clean":"2024-09-18T11:30:22.529172+0000","last_became_active":"2024-09-18T11:30:02.074697+0000","last_became_peered":"2024-09-18T11:30:02.074697+0000","last_unstale":"2024-09-18T11:30:22.529172+0000","last_undegraded":"2024-09-18T11:30:22.529172+0000","last_fullsized":"2024-09-18T11:30:22.529172+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.577548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041201270000000003,"stat_sum":{"num_bytes":13607941,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":67,"num_read_kb":67280,"num_write":31,"num_write_kb":9495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"1.0","version":"16'72","reported_seq":123,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.178541+0000","last_change":"2024-09-18T11:29:48.974790+0000","last_active":"2024-09-18T11:30:21.178541+0000","last_peered":"2024-09-18T11:30:21.178541+0000","last_clean":"2024-09-18T11:30:21.178541+0000","last_became_active":"2024-09-18T11:29:48.974043+0000","last_became_peered":"2024-09-18T11:29:48.974043+0000","last_unstale":"2024-09-18T11:30:21.178541+0000","last_undegraded":"2024-09-18T11:30:21.178541+0000","last_fullsized":"2024-09-18T11:30:21.178541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_clean_scrub_stamp":"2024-09-18T11:29:47.951437+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:13.784759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":37,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.229181+0000","last_change":"2024-09-18T11:29:52.047782+0000","last_active":"2024-09-18T11:30:21.229181+0000","last_peered":"2024-09-18T11:30:21.229181+0000","last_clean":"2024-09-18T11:30:21.229181+0000","last_became_active":"2024-09-18T11:29:50.028407+0000","last_became_peered":"2024-09-18T11:29:50.028407+0000","last_unstale":"2024-09-18T11:30:21.229181+0000","last_undegraded":"2024-09-18T11:30:21.229181+0000","last_fullsized":"2024-09-18T11:30:21.229181+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.851663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"41'90","reported_seq":119,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.263168+0000","last_change":"2024-09-18T11:30:01.077948+0000","last_active":"2024-09-18T11:30:22.263168+0000","last_peered":"2024-09-18T11:30:22.263168+0000","last_clean":"2024-09-18T11:30:22.263168+0000","last_became_active":"2024-09-18T11:30:01.077612+0000","last_became_peered":"2024-09-18T11:30:01.077612+0000","last_unstale":"2024-09-18T11:30:22.263168+0000","last_undegraded":"2024-09-18T11:30:22.263168+0000","last_fullsized":"2024-09-18T11:30:22.263168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:30:00.059457+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:11.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26266683,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":71,"num_read_kb":1434,"num_write":106,"num_write_kb":41966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"40'17","reported_seq":71,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.229931+0000","last_change":"2024-09-18T11:30:19.172754+0000","last_active":"2024-09-18T11:30:21.229931+0000","last_peered":"2024-09-18T11:30:21.229931+0000","last_clean":"2024-09-18T11:30:21.229931+0000","last_became_active":"2024-09-18T11:30:02.076812+0000","last_became_peered":"2024-09-18T11:30:02.076812+0000","last_unstale":"2024-09-18T11:30:21.229931+0000","last_undegraded":"2024-09-18T11:30:21.229931+0000","last_fullsized":"2024-09-18T11:30:21.229931+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:28.398569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053628970000000002,"stat_sum":{"num_bytes":6954776,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":33,"num_read_kb":30014,"num_write":13,"num_write_kb":3429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.178518+0000","last_change":"2024-09-18T11:29:52.420488+0000","last_active":"2024-09-18T11:30:21.178518+0000","last_peered":"2024-09-18T11:30:21.178518+0000","last_clean":"2024-09-18T11:30:21.178518+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:30:21.178518+0000","last_undegraded":"2024-09-18T11:30:21.178518+0000","last_fullsized":"2024-09-18T11:30:21.178518+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:02.759929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"41'71","reported_seq":104,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.207106+0000","last_change":"2024-09-18T11:30:01.073376+0000","last_active":"2024-09-18T11:30:22.207106+0000","last_peered":"2024-09-18T11:30:22.207106+0000","last_clean":"2024-09-18T11:30:22.207106+0000","last_became_active":"2024-09-18T11:30:01.073106+0000","last_became_peered":"2024-09-18T11:30:01.073106+0000","last_unstale":"2024-09-18T11:30:22.207106+0000","last_undegraded":"2024-09-18T11:30:22.207106+0000","last_fullsized":"2024-09-18T11:30:22.207106+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:30:00.059457+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:10.905311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30877265,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":61,"num_read_kb":2237,"num_write":78,"num_write_kb":39235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"39'48","reported_seq":120,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.218504+0000","last_change":"2024-09-18T11:30:19.179124+0000","last_active":"2024-09-18T11:30:21.218504+0000","last_peered":"2024-09-18T11:30:21.218504+0000","last_clean":"2024-09-18T11:30:21.218504+0000","last_became_active":"2024-09-18T11:30:02.076465+0000","last_became_peered":"2024-09-18T11:30:02.076465+0000","last_unstale":"2024-09-18T11:30:21.218504+0000","last_undegraded":"2024-09-18T11:30:21.218504+0000","last_fullsized":"2024-09-18T11:30:21.218504+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:15.876699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062013570000000002,"stat_sum":{"num_bytes":14854954,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":65,"num_read_kb":39994,"num_write":40,"num_write_kb":10429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"2.1","version":"0'0","reported_seq":38,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.178008+0000","last_change":"2024-09-18T11:29:52.348137+0000","last_active":"2024-09-18T11:30:21.178008+0000","last_peered":"2024-09-18T11:30:21.178008+0000","last_clean":"2024-09-18T11:30:21.178008+0000","last_became_active":"2024-09-18T11:29:50.180029+0000","last_became_peered":"2024-09-18T11:29:50.180029+0000","last_unstale":"2024-09-18T11:30:21.178008+0000","last_undegraded":"2024-09-18T11:30:21.178008+0000","last_fullsized":"2024-09-18T11:30:21.178008+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.269389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023063399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"39'58","reported_seq":85,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.218289+0000","last_change":"2024-09-18T11:30:01.071704+0000","last_active":"2024-09-18T11:30:21.218289+0000","last_peered":"2024-09-18T11:30:21.218289+0000","last_clean":"2024-09-18T11:30:21.218289+0000","last_became_active":"2024-09-18T11:30:01.071435+0000","last_became_peered":"2024-09-18T11:30:01.071435+0000","last_unstale":"2024-09-18T11:30:21.218289+0000","last_undegraded":"2024-09-18T11:30:21.218289+0000","last_fullsized":"2024-09-18T11:30:21.218289+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:30:00.059457+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:01.083180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18046455,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":44,"num_read_kb":29,"num_write":61,"num_write_kb":24921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"41'38","reported_seq":96,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.206678+0000","last_change":"2024-09-18T11:30:19.185275+0000","last_active":"2024-09-18T11:30:22.206678+0000","last_peered":"2024-09-18T11:30:22.206678+0000","last_clean":"2024-09-18T11:30:22.206678+0000","last_became_active":"2024-09-18T11:30:02.334751+0000","last_became_peered":"2024-09-18T11:30:02.334751+0000","last_unstale":"2024-09-18T11:30:22.206678+0000","last_undegraded":"2024-09-18T11:30:22.206678+0000","last_fullsized":"2024-09-18T11:30:22.206678+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:20.359490+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.014869475,"stat_sum":{"num_bytes":11503078,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":37,"num_read_kb":28810,"num_write":32,"num_write_kb":9028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":5,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"2.2","version":"18'2","reported_seq":38,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.228901+0000","last_change":"2024-09-18T11:29:52.048389+0000","last_active":"2024-09-18T11:30:21.228901+0000","last_peered":"2024-09-18T11:30:21.228901+0000","last_clean":"2024-09-18T11:30:21.228901+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:30:21.228901+0000","last_undegraded":"2024-09-18T11:30:21.228901+0000","last_fullsized":"2024-09-18T11:30:21.228901+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:13.358402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"39'75","reported_seq":105,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.179660+0000","last_change":"2024-09-18T11:30:01.073376+0000","last_active":"2024-09-18T11:30:21.179660+0000","last_peered":"2024-09-18T11:30:21.179660+0000","last_clean":"2024-09-18T11:30:21.179660+0000","last_became_active":"2024-09-18T11:30:01.073090+0000","last_became_peered":"2024-09-18T11:30:01.073090+0000","last_unstale":"2024-09-18T11:30:21.179660+0000","last_undegraded":"2024-09-18T11:30:21.179660+0000","last_fullsized":"2024-09-18T11:30:21.179660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:30:00.059457+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:13.135139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21807244,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":60,"num_read_kb":1242,"num_write":80,"num_write_kb":35256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"39'26","reported_seq":78,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.218599+0000","last_change":"2024-09-18T11:30:19.177978+0000","last_active":"2024-09-18T11:30:21.218599+0000","last_peered":"2024-09-18T11:30:21.218599+0000","last_clean":"2024-09-18T11:30:21.218599+0000","last_became_active":"2024-09-18T11:30:02.076874+0000","last_became_peered":"2024-09-18T11:30:02.076874+0000","last_unstale":"2024-09-18T11:30:21.218599+0000","last_undegraded":"2024-09-18T11:30:21.218599+0000","last_fullsized":"2024-09-18T11:30:21.218599+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:06.825500+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0077089469999999998,"stat_sum":{"num_bytes":5261910,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":19145,"num_write":23,"num_write_kb":6763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.218440+0000","last_change":"2024-09-18T11:29:52.044757+0000","last_active":"2024-09-18T11:30:21.218440+0000","last_peered":"2024-09-18T11:30:21.218440+0000","last_clean":"2024-09-18T11:30:21.218440+0000","last_became_active":"2024-09-18T11:29:50.034886+0000","last_became_peered":"2024-09-18T11:29:50.034886+0000","last_unstale":"2024-09-18T11:30:21.218440+0000","last_undegraded":"2024-09-18T11:30:21.218440+0000","last_fullsized":"2024-09-18T11:30:21.218440+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:18.177634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"40'37","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.264807+0000","last_change":"2024-09-18T11:30:19.182225+0000","last_active":"2024-09-18T11:30:22.264807+0000","last_peered":"2024-09-18T11:30:22.264807+0000","last_clean":"2024-09-18T11:30:22.264807+0000","last_became_active":"2024-09-18T11:30:02.075960+0000","last_became_peered":"2024-09-18T11:30:02.075960+0000","last_unstale":"2024-09-18T11:30:22.264807+0000","last_undegraded":"2024-09-18T11:30:22.264807+0000","last_fullsized":"2024-09-18T11:30:22.264807+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:20.654025+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.014664574999999999,"stat_sum":{"num_bytes":8941225,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":40,"num_read_kb":37636,"num_write":28,"num_write_kb":7858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.178462+0000","last_change":"2024-09-18T11:29:52.420604+0000","last_active":"2024-09-18T11:30:21.178462+0000","last_peered":"2024-09-18T11:30:21.178462+0000","last_clean":"2024-09-18T11:30:21.178462+0000","last_became_active":"2024-09-18T11:29:50.034939+0000","last_became_peered":"2024-09-18T11:29:50.034939+0000","last_unstale":"2024-09-18T11:30:21.178462+0000","last_undegraded":"2024-09-18T11:30:21.178462+0000","last_fullsized":"2024-09-18T11:30:21.178462+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:16.259587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"40'55","reported_seq":119,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.208641+0000","last_change":"2024-09-18T11:30:19.195513+0000","last_active":"2024-09-18T11:30:22.208641+0000","last_peered":"2024-09-18T11:30:22.208641+0000","last_clean":"2024-09-18T11:30:22.208641+0000","last_became_active":"2024-09-18T11:30:02.072165+0000","last_became_peered":"2024-09-18T11:30:02.072165+0000","last_unstale":"2024-09-18T11:30:22.208641+0000","last_undegraded":"2024-09-18T11:30:22.208641+0000","last_fullsized":"2024-09-18T11:30:22.208641+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:04.270484+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0064590749999999999,"stat_sum":{"num_bytes":13000709,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":53,"num_read_kb":50908,"num_write":48,"num_write_kb":15688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":5,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.219008+0000","last_change":"2024-09-18T11:29:52.044763+0000","last_active":"2024-09-18T11:30:21.219008+0000","last_peered":"2024-09-18T11:30:21.219008+0000","last_clean":"2024-09-18T11:30:21.219008+0000","last_became_active":"2024-09-18T11:29:50.028674+0000","last_became_peered":"2024-09-18T11:29:50.028674+0000","last_unstale":"2024-09-18T11:30:21.219008+0000","last_undegraded":"2024-09-18T11:30:21.219008+0000","last_fullsized":"2024-09-18T11:30:21.219008+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:53.878233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031621200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"41'53","reported_seq":121,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T11:30:22.530919+0000","last_change":"2024-09-18T11:30:19.170173+0000","last_active":"2024-09-18T11:30:22.530919+0000","last_peered":"2024-09-18T11:30:22.530919+0000","last_clean":"2024-09-18T11:30:22.530919+0000","last_became_active":"2024-09-18T11:30:02.075529+0000","last_became_peered":"2024-09-18T11:30:02.075529+0000","last_unstale":"2024-09-18T11:30:22.530919+0000","last_undegraded":"2024-09-18T11:30:22.530919+0000","last_fullsized":"2024-09-18T11:30:22.530919+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:21.261787+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0026356589999999998,"stat_sum":{"num_bytes":15540630,"num_objects":7,"num_object_clones":3,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":63,"num_read_kb":62260,"num_write":42,"num_write_kb":11007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"6","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":37,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T11:30:21.774574+0000","last_change":"2024-09-18T11:29:52.042474+0000","last_active":"2024-09-18T11:30:21.774574+0000","last_peered":"2024-09-18T11:30:21.774574+0000","last_clean":"2024-09-18T11:30:21.774574+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:30:21.774574+0000","last_undegraded":"2024-09-18T11:30:21.774574+0000","last_fullsized":"2024-09-18T11:30:21.774574+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:56.288858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":161220740,"num_objects":65,"num_object_clones":17,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":2,"num_read":748,"num_read_kb":679396,"num_write":437,"num_write_kb":125573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":45,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":190930944,"data_stored":206016468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":526,"ondisk_log_size":526,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":96997647,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":236,"num_read_kb":4942,"num_write":325,"num_write_kb":141378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":104857600,"data_stored":104640250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":294,"ondisk_log_size":294,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309168,"kb_used_data":11808,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94062672,"statfs":{"total":96636764160,"available":96320176128,"internally_reserved":0,"allocated":12091392,"data_stored":14078460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312904,"kb_used_data":15548,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94058936,"statfs":{"total":96636764160,"available":96316350464,"internally_reserved":0,"allocated":15921152,"data_stored":15825369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":340160,"kb_used_data":41752,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94031680,"statfs":{"total":96636764160,"available":96288440320,"internally_reserved":0,"allocated":42754048,"data_stored":44473385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":354168,"kb_used_data":56812,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94017672,"statfs":{"total":96636764160,"available":96274096128,"internally_reserved":0,"allocated":58175488,"data_stored":58895244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":329356,"kb_used_data":31740,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94042484,"statfs":{"total":96636764160,"available":96299503616,"internally_reserved":0,"allocated":32501760,"data_stored":34409337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.247}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322292,"kb_used_data":24932,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94049548,"statfs":{"total":96636764160,"available":96306737152,"internally_reserved":0,"allocated":25530368,"data_stored":25936579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.249}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":351328,"kb_used_data":52664,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94020512,"statfs":{"total":96636764160,"available":96277004288,"internally_reserved":0,"allocated":53927936,"data_stored":56096942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":343016,"kb_used_data":45656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94028824,"statfs":{"total":96636764160,"available":96285515776,"internally_reserved":0,"allocated":46751744,"data_stored":47207479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28450816,"data_stored":28402658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10584064,"data_stored":10565145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10584064,"data_stored":10565145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13393920,"data_stored":13352322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28450816,"data_stored":28402658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":13393920,"data_stored":13352322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17313792,"data_stored":18065286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":43937792,"data_stored":46831636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18186240,"data_stored":20145598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":18661376,"data_stored":20849847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36278272,"data_stored":38737716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28921856,"data_stored":30915857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16523264,"data_stored":17131054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":11108352,"data_stored":13339474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:30:27.938 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T11:30:27.945 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:27.946 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:27.954 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:30:27.977 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:27.976+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:28.047 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:28.046+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:28.051 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:28.052 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:28.060 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:30:28.132 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:30:28.148 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:28.147+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:28.156 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:28.157 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:28.165 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:30:28.210 INFO:tasks.rados.rados.0.smithi119.stdout:824: done (7 left) 2024-09-18T11:30:28.210 INFO:tasks.rados.rados.0.smithi119.stdout:825: snap_create 2024-09-18T11:30:28.210 INFO:tasks.rados.rados.0.smithi119.stdout:817: finishing write tid 1 to smithi11939506-24 2024-09-18T11:30:28.210 INFO:tasks.rados.rados.0.smithi119.stdout:817: finishing write tid 2 to smithi11939506-24 2024-09-18T11:30:28.210 INFO:tasks.rados.rados.0.smithi119.stdout:817: finishing write tid 3 to smithi11939506-24 2024-09-18T11:30:28.210 INFO:tasks.rados.rados.0.smithi119.stdout:817: finishing write tid 4 to smithi11939506-24 2024-09-18T11:30:28.210 INFO:tasks.rados.rados.0.smithi119.stdout:817: finishing write tid 5 to smithi11939506-24 2024-09-18T11:30:28.210 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 54 (ObjNum 109 snap 8 seq_num 109) dirty exists 2024-09-18T11:30:28.210 INFO:tasks.rados.rados.0.smithi119.stdout:817: left oid 24 (ObjNum 109 snap 8 seq_num 109) 2024-09-18T11:30:28.210 INFO:tasks.rados.rados.0.smithi119.stdout:818: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-18T11:30:28.250 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:28.249+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:28.256 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:30:28.256 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:30:28.271 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:28.271 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:28.286 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:28.286 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:28.288 INFO:tasks.rados.rados.0.smithi119.stdout:820: expect (ObjNum 101 snap 6 seq_num 101) 2024-09-18T11:30:28.300 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:28.301 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:28.307 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:28.307 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:28.316 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:30:28.317 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:28.317 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:28.331 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:28.332 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:28.346 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:28.347 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:28.351 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:28.350+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:28.362 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:28.362 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:28.415 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:28.416 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:28.416 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:28.416 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:28.424 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:30:28.467 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:28.466+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:28.489 INFO:tasks.rados.rados.0.smithi119.stdout:821: expect (ObjNum 98 snap 6 seq_num 98) 2024-09-18T11:30:28.521 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:28.521 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:28.522 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:28.522 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:28.522 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:28.530 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:30:28.553 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:28.552+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:28.643 INFO:tasks.rados.rados.0.smithi119.stdout:819: finishing set_chunk to oid 20 2024-09-18T11:30:28.643 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 20 v 35 (ObjNum 80 snap 4 seq_num 80) dirty exists 2024-09-18T11:30:28.643 INFO:tasks.rados.rados.0.smithi119.stdout:822: expect (ObjNum 91 snap 5 seq_num 91) 2024-09-18T11:30:28.653 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:28.652+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:28.670 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:28.670 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:28.670 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:28.670 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:28.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.670 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:28.671 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:28.671 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.551605372000001, 2024-09-18T11:30:28.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:30:28.671 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:28.671 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:28.671 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:28.671 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:28.671 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:28.671 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:30:28.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.671 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:28.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:28.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:28.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:28.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:28.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:28.673 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:30:28.673 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:28.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.673 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.673 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:28.673 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:30:28.673 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:30:28.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.673 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.673 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:28.673 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:30:28.673 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:30:28.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.674 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:28.674 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:30:28.674 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:30:28.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.674 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.674 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:30:28.674 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:30:28.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:28.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:30:28.675 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:30:28.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:30:28.675 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:30:28.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:28.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:30:28.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:30:28.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.676 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:28.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:30:28.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:30:28.676 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.676 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:28.676 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.677 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:896 4.5 4:aa575018:::smithi11939506-11:2 [read 0~2941192,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:28.677 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.132995+0000", 2024-09-18T11:30:28.677 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.534929730999998, 2024-09-18T11:30:28.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047874989999999997, 2024-09-18T11:30:28.677 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:28.677 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:28.677 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:28.677 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:28.677 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:28.677 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 896 2024-09-18T11:30:28.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.677 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:28.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.678 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:28.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.132995+0000", 2024-09-18T11:30:28.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.678 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:28.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.132995+0000", 2024-09-18T11:30:28.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.678 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:28.679 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.132993+0000", 2024-09-18T11:30:28.679 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:28.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.679 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:28.679 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133000+0000", 2024-09-18T11:30:28.679 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5509999999999996e-06 2024-09-18T11:30:28.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.679 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:28.679 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133010+0000", 2024-09-18T11:30:28.680 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0254e-05 2024-09-18T11:30:28.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.680 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:28.680 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133023+0000", 2024-09-18T11:30:28.680 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3458999999999999e-05 2024-09-18T11:30:28.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.681 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133069+0000", 2024-09-18T11:30:28.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5324000000000002e-05 2024-09-18T11:30:28.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.681 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:28.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133161+0000", 2024-09-18T11:30:28.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2773000000000001e-05 2024-09-18T11:30:28.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.681 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:28.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.137782+0000", 2024-09-18T11:30:28.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046208680000000002 2024-09-18T11:30:28.682 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.682 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:28.682 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.682 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:897 4.5 4:aa575018:::smithi11939506-11:2 [sparse-read 0~1343176] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:28.683 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.133034+0000", 2024-09-18T11:30:28.683 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.534890372, 2024-09-18T11:30:28.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051617950000000003, 2024-09-18T11:30:28.683 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:28.683 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:28.683 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:28.683 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:28.683 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:28.683 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 897 2024-09-18T11:30:28.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.683 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:28.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.684 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:28.684 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133034+0000", 2024-09-18T11:30:28.684 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.684 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.685 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:28.685 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133034+0000", 2024-09-18T11:30:28.685 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:28.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133033+0000", 2024-09-18T11:30:28.686 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:28.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:28.687 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133037+0000", 2024-09-18T11:30:28.687 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.118e-06 2024-09-18T11:30:28.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.687 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:28.687 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133045+0000", 2024-09-18T11:30:28.687 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1780000000000003e-06 2024-09-18T11:30:28.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.687 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:28.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133049+0000", 2024-09-18T11:30:28.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7730000000000001e-06 2024-09-18T11:30:28.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.688 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.137838+0000", 2024-09-18T11:30:28.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00478912 2024-09-18T11:30:28.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.688 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:28.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.137936+0000", 2024-09-18T11:30:28.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.823e-05 2024-09-18T11:30:28.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.689 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:28.689 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138196+0000", 2024-09-18T11:30:28.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00025993799999999998 2024-09-18T11:30:28.689 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.689 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:28.689 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.689 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:898 4.5 4:aa575018:::smithi11939506-11:2 [sparse-read 0~1343176] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:28.689 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:28.689 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.534861333999999, 2024-09-18T11:30:28.690 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055588349999999998, 2024-09-18T11:30:28.690 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:28.690 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:28.690 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:28.690 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:28.690 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:28.690 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 898 2024-09-18T11:30:28.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.690 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:28.690 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.690 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:28.690 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:28.690 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.691 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:28.691 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:28.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.691 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:28.691 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:28.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:28.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.691 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:28.691 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133066+0000", 2024-09-18T11:30:28.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2729999999999998e-06 2024-09-18T11:30:28.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:28.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133072+0000", 2024-09-18T11:30:28.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0680000000000002e-06 2024-09-18T11:30:28.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:28.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133077+0000", 2024-09-18T11:30:28.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3569999999999997e-06 2024-09-18T11:30:28.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138226+0000", 2024-09-18T11:30:28.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051480980000000003 2024-09-18T11:30:28.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.693 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:28.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138324+0000", 2024-09-18T11:30:28.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8639000000000002e-05 2024-09-18T11:30:28.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.693 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:28.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138622+0000", 2024-09-18T11:30:28.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029774100000000002 2024-09-18T11:30:28.693 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.693 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:28.694 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.694 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1034 4.a 4:588debc2:::smithi11939506-18:head [sparse-read 0~1476819,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:28.694 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.700666+0000", 2024-09-18T11:30:28.694 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.967258363999999, 2024-09-18T11:30:28.694 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002576254, 2024-09-18T11:30:28.694 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:28.694 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:28.695 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:28.695 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:28.695 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:28.695 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1034 2024-09-18T11:30:28.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.695 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:28.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:28.696 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700666+0000", 2024-09-18T11:30:28.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.696 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.696 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:28.696 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700666+0000", 2024-09-18T11:30:28.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.696 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.696 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:28.697 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700665+0000", 2024-09-18T11:30:28.697 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:28.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.697 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:28.697 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700671+0000", 2024-09-18T11:30:28.697 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3839999999999996e-06 2024-09-18T11:30:28.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.698 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:28.698 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700683+0000", 2024-09-18T11:30:28.698 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1912999999999999e-05 2024-09-18T11:30:28.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.698 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.698 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:28.698 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700687+0000", 2024-09-18T11:30:28.698 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6059999999999996e-06 2024-09-18T11:30:28.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.698 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.698 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.698 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700720+0000", 2024-09-18T11:30:28.698 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2642999999999999e-05 2024-09-18T11:30:28.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.699 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.699 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:28.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700779+0000", 2024-09-18T11:30:28.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8992e-05 2024-09-18T11:30:28.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.699 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.699 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:28.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.703242+0000", 2024-09-18T11:30:28.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024635479999999999 2024-09-18T11:30:28.699 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.699 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:28.699 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.700 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1035 4.a 4:588debc2:::smithi11939506-18:head [read 0~3008420,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:28.700 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.700693+0000", 2024-09-18T11:30:28.700 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.967231743999999, 2024-09-18T11:30:28.700 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043653629999999997, 2024-09-18T11:30:28.700 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:28.700 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:28.700 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:28.701 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:28.701 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:28.702 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1035 2024-09-18T11:30:28.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.702 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:28.702 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.703 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:28.703 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700693+0000", 2024-09-18T11:30:28.703 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.703 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.703 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:28.704 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700693+0000", 2024-09-18T11:30:28.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.704 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.704 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:28.704 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700692+0000", 2024-09-18T11:30:28.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:28.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.704 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.704 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:28.704 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700695+0000", 2024-09-18T11:30:28.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3e-06 2024-09-18T11:30:28.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.705 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:28.705 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700698+0000", 2024-09-18T11:30:28.705 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6799999999999999e-06 2024-09-18T11:30:28.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.705 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:28.705 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700704+0000", 2024-09-18T11:30:28.705 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7010000000000001e-06 2024-09-18T11:30:28.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.705 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.703282+0000", 2024-09-18T11:30:28.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002577686 2024-09-18T11:30:28.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:28.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.703363+0000", 2024-09-18T11:30:28.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0962000000000002e-05 2024-09-18T11:30:28.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:28.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.705058+0000", 2024-09-18T11:30:28.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016953179999999999 2024-09-18T11:30:28.706 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.707 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:28.707 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.707 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1036 4.a 4:588debc2:::smithi11939506-18:head [sparse-read 0~1476819] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:28.707 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.700715+0000", 2024-09-18T11:30:28.707 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.967209384, 2024-09-18T11:30:28.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0045971959999999996, 2024-09-18T11:30:28.707 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:28.707 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:28.707 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:28.707 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:28.707 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:28.707 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1036 2024-09-18T11:30:28.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.708 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:28.708 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.708 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:28.708 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700715+0000", 2024-09-18T11:30:28.708 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.708 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.708 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:28.708 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700715+0000", 2024-09-18T11:30:28.708 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.709 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:28.709 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700715+0000", 2024-09-18T11:30:28.709 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:28.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.709 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:28.709 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700717+0000", 2024-09-18T11:30:28.709 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9530000000000002e-06 2024-09-18T11:30:28.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.710 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:28.710 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700721+0000", 2024-09-18T11:30:28.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9729999999999999e-06 2024-09-18T11:30:28.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.710 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:28.710 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700723+0000", 2024-09-18T11:30:28.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.153e-06 2024-09-18T11:30:28.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.710 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.710 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.705107+0000", 2024-09-18T11:30:28.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043840459999999999 2024-09-18T11:30:28.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:28.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.705156+0000", 2024-09-18T11:30:28.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9378000000000001e-05 2024-09-18T11:30:28.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:28.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.705312+0000", 2024-09-18T11:30:28.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015603200000000001 2024-09-18T11:30:28.711 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.711 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:28.711 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.712 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1108 4.a 4:552523b6:::smithi11939506-38:head [read 0~2621859,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:28.712 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.851579+0000", 2024-09-18T11:30:28.712 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.8163455289999995, 2024-09-18T11:30:28.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0033177340000000001, 2024-09-18T11:30:28.712 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:28.712 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:28.712 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:28.712 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:28.712 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:28.712 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1108 2024-09-18T11:30:28.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.712 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:28.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:28.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851579+0000", 2024-09-18T11:30:28.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:28.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851579+0000", 2024-09-18T11:30:28.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:28.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851579+0000", 2024-09-18T11:30:28.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:28.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.714 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:28.714 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851581+0000", 2024-09-18T11:30:28.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.48e-06 2024-09-18T11:30:28.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.714 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:28.714 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851585+0000", 2024-09-18T11:30:28.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0860000000000004e-06 2024-09-18T11:30:28.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.714 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:28.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851589+0000", 2024-09-18T11:30:28.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.968e-06 2024-09-18T11:30:28.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.715 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.853620+0000", 2024-09-18T11:30:28.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020308750000000001 2024-09-18T11:30:28.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.715 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:28.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.853668+0000", 2024-09-18T11:30:28.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8544999999999997e-05 2024-09-18T11:30:28.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.716 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:28.716 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.854897+0000", 2024-09-18T11:30:28.716 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001228125 2024-09-18T11:30:28.716 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.716 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:28.716 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.716 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1109 4.a 4:552523b6:::smithi11939506-38:head [read 0~2621859,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:28.716 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.851616+0000", 2024-09-18T11:30:28.716 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.8163087390000001, 2024-09-18T11:30:28.716 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041528579999999997, 2024-09-18T11:30:28.717 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:28.717 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:28.717 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:28.717 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:28.717 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:28.717 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1109 2024-09-18T11:30:28.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.717 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:28.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.717 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:28.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851616+0000", 2024-09-18T11:30:28.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:28.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851616+0000", 2024-09-18T11:30:28.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:28.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851615+0000", 2024-09-18T11:30:28.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:28.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:28.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851617+0000", 2024-09-18T11:30:28.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0020000000000001e-06 2024-09-18T11:30:28.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:28.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851624+0000", 2024-09-18T11:30:28.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5710000000000003e-06 2024-09-18T11:30:28.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:28.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851631+0000", 2024-09-18T11:30:28.720 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0659999999999998e-06 2024-09-18T11:30:28.720 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.720 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.720 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.720 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.854937+0000", 2024-09-18T11:30:28.720 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003306284 2024-09-18T11:30:28.720 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.720 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.720 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:28.720 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.855040+0000", 2024-09-18T11:30:28.720 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010267099999999999 2024-09-18T11:30:28.720 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.720 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.721 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:28.721 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.855769+0000", 2024-09-18T11:30:28.721 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00072857200000000003 2024-09-18T11:30:28.721 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.721 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:28.721 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.721 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.721 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.721 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1113 4.5 4:a087a566:::smithi11939506-46:head [read 0~2756315,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:28.721 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.851696+0000", 2024-09-18T11:30:28.721 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.816228615, 2024-09-18T11:30:28.721 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028874479999999999, 2024-09-18T11:30:28.721 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:28.721 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:28.722 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:28.722 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:28.722 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:28.722 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1113 2024-09-18T11:30:28.722 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.722 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:28.722 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.722 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:28.722 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851696+0000", 2024-09-18T11:30:28.722 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.722 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.722 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.722 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:28.722 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851696+0000", 2024-09-18T11:30:28.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.723 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:28.723 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851696+0000", 2024-09-18T11:30:28.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:28.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.723 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:28.723 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851698+0000", 2024-09-18T11:30:28.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.979e-06 2024-09-18T11:30:28.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.723 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:28.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851701+0000", 2024-09-18T11:30:28.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9319999999999998e-06 2024-09-18T11:30:28.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:28.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851706+0000", 2024-09-18T11:30:28.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3150000000000002e-06 2024-09-18T11:30:28.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.853523+0000", 2024-09-18T11:30:28.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018174350000000001 2024-09-18T11:30:28.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:28.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.853578+0000", 2024-09-18T11:30:28.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5072999999999999e-05 2024-09-18T11:30:28.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:28.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.854583+0000", 2024-09-18T11:30:28.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001004958 2024-09-18T11:30:28.725 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.725 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:28.725 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.726 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1130 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2605052,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:28.726 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.178984+0000", 2024-09-18T11:30:28.726 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.4889406740000002, 2024-09-18T11:30:28.726 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041647380000000003, 2024-09-18T11:30:28.726 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:28.726 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:28.726 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:28.726 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:28.726 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:28.726 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1130 2024-09-18T11:30:28.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.727 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:28.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.727 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:28.727 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178984+0000", 2024-09-18T11:30:28.727 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.727 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:28.727 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178984+0000", 2024-09-18T11:30:28.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.728 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:28.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178981+0000", 2024-09-18T11:30:28.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:28.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.728 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:28.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178998+0000", 2024-09-18T11:30:28.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7493e-05 2024-09-18T11:30:28.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.729 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:28.729 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179032+0000", 2024-09-18T11:30:28.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3562000000000002e-05 2024-09-18T11:30:28.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.729 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:28.729 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179048+0000", 2024-09-18T11:30:28.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6552999999999999e-05 2024-09-18T11:30:28.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.729 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.729 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179175+0000", 2024-09-18T11:30:28.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012698399999999999 2024-09-18T11:30:28.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.730 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:28.730 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179193+0000", 2024-09-18T11:30:28.730 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7558999999999999e-05 2024-09-18T11:30:28.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.730 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.730 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.181969+0000", 2024-09-18T11:30:28.730 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027756479999999999 2024-09-18T11:30:28.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.730 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:28.730 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.182096+0000", 2024-09-18T11:30:28.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012708700000000001 2024-09-18T11:30:28.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.731 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:28.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.183148+0000", 2024-09-18T11:30:28.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0010528250000000001 2024-09-18T11:30:28.731 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.731 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:28.731 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.731 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1131 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2605052,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:28.731 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179072+0000", 2024-09-18T11:30:28.731 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.488852713, 2024-09-18T11:30:28.732 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050097329999999997, 2024-09-18T11:30:28.732 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:28.732 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:28.732 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:28.732 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:28.732 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:28.732 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1131 2024-09-18T11:30:28.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.733 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:28.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.733 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:28.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179072+0000", 2024-09-18T11:30:28.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.734 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:28.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179072+0000", 2024-09-18T11:30:28.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.735 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:28.735 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179070+0000", 2024-09-18T11:30:28.735 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:28.735 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.735 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.735 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:28.736 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179084+0000", 2024-09-18T11:30:28.736 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3689999999999999e-05 2024-09-18T11:30:28.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.736 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:28.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179105+0000", 2024-09-18T11:30:28.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0605999999999999e-05 2024-09-18T11:30:28.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.737 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:28.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179115+0000", 2024-09-18T11:30:28.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0862999999999999e-05 2024-09-18T11:30:28.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.737 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179365+0000", 2024-09-18T11:30:28.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00024911399999999999 2024-09-18T11:30:28.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:28.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179382+0000", 2024-09-18T11:30:28.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7856999999999999e-05 2024-09-18T11:30:28.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.183178+0000", 2024-09-18T11:30:28.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037957780000000001 2024-09-18T11:30:28.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.739 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.739 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:28.739 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.183256+0000", 2024-09-18T11:30:28.739 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7476000000000002e-05 2024-09-18T11:30:28.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.739 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.739 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:28.739 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184081+0000", 2024-09-18T11:30:28.739 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00082578199999999995 2024-09-18T11:30:28.739 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.739 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:28.739 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.740 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1132 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2605052,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:28.740 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179179+0000", 2024-09-18T11:30:28.740 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.488745368, 2024-09-18T11:30:28.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059816840000000001, 2024-09-18T11:30:28.740 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:28.740 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:28.740 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:28.740 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:28.740 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:28.740 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1132 2024-09-18T11:30:28.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.741 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:28.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.741 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:28.741 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179179+0000", 2024-09-18T11:30:28.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:28.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179179+0000", 2024-09-18T11:30:28.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:28.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179178+0000", 2024-09-18T11:30:28.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:28.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:28.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179193+0000", 2024-09-18T11:30:28.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5446000000000001e-05 2024-09-18T11:30:28.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.744 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:28.744 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179215+0000", 2024-09-18T11:30:28.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1253000000000001e-05 2024-09-18T11:30:28.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.744 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:28.744 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179227+0000", 2024-09-18T11:30:28.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2612999999999999e-05 2024-09-18T11:30:28.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.744 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.744 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179447+0000", 2024-09-18T11:30:28.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021972199999999999 2024-09-18T11:30:28.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:28.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179453+0000", 2024-09-18T11:30:28.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8830000000000003e-06 2024-09-18T11:30:28.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184113+0000", 2024-09-18T11:30:28.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046603180000000001 2024-09-18T11:30:28.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:28.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184229+0000", 2024-09-18T11:30:28.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000115516 2024-09-18T11:30:28.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.746 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:28.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185161+0000", 2024-09-18T11:30:28.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00093216000000000002 2024-09-18T11:30:28.746 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.746 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:28.746 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.746 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:28.746 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:30:28.747 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.9985799709999998, 2024-09-18T11:30:28.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:30:28.747 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:28.747 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:28.747 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:28.747 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:28.747 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:28.747 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:30:28.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.747 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:28.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:28.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:30:28.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:28.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:30:28.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:28.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:30:28.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:28.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:28.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:30:28.749 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:30:28.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.749 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:28.749 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:30:28.749 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:30:28.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.749 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:28.749 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:30:28.749 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:30:28.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:30:28.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:30:28.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:28.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:30:28.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:30:28.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:28.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:30:28.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:30:28.750 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.750 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:28.751 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.751 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1173 4.5 4:a087a566:::smithi11939506-46:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:28.751 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.027238+0000", 2024-09-18T11:30:28.751 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.6406863559999998, 2024-09-18T11:30:28.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038968649999999998, 2024-09-18T11:30:28.751 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:28.751 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:28.751 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:28.751 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:28.751 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:28.751 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1173 2024-09-18T11:30:28.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.752 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:28.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:28.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027238+0000", 2024-09-18T11:30:28.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:28.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027238+0000", 2024-09-18T11:30:28.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:28.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027237+0000", 2024-09-18T11:30:28.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:28.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.753 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:28.753 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027243+0000", 2024-09-18T11:30:28.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5439999999999998e-06 2024-09-18T11:30:28.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.753 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:28.753 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027254+0000", 2024-09-18T11:30:28.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0851e-05 2024-09-18T11:30:28.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.754 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:28.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027260+0000", 2024-09-18T11:30:28.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.212e-06 2024-09-18T11:30:28.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.754 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027289+0000", 2024-09-18T11:30:28.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9499e-05 2024-09-18T11:30:28.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.754 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:28.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027347+0000", 2024-09-18T11:30:28.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8244e-05 2024-09-18T11:30:28.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:28.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.028549+0000", 2024-09-18T11:30:28.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001201543 2024-09-18T11:30:28.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:28.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.029048+0000", 2024-09-18T11:30:28.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00049897600000000004 2024-09-18T11:30:28.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:28.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.031051+0000", 2024-09-18T11:30:28.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020031889999999998 2024-09-18T11:30:28.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:28.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.031096+0000", 2024-09-18T11:30:28.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4959000000000001e-05 2024-09-18T11:30:28.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:28.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.031135+0000", 2024-09-18T11:30:28.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8769000000000001e-05 2024-09-18T11:30:28.756 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.756 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:28.757 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.757 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1185 4.5 4:af4ba75a:::smithi11939506-37:head [write 729437~527542] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:28.757 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.156648+0000", 2024-09-18T11:30:28.757 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.511276047, 2024-09-18T11:30:28.757 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057485540000000003, 2024-09-18T11:30:28.757 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:28.757 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:28.757 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:28.757 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:28.757 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:28.757 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1185 2024-09-18T11:30:28.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.758 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:28.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.758 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:28.758 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.156648+0000", 2024-09-18T11:30:28.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.758 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:28.758 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.156648+0000", 2024-09-18T11:30:28.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.758 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:28.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.156647+0000", 2024-09-18T11:30:28.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:28.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:28.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157166+0000", 2024-09-18T11:30:28.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00051975999999999997 2024-09-18T11:30:28.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:28.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157183+0000", 2024-09-18T11:30:28.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.611e-05 2024-09-18T11:30:28.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.760 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:28.760 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157188+0000", 2024-09-18T11:30:28.760 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.648e-06 2024-09-18T11:30:28.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.760 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.760 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.760 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157253+0000", 2024-09-18T11:30:28.760 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4503000000000002e-05 2024-09-18T11:30:28.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.760 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.760 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:28.760 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157322+0000", 2024-09-18T11:30:28.760 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9031999999999999e-05 2024-09-18T11:30:28.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.761 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:28.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157437+0000", 2024-09-18T11:30:28.761 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011543400000000001 2024-09-18T11:30:28.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.761 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:28.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162104+0000", 2024-09-18T11:30:28.761 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046668009999999999 2024-09-18T11:30:28.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.761 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:28.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162321+0000", 2024-09-18T11:30:28.762 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021706699999999999 2024-09-18T11:30:28.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:28.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162355+0000", 2024-09-18T11:30:28.762 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3670000000000001e-05 2024-09-18T11:30:28.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:28.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162397+0000", 2024-09-18T11:30:28.762 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2168e-05 2024-09-18T11:30:28.762 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.762 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:28.762 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.763 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1186 4.5 4:af4ba75a:::smithi11939506-37:head [write 1925626~643351] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:28.763 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.163566+0000", 2024-09-18T11:30:28.763 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.5043584609999998, 2024-09-18T11:30:28.763 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051394919999999998, 2024-09-18T11:30:28.763 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:28.763 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:28.763 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:28.763 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:28.763 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:28.763 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1186 2024-09-18T11:30:28.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.763 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:28.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:28.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.163566+0000", 2024-09-18T11:30:28.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:28.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.163566+0000", 2024-09-18T11:30:28.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:28.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.163565+0000", 2024-09-18T11:30:28.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:28.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.765 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:28.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164172+0000", 2024-09-18T11:30:28.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060731000000000001 2024-09-18T11:30:28.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.765 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:28.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164184+0000", 2024-09-18T11:30:28.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1202999999999999e-05 2024-09-18T11:30:28.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.765 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:28.766 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164188+0000", 2024-09-18T11:30:28.766 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2810000000000002e-06 2024-09-18T11:30:28.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.766 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.766 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164212+0000", 2024-09-18T11:30:28.766 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4227999999999999e-05 2024-09-18T11:30:28.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.766 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:28.766 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164275+0000", 2024-09-18T11:30:28.766 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3264999999999998e-05 2024-09-18T11:30:28.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.767 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:28.767 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164376+0000", 2024-09-18T11:30:28.767 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000100821 2024-09-18T11:30:28.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.767 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.767 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:28.767 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168331+0000", 2024-09-18T11:30:28.767 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039549779999999996 2024-09-18T11:30:28.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.767 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.767 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:28.767 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168621+0000", 2024-09-18T11:30:28.767 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029003900000000002 2024-09-18T11:30:28.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.768 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:28.768 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168662+0000", 2024-09-18T11:30:28.768 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1307999999999998e-05 2024-09-18T11:30:28.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.768 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:28.768 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168705+0000", 2024-09-18T11:30:28.768 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2994999999999997e-05 2024-09-18T11:30:28.768 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.768 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:28.768 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.769 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1188 4.5 4:af4ba75a:::smithi11939506-37:head [setxattr _header (56) in=7b,truncate 3529437] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:28.769 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.168793+0000", 2024-09-18T11:30:28.769 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.4991312270000003, 2024-09-18T11:30:28.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040182960000000002, 2024-09-18T11:30:28.769 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:28.769 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:28.769 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:28.769 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:28.769 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:28.769 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1188 2024-09-18T11:30:28.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.769 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:28.769 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.769 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:28.770 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168793+0000", 2024-09-18T11:30:28.770 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.770 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:28.770 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168793+0000", 2024-09-18T11:30:28.770 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.770 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:28.770 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168793+0000", 2024-09-18T11:30:28.770 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:28.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:28.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168797+0000", 2024-09-18T11:30:28.771 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5510000000000001e-06 2024-09-18T11:30:28.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:28.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168802+0000", 2024-09-18T11:30:28.771 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.023e-06 2024-09-18T11:30:28.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:28.772 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168806+0000", 2024-09-18T11:30:28.772 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4180000000000001e-06 2024-09-18T11:30:28.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.772 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.772 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.169485+0000", 2024-09-18T11:30:28.772 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00067961699999999998 2024-09-18T11:30:28.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.772 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:28.772 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.169580+0000", 2024-09-18T11:30:28.772 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4827000000000005e-05 2024-09-18T11:30:28.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.773 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.773 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:28.773 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.169713+0000", 2024-09-18T11:30:28.773 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000133285 2024-09-18T11:30:28.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.773 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.773 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:28.773 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.170114+0000", 2024-09-18T11:30:28.773 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000400381 2024-09-18T11:30:28.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.773 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.773 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:28.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.172752+0000", 2024-09-18T11:30:28.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002638793 2024-09-18T11:30:28.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.774 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:28.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.172783+0000", 2024-09-18T11:30:28.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0275999999999999e-05 2024-09-18T11:30:28.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.774 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:28.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.172811+0000", 2024-09-18T11:30:28.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8759999999999999e-05 2024-09-18T11:30:28.774 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.774 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:28.775 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.775 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1189 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:28.775 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.168812+0000", 2024-09-18T11:30:28.775 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.4991125759999999, 2024-09-18T11:30:28.775 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004129377, 2024-09-18T11:30:28.775 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:28.775 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:28.775 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:28.775 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:28.775 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:28.775 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1189 2024-09-18T11:30:28.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.776 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:28.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.776 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:28.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168812+0000", 2024-09-18T11:30:28.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.776 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:28.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168812+0000", 2024-09-18T11:30:28.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.776 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:28.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168812+0000", 2024-09-18T11:30:28.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:28.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:28.777 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168814+0000", 2024-09-18T11:30:28.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1399999999999998e-06 2024-09-18T11:30:28.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:28.777 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168820+0000", 2024-09-18T11:30:28.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6320000000000002e-06 2024-09-18T11:30:28.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.778 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:28.778 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168823+0000", 2024-09-18T11:30:28.778 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3599999999999999e-06 2024-09-18T11:30:28.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.169903+0000", 2024-09-18T11:30:28.780 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001080136 2024-09-18T11:30:28.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.781 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:28.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.169978+0000", 2024-09-18T11:30:28.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5298999999999999e-05 2024-09-18T11:30:28.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.172837+0000", 2024-09-18T11:30:28.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028588620000000002 2024-09-18T11:30:28.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:28.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.172884+0000", 2024-09-18T11:30:28.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6767999999999997e-05 2024-09-18T11:30:28.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.783 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:28.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.172941+0000", 2024-09-18T11:30:28.783 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7515000000000001e-05 2024-09-18T11:30:28.783 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.783 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:28.784 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.784 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1372 4.a 4:588debc2:::smithi11939506-18:head [read 0~3008420,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-18T11:30:28.784 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:24.316548+0000", 2024-09-18T11:30:28.784 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.3513767290000001, 2024-09-18T11:30:28.784 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023069980000000002, 2024-09-18T11:30:28.784 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:28.784 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1372 2024-09-18T11:30:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:24.316548+0000", 2024-09-18T11:30:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:24.316548+0000", 2024-09-18T11:30:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:24.316547+0000", 2024-09-18T11:30:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:24.316550+0000", 2024-09-18T11:30:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3579999999999999e-06 2024-09-18T11:30:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:24.316556+0000", 2024-09-18T11:30:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7810000000000002e-06 2024-09-18T11:30:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:24.316561+0000", 2024-09-18T11:30:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1769999999999997e-06 2024-09-18T11:30:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:24.318024+0000", 2024-09-18T11:30:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001463057 2024-09-18T11:30:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:24.318133+0000", 2024-09-18T11:30:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000108961 2024-09-18T11:30:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:24.318855+0000", 2024-09-18T11:30:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00072140699999999997 2024-09-18T11:30:28.789 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.789 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:28.789 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.789 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:28.789 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:28.789 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:28.790 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:30:28.791 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:28.789+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:28.794 INFO:tasks.rados.rados.0.smithi119.stdout:823: finishing write tid 1 to smithi11939506-4 2024-09-18T11:30:28.794 INFO:tasks.rados.rados.0.smithi119.stdout:823: finishing write tid 2 to smithi11939506-4 2024-09-18T11:30:28.794 INFO:tasks.rados.rados.0.smithi119.stdout:823: finishing write tid 3 to smithi11939506-4 2024-09-18T11:30:28.794 INFO:tasks.rados.rados.0.smithi119.stdout:823: finishing write tid 4 to smithi11939506-4 2024-09-18T11:30:28.794 INFO:tasks.rados.rados.0.smithi119.stdout:823: finishing write tid 5 to smithi11939506-4 2024-09-18T11:30:28.794 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 4 v 23 (ObjNum 110 snap 8 seq_num 110) dirty exists 2024-09-18T11:30:28.795 INFO:tasks.rados.rados.0.smithi119.stdout:823: left oid 4 (ObjNum 110 snap 8 seq_num 110) 2024-09-18T11:30:28.889 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:28.889 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:28.889 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:28.889 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:28.891 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:28.889+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:28.898 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:30:28.992 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:28.991+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:29.036 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:29.036 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:29.036 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:29.037 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:29.037 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:29.045 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:30:29.092 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:29.091+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:29.186 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:29.186 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:29.186 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:29.186 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:29.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.186 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:41 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:29.186 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157303+0000", 2024-09-18T11:30:29.186 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.026216021000003, 2024-09-18T11:30:29.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054616040000000001, 2024-09-18T11:30:29.187 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:29.187 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.187 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:29.187 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:29.187 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:29.187 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 41 2024-09-18T11:30:29.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.187 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:29.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:29.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157303+0000", 2024-09-18T11:30:29.188 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.188 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.188 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:29.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157303+0000", 2024-09-18T11:30:29.188 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.188 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.188 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:29.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157303+0000", 2024-09-18T11:30:29.188 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:29.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.188 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.189 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:29.189 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157388+0000", 2024-09-18T11:30:29.189 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4938999999999994e-05 2024-09-18T11:30:29.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.189 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.189 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:29.189 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157395+0000", 2024-09-18T11:30:29.189 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1130000000000003e-06 2024-09-18T11:30:29.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.189 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.189 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.189 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157405+0000", 2024-09-18T11:30:29.189 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7510000000000003e-06 2024-09-18T11:30:29.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.190 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.190 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:29.191 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158213+0000", 2024-09-18T11:30:29.191 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00080873000000000002 2024-09-18T11:30:29.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.192 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.192 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:29.192 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158267+0000", 2024-09-18T11:30:29.192 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3709999999999999e-05 2024-09-18T11:30:29.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.192 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.192 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:29.192 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158352+0000", 2024-09-18T11:30:29.192 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4782000000000002e-05 2024-09-18T11:30:29.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.192 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.192 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:29.192 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160262+0000", 2024-09-18T11:30:29.193 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001910068 2024-09-18T11:30:29.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.193 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.193 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.193 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162672+0000", 2024-09-18T11:30:29.193 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024101159999999999 2024-09-18T11:30:29.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.193 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.193 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:29.193 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162724+0000", 2024-09-18T11:30:29.193 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1637e-05 2024-09-18T11:30:29.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.193 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.194 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:29.194 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162765+0000", 2024-09-18T11:30:29.194 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1282000000000002e-05 2024-09-18T11:30:29.194 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.194 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:29.194 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.194 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.194 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.194 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:29.194 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:29.194 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.026106308000003, 2024-09-18T11:30:29.194 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054864400000000004, 2024-09-18T11:30:29.194 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:29.194 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.195 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:29.195 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:29.195 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:29.195 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 42 2024-09-18T11:30:29.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.195 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:29.195 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.195 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:29.195 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:29.195 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.195 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.195 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:29.196 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:29.196 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.196 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.196 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.196 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:29.196 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:29.196 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:29.196 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.196 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.196 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:29.196 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157416+0000", 2024-09-18T11:30:29.196 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.721e-06 2024-09-18T11:30:29.196 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.196 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.197 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:29.197 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157423+0000", 2024-09-18T11:30:29.197 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4049999999999997e-06 2024-09-18T11:30:29.197 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.197 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.197 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.197 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157425+0000", 2024-09-18T11:30:29.197 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.481e-06 2024-09-18T11:30:29.197 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.197 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.197 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:29.197 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158649+0000", 2024-09-18T11:30:29.197 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001223453 2024-09-18T11:30:29.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.198 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.198 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:29.198 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158729+0000", 2024-09-18T11:30:29.198 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0205999999999997e-05 2024-09-18T11:30:29.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.198 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.198 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:29.198 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158875+0000", 2024-09-18T11:30:29.198 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014628800000000001 2024-09-18T11:30:29.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.198 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.198 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:29.198 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161518+0000", 2024-09-18T11:30:29.199 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026427949999999999 2024-09-18T11:30:29.199 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.199 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.199 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.199 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162821+0000", 2024-09-18T11:30:29.199 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013028099999999999 2024-09-18T11:30:29.199 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.199 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.199 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:29.199 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162864+0000", 2024-09-18T11:30:29.199 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2839000000000001e-05 2024-09-18T11:30:29.199 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.199 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.200 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:29.200 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162900+0000", 2024-09-18T11:30:29.200 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5948999999999998e-05 2024-09-18T11:30:29.200 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.200 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:29.200 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.200 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.200 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.200 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:29.200 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:29.200 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.026019943999998, 2024-09-18T11:30:29.200 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055400850000000001, 2024-09-18T11:30:29.200 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:29.201 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.201 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:29.201 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:29.201 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:29.201 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 43 2024-09-18T11:30:29.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.201 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:29.201 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.201 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:29.201 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:29.201 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.201 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.201 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:29.202 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:29.202 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.202 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.202 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:29.202 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:29.202 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:29.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.202 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.202 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:29.202 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157504+0000", 2024-09-18T11:30:29.202 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4519999999999996e-06 2024-09-18T11:30:29.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.203 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.203 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:29.203 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157534+0000", 2024-09-18T11:30:29.203 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.97e-05 2024-09-18T11:30:29.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.203 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.203 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.203 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157539+0000", 2024-09-18T11:30:29.203 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4360000000000001e-06 2024-09-18T11:30:29.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.203 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.203 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:29.203 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159428+0000", 2024-09-18T11:30:29.203 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018888220000000001 2024-09-18T11:30:29.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.204 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.204 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:29.204 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159479+0000", 2024-09-18T11:30:29.204 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0608000000000001e-05 2024-09-18T11:30:29.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.204 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.204 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:29.204 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159559+0000", 2024-09-18T11:30:29.204 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0667000000000005e-05 2024-09-18T11:30:29.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.204 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.205 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:29.205 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162779+0000", 2024-09-18T11:30:29.205 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032197459999999999 2024-09-18T11:30:29.205 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.205 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.205 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.205 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162950+0000", 2024-09-18T11:30:29.205 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017059599999999999 2024-09-18T11:30:29.205 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.205 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.205 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:29.205 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163001+0000", 2024-09-18T11:30:29.205 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0968999999999999e-05 2024-09-18T11:30:29.206 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.206 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.206 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:29.206 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163040+0000", 2024-09-18T11:30:29.206 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8843000000000001e-05 2024-09-18T11:30:29.206 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.206 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:29.206 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.206 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.206 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.206 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:29.206 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:29.206 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.02596879, 2024-09-18T11:30:29.206 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:30:29.207 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:29.207 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.207 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:29.207 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:29.207 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:29.207 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:30:29.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.207 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:29.207 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.207 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:29.207 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:29.207 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.207 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.208 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:29.208 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:29.208 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.208 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.208 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.208 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:29.208 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:30:29.208 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:29.208 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.208 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.208 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:29.208 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:30:29.208 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:30:29.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.209 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.209 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:29.209 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:30:29.209 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:30:29.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.209 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.209 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.209 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:30:29.209 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:30:29.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.210 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.210 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:29.210 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:30:29.210 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:30:29.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.210 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.210 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:29.210 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:30:29.210 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:30:29.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.210 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.210 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:29.210 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:30:29.210 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:30:29.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.211 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.211 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:29.211 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:30:29.211 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:30:29.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.211 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.211 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.211 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:30:29.211 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:30:29.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.211 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.211 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:29.211 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:30:29.212 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:30:29.212 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.212 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.212 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:29.212 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:30:29.212 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:30:29.212 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.212 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:29.212 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.212 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.212 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.212 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:29.212 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:29.213 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.025842527999998, 2024-09-18T11:30:29.213 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055717040000000002, 2024-09-18T11:30:29.213 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:29.213 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.213 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:29.213 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:29.213 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:29.213 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 45 2024-09-18T11:30:29.213 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.213 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:29.213 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.213 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:29.213 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:29.213 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.214 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.214 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:29.214 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:29.214 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.214 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.214 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:29.214 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157676+0000", 2024-09-18T11:30:29.214 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:29.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.214 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.214 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:29.214 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157694+0000", 2024-09-18T11:30:29.215 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7422999999999999e-05 2024-09-18T11:30:29.215 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.215 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.215 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:29.215 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157704+0000", 2024-09-18T11:30:29.215 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9000000000000001e-06 2024-09-18T11:30:29.215 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.215 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.215 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.215 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157718+0000", 2024-09-18T11:30:29.215 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4788e-05 2024-09-18T11:30:29.215 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.215 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.215 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:29.216 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160937+0000", 2024-09-18T11:30:29.216 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00321886 2024-09-18T11:30:29.216 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.216 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.216 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:29.216 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161023+0000", 2024-09-18T11:30:29.216 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6063999999999995e-05 2024-09-18T11:30:29.216 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.216 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.216 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:29.216 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161202+0000", 2024-09-18T11:30:29.216 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017912599999999999 2024-09-18T11:30:29.216 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.217 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.217 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:29.217 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163052+0000", 2024-09-18T11:30:29.217 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001849554 2024-09-18T11:30:29.217 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.217 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.217 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.217 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163204+0000", 2024-09-18T11:30:29.217 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015186900000000001 2024-09-18T11:30:29.217 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.217 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.217 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:29.217 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163229+0000", 2024-09-18T11:30:29.217 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:30:29.218 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.218 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.218 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:29.218 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163249+0000", 2024-09-18T11:30:29.218 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9992000000000001e-05 2024-09-18T11:30:29.218 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.218 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:29.218 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.218 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.218 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.218 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:29.218 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:29.218 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.025793415000003, 2024-09-18T11:30:29.218 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:30:29.219 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:29.219 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.219 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:29.219 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:29.219 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:29.219 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:30:29.219 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.219 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:29.219 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.219 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:29.219 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:29.219 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.219 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.219 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.220 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:29.220 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:29.220 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.220 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.220 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.220 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:29.220 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:29.220 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:29.220 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.220 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.220 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:29.220 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:30:29.220 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:30:29.221 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.221 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.221 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:29.221 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:30:29.221 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:30:29.221 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.221 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.221 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.221 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:30:29.221 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:30:29.221 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.221 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.221 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:29.222 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:30:29.222 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:30:29.222 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.222 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.222 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:29.222 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:30:29.222 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:30:29.222 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.222 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.222 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:29.222 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:30:29.222 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:30:29.223 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.223 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.223 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:29.223 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:30:29.223 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:30:29.223 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.223 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.223 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.223 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:30:29.223 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:30:29.223 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.223 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.223 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:29.223 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:30:29.224 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:30:29.224 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.224 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.224 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:29.224 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:30:29.224 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:30:29.224 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.224 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:29.224 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.224 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.224 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.224 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:29.224 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:29.225 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.025750883999997, 2024-09-18T11:30:29.225 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:30:29.225 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:29.225 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.225 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:29.225 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:29.225 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:29.225 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:30:29.225 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.225 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:29.225 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.225 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:29.225 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:29.225 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.226 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.226 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.226 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:29.226 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:29.226 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.226 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.226 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.226 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:29.226 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:30:29.226 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:29.226 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.226 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.226 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:29.227 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:30:29.227 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:30:29.227 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.227 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.227 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:29.227 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:30:29.227 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:30:29.227 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.227 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.227 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.227 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:30:29.227 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:30:29.227 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.227 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.228 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:29.228 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:30:29.228 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:30:29.228 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.228 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.228 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:29.228 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:30:29.228 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:30:29.228 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.228 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.228 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:29.228 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:30:29.228 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:30:29.228 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.229 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.229 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.229 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:30:29.229 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:30:29.229 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.229 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.229 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:29.229 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:30:29.229 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:30:29.229 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.229 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.229 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:29.229 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:30:29.230 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:30:29.230 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.230 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.230 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:29.230 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:30:29.230 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:30:29.230 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.230 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:29.230 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.230 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.230 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.230 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:29.230 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:29.230 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.025695954, 2024-09-18T11:30:29.231 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:30:29.231 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:29.231 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.231 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:29.231 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:29.231 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:29.231 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:30:29.231 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.231 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:29.231 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.231 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:29.231 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:29.231 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.231 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.232 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.232 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:29.232 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:29.232 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.232 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.232 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.232 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:29.232 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:29.232 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:29.232 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.232 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.232 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:29.232 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:30:29.233 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:30:29.233 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.233 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.233 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:29.233 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:30:29.233 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:30:29.233 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.233 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.233 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.233 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:30:29.233 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:30:29.233 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.233 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.233 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:29.234 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:30:29.234 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:30:29.234 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.234 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.234 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:29.234 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:30:29.234 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:30:29.234 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.234 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.234 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:29.234 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:30:29.234 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:30:29.234 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.234 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.235 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:29.235 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:30:29.235 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:30:29.235 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.235 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.235 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.235 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:30:29.235 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:30:29.235 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.235 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.235 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:29.235 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:30:29.235 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:30:29.236 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.236 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.236 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:29.236 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:30:29.236 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:30:29.236 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.236 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:29.236 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.236 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.236 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.236 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:29.236 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:29.236 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.025685973999998, 2024-09-18T11:30:29.236 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:30:29.237 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:29.237 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.237 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:29.237 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:29.237 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:29.237 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:30:29.237 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.237 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:29.237 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.237 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:29.237 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:29.237 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.237 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.238 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.238 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:29.238 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:29.238 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.238 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.238 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.238 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:29.238 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:29.238 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:29.238 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.238 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.238 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:29.238 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:30:29.239 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:30:29.239 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.239 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.239 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:29.239 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:30:29.239 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:30:29.239 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.239 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.239 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.239 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:30:29.239 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:30:29.240 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.240 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.240 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:29.240 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:30:29.240 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:30:29.240 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.240 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.240 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:29.240 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:30:29.240 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:30:29.240 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.241 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.241 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:29.241 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:30:29.241 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:30:29.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.241 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.241 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.241 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:30:29.241 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:30:29.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.241 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.241 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:29.241 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:30:29.242 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:30:29.242 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.242 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.242 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:29.242 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:30:29.242 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:30:29.242 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.242 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.242 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:29.243 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:30:29.243 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:30:29.243 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.243 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:29.243 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.244 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.244 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:29.244 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:29.244 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.025674375000001, 2024-09-18T11:30:29.244 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:30:29.244 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:29.244 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.245 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:29.245 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:29.245 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:29.245 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:30:29.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.245 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:29.245 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.245 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:29.246 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:29.246 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.246 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.246 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:29.246 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:29.246 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.247 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.247 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:29.247 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:29.247 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:29.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.247 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.247 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:29.247 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:30:29.247 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:30:29.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.248 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.248 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:29.248 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:30:29.248 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:30:29.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.248 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.248 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.248 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:30:29.249 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:30:29.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.249 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.249 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:29.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:30:29.249 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:30:29.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.249 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.249 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:29.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:30:29.249 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:30:29.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.249 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.250 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:29.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:30:29.250 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:30:29.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.250 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.250 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:29.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:30:29.250 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:30:29.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.250 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.250 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:30:29.250 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:30:29.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.251 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.251 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:29.251 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:30:29.251 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:30:29.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.251 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.251 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:29.251 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:30:29.251 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:30:29.251 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.251 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:29.251 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.252 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.252 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:29.252 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:29.252 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.025617894, 2024-09-18T11:30:29.252 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:30:29.252 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:29.252 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.252 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:29.252 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:29.252 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:29.252 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:30:29.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.253 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:29.253 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.253 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:29.253 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:29.253 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.253 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.253 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:29.253 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:29.253 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.253 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.253 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:29.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:30:29.254 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:29.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.254 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.254 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:29.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:30:29.254 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:30:29.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.254 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.254 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:29.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:30:29.254 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:30:29.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.255 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.255 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:30:29.255 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:30:29.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.255 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.255 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:29.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:30:29.255 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:30:29.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.255 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.255 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:29.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:30:29.256 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:30:29.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.256 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.256 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:29.256 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:30:29.256 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:30:29.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.256 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.256 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.256 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:30:29.256 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:30:29.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.257 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.257 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:29.257 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:30:29.257 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:30:29.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.257 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.257 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:29.257 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:30:29.257 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:30:29.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.257 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.257 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:29.257 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:30:29.258 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:30:29.258 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.258 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:29.258 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.258 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.258 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:29.258 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:29.258 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.02560407, 2024-09-18T11:30:29.258 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:30:29.258 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:29.258 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.258 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:29.258 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:29.259 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:29.259 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:30:29.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.259 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:29.259 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.259 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:29.259 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:29.259 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.259 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.259 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:29.259 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:29.259 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.260 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.260 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:29.260 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:29.260 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:29.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.260 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.260 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:29.260 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:30:29.260 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:30:29.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.260 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.260 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:29.260 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:30:29.261 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:30:29.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.261 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.261 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.261 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:30:29.261 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:30:29.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.261 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.261 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:29.261 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:30:29.261 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:30:29.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.261 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.261 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:29.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:30:29.262 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:30:29.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.262 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.262 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:29.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:30:29.262 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:30:29.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.262 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.262 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:30:29.262 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:30:29.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.263 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.263 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:29.263 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:30:29.263 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:30:29.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.263 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.263 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:29.263 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:30:29.263 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:30:29.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.263 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.263 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:29.264 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:30:29.264 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:30:29.264 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.264 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:29.264 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.264 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.264 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:29.264 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:29.264 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 39.609425162999997, 2024-09-18T11:30:29.264 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:30:29.264 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:29.265 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.265 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:29.265 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:29.265 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:29.265 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:30:29.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.265 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:29.265 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.265 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:29.265 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:29.265 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.265 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.265 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:29.266 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:29.266 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.266 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.266 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:29.266 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:29.266 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:29.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.266 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.266 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:29.266 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:30:29.267 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:30:29.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.267 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.267 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:29.267 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:30:29.267 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:30:29.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.267 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.267 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.267 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:30:29.267 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:30:29.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.267 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.267 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:29.268 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:30:29.268 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:30:29.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.268 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.268 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:29.268 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:30:29.268 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:30:29.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.268 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.268 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:29.268 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:30:29.268 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:30:29.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.269 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.269 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:29.269 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:30:29.269 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:30:29.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.269 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.269 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.269 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:30:29.270 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:30:29.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.270 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.270 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:29.270 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:30:29.270 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:30:29.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.271 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.271 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:29.271 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:30:29.271 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:30:29.271 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.271 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:29.271 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.272 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.272 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:29.272 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:29.272 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 39.609401863000002, 2024-09-18T11:30:29.272 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:30:29.272 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:29.272 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.273 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:29.273 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:29.273 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:29.273 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:30:29.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.273 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:29.273 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.273 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:29.273 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:29.273 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.273 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.273 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:29.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:29.274 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.274 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.274 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:29.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:30:29.274 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:29.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.274 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.274 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:29.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:30:29.274 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:30:29.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.275 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.275 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:29.275 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:30:29.275 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:30:29.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.275 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.275 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.275 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:30:29.275 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:30:29.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.275 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.275 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:29.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:30:29.276 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:30:29.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.276 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.276 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:29.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:30:29.276 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:30:29.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.276 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.276 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:29.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:30:29.276 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:30:29.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.277 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:29.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:30:29.277 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:30:29.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.277 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:30:29.277 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:30:29.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.277 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:29.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:30:29.277 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:30:29.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.278 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.278 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:29.278 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:30:29.278 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:30:29.278 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.278 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:29.278 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.278 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.278 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:29.279 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:29.279 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 39.609316253999999, 2024-09-18T11:30:29.279 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:30:29.279 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:29.279 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.279 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:29.279 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:29.279 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:29.279 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:30:29.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.279 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:29.279 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.279 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:29.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:29.280 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.280 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:29.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:29.280 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.280 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:29.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:29.280 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:29.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.281 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:29.281 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:30:29.281 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:30:29.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.281 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:29.281 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:30:29.281 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:30:29.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:30:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:30:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:30:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:30:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:30:29.282 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:30:29.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.283 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:29.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:30:29.283 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:30:29.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.284 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.284 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:29.284 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:30:29.284 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:30:29.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.284 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.284 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.284 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:30:29.284 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:30:29.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.284 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.285 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:29.285 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:30:29.285 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:30:29.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.285 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:29.285 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:30:29.285 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:30:29.286 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.286 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:29.286 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.286 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.287 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:29.287 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:29.287 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 39.609305288999998, 2024-09-18T11:30:29.287 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:30:29.287 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:29.287 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.288 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:29.288 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:29.288 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:29.288 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:30:29.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.288 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:29.288 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.289 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:29.289 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:29.289 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.289 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.289 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:29.289 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:29.290 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.290 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.290 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:29.290 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:29.290 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:29.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.291 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:29.291 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:30:29.291 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:30:29.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.291 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:29.291 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:30:29.291 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:30:29.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.291 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.291 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:30:29.292 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:30:29.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.292 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.292 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:29.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:30:29.292 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:30:29.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.292 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.292 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:29.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:30:29.292 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:30:29.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.293 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:29.293 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:30:29.293 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:30:29.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.293 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:29.293 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:30:29.293 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:30:29.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.293 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:30:29.294 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:30:29.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.294 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:29.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:30:29.294 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:30:29.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.294 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:29.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:30:29.294 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:30:29.294 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.294 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:29.295 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.295 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:29.295 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:29.295 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 39.609294120999998, 2024-09-18T11:30:29.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:30:29.295 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:29.295 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.295 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:29.295 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:29.296 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:29.296 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:30:29.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.296 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:29.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.296 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:29.296 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:29.296 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.296 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:29.296 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:29.296 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.297 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:29.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:29.297 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:29.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.297 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:29.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:30:29.297 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:30:29.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.297 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:29.298 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:30:29.298 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:30:29.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.298 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.298 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:30:29.298 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:30:29.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.298 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:29.298 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:30:29.298 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:30:29.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.299 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:29.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:30:29.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:30:29.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.299 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:29.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:30:29.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:30:29.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.299 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:30:29.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:30:29.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:29.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:30:29.300 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:30:29.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:29.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:30:29.300 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:30:29.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:29.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:30:29.301 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:30:29.301 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.301 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:29.301 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.301 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:29.301 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:29.301 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 39.609228792000003, 2024-09-18T11:30:29.301 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:30:29.301 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:29.302 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.302 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:29.302 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:29.302 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:29.302 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:30:29.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.302 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:29.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.302 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:29.302 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:29.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.303 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:29.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:29.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.303 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:29.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:30:29.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:29.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.303 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:29.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:30:29.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:30:29.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.304 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:29.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:30:29.304 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:30:29.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.304 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:30:29.304 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:30:29.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:29.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:30:29.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:30:29.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:29.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:30:29.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:30:29.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:29.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:30:29.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:30:29.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:30:29.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:30:29.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:29.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:30:29.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:30:29.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:29.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:30:29.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:30:29.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:29.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:30:29.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:30:29.307 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.307 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:29.307 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.307 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:29.308 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:29.308 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 39.599066813999997, 2024-09-18T11:30:29.308 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:30:29.308 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:29.308 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.308 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:29.308 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:29.308 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:29.308 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:30:29.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.308 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:29.308 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.308 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:29.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:29.309 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.309 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:29.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:29.309 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.309 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:29.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:30:29.309 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:29.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:29.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:30:29.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:30:29.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:29.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:30:29.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:30:29.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:30:29.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:30:29.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:29.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:30:29.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:30:29.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:29.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:30:29.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:30:29.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.312 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:29.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:30:29.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:30:29.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.312 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:29.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:30:29.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:30:29.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.313 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:30:29.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:30:29.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.313 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:29.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:30:29.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:30:29.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.313 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:29.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:30:29.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:30:29.314 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.314 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:29.314 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.314 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:29.314 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:29.314 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 39.598974335000001, 2024-09-18T11:30:29.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:30:29.314 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:29.314 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.314 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:29.314 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:29.314 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:29.315 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:30:29.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.315 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:29.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:29.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:29.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:29.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:29.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:29.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:29.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:29.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:29.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:29.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:30:29.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:30:29.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:29.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:30:29.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:30:29.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:30:29.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:30:29.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:29.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:30:29.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:30:29.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:29.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:30:29.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:30:29.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:29.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:30:29.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:30:29.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:29.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:30:29.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:30:29.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:30:29.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:30:29.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:29.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:30:29.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:30:29.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:29.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:29.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:29.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:30:29.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:30:29.320 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.320 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:29.320 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.320 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:29.320 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:29.320 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:29.321 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T11:30:29.323 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:30:29.324 INFO:tasks.rados.rados.0.smithi119.stdout:817: done (7 left) 2024-09-18T11:30:29.324 INFO:tasks.rados.rados.0.smithi119.stdout:818: done (6 left) 2024-09-18T11:30:29.324 INFO:tasks.rados.rados.0.smithi119.stdout:819: done (5 left) 2024-09-18T11:30:29.324 INFO:tasks.rados.rados.0.smithi119.stdout:820: done (4 left) 2024-09-18T11:30:29.324 INFO:tasks.rados.rados.0.smithi119.stdout:821: done (3 left) 2024-09-18T11:30:29.324 INFO:tasks.rados.rados.0.smithi119.stdout:822: done (2 left) 2024-09-18T11:30:29.324 INFO:tasks.rados.rados.0.smithi119.stdout:823: done (1 left) 2024-09-18T11:30:29.324 INFO:tasks.rados.rados.0.smithi119.stdout:825: done (0 left) 2024-09-18T11:30:29.324 INFO:tasks.rados.rados.0.smithi119.stdout:826: read oid 13 snap 6 2024-09-18T11:30:29.324 INFO:tasks.rados.rados.0.smithi119.stdout:826: expect (ObjNum 74 snap 3 seq_num 74) 2024-09-18T11:30:29.325 INFO:tasks.rados.rados.0.smithi119.stdout:827: write oid 20 current snap is 9 2024-09-18T11:30:29.325 INFO:tasks.rados.rados.0.smithi119.stdout:827: seq_num 111 ranges {665577=781581,2062560=563688,3182305=643664} 2024-09-18T11:30:29.325 INFO:tasks.rados.rados.0.smithi119.stdout:827: writing smithi11939506-20 from 665577 to 1447158 tid 1 2024-09-18T11:30:29.325 INFO:tasks.rados.rados.0.smithi119.stdout:827: writing smithi11939506-20 from 2062560 to 2626248 tid 2 2024-09-18T11:30:29.325 INFO:tasks.rados.rados.0.smithi119.stdout:827: writing smithi11939506-20 from 3182305 to 3825969 tid 3 2024-09-18T11:30:29.325 INFO:tasks.rados.rados.0.smithi119.stdout:828: delete oid 33 current snap is 9 2024-09-18T11:30:29.325 INFO:tasks.rados.rados.0.smithi119.stdout:826: expect (ObjNum 74 snap 3 seq_num 74) 2024-09-18T11:30:29.325 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:29.320+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:29.403 INFO:tasks.rados.rados.0.smithi119.stdout:827: finishing write tid 1 to smithi11939506-20 2024-09-18T11:30:29.403 INFO:tasks.rados.rados.0.smithi119.stdout:827: finishing write tid 2 to smithi11939506-20 2024-09-18T11:30:29.403 INFO:tasks.rados.rados.0.smithi119.stdout:827: finishing write tid 3 to smithi11939506-20 2024-09-18T11:30:29.403 INFO:tasks.rados.rados.0.smithi119.stdout:827: finishing write tid 4 to smithi11939506-20 2024-09-18T11:30:29.403 INFO:tasks.rados.rados.0.smithi119.stdout:826: done (2 left) 2024-09-18T11:30:29.403 INFO:tasks.rados.rados.0.smithi119.stdout:828: done (1 left) 2024-09-18T11:30:29.404 INFO:tasks.rados.rados.0.smithi119.stdout:829: rollback oid 46 current snap is 9 2024-09-18T11:30:29.404 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 46 to 6 2024-09-18T11:30:29.404 INFO:tasks.rados.rados.0.smithi119.stdout:827: finishing write tid 5 to smithi11939506-20 2024-09-18T11:30:29.404 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 20 v 66 (ObjNum 111 snap 9 seq_num 111) dirty exists 2024-09-18T11:30:29.404 INFO:tasks.rados.rados.0.smithi119.stdout:827: left oid 20 (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:30:29.404 INFO:tasks.rados.rados.0.smithi119.stdout:827: done (1 left) 2024-09-18T11:30:29.404 INFO:tasks.rados.rados.0.smithi119.stdout:830: read oid 33 snap -1 2024-09-18T11:30:29.404 INFO:tasks.rados.rados.0.smithi119.stdout:830: expect deleted 2024-09-18T11:30:29.404 INFO:tasks.rados.rados.0.smithi119.stdout:831: read oid 43 snap 8 2024-09-18T11:30:29.404 INFO:tasks.rados.rados.0.smithi119.stdout:831: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:30:29.404 INFO:tasks.rados.rados.0.smithi119.stdout:832: snap_remove snap 7 2024-09-18T11:30:29.422 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:29.421+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:29.426 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:29.426 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:29.426 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:29.426 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:29.438 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:30:29.524 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:29.523+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:29.577 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:29.578 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:29.578 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:29.578 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:29.578 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:29.586 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:30:29.627 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:29.626+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:29.724 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:79 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'57, mlcod=15'53)", 2024-09-18T11:30:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.586176+0000", 2024-09-18T11:30:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.135967858000001, 2024-09-18T11:30:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043147059999999997, 2024-09-18T11:30:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:29.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586176+0000", 2024-09-18T11:30:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586176+0000", 2024-09-18T11:30:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586170+0000", 2024-09-18T11:30:29.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999938 2024-09-18T11:30:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586282+0000", 2024-09-18T11:30:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112254 2024-09-18T11:30:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586307+0000", 2024-09-18T11:30:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5296e-05 2024-09-18T11:30:29.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586321+0000", 2024-09-18T11:30:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3723000000000001e-05 2024-09-18T11:30:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.587754+0000", 2024-09-18T11:30:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014330829999999999 2024-09-18T11:30:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:29.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.587785+0000", 2024-09-18T11:30:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1176000000000001e-05 2024-09-18T11:30:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.590467+0000", 2024-09-18T11:30:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026815089999999999 2024-09-18T11:30:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.590491+0000", 2024-09-18T11:30:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3941e-05 2024-09-18T11:30:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.729 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:21 3.1 3:9f1cb8cf:::smithi11939506-38 low_tier:head [read 336384~173568 [fadvise_sequential]] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.118534+0000", 2024-09-18T11:30:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.603609667000001, 2024-09-18T11:30:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006633842, 2024-09-18T11:30:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:29.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118534+0000", 2024-09-18T11:30:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118534+0000", 2024-09-18T11:30:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118532+0000", 2024-09-18T11:30:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118542+0000", 2024-09-18T11:30:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0151e-05 2024-09-18T11:30:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118563+0000", 2024-09-18T11:30:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1089000000000001e-05 2024-09-18T11:30:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118572+0000", 2024-09-18T11:30:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1090000000000004e-06 2024-09-18T11:30:29.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118633+0000", 2024-09-18T11:30:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0940000000000003e-05 2024-09-18T11:30:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118640+0000", 2024-09-18T11:30:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3690000000000003e-06 2024-09-18T11:30:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124910+0000", 2024-09-18T11:30:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062697569999999999 2024-09-18T11:30:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124980+0000", 2024-09-18T11:30:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0846000000000006e-05 2024-09-18T11:30:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.125168+0000", 2024-09-18T11:30:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018786200000000001 2024-09-18T11:30:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:29.734 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:22 3.1 3:9f1cb8cf:::smithi11939506-38 low_tier:head [read 1235456~1340416 [fadvise_sequential]] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.603556939000001, 2024-09-18T11:30:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0090773199999999998, 2024-09-18T11:30:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:29.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118594+0000", 2024-09-18T11:30:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1590000000000004e-06 2024-09-18T11:30:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118601+0000", 2024-09-18T11:30:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3900000000000004e-06 2024-09-18T11:30:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118605+0000", 2024-09-18T11:30:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2479999999999998e-06 2024-09-18T11:30:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118697+0000", 2024-09-18T11:30:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1447999999999996e-05 2024-09-18T11:30:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118702+0000", 2024-09-18T11:30:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6670000000000001e-06 2024-09-18T11:30:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.125285+0000", 2024-09-18T11:30:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006582124 2024-09-18T11:30:29.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.125349+0000", 2024-09-18T11:30:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4777000000000007e-05 2024-09-18T11:30:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.127664+0000", 2024-09-18T11:30:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023151069999999998 2024-09-18T11:30:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:911 4.3 4:cc46532a:::smithi11939506-33:head [write 1904434~719345] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.166081+0000", 2024-09-18T11:30:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.556062872999998, 2024-09-18T11:30:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081214100000000008, 2024-09-18T11:30:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 911 2024-09-18T11:30:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.740 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166081+0000", 2024-09-18T11:30:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166081+0000", 2024-09-18T11:30:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:29.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166080+0000", 2024-09-18T11:30:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166386+0000", 2024-09-18T11:30:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030589499999999998 2024-09-18T11:30:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166398+0000", 2024-09-18T11:30:29.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1678000000000001e-05 2024-09-18T11:30:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166402+0000", 2024-09-18T11:30:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6059999999999996e-06 2024-09-18T11:30:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166443+0000", 2024-09-18T11:30:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1044000000000002e-05 2024-09-18T11:30:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:29.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166526+0000", 2024-09-18T11:30:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2281999999999996e-05 2024-09-18T11:30:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166661+0000", 2024-09-18T11:30:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013516299999999999 2024-09-18T11:30:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.168437+0000", 2024-09-18T11:30:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001775794 2024-09-18T11:30:29.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174126+0000", 2024-09-18T11:30:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056894859999999997 2024-09-18T11:30:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174166+0000", 2024-09-18T11:30:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9787999999999999e-05 2024-09-18T11:30:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:29.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174203+0000", 2024-09-18T11:30:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6644999999999999e-05 2024-09-18T11:30:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:912 4.3 4:cc46532a:::smithi11939506-33:head [write 3128657~31026] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169139+0000", 2024-09-18T11:30:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.553005199000001, 2024-09-18T11:30:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051669280000000003, 2024-09-18T11:30:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.746 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 912 2024-09-18T11:30:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169139+0000", 2024-09-18T11:30:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169139+0000", 2024-09-18T11:30:29.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169138+0000", 2024-09-18T11:30:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169176+0000", 2024-09-18T11:30:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7679000000000003e-05 2024-09-18T11:30:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169187+0000", 2024-09-18T11:30:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1353e-05 2024-09-18T11:30:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169193+0000", 2024-09-18T11:30:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0390000000000001e-06 2024-09-18T11:30:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169230+0000", 2024-09-18T11:30:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6397999999999999e-05 2024-09-18T11:30:29.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169302+0000", 2024-09-18T11:30:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2922000000000004e-05 2024-09-18T11:30:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169426+0000", 2024-09-18T11:30:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012302600000000001 2024-09-18T11:30:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174078+0000", 2024-09-18T11:30:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046524330000000001 2024-09-18T11:30:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174240+0000", 2024-09-18T11:30:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016188899999999999 2024-09-18T11:30:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174275+0000", 2024-09-18T11:30:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5556000000000003e-05 2024-09-18T11:30:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174306+0000", 2024-09-18T11:30:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0408000000000001e-05 2024-09-18T11:30:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:913 4.3 4:cc46532a:::smithi11939506-33:head [setxattr _header (56) in=7b,truncate 3159683] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169203+0000", 2024-09-18T11:30:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.552941434000001, 2024-09-18T11:30:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052201790000000001, 2024-09-18T11:30:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.752 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 913 2024-09-18T11:30:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169203+0000", 2024-09-18T11:30:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:29.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169203+0000", 2024-09-18T11:30:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169202+0000", 2024-09-18T11:30:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169206+0000", 2024-09-18T11:30:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8160000000000004e-06 2024-09-18T11:30:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169212+0000", 2024-09-18T11:30:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3770000000000001e-06 2024-09-18T11:30:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169216+0000", 2024-09-18T11:30:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0060000000000003e-06 2024-09-18T11:30:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169888+0000", 2024-09-18T11:30:29.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00067139700000000001 2024-09-18T11:30:29.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:29.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169963+0000", 2024-09-18T11:30:29.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4999000000000005e-05 2024-09-18T11:30:29.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:29.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.170077+0000", 2024-09-18T11:30:29.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011401899999999999 2024-09-18T11:30:29.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:29.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174084+0000", 2024-09-18T11:30:29.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040071550000000001 2024-09-18T11:30:29.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174336+0000", 2024-09-18T11:30:29.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00025225300000000003 2024-09-18T11:30:29.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:29.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174371+0000", 2024-09-18T11:30:29.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5361000000000002e-05 2024-09-18T11:30:29.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174423+0000", 2024-09-18T11:30:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1347999999999997e-05 2024-09-18T11:30:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:914 4.3 4:cc46532a:::smithi11939506-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169236+0000", 2024-09-18T11:30:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.552907913999999, 2024-09-18T11:30:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005369965, 2024-09-18T11:30:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:29.758 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 914 2024-09-18T11:30:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169236+0000", 2024-09-18T11:30:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:29.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169236+0000", 2024-09-18T11:30:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169236+0000", 2024-09-18T11:30:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169241+0000", 2024-09-18T11:30:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8239999999999996e-06 2024-09-18T11:30:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169249+0000", 2024-09-18T11:30:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9319999999999996e-06 2024-09-18T11:30:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169254+0000", 2024-09-18T11:30:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3220000000000003e-06 2024-09-18T11:30:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.170304+0000", 2024-09-18T11:30:29.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001050184 2024-09-18T11:30:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.170390+0000", 2024-09-18T11:30:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5839000000000003e-05 2024-09-18T11:30:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174455+0000", 2024-09-18T11:30:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040656119999999997 2024-09-18T11:30:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:29.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174525+0000", 2024-09-18T11:30:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9151000000000003e-05 2024-09-18T11:30:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174606+0000", 2024-09-18T11:30:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1588000000000006e-05 2024-09-18T11:30:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:923 4.b 4:db5b7794:::smithi11939506-13:head [read 0~2873964,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169380+0000", 2024-09-18T11:30:29.763 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.55276379, 2024-09-18T11:30:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047044239999999996, 2024-09-18T11:30:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 923 2024-09-18T11:30:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169380+0000", 2024-09-18T11:30:29.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169380+0000", 2024-09-18T11:30:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169380+0000", 2024-09-18T11:30:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:29.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169383+0000", 2024-09-18T11:30:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1429999999999998e-06 2024-09-18T11:30:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169389+0000", 2024-09-18T11:30:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4579999999999998e-06 2024-09-18T11:30:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169393+0000", 2024-09-18T11:30:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7840000000000001e-06 2024-09-18T11:30:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.172647+0000", 2024-09-18T11:30:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032532849999999999 2024-09-18T11:30:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.172724+0000", 2024-09-18T11:30:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7836999999999994e-05 2024-09-18T11:30:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174085+0000", 2024-09-18T11:30:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013603580000000001 2024-09-18T11:30:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.767 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:924 4.b 4:db5b7794:::smithi11939506-13:head [sparse-read 0~1546056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169399+0000", 2024-09-18T11:30:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.552745445999999, 2024-09-18T11:30:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054498719999999997, 2024-09-18T11:30:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:29.768 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 924 2024-09-18T11:30:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169399+0000", 2024-09-18T11:30:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169399+0000", 2024-09-18T11:30:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169398+0000", 2024-09-18T11:30:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169401+0000", 2024-09-18T11:30:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4490000000000002e-06 2024-09-18T11:30:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169407+0000", 2024-09-18T11:30:29.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9490000000000001e-06 2024-09-18T11:30:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169411+0000", 2024-09-18T11:30:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.924e-06 2024-09-18T11:30:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174189+0000", 2024-09-18T11:30:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047787039999999999 2024-09-18T11:30:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:29.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174263+0000", 2024-09-18T11:30:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3301999999999997e-05 2024-09-18T11:30:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174849+0000", 2024-09-18T11:30:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00058598500000000002 2024-09-18T11:30:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:989 4.3 4:ca2b2309:::smithi11939506-44:head [write 410104~465105,stat] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.151379+0000", 2024-09-18T11:30:29.772 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.570764908999999, 2024-09-18T11:30:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062831420000000002, 2024-09-18T11:30:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 989 2024-09-18T11:30:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151379+0000", 2024-09-18T11:30:29.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151379+0000", 2024-09-18T11:30:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151376+0000", 2024-09-18T11:30:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151666+0000", 2024-09-18T11:30:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028977899999999999 2024-09-18T11:30:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151697+0000", 2024-09-18T11:30:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0516000000000001e-05 2024-09-18T11:30:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151707+0000", 2024-09-18T11:30:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0322999999999999e-05 2024-09-18T11:30:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151770+0000", 2024-09-18T11:30:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879000000000006e-05 2024-09-18T11:30:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151783+0000", 2024-09-18T11:30:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3154e-05 2024-09-18T11:30:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.153878+0000", 2024-09-18T11:30:29.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002095155 2024-09-18T11:30:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.153957+0000", 2024-09-18T11:30:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8877999999999999e-05 2024-09-18T11:30:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.154073+0000", 2024-09-18T11:30:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116045 2024-09-18T11:30:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:29.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.156031+0000", 2024-09-18T11:30:29.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019581099999999999 2024-09-18T11:30:29.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.157569+0000", 2024-09-18T11:30:29.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015383199999999999 2024-09-18T11:30:29.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:29.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.157606+0000", 2024-09-18T11:30:29.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6278000000000001e-05 2024-09-18T11:30:29.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:29.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.157662+0000", 2024-09-18T11:30:29.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6589000000000003e-05 2024-09-18T11:30:29.779 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.779 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:29.779 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.779 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:990 4.3 4:ca2b2309:::smithi11939506-44:head [write 1563527~583025] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:29.779 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.158301+0000", 2024-09-18T11:30:29.779 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.563843399, 2024-09-18T11:30:29.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049105670000000002, 2024-09-18T11:30:29.779 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:29.779 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.780 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:29.780 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:29.780 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:29.780 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 990 2024-09-18T11:30:29.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.780 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:29.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:29.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158301+0000", 2024-09-18T11:30:29.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:29.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158301+0000", 2024-09-18T11:30:29.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:29.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158299+0000", 2024-09-18T11:30:29.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:29.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:29.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158457+0000", 2024-09-18T11:30:29.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000157374 2024-09-18T11:30:29.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:29.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158469+0000", 2024-09-18T11:30:29.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1969e-05 2024-09-18T11:30:29.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158475+0000", 2024-09-18T11:30:29.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4450000000000001e-06 2024-09-18T11:30:29.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158504+0000", 2024-09-18T11:30:29.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8924999999999998e-05 2024-09-18T11:30:29.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:29.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158568+0000", 2024-09-18T11:30:29.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3781000000000005e-05 2024-09-18T11:30:29.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:29.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158677+0000", 2024-09-18T11:30:29.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010893899999999999 2024-09-18T11:30:29.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:29.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.160238+0000", 2024-09-18T11:30:29.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015608340000000001 2024-09-18T11:30:29.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.163135+0000", 2024-09-18T11:30:29.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028975400000000001 2024-09-18T11:30:29.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:29.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.163166+0000", 2024-09-18T11:30:29.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1164999999999997e-05 2024-09-18T11:30:29.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:29.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.163211+0000", 2024-09-18T11:30:29.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4836000000000002e-05 2024-09-18T11:30:29.785 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.785 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:29.785 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.785 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1012 4.b 4:db5b7794:::smithi11939506-13:head [write 443718~615603] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:29.785 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.655768+0000", 2024-09-18T11:30:29.785 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.066376394000001, 2024-09-18T11:30:29.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055941680000000001, 2024-09-18T11:30:29.785 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:29.785 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:29.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:29.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:29.786 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1012 2024-09-18T11:30:29.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.786 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:29.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:29.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.655768+0000", 2024-09-18T11:30:29.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:29.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.655768+0000", 2024-09-18T11:30:29.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:29.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.655765+0000", 2024-09-18T11:30:29.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:29.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:29.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656140+0000", 2024-09-18T11:30:29.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00037503800000000001 2024-09-18T11:30:29.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:29.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656166+0000", 2024-09-18T11:30:29.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5451000000000001e-05 2024-09-18T11:30:29.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656176+0000", 2024-09-18T11:30:29.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0489000000000001e-05 2024-09-18T11:30:29.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656255+0000", 2024-09-18T11:30:29.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8880000000000004e-05 2024-09-18T11:30:29.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:29.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656322+0000", 2024-09-18T11:30:29.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6500000000000004e-05 2024-09-18T11:30:29.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:29.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656436+0000", 2024-09-18T11:30:29.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114945 2024-09-18T11:30:29.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.658992+0000", 2024-09-18T11:30:29.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025551419999999998 2024-09-18T11:30:29.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:29.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.661302+0000", 2024-09-18T11:30:29.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002310543 2024-09-18T11:30:29.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:29.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.661335+0000", 2024-09-18T11:30:29.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2716999999999999e-05 2024-09-18T11:30:29.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:29.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.661362+0000", 2024-09-18T11:30:29.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6998000000000001e-05 2024-09-18T11:30:29.791 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.791 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:29.791 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.791 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1016 4.b 4:db5b7794:::smithi11939506-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:29.791 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.668055+0000", 2024-09-18T11:30:29.791 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.054088822000001, 2024-09-18T11:30:29.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044191300000000003, 2024-09-18T11:30:29.791 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:29.791 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:29.792 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:29.792 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:29.792 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:29.792 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1016 2024-09-18T11:30:29.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.792 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:29.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:29.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.668055+0000", 2024-09-18T11:30:29.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:29.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.668055+0000", 2024-09-18T11:30:29.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:29.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.668055+0000", 2024-09-18T11:30:29.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:29.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:29.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.668057+0000", 2024-09-18T11:30:29.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6340000000000001e-06 2024-09-18T11:30:29.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:29.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.668064+0000", 2024-09-18T11:30:29.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5150000000000003e-06 2024-09-18T11:30:29.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.668067+0000", 2024-09-18T11:30:29.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.552e-06 2024-09-18T11:30:29.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.669678+0000", 2024-09-18T11:30:29.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016104699999999999 2024-09-18T11:30:29.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:29.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.669752+0000", 2024-09-18T11:30:29.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4084000000000004e-05 2024-09-18T11:30:29.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.672307+0000", 2024-09-18T11:30:29.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002554551 2024-09-18T11:30:29.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:29.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.672379+0000", 2024-09-18T11:30:29.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2256999999999996e-05 2024-09-18T11:30:29.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:29.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.672474+0000", 2024-09-18T11:30:29.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5561000000000002e-05 2024-09-18T11:30:29.796 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.796 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:29.796 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.796 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1150 4.b 4:d3ae8215:::smithi11939506-41:head [write 1807440~632344] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:29.796 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.620640+0000", 2024-09-18T11:30:29.796 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.1015042220000009, 2024-09-18T11:30:29.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068244000000000004, 2024-09-18T11:30:29.797 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:29.797 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.797 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:29.797 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:29.797 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:29.797 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1150 2024-09-18T11:30:29.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.797 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:29.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:29.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.620640+0000", 2024-09-18T11:30:29.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:29.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.620640+0000", 2024-09-18T11:30:29.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:29.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.620638+0000", 2024-09-18T11:30:29.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:29.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:29.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621224+0000", 2024-09-18T11:30:29.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00058580899999999996 2024-09-18T11:30:29.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:29.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621235+0000", 2024-09-18T11:30:29.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:30:29.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621240+0000", 2024-09-18T11:30:29.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4270000000000001e-06 2024-09-18T11:30:29.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621268+0000", 2024-09-18T11:30:29.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8252999999999999e-05 2024-09-18T11:30:29.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:29.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621344+0000", 2024-09-18T11:30:29.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6154000000000003e-05 2024-09-18T11:30:29.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:29.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621464+0000", 2024-09-18T11:30:29.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012021 2024-09-18T11:30:29.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.624135+0000", 2024-09-18T11:30:29.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026711310000000002 2024-09-18T11:30:29.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:29.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.627420+0000", 2024-09-18T11:30:29.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032846799999999999 2024-09-18T11:30:29.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:29.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.627443+0000", 2024-09-18T11:30:29.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3153e-05 2024-09-18T11:30:29.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:29.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.627464+0000", 2024-09-18T11:30:29.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0967000000000001e-05 2024-09-18T11:30:29.802 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.802 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:29.802 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.802 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1152 4.b 4:d3ae8215:::smithi11939506-41:head [setxattr _header (56) in=7b,truncate 3428595] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:29.802 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.625951+0000", 2024-09-18T11:30:29.802 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0961935369999996, 2024-09-18T11:30:29.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044856540000000004, 2024-09-18T11:30:29.803 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:29.803 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.803 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:29.803 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:29.803 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:29.803 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1152 2024-09-18T11:30:29.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.803 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:29.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:29.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625951+0000", 2024-09-18T11:30:29.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:29.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625951+0000", 2024-09-18T11:30:29.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:29.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625950+0000", 2024-09-18T11:30:29.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:29.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:29.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625954+0000", 2024-09-18T11:30:29.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7090000000000001e-06 2024-09-18T11:30:29.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:29.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625960+0000", 2024-09-18T11:30:29.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9209999999999997e-06 2024-09-18T11:30:29.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625963+0000", 2024-09-18T11:30:29.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0189999999999998e-06 2024-09-18T11:30:29.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.626714+0000", 2024-09-18T11:30:29.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00075054400000000002 2024-09-18T11:30:29.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:29.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.626784+0000", 2024-09-18T11:30:29.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9961000000000003e-05 2024-09-18T11:30:29.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:29.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.626899+0000", 2024-09-18T11:30:29.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011521100000000001 2024-09-18T11:30:29.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:29.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.627477+0000", 2024-09-18T11:30:29.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00057745099999999998 2024-09-18T11:30:29.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.630329+0000", 2024-09-18T11:30:29.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028520529999999998 2024-09-18T11:30:29.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:29.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.630377+0000", 2024-09-18T11:30:29.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7939999999999998e-05 2024-09-18T11:30:29.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:29.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.630436+0000", 2024-09-18T11:30:29.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9676000000000003e-05 2024-09-18T11:30:29.808 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.808 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:29.808 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.808 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1153 4.b 4:d3ae8215:::smithi11939506-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:29.808 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.625968+0000", 2024-09-18T11:30:29.808 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0961759690000008, 2024-09-18T11:30:29.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046073240000000003, 2024-09-18T11:30:29.809 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:29.809 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:29.809 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:29.809 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:29.809 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:29.809 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1153 2024-09-18T11:30:29.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.809 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:29.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:29.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625968+0000", 2024-09-18T11:30:29.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:29.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625968+0000", 2024-09-18T11:30:29.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:29.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625968+0000", 2024-09-18T11:30:29.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:29.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:29.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625969+0000", 2024-09-18T11:30:29.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.468e-06 2024-09-18T11:30:29.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:29.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625978+0000", 2024-09-18T11:30:29.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4309999999999994e-06 2024-09-18T11:30:29.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625981+0000", 2024-09-18T11:30:29.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5250000000000001e-06 2024-09-18T11:30:29.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.627082+0000", 2024-09-18T11:30:29.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0011011149999999999 2024-09-18T11:30:29.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:29.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.627163+0000", 2024-09-18T11:30:29.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1112999999999994e-05 2024-09-18T11:30:29.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.630468+0000", 2024-09-18T11:30:29.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0033042309999999999 2024-09-18T11:30:29.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:29.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.630521+0000", 2024-09-18T11:30:29.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3108e-05 2024-09-18T11:30:29.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:29.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.630575+0000", 2024-09-18T11:30:29.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4599999999999999e-05 2024-09-18T11:30:29.813 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.813 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:29.814 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.814 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:31 3.1 3:88ab294d:::972cde73ee90bbfa4e1e8219b21f333678dc7dbfb3d5501c6973feb14420b94c:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:29.814 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.678405+0000", 2024-09-18T11:30:29.814 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0437395519999999, 2024-09-18T11:30:29.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046395250000000002, 2024-09-18T11:30:29.814 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:29.814 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.814 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:29.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:29.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.678405+0000", 2024-09-18T11:30:29.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:29.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.678405+0000", 2024-09-18T11:30:29.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:29.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.678402+0000", 2024-09-18T11:30:29.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:29.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:29.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679284+0000", 2024-09-18T11:30:29.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00088187200000000002 2024-09-18T11:30:29.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:29.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679305+0000", 2024-09-18T11:30:29.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1183999999999999e-05 2024-09-18T11:30:29.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679313+0000", 2024-09-18T11:30:29.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2649999999999996e-06 2024-09-18T11:30:29.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679385+0000", 2024-09-18T11:30:29.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2119000000000005e-05 2024-09-18T11:30:29.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:29.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679510+0000", 2024-09-18T11:30:29.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000124831 2024-09-18T11:30:29.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:29.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679727+0000", 2024-09-18T11:30:29.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000217061 2024-09-18T11:30:29.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:29.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680985+0000", 2024-09-18T11:30:29.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001258295 2024-09-18T11:30:29.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682957+0000", 2024-09-18T11:30:29.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019718740000000002 2024-09-18T11:30:29.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:29.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.683007+0000", 2024-09-18T11:30:29.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9740000000000001e-05 2024-09-18T11:30:29.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:29.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.683044+0000", 2024-09-18T11:30:29.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7082999999999997e-05 2024-09-18T11:30:29.819 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.819 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:29.819 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.820 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:32 3.1 3:94708c4c:::888f815178b2a748104fffa559dcb609f143f519d8737c6aa160c9cce13a6441:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:29.820 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.679331+0000", 2024-09-18T11:30:29.820 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0428134539999991, 2024-09-18T11:30:29.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063965929999999999, 2024-09-18T11:30:29.820 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:29.820 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.820 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:29.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:29.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679331+0000", 2024-09-18T11:30:29.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:29.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679331+0000", 2024-09-18T11:30:29.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:29.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679330+0000", 2024-09-18T11:30:29.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:29.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:29.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681259+0000", 2024-09-18T11:30:29.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019287079999999999 2024-09-18T11:30:29.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:29.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681274+0000", 2024-09-18T11:30:29.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5903e-05 2024-09-18T11:30:29.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681281+0000", 2024-09-18T11:30:29.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5640000000000002e-06 2024-09-18T11:30:29.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681325+0000", 2024-09-18T11:30:29.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3925999999999999e-05 2024-09-18T11:30:29.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:29.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681435+0000", 2024-09-18T11:30:29.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109739 2024-09-18T11:30:29.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:29.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681642+0000", 2024-09-18T11:30:29.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000206901 2024-09-18T11:30:29.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:29.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.683180+0000", 2024-09-18T11:30:29.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015380929999999999 2024-09-18T11:30:29.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.685648+0000", 2024-09-18T11:30:29.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024687419999999999 2024-09-18T11:30:29.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:29.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.685695+0000", 2024-09-18T11:30:29.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7030000000000002e-05 2024-09-18T11:30:29.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:29.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.685727+0000", 2024-09-18T11:30:29.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1792000000000003e-05 2024-09-18T11:30:29.825 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.825 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:29.825 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.825 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1204 4.6 4:6f86ffcc:::smithi11939506-15:head [write 779858~469642,stat] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-18T11:30:29.825 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:21.214976+0000", 2024-09-18T11:30:29.825 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.5071684609999991, 2024-09-18T11:30:29.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050506320000000002, 2024-09-18T11:30:29.826 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:29.826 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:29.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:29.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:29.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:29.826 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1204 2024-09-18T11:30:29.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.826 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:29.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:29.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.214976+0000", 2024-09-18T11:30:29.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:29.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.214976+0000", 2024-09-18T11:30:29.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:29.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.214974+0000", 2024-09-18T11:30:29.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:29.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:29.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.215075+0000", 2024-09-18T11:30:29.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100777 2024-09-18T11:30:29.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:29.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.215091+0000", 2024-09-18T11:30:29.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6413e-05 2024-09-18T11:30:29.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.215097+0000", 2024-09-18T11:30:29.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4179999999999996e-06 2024-09-18T11:30:29.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.215148+0000", 2024-09-18T11:30:29.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1162000000000002e-05 2024-09-18T11:30:29.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:29.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.215153+0000", 2024-09-18T11:30:29.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0849999999999996e-06 2024-09-18T11:30:29.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.216731+0000", 2024-09-18T11:30:29.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015780670000000001 2024-09-18T11:30:29.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:29.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.216818+0000", 2024-09-18T11:30:29.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7121999999999997e-05 2024-09-18T11:30:29.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:29.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.216999+0000", 2024-09-18T11:30:29.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000181051 2024-09-18T11:30:29.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:29.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.219747+0000", 2024-09-18T11:30:29.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027480529999999999 2024-09-18T11:30:29.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:29.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.219955+0000", 2024-09-18T11:30:29.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020780399999999999 2024-09-18T11:30:29.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:29.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.219987+0000", 2024-09-18T11:30:29.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2320000000000002e-05 2024-09-18T11:30:29.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:29.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.220026+0000", 2024-09-18T11:30:29.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8962999999999999e-05 2024-09-18T11:30:29.832 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.832 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:29.832 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.832 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:43 3.1 3:8abdb616:::smithi11939506-46 low_tier:head [read 2311680~395776 [fadvise_sequential]] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected+supports_pool_eio e42)", 2024-09-18T11:30:29.832 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:23.795465+0000", 2024-09-18T11:30:29.832 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.9266790110000001, 2024-09-18T11:30:29.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004375416, 2024-09-18T11:30:29.832 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:29.832 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:29.833 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:29.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:29.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:23.795465+0000", 2024-09-18T11:30:29.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:29.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:23.795465+0000", 2024-09-18T11:30:29.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:29.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:29.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:23.795465+0000", 2024-09-18T11:30:29.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:29.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:29.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:23.795467+0000", 2024-09-18T11:30:29.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.791e-06 2024-09-18T11:30:29.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:29.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:23.795474+0000", 2024-09-18T11:30:29.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6220000000000003e-06 2024-09-18T11:30:29.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:29.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:23.795478+0000", 2024-09-18T11:30:29.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9160000000000003e-06 2024-09-18T11:30:29.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:29.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:23.798921+0000", 2024-09-18T11:30:29.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034431010000000001 2024-09-18T11:30:29.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:29.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:23.799026+0000", 2024-09-18T11:30:29.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105447 2024-09-18T11:30:29.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:29.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:29.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:29.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:23.799840+0000", 2024-09-18T11:30:29.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00081406100000000004 2024-09-18T11:30:29.836 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.836 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:29.836 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.836 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:29.836 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:29.836 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:29.838 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:30:29.838 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:29.836+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:29.935 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:29.936 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:29.936 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:29.936 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:29.938 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:29.937+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:29.944 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:30:30.039 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:30.038+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:30.042 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:30.042 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:30.043 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:30.043 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:30.043 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:30.051 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:30:30.140 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:30.139+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:30.150 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:30.151 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:30.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:30.151 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:30.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.151 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:887 4.7 4:e0d95e10:::smithi11939506-22:head [read 0~2352947,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:30.151 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:30:30.151 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.031667505000001, 2024-09-18T11:30:30.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079115169999999999, 2024-09-18T11:30:30.151 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:30.151 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:30.152 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:30.152 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:30.152 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:30.152 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 887 2024-09-18T11:30:30.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.152 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:30.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:30.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:30:30.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:30.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:30:30.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:30.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116365+0000", 2024-09-18T11:30:30.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:30.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:30.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116376+0000", 2024-09-18T11:30:30.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1471000000000001e-05 2024-09-18T11:30:30.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:30.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116394+0000", 2024-09-18T11:30:30.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8088e-05 2024-09-18T11:30:30.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:30.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116402+0000", 2024-09-18T11:30:30.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2320000000000004e-06 2024-09-18T11:30:30.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116458+0000", 2024-09-18T11:30:30.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6536000000000002e-05 2024-09-18T11:30:30.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:30.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116466+0000", 2024-09-18T11:30:30.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0819999999999999e-06 2024-09-18T11:30:30.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.122509+0000", 2024-09-18T11:30:30.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060431879999999997 2024-09-18T11:30:30.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:30.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.122577+0000", 2024-09-18T11:30:30.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7646999999999998e-05 2024-09-18T11:30:30.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:30.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124278+0000", 2024-09-18T11:30:30.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017013200000000001 2024-09-18T11:30:30.156 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.156 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:30.156 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.157 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:888 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:30.157 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:30.157 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.031614042000001, 2024-09-18T11:30:30.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082282120000000004, 2024-09-18T11:30:30.157 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:30.157 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:30.157 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:30.157 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:30.157 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:30.157 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 888 2024-09-18T11:30:30.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.157 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:30.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:30.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:30.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:30.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:30.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:30.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:30.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:30.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:30.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116423+0000", 2024-09-18T11:30:30.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1e-06 2024-09-18T11:30:30.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:30.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:30:30.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5510000000000001e-06 2024-09-18T11:30:30.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:30.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116431+0000", 2024-09-18T11:30:30.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6109999999999998e-06 2024-09-18T11:30:30.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116527+0000", 2024-09-18T11:30:30.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6212999999999995e-05 2024-09-18T11:30:30.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:30.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116532+0000", 2024-09-18T11:30:30.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6140000000000002e-06 2024-09-18T11:30:30.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124314+0000", 2024-09-18T11:30:30.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077818929999999998 2024-09-18T11:30:30.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:30.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124411+0000", 2024-09-18T11:30:30.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6992999999999998e-05 2024-09-18T11:30:30.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:30.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124649+0000", 2024-09-18T11:30:30.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237729 2024-09-18T11:30:30.161 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.162 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:30.162 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.162 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:889 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:30.162 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:30.162 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.031589392000001, 2024-09-18T11:30:30.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085383349999999993, 2024-09-18T11:30:30.162 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:30.162 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:30.162 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:30.162 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:30.162 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:30.163 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 889 2024-09-18T11:30:30.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.163 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:30.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:30.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:30.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:30.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:30.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:30.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:30.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:30.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:30.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116447+0000", 2024-09-18T11:30:30.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.854e-06 2024-09-18T11:30:30.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:30.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116452+0000", 2024-09-18T11:30:30.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0250000000000002e-06 2024-09-18T11:30:30.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:30.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116455+0000", 2024-09-18T11:30:30.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8739999999999999e-06 2024-09-18T11:30:30.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116565+0000", 2024-09-18T11:30:30.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110741 2024-09-18T11:30:30.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:30.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116571+0000", 2024-09-18T11:30:30.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4990000000000002e-06 2024-09-18T11:30:30.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124691+0000", 2024-09-18T11:30:30.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081200339999999999 2024-09-18T11:30:30.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:30.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124784+0000", 2024-09-18T11:30:30.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2886999999999993e-05 2024-09-18T11:30:30.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:30.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124983+0000", 2024-09-18T11:30:30.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019969000000000001 2024-09-18T11:30:30.167 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.167 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:30.167 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.167 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:46 4.a e31/22 4:552523b6:::smithi11939506-38:head v 31'20, mlcod=30'18)", 2024-09-18T11:30:30.167 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:30:30.167 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.01763776, 2024-09-18T11:30:30.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080208420000000002, 2024-09-18T11:30:30.167 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:30.167 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:30.168 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:30.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:30.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:30:30.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:30.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:30:30.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:30.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130396+0000", 2024-09-18T11:30:30.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:30.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:30.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133245+0000", 2024-09-18T11:30:30.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028486929999999998 2024-09-18T11:30:30.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:30.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133253+0000", 2024-09-18T11:30:30.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1540000000000002e-06 2024-09-18T11:30:30.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:30.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133261+0000", 2024-09-18T11:30:30.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3620000000000006e-06 2024-09-18T11:30:30.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133319+0000", 2024-09-18T11:30:30.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7719000000000001e-05 2024-09-18T11:30:30.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:30.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133341+0000", 2024-09-18T11:30:30.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1946999999999998e-05 2024-09-18T11:30:30.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:30.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138368+0000", 2024-09-18T11:30:30.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005026712 2024-09-18T11:30:30.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:30.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138418+0000", 2024-09-18T11:30:30.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9864000000000002e-05 2024-09-18T11:30:30.171 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.171 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:30.172 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.172 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "pg_update_log_missing(4.a epoch 31/22 rep_tid 47 entries 31'21 (0'0) error 4:552523b6:::smithi11939506-38:head by client.4590.0:883 0.000000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 30'18)", 2024-09-18T11:30:30.172 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.133275+0000", 2024-09-18T11:30:30.172 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.014759438999999, 2024-09-18T11:30:30.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051654209999999999, 2024-09-18T11:30:30.172 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:30.172 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:30:30.172 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:30.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:30.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133275+0000", 2024-09-18T11:30:30.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:30.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133275+0000", 2024-09-18T11:30:30.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:30.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133275+0000", 2024-09-18T11:30:30.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:30.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:30.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133283+0000", 2024-09-18T11:30:30.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3210000000000005e-06 2024-09-18T11:30:30.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:30.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133294+0000", 2024-09-18T11:30:30.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0849000000000001e-05 2024-09-18T11:30:30.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:30.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133300+0000", 2024-09-18T11:30:30.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3110000000000002e-06 2024-09-18T11:30:30.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.134908+0000", 2024-09-18T11:30:30.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016080339999999999 2024-09-18T11:30:30.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:30.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138440+0000", 2024-09-18T11:30:30.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035323059999999998 2024-09-18T11:30:30.175 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.175 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:30.175 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.176 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:905 4.7 4:e73ba68f:::smithi11939506-1:head [write 742876~447756,stat] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:30.176 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.140411+0000", 2024-09-18T11:30:30.176 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.007623063, 2024-09-18T11:30:30.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060845960000000003, 2024-09-18T11:30:30.176 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:30.176 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:30.176 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:30.176 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:30.176 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:30.176 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 905 2024-09-18T11:30:30.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.176 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:30.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:30.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140411+0000", 2024-09-18T11:30:30.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:30.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140411+0000", 2024-09-18T11:30:30.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:30.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140408+0000", 2024-09-18T11:30:30.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:30:30.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:30.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140676+0000", 2024-09-18T11:30:30.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026841300000000001 2024-09-18T11:30:30.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:30.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140708+0000", 2024-09-18T11:30:30.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1952000000000002e-05 2024-09-18T11:30:30.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:30.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140721+0000", 2024-09-18T11:30:30.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3005e-05 2024-09-18T11:30:30.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140788+0000", 2024-09-18T11:30:30.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6839000000000002e-05 2024-09-18T11:30:30.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:30.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140923+0000", 2024-09-18T11:30:30.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000135258 2024-09-18T11:30:30.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:30.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.141159+0000", 2024-09-18T11:30:30.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023572900000000001 2024-09-18T11:30:30.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:30.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.143764+0000", 2024-09-18T11:30:30.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026044919999999999 2024-09-18T11:30:30.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:30.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.146411+0000", 2024-09-18T11:30:30.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026475130000000002 2024-09-18T11:30:30.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:30.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.146448+0000", 2024-09-18T11:30:30.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6650999999999999e-05 2024-09-18T11:30:30.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:30.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.146496+0000", 2024-09-18T11:30:30.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8177000000000003e-05 2024-09-18T11:30:30.181 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.181 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:30.181 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.182 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:917 4.f 4:f5d7df41:::smithi11939506-16:2 [sparse-read 0~1650442] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:30.182 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169348+0000", 2024-09-18T11:30:30.182 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.978686738, 2024-09-18T11:30:30.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050103350000000003, 2024-09-18T11:30:30.182 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:30.182 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:30.182 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:30.182 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:30.182 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:30.182 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 917 2024-09-18T11:30:30.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.182 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:30.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:30.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169348+0000", 2024-09-18T11:30:30.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:30.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169348+0000", 2024-09-18T11:30:30.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:30.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169347+0000", 2024-09-18T11:30:30.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:30.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:30.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169350+0000", 2024-09-18T11:30:30.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.537e-06 2024-09-18T11:30:30.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:30.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169356+0000", 2024-09-18T11:30:30.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1129999999999997e-06 2024-09-18T11:30:30.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:30.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169359+0000", 2024-09-18T11:30:30.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.444e-06 2024-09-18T11:30:30.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.173900+0000", 2024-09-18T11:30:30.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045409079999999998 2024-09-18T11:30:30.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:30.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174024+0000", 2024-09-18T11:30:30.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012400399999999999 2024-09-18T11:30:30.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:30.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174358+0000", 2024-09-18T11:30:30.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00033376199999999998 2024-09-18T11:30:30.186 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.186 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:30.186 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.186 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:995 4.f 4:f269e3d0:::smithi11939506-24:head [read 0~2386561,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:30.186 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164480+0000", 2024-09-18T11:30:30.186 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.983554733, 2024-09-18T11:30:30.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057271170000000003, 2024-09-18T11:30:30.186 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:30.186 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:30.186 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:30.186 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:30.187 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:30.187 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 995 2024-09-18T11:30:30.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.187 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:30.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:30.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164480+0000", 2024-09-18T11:30:30.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:30.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164480+0000", 2024-09-18T11:30:30.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:30.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164479+0000", 2024-09-18T11:30:30.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:30.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:30.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164483+0000", 2024-09-18T11:30:30.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9859999999999996e-06 2024-09-18T11:30:30.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:30.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164497+0000", 2024-09-18T11:30:30.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3559e-05 2024-09-18T11:30:30.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:30.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164506+0000", 2024-09-18T11:30:30.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3959999999999995e-06 2024-09-18T11:30:30.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164573+0000", 2024-09-18T11:30:30.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7123000000000001e-05 2024-09-18T11:30:30.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:30.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164577+0000", 2024-09-18T11:30:30.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9999999999999998e-06 2024-09-18T11:30:30.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.168696+0000", 2024-09-18T11:30:30.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041188379999999997 2024-09-18T11:30:30.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:30.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.168824+0000", 2024-09-18T11:30:30.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012762099999999999 2024-09-18T11:30:30.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:30.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170207+0000", 2024-09-18T11:30:30.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001383076 2024-09-18T11:30:30.191 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.191 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:30.191 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.191 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:996 4.f 4:f269e3d0:::smithi11939506-24:head [read 0~2386561,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:30.191 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164511+0000", 2024-09-18T11:30:30.191 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.983523162000001, 2024-09-18T11:30:30.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069965679999999999, 2024-09-18T11:30:30.192 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:30.192 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:30.192 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:30.192 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:30.192 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:30.192 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 996 2024-09-18T11:30:30.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.192 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:30.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:30.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164511+0000", 2024-09-18T11:30:30.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:30.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164511+0000", 2024-09-18T11:30:30.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:30.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164511+0000", 2024-09-18T11:30:30.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:30.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:30.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164514+0000", 2024-09-18T11:30:30.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3210000000000001e-06 2024-09-18T11:30:30.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:30.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164521+0000", 2024-09-18T11:30:30.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5169999999999996e-06 2024-09-18T11:30:30.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:30.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164524+0000", 2024-09-18T11:30:30.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:30:30.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164674+0000", 2024-09-18T11:30:30.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014943199999999999 2024-09-18T11:30:30.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:30.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:30:30.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8579999999999996e-06 2024-09-18T11:30:30.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170262+0000", 2024-09-18T11:30:30.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055827719999999997 2024-09-18T11:30:30.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:30.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170405+0000", 2024-09-18T11:30:30.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000143115 2024-09-18T11:30:30.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:30.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.171508+0000", 2024-09-18T11:30:30.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001102448 2024-09-18T11:30:30.198 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.198 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:30.198 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.199 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1046 4.4 4:2cb3e6a5:::smithi11939506-3:head [write 1455937~734021] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:30.199 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.173120+0000", 2024-09-18T11:30:30.199 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.974914775, 2024-09-18T11:30:30.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051191429999999996, 2024-09-18T11:30:30.199 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:30.199 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:30.199 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:30.199 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:30.200 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:30.200 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1046 2024-09-18T11:30:30.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.200 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:30.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:30.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173120+0000", 2024-09-18T11:30:30.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:30.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173120+0000", 2024-09-18T11:30:30.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:30.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173119+0000", 2024-09-18T11:30:30.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:30.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:30.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173464+0000", 2024-09-18T11:30:30.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00034509200000000002 2024-09-18T11:30:30.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:30.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173475+0000", 2024-09-18T11:30:30.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1625e-05 2024-09-18T11:30:30.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:30.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173481+0000", 2024-09-18T11:30:30.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0639999999999999e-06 2024-09-18T11:30:30.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173517+0000", 2024-09-18T11:30:30.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5540000000000002e-05 2024-09-18T11:30:30.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:30.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173581+0000", 2024-09-18T11:30:30.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3862000000000006e-05 2024-09-18T11:30:30.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:30.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173692+0000", 2024-09-18T11:30:30.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000111487 2024-09-18T11:30:30.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:30.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.177793+0000", 2024-09-18T11:30:30.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041005879999999996 2024-09-18T11:30:30.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:30.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.178146+0000", 2024-09-18T11:30:30.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00035280100000000003 2024-09-18T11:30:30.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:30.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.178196+0000", 2024-09-18T11:30:30.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0132e-05 2024-09-18T11:30:30.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:30.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.178239+0000", 2024-09-18T11:30:30.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3024e-05 2024-09-18T11:30:30.206 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.207 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:30.207 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.207 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1121 4.f 4:f5d7df41:::smithi11939506-16:head [write 1515641~485017] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:30.207 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.965485+0000", 2024-09-18T11:30:30.207 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 10.182549129, 2024-09-18T11:30:30.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006622869, 2024-09-18T11:30:30.207 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:30.207 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:30.207 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:30.207 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:30.207 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:30.208 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1121 2024-09-18T11:30:30.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.208 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:30.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:30.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965485+0000", 2024-09-18T11:30:30.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:30.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965485+0000", 2024-09-18T11:30:30.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:30.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965484+0000", 2024-09-18T11:30:30.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:30.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:30.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965572+0000", 2024-09-18T11:30:30.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7934000000000002e-05 2024-09-18T11:30:30.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:30.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965583+0000", 2024-09-18T11:30:30.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0594999999999999e-05 2024-09-18T11:30:30.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:30.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965587+0000", 2024-09-18T11:30:30.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3800000000000004e-06 2024-09-18T11:30:30.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965611+0000", 2024-09-18T11:30:30.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.404e-05 2024-09-18T11:30:30.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:30.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965665+0000", 2024-09-18T11:30:30.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3448000000000001e-05 2024-09-18T11:30:30.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:30:30.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965741+0000", 2024-09-18T11:30:30.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6264e-05 2024-09-18T11:30:30.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:30.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.969188+0000", 2024-09-18T11:30:30.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034471129999999999 2024-09-18T11:30:30.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:30.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.971990+0000", 2024-09-18T11:30:30.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028016809999999999 2024-09-18T11:30:30.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:30.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.972068+0000", 2024-09-18T11:30:30.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8671000000000006e-05 2024-09-18T11:30:30.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:30.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.972108+0000", 2024-09-18T11:30:30.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9765000000000003e-05 2024-09-18T11:30:30.219 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.220 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:30.220 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.220 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1126 3.0 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head [write 0~2520064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:30.221 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:30:30.221 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 10.166147413999999, 2024-09-18T11:30:30.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086197050000000001, 2024-09-18T11:30:30.221 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:30.221 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:30.222 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:30.222 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:30.222 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:30.222 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1126 2024-09-18T11:30:30.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.223 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:30.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:30.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:30:30.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:30.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:30:30.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:30.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981886+0000", 2024-09-18T11:30:30.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:30.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:30.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983017+0000", 2024-09-18T11:30:30.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00113131 2024-09-18T11:30:30.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:30.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983037+0000", 2024-09-18T11:30:30.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0231e-05 2024-09-18T11:30:30.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:30.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983044+0000", 2024-09-18T11:30:30.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9539999999999998e-06 2024-09-18T11:30:30.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983087+0000", 2024-09-18T11:30:30.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3040000000000001e-05 2024-09-18T11:30:30.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:30.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983193+0000", 2024-09-18T11:30:30.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105682 2024-09-18T11:30:30.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:30.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983287+0000", 2024-09-18T11:30:30.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4066000000000004e-05 2024-09-18T11:30:30.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:30.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.986918+0000", 2024-09-18T11:30:30.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036311970000000001 2024-09-18T11:30:30.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:30.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990442+0000", 2024-09-18T11:30:30.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035243119999999999 2024-09-18T11:30:30.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:30.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990476+0000", 2024-09-18T11:30:30.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4028e-05 2024-09-18T11:30:30.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:30.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990507+0000", 2024-09-18T11:30:30.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0284999999999999e-05 2024-09-18T11:30:30.231 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.231 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:30.231 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.231 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1136 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~1201068,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:30.231 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.179006+0000", 2024-09-18T11:30:30.231 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9690285890000006, 2024-09-18T11:30:30.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006872197, 2024-09-18T11:30:30.231 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:30.232 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:30.232 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:30.232 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:30.232 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:30.232 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1136 2024-09-18T11:30:30.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.232 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:30.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:30.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179006+0000", 2024-09-18T11:30:30.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:30.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179006+0000", 2024-09-18T11:30:30.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:30.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179004+0000", 2024-09-18T11:30:30.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:30.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:30.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179013+0000", 2024-09-18T11:30:30.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3340000000000002e-06 2024-09-18T11:30:30.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:30.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179028+0000", 2024-09-18T11:30:30.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5150999999999999e-05 2024-09-18T11:30:30.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:30.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179035+0000", 2024-09-18T11:30:30.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4919999999999996e-06 2024-09-18T11:30:30.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179086+0000", 2024-09-18T11:30:30.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0402000000000003e-05 2024-09-18T11:30:30.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:30.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179092+0000", 2024-09-18T11:30:30.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1890000000000004e-06 2024-09-18T11:30:30.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.181504+0000", 2024-09-18T11:30:30.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024116340000000002 2024-09-18T11:30:30.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:30.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.181609+0000", 2024-09-18T11:30:30.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105337 2024-09-18T11:30:30.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:30.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.185878+0000", 2024-09-18T11:30:30.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042690779999999999 2024-09-18T11:30:30.236 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.236 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:30.236 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.237 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1137 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~1201068] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:30.237 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.179047+0000", 2024-09-18T11:30:30.237 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9689877629999994, 2024-09-18T11:30:30.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073446830000000003, 2024-09-18T11:30:30.237 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:30.237 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:30.237 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:30.237 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:30.237 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:30.237 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1137 2024-09-18T11:30:30.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.238 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:30.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:30.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179047+0000", 2024-09-18T11:30:30.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:30.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179047+0000", 2024-09-18T11:30:30.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:30.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179046+0000", 2024-09-18T11:30:30.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:30.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:30.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179049+0000", 2024-09-18T11:30:30.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.269e-06 2024-09-18T11:30:30.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:30.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179057+0000", 2024-09-18T11:30:30.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7279999999999995e-06 2024-09-18T11:30:30.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:30.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179061+0000", 2024-09-18T11:30:30.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3309999999999997e-06 2024-09-18T11:30:30.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179159+0000", 2024-09-18T11:30:30.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7027999999999993e-05 2024-09-18T11:30:30.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:30.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179166+0000", 2024-09-18T11:30:30.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8520000000000004e-06 2024-09-18T11:30:30.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.185971+0000", 2024-09-18T11:30:30.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068045570000000001 2024-09-18T11:30:30.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:30.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186111+0000", 2024-09-18T11:30:30.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000139906 2024-09-18T11:30:30.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:30.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186391+0000", 2024-09-18T11:30:30.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028053000000000003 2024-09-18T11:30:30.242 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.242 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:30.242 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.242 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1138 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~1201068] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:30.242 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.179081+0000", 2024-09-18T11:30:30.242 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9689538009999996, 2024-09-18T11:30:30.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007692099, 2024-09-18T11:30:30.242 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:30.242 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:30.243 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:30.243 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:30.243 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:30.243 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1138 2024-09-18T11:30:30.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.243 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:30.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:30.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179081+0000", 2024-09-18T11:30:30.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:30.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179081+0000", 2024-09-18T11:30:30.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:30.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179080+0000", 2024-09-18T11:30:30.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:30.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:30.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179083+0000", 2024-09-18T11:30:30.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.266e-06 2024-09-18T11:30:30.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:30.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179089+0000", 2024-09-18T11:30:30.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3589999999999996e-06 2024-09-18T11:30:30.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:30.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:30.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1999999999999996e-06 2024-09-18T11:30:30.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179194+0000", 2024-09-18T11:30:30.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101093 2024-09-18T11:30:30.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:30.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179199+0000", 2024-09-18T11:30:30.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8960000000000002e-06 2024-09-18T11:30:30.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186435+0000", 2024-09-18T11:30:30.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072360979999999998 2024-09-18T11:30:30.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:30.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186530+0000", 2024-09-18T11:30:30.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5006999999999993e-05 2024-09-18T11:30:30.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:30.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186773+0000", 2024-09-18T11:30:30.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000242496 2024-09-18T11:30:30.247 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.247 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:30.247 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.247 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1139 4.2 4:425afe9c:::smithi11939506-29:head [write 594981~609197] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:30.247 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.186267+0000", 2024-09-18T11:30:30.247 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9617669739999997, 2024-09-18T11:30:30.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051309839999999999, 2024-09-18T11:30:30.248 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:30.248 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:30.248 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:30.248 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:30.248 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:30.248 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1139 2024-09-18T11:30:30.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.248 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:30.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:30.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186267+0000", 2024-09-18T11:30:30.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:30.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186267+0000", 2024-09-18T11:30:30.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:30.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186265+0000", 2024-09-18T11:30:30.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:30.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:30.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186460+0000", 2024-09-18T11:30:30.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019487700000000001 2024-09-18T11:30:30.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:30.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186478+0000", 2024-09-18T11:30:30.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8376000000000002e-05 2024-09-18T11:30:30.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:30.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186487+0000", 2024-09-18T11:30:30.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2039999999999997e-06 2024-09-18T11:30:30.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186543+0000", 2024-09-18T11:30:30.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6756999999999999e-05 2024-09-18T11:30:30.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:30.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186652+0000", 2024-09-18T11:30:30.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010819600000000001 2024-09-18T11:30:30.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:30.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186859+0000", 2024-09-18T11:30:30.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000207548 2024-09-18T11:30:30.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:30.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.190269+0000", 2024-09-18T11:30:30.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003409586 2024-09-18T11:30:30.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:30.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.191292+0000", 2024-09-18T11:30:30.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0010231400000000001 2024-09-18T11:30:30.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:30.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.191347+0000", 2024-09-18T11:30:30.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5435e-05 2024-09-18T11:30:30.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:30.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.191398+0000", 2024-09-18T11:30:30.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1221999999999998e-05 2024-09-18T11:30:30.253 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.253 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:30.253 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.254 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1140 4.2 4:425afe9c:::smithi11939506-29:head [write 1974123~745028] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:30.254 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.194143+0000", 2024-09-18T11:30:30.254 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9538915130000003, 2024-09-18T11:30:30.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055647580000000004, 2024-09-18T11:30:30.254 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:30.254 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:30.254 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:30.254 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:30.254 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:30.254 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1140 2024-09-18T11:30:30.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.255 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:30.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:30.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.194143+0000", 2024-09-18T11:30:30.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:30.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.194143+0000", 2024-09-18T11:30:30.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:30.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.194140+0000", 2024-09-18T11:30:30.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:30.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:30.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.194676+0000", 2024-09-18T11:30:30.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00053547800000000004 2024-09-18T11:30:30.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:30.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.194720+0000", 2024-09-18T11:30:30.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3961999999999997e-05 2024-09-18T11:30:30.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:30.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.194732+0000", 2024-09-18T11:30:30.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1853e-05 2024-09-18T11:30:30.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.194813+0000", 2024-09-18T11:30:30.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1005000000000001e-05 2024-09-18T11:30:30.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:30.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.194920+0000", 2024-09-18T11:30:30.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010731 2024-09-18T11:30:30.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:30.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.195074+0000", 2024-09-18T11:30:30.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015366699999999999 2024-09-18T11:30:30.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:30.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.197750+0000", 2024-09-18T11:30:30.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00267659 2024-09-18T11:30:30.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:30.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.199646+0000", 2024-09-18T11:30:30.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018959020000000001 2024-09-18T11:30:30.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:30.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.199675+0000", 2024-09-18T11:30:30.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9128e-05 2024-09-18T11:30:30.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:30.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.199708+0000", 2024-09-18T11:30:30.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2391000000000002e-05 2024-09-18T11:30:30.260 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.260 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:30.260 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.260 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1165 4.7 4:e73ba68f:::smithi11939506-1:head [write 662209~626546] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:30.260 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.657280+0000", 2024-09-18T11:30:30.260 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4907543560000001, 2024-09-18T11:30:30.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049684990000000004, 2024-09-18T11:30:30.261 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:30.261 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:30.261 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:30.261 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:30.261 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:30.261 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1165 2024-09-18T11:30:30.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.261 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:30.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:30.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.657280+0000", 2024-09-18T11:30:30.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:30.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.657280+0000", 2024-09-18T11:30:30.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:30.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.657278+0000", 2024-09-18T11:30:30.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:30.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:30.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.657456+0000", 2024-09-18T11:30:30.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000177911 2024-09-18T11:30:30.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:30.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.657475+0000", 2024-09-18T11:30:30.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8873999999999999e-05 2024-09-18T11:30:30.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:30.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.657482+0000", 2024-09-18T11:30:30.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6000000000000001e-06 2024-09-18T11:30:30.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.657526+0000", 2024-09-18T11:30:30.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4175000000000003e-05 2024-09-18T11:30:30.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:30.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.657591+0000", 2024-09-18T11:30:30.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4746999999999995e-05 2024-09-18T11:30:30.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:30.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.657719+0000", 2024-09-18T11:30:30.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000127792 2024-09-18T11:30:30.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:30.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.659995+0000", 2024-09-18T11:30:30.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002276311 2024-09-18T11:30:30.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:30.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.662186+0000", 2024-09-18T11:30:30.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0021902720000000001 2024-09-18T11:30:30.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:30.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.662220+0000", 2024-09-18T11:30:30.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4029000000000002e-05 2024-09-18T11:30:30.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:30.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.662249+0000", 2024-09-18T11:30:30.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.904e-05 2024-09-18T11:30:30.266 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.266 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:30.266 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.266 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1194 4.2 4:425afe9c:::smithi11939506-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:30.266 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:21.188172+0000", 2024-09-18T11:30:30.266 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.9598624789999999, 2024-09-18T11:30:30.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049333449999999996, 2024-09-18T11:30:30.267 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:30.267 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:30.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:30.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:30.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:30.267 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1194 2024-09-18T11:30:30.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.267 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:30.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:30.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188172+0000", 2024-09-18T11:30:30.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:30.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188172+0000", 2024-09-18T11:30:30.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:30.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188172+0000", 2024-09-18T11:30:30.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:30.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:30.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188174+0000", 2024-09-18T11:30:30.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:30:30.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:30.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188180+0000", 2024-09-18T11:30:30.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7209999999999999e-06 2024-09-18T11:30:30.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:30.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188183+0000", 2024-09-18T11:30:30.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1690000000000001e-06 2024-09-18T11:30:30.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.190148+0000", 2024-09-18T11:30:30.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019649450000000001 2024-09-18T11:30:30.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:30.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.190291+0000", 2024-09-18T11:30:30.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014339800000000001 2024-09-18T11:30:30.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.192937+0000", 2024-09-18T11:30:30.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026458459999999999 2024-09-18T11:30:30.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:30.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.193002+0000", 2024-09-18T11:30:30.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5168999999999999e-05 2024-09-18T11:30:30.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:30.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.193105+0000", 2024-09-18T11:30:30.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103224 2024-09-18T11:30:30.272 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.272 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:30.272 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.272 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1198 4.f 4:f269e3d0:::smithi11939506-24:head [tier-promote] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:30.272 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:21.188205+0000", 2024-09-18T11:30:30.272 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.9598291279999991, 2024-09-18T11:30:30.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0071523649999999999, 2024-09-18T11:30:30.272 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:30.272 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:30.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:30.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:30.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:30.273 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1198 2024-09-18T11:30:30.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.273 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:30.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:30.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188205+0000", 2024-09-18T11:30:30.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:30.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188205+0000", 2024-09-18T11:30:30.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:30.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:30.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188205+0000", 2024-09-18T11:30:30.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:30.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:30.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188207+0000", 2024-09-18T11:30:30.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5730000000000002e-06 2024-09-18T11:30:30.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:30.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188213+0000", 2024-09-18T11:30:30.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7640000000000002e-06 2024-09-18T11:30:30.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:30.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188218+0000", 2024-09-18T11:30:30.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0499999999999999e-06 2024-09-18T11:30:30.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:30.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188254+0000", 2024-09-18T11:30:30.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5407999999999997e-05 2024-09-18T11:30:30.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:30.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188321+0000", 2024-09-18T11:30:30.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7682999999999996e-05 2024-09-18T11:30:30.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:30.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:30.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:30.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.195358+0000", 2024-09-18T11:30:30.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070362890000000003 2024-09-18T11:30:30.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.276 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:30.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:30.276 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:30.276 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:30.278 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:30:30.278 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is unset 2024-09-18T11:30:30.279 INFO:tasks.rados.rados.0.smithi119.stdout:832: done (3 left) 2024-09-18T11:30:30.279 INFO:tasks.rados.rados.0.smithi119.stdout:833: snap_create 2024-09-18T11:30:30.279 INFO:tasks.rados.rados.0.smithi119.stdout:829: finishing rollback tid 0 to smithi11939506-46 2024-09-18T11:30:30.279 INFO:tasks.rados.rados.0.smithi119.stdout:829: finishing rollback tid 1 to smithi11939506-46 2024-09-18T11:30:30.279 INFO:tasks.rados.rados.0.smithi119.stdout:829: finishing rollback tid 2 to smithi11939506-46 2024-09-18T11:30:30.279 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 45 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-18T11:30:30.280 INFO:tasks.rados.rados.0.smithi119.stdout:831: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:30:30.281 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:30.277+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:30.373 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:30:30.417 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:30.416+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:30.419 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:30.419 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:30.419 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:30.419 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:30.428 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:30:30.480 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:30.478+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:30.489 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:30:30.490 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:30:30.506 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:30.507 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:30.521 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:30.522 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:30.536 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:30.537 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:30.552 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:30.553 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:30.568 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:30.568 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:30.571 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:30.571 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:30.579 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:30:30.584 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:30.584 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:30.585 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:30.583+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:30.600 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:30.601 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:30.682 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:30.681+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:30.682 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:30.683 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:30.691 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:30:30.783 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:30.782+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:30.789 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:30.789 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:30.798 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:30:30.884 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:30.883+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:30.895 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:30.895 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:30.904 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:30:30.988 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:30.987+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:31.043 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:31.044 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:31.052 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:30:31.093 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:31.092+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:31.162 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:31.162 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:31.171 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:30:31.188 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:31.187+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:31.226 INFO:tasks.rados.rados.0.smithi119.stdout:829: done (3 left) 2024-09-18T11:30:31.226 INFO:tasks.rados.rados.0.smithi119.stdout:830: done (2 left) 2024-09-18T11:30:31.226 INFO:tasks.rados.rados.0.smithi119.stdout:831: done (1 left) 2024-09-18T11:30:31.226 INFO:tasks.rados.rados.0.smithi119.stdout:833: done (0 left) 2024-09-18T11:30:31.226 INFO:tasks.rados.rados.0.smithi119.stdout:834: snap_create 2024-09-18T11:30:31.272 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:31.272 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:31.281 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:30:31.324 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:31.323+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:31.378 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:31.378 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:31.387 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:30:31.390 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:31.389+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:31.491 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:31.490+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:31.526 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:31.526 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:31.534 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:30:31.592 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:31.591+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:31.631 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:31.632 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:31.640 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:30:31.692 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:31.691+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:31.738 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:31.738 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:31.747 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:30:31.794 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:31.793+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:31.845 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:31.846 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:31.854 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:30:31.895 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:31.894+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:31.952 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:31.952 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:31.960 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:30:32.004 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:32.003+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:32.058 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:32.058 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:32.067 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:30:32.096 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:32.095+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:32.198 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:32.196+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:32.206 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:32.206 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:32.214 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:30:32.230 INFO:tasks.rados.rados.0.smithi119.stdout:834: done (0 left) 2024-09-18T11:30:32.230 INFO:tasks.rados.rados.0.smithi119.stdout:835: tier_evict oid 12 2024-09-18T11:30:32.230 INFO:tasks.rados.rados.0.smithi119.stdout:835: tier_evict oid 12 snap -1 2024-09-18T11:30:32.230 INFO:tasks.rados.rados.0.smithi119.stdout:836: tier_promote oid 27 2024-09-18T11:30:32.230 INFO:tasks.rados.rados.0.smithi119.stdout:837: write (excl) oid 45 current snap is 11 2024-09-18T11:30:32.230 INFO:tasks.rados.rados.0.smithi119.stdout:837: seq_num 113 ranges {699191=532079,1779524=567357,2837151=666208} 2024-09-18T11:30:32.237 INFO:tasks.rados.rados.0.smithi119.stdout:837: writing smithi11939506-45 from 699191 to 1231270 tid 1 2024-09-18T11:30:32.243 INFO:tasks.rados.rados.0.smithi119.stdout:837: writing smithi11939506-45 from 1779524 to 2346881 tid 2 2024-09-18T11:30:32.250 INFO:tasks.rados.rados.0.smithi119.stdout:837: writing smithi11939506-45 from 2837151 to 3503359 tid 3 2024-09-18T11:30:32.250 INFO:tasks.rados.rados.0.smithi119.stdout:838: write oid 37 current snap is 11 2024-09-18T11:30:32.251 INFO:tasks.rados.rados.0.smithi119.stdout:835: got (22) Invalid argument 2024-09-18T11:30:32.251 INFO:tasks.rados.rados.0.smithi119.stdout:838: seq_num 114 ranges {715998=607328,1838006=427368,2864574=477480,3863835=52163} 2024-09-18T11:30:32.257 INFO:tasks.rados.rados.0.smithi119.stdout:838: writing smithi11939506-37 from 715998 to 1323326 tid 1 2024-09-18T11:30:32.260 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:32.257+0000 7fd2785a3640 -1 osd.6 50 handle_osd_map injecting map crc failure 2024-09-18T11:30:32.262 INFO:tasks.rados.rados.0.smithi119.stdout:838: writing smithi11939506-37 from 1838006 to 2265374 tid 2 2024-09-18T11:30:32.268 INFO:tasks.rados.rados.0.smithi119.stdout:838: writing smithi11939506-37 from 2864574 to 3342054 tid 3 2024-09-18T11:30:32.271 INFO:tasks.rados.rados.0.smithi119.stdout:838: writing smithi11939506-37 from 3863835 to 3915998 tid 4 2024-09-18T11:30:32.271 INFO:tasks.rados.rados.0.smithi119.stdout:835: done (3 left) 2024-09-18T11:30:32.271 INFO:tasks.rados.rados.0.smithi119.stdout:839: write oid 26 current snap is 11 2024-09-18T11:30:32.271 INFO:tasks.rados.rados.0.smithi119.stdout:839: seq_num 115 ranges {732805=798930,2296488=571026,3291997=405105} 2024-09-18T11:30:32.279 INFO:tasks.rados.rados.0.smithi119.stdout:839: writing smithi11939506-26 from 732805 to 1531735 tid 1 2024-09-18T11:30:32.281 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T11:30:32.286 INFO:tasks.rados.rados.0.smithi119.stdout:839: writing smithi11939506-26 from 2296488 to 2867514 tid 2 2024-09-18T11:30:32.291 INFO:tasks.rados.rados.0.smithi119.stdout:839: writing smithi11939506-26 from 3291997 to 3697102 tid 3 2024-09-18T11:30:32.291 INFO:tasks.rados.rados.0.smithi119.stdout:840: write oid 18 current snap is 11 2024-09-18T11:30:32.291 INFO:tasks.rados.rados.0.smithi119.stdout:840: seq_num 116 ranges {749612=474179,1954970=431037,2919420=616377} 2024-09-18T11:30:32.298 INFO:tasks.rados.rados.0.smithi119.stdout:840: writing smithi11939506-18 from 749612 to 1223791 tid 1 2024-09-18T11:30:32.298 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:32.297+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:32.304 INFO:tasks.rados.rados.0.smithi119.stdout:840: writing smithi11939506-18 from 1954970 to 2386007 tid 2 2024-09-18T11:30:32.311 INFO:tasks.rados.rados.0.smithi119.stdout:840: writing smithi11939506-18 from 2919420 to 3535797 tid 3 2024-09-18T11:30:32.311 INFO:tasks.rados.rados.0.smithi119.stdout:841: read oid 7 snap -1 2024-09-18T11:30:32.311 INFO:tasks.rados.rados.0.smithi119.stdout:841: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-18T11:30:32.312 INFO:tasks.rados.rados.0.smithi119.stdout:842: read oid 42 snap -1 2024-09-18T11:30:32.312 INFO:tasks.rados.rados.0.smithi119.stdout:842: expect (ObjNum 99 snap 6 seq_num 99) 2024-09-18T11:30:32.312 INFO:tasks.rados.rados.0.smithi119.stdout:843: tier_promote oid 3 2024-09-18T11:30:32.312 INFO:tasks.rados.rados.0.smithi119.stdout:837: finishing write tid 1 to smithi11939506-45 2024-09-18T11:30:32.312 INFO:tasks.rados.rados.0.smithi119.stdout:836: got (0) Success 2024-09-18T11:30:32.312 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 6 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T11:30:32.312 INFO:tasks.rados.rados.0.smithi119.stdout:837: finishing write tid 2 to smithi11939506-45 2024-09-18T11:30:32.312 INFO:tasks.rados.rados.0.smithi119.stdout:836: done (7 left) 2024-09-18T11:30:32.312 INFO:tasks.rados.rados.0.smithi119.stdout:844: read oid 39 snap -1 2024-09-18T11:30:32.312 INFO:tasks.rados.rados.0.smithi119.stdout:844: expect deleted 2024-09-18T11:30:32.312 INFO:tasks.rados.rados.0.smithi119.stdout:837: finishing write tid 3 to smithi11939506-45 2024-09-18T11:30:32.312 INFO:tasks.rados.rados.0.smithi119.stdout:837: finishing write tid 4 to smithi11939506-45 2024-09-18T11:30:32.313 INFO:tasks.rados.rados.0.smithi119.stdout:837: finishing write tid 5 to smithi11939506-45 2024-09-18T11:30:32.313 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 15 (ObjNum 113 snap 11 seq_num 113) dirty exists 2024-09-18T11:30:32.313 INFO:tasks.rados.rados.0.smithi119.stdout:837: left oid 45 (ObjNum 113 snap 11 seq_num 113) 2024-09-18T11:30:32.313 INFO:tasks.rados.rados.0.smithi119.stdout:838: finishing write tid 1 to smithi11939506-37 2024-09-18T11:30:32.313 INFO:tasks.rados.rados.0.smithi119.stdout:845: read oid 27 snap -1 2024-09-18T11:30:32.313 INFO:tasks.rados.rados.0.smithi119.stdout:845: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-18T11:30:32.313 INFO:tasks.rados.rados.0.smithi119.stdout:837: done (8 left) 2024-09-18T11:30:32.313 INFO:tasks.rados.rados.0.smithi119.stdout:846: write (excl) oid 6 current snap is 11 2024-09-18T11:30:32.313 INFO:tasks.rados.rados.0.smithi119.stdout:846: seq_num 117 ranges {766419=549428,2013452=574695,3346843=427649} 2024-09-18T11:30:32.319 INFO:tasks.rados.rados.0.smithi119.stdout:846: writing smithi11939506-6 from 766419 to 1315847 tid 1 2024-09-18T11:30:32.328 INFO:tasks.rados.rados.0.smithi119.stdout:846: writing smithi11939506-6 from 2013452 to 2588147 tid 2 2024-09-18T11:30:32.336 INFO:tasks.rados.rados.0.smithi119.stdout:846: writing smithi11939506-6 from 3346843 to 3774492 tid 3 2024-09-18T11:30:32.336 INFO:tasks.rados.rados.0.smithi119.stdout:847: set_chunk oid 11 target oid 2024-09-18T11:30:32.349 INFO:tasks.rados.rados.0.smithi119.stdout:847: set_chunk oid 11 offset: 1363456 length: 1854976 target oid 2d6b732d315f6106b1a30740b78802c0014a2c0700a0c6eb7d53c5b00ebff73b offset: 0 snap -1 2024-09-18T11:30:32.349 INFO:tasks.rados.rados.0.smithi119.stdout:848: rollback oid 38 current snap is 11 2024-09-18T11:30:32.350 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 38 to 9 2024-09-18T11:30:32.350 INFO:tasks.rados.rados.0.smithi119.stdout:838: finishing write tid 2 to smithi11939506-37 2024-09-18T11:30:32.350 INFO:tasks.rados.rados.0.smithi119.stdout:838: finishing write tid 3 to smithi11939506-37 2024-09-18T11:30:32.350 INFO:tasks.rados.rados.0.smithi119.stdout:838: finishing write tid 4 to smithi11939506-37 2024-09-18T11:30:32.350 INFO:tasks.rados.rados.0.smithi119.stdout:849: write (excl) oid 1 current snap is 11 2024-09-18T11:30:32.350 INFO:tasks.rados.rados.0.smithi119.stdout:838: finishing write tid 5 to smithi11939506-37 2024-09-18T11:30:32.350 INFO:tasks.rados.rados.0.smithi119.stdout:849: seq_num 118 ranges {783226=624677,2071934=434706,2974266=638921} 2024-09-18T11:30:32.357 INFO:tasks.rados.rados.0.smithi119.stdout:849: writing smithi11939506-1 from 783226 to 1407903 tid 1 2024-09-18T11:30:32.363 INFO:tasks.rados.rados.0.smithi119.stdout:849: writing smithi11939506-1 from 2071934 to 2506640 tid 2 2024-09-18T11:30:32.365 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:32.365 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:32.369 INFO:tasks.rados.rados.0.smithi119.stdout:849: writing smithi11939506-1 from 2974266 to 3613187 tid 3 2024-09-18T11:30:32.369 INFO:tasks.rados.rados.0.smithi119.stdout:850: read oid 48 snap -1 2024-09-18T11:30:32.370 INFO:tasks.rados.rados.0.smithi119.stdout:850: expect (ObjNum 47 snap 0 seq_num 47) 2024-09-18T11:30:32.370 INFO:tasks.rados.rados.0.smithi119.stdout:851: tier_evict oid 21 2024-09-18T11:30:32.370 INFO:tasks.rados.rados.0.smithi119.stdout:851: tier_evict oid 21 snap -1 2024-09-18T11:30:32.370 INFO:tasks.rados.rados.0.smithi119.stdout:838: finishing write tid 6 to smithi11939506-37 2024-09-18T11:30:32.370 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 53 (ObjNum 114 snap 11 seq_num 114) dirty exists 2024-09-18T11:30:32.370 INFO:tasks.rados.rados.0.smithi119.stdout:838: left oid 37 (ObjNum 114 snap 11 seq_num 114) 2024-09-18T11:30:32.370 INFO:tasks.rados.rados.0.smithi119.stdout:839: finishing write tid 1 to smithi11939506-26 2024-09-18T11:30:32.370 INFO:tasks.rados.rados.0.smithi119.stdout:839: finishing write tid 2 to smithi11939506-26 2024-09-18T11:30:32.370 INFO:tasks.rados.rados.0.smithi119.stdout:838: done (13 left) 2024-09-18T11:30:32.370 INFO:tasks.rados.rados.0.smithi119.stdout:852: write oid 17 current snap is 11 2024-09-18T11:30:32.370 INFO:tasks.rados.rados.0.smithi119.stdout:839: finishing write tid 3 to smithi11939506-26 2024-09-18T11:30:32.370 INFO:tasks.rados.rados.0.smithi119.stdout:852: seq_num 119 ranges {400033=699926,1614063=385970} 2024-09-18T11:30:32.375 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:30:32.375 INFO:tasks.rados.rados.0.smithi119.stdout:852: writing smithi11939506-17 from 400033 to 1099959 tid 1 2024-09-18T11:30:32.380 INFO:tasks.rados.rados.0.smithi119.stdout:852: writing smithi11939506-17 from 1614063 to 2000033 tid 2 2024-09-18T11:30:32.381 INFO:tasks.rados.rados.0.smithi119.stdout:853: write oid 29 current snap is 11 2024-09-18T11:30:32.381 INFO:tasks.rados.rados.0.smithi119.stdout:839: finishing write tid 4 to smithi11939506-26 2024-09-18T11:30:32.381 INFO:tasks.rados.rados.0.smithi119.stdout:839: finishing write tid 5 to smithi11939506-26 2024-09-18T11:30:32.381 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 67 (ObjNum 115 snap 11 seq_num 115) dirty exists 2024-09-18T11:30:32.381 INFO:tasks.rados.rados.0.smithi119.stdout:839: left oid 26 (ObjNum 115 snap 11 seq_num 115) 2024-09-18T11:30:32.381 INFO:tasks.rados.rados.0.smithi119.stdout:840: finishing write tid 1 to smithi11939506-18 2024-09-18T11:30:32.381 INFO:tasks.rados.rados.0.smithi119.stdout:840: finishing write tid 2 to smithi11939506-18 2024-09-18T11:30:32.381 INFO:tasks.rados.rados.0.smithi119.stdout:853: seq_num 120 ranges {416840=775175,1672545=344295} 2024-09-18T11:30:32.386 INFO:tasks.rados.rados.0.smithi119.stdout:853: writing smithi11939506-29 from 416840 to 1192015 tid 1 2024-09-18T11:30:32.392 INFO:tasks.rados.rados.0.smithi119.stdout:853: writing smithi11939506-29 from 1672545 to 2016840 tid 2 2024-09-18T11:30:32.392 INFO:tasks.rados.rados.0.smithi119.stdout:839: done (14 left) 2024-09-18T11:30:32.392 INFO:tasks.rados.rados.0.smithi119.stdout:854: tier_promote oid 23 2024-09-18T11:30:32.392 INFO:tasks.rados.rados.0.smithi119.stdout:844: done (14 left) 2024-09-18T11:30:32.392 INFO:tasks.rados.rados.0.smithi119.stdout:855: read oid 9 snap 9 2024-09-18T11:30:32.392 INFO:tasks.rados.rados.0.smithi119.stdout:855: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T11:30:32.393 INFO:tasks.rados.rados.0.smithi119.stdout:856: write (excl) oid 31 current snap is 11 2024-09-18T11:30:32.393 INFO:tasks.rados.rados.0.smithi119.stdout:843: got (0) Success 2024-09-18T11:30:32.393 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 26 (ObjNum 77 snap 3 seq_num 77) dirty exists 2024-09-18T11:30:32.393 INFO:tasks.rados.rados.0.smithi119.stdout:840: finishing write tid 3 to smithi11939506-18 2024-09-18T11:30:32.393 INFO:tasks.rados.rados.0.smithi119.stdout:840: finishing write tid 4 to smithi11939506-18 2024-09-18T11:30:32.393 INFO:tasks.rados.rados.0.smithi119.stdout:840: finishing write tid 5 to smithi11939506-18 2024-09-18T11:30:32.393 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 39 (ObjNum 116 snap 11 seq_num 116) dirty exists 2024-09-18T11:30:32.393 INFO:tasks.rados.rados.0.smithi119.stdout:840: left oid 18 (ObjNum 116 snap 11 seq_num 116) 2024-09-18T11:30:32.393 INFO:tasks.rados.rados.0.smithi119.stdout:841: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-18T11:30:32.416 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:32.415+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:32.499 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:32.498+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:32.511 INFO:tasks.rados.rados.0.smithi119.stdout:842: expect (ObjNum 99 snap 6 seq_num 99) 2024-09-18T11:30:32.514 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:32.514 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:32.523 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:30:32.600 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:32.599+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:32.612 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:30:32.622 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:32.622 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:32.631 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:30:32.679 INFO:tasks.rados.rados.0.smithi119.stdout:845: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-18T11:30:32.701 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:32.700+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:32.729 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:30:32.729 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:30:32.744 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:32.745 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:32.759 INFO:tasks.rados.rados.0.smithi119.stdout:846: finishing write tid 1 to smithi11939506-6 2024-09-18T11:30:32.759 INFO:tasks.rados.rados.0.smithi119.stdout:846: finishing write tid 2 to smithi11939506-6 2024-09-18T11:30:32.759 INFO:tasks.rados.rados.0.smithi119.stdout:846: finishing write tid 3 to smithi11939506-6 2024-09-18T11:30:32.759 INFO:tasks.rados.rados.0.smithi119.stdout:846: finishing write tid 4 to smithi11939506-6 2024-09-18T11:30:32.759 INFO:tasks.rados.rados.0.smithi119.stdout:846: finishing write tid 5 to smithi11939506-6 2024-09-18T11:30:32.759 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 6 v 26 (ObjNum 117 snap 11 seq_num 117) dirty exists 2024-09-18T11:30:32.760 INFO:tasks.rados.rados.0.smithi119.stdout:846: left oid 6 (ObjNum 117 snap 11 seq_num 117) 2024-09-18T11:30:32.760 INFO:tasks.rados.rados.0.smithi119.stdout:848: finishing rollback tid 0 to smithi11939506-38 2024-09-18T11:30:32.760 INFO:tasks.rados.rados.0.smithi119.stdout:848: finishing rollback tid 1 to smithi11939506-38 2024-09-18T11:30:32.760 INFO:tasks.rados.rados.0.smithi119.stdout:856: seq_num 121 ranges {433647=450424,1331027=582033} 2024-09-18T11:30:32.761 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:32.761 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:32.764 INFO:tasks.rados.rados.0.smithi119.stdout:856: writing smithi11939506-31 from 433647 to 884071 tid 1 2024-09-18T11:30:32.770 INFO:tasks.rados.rados.0.smithi119.stdout:856: writing smithi11939506-31 from 1331027 to 1913060 tid 2 2024-09-18T11:30:32.770 INFO:tasks.rados.rados.0.smithi119.stdout:840: done (15 left) 2024-09-18T11:30:32.770 INFO:tasks.rados.rados.0.smithi119.stdout:841: done (14 left) 2024-09-18T11:30:32.770 INFO:tasks.rados.rados.0.smithi119.stdout:842: done (13 left) 2024-09-18T11:30:32.770 INFO:tasks.rados.rados.0.smithi119.stdout:843: done (12 left) 2024-09-18T11:30:32.770 INFO:tasks.rados.rados.0.smithi119.stdout:845: done (11 left) 2024-09-18T11:30:32.770 INFO:tasks.rados.rados.0.smithi119.stdout:846: done (10 left) 2024-09-18T11:30:32.770 INFO:tasks.rados.rados.0.smithi119.stdout:857: write (excl) oid 28 current snap is 11 2024-09-18T11:30:32.771 INFO:tasks.rados.rados.0.smithi119.stdout:857: seq_num 122 ranges {450454=642026,1505862=544592} 2024-09-18T11:30:32.776 INFO:tasks.rados.rados.0.smithi119.stdout:857: writing smithi11939506-28 from 450454 to 1092480 tid 1 2024-09-18T11:30:32.776 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:32.776 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:32.776 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:32.777 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:32.777 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:32.781 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:32.781 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:32.782 INFO:tasks.rados.rados.0.smithi119.stdout:857: writing smithi11939506-28 from 1505862 to 2050454 tid 2 2024-09-18T11:30:32.782 INFO:tasks.rados.rados.0.smithi119.stdout:858: read oid 3 snap 9 2024-09-18T11:30:32.783 INFO:tasks.rados.rados.0.smithi119.stdout:858: expect (ObjNum 77 snap 3 seq_num 77) 2024-09-18T11:30:32.783 INFO:tasks.rados.rados.0.smithi119.stdout:859: snap_create 2024-09-18T11:30:32.783 INFO:tasks.rados.rados.0.smithi119.stdout:847: finishing set_chunk to oid 11 2024-09-18T11:30:32.783 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 11 v 31 (ObjNum 81 snap 4 seq_num 81) dirty exists 2024-09-18T11:30:32.783 INFO:tasks.rados.rados.0.smithi119.stdout:848: finishing rollback tid 2 to smithi11939506-38 2024-09-18T11:30:32.783 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 43 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-18T11:30:32.783 INFO:tasks.rados.rados.0.smithi119.stdout:849: finishing write tid 1 to smithi11939506-1 2024-09-18T11:30:32.783 INFO:tasks.rados.rados.0.smithi119.stdout:849: finishing write tid 2 to smithi11939506-1 2024-09-18T11:30:32.783 INFO:tasks.rados.rados.0.smithi119.stdout:851: got (22) Invalid argument 2024-09-18T11:30:32.783 INFO:tasks.rados.rados.0.smithi119.stdout:849: finishing write tid 3 to smithi11939506-1 2024-09-18T11:30:32.783 INFO:tasks.rados.rados.0.smithi119.stdout:849: finishing write tid 4 to smithi11939506-1 2024-09-18T11:30:32.783 INFO:tasks.rados.rados.0.smithi119.stdout:849: finishing write tid 5 to smithi11939506-1 2024-09-18T11:30:32.783 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 64 (ObjNum 118 snap 11 seq_num 118) dirty exists 2024-09-18T11:30:32.784 INFO:tasks.rados.rados.0.smithi119.stdout:849: left oid 1 (ObjNum 118 snap 11 seq_num 118) 2024-09-18T11:30:32.784 INFO:tasks.rados.rados.0.smithi119.stdout:850: expect (ObjNum 47 snap 0 seq_num 47) 2024-09-18T11:30:32.789 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:30:32.800 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:32.800 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:32.802 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:32.800+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:32.815 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:32.815 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:32.831 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:32.831 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:32.846 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:32.846 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:32.876 INFO:tasks.rados.rados.0.smithi119.stdout:852: finishing write tid 1 to smithi11939506-17 2024-09-18T11:30:32.876 INFO:tasks.rados.rados.0.smithi119.stdout:852: finishing write tid 2 to smithi11939506-17 2024-09-18T11:30:32.876 INFO:tasks.rados.rados.0.smithi119.stdout:852: finishing write tid 3 to smithi11939506-17 2024-09-18T11:30:32.876 INFO:tasks.rados.rados.0.smithi119.stdout:852: finishing write tid 4 to smithi11939506-17 2024-09-18T11:30:32.877 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 75 (ObjNum 119 snap 11 seq_num 119) dirty exists 2024-09-18T11:30:32.877 INFO:tasks.rados.rados.0.smithi119.stdout:852: left oid 17 (ObjNum 119 snap 11 seq_num 119) 2024-09-18T11:30:32.877 INFO:tasks.rados.rados.0.smithi119.stdout:853: finishing write tid 1 to smithi11939506-29 2024-09-18T11:30:32.877 INFO:tasks.rados.rados.0.smithi119.stdout:855: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-18T11:30:32.902 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:32.902 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:32.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:32.902 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:32.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.902 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4206.0:79 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'57, mlcod=15'53)", 2024-09-18T11:30:32.902 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:49.586176+0000", 2024-09-18T11:30:32.902 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.312683036999999, 2024-09-18T11:30:32.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043147059999999997, 2024-09-18T11:30:32.903 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:32.903 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:32.903 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:32.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:32.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586176+0000", 2024-09-18T11:30:32.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:32.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586176+0000", 2024-09-18T11:30:32.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:32.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586170+0000", 2024-09-18T11:30:32.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999938 2024-09-18T11:30:32.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:32.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586282+0000", 2024-09-18T11:30:32.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112254 2024-09-18T11:30:32.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:32.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586307+0000", 2024-09-18T11:30:32.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5296e-05 2024-09-18T11:30:32.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:32.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.586321+0000", 2024-09-18T11:30:32.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3723000000000001e-05 2024-09-18T11:30:32.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:32.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.587754+0000", 2024-09-18T11:30:32.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014330829999999999 2024-09-18T11:30:32.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:32.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.587785+0000", 2024-09-18T11:30:32.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1176000000000001e-05 2024-09-18T11:30:32.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:32.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.590467+0000", 2024-09-18T11:30:32.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026815089999999999 2024-09-18T11:30:32.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:32.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:49.590491+0000", 2024-09-18T11:30:32.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3941e-05 2024-09-18T11:30:32.906 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.906 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:32.906 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.907 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:21 3.1 3:9f1cb8cf:::smithi11939506-38 low_tier:head [read 336384~173568 [fadvise_sequential]] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:32.907 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.118534+0000", 2024-09-18T11:30:32.907 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.780324845999999, 2024-09-18T11:30:32.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006633842, 2024-09-18T11:30:32.907 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:32.907 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:32.907 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:32.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:32.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118534+0000", 2024-09-18T11:30:32.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:32.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118534+0000", 2024-09-18T11:30:32.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:32.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118532+0000", 2024-09-18T11:30:32.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:32.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:32.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118542+0000", 2024-09-18T11:30:32.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0151e-05 2024-09-18T11:30:32.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:32.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118563+0000", 2024-09-18T11:30:32.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1089000000000001e-05 2024-09-18T11:30:32.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:32.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118572+0000", 2024-09-18T11:30:32.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1090000000000004e-06 2024-09-18T11:30:32.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:32.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118633+0000", 2024-09-18T11:30:32.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0940000000000003e-05 2024-09-18T11:30:32.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:32.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118640+0000", 2024-09-18T11:30:32.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3690000000000003e-06 2024-09-18T11:30:32.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:32.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124910+0000", 2024-09-18T11:30:32.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062697569999999999 2024-09-18T11:30:32.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:32.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124980+0000", 2024-09-18T11:30:32.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0846000000000006e-05 2024-09-18T11:30:32.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:32.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.125168+0000", 2024-09-18T11:30:32.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018786200000000001 2024-09-18T11:30:32.912 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.912 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:32.912 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.912 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:22 3.1 3:9f1cb8cf:::smithi11939506-38 low_tier:head [read 1235456~1340416 [fadvise_sequential]] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:32.912 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:32.912 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.780272117999999, 2024-09-18T11:30:32.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0090773199999999998, 2024-09-18T11:30:32.913 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:32.913 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:32.913 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:32.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:32.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:32.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:32.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:32.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:32.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:32.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:32.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:32.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118594+0000", 2024-09-18T11:30:32.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1590000000000004e-06 2024-09-18T11:30:32.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:32.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118601+0000", 2024-09-18T11:30:32.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3900000000000004e-06 2024-09-18T11:30:32.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:32.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118605+0000", 2024-09-18T11:30:32.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2479999999999998e-06 2024-09-18T11:30:32.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:32.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118697+0000", 2024-09-18T11:30:32.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1447999999999996e-05 2024-09-18T11:30:32.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:32.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118702+0000", 2024-09-18T11:30:32.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6670000000000001e-06 2024-09-18T11:30:32.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:32.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.125285+0000", 2024-09-18T11:30:32.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006582124 2024-09-18T11:30:32.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:32.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.125349+0000", 2024-09-18T11:30:32.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4777000000000007e-05 2024-09-18T11:30:32.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:32.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.127664+0000", 2024-09-18T11:30:32.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023151069999999998 2024-09-18T11:30:32.917 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.917 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:32.917 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.917 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:911 4.3 4:cc46532a:::smithi11939506-33:head [write 1904434~719345] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:32.917 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.166081+0000", 2024-09-18T11:30:32.917 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.732778052, 2024-09-18T11:30:32.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081214100000000008, 2024-09-18T11:30:32.917 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:32.918 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:32.918 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:32.918 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:32.918 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:32.918 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 911 2024-09-18T11:30:32.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.918 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:32.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:32.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166081+0000", 2024-09-18T11:30:32.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:32.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166081+0000", 2024-09-18T11:30:32.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:32.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166080+0000", 2024-09-18T11:30:32.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:32.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:32.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166386+0000", 2024-09-18T11:30:32.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030589499999999998 2024-09-18T11:30:32.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:32.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166398+0000", 2024-09-18T11:30:32.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1678000000000001e-05 2024-09-18T11:30:32.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:32.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166402+0000", 2024-09-18T11:30:32.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6059999999999996e-06 2024-09-18T11:30:32.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:32.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166443+0000", 2024-09-18T11:30:32.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1044000000000002e-05 2024-09-18T11:30:32.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:32.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166526+0000", 2024-09-18T11:30:32.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2281999999999996e-05 2024-09-18T11:30:32.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:32.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166661+0000", 2024-09-18T11:30:32.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013516299999999999 2024-09-18T11:30:32.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:32.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.168437+0000", 2024-09-18T11:30:32.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001775794 2024-09-18T11:30:32.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:32.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174126+0000", 2024-09-18T11:30:32.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056894859999999997 2024-09-18T11:30:32.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:32.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174166+0000", 2024-09-18T11:30:32.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9787999999999999e-05 2024-09-18T11:30:32.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:32.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174203+0000", 2024-09-18T11:30:32.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6644999999999999e-05 2024-09-18T11:30:32.924 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.924 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:32.924 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.924 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:912 4.3 4:cc46532a:::smithi11939506-33:head [write 3128657~31026] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:32.924 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169139+0000", 2024-09-18T11:30:32.924 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.729720378, 2024-09-18T11:30:32.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051669280000000003, 2024-09-18T11:30:32.924 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:32.925 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:32.925 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:32.925 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:32.925 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:32.925 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 912 2024-09-18T11:30:32.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.925 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:32.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:32.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169139+0000", 2024-09-18T11:30:32.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:32.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169139+0000", 2024-09-18T11:30:32.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:32.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169138+0000", 2024-09-18T11:30:32.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:32.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:32.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169176+0000", 2024-09-18T11:30:32.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7679000000000003e-05 2024-09-18T11:30:32.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:32.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169187+0000", 2024-09-18T11:30:32.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1353e-05 2024-09-18T11:30:32.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:32.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169193+0000", 2024-09-18T11:30:32.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0390000000000001e-06 2024-09-18T11:30:32.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:32.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169230+0000", 2024-09-18T11:30:32.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6397999999999999e-05 2024-09-18T11:30:32.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:32.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169302+0000", 2024-09-18T11:30:32.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2922000000000004e-05 2024-09-18T11:30:32.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:32.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169426+0000", 2024-09-18T11:30:32.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012302600000000001 2024-09-18T11:30:32.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:32.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174078+0000", 2024-09-18T11:30:32.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046524330000000001 2024-09-18T11:30:32.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:32.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174240+0000", 2024-09-18T11:30:32.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016188899999999999 2024-09-18T11:30:32.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:32.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174275+0000", 2024-09-18T11:30:32.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5556000000000003e-05 2024-09-18T11:30:32.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:32.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174306+0000", 2024-09-18T11:30:32.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0408000000000001e-05 2024-09-18T11:30:32.932 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.932 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:32.932 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.932 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:913 4.3 4:cc46532a:::smithi11939506-33:head [setxattr _header (56) in=7b,truncate 3159683] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:32.932 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169203+0000", 2024-09-18T11:30:32.932 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.729656613, 2024-09-18T11:30:32.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052201790000000001, 2024-09-18T11:30:32.932 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:32.933 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:32.933 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:32.933 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:32.933 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:32.933 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 913 2024-09-18T11:30:32.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.933 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:32.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:32.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169203+0000", 2024-09-18T11:30:32.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:32.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169203+0000", 2024-09-18T11:30:32.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:32.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169202+0000", 2024-09-18T11:30:32.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:32.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:32.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169206+0000", 2024-09-18T11:30:32.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8160000000000004e-06 2024-09-18T11:30:32.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:32.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169212+0000", 2024-09-18T11:30:32.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3770000000000001e-06 2024-09-18T11:30:32.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:32.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169216+0000", 2024-09-18T11:30:32.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0060000000000003e-06 2024-09-18T11:30:32.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:32.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169888+0000", 2024-09-18T11:30:32.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00067139700000000001 2024-09-18T11:30:32.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:32.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169963+0000", 2024-09-18T11:30:32.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4999000000000005e-05 2024-09-18T11:30:32.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:32.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.170077+0000", 2024-09-18T11:30:32.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011401899999999999 2024-09-18T11:30:32.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:32.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174084+0000", 2024-09-18T11:30:32.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040071550000000001 2024-09-18T11:30:32.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:32.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174336+0000", 2024-09-18T11:30:32.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00025225300000000003 2024-09-18T11:30:32.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:32.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174371+0000", 2024-09-18T11:30:32.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5361000000000002e-05 2024-09-18T11:30:32.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:32.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174423+0000", 2024-09-18T11:30:32.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1347999999999997e-05 2024-09-18T11:30:32.939 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.939 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:32.939 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.939 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:914 4.3 4:cc46532a:::smithi11939506-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:32.939 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169236+0000", 2024-09-18T11:30:32.939 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.729623093000001, 2024-09-18T11:30:32.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005369965, 2024-09-18T11:30:32.939 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:32.939 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:32.939 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:32.939 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:32.939 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:32.940 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 914 2024-09-18T11:30:32.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.940 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:32.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:32.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169236+0000", 2024-09-18T11:30:32.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:32.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169236+0000", 2024-09-18T11:30:32.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:32.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169236+0000", 2024-09-18T11:30:32.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:32.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:32.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169241+0000", 2024-09-18T11:30:32.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8239999999999996e-06 2024-09-18T11:30:32.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:32.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169249+0000", 2024-09-18T11:30:32.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9319999999999996e-06 2024-09-18T11:30:32.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:32.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169254+0000", 2024-09-18T11:30:32.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3220000000000003e-06 2024-09-18T11:30:32.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:32.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.170304+0000", 2024-09-18T11:30:32.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001050184 2024-09-18T11:30:32.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:32.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.170390+0000", 2024-09-18T11:30:32.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5839000000000003e-05 2024-09-18T11:30:32.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:32.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174455+0000", 2024-09-18T11:30:32.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040656119999999997 2024-09-18T11:30:32.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:32.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174525+0000", 2024-09-18T11:30:32.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9151000000000003e-05 2024-09-18T11:30:32.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:32.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174606+0000", 2024-09-18T11:30:32.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1588000000000006e-05 2024-09-18T11:30:32.944 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.944 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:32.944 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.944 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:923 4.b 4:db5b7794:::smithi11939506-13:head [read 0~2873964,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:32.944 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169380+0000", 2024-09-18T11:30:32.945 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.729478968999999, 2024-09-18T11:30:32.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047044239999999996, 2024-09-18T11:30:32.945 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:32.945 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:32.945 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:32.945 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:32.945 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:32.945 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 923 2024-09-18T11:30:32.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.945 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:32.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:32.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169380+0000", 2024-09-18T11:30:32.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:32.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169380+0000", 2024-09-18T11:30:32.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:32.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169380+0000", 2024-09-18T11:30:32.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:32.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:32.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169383+0000", 2024-09-18T11:30:32.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1429999999999998e-06 2024-09-18T11:30:32.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:32.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169389+0000", 2024-09-18T11:30:32.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4579999999999998e-06 2024-09-18T11:30:32.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:32.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169393+0000", 2024-09-18T11:30:32.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7840000000000001e-06 2024-09-18T11:30:32.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:32.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.172647+0000", 2024-09-18T11:30:32.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032532849999999999 2024-09-18T11:30:32.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:32.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.172724+0000", 2024-09-18T11:30:32.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7836999999999994e-05 2024-09-18T11:30:32.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:32.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174085+0000", 2024-09-18T11:30:32.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013603580000000001 2024-09-18T11:30:32.948 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.949 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:32.949 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.949 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:924 4.b 4:db5b7794:::smithi11939506-13:head [sparse-read 0~1546056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:32.949 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169399+0000", 2024-09-18T11:30:32.949 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.729460625000002, 2024-09-18T11:30:32.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054498719999999997, 2024-09-18T11:30:32.949 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:32.949 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:32.949 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:32.949 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:32.949 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:32.950 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 924 2024-09-18T11:30:32.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.950 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:32.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:32.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169399+0000", 2024-09-18T11:30:32.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:32.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169399+0000", 2024-09-18T11:30:32.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:32.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169398+0000", 2024-09-18T11:30:32.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:32.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:32.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169401+0000", 2024-09-18T11:30:32.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4490000000000002e-06 2024-09-18T11:30:32.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:32.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169407+0000", 2024-09-18T11:30:32.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9490000000000001e-06 2024-09-18T11:30:32.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:32.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169411+0000", 2024-09-18T11:30:32.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.924e-06 2024-09-18T11:30:32.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:32.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174189+0000", 2024-09-18T11:30:32.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047787039999999999 2024-09-18T11:30:32.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:32.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174263+0000", 2024-09-18T11:30:32.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3301999999999997e-05 2024-09-18T11:30:32.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:32.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174849+0000", 2024-09-18T11:30:32.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00058598500000000002 2024-09-18T11:30:32.953 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.953 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:32.954 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.954 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:989 4.3 4:ca2b2309:::smithi11939506-44:head [write 410104~465105,stat] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:32.954 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.151379+0000", 2024-09-18T11:30:32.954 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.747480088, 2024-09-18T11:30:32.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062831420000000002, 2024-09-18T11:30:32.954 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:32.954 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:32.954 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:32.954 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:32.954 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:32.954 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 989 2024-09-18T11:30:32.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.955 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:32.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:32.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151379+0000", 2024-09-18T11:30:32.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:32.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151379+0000", 2024-09-18T11:30:32.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:32.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151376+0000", 2024-09-18T11:30:32.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:32.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:32.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151666+0000", 2024-09-18T11:30:32.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028977899999999999 2024-09-18T11:30:32.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:32.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151697+0000", 2024-09-18T11:30:32.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0516000000000001e-05 2024-09-18T11:30:32.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:32.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151707+0000", 2024-09-18T11:30:32.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0322999999999999e-05 2024-09-18T11:30:32.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:32.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151770+0000", 2024-09-18T11:30:32.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879000000000006e-05 2024-09-18T11:30:32.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:32.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151783+0000", 2024-09-18T11:30:32.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3154e-05 2024-09-18T11:30:32.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:32.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.153878+0000", 2024-09-18T11:30:32.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002095155 2024-09-18T11:30:32.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:32.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.153957+0000", 2024-09-18T11:30:32.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8877999999999999e-05 2024-09-18T11:30:32.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:32.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.154073+0000", 2024-09-18T11:30:32.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116045 2024-09-18T11:30:32.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:32.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.156031+0000", 2024-09-18T11:30:32.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019581099999999999 2024-09-18T11:30:32.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:32.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.157569+0000", 2024-09-18T11:30:32.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015383199999999999 2024-09-18T11:30:32.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:32.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.157606+0000", 2024-09-18T11:30:32.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6278000000000001e-05 2024-09-18T11:30:32.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:32.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.157662+0000", 2024-09-18T11:30:32.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6589000000000003e-05 2024-09-18T11:30:32.960 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.960 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:32.960 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.961 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:990 4.3 4:ca2b2309:::smithi11939506-44:head [write 1563527~583025] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:32.961 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.158301+0000", 2024-09-18T11:30:32.961 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.740558578000002, 2024-09-18T11:30:32.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049105670000000002, 2024-09-18T11:30:32.961 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:32.961 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:32.961 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:32.961 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:32.961 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:32.961 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 990 2024-09-18T11:30:32.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.961 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:32.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:32.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158301+0000", 2024-09-18T11:30:32.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:32.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158301+0000", 2024-09-18T11:30:32.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:32.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158299+0000", 2024-09-18T11:30:32.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:32.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:32.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158457+0000", 2024-09-18T11:30:32.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000157374 2024-09-18T11:30:32.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:32.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158469+0000", 2024-09-18T11:30:32.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1969e-05 2024-09-18T11:30:32.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:32.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158475+0000", 2024-09-18T11:30:32.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4450000000000001e-06 2024-09-18T11:30:32.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:32.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158504+0000", 2024-09-18T11:30:32.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8924999999999998e-05 2024-09-18T11:30:32.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:32.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158568+0000", 2024-09-18T11:30:32.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3781000000000005e-05 2024-09-18T11:30:32.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:32.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.158677+0000", 2024-09-18T11:30:32.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010893899999999999 2024-09-18T11:30:32.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:32.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.160238+0000", 2024-09-18T11:30:32.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015608340000000001 2024-09-18T11:30:32.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:32.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.163135+0000", 2024-09-18T11:30:32.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028975400000000001 2024-09-18T11:30:32.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:32.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.163166+0000", 2024-09-18T11:30:32.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1164999999999997e-05 2024-09-18T11:30:32.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:32.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.163211+0000", 2024-09-18T11:30:32.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4836000000000002e-05 2024-09-18T11:30:32.967 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.967 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:32.967 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.967 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1012 4.b 4:db5b7794:::smithi11939506-13:head [write 443718~615603] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:32.967 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.655768+0000", 2024-09-18T11:30:32.967 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.243091573000001, 2024-09-18T11:30:32.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055941680000000001, 2024-09-18T11:30:32.967 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:32.967 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:32.967 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:32.967 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:32.967 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:32.967 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1012 2024-09-18T11:30:32.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.968 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:32.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:32.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.655768+0000", 2024-09-18T11:30:32.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:32.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.655768+0000", 2024-09-18T11:30:32.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:32.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.655765+0000", 2024-09-18T11:30:32.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:32.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:32.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656140+0000", 2024-09-18T11:30:32.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00037503800000000001 2024-09-18T11:30:32.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:32.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656166+0000", 2024-09-18T11:30:32.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5451000000000001e-05 2024-09-18T11:30:32.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:32.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656176+0000", 2024-09-18T11:30:32.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0489000000000001e-05 2024-09-18T11:30:32.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:32.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656255+0000", 2024-09-18T11:30:32.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8880000000000004e-05 2024-09-18T11:30:32.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:32.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656322+0000", 2024-09-18T11:30:32.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6500000000000004e-05 2024-09-18T11:30:32.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:32.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656436+0000", 2024-09-18T11:30:32.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114945 2024-09-18T11:30:32.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:32.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.658992+0000", 2024-09-18T11:30:32.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025551419999999998 2024-09-18T11:30:32.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:32.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.661302+0000", 2024-09-18T11:30:32.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002310543 2024-09-18T11:30:32.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:32.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.661335+0000", 2024-09-18T11:30:32.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2716999999999999e-05 2024-09-18T11:30:32.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:32.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.661362+0000", 2024-09-18T11:30:32.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6998000000000001e-05 2024-09-18T11:30:32.972 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.973 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:32.973 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.973 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1016 4.b 4:db5b7794:::smithi11939506-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:32.973 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.668055+0000", 2024-09-18T11:30:32.973 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.230804000999999, 2024-09-18T11:30:32.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044191300000000003, 2024-09-18T11:30:32.973 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:32.973 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:32.973 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:32.973 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:32.974 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:32.974 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1016 2024-09-18T11:30:32.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.974 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:32.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:32.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.668055+0000", 2024-09-18T11:30:32.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:32.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.668055+0000", 2024-09-18T11:30:32.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:32.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.668055+0000", 2024-09-18T11:30:32.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:32.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:32.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.668057+0000", 2024-09-18T11:30:32.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6340000000000001e-06 2024-09-18T11:30:32.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:32.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.668064+0000", 2024-09-18T11:30:32.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5150000000000003e-06 2024-09-18T11:30:32.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:32.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.668067+0000", 2024-09-18T11:30:32.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.552e-06 2024-09-18T11:30:32.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:32.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.669678+0000", 2024-09-18T11:30:32.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016104699999999999 2024-09-18T11:30:32.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:32.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.669752+0000", 2024-09-18T11:30:32.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4084000000000004e-05 2024-09-18T11:30:32.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:32.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.672307+0000", 2024-09-18T11:30:32.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002554551 2024-09-18T11:30:32.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:32.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.672379+0000", 2024-09-18T11:30:32.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2256999999999996e-05 2024-09-18T11:30:32.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:32.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.672474+0000", 2024-09-18T11:30:32.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5561000000000002e-05 2024-09-18T11:30:32.981 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.981 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:32.981 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.981 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1150 4.b 4:d3ae8215:::smithi11939506-41:head [write 1807440~632344] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:32.982 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.620640+0000", 2024-09-18T11:30:32.982 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.278219400999999, 2024-09-18T11:30:32.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068244000000000004, 2024-09-18T11:30:32.982 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:32.982 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:32.982 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:32.983 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:32.983 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:32.983 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1150 2024-09-18T11:30:32.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.983 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:32.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:32.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.620640+0000", 2024-09-18T11:30:32.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:32.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.620640+0000", 2024-09-18T11:30:32.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:32.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.620638+0000", 2024-09-18T11:30:32.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:32.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:32.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621224+0000", 2024-09-18T11:30:32.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00058580899999999996 2024-09-18T11:30:32.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:32.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621235+0000", 2024-09-18T11:30:32.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:30:32.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:32.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621240+0000", 2024-09-18T11:30:32.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4270000000000001e-06 2024-09-18T11:30:32.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:32.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621268+0000", 2024-09-18T11:30:32.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8252999999999999e-05 2024-09-18T11:30:32.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:32.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621344+0000", 2024-09-18T11:30:32.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6154000000000003e-05 2024-09-18T11:30:32.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:32.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621464+0000", 2024-09-18T11:30:32.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012021 2024-09-18T11:30:32.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:32.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.624135+0000", 2024-09-18T11:30:32.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026711310000000002 2024-09-18T11:30:32.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:32.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.627420+0000", 2024-09-18T11:30:32.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032846799999999999 2024-09-18T11:30:32.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:32.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.627443+0000", 2024-09-18T11:30:32.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3153e-05 2024-09-18T11:30:32.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:32.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.627464+0000", 2024-09-18T11:30:32.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0967000000000001e-05 2024-09-18T11:30:32.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.994 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:32.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:32.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.995 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1152 4.b 4:d3ae8215:::smithi11939506-41:head [setxattr _header (56) in=7b,truncate 3428595] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:32.995 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.625951+0000", 2024-09-18T11:30:32.995 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.272908716, 2024-09-18T11:30:32.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044856540000000004, 2024-09-18T11:30:32.995 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:32.995 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:32.996 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:32.996 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:32.996 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:32.996 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1152 2024-09-18T11:30:32.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.996 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:32.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:32.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625951+0000", 2024-09-18T11:30:32.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:32.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625951+0000", 2024-09-18T11:30:32.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:32.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:32.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625950+0000", 2024-09-18T11:30:32.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:32.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:32.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:32.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625954+0000", 2024-09-18T11:30:33.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7090000000000001e-06 2024-09-18T11:30:33.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625960+0000", 2024-09-18T11:30:33.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9209999999999997e-06 2024-09-18T11:30:33.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625963+0000", 2024-09-18T11:30:33.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0189999999999998e-06 2024-09-18T11:30:33.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.626714+0000", 2024-09-18T11:30:33.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00075054400000000002 2024-09-18T11:30:33.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.626784+0000", 2024-09-18T11:30:33.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9961000000000003e-05 2024-09-18T11:30:33.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:33.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.626899+0000", 2024-09-18T11:30:33.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011521100000000001 2024-09-18T11:30:33.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:33.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.627477+0000", 2024-09-18T11:30:33.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00057745099999999998 2024-09-18T11:30:33.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:33.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.630329+0000", 2024-09-18T11:30:33.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028520529999999998 2024-09-18T11:30:33.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.630377+0000", 2024-09-18T11:30:33.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7939999999999998e-05 2024-09-18T11:30:33.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.630436+0000", 2024-09-18T11:30:33.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9676000000000003e-05 2024-09-18T11:30:33.008 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.008 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.008 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.009 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1153 4.b 4:d3ae8215:::smithi11939506-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:33.009 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.625968+0000", 2024-09-18T11:30:33.009 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.272891147999999, 2024-09-18T11:30:33.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046073240000000003, 2024-09-18T11:30:33.009 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.010 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:33.010 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:33.010 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:33.010 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:33.010 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1153 2024-09-18T11:30:33.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.011 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625968+0000", 2024-09-18T11:30:33.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625968+0000", 2024-09-18T11:30:33.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625968+0000", 2024-09-18T11:30:33.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625969+0000", 2024-09-18T11:30:33.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.468e-06 2024-09-18T11:30:33.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625978+0000", 2024-09-18T11:30:33.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4309999999999994e-06 2024-09-18T11:30:33.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.625981+0000", 2024-09-18T11:30:33.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5250000000000001e-06 2024-09-18T11:30:33.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.627082+0000", 2024-09-18T11:30:33.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0011011149999999999 2024-09-18T11:30:33.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:33.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.627163+0000", 2024-09-18T11:30:33.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1112999999999994e-05 2024-09-18T11:30:33.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.630468+0000", 2024-09-18T11:30:33.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0033042309999999999 2024-09-18T11:30:33.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.630521+0000", 2024-09-18T11:30:33.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3108e-05 2024-09-18T11:30:33.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.630575+0000", 2024-09-18T11:30:33.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4599999999999999e-05 2024-09-18T11:30:33.020 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.020 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.020 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.020 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:31 3.1 3:88ab294d:::972cde73ee90bbfa4e1e8219b21f333678dc7dbfb3d5501c6973feb14420b94c:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:33.020 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.678405+0000", 2024-09-18T11:30:33.020 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.220454731, 2024-09-18T11:30:33.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046395250000000002, 2024-09-18T11:30:33.021 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.021 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.021 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.678405+0000", 2024-09-18T11:30:33.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.678405+0000", 2024-09-18T11:30:33.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.678402+0000", 2024-09-18T11:30:33.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:33.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679284+0000", 2024-09-18T11:30:33.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00088187200000000002 2024-09-18T11:30:33.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679305+0000", 2024-09-18T11:30:33.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1183999999999999e-05 2024-09-18T11:30:33.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679313+0000", 2024-09-18T11:30:33.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2649999999999996e-06 2024-09-18T11:30:33.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679385+0000", 2024-09-18T11:30:33.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2119000000000005e-05 2024-09-18T11:30:33.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679510+0000", 2024-09-18T11:30:33.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000124831 2024-09-18T11:30:33.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:33.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679727+0000", 2024-09-18T11:30:33.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000217061 2024-09-18T11:30:33.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:33.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680985+0000", 2024-09-18T11:30:33.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001258295 2024-09-18T11:30:33.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:33.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682957+0000", 2024-09-18T11:30:33.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019718740000000002 2024-09-18T11:30:33.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.683007+0000", 2024-09-18T11:30:33.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9740000000000001e-05 2024-09-18T11:30:33.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.683044+0000", 2024-09-18T11:30:33.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7082999999999997e-05 2024-09-18T11:30:33.026 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.026 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.026 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.026 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:32 3.1 3:94708c4c:::888f815178b2a748104fffa559dcb609f143f519d8737c6aa160c9cce13a6441:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:33.026 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.679331+0000", 2024-09-18T11:30:33.026 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.219528632999999, 2024-09-18T11:30:33.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063965929999999999, 2024-09-18T11:30:33.026 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.027 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.027 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679331+0000", 2024-09-18T11:30:33.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679331+0000", 2024-09-18T11:30:33.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679330+0000", 2024-09-18T11:30:33.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:33.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681259+0000", 2024-09-18T11:30:33.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019287079999999999 2024-09-18T11:30:33.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681274+0000", 2024-09-18T11:30:33.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5903e-05 2024-09-18T11:30:33.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681281+0000", 2024-09-18T11:30:33.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5640000000000002e-06 2024-09-18T11:30:33.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681325+0000", 2024-09-18T11:30:33.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3925999999999999e-05 2024-09-18T11:30:33.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681435+0000", 2024-09-18T11:30:33.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109739 2024-09-18T11:30:33.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:33.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681642+0000", 2024-09-18T11:30:33.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000206901 2024-09-18T11:30:33.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:33.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.683180+0000", 2024-09-18T11:30:33.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015380929999999999 2024-09-18T11:30:33.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:33.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.685648+0000", 2024-09-18T11:30:33.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024687419999999999 2024-09-18T11:30:33.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.685695+0000", 2024-09-18T11:30:33.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7030000000000002e-05 2024-09-18T11:30:33.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.685727+0000", 2024-09-18T11:30:33.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1792000000000003e-05 2024-09-18T11:30:33.032 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.032 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.032 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.033 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1204 4.6 4:6f86ffcc:::smithi11939506-15:head [write 779858~469642,stat] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-18T11:30:33.033 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:21.214976+0000", 2024-09-18T11:30:33.033 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 11.683883639999999, 2024-09-18T11:30:33.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050506320000000002, 2024-09-18T11:30:33.033 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.033 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.033 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:33.033 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:33.033 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:33.033 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1204 2024-09-18T11:30:33.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.033 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.214976+0000", 2024-09-18T11:30:33.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.214976+0000", 2024-09-18T11:30:33.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.214974+0000", 2024-09-18T11:30:33.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:33.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.215075+0000", 2024-09-18T11:30:33.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100777 2024-09-18T11:30:33.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.215091+0000", 2024-09-18T11:30:33.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6413e-05 2024-09-18T11:30:33.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.215097+0000", 2024-09-18T11:30:33.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4179999999999996e-06 2024-09-18T11:30:33.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.215148+0000", 2024-09-18T11:30:33.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1162000000000002e-05 2024-09-18T11:30:33.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:33.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.215153+0000", 2024-09-18T11:30:33.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0849999999999996e-06 2024-09-18T11:30:33.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.216731+0000", 2024-09-18T11:30:33.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015780670000000001 2024-09-18T11:30:33.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.216818+0000", 2024-09-18T11:30:33.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7121999999999997e-05 2024-09-18T11:30:33.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:33.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.216999+0000", 2024-09-18T11:30:33.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000181051 2024-09-18T11:30:33.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:33.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.219747+0000", 2024-09-18T11:30:33.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027480529999999999 2024-09-18T11:30:33.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:33.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.219955+0000", 2024-09-18T11:30:33.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020780399999999999 2024-09-18T11:30:33.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.219987+0000", 2024-09-18T11:30:33.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2320000000000002e-05 2024-09-18T11:30:33.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.220026+0000", 2024-09-18T11:30:33.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8962999999999999e-05 2024-09-18T11:30:33.039 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.039 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.039 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.040 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:43 3.1 3:8abdb616:::smithi11939506-46 low_tier:head [read 2311680~395776 [fadvise_sequential]] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected+supports_pool_eio e42)", 2024-09-18T11:30:33.040 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:23.795465+0000", 2024-09-18T11:30:33.040 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.1033941899999995, 2024-09-18T11:30:33.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004375416, 2024-09-18T11:30:33.040 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.040 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:33.040 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:23.795465+0000", 2024-09-18T11:30:33.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:23.795465+0000", 2024-09-18T11:30:33.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:23.795465+0000", 2024-09-18T11:30:33.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:23.795467+0000", 2024-09-18T11:30:33.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.791e-06 2024-09-18T11:30:33.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:23.795474+0000", 2024-09-18T11:30:33.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6220000000000003e-06 2024-09-18T11:30:33.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:23.795478+0000", 2024-09-18T11:30:33.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9160000000000003e-06 2024-09-18T11:30:33.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:23.798921+0000", 2024-09-18T11:30:33.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034431010000000001 2024-09-18T11:30:33.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:23.799026+0000", 2024-09-18T11:30:33.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105447 2024-09-18T11:30:33.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:23.799840+0000", 2024-09-18T11:30:33.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00081406100000000004 2024-09-18T11:30:33.044 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.044 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.044 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.044 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.044 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.044 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:33.046 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:30:33.047 INFO:tasks.rados.rados.0.smithi119.stdout:853: finishing write tid 2 to smithi11939506-29 2024-09-18T11:30:33.047 INFO:tasks.rados.rados.0.smithi119.stdout:853: finishing write tid 3 to smithi11939506-29 2024-09-18T11:30:33.047 INFO:tasks.rados.rados.0.smithi119.stdout:853: finishing write tid 4 to smithi11939506-29 2024-09-18T11:30:33.047 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 32 (ObjNum 120 snap 11 seq_num 120) dirty exists 2024-09-18T11:30:33.047 INFO:tasks.rados.rados.0.smithi119.stdout:853: left oid 29 (ObjNum 120 snap 11 seq_num 120) 2024-09-18T11:30:33.047 INFO:tasks.rados.rados.0.smithi119.stdout:854: got (0) Success 2024-09-18T11:30:33.047 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 48 (ObjNum 90 snap 5 seq_num 90) dirty exists 2024-09-18T11:30:33.047 INFO:tasks.rados.rados.0.smithi119.stdout:856: finishing write tid 1 to smithi11939506-31 2024-09-18T11:30:33.047 INFO:tasks.rados.rados.0.smithi119.stdout:856: finishing write tid 2 to smithi11939506-31 2024-09-18T11:30:33.047 INFO:tasks.rados.rados.0.smithi119.stdout:856: finishing write tid 3 to smithi11939506-31 2024-09-18T11:30:33.047 INFO:tasks.rados.rados.0.smithi119.stdout:856: finishing write tid 4 to smithi11939506-31 2024-09-18T11:30:33.047 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 66 (ObjNum 121 snap 11 seq_num 121) dirty exists 2024-09-18T11:30:33.048 INFO:tasks.rados.rados.0.smithi119.stdout:856: left oid 31 (ObjNum 121 snap 11 seq_num 121) 2024-09-18T11:30:33.048 INFO:tasks.rados.rados.0.smithi119.stdout:857: finishing write tid 1 to smithi11939506-28 2024-09-18T11:30:33.048 INFO:tasks.rados.rados.0.smithi119.stdout:858: expect (ObjNum 77 snap 3 seq_num 77) 2024-09-18T11:30:33.048 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is unset 2024-09-18T11:30:33.049 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:33.044+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:33.133 INFO:tasks.rados.rados.0.smithi119.stdout:857: finishing write tid 2 to smithi11939506-28 2024-09-18T11:30:33.133 INFO:tasks.rados.rados.0.smithi119.stdout:857: finishing write tid 3 to smithi11939506-28 2024-09-18T11:30:33.133 INFO:tasks.rados.rados.0.smithi119.stdout:857: finishing write tid 4 to smithi11939506-28 2024-09-18T11:30:33.134 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 28 v 52 (ObjNum 122 snap 11 seq_num 122) dirty exists 2024-09-18T11:30:33.134 INFO:tasks.rados.rados.0.smithi119.stdout:857: left oid 28 (ObjNum 122 snap 11 seq_num 122) 2024-09-18T11:30:33.134 INFO:tasks.rados.rados.0.smithi119.stdout:847: done (12 left) 2024-09-18T11:30:33.134 INFO:tasks.rados.rados.0.smithi119.stdout:848: done (11 left) 2024-09-18T11:30:33.134 INFO:tasks.rados.rados.0.smithi119.stdout:849: done (10 left) 2024-09-18T11:30:33.134 INFO:tasks.rados.rados.0.smithi119.stdout:850: done (9 left) 2024-09-18T11:30:33.134 INFO:tasks.rados.rados.0.smithi119.stdout:851: done (8 left) 2024-09-18T11:30:33.134 INFO:tasks.rados.rados.0.smithi119.stdout:852: done (7 left) 2024-09-18T11:30:33.134 INFO:tasks.rados.rados.0.smithi119.stdout:853: done (6 left) 2024-09-18T11:30:33.134 INFO:tasks.rados.rados.0.smithi119.stdout:854: done (5 left) 2024-09-18T11:30:33.134 INFO:tasks.rados.rados.0.smithi119.stdout:855: done (4 left) 2024-09-18T11:30:33.134 INFO:tasks.rados.rados.0.smithi119.stdout:856: done (3 left) 2024-09-18T11:30:33.134 INFO:tasks.rados.rados.0.smithi119.stdout:857: done (2 left) 2024-09-18T11:30:33.134 INFO:tasks.rados.rados.0.smithi119.stdout:858: done (1 left) 2024-09-18T11:30:33.135 INFO:tasks.rados.rados.0.smithi119.stdout:859: done (0 left) 2024-09-18T11:30:33.135 INFO:tasks.rados.rados.0.smithi119.stdout:860: tier_flush oid 1 2024-09-18T11:30:33.135 INFO:tasks.rados.rados.0.smithi119.stdout:860: tier_flush oid 1 snap -1 2024-09-18T11:30:33.135 INFO:tasks.rados.rados.0.smithi119.stdout:861: read oid 4 snap -1 2024-09-18T11:30:33.135 INFO:tasks.rados.rados.0.smithi119.stdout:861: expect (ObjNum 110 snap 8 seq_num 110) 2024-09-18T11:30:33.135 INFO:tasks.rados.rados.0.smithi119.stdout:862: read oid 24 snap -1 2024-09-18T11:30:33.135 INFO:tasks.rados.rados.0.smithi119.stdout:862: expect (ObjNum 109 snap 8 seq_num 109) 2024-09-18T11:30:33.135 INFO:tasks.rados.rados.0.smithi119.stdout:863: snap_remove snap 9 2024-09-18T11:30:33.146 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:33.145+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:33.188 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:33.188 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:33.189 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:33.189 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:33.197 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:30:33.247 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:33.245+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:33.295 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:33.295 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:33.295 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:33.295 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:33.295 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:33.303 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:30:33.348 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:33.346+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:33.403 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:33.403 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:33.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:33.403 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:33.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.404 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:887 4.7 4:e0d95e10:::smithi11939506-22:head [read 0~2352947,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:33.404 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:30:33.404 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.284228565999999, 2024-09-18T11:30:33.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079115169999999999, 2024-09-18T11:30:33.404 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.404 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:33.404 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:33.404 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:33.404 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:33.404 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 887 2024-09-18T11:30:33.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.404 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:30:33.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:30:33.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116365+0000", 2024-09-18T11:30:33.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:33.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116376+0000", 2024-09-18T11:30:33.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1471000000000001e-05 2024-09-18T11:30:33.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116394+0000", 2024-09-18T11:30:33.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8088e-05 2024-09-18T11:30:33.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116402+0000", 2024-09-18T11:30:33.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2320000000000004e-06 2024-09-18T11:30:33.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116458+0000", 2024-09-18T11:30:33.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6536000000000002e-05 2024-09-18T11:30:33.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:33.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116466+0000", 2024-09-18T11:30:33.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0819999999999999e-06 2024-09-18T11:30:33.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.122509+0000", 2024-09-18T11:30:33.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060431879999999997 2024-09-18T11:30:33.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.122577+0000", 2024-09-18T11:30:33.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7646999999999998e-05 2024-09-18T11:30:33.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124278+0000", 2024-09-18T11:30:33.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017013200000000001 2024-09-18T11:30:33.408 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.408 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.409 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.409 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:888 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:33.409 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:33.409 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.284175102999999, 2024-09-18T11:30:33.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082282120000000004, 2024-09-18T11:30:33.409 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.409 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:33.409 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:33.409 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:33.409 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:33.409 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 888 2024-09-18T11:30:33.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.410 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:33.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:33.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:33.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116423+0000", 2024-09-18T11:30:33.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1e-06 2024-09-18T11:30:33.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:30:33.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5510000000000001e-06 2024-09-18T11:30:33.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116431+0000", 2024-09-18T11:30:33.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6109999999999998e-06 2024-09-18T11:30:33.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116527+0000", 2024-09-18T11:30:33.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6212999999999995e-05 2024-09-18T11:30:33.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:33.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116532+0000", 2024-09-18T11:30:33.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6140000000000002e-06 2024-09-18T11:30:33.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124314+0000", 2024-09-18T11:30:33.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077818929999999998 2024-09-18T11:30:33.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124411+0000", 2024-09-18T11:30:33.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6992999999999998e-05 2024-09-18T11:30:33.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124649+0000", 2024-09-18T11:30:33.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237729 2024-09-18T11:30:33.414 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.415 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.415 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.415 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:889 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:33.415 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:33.416 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.284150452999999, 2024-09-18T11:30:33.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085383349999999993, 2024-09-18T11:30:33.416 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.416 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:33.417 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:33.417 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:33.417 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:33.417 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 889 2024-09-18T11:30:33.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.417 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:33.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:33.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:33.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116447+0000", 2024-09-18T11:30:33.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.854e-06 2024-09-18T11:30:33.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116452+0000", 2024-09-18T11:30:33.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0250000000000002e-06 2024-09-18T11:30:33.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116455+0000", 2024-09-18T11:30:33.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8739999999999999e-06 2024-09-18T11:30:33.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116565+0000", 2024-09-18T11:30:33.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110741 2024-09-18T11:30:33.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:33.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116571+0000", 2024-09-18T11:30:33.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4990000000000002e-06 2024-09-18T11:30:33.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124691+0000", 2024-09-18T11:30:33.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081200339999999999 2024-09-18T11:30:33.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124784+0000", 2024-09-18T11:30:33.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2886999999999993e-05 2024-09-18T11:30:33.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124983+0000", 2024-09-18T11:30:33.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019969000000000001 2024-09-18T11:30:33.422 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.423 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.423 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.423 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:46 4.a e31/22 4:552523b6:::smithi11939506-38:head v 31'20, mlcod=30'18)", 2024-09-18T11:30:33.423 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:30:33.423 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.270198821000001, 2024-09-18T11:30:33.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080208420000000002, 2024-09-18T11:30:33.423 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.423 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.423 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:30:33.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:30:33.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130396+0000", 2024-09-18T11:30:33.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133245+0000", 2024-09-18T11:30:33.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028486929999999998 2024-09-18T11:30:33.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133253+0000", 2024-09-18T11:30:33.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1540000000000002e-06 2024-09-18T11:30:33.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133261+0000", 2024-09-18T11:30:33.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3620000000000006e-06 2024-09-18T11:30:33.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133319+0000", 2024-09-18T11:30:33.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7719000000000001e-05 2024-09-18T11:30:33.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133341+0000", 2024-09-18T11:30:33.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1946999999999998e-05 2024-09-18T11:30:33.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138368+0000", 2024-09-18T11:30:33.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005026712 2024-09-18T11:30:33.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138418+0000", 2024-09-18T11:30:33.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9864000000000002e-05 2024-09-18T11:30:33.427 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.427 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.427 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.427 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:905 4.7 4:e73ba68f:::smithi11939506-1:head [write 742876~447756,stat] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:33.427 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.140411+0000", 2024-09-18T11:30:33.428 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.260184123999998, 2024-09-18T11:30:33.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060845960000000003, 2024-09-18T11:30:33.428 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.428 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.428 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:33.428 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:33.428 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:33.428 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 905 2024-09-18T11:30:33.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.428 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140411+0000", 2024-09-18T11:30:33.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140411+0000", 2024-09-18T11:30:33.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140408+0000", 2024-09-18T11:30:33.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:30:33.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140676+0000", 2024-09-18T11:30:33.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026841300000000001 2024-09-18T11:30:33.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140708+0000", 2024-09-18T11:30:33.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1952000000000002e-05 2024-09-18T11:30:33.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140721+0000", 2024-09-18T11:30:33.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3005e-05 2024-09-18T11:30:33.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140788+0000", 2024-09-18T11:30:33.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6839000000000002e-05 2024-09-18T11:30:33.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.140923+0000", 2024-09-18T11:30:33.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000135258 2024-09-18T11:30:33.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:33.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.141159+0000", 2024-09-18T11:30:33.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023572900000000001 2024-09-18T11:30:33.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:33.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.143764+0000", 2024-09-18T11:30:33.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026044919999999999 2024-09-18T11:30:33.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:33.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.146411+0000", 2024-09-18T11:30:33.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026475130000000002 2024-09-18T11:30:33.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.146448+0000", 2024-09-18T11:30:33.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6650999999999999e-05 2024-09-18T11:30:33.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.146496+0000", 2024-09-18T11:30:33.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8177000000000003e-05 2024-09-18T11:30:33.433 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.433 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.434 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.434 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:995 4.f 4:f269e3d0:::smithi11939506-24:head [read 0~2386561,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:33.434 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164480+0000", 2024-09-18T11:30:33.434 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.236115794, 2024-09-18T11:30:33.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057271170000000003, 2024-09-18T11:30:33.434 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.434 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:33.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:33.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:33.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:33.434 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 995 2024-09-18T11:30:33.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.435 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164480+0000", 2024-09-18T11:30:33.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164480+0000", 2024-09-18T11:30:33.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164479+0000", 2024-09-18T11:30:33.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164483+0000", 2024-09-18T11:30:33.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9859999999999996e-06 2024-09-18T11:30:33.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164497+0000", 2024-09-18T11:30:33.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3559e-05 2024-09-18T11:30:33.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164506+0000", 2024-09-18T11:30:33.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3959999999999995e-06 2024-09-18T11:30:33.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164573+0000", 2024-09-18T11:30:33.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7123000000000001e-05 2024-09-18T11:30:33.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:33.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164577+0000", 2024-09-18T11:30:33.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9999999999999998e-06 2024-09-18T11:30:33.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.168696+0000", 2024-09-18T11:30:33.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041188379999999997 2024-09-18T11:30:33.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.168824+0000", 2024-09-18T11:30:33.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012762099999999999 2024-09-18T11:30:33.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170207+0000", 2024-09-18T11:30:33.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001383076 2024-09-18T11:30:33.439 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.439 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.439 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.439 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:996 4.f 4:f269e3d0:::smithi11939506-24:head [read 0~2386561,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:33.439 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164511+0000", 2024-09-18T11:30:33.439 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.236084222999999, 2024-09-18T11:30:33.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069965679999999999, 2024-09-18T11:30:33.439 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.439 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:33.440 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:33.440 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:33.440 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:33.440 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 996 2024-09-18T11:30:33.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.440 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164511+0000", 2024-09-18T11:30:33.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164511+0000", 2024-09-18T11:30:33.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164511+0000", 2024-09-18T11:30:33.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164514+0000", 2024-09-18T11:30:33.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3210000000000001e-06 2024-09-18T11:30:33.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164521+0000", 2024-09-18T11:30:33.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5169999999999996e-06 2024-09-18T11:30:33.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164524+0000", 2024-09-18T11:30:33.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:30:33.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164674+0000", 2024-09-18T11:30:33.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014943199999999999 2024-09-18T11:30:33.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:33.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:30:33.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8579999999999996e-06 2024-09-18T11:30:33.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170262+0000", 2024-09-18T11:30:33.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055827719999999997 2024-09-18T11:30:33.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170405+0000", 2024-09-18T11:30:33.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000143115 2024-09-18T11:30:33.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.171508+0000", 2024-09-18T11:30:33.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001102448 2024-09-18T11:30:33.446 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.446 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.447 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.447 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1121 4.f 4:f5d7df41:::smithi11939506-16:head [write 1515641~485017] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:33.447 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.965485+0000", 2024-09-18T11:30:33.447 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.43511019, 2024-09-18T11:30:33.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006622869, 2024-09-18T11:30:33.447 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.447 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.447 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:33.447 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:33.447 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:33.447 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1121 2024-09-18T11:30:33.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.448 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965485+0000", 2024-09-18T11:30:33.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965485+0000", 2024-09-18T11:30:33.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965484+0000", 2024-09-18T11:30:33.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:33.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965572+0000", 2024-09-18T11:30:33.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7934000000000002e-05 2024-09-18T11:30:33.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965583+0000", 2024-09-18T11:30:33.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0594999999999999e-05 2024-09-18T11:30:33.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965587+0000", 2024-09-18T11:30:33.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3800000000000004e-06 2024-09-18T11:30:33.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965611+0000", 2024-09-18T11:30:33.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.404e-05 2024-09-18T11:30:33.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965665+0000", 2024-09-18T11:30:33.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3448000000000001e-05 2024-09-18T11:30:33.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:30:33.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965741+0000", 2024-09-18T11:30:33.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6264e-05 2024-09-18T11:30:33.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:33.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.969188+0000", 2024-09-18T11:30:33.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034471129999999999 2024-09-18T11:30:33.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:33.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.971990+0000", 2024-09-18T11:30:33.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028016809999999999 2024-09-18T11:30:33.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.972068+0000", 2024-09-18T11:30:33.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8671000000000006e-05 2024-09-18T11:30:33.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.972108+0000", 2024-09-18T11:30:33.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9765000000000003e-05 2024-09-18T11:30:33.453 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.453 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.453 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.453 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1126 3.0 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head [write 0~2520064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:33.453 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:30:33.453 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.418708475000001, 2024-09-18T11:30:33.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086197050000000001, 2024-09-18T11:30:33.454 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.454 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.454 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:33.454 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:33.454 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:33.454 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1126 2024-09-18T11:30:33.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.454 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:30:33.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:30:33.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981886+0000", 2024-09-18T11:30:33.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:33.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983017+0000", 2024-09-18T11:30:33.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00113131 2024-09-18T11:30:33.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983037+0000", 2024-09-18T11:30:33.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0231e-05 2024-09-18T11:30:33.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983044+0000", 2024-09-18T11:30:33.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9539999999999998e-06 2024-09-18T11:30:33.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983087+0000", 2024-09-18T11:30:33.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3040000000000001e-05 2024-09-18T11:30:33.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983193+0000", 2024-09-18T11:30:33.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105682 2024-09-18T11:30:33.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:33.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983287+0000", 2024-09-18T11:30:33.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4066000000000004e-05 2024-09-18T11:30:33.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:33.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.986918+0000", 2024-09-18T11:30:33.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036311970000000001 2024-09-18T11:30:33.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:33.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990442+0000", 2024-09-18T11:30:33.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035243119999999999 2024-09-18T11:30:33.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990476+0000", 2024-09-18T11:30:33.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4028e-05 2024-09-18T11:30:33.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990507+0000", 2024-09-18T11:30:33.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0284999999999999e-05 2024-09-18T11:30:33.460 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.460 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.460 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.460 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1136 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~1201068,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:33.460 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.179006+0000", 2024-09-18T11:30:33.460 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.22158965, 2024-09-18T11:30:33.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006872197, 2024-09-18T11:30:33.461 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.461 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:33.461 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:33.461 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:33.461 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:33.461 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1136 2024-09-18T11:30:33.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.461 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179006+0000", 2024-09-18T11:30:33.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179006+0000", 2024-09-18T11:30:33.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179004+0000", 2024-09-18T11:30:33.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:33.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179013+0000", 2024-09-18T11:30:33.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3340000000000002e-06 2024-09-18T11:30:33.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179028+0000", 2024-09-18T11:30:33.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5150999999999999e-05 2024-09-18T11:30:33.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179035+0000", 2024-09-18T11:30:33.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4919999999999996e-06 2024-09-18T11:30:33.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179086+0000", 2024-09-18T11:30:33.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0402000000000003e-05 2024-09-18T11:30:33.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:33.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179092+0000", 2024-09-18T11:30:33.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1890000000000004e-06 2024-09-18T11:30:33.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.181504+0000", 2024-09-18T11:30:33.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024116340000000002 2024-09-18T11:30:33.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.181609+0000", 2024-09-18T11:30:33.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105337 2024-09-18T11:30:33.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.185878+0000", 2024-09-18T11:30:33.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042690779999999999 2024-09-18T11:30:33.466 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.466 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.466 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.466 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1137 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~1201068] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:33.466 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.179047+0000", 2024-09-18T11:30:33.466 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.221548823999999, 2024-09-18T11:30:33.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073446830000000003, 2024-09-18T11:30:33.467 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.467 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:33.467 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:33.467 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:33.467 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:33.467 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1137 2024-09-18T11:30:33.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.467 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179047+0000", 2024-09-18T11:30:33.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179047+0000", 2024-09-18T11:30:33.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179046+0000", 2024-09-18T11:30:33.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179049+0000", 2024-09-18T11:30:33.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.269e-06 2024-09-18T11:30:33.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179057+0000", 2024-09-18T11:30:33.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7279999999999995e-06 2024-09-18T11:30:33.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179061+0000", 2024-09-18T11:30:33.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3309999999999997e-06 2024-09-18T11:30:33.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179159+0000", 2024-09-18T11:30:33.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7027999999999993e-05 2024-09-18T11:30:33.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:33.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179166+0000", 2024-09-18T11:30:33.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8520000000000004e-06 2024-09-18T11:30:33.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.185971+0000", 2024-09-18T11:30:33.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068045570000000001 2024-09-18T11:30:33.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186111+0000", 2024-09-18T11:30:33.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000139906 2024-09-18T11:30:33.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186391+0000", 2024-09-18T11:30:33.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028053000000000003 2024-09-18T11:30:33.476 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.476 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.476 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.477 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1138 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~1201068] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:33.477 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.179081+0000", 2024-09-18T11:30:33.477 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.221514861999999, 2024-09-18T11:30:33.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007692099, 2024-09-18T11:30:33.478 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.478 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:33.478 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:33.478 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:33.478 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:33.479 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1138 2024-09-18T11:30:33.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.479 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179081+0000", 2024-09-18T11:30:33.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179081+0000", 2024-09-18T11:30:33.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179080+0000", 2024-09-18T11:30:33.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179083+0000", 2024-09-18T11:30:33.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.266e-06 2024-09-18T11:30:33.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179089+0000", 2024-09-18T11:30:33.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3589999999999996e-06 2024-09-18T11:30:33.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:33.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1999999999999996e-06 2024-09-18T11:30:33.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179194+0000", 2024-09-18T11:30:33.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101093 2024-09-18T11:30:33.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:33.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179199+0000", 2024-09-18T11:30:33.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8960000000000002e-06 2024-09-18T11:30:33.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186435+0000", 2024-09-18T11:30:33.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072360979999999998 2024-09-18T11:30:33.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186530+0000", 2024-09-18T11:30:33.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5006999999999993e-05 2024-09-18T11:30:33.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186773+0000", 2024-09-18T11:30:33.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000242496 2024-09-18T11:30:33.487 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.487 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.487 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.487 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1198 4.f 4:f269e3d0:::smithi11939506-24:head [tier-promote] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:33.487 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:21.188205+0000", 2024-09-18T11:30:33.487 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.212390189000001, 2024-09-18T11:30:33.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0071523649999999999, 2024-09-18T11:30:33.487 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.487 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:33.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:33.488 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:33.488 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:33.488 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1198 2024-09-18T11:30:33.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.488 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188205+0000", 2024-09-18T11:30:33.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188205+0000", 2024-09-18T11:30:33.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188205+0000", 2024-09-18T11:30:33.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188207+0000", 2024-09-18T11:30:33.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5730000000000002e-06 2024-09-18T11:30:33.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188213+0000", 2024-09-18T11:30:33.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7640000000000002e-06 2024-09-18T11:30:33.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188218+0000", 2024-09-18T11:30:33.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0499999999999999e-06 2024-09-18T11:30:33.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188254+0000", 2024-09-18T11:30:33.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5407999999999997e-05 2024-09-18T11:30:33.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188321+0000", 2024-09-18T11:30:33.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7682999999999996e-05 2024-09-18T11:30:33.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.195358+0000", 2024-09-18T11:30:33.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070362890000000003 2024-09-18T11:30:33.491 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.492 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.492 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.492 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1507 4.f 4:fc192b81:::smithi11939506-31:head [write 433647~450424,stat] snapc 13=[13,12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T11:30:33.492 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:32.763861+0000", 2024-09-18T11:30:33.492 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.636734468, 2024-09-18T11:30:33.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075480879999999997, 2024-09-18T11:30:33.492 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.492 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.492 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:33.492 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:33.492 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:33.493 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1507 2024-09-18T11:30:33.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.493 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.763861+0000", 2024-09-18T11:30:33.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.763861+0000", 2024-09-18T11:30:33.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.763860+0000", 2024-09-18T11:30:33.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:33.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.764059+0000", 2024-09-18T11:30:33.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019900500000000001 2024-09-18T11:30:33.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.764076+0000", 2024-09-18T11:30:33.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7558e-05 2024-09-18T11:30:33.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.764085+0000", 2024-09-18T11:30:33.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3119999999999994e-06 2024-09-18T11:30:33.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.764144+0000", 2024-09-18T11:30:33.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9880999999999998e-05 2024-09-18T11:30:33.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.764260+0000", 2024-09-18T11:30:33.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011586099999999999 2024-09-18T11:30:33.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:30:33.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.764485+0000", 2024-09-18T11:30:33.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022445200000000001 2024-09-18T11:30:33.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:33.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.771268+0000", 2024-09-18T11:30:33.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067833149999999998 2024-09-18T11:30:33.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:33.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.771322+0000", 2024-09-18T11:30:33.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4361999999999999e-05 2024-09-18T11:30:33.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.771354+0000", 2024-09-18T11:30:33.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1451000000000001e-05 2024-09-18T11:30:33.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.771409+0000", 2024-09-18T11:30:33.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5287000000000001e-05 2024-09-18T11:30:33.498 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.498 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.498 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.499 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1511 4.4 4:2c93196f:::smithi11939506-28:head [write 450454~642026,stat] snapc 13=[13,12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T11:30:33.499 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:32.776036+0000", 2024-09-18T11:30:33.499 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.624558999, 2024-09-18T11:30:33.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005866624, 2024-09-18T11:30:33.499 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.499 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.499 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:33.499 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:33.499 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:33.499 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1511 2024-09-18T11:30:33.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.500 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.776036+0000", 2024-09-18T11:30:33.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.776036+0000", 2024-09-18T11:30:33.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.776035+0000", 2024-09-18T11:30:33.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:33.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.776419+0000", 2024-09-18T11:30:33.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038366499999999999 2024-09-18T11:30:33.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.776437+0000", 2024-09-18T11:30:33.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8252000000000001e-05 2024-09-18T11:30:33.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.776444+0000", 2024-09-18T11:30:33.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0389999999999999e-06 2024-09-18T11:30:33.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.776500+0000", 2024-09-18T11:30:33.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5577e-05 2024-09-18T11:30:33.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.776602+0000", 2024-09-18T11:30:33.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102661 2024-09-18T11:30:33.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:33.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.776823+0000", 2024-09-18T11:30:33.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022037200000000001 2024-09-18T11:30:33.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:33.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.779532+0000", 2024-09-18T11:30:33.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027095119999999999 2024-09-18T11:30:33.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:33.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781787+0000", 2024-09-18T11:30:33.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022546020000000001 2024-09-18T11:30:33.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781849+0000", 2024-09-18T11:30:33.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2209e-05 2024-09-18T11:30:33.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781903+0000", 2024-09-18T11:30:33.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4160999999999999e-05 2024-09-18T11:30:33.505 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.505 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.506 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.506 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1512 4.4 4:2c93196f:::smithi11939506-28:head [write 1505862~544592] snapc 13=[13,12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T11:30:33.506 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:32.781533+0000", 2024-09-18T11:30:33.506 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.61906208500000004, 2024-09-18T11:30:33.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059120129999999998, 2024-09-18T11:30:33.507 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.507 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.507 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:33.507 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:33.507 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:33.507 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1512 2024-09-18T11:30:33.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.507 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781533+0000", 2024-09-18T11:30:33.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781533+0000", 2024-09-18T11:30:33.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781531+0000", 2024-09-18T11:30:33.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:33.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781865+0000", 2024-09-18T11:30:33.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000334111 2024-09-18T11:30:33.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781880+0000", 2024-09-18T11:30:33.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4630000000000001e-05 2024-09-18T11:30:33.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781898+0000", 2024-09-18T11:30:33.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7688000000000001e-05 2024-09-18T11:30:33.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781931+0000", 2024-09-18T11:30:33.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3092000000000001e-05 2024-09-18T11:30:33.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782006+0000", 2024-09-18T11:30:33.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5530000000000004e-05 2024-09-18T11:30:33.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:33.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782130+0000", 2024-09-18T11:30:33.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000123127 2024-09-18T11:30:33.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:33.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.787303+0000", 2024-09-18T11:30:33.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051732790000000002 2024-09-18T11:30:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.787361+0000", 2024-09-18T11:30:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7738999999999997e-05 2024-09-18T11:30:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.787406+0000", 2024-09-18T11:30:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5544000000000003e-05 2024-09-18T11:30:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.787445+0000", 2024-09-18T11:30:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9353000000000001e-05 2024-09-18T11:30:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1513 4.4 4:2c93196f:::smithi11939506-28:head [setxattr _header (57) in=7b,truncate 2050454] snapc 13=[13,12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T11:30:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:32.781909+0000", 2024-09-18T11:30:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.61868637500000001, 2024-09-18T11:30:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056293009999999997, 2024-09-18T11:30:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.515 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1513 2024-09-18T11:30:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781909+0000", 2024-09-18T11:30:33.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781909+0000", 2024-09-18T11:30:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781909+0000", 2024-09-18T11:30:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781912+0000", 2024-09-18T11:30:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4209999999999999e-06 2024-09-18T11:30:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781916+0000", 2024-09-18T11:30:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4880000000000001e-06 2024-09-18T11:30:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781919+0000", 2024-09-18T11:30:33.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9349999999999999e-06 2024-09-18T11:30:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783121+0000", 2024-09-18T11:30:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001201971 2024-09-18T11:30:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783205+0000", 2024-09-18T11:30:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3472000000000003e-05 2024-09-18T11:30:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783326+0000", 2024-09-18T11:30:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012066999999999999 2024-09-18T11:30:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.787352+0000", 2024-09-18T11:30:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040265029999999999 2024-09-18T11:30:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.787455+0000", 2024-09-18T11:30:33.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102599 2024-09-18T11:30:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.787493+0000", 2024-09-18T11:30:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8229000000000003e-05 2024-09-18T11:30:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.787538+0000", 2024-09-18T11:30:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5544000000000003e-05 2024-09-18T11:30:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.524 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1514 4.4 4:2c93196f:::smithi11939506-28:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-09-18T11:30:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:32.781927+0000", 2024-09-18T11:30:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.61866867699999994, 2024-09-18T11:30:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0058364829999999999, 2024-09-18T11:30:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:33.525 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1514 2024-09-18T11:30:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781927+0000", 2024-09-18T11:30:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781927+0000", 2024-09-18T11:30:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781927+0000", 2024-09-18T11:30:33.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781928+0000", 2024-09-18T11:30:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.934e-06 2024-09-18T11:30:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781932+0000", 2024-09-18T11:30:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6909999999999999e-06 2024-09-18T11:30:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.781935+0000", 2024-09-18T11:30:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7159999999999999e-06 2024-09-18T11:30:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783613+0000", 2024-09-18T11:30:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001678173 2024-09-18T11:30:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:33.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783712+0000", 2024-09-18T11:30:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8805e-05 2024-09-18T11:30:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.787571+0000", 2024-09-18T11:30:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003858761 2024-09-18T11:30:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.787667+0000", 2024-09-18T11:30:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5889000000000003e-05 2024-09-18T11:30:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.787763+0000", 2024-09-18T11:30:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6796e-05 2024-09-18T11:30:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1524 4.f 4:f269e3d0:::smithi11939506-24:head [read 0~3831963,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-18T11:30:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.132647+0000", 2024-09-18T11:30:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.26794836100000002, 2024-09-18T11:30:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0066940790000000003, 2024-09-18T11:30:33.532 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1524 2024-09-18T11:30:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132647+0000", 2024-09-18T11:30:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132647+0000", 2024-09-18T11:30:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132647+0000", 2024-09-18T11:30:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132651+0000", 2024-09-18T11:30:33.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9310000000000001e-06 2024-09-18T11:30:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132657+0000", 2024-09-18T11:30:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6540000000000002e-06 2024-09-18T11:30:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132661+0000", 2024-09-18T11:30:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1999999999999996e-06 2024-09-18T11:30:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132710+0000", 2024-09-18T11:30:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8711999999999997e-05 2024-09-18T11:30:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132714+0000", 2024-09-18T11:30:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4599999999999999e-06 2024-09-18T11:30:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.137571+0000", 2024-09-18T11:30:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004857765 2024-09-18T11:30:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.137683+0000", 2024-09-18T11:30:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011120600000000001 2024-09-18T11:30:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.139341+0000", 2024-09-18T11:30:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001658613 2024-09-18T11:30:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.537 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1525 4.f 4:f269e3d0:::smithi11939506-24:head [sparse-read 0~2011444] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-18T11:30:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.132667+0000", 2024-09-18T11:30:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.26792834900000001, 2024-09-18T11:30:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072757350000000002, 2024-09-18T11:30:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:33.538 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1525 2024-09-18T11:30:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132667+0000", 2024-09-18T11:30:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132667+0000", 2024-09-18T11:30:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132667+0000", 2024-09-18T11:30:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132669+0000", 2024-09-18T11:30:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3130000000000001e-06 2024-09-18T11:30:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132675+0000", 2024-09-18T11:30:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8109999999999999e-06 2024-09-18T11:30:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132678+0000", 2024-09-18T11:30:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.579e-06 2024-09-18T11:30:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132731+0000", 2024-09-18T11:30:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2809999999999998e-05 2024-09-18T11:30:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132737+0000", 2024-09-18T11:30:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6200000000000004e-06 2024-09-18T11:30:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.139405+0000", 2024-09-18T11:30:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0066678650000000003 2024-09-18T11:30:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.139521+0000", 2024-09-18T11:30:33.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115987 2024-09-18T11:30:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.139943+0000", 2024-09-18T11:30:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00042216999999999998 2024-09-18T11:30:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.543 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.543 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:33.545 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:30:33.546 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:33.543+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:33.645 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:33.645 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:33.645 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:33.645 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:33.645 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:33.643+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:33.653 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:30:33.746 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:33.744+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:33.750 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:33.750 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:33.750 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:33.751 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:33.751 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:33.759 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:30:33.846 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:33.845+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:33.859 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:33.859 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:33.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:33.859 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:33.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.859 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4266.0:1 2.3 e16/15 2:c4c92e5a:::rbd_trash:head v 16'1, mlcod=0'0)", 2024-09-18T11:30:33.859 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:29:50.028963+0000", 2024-09-18T11:30:33.859 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.827713768999999, 2024-09-18T11:30:33.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001253021, 2024-09-18T11:30:33.859 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.859 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.859 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.028963+0000", 2024-09-18T11:30:33.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.028963+0000", 2024-09-18T11:30:33.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.028963+0000", 2024-09-18T11:30:33.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.028967+0000", 2024-09-18T11:30:33.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2119999999999997e-06 2024-09-18T11:30:33.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.029011+0000", 2024-09-18T11:30:33.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4011000000000003e-05 2024-09-18T11:30:33.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.029019+0000", 2024-09-18T11:30:33.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.729e-06 2024-09-18T11:30:33.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.029185+0000", 2024-09-18T11:30:33.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016637600000000001 2024-09-18T11:30:33.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.029211+0000", 2024-09-18T11:30:33.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6211e-05 2024-09-18T11:30:33.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.030175+0000", 2024-09-18T11:30:33.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096347000000000002 2024-09-18T11:30:33.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:29:50.030216+0000", 2024-09-18T11:30:33.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.138e-05 2024-09-18T11:30:33.865 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.865 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.866 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.866 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1045 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'23, mlcod=30'20)", 2024-09-18T11:30:33.866 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.167583+0000", 2024-09-18T11:30:33.866 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.689093346, 2024-09-18T11:30:33.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031033380000000002, 2024-09-18T11:30:33.867 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.867 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.867 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167583+0000", 2024-09-18T11:30:33.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167583+0000", 2024-09-18T11:30:33.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167582+0000", 2024-09-18T11:30:33.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:33.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167934+0000", 2024-09-18T11:30:33.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000352262 2024-09-18T11:30:33.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167952+0000", 2024-09-18T11:30:33.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7859999999999998e-05 2024-09-18T11:30:33.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167960+0000", 2024-09-18T11:30:33.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9729999999999997e-06 2024-09-18T11:30:33.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.168008+0000", 2024-09-18T11:30:33.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7531000000000003e-05 2024-09-18T11:30:33.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.168032+0000", 2024-09-18T11:30:33.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4443e-05 2024-09-18T11:30:33.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.170646+0000", 2024-09-18T11:30:33.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026136739999999999 2024-09-18T11:30:33.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.170687+0000", 2024-09-18T11:30:33.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0859000000000003e-05 2024-09-18T11:30:33.874 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.875 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.875 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.875 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1046 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'24, mlcod=30'21)", 2024-09-18T11:30:33.875 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:30:33.875 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.682893566000001, 2024-09-18T11:30:33.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042519100000000002, 2024-09-18T11:30:33.876 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.876 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.876 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:30:33.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:30:33.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.878 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173782+0000", 2024-09-18T11:30:33.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:33.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174207+0000", 2024-09-18T11:30:33.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00042479799999999999 2024-09-18T11:30:33.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174224+0000", 2024-09-18T11:30:33.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7513999999999999e-05 2024-09-18T11:30:33.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174232+0000", 2024-09-18T11:30:33.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0840000000000003e-06 2024-09-18T11:30:33.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174297+0000", 2024-09-18T11:30:33.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5215000000000005e-05 2024-09-18T11:30:33.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.882 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.882 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174336+0000", 2024-09-18T11:30:33.882 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9403000000000002e-05 2024-09-18T11:30:33.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.882 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.883 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.177990+0000", 2024-09-18T11:30:33.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036533709999999999 2024-09-18T11:30:33.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.883 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.883 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.178035+0000", 2024-09-18T11:30:33.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5495999999999999e-05 2024-09-18T11:30:33.883 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.884 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.884 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.884 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1047 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'25, mlcod=35'23)", 2024-09-18T11:30:33.884 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:30:33.885 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.676921367999999, 2024-09-18T11:30:33.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038129129999999998, 2024-09-18T11:30:33.885 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.885 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.885 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:30:33.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:30:33.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179754+0000", 2024-09-18T11:30:33.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:33.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.887 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.887 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180088+0000", 2024-09-18T11:30:33.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00033395000000000001 2024-09-18T11:30:33.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.887 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.887 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180105+0000", 2024-09-18T11:30:33.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7039000000000001e-05 2024-09-18T11:30:33.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.887 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.887 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180112+0000", 2024-09-18T11:30:33.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3710000000000002e-06 2024-09-18T11:30:33.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180176+0000", 2024-09-18T11:30:33.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3178999999999999e-05 2024-09-18T11:30:33.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180197+0000", 2024-09-18T11:30:33.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1687000000000001e-05 2024-09-18T11:30:33.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183542+0000", 2024-09-18T11:30:33.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003344877 2024-09-18T11:30:33.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183568+0000", 2024-09-18T11:30:33.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6146e-05 2024-09-18T11:30:33.889 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.889 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.889 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.890 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1048 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'26, mlcod=35'23)", 2024-09-18T11:30:33.890 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.180697+0000", 2024-09-18T11:30:33.890 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.675979242, 2024-09-18T11:30:33.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031436290000000002, 2024-09-18T11:30:33.890 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.890 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.890 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180697+0000", 2024-09-18T11:30:33.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180697+0000", 2024-09-18T11:30:33.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180697+0000", 2024-09-18T11:30:33.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:33.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180704+0000", 2024-09-18T11:30:33.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8360000000000003e-06 2024-09-18T11:30:33.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180716+0000", 2024-09-18T11:30:33.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2308e-05 2024-09-18T11:30:33.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180724+0000", 2024-09-18T11:30:33.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6499999999999996e-06 2024-09-18T11:30:33.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180884+0000", 2024-09-18T11:30:33.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016041699999999999 2024-09-18T11:30:33.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180905+0000", 2024-09-18T11:30:33.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1103000000000001e-05 2024-09-18T11:30:33.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183819+0000", 2024-09-18T11:30:33.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029141800000000002 2024-09-18T11:30:33.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183841+0000", 2024-09-18T11:30:33.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1869e-05 2024-09-18T11:30:33.894 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.894 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.894 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.894 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:30:33.894 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:30:33.894 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.873302699, 2024-09-18T11:30:33.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:30:33.894 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.894 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.895 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:30:33.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:30:33.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:30:33.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:33.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:30:33.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:30:33.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:30:33.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:30:33.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:30:33.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:30:33.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:30:33.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:30:33.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:30:33.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:30:33.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:30:33.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:30:33.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:30:33.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:30:33.899 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.899 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.900 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.900 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1144 4.4 e39/22 4:2d9c3f7f:::smithi11939506-5:head v 39'30, mlcod=35'26)", 2024-09-18T11:30:33.900 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.204195+0000", 2024-09-18T11:30:33.900 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.65248195, 2024-09-18T11:30:33.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002568945, 2024-09-18T11:30:33.900 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.900 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.900 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204195+0000", 2024-09-18T11:30:33.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204195+0000", 2024-09-18T11:30:33.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204194+0000", 2024-09-18T11:30:33.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204431+0000", 2024-09-18T11:30:33.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237013 2024-09-18T11:30:33.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204446+0000", 2024-09-18T11:30:33.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4238e-05 2024-09-18T11:30:33.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204453+0000", 2024-09-18T11:30:33.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.401e-06 2024-09-18T11:30:33.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204509+0000", 2024-09-18T11:30:33.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5754000000000002e-05 2024-09-18T11:30:33.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204535+0000", 2024-09-18T11:30:33.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5936999999999999e-05 2024-09-18T11:30:33.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.206686+0000", 2024-09-18T11:30:33.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0021507829999999999 2024-09-18T11:30:33.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.206764+0000", 2024-09-18T11:30:33.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8135999999999999e-05 2024-09-18T11:30:33.904 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.904 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.905 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.905 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1145 4.4 e39/22 4:2d9c3f7f:::smithi11939506-5:head v 39'31, mlcod=38'28)", 2024-09-18T11:30:33.905 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.211000+0000", 2024-09-18T11:30:33.905 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.645676663, 2024-09-18T11:30:33.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0021574469999999998, 2024-09-18T11:30:33.905 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.905 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.905 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.211000+0000", 2024-09-18T11:30:33.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.211000+0000", 2024-09-18T11:30:33.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.210998+0000", 2024-09-18T11:30:33.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:33.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.211183+0000", 2024-09-18T11:30:33.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018480699999999999 2024-09-18T11:30:33.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.211203+0000", 2024-09-18T11:30:33.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9652000000000001e-05 2024-09-18T11:30:33.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.211210+0000", 2024-09-18T11:30:33.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8390000000000007e-06 2024-09-18T11:30:33.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.211270+0000", 2024-09-18T11:30:33.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9076000000000002e-05 2024-09-18T11:30:33.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.211294+0000", 2024-09-18T11:30:33.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4107000000000001e-05 2024-09-18T11:30:33.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.213120+0000", 2024-09-18T11:30:33.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018266000000000001 2024-09-18T11:30:33.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.213158+0000", 2024-09-18T11:30:33.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7249999999999997e-05 2024-09-18T11:30:33.909 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.909 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.909 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.910 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1146 4.4 e39/22 4:2d9c3f7f:::smithi11939506-5:head v 39'32, mlcod=39'30)", 2024-09-18T11:30:33.910 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.217314+0000", 2024-09-18T11:30:33.910 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.639362671000001, 2024-09-18T11:30:33.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020364070000000001, 2024-09-18T11:30:33.910 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.910 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.910 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.217314+0000", 2024-09-18T11:30:33.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.217314+0000", 2024-09-18T11:30:33.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.217313+0000", 2024-09-18T11:30:33.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:33.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.217530+0000", 2024-09-18T11:30:33.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021718599999999999 2024-09-18T11:30:33.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.217540+0000", 2024-09-18T11:30:33.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9010000000000006e-06 2024-09-18T11:30:33.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.217546+0000", 2024-09-18T11:30:33.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0179999999999998e-06 2024-09-18T11:30:33.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.217594+0000", 2024-09-18T11:30:33.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7341e-05 2024-09-18T11:30:33.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.217618+0000", 2024-09-18T11:30:33.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:30:33.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.219317+0000", 2024-09-18T11:30:33.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001698864 2024-09-18T11:30:33.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.219350+0000", 2024-09-18T11:30:33.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3194000000000001e-05 2024-09-18T11:30:33.914 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.914 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.914 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.914 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1147 4.4 e39/22 4:2d9c3f7f:::smithi11939506-5:head v 39'33, mlcod=39'30)", 2024-09-18T11:30:33.914 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.218465+0000", 2024-09-18T11:30:33.915 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.638211481000001, 2024-09-18T11:30:33.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013437880000000001, 2024-09-18T11:30:33.915 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.915 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.915 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.218465+0000", 2024-09-18T11:30:33.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.218465+0000", 2024-09-18T11:30:33.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.218465+0000", 2024-09-18T11:30:33.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.218473+0000", 2024-09-18T11:30:33.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6270000000000006e-06 2024-09-18T11:30:33.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.218483+0000", 2024-09-18T11:30:33.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9029999999999999e-06 2024-09-18T11:30:33.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.218489+0000", 2024-09-18T11:30:33.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.648e-06 2024-09-18T11:30:33.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.218689+0000", 2024-09-18T11:30:33.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000199965 2024-09-18T11:30:33.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.218710+0000", 2024-09-18T11:30:33.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0939000000000001e-05 2024-09-18T11:30:33.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.219779+0000", 2024-09-18T11:30:33.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001069665 2024-09-18T11:30:33.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.219809+0000", 2024-09-18T11:30:33.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9686999999999999e-05 2024-09-18T11:30:33.920 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.920 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.920 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.920 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1378 4.d e44/22 4:b81644f2:::smithi11939506-34:head v 44'49, mlcod=42'46)", 2024-09-18T11:30:33.920 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:25.202401+0000", 2024-09-18T11:30:33.920 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.654275384, 2024-09-18T11:30:33.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028449030000000002, 2024-09-18T11:30:33.920 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.920 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.921 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.202401+0000", 2024-09-18T11:30:33.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.202401+0000", 2024-09-18T11:30:33.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.202401+0000", 2024-09-18T11:30:33.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.203222+0000", 2024-09-18T11:30:33.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00082101099999999996 2024-09-18T11:30:33.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.203236+0000", 2024-09-18T11:30:33.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3451999999999999e-05 2024-09-18T11:30:33.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.203244+0000", 2024-09-18T11:30:33.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6400000000000003e-06 2024-09-18T11:30:33.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.203285+0000", 2024-09-18T11:30:33.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1254000000000002e-05 2024-09-18T11:30:33.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.203302+0000", 2024-09-18T11:30:33.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6769000000000002e-05 2024-09-18T11:30:33.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.205219+0000", 2024-09-18T11:30:33.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019171469999999999 2024-09-18T11:30:33.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.205246+0000", 2024-09-18T11:30:33.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6891999999999999e-05 2024-09-18T11:30:33.924 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.924 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.924 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.925 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1379 4.d e44/22 4:b81644f2:::smithi11939506-34:head v 44'50, mlcod=42'46)", 2024-09-18T11:30:33.925 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:25.206786+0000", 2024-09-18T11:30:33.925 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.6498905690000001, 2024-09-18T11:30:33.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001400245, 2024-09-18T11:30:33.925 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.925 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.925 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.206786+0000", 2024-09-18T11:30:33.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.206786+0000", 2024-09-18T11:30:33.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.206786+0000", 2024-09-18T11:30:33.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.207170+0000", 2024-09-18T11:30:33.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038410799999999998 2024-09-18T11:30:33.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.207181+0000", 2024-09-18T11:30:33.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1216000000000001e-05 2024-09-18T11:30:33.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.207186+0000", 2024-09-18T11:30:33.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8509999999999998e-06 2024-09-18T11:30:33.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.207223+0000", 2024-09-18T11:30:33.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7251e-05 2024-09-18T11:30:33.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.207235+0000", 2024-09-18T11:30:33.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2266000000000001e-05 2024-09-18T11:30:33.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.208162+0000", 2024-09-18T11:30:33.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00092648999999999997 2024-09-18T11:30:33.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.208186+0000", 2024-09-18T11:30:33.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4698999999999998e-05 2024-09-18T11:30:33.929 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.929 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.929 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.930 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1380 4.d e44/22 4:b81644f2:::smithi11939506-34:head v 44'51, mlcod=44'49)", 2024-09-18T11:30:33.930 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:25.215023+0000", 2024-09-18T11:30:33.930 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.6416536599999993, 2024-09-18T11:30:33.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001941852, 2024-09-18T11:30:33.930 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.931 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.931 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.215023+0000", 2024-09-18T11:30:33.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.215023+0000", 2024-09-18T11:30:33.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.215022+0000", 2024-09-18T11:30:33.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.215656+0000", 2024-09-18T11:30:33.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00063338800000000003 2024-09-18T11:30:33.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.215665+0000", 2024-09-18T11:30:33.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5259999999999999e-06 2024-09-18T11:30:33.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.215669+0000", 2024-09-18T11:30:33.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0289999999999999e-06 2024-09-18T11:30:33.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.215703+0000", 2024-09-18T11:30:33.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3878000000000003e-05 2024-09-18T11:30:33.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.215715+0000", 2024-09-18T11:30:33.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1909000000000001e-05 2024-09-18T11:30:33.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.216942+0000", 2024-09-18T11:30:33.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012271910000000001 2024-09-18T11:30:33.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.216965+0000", 2024-09-18T11:30:33.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2507e-05 2024-09-18T11:30:33.939 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.939 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.939 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.939 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:30:33.939 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:30:33.939 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.6384717460000005, 2024-09-18T11:30:33.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:30:33.940 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.940 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.940 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:30:33.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:30:33.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:30:33.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:30:33.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:30:33.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:30:33.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:30:33.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:30:33.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:30:33.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:30:33.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:30:33.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:30:33.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:30:33.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:30:33.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:30:33.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:30:33.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:30:33.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:30:33.948 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.948 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.948 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.948 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1422 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'64, mlcod=47'61)", 2024-09-18T11:30:33.948 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.224175+0000", 2024-09-18T11:30:33.949 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.6325015010000001, 2024-09-18T11:30:33.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001695776, 2024-09-18T11:30:33.949 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.949 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.949 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.224175+0000", 2024-09-18T11:30:33.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.224175+0000", 2024-09-18T11:30:33.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.224175+0000", 2024-09-18T11:30:33.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.224679+0000", 2024-09-18T11:30:33.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050417699999999997 2024-09-18T11:30:33.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.224689+0000", 2024-09-18T11:30:33.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0319000000000001e-05 2024-09-18T11:30:33.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.224693+0000", 2024-09-18T11:30:33.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4399999999999998e-06 2024-09-18T11:30:33.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.224720+0000", 2024-09-18T11:30:33.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6330999999999999e-05 2024-09-18T11:30:33.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.224734+0000", 2024-09-18T11:30:33.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4431e-05 2024-09-18T11:30:33.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.225848+0000", 2024-09-18T11:30:33.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0011133810000000001 2024-09-18T11:30:33.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.225871+0000", 2024-09-18T11:30:33.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.336e-05 2024-09-18T11:30:33.954 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.954 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.954 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.955 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1423 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'65, mlcod=48'63)", 2024-09-18T11:30:33.955 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.230841+0000", 2024-09-18T11:30:33.955 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.625836177, 2024-09-18T11:30:33.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002175808, 2024-09-18T11:30:33.955 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.955 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.955 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.230841+0000", 2024-09-18T11:30:33.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.230841+0000", 2024-09-18T11:30:33.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.230840+0000", 2024-09-18T11:30:33.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:33.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.231438+0000", 2024-09-18T11:30:33.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059785999999999997 2024-09-18T11:30:33.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.231447+0000", 2024-09-18T11:30:33.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3999999999999998e-06 2024-09-18T11:30:33.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.231451+0000", 2024-09-18T11:30:33.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0849999999999999e-06 2024-09-18T11:30:33.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.231490+0000", 2024-09-18T11:30:33.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8306999999999998e-05 2024-09-18T11:30:33.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.231510+0000", 2024-09-18T11:30:33.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0296999999999999e-05 2024-09-18T11:30:33.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.232984+0000", 2024-09-18T11:30:33.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014746080000000001 2024-09-18T11:30:33.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.233016+0000", 2024-09-18T11:30:33.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1919999999999999e-05 2024-09-18T11:30:33.959 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.959 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.959 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.959 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:244 4.d e49/22 4:b5a463bf:::smithi11939506-20:10 v 49'68, mlcod=48'65)", 2024-09-18T11:30:33.959 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:30.220263+0000", 2024-09-18T11:30:33.959 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.6364141559999998, 2024-09-18T11:30:33.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032524960000000001, 2024-09-18T11:30:33.959 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.959 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.959 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220263+0000", 2024-09-18T11:30:33.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220263+0000", 2024-09-18T11:30:33.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220262+0000", 2024-09-18T11:30:33.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:30:33.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220266+0000", 2024-09-18T11:30:33.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.895e-06 2024-09-18T11:30:33.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220278+0000", 2024-09-18T11:30:33.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1962e-05 2024-09-18T11:30:33.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220284+0000", 2024-09-18T11:30:33.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1299999999999998e-06 2024-09-18T11:30:33.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220334+0000", 2024-09-18T11:30:33.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9789000000000001e-05 2024-09-18T11:30:33.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220356+0000", 2024-09-18T11:30:33.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1897000000000001e-05 2024-09-18T11:30:33.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.223464+0000", 2024-09-18T11:30:33.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031082789999999998 2024-09-18T11:30:33.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.223515+0000", 2024-09-18T11:30:33.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0748999999999998e-05 2024-09-18T11:30:33.964 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.964 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.964 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.964 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1511 4.4 e51/22 4:2c93196f:::smithi11939506-28:head v 51'50, mlcod=51'47)", 2024-09-18T11:30:33.964 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:32.776917+0000", 2024-09-18T11:30:33.965 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.0797602070000001, 2024-09-18T11:30:33.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024761599999999998, 2024-09-18T11:30:33.965 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.965 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.965 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.776917+0000", 2024-09-18T11:30:33.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.776917+0000", 2024-09-18T11:30:33.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.776915+0000", 2024-09-18T11:30:33.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:33.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.777297+0000", 2024-09-18T11:30:33.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038211499999999998 2024-09-18T11:30:33.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.777317+0000", 2024-09-18T11:30:33.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0043000000000001e-05 2024-09-18T11:30:33.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.777325+0000", 2024-09-18T11:30:33.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9500000000000001e-06 2024-09-18T11:30:33.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.777380+0000", 2024-09-18T11:30:33.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4471000000000001e-05 2024-09-18T11:30:33.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.777403+0000", 2024-09-18T11:30:33.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3703999999999999e-05 2024-09-18T11:30:33.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.779349+0000", 2024-09-18T11:30:33.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001945312 2024-09-18T11:30:33.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.779393+0000", 2024-09-18T11:30:33.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3915000000000002e-05 2024-09-18T11:30:33.969 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.969 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.969 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.969 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1512 4.4 e51/22 4:2c93196f:::smithi11939506-28:head v 51'51, mlcod=51'48)", 2024-09-18T11:30:33.969 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:32.782201+0000", 2024-09-18T11:30:33.969 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.0744757920000001, 2024-09-18T11:30:33.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00363554, 2024-09-18T11:30:33.969 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.969 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.969 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782201+0000", 2024-09-18T11:30:33.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782201+0000", 2024-09-18T11:30:33.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782200+0000", 2024-09-18T11:30:33.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:33.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782564+0000", 2024-09-18T11:30:33.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000364612 2024-09-18T11:30:33.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782585+0000", 2024-09-18T11:30:33.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0418999999999999e-05 2024-09-18T11:30:33.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782594+0000", 2024-09-18T11:30:33.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9570000000000008e-06 2024-09-18T11:30:33.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782663+0000", 2024-09-18T11:30:33.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9330000000000002e-05 2024-09-18T11:30:33.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782688+0000", 2024-09-18T11:30:33.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5420000000000001e-05 2024-09-18T11:30:33.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.785790+0000", 2024-09-18T11:30:33.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031016160000000002 2024-09-18T11:30:33.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.785836+0000", 2024-09-18T11:30:33.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6360999999999999e-05 2024-09-18T11:30:33.973 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.973 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.973 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.974 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1513 4.4 e51/22 4:2c93196f:::smithi11939506-28:head v 51'52, mlcod=51'48)", 2024-09-18T11:30:33.974 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:32.783397+0000", 2024-09-18T11:30:33.974 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.073279254, 2024-09-18T11:30:33.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003017198, 2024-09-18T11:30:33.974 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:33.974 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:33.974 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:33.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:33.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783397+0000", 2024-09-18T11:30:33.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:33.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783397+0000", 2024-09-18T11:30:33.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:33.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:33.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783397+0000", 2024-09-18T11:30:33.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:33.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:33.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783404+0000", 2024-09-18T11:30:33.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.075e-06 2024-09-18T11:30:33.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:33.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783417+0000", 2024-09-18T11:30:33.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3234000000000001e-05 2024-09-18T11:30:33.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:33.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783422+0000", 2024-09-18T11:30:33.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5260000000000001e-06 2024-09-18T11:30:33.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:33.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783605+0000", 2024-09-18T11:30:33.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018248099999999999 2024-09-18T11:30:33.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:33.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783635+0000", 2024-09-18T11:30:33.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0309000000000001e-05 2024-09-18T11:30:33.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:33.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.786378+0000", 2024-09-18T11:30:33.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027433119999999999 2024-09-18T11:30:33.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:33.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:33.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:33.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.786415+0000", 2024-09-18T11:30:33.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6192999999999997e-05 2024-09-18T11:30:33.978 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.978 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.978 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.979 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:33.979 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:33.979 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:33.980 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:30:33.981 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:33.979+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:34.000 INFO:tasks.rados.rados.0.smithi119.stdout:860: got (0) Success 2024-09-18T11:30:34.000 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 64 (ObjNum 118 snap 11 seq_num 118) dirty exists 2024-09-18T11:30:34.000 INFO:tasks.rados.rados.0.smithi119.stdout:861: expect (ObjNum 110 snap 8 seq_num 110) 2024-09-18T11:30:34.080 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:34.079+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:34.101 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:34.101 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:34.101 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:34.101 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:34.110 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:30:34.184 INFO:tasks.rados.rados.0.smithi119.stdout:862: expect (ObjNum 109 snap 8 seq_num 109) 2024-09-18T11:30:34.185 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:34.183+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:34.251 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:34.251 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:34.251 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:34.251 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:34.251 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:34.259 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:30:34.282 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:34.280+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:34.371 INFO:tasks.rados.rados.0.smithi119.stdout:860: done (3 left) 2024-09-18T11:30:34.371 INFO:tasks.rados.rados.0.smithi119.stdout:861: done (2 left) 2024-09-18T11:30:34.371 INFO:tasks.rados.rados.0.smithi119.stdout:862: done (1 left) 2024-09-18T11:30:34.371 INFO:tasks.rados.rados.0.smithi119.stdout:863: done (0 left) 2024-09-18T11:30:34.371 INFO:tasks.rados.rados.0.smithi119.stdout:864: set_chunk oid 44 target oid 2024-09-18T11:30:34.382 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:34.381+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:34.384 INFO:tasks.rados.rados.0.smithi119.stdout:864: set_chunk oid 44 offset: 461312 length: 1647104 target oid d85d35018f93ce704e0c86e65d6c262758d2b38883b05fc1a96898c5a961f23a offset: 0 snap 11 2024-09-18T11:30:34.384 INFO:tasks.rados.rados.0.smithi119.stdout:865: write oid 34 current snap is 12 2024-09-18T11:30:34.384 INFO:tasks.rados.rados.0.smithi119.stdout:865: seq_num 123 ranges {467261=717275,1847991=219270} 2024-09-18T11:30:34.390 INFO:tasks.rados.rados.0.smithi119.stdout:865: writing smithi11939506-34 from 467261 to 1184536 tid 1 2024-09-18T11:30:34.395 INFO:tasks.rados.rados.0.smithi119.stdout:865: writing smithi11939506-34 from 1847991 to 2067261 tid 2 2024-09-18T11:30:34.395 INFO:tasks.rados.rados.0.smithi119.stdout:866: set_chunk oid 25 target oid 2024-09-18T11:30:34.403 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:34.403 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:34.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:34.403 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:34.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.403 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:900 4.0 4:0aec5e1c:::smithi11939506-50:head [read 0~2974806,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:34.403 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.133226+0000", 2024-09-18T11:30:34.403 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.267311337999999, 2024-09-18T11:30:34.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059578069999999999, 2024-09-18T11:30:34.403 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:34.403 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:34.404 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:34.404 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:34.404 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.404 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 900 2024-09-18T11:30:34.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.404 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:34.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:34.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133226+0000", 2024-09-18T11:30:34.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:34.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133226+0000", 2024-09-18T11:30:34.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:34.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133225+0000", 2024-09-18T11:30:34.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:34.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:34.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133234+0000", 2024-09-18T11:30:34.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5799999999999992e-06 2024-09-18T11:30:34.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:34.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133242+0000", 2024-09-18T11:30:34.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4039999999999995e-06 2024-09-18T11:30:34.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133256+0000", 2024-09-18T11:30:34.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3278000000000001e-05 2024-09-18T11:30:34.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133293+0000", 2024-09-18T11:30:34.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7799000000000002e-05 2024-09-18T11:30:34.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:34.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133298+0000", 2024-09-18T11:30:34.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9110000000000001e-06 2024-09-18T11:30:34.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.137554+0000", 2024-09-18T11:30:34.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042556099999999999 2024-09-18T11:30:34.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:34.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.137713+0000", 2024-09-18T11:30:34.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015937199999999999 2024-09-18T11:30:34.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:34.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139184+0000", 2024-09-18T11:30:34.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001470694 2024-09-18T11:30:34.409 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.409 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:34.409 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.409 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:901 4.0 4:0aec5e1c:::smithi11939506-50:head [sparse-read 0~1781150] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:34.409 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:34.409 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.267272127999998, 2024-09-18T11:30:34.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063828720000000004, 2024-09-18T11:30:34.409 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:34.409 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:34.409 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:34.409 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:34.409 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.410 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 901 2024-09-18T11:30:34.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.410 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:34.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:34.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:34.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:34.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:34.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:34.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:34.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:34.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:34.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133268+0000", 2024-09-18T11:30:34.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9610000000000001e-06 2024-09-18T11:30:34.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:34.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133280+0000", 2024-09-18T11:30:34.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2481e-05 2024-09-18T11:30:34.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133285+0000", 2024-09-18T11:30:34.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5469999999999998e-06 2024-09-18T11:30:34.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133317+0000", 2024-09-18T11:30:34.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1838999999999998e-05 2024-09-18T11:30:34.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:34.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133320+0000", 2024-09-18T11:30:34.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3009999999999998e-06 2024-09-18T11:30:34.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139225+0000", 2024-09-18T11:30:34.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059047919999999999 2024-09-18T11:30:34.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:34.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139330+0000", 2024-09-18T11:30:34.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104924 2024-09-18T11:30:34.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:34.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139648+0000", 2024-09-18T11:30:34.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00031854799999999999 2024-09-18T11:30:34.414 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.414 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:34.414 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.414 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:34.414 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:30:34.414 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.231118087999999, 2024-09-18T11:30:34.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:30:34.414 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:34.414 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:34.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:34.415 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:34.415 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.415 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:30:34.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.415 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:34.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:34.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:30:34.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:34.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:30:34.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:34.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:30:34.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:34.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:34.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:30:34.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:30:34.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:34.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:30:34.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:30:34.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:30:34.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:30:34.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:30:34.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:30:34.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:34.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:30:34.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:30:34.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:34.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:30:34.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:30:34.418 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.418 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:34.418 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.418 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1000 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:34.419 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164518+0000", 2024-09-18T11:30:34.419 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.236019981999998, 2024-09-18T11:30:34.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005734093, 2024-09-18T11:30:34.419 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:34.419 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:34.419 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:34.419 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:34.419 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.419 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1000 2024-09-18T11:30:34.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.419 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:34.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:34.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164518+0000", 2024-09-18T11:30:34.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:34.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164518+0000", 2024-09-18T11:30:34.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:34.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164515+0000", 2024-09-18T11:30:34.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:34.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:34.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164525+0000", 2024-09-18T11:30:34.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5480000000000007e-06 2024-09-18T11:30:34.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:34.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164542+0000", 2024-09-18T11:30:34.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7643e-05 2024-09-18T11:30:34.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164553+0000", 2024-09-18T11:30:34.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0581000000000001e-05 2024-09-18T11:30:34.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164637+0000", 2024-09-18T11:30:34.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3825999999999993e-05 2024-09-18T11:30:34.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:34.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164644+0000", 2024-09-18T11:30:34.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5499999999999997e-06 2024-09-18T11:30:34.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.166446+0000", 2024-09-18T11:30:34.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018018789999999999 2024-09-18T11:30:34.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:34.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.166537+0000", 2024-09-18T11:30:34.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0325e-05 2024-09-18T11:30:34.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:34.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170252+0000", 2024-09-18T11:30:34.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037150820000000002 2024-09-18T11:30:34.423 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.424 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:34.424 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.424 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1001 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:34.424 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:30:34.424 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.235966810000001, 2024-09-18T11:30:34.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007461082, 2024-09-18T11:30:34.424 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:34.424 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:34.424 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:34.424 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:34.424 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.424 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1001 2024-09-18T11:30:34.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.425 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:34.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:34.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:30:34.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:34.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:30:34.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:34.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164570+0000", 2024-09-18T11:30:34.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:34.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:34.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164574+0000", 2024-09-18T11:30:34.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.822e-06 2024-09-18T11:30:34.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:34.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164582+0000", 2024-09-18T11:30:34.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3410000000000003e-06 2024-09-18T11:30:34.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164587+0000", 2024-09-18T11:30:34.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769999999999999e-06 2024-09-18T11:30:34.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164702+0000", 2024-09-18T11:30:34.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115705 2024-09-18T11:30:34.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:34.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164709+0000", 2024-09-18T11:30:34.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-09-18T11:30:34.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170296+0000", 2024-09-18T11:30:34.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055876390000000001 2024-09-18T11:30:34.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:34.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170418+0000", 2024-09-18T11:30:34.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121549 2024-09-18T11:30:34.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:34.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172032+0000", 2024-09-18T11:30:34.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00161407 2024-09-18T11:30:34.429 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.429 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:34.429 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.429 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:34.429 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:34.429 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.235940094, 2024-09-18T11:30:34.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:30:34.429 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:34.429 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:34.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:34.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:34.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.430 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:30:34.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.430 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:34.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:34.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:34.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:34.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:34.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:34.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:34.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:34.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:34.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:30:34.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:30:34.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:34.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:30:34.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:30:34.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:30:34.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:30:34.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:30:34.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:30:34.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:34.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:30:34.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:30:34.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:30:34.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:30:34.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:34.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:30:34.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:30:34.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:34.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:30:34.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:30:34.434 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.434 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:34.434 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.435 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:34.435 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:30:34.435 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.235921805, 2024-09-18T11:30:34.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:30:34.435 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:34.435 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:34.435 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:34.435 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:34.435 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.435 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:30:34.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.435 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:34.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:34.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:30:34.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:34.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:30:34.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:34.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:30:34.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:34.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:34.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:30:34.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:30:34.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:34.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:30:34.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:30:34.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:30:34.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:30:34.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:30:34.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:30:34.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:34.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:30:34.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:30:34.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:30:34.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:30:34.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:34.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:30:34.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:30:34.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:34.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:30:34.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:30:34.440 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.440 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:34.440 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.440 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:34.440 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:30:34.441 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.235861035999999, 2024-09-18T11:30:34.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:30:34.441 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:34.441 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:34.441 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:34.441 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:34.441 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.441 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:30:34.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.441 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:34.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:34.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:30:34.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:34.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:30:34.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:34.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:30:34.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:34.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:34.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:30:34.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:30:34.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:34.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:30:34.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:30:34.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:30:34.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:30:34.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:30:34.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:30:34.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:34.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:30:34.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:30:34.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:30:34.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:30:34.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:34.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:30:34.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:30:34.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:34.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:30:34.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:30:34.446 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.446 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:34.446 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.446 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:34.446 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:30:34.446 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.235838715, 2024-09-18T11:30:34.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:30:34.446 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:34.446 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:34.446 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:34.447 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:34.447 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.447 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:30:34.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.447 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:34.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:34.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:30:34.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:34.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:30:34.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:34.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:30:34.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:34.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:34.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:30:34.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:30:34.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:34.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:30:34.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:30:34.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:30:34.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:30:34.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:30:34.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:30:34.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:34.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:30:34.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:30:34.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:30:34.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:30:34.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:34.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:30:34.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:30:34.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:34.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:30:34.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:30:34.451 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.451 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:34.451 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.452 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1018 4.0 4:0aec5e1c:::smithi11939506-50:head [write 1622620~730352] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:34.452 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:30:34.452 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.719604158999999, 2024-09-18T11:30:34.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064405160000000003, 2024-09-18T11:30:34.452 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:34.452 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:34.452 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:34.452 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:34.452 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.452 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1018 2024-09-18T11:30:34.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.452 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:34.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:34.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:30:34.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:34.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:30:34.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:34.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680932+0000", 2024-09-18T11:30:34.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:34.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:34.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681334+0000", 2024-09-18T11:30:34.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000401803 2024-09-18T11:30:34.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:34.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681350+0000", 2024-09-18T11:30:34.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6218999999999998e-05 2024-09-18T11:30:34.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681374+0000", 2024-09-18T11:30:34.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4088e-05 2024-09-18T11:30:34.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681415+0000", 2024-09-18T11:30:34.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1307000000000003e-05 2024-09-18T11:30:34.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:34.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681507+0000", 2024-09-18T11:30:34.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1867999999999995e-05 2024-09-18T11:30:34.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:34.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681671+0000", 2024-09-18T11:30:34.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016406 2024-09-18T11:30:34.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:34.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.683856+0000", 2024-09-18T11:30:34.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002184624 2024-09-18T11:30:34.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:34.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687266+0000", 2024-09-18T11:30:34.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034105490000000001 2024-09-18T11:30:34.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:34.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687321+0000", 2024-09-18T11:30:34.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4960999999999998e-05 2024-09-18T11:30:34.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:34.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687374+0000", 2024-09-18T11:30:34.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2553000000000003e-05 2024-09-18T11:30:34.463 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.463 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:34.463 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.464 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1037 4.0 4:019b6e09:::smithi11939506-40:head [write 477332~766101] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:34.464 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.707508+0000", 2024-09-18T11:30:34.464 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.693029433, 2024-09-18T11:30:34.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050976160000000001, 2024-09-18T11:30:34.464 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:34.465 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:34.465 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:34.465 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:34.465 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.465 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1037 2024-09-18T11:30:34.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.466 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:34.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:34.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.707508+0000", 2024-09-18T11:30:34.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:34.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.707508+0000", 2024-09-18T11:30:34.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:34.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.707506+0000", 2024-09-18T11:30:34.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:34.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:34.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708363+0000", 2024-09-18T11:30:34.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00085727299999999998 2024-09-18T11:30:34.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:34.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708388+0000", 2024-09-18T11:30:34.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5031000000000001e-05 2024-09-18T11:30:34.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708396+0000", 2024-09-18T11:30:34.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0870000000000007e-06 2024-09-18T11:30:34.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708452+0000", 2024-09-18T11:30:34.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.533e-05 2024-09-18T11:30:34.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:34.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708563+0000", 2024-09-18T11:30:34.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011086600000000001 2024-09-18T11:30:34.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:34.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708761+0000", 2024-09-18T11:30:34.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019803 2024-09-18T11:30:34.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:34.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.711546+0000", 2024-09-18T11:30:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002785689 2024-09-18T11:30:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712507+0000", 2024-09-18T11:30:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096094300000000002 2024-09-18T11:30:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712557+0000", 2024-09-18T11:30:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9701999999999999e-05 2024-09-18T11:30:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:34.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712606+0000", 2024-09-18T11:30:34.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.888e-05 2024-09-18T11:30:34.474 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.474 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:34.474 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.474 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1040 4.0 4:019b6e09:::smithi11939506-40:head [setxattr _header (56) in=7b,truncate 3277332] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:34.474 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.719140+0000", 2024-09-18T11:30:34.474 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.681397459999999, 2024-09-18T11:30:34.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049512000000000002, 2024-09-18T11:30:34.474 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:34.474 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1040 2024-09-18T11:30:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719140+0000", 2024-09-18T11:30:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:34.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719140+0000", 2024-09-18T11:30:34.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:34.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719140+0000", 2024-09-18T11:30:34.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:34.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:34.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719143+0000", 2024-09-18T11:30:34.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5389999999999999e-06 2024-09-18T11:30:34.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:34.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719149+0000", 2024-09-18T11:30:34.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0249999999999999e-06 2024-09-18T11:30:34.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719153+0000", 2024-09-18T11:30:34.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4910000000000001e-06 2024-09-18T11:30:34.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.720365+0000", 2024-09-18T11:30:34.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001212346 2024-09-18T11:30:34.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:34.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.720476+0000", 2024-09-18T11:30:34.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011156500000000001 2024-09-18T11:30:34.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:34.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.720662+0000", 2024-09-18T11:30:34.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018538199999999999 2024-09-18T11:30:34.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:34.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.722050+0000", 2024-09-18T11:30:34.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001388291 2024-09-18T11:30:34.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:34.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.724024+0000", 2024-09-18T11:30:34.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019738860000000002 2024-09-18T11:30:34.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:34.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.724055+0000", 2024-09-18T11:30:34.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1288000000000003e-05 2024-09-18T11:30:34.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:34.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.724091+0000", 2024-09-18T11:30:34.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5960999999999997e-05 2024-09-18T11:30:34.480 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.480 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:34.480 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.481 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1041 4.0 4:019b6e09:::smithi11939506-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:34.481 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.719166+0000", 2024-09-18T11:30:34.481 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.681371815999999, 2024-09-18T11:30:34.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050746640000000004, 2024-09-18T11:30:34.481 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:34.481 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:34.481 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:34.481 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:34.481 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.481 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1041 2024-09-18T11:30:34.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.482 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:34.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:34.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719166+0000", 2024-09-18T11:30:34.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:34.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719166+0000", 2024-09-18T11:30:34.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:34.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719165+0000", 2024-09-18T11:30:34.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:34.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:34.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719168+0000", 2024-09-18T11:30:34.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6220000000000001e-06 2024-09-18T11:30:34.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:34.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719174+0000", 2024-09-18T11:30:34.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2020000000000001e-06 2024-09-18T11:30:34.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719178+0000", 2024-09-18T11:30:34.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6890000000000002e-06 2024-09-18T11:30:34.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.720928+0000", 2024-09-18T11:30:34.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017504490000000001 2024-09-18T11:30:34.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:34.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.721050+0000", 2024-09-18T11:30:34.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121477 2024-09-18T11:30:34.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.724120+0000", 2024-09-18T11:30:34.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0030705569999999998 2024-09-18T11:30:34.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:34.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.724176+0000", 2024-09-18T11:30:34.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5327000000000001e-05 2024-09-18T11:30:34.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:34.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.724240+0000", 2024-09-18T11:30:34.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4806000000000003e-05 2024-09-18T11:30:34.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.486 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:34.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.486 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1050 4.0 4:0aec5e1c:::smithi11939506-50:head [write 510946~632952] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:34.486 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.185718+0000", 2024-09-18T11:30:34.486 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.214819187, 2024-09-18T11:30:34.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055213279999999998, 2024-09-18T11:30:34.486 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:34.486 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:34.486 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:34.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:34.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.487 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1050 2024-09-18T11:30:34.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.487 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:34.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:34.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185718+0000", 2024-09-18T11:30:34.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:34.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185718+0000", 2024-09-18T11:30:34.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:34.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185717+0000", 2024-09-18T11:30:34.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:34.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:34.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185850+0000", 2024-09-18T11:30:34.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000133616 2024-09-18T11:30:34.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:34.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185867+0000", 2024-09-18T11:30:34.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7047000000000001e-05 2024-09-18T11:30:34.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185874+0000", 2024-09-18T11:30:34.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4250000000000003e-06 2024-09-18T11:30:34.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185926+0000", 2024-09-18T11:30:34.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1867999999999999e-05 2024-09-18T11:30:34.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:34.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185985+0000", 2024-09-18T11:30:34.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9769000000000003e-05 2024-09-18T11:30:34.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:34.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.186079+0000", 2024-09-18T11:30:34.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3109000000000006e-05 2024-09-18T11:30:34.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:34.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.187878+0000", 2024-09-18T11:30:34.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001799557 2024-09-18T11:30:34.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:34.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191165+0000", 2024-09-18T11:30:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003286892 2024-09-18T11:30:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191196+0000", 2024-09-18T11:30:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0607e-05 2024-09-18T11:30:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191240+0000", 2024-09-18T11:30:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4078e-05 2024-09-18T11:30:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1051 4.0 4:0aec5e1c:::smithi11939506-50:head [write 1798066~477679] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.191955+0000", 2024-09-18T11:30:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.20858273, 2024-09-18T11:30:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004674246, 2024-09-18T11:30:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1051 2024-09-18T11:30:34.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.494 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:34.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:34.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191955+0000", 2024-09-18T11:30:34.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:34.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191955+0000", 2024-09-18T11:30:34.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:34.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191954+0000", 2024-09-18T11:30:34.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:34.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:34.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.192068+0000", 2024-09-18T11:30:34.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113999 2024-09-18T11:30:34.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:34.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.192079+0000", 2024-09-18T11:30:34.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1477999999999999e-05 2024-09-18T11:30:34.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.192083+0000", 2024-09-18T11:30:34.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4170000000000002e-06 2024-09-18T11:30:34.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.192115+0000", 2024-09-18T11:30:34.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1378999999999999e-05 2024-09-18T11:30:34.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:34.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.192170+0000", 2024-09-18T11:30:34.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5631000000000003e-05 2024-09-18T11:30:34.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:34.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.192261+0000", 2024-09-18T11:30:34.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0748000000000006e-05 2024-09-18T11:30:34.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:34.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.193630+0000", 2024-09-18T11:30:34.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013684999999999999 2024-09-18T11:30:34.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:34.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.196567+0000", 2024-09-18T11:30:34.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029371279999999998 2024-09-18T11:30:34.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:34.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.196597+0000", 2024-09-18T11:30:34.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0480999999999998e-05 2024-09-18T11:30:34.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:34.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.196629+0000", 2024-09-18T11:30:34.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1767999999999998e-05 2024-09-18T11:30:34.499 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.499 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:34.499 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.500 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:34.500 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:30:34.500 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.454713831999999, 2024-09-18T11:30:34.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:30:34.501 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:34.501 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:34.501 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:34.501 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:34.501 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.502 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:30:34.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.502 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:34.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:34.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:30:34.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:34.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:30:34.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:34.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:30:34.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:30:34.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:34.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:30:34.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:30:34.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:34.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:30:34.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:30:34.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:30:34.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:30:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:30:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:30:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:30:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:30:34.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:30:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:30:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:30:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:30:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:34.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:30:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:30:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:30:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:30:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:30:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:30:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:34.510 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1121 4.f e38/22 4:f5d7df41:::smithi11939506-16:head v 38'34, mlcod=30'32)", 2024-09-18T11:30:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:30:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.434726573000001, 2024-09-18T11:30:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060953680000000003, 2024-09-18T11:30:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:34.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:30:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:30:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965810+0000", 2024-09-18T11:30:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966142+0000", 2024-09-18T11:30:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000331591 2024-09-18T11:30:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966154+0000", 2024-09-18T11:30:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.253e-05 2024-09-18T11:30:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966160+0000", 2024-09-18T11:30:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7130000000000002e-06 2024-09-18T11:30:34.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966196+0000", 2024-09-18T11:30:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6276999999999998e-05 2024-09-18T11:30:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966217+0000", 2024-09-18T11:30:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0335000000000001e-05 2024-09-18T11:30:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.971872+0000", 2024-09-18T11:30:34.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056549290000000004 2024-09-18T11:30:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.971906+0000", 2024-09-18T11:30:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4702000000000001e-05 2024-09-18T11:30:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:29 3.2 3:5fa8fcfa:::ec59520759824f0488c7a3ca135dacb324de2330e16b7f54053de5719c4e96b6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:30:34.515 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.720355916000001, 2024-09-18T11:30:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064126499999999998, 2024-09-18T11:30:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:30:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:30:34.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680181+0000", 2024-09-18T11:30:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680654+0000", 2024-09-18T11:30:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047334900000000002 2024-09-18T11:30:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:34.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680666+0000", 2024-09-18T11:30:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2232000000000001e-05 2024-09-18T11:30:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680672+0000", 2024-09-18T11:30:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:30:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681222+0000", 2024-09-18T11:30:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00054928699999999997 2024-09-18T11:30:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681332+0000", 2024-09-18T11:30:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110634 2024-09-18T11:30:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681552+0000", 2024-09-18T11:30:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021926899999999999 2024-09-18T11:30:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682950+0000", 2024-09-18T11:30:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013982669999999999 2024-09-18T11:30:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686503+0000", 2024-09-18T11:30:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035536130000000002 2024-09-18T11:30:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686553+0000", 2024-09-18T11:30:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9801e-05 2024-09-18T11:30:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686594+0000", 2024-09-18T11:30:34.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1022000000000001e-05 2024-09-18T11:30:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:30 3.2 3:49074e56:::7fc21ffee1ba17f2a9197c61ff60ef0b03804c121c8aeb83ea424b7ac77081e9:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.680698+0000", 2024-09-18T11:30:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.719839598, 2024-09-18T11:30:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060349130000000003, 2024-09-18T11:30:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680698+0000", 2024-09-18T11:30:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680698+0000", 2024-09-18T11:30:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680697+0000", 2024-09-18T11:30:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:34.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680750+0000", 2024-09-18T11:30:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3170000000000001e-05 2024-09-18T11:30:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680760+0000", 2024-09-18T11:30:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3379999999999994e-06 2024-09-18T11:30:34.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680766+0000", 2024-09-18T11:30:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8249999999999999e-06 2024-09-18T11:30:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682201+0000", 2024-09-18T11:30:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001434868 2024-09-18T11:30:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:34.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682294+0000", 2024-09-18T11:30:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2965000000000002e-05 2024-09-18T11:30:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682445+0000", 2024-09-18T11:30:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015103600000000001 2024-09-18T11:30:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.683524+0000", 2024-09-18T11:30:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0010789980000000001 2024-09-18T11:30:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686687+0000", 2024-09-18T11:30:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003163052 2024-09-18T11:30:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686714+0000", 2024-09-18T11:30:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6369000000000001e-05 2024-09-18T11:30:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686733+0000", 2024-09-18T11:30:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9117999999999999e-05 2024-09-18T11:30:34.526 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1199 4.0 4:0aec5e1c:::smithi11939506-50:head [write 763051~794393] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-18T11:30:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:30:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.203541791999999, 2024-09-18T11:30:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067768530000000002, 2024-09-18T11:30:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:34.527 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1199 2024-09-18T11:30:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:30:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:30:34.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196994+0000", 2024-09-18T11:30:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197355+0000", 2024-09-18T11:30:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036035199999999998 2024-09-18T11:30:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:34.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197366+0000", 2024-09-18T11:30:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.17e-05 2024-09-18T11:30:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197371+0000", 2024-09-18T11:30:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6099999999999999e-06 2024-09-18T11:30:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197416+0000", 2024-09-18T11:30:34.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5099000000000003e-05 2024-09-18T11:30:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197420+0000", 2024-09-18T11:30:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3189999999999996e-06 2024-09-18T11:30:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199392+0000", 2024-09-18T11:30:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019720139999999998 2024-09-18T11:30:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199452+0000", 2024-09-18T11:30:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9929999999999997e-05 2024-09-18T11:30:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199565+0000", 2024-09-18T11:30:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112811 2024-09-18T11:30:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.202108+0000", 2024-09-18T11:30:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025429060000000002 2024-09-18T11:30:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203714+0000", 2024-09-18T11:30:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001606335 2024-09-18T11:30:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203745+0000", 2024-09-18T11:30:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0837999999999998e-05 2024-09-18T11:30:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203773+0000", 2024-09-18T11:30:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7562000000000001e-05 2024-09-18T11:30:34.533 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.534 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:34.534 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.534 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:34.534 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:34.534 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:34.536 INFO:tasks.rados.rados.0.smithi119.stdout:866: set_chunk oid 25 offset: 623104 length: 813056 target oid 606d95f8fde1c0fdb94da22d76b8477d175d605114d2fa08817c654f24d040f4 offset: 0 snap 8 2024-09-18T11:30:34.536 INFO:tasks.rados.rados.0.smithi119.stdout:867: read oid 14 snap 6 2024-09-18T11:30:34.536 INFO:tasks.rados.rados.0.smithi119.stdout:867: expect (ObjNum 66 snap 0 seq_num 66) 2024-09-18T11:30:34.536 INFO:tasks.rados.rados.0.smithi119.stdout:868: read oid 36 snap -1 2024-09-18T11:30:34.536 INFO:tasks.rados.rados.0.smithi119.stdout:868: expect (ObjNum 100 snap 6 seq_num 100) 2024-09-18T11:30:34.536 INFO:tasks.rados.rados.0.smithi119.stdout:869: tier_promote oid 5 2024-09-18T11:30:34.536 INFO:tasks.rados.rados.0.smithi119.stdout:864: finishing set_chunk to oid 44 2024-09-18T11:30:34.536 INFO:tasks.rados.rados.0.smithi119.stdout:865: finishing write tid 1 to smithi11939506-34 2024-09-18T11:30:34.537 INFO:tasks.rados.rados.0.smithi119.stdout:865: finishing write tid 2 to smithi11939506-34 2024-09-18T11:30:34.537 INFO:tasks.rados.rados.0.smithi119.stdout:865: finishing write tid 3 to smithi11939506-34 2024-09-18T11:30:34.537 INFO:tasks.rados.rados.0.smithi119.stdout:865: finishing write tid 4 to smithi11939506-34 2024-09-18T11:30:34.537 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 74 (ObjNum 123 snap 12 seq_num 123) dirty exists 2024-09-18T11:30:34.537 INFO:tasks.rados.rados.0.smithi119.stdout:865: left oid 34 (ObjNum 123 snap 12 seq_num 123) 2024-09-18T11:30:34.537 INFO:tasks.rados.rados.0.smithi119.stdout:864: done (5 left) 2024-09-18T11:30:34.537 INFO:tasks.rados.rados.0.smithi119.stdout:865: done (4 left) 2024-09-18T11:30:34.537 INFO:tasks.rados.rados.0.smithi119.stdout:870: read oid 12 snap -1 2024-09-18T11:30:34.537 INFO:tasks.rados.rados.0.smithi119.stdout:870: expect (ObjNum 70 snap 2 seq_num 70) 2024-09-18T11:30:34.537 INFO:tasks.rados.rados.0.smithi119.stdout:871: set_chunk oid 50 target oid 2024-09-18T11:30:34.537 INFO:tasks.rados.rados.0.smithi119.stdout:871: set_chunk oid 50 offset: 2117120 length: 608768 target oid ac47b8623647b4cd1851128a75d93065bd2a2122c8cd6774e7ae0a9937ae73a7 offset: 0 snap 10 2024-09-18T11:30:34.537 INFO:tasks.rados.rados.0.smithi119.stdout:872: read oid 48 snap 6 2024-09-18T11:30:34.537 INFO:tasks.rados.rados.0.smithi119.stdout:872: expect (ObjNum 47 snap 0 seq_num 47) 2024-09-18T11:30:34.537 INFO:tasks.rados.rados.0.smithi119.stdout:873: set_chunk oid 44 target oid 2024-09-18T11:30:34.538 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:30:34.539 INFO:tasks.rados.rados.0.smithi119.stdout:873: set_chunk oid 44 offset: 116736 length: 1404928 target oid 287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03 offset: 0 snap -1 2024-09-18T11:30:34.539 INFO:tasks.rados.rados.0.smithi119.stdout:874: snap_create 2024-09-18T11:30:34.540 INFO:tasks.rados.rados.0.smithi119.stdout:866: finishing set_chunk to oid 25 2024-09-18T11:30:34.540 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 25 set_chunk returned error code -95 offset: 623104 length: 813056 2024-09-18T11:30:34.540 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 0 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-18T11:30:34.540 INFO:tasks.rados.rados.0.smithi119.stdout:869: got (0) Success 2024-09-18T11:30:34.540 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 42 (ObjNum 105 snap 6 seq_num 105) dirty exists 2024-09-18T11:30:34.540 INFO:tasks.rados.rados.0.smithi119.stdout:871: finishing set_chunk to oid 50 2024-09-18T11:30:34.540 INFO:tasks.rados.rados.0.smithi119.stdout:868: expect (ObjNum 100 snap 6 seq_num 100) 2024-09-18T11:30:34.540 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:34.534+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:34.597 INFO:tasks.rados.rados.0.smithi119.stdout:872: expect (ObjNum 47 snap 0 seq_num 47) 2024-09-18T11:30:34.636 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:34.633+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:34.640 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:34.640 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:34.640 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:34.640 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:34.648 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:30:34.684 INFO:tasks.rados.rados.0.smithi119.stdout:870: expect (ObjNum 70 snap 2 seq_num 70) 2024-09-18T11:30:34.736 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:34.734+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:34.787 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:34.787 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:34.787 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:34.787 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:34.787 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:34.796 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:30:34.820 INFO:tasks.rados.rados.0.smithi119.stdout:867: expect (ObjNum 66 snap 0 seq_num 66) 2024-09-18T11:30:34.837 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:34.836+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:34.858 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:30:34.936 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:34.936 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:34.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:34.936 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:34.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.936 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:878 4.e 4:7410c0e0:::smithi11939506-47:2 [read 0~2773122,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116276+0000", 2024-09-18T11:30:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.817346987000001, 2024-09-18T11:30:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041562079999999998, 2024-09-18T11:30:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 878 2024-09-18T11:30:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116276+0000", 2024-09-18T11:30:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116276+0000", 2024-09-18T11:30:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116276+0000", 2024-09-18T11:30:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116279+0000", 2024-09-18T11:30:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.704e-06 2024-09-18T11:30:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116289+0000", 2024-09-18T11:30:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7210000000000006e-06 2024-09-18T11:30:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116293+0000", 2024-09-18T11:30:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8739999999999997e-06 2024-09-18T11:30:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118772+0000", 2024-09-18T11:30:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024794560000000001 2024-09-18T11:30:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118852+0000", 2024-09-18T11:30:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9338999999999994e-05 2024-09-18T11:30:34.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.120432+0000", 2024-09-18T11:30:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015806749999999999 2024-09-18T11:30:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:879 4.e 4:7410c0e0:::smithi11939506-47:2 [sparse-read 0~1211253] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116299+0000", 2024-09-18T11:30:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.817323992999999, 2024-09-18T11:30:34.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043627500000000003, 2024-09-18T11:30:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 879 2024-09-18T11:30:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116299+0000", 2024-09-18T11:30:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116299+0000", 2024-09-18T11:30:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116299+0000", 2024-09-18T11:30:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116301+0000", 2024-09-18T11:30:34.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6860000000000002e-06 2024-09-18T11:30:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116309+0000", 2024-09-18T11:30:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1300000000000003e-06 2024-09-18T11:30:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116315+0000", 2024-09-18T11:30:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3219999999999998e-06 2024-09-18T11:30:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.120452+0000", 2024-09-18T11:30:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041371539999999997 2024-09-18T11:30:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.120521+0000", 2024-09-18T11:30:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8674e-05 2024-09-18T11:30:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.120662+0000", 2024-09-18T11:30:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000141225 2024-09-18T11:30:34.945 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1026 4.c 4:323f30dd:::smithi11939506-12:head [read 0~3176490,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.686142+0000", 2024-09-18T11:30:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.247481278999999, 2024-09-18T11:30:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048199560000000002, 2024-09-18T11:30:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:34.946 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1026 2024-09-18T11:30:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686142+0000", 2024-09-18T11:30:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686142+0000", 2024-09-18T11:30:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686141+0000", 2024-09-18T11:30:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686145+0000", 2024-09-18T11:30:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0999999999999997e-06 2024-09-18T11:30:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686153+0000", 2024-09-18T11:30:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0250000000000002e-06 2024-09-18T11:30:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686158+0000", 2024-09-18T11:30:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1389999999999997e-06 2024-09-18T11:30:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689573+0000", 2024-09-18T11:30:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034151369999999999 2024-09-18T11:30:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689630+0000", 2024-09-18T11:30:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6830000000000003e-05 2024-09-18T11:30:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690962+0000", 2024-09-18T11:30:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013323090000000001 2024-09-18T11:30:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1027 4.c 4:323f30dd:::smithi11939506-12:head [sparse-read 0~2047579] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.686162+0000", 2024-09-18T11:30:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.247461188999999, 2024-09-18T11:30:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050932319999999996, 2024-09-18T11:30:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1027 2024-09-18T11:30:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:34.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686162+0000", 2024-09-18T11:30:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686162+0000", 2024-09-18T11:30:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:34.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686162+0000", 2024-09-18T11:30:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686164+0000", 2024-09-18T11:30:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3489999999999999e-06 2024-09-18T11:30:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686169+0000", 2024-09-18T11:30:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1039999999999998e-06 2024-09-18T11:30:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686172+0000", 2024-09-18T11:30:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3680000000000002e-06 2024-09-18T11:30:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690990+0000", 2024-09-18T11:30:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048174460000000004 2024-09-18T11:30:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.691056+0000", 2024-09-18T11:30:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6192000000000006e-05 2024-09-18T11:30:34.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.691255+0000", 2024-09-18T11:30:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000199142 2024-09-18T11:30:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1029 3.3 3:df268f6c:::8da2985a35cccabf5edc5a8900d2327bcfe6ed768fa38d466fa9a390ce713320:head [write 0~356864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.689033+0000", 2024-09-18T11:30:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.244590024000001, 2024-09-18T11:30:34.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053345750000000003, 2024-09-18T11:30:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1029 2024-09-18T11:30:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689033+0000", 2024-09-18T11:30:34.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689033+0000", 2024-09-18T11:30:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689031+0000", 2024-09-18T11:30:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690898+0000", 2024-09-18T11:30:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001867032 2024-09-18T11:30:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690914+0000", 2024-09-18T11:30:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5806000000000001e-05 2024-09-18T11:30:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690921+0000", 2024-09-18T11:30:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8859999999999998e-06 2024-09-18T11:30:34.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690977+0000", 2024-09-18T11:30:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5612999999999998e-05 2024-09-18T11:30:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.691142+0000", 2024-09-18T11:30:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000165112 2024-09-18T11:30:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:34.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.691351+0000", 2024-09-18T11:30:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020942700000000001 2024-09-18T11:30:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.692616+0000", 2024-09-18T11:30:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012648200000000001 2024-09-18T11:30:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.694303+0000", 2024-09-18T11:30:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016875320000000001 2024-09-18T11:30:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.694335+0000", 2024-09-18T11:30:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1971000000000003e-05 2024-09-18T11:30:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.694368+0000", 2024-09-18T11:30:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2403000000000001e-05 2024-09-18T11:30:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1133 4.1 4:8b9a985d:::smithi11939506-23:a [read 0~3327753,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.178999+0000", 2024-09-18T11:30:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.754624052, 2024-09-18T11:30:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059312619999999996, 2024-09-18T11:30:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1133 2024-09-18T11:30:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.962 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178999+0000", 2024-09-18T11:30:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178999+0000", 2024-09-18T11:30:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178997+0000", 2024-09-18T11:30:34.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179012+0000", 2024-09-18T11:30:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5166e-05 2024-09-18T11:30:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179043+0000", 2024-09-18T11:30:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0892999999999997e-05 2024-09-18T11:30:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179057+0000", 2024-09-18T11:30:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4212e-05 2024-09-18T11:30:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179217+0000", 2024-09-18T11:30:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016031699999999999 2024-09-18T11:30:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179235+0000", 2024-09-18T11:30:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7903000000000001e-05 2024-09-18T11:30:34.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:34.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.181282+0000", 2024-09-18T11:30:34.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002046614 2024-09-18T11:30:34.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:34.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.181350+0000", 2024-09-18T11:30:34.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8190999999999998e-05 2024-09-18T11:30:34.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:34.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184930+0000", 2024-09-18T11:30:34.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035805960000000001 2024-09-18T11:30:34.967 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.967 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:34.967 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.967 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1134 4.1 4:8b9a985d:::smithi11939506-23:a [sparse-read 0~1709094] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:34.967 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:34.967 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.754530271, 2024-09-18T11:30:34.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061866400000000002, 2024-09-18T11:30:34.967 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:34.967 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:34.967 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:34.968 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:34.968 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.968 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1134 2024-09-18T11:30:34.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.968 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:34.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:34.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:34.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:34.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:34.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:34.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179092+0000", 2024-09-18T11:30:34.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:34.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:34.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179104+0000", 2024-09-18T11:30:34.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2346e-05 2024-09-18T11:30:34.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:34.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179128+0000", 2024-09-18T11:30:34.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4046999999999999e-05 2024-09-18T11:30:34.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179141+0000", 2024-09-18T11:30:34.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3149000000000001e-05 2024-09-18T11:30:34.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:34.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179406+0000", 2024-09-18T11:30:34.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026496100000000003 2024-09-18T11:30:34.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:34.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179431+0000", 2024-09-18T11:30:34.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5242999999999999e-05 2024-09-18T11:30:34.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:34.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184981+0000", 2024-09-18T11:30:34.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055495969999999999 2024-09-18T11:30:34.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:34.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185055+0000", 2024-09-18T11:30:34.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4375999999999994e-05 2024-09-18T11:30:34.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:34.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185280+0000", 2024-09-18T11:30:34.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022433800000000001 2024-09-18T11:30:34.972 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.972 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:34.972 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.972 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1135 4.1 4:8b9a985d:::smithi11939506-23:a [sparse-read 0~1709094] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:34.972 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179201+0000", 2024-09-18T11:30:34.972 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.754421824, 2024-09-18T11:30:34.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063123149999999998, 2024-09-18T11:30:34.972 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:34.973 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:34.973 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:34.973 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:34.973 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.973 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1135 2024-09-18T11:30:34.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.973 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:34.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:34.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179201+0000", 2024-09-18T11:30:34.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:34.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179201+0000", 2024-09-18T11:30:34.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:34.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179200+0000", 2024-09-18T11:30:34.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:34.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:34.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179207+0000", 2024-09-18T11:30:34.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1960000000000002e-06 2024-09-18T11:30:34.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:34.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179229+0000", 2024-09-18T11:30:34.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1322999999999999e-05 2024-09-18T11:30:34.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179240+0000", 2024-09-18T11:30:34.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1793e-05 2024-09-18T11:30:34.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:34.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179460+0000", 2024-09-18T11:30:34.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000219407 2024-09-18T11:30:34.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:34.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179466+0000", 2024-09-18T11:30:34.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7490000000000004e-06 2024-09-18T11:30:34.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:34.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185300+0000", 2024-09-18T11:30:34.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058347490000000002 2024-09-18T11:30:34.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:34.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185345+0000", 2024-09-18T11:30:34.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4316000000000001e-05 2024-09-18T11:30:34.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:34.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185514+0000", 2024-09-18T11:30:34.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016905800000000001 2024-09-18T11:30:34.977 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.977 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:34.977 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.977 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.6.13:26 3.3 3:f159e761:::e59e93d7c4f7efebda4afc5896e5fcf5614bf8edbf483060bccde004b8ac774d:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:34.978 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.678107+0000", 2024-09-18T11:30:34.978 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.255516348, 2024-09-18T11:30:34.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0044766939999999998, 2024-09-18T11:30:34.978 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:34.978 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:34.978 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:34.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:34.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678107+0000", 2024-09-18T11:30:34.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:34.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678107+0000", 2024-09-18T11:30:34.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:34.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678105+0000", 2024-09-18T11:30:34.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:34.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:34.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678351+0000", 2024-09-18T11:30:34.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000245751 2024-09-18T11:30:34.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:34.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678391+0000", 2024-09-18T11:30:34.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0695000000000003e-05 2024-09-18T11:30:34.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678405+0000", 2024-09-18T11:30:34.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.309e-05 2024-09-18T11:30:34.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:34.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678476+0000", 2024-09-18T11:30:34.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0938000000000004e-05 2024-09-18T11:30:34.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:34.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678573+0000", 2024-09-18T11:30:34.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7331999999999996e-05 2024-09-18T11:30:34.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:34.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.678751+0000", 2024-09-18T11:30:34.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017789100000000001 2024-09-18T11:30:34.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:34.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.680035+0000", 2024-09-18T11:30:34.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012847469999999999 2024-09-18T11:30:34.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:34.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682495+0000", 2024-09-18T11:30:34.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024598599999999999 2024-09-18T11:30:34.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:34.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682535+0000", 2024-09-18T11:30:34.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9187999999999998e-05 2024-09-18T11:30:34.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:34.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682584+0000", 2024-09-18T11:30:34.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9048000000000002e-05 2024-09-18T11:30:34.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.983 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:34.983 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.983 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1185 4.5 e39/22 4:af4ba75a:::smithi11939506-37:head v 39'34, mlcod=38'31)", 2024-09-18T11:30:34.983 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.157513+0000", 2024-09-18T11:30:34.983 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.776109884, 2024-09-18T11:30:34.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047192279999999998, 2024-09-18T11:30:34.983 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:34.983 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:34.983 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:34.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:34.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157513+0000", 2024-09-18T11:30:34.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:34.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157513+0000", 2024-09-18T11:30:34.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:34.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157512+0000", 2024-09-18T11:30:34.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:34.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:34.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.158094+0000", 2024-09-18T11:30:34.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00058189099999999996 2024-09-18T11:30:34.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:34.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.158110+0000", 2024-09-18T11:30:34.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5345e-05 2024-09-18T11:30:34.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.158117+0000", 2024-09-18T11:30:34.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.002e-06 2024-09-18T11:30:34.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:34.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.158151+0000", 2024-09-18T11:30:34.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4335000000000002e-05 2024-09-18T11:30:34.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:34.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.158165+0000", 2024-09-18T11:30:34.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3677e-05 2024-09-18T11:30:34.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:34.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162208+0000", 2024-09-18T11:30:34.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040431829999999997 2024-09-18T11:30:34.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:34.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162233+0000", 2024-09-18T11:30:34.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4783999999999999e-05 2024-09-18T11:30:34.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.987 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:34.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.987 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1186 4.5 e39/22 4:af4ba75a:::smithi11939506-37:head v 39'35, mlcod=39'32)", 2024-09-18T11:30:34.987 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.164439+0000", 2024-09-18T11:30:34.987 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.769184591, 2024-09-18T11:30:34.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004086569, 2024-09-18T11:30:34.987 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:34.987 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:34.987 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:34.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:34.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164439+0000", 2024-09-18T11:30:34.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:34.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164439+0000", 2024-09-18T11:30:34.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:34.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164438+0000", 2024-09-18T11:30:34.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:34.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:34.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164613+0000", 2024-09-18T11:30:34.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017544300000000001 2024-09-18T11:30:34.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:34.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164625+0000", 2024-09-18T11:30:34.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2088e-05 2024-09-18T11:30:34.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164631+0000", 2024-09-18T11:30:34.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7610000000000004e-06 2024-09-18T11:30:34.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:34.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164659+0000", 2024-09-18T11:30:34.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7770999999999999e-05 2024-09-18T11:30:34.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:34.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164674+0000", 2024-09-18T11:30:34.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4620000000000001e-05 2024-09-18T11:30:34.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:34.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168491+0000", 2024-09-18T11:30:34.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038169810000000001 2024-09-18T11:30:34.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:34.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168525+0000", 2024-09-18T11:30:34.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4579000000000002e-05 2024-09-18T11:30:34.991 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.991 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:34.991 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.991 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1378 4.d 4:b81644f2:::smithi11939506-34:head [write 615156~555834,stat] snapc c=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-18T11:30:34.992 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:25.201447+0000", 2024-09-18T11:30:34.992 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.7321762700000001, 2024-09-18T11:30:34.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060023309999999996, 2024-09-18T11:30:34.992 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:34.992 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:34.992 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:34.992 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:34.992 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.992 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1378 2024-09-18T11:30:34.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.992 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:34.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:34.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.201447+0000", 2024-09-18T11:30:34.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:34.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.201447+0000", 2024-09-18T11:30:34.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:34.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.201445+0000", 2024-09-18T11:30:34.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:34.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:34.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202027+0000", 2024-09-18T11:30:34.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00058162700000000003 2024-09-18T11:30:34.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:34.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202044+0000", 2024-09-18T11:30:34.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6685e-05 2024-09-18T11:30:34.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:34.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202051+0000", 2024-09-18T11:30:34.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0890000000000002e-06 2024-09-18T11:30:34.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:34.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202086+0000", 2024-09-18T11:30:34.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5593999999999998e-05 2024-09-18T11:30:34.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:34.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202153+0000", 2024-09-18T11:30:34.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7237999999999996e-05 2024-09-18T11:30:34.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:34.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202277+0000", 2024-09-18T11:30:34.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012383899999999999 2024-09-18T11:30:34.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:34.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.205380+0000", 2024-09-18T11:30:34.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031026109999999999 2024-09-18T11:30:34.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:34.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.207341+0000", 2024-09-18T11:30:34.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019606419999999999 2024-09-18T11:30:34.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:34.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.207403+0000", 2024-09-18T11:30:34.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2921000000000003e-05 2024-09-18T11:30:34.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:34.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.207449+0000", 2024-09-18T11:30:34.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5824e-05 2024-09-18T11:30:34.997 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.997 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:34.997 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:34.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.997 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1421 4.d 4:b5a463bf:::smithi11939506-20:head [write 665577~781581] snapc 10=[10,e,c] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T11:30:34.998 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:29.216974+0000", 2024-09-18T11:30:34.998 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.71664964, 2024-09-18T11:30:34.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006459869, 2024-09-18T11:30:34.998 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:34.998 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:34.998 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:34.998 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:34.998 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:34.998 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1421 2024-09-18T11:30:34.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.998 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:34.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:34.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.216974+0000", 2024-09-18T11:30:34.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:34.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.216974+0000", 2024-09-18T11:30:34.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:34.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:34.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:34.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:34.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.216972+0000", 2024-09-18T11:30:34.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:34.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:35.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217766+0000", 2024-09-18T11:30:35.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079344699999999997 2024-09-18T11:30:35.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:35.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217780+0000", 2024-09-18T11:30:35.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.428e-05 2024-09-18T11:30:35.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:35.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217786+0000", 2024-09-18T11:30:35.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9050000000000002e-06 2024-09-18T11:30:35.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:35.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217828+0000", 2024-09-18T11:30:35.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1839999999999999e-05 2024-09-18T11:30:35.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:35.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217897+0000", 2024-09-18T11:30:35.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9663e-05 2024-09-18T11:30:35.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:35.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.218013+0000", 2024-09-18T11:30:35.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011509 2024-09-18T11:30:35.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:35.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.220250+0000", 2024-09-18T11:30:35.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022370150000000002 2024-09-18T11:30:35.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:35.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.223359+0000", 2024-09-18T11:30:35.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031092519999999998 2024-09-18T11:30:35.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:35.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.223396+0000", 2024-09-18T11:30:35.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7531999999999999e-05 2024-09-18T11:30:35.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:35.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.223433+0000", 2024-09-18T11:30:35.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7098999999999998e-05 2024-09-18T11:30:35.003 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:35.003 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:35.003 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:35.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.004 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1528 3.1 e53/21 3:97025964:::d85d35018f93ce704e0c86e65d6c262758d2b38883b05fc1a96898c5a961f23a:head v 53'86, mlcod=51'84)", 2024-09-18T11:30:35.004 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.377855+0000", 2024-09-18T11:30:35.004 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.55576773599999996, 2024-09-18T11:30:35.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058274199999999998, 2024-09-18T11:30:35.004 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:35.004 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:35.004 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:35.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:35.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.377855+0000", 2024-09-18T11:30:35.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:35.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:35.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.377855+0000", 2024-09-18T11:30:35.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:35.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:35.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.377853+0000", 2024-09-18T11:30:35.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:35.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:35.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.379683+0000", 2024-09-18T11:30:35.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018298139999999999 2024-09-18T11:30:35.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:35.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.379698+0000", 2024-09-18T11:30:35.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4620000000000001e-05 2024-09-18T11:30:35.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:35.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.379722+0000", 2024-09-18T11:30:35.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4505999999999999e-05 2024-09-18T11:30:35.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:35.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.379796+0000", 2024-09-18T11:30:35.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3527000000000003e-05 2024-09-18T11:30:35.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:35.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.379825+0000", 2024-09-18T11:30:35.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.953e-05 2024-09-18T11:30:35.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:35.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.383640+0000", 2024-09-18T11:30:35.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038143629999999999 2024-09-18T11:30:35.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:35.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.383683+0000", 2024-09-18T11:30:35.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3053000000000003e-05 2024-09-18T11:30:35.008 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:35.008 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:35.008 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:35.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.009 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1530 4.d 4:b81644f2:::smithi11939506-34:head [write 467261~717275] snapc 14=[14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:30:35.009 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:30:35.009 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.542938692, 2024-09-18T11:30:35.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075660950000000001, 2024-09-18T11:30:35.009 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:35.009 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:35.009 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:35.009 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:35.009 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:35.009 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1530 2024-09-18T11:30:35.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.010 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:35.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:35.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:30:35.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:35.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:35.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:30:35.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:35.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:35.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390682+0000", 2024-09-18T11:30:35.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:35.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:35.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391554+0000", 2024-09-18T11:30:35.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00087220600000000004 2024-09-18T11:30:35.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:35.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391570+0000", 2024-09-18T11:30:35.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6002999999999999e-05 2024-09-18T11:30:35.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:35.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391575+0000", 2024-09-18T11:30:35.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2549999999999997e-06 2024-09-18T11:30:35.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:35.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391631+0000", 2024-09-18T11:30:35.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5671000000000003e-05 2024-09-18T11:30:35.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:35.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391751+0000", 2024-09-18T11:30:35.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000119821 2024-09-18T11:30:35.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:35.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391974+0000", 2024-09-18T11:30:35.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022309900000000001 2024-09-18T11:30:35.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:35.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398136+0000", 2024-09-18T11:30:35.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006161573 2024-09-18T11:30:35.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:35.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398184+0000", 2024-09-18T11:30:35.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8458000000000003e-05 2024-09-18T11:30:35.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:35.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398219+0000", 2024-09-18T11:30:35.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4573999999999998e-05 2024-09-18T11:30:35.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:35.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398251+0000", 2024-09-18T11:30:35.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2085000000000002e-05 2024-09-18T11:30:35.015 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:35.015 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:35.015 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:35.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.015 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1532 4.d 4:b81644f2:::smithi11939506-34:head [setxattr _header (57) in=7b,truncate 2067261] snapc 14=[14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:30:35.015 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.395599+0000", 2024-09-18T11:30:35.015 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.53802377499999998, 2024-09-18T11:30:35.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054503360000000001, 2024-09-18T11:30:35.015 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:35.015 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1532 2024-09-18T11:30:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395599+0000", 2024-09-18T11:30:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395599+0000", 2024-09-18T11:30:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395599+0000", 2024-09-18T11:30:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395603+0000", 2024-09-18T11:30:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3640000000000001e-06 2024-09-18T11:30:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395610+0000", 2024-09-18T11:30:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9090000000000003e-06 2024-09-18T11:30:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395615+0000", 2024-09-18T11:30:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6269999999999999e-06 2024-09-18T11:30:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.397243+0000", 2024-09-18T11:30:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016279350000000001 2024-09-18T11:30:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.397343+0000", 2024-09-18T11:30:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000100054 2024-09-18T11:30:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.397496+0000", 2024-09-18T11:30:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000153578 2024-09-18T11:30:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398472+0000", 2024-09-18T11:30:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00097554100000000002 2024-09-18T11:30:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.400978+0000", 2024-09-18T11:30:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025066590000000001 2024-09-18T11:30:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.401013+0000", 2024-09-18T11:30:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4357999999999999e-05 2024-09-18T11:30:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.401050+0000", 2024-09-18T11:30:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7052999999999999e-05 2024-09-18T11:30:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1533 4.d 4:b81644f2:::smithi11939506-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:30:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.395621+0000", 2024-09-18T11:30:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.53800230000000004, 2024-09-18T11:30:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055874380000000001, 2024-09-18T11:30:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1533 2024-09-18T11:30:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395621+0000", 2024-09-18T11:30:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395621+0000", 2024-09-18T11:30:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395621+0000", 2024-09-18T11:30:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395623+0000", 2024-09-18T11:30:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.841e-06 2024-09-18T11:30:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395630+0000", 2024-09-18T11:30:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.285e-06 2024-09-18T11:30:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395633+0000", 2024-09-18T11:30:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.614e-06 2024-09-18T11:30:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398334+0000", 2024-09-18T11:30:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027003460000000002 2024-09-18T11:30:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398422+0000", 2024-09-18T11:30:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8506999999999998e-05 2024-09-18T11:30:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.401087+0000", 2024-09-18T11:30:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026650900000000002 2024-09-18T11:30:35.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:35.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.401138+0000", 2024-09-18T11:30:35.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0566999999999999e-05 2024-09-18T11:30:35.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:35.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.401208+0000", 2024-09-18T11:30:35.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0603000000000001e-05 2024-09-18T11:30:35.026 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:35.026 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:35.026 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:35.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.027 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1539 4.1 4:87078dfe:::smithi11939506-14:c [read 0~3109262,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:30:35.027 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.403371+0000", 2024-09-18T11:30:35.027 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.53025201, 2024-09-18T11:30:35.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042633300000000001, 2024-09-18T11:30:35.027 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:35.027 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:35.027 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:35.027 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:35.027 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:35.027 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1539 2024-09-18T11:30:35.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.028 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:35.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:35.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.403371+0000", 2024-09-18T11:30:35.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:35.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:35.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.403371+0000", 2024-09-18T11:30:35.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:35.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:35.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.403371+0000", 2024-09-18T11:30:35.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:35.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:35.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.403376+0000", 2024-09-18T11:30:35.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0830000000000003e-06 2024-09-18T11:30:35.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:35.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.403385+0000", 2024-09-18T11:30:35.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2259999999999994e-06 2024-09-18T11:30:35.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:35.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.403390+0000", 2024-09-18T11:30:35.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4770000000000002e-06 2024-09-18T11:30:35.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:35.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.405913+0000", 2024-09-18T11:30:35.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025230280000000001 2024-09-18T11:30:35.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:35.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.406000+0000", 2024-09-18T11:30:35.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6570000000000006e-05 2024-09-18T11:30:35.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:35.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.407635+0000", 2024-09-18T11:30:35.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001634588 2024-09-18T11:30:35.031 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:35.031 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:35.031 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:35.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.031 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1546 4.c 4:323f30dd:::smithi11939506-12:head [sparse-read 0~2047579] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:30:35.031 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.403492+0000", 2024-09-18T11:30:35.031 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.53013171199999998, 2024-09-18T11:30:35.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040109000000000004, 2024-09-18T11:30:35.032 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:35.032 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:35.032 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:35.032 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:35.032 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:35.032 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1546 2024-09-18T11:30:35.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.032 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:35.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.032 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:35.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.403492+0000", 2024-09-18T11:30:35.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:35.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:35.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.403492+0000", 2024-09-18T11:30:35.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:35.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:35.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.403491+0000", 2024-09-18T11:30:35.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:35.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:35.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.403494+0000", 2024-09-18T11:30:35.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0510000000000001e-06 2024-09-18T11:30:35.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:35.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.403502+0000", 2024-09-18T11:30:35.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6839999999999996e-06 2024-09-18T11:30:35.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:35.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.403508+0000", 2024-09-18T11:30:35.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8390000000000003e-06 2024-09-18T11:30:35.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:35.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.407174+0000", 2024-09-18T11:30:35.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036667330000000001 2024-09-18T11:30:35.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:35.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.407248+0000", 2024-09-18T11:30:35.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3875999999999995e-05 2024-09-18T11:30:35.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:35.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.407502+0000", 2024-09-18T11:30:35.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00025420100000000001 2024-09-18T11:30:35.035 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:35.035 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:35.036 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:35.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.036 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:30:35.036 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:30:35.036 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.51753636800000002, 2024-09-18T11:30:35.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:30:35.036 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:35.036 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:35.036 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:35.036 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:35.036 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:35.036 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:30:35.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.037 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:35.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:35.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:30:35.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:35.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:35.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:30:35.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:35.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:35.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:30:35.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:35.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:35.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:30:35.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:30:35.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:35.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:30:35.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:30:35.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:35.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:30:35.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:30:35.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:35.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:30:35.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:30:35.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:35.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:30:35.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:30:35.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:35.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:30:35.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:30:35.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:35.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:30:35.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:30:35.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:35.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:30:35.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:30:35.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:35.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:30:35.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:30:35.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:35.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:35.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:35.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:30:35.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:30:35.042 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:35.042 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:35.042 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:35.042 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:35.042 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:35.042 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:35.043 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:30:35.043 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:30:35.044 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:35.044 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:35.045 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:30:35.045 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:35.041+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:35.045 INFO:tasks.rados.rados.0.smithi119.stdout:873: finishing set_chunk to oid 44 2024-09-18T11:30:35.045 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 44 set_chunk returned error code -95 offset: 116736 length: 1404928 2024-09-18T11:30:35.045 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 44 v 0 (ObjNum 72 snap 3 seq_num 72) dirty exists 2024-09-18T11:30:35.046 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:35.046 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:35.046 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:35.046 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:35.047 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:35.047 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:35.049 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T11:30:35.050 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:35.050 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:35.065 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:35.065 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:35.080 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:35.081 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:35.144 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:35.142+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:35.183 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:35.184 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:35.192 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:30:35.245 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:35.243+0000 7fcb682b1640 -1 osd.7 53 handle_osd_map injecting map crc failure 2024-09-18T11:30:35.246 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:35.244+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:35.248 INFO:tasks.rados.rados.0.smithi119.stdout:866: done (8 left) 2024-09-18T11:30:35.248 INFO:tasks.rados.rados.0.smithi119.stdout:867: done (7 left) 2024-09-18T11:30:35.248 INFO:tasks.rados.rados.0.smithi119.stdout:868: done (6 left) 2024-09-18T11:30:35.249 INFO:tasks.rados.rados.0.smithi119.stdout:869: done (5 left) 2024-09-18T11:30:35.249 INFO:tasks.rados.rados.0.smithi119.stdout:870: done (4 left) 2024-09-18T11:30:35.249 INFO:tasks.rados.rados.0.smithi119.stdout:871: done (3 left) 2024-09-18T11:30:35.249 INFO:tasks.rados.rados.0.smithi119.stdout:872: done (2 left) 2024-09-18T11:30:35.249 INFO:tasks.rados.rados.0.smithi119.stdout:873: done (1 left) 2024-09-18T11:30:35.249 INFO:tasks.rados.rados.0.smithi119.stdout:874: done (0 left) 2024-09-18T11:30:35.249 INFO:tasks.rados.rados.0.smithi119.stdout:875: read oid 45 snap -1 2024-09-18T11:30:35.249 INFO:tasks.rados.rados.0.smithi119.stdout:875: expect (ObjNum 113 snap 11 seq_num 113) 2024-09-18T11:30:35.249 INFO:tasks.rados.rados.0.smithi119.stdout:876: read oid 3 snap -1 2024-09-18T11:30:35.249 INFO:tasks.rados.rados.0.smithi119.stdout:876: expect (ObjNum 77 snap 3 seq_num 77) 2024-09-18T11:30:35.249 INFO:tasks.rados.rados.0.smithi119.stdout:877: read oid 10 snap -1 2024-09-18T11:30:35.249 INFO:tasks.rados.rados.0.smithi119.stdout:877: expect deleted 2024-09-18T11:30:35.249 INFO:tasks.rados.rados.0.smithi119.stdout:878: read oid 26 snap 10 2024-09-18T11:30:35.249 INFO:tasks.rados.rados.0.smithi119.stdout:878: expect (ObjNum 98 snap 6 seq_num 98) 2024-09-18T11:30:35.250 INFO:tasks.rados.rados.0.smithi119.stdout:879: write oid 37 current snap is 13 2024-09-18T11:30:35.250 INFO:tasks.rados.rados.0.smithi119.stdout:879: seq_num 124 ranges {484068=792524,1906473=177595} 2024-09-18T11:30:35.255 INFO:tasks.rados.rados.0.smithi119.stdout:879: writing smithi11939506-37 from 484068 to 1276592 tid 1 2024-09-18T11:30:35.260 INFO:tasks.rados.rados.0.smithi119.stdout:879: writing smithi11939506-37 from 1906473 to 2084068 tid 2 2024-09-18T11:30:35.260 INFO:tasks.rados.rados.0.smithi119.stdout:880: snap_create 2024-09-18T11:30:35.260 INFO:tasks.rados.rados.0.smithi119.stdout:876: expect (ObjNum 77 snap 3 seq_num 77) 2024-09-18T11:30:35.335 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:35.335 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:35.344 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:30:35.385 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:35.383+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:35.446 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:35.445+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:35.457 INFO:tasks.rados.rados.0.smithi119.stdout:875: expect (ObjNum 113 snap 11 seq_num 113) 2024-09-18T11:30:35.482 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:35.482 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:35.490 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:30:35.547 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:35.546+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:35.590 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:35.590 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:35.599 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:30:35.647 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:35.645+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:35.671 INFO:tasks.rados.rados.0.smithi119.stdout:879: finishing write tid 1 to smithi11939506-37 2024-09-18T11:30:35.671 INFO:tasks.rados.rados.0.smithi119.stdout:879: finishing write tid 2 to smithi11939506-37 2024-09-18T11:30:35.671 INFO:tasks.rados.rados.0.smithi119.stdout:879: finishing write tid 3 to smithi11939506-37 2024-09-18T11:30:35.671 INFO:tasks.rados.rados.0.smithi119.stdout:879: finishing write tid 4 to smithi11939506-37 2024-09-18T11:30:35.671 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 60 (ObjNum 124 snap 13 seq_num 124) dirty exists 2024-09-18T11:30:35.671 INFO:tasks.rados.rados.0.smithi119.stdout:879: left oid 37 (ObjNum 124 snap 13 seq_num 124) 2024-09-18T11:30:35.671 INFO:tasks.rados.rados.0.smithi119.stdout:878: expect (ObjNum 98 snap 6 seq_num 98) 2024-09-18T11:30:35.698 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:35.698 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:35.706 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:30:35.748 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:35.747+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:35.845 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:35.845 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:35.849 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:35.848+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:35.854 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:30:35.949 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:35.948+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:35.952 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:35.952 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:35.961 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:30:36.050 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:36.049+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:36.058 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:36.058 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:36.067 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:30:36.151 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:36.150+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:36.164 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:36.164 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:36.173 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:30:36.251 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:36.250+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:36.255 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is set 2024-09-18T11:30:36.255 INFO:tasks.rados.rados.0.smithi119.stdout:875: done (5 left) 2024-09-18T11:30:36.255 INFO:tasks.rados.rados.0.smithi119.stdout:876: done (4 left) 2024-09-18T11:30:36.256 INFO:tasks.rados.rados.0.smithi119.stdout:877: done (3 left) 2024-09-18T11:30:36.256 INFO:tasks.rados.rados.0.smithi119.stdout:878: done (2 left) 2024-09-18T11:30:36.256 INFO:tasks.rados.rados.0.smithi119.stdout:879: done (1 left) 2024-09-18T11:30:36.256 INFO:tasks.rados.rados.0.smithi119.stdout:880: done (0 left) 2024-09-18T11:30:36.256 INFO:tasks.rados.rados.0.smithi119.stdout:881: write (excl) oid 30 current snap is 14 2024-09-18T11:30:36.256 INFO:tasks.rados.rados.0.smithi119.stdout:881: seq_num 125 ranges {500875=467773,1564955=535920} 2024-09-18T11:30:36.261 INFO:tasks.rados.rados.0.smithi119.stdout:881: writing smithi11939506-30 from 500875 to 968648 tid 1 2024-09-18T11:30:36.267 INFO:tasks.rados.rados.0.smithi119.stdout:881: writing smithi11939506-30 from 1564955 to 2100875 tid 2 2024-09-18T11:30:36.267 INFO:tasks.rados.rados.0.smithi119.stdout:882: rollback oid 21 current snap is 14 2024-09-18T11:30:36.268 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 21 to 8 2024-09-18T11:30:36.268 INFO:tasks.rados.rados.0.smithi119.stdout:881: finishing write tid 1 to smithi11939506-30 2024-09-18T11:30:36.268 INFO:tasks.rados.rados.0.smithi119.stdout:883: tier_flush oid 25 2024-09-18T11:30:36.268 INFO:tasks.rados.rados.0.smithi119.stdout:883: tier_flush oid 25 snap -1 2024-09-18T11:30:36.268 INFO:tasks.rados.rados.0.smithi119.stdout:884: read oid 43 snap -1 2024-09-18T11:30:36.268 INFO:tasks.rados.rados.0.smithi119.stdout:884: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:30:36.268 INFO:tasks.rados.rados.0.smithi119.stdout:885: write oid 16 current snap is 14 2024-09-18T11:30:36.268 INFO:tasks.rados.rados.0.smithi119.stdout:885: seq_num 126 ranges {517682=543022,1623437=494245} 2024-09-18T11:30:36.273 INFO:tasks.rados.rados.0.smithi119.stdout:885: writing smithi11939506-16 from 517682 to 1060704 tid 1 2024-09-18T11:30:36.280 INFO:tasks.rados.rados.0.smithi119.stdout:885: writing smithi11939506-16 from 1623437 to 2117682 tid 2 2024-09-18T11:30:36.280 INFO:tasks.rados.rados.0.smithi119.stdout:886: write oid 24 current snap is 14 2024-09-18T11:30:36.280 INFO:tasks.rados.rados.0.smithi119.stdout:886: seq_num 127 ranges {534489=618271,1565566=568923} 2024-09-18T11:30:36.283 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:36.284 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:36.287 INFO:tasks.rados.rados.0.smithi119.stdout:886: writing smithi11939506-24 from 534489 to 1152760 tid 1 2024-09-18T11:30:36.293 INFO:tasks.rados.rados.0.smithi119.stdout:886: writing smithi11939506-24 from 1565566 to 2134489 tid 2 2024-09-18T11:30:36.293 INFO:tasks.rados.rados.0.smithi119.stdout:887: delete oid 11 current snap is 14 2024-09-18T11:30:36.293 INFO:tasks.rados.rados.0.smithi119.stdout:882: finishing rollback tid 1 to smithi11939506-21 2024-09-18T11:30:36.294 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 0 (ObjNum 1254071520 snap 32633 seq_num 304) dirty dne 2024-09-18T11:30:36.294 INFO:tasks.rados.rados.0.smithi119.stdout:881: finishing write tid 2 to smithi11939506-30 2024-09-18T11:30:36.294 INFO:tasks.rados.rados.0.smithi119.stdout:881: finishing write tid 3 to smithi11939506-30 2024-09-18T11:30:36.294 INFO:tasks.rados.rados.0.smithi119.stdout:881: finishing write tid 4 to smithi11939506-30 2024-09-18T11:30:36.294 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 30 v 29 (ObjNum 125 snap 14 seq_num 125) dirty exists 2024-09-18T11:30:36.294 INFO:tasks.rados.rados.0.smithi119.stdout:881: left oid 30 (ObjNum 125 snap 14 seq_num 125) 2024-09-18T11:30:36.294 INFO:tasks.rados.rados.0.smithi119.stdout:885: finishing write tid 1 to smithi11939506-16 2024-09-18T11:30:36.294 INFO:tasks.rados.rados.0.smithi119.stdout:884: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:30:36.295 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:30:36.353 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:36.351+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:36.385 INFO:tasks.rados.rados.0.smithi119.stdout:885: finishing write tid 2 to smithi11939506-16 2024-09-18T11:30:36.385 INFO:tasks.rados.rados.0.smithi119.stdout:881: done (6 left) 2024-09-18T11:30:36.386 INFO:tasks.rados.rados.0.smithi119.stdout:882: done (5 left) 2024-09-18T11:30:36.386 INFO:tasks.rados.rados.0.smithi119.stdout:884: done (4 left) 2024-09-18T11:30:36.386 INFO:tasks.rados.rados.0.smithi119.stdout:887: done (3 left) 2024-09-18T11:30:36.386 INFO:tasks.rados.rados.0.smithi119.stdout:888: read oid 22 snap -1 2024-09-18T11:30:36.386 INFO:tasks.rados.rados.0.smithi119.stdout:888: expect (ObjNum 101 snap 6 seq_num 101) 2024-09-18T11:30:36.386 INFO:tasks.rados.rados.0.smithi119.stdout:889: read oid 38 snap -1 2024-09-18T11:30:36.386 INFO:tasks.rados.rados.0.smithi119.stdout:889: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-18T11:30:36.386 INFO:tasks.rados.rados.0.smithi119.stdout:890: snap_create 2024-09-18T11:30:36.386 INFO:tasks.rados.rados.0.smithi119.stdout:885: finishing write tid 3 to smithi11939506-16 2024-09-18T11:30:36.386 INFO:tasks.rados.rados.0.smithi119.stdout:885: finishing write tid 4 to smithi11939506-16 2024-09-18T11:30:36.386 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 72 (ObjNum 126 snap 14 seq_num 126) dirty exists 2024-09-18T11:30:36.386 INFO:tasks.rados.rados.0.smithi119.stdout:885: left oid 16 (ObjNum 126 snap 14 seq_num 126) 2024-09-18T11:30:36.386 INFO:tasks.rados.rados.0.smithi119.stdout:883: got (0) Success 2024-09-18T11:30:36.386 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 0 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-18T11:30:36.387 INFO:tasks.rados.rados.0.smithi119.stdout:886: finishing write tid 1 to smithi11939506-24 2024-09-18T11:30:36.387 INFO:tasks.rados.rados.0.smithi119.stdout:886: finishing write tid 2 to smithi11939506-24 2024-09-18T11:30:36.387 INFO:tasks.rados.rados.0.smithi119.stdout:886: finishing write tid 3 to smithi11939506-24 2024-09-18T11:30:36.387 INFO:tasks.rados.rados.0.smithi119.stdout:886: finishing write tid 4 to smithi11939506-24 2024-09-18T11:30:36.387 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 76 (ObjNum 127 snap 14 seq_num 127) dirty exists 2024-09-18T11:30:36.387 INFO:tasks.rados.rados.0.smithi119.stdout:886: left oid 24 (ObjNum 127 snap 14 seq_num 127) 2024-09-18T11:30:36.387 INFO:tasks.rados.rados.0.smithi119.stdout:888: expect (ObjNum 101 snap 6 seq_num 101) 2024-09-18T11:30:36.443 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:36.443 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:36.451 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:30:36.453 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:36.451+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:36.549 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:36.549 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:36.554 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:36.553+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:36.558 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:30:36.597 INFO:tasks.rados.rados.0.smithi119.stdout:889: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-18T11:30:36.655 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:36.652+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:36.695 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:36.695 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:36.704 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:30:36.755 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:36.754+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:36.802 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:36.802 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:36.810 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:30:36.856 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:36.854+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:36.950 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:36.950 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:36.957 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:36.956+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:36.959 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:30:37.058 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:37.057+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:37.092 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:30:37.097 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:37.098 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:37.107 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:30:37.191 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:37.190+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:37.220 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:30:37.220 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:30:37.236 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:37.236 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:37.251 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:37.252 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:37.254 INFO:tasks.rados.rados.0.smithi119.stdout:883: done (5 left) 2024-09-18T11:30:37.254 INFO:tasks.rados.rados.0.smithi119.stdout:885: done (4 left) 2024-09-18T11:30:37.254 INFO:tasks.rados.rados.0.smithi119.stdout:886: done (3 left) 2024-09-18T11:30:37.254 INFO:tasks.rados.rados.0.smithi119.stdout:888: done (2 left) 2024-09-18T11:30:37.254 INFO:tasks.rados.rados.0.smithi119.stdout:889: done (1 left) 2024-09-18T11:30:37.254 INFO:tasks.rados.rados.0.smithi119.stdout:890: done (0 left) 2024-09-18T11:30:37.254 INFO:tasks.rados.rados.0.smithi119.stdout:891: set_chunk oid 45 target oid 2024-09-18T11:30:37.258 INFO:tasks.rados.rados.0.smithi119.stdout:891: set_chunk oid 45 offset: 2383360 length: 258048 target oid 8f89fa43d45dad266eeeafd77d323b5639d6659229c5b44652420b4cd210cef7 offset: 0 snap -1 2024-09-18T11:30:37.259 INFO:tasks.rados.rados.0.smithi119.stdout:892: tier_flush oid 33 2024-09-18T11:30:37.259 INFO:tasks.rados.rados.0.smithi119.stdout:892: tier_flush oid 33 snap -1 2024-09-18T11:30:37.259 INFO:tasks.rados.rados.0.smithi119.stdout:893: write (excl) oid 17 current snap is 15 2024-09-18T11:30:37.259 INFO:tasks.rados.rados.0.smithi119.stdout:893: seq_num 129 ranges {568103=768769,2082530=85573} 2024-09-18T11:30:37.260 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:37.260 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:37.260 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:37.260 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:37.260 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:37.260 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:37.258+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:37.266 INFO:tasks.rados.rados.0.smithi119.stdout:893: writing smithi11939506-17 from 568103 to 1336872 tid 1 2024-09-18T11:30:37.269 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:37.269 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:37.270 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:30:37.271 INFO:tasks.rados.rados.0.smithi119.stdout:893: writing smithi11939506-17 from 2082530 to 2168103 tid 2 2024-09-18T11:30:37.271 INFO:tasks.rados.rados.0.smithi119.stdout:894: set_chunk oid 24 target oid 2024-09-18T11:30:37.282 INFO:tasks.rados.rados.0.smithi119.stdout:894: set_chunk oid 24 offset: 135680 length: 1043456 target oid 332b932314d630c5b9ab235005777e70280c65f6e38fd3b9acceb70469db8a91 offset: 0 snap -1 2024-09-18T11:30:37.282 INFO:tasks.rados.rados.0.smithi119.stdout:895: set_chunk oid 46 target oid 2024-09-18T11:30:37.287 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:37.287 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:37.295 INFO:tasks.rados.rados.0.smithi119.stdout:895: set_chunk oid 46 offset: 859648 length: 1221120 target oid c2b3fc9c1ba640cc8bcddb60552d5559ebeb3b68ed444eb13795808d4ae6858d offset: 0 snap 12 2024-09-18T11:30:37.295 INFO:tasks.rados.rados.0.smithi119.stdout:896: read oid 6 snap -1 2024-09-18T11:30:37.295 INFO:tasks.rados.rados.0.smithi119.stdout:896: expect (ObjNum 117 snap 11 seq_num 117) 2024-09-18T11:30:37.295 INFO:tasks.rados.rados.0.smithi119.stdout:897: read oid 18 snap -1 2024-09-18T11:30:37.295 INFO:tasks.rados.rados.0.smithi119.stdout:897: expect (ObjNum 116 snap 11 seq_num 116) 2024-09-18T11:30:37.295 INFO:tasks.rados.rados.0.smithi119.stdout:898: tier_evict oid 31 2024-09-18T11:30:37.295 INFO:tasks.rados.rados.0.smithi119.stdout:898: tier_evict oid 31 snap -1 2024-09-18T11:30:37.295 INFO:tasks.rados.rados.0.smithi119.stdout:892: got (2) No such file or directory 2024-09-18T11:30:37.296 INFO:tasks.rados.rados.0.smithi119.stdout:892: got expected ENOENT (src dne) 2024-09-18T11:30:37.296 INFO:tasks.rados.rados.0.smithi119.stdout:892: done (7 left) 2024-09-18T11:30:37.296 INFO:tasks.rados.rados.0.smithi119.stdout:899: read oid 42 snap -1 2024-09-18T11:30:37.296 INFO:tasks.rados.rados.0.smithi119.stdout:899: expect (ObjNum 99 snap 6 seq_num 99) 2024-09-18T11:30:37.296 INFO:tasks.rados.rados.0.smithi119.stdout:900: write (excl) oid 35 current snap is 15 2024-09-18T11:30:37.296 INFO:tasks.rados.rados.0.smithi119.stdout:900: seq_num 130 ranges {584910=560371,1857365=327545} 2024-09-18T11:30:37.301 INFO:tasks.rados.rados.0.smithi119.stdout:900: writing smithi11939506-35 from 584910 to 1145281 tid 1 2024-09-18T11:30:37.305 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:37.305 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:37.307 INFO:tasks.rados.rados.0.smithi119.stdout:900: writing smithi11939506-35 from 1857365 to 2184910 tid 2 2024-09-18T11:30:37.307 INFO:tasks.rados.rados.0.smithi119.stdout:901: set_chunk oid 50 target oid 2024-09-18T11:30:37.320 INFO:tasks.rados.rados.0.smithi119.stdout:901: set_chunk oid 50 offset: 2660864 length: 582144 target oid 77cdbf3abbada57d0636b2198046a030673818705cbf13bb77b30c3e68b24345 offset: 0 snap -1 2024-09-18T11:30:37.320 INFO:tasks.rados.rados.0.smithi119.stdout:902: rollback oid 36 current snap is 15 2024-09-18T11:30:37.320 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 36 to 6 2024-09-18T11:30:37.320 INFO:tasks.rados.rados.0.smithi119.stdout:891: finishing set_chunk to oid 45 2024-09-18T11:30:37.320 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 15 (ObjNum 113 snap 11 seq_num 113) dirty exists 2024-09-18T11:30:37.320 INFO:tasks.rados.rados.0.smithi119.stdout:891: done (10 left) 2024-09-18T11:30:37.320 INFO:tasks.rados.rados.0.smithi119.stdout:903: write oid 2 current snap is 15 2024-09-18T11:30:37.320 INFO:tasks.rados.rados.0.smithi119.stdout:893: finishing write tid 1 to smithi11939506-17 2024-09-18T11:30:37.321 INFO:tasks.rados.rados.0.smithi119.stdout:893: finishing write tid 2 to smithi11939506-17 2024-09-18T11:30:37.321 INFO:tasks.rados.rados.0.smithi119.stdout:903: seq_num 131 ranges {601717=635620,1799494=402223} 2024-09-18T11:30:37.325 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:37.325 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:37.326 INFO:tasks.rados.rados.0.smithi119.stdout:903: writing smithi11939506-2 from 601717 to 1237337 tid 1 2024-09-18T11:30:37.334 INFO:tasks.rados.rados.0.smithi119.stdout:903: writing smithi11939506-2 from 1799494 to 2201717 tid 2 2024-09-18T11:30:37.334 INFO:tasks.rados.rados.0.smithi119.stdout:904: read oid 21 snap -1 2024-09-18T11:30:37.334 INFO:tasks.rados.rados.0.smithi119.stdout:904: expect deleted 2024-09-18T11:30:37.334 INFO:tasks.rados.rados.0.smithi119.stdout:905: read oid 19 snap -1 2024-09-18T11:30:37.334 INFO:tasks.rados.rados.0.smithi119.stdout:905: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-18T11:30:37.335 INFO:tasks.rados.rados.0.smithi119.stdout:906: read oid 1 snap -1 2024-09-18T11:30:37.335 INFO:tasks.rados.rados.0.smithi119.stdout:906: expect (ObjNum 118 snap 11 seq_num 118) 2024-09-18T11:30:37.335 INFO:tasks.rados.rados.0.smithi119.stdout:907: read oid 4 snap -1 2024-09-18T11:30:37.335 INFO:tasks.rados.rados.0.smithi119.stdout:907: expect (ObjNum 110 snap 8 seq_num 110) 2024-09-18T11:30:37.335 INFO:tasks.rados.rados.0.smithi119.stdout:908: write oid 9 current snap is 15 2024-09-18T11:30:37.335 INFO:tasks.rados.rados.0.smithi119.stdout:893: finishing write tid 3 to smithi11939506-17 2024-09-18T11:30:37.335 INFO:tasks.rados.rados.0.smithi119.stdout:893: finishing write tid 4 to smithi11939506-17 2024-09-18T11:30:37.335 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 81 (ObjNum 129 snap 15 seq_num 129) dirty exists 2024-09-18T11:30:37.335 INFO:tasks.rados.rados.0.smithi119.stdout:893: left oid 17 (ObjNum 129 snap 15 seq_num 129) 2024-09-18T11:30:37.335 INFO:tasks.rados.rados.0.smithi119.stdout:894: finishing set_chunk to oid 24 2024-09-18T11:30:37.335 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 24 set_chunk returned error code -95 offset: 135680 length: 1043456 2024-09-18T11:30:37.335 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 0 (ObjNum 127 snap 14 seq_num 127) dirty exists 2024-09-18T11:30:37.335 INFO:tasks.rados.rados.0.smithi119.stdout:908: seq_num 132 ranges {618524=710869,1857976=360548} 2024-09-18T11:30:37.344 INFO:tasks.rados.rados.0.smithi119.stdout:908: writing smithi11939506-9 from 618524 to 1329393 tid 1 2024-09-18T11:30:37.350 INFO:tasks.rados.rados.0.smithi119.stdout:908: writing smithi11939506-9 from 1857976 to 2218524 tid 2 2024-09-18T11:30:37.350 INFO:tasks.rados.rados.0.smithi119.stdout:893: done (15 left) 2024-09-18T11:30:37.350 INFO:tasks.rados.rados.0.smithi119.stdout:894: done (14 left) 2024-09-18T11:30:37.350 INFO:tasks.rados.rados.0.smithi119.stdout:909: read oid 24 snap -1 2024-09-18T11:30:37.351 INFO:tasks.rados.rados.0.smithi119.stdout:909: expect (ObjNum 127 snap 14 seq_num 127) 2024-09-18T11:30:37.351 INFO:tasks.rados.rados.0.smithi119.stdout:910: read oid 30 snap -1 2024-09-18T11:30:37.351 INFO:tasks.rados.rados.0.smithi119.stdout:910: expect (ObjNum 125 snap 14 seq_num 125) 2024-09-18T11:30:37.351 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:37.351 INFO:tasks.rados.rados.0.smithi119.stdout:898: got (22) Invalid argument 2024-09-18T11:30:37.351 INFO:tasks.rados.rados.0.smithi119.stdout:895: finishing set_chunk to oid 46 2024-09-18T11:30:37.351 INFO:tasks.rados.rados.0.smithi119.stdout:900: finishing write tid 1 to smithi11939506-35 2024-09-18T11:30:37.351 INFO:tasks.rados.rados.0.smithi119.stdout:900: finishing write tid 2 to smithi11939506-35 2024-09-18T11:30:37.351 INFO:tasks.rados.rados.0.smithi119.stdout:895: done (15 left) 2024-09-18T11:30:37.351 INFO:tasks.rados.rados.0.smithi119.stdout:898: done (14 left) 2024-09-18T11:30:37.351 INFO:tasks.rados.rados.0.smithi119.stdout:911: read oid 40 snap 13 2024-09-18T11:30:37.351 INFO:tasks.rados.rados.0.smithi119.stdout:911: expect (ObjNum 86 snap 5 seq_num 86) 2024-09-18T11:30:37.351 INFO:tasks.rados.rados.0.smithi119.stdout:912: write (excl) oid 14 current snap is 15 2024-09-18T11:30:37.351 INFO:tasks.rados.rados.0.smithi119.stdout:912: seq_num 133 ranges {635331=786118,1916458=318873} 2024-09-18T11:30:37.352 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:37.353 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:37.358 INFO:tasks.rados.rados.0.smithi119.stdout:912: writing smithi11939506-14 from 635331 to 1421449 tid 1 2024-09-18T11:30:37.361 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:37.359+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:37.364 INFO:tasks.rados.rados.0.smithi119.stdout:912: writing smithi11939506-14 from 1916458 to 2235331 tid 2 2024-09-18T11:30:37.364 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:37.364 INFO:tasks.rados.rados.0.smithi119.stdout:899: expect (ObjNum 99 snap 6 seq_num 99) 2024-09-18T11:30:37.397 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:37.397 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:37.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:37.397 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:37.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.397 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1045 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'23, mlcod=30'20)", 2024-09-18T11:30:37.397 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.167583+0000", 2024-09-18T11:30:37.397 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.227089298999999, 2024-09-18T11:30:37.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031033380000000002, 2024-09-18T11:30:37.397 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.398 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.398 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167583+0000", 2024-09-18T11:30:37.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167583+0000", 2024-09-18T11:30:37.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167582+0000", 2024-09-18T11:30:37.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:37.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167934+0000", 2024-09-18T11:30:37.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000352262 2024-09-18T11:30:37.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167952+0000", 2024-09-18T11:30:37.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7859999999999998e-05 2024-09-18T11:30:37.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167960+0000", 2024-09-18T11:30:37.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9729999999999997e-06 2024-09-18T11:30:37.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.168008+0000", 2024-09-18T11:30:37.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7531000000000003e-05 2024-09-18T11:30:37.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.168032+0000", 2024-09-18T11:30:37.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4443e-05 2024-09-18T11:30:37.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.170646+0000", 2024-09-18T11:30:37.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026136739999999999 2024-09-18T11:30:37.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.170687+0000", 2024-09-18T11:30:37.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0859000000000003e-05 2024-09-18T11:30:37.402 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.402 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.402 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.402 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1046 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'24, mlcod=30'21)", 2024-09-18T11:30:37.402 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:30:37.402 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.220889519, 2024-09-18T11:30:37.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042519100000000002, 2024-09-18T11:30:37.403 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.403 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.403 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:30:37.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:30:37.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173782+0000", 2024-09-18T11:30:37.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:37.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174207+0000", 2024-09-18T11:30:37.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00042479799999999999 2024-09-18T11:30:37.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174224+0000", 2024-09-18T11:30:37.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7513999999999999e-05 2024-09-18T11:30:37.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174232+0000", 2024-09-18T11:30:37.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0840000000000003e-06 2024-09-18T11:30:37.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174297+0000", 2024-09-18T11:30:37.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5215000000000005e-05 2024-09-18T11:30:37.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174336+0000", 2024-09-18T11:30:37.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9403000000000002e-05 2024-09-18T11:30:37.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.177990+0000", 2024-09-18T11:30:37.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036533709999999999 2024-09-18T11:30:37.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.178035+0000", 2024-09-18T11:30:37.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5495999999999999e-05 2024-09-18T11:30:37.406 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.406 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.406 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.407 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1047 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'25, mlcod=35'23)", 2024-09-18T11:30:37.407 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:30:37.407 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.214917321000001, 2024-09-18T11:30:37.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038129129999999998, 2024-09-18T11:30:37.407 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.407 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.407 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:30:37.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:30:37.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179754+0000", 2024-09-18T11:30:37.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:37.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180088+0000", 2024-09-18T11:30:37.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00033395000000000001 2024-09-18T11:30:37.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180105+0000", 2024-09-18T11:30:37.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7039000000000001e-05 2024-09-18T11:30:37.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180112+0000", 2024-09-18T11:30:37.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3710000000000002e-06 2024-09-18T11:30:37.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180176+0000", 2024-09-18T11:30:37.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3178999999999999e-05 2024-09-18T11:30:37.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180197+0000", 2024-09-18T11:30:37.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1687000000000001e-05 2024-09-18T11:30:37.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183542+0000", 2024-09-18T11:30:37.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003344877 2024-09-18T11:30:37.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183568+0000", 2024-09-18T11:30:37.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6146e-05 2024-09-18T11:30:37.411 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.411 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.411 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.411 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1048 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'26, mlcod=35'23)", 2024-09-18T11:30:37.411 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.180697+0000", 2024-09-18T11:30:37.411 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.213975195, 2024-09-18T11:30:37.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031436290000000002, 2024-09-18T11:30:37.411 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.411 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.411 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180697+0000", 2024-09-18T11:30:37.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180697+0000", 2024-09-18T11:30:37.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180697+0000", 2024-09-18T11:30:37.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:37.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180704+0000", 2024-09-18T11:30:37.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8360000000000003e-06 2024-09-18T11:30:37.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180716+0000", 2024-09-18T11:30:37.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2308e-05 2024-09-18T11:30:37.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180724+0000", 2024-09-18T11:30:37.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6499999999999996e-06 2024-09-18T11:30:37.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180884+0000", 2024-09-18T11:30:37.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016041699999999999 2024-09-18T11:30:37.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180905+0000", 2024-09-18T11:30:37.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1103000000000001e-05 2024-09-18T11:30:37.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183819+0000", 2024-09-18T11:30:37.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029141800000000002 2024-09-18T11:30:37.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183841+0000", 2024-09-18T11:30:37.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1869e-05 2024-09-18T11:30:37.415 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.415 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.415 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.416 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:30:37.416 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:30:37.416 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.411298651999999, 2024-09-18T11:30:37.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:30:37.416 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.416 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.416 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:30:37.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:30:37.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:30:37.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:37.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:30:37.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:30:37.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:30:37.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:30:37.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:30:37.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:30:37.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:30:37.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:30:37.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:30:37.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:30:37.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:30:37.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:30:37.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:30:37.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:30:37.420 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.420 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.420 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.420 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1144 4.4 e39/22 4:2d9c3f7f:::smithi11939506-5:head v 39'30, mlcod=35'26)", 2024-09-18T11:30:37.420 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.204195+0000", 2024-09-18T11:30:37.420 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.190477903000001, 2024-09-18T11:30:37.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002568945, 2024-09-18T11:30:37.420 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.421 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.421 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204195+0000", 2024-09-18T11:30:37.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204195+0000", 2024-09-18T11:30:37.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204194+0000", 2024-09-18T11:30:37.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:37.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204431+0000", 2024-09-18T11:30:37.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237013 2024-09-18T11:30:37.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204446+0000", 2024-09-18T11:30:37.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4238e-05 2024-09-18T11:30:37.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204453+0000", 2024-09-18T11:30:37.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.401e-06 2024-09-18T11:30:37.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204509+0000", 2024-09-18T11:30:37.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5754000000000002e-05 2024-09-18T11:30:37.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204535+0000", 2024-09-18T11:30:37.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5936999999999999e-05 2024-09-18T11:30:37.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.206686+0000", 2024-09-18T11:30:37.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0021507829999999999 2024-09-18T11:30:37.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.206764+0000", 2024-09-18T11:30:37.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8135999999999999e-05 2024-09-18T11:30:37.424 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.424 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.424 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.425 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1145 4.4 e39/22 4:2d9c3f7f:::smithi11939506-5:head v 39'31, mlcod=38'28)", 2024-09-18T11:30:37.425 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.211000+0000", 2024-09-18T11:30:37.425 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.183672615999999, 2024-09-18T11:30:37.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0021574469999999998, 2024-09-18T11:30:37.425 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.425 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.425 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.211000+0000", 2024-09-18T11:30:37.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.211000+0000", 2024-09-18T11:30:37.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.210998+0000", 2024-09-18T11:30:37.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:37.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.211183+0000", 2024-09-18T11:30:37.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018480699999999999 2024-09-18T11:30:37.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.211203+0000", 2024-09-18T11:30:37.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9652000000000001e-05 2024-09-18T11:30:37.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.211210+0000", 2024-09-18T11:30:37.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8390000000000007e-06 2024-09-18T11:30:37.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.211270+0000", 2024-09-18T11:30:37.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9076000000000002e-05 2024-09-18T11:30:37.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.211294+0000", 2024-09-18T11:30:37.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4107000000000001e-05 2024-09-18T11:30:37.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.213120+0000", 2024-09-18T11:30:37.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018266000000000001 2024-09-18T11:30:37.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.213158+0000", 2024-09-18T11:30:37.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7249999999999997e-05 2024-09-18T11:30:37.429 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.429 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.429 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.429 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1146 4.4 e39/22 4:2d9c3f7f:::smithi11939506-5:head v 39'32, mlcod=39'30)", 2024-09-18T11:30:37.429 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.217314+0000", 2024-09-18T11:30:37.429 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.177358624, 2024-09-18T11:30:37.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020364070000000001, 2024-09-18T11:30:37.430 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.430 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.430 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.217314+0000", 2024-09-18T11:30:37.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.217314+0000", 2024-09-18T11:30:37.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.217313+0000", 2024-09-18T11:30:37.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:37.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.217530+0000", 2024-09-18T11:30:37.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021718599999999999 2024-09-18T11:30:37.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.217540+0000", 2024-09-18T11:30:37.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9010000000000006e-06 2024-09-18T11:30:37.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.217546+0000", 2024-09-18T11:30:37.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0179999999999998e-06 2024-09-18T11:30:37.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.217594+0000", 2024-09-18T11:30:37.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7341e-05 2024-09-18T11:30:37.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.217618+0000", 2024-09-18T11:30:37.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:30:37.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.219317+0000", 2024-09-18T11:30:37.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001698864 2024-09-18T11:30:37.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.219350+0000", 2024-09-18T11:30:37.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3194000000000001e-05 2024-09-18T11:30:37.433 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.434 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.434 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.434 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1378 4.d e44/22 4:b81644f2:::smithi11939506-34:head v 44'49, mlcod=42'46)", 2024-09-18T11:30:37.434 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:25.202401+0000", 2024-09-18T11:30:37.434 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.192271336999999, 2024-09-18T11:30:37.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028449030000000002, 2024-09-18T11:30:37.434 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.434 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.434 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.202401+0000", 2024-09-18T11:30:37.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.202401+0000", 2024-09-18T11:30:37.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.202401+0000", 2024-09-18T11:30:37.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:37.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.203222+0000", 2024-09-18T11:30:37.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00082101099999999996 2024-09-18T11:30:37.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.203236+0000", 2024-09-18T11:30:37.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3451999999999999e-05 2024-09-18T11:30:37.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.203244+0000", 2024-09-18T11:30:37.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6400000000000003e-06 2024-09-18T11:30:37.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.203285+0000", 2024-09-18T11:30:37.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1254000000000002e-05 2024-09-18T11:30:37.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.203302+0000", 2024-09-18T11:30:37.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6769000000000002e-05 2024-09-18T11:30:37.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.205219+0000", 2024-09-18T11:30:37.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019171469999999999 2024-09-18T11:30:37.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.205246+0000", 2024-09-18T11:30:37.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6891999999999999e-05 2024-09-18T11:30:37.438 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.438 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.438 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.438 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1380 4.d e44/22 4:b81644f2:::smithi11939506-34:head v 44'51, mlcod=44'49)", 2024-09-18T11:30:37.439 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:25.215023+0000", 2024-09-18T11:30:37.439 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.179649613, 2024-09-18T11:30:37.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001941852, 2024-09-18T11:30:37.439 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.439 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.439 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.215023+0000", 2024-09-18T11:30:37.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.215023+0000", 2024-09-18T11:30:37.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.215022+0000", 2024-09-18T11:30:37.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:37.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.215656+0000", 2024-09-18T11:30:37.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00063338800000000003 2024-09-18T11:30:37.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.215665+0000", 2024-09-18T11:30:37.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5259999999999999e-06 2024-09-18T11:30:37.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.215669+0000", 2024-09-18T11:30:37.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0289999999999999e-06 2024-09-18T11:30:37.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.215703+0000", 2024-09-18T11:30:37.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3878000000000003e-05 2024-09-18T11:30:37.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.215715+0000", 2024-09-18T11:30:37.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1909000000000001e-05 2024-09-18T11:30:37.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.216942+0000", 2024-09-18T11:30:37.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012271910000000001 2024-09-18T11:30:37.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.216965+0000", 2024-09-18T11:30:37.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2507e-05 2024-09-18T11:30:37.443 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.443 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.443 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.443 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:30:37.443 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:30:37.443 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1764676989999998, 2024-09-18T11:30:37.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:30:37.443 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.443 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.443 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:30:37.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:30:37.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:30:37.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:30:37.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:30:37.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:30:37.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:30:37.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:30:37.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:30:37.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:30:37.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:30:37.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:30:37.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:30:37.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:30:37.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:30:37.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:30:37.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:30:37.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:30:37.447 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.447 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.447 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.447 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1422 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'64, mlcod=47'61)", 2024-09-18T11:30:37.447 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.224175+0000", 2024-09-18T11:30:37.448 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1704974539999995, 2024-09-18T11:30:37.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001695776, 2024-09-18T11:30:37.448 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.448 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.448 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.224175+0000", 2024-09-18T11:30:37.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.224175+0000", 2024-09-18T11:30:37.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.224175+0000", 2024-09-18T11:30:37.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:37.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.224679+0000", 2024-09-18T11:30:37.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050417699999999997 2024-09-18T11:30:37.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.224689+0000", 2024-09-18T11:30:37.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0319000000000001e-05 2024-09-18T11:30:37.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.224693+0000", 2024-09-18T11:30:37.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4399999999999998e-06 2024-09-18T11:30:37.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.224720+0000", 2024-09-18T11:30:37.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6330999999999999e-05 2024-09-18T11:30:37.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.224734+0000", 2024-09-18T11:30:37.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4431e-05 2024-09-18T11:30:37.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.225848+0000", 2024-09-18T11:30:37.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0011133810000000001 2024-09-18T11:30:37.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.225871+0000", 2024-09-18T11:30:37.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.336e-05 2024-09-18T11:30:37.452 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.452 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.452 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.452 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1423 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'65, mlcod=48'63)", 2024-09-18T11:30:37.452 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.230841+0000", 2024-09-18T11:30:37.452 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1638321299999994, 2024-09-18T11:30:37.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002175808, 2024-09-18T11:30:37.452 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.452 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.452 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.230841+0000", 2024-09-18T11:30:37.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.230841+0000", 2024-09-18T11:30:37.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.230840+0000", 2024-09-18T11:30:37.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:37.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.231438+0000", 2024-09-18T11:30:37.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059785999999999997 2024-09-18T11:30:37.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.231447+0000", 2024-09-18T11:30:37.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3999999999999998e-06 2024-09-18T11:30:37.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.231451+0000", 2024-09-18T11:30:37.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0849999999999999e-06 2024-09-18T11:30:37.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.231490+0000", 2024-09-18T11:30:37.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8306999999999998e-05 2024-09-18T11:30:37.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.231510+0000", 2024-09-18T11:30:37.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0296999999999999e-05 2024-09-18T11:30:37.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.232984+0000", 2024-09-18T11:30:37.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014746080000000001 2024-09-18T11:30:37.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.233016+0000", 2024-09-18T11:30:37.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1919999999999999e-05 2024-09-18T11:30:37.456 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.456 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.456 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.457 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:244 4.d e49/22 4:b5a463bf:::smithi11939506-20:10 v 49'68, mlcod=48'65)", 2024-09-18T11:30:37.457 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:30.220263+0000", 2024-09-18T11:30:37.457 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.1744101090000001, 2024-09-18T11:30:37.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032524960000000001, 2024-09-18T11:30:37.457 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.457 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.457 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220263+0000", 2024-09-18T11:30:37.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220263+0000", 2024-09-18T11:30:37.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220262+0000", 2024-09-18T11:30:37.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:30:37.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220266+0000", 2024-09-18T11:30:37.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.895e-06 2024-09-18T11:30:37.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220278+0000", 2024-09-18T11:30:37.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1962e-05 2024-09-18T11:30:37.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220284+0000", 2024-09-18T11:30:37.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1299999999999998e-06 2024-09-18T11:30:37.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220334+0000", 2024-09-18T11:30:37.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9789000000000001e-05 2024-09-18T11:30:37.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220356+0000", 2024-09-18T11:30:37.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1897000000000001e-05 2024-09-18T11:30:37.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.223464+0000", 2024-09-18T11:30:37.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031082789999999998 2024-09-18T11:30:37.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.223515+0000", 2024-09-18T11:30:37.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0748999999999998e-05 2024-09-18T11:30:37.460 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.461 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.461 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.461 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1511 4.4 e51/22 4:2c93196f:::smithi11939506-28:head v 51'50, mlcod=51'47)", 2024-09-18T11:30:37.461 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:32.776917+0000", 2024-09-18T11:30:37.461 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.6177561599999999, 2024-09-18T11:30:37.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024761599999999998, 2024-09-18T11:30:37.461 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.461 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.461 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.776917+0000", 2024-09-18T11:30:37.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.776917+0000", 2024-09-18T11:30:37.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.776915+0000", 2024-09-18T11:30:37.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:37.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.777297+0000", 2024-09-18T11:30:37.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038211499999999998 2024-09-18T11:30:37.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.777317+0000", 2024-09-18T11:30:37.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0043000000000001e-05 2024-09-18T11:30:37.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.777325+0000", 2024-09-18T11:30:37.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9500000000000001e-06 2024-09-18T11:30:37.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.777380+0000", 2024-09-18T11:30:37.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4471000000000001e-05 2024-09-18T11:30:37.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.777403+0000", 2024-09-18T11:30:37.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3703999999999999e-05 2024-09-18T11:30:37.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.779349+0000", 2024-09-18T11:30:37.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001945312 2024-09-18T11:30:37.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.779393+0000", 2024-09-18T11:30:37.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3915000000000002e-05 2024-09-18T11:30:37.465 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.465 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.465 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.465 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1512 4.4 e51/22 4:2c93196f:::smithi11939506-28:head v 51'51, mlcod=51'48)", 2024-09-18T11:30:37.465 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:32.782201+0000", 2024-09-18T11:30:37.465 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.6124717449999997, 2024-09-18T11:30:37.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00363554, 2024-09-18T11:30:37.465 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.466 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.466 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782201+0000", 2024-09-18T11:30:37.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782201+0000", 2024-09-18T11:30:37.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782200+0000", 2024-09-18T11:30:37.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:37.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782564+0000", 2024-09-18T11:30:37.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000364612 2024-09-18T11:30:37.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782585+0000", 2024-09-18T11:30:37.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0418999999999999e-05 2024-09-18T11:30:37.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782594+0000", 2024-09-18T11:30:37.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9570000000000008e-06 2024-09-18T11:30:37.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782663+0000", 2024-09-18T11:30:37.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9330000000000002e-05 2024-09-18T11:30:37.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782688+0000", 2024-09-18T11:30:37.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5420000000000001e-05 2024-09-18T11:30:37.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.785790+0000", 2024-09-18T11:30:37.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031016160000000002 2024-09-18T11:30:37.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.785836+0000", 2024-09-18T11:30:37.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6360999999999999e-05 2024-09-18T11:30:37.469 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.469 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.469 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.470 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1513 4.4 e51/22 4:2c93196f:::smithi11939506-28:head v 51'52, mlcod=51'48)", 2024-09-18T11:30:37.470 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:32.783397+0000", 2024-09-18T11:30:37.470 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.6112752070000003, 2024-09-18T11:30:37.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003017198, 2024-09-18T11:30:37.470 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.470 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.470 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783397+0000", 2024-09-18T11:30:37.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783397+0000", 2024-09-18T11:30:37.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783397+0000", 2024-09-18T11:30:37.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:37.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783404+0000", 2024-09-18T11:30:37.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.075e-06 2024-09-18T11:30:37.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783417+0000", 2024-09-18T11:30:37.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3234000000000001e-05 2024-09-18T11:30:37.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783422+0000", 2024-09-18T11:30:37.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5260000000000001e-06 2024-09-18T11:30:37.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783605+0000", 2024-09-18T11:30:37.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018248099999999999 2024-09-18T11:30:37.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783635+0000", 2024-09-18T11:30:37.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0309000000000001e-05 2024-09-18T11:30:37.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.786378+0000", 2024-09-18T11:30:37.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027433119999999999 2024-09-18T11:30:37.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.786415+0000", 2024-09-18T11:30:37.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6192999999999997e-05 2024-09-18T11:30:37.473 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.473 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.473 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.474 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:30:37.474 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:30:37.474 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.0025614140000001, 2024-09-18T11:30:37.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:30:37.474 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.474 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.474 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:30:37.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:30:37.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:30:37.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:37.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:30:37.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:30:37.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:30:37.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:30:37.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:30:37.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:30:37.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:30:37.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:30:37.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:30:37.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:30:37.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:30:37.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:30:37.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:30:37.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:30:37.478 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.478 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.478 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.478 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1531 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'73, mlcod=51'69)", 2024-09-18T11:30:37.478 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.396013+0000", 2024-09-18T11:30:37.478 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.9986599539999998, 2024-09-18T11:30:37.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015166929999999999, 2024-09-18T11:30:37.478 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.478 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.478 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.396013+0000", 2024-09-18T11:30:37.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.396013+0000", 2024-09-18T11:30:37.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.396012+0000", 2024-09-18T11:30:37.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:37.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.396216+0000", 2024-09-18T11:30:37.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020350500000000001 2024-09-18T11:30:37.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.396223+0000", 2024-09-18T11:30:37.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7759999999999997e-06 2024-09-18T11:30:37.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.396227+0000", 2024-09-18T11:30:37.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6669999999999998e-06 2024-09-18T11:30:37.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.396267+0000", 2024-09-18T11:30:37.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0227e-05 2024-09-18T11:30:37.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.396320+0000", 2024-09-18T11:30:37.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2682000000000001e-05 2024-09-18T11:30:37.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397496+0000", 2024-09-18T11:30:37.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0011764099999999999 2024-09-18T11:30:37.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397529+0000", 2024-09-18T11:30:37.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3052000000000001e-05 2024-09-18T11:30:37.482 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.482 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.482 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.482 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1532 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'74, mlcod=51'69)", 2024-09-18T11:30:37.482 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.397573+0000", 2024-09-18T11:30:37.482 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.9970993259999998, 2024-09-18T11:30:37.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032675960000000002, 2024-09-18T11:30:37.483 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.483 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.483 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397573+0000", 2024-09-18T11:30:37.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397573+0000", 2024-09-18T11:30:37.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397573+0000", 2024-09-18T11:30:37.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:37.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397577+0000", 2024-09-18T11:30:37.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5499999999999996e-06 2024-09-18T11:30:37.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397585+0000", 2024-09-18T11:30:37.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1470000000000001e-06 2024-09-18T11:30:37.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397591+0000", 2024-09-18T11:30:37.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5369999999999997e-06 2024-09-18T11:30:37.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397635+0000", 2024-09-18T11:30:37.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4308000000000003e-05 2024-09-18T11:30:37.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397656+0000", 2024-09-18T11:30:37.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1138e-05 2024-09-18T11:30:37.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.400817+0000", 2024-09-18T11:30:37.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031607499999999999 2024-09-18T11:30:37.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.400841+0000", 2024-09-18T11:30:37.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3717e-05 2024-09-18T11:30:37.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.486 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.486 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.487 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:37.488 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:30:37.488 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:37.486+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:37.529 INFO:tasks.rados.rados.0.smithi119.stdout:900: finishing write tid 3 to smithi11939506-35 2024-09-18T11:30:37.529 INFO:tasks.rados.rados.0.smithi119.stdout:900: finishing write tid 4 to smithi11939506-35 2024-09-18T11:30:37.529 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 45 (ObjNum 130 snap 15 seq_num 130) dirty exists 2024-09-18T11:30:37.529 INFO:tasks.rados.rados.0.smithi119.stdout:900: left oid 35 (ObjNum 130 snap 15 seq_num 130) 2024-09-18T11:30:37.529 INFO:tasks.rados.rados.0.smithi119.stdout:899: done (15 left) 2024-09-18T11:30:37.529 INFO:tasks.rados.rados.0.smithi119.stdout:900: done (14 left) 2024-09-18T11:30:37.529 INFO:tasks.rados.rados.0.smithi119.stdout:913: snap_create 2024-09-18T11:30:37.529 INFO:tasks.rados.rados.0.smithi119.stdout:896: expect (ObjNum 117 snap 11 seq_num 117) 2024-09-18T11:30:37.588 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:37.586+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:37.628 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:37.628 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:37.628 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:37.628 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:37.637 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:30:37.689 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:37.688+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:37.722 INFO:tasks.rados.rados.0.smithi119.stdout:901: finishing set_chunk to oid 50 2024-09-18T11:30:37.722 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 50 set_chunk returned error code -95 offset: 2660864 length: 582144 2024-09-18T11:30:37.722 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 0 (ObjNum 93 snap 5 seq_num 93) dirty exists 2024-09-18T11:30:37.722 INFO:tasks.rados.rados.0.smithi119.stdout:903: finishing write tid 1 to smithi11939506-2 2024-09-18T11:30:37.722 INFO:tasks.rados.rados.0.smithi119.stdout:897: expect (ObjNum 116 snap 11 seq_num 116) 2024-09-18T11:30:37.734 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:37.734 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:37.735 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:37.735 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:37.735 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:37.743 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:30:37.827 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:37.826+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:37.883 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:37.883 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:37.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:37.883 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:37.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.883 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:900 4.0 4:0aec5e1c:::smithi11939506-50:head [read 0~2974806,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:37.883 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.133226+0000", 2024-09-18T11:30:37.883 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.74724694, 2024-09-18T11:30:37.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059578069999999999, 2024-09-18T11:30:37.883 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.884 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:37.884 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:37.884 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:37.884 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:37.884 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 900 2024-09-18T11:30:37.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.884 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133226+0000", 2024-09-18T11:30:37.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133226+0000", 2024-09-18T11:30:37.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133225+0000", 2024-09-18T11:30:37.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:37.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133234+0000", 2024-09-18T11:30:37.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5799999999999992e-06 2024-09-18T11:30:37.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133242+0000", 2024-09-18T11:30:37.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4039999999999995e-06 2024-09-18T11:30:37.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133256+0000", 2024-09-18T11:30:37.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3278000000000001e-05 2024-09-18T11:30:37.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.887 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.887 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133293+0000", 2024-09-18T11:30:37.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7799000000000002e-05 2024-09-18T11:30:37.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.887 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:37.887 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133298+0000", 2024-09-18T11:30:37.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9110000000000001e-06 2024-09-18T11:30:37.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.137554+0000", 2024-09-18T11:30:37.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042556099999999999 2024-09-18T11:30:37.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.137713+0000", 2024-09-18T11:30:37.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015937199999999999 2024-09-18T11:30:37.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139184+0000", 2024-09-18T11:30:37.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001470694 2024-09-18T11:30:37.889 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.889 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.889 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.889 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:901 4.0 4:0aec5e1c:::smithi11939506-50:head [sparse-read 0~1781150] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:37.889 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:37.889 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.74720773, 2024-09-18T11:30:37.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063828720000000004, 2024-09-18T11:30:37.889 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.889 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:37.889 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:37.890 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:37.890 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:37.890 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 901 2024-09-18T11:30:37.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.890 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:37.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:37.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:37.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:37.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133268+0000", 2024-09-18T11:30:37.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9610000000000001e-06 2024-09-18T11:30:37.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133280+0000", 2024-09-18T11:30:37.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2481e-05 2024-09-18T11:30:37.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133285+0000", 2024-09-18T11:30:37.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5469999999999998e-06 2024-09-18T11:30:37.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133317+0000", 2024-09-18T11:30:37.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1838999999999998e-05 2024-09-18T11:30:37.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:37.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133320+0000", 2024-09-18T11:30:37.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3009999999999998e-06 2024-09-18T11:30:37.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139225+0000", 2024-09-18T11:30:37.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059047919999999999 2024-09-18T11:30:37.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139330+0000", 2024-09-18T11:30:37.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104924 2024-09-18T11:30:37.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139648+0000", 2024-09-18T11:30:37.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00031854799999999999 2024-09-18T11:30:37.894 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.894 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.894 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.894 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:37.894 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:30:37.894 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.71105369, 2024-09-18T11:30:37.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:30:37.894 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.894 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:37.894 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:37.894 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:37.895 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:37.895 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:30:37.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.895 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:30:37.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:30:37.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:30:37.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:37.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:30:37.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:30:37.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:30:37.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:30:37.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:30:37.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:30:37.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:30:37.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:30:37.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:30:37.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:30:37.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:30:37.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:30:37.899 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.899 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.899 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.900 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1000 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:37.900 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164518+0000", 2024-09-18T11:30:37.900 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.715955584, 2024-09-18T11:30:37.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005734093, 2024-09-18T11:30:37.900 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.900 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:37.900 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:37.900 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:37.900 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:37.900 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1000 2024-09-18T11:30:37.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.900 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164518+0000", 2024-09-18T11:30:37.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164518+0000", 2024-09-18T11:30:37.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164515+0000", 2024-09-18T11:30:37.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:37.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164525+0000", 2024-09-18T11:30:37.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5480000000000007e-06 2024-09-18T11:30:37.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164542+0000", 2024-09-18T11:30:37.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7643e-05 2024-09-18T11:30:37.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164553+0000", 2024-09-18T11:30:37.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0581000000000001e-05 2024-09-18T11:30:37.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164637+0000", 2024-09-18T11:30:37.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3825999999999993e-05 2024-09-18T11:30:37.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:37.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164644+0000", 2024-09-18T11:30:37.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5499999999999997e-06 2024-09-18T11:30:37.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.166446+0000", 2024-09-18T11:30:37.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018018789999999999 2024-09-18T11:30:37.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.166537+0000", 2024-09-18T11:30:37.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0325e-05 2024-09-18T11:30:37.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170252+0000", 2024-09-18T11:30:37.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037150820000000002 2024-09-18T11:30:37.904 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.904 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.905 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.905 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1001 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:37.905 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:30:37.905 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.715902411999998, 2024-09-18T11:30:37.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007461082, 2024-09-18T11:30:37.905 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.905 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:37.905 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:37.905 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:37.905 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:37.905 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1001 2024-09-18T11:30:37.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.906 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:30:37.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:30:37.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164570+0000", 2024-09-18T11:30:37.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:37.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164574+0000", 2024-09-18T11:30:37.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.822e-06 2024-09-18T11:30:37.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164582+0000", 2024-09-18T11:30:37.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3410000000000003e-06 2024-09-18T11:30:37.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164587+0000", 2024-09-18T11:30:37.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769999999999999e-06 2024-09-18T11:30:37.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164702+0000", 2024-09-18T11:30:37.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115705 2024-09-18T11:30:37.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:37.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164709+0000", 2024-09-18T11:30:37.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-09-18T11:30:37.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170296+0000", 2024-09-18T11:30:37.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055876390000000001 2024-09-18T11:30:37.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170418+0000", 2024-09-18T11:30:37.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121549 2024-09-18T11:30:37.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172032+0000", 2024-09-18T11:30:37.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00161407 2024-09-18T11:30:37.910 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.910 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.910 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.910 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:37.910 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:37.910 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.715875696000001, 2024-09-18T11:30:37.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:30:37.910 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.910 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:37.910 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:37.910 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:37.910 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:37.911 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:30:37.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.911 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:37.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:37.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:37.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:37.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:30:37.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:30:37.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:30:37.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:30:37.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:30:37.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:30:37.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:30:37.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:30:37.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:37.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:30:37.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:30:37.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:30:37.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:30:37.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:30:37.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:30:37.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:30:37.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:30:37.915 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.915 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.915 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.915 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:37.915 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:30:37.915 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.715857407000001, 2024-09-18T11:30:37.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:30:37.915 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.915 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:37.915 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:37.916 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:37.916 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:37.916 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:30:37.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.916 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:30:37.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:30:37.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:30:37.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:37.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:30:37.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:30:37.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:30:37.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:30:37.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:30:37.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:30:37.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:30:37.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:30:37.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:37.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:30:37.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:30:37.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:30:37.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:30:37.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:30:37.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:30:37.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:30:37.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:30:37.920 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.920 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.920 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.920 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:37.920 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:30:37.920 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.715796638, 2024-09-18T11:30:37.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:30:37.920 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.921 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:37.921 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:37.921 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:37.921 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:37.921 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:30:37.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.921 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:30:37.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:30:37.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:30:37.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:37.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:30:37.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:30:37.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:30:37.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:30:37.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:30:37.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:30:37.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:30:37.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:30:37.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:37.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:30:37.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:30:37.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:30:37.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:30:37.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:30:37.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:30:37.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:30:37.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:30:37.925 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.925 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.925 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.925 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:37.925 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:30:37.925 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.715774317000001, 2024-09-18T11:30:37.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:30:37.925 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.925 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:37.925 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:37.925 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:37.926 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:37.926 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:30:37.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.926 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:30:37.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:30:37.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:30:37.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:37.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:30:37.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:30:37.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:30:37.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:30:37.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:30:37.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:30:37.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:30:37.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:30:37.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:37.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:30:37.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:30:37.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:30:37.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:30:37.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:30:37.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:30:37.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:30:37.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:30:37.930 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.930 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.930 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.930 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1018 4.0 4:0aec5e1c:::smithi11939506-50:head [write 1622620~730352] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:37.930 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:30:37.930 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.199539761, 2024-09-18T11:30:37.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064405160000000003, 2024-09-18T11:30:37.931 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.931 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.931 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:37.931 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:37.931 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:37.931 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1018 2024-09-18T11:30:37.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.931 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:30:37.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:30:37.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680932+0000", 2024-09-18T11:30:37.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:37.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681334+0000", 2024-09-18T11:30:37.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000401803 2024-09-18T11:30:37.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681350+0000", 2024-09-18T11:30:37.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6218999999999998e-05 2024-09-18T11:30:37.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681374+0000", 2024-09-18T11:30:37.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4088e-05 2024-09-18T11:30:37.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681415+0000", 2024-09-18T11:30:37.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1307000000000003e-05 2024-09-18T11:30:37.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681507+0000", 2024-09-18T11:30:37.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1867999999999995e-05 2024-09-18T11:30:37.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:37.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681671+0000", 2024-09-18T11:30:37.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016406 2024-09-18T11:30:37.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:37.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.683856+0000", 2024-09-18T11:30:37.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002184624 2024-09-18T11:30:37.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:37.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687266+0000", 2024-09-18T11:30:37.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034105490000000001 2024-09-18T11:30:37.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687321+0000", 2024-09-18T11:30:37.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4960999999999998e-05 2024-09-18T11:30:37.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687374+0000", 2024-09-18T11:30:37.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2553000000000003e-05 2024-09-18T11:30:37.936 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.936 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.936 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.936 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1037 4.0 4:019b6e09:::smithi11939506-40:head [write 477332~766101] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:37.936 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.707508+0000", 2024-09-18T11:30:37.936 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.172965035000001, 2024-09-18T11:30:37.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050976160000000001, 2024-09-18T11:30:37.937 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.937 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.937 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:37.937 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:37.937 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:37.937 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1037 2024-09-18T11:30:37.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.937 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.707508+0000", 2024-09-18T11:30:37.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.707508+0000", 2024-09-18T11:30:37.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.707506+0000", 2024-09-18T11:30:37.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:37.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708363+0000", 2024-09-18T11:30:37.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00085727299999999998 2024-09-18T11:30:37.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708388+0000", 2024-09-18T11:30:37.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5031000000000001e-05 2024-09-18T11:30:37.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708396+0000", 2024-09-18T11:30:37.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0870000000000007e-06 2024-09-18T11:30:37.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708452+0000", 2024-09-18T11:30:37.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.533e-05 2024-09-18T11:30:37.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708563+0000", 2024-09-18T11:30:37.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011086600000000001 2024-09-18T11:30:37.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:37.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708761+0000", 2024-09-18T11:30:37.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019803 2024-09-18T11:30:37.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:37.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.711546+0000", 2024-09-18T11:30:37.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002785689 2024-09-18T11:30:37.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:37.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712507+0000", 2024-09-18T11:30:37.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096094300000000002 2024-09-18T11:30:37.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712557+0000", 2024-09-18T11:30:37.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9701999999999999e-05 2024-09-18T11:30:37.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712606+0000", 2024-09-18T11:30:37.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.888e-05 2024-09-18T11:30:37.942 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.942 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.942 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.942 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1041 4.0 4:019b6e09:::smithi11939506-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:37.942 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.719166+0000", 2024-09-18T11:30:37.942 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.161307418, 2024-09-18T11:30:37.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050746640000000004, 2024-09-18T11:30:37.943 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.943 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:37.943 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:37.943 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:37.943 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:37.943 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1041 2024-09-18T11:30:37.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.943 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719166+0000", 2024-09-18T11:30:37.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719166+0000", 2024-09-18T11:30:37.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719165+0000", 2024-09-18T11:30:37.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:37.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719168+0000", 2024-09-18T11:30:37.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6220000000000001e-06 2024-09-18T11:30:37.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719174+0000", 2024-09-18T11:30:37.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2020000000000001e-06 2024-09-18T11:30:37.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.719178+0000", 2024-09-18T11:30:37.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6890000000000002e-06 2024-09-18T11:30:37.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.720928+0000", 2024-09-18T11:30:37.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017504490000000001 2024-09-18T11:30:37.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:37.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.721050+0000", 2024-09-18T11:30:37.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121477 2024-09-18T11:30:37.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.724120+0000", 2024-09-18T11:30:37.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0030705569999999998 2024-09-18T11:30:37.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.724176+0000", 2024-09-18T11:30:37.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5327000000000001e-05 2024-09-18T11:30:37.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.724240+0000", 2024-09-18T11:30:37.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4806000000000003e-05 2024-09-18T11:30:37.947 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.947 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.947 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.948 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1050 4.0 4:0aec5e1c:::smithi11939506-50:head [write 510946~632952] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:37.948 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.185718+0000", 2024-09-18T11:30:37.948 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.694754789000001, 2024-09-18T11:30:37.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055213279999999998, 2024-09-18T11:30:37.948 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.948 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.948 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:37.948 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:37.948 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:37.948 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1050 2024-09-18T11:30:37.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.949 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185718+0000", 2024-09-18T11:30:37.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185718+0000", 2024-09-18T11:30:37.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185717+0000", 2024-09-18T11:30:37.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:37.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185850+0000", 2024-09-18T11:30:37.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000133616 2024-09-18T11:30:37.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185867+0000", 2024-09-18T11:30:37.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7047000000000001e-05 2024-09-18T11:30:37.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185874+0000", 2024-09-18T11:30:37.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4250000000000003e-06 2024-09-18T11:30:37.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185926+0000", 2024-09-18T11:30:37.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1867999999999999e-05 2024-09-18T11:30:37.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185985+0000", 2024-09-18T11:30:37.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9769000000000003e-05 2024-09-18T11:30:37.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:37.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.186079+0000", 2024-09-18T11:30:37.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3109000000000006e-05 2024-09-18T11:30:37.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:37.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.187878+0000", 2024-09-18T11:30:37.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001799557 2024-09-18T11:30:37.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:37.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191165+0000", 2024-09-18T11:30:37.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003286892 2024-09-18T11:30:37.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191196+0000", 2024-09-18T11:30:37.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0607e-05 2024-09-18T11:30:37.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191240+0000", 2024-09-18T11:30:37.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4078e-05 2024-09-18T11:30:37.954 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.954 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.954 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.954 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:37.954 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:30:37.954 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.934649434000001, 2024-09-18T11:30:37.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:30:37.954 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.954 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.954 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:37.954 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:37.954 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:37.955 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:30:37.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.955 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:30:37.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:30:37.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:30:37.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:30:37.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:30:37.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:30:37.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:30:37.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:30:37.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:30:37.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:30:37.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:30:37.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:30:37.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:30:37.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:30:37.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:37.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:30:37.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:30:37.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:37.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:30:37.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:30:37.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:37.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:30:37.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:30:37.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:30:37.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:30:37.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:30:37.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:30:37.960 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.960 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.960 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.960 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1121 4.f e38/22 4:f5d7df41:::smithi11939506-16:head v 38'34, mlcod=30'32)", 2024-09-18T11:30:37.960 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:30:37.960 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.914662175, 2024-09-18T11:30:37.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060953680000000003, 2024-09-18T11:30:37.960 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.961 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.961 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:30:37.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:30:37.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965810+0000", 2024-09-18T11:30:37.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:37.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966142+0000", 2024-09-18T11:30:37.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000331591 2024-09-18T11:30:37.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966154+0000", 2024-09-18T11:30:37.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.253e-05 2024-09-18T11:30:37.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966160+0000", 2024-09-18T11:30:37.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7130000000000002e-06 2024-09-18T11:30:37.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966196+0000", 2024-09-18T11:30:37.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6276999999999998e-05 2024-09-18T11:30:37.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966217+0000", 2024-09-18T11:30:37.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0335000000000001e-05 2024-09-18T11:30:37.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.971872+0000", 2024-09-18T11:30:37.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056549290000000004 2024-09-18T11:30:37.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.971906+0000", 2024-09-18T11:30:37.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4702000000000001e-05 2024-09-18T11:30:37.964 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.964 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.964 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.965 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:29 3.2 3:5fa8fcfa:::ec59520759824f0488c7a3ca135dacb324de2330e16b7f54053de5719c4e96b6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:37.965 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:30:37.965 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.200291518, 2024-09-18T11:30:37.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064126499999999998, 2024-09-18T11:30:37.965 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.965 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.965 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:30:37.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:30:37.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680181+0000", 2024-09-18T11:30:37.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:37.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680654+0000", 2024-09-18T11:30:37.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047334900000000002 2024-09-18T11:30:37.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680666+0000", 2024-09-18T11:30:37.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2232000000000001e-05 2024-09-18T11:30:37.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680672+0000", 2024-09-18T11:30:37.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:30:37.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681222+0000", 2024-09-18T11:30:37.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00054928699999999997 2024-09-18T11:30:37.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681332+0000", 2024-09-18T11:30:37.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110634 2024-09-18T11:30:37.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:37.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681552+0000", 2024-09-18T11:30:37.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021926899999999999 2024-09-18T11:30:37.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:37.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682950+0000", 2024-09-18T11:30:37.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013982669999999999 2024-09-18T11:30:37.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:37.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686503+0000", 2024-09-18T11:30:37.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035536130000000002 2024-09-18T11:30:37.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686553+0000", 2024-09-18T11:30:37.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9801e-05 2024-09-18T11:30:37.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686594+0000", 2024-09-18T11:30:37.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1022000000000001e-05 2024-09-18T11:30:37.970 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.970 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.970 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.970 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:30 3.2 3:49074e56:::7fc21ffee1ba17f2a9197c61ff60ef0b03804c121c8aeb83ea424b7ac77081e9:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:37.970 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.680698+0000", 2024-09-18T11:30:37.970 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.199775200000001, 2024-09-18T11:30:37.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060349130000000003, 2024-09-18T11:30:37.971 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.971 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.971 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680698+0000", 2024-09-18T11:30:37.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680698+0000", 2024-09-18T11:30:37.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680697+0000", 2024-09-18T11:30:37.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:37.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680750+0000", 2024-09-18T11:30:37.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3170000000000001e-05 2024-09-18T11:30:37.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680760+0000", 2024-09-18T11:30:37.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3379999999999994e-06 2024-09-18T11:30:37.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680766+0000", 2024-09-18T11:30:37.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8249999999999999e-06 2024-09-18T11:30:37.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682201+0000", 2024-09-18T11:30:37.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001434868 2024-09-18T11:30:37.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682294+0000", 2024-09-18T11:30:37.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2965000000000002e-05 2024-09-18T11:30:37.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:37.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682445+0000", 2024-09-18T11:30:37.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015103600000000001 2024-09-18T11:30:37.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:37.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.683524+0000", 2024-09-18T11:30:37.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0010789980000000001 2024-09-18T11:30:37.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:37.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686687+0000", 2024-09-18T11:30:37.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003163052 2024-09-18T11:30:37.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686714+0000", 2024-09-18T11:30:37.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6369000000000001e-05 2024-09-18T11:30:37.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686733+0000", 2024-09-18T11:30:37.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9117999999999999e-05 2024-09-18T11:30:37.975 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.975 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.975 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.975 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1199 4.0 4:0aec5e1c:::smithi11939506-50:head [write 763051~794393] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-18T11:30:37.976 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:30:37.976 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 16.683477394000001, 2024-09-18T11:30:37.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067768530000000002, 2024-09-18T11:30:37.976 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.976 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:37.976 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:37.976 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:37.976 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:37.976 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1199 2024-09-18T11:30:37.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.976 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:30:37.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:30:37.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196994+0000", 2024-09-18T11:30:37.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:37.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197355+0000", 2024-09-18T11:30:37.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036035199999999998 2024-09-18T11:30:37.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197366+0000", 2024-09-18T11:30:37.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.17e-05 2024-09-18T11:30:37.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197371+0000", 2024-09-18T11:30:37.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6099999999999999e-06 2024-09-18T11:30:37.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197416+0000", 2024-09-18T11:30:37.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5099000000000003e-05 2024-09-18T11:30:37.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:37.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197420+0000", 2024-09-18T11:30:37.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3189999999999996e-06 2024-09-18T11:30:37.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199392+0000", 2024-09-18T11:30:37.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019720139999999998 2024-09-18T11:30:37.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199452+0000", 2024-09-18T11:30:37.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9929999999999997e-05 2024-09-18T11:30:37.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:37.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199565+0000", 2024-09-18T11:30:37.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112811 2024-09-18T11:30:37.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:37.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.202108+0000", 2024-09-18T11:30:37.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025429060000000002 2024-09-18T11:30:37.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:37.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203714+0000", 2024-09-18T11:30:37.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001606335 2024-09-18T11:30:37.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:37.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203745+0000", 2024-09-18T11:30:37.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0837999999999998e-05 2024-09-18T11:30:37.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203773+0000", 2024-09-18T11:30:37.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7562000000000001e-05 2024-09-18T11:30:37.982 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.982 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.982 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.982 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1660 4.0 4:019b6e09:::smithi11939506-40:17 [read 0~3445402,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:37.982 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349874+0000", 2024-09-18T11:30:37.982 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.53059904400000002, 2024-09-18T11:30:37.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057173650000000003, 2024-09-18T11:30:37.982 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.983 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:37.983 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:37.983 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:37.983 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:37.983 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1660 2024-09-18T11:30:37.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.983 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349874+0000", 2024-09-18T11:30:37.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349874+0000", 2024-09-18T11:30:37.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349873+0000", 2024-09-18T11:30:37.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:37.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349878+0000", 2024-09-18T11:30:37.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0860000000000004e-06 2024-09-18T11:30:37.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349886+0000", 2024-09-18T11:30:37.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4249999999999998e-06 2024-09-18T11:30:37.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349891+0000", 2024-09-18T11:30:37.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8119999999999998e-06 2024-09-18T11:30:37.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.353312+0000", 2024-09-18T11:30:37.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003420799 2024-09-18T11:30:37.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.353437+0000", 2024-09-18T11:30:37.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012563799999999999 2024-09-18T11:30:37.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355591+0000", 2024-09-18T11:30:37.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0021542369999999998 2024-09-18T11:30:37.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.986 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.986 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1661 4.0 4:019b6e09:::smithi11939506-40:17 [read 0~3445402,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:37.987 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:30:37.987 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.53057673100000002, 2024-09-18T11:30:37.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075190880000000002, 2024-09-18T11:30:37.987 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:37.987 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:37.987 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:37.987 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:37.987 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:37.987 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1661 2024-09-18T11:30:37.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.988 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:37.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:37.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:30:37.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:37.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:30:37.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:37.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:37.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:30:37.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:37.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:37.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349899+0000", 2024-09-18T11:30:37.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9749999999999999e-06 2024-09-18T11:30:37.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:37.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349906+0000", 2024-09-18T11:30:37.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9290000000000001e-06 2024-09-18T11:30:37.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:37.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349912+0000", 2024-09-18T11:30:37.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9900000000000002e-06 2024-09-18T11:30:37.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:37.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355645+0000", 2024-09-18T11:30:37.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057329549999999996 2024-09-18T11:30:37.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:37.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355776+0000", 2024-09-18T11:30:37.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130975 2024-09-18T11:30:37.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:37.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:37.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:37.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357416+0000", 2024-09-18T11:30:37.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001639649 2024-09-18T11:30:37.991 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.991 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.991 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.991 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:37.991 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:37.991 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:37.993 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:30:37.993 INFO:tasks.rados.rados.0.smithi119.stdout:902: finishing rollback tid 0 to smithi11939506-36 2024-09-18T11:30:37.994 INFO:tasks.rados.rados.0.smithi119.stdout:902: finishing rollback tid 1 to smithi11939506-36 2024-09-18T11:30:37.994 INFO:tasks.rados.rados.0.smithi119.stdout:902: finishing rollback tid 2 to smithi11939506-36 2024-09-18T11:30:37.994 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 36 v 51 (ObjNum 100 snap 6 seq_num 100) dirty exists 2024-09-18T11:30:37.994 INFO:tasks.rados.rados.0.smithi119.stdout:905: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-18T11:30:37.994 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:37.991+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:38.020 INFO:tasks.rados.rados.0.smithi119.stdout:903: finishing write tid 2 to smithi11939506-2 2024-09-18T11:30:38.020 INFO:tasks.rados.rados.0.smithi119.stdout:903: finishing write tid 3 to smithi11939506-2 2024-09-18T11:30:38.020 INFO:tasks.rados.rados.0.smithi119.stdout:903: finishing write tid 4 to smithi11939506-2 2024-09-18T11:30:38.020 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 85 (ObjNum 131 snap 15 seq_num 131) dirty exists 2024-09-18T11:30:38.020 INFO:tasks.rados.rados.0.smithi119.stdout:903: left oid 2 (ObjNum 131 snap 15 seq_num 131) 2024-09-18T11:30:38.020 INFO:tasks.rados.rados.0.smithi119.stdout:907: expect (ObjNum 110 snap 8 seq_num 110) 2024-09-18T11:30:38.093 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:38.092+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:38.100 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:38.100 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:38.100 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:38.100 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:38.109 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:30:38.194 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:38.192+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:38.197 INFO:tasks.rados.rados.0.smithi119.stdout:906: expect (ObjNum 118 snap 11 seq_num 118) 2024-09-18T11:30:38.208 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:38.209 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:38.209 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:38.209 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:38.209 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:38.216 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:30:38.270 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T11:30:38.300 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:38.298+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:38.356 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:38.356 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:38.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:38.357 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:38.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.357 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1026 4.c 4:323f30dd:::smithi11939506-12:head [read 0~3176490,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:38.357 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.686142+0000", 2024-09-18T11:30:38.357 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.667881196, 2024-09-18T11:30:38.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048199560000000002, 2024-09-18T11:30:38.357 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.357 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:38.357 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.357 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:38.357 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:38.357 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1026 2024-09-18T11:30:38.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.358 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686142+0000", 2024-09-18T11:30:38.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686142+0000", 2024-09-18T11:30:38.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686141+0000", 2024-09-18T11:30:38.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:38.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686145+0000", 2024-09-18T11:30:38.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0999999999999997e-06 2024-09-18T11:30:38.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686153+0000", 2024-09-18T11:30:38.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0250000000000002e-06 2024-09-18T11:30:38.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686158+0000", 2024-09-18T11:30:38.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1389999999999997e-06 2024-09-18T11:30:38.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689573+0000", 2024-09-18T11:30:38.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034151369999999999 2024-09-18T11:30:38.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689630+0000", 2024-09-18T11:30:38.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6830000000000003e-05 2024-09-18T11:30:38.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690962+0000", 2024-09-18T11:30:38.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013323090000000001 2024-09-18T11:30:38.361 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.361 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.361 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.361 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1027 4.c 4:323f30dd:::smithi11939506-12:head [sparse-read 0~2047579] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:38.361 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.686162+0000", 2024-09-18T11:30:38.362 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.667861106, 2024-09-18T11:30:38.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050932319999999996, 2024-09-18T11:30:38.362 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.362 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:38.362 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.362 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:38.362 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:38.362 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1027 2024-09-18T11:30:38.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.362 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686162+0000", 2024-09-18T11:30:38.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686162+0000", 2024-09-18T11:30:38.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686162+0000", 2024-09-18T11:30:38.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:38.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686164+0000", 2024-09-18T11:30:38.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3489999999999999e-06 2024-09-18T11:30:38.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686169+0000", 2024-09-18T11:30:38.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1039999999999998e-06 2024-09-18T11:30:38.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.686172+0000", 2024-09-18T11:30:38.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3680000000000002e-06 2024-09-18T11:30:38.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690990+0000", 2024-09-18T11:30:38.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048174460000000004 2024-09-18T11:30:38.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.691056+0000", 2024-09-18T11:30:38.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6192000000000006e-05 2024-09-18T11:30:38.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.691255+0000", 2024-09-18T11:30:38.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000199142 2024-09-18T11:30:38.366 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.366 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.366 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.366 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1029 3.3 3:df268f6c:::8da2985a35cccabf5edc5a8900d2327bcfe6ed768fa38d466fa9a390ce713320:head [write 0~356864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:38.366 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.689033+0000", 2024-09-18T11:30:38.366 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.664989941000002, 2024-09-18T11:30:38.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053345750000000003, 2024-09-18T11:30:38.366 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.366 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.367 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.367 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:38.367 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:38.367 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1029 2024-09-18T11:30:38.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.367 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689033+0000", 2024-09-18T11:30:38.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689033+0000", 2024-09-18T11:30:38.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689031+0000", 2024-09-18T11:30:38.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:38.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690898+0000", 2024-09-18T11:30:38.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001867032 2024-09-18T11:30:38.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690914+0000", 2024-09-18T11:30:38.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5806000000000001e-05 2024-09-18T11:30:38.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690921+0000", 2024-09-18T11:30:38.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8859999999999998e-06 2024-09-18T11:30:38.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690977+0000", 2024-09-18T11:30:38.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5612999999999998e-05 2024-09-18T11:30:38.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.691142+0000", 2024-09-18T11:30:38.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000165112 2024-09-18T11:30:38.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:38.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.691351+0000", 2024-09-18T11:30:38.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020942700000000001 2024-09-18T11:30:38.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:38.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.692616+0000", 2024-09-18T11:30:38.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012648200000000001 2024-09-18T11:30:38.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:38.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.694303+0000", 2024-09-18T11:30:38.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016875320000000001 2024-09-18T11:30:38.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.694335+0000", 2024-09-18T11:30:38.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1971000000000003e-05 2024-09-18T11:30:38.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.694368+0000", 2024-09-18T11:30:38.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2403000000000001e-05 2024-09-18T11:30:38.372 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.372 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.372 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.372 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1133 4.1 4:8b9a985d:::smithi11939506-23:a [read 0~3327753,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:38.372 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.178999+0000", 2024-09-18T11:30:38.372 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.175023969000002, 2024-09-18T11:30:38.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059312619999999996, 2024-09-18T11:30:38.373 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.373 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:38.373 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.373 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:38.373 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:38.373 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1133 2024-09-18T11:30:38.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.373 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178999+0000", 2024-09-18T11:30:38.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178999+0000", 2024-09-18T11:30:38.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178997+0000", 2024-09-18T11:30:38.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:38.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179012+0000", 2024-09-18T11:30:38.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5166e-05 2024-09-18T11:30:38.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179043+0000", 2024-09-18T11:30:38.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0892999999999997e-05 2024-09-18T11:30:38.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179057+0000", 2024-09-18T11:30:38.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4212e-05 2024-09-18T11:30:38.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179217+0000", 2024-09-18T11:30:38.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016031699999999999 2024-09-18T11:30:38.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:38.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179235+0000", 2024-09-18T11:30:38.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7903000000000001e-05 2024-09-18T11:30:38.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.181282+0000", 2024-09-18T11:30:38.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002046614 2024-09-18T11:30:38.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.181350+0000", 2024-09-18T11:30:38.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8190999999999998e-05 2024-09-18T11:30:38.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184930+0000", 2024-09-18T11:30:38.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035805960000000001 2024-09-18T11:30:38.377 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.377 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.377 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.378 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1134 4.1 4:8b9a985d:::smithi11939506-23:a [sparse-read 0~1709094] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:38.378 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:38.378 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.174930188000001, 2024-09-18T11:30:38.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061866400000000002, 2024-09-18T11:30:38.378 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.378 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:38.378 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.378 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:38.378 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:38.378 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1134 2024-09-18T11:30:38.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.379 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:38.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:38.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179092+0000", 2024-09-18T11:30:38.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:38.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179104+0000", 2024-09-18T11:30:38.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2346e-05 2024-09-18T11:30:38.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179128+0000", 2024-09-18T11:30:38.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4046999999999999e-05 2024-09-18T11:30:38.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179141+0000", 2024-09-18T11:30:38.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3149000000000001e-05 2024-09-18T11:30:38.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179406+0000", 2024-09-18T11:30:38.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026496100000000003 2024-09-18T11:30:38.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:38.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179431+0000", 2024-09-18T11:30:38.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5242999999999999e-05 2024-09-18T11:30:38.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184981+0000", 2024-09-18T11:30:38.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055495969999999999 2024-09-18T11:30:38.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185055+0000", 2024-09-18T11:30:38.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4375999999999994e-05 2024-09-18T11:30:38.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185280+0000", 2024-09-18T11:30:38.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022433800000000001 2024-09-18T11:30:38.383 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.383 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.383 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.383 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1135 4.1 4:8b9a985d:::smithi11939506-23:a [sparse-read 0~1709094] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:38.383 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179201+0000", 2024-09-18T11:30:38.383 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.174821740999999, 2024-09-18T11:30:38.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063123149999999998, 2024-09-18T11:30:38.383 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.383 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:38.383 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.383 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:38.383 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:38.384 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1135 2024-09-18T11:30:38.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.384 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179201+0000", 2024-09-18T11:30:38.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179201+0000", 2024-09-18T11:30:38.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179200+0000", 2024-09-18T11:30:38.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:38.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179207+0000", 2024-09-18T11:30:38.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1960000000000002e-06 2024-09-18T11:30:38.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179229+0000", 2024-09-18T11:30:38.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1322999999999999e-05 2024-09-18T11:30:38.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179240+0000", 2024-09-18T11:30:38.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1793e-05 2024-09-18T11:30:38.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179460+0000", 2024-09-18T11:30:38.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000219407 2024-09-18T11:30:38.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:38.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179466+0000", 2024-09-18T11:30:38.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7490000000000004e-06 2024-09-18T11:30:38.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185300+0000", 2024-09-18T11:30:38.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058347490000000002 2024-09-18T11:30:38.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185345+0000", 2024-09-18T11:30:38.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4316000000000001e-05 2024-09-18T11:30:38.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185514+0000", 2024-09-18T11:30:38.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016905800000000001 2024-09-18T11:30:38.388 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.388 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.388 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.388 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1378 4.d 4:b81644f2:::smithi11939506-34:head [write 615156~555834,stat] snapc c=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-18T11:30:38.388 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:25.201447+0000", 2024-09-18T11:30:38.388 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.152576186999999, 2024-09-18T11:30:38.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060023309999999996, 2024-09-18T11:30:38.389 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.389 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.389 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.389 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:38.389 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:38.389 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1378 2024-09-18T11:30:38.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.389 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.201447+0000", 2024-09-18T11:30:38.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.201447+0000", 2024-09-18T11:30:38.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.201445+0000", 2024-09-18T11:30:38.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:38.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202027+0000", 2024-09-18T11:30:38.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00058162700000000003 2024-09-18T11:30:38.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202044+0000", 2024-09-18T11:30:38.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6685e-05 2024-09-18T11:30:38.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202051+0000", 2024-09-18T11:30:38.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0890000000000002e-06 2024-09-18T11:30:38.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202086+0000", 2024-09-18T11:30:38.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5593999999999998e-05 2024-09-18T11:30:38.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202153+0000", 2024-09-18T11:30:38.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7237999999999996e-05 2024-09-18T11:30:38.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:38.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202277+0000", 2024-09-18T11:30:38.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012383899999999999 2024-09-18T11:30:38.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:38.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.205380+0000", 2024-09-18T11:30:38.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031026109999999999 2024-09-18T11:30:38.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:38.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.207341+0000", 2024-09-18T11:30:38.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019606419999999999 2024-09-18T11:30:38.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.207403+0000", 2024-09-18T11:30:38.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2921000000000003e-05 2024-09-18T11:30:38.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.207449+0000", 2024-09-18T11:30:38.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5824e-05 2024-09-18T11:30:38.394 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.394 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.394 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.394 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1421 4.d 4:b5a463bf:::smithi11939506-20:head [write 665577~781581] snapc 10=[10,e,c] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T11:30:38.394 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:29.216974+0000", 2024-09-18T11:30:38.394 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.1370495569999992, 2024-09-18T11:30:38.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006459869, 2024-09-18T11:30:38.394 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.394 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.394 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.395 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:38.395 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:38.395 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1421 2024-09-18T11:30:38.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.395 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.216974+0000", 2024-09-18T11:30:38.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.216974+0000", 2024-09-18T11:30:38.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.216972+0000", 2024-09-18T11:30:38.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:38.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217766+0000", 2024-09-18T11:30:38.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079344699999999997 2024-09-18T11:30:38.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217780+0000", 2024-09-18T11:30:38.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.428e-05 2024-09-18T11:30:38.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217786+0000", 2024-09-18T11:30:38.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9050000000000002e-06 2024-09-18T11:30:38.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217828+0000", 2024-09-18T11:30:38.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1839999999999999e-05 2024-09-18T11:30:38.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217897+0000", 2024-09-18T11:30:38.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9663e-05 2024-09-18T11:30:38.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:38.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.218013+0000", 2024-09-18T11:30:38.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011509 2024-09-18T11:30:38.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:38.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.220250+0000", 2024-09-18T11:30:38.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022370150000000002 2024-09-18T11:30:38.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:38.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.223359+0000", 2024-09-18T11:30:38.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031092519999999998 2024-09-18T11:30:38.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.223396+0000", 2024-09-18T11:30:38.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7531999999999999e-05 2024-09-18T11:30:38.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.223433+0000", 2024-09-18T11:30:38.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7098999999999998e-05 2024-09-18T11:30:38.399 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.400 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.400 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.400 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1528 3.1 e53/21 3:97025964:::d85d35018f93ce704e0c86e65d6c262758d2b38883b05fc1a96898c5a961f23a:head v 53'86, mlcod=51'84)", 2024-09-18T11:30:38.400 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.377855+0000", 2024-09-18T11:30:38.400 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.9761676530000001, 2024-09-18T11:30:38.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058274199999999998, 2024-09-18T11:30:38.400 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.400 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.400 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.377855+0000", 2024-09-18T11:30:38.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.377855+0000", 2024-09-18T11:30:38.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.377853+0000", 2024-09-18T11:30:38.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:38.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.379683+0000", 2024-09-18T11:30:38.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018298139999999999 2024-09-18T11:30:38.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.379698+0000", 2024-09-18T11:30:38.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4620000000000001e-05 2024-09-18T11:30:38.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.379722+0000", 2024-09-18T11:30:38.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4505999999999999e-05 2024-09-18T11:30:38.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.379796+0000", 2024-09-18T11:30:38.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3527000000000003e-05 2024-09-18T11:30:38.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.379825+0000", 2024-09-18T11:30:38.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.953e-05 2024-09-18T11:30:38.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.383640+0000", 2024-09-18T11:30:38.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038143629999999999 2024-09-18T11:30:38.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.383683+0000", 2024-09-18T11:30:38.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3053000000000003e-05 2024-09-18T11:30:38.404 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.404 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.404 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.405 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1530 4.d 4:b81644f2:::smithi11939506-34:head [write 467261~717275] snapc 14=[14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:30:38.405 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:30:38.405 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.963338609, 2024-09-18T11:30:38.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075660950000000001, 2024-09-18T11:30:38.405 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.405 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.405 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.405 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:38.405 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:38.405 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1530 2024-09-18T11:30:38.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.406 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:30:38.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:30:38.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390682+0000", 2024-09-18T11:30:38.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:38.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391554+0000", 2024-09-18T11:30:38.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00087220600000000004 2024-09-18T11:30:38.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391570+0000", 2024-09-18T11:30:38.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6002999999999999e-05 2024-09-18T11:30:38.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391575+0000", 2024-09-18T11:30:38.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2549999999999997e-06 2024-09-18T11:30:38.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391631+0000", 2024-09-18T11:30:38.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5671000000000003e-05 2024-09-18T11:30:38.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391751+0000", 2024-09-18T11:30:38.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000119821 2024-09-18T11:30:38.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:38.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391974+0000", 2024-09-18T11:30:38.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022309900000000001 2024-09-18T11:30:38.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:38.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398136+0000", 2024-09-18T11:30:38.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006161573 2024-09-18T11:30:38.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:38.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398184+0000", 2024-09-18T11:30:38.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8458000000000003e-05 2024-09-18T11:30:38.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398219+0000", 2024-09-18T11:30:38.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4573999999999998e-05 2024-09-18T11:30:38.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398251+0000", 2024-09-18T11:30:38.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2085000000000002e-05 2024-09-18T11:30:38.414 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.414 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.414 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.414 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1532 4.d 4:b81644f2:::smithi11939506-34:head [setxattr _header (57) in=7b,truncate 2067261] snapc 14=[14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:30:38.414 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.395599+0000", 2024-09-18T11:30:38.415 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.9584236920000002, 2024-09-18T11:30:38.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054503360000000001, 2024-09-18T11:30:38.415 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.415 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.415 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.415 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:38.415 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:38.415 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1532 2024-09-18T11:30:38.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.415 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395599+0000", 2024-09-18T11:30:38.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395599+0000", 2024-09-18T11:30:38.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395599+0000", 2024-09-18T11:30:38.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:38.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395603+0000", 2024-09-18T11:30:38.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3640000000000001e-06 2024-09-18T11:30:38.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395610+0000", 2024-09-18T11:30:38.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9090000000000003e-06 2024-09-18T11:30:38.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395615+0000", 2024-09-18T11:30:38.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6269999999999999e-06 2024-09-18T11:30:38.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.397243+0000", 2024-09-18T11:30:38.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016279350000000001 2024-09-18T11:30:38.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.397343+0000", 2024-09-18T11:30:38.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000100054 2024-09-18T11:30:38.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:38.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.397496+0000", 2024-09-18T11:30:38.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000153578 2024-09-18T11:30:38.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:38.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398472+0000", 2024-09-18T11:30:38.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00097554100000000002 2024-09-18T11:30:38.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:38.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.400978+0000", 2024-09-18T11:30:38.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025066590000000001 2024-09-18T11:30:38.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.401013+0000", 2024-09-18T11:30:38.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4357999999999999e-05 2024-09-18T11:30:38.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.401050+0000", 2024-09-18T11:30:38.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7052999999999999e-05 2024-09-18T11:30:38.420 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.420 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.420 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.421 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1533 4.d 4:b81644f2:::smithi11939506-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:30:38.421 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.395621+0000", 2024-09-18T11:30:38.421 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.9584022169999997, 2024-09-18T11:30:38.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055874380000000001, 2024-09-18T11:30:38.421 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.421 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:38.421 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.421 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:38.421 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:38.421 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1533 2024-09-18T11:30:38.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.421 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395621+0000", 2024-09-18T11:30:38.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395621+0000", 2024-09-18T11:30:38.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395621+0000", 2024-09-18T11:30:38.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:38.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395623+0000", 2024-09-18T11:30:38.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.841e-06 2024-09-18T11:30:38.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395630+0000", 2024-09-18T11:30:38.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.285e-06 2024-09-18T11:30:38.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395633+0000", 2024-09-18T11:30:38.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.614e-06 2024-09-18T11:30:38.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398334+0000", 2024-09-18T11:30:38.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027003460000000002 2024-09-18T11:30:38.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:38.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398422+0000", 2024-09-18T11:30:38.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8506999999999998e-05 2024-09-18T11:30:38.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.401087+0000", 2024-09-18T11:30:38.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026650900000000002 2024-09-18T11:30:38.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.401138+0000", 2024-09-18T11:30:38.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0566999999999999e-05 2024-09-18T11:30:38.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.401208+0000", 2024-09-18T11:30:38.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0603000000000001e-05 2024-09-18T11:30:38.426 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.426 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.426 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.426 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:30:38.426 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:30:38.426 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.9379362850000001, 2024-09-18T11:30:38.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:30:38.426 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.426 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.427 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.427 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:38.427 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:38.427 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:30:38.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.427 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:30:38.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:30:38.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:30:38.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:38.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:30:38.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:30:38.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:30:38.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:30:38.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:30:38.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:30:38.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:30:38.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:30:38.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:30:38.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:30:38.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:38.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:30:38.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:30:38.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:38.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:30:38.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:30:38.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:38.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:30:38.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:30:38.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:30:38.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:30:38.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:30:38.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:30:38.431 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.431 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.431 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.432 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1614 4.e 4:73745217:::smithi11939506-6:head [read 0~3966419,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:38.432 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.294355+0000", 2024-09-18T11:30:38.432 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.0596680430000001, 2024-09-18T11:30:38.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057731620000000001, 2024-09-18T11:30:38.432 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.432 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:38.432 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.432 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:38.432 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:38.432 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1614 2024-09-18T11:30:38.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.433 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294355+0000", 2024-09-18T11:30:38.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294355+0000", 2024-09-18T11:30:38.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294355+0000", 2024-09-18T11:30:38.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:38.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294359+0000", 2024-09-18T11:30:38.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5199999999999999e-06 2024-09-18T11:30:38.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294367+0000", 2024-09-18T11:30:38.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8189999999999992e-06 2024-09-18T11:30:38.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294371+0000", 2024-09-18T11:30:38.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3490000000000002e-06 2024-09-18T11:30:38.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.297333+0000", 2024-09-18T11:30:38.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029618800000000001 2024-09-18T11:30:38.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.297431+0000", 2024-09-18T11:30:38.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8042999999999996e-05 2024-09-18T11:30:38.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.300128+0000", 2024-09-18T11:30:38.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026971180000000001 2024-09-18T11:30:38.436 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.436 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.436 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.436 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1615 4.e 4:73745217:::smithi11939506-6:head [sparse-read 0~1834156] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:38.436 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.294431+0000", 2024-09-18T11:30:38.436 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.0595922419999999, 2024-09-18T11:30:38.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060124970000000003, 2024-09-18T11:30:38.436 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.436 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:38.437 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.437 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:38.437 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:38.437 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1615 2024-09-18T11:30:38.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.437 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294431+0000", 2024-09-18T11:30:38.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294431+0000", 2024-09-18T11:30:38.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294430+0000", 2024-09-18T11:30:38.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:38.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294439+0000", 2024-09-18T11:30:38.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9129999999999992e-06 2024-09-18T11:30:38.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294454+0000", 2024-09-18T11:30:38.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.524e-05 2024-09-18T11:30:38.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294460+0000", 2024-09-18T11:30:38.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6810000000000003e-06 2024-09-18T11:30:38.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.300155+0000", 2024-09-18T11:30:38.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056949779999999998 2024-09-18T11:30:38.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.300230+0000", 2024-09-18T11:30:38.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5593000000000006e-05 2024-09-18T11:30:38.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.300443+0000", 2024-09-18T11:30:38.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021289599999999999 2024-09-18T11:30:38.440 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.440 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.440 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.441 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1628 3.3 3:f80aa31b:::77cdbf3abbada57d0636b2198046a030673818705cbf13bb77b30c3e68b24345:head [write 0~582144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:38.441 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:30:38.441 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.043046285, 2024-09-18T11:30:38.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081714639999999998, 2024-09-18T11:30:38.441 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.441 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.441 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.441 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:38.441 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:38.441 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1628 2024-09-18T11:30:38.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.441 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:30:38.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:30:38.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310975+0000", 2024-09-18T11:30:38.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:38.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312086+0000", 2024-09-18T11:30:38.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001111288 2024-09-18T11:30:38.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312104+0000", 2024-09-18T11:30:38.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7427000000000001e-05 2024-09-18T11:30:38.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312110+0000", 2024-09-18T11:30:38.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5320000000000003e-06 2024-09-18T11:30:38.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312179+0000", 2024-09-18T11:30:38.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9196000000000006e-05 2024-09-18T11:30:38.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312375+0000", 2024-09-18T11:30:38.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00019550899999999999 2024-09-18T11:30:38.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:38.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312558+0000", 2024-09-18T11:30:38.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000182566 2024-09-18T11:30:38.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:38.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.314286+0000", 2024-09-18T11:30:38.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00172831 2024-09-18T11:30:38.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:38.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319079+0000", 2024-09-18T11:30:38.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047929340000000004 2024-09-18T11:30:38.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319116+0000", 2024-09-18T11:30:38.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6993999999999999e-05 2024-09-18T11:30:38.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319148+0000", 2024-09-18T11:30:38.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2539999999999997e-05 2024-09-18T11:30:38.446 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.446 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.446 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.447 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1662 4.1 4:87078dfe:::smithi11939506-14:head [write 635331~786118,stat] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:38.447 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.360496+0000", 2024-09-18T11:30:38.447 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.99352753999999999, 2024-09-18T11:30:38.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071849519999999997, 2024-09-18T11:30:38.447 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.447 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.447 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.447 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:38.447 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:38.447 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1662 2024-09-18T11:30:38.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.448 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.360496+0000", 2024-09-18T11:30:38.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.360496+0000", 2024-09-18T11:30:38.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.360493+0000", 2024-09-18T11:30:38.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:38.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361139+0000", 2024-09-18T11:30:38.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064615400000000002 2024-09-18T11:30:38.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361154+0000", 2024-09-18T11:30:38.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4970000000000001e-05 2024-09-18T11:30:38.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361161+0000", 2024-09-18T11:30:38.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4139999999999999e-06 2024-09-18T11:30:38.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361201+0000", 2024-09-18T11:30:38.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0385e-05 2024-09-18T11:30:38.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361273+0000", 2024-09-18T11:30:38.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2224999999999994e-05 2024-09-18T11:30:38.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:38.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361406+0000", 2024-09-18T11:30:38.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013257699999999999 2024-09-18T11:30:38.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:38.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367415+0000", 2024-09-18T11:30:38.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006009655 2024-09-18T11:30:38.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:38.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367586+0000", 2024-09-18T11:30:38.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017109800000000001 2024-09-18T11:30:38.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367636+0000", 2024-09-18T11:30:38.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9721e-05 2024-09-18T11:30:38.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367681+0000", 2024-09-18T11:30:38.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4382999999999998e-05 2024-09-18T11:30:38.453 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.453 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.453 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.453 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1663 4.1 4:87078dfe:::smithi11939506-14:head [write 1916458~318873] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:38.453 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:30:38.453 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.99009972199999996, 2024-09-18T11:30:38.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079505749999999997, 2024-09-18T11:30:38.453 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.453 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.454 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.454 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:38.454 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:38.454 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1663 2024-09-18T11:30:38.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.454 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:30:38.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:30:38.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363922+0000", 2024-09-18T11:30:38.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:38.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364211+0000", 2024-09-18T11:30:38.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000288385 2024-09-18T11:30:38.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364223+0000", 2024-09-18T11:30:38.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2582000000000001e-05 2024-09-18T11:30:38.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364228+0000", 2024-09-18T11:30:38.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0000000000000004e-06 2024-09-18T11:30:38.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364268+0000", 2024-09-18T11:30:38.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0235999999999999e-05 2024-09-18T11:30:38.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364341+0000", 2024-09-18T11:30:38.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2266000000000002e-05 2024-09-18T11:30:38.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:38.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364444+0000", 2024-09-18T11:30:38.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103632 2024-09-18T11:30:38.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:38.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369461+0000", 2024-09-18T11:30:38.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005016848 2024-09-18T11:30:38.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:38.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371820+0000", 2024-09-18T11:30:38.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023586169999999999 2024-09-18T11:30:38.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371851+0000", 2024-09-18T11:30:38.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1297999999999997e-05 2024-09-18T11:30:38.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371874+0000", 2024-09-18T11:30:38.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2841e-05 2024-09-18T11:30:38.459 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.459 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.459 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.459 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1664 4.1 4:87078dfe:::smithi11939506-14:head [setxattr _header (57) in=7b,truncate 2235331] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:38.459 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:30:38.459 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.98978606499999999, 2024-09-18T11:30:38.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010368235999999999, 2024-09-18T11:30:38.459 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.459 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.459 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.459 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:38.460 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:38.460 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1664 2024-09-18T11:30:38.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.460 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:30:38.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:30:38.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364236+0000", 2024-09-18T11:30:38.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:38.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364240+0000", 2024-09-18T11:30:38.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9829999999999998e-06 2024-09-18T11:30:38.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364247+0000", 2024-09-18T11:30:38.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2140000000000001e-06 2024-09-18T11:30:38.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364256+0000", 2024-09-18T11:30:38.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1509999999999993e-06 2024-09-18T11:30:38.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366529+0000", 2024-09-18T11:30:38.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022731409999999998 2024-09-18T11:30:38.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366621+0000", 2024-09-18T11:30:38.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2408e-05 2024-09-18T11:30:38.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:38.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366801+0000", 2024-09-18T11:30:38.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018002200000000001 2024-09-18T11:30:38.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:38.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372155+0000", 2024-09-18T11:30:38.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053531519999999999 2024-09-18T11:30:38.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:38.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374539+0000", 2024-09-18T11:30:38.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023842580000000002 2024-09-18T11:30:38.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374573+0000", 2024-09-18T11:30:38.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4205999999999997e-05 2024-09-18T11:30:38.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374605+0000", 2024-09-18T11:30:38.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2283999999999998e-05 2024-09-18T11:30:38.465 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.466 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.466 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.466 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1665 4.1 4:87078dfe:::smithi11939506-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:38.466 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:30:38.466 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.989761891, 2024-09-18T11:30:38.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010471897000000001, 2024-09-18T11:30:38.466 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.466 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:38.467 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.467 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:38.467 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:38.467 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1665 2024-09-18T11:30:38.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.467 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:30:38.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:30:38.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:30:38.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:38.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364263+0000", 2024-09-18T11:30:38.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6400000000000001e-06 2024-09-18T11:30:38.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364270+0000", 2024-09-18T11:30:38.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:30:38.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364274+0000", 2024-09-18T11:30:38.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.968e-06 2024-09-18T11:30:38.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367354+0000", 2024-09-18T11:30:38.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030803430000000001 2024-09-18T11:30:38.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:38.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367406+0000", 2024-09-18T11:30:38.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2330999999999998e-05 2024-09-18T11:30:38.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374625+0000", 2024-09-18T11:30:38.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072190689999999998 2024-09-18T11:30:38.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374673+0000", 2024-09-18T11:30:38.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7525999999999999e-05 2024-09-18T11:30:38.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374733+0000", 2024-09-18T11:30:38.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0359000000000003e-05 2024-09-18T11:30:38.471 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.471 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.471 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.471 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.471 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.471 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:38.473 INFO:tasks.rados.rados.0.smithi119.stdout:908: finishing write tid 1 to smithi11939506-9 2024-09-18T11:30:38.473 INFO:tasks.rados.rados.0.smithi119.stdout:910: expect (ObjNum 125 snap 14 seq_num 125) 2024-09-18T11:30:38.474 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:30:38.474 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:38.471+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:38.538 INFO:tasks.rados.rados.0.smithi119.stdout:908: finishing write tid 2 to smithi11939506-9 2024-09-18T11:30:38.538 INFO:tasks.rados.rados.0.smithi119.stdout:909: expect (ObjNum 127 snap 14 seq_num 127) 2024-09-18T11:30:38.574 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:38.571+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:38.614 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:38.614 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:38.614 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:38.614 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:38.622 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:30:38.641 INFO:tasks.rados.rados.0.smithi119.stdout:908: finishing write tid 3 to smithi11939506-9 2024-09-18T11:30:38.641 INFO:tasks.rados.rados.0.smithi119.stdout:908: finishing write tid 4 to smithi11939506-9 2024-09-18T11:30:38.641 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 89 (ObjNum 132 snap 15 seq_num 132) dirty exists 2024-09-18T11:30:38.641 INFO:tasks.rados.rados.0.smithi119.stdout:908: left oid 9 (ObjNum 132 snap 15 seq_num 132) 2024-09-18T11:30:38.641 INFO:tasks.rados.rados.0.smithi119.stdout:911: expect (ObjNum 86 snap 5 seq_num 86) 2024-09-18T11:30:38.706 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:38.704+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:38.760 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:38.760 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:38.760 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:38.760 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:38.760 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:38.768 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:30:38.809 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:38.807+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:38.823 INFO:tasks.rados.rados.0.smithi119.stdout:912: finishing write tid 1 to smithi11939506-14 2024-09-18T11:30:38.823 INFO:tasks.rados.rados.0.smithi119.stdout:912: finishing write tid 2 to smithi11939506-14 2024-09-18T11:30:38.823 INFO:tasks.rados.rados.0.smithi119.stdout:912: finishing write tid 3 to smithi11939506-14 2024-09-18T11:30:38.823 INFO:tasks.rados.rados.0.smithi119.stdout:912: finishing write tid 4 to smithi11939506-14 2024-09-18T11:30:38.823 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 75 (ObjNum 133 snap 15 seq_num 133) dirty exists 2024-09-18T11:30:38.823 INFO:tasks.rados.rados.0.smithi119.stdout:912: left oid 14 (ObjNum 133 snap 15 seq_num 133) 2024-09-18T11:30:38.823 INFO:tasks.rados.rados.0.smithi119.stdout:896: done (14 left) 2024-09-18T11:30:38.823 INFO:tasks.rados.rados.0.smithi119.stdout:897: done (13 left) 2024-09-18T11:30:38.824 INFO:tasks.rados.rados.0.smithi119.stdout:901: done (12 left) 2024-09-18T11:30:38.824 INFO:tasks.rados.rados.0.smithi119.stdout:902: done (11 left) 2024-09-18T11:30:38.824 INFO:tasks.rados.rados.0.smithi119.stdout:903: done (10 left) 2024-09-18T11:30:38.824 INFO:tasks.rados.rados.0.smithi119.stdout:904: done (9 left) 2024-09-18T11:30:38.824 INFO:tasks.rados.rados.0.smithi119.stdout:905: done (8 left) 2024-09-18T11:30:38.824 INFO:tasks.rados.rados.0.smithi119.stdout:906: done (7 left) 2024-09-18T11:30:38.824 INFO:tasks.rados.rados.0.smithi119.stdout:907: done (6 left) 2024-09-18T11:30:38.824 INFO:tasks.rados.rados.0.smithi119.stdout:908: done (5 left) 2024-09-18T11:30:38.824 INFO:tasks.rados.rados.0.smithi119.stdout:909: done (4 left) 2024-09-18T11:30:38.824 INFO:tasks.rados.rados.0.smithi119.stdout:910: done (3 left) 2024-09-18T11:30:38.824 INFO:tasks.rados.rados.0.smithi119.stdout:911: done (2 left) 2024-09-18T11:30:38.824 INFO:tasks.rados.rados.0.smithi119.stdout:912: done (1 left) 2024-09-18T11:30:38.824 INFO:tasks.rados.rados.0.smithi119.stdout:913: done (0 left) 2024-09-18T11:30:38.824 INFO:tasks.rados.rados.0.smithi119.stdout:914: read oid 5 snap -1 2024-09-18T11:30:38.826 INFO:tasks.rados.rados.0.smithi119.stdout:914: expect (ObjNum 105 snap 6 seq_num 105) 2024-09-18T11:30:38.826 INFO:tasks.rados.rados.0.smithi119.stdout:915: tier_promote oid 4 2024-09-18T11:30:38.826 INFO:tasks.rados.rados.0.smithi119.stdout:916: write (excl) oid 1 current snap is 16 2024-09-18T11:30:38.826 INFO:tasks.rados.rados.0.smithi119.stdout:916: seq_num 134 ranges {652138=461367,1574940=677198} 2024-09-18T11:30:38.829 INFO:tasks.rados.rados.0.smithi119.stdout:916: writing smithi11939506-1 from 652138 to 1113505 tid 1 2024-09-18T11:30:38.835 INFO:tasks.rados.rados.0.smithi119.stdout:916: writing smithi11939506-1 from 1574940 to 2252138 tid 2 2024-09-18T11:30:38.835 INFO:tasks.rados.rados.0.smithi119.stdout:917: read oid 23 snap -1 2024-09-18T11:30:38.835 INFO:tasks.rados.rados.0.smithi119.stdout:917: expect (ObjNum 90 snap 5 seq_num 90) 2024-09-18T11:30:38.836 INFO:tasks.rados.rados.0.smithi119.stdout:918: snap_remove snap 6 2024-09-18T11:30:38.876 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:38.874+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:38.910 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:38.910 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:38.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:38.911 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:38.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.911 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:30:38.911 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:30:38.911 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 48.881840310999998, 2024-09-18T11:30:38.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073601550000000002, 2024-09-18T11:30:38.911 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.911 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.911 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.911 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:38.911 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:38.911 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1 2024-09-18T11:30:38.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.912 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:30:38.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:30:38.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026524+0000", 2024-09-18T11:30:38.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999905 2024-09-18T11:30:38.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026541+0000", 2024-09-18T11:30:38.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7070000000000001e-05 2024-09-18T11:30:38.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026554+0000", 2024-09-18T11:30:38.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.288e-05 2024-09-18T11:30:38.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026564+0000", 2024-09-18T11:30:38.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0197000000000001e-05 2024-09-18T11:30:38.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026826+0000", 2024-09-18T11:30:38.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026212400000000003 2024-09-18T11:30:38.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:30:38.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026839+0000", 2024-09-18T11:30:38.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2831e-05 2024-09-18T11:30:38.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028605+0000", 2024-09-18T11:30:38.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001765794 2024-09-18T11:30:38.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028709+0000", 2024-09-18T11:30:38.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104348 2024-09-18T11:30:38.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:38.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028824+0000", 2024-09-18T11:30:38.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011527399999999999 2024-09-18T11:30:38.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:38.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030319+0000", 2024-09-18T11:30:38.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001494439 2024-09-18T11:30:38.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:38.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033805+0000", 2024-09-18T11:30:38.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034857820000000002 2024-09-18T11:30:38.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033856+0000", 2024-09-18T11:30:38.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1687999999999997e-05 2024-09-18T11:30:38.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033894+0000", 2024-09-18T11:30:38.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7481000000000002e-05 2024-09-18T11:30:38.917 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.917 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.917 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.917 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T11:30:38.917 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:51.038968+0000", 2024-09-18T11:30:38.917 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 47.869406116999997, 2024-09-18T11:30:38.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028722909999999999, 2024-09-18T11:30:38.917 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.918 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.918 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.918 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:38.918 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:38.918 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 5 2024-09-18T11:30:38.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.918 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.038968+0000", 2024-09-18T11:30:38.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.038968+0000", 2024-09-18T11:30:38.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.038966+0000", 2024-09-18T11:30:38.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:38.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.038983+0000", 2024-09-18T11:30:38.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7099e-05 2024-09-18T11:30:38.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.039002+0000", 2024-09-18T11:30:38.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8938000000000001e-05 2024-09-18T11:30:38.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.039012+0000", 2024-09-18T11:30:38.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9529999999999994e-06 2024-09-18T11:30:38.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.039068+0000", 2024-09-18T11:30:38.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5893999999999997e-05 2024-09-18T11:30:38.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:38.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.039074+0000", 2024-09-18T11:30:38.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8510000000000001e-06 2024-09-18T11:30:38.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.040586+0000", 2024-09-18T11:30:38.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015119809999999999 2024-09-18T11:30:38.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.040705+0000", 2024-09-18T11:30:38.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000119054 2024-09-18T11:30:38.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:38.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.040812+0000", 2024-09-18T11:30:38.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010652700000000001 2024-09-18T11:30:38.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:38.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041405+0000", 2024-09-18T11:30:38.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059319099999999997 2024-09-18T11:30:38.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:38.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041787+0000", 2024-09-18T11:30:38.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00038150800000000002 2024-09-18T11:30:38.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041817+0000", 2024-09-18T11:30:38.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0091e-05 2024-09-18T11:30:38.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:51.041840+0000", 2024-09-18T11:30:38.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3385000000000001e-05 2024-09-18T11:30:38.924 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.924 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.924 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.924 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T11:30:38.924 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:30:38.924 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 46.862561880000001, 2024-09-18T11:30:38.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055107869999999996, 2024-09-18T11:30:38.925 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.925 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.925 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:38.925 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:38.925 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:38.925 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 7 2024-09-18T11:30:38.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.925 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:30:38.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:30:38.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045810+0000", 2024-09-18T11:30:38.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:38.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045820+0000", 2024-09-18T11:30:38.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0215e-05 2024-09-18T11:30:38.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045838+0000", 2024-09-18T11:30:38.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7849000000000001e-05 2024-09-18T11:30:38.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045845+0000", 2024-09-18T11:30:38.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4229999999999999e-06 2024-09-18T11:30:38.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045898+0000", 2024-09-18T11:30:38.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2537999999999997e-05 2024-09-18T11:30:38.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:38.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045904+0000", 2024-09-18T11:30:38.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4019999999999999e-06 2024-09-18T11:30:38.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047663+0000", 2024-09-18T11:30:38.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017584079999999999 2024-09-18T11:30:38.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047736+0000", 2024-09-18T11:30:38.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3509000000000004e-05 2024-09-18T11:30:38.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:38.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047857+0000", 2024-09-18T11:30:38.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012037800000000001 2024-09-18T11:30:38.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:38.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.050868+0000", 2024-09-18T11:30:38.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030111420000000001 2024-09-18T11:30:38.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:38.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051239+0000", 2024-09-18T11:30:38.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037141000000000002 2024-09-18T11:30:38.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051288+0000", 2024-09-18T11:30:38.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9051000000000002e-05 2024-09-18T11:30:38.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051323+0000", 2024-09-18T11:30:38.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4545000000000002e-05 2024-09-18T11:30:38.931 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.931 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.931 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.931 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2 4.7 e28/22 4:e73ba68f:::smithi11939506-1:head v 28'2, mlcod=0'0)", 2024-09-18T11:30:38.931 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:08.105343+0000", 2024-09-18T11:30:38.931 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 30.803031104999999, 2024-09-18T11:30:38.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040860050000000002, 2024-09-18T11:30:38.931 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.931 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.931 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105343+0000", 2024-09-18T11:30:38.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105343+0000", 2024-09-18T11:30:38.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105342+0000", 2024-09-18T11:30:38.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:38.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105801+0000", 2024-09-18T11:30:38.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00045840099999999999 2024-09-18T11:30:38.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105813+0000", 2024-09-18T11:30:38.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2461e-05 2024-09-18T11:30:38.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105818+0000", 2024-09-18T11:30:38.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.989e-06 2024-09-18T11:30:38.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105856+0000", 2024-09-18T11:30:38.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7839999999999997e-05 2024-09-18T11:30:38.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105872+0000", 2024-09-18T11:30:38.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6582000000000001e-05 2024-09-18T11:30:38.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.109389+0000", 2024-09-18T11:30:38.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003516211 2024-09-18T11:30:38.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.109429+0000", 2024-09-18T11:30:38.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0261e-05 2024-09-18T11:30:38.935 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.935 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.935 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.935 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3 4.7 e28/22 4:e73ba68f:::smithi11939506-1:head v 28'3, mlcod=0'0)", 2024-09-18T11:30:38.936 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:08.106522+0000", 2024-09-18T11:30:38.936 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 30.801852089, 2024-09-18T11:30:38.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031398509999999999, 2024-09-18T11:30:38.936 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.936 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.936 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106522+0000", 2024-09-18T11:30:38.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106522+0000", 2024-09-18T11:30:38.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106521+0000", 2024-09-18T11:30:38.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:38.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106526+0000", 2024-09-18T11:30:38.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9799999999999998e-06 2024-09-18T11:30:38.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106533+0000", 2024-09-18T11:30:38.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1229999999999997e-06 2024-09-18T11:30:38.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106536+0000", 2024-09-18T11:30:38.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.055e-06 2024-09-18T11:30:38.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106577+0000", 2024-09-18T11:30:38.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0932e-05 2024-09-18T11:30:38.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106588+0000", 2024-09-18T11:30:38.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1486e-05 2024-09-18T11:30:38.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.109630+0000", 2024-09-18T11:30:38.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030422159999999999 2024-09-18T11:30:38.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.109662+0000", 2024-09-18T11:30:38.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1498999999999998e-05 2024-09-18T11:30:38.940 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.940 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.940 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.940 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:905 4.7 e31/22 4:e73ba68f:::smithi11939506-1:head v 31'32, mlcod=30'29)", 2024-09-18T11:30:38.940 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:30:38.940 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.766905669, 2024-09-18T11:30:38.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048351990000000001, 2024-09-18T11:30:38.940 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.940 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.941 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:30:38.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:30:38.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:30:38.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:38.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144912+0000", 2024-09-18T11:30:38.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003444 2024-09-18T11:30:38.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144921+0000", 2024-09-18T11:30:38.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1900000000000001e-06 2024-09-18T11:30:38.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144927+0000", 2024-09-18T11:30:38.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4940000000000003e-06 2024-09-18T11:30:38.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144952+0000", 2024-09-18T11:30:38.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5845000000000001e-05 2024-09-18T11:30:38.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144966+0000", 2024-09-18T11:30:38.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.396e-05 2024-09-18T11:30:38.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.146274+0000", 2024-09-18T11:30:38.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013073550000000001 2024-09-18T11:30:38.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.146303+0000", 2024-09-18T11:30:38.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9754e-05 2024-09-18T11:30:38.944 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.944 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.944 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.944 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:911 4.3 e31/22 4:cc46532a:::smithi11939506-33:head v 31'24, mlcod=30'21)", 2024-09-18T11:30:38.945 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.166801+0000", 2024-09-18T11:30:38.945 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.741573342999999, 2024-09-18T11:30:38.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048144349999999997, 2024-09-18T11:30:38.945 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.945 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.945 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.166801+0000", 2024-09-18T11:30:38.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.166801+0000", 2024-09-18T11:30:38.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.166799+0000", 2024-09-18T11:30:38.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:38.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167413+0000", 2024-09-18T11:30:38.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000613259 2024-09-18T11:30:38.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167431+0000", 2024-09-18T11:30:38.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7906000000000001e-05 2024-09-18T11:30:38.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167438+0000", 2024-09-18T11:30:38.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.883e-06 2024-09-18T11:30:38.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167468+0000", 2024-09-18T11:30:38.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0168e-05 2024-09-18T11:30:38.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167483+0000", 2024-09-18T11:30:38.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4997999999999999e-05 2024-09-18T11:30:38.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.171574+0000", 2024-09-18T11:30:38.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040909589999999999 2024-09-18T11:30:38.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.171615+0000", 2024-09-18T11:30:38.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1371000000000001e-05 2024-09-18T11:30:38.949 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.949 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.949 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.949 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:989 4.3 e35/22 4:ca2b2309:::smithi11939506-44:head v 35'30, mlcod=31'26)", 2024-09-18T11:30:38.949 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.154272+0000", 2024-09-18T11:30:38.949 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.754102190000001, 2024-09-18T11:30:38.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031808090000000002, 2024-09-18T11:30:38.949 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.949 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.949 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.154272+0000", 2024-09-18T11:30:38.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.154272+0000", 2024-09-18T11:30:38.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.154272+0000", 2024-09-18T11:30:38.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:38.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.155085+0000", 2024-09-18T11:30:38.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000813921 2024-09-18T11:30:38.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.155097+0000", 2024-09-18T11:30:38.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1881e-05 2024-09-18T11:30:38.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.155103+0000", 2024-09-18T11:30:38.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6250000000000004e-06 2024-09-18T11:30:38.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.155161+0000", 2024-09-18T11:30:38.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8514999999999998e-05 2024-09-18T11:30:38.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.155185+0000", 2024-09-18T11:30:38.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3841000000000001e-05 2024-09-18T11:30:38.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.157411+0000", 2024-09-18T11:30:38.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022258719999999998 2024-09-18T11:30:38.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.157453+0000", 2024-09-18T11:30:38.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1387999999999997e-05 2024-09-18T11:30:38.953 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.953 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.953 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.953 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:990 4.3 e35/22 4:ca2b2309:::smithi11939506-44:head v 35'31, mlcod=34'28)", 2024-09-18T11:30:38.953 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.158794+0000", 2024-09-18T11:30:38.953 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.749580295000001, 2024-09-18T11:30:38.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042163180000000001, 2024-09-18T11:30:38.954 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.954 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.954 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.158794+0000", 2024-09-18T11:30:38.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.158794+0000", 2024-09-18T11:30:38.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.158793+0000", 2024-09-18T11:30:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159307+0000", 2024-09-18T11:30:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00051453999999999996 2024-09-18T11:30:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159320+0000", 2024-09-18T11:30:38.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2639e-05 2024-09-18T11:30:38.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159327+0000", 2024-09-18T11:30:38.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3200000000000002e-06 2024-09-18T11:30:38.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159359+0000", 2024-09-18T11:30:38.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2147999999999998e-05 2024-09-18T11:30:38.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159380+0000", 2024-09-18T11:30:38.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0625999999999999e-05 2024-09-18T11:30:38.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.162980+0000", 2024-09-18T11:30:38.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003600125 2024-09-18T11:30:38.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.163010+0000", 2024-09-18T11:30:38.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0020999999999999e-05 2024-09-18T11:30:38.957 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.958 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.958 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.958 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1118 3.2 e38/21 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head v 38'76, mlcod=38'74)", 2024-09-18T11:30:38.958 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:30:38.958 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.960430159000001, 2024-09-18T11:30:38.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006048688, 2024-09-18T11:30:38.958 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.958 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.958 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:30:38.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:30:38.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947941+0000", 2024-09-18T11:30:38.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:38.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949868+0000", 2024-09-18T11:30:38.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019267659999999999 2024-09-18T11:30:38.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949893+0000", 2024-09-18T11:30:38.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5239999999999999e-05 2024-09-18T11:30:38.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949901+0000", 2024-09-18T11:30:38.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8420000000000005e-06 2024-09-18T11:30:38.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949954+0000", 2024-09-18T11:30:38.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3464999999999997e-05 2024-09-18T11:30:38.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949974+0000", 2024-09-18T11:30:38.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9731000000000001e-05 2024-09-18T11:30:38.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953948+0000", 2024-09-18T11:30:38.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039735489999999998 2024-09-18T11:30:38.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953992+0000", 2024-09-18T11:30:38.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4700999999999998e-05 2024-09-18T11:30:38.962 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.962 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.962 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.962 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1139 4.2 e39/22 4:425afe9c:::smithi11939506-29:head v 39'19, mlcod=35'15)", 2024-09-18T11:30:38.962 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.187224+0000", 2024-09-18T11:30:38.963 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.721149881999999, 2024-09-18T11:30:38.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036968130000000002, 2024-09-18T11:30:38.963 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.963 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.963 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.187224+0000", 2024-09-18T11:30:38.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.187224+0000", 2024-09-18T11:30:38.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.187224+0000", 2024-09-18T11:30:38.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:38.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.188633+0000", 2024-09-18T11:30:38.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001409158 2024-09-18T11:30:38.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.188646+0000", 2024-09-18T11:30:38.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3458999999999999e-05 2024-09-18T11:30:38.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.188653+0000", 2024-09-18T11:30:38.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8569999999999997e-06 2024-09-18T11:30:38.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.188700+0000", 2024-09-18T11:30:38.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7176999999999999e-05 2024-09-18T11:30:38.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.188720+0000", 2024-09-18T11:30:38.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0043000000000001e-05 2024-09-18T11:30:38.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.190874+0000", 2024-09-18T11:30:38.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021537739999999998 2024-09-18T11:30:38.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.190921+0000", 2024-09-18T11:30:38.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6864999999999999e-05 2024-09-18T11:30:38.966 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.967 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.967 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.967 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1165 4.7 e39/22 4:e73ba68f:::smithi11939506-1:head v 39'40, mlcod=34'37)", 2024-09-18T11:30:38.967 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.657838+0000", 2024-09-18T11:30:38.967 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.250535497000001, 2024-09-18T11:30:38.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042136450000000002, 2024-09-18T11:30:38.967 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.967 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.967 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.657838+0000", 2024-09-18T11:30:38.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.657838+0000", 2024-09-18T11:30:38.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.657837+0000", 2024-09-18T11:30:38.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:38.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659431+0000", 2024-09-18T11:30:38.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015942479999999999 2024-09-18T11:30:38.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659450+0000", 2024-09-18T11:30:38.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8909000000000001e-05 2024-09-18T11:30:38.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659457+0000", 2024-09-18T11:30:38.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8390000000000001e-06 2024-09-18T11:30:38.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659500+0000", 2024-09-18T11:30:38.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3101e-05 2024-09-18T11:30:38.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659518+0000", 2024-09-18T11:30:38.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8020999999999999e-05 2024-09-18T11:30:38.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.662024+0000", 2024-09-18T11:30:38.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025059399999999999 2024-09-18T11:30:38.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.662052+0000", 2024-09-18T11:30:38.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8206999999999999e-05 2024-09-18T11:30:38.972 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.972 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.972 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.972 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.13:29 3.2 e39/21 3:5fa8fcfa:::ec59520759824f0488c7a3ca135dacb324de2330e16b7f54053de5719c4e96b6:head v 39'81, mlcod=38'76)", 2024-09-18T11:30:38.972 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:30:38.972 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.226729682999999, 2024-09-18T11:30:38.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004755858, 2024-09-18T11:30:38.972 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.972 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.972 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:30:38.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:30:38.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:30:38.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:38.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681961+0000", 2024-09-18T11:30:38.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00031696799999999999 2024-09-18T11:30:38.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681969+0000", 2024-09-18T11:30:38.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4800000000000001e-06 2024-09-18T11:30:38.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681973+0000", 2024-09-18T11:30:38.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4819999999999999e-06 2024-09-18T11:30:38.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682007+0000", 2024-09-18T11:30:38.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3961999999999998e-05 2024-09-18T11:30:38.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682033+0000", 2024-09-18T11:30:38.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6103999999999999e-05 2024-09-18T11:30:38.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686375+0000", 2024-09-18T11:30:38.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043422019999999999 2024-09-18T11:30:38.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686400+0000", 2024-09-18T11:30:38.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5265e-05 2024-09-18T11:30:38.976 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.976 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.976 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.977 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.13:30 3.2 e39/21 3:49074e56:::7fc21ffee1ba17f2a9197c61ff60ef0b03804c121c8aeb83ea424b7ac77081e9:head v 39'82, mlcod=39'77)", 2024-09-18T11:30:38.977 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.682532+0000", 2024-09-18T11:30:38.977 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.225841621000001, 2024-09-18T11:30:38.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004072694, 2024-09-18T11:30:38.977 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.977 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.977 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682532+0000", 2024-09-18T11:30:38.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682532+0000", 2024-09-18T11:30:38.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682532+0000", 2024-09-18T11:30:38.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:38.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682681+0000", 2024-09-18T11:30:38.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014942500000000001 2024-09-18T11:30:38.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682691+0000", 2024-09-18T11:30:38.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0125e-05 2024-09-18T11:30:38.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682695+0000", 2024-09-18T11:30:38.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9339999999999999e-06 2024-09-18T11:30:38.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682724+0000", 2024-09-18T11:30:38.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8918e-05 2024-09-18T11:30:38.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682738+0000", 2024-09-18T11:30:38.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3793e-05 2024-09-18T11:30:38.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686590+0000", 2024-09-18T11:30:38.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038524380000000001 2024-09-18T11:30:38.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686605+0000", 2024-09-18T11:30:38.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4827999999999999e-05 2024-09-18T11:30:38.981 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.981 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.981 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.981 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1174 4.7 e39/22 4:e0d95e10:::smithi11939506-22:head v 39'45, mlcod=39'42)", 2024-09-18T11:30:38.981 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.035250+0000", 2024-09-18T11:30:38.982 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.873124130000001, 2024-09-18T11:30:38.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028686929999999999, 2024-09-18T11:30:38.982 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.982 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.982 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035250+0000", 2024-09-18T11:30:38.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035250+0000", 2024-09-18T11:30:38.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035249+0000", 2024-09-18T11:30:38.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:38.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035852+0000", 2024-09-18T11:30:38.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060368100000000001 2024-09-18T11:30:38.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035866+0000", 2024-09-18T11:30:38.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3448999999999999e-05 2024-09-18T11:30:38.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035873+0000", 2024-09-18T11:30:38.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5739999999999998e-06 2024-09-18T11:30:38.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035918+0000", 2024-09-18T11:30:38.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4648999999999999e-05 2024-09-18T11:30:38.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.035940+0000", 2024-09-18T11:30:38.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2166000000000001e-05 2024-09-18T11:30:38.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.038085+0000", 2024-09-18T11:30:38.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021446210000000002 2024-09-18T11:30:38.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.038119+0000", 2024-09-18T11:30:38.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3800000000000002e-05 2024-09-18T11:30:38.986 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.986 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.986 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.986 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1623 4.3 e56/22 4:c70c3162:::smithi11939506-35:head v 56'43, mlcod=48'40)", 2024-09-18T11:30:38.986 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.302560+0000", 2024-09-18T11:30:38.986 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.6058143459999998, 2024-09-18T11:30:38.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046159060000000003, 2024-09-18T11:30:38.986 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.986 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.986 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.302560+0000", 2024-09-18T11:30:38.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.302560+0000", 2024-09-18T11:30:38.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.302559+0000", 2024-09-18T11:30:38.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:38.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303837+0000", 2024-09-18T11:30:38.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012781089999999999 2024-09-18T11:30:38.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303853+0000", 2024-09-18T11:30:38.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5551999999999999e-05 2024-09-18T11:30:38.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303862+0000", 2024-09-18T11:30:38.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0489999999999993e-06 2024-09-18T11:30:38.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303914+0000", 2024-09-18T11:30:38.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1697999999999999e-05 2024-09-18T11:30:38.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303938+0000", 2024-09-18T11:30:38.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4369e-05 2024-09-18T11:30:38.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.307122+0000", 2024-09-18T11:30:38.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031841780000000002 2024-09-18T11:30:38.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.307176+0000", 2024-09-18T11:30:38.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3400999999999999e-05 2024-09-18T11:30:38.990 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.990 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.991 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.991 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1625 4.3 e56/22 4:c70c3162:::smithi11939506-35:head v 56'45, mlcod=53'41)", 2024-09-18T11:30:38.991 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.309486+0000", 2024-09-18T11:30:38.991 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.598887604, 2024-09-18T11:30:38.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030400660000000001, 2024-09-18T11:30:38.991 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.991 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.991 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.309486+0000", 2024-09-18T11:30:38.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.309486+0000", 2024-09-18T11:30:38.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.309485+0000", 2024-09-18T11:30:38.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:38.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.309498+0000", 2024-09-18T11:30:38.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.313e-05 2024-09-18T11:30:38.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.309510+0000", 2024-09-18T11:30:38.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1972e-05 2024-09-18T11:30:38.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.309517+0000", 2024-09-18T11:30:38.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.058e-06 2024-09-18T11:30:38.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.309569+0000", 2024-09-18T11:30:38.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1737999999999998e-05 2024-09-18T11:30:38.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.309593+0000", 2024-09-18T11:30:38.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4613e-05 2024-09-18T11:30:38.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312472+0000", 2024-09-18T11:30:38.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028784909999999999 2024-09-18T11:30:38.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312526+0000", 2024-09-18T11:30:38.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4432999999999999e-05 2024-09-18T11:30:38.995 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.995 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.995 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.995 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1662 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'73, mlcod=51'69)", 2024-09-18T11:30:38.995 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:30:38.995 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.5468251240000002, 2024-09-18T11:30:38.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059408539999999998, 2024-09-18T11:30:38.995 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:38.995 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:38.996 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:38.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:38.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:30:38.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:38.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:30:38.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:38.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:38.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:30:38.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:38.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:38.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361914+0000", 2024-09-18T11:30:38.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036569 2024-09-18T11:30:38.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:38.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361926+0000", 2024-09-18T11:30:38.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1960999999999999e-05 2024-09-18T11:30:38.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:38.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361937+0000", 2024-09-18T11:30:38.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0876999999999999e-05 2024-09-18T11:30:38.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:38.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361986+0000", 2024-09-18T11:30:38.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8652000000000001e-05 2024-09-18T11:30:38.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:38.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.362012+0000", 2024-09-18T11:30:38.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5854999999999999e-05 2024-09-18T11:30:38.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:38.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367460+0000", 2024-09-18T11:30:38.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054482100000000002 2024-09-18T11:30:38.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:38.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:38.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367490+0000", 2024-09-18T11:30:38.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9909000000000001e-05 2024-09-18T11:30:38.999 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.999 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:38.999 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:38.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:38.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.000 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1663 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'74, mlcod=51'69)", 2024-09-18T11:30:39.000 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364521+0000", 2024-09-18T11:30:39.000 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.5438531260000001, 2024-09-18T11:30:39.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048541399999999998, 2024-09-18T11:30:39.000 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.000 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:39.000 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364521+0000", 2024-09-18T11:30:39.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364521+0000", 2024-09-18T11:30:39.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364520+0000", 2024-09-18T11:30:39.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:39.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364616+0000", 2024-09-18T11:30:39.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5569999999999995e-05 2024-09-18T11:30:39.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364625+0000", 2024-09-18T11:30:39.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3379999999999994e-06 2024-09-18T11:30:39.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364629+0000", 2024-09-18T11:30:39.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2490000000000003e-06 2024-09-18T11:30:39.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364664+0000", 2024-09-18T11:30:39.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5132999999999997e-05 2024-09-18T11:30:39.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364687+0000", 2024-09-18T11:30:39.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2691999999999999e-05 2024-09-18T11:30:39.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:39.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369349+0000", 2024-09-18T11:30:39.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046622950000000003 2024-09-18T11:30:39.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369375+0000", 2024-09-18T11:30:39.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5534e-05 2024-09-18T11:30:39.004 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.004 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.004 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.004 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1664 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'75, mlcod=51'69)", 2024-09-18T11:30:39.004 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:30:39.004 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.5414866169999999, 2024-09-18T11:30:39.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051832229999999998, 2024-09-18T11:30:39.004 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.004 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:39.004 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:30:39.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:30:39.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:30:39.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:39.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366894+0000", 2024-09-18T11:30:39.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2219999999999996e-06 2024-09-18T11:30:39.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366906+0000", 2024-09-18T11:30:39.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2278e-05 2024-09-18T11:30:39.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366912+0000", 2024-09-18T11:30:39.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2430000000000002e-06 2024-09-18T11:30:39.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366952+0000", 2024-09-18T11:30:39.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9428999999999998e-05 2024-09-18T11:30:39.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366972+0000", 2024-09-18T11:30:39.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0554999999999999e-05 2024-09-18T11:30:39.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:39.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372045+0000", 2024-09-18T11:30:39.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050727319999999999 2024-09-18T11:30:39.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372071+0000", 2024-09-18T11:30:39.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5477e-05 2024-09-18T11:30:39.008 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.008 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.008 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.008 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.008 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.008 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:39.010 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:30:39.010 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:39.007+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:39.110 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:39.108+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:39.149 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:39.149 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:39.150 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:39.150 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:39.158 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:30:39.242 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:39.240+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:39.258 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:39.255+0000 7fcb682b1640 -1 osd.7 57 handle_osd_map injecting map crc failure 2024-09-18T11:30:39.260 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is set 2024-09-18T11:30:39.261 INFO:tasks.rados.rados.0.smithi119.stdout:915: got (0) Success 2024-09-18T11:30:39.261 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 4 v 23 (ObjNum 110 snap 8 seq_num 110) dirty exists 2024-09-18T11:30:39.261 INFO:tasks.rados.rados.0.smithi119.stdout:914: expect (ObjNum 105 snap 6 seq_num 105) 2024-09-18T11:30:39.302 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:39.302 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:39.302 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:39.302 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:39.302 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:39.311 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:30:39.312 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:39.310+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:39.371 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:30:39.411 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:39.409+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:39.451 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:39.451 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:39.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:39.451 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 28.332716776000002, 2024-09-18T11:30:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:30:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:30:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:30:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:30:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:30:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:30:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:30:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:30:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:30:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:30:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:30:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:30:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:30:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:30:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:30:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:30:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:30:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:30:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:30:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:896 4.5 4:aa575018:::smithi11939506-11:2 [read 0~2941192,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.132995+0000", 2024-09-18T11:30:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 28.316041134999999, 2024-09-18T11:30:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047874989999999997, 2024-09-18T11:30:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 896 2024-09-18T11:30:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.132995+0000", 2024-09-18T11:30:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.132995+0000", 2024-09-18T11:30:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.132993+0000", 2024-09-18T11:30:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133000+0000", 2024-09-18T11:30:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5509999999999996e-06 2024-09-18T11:30:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133010+0000", 2024-09-18T11:30:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0254e-05 2024-09-18T11:30:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133023+0000", 2024-09-18T11:30:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3458999999999999e-05 2024-09-18T11:30:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133069+0000", 2024-09-18T11:30:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5324000000000002e-05 2024-09-18T11:30:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133161+0000", 2024-09-18T11:30:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2773000000000001e-05 2024-09-18T11:30:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.137782+0000", 2024-09-18T11:30:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046208680000000002 2024-09-18T11:30:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:897 4.5 4:aa575018:::smithi11939506-11:2 [sparse-read 0~1343176] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.133034+0000", 2024-09-18T11:30:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 28.316001776, 2024-09-18T11:30:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051617950000000003, 2024-09-18T11:30:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 897 2024-09-18T11:30:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133034+0000", 2024-09-18T11:30:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133034+0000", 2024-09-18T11:30:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133033+0000", 2024-09-18T11:30:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133037+0000", 2024-09-18T11:30:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.118e-06 2024-09-18T11:30:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133045+0000", 2024-09-18T11:30:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1780000000000003e-06 2024-09-18T11:30:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133049+0000", 2024-09-18T11:30:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7730000000000001e-06 2024-09-18T11:30:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.137838+0000", 2024-09-18T11:30:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00478912 2024-09-18T11:30:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.137936+0000", 2024-09-18T11:30:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.823e-05 2024-09-18T11:30:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138196+0000", 2024-09-18T11:30:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00025993799999999998 2024-09-18T11:30:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:898 4.5 4:aa575018:::smithi11939506-11:2 [sparse-read 0~1343176] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 28.315972737999999, 2024-09-18T11:30:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055588349999999998, 2024-09-18T11:30:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 898 2024-09-18T11:30:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133066+0000", 2024-09-18T11:30:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2729999999999998e-06 2024-09-18T11:30:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133072+0000", 2024-09-18T11:30:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0680000000000002e-06 2024-09-18T11:30:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133077+0000", 2024-09-18T11:30:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3569999999999997e-06 2024-09-18T11:30:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138226+0000", 2024-09-18T11:30:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051480980000000003 2024-09-18T11:30:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138324+0000", 2024-09-18T11:30:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8639000000000002e-05 2024-09-18T11:30:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138622+0000", 2024-09-18T11:30:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029774100000000002 2024-09-18T11:30:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1035 4.a 4:588debc2:::smithi11939506-18:head [read 0~3008420,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.700693+0000", 2024-09-18T11:30:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.748343148, 2024-09-18T11:30:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043653629999999997, 2024-09-18T11:30:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1035 2024-09-18T11:30:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700693+0000", 2024-09-18T11:30:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700693+0000", 2024-09-18T11:30:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700692+0000", 2024-09-18T11:30:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700695+0000", 2024-09-18T11:30:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3e-06 2024-09-18T11:30:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700698+0000", 2024-09-18T11:30:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6799999999999999e-06 2024-09-18T11:30:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700704+0000", 2024-09-18T11:30:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7010000000000001e-06 2024-09-18T11:30:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.703282+0000", 2024-09-18T11:30:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002577686 2024-09-18T11:30:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.703363+0000", 2024-09-18T11:30:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0962000000000002e-05 2024-09-18T11:30:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.705058+0000", 2024-09-18T11:30:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016953179999999999 2024-09-18T11:30:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1036 4.a 4:588debc2:::smithi11939506-18:head [sparse-read 0~1476819] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.700715+0000", 2024-09-18T11:30:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.748320788000001, 2024-09-18T11:30:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0045971959999999996, 2024-09-18T11:30:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1036 2024-09-18T11:30:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700715+0000", 2024-09-18T11:30:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700715+0000", 2024-09-18T11:30:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700715+0000", 2024-09-18T11:30:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700717+0000", 2024-09-18T11:30:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9530000000000002e-06 2024-09-18T11:30:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700721+0000", 2024-09-18T11:30:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9729999999999999e-06 2024-09-18T11:30:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700723+0000", 2024-09-18T11:30:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.153e-06 2024-09-18T11:30:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.705107+0000", 2024-09-18T11:30:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043840459999999999 2024-09-18T11:30:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.705156+0000", 2024-09-18T11:30:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9378000000000001e-05 2024-09-18T11:30:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.705312+0000", 2024-09-18T11:30:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015603200000000001 2024-09-18T11:30:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1108 4.a 4:552523b6:::smithi11939506-38:head [read 0~2621859,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.851579+0000", 2024-09-18T11:30:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.597456933, 2024-09-18T11:30:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0033177340000000001, 2024-09-18T11:30:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1108 2024-09-18T11:30:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851579+0000", 2024-09-18T11:30:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851579+0000", 2024-09-18T11:30:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851579+0000", 2024-09-18T11:30:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851581+0000", 2024-09-18T11:30:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.48e-06 2024-09-18T11:30:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851585+0000", 2024-09-18T11:30:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0860000000000004e-06 2024-09-18T11:30:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851589+0000", 2024-09-18T11:30:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.968e-06 2024-09-18T11:30:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.853620+0000", 2024-09-18T11:30:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020308750000000001 2024-09-18T11:30:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.853668+0000", 2024-09-18T11:30:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8544999999999997e-05 2024-09-18T11:30:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.854897+0000", 2024-09-18T11:30:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001228125 2024-09-18T11:30:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1109 4.a 4:552523b6:::smithi11939506-38:head [read 0~2621859,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.851616+0000", 2024-09-18T11:30:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.597420143000001, 2024-09-18T11:30:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041528579999999997, 2024-09-18T11:30:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1109 2024-09-18T11:30:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851616+0000", 2024-09-18T11:30:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851616+0000", 2024-09-18T11:30:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851615+0000", 2024-09-18T11:30:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851617+0000", 2024-09-18T11:30:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0020000000000001e-06 2024-09-18T11:30:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851624+0000", 2024-09-18T11:30:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5710000000000003e-06 2024-09-18T11:30:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.851631+0000", 2024-09-18T11:30:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0659999999999998e-06 2024-09-18T11:30:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.854937+0000", 2024-09-18T11:30:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003306284 2024-09-18T11:30:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.855040+0000", 2024-09-18T11:30:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010267099999999999 2024-09-18T11:30:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.855769+0000", 2024-09-18T11:30:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00072857200000000003 2024-09-18T11:30:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1130 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2605052,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.178984+0000", 2024-09-18T11:30:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.270052077999999, 2024-09-18T11:30:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041647380000000003, 2024-09-18T11:30:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1130 2024-09-18T11:30:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178984+0000", 2024-09-18T11:30:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178984+0000", 2024-09-18T11:30:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178981+0000", 2024-09-18T11:30:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178998+0000", 2024-09-18T11:30:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7493e-05 2024-09-18T11:30:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179032+0000", 2024-09-18T11:30:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3562000000000002e-05 2024-09-18T11:30:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179048+0000", 2024-09-18T11:30:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6552999999999999e-05 2024-09-18T11:30:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179175+0000", 2024-09-18T11:30:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012698399999999999 2024-09-18T11:30:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179193+0000", 2024-09-18T11:30:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7558999999999999e-05 2024-09-18T11:30:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.181969+0000", 2024-09-18T11:30:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027756479999999999 2024-09-18T11:30:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.182096+0000", 2024-09-18T11:30:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012708700000000001 2024-09-18T11:30:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.183148+0000", 2024-09-18T11:30:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0010528250000000001 2024-09-18T11:30:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1131 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2605052,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179072+0000", 2024-09-18T11:30:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.269964117000001, 2024-09-18T11:30:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050097329999999997, 2024-09-18T11:30:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1131 2024-09-18T11:30:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179072+0000", 2024-09-18T11:30:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179072+0000", 2024-09-18T11:30:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179070+0000", 2024-09-18T11:30:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179084+0000", 2024-09-18T11:30:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3689999999999999e-05 2024-09-18T11:30:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179105+0000", 2024-09-18T11:30:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0605999999999999e-05 2024-09-18T11:30:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179115+0000", 2024-09-18T11:30:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0862999999999999e-05 2024-09-18T11:30:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179365+0000", 2024-09-18T11:30:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00024911399999999999 2024-09-18T11:30:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179382+0000", 2024-09-18T11:30:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7856999999999999e-05 2024-09-18T11:30:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.183178+0000", 2024-09-18T11:30:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037957780000000001 2024-09-18T11:30:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.183256+0000", 2024-09-18T11:30:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7476000000000002e-05 2024-09-18T11:30:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184081+0000", 2024-09-18T11:30:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00082578199999999995 2024-09-18T11:30:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1132 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2605052,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179179+0000", 2024-09-18T11:30:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.269856772000001, 2024-09-18T11:30:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059816840000000001, 2024-09-18T11:30:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1132 2024-09-18T11:30:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179179+0000", 2024-09-18T11:30:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179179+0000", 2024-09-18T11:30:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179178+0000", 2024-09-18T11:30:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179193+0000", 2024-09-18T11:30:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5446000000000001e-05 2024-09-18T11:30:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179215+0000", 2024-09-18T11:30:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1253000000000001e-05 2024-09-18T11:30:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179227+0000", 2024-09-18T11:30:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2612999999999999e-05 2024-09-18T11:30:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179447+0000", 2024-09-18T11:30:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021972199999999999 2024-09-18T11:30:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179453+0000", 2024-09-18T11:30:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8830000000000003e-06 2024-09-18T11:30:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184113+0000", 2024-09-18T11:30:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046603180000000001 2024-09-18T11:30:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184229+0000", 2024-09-18T11:30:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000115516 2024-09-18T11:30:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185161+0000", 2024-09-18T11:30:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00093216000000000002 2024-09-18T11:30:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:30:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.779691374999999, 2024-09-18T11:30:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:30:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:30:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:30:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:30:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:30:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:30:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:30:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:30:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:30:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:30:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:30:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:30:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:30:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:30:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:30:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:30:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:30:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1173 4.5 4:a087a566:::smithi11939506-46:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.027238+0000", 2024-09-18T11:30:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.42179776, 2024-09-18T11:30:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038968649999999998, 2024-09-18T11:30:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1173 2024-09-18T11:30:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027238+0000", 2024-09-18T11:30:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027238+0000", 2024-09-18T11:30:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027237+0000", 2024-09-18T11:30:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027243+0000", 2024-09-18T11:30:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5439999999999998e-06 2024-09-18T11:30:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027254+0000", 2024-09-18T11:30:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0851e-05 2024-09-18T11:30:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027260+0000", 2024-09-18T11:30:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.212e-06 2024-09-18T11:30:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027289+0000", 2024-09-18T11:30:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9499e-05 2024-09-18T11:30:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.027347+0000", 2024-09-18T11:30:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8244e-05 2024-09-18T11:30:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.028549+0000", 2024-09-18T11:30:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001201543 2024-09-18T11:30:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.029048+0000", 2024-09-18T11:30:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00049897600000000004 2024-09-18T11:30:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.031051+0000", 2024-09-18T11:30:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020031889999999998 2024-09-18T11:30:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.031096+0000", 2024-09-18T11:30:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4959000000000001e-05 2024-09-18T11:30:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.031135+0000", 2024-09-18T11:30:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8769000000000001e-05 2024-09-18T11:30:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1185 4.5 4:af4ba75a:::smithi11939506-37:head [write 729437~527542] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.156648+0000", 2024-09-18T11:30:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.292387451, 2024-09-18T11:30:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057485540000000003, 2024-09-18T11:30:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1185 2024-09-18T11:30:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.156648+0000", 2024-09-18T11:30:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.156648+0000", 2024-09-18T11:30:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.156647+0000", 2024-09-18T11:30:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157166+0000", 2024-09-18T11:30:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00051975999999999997 2024-09-18T11:30:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157183+0000", 2024-09-18T11:30:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.611e-05 2024-09-18T11:30:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157188+0000", 2024-09-18T11:30:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.648e-06 2024-09-18T11:30:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157253+0000", 2024-09-18T11:30:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4503000000000002e-05 2024-09-18T11:30:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157322+0000", 2024-09-18T11:30:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9031999999999999e-05 2024-09-18T11:30:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157437+0000", 2024-09-18T11:30:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011543400000000001 2024-09-18T11:30:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162104+0000", 2024-09-18T11:30:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046668009999999999 2024-09-18T11:30:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162321+0000", 2024-09-18T11:30:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021706699999999999 2024-09-18T11:30:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162355+0000", 2024-09-18T11:30:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3670000000000001e-05 2024-09-18T11:30:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162397+0000", 2024-09-18T11:30:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2168e-05 2024-09-18T11:30:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1186 4.5 4:af4ba75a:::smithi11939506-37:head [write 1925626~643351] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.163566+0000", 2024-09-18T11:30:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.285469865, 2024-09-18T11:30:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051394919999999998, 2024-09-18T11:30:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1186 2024-09-18T11:30:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.163566+0000", 2024-09-18T11:30:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.163566+0000", 2024-09-18T11:30:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.163565+0000", 2024-09-18T11:30:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164172+0000", 2024-09-18T11:30:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060731000000000001 2024-09-18T11:30:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164184+0000", 2024-09-18T11:30:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1202999999999999e-05 2024-09-18T11:30:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164188+0000", 2024-09-18T11:30:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2810000000000002e-06 2024-09-18T11:30:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164212+0000", 2024-09-18T11:30:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4227999999999999e-05 2024-09-18T11:30:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164275+0000", 2024-09-18T11:30:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3264999999999998e-05 2024-09-18T11:30:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164376+0000", 2024-09-18T11:30:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000100821 2024-09-18T11:30:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168331+0000", 2024-09-18T11:30:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039549779999999996 2024-09-18T11:30:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168621+0000", 2024-09-18T11:30:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029003900000000002 2024-09-18T11:30:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168662+0000", 2024-09-18T11:30:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1307999999999998e-05 2024-09-18T11:30:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168705+0000", 2024-09-18T11:30:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2994999999999997e-05 2024-09-18T11:30:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1188 4.5 4:af4ba75a:::smithi11939506-37:head [setxattr _header (56) in=7b,truncate 3529437] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.168793+0000", 2024-09-18T11:30:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.280242631, 2024-09-18T11:30:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040182960000000002, 2024-09-18T11:30:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1188 2024-09-18T11:30:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168793+0000", 2024-09-18T11:30:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168793+0000", 2024-09-18T11:30:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168793+0000", 2024-09-18T11:30:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168797+0000", 2024-09-18T11:30:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5510000000000001e-06 2024-09-18T11:30:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168802+0000", 2024-09-18T11:30:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.023e-06 2024-09-18T11:30:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168806+0000", 2024-09-18T11:30:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4180000000000001e-06 2024-09-18T11:30:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.169485+0000", 2024-09-18T11:30:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00067961699999999998 2024-09-18T11:30:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.169580+0000", 2024-09-18T11:30:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4827000000000005e-05 2024-09-18T11:30:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.169713+0000", 2024-09-18T11:30:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000133285 2024-09-18T11:30:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.170114+0000", 2024-09-18T11:30:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000400381 2024-09-18T11:30:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.172752+0000", 2024-09-18T11:30:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002638793 2024-09-18T11:30:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.172783+0000", 2024-09-18T11:30:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0275999999999999e-05 2024-09-18T11:30:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.172811+0000", 2024-09-18T11:30:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8759999999999999e-05 2024-09-18T11:30:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1189 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.168812+0000", 2024-09-18T11:30:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.280223979999999, 2024-09-18T11:30:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004129377, 2024-09-18T11:30:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1189 2024-09-18T11:30:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168812+0000", 2024-09-18T11:30:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168812+0000", 2024-09-18T11:30:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168812+0000", 2024-09-18T11:30:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168814+0000", 2024-09-18T11:30:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1399999999999998e-06 2024-09-18T11:30:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168820+0000", 2024-09-18T11:30:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6320000000000002e-06 2024-09-18T11:30:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168823+0000", 2024-09-18T11:30:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3599999999999999e-06 2024-09-18T11:30:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.169903+0000", 2024-09-18T11:30:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001080136 2024-09-18T11:30:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.169978+0000", 2024-09-18T11:30:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5298999999999999e-05 2024-09-18T11:30:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.172837+0000", 2024-09-18T11:30:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028588620000000002 2024-09-18T11:30:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.172884+0000", 2024-09-18T11:30:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6767999999999997e-05 2024-09-18T11:30:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.172941+0000", 2024-09-18T11:30:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7515000000000001e-05 2024-09-18T11:30:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1427 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 10=[10,e,c] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T11:30:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:29.402527+0000", 2024-09-18T11:30:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 10.046509271, 2024-09-18T11:30:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0044713139999999997, 2024-09-18T11:30:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1427 2024-09-18T11:30:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402527+0000", 2024-09-18T11:30:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402527+0000", 2024-09-18T11:30:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402525+0000", 2024-09-18T11:30:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402551+0000", 2024-09-18T11:30:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5714000000000001e-05 2024-09-18T11:30:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402568+0000", 2024-09-18T11:30:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7190999999999999e-05 2024-09-18T11:30:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402575+0000", 2024-09-18T11:30:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1189999999999999e-06 2024-09-18T11:30:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402620+0000", 2024-09-18T11:30:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4622000000000001e-05 2024-09-18T11:30:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402626+0000", 2024-09-18T11:30:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:30:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.404472+0000", 2024-09-18T11:30:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018458610000000001 2024-09-18T11:30:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.404539+0000", 2024-09-18T11:30:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6583999999999998e-05 2024-09-18T11:30:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.404716+0000", 2024-09-18T11:30:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017717699999999999 2024-09-18T11:30:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.406903+0000", 2024-09-18T11:30:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021871210000000002 2024-09-18T11:30:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.406925+0000", 2024-09-18T11:30:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1749000000000001e-05 2024-09-18T11:30:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.406954+0000", 2024-09-18T11:30:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9813000000000001e-05 2024-09-18T11:30:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.406998+0000", 2024-09-18T11:30:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3368000000000002e-05 2024-09-18T11:30:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1428 4.5 4:a087a566:::smithi11939506-46:head [rollback c] snapc 10=[10,e,c] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T11:30:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:29.402585+0000", 2024-09-18T11:30:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 10.046451039000001, 2024-09-18T11:30:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046759820000000004, 2024-09-18T11:30:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1428 2024-09-18T11:30:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402585+0000", 2024-09-18T11:30:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402585+0000", 2024-09-18T11:30:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402584+0000", 2024-09-18T11:30:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402597+0000", 2024-09-18T11:30:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2830000000000001e-05 2024-09-18T11:30:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402614+0000", 2024-09-18T11:30:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7000999999999999e-05 2024-09-18T11:30:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402618+0000", 2024-09-18T11:30:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8789999999999996e-06 2024-09-18T11:30:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402678+0000", 2024-09-18T11:30:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0312999999999997e-05 2024-09-18T11:30:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402682+0000", 2024-09-18T11:30:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4139999999999998e-06 2024-09-18T11:30:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.405688+0000", 2024-09-18T11:30:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003005939 2024-09-18T11:30:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.405753+0000", 2024-09-18T11:30:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5659000000000003e-05 2024-09-18T11:30:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.405912+0000", 2024-09-18T11:30:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015856799999999999 2024-09-18T11:30:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.407014+0000", 2024-09-18T11:30:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001101625 2024-09-18T11:30:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.407205+0000", 2024-09-18T11:30:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000190927 2024-09-18T11:30:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.407228+0000", 2024-09-18T11:30:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3850999999999999e-05 2024-09-18T11:30:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.407261+0000", 2024-09-18T11:30:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2376000000000003e-05 2024-09-18T11:30:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1429 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 10=[10,e,c] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T11:30:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:29.402624+0000", 2024-09-18T11:30:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 10.046411459, 2024-09-18T11:30:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0052753469999999997, 2024-09-18T11:30:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1429 2024-09-18T11:30:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402624+0000", 2024-09-18T11:30:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402624+0000", 2024-09-18T11:30:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402624+0000", 2024-09-18T11:30:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402627+0000", 2024-09-18T11:30:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7870000000000002e-06 2024-09-18T11:30:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402632+0000", 2024-09-18T11:30:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9520000000000002e-06 2024-09-18T11:30:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402636+0000", 2024-09-18T11:30:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.809e-06 2024-09-18T11:30:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402696+0000", 2024-09-18T11:30:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9150000000000001e-05 2024-09-18T11:30:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402698+0000", 2024-09-18T11:30:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4880000000000001e-06 2024-09-18T11:30:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.406581+0000", 2024-09-18T11:30:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038825840000000001 2024-09-18T11:30:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.406633+0000", 2024-09-18T11:30:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2602000000000002e-05 2024-09-18T11:30:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.406762+0000", 2024-09-18T11:30:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000128804 2024-09-18T11:30:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.407717+0000", 2024-09-18T11:30:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00095522800000000002 2024-09-18T11:30:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.407853+0000", 2024-09-18T11:30:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000135938 2024-09-18T11:30:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.407879+0000", 2024-09-18T11:30:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5476000000000001e-05 2024-09-18T11:30:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.407900+0000", 2024-09-18T11:30:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0941000000000002e-05 2024-09-18T11:30:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:39.568 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:39.569 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:30:39.569 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:30:39.570 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:39.570 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:39.570 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:30:39.571 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:39.571 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:39.572 INFO:tasks.rados.rados.0.smithi119.stdout:916: finishing write tid 1 to smithi11939506-1 2024-09-18T11:30:39.572 INFO:tasks.rados.rados.0.smithi119.stdout:916: finishing write tid 2 to smithi11939506-1 2024-09-18T11:30:39.572 INFO:tasks.rados.rados.0.smithi119.stdout:916: finishing write tid 3 to smithi11939506-1 2024-09-18T11:30:39.572 INFO:tasks.rados.rados.0.smithi119.stdout:916: finishing write tid 4 to smithi11939506-1 2024-09-18T11:30:39.572 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 70 (ObjNum 134 snap 16 seq_num 134) dirty exists 2024-09-18T11:30:39.573 INFO:tasks.rados.rados.0.smithi119.stdout:916: left oid 1 (ObjNum 134 snap 16 seq_num 134) 2024-09-18T11:30:39.573 INFO:tasks.rados.rados.0.smithi119.stdout:914: done (4 left) 2024-09-18T11:30:39.573 INFO:tasks.rados.rados.0.smithi119.stdout:915: done (3 left) 2024-09-18T11:30:39.573 INFO:tasks.rados.rados.0.smithi119.stdout:916: done (2 left) 2024-09-18T11:30:39.573 INFO:tasks.rados.rados.0.smithi119.stdout:918: done (1 left) 2024-09-18T11:30:39.573 INFO:tasks.rados.rados.0.smithi119.stdout:919: write oid 24 current snap is 16 2024-09-18T11:30:39.573 INFO:tasks.rados.rados.0.smithi119.stdout:919: seq_num 135 ranges {668945=536616,1633422=491363} 2024-09-18T11:30:39.573 INFO:tasks.rados.rados.0.smithi119.stdout:919: writing smithi11939506-24 from 668945 to 1205561 tid 1 2024-09-18T11:30:39.573 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:39.574 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:39.574 INFO:tasks.rados.rados.0.smithi119.stdout:919: writing smithi11939506-24 from 1633422 to 2124785 tid 2 2024-09-18T11:30:39.574 INFO:tasks.rados.rados.0.smithi119.stdout:920: read oid 49 snap 13 2024-09-18T11:30:39.574 INFO:tasks.rados.rados.0.smithi119.stdout:920: expect deleted 2024-09-18T11:30:39.574 INFO:tasks.rados.rados.0.smithi119.stdout:921: rollback oid 46 current snap is 16 2024-09-18T11:30:39.574 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 46 to 12 2024-09-18T11:30:39.574 INFO:tasks.rados.rados.0.smithi119.stdout:917: expect (ObjNum 90 snap 5 seq_num 90) 2024-09-18T11:30:39.575 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:39.575 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:39.575 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:39.576 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:39.576 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:39.568+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:39.579 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:39.579 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:39.594 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:39.594 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:39.670 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:39.668+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:39.708 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:39.708 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:39.716 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:30:39.737 INFO:tasks.rados.rados.0.smithi119.stdout:919: finishing write tid 1 to smithi11939506-24 2024-09-18T11:30:39.737 INFO:tasks.rados.rados.0.smithi119.stdout:921: finishing rollback tid 0 to smithi11939506-46 2024-09-18T11:30:39.738 INFO:tasks.rados.rados.0.smithi119.stdout:919: finishing write tid 2 to smithi11939506-24 2024-09-18T11:30:39.738 INFO:tasks.rados.rados.0.smithi119.stdout:919: finishing write tid 3 to smithi11939506-24 2024-09-18T11:30:39.738 INFO:tasks.rados.rados.0.smithi119.stdout:919: finishing write tid 4 to smithi11939506-24 2024-09-18T11:30:39.738 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 86 (ObjNum 135 snap 16 seq_num 135) dirty exists 2024-09-18T11:30:39.738 INFO:tasks.rados.rados.0.smithi119.stdout:919: left oid 24 (ObjNum 135 snap 16 seq_num 135) 2024-09-18T11:30:39.738 INFO:tasks.rados.rados.0.smithi119.stdout:921: finishing rollback tid 1 to smithi11939506-46 2024-09-18T11:30:39.738 INFO:tasks.rados.rados.0.smithi119.stdout:917: done (3 left) 2024-09-18T11:30:39.738 INFO:tasks.rados.rados.0.smithi119.stdout:919: done (2 left) 2024-09-18T11:30:39.738 INFO:tasks.rados.rados.0.smithi119.stdout:920: done (1 left) 2024-09-18T11:30:39.738 INFO:tasks.rados.rados.0.smithi119.stdout:922: tier_evict oid 1 2024-09-18T11:30:39.738 INFO:tasks.rados.rados.0.smithi119.stdout:922: tier_evict oid 1 snap -1 2024-09-18T11:30:39.738 INFO:tasks.rados.rados.0.smithi119.stdout:921: finishing rollback tid 2 to smithi11939506-46 2024-09-18T11:30:39.738 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 73 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-18T11:30:39.739 INFO:tasks.rados.rados.0.smithi119.stdout:921: done (1 left) 2024-09-18T11:30:39.739 INFO:tasks.rados.rados.0.smithi119.stdout:923: tier_evict oid 42 2024-09-18T11:30:39.739 INFO:tasks.rados.rados.0.smithi119.stdout:923: tier_evict oid 42 snap -1 2024-09-18T11:30:39.739 INFO:tasks.rados.rados.0.smithi119.stdout:924: write oid 13 current snap is 16 2024-09-18T11:30:39.739 INFO:tasks.rados.rados.0.smithi119.stdout:924: seq_num 136 ranges {685752=611865,1975551=310201} 2024-09-18T11:30:39.743 INFO:tasks.rados.rados.0.smithi119.stdout:924: writing smithi11939506-13 from 685752 to 1297617 tid 1 2024-09-18T11:30:39.750 INFO:tasks.rados.rados.0.smithi119.stdout:924: writing smithi11939506-13 from 1975551 to 2285752 tid 2 2024-09-18T11:30:39.750 INFO:tasks.rados.rados.0.smithi119.stdout:925: tier_promote oid 38 2024-09-18T11:30:39.750 INFO:tasks.rados.rados.0.smithi119.stdout:926: read oid 8 snap -1 2024-09-18T11:30:39.750 INFO:tasks.rados.rados.0.smithi119.stdout:926: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-18T11:30:39.750 INFO:tasks.rados.rados.0.smithi119.stdout:927: read oid 44 snap 12 2024-09-18T11:30:39.750 INFO:tasks.rados.rados.0.smithi119.stdout:927: expect (ObjNum 72 snap 3 seq_num 72) 2024-09-18T11:30:39.750 INFO:tasks.rados.rados.0.smithi119.stdout:928: tier_promote oid 4 2024-09-18T11:30:39.750 INFO:tasks.rados.rados.0.smithi119.stdout:922: got (22) Invalid argument 2024-09-18T11:30:39.750 INFO:tasks.rados.rados.0.smithi119.stdout:923: got (22) Invalid argument 2024-09-18T11:30:39.750 INFO:tasks.rados.rados.0.smithi119.stdout:922: done (6 left) 2024-09-18T11:30:39.750 INFO:tasks.rados.rados.0.smithi119.stdout:923: done (5 left) 2024-09-18T11:30:39.751 INFO:tasks.rados.rados.0.smithi119.stdout:929: write (excl) oid 42 current snap is 16 2024-09-18T11:30:39.751 INFO:tasks.rados.rados.0.smithi119.stdout:929: seq_num 137 ranges {702559=403467,1750386=495032} 2024-09-18T11:30:39.756 INFO:tasks.rados.rados.0.smithi119.stdout:929: writing smithi11939506-42 from 702559 to 1106026 tid 1 2024-09-18T11:30:39.763 INFO:tasks.rados.rados.0.smithi119.stdout:929: writing smithi11939506-42 from 1750386 to 2245418 tid 2 2024-09-18T11:30:39.763 INFO:tasks.rados.rados.0.smithi119.stdout:930: read oid 16 snap -1 2024-09-18T11:30:39.763 INFO:tasks.rados.rados.0.smithi119.stdout:930: expect (ObjNum 126 snap 14 seq_num 126) 2024-09-18T11:30:39.763 INFO:tasks.rados.rados.0.smithi119.stdout:931: read oid 45 snap -1 2024-09-18T11:30:39.763 INFO:tasks.rados.rados.0.smithi119.stdout:931: expect (ObjNum 113 snap 11 seq_num 113) 2024-09-18T11:30:39.763 INFO:tasks.rados.rados.0.smithi119.stdout:932: write oid 37 current snap is 16 2024-09-18T11:30:39.763 INFO:tasks.rados.rados.0.smithi119.stdout:932: seq_num 138 ranges {719366=478716,1808868=510498} 2024-09-18T11:30:39.769 INFO:tasks.rados.rados.0.smithi119.stdout:932: writing smithi11939506-37 from 719366 to 1198082 tid 1 2024-09-18T11:30:39.771 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:39.770+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:39.775 INFO:tasks.rados.rados.0.smithi119.stdout:932: writing smithi11939506-37 from 1808868 to 2319366 tid 2 2024-09-18T11:30:39.776 INFO:tasks.rados.rados.0.smithi119.stdout:933: rollback oid 27 current snap is 16 2024-09-18T11:30:39.776 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 27 to 10 2024-09-18T11:30:39.776 INFO:tasks.rados.rados.0.smithi119.stdout:926: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-18T11:30:39.856 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:39.856 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:39.864 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:30:39.904 INFO:tasks.rados.rados.0.smithi119.stdout:928: got (0) Success 2024-09-18T11:30:39.904 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 4 v 23 (ObjNum 110 snap 8 seq_num 110) dirty exists 2024-09-18T11:30:39.904 INFO:tasks.rados.rados.0.smithi119.stdout:926: done (9 left) 2024-09-18T11:30:39.905 INFO:tasks.rados.rados.0.smithi119.stdout:928: done (8 left) 2024-09-18T11:30:39.905 INFO:tasks.rados.rados.0.smithi119.stdout:934: set_chunk oid 34 target oid 2024-09-18T11:30:39.907 INFO:tasks.rados.rados.0.smithi119.stdout:934: set_chunk oid 34 offset: 1162240 length: 113664 target oid cae62e4d4aefd9adf6915fa425f399041ff97322484dcdeffdd3684e60bdc162 offset: 0 snap -1 2024-09-18T11:30:39.907 INFO:tasks.rados.rados.0.smithi119.stdout:935: write (excl) oid 25 current snap is 16 2024-09-18T11:30:39.907 INFO:tasks.rados.rados.0.smithi119.stdout:935: seq_num 139 ranges {736173=553965,1867350=468823} 2024-09-18T11:30:39.908 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:39.906+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:39.914 INFO:tasks.rados.rados.0.smithi119.stdout:935: writing smithi11939506-25 from 736173 to 1290138 tid 1 2024-09-18T11:30:39.920 INFO:tasks.rados.rados.0.smithi119.stdout:935: writing smithi11939506-25 from 1867350 to 2336173 tid 2 2024-09-18T11:30:39.920 INFO:tasks.rados.rados.0.smithi119.stdout:936: read oid 19 snap 13 2024-09-18T11:30:39.920 INFO:tasks.rados.rados.0.smithi119.stdout:936: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-18T11:30:39.920 INFO:tasks.rados.rados.0.smithi119.stdout:937: read oid 6 snap -1 2024-09-18T11:30:39.920 INFO:tasks.rados.rados.0.smithi119.stdout:937: expect (ObjNum 117 snap 11 seq_num 117) 2024-09-18T11:30:39.920 INFO:tasks.rados.rados.0.smithi119.stdout:938: write oid 12 current snap is 16 2024-09-18T11:30:39.920 INFO:tasks.rados.rados.0.smithi119.stdout:924: finishing write tid 1 to smithi11939506-13 2024-09-18T11:30:39.920 INFO:tasks.rados.rados.0.smithi119.stdout:924: finishing write tid 2 to smithi11939506-13 2024-09-18T11:30:39.921 INFO:tasks.rados.rados.0.smithi119.stdout:924: finishing write tid 3 to smithi11939506-13 2024-09-18T11:30:39.921 INFO:tasks.rados.rados.0.smithi119.stdout:924: finishing write tid 4 to smithi11939506-13 2024-09-18T11:30:39.921 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 13 v 99 (ObjNum 136 snap 16 seq_num 136) dirty exists 2024-09-18T11:30:39.921 INFO:tasks.rados.rados.0.smithi119.stdout:924: left oid 13 (ObjNum 136 snap 16 seq_num 136) 2024-09-18T11:30:39.921 INFO:tasks.rados.rados.0.smithi119.stdout:927: expect (ObjNum 72 snap 3 seq_num 72) 2024-09-18T11:30:39.966 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:39.966 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:39.972 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:39.970+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:39.975 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:30:40.059 INFO:tasks.rados.rados.0.smithi119.stdout:925: got (0) Success 2024-09-18T11:30:40.059 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 43 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-18T11:30:40.059 INFO:tasks.rados.rados.0.smithi119.stdout:929: finishing write tid 1 to smithi11939506-42 2024-09-18T11:30:40.059 INFO:tasks.rados.rados.0.smithi119.stdout:938: seq_num 140 ranges {752980=629214,1809479=543501} 2024-09-18T11:30:40.068 INFO:tasks.rados.rados.0.smithi119.stdout:938: writing smithi11939506-12 from 752980 to 1382194 tid 1 2024-09-18T11:30:40.073 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:40.071+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:40.074 INFO:tasks.rados.rados.0.smithi119.stdout:938: writing smithi11939506-12 from 1809479 to 2352980 tid 2 2024-09-18T11:30:40.075 INFO:tasks.rados.rados.0.smithi119.stdout:924: done (12 left) 2024-09-18T11:30:40.075 INFO:tasks.rados.rados.0.smithi119.stdout:925: done (11 left) 2024-09-18T11:30:40.075 INFO:tasks.rados.rados.0.smithi119.stdout:927: done (10 left) 2024-09-18T11:30:40.075 INFO:tasks.rados.rados.0.smithi119.stdout:939: tier_promote oid 28 2024-09-18T11:30:40.075 INFO:tasks.rados.rados.0.smithi119.stdout:930: expect (ObjNum 126 snap 14 seq_num 126) 2024-09-18T11:30:40.080 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:40.080 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:40.089 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:30:40.174 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:40.173+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:40.187 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:40.187 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:40.195 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:30:40.208 INFO:tasks.rados.rados.0.smithi119.stdout:931: expect (ObjNum 113 snap 11 seq_num 113) 2024-09-18T11:30:40.274 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:40.272+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:40.307 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:40.307 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:40.315 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:30:40.376 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:40.374+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:40.413 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:40.414 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:40.422 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:30:40.432 INFO:tasks.rados.rados.0.smithi119.stdout:929: finishing write tid 2 to smithi11939506-42 2024-09-18T11:30:40.432 INFO:tasks.rados.rados.0.smithi119.stdout:929: finishing write tid 3 to smithi11939506-42 2024-09-18T11:30:40.432 INFO:tasks.rados.rados.0.smithi119.stdout:930: done (10 left) 2024-09-18T11:30:40.432 INFO:tasks.rados.rados.0.smithi119.stdout:931: done (9 left) 2024-09-18T11:30:40.432 INFO:tasks.rados.rados.0.smithi119.stdout:940: read oid 14 snap 13 2024-09-18T11:30:40.433 INFO:tasks.rados.rados.0.smithi119.stdout:940: expect (ObjNum 66 snap 0 seq_num 66) 2024-09-18T11:30:40.433 INFO:tasks.rados.rados.0.smithi119.stdout:941: read oid 8 snap -1 2024-09-18T11:30:40.433 INFO:tasks.rados.rados.0.smithi119.stdout:941: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-18T11:30:40.433 INFO:tasks.rados.rados.0.smithi119.stdout:942: snap_create 2024-09-18T11:30:40.433 INFO:tasks.rados.rados.0.smithi119.stdout:929: finishing write tid 4 to smithi11939506-42 2024-09-18T11:30:40.433 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 82 (ObjNum 137 snap 16 seq_num 137) dirty exists 2024-09-18T11:30:40.433 INFO:tasks.rados.rados.0.smithi119.stdout:929: left oid 42 (ObjNum 137 snap 16 seq_num 137) 2024-09-18T11:30:40.433 INFO:tasks.rados.rados.0.smithi119.stdout:932: finishing write tid 1 to smithi11939506-37 2024-09-18T11:30:40.433 INFO:tasks.rados.rados.0.smithi119.stdout:933: finishing rollback tid 0 to smithi11939506-27 2024-09-18T11:30:40.433 INFO:tasks.rados.rados.0.smithi119.stdout:932: finishing write tid 2 to smithi11939506-37 2024-09-18T11:30:40.433 INFO:tasks.rados.rados.0.smithi119.stdout:933: finishing rollback tid 1 to smithi11939506-27 2024-09-18T11:30:40.433 INFO:tasks.rados.rados.0.smithi119.stdout:933: finishing rollback tid 2 to smithi11939506-27 2024-09-18T11:30:40.433 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 86 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T11:30:40.434 INFO:tasks.rados.rados.0.smithi119.stdout:932: finishing write tid 3 to smithi11939506-37 2024-09-18T11:30:40.434 INFO:tasks.rados.rados.0.smithi119.stdout:932: finishing write tid 4 to smithi11939506-37 2024-09-18T11:30:40.434 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 77 (ObjNum 138 snap 16 seq_num 138) dirty exists 2024-09-18T11:30:40.434 INFO:tasks.rados.rados.0.smithi119.stdout:932: left oid 37 (ObjNum 138 snap 16 seq_num 138) 2024-09-18T11:30:40.434 INFO:tasks.rados.rados.0.smithi119.stdout:934: finishing set_chunk to oid 34 2024-09-18T11:30:40.434 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 74 (ObjNum 123 snap 12 seq_num 123) dirty exists 2024-09-18T11:30:40.434 INFO:tasks.rados.rados.0.smithi119.stdout:935: finishing write tid 1 to smithi11939506-25 2024-09-18T11:30:40.434 INFO:tasks.rados.rados.0.smithi119.stdout:935: finishing write tid 2 to smithi11939506-25 2024-09-18T11:30:40.434 INFO:tasks.rados.rados.0.smithi119.stdout:935: finishing write tid 3 to smithi11939506-25 2024-09-18T11:30:40.434 INFO:tasks.rados.rados.0.smithi119.stdout:935: finishing write tid 4 to smithi11939506-25 2024-09-18T11:30:40.434 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 10 (ObjNum 139 snap 16 seq_num 139) dirty exists 2024-09-18T11:30:40.434 INFO:tasks.rados.rados.0.smithi119.stdout:935: left oid 25 (ObjNum 139 snap 16 seq_num 139) 2024-09-18T11:30:40.434 INFO:tasks.rados.rados.0.smithi119.stdout:936: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-18T11:30:40.477 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:40.476+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:40.507 INFO:tasks.rados.rados.0.smithi119.stdout:937: expect (ObjNum 117 snap 11 seq_num 117) 2024-09-18T11:30:40.523 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:40.523 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:40.532 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:30:40.616 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:40.615+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:40.670 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:40.671 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:40.679 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:30:40.679 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:40.677+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:40.700 INFO:tasks.rados.rados.0.smithi119.stdout:938: finishing write tid 1 to smithi11939506-12 2024-09-18T11:30:40.700 INFO:tasks.rados.rados.0.smithi119.stdout:939: got (0) Success 2024-09-18T11:30:40.700 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 28 v 52 (ObjNum 122 snap 11 seq_num 122) dirty exists 2024-09-18T11:30:40.700 INFO:tasks.rados.rados.0.smithi119.stdout:938: finishing write tid 2 to smithi11939506-12 2024-09-18T11:30:40.700 INFO:tasks.rados.rados.0.smithi119.stdout:938: finishing write tid 3 to smithi11939506-12 2024-09-18T11:30:40.701 INFO:tasks.rados.rados.0.smithi119.stdout:938: finishing write tid 4 to smithi11939506-12 2024-09-18T11:30:40.701 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 12 v 37 (ObjNum 140 snap 16 seq_num 140) dirty exists 2024-09-18T11:30:40.701 INFO:tasks.rados.rados.0.smithi119.stdout:938: left oid 12 (ObjNum 140 snap 16 seq_num 140) 2024-09-18T11:30:40.701 INFO:tasks.rados.rados.0.smithi119.stdout:941: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-18T11:30:40.780 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:40.779+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:40.799 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:40.800 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:40.808 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:30:40.835 INFO:tasks.rados.rados.0.smithi119.stdout:940: expect (ObjNum 66 snap 0 seq_num 66) 2024-09-18T11:30:40.881 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:40.880+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:40.907 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:40.907 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:40.916 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:30:41.000 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:40.999+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:41.053 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:41.053 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:41.062 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:30:41.083 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:41.082+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:41.184 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:41.183+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:41.200 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:41.201 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:41.210 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:30:41.275 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:41.274+0000 7f2d449e8640 -1 osd.1 58 handle_osd_map injecting map crc failure 2024-09-18T11:30:41.277 INFO:tasks.rados.rados.0.smithi119.stdout:929: done (11 left) 2024-09-18T11:30:41.277 INFO:tasks.rados.rados.0.smithi119.stdout:932: done (10 left) 2024-09-18T11:30:41.277 INFO:tasks.rados.rados.0.smithi119.stdout:933: done (9 left) 2024-09-18T11:30:41.277 INFO:tasks.rados.rados.0.smithi119.stdout:934: done (8 left) 2024-09-18T11:30:41.277 INFO:tasks.rados.rados.0.smithi119.stdout:935: done (7 left) 2024-09-18T11:30:41.277 INFO:tasks.rados.rados.0.smithi119.stdout:936: done (6 left) 2024-09-18T11:30:41.277 INFO:tasks.rados.rados.0.smithi119.stdout:937: done (5 left) 2024-09-18T11:30:41.277 INFO:tasks.rados.rados.0.smithi119.stdout:938: done (4 left) 2024-09-18T11:30:41.278 INFO:tasks.rados.rados.0.smithi119.stdout:939: done (3 left) 2024-09-18T11:30:41.278 INFO:tasks.rados.rados.0.smithi119.stdout:940: done (2 left) 2024-09-18T11:30:41.278 INFO:tasks.rados.rados.0.smithi119.stdout:941: done (1 left) 2024-09-18T11:30:41.278 INFO:tasks.rados.rados.0.smithi119.stdout:942: done (0 left) 2024-09-18T11:30:41.278 INFO:tasks.rados.rados.0.smithi119.stdout:943: snap_create 2024-09-18T11:30:41.280 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T11:30:41.285 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:41.284+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:41.351 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:41.351 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:41.360 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:30:41.386 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:41.384+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:41.461 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:41.461 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:41.470 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:30:41.486 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:41.485+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:41.581 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:41.581 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:41.588 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:41.587+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:41.590 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:30:41.605 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:30:41.621 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:41.619+0000 7fa4676ed640 -1 osd.0 58 handle_osd_map injecting map crc failure 2024-09-18T11:30:41.689 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:41.687+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:41.690 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:41.690 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:41.699 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:30:41.719 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:30:41.720 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:30:41.735 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:41.735 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:41.749 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:41.749 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:41.764 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:41.764 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:41.779 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:41.779 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:41.789 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:41.788+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:41.794 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:41.794 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:41.809 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:41.809 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:41.825 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:41.825 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:41.840 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:41.840 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:41.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:41.840 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:41.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.841 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1029 3.3 3:df268f6c:::8da2985a35cccabf5edc5a8900d2327bcfe6ed768fa38d466fa9a390ce713320:head [write 0~356864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:41.841 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.689033+0000", 2024-09-18T11:30:41.841 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.149222434999999, 2024-09-18T11:30:41.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053345750000000003, 2024-09-18T11:30:41.841 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:41.841 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:41.841 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:41.841 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:41.841 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:41.841 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1029 2024-09-18T11:30:41.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.841 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:41.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:41.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689033+0000", 2024-09-18T11:30:41.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:41.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689033+0000", 2024-09-18T11:30:41.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:41.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.689031+0000", 2024-09-18T11:30:41.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:41.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:41.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690898+0000", 2024-09-18T11:30:41.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001867032 2024-09-18T11:30:41.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:41.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690914+0000", 2024-09-18T11:30:41.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5806000000000001e-05 2024-09-18T11:30:41.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:41.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690921+0000", 2024-09-18T11:30:41.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8859999999999998e-06 2024-09-18T11:30:41.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.690977+0000", 2024-09-18T11:30:41.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5612999999999998e-05 2024-09-18T11:30:41.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:41.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.691142+0000", 2024-09-18T11:30:41.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000165112 2024-09-18T11:30:41.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:41.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.691351+0000", 2024-09-18T11:30:41.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020942700000000001 2024-09-18T11:30:41.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:41.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.692616+0000", 2024-09-18T11:30:41.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012648200000000001 2024-09-18T11:30:41.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:41.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.694303+0000", 2024-09-18T11:30:41.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016875320000000001 2024-09-18T11:30:41.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:41.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.694335+0000", 2024-09-18T11:30:41.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1971000000000003e-05 2024-09-18T11:30:41.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:41.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.694368+0000", 2024-09-18T11:30:41.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2403000000000001e-05 2024-09-18T11:30:41.848 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.848 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:41.848 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.848 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1133 4.1 4:8b9a985d:::smithi11939506-23:a [read 0~3327753,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:41.848 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.178999+0000", 2024-09-18T11:30:41.848 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.659256462999998, 2024-09-18T11:30:41.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059312619999999996, 2024-09-18T11:30:41.848 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:41.848 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:41.849 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:41.849 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:41.849 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:41.849 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1133 2024-09-18T11:30:41.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.849 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:41.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:41.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178999+0000", 2024-09-18T11:30:41.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:41.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178999+0000", 2024-09-18T11:30:41.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:41.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178997+0000", 2024-09-18T11:30:41.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:41.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:41.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179012+0000", 2024-09-18T11:30:41.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5166e-05 2024-09-18T11:30:41.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:41.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179043+0000", 2024-09-18T11:30:41.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0892999999999997e-05 2024-09-18T11:30:41.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:41.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179057+0000", 2024-09-18T11:30:41.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4212e-05 2024-09-18T11:30:41.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179217+0000", 2024-09-18T11:30:41.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016031699999999999 2024-09-18T11:30:41.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:41.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179235+0000", 2024-09-18T11:30:41.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7903000000000001e-05 2024-09-18T11:30:41.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.181282+0000", 2024-09-18T11:30:41.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002046614 2024-09-18T11:30:41.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:41.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.181350+0000", 2024-09-18T11:30:41.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8190999999999998e-05 2024-09-18T11:30:41.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:41.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184930+0000", 2024-09-18T11:30:41.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035805960000000001 2024-09-18T11:30:41.853 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.853 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:41.853 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.854 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1134 4.1 4:8b9a985d:::smithi11939506-23:a [sparse-read 0~1709094] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:41.854 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:41.854 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.659162682000002, 2024-09-18T11:30:41.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061866400000000002, 2024-09-18T11:30:41.854 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:41.854 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:41.854 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:41.854 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:41.854 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:41.854 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1134 2024-09-18T11:30:41.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.854 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:41.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:41.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:41.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:41.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:41.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:41.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179092+0000", 2024-09-18T11:30:41.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:41.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:41.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179104+0000", 2024-09-18T11:30:41.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2346e-05 2024-09-18T11:30:41.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:41.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179128+0000", 2024-09-18T11:30:41.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4046999999999999e-05 2024-09-18T11:30:41.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:41.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179141+0000", 2024-09-18T11:30:41.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3149000000000001e-05 2024-09-18T11:30:41.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179406+0000", 2024-09-18T11:30:41.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026496100000000003 2024-09-18T11:30:41.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:41.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179431+0000", 2024-09-18T11:30:41.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5242999999999999e-05 2024-09-18T11:30:41.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184981+0000", 2024-09-18T11:30:41.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055495969999999999 2024-09-18T11:30:41.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:41.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185055+0000", 2024-09-18T11:30:41.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4375999999999994e-05 2024-09-18T11:30:41.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:41.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185280+0000", 2024-09-18T11:30:41.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022433800000000001 2024-09-18T11:30:41.864 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.864 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:41.864 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.864 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1135 4.1 4:8b9a985d:::smithi11939506-23:a [sparse-read 0~1709094] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:41.864 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179201+0000", 2024-09-18T11:30:41.865 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.659054234999999, 2024-09-18T11:30:41.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063123149999999998, 2024-09-18T11:30:41.865 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:41.865 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:41.865 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:41.865 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:41.865 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:41.866 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1135 2024-09-18T11:30:41.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.866 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:41.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:41.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179201+0000", 2024-09-18T11:30:41.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:41.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179201+0000", 2024-09-18T11:30:41.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:41.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179200+0000", 2024-09-18T11:30:41.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:41.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:41.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179207+0000", 2024-09-18T11:30:41.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1960000000000002e-06 2024-09-18T11:30:41.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:41.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179229+0000", 2024-09-18T11:30:41.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1322999999999999e-05 2024-09-18T11:30:41.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:41.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179240+0000", 2024-09-18T11:30:41.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1793e-05 2024-09-18T11:30:41.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179460+0000", 2024-09-18T11:30:41.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000219407 2024-09-18T11:30:41.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:41.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179466+0000", 2024-09-18T11:30:41.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7490000000000004e-06 2024-09-18T11:30:41.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185300+0000", 2024-09-18T11:30:41.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058347490000000002 2024-09-18T11:30:41.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:41.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185345+0000", 2024-09-18T11:30:41.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4316000000000001e-05 2024-09-18T11:30:41.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:41.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185514+0000", 2024-09-18T11:30:41.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016905800000000001 2024-09-18T11:30:41.874 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.874 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:41.874 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.875 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1378 4.d 4:b81644f2:::smithi11939506-34:head [write 615156~555834,stat] snapc c=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-18T11:30:41.876 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:25.201447+0000", 2024-09-18T11:30:41.876 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 16.636808681000002, 2024-09-18T11:30:41.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060023309999999996, 2024-09-18T11:30:41.876 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:41.876 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:41.876 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:41.876 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:41.877 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:41.877 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1378 2024-09-18T11:30:41.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.877 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:41.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:41.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.201447+0000", 2024-09-18T11:30:41.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:41.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.201447+0000", 2024-09-18T11:30:41.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:41.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.201445+0000", 2024-09-18T11:30:41.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:41.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:41.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202027+0000", 2024-09-18T11:30:41.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00058162700000000003 2024-09-18T11:30:41.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:41.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202044+0000", 2024-09-18T11:30:41.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6685e-05 2024-09-18T11:30:41.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:41.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202051+0000", 2024-09-18T11:30:41.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0890000000000002e-06 2024-09-18T11:30:41.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202086+0000", 2024-09-18T11:30:41.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5593999999999998e-05 2024-09-18T11:30:41.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:41.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202153+0000", 2024-09-18T11:30:41.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7237999999999996e-05 2024-09-18T11:30:41.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:41.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202277+0000", 2024-09-18T11:30:41.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012383899999999999 2024-09-18T11:30:41.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:41.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.205380+0000", 2024-09-18T11:30:41.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031026109999999999 2024-09-18T11:30:41.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:41.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.207341+0000", 2024-09-18T11:30:41.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019606419999999999 2024-09-18T11:30:41.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:41.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.207403+0000", 2024-09-18T11:30:41.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2921000000000003e-05 2024-09-18T11:30:41.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:41.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.207449+0000", 2024-09-18T11:30:41.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5824e-05 2024-09-18T11:30:41.883 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.883 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:41.883 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.883 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1421 4.d 4:b5a463bf:::smithi11939506-20:head [write 665577~781581] snapc 10=[10,e,c] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T11:30:41.883 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:29.216974+0000", 2024-09-18T11:30:41.883 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.621282051, 2024-09-18T11:30:41.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006459869, 2024-09-18T11:30:41.883 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:41.883 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:41.883 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:41.884 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:41.884 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:41.884 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1421 2024-09-18T11:30:41.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.884 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:41.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:41.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.216974+0000", 2024-09-18T11:30:41.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:41.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.216974+0000", 2024-09-18T11:30:41.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:41.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.216972+0000", 2024-09-18T11:30:41.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:41.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:41.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217766+0000", 2024-09-18T11:30:41.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079344699999999997 2024-09-18T11:30:41.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:41.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217780+0000", 2024-09-18T11:30:41.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.428e-05 2024-09-18T11:30:41.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:41.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217786+0000", 2024-09-18T11:30:41.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9050000000000002e-06 2024-09-18T11:30:41.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217828+0000", 2024-09-18T11:30:41.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1839999999999999e-05 2024-09-18T11:30:41.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:41.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217897+0000", 2024-09-18T11:30:41.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9663e-05 2024-09-18T11:30:41.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:41.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.218013+0000", 2024-09-18T11:30:41.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011509 2024-09-18T11:30:41.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:41.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.220250+0000", 2024-09-18T11:30:41.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022370150000000002 2024-09-18T11:30:41.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:41.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.223359+0000", 2024-09-18T11:30:41.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031092519999999998 2024-09-18T11:30:41.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:41.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.223396+0000", 2024-09-18T11:30:41.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7531999999999999e-05 2024-09-18T11:30:41.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:41.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.223433+0000", 2024-09-18T11:30:41.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7098999999999998e-05 2024-09-18T11:30:41.892 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.892 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:41.893 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.893 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1528 3.1 e53/21 3:97025964:::d85d35018f93ce704e0c86e65d6c262758d2b38883b05fc1a96898c5a961f23a:head v 53'86, mlcod=51'84)", 2024-09-18T11:30:41.893 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.377855+0000", 2024-09-18T11:30:41.893 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.4604001469999996, 2024-09-18T11:30:41.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058274199999999998, 2024-09-18T11:30:41.893 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:41.893 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:41.893 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:41.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:41.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.377855+0000", 2024-09-18T11:30:41.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:41.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.377855+0000", 2024-09-18T11:30:41.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:41.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.377853+0000", 2024-09-18T11:30:41.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:41.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:41.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.379683+0000", 2024-09-18T11:30:41.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018298139999999999 2024-09-18T11:30:41.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:41.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.379698+0000", 2024-09-18T11:30:41.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4620000000000001e-05 2024-09-18T11:30:41.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:41.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.379722+0000", 2024-09-18T11:30:41.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4505999999999999e-05 2024-09-18T11:30:41.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.379796+0000", 2024-09-18T11:30:41.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3527000000000003e-05 2024-09-18T11:30:41.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:41.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.379825+0000", 2024-09-18T11:30:41.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.953e-05 2024-09-18T11:30:41.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:41.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.383640+0000", 2024-09-18T11:30:41.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038143629999999999 2024-09-18T11:30:41.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:41.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.383683+0000", 2024-09-18T11:30:41.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3053000000000003e-05 2024-09-18T11:30:41.897 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.897 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:41.897 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.897 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1530 4.d 4:b81644f2:::smithi11939506-34:head [write 467261~717275] snapc 14=[14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:30:41.898 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:30:41.898 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.4475711029999996, 2024-09-18T11:30:41.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075660950000000001, 2024-09-18T11:30:41.898 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:41.898 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:41.898 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:41.898 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:41.898 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:41.898 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1530 2024-09-18T11:30:41.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.898 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:41.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:41.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:30:41.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:41.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:30:41.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:41.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390682+0000", 2024-09-18T11:30:41.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:41.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:41.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391554+0000", 2024-09-18T11:30:41.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00087220600000000004 2024-09-18T11:30:41.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:41.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391570+0000", 2024-09-18T11:30:41.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6002999999999999e-05 2024-09-18T11:30:41.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:41.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391575+0000", 2024-09-18T11:30:41.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2549999999999997e-06 2024-09-18T11:30:41.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391631+0000", 2024-09-18T11:30:41.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5671000000000003e-05 2024-09-18T11:30:41.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:41.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391751+0000", 2024-09-18T11:30:41.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000119821 2024-09-18T11:30:41.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:41.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391974+0000", 2024-09-18T11:30:41.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022309900000000001 2024-09-18T11:30:41.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:41.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398136+0000", 2024-09-18T11:30:41.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006161573 2024-09-18T11:30:41.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:41.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398184+0000", 2024-09-18T11:30:41.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8458000000000003e-05 2024-09-18T11:30:41.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:41.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398219+0000", 2024-09-18T11:30:41.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4573999999999998e-05 2024-09-18T11:30:41.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:41.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398251+0000", 2024-09-18T11:30:41.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2085000000000002e-05 2024-09-18T11:30:41.905 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.905 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:41.905 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.905 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1532 4.d 4:b81644f2:::smithi11939506-34:head [setxattr _header (57) in=7b,truncate 2067261] snapc 14=[14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:30:41.905 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.395599+0000", 2024-09-18T11:30:41.905 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.4426561859999998, 2024-09-18T11:30:41.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054503360000000001, 2024-09-18T11:30:41.906 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:41.906 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:41.906 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:41.906 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:41.906 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:41.906 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1532 2024-09-18T11:30:41.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.906 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:41.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:41.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395599+0000", 2024-09-18T11:30:41.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:41.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395599+0000", 2024-09-18T11:30:41.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:41.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395599+0000", 2024-09-18T11:30:41.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:41.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:41.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395603+0000", 2024-09-18T11:30:41.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3640000000000001e-06 2024-09-18T11:30:41.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:41.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395610+0000", 2024-09-18T11:30:41.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9090000000000003e-06 2024-09-18T11:30:41.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:41.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395615+0000", 2024-09-18T11:30:41.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6269999999999999e-06 2024-09-18T11:30:41.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.397243+0000", 2024-09-18T11:30:41.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016279350000000001 2024-09-18T11:30:41.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:41.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.397343+0000", 2024-09-18T11:30:41.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000100054 2024-09-18T11:30:41.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:41.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.397496+0000", 2024-09-18T11:30:41.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000153578 2024-09-18T11:30:41.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:41.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398472+0000", 2024-09-18T11:30:41.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00097554100000000002 2024-09-18T11:30:41.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:41.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.400978+0000", 2024-09-18T11:30:41.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025066590000000001 2024-09-18T11:30:41.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:41.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.401013+0000", 2024-09-18T11:30:41.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4357999999999999e-05 2024-09-18T11:30:41.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:41.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.401050+0000", 2024-09-18T11:30:41.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7052999999999999e-05 2024-09-18T11:30:41.912 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.912 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:41.913 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.913 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1533 4.d 4:b81644f2:::smithi11939506-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:30:41.913 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.395621+0000", 2024-09-18T11:30:41.914 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.4426347110000002, 2024-09-18T11:30:41.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055874380000000001, 2024-09-18T11:30:41.914 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:41.914 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:41.914 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:41.914 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:41.914 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:41.915 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1533 2024-09-18T11:30:41.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.915 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:41.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:41.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395621+0000", 2024-09-18T11:30:41.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:41.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395621+0000", 2024-09-18T11:30:41.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:41.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395621+0000", 2024-09-18T11:30:41.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:41.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:41.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395623+0000", 2024-09-18T11:30:41.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.841e-06 2024-09-18T11:30:41.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:41.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395630+0000", 2024-09-18T11:30:41.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.285e-06 2024-09-18T11:30:41.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:41.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.395633+0000", 2024-09-18T11:30:41.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.614e-06 2024-09-18T11:30:41.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398334+0000", 2024-09-18T11:30:41.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027003460000000002 2024-09-18T11:30:41.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:41.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398422+0000", 2024-09-18T11:30:41.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8506999999999998e-05 2024-09-18T11:30:41.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.401087+0000", 2024-09-18T11:30:41.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026650900000000002 2024-09-18T11:30:41.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:41.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.401138+0000", 2024-09-18T11:30:41.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0566999999999999e-05 2024-09-18T11:30:41.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:41.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.401208+0000", 2024-09-18T11:30:41.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0603000000000001e-05 2024-09-18T11:30:41.923 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.923 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:41.923 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.924 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:30:41.924 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:30:41.924 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.4221687789999997, 2024-09-18T11:30:41.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:30:41.924 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:41.924 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:41.925 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:41.925 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:41.925 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:41.925 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:30:41.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.926 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:41.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:41.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:30:41.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:41.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:30:41.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:41.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:30:41.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:41.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:41.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:30:41.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:30:41.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:41.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:30:41.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:30:41.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:41.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:30:41.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:30:41.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:30:41.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:30:41.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:41.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:30:41.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:30:41.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:41.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:30:41.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:30:41.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:41.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:30:41.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:30:41.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:41.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:30:41.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:30:41.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:41.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:30:41.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:30:41.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:41.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:30:41.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:30:41.935 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.936 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:41.936 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.936 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1614 4.e 4:73745217:::smithi11939506-6:head [read 0~3966419,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:41.936 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.294355+0000", 2024-09-18T11:30:41.936 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.5439005369999999, 2024-09-18T11:30:41.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057731620000000001, 2024-09-18T11:30:41.936 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:41.936 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:41.936 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:41.936 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:41.936 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:41.937 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1614 2024-09-18T11:30:41.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.937 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:41.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:41.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294355+0000", 2024-09-18T11:30:41.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:41.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294355+0000", 2024-09-18T11:30:41.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:41.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294355+0000", 2024-09-18T11:30:41.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:41.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:41.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294359+0000", 2024-09-18T11:30:41.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5199999999999999e-06 2024-09-18T11:30:41.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:41.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294367+0000", 2024-09-18T11:30:41.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8189999999999992e-06 2024-09-18T11:30:41.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:41.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294371+0000", 2024-09-18T11:30:41.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3490000000000002e-06 2024-09-18T11:30:41.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.297333+0000", 2024-09-18T11:30:41.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029618800000000001 2024-09-18T11:30:41.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:41.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.297431+0000", 2024-09-18T11:30:41.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8042999999999996e-05 2024-09-18T11:30:41.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:41.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.300128+0000", 2024-09-18T11:30:41.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026971180000000001 2024-09-18T11:30:41.940 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.940 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:41.940 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.940 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1615 4.e 4:73745217:::smithi11939506-6:head [sparse-read 0~1834156] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:41.940 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.294431+0000", 2024-09-18T11:30:41.940 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.5438247360000004, 2024-09-18T11:30:41.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060124970000000003, 2024-09-18T11:30:41.941 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:41.941 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:41.941 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:41.941 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:41.941 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:41.941 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1615 2024-09-18T11:30:41.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.941 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:41.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:41.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294431+0000", 2024-09-18T11:30:41.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:41.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294431+0000", 2024-09-18T11:30:41.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:41.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294430+0000", 2024-09-18T11:30:41.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:41.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:41.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294439+0000", 2024-09-18T11:30:41.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9129999999999992e-06 2024-09-18T11:30:41.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:41.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294454+0000", 2024-09-18T11:30:41.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.524e-05 2024-09-18T11:30:41.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:41.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294460+0000", 2024-09-18T11:30:41.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6810000000000003e-06 2024-09-18T11:30:41.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.300155+0000", 2024-09-18T11:30:41.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056949779999999998 2024-09-18T11:30:41.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:41.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.300230+0000", 2024-09-18T11:30:41.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5593000000000006e-05 2024-09-18T11:30:41.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:41.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.300443+0000", 2024-09-18T11:30:41.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021289599999999999 2024-09-18T11:30:41.944 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.944 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:41.944 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.945 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1628 3.3 3:f80aa31b:::77cdbf3abbada57d0636b2198046a030673818705cbf13bb77b30c3e68b24345:head [write 0~582144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:41.945 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:30:41.945 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.5272787789999995, 2024-09-18T11:30:41.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081714639999999998, 2024-09-18T11:30:41.945 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:41.945 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:41.945 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:41.945 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:41.945 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:41.946 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1628 2024-09-18T11:30:41.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.946 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:41.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:41.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:30:41.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:41.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:30:41.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:41.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310975+0000", 2024-09-18T11:30:41.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:41.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:41.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312086+0000", 2024-09-18T11:30:41.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001111288 2024-09-18T11:30:41.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:41.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312104+0000", 2024-09-18T11:30:41.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7427000000000001e-05 2024-09-18T11:30:41.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:41.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312110+0000", 2024-09-18T11:30:41.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5320000000000003e-06 2024-09-18T11:30:41.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312179+0000", 2024-09-18T11:30:41.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9196000000000006e-05 2024-09-18T11:30:41.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:41.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312375+0000", 2024-09-18T11:30:41.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00019550899999999999 2024-09-18T11:30:41.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:41.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312558+0000", 2024-09-18T11:30:41.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000182566 2024-09-18T11:30:41.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:41.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.314286+0000", 2024-09-18T11:30:41.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00172831 2024-09-18T11:30:41.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:41.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319079+0000", 2024-09-18T11:30:41.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047929340000000004 2024-09-18T11:30:41.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:41.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319116+0000", 2024-09-18T11:30:41.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6993999999999999e-05 2024-09-18T11:30:41.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:41.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319148+0000", 2024-09-18T11:30:41.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2539999999999997e-05 2024-09-18T11:30:41.951 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.951 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:41.951 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.951 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1662 4.1 4:87078dfe:::smithi11939506-14:head [write 635331~786118,stat] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:41.951 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.360496+0000", 2024-09-18T11:30:41.951 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.4777600340000001, 2024-09-18T11:30:41.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071849519999999997, 2024-09-18T11:30:41.952 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:41.952 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:41.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:41.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:41.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:41.952 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1662 2024-09-18T11:30:41.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.952 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:41.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:41.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.360496+0000", 2024-09-18T11:30:41.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:41.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.360496+0000", 2024-09-18T11:30:41.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:41.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.360493+0000", 2024-09-18T11:30:41.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:41.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:41.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361139+0000", 2024-09-18T11:30:41.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064615400000000002 2024-09-18T11:30:41.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:41.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361154+0000", 2024-09-18T11:30:41.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4970000000000001e-05 2024-09-18T11:30:41.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:41.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361161+0000", 2024-09-18T11:30:41.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4139999999999999e-06 2024-09-18T11:30:41.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361201+0000", 2024-09-18T11:30:41.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0385e-05 2024-09-18T11:30:41.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:41.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361273+0000", 2024-09-18T11:30:41.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2224999999999994e-05 2024-09-18T11:30:41.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:41.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361406+0000", 2024-09-18T11:30:41.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013257699999999999 2024-09-18T11:30:41.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:41.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367415+0000", 2024-09-18T11:30:41.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006009655 2024-09-18T11:30:41.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:41.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367586+0000", 2024-09-18T11:30:41.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017109800000000001 2024-09-18T11:30:41.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:41.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367636+0000", 2024-09-18T11:30:41.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9721e-05 2024-09-18T11:30:41.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:41.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367681+0000", 2024-09-18T11:30:41.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4382999999999998e-05 2024-09-18T11:30:41.958 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.958 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:41.958 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.958 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1663 4.1 4:87078dfe:::smithi11939506-14:head [write 1916458~318873] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:41.958 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:30:41.958 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.4743322159999996, 2024-09-18T11:30:41.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079505749999999997, 2024-09-18T11:30:41.958 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:41.958 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:41.958 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:41.958 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:41.959 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:41.959 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1663 2024-09-18T11:30:41.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.959 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:41.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:41.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:30:41.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:41.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:30:41.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:41.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363922+0000", 2024-09-18T11:30:41.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:41.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:41.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364211+0000", 2024-09-18T11:30:41.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000288385 2024-09-18T11:30:41.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:41.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364223+0000", 2024-09-18T11:30:41.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2582000000000001e-05 2024-09-18T11:30:41.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:41.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364228+0000", 2024-09-18T11:30:41.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0000000000000004e-06 2024-09-18T11:30:41.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364268+0000", 2024-09-18T11:30:41.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0235999999999999e-05 2024-09-18T11:30:41.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:41.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364341+0000", 2024-09-18T11:30:41.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2266000000000002e-05 2024-09-18T11:30:41.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:41.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364444+0000", 2024-09-18T11:30:41.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103632 2024-09-18T11:30:41.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:41.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369461+0000", 2024-09-18T11:30:41.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005016848 2024-09-18T11:30:41.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:41.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371820+0000", 2024-09-18T11:30:41.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023586169999999999 2024-09-18T11:30:41.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:41.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371851+0000", 2024-09-18T11:30:41.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1297999999999997e-05 2024-09-18T11:30:41.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:41.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371874+0000", 2024-09-18T11:30:41.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2841e-05 2024-09-18T11:30:41.964 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.964 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:41.964 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.964 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1664 4.1 4:87078dfe:::smithi11939506-14:head [setxattr _header (57) in=7b,truncate 2235331] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:41.964 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:30:41.964 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.4740185590000001, 2024-09-18T11:30:41.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010368235999999999, 2024-09-18T11:30:41.964 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:41.964 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:41.964 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:41.964 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:41.965 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:41.965 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1664 2024-09-18T11:30:41.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.965 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:41.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:41.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:30:41.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:41.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:30:41.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:41.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364236+0000", 2024-09-18T11:30:41.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:41.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:41.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364240+0000", 2024-09-18T11:30:41.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9829999999999998e-06 2024-09-18T11:30:41.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:41.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364247+0000", 2024-09-18T11:30:41.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2140000000000001e-06 2024-09-18T11:30:41.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:41.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364256+0000", 2024-09-18T11:30:41.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1509999999999993e-06 2024-09-18T11:30:41.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366529+0000", 2024-09-18T11:30:41.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022731409999999998 2024-09-18T11:30:41.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:41.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366621+0000", 2024-09-18T11:30:41.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2408e-05 2024-09-18T11:30:41.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:41.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366801+0000", 2024-09-18T11:30:41.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018002200000000001 2024-09-18T11:30:41.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:41.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372155+0000", 2024-09-18T11:30:41.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053531519999999999 2024-09-18T11:30:41.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:41.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374539+0000", 2024-09-18T11:30:41.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023842580000000002 2024-09-18T11:30:41.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:41.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374573+0000", 2024-09-18T11:30:41.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4205999999999997e-05 2024-09-18T11:30:41.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:41.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374605+0000", 2024-09-18T11:30:41.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2283999999999998e-05 2024-09-18T11:30:41.972 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.972 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:41.972 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.973 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1665 4.1 4:87078dfe:::smithi11939506-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:41.973 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:30:41.973 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.4739943850000001, 2024-09-18T11:30:41.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010471897000000001, 2024-09-18T11:30:41.973 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:41.974 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:41.974 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:41.974 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:41.974 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:41.974 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1665 2024-09-18T11:30:41.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.975 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:41.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:41.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:30:41.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:41.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:30:41.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:41.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:30:41.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:41.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:41.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364263+0000", 2024-09-18T11:30:41.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6400000000000001e-06 2024-09-18T11:30:41.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:41.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364270+0000", 2024-09-18T11:30:41.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:30:41.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:41.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364274+0000", 2024-09-18T11:30:41.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.968e-06 2024-09-18T11:30:41.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367354+0000", 2024-09-18T11:30:41.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030803430000000001 2024-09-18T11:30:41.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:41.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367406+0000", 2024-09-18T11:30:41.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2330999999999998e-05 2024-09-18T11:30:41.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374625+0000", 2024-09-18T11:30:41.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072190689999999998 2024-09-18T11:30:41.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:41.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374673+0000", 2024-09-18T11:30:41.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7525999999999999e-05 2024-09-18T11:30:41.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:41.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374733+0000", 2024-09-18T11:30:41.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0359000000000003e-05 2024-09-18T11:30:41.983 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.983 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:41.983 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.984 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:101 4.5 e58/22 4:aa575018:::smithi11939506-11:17 v 58'67, mlcod=55'62)", 2024-09-18T11:30:41.984 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.273235+0000", 2024-09-18T11:30:41.984 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 2.565021094, 2024-09-18T11:30:41.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0067904050000000002, 2024-09-18T11:30:41.984 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:41.984 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:41.984 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:41.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:41.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273235+0000", 2024-09-18T11:30:41.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:41.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273235+0000", 2024-09-18T11:30:41.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:41.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273234+0000", 2024-09-18T11:30:41.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:41.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:41.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273239+0000", 2024-09-18T11:30:41.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7999999999999998e-06 2024-09-18T11:30:41.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:41.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273247+0000", 2024-09-18T11:30:41.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4700000000000002e-06 2024-09-18T11:30:41.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:41.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273251+0000", 2024-09-18T11:30:41.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7500000000000001e-06 2024-09-18T11:30:41.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.274599+0000", 2024-09-18T11:30:41.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013483900000000001 2024-09-18T11:30:41.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:41.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.274623+0000", 2024-09-18T11:30:41.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3499999999999999e-05 2024-09-18T11:30:41.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:41.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.279994+0000", 2024-09-18T11:30:41.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053713889999999999 2024-09-18T11:30:41.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:41.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.280025+0000", 2024-09-18T11:30:41.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0579e-05 2024-09-18T11:30:41.991 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.991 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:41.991 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.992 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:102 4.5 e58/22 4:af4ba75a:::smithi11939506-37:13 v 58'69, mlcod=58'65)", 2024-09-18T11:30:41.992 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.283196+0000", 2024-09-18T11:30:41.992 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 2.5550597389999998, 2024-09-18T11:30:41.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005180367, 2024-09-18T11:30:41.992 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:41.992 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:41.992 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:41.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:41.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.283196+0000", 2024-09-18T11:30:41.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:41.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.283196+0000", 2024-09-18T11:30:41.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:41.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:41.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.283195+0000", 2024-09-18T11:30:41.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:41.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:41.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.283203+0000", 2024-09-18T11:30:41.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4800000000000004e-06 2024-09-18T11:30:41.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:41.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.283214+0000", 2024-09-18T11:30:41.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0995000000000001e-05 2024-09-18T11:30:41.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:41.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.283219+0000", 2024-09-18T11:30:41.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7219999999999996e-06 2024-09-18T11:30:41.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:41.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.283256+0000", 2024-09-18T11:30:41.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6786999999999999e-05 2024-09-18T11:30:41.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:41.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.283274+0000", 2024-09-18T11:30:41.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7762e-05 2024-09-18T11:30:41.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:41.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.288354+0000", 2024-09-18T11:30:41.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050800690000000004 2024-09-18T11:30:41.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:41.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:41.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:41.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.288376+0000", 2024-09-18T11:30:41.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2382999999999999e-05 2024-09-18T11:30:41.996 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.996 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:41.996 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.996 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:41.996 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:41.996 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:41.998 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:30:41.998 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:41.997+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:42.098 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:42.098 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:42.098 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:42.098 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:42.099 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:42.097+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:42.106 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:30:42.199 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:42.197+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:42.206 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:42.206 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:42.206 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:42.206 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:42.206 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:42.215 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:30:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is unset 2024-09-18T11:30:42.282 INFO:tasks.rados.rados.0.smithi119.stdout:943: done (0 left) 2024-09-18T11:30:42.282 INFO:tasks.rados.rados.0.smithi119.stdout:944: read oid 24 snap -1 2024-09-18T11:30:42.282 INFO:tasks.rados.rados.0.smithi119.stdout:944: expect (ObjNum 135 snap 16 seq_num 135) 2024-09-18T11:30:42.282 INFO:tasks.rados.rados.0.smithi119.stdout:945: read oid 27 snap -1 2024-09-18T11:30:42.282 INFO:tasks.rados.rados.0.smithi119.stdout:945: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-18T11:30:42.283 INFO:tasks.rados.rados.0.smithi119.stdout:946: read oid 45 snap 16 2024-09-18T11:30:42.283 INFO:tasks.rados.rados.0.smithi119.stdout:946: expect (ObjNum 113 snap 11 seq_num 113) 2024-09-18T11:30:42.283 INFO:tasks.rados.rados.0.smithi119.stdout:947: set_chunk oid 6 target oid 2024-09-18T11:30:42.294 INFO:tasks.rados.rados.0.smithi119.stdout:947: set_chunk oid 6 offset: 1423360 length: 627200 target oid 554b36e1a8dfdc0f27ed2d1e3c014381bf16adc7985c58ddbe8e14169aa70c76 offset: 0 snap -1 2024-09-18T11:30:42.294 INFO:tasks.rados.rados.0.smithi119.stdout:948: read oid 37 snap -1 2024-09-18T11:30:42.294 INFO:tasks.rados.rados.0.smithi119.stdout:948: expect (ObjNum 138 snap 16 seq_num 138) 2024-09-18T11:30:42.294 INFO:tasks.rados.rados.0.smithi119.stdout:949: write (excl) oid 50 current snap is 18 2024-09-18T11:30:42.294 INFO:tasks.rados.rados.0.smithi119.stdout:949: seq_num 141 ranges {769787=704463,2267961=101826} 2024-09-18T11:30:42.301 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:42.300+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:42.302 INFO:tasks.rados.rados.0.smithi119.stdout:949: writing smithi11939506-50 from 769787 to 1474250 tid 1 2024-09-18T11:30:42.307 INFO:tasks.rados.rados.0.smithi119.stdout:949: writing smithi11939506-50 from 2267961 to 2369787 tid 2 2024-09-18T11:30:42.307 INFO:tasks.rados.rados.0.smithi119.stdout:950: delete oid 47 current snap is 18 2024-09-18T11:30:42.307 INFO:tasks.rados.rados.0.smithi119.stdout:944: expect (ObjNum 135 snap 16 seq_num 135) 2024-09-18T11:30:42.362 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:42.362 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:42.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:42.362 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:42.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.363 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:30:42.363 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:30:42.363 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 52.333624495999999, 2024-09-18T11:30:42.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073601550000000002, 2024-09-18T11:30:42.363 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.363 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.363 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:42.363 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:42.363 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:42.363 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1 2024-09-18T11:30:42.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.363 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:30:42.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:30:42.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026524+0000", 2024-09-18T11:30:42.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999905 2024-09-18T11:30:42.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026541+0000", 2024-09-18T11:30:42.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7070000000000001e-05 2024-09-18T11:30:42.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026554+0000", 2024-09-18T11:30:42.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.288e-05 2024-09-18T11:30:42.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026564+0000", 2024-09-18T11:30:42.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0197000000000001e-05 2024-09-18T11:30:42.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026826+0000", 2024-09-18T11:30:42.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026212400000000003 2024-09-18T11:30:42.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:30:42.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026839+0000", 2024-09-18T11:30:42.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2831e-05 2024-09-18T11:30:42.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028605+0000", 2024-09-18T11:30:42.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001765794 2024-09-18T11:30:42.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028709+0000", 2024-09-18T11:30:42.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104348 2024-09-18T11:30:42.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:42.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028824+0000", 2024-09-18T11:30:42.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011527399999999999 2024-09-18T11:30:42.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:42.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030319+0000", 2024-09-18T11:30:42.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001494439 2024-09-18T11:30:42.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:42.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033805+0000", 2024-09-18T11:30:42.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034857820000000002 2024-09-18T11:30:42.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033856+0000", 2024-09-18T11:30:42.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1687999999999997e-05 2024-09-18T11:30:42.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033894+0000", 2024-09-18T11:30:42.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7481000000000002e-05 2024-09-18T11:30:42.369 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.369 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.369 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.369 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T11:30:42.369 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:30:42.369 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 50.314346065000002, 2024-09-18T11:30:42.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055107869999999996, 2024-09-18T11:30:42.370 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.370 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.370 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:42.370 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:42.370 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:42.370 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 7 2024-09-18T11:30:42.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.370 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:30:42.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:30:42.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045810+0000", 2024-09-18T11:30:42.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:42.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045820+0000", 2024-09-18T11:30:42.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0215e-05 2024-09-18T11:30:42.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045838+0000", 2024-09-18T11:30:42.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7849000000000001e-05 2024-09-18T11:30:42.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045845+0000", 2024-09-18T11:30:42.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4229999999999999e-06 2024-09-18T11:30:42.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045898+0000", 2024-09-18T11:30:42.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2537999999999997e-05 2024-09-18T11:30:42.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:42.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045904+0000", 2024-09-18T11:30:42.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4019999999999999e-06 2024-09-18T11:30:42.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047663+0000", 2024-09-18T11:30:42.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017584079999999999 2024-09-18T11:30:42.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047736+0000", 2024-09-18T11:30:42.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3509000000000004e-05 2024-09-18T11:30:42.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:42.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047857+0000", 2024-09-18T11:30:42.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012037800000000001 2024-09-18T11:30:42.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:42.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.050868+0000", 2024-09-18T11:30:42.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030111420000000001 2024-09-18T11:30:42.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:42.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051239+0000", 2024-09-18T11:30:42.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037141000000000002 2024-09-18T11:30:42.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051288+0000", 2024-09-18T11:30:42.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9051000000000002e-05 2024-09-18T11:30:42.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051323+0000", 2024-09-18T11:30:42.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4545000000000002e-05 2024-09-18T11:30:42.376 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.376 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.376 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.376 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2 4.7 e28/22 4:e73ba68f:::smithi11939506-1:head v 28'2, mlcod=0'0)", 2024-09-18T11:30:42.376 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:08.105343+0000", 2024-09-18T11:30:42.376 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 34.254815290000003, 2024-09-18T11:30:42.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040860050000000002, 2024-09-18T11:30:42.376 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.376 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.376 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105343+0000", 2024-09-18T11:30:42.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105343+0000", 2024-09-18T11:30:42.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105342+0000", 2024-09-18T11:30:42.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:42.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105801+0000", 2024-09-18T11:30:42.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00045840099999999999 2024-09-18T11:30:42.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105813+0000", 2024-09-18T11:30:42.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2461e-05 2024-09-18T11:30:42.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105818+0000", 2024-09-18T11:30:42.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.989e-06 2024-09-18T11:30:42.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105856+0000", 2024-09-18T11:30:42.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7839999999999997e-05 2024-09-18T11:30:42.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.105872+0000", 2024-09-18T11:30:42.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6582000000000001e-05 2024-09-18T11:30:42.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.109389+0000", 2024-09-18T11:30:42.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003516211 2024-09-18T11:30:42.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.109429+0000", 2024-09-18T11:30:42.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0261e-05 2024-09-18T11:30:42.380 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.380 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.380 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.380 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3 4.7 e28/22 4:e73ba68f:::smithi11939506-1:head v 28'3, mlcod=0'0)", 2024-09-18T11:30:42.381 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:08.106522+0000", 2024-09-18T11:30:42.381 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 34.253636274000002, 2024-09-18T11:30:42.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031398509999999999, 2024-09-18T11:30:42.381 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.381 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.381 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106522+0000", 2024-09-18T11:30:42.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106522+0000", 2024-09-18T11:30:42.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106521+0000", 2024-09-18T11:30:42.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:42.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106526+0000", 2024-09-18T11:30:42.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9799999999999998e-06 2024-09-18T11:30:42.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106533+0000", 2024-09-18T11:30:42.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1229999999999997e-06 2024-09-18T11:30:42.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106536+0000", 2024-09-18T11:30:42.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.055e-06 2024-09-18T11:30:42.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106577+0000", 2024-09-18T11:30:42.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0932e-05 2024-09-18T11:30:42.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.106588+0000", 2024-09-18T11:30:42.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1486e-05 2024-09-18T11:30:42.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.109630+0000", 2024-09-18T11:30:42.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030422159999999999 2024-09-18T11:30:42.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:08.109662+0000", 2024-09-18T11:30:42.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1498999999999998e-05 2024-09-18T11:30:42.384 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.385 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.385 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.385 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:905 4.7 e31/22 4:e73ba68f:::smithi11939506-1:head v 31'32, mlcod=30'29)", 2024-09-18T11:30:42.385 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:30:42.385 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.218689854000001, 2024-09-18T11:30:42.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048351990000000001, 2024-09-18T11:30:42.385 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.385 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.385 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:30:42.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:30:42.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:30:42.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:42.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144912+0000", 2024-09-18T11:30:42.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003444 2024-09-18T11:30:42.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144921+0000", 2024-09-18T11:30:42.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1900000000000001e-06 2024-09-18T11:30:42.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144927+0000", 2024-09-18T11:30:42.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4940000000000003e-06 2024-09-18T11:30:42.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144952+0000", 2024-09-18T11:30:42.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5845000000000001e-05 2024-09-18T11:30:42.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144966+0000", 2024-09-18T11:30:42.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.396e-05 2024-09-18T11:30:42.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.146274+0000", 2024-09-18T11:30:42.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013073550000000001 2024-09-18T11:30:42.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.146303+0000", 2024-09-18T11:30:42.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9754e-05 2024-09-18T11:30:42.389 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.389 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.389 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.389 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:911 4.3 e31/22 4:cc46532a:::smithi11939506-33:head v 31'24, mlcod=30'21)", 2024-09-18T11:30:42.389 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.166801+0000", 2024-09-18T11:30:42.389 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.193357528, 2024-09-18T11:30:42.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048144349999999997, 2024-09-18T11:30:42.390 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.390 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.390 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.166801+0000", 2024-09-18T11:30:42.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.166801+0000", 2024-09-18T11:30:42.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.166799+0000", 2024-09-18T11:30:42.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:42.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167413+0000", 2024-09-18T11:30:42.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000613259 2024-09-18T11:30:42.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167431+0000", 2024-09-18T11:30:42.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7906000000000001e-05 2024-09-18T11:30:42.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167438+0000", 2024-09-18T11:30:42.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.883e-06 2024-09-18T11:30:42.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167468+0000", 2024-09-18T11:30:42.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0168e-05 2024-09-18T11:30:42.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167483+0000", 2024-09-18T11:30:42.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4997999999999999e-05 2024-09-18T11:30:42.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.171574+0000", 2024-09-18T11:30:42.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040909589999999999 2024-09-18T11:30:42.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.171615+0000", 2024-09-18T11:30:42.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1371000000000001e-05 2024-09-18T11:30:42.394 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.394 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.394 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.394 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:989 4.3 e35/22 4:ca2b2309:::smithi11939506-44:head v 35'30, mlcod=31'26)", 2024-09-18T11:30:42.394 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.154272+0000", 2024-09-18T11:30:42.394 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.205886374999999, 2024-09-18T11:30:42.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031808090000000002, 2024-09-18T11:30:42.394 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.394 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.394 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.154272+0000", 2024-09-18T11:30:42.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.154272+0000", 2024-09-18T11:30:42.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.154272+0000", 2024-09-18T11:30:42.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:42.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.155085+0000", 2024-09-18T11:30:42.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000813921 2024-09-18T11:30:42.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.155097+0000", 2024-09-18T11:30:42.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1881e-05 2024-09-18T11:30:42.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.155103+0000", 2024-09-18T11:30:42.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6250000000000004e-06 2024-09-18T11:30:42.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.155161+0000", 2024-09-18T11:30:42.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8514999999999998e-05 2024-09-18T11:30:42.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.155185+0000", 2024-09-18T11:30:42.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3841000000000001e-05 2024-09-18T11:30:42.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.157411+0000", 2024-09-18T11:30:42.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022258719999999998 2024-09-18T11:30:42.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.157453+0000", 2024-09-18T11:30:42.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1387999999999997e-05 2024-09-18T11:30:42.398 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.398 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.398 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.398 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:990 4.3 e35/22 4:ca2b2309:::smithi11939506-44:head v 35'31, mlcod=34'28)", 2024-09-18T11:30:42.399 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.158794+0000", 2024-09-18T11:30:42.399 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.201364479999999, 2024-09-18T11:30:42.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042163180000000001, 2024-09-18T11:30:42.399 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.399 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.399 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.158794+0000", 2024-09-18T11:30:42.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.158794+0000", 2024-09-18T11:30:42.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.158793+0000", 2024-09-18T11:30:42.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:42.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159307+0000", 2024-09-18T11:30:42.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00051453999999999996 2024-09-18T11:30:42.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159320+0000", 2024-09-18T11:30:42.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2639e-05 2024-09-18T11:30:42.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159327+0000", 2024-09-18T11:30:42.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3200000000000002e-06 2024-09-18T11:30:42.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159359+0000", 2024-09-18T11:30:42.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2147999999999998e-05 2024-09-18T11:30:42.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159380+0000", 2024-09-18T11:30:42.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0625999999999999e-05 2024-09-18T11:30:42.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.162980+0000", 2024-09-18T11:30:42.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003600125 2024-09-18T11:30:42.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.163010+0000", 2024-09-18T11:30:42.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0020999999999999e-05 2024-09-18T11:30:42.402 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.402 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.403 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.403 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1118 3.2 e38/21 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head v 38'76, mlcod=38'74)", 2024-09-18T11:30:42.403 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:30:42.403 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.412214343999999, 2024-09-18T11:30:42.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006048688, 2024-09-18T11:30:42.403 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.403 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.403 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:30:42.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:30:42.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947941+0000", 2024-09-18T11:30:42.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:42.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949868+0000", 2024-09-18T11:30:42.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019267659999999999 2024-09-18T11:30:42.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949893+0000", 2024-09-18T11:30:42.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5239999999999999e-05 2024-09-18T11:30:42.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949901+0000", 2024-09-18T11:30:42.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8420000000000005e-06 2024-09-18T11:30:42.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949954+0000", 2024-09-18T11:30:42.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3464999999999997e-05 2024-09-18T11:30:42.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949974+0000", 2024-09-18T11:30:42.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9731000000000001e-05 2024-09-18T11:30:42.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953948+0000", 2024-09-18T11:30:42.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039735489999999998 2024-09-18T11:30:42.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953992+0000", 2024-09-18T11:30:42.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4700999999999998e-05 2024-09-18T11:30:42.407 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.407 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.407 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.407 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1139 4.2 e39/22 4:425afe9c:::smithi11939506-29:head v 39'19, mlcod=35'15)", 2024-09-18T11:30:42.407 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.187224+0000", 2024-09-18T11:30:42.407 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.172934067, 2024-09-18T11:30:42.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036968130000000002, 2024-09-18T11:30:42.408 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.408 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.408 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.187224+0000", 2024-09-18T11:30:42.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.187224+0000", 2024-09-18T11:30:42.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.187224+0000", 2024-09-18T11:30:42.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:42.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.188633+0000", 2024-09-18T11:30:42.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001409158 2024-09-18T11:30:42.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.188646+0000", 2024-09-18T11:30:42.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3458999999999999e-05 2024-09-18T11:30:42.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.188653+0000", 2024-09-18T11:30:42.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8569999999999997e-06 2024-09-18T11:30:42.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.188700+0000", 2024-09-18T11:30:42.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7176999999999999e-05 2024-09-18T11:30:42.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.188720+0000", 2024-09-18T11:30:42.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0043000000000001e-05 2024-09-18T11:30:42.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.190874+0000", 2024-09-18T11:30:42.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021537739999999998 2024-09-18T11:30:42.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.190921+0000", 2024-09-18T11:30:42.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6864999999999999e-05 2024-09-18T11:30:42.411 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.412 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.412 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.412 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1165 4.7 e39/22 4:e73ba68f:::smithi11939506-1:head v 39'40, mlcod=34'37)", 2024-09-18T11:30:42.412 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.657838+0000", 2024-09-18T11:30:42.412 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.702319681999999, 2024-09-18T11:30:42.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042136450000000002, 2024-09-18T11:30:42.412 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.412 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.412 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.657838+0000", 2024-09-18T11:30:42.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.657838+0000", 2024-09-18T11:30:42.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.657837+0000", 2024-09-18T11:30:42.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:42.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659431+0000", 2024-09-18T11:30:42.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015942479999999999 2024-09-18T11:30:42.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659450+0000", 2024-09-18T11:30:42.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8909000000000001e-05 2024-09-18T11:30:42.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659457+0000", 2024-09-18T11:30:42.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8390000000000001e-06 2024-09-18T11:30:42.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659500+0000", 2024-09-18T11:30:42.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3101e-05 2024-09-18T11:30:42.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659518+0000", 2024-09-18T11:30:42.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8020999999999999e-05 2024-09-18T11:30:42.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.662024+0000", 2024-09-18T11:30:42.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025059399999999999 2024-09-18T11:30:42.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.662052+0000", 2024-09-18T11:30:42.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8206999999999999e-05 2024-09-18T11:30:42.416 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.416 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.416 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.416 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.13:29 3.2 e39/21 3:5fa8fcfa:::ec59520759824f0488c7a3ca135dacb324de2330e16b7f54053de5719c4e96b6:head v 39'81, mlcod=38'76)", 2024-09-18T11:30:42.416 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:30:42.416 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.678513868, 2024-09-18T11:30:42.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004755858, 2024-09-18T11:30:42.417 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.417 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.417 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:30:42.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:30:42.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:30:42.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:42.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681961+0000", 2024-09-18T11:30:42.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00031696799999999999 2024-09-18T11:30:42.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681969+0000", 2024-09-18T11:30:42.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4800000000000001e-06 2024-09-18T11:30:42.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681973+0000", 2024-09-18T11:30:42.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4819999999999999e-06 2024-09-18T11:30:42.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682007+0000", 2024-09-18T11:30:42.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3961999999999998e-05 2024-09-18T11:30:42.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682033+0000", 2024-09-18T11:30:42.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6103999999999999e-05 2024-09-18T11:30:42.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686375+0000", 2024-09-18T11:30:42.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043422019999999999 2024-09-18T11:30:42.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686400+0000", 2024-09-18T11:30:42.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5265e-05 2024-09-18T11:30:42.420 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.420 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.420 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.421 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.13:30 3.2 e39/21 3:49074e56:::7fc21ffee1ba17f2a9197c61ff60ef0b03804c121c8aeb83ea424b7ac77081e9:head v 39'82, mlcod=39'77)", 2024-09-18T11:30:42.421 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.682532+0000", 2024-09-18T11:30:42.421 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.677625806000002, 2024-09-18T11:30:42.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004072694, 2024-09-18T11:30:42.421 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.421 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.421 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682532+0000", 2024-09-18T11:30:42.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682532+0000", 2024-09-18T11:30:42.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682532+0000", 2024-09-18T11:30:42.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:42.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682681+0000", 2024-09-18T11:30:42.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014942500000000001 2024-09-18T11:30:42.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682691+0000", 2024-09-18T11:30:42.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0125e-05 2024-09-18T11:30:42.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682695+0000", 2024-09-18T11:30:42.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9339999999999999e-06 2024-09-18T11:30:42.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682724+0000", 2024-09-18T11:30:42.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8918e-05 2024-09-18T11:30:42.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682738+0000", 2024-09-18T11:30:42.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3793e-05 2024-09-18T11:30:42.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686590+0000", 2024-09-18T11:30:42.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038524380000000001 2024-09-18T11:30:42.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686605+0000", 2024-09-18T11:30:42.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4827999999999999e-05 2024-09-18T11:30:42.425 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.425 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.425 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.425 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1623 4.3 e56/22 4:c70c3162:::smithi11939506-35:head v 56'43, mlcod=48'40)", 2024-09-18T11:30:42.425 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.302560+0000", 2024-09-18T11:30:42.425 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.057598531, 2024-09-18T11:30:42.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046159060000000003, 2024-09-18T11:30:42.425 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.425 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.425 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.302560+0000", 2024-09-18T11:30:42.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.302560+0000", 2024-09-18T11:30:42.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.302559+0000", 2024-09-18T11:30:42.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:42.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303837+0000", 2024-09-18T11:30:42.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012781089999999999 2024-09-18T11:30:42.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303853+0000", 2024-09-18T11:30:42.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5551999999999999e-05 2024-09-18T11:30:42.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303862+0000", 2024-09-18T11:30:42.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0489999999999993e-06 2024-09-18T11:30:42.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303914+0000", 2024-09-18T11:30:42.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1697999999999999e-05 2024-09-18T11:30:42.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303938+0000", 2024-09-18T11:30:42.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4369e-05 2024-09-18T11:30:42.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.307122+0000", 2024-09-18T11:30:42.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031841780000000002 2024-09-18T11:30:42.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.307176+0000", 2024-09-18T11:30:42.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3400999999999999e-05 2024-09-18T11:30:42.429 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.429 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.429 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.429 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1625 4.3 e56/22 4:c70c3162:::smithi11939506-35:head v 56'45, mlcod=53'41)", 2024-09-18T11:30:42.429 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.309486+0000", 2024-09-18T11:30:42.429 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.0506717889999999, 2024-09-18T11:30:42.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030400660000000001, 2024-09-18T11:30:42.429 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.430 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.430 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.309486+0000", 2024-09-18T11:30:42.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.309486+0000", 2024-09-18T11:30:42.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.309485+0000", 2024-09-18T11:30:42.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:42.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.309498+0000", 2024-09-18T11:30:42.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.313e-05 2024-09-18T11:30:42.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.309510+0000", 2024-09-18T11:30:42.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1972e-05 2024-09-18T11:30:42.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.309517+0000", 2024-09-18T11:30:42.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.058e-06 2024-09-18T11:30:42.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.309569+0000", 2024-09-18T11:30:42.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1737999999999998e-05 2024-09-18T11:30:42.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.309593+0000", 2024-09-18T11:30:42.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4613e-05 2024-09-18T11:30:42.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312472+0000", 2024-09-18T11:30:42.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028784909999999999 2024-09-18T11:30:42.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312526+0000", 2024-09-18T11:30:42.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4432999999999999e-05 2024-09-18T11:30:42.433 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.433 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.433 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.434 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1662 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'73, mlcod=51'69)", 2024-09-18T11:30:42.434 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:30:42.434 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.9986093089999999, 2024-09-18T11:30:42.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059408539999999998, 2024-09-18T11:30:42.434 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.434 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.434 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:30:42.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:30:42.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:30:42.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:42.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361914+0000", 2024-09-18T11:30:42.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036569 2024-09-18T11:30:42.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361926+0000", 2024-09-18T11:30:42.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1960999999999999e-05 2024-09-18T11:30:42.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361937+0000", 2024-09-18T11:30:42.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0876999999999999e-05 2024-09-18T11:30:42.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361986+0000", 2024-09-18T11:30:42.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8652000000000001e-05 2024-09-18T11:30:42.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.362012+0000", 2024-09-18T11:30:42.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5854999999999999e-05 2024-09-18T11:30:42.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367460+0000", 2024-09-18T11:30:42.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054482100000000002 2024-09-18T11:30:42.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367490+0000", 2024-09-18T11:30:42.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9909000000000001e-05 2024-09-18T11:30:42.438 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.438 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.439 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.439 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1663 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'74, mlcod=51'69)", 2024-09-18T11:30:42.439 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364521+0000", 2024-09-18T11:30:42.439 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.9956373110000003, 2024-09-18T11:30:42.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048541399999999998, 2024-09-18T11:30:42.439 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.439 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.439 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364521+0000", 2024-09-18T11:30:42.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364521+0000", 2024-09-18T11:30:42.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364520+0000", 2024-09-18T11:30:42.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:42.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364616+0000", 2024-09-18T11:30:42.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5569999999999995e-05 2024-09-18T11:30:42.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364625+0000", 2024-09-18T11:30:42.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3379999999999994e-06 2024-09-18T11:30:42.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364629+0000", 2024-09-18T11:30:42.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2490000000000003e-06 2024-09-18T11:30:42.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364664+0000", 2024-09-18T11:30:42.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5132999999999997e-05 2024-09-18T11:30:42.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364687+0000", 2024-09-18T11:30:42.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2691999999999999e-05 2024-09-18T11:30:42.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369349+0000", 2024-09-18T11:30:42.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046622950000000003 2024-09-18T11:30:42.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369375+0000", 2024-09-18T11:30:42.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5534e-05 2024-09-18T11:30:42.443 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.443 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.443 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.444 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1664 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'75, mlcod=51'69)", 2024-09-18T11:30:42.444 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:30:42.444 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.9932708019999996, 2024-09-18T11:30:42.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051832229999999998, 2024-09-18T11:30:42.444 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.444 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.444 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:30:42.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:30:42.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:30:42.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:42.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366894+0000", 2024-09-18T11:30:42.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2219999999999996e-06 2024-09-18T11:30:42.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366906+0000", 2024-09-18T11:30:42.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2278e-05 2024-09-18T11:30:42.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366912+0000", 2024-09-18T11:30:42.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2430000000000002e-06 2024-09-18T11:30:42.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366952+0000", 2024-09-18T11:30:42.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9428999999999998e-05 2024-09-18T11:30:42.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366972+0000", 2024-09-18T11:30:42.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0554999999999999e-05 2024-09-18T11:30:42.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372045+0000", 2024-09-18T11:30:42.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050727319999999999 2024-09-18T11:30:42.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372071+0000", 2024-09-18T11:30:42.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5477e-05 2024-09-18T11:30:42.448 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.448 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.448 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.448 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1673 4.7 e57/22 4:e73ba68f:::smithi11939506-1:head v 57'70, mlcod=53'66)", 2024-09-18T11:30:42.448 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:38.836645+0000", 2024-09-18T11:30:42.448 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.5235127880000001, 2024-09-18T11:30:42.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043162260000000003, 2024-09-18T11:30:42.448 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.448 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.448 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:38.836645+0000", 2024-09-18T11:30:42.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:38.836645+0000", 2024-09-18T11:30:42.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:38.836645+0000", 2024-09-18T11:30:42.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:42.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:38.836650+0000", 2024-09-18T11:30:42.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9559999999999996e-06 2024-09-18T11:30:42.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:38.836657+0000", 2024-09-18T11:30:42.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7490000000000001e-06 2024-09-18T11:30:42.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:38.836662+0000", 2024-09-18T11:30:42.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6980000000000003e-06 2024-09-18T11:30:42.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:38.837225+0000", 2024-09-18T11:30:42.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056309700000000003 2024-09-18T11:30:42.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:38.837248+0000", 2024-09-18T11:30:42.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2972e-05 2024-09-18T11:30:42.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:38.840914+0000", 2024-09-18T11:30:42.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036657859999999999 2024-09-18T11:30:42.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:38.840962+0000", 2024-09-18T11:30:42.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7499e-05 2024-09-18T11:30:42.452 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.452 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.452 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.453 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:269 4.7 e58/22 4:e73ba68f:::smithi11939506-1:13 v 58'72, mlcod=57'69)", 2024-09-18T11:30:42.453 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:30:42.453 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.094777412, 2024-09-18T11:30:42.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065512569999999996, 2024-09-18T11:30:42.453 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.453 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.453 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:30:42.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:30:42.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265380+0000", 2024-09-18T11:30:42.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:42.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265388+0000", 2024-09-18T11:30:42.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.174e-06 2024-09-18T11:30:42.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265398+0000", 2024-09-18T11:30:42.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2599999999999994e-06 2024-09-18T11:30:42.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265402+0000", 2024-09-18T11:30:42.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0289999999999999e-06 2024-09-18T11:30:42.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265434+0000", 2024-09-18T11:30:42.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2511000000000001e-05 2024-09-18T11:30:42.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265447+0000", 2024-09-18T11:30:42.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2947e-05 2024-09-18T11:30:42.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271892+0000", 2024-09-18T11:30:42.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064445930000000002 2024-09-18T11:30:42.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271932+0000", 2024-09-18T11:30:42.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0303999999999999e-05 2024-09-18T11:30:42.457 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.457 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.457 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.457 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.457 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.457 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:42.458 INFO:tasks.rados.rados.0.smithi119.stdout:945: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-18T11:30:42.459 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:30:42.459 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:42.456+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:42.526 INFO:tasks.rados.rados.0.smithi119.stdout:946: expect (ObjNum 113 snap 11 seq_num 113) 2024-09-18T11:30:42.559 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:42.557+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:42.599 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:42.599 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:42.599 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:42.599 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:42.608 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:30:42.692 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:42.690+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:42.740 INFO:tasks.rados.rados.0.smithi119.stdout:947: finishing set_chunk to oid 6 2024-09-18T11:30:42.740 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 6 v 26 (ObjNum 117 snap 11 seq_num 117) dirty exists 2024-09-18T11:30:42.740 INFO:tasks.rados.rados.0.smithi119.stdout:948: expect (ObjNum 138 snap 16 seq_num 138) 2024-09-18T11:30:42.746 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:42.746 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:42.746 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:42.746 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:42.746 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:42.755 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:30:42.796 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:42.795+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:42.861 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:42.860+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:42.873 INFO:tasks.rados.rados.0.smithi119.stdout:949: finishing write tid 1 to smithi11939506-50 2024-09-18T11:30:42.873 INFO:tasks.rados.rados.0.smithi119.stdout:949: finishing write tid 2 to smithi11939506-50 2024-09-18T11:30:42.873 INFO:tasks.rados.rados.0.smithi119.stdout:949: finishing write tid 3 to smithi11939506-50 2024-09-18T11:30:42.873 INFO:tasks.rados.rados.0.smithi119.stdout:949: finishing write tid 4 to smithi11939506-50 2024-09-18T11:30:42.873 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 70 (ObjNum 141 snap 18 seq_num 141) dirty exists 2024-09-18T11:30:42.873 INFO:tasks.rados.rados.0.smithi119.stdout:949: left oid 50 (ObjNum 141 snap 18 seq_num 141) 2024-09-18T11:30:42.873 INFO:tasks.rados.rados.0.smithi119.stdout:944: done (6 left) 2024-09-18T11:30:42.873 INFO:tasks.rados.rados.0.smithi119.stdout:945: done (5 left) 2024-09-18T11:30:42.873 INFO:tasks.rados.rados.0.smithi119.stdout:946: done (4 left) 2024-09-18T11:30:42.873 INFO:tasks.rados.rados.0.smithi119.stdout:947: done (3 left) 2024-09-18T11:30:42.873 INFO:tasks.rados.rados.0.smithi119.stdout:948: done (2 left) 2024-09-18T11:30:42.873 INFO:tasks.rados.rados.0.smithi119.stdout:949: done (1 left) 2024-09-18T11:30:42.874 INFO:tasks.rados.rados.0.smithi119.stdout:950: done (0 left) 2024-09-18T11:30:42.874 INFO:tasks.rados.rados.0.smithi119.stdout:951: write (excl) oid 29 current snap is 18 2024-09-18T11:30:42.874 INFO:tasks.rados.rados.0.smithi119.stdout:951: seq_num 143 ranges {403401=454961,1584925=506039} 2024-09-18T11:30:42.877 INFO:tasks.rados.rados.0.smithi119.stdout:951: writing smithi11939506-29 from 403401 to 858362 tid 1 2024-09-18T11:30:42.884 INFO:tasks.rados.rados.0.smithi119.stdout:951: writing smithi11939506-29 from 1584925 to 2090964 tid 2 2024-09-18T11:30:42.884 INFO:tasks.rados.rados.0.smithi119.stdout:952: tier_evict oid 12 2024-09-18T11:30:42.884 INFO:tasks.rados.rados.0.smithi119.stdout:952: tier_evict oid 12 snap -1 2024-09-18T11:30:42.884 INFO:tasks.rados.rados.0.smithi119.stdout:951: finishing write tid 1 to smithi11939506-29 2024-09-18T11:30:42.884 INFO:tasks.rados.rados.0.smithi119.stdout:953: read oid 23 snap -1 2024-09-18T11:30:42.884 INFO:tasks.rados.rados.0.smithi119.stdout:953: expect (ObjNum 90 snap 5 seq_num 90) 2024-09-18T11:30:42.884 INFO:tasks.rados.rados.0.smithi119.stdout:954: write (excl) oid 18 current snap is 18 2024-09-18T11:30:42.885 INFO:tasks.rados.rados.0.smithi119.stdout:954: seq_num 144 ranges {420208=530210,1527054=766050} 2024-09-18T11:30:42.889 INFO:tasks.rados.rados.0.smithi119.stdout:954: writing smithi11939506-18 from 420208 to 950418 tid 1 2024-09-18T11:30:42.896 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:42.897 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:42.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:42.897 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:42.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.897 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:42.897 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:42.897 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.778169451, 2024-09-18T11:30:42.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:30:42.897 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.897 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:42.897 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:42.897 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:42.898 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:42.898 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:30:42.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.898 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:42.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:42.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:30:42.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:42.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:30:42.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:30:42.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:30:42.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:30:42.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:30:42.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:30:42.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:30:42.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:30:42.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:42.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:30:42.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:30:42.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:30:42.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:30:42.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:30:42.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:30:42.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:30:42.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:30:42.902 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.902 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.902 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.902 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:896 4.5 4:aa575018:::smithi11939506-11:2 [read 0~2941192,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:42.902 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.132995+0000", 2024-09-18T11:30:42.902 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.761493810000001, 2024-09-18T11:30:42.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047874989999999997, 2024-09-18T11:30:42.903 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.903 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:42.903 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:42.903 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:42.903 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:42.903 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 896 2024-09-18T11:30:42.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.903 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.132995+0000", 2024-09-18T11:30:42.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.132995+0000", 2024-09-18T11:30:42.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.132993+0000", 2024-09-18T11:30:42.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:42.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133000+0000", 2024-09-18T11:30:42.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5509999999999996e-06 2024-09-18T11:30:42.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133010+0000", 2024-09-18T11:30:42.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0254e-05 2024-09-18T11:30:42.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133023+0000", 2024-09-18T11:30:42.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3458999999999999e-05 2024-09-18T11:30:42.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133069+0000", 2024-09-18T11:30:42.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5324000000000002e-05 2024-09-18T11:30:42.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133161+0000", 2024-09-18T11:30:42.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2773000000000001e-05 2024-09-18T11:30:42.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.137782+0000", 2024-09-18T11:30:42.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046208680000000002 2024-09-18T11:30:42.906 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.907 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.907 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.907 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:897 4.5 4:aa575018:::smithi11939506-11:2 [sparse-read 0~1343176] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:42.907 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.133034+0000", 2024-09-18T11:30:42.907 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.761454450999999, 2024-09-18T11:30:42.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051617950000000003, 2024-09-18T11:30:42.907 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.907 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:42.907 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:42.907 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:42.907 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:42.908 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 897 2024-09-18T11:30:42.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.908 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133034+0000", 2024-09-18T11:30:42.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133034+0000", 2024-09-18T11:30:42.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133033+0000", 2024-09-18T11:30:42.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:42.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133037+0000", 2024-09-18T11:30:42.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.118e-06 2024-09-18T11:30:42.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133045+0000", 2024-09-18T11:30:42.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1780000000000003e-06 2024-09-18T11:30:42.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133049+0000", 2024-09-18T11:30:42.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7730000000000001e-06 2024-09-18T11:30:42.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.137838+0000", 2024-09-18T11:30:42.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00478912 2024-09-18T11:30:42.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.137936+0000", 2024-09-18T11:30:42.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.823e-05 2024-09-18T11:30:42.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138196+0000", 2024-09-18T11:30:42.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00025993799999999998 2024-09-18T11:30:42.915 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.915 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.915 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.915 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:898 4.5 4:aa575018:::smithi11939506-11:2 [sparse-read 0~1343176] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:42.915 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:42.915 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.761425413000001, 2024-09-18T11:30:42.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055588349999999998, 2024-09-18T11:30:42.916 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.916 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:42.916 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:42.916 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:42.916 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:42.916 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 898 2024-09-18T11:30:42.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.916 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:42.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:42.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:42.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:42.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133066+0000", 2024-09-18T11:30:42.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2729999999999998e-06 2024-09-18T11:30:42.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133072+0000", 2024-09-18T11:30:42.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0680000000000002e-06 2024-09-18T11:30:42.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133077+0000", 2024-09-18T11:30:42.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3569999999999997e-06 2024-09-18T11:30:42.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138226+0000", 2024-09-18T11:30:42.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051480980000000003 2024-09-18T11:30:42.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138324+0000", 2024-09-18T11:30:42.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8639000000000002e-05 2024-09-18T11:30:42.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138622+0000", 2024-09-18T11:30:42.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029774100000000002 2024-09-18T11:30:42.919 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.920 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.920 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.920 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1035 4.a 4:588debc2:::smithi11939506-18:head [read 0~3008420,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:42.920 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.700693+0000", 2024-09-18T11:30:42.920 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.193795822999999, 2024-09-18T11:30:42.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043653629999999997, 2024-09-18T11:30:42.920 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.920 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:42.920 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:42.920 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:42.921 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:42.921 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1035 2024-09-18T11:30:42.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.921 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700693+0000", 2024-09-18T11:30:42.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700693+0000", 2024-09-18T11:30:42.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700692+0000", 2024-09-18T11:30:42.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:42.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700695+0000", 2024-09-18T11:30:42.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3e-06 2024-09-18T11:30:42.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700698+0000", 2024-09-18T11:30:42.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6799999999999999e-06 2024-09-18T11:30:42.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700704+0000", 2024-09-18T11:30:42.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7010000000000001e-06 2024-09-18T11:30:42.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.703282+0000", 2024-09-18T11:30:42.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002577686 2024-09-18T11:30:42.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.703363+0000", 2024-09-18T11:30:42.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0962000000000002e-05 2024-09-18T11:30:42.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.705058+0000", 2024-09-18T11:30:42.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016953179999999999 2024-09-18T11:30:42.924 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.924 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.924 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.924 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1036 4.a 4:588debc2:::smithi11939506-18:head [sparse-read 0~1476819] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:42.925 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.700715+0000", 2024-09-18T11:30:42.925 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.193773462999999, 2024-09-18T11:30:42.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0045971959999999996, 2024-09-18T11:30:42.925 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.925 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:42.925 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:42.925 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:42.925 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:42.925 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1036 2024-09-18T11:30:42.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.925 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700715+0000", 2024-09-18T11:30:42.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700715+0000", 2024-09-18T11:30:42.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700715+0000", 2024-09-18T11:30:42.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:42.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700717+0000", 2024-09-18T11:30:42.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9530000000000002e-06 2024-09-18T11:30:42.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700721+0000", 2024-09-18T11:30:42.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9729999999999999e-06 2024-09-18T11:30:42.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.700723+0000", 2024-09-18T11:30:42.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.153e-06 2024-09-18T11:30:42.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.705107+0000", 2024-09-18T11:30:42.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043840459999999999 2024-09-18T11:30:42.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.705156+0000", 2024-09-18T11:30:42.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9378000000000001e-05 2024-09-18T11:30:42.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.705312+0000", 2024-09-18T11:30:42.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015603200000000001 2024-09-18T11:30:42.929 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.929 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.929 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.929 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1130 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2605052,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:42.929 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.178984+0000", 2024-09-18T11:30:42.929 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.715504753000001, 2024-09-18T11:30:42.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041647380000000003, 2024-09-18T11:30:42.929 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.929 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:42.929 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:42.929 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:42.929 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:42.930 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1130 2024-09-18T11:30:42.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.930 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178984+0000", 2024-09-18T11:30:42.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178984+0000", 2024-09-18T11:30:42.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178981+0000", 2024-09-18T11:30:42.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:42.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178998+0000", 2024-09-18T11:30:42.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7493e-05 2024-09-18T11:30:42.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179032+0000", 2024-09-18T11:30:42.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3562000000000002e-05 2024-09-18T11:30:42.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179048+0000", 2024-09-18T11:30:42.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6552999999999999e-05 2024-09-18T11:30:42.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179175+0000", 2024-09-18T11:30:42.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012698399999999999 2024-09-18T11:30:42.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:42.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179193+0000", 2024-09-18T11:30:42.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7558999999999999e-05 2024-09-18T11:30:42.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.181969+0000", 2024-09-18T11:30:42.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027756479999999999 2024-09-18T11:30:42.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.182096+0000", 2024-09-18T11:30:42.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012708700000000001 2024-09-18T11:30:42.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.183148+0000", 2024-09-18T11:30:42.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0010528250000000001 2024-09-18T11:30:42.934 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.934 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.934 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.934 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1131 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2605052,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:42.934 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179072+0000", 2024-09-18T11:30:42.934 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.715416791999999, 2024-09-18T11:30:42.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050097329999999997, 2024-09-18T11:30:42.934 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.934 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:42.935 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:42.935 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:42.935 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:42.935 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1131 2024-09-18T11:30:42.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.935 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179072+0000", 2024-09-18T11:30:42.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179072+0000", 2024-09-18T11:30:42.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179070+0000", 2024-09-18T11:30:42.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:42.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179084+0000", 2024-09-18T11:30:42.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3689999999999999e-05 2024-09-18T11:30:42.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179105+0000", 2024-09-18T11:30:42.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0605999999999999e-05 2024-09-18T11:30:42.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179115+0000", 2024-09-18T11:30:42.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0862999999999999e-05 2024-09-18T11:30:42.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179365+0000", 2024-09-18T11:30:42.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00024911399999999999 2024-09-18T11:30:42.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:42.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179382+0000", 2024-09-18T11:30:42.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7856999999999999e-05 2024-09-18T11:30:42.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.183178+0000", 2024-09-18T11:30:42.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037957780000000001 2024-09-18T11:30:42.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.183256+0000", 2024-09-18T11:30:42.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7476000000000002e-05 2024-09-18T11:30:42.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184081+0000", 2024-09-18T11:30:42.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00082578199999999995 2024-09-18T11:30:42.939 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.939 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.939 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.940 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1132 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2605052,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:42.940 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179179+0000", 2024-09-18T11:30:42.940 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.715309446999999, 2024-09-18T11:30:42.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059816840000000001, 2024-09-18T11:30:42.940 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.940 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:42.940 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:42.940 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:42.940 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:42.940 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1132 2024-09-18T11:30:42.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.941 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179179+0000", 2024-09-18T11:30:42.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179179+0000", 2024-09-18T11:30:42.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179178+0000", 2024-09-18T11:30:42.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:42.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179193+0000", 2024-09-18T11:30:42.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5446000000000001e-05 2024-09-18T11:30:42.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179215+0000", 2024-09-18T11:30:42.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1253000000000001e-05 2024-09-18T11:30:42.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179227+0000", 2024-09-18T11:30:42.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2612999999999999e-05 2024-09-18T11:30:42.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179447+0000", 2024-09-18T11:30:42.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021972199999999999 2024-09-18T11:30:42.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:42.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179453+0000", 2024-09-18T11:30:42.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8830000000000003e-06 2024-09-18T11:30:42.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184113+0000", 2024-09-18T11:30:42.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046603180000000001 2024-09-18T11:30:42.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184229+0000", 2024-09-18T11:30:42.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000115516 2024-09-18T11:30:42.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185161+0000", 2024-09-18T11:30:42.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00093216000000000002 2024-09-18T11:30:42.944 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.945 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.945 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.945 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:42.945 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:30:42.945 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.225144050000001, 2024-09-18T11:30:42.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:30:42.945 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.945 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:42.945 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:42.945 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:42.946 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:42.946 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:30:42.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.946 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:30:42.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:30:42.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:30:42.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:42.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:30:42.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:30:42.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:30:42.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:30:42.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:30:42.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:30:42.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:30:42.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:30:42.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:30:42.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:30:42.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:30:42.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:30:42.949 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.949 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.949 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.950 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1185 4.5 4:af4ba75a:::smithi11939506-37:head [write 729437~527542] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:42.950 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.156648+0000", 2024-09-18T11:30:42.950 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.737840126000002, 2024-09-18T11:30:42.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057485540000000003, 2024-09-18T11:30:42.950 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.950 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.950 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:42.950 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:42.950 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:42.950 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1185 2024-09-18T11:30:42.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.950 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.156648+0000", 2024-09-18T11:30:42.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.156648+0000", 2024-09-18T11:30:42.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.156647+0000", 2024-09-18T11:30:42.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:42.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157166+0000", 2024-09-18T11:30:42.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00051975999999999997 2024-09-18T11:30:42.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157183+0000", 2024-09-18T11:30:42.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.611e-05 2024-09-18T11:30:42.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157188+0000", 2024-09-18T11:30:42.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.648e-06 2024-09-18T11:30:42.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157253+0000", 2024-09-18T11:30:42.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4503000000000002e-05 2024-09-18T11:30:42.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157322+0000", 2024-09-18T11:30:42.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9031999999999999e-05 2024-09-18T11:30:42.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:42.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157437+0000", 2024-09-18T11:30:42.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011543400000000001 2024-09-18T11:30:42.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:42.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162104+0000", 2024-09-18T11:30:42.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046668009999999999 2024-09-18T11:30:42.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:42.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162321+0000", 2024-09-18T11:30:42.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021706699999999999 2024-09-18T11:30:42.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162355+0000", 2024-09-18T11:30:42.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3670000000000001e-05 2024-09-18T11:30:42.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162397+0000", 2024-09-18T11:30:42.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2168e-05 2024-09-18T11:30:42.955 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.956 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.956 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.956 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1186 4.5 4:af4ba75a:::smithi11939506-37:head [write 1925626~643351] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:42.956 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.163566+0000", 2024-09-18T11:30:42.956 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.730922540000002, 2024-09-18T11:30:42.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051394919999999998, 2024-09-18T11:30:42.956 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.956 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.956 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:42.956 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:42.956 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:42.957 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1186 2024-09-18T11:30:42.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.957 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.163566+0000", 2024-09-18T11:30:42.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.163566+0000", 2024-09-18T11:30:42.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.163565+0000", 2024-09-18T11:30:42.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:42.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164172+0000", 2024-09-18T11:30:42.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060731000000000001 2024-09-18T11:30:42.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164184+0000", 2024-09-18T11:30:42.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1202999999999999e-05 2024-09-18T11:30:42.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164188+0000", 2024-09-18T11:30:42.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2810000000000002e-06 2024-09-18T11:30:42.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164212+0000", 2024-09-18T11:30:42.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4227999999999999e-05 2024-09-18T11:30:42.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164275+0000", 2024-09-18T11:30:42.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3264999999999998e-05 2024-09-18T11:30:42.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:42.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164376+0000", 2024-09-18T11:30:42.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000100821 2024-09-18T11:30:42.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:42.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168331+0000", 2024-09-18T11:30:42.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039549779999999996 2024-09-18T11:30:42.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:42.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168621+0000", 2024-09-18T11:30:42.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029003900000000002 2024-09-18T11:30:42.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168662+0000", 2024-09-18T11:30:42.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1307999999999998e-05 2024-09-18T11:30:42.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168705+0000", 2024-09-18T11:30:42.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2994999999999997e-05 2024-09-18T11:30:42.962 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.962 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.962 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.962 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1427 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 10=[10,e,c] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T11:30:42.962 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:29.402527+0000", 2024-09-18T11:30:42.962 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.491961946, 2024-09-18T11:30:42.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0044713139999999997, 2024-09-18T11:30:42.962 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.962 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.962 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:42.962 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:42.963 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:42.963 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1427 2024-09-18T11:30:42.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.963 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402527+0000", 2024-09-18T11:30:42.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402527+0000", 2024-09-18T11:30:42.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402525+0000", 2024-09-18T11:30:42.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:42.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402551+0000", 2024-09-18T11:30:42.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5714000000000001e-05 2024-09-18T11:30:42.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402568+0000", 2024-09-18T11:30:42.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7190999999999999e-05 2024-09-18T11:30:42.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402575+0000", 2024-09-18T11:30:42.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1189999999999999e-06 2024-09-18T11:30:42.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402620+0000", 2024-09-18T11:30:42.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4622000000000001e-05 2024-09-18T11:30:42.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:42.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402626+0000", 2024-09-18T11:30:42.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:30:42.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.404472+0000", 2024-09-18T11:30:42.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018458610000000001 2024-09-18T11:30:42.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.404539+0000", 2024-09-18T11:30:42.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6583999999999998e-05 2024-09-18T11:30:42.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:42.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.404716+0000", 2024-09-18T11:30:42.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017717699999999999 2024-09-18T11:30:42.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:42.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.406903+0000", 2024-09-18T11:30:42.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021871210000000002 2024-09-18T11:30:42.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:42.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.406925+0000", 2024-09-18T11:30:42.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1749000000000001e-05 2024-09-18T11:30:42.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.406954+0000", 2024-09-18T11:30:42.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9813000000000001e-05 2024-09-18T11:30:42.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.406998+0000", 2024-09-18T11:30:42.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3368000000000002e-05 2024-09-18T11:30:42.968 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.969 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.969 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.969 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1428 4.5 4:a087a566:::smithi11939506-46:head [rollback c] snapc 10=[10,e,c] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T11:30:42.969 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:29.402585+0000", 2024-09-18T11:30:42.969 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.491903713999999, 2024-09-18T11:30:42.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046759820000000004, 2024-09-18T11:30:42.969 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.969 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.969 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:42.969 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:42.969 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:42.970 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1428 2024-09-18T11:30:42.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.970 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402585+0000", 2024-09-18T11:30:42.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402585+0000", 2024-09-18T11:30:42.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402584+0000", 2024-09-18T11:30:42.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:42.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402597+0000", 2024-09-18T11:30:42.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2830000000000001e-05 2024-09-18T11:30:42.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402614+0000", 2024-09-18T11:30:42.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7000999999999999e-05 2024-09-18T11:30:42.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402618+0000", 2024-09-18T11:30:42.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8789999999999996e-06 2024-09-18T11:30:42.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402678+0000", 2024-09-18T11:30:42.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0312999999999997e-05 2024-09-18T11:30:42.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:42.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402682+0000", 2024-09-18T11:30:42.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4139999999999998e-06 2024-09-18T11:30:42.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.405688+0000", 2024-09-18T11:30:42.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003005939 2024-09-18T11:30:42.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.405753+0000", 2024-09-18T11:30:42.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5659000000000003e-05 2024-09-18T11:30:42.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:42.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.405912+0000", 2024-09-18T11:30:42.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015856799999999999 2024-09-18T11:30:42.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:42.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.407014+0000", 2024-09-18T11:30:42.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001101625 2024-09-18T11:30:42.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:42.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.407205+0000", 2024-09-18T11:30:42.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000190927 2024-09-18T11:30:42.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.407228+0000", 2024-09-18T11:30:42.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3850999999999999e-05 2024-09-18T11:30:42.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.407261+0000", 2024-09-18T11:30:42.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2376000000000003e-05 2024-09-18T11:30:42.975 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.975 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.976 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.976 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1429 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 10=[10,e,c] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T11:30:42.976 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:29.402624+0000", 2024-09-18T11:30:42.976 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.491864134, 2024-09-18T11:30:42.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0052753469999999997, 2024-09-18T11:30:42.976 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.976 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.976 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:42.976 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:42.976 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:42.976 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1429 2024-09-18T11:30:42.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.977 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402624+0000", 2024-09-18T11:30:42.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402624+0000", 2024-09-18T11:30:42.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402624+0000", 2024-09-18T11:30:42.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:42.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402627+0000", 2024-09-18T11:30:42.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7870000000000002e-06 2024-09-18T11:30:42.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402632+0000", 2024-09-18T11:30:42.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9520000000000002e-06 2024-09-18T11:30:42.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402636+0000", 2024-09-18T11:30:42.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.809e-06 2024-09-18T11:30:42.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402696+0000", 2024-09-18T11:30:42.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9150000000000001e-05 2024-09-18T11:30:42.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:42.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402698+0000", 2024-09-18T11:30:42.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4880000000000001e-06 2024-09-18T11:30:42.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.406581+0000", 2024-09-18T11:30:42.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038825840000000001 2024-09-18T11:30:42.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.406633+0000", 2024-09-18T11:30:42.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2602000000000002e-05 2024-09-18T11:30:42.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:42.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.406762+0000", 2024-09-18T11:30:42.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000128804 2024-09-18T11:30:42.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:42.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.407717+0000", 2024-09-18T11:30:42.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00095522800000000002 2024-09-18T11:30:42.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:42.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.407853+0000", 2024-09-18T11:30:42.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000135938 2024-09-18T11:30:42.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.407879+0000", 2024-09-18T11:30:42.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5476000000000001e-05 2024-09-18T11:30:42.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.407900+0000", 2024-09-18T11:30:42.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0941000000000002e-05 2024-09-18T11:30:42.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.982 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.982 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1687 4.5 4:a087a566:::smithi11939506-46:head [rollback 16] snapc 19=[19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-18T11:30:42.982 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.532470+0000", 2024-09-18T11:30:42.982 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.3620187850000001, 2024-09-18T11:30:42.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0045542630000000002, 2024-09-18T11:30:42.983 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.983 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.983 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:42.983 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:42.983 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:42.983 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1687 2024-09-18T11:30:42.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.983 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532470+0000", 2024-09-18T11:30:42.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532470+0000", 2024-09-18T11:30:42.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532469+0000", 2024-09-18T11:30:42.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:42.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532471+0000", 2024-09-18T11:30:42.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9350000000000001e-06 2024-09-18T11:30:42.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532494+0000", 2024-09-18T11:30:42.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2756999999999999e-05 2024-09-18T11:30:42.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532496+0000", 2024-09-18T11:30:42.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2359999999999999e-06 2024-09-18T11:30:42.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.533787+0000", 2024-09-18T11:30:42.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00129075 2024-09-18T11:30:42.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.533869+0000", 2024-09-18T11:30:42.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1914000000000002e-05 2024-09-18T11:30:42.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:42.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.534038+0000", 2024-09-18T11:30:42.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016908 2024-09-18T11:30:42.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:42.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.536851+0000", 2024-09-18T11:30:42.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028125490000000001 2024-09-18T11:30:42.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:42.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.536971+0000", 2024-09-18T11:30:42.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120325 2024-09-18T11:30:42.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.536999+0000", 2024-09-18T11:30:42.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7968999999999999e-05 2024-09-18T11:30:42.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.537024+0000", 2024-09-18T11:30:42.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4998999999999999e-05 2024-09-18T11:30:42.988 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.988 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.988 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.988 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1688 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 19=[19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-18T11:30:42.988 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:30:42.988 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.361989457, 2024-09-18T11:30:42.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071575789999999999, 2024-09-18T11:30:42.988 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.988 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:42.988 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:42.989 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:42.989 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:42.989 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1688 2024-09-18T11:30:42.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.989 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:30:42.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:30:42.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:30:42.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:42.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532500+0000", 2024-09-18T11:30:42.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5260000000000001e-06 2024-09-18T11:30:42.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532503+0000", 2024-09-18T11:30:42.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:30:42.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532505+0000", 2024-09-18T11:30:42.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.889e-06 2024-09-18T11:30:42.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.534722+0000", 2024-09-18T11:30:42.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002217119 2024-09-18T11:30:42.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.534827+0000", 2024-09-18T11:30:42.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0001049 2024-09-18T11:30:42.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:42.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.535007+0000", 2024-09-18T11:30:42.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017965599999999999 2024-09-18T11:30:42.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:42.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539463+0000", 2024-09-18T11:30:42.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004455713 2024-09-18T11:30:42.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:42.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539607+0000", 2024-09-18T11:30:42.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014440300000000001 2024-09-18T11:30:42.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:42.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539634+0000", 2024-09-18T11:30:42.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7398000000000001e-05 2024-09-18T11:30:42.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539657+0000", 2024-09-18T11:30:42.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2228000000000001e-05 2024-09-18T11:30:42.994 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.994 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.994 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.994 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1758 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2319366,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:42.994 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.293909+0000", 2024-09-18T11:30:42.994 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.60057902399999996, 2024-09-18T11:30:42.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056075329999999996, 2024-09-18T11:30:42.994 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.994 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:42.994 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:42.994 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:42.994 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:42.994 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1758 2024-09-18T11:30:42.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.995 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:42.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:42.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293909+0000", 2024-09-18T11:30:42.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:42.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293909+0000", 2024-09-18T11:30:42.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:42.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:42.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293907+0000", 2024-09-18T11:30:42.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:42.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:42.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293918+0000", 2024-09-18T11:30:42.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1432e-05 2024-09-18T11:30:42.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:42.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293939+0000", 2024-09-18T11:30:42.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0874999999999999e-05 2024-09-18T11:30:42.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:42.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293948+0000", 2024-09-18T11:30:42.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4700000000000002e-06 2024-09-18T11:30:42.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:30:42.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6805000000000003e-05 2024-09-18T11:30:42.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:42.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294002+0000", 2024-09-18T11:30:42.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2629999999999997e-06 2024-09-18T11:30:42.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:42.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.295647+0000", 2024-09-18T11:30:42.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016449329999999999 2024-09-18T11:30:42.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:42.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.295706+0000", 2024-09-18T11:30:42.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9011999999999997e-05 2024-09-18T11:30:42.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:42.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.299517+0000", 2024-09-18T11:30:42.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038113359999999998 2024-09-18T11:30:42.998 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.999 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:42.999 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:42.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:42.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:42.999 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1759 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2319366,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:42.999 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:30:42.999 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.60052924699999999, 2024-09-18T11:30:42.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064323039999999998, 2024-09-18T11:30:42.999 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:42.999 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:42.999 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:42.999 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:42.999 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:42.999 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1759 2024-09-18T11:30:42.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:30:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:30:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:30:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293963+0000", 2024-09-18T11:30:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0810000000000004e-06 2024-09-18T11:30:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293970+0000", 2024-09-18T11:30:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5109999999999997e-06 2024-09-18T11:30:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293975+0000", 2024-09-18T11:30:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.33e-06 2024-09-18T11:30:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294078+0000", 2024-09-18T11:30:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102871 2024-09-18T11:30:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294084+0000", 2024-09-18T11:30:43.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5470000000000002e-06 2024-09-18T11:30:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.299550+0000", 2024-09-18T11:30:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054660799999999999 2024-09-18T11:30:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.299608+0000", 2024-09-18T11:30:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8307000000000003e-05 2024-09-18T11:30:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300392+0000", 2024-09-18T11:30:43.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078306999999999997 2024-09-18T11:30:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1760 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2319366,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:30:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.60049490699999997, 2024-09-18T11:30:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070132529999999997, 2024-09-18T11:30:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:43.004 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1760 2024-09-18T11:30:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:30:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:30:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293993+0000", 2024-09-18T11:30:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293997+0000", 2024-09-18T11:30:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4999999999999999e-06 2024-09-18T11:30:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294004+0000", 2024-09-18T11:30:43.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3950000000000003e-06 2024-09-18T11:30:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294009+0000", 2024-09-18T11:30:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4989999999999996e-06 2024-09-18T11:30:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294103+0000", 2024-09-18T11:30:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4817999999999999e-05 2024-09-18T11:30:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294110+0000", 2024-09-18T11:30:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7240000000000003e-06 2024-09-18T11:30:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300406+0000", 2024-09-18T11:30:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062964470000000002 2024-09-18T11:30:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300457+0000", 2024-09-18T11:30:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0915000000000003e-05 2024-09-18T11:30:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.301007+0000", 2024-09-18T11:30:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00054942299999999995 2024-09-18T11:30:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.009 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.009 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:43.011 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:30:43.014 INFO:tasks.rados.rados.0.smithi119.stdout:954: writing smithi11939506-18 from 1527054 to 2293104 tid 2 2024-09-18T11:30:43.014 INFO:tasks.rados.rados.0.smithi119.stdout:955: read oid 22 snap -1 2024-09-18T11:30:43.014 INFO:tasks.rados.rados.0.smithi119.stdout:955: expect (ObjNum 101 snap 6 seq_num 101) 2024-09-18T11:30:43.015 INFO:tasks.rados.rados.0.smithi119.stdout:956: read oid 40 snap 15 2024-09-18T11:30:43.015 INFO:tasks.rados.rados.0.smithi119.stdout:956: expect (ObjNum 86 snap 5 seq_num 86) 2024-09-18T11:30:43.015 INFO:tasks.rados.rados.0.smithi119.stdout:957: write (excl) oid 7 current snap is 18 2024-09-18T11:30:43.015 INFO:tasks.rados.rados.0.smithi119.stdout:952: got (22) Invalid argument 2024-09-18T11:30:43.015 INFO:tasks.rados.rados.0.smithi119.stdout:957: seq_num 145 ranges {437015=721812,1701889=509708} 2024-09-18T11:30:43.015 INFO:tasks.rados.rados.0.smithi119.stdout:957: writing smithi11939506-7 from 437015 to 1158827 tid 1 2024-09-18T11:30:43.015 INFO:tasks.rados.rados.0.smithi119.stdout:957: writing smithi11939506-7 from 1701889 to 2211597 tid 2 2024-09-18T11:30:43.015 INFO:tasks.rados.rados.0.smithi119.stdout:952: done (6 left) 2024-09-18T11:30:43.015 INFO:tasks.rados.rados.0.smithi119.stdout:958: write oid 25 current snap is 18 2024-09-18T11:30:43.015 INFO:tasks.rados.rados.0.smithi119.stdout:958: seq_num 146 ranges {453822=797061,1760371=653366} 2024-09-18T11:30:43.015 INFO:tasks.rados.rados.0.smithi119.stdout:958: writing smithi11939506-25 from 453822 to 1250883 tid 1 2024-09-18T11:30:43.015 INFO:tasks.rados.rados.0.smithi119.stdout:958: writing smithi11939506-25 from 1760371 to 2413737 tid 2 2024-09-18T11:30:43.015 INFO:tasks.rados.rados.0.smithi119.stdout:959: read oid 6 snap -1 2024-09-18T11:30:43.016 INFO:tasks.rados.rados.0.smithi119.stdout:959: expect (ObjNum 117 snap 11 seq_num 117) 2024-09-18T11:30:43.016 INFO:tasks.rados.rados.0.smithi119.stdout:960: read oid 19 snap -1 2024-09-18T11:30:43.016 INFO:tasks.rados.rados.0.smithi119.stdout:960: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-18T11:30:43.016 INFO:tasks.rados.rados.0.smithi119.stdout:961: read oid 21 snap 13 2024-09-18T11:30:43.016 INFO:tasks.rados.rados.0.smithi119.stdout:961: expect deleted 2024-09-18T11:30:43.016 INFO:tasks.rados.rados.0.smithi119.stdout:962: read oid 42 snap -1 2024-09-18T11:30:43.016 INFO:tasks.rados.rados.0.smithi119.stdout:962: expect (ObjNum 137 snap 16 seq_num 137) 2024-09-18T11:30:43.016 INFO:tasks.rados.rados.0.smithi119.stdout:963: write oid 26 current snap is 18 2024-09-18T11:30:43.016 INFO:tasks.rados.rados.0.smithi119.stdout:963: seq_num 147 ranges {470629=472310,1418853=513377} 2024-09-18T11:30:43.016 INFO:tasks.rados.rados.0.smithi119.stdout:963: writing smithi11939506-26 from 470629 to 942939 tid 1 2024-09-18T11:30:43.016 INFO:tasks.rados.rados.0.smithi119.stdout:963: writing smithi11939506-26 from 1418853 to 1932230 tid 2 2024-09-18T11:30:43.016 INFO:tasks.rados.rados.0.smithi119.stdout:964: read oid 27 snap -1 2024-09-18T11:30:43.016 INFO:tasks.rados.rados.0.smithi119.stdout:964: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-18T11:30:43.017 INFO:tasks.rados.rados.0.smithi119.stdout:965: read oid 13 snap 14 2024-09-18T11:30:43.017 INFO:tasks.rados.rados.0.smithi119.stdout:965: expect (ObjNum 74 snap 3 seq_num 74) 2024-09-18T11:30:43.017 INFO:tasks.rados.rados.0.smithi119.stdout:966: write oid 11 current snap is 18 2024-09-18T11:30:43.017 INFO:tasks.rados.rados.0.smithi119.stdout:966: seq_num 148 ranges {487436=547559,1760982=657035} 2024-09-18T11:30:43.017 INFO:tasks.rados.rados.0.smithi119.stdout:966: writing smithi11939506-11 from 487436 to 1034995 tid 1 2024-09-18T11:30:43.017 INFO:tasks.rados.rados.0.smithi119.stdout:966: writing smithi11939506-11 from 1760982 to 2418017 tid 2 2024-09-18T11:30:43.017 INFO:tasks.rados.rados.0.smithi119.stdout:967: read oid 47 snap -1 2024-09-18T11:30:43.017 INFO:tasks.rados.rados.0.smithi119.stdout:967: expect deleted 2024-09-18T11:30:43.017 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:43.017 INFO:tasks.rados.rados.0.smithi119.stdout:951: finishing write tid 2 to smithi11939506-29 2024-09-18T11:30:43.017 INFO:tasks.rados.rados.0.smithi119.stdout:951: finishing write tid 3 to smithi11939506-29 2024-09-18T11:30:43.017 INFO:tasks.rados.rados.0.smithi119.stdout:951: finishing write tid 4 to smithi11939506-29 2024-09-18T11:30:43.017 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 40 (ObjNum 143 snap 18 seq_num 143) dirty exists 2024-09-18T11:30:43.017 INFO:tasks.rados.rados.0.smithi119.stdout:951: left oid 29 (ObjNum 143 snap 18 seq_num 143) 2024-09-18T11:30:43.018 INFO:tasks.rados.rados.0.smithi119.stdout:953: expect (ObjNum 90 snap 5 seq_num 90) 2024-09-18T11:30:43.018 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:43.009+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:43.111 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:43.109+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:43.116 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:43.116 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:43.116 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:43.116 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:43.124 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:30:43.150 INFO:tasks.rados.rados.0.smithi119.stdout:954: finishing write tid 1 to smithi11939506-18 2024-09-18T11:30:43.150 INFO:tasks.rados.rados.0.smithi119.stdout:956: expect (ObjNum 86 snap 5 seq_num 86) 2024-09-18T11:30:43.212 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:43.211+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:43.262 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:43.262 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:43.262 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:43.262 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:43.262 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:43.270 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:30:43.313 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:43.312+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:43.335 INFO:tasks.rados.rados.0.smithi119.stdout:955: expect (ObjNum 101 snap 6 seq_num 101) 2024-09-18T11:30:43.410 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:43.411 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:43.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:43.411 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:43.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.411 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:43.411 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:43.411 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.250930025999999, 2024-09-18T11:30:43.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054864400000000004, 2024-09-18T11:30:43.411 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:43.411 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.411 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:43.412 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:43.412 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:43.412 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 42 2024-09-18T11:30:43.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.412 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:43.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:43.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:43.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:43.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157416+0000", 2024-09-18T11:30:43.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.721e-06 2024-09-18T11:30:43.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157423+0000", 2024-09-18T11:30:43.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4049999999999997e-06 2024-09-18T11:30:43.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157425+0000", 2024-09-18T11:30:43.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.481e-06 2024-09-18T11:30:43.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158649+0000", 2024-09-18T11:30:43.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001223453 2024-09-18T11:30:43.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158729+0000", 2024-09-18T11:30:43.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0205999999999997e-05 2024-09-18T11:30:43.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:43.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158875+0000", 2024-09-18T11:30:43.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014628800000000001 2024-09-18T11:30:43.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:43.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161518+0000", 2024-09-18T11:30:43.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026427949999999999 2024-09-18T11:30:43.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162821+0000", 2024-09-18T11:30:43.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013028099999999999 2024-09-18T11:30:43.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:43.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162864+0000", 2024-09-18T11:30:43.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2839000000000001e-05 2024-09-18T11:30:43.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162900+0000", 2024-09-18T11:30:43.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5948999999999998e-05 2024-09-18T11:30:43.417 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.417 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.417 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.417 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:43.417 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:43.417 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.250843662000001, 2024-09-18T11:30:43.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055400850000000001, 2024-09-18T11:30:43.417 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:43.417 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.418 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:43.418 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:43.418 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:43.418 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 43 2024-09-18T11:30:43.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.418 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:43.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:43.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:43.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:43.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157504+0000", 2024-09-18T11:30:43.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4519999999999996e-06 2024-09-18T11:30:43.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157534+0000", 2024-09-18T11:30:43.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.97e-05 2024-09-18T11:30:43.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157539+0000", 2024-09-18T11:30:43.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4360000000000001e-06 2024-09-18T11:30:43.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159428+0000", 2024-09-18T11:30:43.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018888220000000001 2024-09-18T11:30:43.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159479+0000", 2024-09-18T11:30:43.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0608000000000001e-05 2024-09-18T11:30:43.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:43.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159559+0000", 2024-09-18T11:30:43.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0667000000000005e-05 2024-09-18T11:30:43.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:43.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162779+0000", 2024-09-18T11:30:43.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032197459999999999 2024-09-18T11:30:43.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162950+0000", 2024-09-18T11:30:43.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017059599999999999 2024-09-18T11:30:43.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:43.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163001+0000", 2024-09-18T11:30:43.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0968999999999999e-05 2024-09-18T11:30:43.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163040+0000", 2024-09-18T11:30:43.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8843000000000001e-05 2024-09-18T11:30:43.423 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.423 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.423 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.423 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:43.423 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:43.423 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.250792508000004, 2024-09-18T11:30:43.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:30:43.423 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:43.423 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.424 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:43.424 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:43.424 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:43.424 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:30:43.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.424 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:43.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:43.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:30:43.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:43.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:30:43.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:30:43.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:30:43.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:30:43.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:30:43.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:30:43.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:30:43.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:30:43.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:30:43.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:30:43.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:43.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:30:43.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:30:43.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:43.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:30:43.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:30:43.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:30:43.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:30:43.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:43.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:30:43.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:30:43.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:30:43.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:30:43.429 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.429 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.429 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.429 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:43.429 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:43.429 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.250666246000002, 2024-09-18T11:30:43.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055717040000000002, 2024-09-18T11:30:43.429 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:43.430 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.430 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:43.430 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:43.430 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:43.430 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 45 2024-09-18T11:30:43.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.430 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:43.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:43.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157676+0000", 2024-09-18T11:30:43.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:43.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157694+0000", 2024-09-18T11:30:43.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7422999999999999e-05 2024-09-18T11:30:43.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157704+0000", 2024-09-18T11:30:43.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9000000000000001e-06 2024-09-18T11:30:43.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157718+0000", 2024-09-18T11:30:43.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4788e-05 2024-09-18T11:30:43.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160937+0000", 2024-09-18T11:30:43.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00321886 2024-09-18T11:30:43.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161023+0000", 2024-09-18T11:30:43.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6063999999999995e-05 2024-09-18T11:30:43.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:43.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161202+0000", 2024-09-18T11:30:43.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017912599999999999 2024-09-18T11:30:43.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:43.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163052+0000", 2024-09-18T11:30:43.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001849554 2024-09-18T11:30:43.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163204+0000", 2024-09-18T11:30:43.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015186900000000001 2024-09-18T11:30:43.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:43.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163229+0000", 2024-09-18T11:30:43.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:30:43.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163249+0000", 2024-09-18T11:30:43.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9992000000000001e-05 2024-09-18T11:30:43.435 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.435 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.435 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.435 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:43.435 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:43.435 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.250617132999999, 2024-09-18T11:30:43.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:30:43.435 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:43.435 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.435 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:43.435 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:43.436 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:43.436 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:30:43.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.436 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:43.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:43.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:43.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:43.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:30:43.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:30:43.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:30:43.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:30:43.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:30:43.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:30:43.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:30:43.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:30:43.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:30:43.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:30:43.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:43.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:30:43.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:30:43.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:43.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:30:43.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:30:43.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:30:43.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:30:43.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:43.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:30:43.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:30:43.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:30:43.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:30:43.441 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.441 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.441 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.441 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:43.441 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:43.441 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.250574602, 2024-09-18T11:30:43.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:30:43.441 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:43.441 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.442 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:43.442 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:43.442 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:43.442 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:30:43.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.442 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:43.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:43.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:30:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:30:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:30:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:30:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:30:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:30:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:30:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:30:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:30:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:30:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:30:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:30:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:30:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:30:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:30:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:30:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:30:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:30:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:30:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:30:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:30:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.250519672000003, 2024-09-18T11:30:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:30:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:30:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:30:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:30:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:30:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:30:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:30:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:30:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:30:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:30:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:30:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:30:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:30:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:30:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:43.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:30:43.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:30:43.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:30:43.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:30:43.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:43.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:30:43.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:30:43.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:30:43.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:30:43.453 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.453 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.453 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.453 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:43.453 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:43.453 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.250509692000001, 2024-09-18T11:30:43.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:30:43.454 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:43.454 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.454 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:43.454 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:43.454 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:43.454 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:30:43.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.454 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:43.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:43.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:43.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:43.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:30:43.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:30:43.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:30:43.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:30:43.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:30:43.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:30:43.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:30:43.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:30:43.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:30:43.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:30:43.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:43.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:30:43.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:30:43.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:30:43.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:30:43.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:43.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:30:43.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:30:43.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:43.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:30:43.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:30:43.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:30:43.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:30:43.459 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.459 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.459 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.459 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:43.459 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:43.459 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.250498092999997, 2024-09-18T11:30:43.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:30:43.460 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:43.460 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.460 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:43.460 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:43.460 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:43.460 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:30:43.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.460 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:43.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:43.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:43.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:43.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:30:43.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:30:43.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:30:43.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:30:43.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:30:43.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:30:43.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:30:43.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:30:43.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:30:43.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:30:43.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:43.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:30:43.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:30:43.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:43.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:30:43.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:30:43.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:30:43.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:30:43.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:43.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:30:43.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:30:43.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:30:43.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:30:43.465 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.465 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.465 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.465 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:43.465 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:43.465 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.250441612000003, 2024-09-18T11:30:43.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:30:43.466 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:43.466 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.466 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:43.466 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:43.466 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:43.466 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:30:43.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.466 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:43.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:43.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:30:43.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:43.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:30:43.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:30:43.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:30:43.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:30:43.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:30:43.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:30:43.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:30:43.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:30:43.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:30:43.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:30:43.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:43.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:30:43.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:30:43.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:30:43.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:30:43.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:43.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:30:43.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:30:43.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:43.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:30:43.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:30:43.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:30:43.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:30:43.471 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.471 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.471 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.471 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:43.471 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:43.471 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.250427788000003, 2024-09-18T11:30:43.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:30:43.471 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:43.472 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.472 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:43.472 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:43.472 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:43.472 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:30:43.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.472 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:43.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:43.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:43.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:43.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:30:43.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:30:43.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:30:43.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:30:43.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:30:43.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:30:43.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:30:43.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:30:43.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:30:43.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:30:43.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:43.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:30:43.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:30:43.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:30:43.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:30:43.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:43.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:30:43.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:30:43.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:43.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:30:43.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:30:43.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:30:43.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:30:43.477 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.477 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.477 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.477 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:43.477 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:43.477 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.834248881000001, 2024-09-18T11:30:43.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:30:43.477 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:43.477 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.478 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:43.478 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:43.478 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:43.478 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:30:43.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.478 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:43.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:43.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:43.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:43.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:30:43.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:30:43.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:30:43.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:30:43.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:30:43.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:30:43.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:30:43.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:30:43.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:30:43.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:30:43.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:43.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:30:43.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:30:43.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:43.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:30:43.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:30:43.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:30:43.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:30:43.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:43.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:30:43.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:30:43.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:30:43.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:30:43.482 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.482 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.483 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.483 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:43.483 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:43.483 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.834225580999998, 2024-09-18T11:30:43.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:30:43.483 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:43.483 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.483 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:43.483 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:43.483 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:43.483 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:30:43.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.484 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:43.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:43.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:30:43.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:43.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:30:43.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:30:43.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:30:43.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:30:43.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:30:43.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:30:43.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:30:43.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:30:43.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:30:43.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:30:43.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:43.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:30:43.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:30:43.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:43.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:30:43.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:30:43.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:30:43.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:30:43.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:43.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:30:43.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:30:43.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:30:43.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:30:43.489 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.489 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.489 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.489 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:43.489 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:43.489 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.834139972000003, 2024-09-18T11:30:43.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:30:43.489 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:43.489 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.489 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:43.489 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:43.490 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:43.490 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:30:43.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.490 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:43.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:43.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:43.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:43.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:30:43.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:30:43.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:30:43.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:30:43.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:30:43.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:30:43.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:30:43.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:30:43.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:30:43.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:30:43.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.493 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:43.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:30:43.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:30:43.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.493 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:43.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:30:43.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:30:43.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:30:43.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:30:43.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:43.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:30:43.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:30:43.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:30:43.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:30:43.495 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.495 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.495 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.495 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:43.495 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:43.495 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.834129007000001, 2024-09-18T11:30:43.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:30:43.496 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:43.496 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.496 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:43.496 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:43.496 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:43.496 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:30:43.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.496 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:43.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:43.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:43.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:43.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:30:43.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:30:43.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:30:43.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:30:43.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:30:43.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:30:43.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:30:43.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:30:43.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:30:43.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:30:43.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:43.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:30:43.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:30:43.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:43.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:30:43.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:30:43.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:30:43.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:30:43.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:43.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:30:43.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:30:43.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:30:43.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:30:43.501 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.501 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.501 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.501 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:43.501 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:43.501 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.834117839000001, 2024-09-18T11:30:43.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:30:43.501 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:43.501 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.501 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:43.501 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:43.502 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:43.502 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:30:43.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.502 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:43.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:43.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:43.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:43.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:30:43.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:30:43.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:30:43.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:30:43.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.504 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:30:43.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:30:43.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.504 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:30:43.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:30:43.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:30:43.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:30:43.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:43.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:30:43.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:30:43.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:30:43.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:30:43.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:43.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:30:43.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:30:43.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:43.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:30:43.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:30:43.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:30:43.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:30:43.507 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.507 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.507 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.507 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:43.507 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:43.507 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.834052509999999, 2024-09-18T11:30:43.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:30:43.507 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:43.507 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.507 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:43.507 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:43.508 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:43.508 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:30:43.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.508 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:43.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:43.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:30:43.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:43.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:30:43.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:30:43.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:30:43.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:30:43.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:30:43.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:30:43.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:30:43.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:30:43.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:30:43.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:30:43.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:43.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:30:43.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:30:43.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:30:43.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:30:43.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:43.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:30:43.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:30:43.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:43.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:30:43.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:30:43.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:30:43.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:30:43.513 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.513 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.513 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.513 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:43.513 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:43.513 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.823890532, 2024-09-18T11:30:43.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:30:43.513 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:43.514 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.514 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:43.514 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:43.514 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:43.514 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:30:43.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.514 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:43.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:43.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:30:43.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:43.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:30:43.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:30:43.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:30:43.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:30:43.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:30:43.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:30:43.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:30:43.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:30:43.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:30:43.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:30:43.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:43.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:30:43.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:30:43.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:43.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:30:43.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:30:43.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:30:43.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:30:43.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:43.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:30:43.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:30:43.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:30:43.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:30:43.521 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.521 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.521 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.522 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:43.522 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:43.522 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.823798052999997, 2024-09-18T11:30:43.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:30:43.522 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:43.522 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.522 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:43.522 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:43.522 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:43.522 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:30:43.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.523 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:43.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:43.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:43.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:43.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.524 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:30:43.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:30:43.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.524 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:30:43.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:30:43.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:30:43.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:30:43.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:30:43.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:30:43.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:30:43.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:30:43.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:43.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:30:43.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:30:43.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:43.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:30:43.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:30:43.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:30:43.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:30:43.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:43.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:30:43.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:30:43.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:30:43.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:30:43.527 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.527 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.527 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.528 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1788 4.8 e60/22 4:1898b15d:::smithi11939506-25:head v 60'12, mlcod=58'9)", 2024-09-18T11:30:43.528 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:30:43.528 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.49139256399999998, 2024-09-18T11:30:43.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058348599999999999, 2024-09-18T11:30:43.528 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:43.528 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.528 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:43.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:43.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:30:43.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:43.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:30:43.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:43.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:43.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916949+0000", 2024-09-18T11:30:43.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:43.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:43.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917602+0000", 2024-09-18T11:30:43.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00065337100000000003 2024-09-18T11:30:43.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.530 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:43.530 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917621+0000", 2024-09-18T11:30:43.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9287e-05 2024-09-18T11:30:43.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.530 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.530 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917629+0000", 2024-09-18T11:30:43.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3780000000000003e-06 2024-09-18T11:30:43.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.530 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:43.530 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917677+0000", 2024-09-18T11:30:43.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8068000000000001e-05 2024-09-18T11:30:43.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:43.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917699+0000", 2024-09-18T11:30:43.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2149000000000001e-05 2024-09-18T11:30:43.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:43.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922747+0000", 2024-09-18T11:30:43.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050477609999999996 2024-09-18T11:30:43.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:43.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:43.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:43.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922785+0000", 2024-09-18T11:30:43.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8726000000000002e-05 2024-09-18T11:30:43.532 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.532 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.532 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.532 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:43.532 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:43.532 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:43.534 INFO:tasks.rados.rados.0.smithi119.stdout:954: finishing write tid 2 to smithi11939506-18 2024-09-18T11:30:43.534 INFO:tasks.rados.rados.0.smithi119.stdout:954: finishing write tid 3 to smithi11939506-18 2024-09-18T11:30:43.534 INFO:tasks.rados.rados.0.smithi119.stdout:954: finishing write tid 4 to smithi11939506-18 2024-09-18T11:30:43.534 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 63 (ObjNum 144 snap 18 seq_num 144) dirty exists 2024-09-18T11:30:43.534 INFO:tasks.rados.rados.0.smithi119.stdout:954: left oid 18 (ObjNum 144 snap 18 seq_num 144) 2024-09-18T11:30:43.534 INFO:tasks.rados.rados.0.smithi119.stdout:957: finishing write tid 1 to smithi11939506-7 2024-09-18T11:30:43.534 INFO:tasks.rados.rados.0.smithi119.stdout:957: finishing write tid 2 to smithi11939506-7 2024-09-18T11:30:43.534 INFO:tasks.rados.rados.0.smithi119.stdout:957: finishing write tid 3 to smithi11939506-7 2024-09-18T11:30:43.534 INFO:tasks.rados.rados.0.smithi119.stdout:951: done (15 left) 2024-09-18T11:30:43.534 INFO:tasks.rados.rados.0.smithi119.stdout:953: done (14 left) 2024-09-18T11:30:43.534 INFO:tasks.rados.rados.0.smithi119.stdout:954: done (13 left) 2024-09-18T11:30:43.534 INFO:tasks.rados.rados.0.smithi119.stdout:955: done (12 left) 2024-09-18T11:30:43.535 INFO:tasks.rados.rados.0.smithi119.stdout:956: done (11 left) 2024-09-18T11:30:43.535 INFO:tasks.rados.rados.0.smithi119.stdout:968: set_chunk oid 9 target oid 2024-09-18T11:30:43.535 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:43.532+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:43.536 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:30:43.537 INFO:tasks.rados.rados.0.smithi119.stdout:968: set_chunk oid 9 offset: 970240 length: 83968 target oid 9fb14befb08f54939f2c71c890ba0ea89332fb5c1fd9e0b07290d878c4431425 offset: 0 snap -1 2024-09-18T11:30:43.537 INFO:tasks.rados.rados.0.smithi119.stdout:969: read oid 3 snap -1 2024-09-18T11:30:43.537 INFO:tasks.rados.rados.0.smithi119.stdout:969: expect (ObjNum 77 snap 3 seq_num 77) 2024-09-18T11:30:43.537 INFO:tasks.rados.rados.0.smithi119.stdout:970: snap_create 2024-09-18T11:30:43.537 INFO:tasks.rados.rados.0.smithi119.stdout:957: finishing write tid 4 to smithi11939506-7 2024-09-18T11:30:43.537 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 7 v 37 (ObjNum 145 snap 18 seq_num 145) dirty exists 2024-09-18T11:30:43.537 INFO:tasks.rados.rados.0.smithi119.stdout:957: left oid 7 (ObjNum 145 snap 18 seq_num 145) 2024-09-18T11:30:43.537 INFO:tasks.rados.rados.0.smithi119.stdout:960: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-18T11:30:43.610 INFO:tasks.rados.rados.0.smithi119.stdout:958: finishing write tid 1 to smithi11939506-25 2024-09-18T11:30:43.610 INFO:tasks.rados.rados.0.smithi119.stdout:958: finishing write tid 2 to smithi11939506-25 2024-09-18T11:30:43.610 INFO:tasks.rados.rados.0.smithi119.stdout:958: finishing write tid 3 to smithi11939506-25 2024-09-18T11:30:43.610 INFO:tasks.rados.rados.0.smithi119.stdout:958: finishing write tid 4 to smithi11939506-25 2024-09-18T11:30:43.611 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 14 (ObjNum 146 snap 18 seq_num 146) dirty exists 2024-09-18T11:30:43.611 INFO:tasks.rados.rados.0.smithi119.stdout:958: left oid 25 (ObjNum 146 snap 18 seq_num 146) 2024-09-18T11:30:43.611 INFO:tasks.rados.rados.0.smithi119.stdout:965: expect (ObjNum 74 snap 3 seq_num 74) 2024-09-18T11:30:43.634 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:43.633+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:43.636 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:43.636 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:43.636 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:43.636 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:43.644 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:30:43.735 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:43.734+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:43.742 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:43.742 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:43.743 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:43.743 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:43.743 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:43.751 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:30:43.785 INFO:tasks.rados.rados.0.smithi119.stdout:959: expect (ObjNum 117 snap 11 seq_num 117) 2024-09-18T11:30:43.835 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:30:43.836 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:43.835+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:43.851 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:43.851 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:43.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:43.852 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:43.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.852 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:21 3.1 3:9f1cb8cf:::smithi11939506-38 low_tier:head [read 336384~173568 [fadvise_sequential]] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:43.852 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.118534+0000", 2024-09-18T11:30:43.852 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 32.730519305999998, 2024-09-18T11:30:43.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006633842, 2024-09-18T11:30:43.852 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:43.852 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:43.852 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:43.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:43.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118534+0000", 2024-09-18T11:30:43.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:43.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118534+0000", 2024-09-18T11:30:43.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:43.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118532+0000", 2024-09-18T11:30:43.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:43.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:43.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118542+0000", 2024-09-18T11:30:43.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0151e-05 2024-09-18T11:30:43.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:43.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118563+0000", 2024-09-18T11:30:43.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1089000000000001e-05 2024-09-18T11:30:43.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118572+0000", 2024-09-18T11:30:43.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1090000000000004e-06 2024-09-18T11:30:43.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118633+0000", 2024-09-18T11:30:43.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0940000000000003e-05 2024-09-18T11:30:43.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:43.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118640+0000", 2024-09-18T11:30:43.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3690000000000003e-06 2024-09-18T11:30:43.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124910+0000", 2024-09-18T11:30:43.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062697569999999999 2024-09-18T11:30:43.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:43.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124980+0000", 2024-09-18T11:30:43.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0846000000000006e-05 2024-09-18T11:30:43.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:43.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.125168+0000", 2024-09-18T11:30:43.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018786200000000001 2024-09-18T11:30:43.857 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.857 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:43.857 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.857 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:22 3.1 3:9f1cb8cf:::smithi11939506-38 low_tier:head [read 1235456~1340416 [fadvise_sequential]] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:43.857 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:43.857 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 32.730466577999998, 2024-09-18T11:30:43.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0090773199999999998, 2024-09-18T11:30:43.858 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:43.858 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:43.858 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:43.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:43.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:43.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:43.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:43.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:43.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:43.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:43.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:43.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118594+0000", 2024-09-18T11:30:43.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1590000000000004e-06 2024-09-18T11:30:43.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:43.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118601+0000", 2024-09-18T11:30:43.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3900000000000004e-06 2024-09-18T11:30:43.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118605+0000", 2024-09-18T11:30:43.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2479999999999998e-06 2024-09-18T11:30:43.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118697+0000", 2024-09-18T11:30:43.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1447999999999996e-05 2024-09-18T11:30:43.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:43.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118702+0000", 2024-09-18T11:30:43.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6670000000000001e-06 2024-09-18T11:30:43.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.125285+0000", 2024-09-18T11:30:43.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006582124 2024-09-18T11:30:43.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:43.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.125349+0000", 2024-09-18T11:30:43.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4777000000000007e-05 2024-09-18T11:30:43.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:43.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.127664+0000", 2024-09-18T11:30:43.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023151069999999998 2024-09-18T11:30:43.862 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.862 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:43.862 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.862 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:911 4.3 4:cc46532a:::smithi11939506-33:head [write 1904434~719345] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:43.862 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.166081+0000", 2024-09-18T11:30:43.862 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 32.682972511999999, 2024-09-18T11:30:43.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081214100000000008, 2024-09-18T11:30:43.863 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:43.863 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.863 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:43.863 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:43.863 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:43.863 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 911 2024-09-18T11:30:43.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.863 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:43.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:43.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166081+0000", 2024-09-18T11:30:43.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:43.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166081+0000", 2024-09-18T11:30:43.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:43.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166080+0000", 2024-09-18T11:30:43.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:43.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:43.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166386+0000", 2024-09-18T11:30:43.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030589499999999998 2024-09-18T11:30:43.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:43.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166398+0000", 2024-09-18T11:30:43.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1678000000000001e-05 2024-09-18T11:30:43.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166402+0000", 2024-09-18T11:30:43.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6059999999999996e-06 2024-09-18T11:30:43.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166443+0000", 2024-09-18T11:30:43.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1044000000000002e-05 2024-09-18T11:30:43.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:43.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166526+0000", 2024-09-18T11:30:43.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2281999999999996e-05 2024-09-18T11:30:43.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:43.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166661+0000", 2024-09-18T11:30:43.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013516299999999999 2024-09-18T11:30:43.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:43.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.168437+0000", 2024-09-18T11:30:43.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001775794 2024-09-18T11:30:43.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174126+0000", 2024-09-18T11:30:43.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056894859999999997 2024-09-18T11:30:43.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:43.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174166+0000", 2024-09-18T11:30:43.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9787999999999999e-05 2024-09-18T11:30:43.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:43.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174203+0000", 2024-09-18T11:30:43.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6644999999999999e-05 2024-09-18T11:30:43.868 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.868 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:43.868 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.868 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:989 4.3 4:ca2b2309:::smithi11939506-44:head [write 410104~465105,stat] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:43.868 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.151379+0000", 2024-09-18T11:30:43.869 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.697674547999998, 2024-09-18T11:30:43.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062831420000000002, 2024-09-18T11:30:43.869 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:43.869 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.869 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:43.869 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:43.869 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:43.869 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 989 2024-09-18T11:30:43.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.869 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:43.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:43.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151379+0000", 2024-09-18T11:30:43.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:43.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151379+0000", 2024-09-18T11:30:43.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:43.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151376+0000", 2024-09-18T11:30:43.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:43.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:43.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151666+0000", 2024-09-18T11:30:43.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028977899999999999 2024-09-18T11:30:43.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:43.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151697+0000", 2024-09-18T11:30:43.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0516000000000001e-05 2024-09-18T11:30:43.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151707+0000", 2024-09-18T11:30:43.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0322999999999999e-05 2024-09-18T11:30:43.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151770+0000", 2024-09-18T11:30:43.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879000000000006e-05 2024-09-18T11:30:43.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:43.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151783+0000", 2024-09-18T11:30:43.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3154e-05 2024-09-18T11:30:43.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.153878+0000", 2024-09-18T11:30:43.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002095155 2024-09-18T11:30:43.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:43.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.153957+0000", 2024-09-18T11:30:43.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8877999999999999e-05 2024-09-18T11:30:43.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:43.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.154073+0000", 2024-09-18T11:30:43.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116045 2024-09-18T11:30:43.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:43.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.156031+0000", 2024-09-18T11:30:43.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019581099999999999 2024-09-18T11:30:43.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.157569+0000", 2024-09-18T11:30:43.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015383199999999999 2024-09-18T11:30:43.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:43.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.157606+0000", 2024-09-18T11:30:43.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6278000000000001e-05 2024-09-18T11:30:43.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:43.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.157662+0000", 2024-09-18T11:30:43.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6589000000000003e-05 2024-09-18T11:30:43.875 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.875 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:43.875 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.875 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1012 4.b 4:db5b7794:::smithi11939506-13:head [write 443718~615603] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:43.875 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.655768+0000", 2024-09-18T11:30:43.875 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.193286033, 2024-09-18T11:30:43.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055941680000000001, 2024-09-18T11:30:43.875 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:43.875 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.875 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:43.876 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:43.876 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:43.876 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1012 2024-09-18T11:30:43.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.876 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:43.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:43.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.655768+0000", 2024-09-18T11:30:43.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:43.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.655768+0000", 2024-09-18T11:30:43.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:43.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.655765+0000", 2024-09-18T11:30:43.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:43.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:43.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656140+0000", 2024-09-18T11:30:43.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00037503800000000001 2024-09-18T11:30:43.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:43.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656166+0000", 2024-09-18T11:30:43.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5451000000000001e-05 2024-09-18T11:30:43.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.878 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656176+0000", 2024-09-18T11:30:43.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0489000000000001e-05 2024-09-18T11:30:43.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.878 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656255+0000", 2024-09-18T11:30:43.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8880000000000004e-05 2024-09-18T11:30:43.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:43.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656322+0000", 2024-09-18T11:30:43.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6500000000000004e-05 2024-09-18T11:30:43.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:43.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656436+0000", 2024-09-18T11:30:43.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114945 2024-09-18T11:30:43.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.658992+0000", 2024-09-18T11:30:43.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025551419999999998 2024-09-18T11:30:43.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:43.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.661302+0000", 2024-09-18T11:30:43.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002310543 2024-09-18T11:30:43.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:43.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.661335+0000", 2024-09-18T11:30:43.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2716999999999999e-05 2024-09-18T11:30:43.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:43.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.661362+0000", 2024-09-18T11:30:43.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6998000000000001e-05 2024-09-18T11:30:43.881 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.881 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:43.881 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.881 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1150 4.b 4:d3ae8215:::smithi11939506-41:head [write 1807440~632344] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:43.881 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.620640+0000", 2024-09-18T11:30:43.881 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.228413861, 2024-09-18T11:30:43.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068244000000000004, 2024-09-18T11:30:43.881 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:43.882 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.882 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:43.882 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:43.882 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:43.882 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1150 2024-09-18T11:30:43.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.882 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:43.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.882 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:43.882 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.620640+0000", 2024-09-18T11:30:43.882 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.882 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:43.883 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.620640+0000", 2024-09-18T11:30:43.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.883 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:43.883 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.620638+0000", 2024-09-18T11:30:43.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:43.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.883 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:43.883 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621224+0000", 2024-09-18T11:30:43.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00058580899999999996 2024-09-18T11:30:43.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:43.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621235+0000", 2024-09-18T11:30:43.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:30:43.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621240+0000", 2024-09-18T11:30:43.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4270000000000001e-06 2024-09-18T11:30:43.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621268+0000", 2024-09-18T11:30:43.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8252999999999999e-05 2024-09-18T11:30:43.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:43.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621344+0000", 2024-09-18T11:30:43.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6154000000000003e-05 2024-09-18T11:30:43.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:43.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621464+0000", 2024-09-18T11:30:43.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012021 2024-09-18T11:30:43.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.624135+0000", 2024-09-18T11:30:43.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026711310000000002 2024-09-18T11:30:43.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:43.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.627420+0000", 2024-09-18T11:30:43.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032846799999999999 2024-09-18T11:30:43.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:43.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.627443+0000", 2024-09-18T11:30:43.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3153e-05 2024-09-18T11:30:43.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.887 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:43.887 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.627464+0000", 2024-09-18T11:30:43.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0967000000000001e-05 2024-09-18T11:30:43.887 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.887 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:43.887 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.887 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:32 3.1 3:94708c4c:::888f815178b2a748104fffa559dcb609f143f519d8737c6aa160c9cce13a6441:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:43.887 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.679331+0000", 2024-09-18T11:30:43.887 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.169723093000002, 2024-09-18T11:30:43.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063965929999999999, 2024-09-18T11:30:43.887 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:43.887 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.888 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:43.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:43.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679331+0000", 2024-09-18T11:30:43.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:43.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679331+0000", 2024-09-18T11:30:43.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:43.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679330+0000", 2024-09-18T11:30:43.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:43.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:43.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681259+0000", 2024-09-18T11:30:43.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019287079999999999 2024-09-18T11:30:43.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:43.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681274+0000", 2024-09-18T11:30:43.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5903e-05 2024-09-18T11:30:43.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681281+0000", 2024-09-18T11:30:43.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5640000000000002e-06 2024-09-18T11:30:43.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681325+0000", 2024-09-18T11:30:43.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3925999999999999e-05 2024-09-18T11:30:43.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:43.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681435+0000", 2024-09-18T11:30:43.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109739 2024-09-18T11:30:43.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:43.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681642+0000", 2024-09-18T11:30:43.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000206901 2024-09-18T11:30:43.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:43.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.683180+0000", 2024-09-18T11:30:43.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015380929999999999 2024-09-18T11:30:43.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.685648+0000", 2024-09-18T11:30:43.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024687419999999999 2024-09-18T11:30:43.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:43.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.685695+0000", 2024-09-18T11:30:43.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7030000000000002e-05 2024-09-18T11:30:43.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:43.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.685727+0000", 2024-09-18T11:30:43.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1792000000000003e-05 2024-09-18T11:30:43.892 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.892 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:43.893 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.893 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1528 3.1 3:97025964:::d85d35018f93ce704e0c86e65d6c262758d2b38883b05fc1a96898c5a961f23a:head [write 0~1647104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:30:43.893 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.376484+0000", 2024-09-18T11:30:43.893 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4725695390000002, 2024-09-18T11:30:43.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073833279999999998, 2024-09-18T11:30:43.893 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:43.893 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.893 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:43.893 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:43.893 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:43.893 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1528 2024-09-18T11:30:43.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.894 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:43.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:43.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.376484+0000", 2024-09-18T11:30:43.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:43.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.376484+0000", 2024-09-18T11:30:43.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.376483+0000", 2024-09-18T11:30:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.377189+0000", 2024-09-18T11:30:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00070628499999999996 2024-09-18T11:30:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.377208+0000", 2024-09-18T11:30:43.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9261000000000001e-05 2024-09-18T11:30:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.377232+0000", 2024-09-18T11:30:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3794e-05 2024-09-18T11:30:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.377286+0000", 2024-09-18T11:30:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4243999999999998e-05 2024-09-18T11:30:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:43.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.377434+0000", 2024-09-18T11:30:43.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014722699999999999 2024-09-18T11:30:43.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:43.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.377586+0000", 2024-09-18T11:30:43.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015253399999999999 2024-09-18T11:30:43.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:43.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.380417+0000", 2024-09-18T11:30:43.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028304799999999998 2024-09-18T11:30:43.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.383812+0000", 2024-09-18T11:30:43.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003395117 2024-09-18T11:30:43.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:43.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.383843+0000", 2024-09-18T11:30:43.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1615000000000001e-05 2024-09-18T11:30:43.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:43.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.383868+0000", 2024-09-18T11:30:43.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4074999999999999e-05 2024-09-18T11:30:43.899 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.899 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:43.899 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.899 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1554 3.3 e53/21 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head v 53'84, mlcod=47'82)", 2024-09-18T11:30:43.899 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.419942+0000", 2024-09-18T11:30:43.899 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4291121150000006, 2024-09-18T11:30:43.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011179532000000001, 2024-09-18T11:30:43.899 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:43.899 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.899 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:43.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:43.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.419942+0000", 2024-09-18T11:30:43.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:43.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.419942+0000", 2024-09-18T11:30:43.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:43.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.419940+0000", 2024-09-18T11:30:43.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:43.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:43.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.426895+0000", 2024-09-18T11:30:43.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069556970000000003 2024-09-18T11:30:43.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:43.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.426923+0000", 2024-09-18T11:30:43.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8095000000000001e-05 2024-09-18T11:30:43.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.426934+0000", 2024-09-18T11:30:43.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0875e-05 2024-09-18T11:30:43.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.426991+0000", 2024-09-18T11:30:43.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6422999999999998e-05 2024-09-18T11:30:43.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:43.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.427012+0000", 2024-09-18T11:30:43.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1599000000000001e-05 2024-09-18T11:30:43.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:43.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.431068+0000", 2024-09-18T11:30:43.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040558419999999996 2024-09-18T11:30:43.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:43.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.431121+0000", 2024-09-18T11:30:43.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3035000000000003e-05 2024-09-18T11:30:43.903 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.903 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:43.903 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.903 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1603 4.b 4:dba5bd39:::smithi11939506-17:head [write 568103~768769,stat] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:43.903 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.265951+0000", 2024-09-18T11:30:43.903 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.5831031280000003, 2024-09-18T11:30:43.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078467250000000006, 2024-09-18T11:30:43.904 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:43.904 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.904 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:43.904 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:43.904 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:43.904 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1603 2024-09-18T11:30:43.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.904 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:43.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:43.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.265951+0000", 2024-09-18T11:30:43.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:43.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.265951+0000", 2024-09-18T11:30:43.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:43.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.265948+0000", 2024-09-18T11:30:43.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:43.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:43.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.266720+0000", 2024-09-18T11:30:43.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00077223899999999998 2024-09-18T11:30:43.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:43.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.266748+0000", 2024-09-18T11:30:43.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8178999999999999e-05 2024-09-18T11:30:43.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.266758+0000", 2024-09-18T11:30:43.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8309999999999996e-06 2024-09-18T11:30:43.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.266808+0000", 2024-09-18T11:30:43.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0136000000000002e-05 2024-09-18T11:30:43.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:43.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.266881+0000", 2024-09-18T11:30:43.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2449999999999999e-05 2024-09-18T11:30:43.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:43.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.267018+0000", 2024-09-18T11:30:43.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000137656 2024-09-18T11:30:43.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.273693+0000", 2024-09-18T11:30:43.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0066745179999999999 2024-09-18T11:30:43.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:43.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.273701+0000", 2024-09-18T11:30:43.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.729e-06 2024-09-18T11:30:43.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:43.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.273751+0000", 2024-09-18T11:30:43.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9944999999999997e-05 2024-09-18T11:30:43.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:43.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.273797+0000", 2024-09-18T11:30:43.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6717e-05 2024-09-18T11:30:43.909 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.909 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:43.909 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.909 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1623 4.3 4:c70c3162:::smithi11939506-35:head [write 584910~560371,stat] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:43.909 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.301415+0000", 2024-09-18T11:30:43.909 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.5476384129999996, 2024-09-18T11:30:43.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0061674679999999997, 2024-09-18T11:30:43.909 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:43.910 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.910 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:43.910 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:43.910 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:43.910 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1623 2024-09-18T11:30:43.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.910 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:43.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:43.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.301415+0000", 2024-09-18T11:30:43.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:43.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.301415+0000", 2024-09-18T11:30:43.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:43.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.301413+0000", 2024-09-18T11:30:43.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:43.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:43.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.301696+0000", 2024-09-18T11:30:43.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028292999999999998 2024-09-18T11:30:43.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:43.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.301719+0000", 2024-09-18T11:30:43.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3302000000000002e-05 2024-09-18T11:30:43.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.301728+0000", 2024-09-18T11:30:43.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.67e-06 2024-09-18T11:30:43.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.301810+0000", 2024-09-18T11:30:43.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1718000000000006e-05 2024-09-18T11:30:43.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:43.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.302028+0000", 2024-09-18T11:30:43.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021829400000000001 2024-09-18T11:30:43.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:43.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.302381+0000", 2024-09-18T11:30:43.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00035296999999999998 2024-09-18T11:30:43.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:43.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.306463+0000", 2024-09-18T11:30:43.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040817370000000002 2024-09-18T11:30:43.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.307497+0000", 2024-09-18T11:30:43.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0010348639999999999 2024-09-18T11:30:43.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:43.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.307538+0000", 2024-09-18T11:30:43.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0324000000000002e-05 2024-09-18T11:30:43.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:43.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.307583+0000", 2024-09-18T11:30:43.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4997999999999998e-05 2024-09-18T11:30:43.915 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.915 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:43.915 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.915 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1628 3.3 e56/21 3:f80aa31b:::77cdbf3abbada57d0636b2198046a030673818705cbf13bb77b30c3e68b24345:head v 56'91, mlcod=56'89)", 2024-09-18T11:30:43.915 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.312845+0000", 2024-09-18T11:30:43.915 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.536209157, 2024-09-18T11:30:43.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006071908, 2024-09-18T11:30:43.916 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:43.916 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.916 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:43.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:43.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.312845+0000", 2024-09-18T11:30:43.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:43.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.312845+0000", 2024-09-18T11:30:43.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:43.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.312844+0000", 2024-09-18T11:30:43.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:43.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:43.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.314834+0000", 2024-09-18T11:30:43.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001990477 2024-09-18T11:30:43.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:43.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.314846+0000", 2024-09-18T11:30:43.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2300000000000001e-05 2024-09-18T11:30:43.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.314852+0000", 2024-09-18T11:30:43.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4720000000000003e-06 2024-09-18T11:30:43.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.314897+0000", 2024-09-18T11:30:43.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4951999999999999e-05 2024-09-18T11:30:43.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:43.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.314923+0000", 2024-09-18T11:30:43.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6086999999999999e-05 2024-09-18T11:30:43.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:43.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.318868+0000", 2024-09-18T11:30:43.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039451770000000002 2024-09-18T11:30:43.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:43.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.318916+0000", 2024-09-18T11:30:43.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8301999999999999e-05 2024-09-18T11:30:43.919 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.919 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:43.920 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.920 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1634 4.b 4:d13decd8:::smithi11939506-2:head [write 1799494~402223] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:43.920 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333638+0000", 2024-09-18T11:30:43.920 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.5154160030000003, 2024-09-18T11:30:43.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057370850000000003, 2024-09-18T11:30:43.920 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:43.920 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.920 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:43.920 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:43.920 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:43.920 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1634 2024-09-18T11:30:43.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.921 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:43.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:43.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333638+0000", 2024-09-18T11:30:43.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:43.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333638+0000", 2024-09-18T11:30:43.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:43.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333635+0000", 2024-09-18T11:30:43.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:43.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:43.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333775+0000", 2024-09-18T11:30:43.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000139646 2024-09-18T11:30:43.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:43.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333793+0000", 2024-09-18T11:30:43.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8309e-05 2024-09-18T11:30:43.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333800+0000", 2024-09-18T11:30:43.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6359999999999997e-06 2024-09-18T11:30:43.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333853+0000", 2024-09-18T11:30:43.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2908000000000002e-05 2024-09-18T11:30:43.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:43.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333960+0000", 2024-09-18T11:30:43.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000106928 2024-09-18T11:30:43.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:43.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.334140+0000", 2024-09-18T11:30:43.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018008799999999999 2024-09-18T11:30:43.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:43.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.338316+0000", 2024-09-18T11:30:43.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041764330000000002 2024-09-18T11:30:43.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.339263+0000", 2024-09-18T11:30:43.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000946764 2024-09-18T11:30:43.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:43.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.339323+0000", 2024-09-18T11:30:43.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9589000000000002e-05 2024-09-18T11:30:43.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:43.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.339375+0000", 2024-09-18T11:30:43.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2172000000000001e-05 2024-09-18T11:30:43.925 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.926 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:43.926 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.926 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1635 4.b 4:d13decd8:::smithi11939506-2:head [setxattr _header (56) in=7b,truncate 2201717] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:43.926 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333814+0000", 2024-09-18T11:30:43.926 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.5152400650000004, 2024-09-18T11:30:43.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057269290000000004, 2024-09-18T11:30:43.926 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:43.926 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.926 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:43.926 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:43.926 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:43.927 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1635 2024-09-18T11:30:43.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.927 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:43.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:43.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333814+0000", 2024-09-18T11:30:43.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:43.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333814+0000", 2024-09-18T11:30:43.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:43.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333813+0000", 2024-09-18T11:30:43.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:43.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:43.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333817+0000", 2024-09-18T11:30:43.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1810000000000003e-06 2024-09-18T11:30:43.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:43.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333826+0000", 2024-09-18T11:30:43.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4649999999999994e-06 2024-09-18T11:30:43.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333830+0000", 2024-09-18T11:30:43.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5000000000000001e-06 2024-09-18T11:30:43.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.335314+0000", 2024-09-18T11:30:43.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001483769 2024-09-18T11:30:43.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:43.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.335413+0000", 2024-09-18T11:30:43.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9309999999999996e-05 2024-09-18T11:30:43.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:43.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.335584+0000", 2024-09-18T11:30:43.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017072999999999999 2024-09-18T11:30:43.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.339421+0000", 2024-09-18T11:30:43.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038367900000000001 2024-09-18T11:30:43.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:43.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.339428+0000", 2024-09-18T11:30:43.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6000000000000001e-06 2024-09-18T11:30:43.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:43.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.339481+0000", 2024-09-18T11:30:43.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2774e-05 2024-09-18T11:30:43.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:43.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.339541+0000", 2024-09-18T11:30:43.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9483e-05 2024-09-18T11:30:43.933 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.933 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:43.933 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.933 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1636 4.b 4:d13decd8:::smithi11939506-2:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:43.934 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333866+0000", 2024-09-18T11:30:43.934 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.5151876279999996, 2024-09-18T11:30:43.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059017790000000002, 2024-09-18T11:30:43.934 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:43.934 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:43.934 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:43.934 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:43.934 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:43.934 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1636 2024-09-18T11:30:43.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.934 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:43.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:43.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333866+0000", 2024-09-18T11:30:43.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:43.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333866+0000", 2024-09-18T11:30:43.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:43.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333866+0000", 2024-09-18T11:30:43.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:43.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:43.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333869+0000", 2024-09-18T11:30:43.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7069999999999998e-06 2024-09-18T11:30:43.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:43.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333878+0000", 2024-09-18T11:30:43.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5339999999999999e-06 2024-09-18T11:30:43.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333883+0000", 2024-09-18T11:30:43.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9300000000000002e-06 2024-09-18T11:30:43.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.336182+0000", 2024-09-18T11:30:43.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022994579999999999 2024-09-18T11:30:43.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:43.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.336260+0000", 2024-09-18T11:30:43.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8323000000000002e-05 2024-09-18T11:30:43.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.339572+0000", 2024-09-18T11:30:43.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0033116719999999999 2024-09-18T11:30:43.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:43.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.339674+0000", 2024-09-18T11:30:43.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010208300000000001 2024-09-18T11:30:43.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:43.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.339768+0000", 2024-09-18T11:30:43.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3634000000000005e-05 2024-09-18T11:30:43.938 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.938 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:43.939 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.939 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1650 4.b 4:ddcb7ff4:::smithi11939506-9:head [write 1857976~360548] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:43.939 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349569+0000", 2024-09-18T11:30:43.939 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.4994846160000002, 2024-09-18T11:30:43.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065957359999999996, 2024-09-18T11:30:43.939 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:43.939 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.939 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:43.939 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:43.939 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:43.940 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1650 2024-09-18T11:30:43.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.940 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:43.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:43.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349569+0000", 2024-09-18T11:30:43.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:43.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349569+0000", 2024-09-18T11:30:43.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:43.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349568+0000", 2024-09-18T11:30:43.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:43.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:43.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349677+0000", 2024-09-18T11:30:43.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109669 2024-09-18T11:30:43.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:43.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349695+0000", 2024-09-18T11:30:43.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7652e-05 2024-09-18T11:30:43.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349701+0000", 2024-09-18T11:30:43.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5869999999999998e-06 2024-09-18T11:30:43.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349756+0000", 2024-09-18T11:30:43.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4793999999999997e-05 2024-09-18T11:30:43.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:43.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.350439+0000", 2024-09-18T11:30:43.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00068225899999999995 2024-09-18T11:30:43.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:43.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.350646+0000", 2024-09-18T11:30:43.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020736800000000001 2024-09-18T11:30:43.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:43.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355791+0000", 2024-09-18T11:30:43.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051446929999999997 2024-09-18T11:30:43.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.356004+0000", 2024-09-18T11:30:43.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021360800000000001 2024-09-18T11:30:43.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:43.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.356089+0000", 2024-09-18T11:30:43.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4529000000000003e-05 2024-09-18T11:30:43.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:43.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.356165+0000", 2024-09-18T11:30:43.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6085000000000001e-05 2024-09-18T11:30:43.945 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.945 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:43.945 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.945 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1651 4.b 4:ddcb7ff4:::smithi11939506-9:head [setxattr _header (56) in=7b,truncate 2218524] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:43.945 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349712+0000", 2024-09-18T11:30:43.945 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.4993415969999999, 2024-09-18T11:30:43.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078601369999999997, 2024-09-18T11:30:43.945 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:43.945 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.945 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:43.945 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:43.945 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:43.946 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1651 2024-09-18T11:30:43.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.946 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:43.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:43.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349712+0000", 2024-09-18T11:30:43.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:43.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349712+0000", 2024-09-18T11:30:43.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:43.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349712+0000", 2024-09-18T11:30:43.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:43.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:43.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349716+0000", 2024-09-18T11:30:43.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.048e-06 2024-09-18T11:30:43.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:43.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349733+0000", 2024-09-18T11:30:43.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7244999999999998e-05 2024-09-18T11:30:43.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349738+0000", 2024-09-18T11:30:43.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1259999999999997e-06 2024-09-18T11:30:43.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.351839+0000", 2024-09-18T11:30:43.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002101225 2024-09-18T11:30:43.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:43.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.351946+0000", 2024-09-18T11:30:43.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010671300000000001 2024-09-18T11:30:43.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:43.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.352153+0000", 2024-09-18T11:30:43.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020683499999999999 2024-09-18T11:30:43.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:43.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355800+0000", 2024-09-18T11:30:43.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00364687 2024-09-18T11:30:43.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357459+0000", 2024-09-18T11:30:43.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016598979999999999 2024-09-18T11:30:43.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:43.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357514+0000", 2024-09-18T11:30:43.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4274999999999998e-05 2024-09-18T11:30:43.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:43.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357572+0000", 2024-09-18T11:30:43.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8505000000000003e-05 2024-09-18T11:30:43.950 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.950 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:43.950 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.951 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1652 4.b 4:ddcb7ff4:::smithi11939506-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:43.951 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349743+0000", 2024-09-18T11:30:43.951 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.4993103469999998, 2024-09-18T11:30:43.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080765520000000007, 2024-09-18T11:30:43.951 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:43.951 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:43.951 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:43.951 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:43.951 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:43.951 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1652 2024-09-18T11:30:43.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.951 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:43.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:43.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349743+0000", 2024-09-18T11:30:43.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:43.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349743+0000", 2024-09-18T11:30:43.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:43.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349743+0000", 2024-09-18T11:30:43.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:43.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:43.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349746+0000", 2024-09-18T11:30:43.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8890000000000002e-06 2024-09-18T11:30:43.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:43.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349752+0000", 2024-09-18T11:30:43.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2589999999999997e-06 2024-09-18T11:30:43.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349756+0000", 2024-09-18T11:30:43.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.788e-06 2024-09-18T11:30:43.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.352573+0000", 2024-09-18T11:30:43.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028174430000000002 2024-09-18T11:30:43.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:43.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.352669+0000", 2024-09-18T11:30:43.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6082000000000006e-05 2024-09-18T11:30:43.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357604+0000", 2024-09-18T11:30:43.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049343210000000002 2024-09-18T11:30:43.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:43.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357706+0000", 2024-09-18T11:30:43.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102437 2024-09-18T11:30:43.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:43.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357820+0000", 2024-09-18T11:30:43.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113763 2024-09-18T11:30:43.956 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.956 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:43.956 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.956 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:74 3.1 3:b900d23f:::smithi11939506-24 low_tier:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e58)", 2024-09-18T11:30:43.956 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.266263+0000", 2024-09-18T11:30:43.957 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5827911549999998, 2024-09-18T11:30:43.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078322889999999992, 2024-09-18T11:30:43.957 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:43.957 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.957 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:43.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:43.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266263+0000", 2024-09-18T11:30:43.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:43.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266263+0000", 2024-09-18T11:30:43.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266260+0000", 2024-09-18T11:30:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266271+0000", 2024-09-18T11:30:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0839000000000001e-05 2024-09-18T11:30:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266293+0000", 2024-09-18T11:30:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1749000000000001e-05 2024-09-18T11:30:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266301+0000", 2024-09-18T11:30:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8550000000000002e-06 2024-09-18T11:30:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266367+0000", 2024-09-18T11:30:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6371999999999994e-05 2024-09-18T11:30:43.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266468+0000", 2024-09-18T11:30:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101303 2024-09-18T11:30:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266650+0000", 2024-09-18T11:30:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000181922 2024-09-18T11:30:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.269069+0000", 2024-09-18T11:30:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024186189999999999 2024-09-18T11:30:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274003+0000", 2024-09-18T11:30:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049343449999999997 2024-09-18T11:30:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274057+0000", 2024-09-18T11:30:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4008000000000002e-05 2024-09-18T11:30:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274095+0000", 2024-09-18T11:30:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7503000000000003e-05 2024-09-18T11:30:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:75 3.1 3:b900d23f:::smithi11939506-24 low_tier:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e58)", 2024-09-18T11:30:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.266327+0000", 2024-09-18T11:30:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5827268859999997, 2024-09-18T11:30:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078435550000000003, 2024-09-18T11:30:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:43.962 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266327+0000", 2024-09-18T11:30:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266327+0000", 2024-09-18T11:30:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266326+0000", 2024-09-18T11:30:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266330+0000", 2024-09-18T11:30:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.805e-06 2024-09-18T11:30:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266337+0000", 2024-09-18T11:30:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9750000000000001e-06 2024-09-18T11:30:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266341+0000", 2024-09-18T11:30:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:30:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.267141+0000", 2024-09-18T11:30:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00079995900000000004 2024-09-18T11:30:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.267219+0000", 2024-09-18T11:30:43.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8551000000000001e-05 2024-09-18T11:30:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.267378+0000", 2024-09-18T11:30:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000158694 2024-09-18T11:30:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.269110+0000", 2024-09-18T11:30:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001732 2024-09-18T11:30:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:43.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274108+0000", 2024-09-18T11:30:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049980520000000002 2024-09-18T11:30:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274144+0000", 2024-09-18T11:30:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5989999999999999e-05 2024-09-18T11:30:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274170+0000", 2024-09-18T11:30:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6398e-05 2024-09-18T11:30:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:43.967 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.968 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:43.968 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:43.968 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:43.969 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:30:43.970 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:30:43.970 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:30:43.970 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:43.971 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:43.971 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:43.968+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:43.978 INFO:tasks.rados.rados.0.smithi119.stdout:966: finishing write tid 1 to smithi11939506-11 2024-09-18T11:30:43.978 INFO:tasks.rados.rados.0.smithi119.stdout:962: expect (ObjNum 137 snap 16 seq_num 137) 2024-09-18T11:30:43.980 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:43.981 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:43.996 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:43.997 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:44.012 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:44.012 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:44.028 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:44.028 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:44.044 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:44.044 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:44.059 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:44.060 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:44.070 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:44.069+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:44.079 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:44.079 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:44.088 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:30:44.125 INFO:tasks.rados.rados.0.smithi119.stdout:963: finishing write tid 1 to smithi11939506-26 2024-09-18T11:30:44.125 INFO:tasks.rados.rados.0.smithi119.stdout:964: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-18T11:30:44.171 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:44.170+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:44.204 INFO:tasks.rados.rados.0.smithi119.stdout:963: finishing write tid 2 to smithi11939506-26 2024-09-18T11:30:44.204 INFO:tasks.rados.rados.0.smithi119.stdout:963: finishing write tid 3 to smithi11939506-26 2024-09-18T11:30:44.204 INFO:tasks.rados.rados.0.smithi119.stdout:963: finishing write tid 4 to smithi11939506-26 2024-09-18T11:30:44.205 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 83 (ObjNum 147 snap 18 seq_num 147) dirty exists 2024-09-18T11:30:44.205 INFO:tasks.rados.rados.0.smithi119.stdout:963: left oid 26 (ObjNum 147 snap 18 seq_num 147) 2024-09-18T11:30:44.205 INFO:tasks.rados.rados.0.smithi119.stdout:966: finishing write tid 2 to smithi11939506-11 2024-09-18T11:30:44.205 INFO:tasks.rados.rados.0.smithi119.stdout:966: finishing write tid 3 to smithi11939506-11 2024-09-18T11:30:44.205 INFO:tasks.rados.rados.0.smithi119.stdout:966: finishing write tid 4 to smithi11939506-11 2024-09-18T11:30:44.205 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 11 v 80 (ObjNum 148 snap 18 seq_num 148) dirty exists 2024-09-18T11:30:44.205 INFO:tasks.rados.rados.0.smithi119.stdout:966: left oid 11 (ObjNum 148 snap 18 seq_num 148) 2024-09-18T11:30:44.205 INFO:tasks.rados.rados.0.smithi119.stdout:969: expect (ObjNum 77 snap 3 seq_num 77) 2024-09-18T11:30:44.224 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:44.224 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:44.232 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:30:44.276 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:44.275+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:44.302 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T11:30:44.333 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:44.333 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:44.342 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:30:44.372 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:44.371+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:44.393 INFO:tasks.rados.rados.0.smithi119.stdout:968: finishing set_chunk to oid 9 2024-09-18T11:30:44.393 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 89 (ObjNum 132 snap 15 seq_num 132) dirty exists 2024-09-18T11:30:44.393 INFO:tasks.rados.rados.0.smithi119.stdout:957: done (13 left) 2024-09-18T11:30:44.393 INFO:tasks.rados.rados.0.smithi119.stdout:958: done (12 left) 2024-09-18T11:30:44.393 INFO:tasks.rados.rados.0.smithi119.stdout:959: done (11 left) 2024-09-18T11:30:44.393 INFO:tasks.rados.rados.0.smithi119.stdout:960: done (10 left) 2024-09-18T11:30:44.393 INFO:tasks.rados.rados.0.smithi119.stdout:961: done (9 left) 2024-09-18T11:30:44.393 INFO:tasks.rados.rados.0.smithi119.stdout:962: done (8 left) 2024-09-18T11:30:44.393 INFO:tasks.rados.rados.0.smithi119.stdout:963: done (7 left) 2024-09-18T11:30:44.393 INFO:tasks.rados.rados.0.smithi119.stdout:964: done (6 left) 2024-09-18T11:30:44.394 INFO:tasks.rados.rados.0.smithi119.stdout:965: done (5 left) 2024-09-18T11:30:44.394 INFO:tasks.rados.rados.0.smithi119.stdout:966: done (4 left) 2024-09-18T11:30:44.394 INFO:tasks.rados.rados.0.smithi119.stdout:967: done (3 left) 2024-09-18T11:30:44.394 INFO:tasks.rados.rados.0.smithi119.stdout:968: done (2 left) 2024-09-18T11:30:44.394 INFO:tasks.rados.rados.0.smithi119.stdout:969: done (1 left) 2024-09-18T11:30:44.394 INFO:tasks.rados.rados.0.smithi119.stdout:970: done (0 left) 2024-09-18T11:30:44.394 INFO:tasks.rados.rados.0.smithi119.stdout:971: read oid 18 snap -1 2024-09-18T11:30:44.394 INFO:tasks.rados.rados.0.smithi119.stdout:971: expect (ObjNum 144 snap 18 seq_num 144) 2024-09-18T11:30:44.394 INFO:tasks.rados.rados.0.smithi119.stdout:972: set_chunk oid 10 target oid 2024-09-18T11:30:44.394 INFO:tasks.rados.rados.0.smithi119.stdout:972: set_chunk oid 10 offset: 0 length: 4000000 target oid smithi11939506- offset: 0 snap -1 2024-09-18T11:30:44.394 INFO:tasks.rados.rados.0.smithi119.stdout:973: read oid 24 snap -1 2024-09-18T11:30:44.394 INFO:tasks.rados.rados.0.smithi119.stdout:973: expect (ObjNum 135 snap 16 seq_num 135) 2024-09-18T11:30:44.394 INFO:tasks.rados.rados.0.smithi119.stdout:974: delete oid 37 current snap is 19 2024-09-18T11:30:44.395 INFO:tasks.rados.rados.0.smithi119.stdout:972: finishing set_chunk to oid 10 2024-09-18T11:30:44.395 INFO:tasks.rados.rados.0.smithi119.stdout:972: got expected ENOENT (src dne) 2024-09-18T11:30:44.396 INFO:tasks.rados.rados.0.smithi119.stdout:973: expect (ObjNum 135 snap 16 seq_num 135) 2024-09-18T11:30:44.474 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:44.473+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:44.498 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:44.498 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:44.506 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:30:44.545 INFO:tasks.rados.rados.0.smithi119.stdout:971: expect (ObjNum 144 snap 18 seq_num 144) 2024-09-18T11:30:44.575 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:44.574+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:44.646 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:44.646 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:44.654 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:30:44.694 INFO:tasks.rados.rados.0.smithi119.stdout:971: done (3 left) 2024-09-18T11:30:44.694 INFO:tasks.rados.rados.0.smithi119.stdout:972: done (2 left) 2024-09-18T11:30:44.694 INFO:tasks.rados.rados.0.smithi119.stdout:973: done (1 left) 2024-09-18T11:30:44.694 INFO:tasks.rados.rados.0.smithi119.stdout:974: done (0 left) 2024-09-18T11:30:44.694 INFO:tasks.rados.rados.0.smithi119.stdout:975: set_chunk oid 8 target oid 2024-09-18T11:30:44.695 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:44.693+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:44.708 INFO:tasks.rados.rados.0.smithi119.stdout:975: set_chunk oid 8 offset: 531456 length: 1744896 target oid d01ff5a07a68e5351ca985a6c57ddc5ef0d05d539368179784aa67dbeae8d271 offset: 0 snap 16 2024-09-18T11:30:44.708 INFO:tasks.rados.rados.0.smithi119.stdout:976: read oid 39 snap 18 2024-09-18T11:30:44.708 INFO:tasks.rados.rados.0.smithi119.stdout:976: expect deleted 2024-09-18T11:30:44.708 INFO:tasks.rados.rados.0.smithi119.stdout:977: write (excl) oid 9 current snap is 19 2024-09-18T11:30:44.708 INFO:tasks.rados.rados.0.smithi119.stdout:977: seq_num 150 ranges {521050=698057,1877946=643104} 2024-09-18T11:30:44.714 INFO:tasks.rados.rados.0.smithi119.stdout:977: writing smithi11939506-9 from 521050 to 1219107 tid 1 2024-09-18T11:30:44.722 INFO:tasks.rados.rados.0.smithi119.stdout:977: writing smithi11939506-9 from 1877946 to 2521050 tid 2 2024-09-18T11:30:44.722 INFO:tasks.rados.rados.0.smithi119.stdout:978: write oid 5 current snap is 19 2024-09-18T11:30:44.722 INFO:tasks.rados.rados.0.smithi119.stdout:978: seq_num 151 ranges {537857=773306,1936428=520715} 2024-09-18T11:30:44.729 INFO:tasks.rados.rados.0.smithi119.stdout:978: writing smithi11939506-5 from 537857 to 1311163 tid 1 2024-09-18T11:30:44.730 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:44.729+0000 7fb6930fb640 -1 osd.2 61 handle_osd_map injecting map crc failure 2024-09-18T11:30:44.736 INFO:tasks.rados.rados.0.smithi119.stdout:978: writing smithi11939506-5 from 1936428 to 2457143 tid 2 2024-09-18T11:30:44.736 INFO:tasks.rados.rados.0.smithi119.stdout:979: tier_evict oid 50 2024-09-18T11:30:44.736 INFO:tasks.rados.rados.0.smithi119.stdout:979: tier_evict oid 50 snap -1 2024-09-18T11:30:44.736 INFO:tasks.rados.rados.0.smithi119.stdout:975: finishing set_chunk to oid 8 2024-09-18T11:30:44.736 INFO:tasks.rados.rados.0.smithi119.stdout:977: finishing write tid 1 to smithi11939506-9 2024-09-18T11:30:44.736 INFO:tasks.rados.rados.0.smithi119.stdout:975: done (4 left) 2024-09-18T11:30:44.736 INFO:tasks.rados.rados.0.smithi119.stdout:976: done (3 left) 2024-09-18T11:30:44.737 INFO:tasks.rados.rados.0.smithi119.stdout:980: set_chunk oid 20 target oid 2024-09-18T11:30:44.740 INFO:tasks.rados.rados.0.smithi119.stdout:980: set_chunk oid 20 offset: 3527680 length: 145408 target oid 76a1a50e6b37fd7abf3acda4de5f45b3a045ded478e9d4f6ad0c66388319eaa4 offset: 0 snap 10 2024-09-18T11:30:44.740 INFO:tasks.rados.rados.0.smithi119.stdout:981: set_chunk oid 19 target oid 2024-09-18T11:30:44.746 INFO:tasks.rados.rados.0.smithi119.stdout:981: set_chunk oid 19 offset: 1644544 length: 598016 target oid 966944e5760147d7e70a93814579084183f8d9734bb8f5484c4cf731e04da19a offset: 0 snap 15 2024-09-18T11:30:44.746 INFO:tasks.rados.rados.0.smithi119.stdout:982: delete oid 23 current snap is 19 2024-09-18T11:30:44.746 INFO:tasks.rados.rados.0.smithi119.stdout:977: finishing write tid 2 to smithi11939506-9 2024-09-18T11:30:44.746 INFO:tasks.rados.rados.0.smithi119.stdout:977: finishing write tid 3 to smithi11939506-9 2024-09-18T11:30:44.746 INFO:tasks.rados.rados.0.smithi119.stdout:977: finishing write tid 4 to smithi11939506-9 2024-09-18T11:30:44.746 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 104 (ObjNum 150 snap 19 seq_num 150) dirty exists 2024-09-18T11:30:44.746 INFO:tasks.rados.rados.0.smithi119.stdout:977: left oid 9 (ObjNum 150 snap 19 seq_num 150) 2024-09-18T11:30:44.746 INFO:tasks.rados.rados.0.smithi119.stdout:978: finishing write tid 1 to smithi11939506-5 2024-09-18T11:30:44.746 INFO:tasks.rados.rados.0.smithi119.stdout:978: finishing write tid 2 to smithi11939506-5 2024-09-18T11:30:44.746 INFO:tasks.rados.rados.0.smithi119.stdout:979: got (22) Invalid argument 2024-09-18T11:30:44.746 INFO:tasks.rados.rados.0.smithi119.stdout:978: finishing write tid 3 to smithi11939506-5 2024-09-18T11:30:44.746 INFO:tasks.rados.rados.0.smithi119.stdout:978: finishing write tid 4 to smithi11939506-5 2024-09-18T11:30:44.746 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 63 (ObjNum 151 snap 19 seq_num 151) dirty exists 2024-09-18T11:30:44.746 INFO:tasks.rados.rados.0.smithi119.stdout:978: left oid 5 (ObjNum 151 snap 19 seq_num 151) 2024-09-18T11:30:44.747 INFO:tasks.rados.rados.0.smithi119.stdout:980: finishing set_chunk to oid 20 2024-09-18T11:30:44.747 INFO:tasks.rados.rados.0.smithi119.stdout:981: finishing set_chunk to oid 19 2024-09-18T11:30:44.747 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 19 set_chunk returned error code -95 offset: 1644544 length: 598016 2024-09-18T11:30:44.747 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 0 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-18T11:30:44.750 INFO:tasks.rados.rados.0.smithi119.stdout:977: done (5 left) 2024-09-18T11:30:44.750 INFO:tasks.rados.rados.0.smithi119.stdout:978: done (4 left) 2024-09-18T11:30:44.750 INFO:tasks.rados.rados.0.smithi119.stdout:979: done (3 left) 2024-09-18T11:30:44.750 INFO:tasks.rados.rados.0.smithi119.stdout:980: done (2 left) 2024-09-18T11:30:44.750 INFO:tasks.rados.rados.0.smithi119.stdout:981: done (1 left) 2024-09-18T11:30:44.751 INFO:tasks.rados.rados.0.smithi119.stdout:982: done (0 left) 2024-09-18T11:30:44.751 INFO:tasks.rados.rados.0.smithi119.stdout:983: read oid 14 snap -1 2024-09-18T11:30:44.751 INFO:tasks.rados.rados.0.smithi119.stdout:983: expect (ObjNum 133 snap 15 seq_num 133) 2024-09-18T11:30:44.751 INFO:tasks.rados.rados.0.smithi119.stdout:984: snap_create 2024-09-18T11:30:44.760 INFO:tasks.rados.rados.0.smithi119.stdout:983: expect (ObjNum 133 snap 15 seq_num 133) 2024-09-18T11:30:44.761 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:44.761 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:44.772 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:30:44.777 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:44.776+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:44.878 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:44.877+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:44.911 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:44.911 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:44.920 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:30:44.979 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:44.977+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:45.058 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:45.059 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:45.067 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:30:45.079 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:45.078+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:45.164 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:45.165 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:45.173 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:30:45.181 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:45.180+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:45.281 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:45.280+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:45.283 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:45.283 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:45.290 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:45.289+0000 7fd2785a3640 -1 osd.6 62 handle_osd_map injecting map crc failure 2024-09-18T11:30:45.291 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is unset 2024-09-18T11:30:45.292 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:30:45.293 INFO:tasks.rados.rados.0.smithi119.stdout:983: done (1 left) 2024-09-18T11:30:45.293 INFO:tasks.rados.rados.0.smithi119.stdout:984: done (0 left) 2024-09-18T11:30:45.293 INFO:tasks.rados.rados.0.smithi119.stdout:985: write (excl) oid 14 current snap is 20 2024-09-18T11:30:45.293 INFO:tasks.rados.rados.0.smithi119.stdout:985: seq_num 153 ranges {571471=640157,1653392=524384} 2024-09-18T11:30:45.299 INFO:tasks.rados.rados.0.smithi119.stdout:985: writing smithi11939506-14 from 571471 to 1211628 tid 1 2024-09-18T11:30:45.305 INFO:tasks.rados.rados.0.smithi119.stdout:985: writing smithi11939506-14 from 1653392 to 2177776 tid 2 2024-09-18T11:30:45.305 INFO:tasks.rados.rados.0.smithi119.stdout:986: read oid 48 snap 11 2024-09-18T11:30:45.305 INFO:tasks.rados.rados.0.smithi119.stdout:986: expect (ObjNum 47 snap 0 seq_num 47) 2024-09-18T11:30:45.305 INFO:tasks.rados.rados.0.smithi119.stdout:987: write oid 6 current snap is 20 2024-09-18T11:30:45.305 INFO:tasks.rados.rados.0.smithi119.stdout:987: seq_num 154 ranges {588278=715406,1711874=668042} 2024-09-18T11:30:45.312 INFO:tasks.rados.rados.0.smithi119.stdout:987: writing smithi11939506-6 from 588278 to 1303684 tid 1 2024-09-18T11:30:45.318 INFO:tasks.rados.rados.0.smithi119.stdout:987: writing smithi11939506-6 from 1711874 to 2379916 tid 2 2024-09-18T11:30:45.318 INFO:tasks.rados.rados.0.smithi119.stdout:988: set_chunk oid 20 target oid 2024-09-18T11:30:45.338 INFO:tasks.rados.rados.0.smithi119.stdout:988: set_chunk oid 20 offset: 627712 length: 1995264 target oid fdc347be713e13884ffef07bdc68c3ed8ea152f90464c1c40a33240e941fc9a0 offset: 0 snap 14 2024-09-18T11:30:45.338 INFO:tasks.rados.rados.0.smithi119.stdout:989: read oid 27 snap -1 2024-09-18T11:30:45.338 INFO:tasks.rados.rados.0.smithi119.stdout:989: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-18T11:30:45.338 INFO:tasks.rados.rados.0.smithi119.stdout:990: write oid 35 current snap is 20 2024-09-18T11:30:45.338 INFO:tasks.rados.rados.0.smithi119.stdout:985: finishing write tid 1 to smithi11939506-14 2024-09-18T11:30:45.338 INFO:tasks.rados.rados.0.smithi119.stdout:985: finishing write tid 2 to smithi11939506-14 2024-09-18T11:30:45.339 INFO:tasks.rados.rados.0.smithi119.stdout:985: finishing write tid 3 to smithi11939506-14 2024-09-18T11:30:45.339 INFO:tasks.rados.rados.0.smithi119.stdout:985: finishing write tid 4 to smithi11939506-14 2024-09-18T11:30:45.339 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 89 (ObjNum 153 snap 20 seq_num 153) dirty exists 2024-09-18T11:30:45.339 INFO:tasks.rados.rados.0.smithi119.stdout:985: left oid 14 (ObjNum 153 snap 20 seq_num 153) 2024-09-18T11:30:45.339 INFO:tasks.rados.rados.0.smithi119.stdout:986: expect (ObjNum 47 snap 0 seq_num 47) 2024-09-18T11:30:45.382 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:45.381+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:45.426 INFO:tasks.rados.rados.0.smithi119.stdout:987: finishing write tid 1 to smithi11939506-6 2024-09-18T11:30:45.426 INFO:tasks.rados.rados.0.smithi119.stdout:987: finishing write tid 2 to smithi11939506-6 2024-09-18T11:30:45.426 INFO:tasks.rados.rados.0.smithi119.stdout:987: finishing write tid 3 to smithi11939506-6 2024-09-18T11:30:45.426 INFO:tasks.rados.rados.0.smithi119.stdout:987: finishing write tid 4 to smithi11939506-6 2024-09-18T11:30:45.426 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 6 v 41 (ObjNum 154 snap 20 seq_num 154) dirty exists 2024-09-18T11:30:45.427 INFO:tasks.rados.rados.0.smithi119.stdout:987: left oid 6 (ObjNum 154 snap 20 seq_num 154) 2024-09-18T11:30:45.427 INFO:tasks.rados.rados.0.smithi119.stdout:990: seq_num 155 ranges {605085=790655,2170356=434729} 2024-09-18T11:30:45.432 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:45.432 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:45.433 INFO:tasks.rados.rados.0.smithi119.stdout:990: writing smithi11939506-35 from 605085 to 1395740 tid 1 2024-09-18T11:30:45.440 INFO:tasks.rados.rados.0.smithi119.stdout:990: writing smithi11939506-35 from 2170356 to 2605085 tid 2 2024-09-18T11:30:45.440 INFO:tasks.rados.rados.0.smithi119.stdout:985: done (5 left) 2024-09-18T11:30:45.440 INFO:tasks.rados.rados.0.smithi119.stdout:986: done (4 left) 2024-09-18T11:30:45.440 INFO:tasks.rados.rados.0.smithi119.stdout:987: done (3 left) 2024-09-18T11:30:45.440 INFO:tasks.rados.rados.0.smithi119.stdout:991: tier_promote oid 47 2024-09-18T11:30:45.440 INFO:tasks.rados.rados.0.smithi119.stdout:989: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-18T11:30:45.441 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:30:45.483 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:45.482+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:45.517 INFO:tasks.rados.rados.0.smithi119.stdout:988: finishing set_chunk to oid 20 2024-09-18T11:30:45.517 INFO:tasks.rados.rados.0.smithi119.stdout:990: finishing write tid 1 to smithi11939506-35 2024-09-18T11:30:45.518 INFO:tasks.rados.rados.0.smithi119.stdout:991: got (0) Success 2024-09-18T11:30:45.518 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 33 (ObjNum 33 snap 0 seq_num 384198464) clean dne 2024-09-18T11:30:45.518 INFO:tasks.rados.rados.0.smithi119.stdout:990: finishing write tid 2 to smithi11939506-35 2024-09-18T11:30:45.518 INFO:tasks.rados.rados.0.smithi119.stdout:988: done (3 left) 2024-09-18T11:30:45.518 INFO:tasks.rados.rados.0.smithi119.stdout:989: done (2 left) 2024-09-18T11:30:45.518 INFO:tasks.rados.rados.0.smithi119.stdout:991: done (1 left) 2024-09-18T11:30:45.518 INFO:tasks.rados.rados.0.smithi119.stdout:992: read oid 17 snap -1 2024-09-18T11:30:45.518 INFO:tasks.rados.rados.0.smithi119.stdout:992: expect (ObjNum 129 snap 15 seq_num 129) 2024-09-18T11:30:45.518 INFO:tasks.rados.rados.0.smithi119.stdout:993: tier_promote oid 42 2024-09-18T11:30:45.518 INFO:tasks.rados.rados.0.smithi119.stdout:990: finishing write tid 3 to smithi11939506-35 2024-09-18T11:30:45.518 INFO:tasks.rados.rados.0.smithi119.stdout:990: finishing write tid 4 to smithi11939506-35 2024-09-18T11:30:45.518 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 51 (ObjNum 155 snap 20 seq_num 155) dirty exists 2024-09-18T11:30:45.519 INFO:tasks.rados.rados.0.smithi119.stdout:990: left oid 35 (ObjNum 155 snap 20 seq_num 155) 2024-09-18T11:30:45.519 INFO:tasks.rados.rados.0.smithi119.stdout:990: done (2 left) 2024-09-18T11:30:45.519 INFO:tasks.rados.rados.0.smithi119.stdout:994: read oid 6 snap -1 2024-09-18T11:30:45.519 INFO:tasks.rados.rados.0.smithi119.stdout:994: expect (ObjNum 154 snap 20 seq_num 154) 2024-09-18T11:30:45.519 INFO:tasks.rados.rados.0.smithi119.stdout:995: snap_remove snap 18 2024-09-18T11:30:45.541 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:45.541 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:45.550 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:30:45.584 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:45.584+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:45.647 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:45.647 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:45.655 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:30:45.700 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:45.699+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:45.754 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:45.754 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:45.762 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:30:45.787 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:45.786+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:45.888 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:45.887+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:45.899 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:45.900 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:45.908 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:30:45.989 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:45.988+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:46.005 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:46.006 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:46.014 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:30:46.071 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:30:46.099 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:46.098+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:46.152 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:46.152 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:46.160 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:30:46.188 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:30:46.188 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:30:46.190 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:46.189+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:46.203 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:46.204 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:46.218 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:46.218 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:46.233 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:46.233 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:46.250 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:46.250 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:46.265 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:46.265 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:46.281 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:46.281 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:46.291 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:46.290+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:46.296 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:46.296 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:46.302 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:46.302 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:46.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:46.303 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:46.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.303 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:46.303 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:46.303 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.183577036000003, 2024-09-18T11:30:46.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:30:46.303 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.303 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:46.303 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.303 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:46.303 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:46.303 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:30:46.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.304 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.304 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:46.304 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.304 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:46.304 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:30:46.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:46.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:30:46.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:30:46.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:30:46.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:30:46.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:30:46.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:30:46.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:30:46.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:30:46.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:46.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:30:46.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:30:46.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:30:46.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:30:46.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:30:46.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:30:46.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.308 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.308 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:30:46.308 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:30:46.308 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.308 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.308 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.308 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.308 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:897 4.5 4:aa575018:::smithi11939506-11:2 [sparse-read 0~1343176] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:46.308 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.133034+0000", 2024-09-18T11:30:46.308 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.166862035999998, 2024-09-18T11:30:46.308 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051617950000000003, 2024-09-18T11:30:46.309 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.309 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:46.309 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.309 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:46.309 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:46.309 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 897 2024-09-18T11:30:46.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.309 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.309 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133034+0000", 2024-09-18T11:30:46.309 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133034+0000", 2024-09-18T11:30:46.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133033+0000", 2024-09-18T11:30:46.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:46.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133037+0000", 2024-09-18T11:30:46.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.118e-06 2024-09-18T11:30:46.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133045+0000", 2024-09-18T11:30:46.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1780000000000003e-06 2024-09-18T11:30:46.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133049+0000", 2024-09-18T11:30:46.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7730000000000001e-06 2024-09-18T11:30:46.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.312 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.137838+0000", 2024-09-18T11:30:46.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00478912 2024-09-18T11:30:46.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.312 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.137936+0000", 2024-09-18T11:30:46.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.823e-05 2024-09-18T11:30:46.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.313 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138196+0000", 2024-09-18T11:30:46.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00025993799999999998 2024-09-18T11:30:46.313 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.313 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.313 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.313 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:898 4.5 4:aa575018:::smithi11939506-11:2 [sparse-read 0~1343176] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:46.313 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:46.314 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.166832997999997, 2024-09-18T11:30:46.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055588349999999998, 2024-09-18T11:30:46.314 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.314 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:46.314 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.314 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:46.314 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:46.314 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 898 2024-09-18T11:30:46.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.314 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.314 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.314 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:46.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:46.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133063+0000", 2024-09-18T11:30:46.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:46.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133066+0000", 2024-09-18T11:30:46.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2729999999999998e-06 2024-09-18T11:30:46.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133072+0000", 2024-09-18T11:30:46.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0680000000000002e-06 2024-09-18T11:30:46.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.133077+0000", 2024-09-18T11:30:46.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3569999999999997e-06 2024-09-18T11:30:46.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138226+0000", 2024-09-18T11:30:46.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051480980000000003 2024-09-18T11:30:46.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138324+0000", 2024-09-18T11:30:46.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8639000000000002e-05 2024-09-18T11:30:46.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138622+0000", 2024-09-18T11:30:46.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029774100000000002 2024-09-18T11:30:46.318 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.318 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.318 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.318 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1131 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2605052,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:46.318 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179072+0000", 2024-09-18T11:30:46.318 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.120824377000002, 2024-09-18T11:30:46.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050097329999999997, 2024-09-18T11:30:46.318 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.319 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:46.319 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.319 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:46.319 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:46.319 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1131 2024-09-18T11:30:46.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.319 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179072+0000", 2024-09-18T11:30:46.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179072+0000", 2024-09-18T11:30:46.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179070+0000", 2024-09-18T11:30:46.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:46.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179084+0000", 2024-09-18T11:30:46.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3689999999999999e-05 2024-09-18T11:30:46.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179105+0000", 2024-09-18T11:30:46.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0605999999999999e-05 2024-09-18T11:30:46.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179115+0000", 2024-09-18T11:30:46.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0862999999999999e-05 2024-09-18T11:30:46.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179365+0000", 2024-09-18T11:30:46.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00024911399999999999 2024-09-18T11:30:46.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:46.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179382+0000", 2024-09-18T11:30:46.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7856999999999999e-05 2024-09-18T11:30:46.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.183178+0000", 2024-09-18T11:30:46.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037957780000000001 2024-09-18T11:30:46.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.183256+0000", 2024-09-18T11:30:46.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7476000000000002e-05 2024-09-18T11:30:46.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184081+0000", 2024-09-18T11:30:46.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00082578199999999995 2024-09-18T11:30:46.323 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.323 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.323 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.324 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1132 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2605052,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:46.324 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179179+0000", 2024-09-18T11:30:46.324 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.120717032000002, 2024-09-18T11:30:46.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059816840000000001, 2024-09-18T11:30:46.324 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.324 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:46.324 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.324 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:46.324 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:46.324 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1132 2024-09-18T11:30:46.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.325 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179179+0000", 2024-09-18T11:30:46.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179179+0000", 2024-09-18T11:30:46.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179178+0000", 2024-09-18T11:30:46.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:46.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179193+0000", 2024-09-18T11:30:46.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5446000000000001e-05 2024-09-18T11:30:46.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179215+0000", 2024-09-18T11:30:46.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1253000000000001e-05 2024-09-18T11:30:46.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179227+0000", 2024-09-18T11:30:46.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2612999999999999e-05 2024-09-18T11:30:46.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179447+0000", 2024-09-18T11:30:46.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021972199999999999 2024-09-18T11:30:46.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:46.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179453+0000", 2024-09-18T11:30:46.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8830000000000003e-06 2024-09-18T11:30:46.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.328 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184113+0000", 2024-09-18T11:30:46.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046603180000000001 2024-09-18T11:30:46.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.328 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184229+0000", 2024-09-18T11:30:46.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000115516 2024-09-18T11:30:46.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185161+0000", 2024-09-18T11:30:46.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00093216000000000002 2024-09-18T11:30:46.329 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.329 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.329 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.329 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:46.329 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:30:46.329 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.630551635, 2024-09-18T11:30:46.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:30:46.329 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.330 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:46.330 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.330 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:46.330 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:46.330 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:30:46.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.330 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:30:46.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:30:46.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:30:46.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:46.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:30:46.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:30:46.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:30:46.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:30:46.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:30:46.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:30:46.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:30:46.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:30:46.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:30:46.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:30:46.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:30:46.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:30:46.333 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.333 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.333 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.334 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1185 4.5 4:af4ba75a:::smithi11939506-37:head [write 729437~527542] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:46.334 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.156648+0000", 2024-09-18T11:30:46.334 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.143247711000001, 2024-09-18T11:30:46.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057485540000000003, 2024-09-18T11:30:46.334 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.334 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.334 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.334 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:46.334 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:46.334 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1185 2024-09-18T11:30:46.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.335 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.156648+0000", 2024-09-18T11:30:46.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.156648+0000", 2024-09-18T11:30:46.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.156647+0000", 2024-09-18T11:30:46.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:46.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157166+0000", 2024-09-18T11:30:46.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00051975999999999997 2024-09-18T11:30:46.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157183+0000", 2024-09-18T11:30:46.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.611e-05 2024-09-18T11:30:46.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157188+0000", 2024-09-18T11:30:46.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.648e-06 2024-09-18T11:30:46.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157253+0000", 2024-09-18T11:30:46.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4503000000000002e-05 2024-09-18T11:30:46.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157322+0000", 2024-09-18T11:30:46.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9031999999999999e-05 2024-09-18T11:30:46.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:46.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157437+0000", 2024-09-18T11:30:46.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011543400000000001 2024-09-18T11:30:46.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162104+0000", 2024-09-18T11:30:46.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046668009999999999 2024-09-18T11:30:46.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162321+0000", 2024-09-18T11:30:46.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021706699999999999 2024-09-18T11:30:46.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162355+0000", 2024-09-18T11:30:46.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3670000000000001e-05 2024-09-18T11:30:46.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162397+0000", 2024-09-18T11:30:46.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2168e-05 2024-09-18T11:30:46.340 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.340 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.340 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.340 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1186 4.5 4:af4ba75a:::smithi11939506-37:head [write 1925626~643351] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:46.340 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.163566+0000", 2024-09-18T11:30:46.340 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.136330125000001, 2024-09-18T11:30:46.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051394919999999998, 2024-09-18T11:30:46.340 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.340 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.340 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.340 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:46.341 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:46.341 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1186 2024-09-18T11:30:46.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.341 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.163566+0000", 2024-09-18T11:30:46.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.163566+0000", 2024-09-18T11:30:46.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.163565+0000", 2024-09-18T11:30:46.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:46.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164172+0000", 2024-09-18T11:30:46.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060731000000000001 2024-09-18T11:30:46.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164184+0000", 2024-09-18T11:30:46.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1202999999999999e-05 2024-09-18T11:30:46.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164188+0000", 2024-09-18T11:30:46.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2810000000000002e-06 2024-09-18T11:30:46.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164212+0000", 2024-09-18T11:30:46.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4227999999999999e-05 2024-09-18T11:30:46.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164275+0000", 2024-09-18T11:30:46.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3264999999999998e-05 2024-09-18T11:30:46.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:46.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.164376+0000", 2024-09-18T11:30:46.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000100821 2024-09-18T11:30:46.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168331+0000", 2024-09-18T11:30:46.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039549779999999996 2024-09-18T11:30:46.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168621+0000", 2024-09-18T11:30:46.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029003900000000002 2024-09-18T11:30:46.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168662+0000", 2024-09-18T11:30:46.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1307999999999998e-05 2024-09-18T11:30:46.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.168705+0000", 2024-09-18T11:30:46.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2994999999999997e-05 2024-09-18T11:30:46.346 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.346 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.346 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.346 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1429 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 10=[10,e,c] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T11:30:46.346 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:29.402624+0000", 2024-09-18T11:30:46.346 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 16.897271718999999, 2024-09-18T11:30:46.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0052753469999999997, 2024-09-18T11:30:46.346 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.346 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.347 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.347 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:46.347 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:46.347 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1429 2024-09-18T11:30:46.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.347 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402624+0000", 2024-09-18T11:30:46.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402624+0000", 2024-09-18T11:30:46.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402624+0000", 2024-09-18T11:30:46.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:46.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402627+0000", 2024-09-18T11:30:46.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7870000000000002e-06 2024-09-18T11:30:46.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402632+0000", 2024-09-18T11:30:46.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9520000000000002e-06 2024-09-18T11:30:46.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402636+0000", 2024-09-18T11:30:46.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.809e-06 2024-09-18T11:30:46.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402696+0000", 2024-09-18T11:30:46.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9150000000000001e-05 2024-09-18T11:30:46.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:46.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.402698+0000", 2024-09-18T11:30:46.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4880000000000001e-06 2024-09-18T11:30:46.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.406581+0000", 2024-09-18T11:30:46.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038825840000000001 2024-09-18T11:30:46.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.406633+0000", 2024-09-18T11:30:46.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2602000000000002e-05 2024-09-18T11:30:46.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:46.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.406762+0000", 2024-09-18T11:30:46.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000128804 2024-09-18T11:30:46.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.407717+0000", 2024-09-18T11:30:46.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00095522800000000002 2024-09-18T11:30:46.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.407853+0000", 2024-09-18T11:30:46.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000135938 2024-09-18T11:30:46.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.407879+0000", 2024-09-18T11:30:46.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5476000000000001e-05 2024-09-18T11:30:46.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.407900+0000", 2024-09-18T11:30:46.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0941000000000002e-05 2024-09-18T11:30:46.353 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.353 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.353 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.353 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1688 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 19=[19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-18T11:30:46.353 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:30:46.353 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 6.7673970419999998, 2024-09-18T11:30:46.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071575789999999999, 2024-09-18T11:30:46.354 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.354 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.354 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.354 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:46.354 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:46.354 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1688 2024-09-18T11:30:46.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.354 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:30:46.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:30:46.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:30:46.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:46.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532500+0000", 2024-09-18T11:30:46.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5260000000000001e-06 2024-09-18T11:30:46.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532503+0000", 2024-09-18T11:30:46.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:30:46.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532505+0000", 2024-09-18T11:30:46.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.889e-06 2024-09-18T11:30:46.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.534722+0000", 2024-09-18T11:30:46.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002217119 2024-09-18T11:30:46.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.534827+0000", 2024-09-18T11:30:46.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0001049 2024-09-18T11:30:46.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:46.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.535007+0000", 2024-09-18T11:30:46.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017965599999999999 2024-09-18T11:30:46.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539463+0000", 2024-09-18T11:30:46.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004455713 2024-09-18T11:30:46.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539607+0000", 2024-09-18T11:30:46.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014440300000000001 2024-09-18T11:30:46.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539634+0000", 2024-09-18T11:30:46.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7398000000000001e-05 2024-09-18T11:30:46.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539657+0000", 2024-09-18T11:30:46.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2228000000000001e-05 2024-09-18T11:30:46.359 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.359 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.359 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.360 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1758 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2319366,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:46.360 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.293909+0000", 2024-09-18T11:30:46.360 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.0059866089999998, 2024-09-18T11:30:46.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056075329999999996, 2024-09-18T11:30:46.360 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.360 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:46.360 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.360 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:46.360 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:46.360 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1758 2024-09-18T11:30:46.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.361 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293909+0000", 2024-09-18T11:30:46.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293909+0000", 2024-09-18T11:30:46.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293907+0000", 2024-09-18T11:30:46.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:46.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293918+0000", 2024-09-18T11:30:46.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1432e-05 2024-09-18T11:30:46.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293939+0000", 2024-09-18T11:30:46.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0874999999999999e-05 2024-09-18T11:30:46.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293948+0000", 2024-09-18T11:30:46.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4700000000000002e-06 2024-09-18T11:30:46.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:30:46.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6805000000000003e-05 2024-09-18T11:30:46.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:46.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294002+0000", 2024-09-18T11:30:46.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2629999999999997e-06 2024-09-18T11:30:46.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.295647+0000", 2024-09-18T11:30:46.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016449329999999999 2024-09-18T11:30:46.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.295706+0000", 2024-09-18T11:30:46.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9011999999999997e-05 2024-09-18T11:30:46.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.299517+0000", 2024-09-18T11:30:46.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038113359999999998 2024-09-18T11:30:46.365 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.365 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.365 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.365 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1759 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2319366,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:46.365 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:30:46.365 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.0059368319999997, 2024-09-18T11:30:46.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064323039999999998, 2024-09-18T11:30:46.366 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.366 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:46.366 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.366 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:46.366 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:46.366 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1759 2024-09-18T11:30:46.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.366 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:30:46.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:30:46.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:30:46.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:46.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293963+0000", 2024-09-18T11:30:46.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0810000000000004e-06 2024-09-18T11:30:46.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293970+0000", 2024-09-18T11:30:46.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5109999999999997e-06 2024-09-18T11:30:46.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293975+0000", 2024-09-18T11:30:46.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.33e-06 2024-09-18T11:30:46.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294078+0000", 2024-09-18T11:30:46.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102871 2024-09-18T11:30:46.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:46.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294084+0000", 2024-09-18T11:30:46.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5470000000000002e-06 2024-09-18T11:30:46.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.299550+0000", 2024-09-18T11:30:46.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054660799999999999 2024-09-18T11:30:46.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.299608+0000", 2024-09-18T11:30:46.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8307000000000003e-05 2024-09-18T11:30:46.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300392+0000", 2024-09-18T11:30:46.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078306999999999997 2024-09-18T11:30:46.370 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.370 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.370 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.371 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1760 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2319366,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:46.371 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:30:46.371 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.0059024919999997, 2024-09-18T11:30:46.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070132529999999997, 2024-09-18T11:30:46.371 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.371 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:46.371 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.371 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:46.371 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:46.371 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1760 2024-09-18T11:30:46.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.371 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:30:46.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:30:46.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293993+0000", 2024-09-18T11:30:46.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:46.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293997+0000", 2024-09-18T11:30:46.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4999999999999999e-06 2024-09-18T11:30:46.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294004+0000", 2024-09-18T11:30:46.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3950000000000003e-06 2024-09-18T11:30:46.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294009+0000", 2024-09-18T11:30:46.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4989999999999996e-06 2024-09-18T11:30:46.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294103+0000", 2024-09-18T11:30:46.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4817999999999999e-05 2024-09-18T11:30:46.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:46.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294110+0000", 2024-09-18T11:30:46.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7240000000000003e-06 2024-09-18T11:30:46.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300406+0000", 2024-09-18T11:30:46.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062964470000000002 2024-09-18T11:30:46.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300457+0000", 2024-09-18T11:30:46.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0915000000000003e-05 2024-09-18T11:30:46.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.301007+0000", 2024-09-18T11:30:46.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00054942299999999995 2024-09-18T11:30:46.376 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.376 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.376 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.376 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1774 4.a 4:588debc2:::smithi11939506-18:head [write 420208~530210,stat] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:46.376 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.889077+0000", 2024-09-18T11:30:46.376 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.4108186370000002, 2024-09-18T11:30:46.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055554150000000002, 2024-09-18T11:30:46.376 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.376 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.376 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.376 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:46.376 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:46.377 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1774 2024-09-18T11:30:46.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.377 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.889077+0000", 2024-09-18T11:30:46.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.889077+0000", 2024-09-18T11:30:46.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.889075+0000", 2024-09-18T11:30:46.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:46.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.890150+0000", 2024-09-18T11:30:46.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001074249 2024-09-18T11:30:46.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.890169+0000", 2024-09-18T11:30:46.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9612000000000001e-05 2024-09-18T11:30:46.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.890176+0000", 2024-09-18T11:30:46.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5490000000000003e-06 2024-09-18T11:30:46.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.890221+0000", 2024-09-18T11:30:46.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5664999999999997e-05 2024-09-18T11:30:46.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.890297+0000", 2024-09-18T11:30:46.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5512999999999993e-05 2024-09-18T11:30:46.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:46.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.890453+0000", 2024-09-18T11:30:46.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015646400000000001 2024-09-18T11:30:46.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.892957+0000", 2024-09-18T11:30:46.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025035309999999998 2024-09-18T11:30:46.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.894568+0000", 2024-09-18T11:30:46.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016112469999999999 2024-09-18T11:30:46.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.894604+0000", 2024-09-18T11:30:46.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.557e-05 2024-09-18T11:30:46.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.894633+0000", 2024-09-18T11:30:46.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9071999999999999e-05 2024-09-18T11:30:46.382 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.382 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.382 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.382 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1775 4.a 4:588debc2:::smithi11939506-18:head [write 1527054~766050] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:46.382 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:30:46.382 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.4035497509999999, 2024-09-18T11:30:46.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070939050000000002, 2024-09-18T11:30:46.382 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.383 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.383 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.383 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:46.383 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:46.383 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1775 2024-09-18T11:30:46.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.383 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:30:46.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:30:46.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896345+0000", 2024-09-18T11:30:46.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:46.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896993+0000", 2024-09-18T11:30:46.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064769999999999997 2024-09-18T11:30:46.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897011+0000", 2024-09-18T11:30:46.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8355999999999998e-05 2024-09-18T11:30:46.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897016+0000", 2024-09-18T11:30:46.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3890000000000004e-06 2024-09-18T11:30:46.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897054+0000", 2024-09-18T11:30:46.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7963000000000002e-05 2024-09-18T11:30:46.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897123+0000", 2024-09-18T11:30:46.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8128999999999998e-05 2024-09-18T11:30:46.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:46.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897224+0000", 2024-09-18T11:30:46.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010168800000000001 2024-09-18T11:30:46.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899515+0000", 2024-09-18T11:30:46.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022910280000000001 2024-09-18T11:30:46.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903333+0000", 2024-09-18T11:30:46.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003817459 2024-09-18T11:30:46.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903388+0000", 2024-09-18T11:30:46.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5651e-05 2024-09-18T11:30:46.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903440+0000", 2024-09-18T11:30:46.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1836999999999999e-05 2024-09-18T11:30:46.388 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.388 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.388 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.389 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1776 4.a 4:588debc2:::smithi11939506-18:head [setxattr _header (57) in=7b,truncate 2420208] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:46.389 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:30:46.389 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.4028719390000002, 2024-09-18T11:30:46.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065840400000000002, 2024-09-18T11:30:46.389 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.389 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.389 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.389 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:46.389 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:46.389 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1776 2024-09-18T11:30:46.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.389 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:30:46.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:30:46.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:30:46.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:46.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897026+0000", 2024-09-18T11:30:46.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9579999999999999e-06 2024-09-18T11:30:46.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897030+0000", 2024-09-18T11:30:46.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5370000000000002e-06 2024-09-18T11:30:46.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897032+0000", 2024-09-18T11:30:46.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.17e-06 2024-09-18T11:30:46.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898415+0000", 2024-09-18T11:30:46.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013823189999999999 2024-09-18T11:30:46.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898527+0000", 2024-09-18T11:30:46.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011236800000000001 2024-09-18T11:30:46.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:46.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898714+0000", 2024-09-18T11:30:46.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018703599999999999 2024-09-18T11:30:46.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899683+0000", 2024-09-18T11:30:46.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00096944700000000002 2024-09-18T11:30:46.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903475+0000", 2024-09-18T11:30:46.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037920129999999999 2024-09-18T11:30:46.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903535+0000", 2024-09-18T11:30:46.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9843999999999998e-05 2024-09-18T11:30:46.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903608+0000", 2024-09-18T11:30:46.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2946000000000003e-05 2024-09-18T11:30:46.394 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.394 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.394 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.394 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1777 4.a 4:588debc2:::smithi11939506-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:46.394 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:30:46.395 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.402845299, 2024-09-18T11:30:46.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0069010809999999999, 2024-09-18T11:30:46.395 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.395 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:46.395 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.395 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:46.395 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:46.395 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1777 2024-09-18T11:30:46.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.395 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:30:46.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:30:46.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897050+0000", 2024-09-18T11:30:46.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:46.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897052+0000", 2024-09-18T11:30:46.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:30:46.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897057+0000", 2024-09-18T11:30:46.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9470000000000002e-06 2024-09-18T11:30:46.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897060+0000", 2024-09-18T11:30:46.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3410000000000001e-06 2024-09-18T11:30:46.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899417+0000", 2024-09-18T11:30:46.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023571170000000002 2024-09-18T11:30:46.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:46.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899501+0000", 2024-09-18T11:30:46.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4160999999999996e-05 2024-09-18T11:30:46.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903661+0000", 2024-09-18T11:30:46.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041599510000000003 2024-09-18T11:30:46.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903790+0000", 2024-09-18T11:30:46.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012941499999999999 2024-09-18T11:30:46.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903952+0000", 2024-09-18T11:30:46.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016146800000000001 2024-09-18T11:30:46.399 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.399 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.399 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.400 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1815 4.5 4:aa575018:::smithi11939506-11:head [write 1760982~657035] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:46.400 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.948313+0000", 2024-09-18T11:30:46.400 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.3515830470000001, 2024-09-18T11:30:46.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056498440000000002, 2024-09-18T11:30:46.400 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.400 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.400 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.400 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:46.400 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:46.400 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1815 2024-09-18T11:30:46.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.400 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.948313+0000", 2024-09-18T11:30:46.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.948313+0000", 2024-09-18T11:30:46.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.948301+0000", 2024-09-18T11:30:46.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999881 2024-09-18T11:30:46.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949192+0000", 2024-09-18T11:30:46.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089072299999999995 2024-09-18T11:30:46.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949207+0000", 2024-09-18T11:30:46.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4705999999999999e-05 2024-09-18T11:30:46.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949211+0000", 2024-09-18T11:30:46.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5020000000000003e-06 2024-09-18T11:30:46.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949274+0000", 2024-09-18T11:30:46.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2951000000000001e-05 2024-09-18T11:30:46.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949331+0000", 2024-09-18T11:30:46.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7343999999999998e-05 2024-09-18T11:30:46.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:46.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949429+0000", 2024-09-18T11:30:46.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7899999999999994e-05 2024-09-18T11:30:46.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.953537+0000", 2024-09-18T11:30:46.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004108006 2024-09-18T11:30:46.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.953885+0000", 2024-09-18T11:30:46.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00034758 2024-09-18T11:30:46.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.953932+0000", 2024-09-18T11:30:46.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7345000000000002e-05 2024-09-18T11:30:46.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.953963+0000", 2024-09-18T11:30:46.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0716999999999997e-05 2024-09-18T11:30:46.405 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.405 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.405 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.406 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1816 4.5 4:aa575018:::smithi11939506-11:head [setxattr _header (57) in=7b,truncate 2487436] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:46.406 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.949217+0000", 2024-09-18T11:30:46.406 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.350678834, 2024-09-18T11:30:46.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049175290000000003, 2024-09-18T11:30:46.406 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.406 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.406 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.406 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:46.406 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:46.406 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1816 2024-09-18T11:30:46.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.406 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949217+0000", 2024-09-18T11:30:46.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949217+0000", 2024-09-18T11:30:46.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949217+0000", 2024-09-18T11:30:46.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:46.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949219+0000", 2024-09-18T11:30:46.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6879999999999999e-06 2024-09-18T11:30:46.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949223+0000", 2024-09-18T11:30:46.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4300000000000002e-06 2024-09-18T11:30:46.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949227+0000", 2024-09-18T11:30:46.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6389999999999998e-06 2024-09-18T11:30:46.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.950450+0000", 2024-09-18T11:30:46.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012233039999999999 2024-09-18T11:30:46.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.950515+0000", 2024-09-18T11:30:46.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5571e-05 2024-09-18T11:30:46.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:46.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.950610+0000", 2024-09-18T11:30:46.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4665000000000003e-05 2024-09-18T11:30:46.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.953977+0000", 2024-09-18T11:30:46.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0033670150000000001 2024-09-18T11:30:46.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.954077+0000", 2024-09-18T11:30:46.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010006299999999999 2024-09-18T11:30:46.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.954106+0000", 2024-09-18T11:30:46.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8574000000000001e-05 2024-09-18T11:30:46.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.954135+0000", 2024-09-18T11:30:46.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9042000000000001e-05 2024-09-18T11:30:46.411 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.411 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.411 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.412 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1817 4.5 4:aa575018:::smithi11939506-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:46.412 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.949229+0000", 2024-09-18T11:30:46.412 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.3506669009999999, 2024-09-18T11:30:46.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050388910000000002, 2024-09-18T11:30:46.412 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.412 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:46.412 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.412 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:46.412 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:46.412 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1817 2024-09-18T11:30:46.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949229+0000", 2024-09-18T11:30:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949229+0000", 2024-09-18T11:30:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949229+0000", 2024-09-18T11:30:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949230+0000", 2024-09-18T11:30:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.502e-06 2024-09-18T11:30:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949235+0000", 2024-09-18T11:30:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.729e-06 2024-09-18T11:30:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949237+0000", 2024-09-18T11:30:46.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0770000000000002e-06 2024-09-18T11:30:46.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.950826+0000", 2024-09-18T11:30:46.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015888040000000001 2024-09-18T11:30:46.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:46.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.950909+0000", 2024-09-18T11:30:46.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2823000000000003e-05 2024-09-18T11:30:46.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.954163+0000", 2024-09-18T11:30:46.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032537149999999999 2024-09-18T11:30:46.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.954206+0000", 2024-09-18T11:30:46.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3409999999999999e-05 2024-09-18T11:30:46.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.954268+0000", 2024-09-18T11:30:46.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2068000000000003e-05 2024-09-18T11:30:46.417 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.417 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.417 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.417 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.417 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.417 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:46.418 INFO:tasks.rados.rados.0.smithi119.stdout:995: done (3 left) 2024-09-18T11:30:46.418 INFO:tasks.rados.rados.0.smithi119.stdout:996: snap_remove snap 13 2024-09-18T11:30:46.419 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:30:46.420 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:46.417+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:46.519 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:46.519 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:46.519 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:46.519 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:46.519 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:30:46.518+0000 7fa4739f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33317) UID: 0 2024-09-18T11:30:46.527 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:30:46.620 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:46.618+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:46.624 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:46.624 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:46.625 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:46.625 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:46.625 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:46.633 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:30:46.721 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:46.720+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:46.772 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:46.772 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:46.772 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:46.772 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:46.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.772 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:46.772 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:46.773 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.612201794999997, 2024-09-18T11:30:46.773 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054864400000000004, 2024-09-18T11:30:46.773 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.773 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.773 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.773 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:46.773 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:46.773 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 42 2024-09-18T11:30:46.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.773 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.773 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.773 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.773 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:46.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.774 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:46.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.774 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:46.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:46.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.775 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.775 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157416+0000", 2024-09-18T11:30:46.775 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.721e-06 2024-09-18T11:30:46.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.775 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.775 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157423+0000", 2024-09-18T11:30:46.775 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4049999999999997e-06 2024-09-18T11:30:46.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.775 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.775 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157425+0000", 2024-09-18T11:30:46.775 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.481e-06 2024-09-18T11:30:46.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.776 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158649+0000", 2024-09-18T11:30:46.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001223453 2024-09-18T11:30:46.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.776 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158729+0000", 2024-09-18T11:30:46.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0205999999999997e-05 2024-09-18T11:30:46.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.776 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:46.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158875+0000", 2024-09-18T11:30:46.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014628800000000001 2024-09-18T11:30:46.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.777 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161518+0000", 2024-09-18T11:30:46.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026427949999999999 2024-09-18T11:30:46.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.777 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162821+0000", 2024-09-18T11:30:46.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013028099999999999 2024-09-18T11:30:46.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.778 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162864+0000", 2024-09-18T11:30:46.778 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2839000000000001e-05 2024-09-18T11:30:46.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.778 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.778 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162900+0000", 2024-09-18T11:30:46.778 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5948999999999998e-05 2024-09-18T11:30:46.778 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.778 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.778 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.778 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:46.779 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:46.779 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.612115430999999, 2024-09-18T11:30:46.779 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055400850000000001, 2024-09-18T11:30:46.779 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.779 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.779 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.779 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:46.779 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:46.779 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 43 2024-09-18T11:30:46.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.779 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.779 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.779 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.779 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:46.780 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:46.780 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:46.780 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:46.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.781 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157504+0000", 2024-09-18T11:30:46.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4519999999999996e-06 2024-09-18T11:30:46.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.781 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157534+0000", 2024-09-18T11:30:46.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.97e-05 2024-09-18T11:30:46.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.781 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157539+0000", 2024-09-18T11:30:46.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4360000000000001e-06 2024-09-18T11:30:46.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159428+0000", 2024-09-18T11:30:46.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018888220000000001 2024-09-18T11:30:46.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159479+0000", 2024-09-18T11:30:46.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0608000000000001e-05 2024-09-18T11:30:46.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:46.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159559+0000", 2024-09-18T11:30:46.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0667000000000005e-05 2024-09-18T11:30:46.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.783 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162779+0000", 2024-09-18T11:30:46.783 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032197459999999999 2024-09-18T11:30:46.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.783 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162950+0000", 2024-09-18T11:30:46.783 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017059599999999999 2024-09-18T11:30:46.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.783 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163001+0000", 2024-09-18T11:30:46.784 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0968999999999999e-05 2024-09-18T11:30:46.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.784 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.784 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163040+0000", 2024-09-18T11:30:46.784 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8843000000000001e-05 2024-09-18T11:30:46.784 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.784 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.784 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.784 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:46.784 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:46.785 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.612064277000002, 2024-09-18T11:30:46.785 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:30:46.785 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.785 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.785 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.785 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:46.785 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:46.785 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:30:46.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.785 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.785 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.785 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:46.785 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:46.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:30:46.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:46.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:30:46.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:30:46.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:30:46.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:30:46.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:30:46.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:30:46.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.788 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.788 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:30:46.788 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:30:46.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.788 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.788 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:30:46.788 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:30:46.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.788 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:46.789 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:30:46.789 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:30:46.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.789 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.789 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:30:46.789 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:30:46.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.789 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.789 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:30:46.789 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:30:46.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.790 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.790 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.790 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:30:46.790 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:30:46.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.790 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.790 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.790 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:30:46.790 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:30:46.790 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.790 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.790 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.791 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.791 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:46.791 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:46.791 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.611938015, 2024-09-18T11:30:46.791 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055717040000000002, 2024-09-18T11:30:46.791 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.791 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.791 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.791 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:46.791 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:46.791 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 45 2024-09-18T11:30:46.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.792 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.792 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.792 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:46.792 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.792 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.792 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:46.792 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157676+0000", 2024-09-18T11:30:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157694+0000", 2024-09-18T11:30:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7422999999999999e-05 2024-09-18T11:30:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157704+0000", 2024-09-18T11:30:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9000000000000001e-06 2024-09-18T11:30:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157718+0000", 2024-09-18T11:30:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4788e-05 2024-09-18T11:30:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160937+0000", 2024-09-18T11:30:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00321886 2024-09-18T11:30:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161023+0000", 2024-09-18T11:30:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6063999999999995e-05 2024-09-18T11:30:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161202+0000", 2024-09-18T11:30:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017912599999999999 2024-09-18T11:30:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163052+0000", 2024-09-18T11:30:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001849554 2024-09-18T11:30:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163204+0000", 2024-09-18T11:30:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015186900000000001 2024-09-18T11:30:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163229+0000", 2024-09-18T11:30:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:30:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163249+0000", 2024-09-18T11:30:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9992000000000001e-05 2024-09-18T11:30:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.611888901999997, 2024-09-18T11:30:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:30:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:30:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:30:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:30:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:30:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:30:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:30:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:30:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:30:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:30:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:30:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:30:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:30:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:30:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:30:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:30:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:30:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:30:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:30:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:30:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:30:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:30:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.611846370999999, 2024-09-18T11:30:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:30:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:30:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:30:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:30:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:30:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:30:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:30:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:30:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:30:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:30:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:30:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:30:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:30:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:30:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:30:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:30:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:30:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:30:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:30:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:30:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:30:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:30:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:30:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.611791441000001, 2024-09-18T11:30:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:30:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:30:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:30:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:30:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:30:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:30:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:30:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:30:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:30:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:30:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:30:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:30:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:30:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:30:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:30:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:30:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:30:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:30:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:30:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:30:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:30:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:30:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.611781461, 2024-09-18T11:30:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:30:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:30:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:30:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:30:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:30:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:30:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:30:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:30:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:30:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:30:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:30:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:30:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:30:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:30:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:30:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:30:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:30:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:30:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:30:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:30:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:30:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:30:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.611769862000003, 2024-09-18T11:30:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:30:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:30:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:30:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:30:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:30:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:30:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:30:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:30:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:30:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:30:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:30:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:30:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:30:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:30:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:30:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:30:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:30:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:30:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:30:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:30:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:30:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:30:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.611713381000001, 2024-09-18T11:30:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:30:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:30:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:30:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:30:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:30:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:30:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:30:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:30:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:30:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:30:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:30:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:30:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:30:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:30:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:30:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:30:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:30:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:30:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:30:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:30:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:30:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:30:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:30:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.611699557000001, 2024-09-18T11:30:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:30:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:30:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:30:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:30:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:30:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:30:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:30:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:30:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:30:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:30:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:30:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:30:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:30:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:30:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:30:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:30:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:30:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:30:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:30:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:30:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:30:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:30:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.195520649999999, 2024-09-18T11:30:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:30:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:30:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:30:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:30:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:30:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:30:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:30:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:30:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:30:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:30:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:30:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:30:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:30:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:30:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:30:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:30:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:30:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:30:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:30:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:30:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:30:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:30:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.195497349999997, 2024-09-18T11:30:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:30:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:30:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:30:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:30:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:30:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:30:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:30:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:30:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:30:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:30:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:30:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:30:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:30:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:30:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:30:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:30:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:30:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:30:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:30:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:30:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:30:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:30:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:30:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.195411741000001, 2024-09-18T11:30:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:30:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:30:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:30:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:30:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:30:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:30:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:30:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:30:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:30:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:30:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:30:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:30:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:30:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:30:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:30:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:30:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:30:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:30:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:30:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:30:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:30:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:30:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.195400776, 2024-09-18T11:30:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:30:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:30:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:30:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:30:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:30:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:30:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:30:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:30:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:30:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:30:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:30:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:30:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:30:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:30:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:30:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:30:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:30:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:30:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:30:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:30:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:30:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:30:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.195389607999999, 2024-09-18T11:30:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:30:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:30:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:30:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:30:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:30:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:30:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:30:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:30:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:30:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:30:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:30:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:30:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:30:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:30:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:30:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:30:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:30:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:30:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:30:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:30:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:30:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:30:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.195324278999998, 2024-09-18T11:30:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:30:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:30:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:30:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:30:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:30:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:30:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:30:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:30:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:30:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:30:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:30:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:30:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:30:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:30:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:30:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:30:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:30:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:30:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:30:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:30:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:30:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:30:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:30:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.185162300999998, 2024-09-18T11:30:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:30:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:30:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:30:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:30:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:30:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:30:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:30:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:30:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:30:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:30:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:30:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:30:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:30:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:30:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:30:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:30:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:30:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:30:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:30:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:30:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:30:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:30:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:30:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.185069822000003, 2024-09-18T11:30:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:30:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:30:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:30:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:30:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:30:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:30:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:30:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:30:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:30:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:30:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:30:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:30:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:30:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:30:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:30:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:30:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:30:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:30:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:30:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:30:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:30:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:30:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1788 4.8 e60/22 4:1898b15d:::smithi11939506-25:head v 60'12, mlcod=58'9)", 2024-09-18T11:30:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:30:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.8526643329999999, 2024-09-18T11:30:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058348599999999999, 2024-09-18T11:30:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:30:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:30:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916949+0000", 2024-09-18T11:30:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917602+0000", 2024-09-18T11:30:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00065337100000000003 2024-09-18T11:30:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917621+0000", 2024-09-18T11:30:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9287e-05 2024-09-18T11:30:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917629+0000", 2024-09-18T11:30:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3780000000000003e-06 2024-09-18T11:30:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917677+0000", 2024-09-18T11:30:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8068000000000001e-05 2024-09-18T11:30:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917699+0000", 2024-09-18T11:30:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2149000000000001e-05 2024-09-18T11:30:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922747+0000", 2024-09-18T11:30:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050477609999999996 2024-09-18T11:30:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922785+0000", 2024-09-18T11:30:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8726000000000002e-05 2024-09-18T11:30:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:46.893 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:46.895 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:46.893+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:46.895 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:30:46.996 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:46.995+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:47.000 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:47.000 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:47.000 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:47.000 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:47.008 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:30:47.096 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:47.095+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:47.146 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:47.146 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:47.146 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:47.146 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:47.146 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:47.155 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:30:47.199 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:47.198+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:47.255 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:47.255 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:47.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:47.255 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:47.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.256 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:21 3.1 3:9f1cb8cf:::smithi11939506-38 low_tier:head [read 336384~173568 [fadvise_sequential]] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:47.256 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.118534+0000", 2024-09-18T11:30:47.256 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.134536367999999, 2024-09-18T11:30:47.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006633842, 2024-09-18T11:30:47.256 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.256 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:47.256 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118534+0000", 2024-09-18T11:30:47.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118534+0000", 2024-09-18T11:30:47.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118532+0000", 2024-09-18T11:30:47.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:47.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118542+0000", 2024-09-18T11:30:47.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0151e-05 2024-09-18T11:30:47.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118563+0000", 2024-09-18T11:30:47.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1089000000000001e-05 2024-09-18T11:30:47.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118572+0000", 2024-09-18T11:30:47.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1090000000000004e-06 2024-09-18T11:30:47.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118633+0000", 2024-09-18T11:30:47.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0940000000000003e-05 2024-09-18T11:30:47.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:47.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118640+0000", 2024-09-18T11:30:47.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3690000000000003e-06 2024-09-18T11:30:47.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124910+0000", 2024-09-18T11:30:47.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062697569999999999 2024-09-18T11:30:47.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124980+0000", 2024-09-18T11:30:47.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0846000000000006e-05 2024-09-18T11:30:47.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.125168+0000", 2024-09-18T11:30:47.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018786200000000001 2024-09-18T11:30:47.260 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.260 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.260 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.260 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:22 3.1 3:9f1cb8cf:::smithi11939506-38 low_tier:head [read 1235456~1340416 [fadvise_sequential]] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:47.261 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:47.261 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.134483639999999, 2024-09-18T11:30:47.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0090773199999999998, 2024-09-18T11:30:47.261 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.261 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:47.261 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:47.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:47.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118587+0000", 2024-09-18T11:30:47.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118594+0000", 2024-09-18T11:30:47.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1590000000000004e-06 2024-09-18T11:30:47.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118601+0000", 2024-09-18T11:30:47.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3900000000000004e-06 2024-09-18T11:30:47.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118605+0000", 2024-09-18T11:30:47.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2479999999999998e-06 2024-09-18T11:30:47.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118697+0000", 2024-09-18T11:30:47.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1447999999999996e-05 2024-09-18T11:30:47.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:47.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.118702+0000", 2024-09-18T11:30:47.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6670000000000001e-06 2024-09-18T11:30:47.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.125285+0000", 2024-09-18T11:30:47.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006582124 2024-09-18T11:30:47.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.125349+0000", 2024-09-18T11:30:47.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4777000000000007e-05 2024-09-18T11:30:47.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.127664+0000", 2024-09-18T11:30:47.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023151069999999998 2024-09-18T11:30:47.265 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.265 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.265 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.265 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:911 4.3 4:cc46532a:::smithi11939506-33:head [write 1904434~719345] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:47.265 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.166081+0000", 2024-09-18T11:30:47.265 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.086989574, 2024-09-18T11:30:47.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081214100000000008, 2024-09-18T11:30:47.266 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.266 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.266 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 911 2024-09-18T11:30:47.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.266 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166081+0000", 2024-09-18T11:30:47.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166081+0000", 2024-09-18T11:30:47.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166080+0000", 2024-09-18T11:30:47.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:47.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166386+0000", 2024-09-18T11:30:47.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030589499999999998 2024-09-18T11:30:47.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166398+0000", 2024-09-18T11:30:47.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1678000000000001e-05 2024-09-18T11:30:47.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166402+0000", 2024-09-18T11:30:47.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6059999999999996e-06 2024-09-18T11:30:47.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166443+0000", 2024-09-18T11:30:47.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1044000000000002e-05 2024-09-18T11:30:47.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166526+0000", 2024-09-18T11:30:47.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2281999999999996e-05 2024-09-18T11:30:47.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:47.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.166661+0000", 2024-09-18T11:30:47.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013516299999999999 2024-09-18T11:30:47.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:47.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.168437+0000", 2024-09-18T11:30:47.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001775794 2024-09-18T11:30:47.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:47.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174126+0000", 2024-09-18T11:30:47.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056894859999999997 2024-09-18T11:30:47.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174166+0000", 2024-09-18T11:30:47.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9787999999999999e-05 2024-09-18T11:30:47.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.174203+0000", 2024-09-18T11:30:47.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6644999999999999e-05 2024-09-18T11:30:47.271 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.271 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.271 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.271 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:989 4.3 4:ca2b2309:::smithi11939506-44:head [write 410104~465105,stat] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:47.271 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.151379+0000", 2024-09-18T11:30:47.271 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 32.101691610000003, 2024-09-18T11:30:47.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062831420000000002, 2024-09-18T11:30:47.272 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.272 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.272 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.272 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 989 2024-09-18T11:30:47.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.272 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151379+0000", 2024-09-18T11:30:47.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151379+0000", 2024-09-18T11:30:47.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151376+0000", 2024-09-18T11:30:47.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:47.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151666+0000", 2024-09-18T11:30:47.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028977899999999999 2024-09-18T11:30:47.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151697+0000", 2024-09-18T11:30:47.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0516000000000001e-05 2024-09-18T11:30:47.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151707+0000", 2024-09-18T11:30:47.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0322999999999999e-05 2024-09-18T11:30:47.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151770+0000", 2024-09-18T11:30:47.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879000000000006e-05 2024-09-18T11:30:47.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:47.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.151783+0000", 2024-09-18T11:30:47.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3154e-05 2024-09-18T11:30:47.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.153878+0000", 2024-09-18T11:30:47.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002095155 2024-09-18T11:30:47.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.153957+0000", 2024-09-18T11:30:47.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8877999999999999e-05 2024-09-18T11:30:47.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:47.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.154073+0000", 2024-09-18T11:30:47.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116045 2024-09-18T11:30:47.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:47.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.156031+0000", 2024-09-18T11:30:47.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019581099999999999 2024-09-18T11:30:47.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:47.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.157569+0000", 2024-09-18T11:30:47.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015383199999999999 2024-09-18T11:30:47.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.157606+0000", 2024-09-18T11:30:47.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6278000000000001e-05 2024-09-18T11:30:47.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.157662+0000", 2024-09-18T11:30:47.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6589000000000003e-05 2024-09-18T11:30:47.277 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.278 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.278 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.278 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1012 4.b 4:db5b7794:::smithi11939506-13:head [write 443718~615603] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:47.278 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.655768+0000", 2024-09-18T11:30:47.278 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.597303095000001, 2024-09-18T11:30:47.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055941680000000001, 2024-09-18T11:30:47.278 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.278 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.278 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.278 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.278 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.278 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1012 2024-09-18T11:30:47.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.279 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.655768+0000", 2024-09-18T11:30:47.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.655768+0000", 2024-09-18T11:30:47.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.655765+0000", 2024-09-18T11:30:47.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:47.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656140+0000", 2024-09-18T11:30:47.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00037503800000000001 2024-09-18T11:30:47.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656166+0000", 2024-09-18T11:30:47.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5451000000000001e-05 2024-09-18T11:30:47.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656176+0000", 2024-09-18T11:30:47.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0489000000000001e-05 2024-09-18T11:30:47.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656255+0000", 2024-09-18T11:30:47.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8880000000000004e-05 2024-09-18T11:30:47.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656322+0000", 2024-09-18T11:30:47.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6500000000000004e-05 2024-09-18T11:30:47.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:47.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.656436+0000", 2024-09-18T11:30:47.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114945 2024-09-18T11:30:47.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:47.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.658992+0000", 2024-09-18T11:30:47.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025551419999999998 2024-09-18T11:30:47.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:47.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.661302+0000", 2024-09-18T11:30:47.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002310543 2024-09-18T11:30:47.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.661335+0000", 2024-09-18T11:30:47.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2716999999999999e-05 2024-09-18T11:30:47.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.661362+0000", 2024-09-18T11:30:47.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6998000000000001e-05 2024-09-18T11:30:47.283 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.283 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.284 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.284 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1150 4.b 4:d3ae8215:::smithi11939506-41:head [write 1807440~632344] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:47.284 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.620640+0000", 2024-09-18T11:30:47.284 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.632430923000001, 2024-09-18T11:30:47.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068244000000000004, 2024-09-18T11:30:47.284 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.284 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.284 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.284 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.284 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.284 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1150 2024-09-18T11:30:47.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.285 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.620640+0000", 2024-09-18T11:30:47.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.620640+0000", 2024-09-18T11:30:47.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.620638+0000", 2024-09-18T11:30:47.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:47.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621224+0000", 2024-09-18T11:30:47.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00058580899999999996 2024-09-18T11:30:47.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621235+0000", 2024-09-18T11:30:47.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:30:47.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621240+0000", 2024-09-18T11:30:47.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4270000000000001e-06 2024-09-18T11:30:47.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621268+0000", 2024-09-18T11:30:47.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8252999999999999e-05 2024-09-18T11:30:47.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621344+0000", 2024-09-18T11:30:47.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6154000000000003e-05 2024-09-18T11:30:47.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:47.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.621464+0000", 2024-09-18T11:30:47.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012021 2024-09-18T11:30:47.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:47.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.624135+0000", 2024-09-18T11:30:47.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026711310000000002 2024-09-18T11:30:47.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:47.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.627420+0000", 2024-09-18T11:30:47.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032846799999999999 2024-09-18T11:30:47.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.627443+0000", 2024-09-18T11:30:47.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3153e-05 2024-09-18T11:30:47.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.627464+0000", 2024-09-18T11:30:47.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0967000000000001e-05 2024-09-18T11:30:47.289 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.290 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.290 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.290 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:32 3.1 3:94708c4c:::888f815178b2a748104fffa559dcb609f143f519d8737c6aa160c9cce13a6441:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:47.290 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.679331+0000", 2024-09-18T11:30:47.290 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.573740154999999, 2024-09-18T11:30:47.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063965929999999999, 2024-09-18T11:30:47.290 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.290 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.290 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679331+0000", 2024-09-18T11:30:47.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679331+0000", 2024-09-18T11:30:47.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.679330+0000", 2024-09-18T11:30:47.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:47.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681259+0000", 2024-09-18T11:30:47.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019287079999999999 2024-09-18T11:30:47.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681274+0000", 2024-09-18T11:30:47.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5903e-05 2024-09-18T11:30:47.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681281+0000", 2024-09-18T11:30:47.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5640000000000002e-06 2024-09-18T11:30:47.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681325+0000", 2024-09-18T11:30:47.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3925999999999999e-05 2024-09-18T11:30:47.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681435+0000", 2024-09-18T11:30:47.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109739 2024-09-18T11:30:47.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:47.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681642+0000", 2024-09-18T11:30:47.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000206901 2024-09-18T11:30:47.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:47.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.683180+0000", 2024-09-18T11:30:47.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015380929999999999 2024-09-18T11:30:47.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:47.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.685648+0000", 2024-09-18T11:30:47.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024687419999999999 2024-09-18T11:30:47.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.685695+0000", 2024-09-18T11:30:47.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7030000000000002e-05 2024-09-18T11:30:47.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.685727+0000", 2024-09-18T11:30:47.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1792000000000003e-05 2024-09-18T11:30:47.295 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.295 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.295 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.295 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1528 3.1 3:97025964:::d85d35018f93ce704e0c86e65d6c262758d2b38883b05fc1a96898c5a961f23a:head [write 0~1647104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:30:47.295 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.376484+0000", 2024-09-18T11:30:47.296 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.876586601, 2024-09-18T11:30:47.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073833279999999998, 2024-09-18T11:30:47.296 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.296 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.296 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.296 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.296 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.296 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1528 2024-09-18T11:30:47.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.296 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.376484+0000", 2024-09-18T11:30:47.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.376484+0000", 2024-09-18T11:30:47.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.376483+0000", 2024-09-18T11:30:47.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:47.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.377189+0000", 2024-09-18T11:30:47.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00070628499999999996 2024-09-18T11:30:47.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.377208+0000", 2024-09-18T11:30:47.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9261000000000001e-05 2024-09-18T11:30:47.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.377232+0000", 2024-09-18T11:30:47.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3794e-05 2024-09-18T11:30:47.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.377286+0000", 2024-09-18T11:30:47.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4243999999999998e-05 2024-09-18T11:30:47.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.377434+0000", 2024-09-18T11:30:47.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014722699999999999 2024-09-18T11:30:47.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:47.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.377586+0000", 2024-09-18T11:30:47.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015253399999999999 2024-09-18T11:30:47.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:47.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.380417+0000", 2024-09-18T11:30:47.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028304799999999998 2024-09-18T11:30:47.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:47.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.383812+0000", 2024-09-18T11:30:47.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003395117 2024-09-18T11:30:47.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.383843+0000", 2024-09-18T11:30:47.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1615000000000001e-05 2024-09-18T11:30:47.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.383868+0000", 2024-09-18T11:30:47.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4074999999999999e-05 2024-09-18T11:30:47.301 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.301 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.301 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.301 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1554 3.3 e53/21 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head v 53'84, mlcod=47'82)", 2024-09-18T11:30:47.302 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.419942+0000", 2024-09-18T11:30:47.302 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.833129177, 2024-09-18T11:30:47.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011179532000000001, 2024-09-18T11:30:47.302 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.302 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.302 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.419942+0000", 2024-09-18T11:30:47.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.419942+0000", 2024-09-18T11:30:47.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.419940+0000", 2024-09-18T11:30:47.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:47.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.426895+0000", 2024-09-18T11:30:47.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069556970000000003 2024-09-18T11:30:47.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.426923+0000", 2024-09-18T11:30:47.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8095000000000001e-05 2024-09-18T11:30:47.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.426934+0000", 2024-09-18T11:30:47.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0875e-05 2024-09-18T11:30:47.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.426991+0000", 2024-09-18T11:30:47.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6422999999999998e-05 2024-09-18T11:30:47.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.427012+0000", 2024-09-18T11:30:47.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1599000000000001e-05 2024-09-18T11:30:47.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.431068+0000", 2024-09-18T11:30:47.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040558419999999996 2024-09-18T11:30:47.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.431121+0000", 2024-09-18T11:30:47.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3035000000000003e-05 2024-09-18T11:30:47.306 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.306 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.306 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.306 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1603 4.b 4:dba5bd39:::smithi11939506-17:head [write 568103~768769,stat] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:47.306 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.265951+0000", 2024-09-18T11:30:47.306 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9871201900000006, 2024-09-18T11:30:47.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078467250000000006, 2024-09-18T11:30:47.306 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.306 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.306 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.306 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.307 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.307 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1603 2024-09-18T11:30:47.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.307 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.265951+0000", 2024-09-18T11:30:47.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.265951+0000", 2024-09-18T11:30:47.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.265948+0000", 2024-09-18T11:30:47.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:47.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.266720+0000", 2024-09-18T11:30:47.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00077223899999999998 2024-09-18T11:30:47.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.266748+0000", 2024-09-18T11:30:47.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8178999999999999e-05 2024-09-18T11:30:47.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.266758+0000", 2024-09-18T11:30:47.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8309999999999996e-06 2024-09-18T11:30:47.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.266808+0000", 2024-09-18T11:30:47.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0136000000000002e-05 2024-09-18T11:30:47.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.266881+0000", 2024-09-18T11:30:47.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2449999999999999e-05 2024-09-18T11:30:47.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:47.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.267018+0000", 2024-09-18T11:30:47.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000137656 2024-09-18T11:30:47.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:47.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.273693+0000", 2024-09-18T11:30:47.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0066745179999999999 2024-09-18T11:30:47.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:47.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.273701+0000", 2024-09-18T11:30:47.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.729e-06 2024-09-18T11:30:47.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.273751+0000", 2024-09-18T11:30:47.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9944999999999997e-05 2024-09-18T11:30:47.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.273797+0000", 2024-09-18T11:30:47.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6717e-05 2024-09-18T11:30:47.311 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.311 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.312 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.312 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1623 4.3 4:c70c3162:::smithi11939506-35:head [write 584910~560371,stat] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:47.312 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.301415+0000", 2024-09-18T11:30:47.312 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9516554750000008, 2024-09-18T11:30:47.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0061674679999999997, 2024-09-18T11:30:47.312 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.312 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.312 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.312 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.312 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.312 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1623 2024-09-18T11:30:47.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.313 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.301415+0000", 2024-09-18T11:30:47.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.301415+0000", 2024-09-18T11:30:47.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.301413+0000", 2024-09-18T11:30:47.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:47.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.301696+0000", 2024-09-18T11:30:47.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028292999999999998 2024-09-18T11:30:47.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.301719+0000", 2024-09-18T11:30:47.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3302000000000002e-05 2024-09-18T11:30:47.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.301728+0000", 2024-09-18T11:30:47.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.67e-06 2024-09-18T11:30:47.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.301810+0000", 2024-09-18T11:30:47.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1718000000000006e-05 2024-09-18T11:30:47.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.302028+0000", 2024-09-18T11:30:47.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021829400000000001 2024-09-18T11:30:47.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:47.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.302381+0000", 2024-09-18T11:30:47.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00035296999999999998 2024-09-18T11:30:47.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:47.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.306463+0000", 2024-09-18T11:30:47.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040817370000000002 2024-09-18T11:30:47.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:47.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.307497+0000", 2024-09-18T11:30:47.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0010348639999999999 2024-09-18T11:30:47.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.307538+0000", 2024-09-18T11:30:47.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0324000000000002e-05 2024-09-18T11:30:47.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.307583+0000", 2024-09-18T11:30:47.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4997999999999998e-05 2024-09-18T11:30:47.317 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.318 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.318 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.318 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1628 3.3 e56/21 3:f80aa31b:::77cdbf3abbada57d0636b2198046a030673818705cbf13bb77b30c3e68b24345:head v 56'91, mlcod=56'89)", 2024-09-18T11:30:47.318 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.312845+0000", 2024-09-18T11:30:47.318 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9402262189999995, 2024-09-18T11:30:47.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006071908, 2024-09-18T11:30:47.318 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.318 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.318 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.312845+0000", 2024-09-18T11:30:47.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.312845+0000", 2024-09-18T11:30:47.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.312844+0000", 2024-09-18T11:30:47.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:47.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.314834+0000", 2024-09-18T11:30:47.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001990477 2024-09-18T11:30:47.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.314846+0000", 2024-09-18T11:30:47.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2300000000000001e-05 2024-09-18T11:30:47.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.314852+0000", 2024-09-18T11:30:47.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4720000000000003e-06 2024-09-18T11:30:47.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.314897+0000", 2024-09-18T11:30:47.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4951999999999999e-05 2024-09-18T11:30:47.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.314923+0000", 2024-09-18T11:30:47.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6086999999999999e-05 2024-09-18T11:30:47.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.318868+0000", 2024-09-18T11:30:47.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039451770000000002 2024-09-18T11:30:47.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.318916+0000", 2024-09-18T11:30:47.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8301999999999999e-05 2024-09-18T11:30:47.322 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.322 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.322 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.322 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1634 4.b 4:d13decd8:::smithi11939506-2:head [write 1799494~402223] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:47.322 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333638+0000", 2024-09-18T11:30:47.323 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9194330649999998, 2024-09-18T11:30:47.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057370850000000003, 2024-09-18T11:30:47.323 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.323 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.323 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.323 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.323 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.323 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1634 2024-09-18T11:30:47.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.323 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333638+0000", 2024-09-18T11:30:47.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333638+0000", 2024-09-18T11:30:47.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333635+0000", 2024-09-18T11:30:47.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:47.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333775+0000", 2024-09-18T11:30:47.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000139646 2024-09-18T11:30:47.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333793+0000", 2024-09-18T11:30:47.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8309e-05 2024-09-18T11:30:47.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333800+0000", 2024-09-18T11:30:47.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6359999999999997e-06 2024-09-18T11:30:47.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333853+0000", 2024-09-18T11:30:47.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2908000000000002e-05 2024-09-18T11:30:47.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333960+0000", 2024-09-18T11:30:47.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000106928 2024-09-18T11:30:47.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:47.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.334140+0000", 2024-09-18T11:30:47.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018008799999999999 2024-09-18T11:30:47.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:47.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.338316+0000", 2024-09-18T11:30:47.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041764330000000002 2024-09-18T11:30:47.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:47.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.339263+0000", 2024-09-18T11:30:47.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000946764 2024-09-18T11:30:47.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.339323+0000", 2024-09-18T11:30:47.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9589000000000002e-05 2024-09-18T11:30:47.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.339375+0000", 2024-09-18T11:30:47.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2172000000000001e-05 2024-09-18T11:30:47.328 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.328 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.328 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.328 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1635 4.b 4:d13decd8:::smithi11939506-2:head [setxattr _header (56) in=7b,truncate 2201717] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:47.329 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333814+0000", 2024-09-18T11:30:47.329 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9192571269999998, 2024-09-18T11:30:47.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057269290000000004, 2024-09-18T11:30:47.329 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.329 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.329 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.329 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.329 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.329 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1635 2024-09-18T11:30:47.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.330 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333814+0000", 2024-09-18T11:30:47.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333814+0000", 2024-09-18T11:30:47.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333813+0000", 2024-09-18T11:30:47.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333817+0000", 2024-09-18T11:30:47.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1810000000000003e-06 2024-09-18T11:30:47.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333826+0000", 2024-09-18T11:30:47.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4649999999999994e-06 2024-09-18T11:30:47.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333830+0000", 2024-09-18T11:30:47.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5000000000000001e-06 2024-09-18T11:30:47.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.335314+0000", 2024-09-18T11:30:47.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001483769 2024-09-18T11:30:47.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.335413+0000", 2024-09-18T11:30:47.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9309999999999996e-05 2024-09-18T11:30:47.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:47.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.335584+0000", 2024-09-18T11:30:47.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017072999999999999 2024-09-18T11:30:47.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:47.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.339421+0000", 2024-09-18T11:30:47.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038367900000000001 2024-09-18T11:30:47.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:47.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.339428+0000", 2024-09-18T11:30:47.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6000000000000001e-06 2024-09-18T11:30:47.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.339481+0000", 2024-09-18T11:30:47.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2774e-05 2024-09-18T11:30:47.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.339541+0000", 2024-09-18T11:30:47.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9483e-05 2024-09-18T11:30:47.334 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.334 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.335 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.335 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1636 4.b 4:d13decd8:::smithi11939506-2:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:47.335 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333866+0000", 2024-09-18T11:30:47.335 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9192046900000008, 2024-09-18T11:30:47.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059017790000000002, 2024-09-18T11:30:47.335 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.335 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:47.335 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.335 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.335 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.335 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1636 2024-09-18T11:30:47.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.336 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333866+0000", 2024-09-18T11:30:47.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333866+0000", 2024-09-18T11:30:47.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333866+0000", 2024-09-18T11:30:47.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333869+0000", 2024-09-18T11:30:47.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7069999999999998e-06 2024-09-18T11:30:47.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333878+0000", 2024-09-18T11:30:47.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5339999999999999e-06 2024-09-18T11:30:47.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333883+0000", 2024-09-18T11:30:47.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9300000000000002e-06 2024-09-18T11:30:47.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.336182+0000", 2024-09-18T11:30:47.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022994579999999999 2024-09-18T11:30:47.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:47.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.336260+0000", 2024-09-18T11:30:47.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8323000000000002e-05 2024-09-18T11:30:47.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.339572+0000", 2024-09-18T11:30:47.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0033116719999999999 2024-09-18T11:30:47.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.339674+0000", 2024-09-18T11:30:47.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010208300000000001 2024-09-18T11:30:47.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.339768+0000", 2024-09-18T11:30:47.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3634000000000005e-05 2024-09-18T11:30:47.340 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.340 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.340 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.340 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1650 4.b 4:ddcb7ff4:::smithi11939506-9:head [write 1857976~360548] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:47.340 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349569+0000", 2024-09-18T11:30:47.340 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9035016779999996, 2024-09-18T11:30:47.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065957359999999996, 2024-09-18T11:30:47.340 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.340 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.340 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.340 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.341 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.341 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1650 2024-09-18T11:30:47.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.341 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349569+0000", 2024-09-18T11:30:47.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349569+0000", 2024-09-18T11:30:47.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349568+0000", 2024-09-18T11:30:47.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:47.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349677+0000", 2024-09-18T11:30:47.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109669 2024-09-18T11:30:47.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349695+0000", 2024-09-18T11:30:47.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7652e-05 2024-09-18T11:30:47.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349701+0000", 2024-09-18T11:30:47.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5869999999999998e-06 2024-09-18T11:30:47.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349756+0000", 2024-09-18T11:30:47.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4793999999999997e-05 2024-09-18T11:30:47.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.350439+0000", 2024-09-18T11:30:47.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00068225899999999995 2024-09-18T11:30:47.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:47.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.350646+0000", 2024-09-18T11:30:47.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020736800000000001 2024-09-18T11:30:47.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:47.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355791+0000", 2024-09-18T11:30:47.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051446929999999997 2024-09-18T11:30:47.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:47.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.356004+0000", 2024-09-18T11:30:47.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021360800000000001 2024-09-18T11:30:47.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.356089+0000", 2024-09-18T11:30:47.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4529000000000003e-05 2024-09-18T11:30:47.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.356165+0000", 2024-09-18T11:30:47.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6085000000000001e-05 2024-09-18T11:30:47.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.346 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.346 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1651 4.b 4:ddcb7ff4:::smithi11939506-9:head [setxattr _header (56) in=7b,truncate 2218524] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:47.346 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349712+0000", 2024-09-18T11:30:47.346 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9033586590000002, 2024-09-18T11:30:47.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078601369999999997, 2024-09-18T11:30:47.346 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.346 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.346 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.347 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1651 2024-09-18T11:30:47.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.347 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349712+0000", 2024-09-18T11:30:47.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349712+0000", 2024-09-18T11:30:47.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349712+0000", 2024-09-18T11:30:47.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349716+0000", 2024-09-18T11:30:47.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.048e-06 2024-09-18T11:30:47.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349733+0000", 2024-09-18T11:30:47.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7244999999999998e-05 2024-09-18T11:30:47.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349738+0000", 2024-09-18T11:30:47.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1259999999999997e-06 2024-09-18T11:30:47.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.351839+0000", 2024-09-18T11:30:47.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002101225 2024-09-18T11:30:47.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.351946+0000", 2024-09-18T11:30:47.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010671300000000001 2024-09-18T11:30:47.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:47.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.352153+0000", 2024-09-18T11:30:47.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020683499999999999 2024-09-18T11:30:47.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:47.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355800+0000", 2024-09-18T11:30:47.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00364687 2024-09-18T11:30:47.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:47.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357459+0000", 2024-09-18T11:30:47.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016598979999999999 2024-09-18T11:30:47.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357514+0000", 2024-09-18T11:30:47.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4274999999999998e-05 2024-09-18T11:30:47.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357572+0000", 2024-09-18T11:30:47.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8505000000000003e-05 2024-09-18T11:30:47.352 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.352 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.352 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.352 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1652 4.b 4:ddcb7ff4:::smithi11939506-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:47.352 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349743+0000", 2024-09-18T11:30:47.353 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9033274089999992, 2024-09-18T11:30:47.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080765520000000007, 2024-09-18T11:30:47.353 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.353 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:47.353 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.353 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.353 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.353 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1652 2024-09-18T11:30:47.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.353 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349743+0000", 2024-09-18T11:30:47.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349743+0000", 2024-09-18T11:30:47.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349743+0000", 2024-09-18T11:30:47.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349746+0000", 2024-09-18T11:30:47.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8890000000000002e-06 2024-09-18T11:30:47.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349752+0000", 2024-09-18T11:30:47.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2589999999999997e-06 2024-09-18T11:30:47.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349756+0000", 2024-09-18T11:30:47.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.788e-06 2024-09-18T11:30:47.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.352573+0000", 2024-09-18T11:30:47.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028174430000000002 2024-09-18T11:30:47.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:47.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.352669+0000", 2024-09-18T11:30:47.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6082000000000006e-05 2024-09-18T11:30:47.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357604+0000", 2024-09-18T11:30:47.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049343210000000002 2024-09-18T11:30:47.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357706+0000", 2024-09-18T11:30:47.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102437 2024-09-18T11:30:47.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357820+0000", 2024-09-18T11:30:47.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113763 2024-09-18T11:30:47.357 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.357 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.357 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.358 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:74 3.1 3:b900d23f:::smithi11939506-24 low_tier:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e58)", 2024-09-18T11:30:47.358 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.266263+0000", 2024-09-18T11:30:47.358 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.9868082170000001, 2024-09-18T11:30:47.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078322889999999992, 2024-09-18T11:30:47.358 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.358 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.358 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266263+0000", 2024-09-18T11:30:47.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266263+0000", 2024-09-18T11:30:47.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266260+0000", 2024-09-18T11:30:47.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:47.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266271+0000", 2024-09-18T11:30:47.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0839000000000001e-05 2024-09-18T11:30:47.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266293+0000", 2024-09-18T11:30:47.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1749000000000001e-05 2024-09-18T11:30:47.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266301+0000", 2024-09-18T11:30:47.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8550000000000002e-06 2024-09-18T11:30:47.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266367+0000", 2024-09-18T11:30:47.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6371999999999994e-05 2024-09-18T11:30:47.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266468+0000", 2024-09-18T11:30:47.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101303 2024-09-18T11:30:47.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:47.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266650+0000", 2024-09-18T11:30:47.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000181922 2024-09-18T11:30:47.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:47.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.269069+0000", 2024-09-18T11:30:47.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024186189999999999 2024-09-18T11:30:47.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:47.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274003+0000", 2024-09-18T11:30:47.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049343449999999997 2024-09-18T11:30:47.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274057+0000", 2024-09-18T11:30:47.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4008000000000002e-05 2024-09-18T11:30:47.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274095+0000", 2024-09-18T11:30:47.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7503000000000003e-05 2024-09-18T11:30:47.363 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.363 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.363 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.363 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:75 3.1 3:b900d23f:::smithi11939506-24 low_tier:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e58)", 2024-09-18T11:30:47.363 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.266327+0000", 2024-09-18T11:30:47.363 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.986743948, 2024-09-18T11:30:47.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078435550000000003, 2024-09-18T11:30:47.364 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.364 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.364 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266327+0000", 2024-09-18T11:30:47.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266327+0000", 2024-09-18T11:30:47.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266326+0000", 2024-09-18T11:30:47.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266330+0000", 2024-09-18T11:30:47.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.805e-06 2024-09-18T11:30:47.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266337+0000", 2024-09-18T11:30:47.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9750000000000001e-06 2024-09-18T11:30:47.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.266341+0000", 2024-09-18T11:30:47.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:30:47.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.267141+0000", 2024-09-18T11:30:47.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00079995900000000004 2024-09-18T11:30:47.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.267219+0000", 2024-09-18T11:30:47.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8551000000000001e-05 2024-09-18T11:30:47.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:47.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.267378+0000", 2024-09-18T11:30:47.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000158694 2024-09-18T11:30:47.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:47.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.269110+0000", 2024-09-18T11:30:47.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001732 2024-09-18T11:30:47.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:47.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274108+0000", 2024-09-18T11:30:47.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049980520000000002 2024-09-18T11:30:47.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274144+0000", 2024-09-18T11:30:47.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5989999999999999e-05 2024-09-18T11:30:47.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274170+0000", 2024-09-18T11:30:47.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6398e-05 2024-09-18T11:30:47.368 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.369 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.369 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.369 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.369 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.369 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:47.369 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T11:30:47.370 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:47.306+0000 7fcb662ad640 -1 osd.7 64 handle_osd_map injecting map crc failure 2024-09-18T11:30:47.371 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:30:47.371 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:47.303+0000 7fb6930fb640 -1 osd.2 64 handle_osd_map injecting map crc failure 2024-09-18T11:30:47.374 INFO:tasks.rados.rados.0.smithi119.stdout:996: done (3 left) 2024-09-18T11:30:47.374 INFO:tasks.rados.rados.0.smithi119.stdout:997: read oid 46 snap -1 2024-09-18T11:30:47.374 INFO:tasks.rados.rados.0.smithi119.stdout:997: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-18T11:30:47.374 INFO:tasks.rados.rados.0.smithi119.stdout:998: read oid 43 snap -1 2024-09-18T11:30:47.374 INFO:tasks.rados.rados.0.smithi119.stdout:998: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:30:47.374 INFO:tasks.rados.rados.0.smithi119.stdout:999: read oid 19 snap -1 2024-09-18T11:30:47.374 INFO:tasks.rados.rados.0.smithi119.stdout:999: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-18T11:30:47.374 INFO:tasks.rados.rados.0.smithi119.stdout:1000: set_chunk oid 15 target oid 2024-09-18T11:30:47.374 INFO:tasks.rados.rados.0.smithi119.stdout:1000: set_chunk oid 15 offset: 640512 length: 181248 target oid 892bfb3c63f276352f313b119c63ff98649cbcd69fc0cae9a7923283fc77ea90 offset: 0 snap 15 2024-09-18T11:30:47.374 INFO:tasks.rados.rados.0.smithi119.stdout:1001: read oid 48 snap -1 2024-09-18T11:30:47.375 INFO:tasks.rados.rados.0.smithi119.stdout:1001: expect (ObjNum 47 snap 0 seq_num 47) 2024-09-18T11:30:47.375 INFO:tasks.rados.rados.0.smithi119.stdout:1002: read oid 31 snap -1 2024-09-18T11:30:47.375 INFO:tasks.rados.rados.0.smithi119.stdout:1002: expect (ObjNum 121 snap 11 seq_num 121) 2024-09-18T11:30:47.375 INFO:tasks.rados.rados.0.smithi119.stdout:1003: tier_promote oid 9 2024-09-18T11:30:47.375 INFO:tasks.rados.rados.0.smithi119.stdout:1004: read oid 50 snap 12 2024-09-18T11:30:47.375 INFO:tasks.rados.rados.0.smithi119.stdout:1004: expect (ObjNum 93 snap 5 seq_num 93) 2024-09-18T11:30:47.375 INFO:tasks.rados.rados.0.smithi119.stdout:1005: write (excl) oid 49 current snap is 20 2024-09-18T11:30:47.375 INFO:tasks.rados.rados.0.smithi119.stdout:1005: seq_num 156 ranges {621892=465904,1712485=671711} 2024-09-18T11:30:47.375 INFO:tasks.rados.rados.0.smithi119.stdout:1005: writing smithi11939506-49 from 621892 to 1087796 tid 1 2024-09-18T11:30:47.375 INFO:tasks.rados.rados.0.smithi119.stdout:1005: writing smithi11939506-49 from 1712485 to 2384196 tid 2 2024-09-18T11:30:47.375 INFO:tasks.rados.rados.0.smithi119.stdout:1006: write oid 45 current snap is 20 2024-09-18T11:30:47.375 INFO:tasks.rados.rados.0.smithi119.stdout:1006: seq_num 157 ranges {638699=541153,1770967=531722} 2024-09-18T11:30:47.375 INFO:tasks.rados.rados.0.smithi119.stdout:1006: writing smithi11939506-45 from 638699 to 1179852 tid 1 2024-09-18T11:30:47.375 INFO:tasks.rados.rados.0.smithi119.stdout:1006: writing smithi11939506-45 from 1770967 to 2302689 tid 2 2024-09-18T11:30:47.376 INFO:tasks.rados.rados.0.smithi119.stdout:1007: write oid 37 current snap is 20 2024-09-18T11:30:47.376 INFO:tasks.rados.rados.0.smithi119.stdout:1007: seq_num 158 ranges {655506=616402,1829449=675380} 2024-09-18T11:30:47.376 INFO:tasks.rados.rados.0.smithi119.stdout:1007: writing smithi11939506-37 from 655506 to 1271908 tid 1 2024-09-18T11:30:47.376 INFO:tasks.rados.rados.0.smithi119.stdout:1007: writing smithi11939506-37 from 1829449 to 2504829 tid 2 2024-09-18T11:30:47.376 INFO:tasks.rados.rados.0.smithi119.stdout:1008: read oid 34 snap -1 2024-09-18T11:30:47.376 INFO:tasks.rados.rados.0.smithi119.stdout:1008: expect (ObjNum 123 snap 12 seq_num 123) 2024-09-18T11:30:47.376 INFO:tasks.rados.rados.0.smithi119.stdout:1009: set_chunk oid 14 target oid 2024-09-18T11:30:47.376 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:47.369+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:47.377 INFO:tasks.rados.rados.0.smithi119.stdout:1009: set_chunk oid 14 offset: 444416 length: 1003008 target oid 866d65857c51f67d56ecbcba33930954810c64ced2a213ca9705911438b0dfb7 offset: 0 snap -1 2024-09-18T11:30:47.377 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:30:47.377 INFO:tasks.rados.rados.0.smithi119.stdout:992: expect (ObjNum 129 snap 15 seq_num 129) 2024-09-18T11:30:47.471 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:47.470+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:47.473 INFO:tasks.rados.rados.0.smithi119.stdout:994: expect (ObjNum 154 snap 20 seq_num 154) 2024-09-18T11:30:47.476 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:47.477 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:47.477 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:47.477 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:47.488 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:30:47.572 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:47.571+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:47.626 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:47.627 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:47.627 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:47.627 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:47.627 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:47.635 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:30:47.638 INFO:tasks.rados.rados.0.smithi119.stdout:993: got (0) Success 2024-09-18T11:30:47.638 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 82 (ObjNum 137 snap 16 seq_num 137) dirty exists 2024-09-18T11:30:47.638 INFO:tasks.rados.rados.0.smithi119.stdout:992: done (15 left) 2024-09-18T11:30:47.638 INFO:tasks.rados.rados.0.smithi119.stdout:993: done (14 left) 2024-09-18T11:30:47.638 INFO:tasks.rados.rados.0.smithi119.stdout:994: done (13 left) 2024-09-18T11:30:47.638 INFO:tasks.rados.rados.0.smithi119.stdout:1010: tier_evict oid 20 2024-09-18T11:30:47.638 INFO:tasks.rados.rados.0.smithi119.stdout:1010: tier_evict oid 20 snap -1 2024-09-18T11:30:47.638 INFO:tasks.rados.rados.0.smithi119.stdout:999: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-18T11:30:47.679 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:47.678+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:47.712 INFO:tasks.rados.rados.0.smithi119.stdout:998: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:30:47.736 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:47.736 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:47.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:47.736 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:47.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.736 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:887 4.7 4:e0d95e10:::smithi11939506-22:head [read 0~2352947,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:47.736 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:30:47.736 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.616822102999997, 2024-09-18T11:30:47.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079115169999999999, 2024-09-18T11:30:47.736 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.737 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:47.737 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.737 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.737 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.737 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 887 2024-09-18T11:30:47.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.737 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:30:47.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:30:47.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116365+0000", 2024-09-18T11:30:47.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:47.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116376+0000", 2024-09-18T11:30:47.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1471000000000001e-05 2024-09-18T11:30:47.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116394+0000", 2024-09-18T11:30:47.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8088e-05 2024-09-18T11:30:47.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116402+0000", 2024-09-18T11:30:47.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2320000000000004e-06 2024-09-18T11:30:47.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116458+0000", 2024-09-18T11:30:47.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6536000000000002e-05 2024-09-18T11:30:47.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:47.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116466+0000", 2024-09-18T11:30:47.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0819999999999999e-06 2024-09-18T11:30:47.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.122509+0000", 2024-09-18T11:30:47.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060431879999999997 2024-09-18T11:30:47.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.122577+0000", 2024-09-18T11:30:47.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7646999999999998e-05 2024-09-18T11:30:47.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124278+0000", 2024-09-18T11:30:47.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017013200000000001 2024-09-18T11:30:47.741 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.741 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.741 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.741 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:888 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:47.742 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:47.742 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.616768639999997, 2024-09-18T11:30:47.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082282120000000004, 2024-09-18T11:30:47.742 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.742 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:47.742 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.742 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.742 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.742 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 888 2024-09-18T11:30:47.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.742 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:47.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:47.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:47.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116423+0000", 2024-09-18T11:30:47.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1e-06 2024-09-18T11:30:47.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:30:47.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5510000000000001e-06 2024-09-18T11:30:47.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116431+0000", 2024-09-18T11:30:47.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6109999999999998e-06 2024-09-18T11:30:47.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116527+0000", 2024-09-18T11:30:47.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6212999999999995e-05 2024-09-18T11:30:47.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:47.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116532+0000", 2024-09-18T11:30:47.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6140000000000002e-06 2024-09-18T11:30:47.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124314+0000", 2024-09-18T11:30:47.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077818929999999998 2024-09-18T11:30:47.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124411+0000", 2024-09-18T11:30:47.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6992999999999998e-05 2024-09-18T11:30:47.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124649+0000", 2024-09-18T11:30:47.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237729 2024-09-18T11:30:47.747 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.747 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.747 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.747 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:889 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:47.747 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:47.747 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.616743990000003, 2024-09-18T11:30:47.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085383349999999993, 2024-09-18T11:30:47.748 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.748 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:47.748 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.748 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.748 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.748 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 889 2024-09-18T11:30:47.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.748 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:47.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:47.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:47.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116447+0000", 2024-09-18T11:30:47.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.854e-06 2024-09-18T11:30:47.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116452+0000", 2024-09-18T11:30:47.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0250000000000002e-06 2024-09-18T11:30:47.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116455+0000", 2024-09-18T11:30:47.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8739999999999999e-06 2024-09-18T11:30:47.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116565+0000", 2024-09-18T11:30:47.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110741 2024-09-18T11:30:47.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:47.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116571+0000", 2024-09-18T11:30:47.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4990000000000002e-06 2024-09-18T11:30:47.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124691+0000", 2024-09-18T11:30:47.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081200339999999999 2024-09-18T11:30:47.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124784+0000", 2024-09-18T11:30:47.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2886999999999993e-05 2024-09-18T11:30:47.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124983+0000", 2024-09-18T11:30:47.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019969000000000001 2024-09-18T11:30:47.753 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.753 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.753 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.753 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:46 4.a e31/22 4:552523b6:::smithi11939506-38:head v 31'20, mlcod=30'18)", 2024-09-18T11:30:47.753 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:30:47.753 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.602792358000002, 2024-09-18T11:30:47.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080208420000000002, 2024-09-18T11:30:47.754 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.754 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.754 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:30:47.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:30:47.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130396+0000", 2024-09-18T11:30:47.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133245+0000", 2024-09-18T11:30:47.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028486929999999998 2024-09-18T11:30:47.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133253+0000", 2024-09-18T11:30:47.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1540000000000002e-06 2024-09-18T11:30:47.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133261+0000", 2024-09-18T11:30:47.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3620000000000006e-06 2024-09-18T11:30:47.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133319+0000", 2024-09-18T11:30:47.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7719000000000001e-05 2024-09-18T11:30:47.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133341+0000", 2024-09-18T11:30:47.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1946999999999998e-05 2024-09-18T11:30:47.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138368+0000", 2024-09-18T11:30:47.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005026712 2024-09-18T11:30:47.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138418+0000", 2024-09-18T11:30:47.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9864000000000002e-05 2024-09-18T11:30:47.758 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.758 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.758 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.758 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1126 3.0 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head [write 0~2520064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:47.758 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:30:47.758 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.751302012, 2024-09-18T11:30:47.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086197050000000001, 2024-09-18T11:30:47.758 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.758 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.759 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.759 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.759 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.759 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1126 2024-09-18T11:30:47.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.759 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:30:47.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:30:47.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981886+0000", 2024-09-18T11:30:47.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:47.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983017+0000", 2024-09-18T11:30:47.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00113131 2024-09-18T11:30:47.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983037+0000", 2024-09-18T11:30:47.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0231e-05 2024-09-18T11:30:47.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983044+0000", 2024-09-18T11:30:47.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9539999999999998e-06 2024-09-18T11:30:47.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983087+0000", 2024-09-18T11:30:47.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3040000000000001e-05 2024-09-18T11:30:47.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983193+0000", 2024-09-18T11:30:47.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105682 2024-09-18T11:30:47.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:47.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983287+0000", 2024-09-18T11:30:47.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4066000000000004e-05 2024-09-18T11:30:47.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:47.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.986918+0000", 2024-09-18T11:30:47.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036311970000000001 2024-09-18T11:30:47.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:47.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990442+0000", 2024-09-18T11:30:47.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035243119999999999 2024-09-18T11:30:47.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990476+0000", 2024-09-18T11:30:47.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4028e-05 2024-09-18T11:30:47.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990507+0000", 2024-09-18T11:30:47.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0284999999999999e-05 2024-09-18T11:30:47.764 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.764 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.764 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.764 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1137 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~1201068] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:47.764 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.179047+0000", 2024-09-18T11:30:47.764 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.554142361, 2024-09-18T11:30:47.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073446830000000003, 2024-09-18T11:30:47.764 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.764 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:47.764 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.765 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.765 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.765 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1137 2024-09-18T11:30:47.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.765 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179047+0000", 2024-09-18T11:30:47.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179047+0000", 2024-09-18T11:30:47.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179046+0000", 2024-09-18T11:30:47.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179049+0000", 2024-09-18T11:30:47.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.269e-06 2024-09-18T11:30:47.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179057+0000", 2024-09-18T11:30:47.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7279999999999995e-06 2024-09-18T11:30:47.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179061+0000", 2024-09-18T11:30:47.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3309999999999997e-06 2024-09-18T11:30:47.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179159+0000", 2024-09-18T11:30:47.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7027999999999993e-05 2024-09-18T11:30:47.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:47.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179166+0000", 2024-09-18T11:30:47.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8520000000000004e-06 2024-09-18T11:30:47.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.185971+0000", 2024-09-18T11:30:47.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068045570000000001 2024-09-18T11:30:47.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186111+0000", 2024-09-18T11:30:47.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000139906 2024-09-18T11:30:47.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186391+0000", 2024-09-18T11:30:47.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028053000000000003 2024-09-18T11:30:47.769 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.769 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.769 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.770 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1138 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~1201068] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:47.770 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.179081+0000", 2024-09-18T11:30:47.770 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.554108399, 2024-09-18T11:30:47.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007692099, 2024-09-18T11:30:47.770 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.770 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:47.770 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.770 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.770 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.770 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1138 2024-09-18T11:30:47.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.770 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179081+0000", 2024-09-18T11:30:47.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179081+0000", 2024-09-18T11:30:47.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179080+0000", 2024-09-18T11:30:47.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179083+0000", 2024-09-18T11:30:47.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.266e-06 2024-09-18T11:30:47.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179089+0000", 2024-09-18T11:30:47.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3589999999999996e-06 2024-09-18T11:30:47.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:47.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1999999999999996e-06 2024-09-18T11:30:47.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179194+0000", 2024-09-18T11:30:47.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101093 2024-09-18T11:30:47.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:47.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179199+0000", 2024-09-18T11:30:47.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8960000000000002e-06 2024-09-18T11:30:47.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186435+0000", 2024-09-18T11:30:47.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072360979999999998 2024-09-18T11:30:47.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186530+0000", 2024-09-18T11:30:47.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5006999999999993e-05 2024-09-18T11:30:47.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186773+0000", 2024-09-18T11:30:47.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000242496 2024-09-18T11:30:47.775 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.775 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.775 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.775 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1198 4.f 4:f269e3d0:::smithi11939506-24:head [tier-promote] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:47.775 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:21.188205+0000", 2024-09-18T11:30:47.775 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.544983726000002, 2024-09-18T11:30:47.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0071523649999999999, 2024-09-18T11:30:47.775 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.775 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:47.775 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.775 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.775 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.775 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1198 2024-09-18T11:30:47.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.776 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188205+0000", 2024-09-18T11:30:47.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188205+0000", 2024-09-18T11:30:47.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188205+0000", 2024-09-18T11:30:47.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188207+0000", 2024-09-18T11:30:47.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5730000000000002e-06 2024-09-18T11:30:47.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188213+0000", 2024-09-18T11:30:47.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7640000000000002e-06 2024-09-18T11:30:47.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188218+0000", 2024-09-18T11:30:47.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0499999999999999e-06 2024-09-18T11:30:47.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188254+0000", 2024-09-18T11:30:47.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5407999999999997e-05 2024-09-18T11:30:47.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.188321+0000", 2024-09-18T11:30:47.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7682999999999996e-05 2024-09-18T11:30:47.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.195358+0000", 2024-09-18T11:30:47.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070362890000000003 2024-09-18T11:30:47.779 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.779 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.779 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.779 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1507 4.f 4:fc192b81:::smithi11939506-31:head [write 433647~450424,stat] snapc 13=[13,12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T11:30:47.779 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:32.763861+0000", 2024-09-18T11:30:47.779 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.969328004999999, 2024-09-18T11:30:47.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075480879999999997, 2024-09-18T11:30:47.779 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.780 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.780 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.780 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.780 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.780 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1507 2024-09-18T11:30:47.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.780 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.763861+0000", 2024-09-18T11:30:47.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.763861+0000", 2024-09-18T11:30:47.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.763860+0000", 2024-09-18T11:30:47.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:47.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.764059+0000", 2024-09-18T11:30:47.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019900500000000001 2024-09-18T11:30:47.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.764076+0000", 2024-09-18T11:30:47.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7558e-05 2024-09-18T11:30:47.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.764085+0000", 2024-09-18T11:30:47.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3119999999999994e-06 2024-09-18T11:30:47.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.764144+0000", 2024-09-18T11:30:47.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9880999999999998e-05 2024-09-18T11:30:47.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.764260+0000", 2024-09-18T11:30:47.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011586099999999999 2024-09-18T11:30:47.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:30:47.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.764485+0000", 2024-09-18T11:30:47.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022445200000000001 2024-09-18T11:30:47.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:47.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.771268+0000", 2024-09-18T11:30:47.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067833149999999998 2024-09-18T11:30:47.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:47.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.771322+0000", 2024-09-18T11:30:47.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4361999999999999e-05 2024-09-18T11:30:47.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.771354+0000", 2024-09-18T11:30:47.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1451000000000001e-05 2024-09-18T11:30:47.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.771409+0000", 2024-09-18T11:30:47.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5287000000000001e-05 2024-09-18T11:30:47.785 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.785 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.785 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.785 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1525 4.f 4:f269e3d0:::smithi11939506-24:head [sparse-read 0~2011444] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-18T11:30:47.786 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.132667+0000", 2024-09-18T11:30:47.786 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.600521885999999, 2024-09-18T11:30:47.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072757350000000002, 2024-09-18T11:30:47.786 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.786 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:47.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.786 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1525 2024-09-18T11:30:47.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.786 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132667+0000", 2024-09-18T11:30:47.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132667+0000", 2024-09-18T11:30:47.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132667+0000", 2024-09-18T11:30:47.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132669+0000", 2024-09-18T11:30:47.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3130000000000001e-06 2024-09-18T11:30:47.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132675+0000", 2024-09-18T11:30:47.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8109999999999999e-06 2024-09-18T11:30:47.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132678+0000", 2024-09-18T11:30:47.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.579e-06 2024-09-18T11:30:47.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132731+0000", 2024-09-18T11:30:47.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2809999999999998e-05 2024-09-18T11:30:47.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:47.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.132737+0000", 2024-09-18T11:30:47.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6200000000000004e-06 2024-09-18T11:30:47.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.139405+0000", 2024-09-18T11:30:47.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0066678650000000003 2024-09-18T11:30:47.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.139521+0000", 2024-09-18T11:30:47.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115987 2024-09-18T11:30:47.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.139943+0000", 2024-09-18T11:30:47.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00042216999999999998 2024-09-18T11:30:47.791 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.791 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.791 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.791 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:262 4.9 e53/22 4:99feffe9:::smithi11939506-45:13 v 53'17, mlcod=51'14)", 2024-09-18T11:30:47.792 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:30:47.792 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.734198197, 2024-09-18T11:30:47.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013878679, 2024-09-18T11:30:47.792 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.792 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.792 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:30:47.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:30:47.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:30:47.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998997+0000", 2024-09-18T11:30:47.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1099999999999999e-06 2024-09-18T11:30:47.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999008+0000", 2024-09-18T11:30:47.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1321000000000001e-05 2024-09-18T11:30:47.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999015+0000", 2024-09-18T11:30:47.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3000000000000004e-06 2024-09-18T11:30:47.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999055+0000", 2024-09-18T11:30:47.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0058000000000002e-05 2024-09-18T11:30:47.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999070+0000", 2024-09-18T11:30:47.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4328000000000001e-05 2024-09-18T11:30:47.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012465+0000", 2024-09-18T11:30:47.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013395638 2024-09-18T11:30:47.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012870+0000", 2024-09-18T11:30:47.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040423400000000001 2024-09-18T11:30:47.796 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.796 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.796 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.796 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:205 4.b e53/22 4:dba5bd39:::smithi11939506-17:13 v 53'77, mlcod=51'74)", 2024-09-18T11:30:47.796 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:30:47.796 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.734148936, 2024-09-18T11:30:47.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013359586, 2024-09-18T11:30:47.797 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.797 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.797 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:30:47.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:30:47.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:30:47.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999048+0000", 2024-09-18T11:30:47.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3550000000000005e-06 2024-09-18T11:30:47.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999056+0000", 2024-09-18T11:30:47.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3259999999999996e-06 2024-09-18T11:30:47.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999063+0000", 2024-09-18T11:30:47.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6409999999999996e-06 2024-09-18T11:30:47.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999098+0000", 2024-09-18T11:30:47.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5203999999999997e-05 2024-09-18T11:30:47.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999115+0000", 2024-09-18T11:30:47.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6478999999999999e-05 2024-09-18T11:30:47.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012362+0000", 2024-09-18T11:30:47.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01324768 2024-09-18T11:30:47.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012400+0000", 2024-09-18T11:30:47.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.731e-05 2024-09-18T11:30:47.800 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.800 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.800 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.800 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:88 4.a e53/22 4:588debc2:::smithi11939506-18:13 v 53'45, mlcod=51'42)", 2024-09-18T11:30:47.800 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:30:47.801 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.731366512999999, 2024-09-18T11:30:47.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011307667, 2024-09-18T11:30:47.801 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.801 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.801 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:30:47.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:30:47.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001822+0000", 2024-09-18T11:30:47.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001827+0000", 2024-09-18T11:30:47.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.378e-06 2024-09-18T11:30:47.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001839+0000", 2024-09-18T11:30:47.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1225e-05 2024-09-18T11:30:47.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001844+0000", 2024-09-18T11:30:47.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2299999999999999e-06 2024-09-18T11:30:47.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001891+0000", 2024-09-18T11:30:47.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6961e-05 2024-09-18T11:30:47.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001906+0000", 2024-09-18T11:30:47.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5158e-05 2024-09-18T11:30:47.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013103+0000", 2024-09-18T11:30:47.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011197314 2024-09-18T11:30:47.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013130+0000", 2024-09-18T11:30:47.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6900999999999999e-05 2024-09-18T11:30:47.804 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.804 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.805 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.805 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:89 4.a e53/22 4:552523b6:::smithi11939506-38:13 v 53'47, mlcod=51'42)", 2024-09-18T11:30:47.805 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:30:47.805 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.730807293, 2024-09-18T11:30:47.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011327614, 2024-09-18T11:30:47.805 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.805 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.805 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:30:47.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:30:47.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002381+0000", 2024-09-18T11:30:47.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002397+0000", 2024-09-18T11:30:47.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5688e-05 2024-09-18T11:30:47.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002406+0000", 2024-09-18T11:30:47.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0229999999999998e-06 2024-09-18T11:30:47.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002415+0000", 2024-09-18T11:30:47.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8959999999999992e-06 2024-09-18T11:30:47.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012386+0000", 2024-09-18T11:30:47.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0099711450000000007 2024-09-18T11:30:47.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012405+0000", 2024-09-18T11:30:47.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9389e-05 2024-09-18T11:30:47.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013676+0000", 2024-09-18T11:30:47.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012708649999999999 2024-09-18T11:30:47.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013709+0000", 2024-09-18T11:30:47.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3207000000000002e-05 2024-09-18T11:30:47.809 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.809 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.809 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.809 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1643 4.7 4:e73ba68f:::smithi11939506-1:head [read 0~3983226,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:47.809 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:30:47.809 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 10.399302117, 2024-09-18T11:30:47.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079617669999999998, 2024-09-18T11:30:47.809 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.809 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:47.810 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.810 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.810 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.810 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1643 2024-09-18T11:30:47.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.810 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:30:47.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:30:47.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:30:47.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333889+0000", 2024-09-18T11:30:47.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8890000000000002e-06 2024-09-18T11:30:47.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333896+0000", 2024-09-18T11:30:47.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.156e-06 2024-09-18T11:30:47.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333900+0000", 2024-09-18T11:30:47.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.138e-06 2024-09-18T11:30:47.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.336677+0000", 2024-09-18T11:30:47.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027774169999999999 2024-09-18T11:30:47.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.336768+0000", 2024-09-18T11:30:47.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0983999999999995e-05 2024-09-18T11:30:47.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341849+0000", 2024-09-18T11:30:47.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050805140000000004 2024-09-18T11:30:47.814 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.814 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.814 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.814 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1644 4.7 4:e73ba68f:::smithi11939506-1:head [sparse-read 0~2695582] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:47.814 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:30:47.814 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 10.399284995, 2024-09-18T11:30:47.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084764039999999999, 2024-09-18T11:30:47.814 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.814 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:47.814 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.814 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.814 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.815 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1644 2024-09-18T11:30:47.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.815 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:30:47.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:30:47.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:30:47.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333906+0000", 2024-09-18T11:30:47.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.243e-06 2024-09-18T11:30:47.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333911+0000", 2024-09-18T11:30:47.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2959999999999998e-06 2024-09-18T11:30:47.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333915+0000", 2024-09-18T11:30:47.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6720000000000002e-06 2024-09-18T11:30:47.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341879+0000", 2024-09-18T11:30:47.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079638699999999996 2024-09-18T11:30:47.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341978+0000", 2024-09-18T11:30:47.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9562000000000007e-05 2024-09-18T11:30:47.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342380+0000", 2024-09-18T11:30:47.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000402113 2024-09-18T11:30:47.818 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.818 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.818 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.818 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1645 4.7 4:e73ba68f:::smithi11939506-1:head [read 0~3983226,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:47.818 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:30:47.818 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 10.399270101000001, 2024-09-18T11:30:47.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010539468999999999, 2024-09-18T11:30:47.819 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.819 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:47.819 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.819 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.819 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.819 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1645 2024-09-18T11:30:47.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.819 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:30:47.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:30:47.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:30:47.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333921+0000", 2024-09-18T11:30:47.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8909999999999999e-06 2024-09-18T11:30:47.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333927+0000", 2024-09-18T11:30:47.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4229999999999996e-06 2024-09-18T11:30:47.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333932+0000", 2024-09-18T11:30:47.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8280000000000002e-06 2024-09-18T11:30:47.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342418+0000", 2024-09-18T11:30:47.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084862219999999999 2024-09-18T11:30:47.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342515+0000", 2024-09-18T11:30:47.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7287999999999994e-05 2024-09-18T11:30:47.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.344458+0000", 2024-09-18T11:30:47.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019431660000000001 2024-09-18T11:30:47.823 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.823 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.823 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.823 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1669 4.4 4:2d9c3f7f:::smithi11939506-5:head [read 0~3764735,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-18T11:30:47.823 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:30:47.823 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.9103212030000005, 2024-09-18T11:30:47.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0083694990000000007, 2024-09-18T11:30:47.823 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.823 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:47.823 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:47.823 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:47.823 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:47.824 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1669 2024-09-18T11:30:47.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.824 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:30:47.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:30:47.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822867+0000", 2024-09-18T11:30:47.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822871+0000", 2024-09-18T11:30:47.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:30:47.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822877+0000", 2024-09-18T11:30:47.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8000000000000004e-06 2024-09-18T11:30:47.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822880+0000", 2024-09-18T11:30:47.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5669999999999999e-06 2024-09-18T11:30:47.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822974+0000", 2024-09-18T11:30:47.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4048999999999993e-05 2024-09-18T11:30:47.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:47.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822980+0000", 2024-09-18T11:30:47.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2789999999999998e-06 2024-09-18T11:30:47.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.829160+0000", 2024-09-18T11:30:47.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0061800179999999998 2024-09-18T11:30:47.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.829248+0000", 2024-09-18T11:30:47.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7707000000000006e-05 2024-09-18T11:30:47.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.831237+0000", 2024-09-18T11:30:47.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001989827 2024-09-18T11:30:47.828 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.828 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.828 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.828 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:100 4.a e58/22 4:5975a5ec:::smithi11939506-36:18 v 58'55, mlcod=56'50)", 2024-09-18T11:30:47.828 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:30:47.828 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.460039965, 2024-09-18T11:30:47.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0088255, 2024-09-18T11:30:47.829 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.829 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.829 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:30:47.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:30:47.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:30:47.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:47.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273155+0000", 2024-09-18T11:30:47.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.212e-06 2024-09-18T11:30:47.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273164+0000", 2024-09-18T11:30:47.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7609999999999996e-06 2024-09-18T11:30:47.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273168+0000", 2024-09-18T11:30:47.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2819999999999998e-06 2024-09-18T11:30:47.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274162+0000", 2024-09-18T11:30:47.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099398600000000009 2024-09-18T11:30:47.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274180+0000", 2024-09-18T11:30:47.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8236e-05 2024-09-18T11:30:47.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281943+0000", 2024-09-18T11:30:47.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007763308 2024-09-18T11:30:47.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281975+0000", 2024-09-18T11:30:47.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1202999999999999e-05 2024-09-18T11:30:47.832 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.832 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.832 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.833 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:236 4.b e58/22 4:dba5bd39:::smithi11939506-17:13 v 58'95, mlcod=58'91)", 2024-09-18T11:30:47.833 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.274449+0000", 2024-09-18T11:30:47.833 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.458739929, 2024-09-18T11:30:47.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075435490000000001, 2024-09-18T11:30:47.833 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:47.833 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:47.833 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:47.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:47.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274449+0000", 2024-09-18T11:30:47.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:47.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274449+0000", 2024-09-18T11:30:47.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:47.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:47.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274448+0000", 2024-09-18T11:30:47.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:47.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:47.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274455+0000", 2024-09-18T11:30:47.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8700000000000003e-06 2024-09-18T11:30:47.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:47.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274467+0000", 2024-09-18T11:30:47.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1871e-05 2024-09-18T11:30:47.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:47.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274474+0000", 2024-09-18T11:30:47.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0439999999999998e-06 2024-09-18T11:30:47.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:47.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274512+0000", 2024-09-18T11:30:47.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8160000000000001e-05 2024-09-18T11:30:47.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:47.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274526+0000", 2024-09-18T11:30:47.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3478e-05 2024-09-18T11:30:47.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:47.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281961+0000", 2024-09-18T11:30:47.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007435629 2024-09-18T11:30:47.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:47.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:47.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:47.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281993+0000", 2024-09-18T11:30:47.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1242999999999999e-05 2024-09-18T11:30:47.837 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.837 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.837 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.837 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:47.837 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:47.837 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:47.839 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:30:47.839 INFO:tasks.rados.rados.0.smithi119.stdout:1000: finishing set_chunk to oid 15 2024-09-18T11:30:47.839 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 15 set_chunk returned error code -95 offset: 640512 length: 181248 2024-09-18T11:30:47.839 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 0 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-18T11:30:47.839 INFO:tasks.rados.rados.0.smithi119.stdout:997: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-18T11:30:47.839 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:47.837+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:47.886 INFO:tasks.rados.rados.0.smithi119.stdout:1003: got (0) Success 2024-09-18T11:30:47.886 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 104 (ObjNum 150 snap 19 seq_num 150) dirty exists 2024-09-18T11:30:47.886 INFO:tasks.rados.rados.0.smithi119.stdout:1002: expect (ObjNum 121 snap 11 seq_num 121) 2024-09-18T11:30:47.939 INFO:tasks.thrashosds.thrasher:Recovered, killing an osd 2024-09-18T11:30:47.939 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T11:30:47.939 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-18T11:30:47.939 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T11:30:47.939 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:47.938+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:47.971 INFO:tasks.ceph.osd.0:Stopped 2024-09-18T11:30:47.971 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2024-09-18T11:30:47.981 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:47.981 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:47.981 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:47.981 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:47.983 INFO:tasks.rados.rados.0.smithi119.stdout:1004: expect (ObjNum 93 snap 5 seq_num 93) 2024-09-18T11:30:47.990 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:30:48.074 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:48.073+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:48.131 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:48.132 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:48.132 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:48.132 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:48.132 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:48.141 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:30:48.141 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:48.140+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:48.200 INFO:tasks.rados.rados.0.smithi119.stdout:1005: finishing write tid 1 to smithi11939506-49 2024-09-18T11:30:48.200 INFO:tasks.rados.rados.0.smithi119.stdout:1001: expect (ObjNum 47 snap 0 seq_num 47) 2024-09-18T11:30:48.242 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:48.241+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:48.261 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-09-18T11:30:48.260+0000 7ff6f6bf1640 -1 mon.a@0(leader).osd e65 definitely_dead 0 2024-09-18T11:30:48.281 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:48.281 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:48.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:48.282 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:48.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.282 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1045 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'23, mlcod=30'20)", 2024-09-18T11:30:48.282 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.167583+0000", 2024-09-18T11:30:48.283 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 32.111434351, 2024-09-18T11:30:48.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031033380000000002, 2024-09-18T11:30:48.283 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:48.283 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:48.283 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:48.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:48.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167583+0000", 2024-09-18T11:30:48.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:48.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167583+0000", 2024-09-18T11:30:48.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:48.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167582+0000", 2024-09-18T11:30:48.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:48.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:48.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167934+0000", 2024-09-18T11:30:48.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000352262 2024-09-18T11:30:48.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:48.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167952+0000", 2024-09-18T11:30:48.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7859999999999998e-05 2024-09-18T11:30:48.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:48.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.167960+0000", 2024-09-18T11:30:48.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9729999999999997e-06 2024-09-18T11:30:48.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:48.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.168008+0000", 2024-09-18T11:30:48.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7531000000000003e-05 2024-09-18T11:30:48.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:48.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.168032+0000", 2024-09-18T11:30:48.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4443e-05 2024-09-18T11:30:48.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:48.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.170646+0000", 2024-09-18T11:30:48.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026136739999999999 2024-09-18T11:30:48.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:48.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.170687+0000", 2024-09-18T11:30:48.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0859000000000003e-05 2024-09-18T11:30:48.289 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.290 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:48.290 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.290 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1046 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'24, mlcod=30'21)", 2024-09-18T11:30:48.290 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:30:48.290 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 32.105234570999997, 2024-09-18T11:30:48.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042519100000000002, 2024-09-18T11:30:48.291 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:48.291 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:48.291 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:48.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:48.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:30:48.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:48.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:30:48.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:48.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173782+0000", 2024-09-18T11:30:48.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:48.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:48.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174207+0000", 2024-09-18T11:30:48.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00042479799999999999 2024-09-18T11:30:48.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:48.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174224+0000", 2024-09-18T11:30:48.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7513999999999999e-05 2024-09-18T11:30:48.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:48.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174232+0000", 2024-09-18T11:30:48.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0840000000000003e-06 2024-09-18T11:30:48.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:48.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174297+0000", 2024-09-18T11:30:48.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5215000000000005e-05 2024-09-18T11:30:48.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:48.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174336+0000", 2024-09-18T11:30:48.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9403000000000002e-05 2024-09-18T11:30:48.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:48.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.177990+0000", 2024-09-18T11:30:48.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036533709999999999 2024-09-18T11:30:48.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:48.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.178035+0000", 2024-09-18T11:30:48.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5495999999999999e-05 2024-09-18T11:30:48.298 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.299 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:48.299 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.299 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1047 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'25, mlcod=35'23)", 2024-09-18T11:30:48.299 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:30:48.300 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 32.099262373000002, 2024-09-18T11:30:48.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038129129999999998, 2024-09-18T11:30:48.300 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:48.300 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:48.300 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:48.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:48.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:30:48.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:48.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:30:48.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:48.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179754+0000", 2024-09-18T11:30:48.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:48.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:48.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180088+0000", 2024-09-18T11:30:48.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00033395000000000001 2024-09-18T11:30:48.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:48.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180105+0000", 2024-09-18T11:30:48.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7039000000000001e-05 2024-09-18T11:30:48.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:48.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180112+0000", 2024-09-18T11:30:48.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3710000000000002e-06 2024-09-18T11:30:48.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:48.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180176+0000", 2024-09-18T11:30:48.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3178999999999999e-05 2024-09-18T11:30:48.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:48.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180197+0000", 2024-09-18T11:30:48.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1687000000000001e-05 2024-09-18T11:30:48.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:48.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183542+0000", 2024-09-18T11:30:48.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003344877 2024-09-18T11:30:48.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:48.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183568+0000", 2024-09-18T11:30:48.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6146e-05 2024-09-18T11:30:48.305 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.305 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:48.305 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.306 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1048 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'26, mlcod=35'23)", 2024-09-18T11:30:48.306 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.180697+0000", 2024-09-18T11:30:48.306 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 32.098320246999997, 2024-09-18T11:30:48.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031436290000000002, 2024-09-18T11:30:48.306 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:48.306 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:48.306 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:48.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:48.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180697+0000", 2024-09-18T11:30:48.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:48.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180697+0000", 2024-09-18T11:30:48.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:48.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180697+0000", 2024-09-18T11:30:48.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:48.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:48.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180704+0000", 2024-09-18T11:30:48.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8360000000000003e-06 2024-09-18T11:30:48.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:48.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180716+0000", 2024-09-18T11:30:48.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2308e-05 2024-09-18T11:30:48.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:48.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180724+0000", 2024-09-18T11:30:48.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6499999999999996e-06 2024-09-18T11:30:48.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:48.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180884+0000", 2024-09-18T11:30:48.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016041699999999999 2024-09-18T11:30:48.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:48.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180905+0000", 2024-09-18T11:30:48.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1103000000000001e-05 2024-09-18T11:30:48.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:48.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183819+0000", 2024-09-18T11:30:48.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029141800000000002 2024-09-18T11:30:48.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:48.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183841+0000", 2024-09-18T11:30:48.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1869e-05 2024-09-18T11:30:48.310 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.310 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:48.310 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.310 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:30:48.310 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:30:48.310 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.295643704, 2024-09-18T11:30:48.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:30:48.310 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:48.310 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:48.310 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:48.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:48.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:30:48.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:48.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:30:48.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:48.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:30:48.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:48.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:48.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:30:48.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:30:48.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:48.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:30:48.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:30:48.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:48.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:30:48.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:30:48.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:48.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:30:48.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:30:48.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:48.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:30:48.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:30:48.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:48.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:30:48.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:30:48.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:48.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:30:48.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:30:48.314 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.314 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:48.314 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.314 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1144 4.4 e39/22 4:2d9c3f7f:::smithi11939506-5:head v 39'30, mlcod=35'26)", 2024-09-18T11:30:48.314 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.204195+0000", 2024-09-18T11:30:48.314 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.074822954999998, 2024-09-18T11:30:48.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002568945, 2024-09-18T11:30:48.314 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:48.314 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:48.315 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:48.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:48.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204195+0000", 2024-09-18T11:30:48.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:48.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204195+0000", 2024-09-18T11:30:48.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:48.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204194+0000", 2024-09-18T11:30:48.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:48.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:48.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204431+0000", 2024-09-18T11:30:48.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237013 2024-09-18T11:30:48.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:48.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204446+0000", 2024-09-18T11:30:48.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4238e-05 2024-09-18T11:30:48.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:48.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204453+0000", 2024-09-18T11:30:48.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.401e-06 2024-09-18T11:30:48.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:48.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204509+0000", 2024-09-18T11:30:48.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5754000000000002e-05 2024-09-18T11:30:48.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:48.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.204535+0000", 2024-09-18T11:30:48.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5936999999999999e-05 2024-09-18T11:30:48.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:48.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.206686+0000", 2024-09-18T11:30:48.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0021507829999999999 2024-09-18T11:30:48.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:48.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.206764+0000", 2024-09-18T11:30:48.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8135999999999999e-05 2024-09-18T11:30:48.318 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.318 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:48.318 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.318 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1145 4.4 e39/22 4:2d9c3f7f:::smithi11939506-5:head v 39'31, mlcod=38'28)", 2024-09-18T11:30:48.319 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.211000+0000", 2024-09-18T11:30:48.319 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.068017668, 2024-09-18T11:30:48.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0021574469999999998, 2024-09-18T11:30:48.319 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:48.319 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:48.319 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:48.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:48.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.211000+0000", 2024-09-18T11:30:48.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:48.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.211000+0000", 2024-09-18T11:30:48.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:48.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.210998+0000", 2024-09-18T11:30:48.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:48.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:48.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.211183+0000", 2024-09-18T11:30:48.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018480699999999999 2024-09-18T11:30:48.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:48.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.211203+0000", 2024-09-18T11:30:48.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9652000000000001e-05 2024-09-18T11:30:48.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:48.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.211210+0000", 2024-09-18T11:30:48.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8390000000000007e-06 2024-09-18T11:30:48.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:48.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.211270+0000", 2024-09-18T11:30:48.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9076000000000002e-05 2024-09-18T11:30:48.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:48.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.211294+0000", 2024-09-18T11:30:48.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4107000000000001e-05 2024-09-18T11:30:48.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:48.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.213120+0000", 2024-09-18T11:30:48.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018266000000000001 2024-09-18T11:30:48.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:48.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.213158+0000", 2024-09-18T11:30:48.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7249999999999997e-05 2024-09-18T11:30:48.323 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.323 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:48.323 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.323 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1378 4.d e44/22 4:b81644f2:::smithi11939506-34:head v 44'49, mlcod=42'46)", 2024-09-18T11:30:48.323 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:25.202401+0000", 2024-09-18T11:30:48.323 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.076616389000002, 2024-09-18T11:30:48.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028449030000000002, 2024-09-18T11:30:48.323 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:48.323 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:48.323 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:48.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:48.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.202401+0000", 2024-09-18T11:30:48.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:48.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.202401+0000", 2024-09-18T11:30:48.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:48.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.202401+0000", 2024-09-18T11:30:48.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:48.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:48.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.203222+0000", 2024-09-18T11:30:48.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00082101099999999996 2024-09-18T11:30:48.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:48.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.203236+0000", 2024-09-18T11:30:48.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3451999999999999e-05 2024-09-18T11:30:48.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:48.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.203244+0000", 2024-09-18T11:30:48.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6400000000000003e-06 2024-09-18T11:30:48.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:48.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.203285+0000", 2024-09-18T11:30:48.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1254000000000002e-05 2024-09-18T11:30:48.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:48.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.203302+0000", 2024-09-18T11:30:48.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6769000000000002e-05 2024-09-18T11:30:48.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:48.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.205219+0000", 2024-09-18T11:30:48.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019171469999999999 2024-09-18T11:30:48.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:48.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:25.205246+0000", 2024-09-18T11:30:48.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6891999999999999e-05 2024-09-18T11:30:48.329 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.329 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:48.329 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.329 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:30:48.329 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:30:48.329 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.060812751, 2024-09-18T11:30:48.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:30:48.330 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:48.330 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:48.330 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:48.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:48.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:30:48.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:48.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:30:48.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:48.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:30:48.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:30:48.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:48.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:30:48.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:30:48.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:48.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:30:48.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:30:48.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:48.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:30:48.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:30:48.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:48.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:30:48.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:30:48.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:48.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:30:48.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:30:48.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:48.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:30:48.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:30:48.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:48.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:30:48.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:30:48.334 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.334 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:48.334 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.334 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1423 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'65, mlcod=48'63)", 2024-09-18T11:30:48.334 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.230841+0000", 2024-09-18T11:30:48.334 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.048177182, 2024-09-18T11:30:48.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002175808, 2024-09-18T11:30:48.334 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:48.335 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:48.335 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:48.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:48.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.230841+0000", 2024-09-18T11:30:48.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:48.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.230841+0000", 2024-09-18T11:30:48.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:48.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.230840+0000", 2024-09-18T11:30:48.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:48.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:48.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.231438+0000", 2024-09-18T11:30:48.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059785999999999997 2024-09-18T11:30:48.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:48.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.231447+0000", 2024-09-18T11:30:48.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3999999999999998e-06 2024-09-18T11:30:48.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:48.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.231451+0000", 2024-09-18T11:30:48.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0849999999999999e-06 2024-09-18T11:30:48.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:48.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.231490+0000", 2024-09-18T11:30:48.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8306999999999998e-05 2024-09-18T11:30:48.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:48.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.231510+0000", 2024-09-18T11:30:48.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0296999999999999e-05 2024-09-18T11:30:48.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:48.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.232984+0000", 2024-09-18T11:30:48.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014746080000000001 2024-09-18T11:30:48.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:48.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.233016+0000", 2024-09-18T11:30:48.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1919999999999999e-05 2024-09-18T11:30:48.340 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.340 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:48.340 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.341 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:244 4.d e49/22 4:b5a463bf:::smithi11939506-20:10 v 49'68, mlcod=48'65)", 2024-09-18T11:30:48.341 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:30.220263+0000", 2024-09-18T11:30:48.341 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.058755161000001, 2024-09-18T11:30:48.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032524960000000001, 2024-09-18T11:30:48.341 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:48.341 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:48.341 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:48.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:48.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220263+0000", 2024-09-18T11:30:48.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:48.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220263+0000", 2024-09-18T11:30:48.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:48.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220262+0000", 2024-09-18T11:30:48.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:30:48.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:48.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220266+0000", 2024-09-18T11:30:48.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.895e-06 2024-09-18T11:30:48.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:48.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220278+0000", 2024-09-18T11:30:48.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1962e-05 2024-09-18T11:30:48.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:48.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220284+0000", 2024-09-18T11:30:48.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1299999999999998e-06 2024-09-18T11:30:48.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:48.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220334+0000", 2024-09-18T11:30:48.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9789000000000001e-05 2024-09-18T11:30:48.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:48.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220356+0000", 2024-09-18T11:30:48.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1897000000000001e-05 2024-09-18T11:30:48.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:48.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.223464+0000", 2024-09-18T11:30:48.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031082789999999998 2024-09-18T11:30:48.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:48.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.223515+0000", 2024-09-18T11:30:48.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0748999999999998e-05 2024-09-18T11:30:48.345 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.345 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:48.345 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.345 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1511 4.4 e51/22 4:2c93196f:::smithi11939506-28:head v 51'50, mlcod=51'47)", 2024-09-18T11:30:48.345 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:32.776917+0000", 2024-09-18T11:30:48.346 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.502101211999999, 2024-09-18T11:30:48.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024761599999999998, 2024-09-18T11:30:48.346 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:48.346 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:48.346 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:48.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:48.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.776917+0000", 2024-09-18T11:30:48.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:48.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.776917+0000", 2024-09-18T11:30:48.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:48.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.776915+0000", 2024-09-18T11:30:48.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:48.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:48.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.777297+0000", 2024-09-18T11:30:48.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00038211499999999998 2024-09-18T11:30:48.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:48.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.777317+0000", 2024-09-18T11:30:48.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0043000000000001e-05 2024-09-18T11:30:48.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:48.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.777325+0000", 2024-09-18T11:30:48.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9500000000000001e-06 2024-09-18T11:30:48.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:48.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.777380+0000", 2024-09-18T11:30:48.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4471000000000001e-05 2024-09-18T11:30:48.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:48.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.777403+0000", 2024-09-18T11:30:48.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3703999999999999e-05 2024-09-18T11:30:48.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:48.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.779349+0000", 2024-09-18T11:30:48.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001945312 2024-09-18T11:30:48.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:48.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.779393+0000", 2024-09-18T11:30:48.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3915000000000002e-05 2024-09-18T11:30:48.350 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.350 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:48.350 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.350 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1512 4.4 e51/22 4:2c93196f:::smithi11939506-28:head v 51'51, mlcod=51'48)", 2024-09-18T11:30:48.350 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:32.782201+0000", 2024-09-18T11:30:48.351 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.496816796999999, 2024-09-18T11:30:48.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00363554, 2024-09-18T11:30:48.351 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:48.351 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:48.351 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:48.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:48.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782201+0000", 2024-09-18T11:30:48.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:48.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782201+0000", 2024-09-18T11:30:48.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:48.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782200+0000", 2024-09-18T11:30:48.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:48.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:48.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782564+0000", 2024-09-18T11:30:48.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000364612 2024-09-18T11:30:48.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:48.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782585+0000", 2024-09-18T11:30:48.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0418999999999999e-05 2024-09-18T11:30:48.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:48.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782594+0000", 2024-09-18T11:30:48.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9570000000000008e-06 2024-09-18T11:30:48.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:48.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782663+0000", 2024-09-18T11:30:48.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9330000000000002e-05 2024-09-18T11:30:48.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:48.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782688+0000", 2024-09-18T11:30:48.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5420000000000001e-05 2024-09-18T11:30:48.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:48.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.785790+0000", 2024-09-18T11:30:48.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031016160000000002 2024-09-18T11:30:48.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:48.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.785836+0000", 2024-09-18T11:30:48.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6360999999999999e-05 2024-09-18T11:30:48.355 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.355 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:48.356 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.356 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1513 4.4 e51/22 4:2c93196f:::smithi11939506-28:head v 51'52, mlcod=51'48)", 2024-09-18T11:30:48.356 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:32.783397+0000", 2024-09-18T11:30:48.356 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.495620259000001, 2024-09-18T11:30:48.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003017198, 2024-09-18T11:30:48.357 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:48.357 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:48.357 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:48.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:48.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783397+0000", 2024-09-18T11:30:48.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:48.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783397+0000", 2024-09-18T11:30:48.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:48.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783397+0000", 2024-09-18T11:30:48.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:48.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:48.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783404+0000", 2024-09-18T11:30:48.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.075e-06 2024-09-18T11:30:48.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:48.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783417+0000", 2024-09-18T11:30:48.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3234000000000001e-05 2024-09-18T11:30:48.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:48.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783422+0000", 2024-09-18T11:30:48.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5260000000000001e-06 2024-09-18T11:30:48.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:48.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783605+0000", 2024-09-18T11:30:48.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018248099999999999 2024-09-18T11:30:48.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:48.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.783635+0000", 2024-09-18T11:30:48.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0309000000000001e-05 2024-09-18T11:30:48.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:48.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.786378+0000", 2024-09-18T11:30:48.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027433119999999999 2024-09-18T11:30:48.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:48.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.786415+0000", 2024-09-18T11:30:48.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6192999999999997e-05 2024-09-18T11:30:48.360 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.360 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:48.360 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.360 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:30:48.361 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:30:48.361 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.886906465999999, 2024-09-18T11:30:48.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:30:48.361 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:48.361 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:48.361 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:48.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:48.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:30:48.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:48.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:30:48.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:48.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:30:48.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:48.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:48.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:30:48.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:30:48.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:48.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:30:48.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:30:48.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:48.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:30:48.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:30:48.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:48.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:30:48.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:30:48.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:48.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:30:48.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:30:48.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:48.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:30:48.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:30:48.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:48.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:30:48.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:30:48.364 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.364 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:48.364 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.365 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1532 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'74, mlcod=51'69)", 2024-09-18T11:30:48.365 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.397573+0000", 2024-09-18T11:30:48.365 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.881444377999999, 2024-09-18T11:30:48.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032675960000000002, 2024-09-18T11:30:48.365 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:48.365 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:48.365 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:48.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:48.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397573+0000", 2024-09-18T11:30:48.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:48.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397573+0000", 2024-09-18T11:30:48.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:48.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397573+0000", 2024-09-18T11:30:48.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:48.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:48.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397577+0000", 2024-09-18T11:30:48.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5499999999999996e-06 2024-09-18T11:30:48.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:48.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397585+0000", 2024-09-18T11:30:48.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1470000000000001e-06 2024-09-18T11:30:48.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:48.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397591+0000", 2024-09-18T11:30:48.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5369999999999997e-06 2024-09-18T11:30:48.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:48.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397635+0000", 2024-09-18T11:30:48.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4308000000000003e-05 2024-09-18T11:30:48.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:48.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397656+0000", 2024-09-18T11:30:48.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1138e-05 2024-09-18T11:30:48.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:48.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.400817+0000", 2024-09-18T11:30:48.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031607499999999999 2024-09-18T11:30:48.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:48.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.400841+0000", 2024-09-18T11:30:48.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3717e-05 2024-09-18T11:30:48.368 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.368 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:48.368 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.369 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.13:87 3.0 e58/21 3:099aa86f:::smithi11939506-4 low_tier:head v 58'69, mlcod=55'66)", 2024-09-18T11:30:48.369 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.270180+0000", 2024-09-18T11:30:48.369 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0088379399999994, 2024-09-18T11:30:48.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037855440000000001, 2024-09-18T11:30:48.369 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:48.369 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:48.369 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:48.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:48.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270180+0000", 2024-09-18T11:30:48.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:48.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270180+0000", 2024-09-18T11:30:48.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:48.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270179+0000", 2024-09-18T11:30:48.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:48.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:48.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270187+0000", 2024-09-18T11:30:48.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5610000000000001e-06 2024-09-18T11:30:48.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:48.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270195+0000", 2024-09-18T11:30:48.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5909999999999996e-06 2024-09-18T11:30:48.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:48.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270200+0000", 2024-09-18T11:30:48.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2599999999999999e-06 2024-09-18T11:30:48.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:48.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270252+0000", 2024-09-18T11:30:48.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1891000000000002e-05 2024-09-18T11:30:48.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:48.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270277+0000", 2024-09-18T11:30:48.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5134e-05 2024-09-18T11:30:48.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:48.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273928+0000", 2024-09-18T11:30:48.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00365132 2024-09-18T11:30:48.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:48.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273965+0000", 2024-09-18T11:30:48.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7277000000000002e-05 2024-09-18T11:30:48.373 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.373 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:48.373 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.373 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.13:88 3.0 e58/21 3:099aa86f:::smithi11939506-4 low_tier:head v 58'70, mlcod=55'66)", 2024-09-18T11:30:48.373 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.270488+0000", 2024-09-18T11:30:48.373 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0085300650000004, 2024-09-18T11:30:48.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034966020000000001, 2024-09-18T11:30:48.373 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:48.373 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:48.374 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:48.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:48.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270488+0000", 2024-09-18T11:30:48.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:48.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270488+0000", 2024-09-18T11:30:48.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:48.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270487+0000", 2024-09-18T11:30:48.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:48.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:48.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270493+0000", 2024-09-18T11:30:48.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9780000000000002e-06 2024-09-18T11:30:48.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:48.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270503+0000", 2024-09-18T11:30:48.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0183000000000001e-05 2024-09-18T11:30:48.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:48.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270507+0000", 2024-09-18T11:30:48.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9960000000000004e-06 2024-09-18T11:30:48.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:48.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270542+0000", 2024-09-18T11:30:48.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5077000000000003e-05 2024-09-18T11:30:48.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:48.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270562+0000", 2024-09-18T11:30:48.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9456000000000001e-05 2024-09-18T11:30:48.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:48.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273967+0000", 2024-09-18T11:30:48.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003405805 2024-09-18T11:30:48.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:48.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273984+0000", 2024-09-18T11:30:48.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6824e-05 2024-09-18T11:30:48.377 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.377 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:48.377 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.378 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:289 4.d e58/22 4:b5a463bf:::smithi11939506-20:10 v 58'78, mlcod=53'73)", 2024-09-18T11:30:48.378 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.275682+0000", 2024-09-18T11:30:48.378 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0033355759999996, 2024-09-18T11:30:48.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003584318, 2024-09-18T11:30:48.378 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:48.378 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:48.378 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:48.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:48.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275682+0000", 2024-09-18T11:30:48.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:48.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275682+0000", 2024-09-18T11:30:48.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:48.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275681+0000", 2024-09-18T11:30:48.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:48.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:48.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275689+0000", 2024-09-18T11:30:48.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-18T11:30:48.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:48.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275698+0000", 2024-09-18T11:30:48.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1120000000000002e-06 2024-09-18T11:30:48.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:48.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275702+0000", 2024-09-18T11:30:48.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1450000000000001e-06 2024-09-18T11:30:48.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:48.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275731+0000", 2024-09-18T11:30:48.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8782999999999999e-05 2024-09-18T11:30:48.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:48.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275743+0000", 2024-09-18T11:30:48.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1436e-05 2024-09-18T11:30:48.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:48.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.279231+0000", 2024-09-18T11:30:48.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034884339999999999 2024-09-18T11:30:48.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:48.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.279266+0000", 2024-09-18T11:30:48.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5488000000000003e-05 2024-09-18T11:30:48.381 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.381 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:48.382 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.382 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1931 3.0 e65/21 3:3c05e498:::866d65857c51f67d56ecbcba33930954810c64ced2a213ca9705911438b0dfb7:head v 65'72, mlcod=58'70)", 2024-09-18T11:30:48.382 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:30:48.382 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.91089644800000003, 2024-09-18T11:30:48.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040630859999999996, 2024-09-18T11:30:48.382 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:48.382 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:48.382 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:48.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:48.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:30:48.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:48.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:30:48.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:48.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:48.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368120+0000", 2024-09-18T11:30:48.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:48.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:48.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368427+0000", 2024-09-18T11:30:48.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030742600000000002 2024-09-18T11:30:48.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:48.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368446+0000", 2024-09-18T11:30:48.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8046e-05 2024-09-18T11:30:48.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:48.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368453+0000", 2024-09-18T11:30:48.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1380000000000001e-06 2024-09-18T11:30:48.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:48.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368519+0000", 2024-09-18T11:30:48.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6470000000000006e-05 2024-09-18T11:30:48.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:48.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368550+0000", 2024-09-18T11:30:48.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0707999999999998e-05 2024-09-18T11:30:48.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:48.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.372151+0000", 2024-09-18T11:30:48.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036015470000000001 2024-09-18T11:30:48.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:48.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:48.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:48.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.372184+0000", 2024-09-18T11:30:48.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2994999999999998e-05 2024-09-18T11:30:48.386 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.386 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:48.386 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.386 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:48.386 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:48.386 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:48.388 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:30:48.389 INFO:tasks.rados.rados.0.smithi119.stdout:1005: finishing write tid 2 to smithi11939506-49 2024-09-18T11:30:48.389 INFO:tasks.rados.rados.0.smithi119.stdout:1005: finishing write tid 3 to smithi11939506-49 2024-09-18T11:30:48.389 INFO:tasks.rados.rados.0.smithi119.stdout:1005: finishing write tid 4 to smithi11939506-49 2024-09-18T11:30:48.390 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 119 (ObjNum 156 snap 20 seq_num 156) dirty exists 2024-09-18T11:30:48.390 INFO:tasks.rados.rados.0.smithi119.stdout:1005: left oid 49 (ObjNum 156 snap 20 seq_num 156) 2024-09-18T11:30:48.390 INFO:tasks.rados.rados.0.smithi119.stdout:1006: finishing write tid 1 to smithi11939506-45 2024-09-18T11:30:48.390 INFO:tasks.rados.rados.0.smithi119.stdout:1006: finishing write tid 2 to smithi11939506-45 2024-09-18T11:30:48.390 INFO:tasks.rados.rados.0.smithi119.stdout:1006: finishing write tid 3 to smithi11939506-45 2024-09-18T11:30:48.390 INFO:tasks.rados.rados.0.smithi119.stdout:1006: finishing write tid 4 to smithi11939506-45 2024-09-18T11:30:48.390 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 24 (ObjNum 157 snap 20 seq_num 157) dirty exists 2024-09-18T11:30:48.390 INFO:tasks.rados.rados.0.smithi119.stdout:1006: left oid 45 (ObjNum 157 snap 20 seq_num 157) 2024-09-18T11:30:48.390 INFO:tasks.rados.rados.0.smithi119.stdout:1007: finishing write tid 1 to smithi11939506-37 2024-09-18T11:30:48.390 INFO:tasks.rados.rados.0.smithi119.stdout:997: done (13 left) 2024-09-18T11:30:48.390 INFO:tasks.rados.rados.0.smithi119.stdout:998: done (12 left) 2024-09-18T11:30:48.390 INFO:tasks.rados.rados.0.smithi119.stdout:999: done (11 left) 2024-09-18T11:30:48.390 INFO:tasks.rados.rados.0.smithi119.stdout:1000: done (10 left) 2024-09-18T11:30:48.390 INFO:tasks.rados.rados.0.smithi119.stdout:1001: done (9 left) 2024-09-18T11:30:48.391 INFO:tasks.rados.rados.0.smithi119.stdout:1002: done (8 left) 2024-09-18T11:30:48.391 INFO:tasks.rados.rados.0.smithi119.stdout:1003: done (7 left) 2024-09-18T11:30:48.391 INFO:tasks.rados.rados.0.smithi119.stdout:1004: done (6 left) 2024-09-18T11:30:48.391 INFO:tasks.rados.rados.0.smithi119.stdout:1005: done (5 left) 2024-09-18T11:30:48.391 INFO:tasks.rados.rados.0.smithi119.stdout:1006: done (4 left) 2024-09-18T11:30:48.391 INFO:tasks.rados.rados.0.smithi119.stdout:1011: write oid 29 current snap is 20 2024-09-18T11:30:48.391 INFO:tasks.rados.rados.0.smithi119.stdout:1011: seq_num 159 ranges {672313=691651,1887931=419038} 2024-09-18T11:30:48.391 INFO:tasks.rados.rados.0.smithi119.stdout:1011: writing smithi11939506-29 from 672313 to 1363964 tid 1 2024-09-18T11:30:48.391 INFO:tasks.rados.rados.0.smithi119.stdout:1011: writing smithi11939506-29 from 1887931 to 2306969 tid 2 2024-09-18T11:30:48.391 INFO:tasks.rados.rados.0.smithi119.stdout:1012: tier_promote oid 3 2024-09-18T11:30:48.391 INFO:tasks.rados.rados.0.smithi119.stdout:1007: finishing write tid 2 to smithi11939506-37 2024-09-18T11:30:48.391 INFO:tasks.rados.rados.0.smithi119.stdout:1013: delete oid 41 current snap is 20 2024-09-18T11:30:48.391 INFO:tasks.rados.rados.0.smithi119.stdout:1007: finishing write tid 3 to smithi11939506-37 2024-09-18T11:30:48.391 INFO:tasks.rados.rados.0.smithi119.stdout:1007: finishing write tid 4 to smithi11939506-37 2024-09-18T11:30:48.392 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 93 (ObjNum 158 snap 20 seq_num 158) dirty exists 2024-09-18T11:30:48.392 INFO:tasks.rados.rados.0.smithi119.stdout:1007: left oid 37 (ObjNum 158 snap 20 seq_num 158) 2024-09-18T11:30:48.392 INFO:tasks.rados.rados.0.smithi119.stdout:1008: expect (ObjNum 123 snap 12 seq_num 123) 2024-09-18T11:30:48.392 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is set 2024-09-18T11:30:48.392 INFO:teuthology.orchestra.run.smithi119.stderr:marked down osd.0. 2024-09-18T11:30:48.394 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T11:30:48.394 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-09-18T11:30:48.394 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:48.386+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:48.420 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:30:48.432 INFO:tasks.rados.rados.0.smithi119.stdout:1009: finishing set_chunk to oid 14 2024-09-18T11:30:48.433 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 89 (ObjNum 153 snap 20 seq_num 153) dirty exists 2024-09-18T11:30:48.433 INFO:tasks.rados.rados.0.smithi119.stdout:1010: got (22) Invalid argument 2024-09-18T11:30:48.433 INFO:tasks.rados.rados.0.smithi119.stdout:1011: finishing write tid 1 to smithi11939506-29 2024-09-18T11:30:48.434 INFO:tasks.rados.rados.0.smithi119.stdout:1012: got (0) Success 2024-09-18T11:30:48.434 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 26 (ObjNum 77 snap 3 seq_num 77) dirty exists 2024-09-18T11:30:48.434 INFO:tasks.rados.rados.0.smithi119.stdout:1011: finishing write tid 2 to smithi11939506-29 2024-09-18T11:30:48.434 INFO:tasks.rados.rados.0.smithi119.stdout:1011: finishing write tid 3 to smithi11939506-29 2024-09-18T11:30:48.434 INFO:tasks.rados.rados.0.smithi119.stdout:1011: finishing write tid 4 to smithi11939506-29 2024-09-18T11:30:48.434 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 46 (ObjNum 159 snap 20 seq_num 159) dirty exists 2024-09-18T11:30:48.435 INFO:tasks.rados.rados.0.smithi119.stdout:1011: left oid 29 (ObjNum 159 snap 20 seq_num 159) 2024-09-18T11:30:48.488 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:48.487+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:48.500 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:48.500 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:48.500 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:48.500 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:48.509 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:30:48.529 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:30:48.554 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:48.554 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:48.575 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:48.575 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:48.589 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:48.588+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:48.590 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:48.590 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:48.605 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:48.605 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:48.608 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:48.608 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:48.617 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:30:48.621 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:48.621 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:48.636 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:48.636 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:48.651 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:48.652 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:48.655 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:30:48.655 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:30:48.696 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:48.695+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:48.723 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-09-18T11:30:48.721+0000 7ff6f6bf1640 -1 mon.a@0(leader).osd e66 definitely_dead 0 2024-09-18T11:30:48.754 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:48.755 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:48.763 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:30:48.806 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:48.805+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:48.861 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:48.861 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:48.869 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:30:48.910 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:48.909+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:48.992 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:48.991+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:49.007 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:49.007 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:49.015 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:30:49.099 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:49.098+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:49.154 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:49.154 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:49.162 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:30:49.203 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:49.202+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:49.295 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:49.294+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:49.300 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:49.300 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:49.309 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:30:49.317 INFO:teuthology.orchestra.run.smithi119.stderr:marked out osd.0. 2024-09-18T11:30:49.335 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.0 2024-09-18T11:30:49.335 INFO:tasks.thrashosds.thrasher:remote for osd 0 is ubuntu@smithi119.front.sepia.ceph.com 2024-09-18T11:30:49.335 DEBUG:teuthology.orchestra.run.smithi119:> type ceph-objectstore-tool 2024-09-18T11:30:49.392 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T11:30:49.392 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op list-pgs 2024-09-18T11:30:49.396 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:49.395+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:49.407 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:49.407 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:49.416 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:30:49.500 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:49.499+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:49.554 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:49.554 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:49.562 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:30:49.598 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:49.595+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:49.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:30:49.616+0000 7fadcecd6200 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file) 2024-09-18T11:30:49.698 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:49.697+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:49.700 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:49.700 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:49.709 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:30:49.799 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:49.797+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:49.847 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:49.847 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:49.856 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:30:49.900 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:49.898+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:49.954 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:49.954 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:49.962 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:30:50.006 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:50.004+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:50.065 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:50.065 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:50.072 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:30:50.102 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:50.101+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:50.203 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:50.201+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:50.209 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:50.209 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:50.218 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:30:50.303 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:50.302+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:50.358 INFO:tasks.rados.rados.0.smithi119.stdout:1007: done (6 left) 2024-09-18T11:30:50.358 INFO:tasks.rados.rados.0.smithi119.stdout:1008: done (5 left) 2024-09-18T11:30:50.359 INFO:tasks.rados.rados.0.smithi119.stdout:1009: done (4 left) 2024-09-18T11:30:50.359 INFO:tasks.rados.rados.0.smithi119.stdout:1010: done (3 left) 2024-09-18T11:30:50.359 INFO:tasks.rados.rados.0.smithi119.stdout:1011: done (2 left) 2024-09-18T11:30:50.359 INFO:tasks.rados.rados.0.smithi119.stdout:1012: done (1 left) 2024-09-18T11:30:50.359 INFO:tasks.rados.rados.0.smithi119.stdout:1013: done (0 left) 2024-09-18T11:30:50.359 INFO:tasks.rados.rados.0.smithi119.stdout:1014: tier_flush oid 41 2024-09-18T11:30:50.359 INFO:tasks.rados.rados.0.smithi119.stdout:1014: tier_flush oid 41 snap -1 2024-09-18T11:30:50.359 INFO:tasks.rados.rados.0.smithi119.stdout:1015: tier_flush oid 12 2024-09-18T11:30:50.359 INFO:tasks.rados.rados.0.smithi119.stdout:1015: tier_flush oid 12 snap -1 2024-09-18T11:30:50.359 INFO:tasks.rados.rados.0.smithi119.stdout:1016: rollback oid 1 current snap is 20 2024-09-18T11:30:50.359 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 1 to 17 2024-09-18T11:30:50.359 INFO:tasks.rados.rados.0.smithi119.stdout:1017: snap_remove snap 11 2024-09-18T11:30:50.367 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:50.367 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:50.377 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:30:50.394 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T11:30:50.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:30:50.398+0000 7fadcecd6200 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file) 2024-09-18T11:30:50.405 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:50.403+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:50.492 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:50.492 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:50.500 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:30:50.545 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:50.543+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:50.605 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:50.605 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:50.607 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:50.606+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:50.614 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:30:50.656 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:30:50.663 INFO:teuthology.orchestra.run.smithi119.stdout:4.b 2024-09-18T11:30:50.663 INFO:teuthology.orchestra.run.smithi119.stdout:4.6 2024-09-18T11:30:50.663 INFO:teuthology.orchestra.run.smithi119.stdout:4.3 2024-09-18T11:30:50.663 INFO:teuthology.orchestra.run.smithi119.stdout:3.3 2024-09-18T11:30:50.663 INFO:teuthology.orchestra.run.smithi119.stdout:3.1 2024-09-18T11:30:50.663 INFO:teuthology.orchestra.run.smithi119.stdout:2.5 2024-09-18T11:30:50.663 INFO:teuthology.orchestra.run.smithi119.stdout:2.4 2024-09-18T11:30:50.663 INFO:teuthology.orchestra.run.smithi119.stdout:1.0 2024-09-18T11:30:50.708 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:50.706+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:50.738 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:50.736+0000 7fb6930fb640 -1 osd.2 67 handle_osd_map injecting map crc failure 2024-09-18T11:30:50.759 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:30:50.759 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:50.759 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:50.770 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:30:50.776 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:50.776 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:50.792 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:50.792 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:50.808 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:50.808 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:50.808 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:50.807+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:50.823 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:50.823 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:50.838 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:50.838 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:50.853 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:50.854 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:50.869 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:50.869 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:50.873 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:30:50.873 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:30:50.877 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:50.877 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:50.877 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:50.877 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:50.886 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:30:50.927 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:50.925+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:51.010 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:51.008+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:51.024 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:51.024 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:51.024 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:51.024 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:51.024 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:51.032 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:30:51.110 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:51.109+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:51.172 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:51.172 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:51.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:51.172 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:51.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.172 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:900 4.0 4:0aec5e1c:::smithi11939506-50:head [read 0~2974806,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:51.172 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.133226+0000", 2024-09-18T11:30:51.172 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.035898381999999, 2024-09-18T11:30:51.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059578069999999999, 2024-09-18T11:30:51.172 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:51.173 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:51.173 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:51.173 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:51.173 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.173 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 900 2024-09-18T11:30:51.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.173 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:51.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:51.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133226+0000", 2024-09-18T11:30:51.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:51.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133226+0000", 2024-09-18T11:30:51.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:51.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133225+0000", 2024-09-18T11:30:51.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:51.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:51.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133234+0000", 2024-09-18T11:30:51.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5799999999999992e-06 2024-09-18T11:30:51.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:51.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133242+0000", 2024-09-18T11:30:51.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4039999999999995e-06 2024-09-18T11:30:51.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133256+0000", 2024-09-18T11:30:51.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3278000000000001e-05 2024-09-18T11:30:51.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133293+0000", 2024-09-18T11:30:51.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7799000000000002e-05 2024-09-18T11:30:51.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:51.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133298+0000", 2024-09-18T11:30:51.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9110000000000001e-06 2024-09-18T11:30:51.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.137554+0000", 2024-09-18T11:30:51.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042556099999999999 2024-09-18T11:30:51.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:51.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.137713+0000", 2024-09-18T11:30:51.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015937199999999999 2024-09-18T11:30:51.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:51.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139184+0000", 2024-09-18T11:30:51.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001470694 2024-09-18T11:30:51.177 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.178 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:51.178 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.178 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:901 4.0 4:0aec5e1c:::smithi11939506-50:head [sparse-read 0~1781150] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:51.178 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:51.178 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.035859172000002, 2024-09-18T11:30:51.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063828720000000004, 2024-09-18T11:30:51.178 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:51.178 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:51.178 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:51.178 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:51.179 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.179 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 901 2024-09-18T11:30:51.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.179 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:51.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:51.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:51.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:51.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:51.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:51.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:51.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:51.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:51.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133268+0000", 2024-09-18T11:30:51.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9610000000000001e-06 2024-09-18T11:30:51.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:51.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133280+0000", 2024-09-18T11:30:51.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2481e-05 2024-09-18T11:30:51.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133285+0000", 2024-09-18T11:30:51.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5469999999999998e-06 2024-09-18T11:30:51.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133317+0000", 2024-09-18T11:30:51.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1838999999999998e-05 2024-09-18T11:30:51.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:51.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133320+0000", 2024-09-18T11:30:51.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3009999999999998e-06 2024-09-18T11:30:51.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139225+0000", 2024-09-18T11:30:51.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059047919999999999 2024-09-18T11:30:51.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:51.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139330+0000", 2024-09-18T11:30:51.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104924 2024-09-18T11:30:51.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:51.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139648+0000", 2024-09-18T11:30:51.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00031854799999999999 2024-09-18T11:30:51.183 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.183 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:51.183 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.183 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:51.184 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:30:51.184 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.999705132000003, 2024-09-18T11:30:51.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:30:51.184 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:51.184 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:51.184 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:51.184 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:51.184 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.184 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:30:51.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.184 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:51.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:51.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:30:51.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:51.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:30:51.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:51.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:30:51.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:51.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:51.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:30:51.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:30:51.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:51.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:30:51.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:30:51.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:30:51.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:30:51.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:30:51.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:30:51.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:51.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:30:51.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:30:51.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:51.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:30:51.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:30:51.188 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.188 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:51.188 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.188 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1000 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:51.188 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164518+0000", 2024-09-18T11:30:51.188 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.004607026000002, 2024-09-18T11:30:51.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005734093, 2024-09-18T11:30:51.188 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:51.188 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:51.189 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:51.189 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:51.189 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.189 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1000 2024-09-18T11:30:51.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.189 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:51.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:51.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164518+0000", 2024-09-18T11:30:51.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:51.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164518+0000", 2024-09-18T11:30:51.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:51.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164515+0000", 2024-09-18T11:30:51.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:51.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:51.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164525+0000", 2024-09-18T11:30:51.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5480000000000007e-06 2024-09-18T11:30:51.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:51.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164542+0000", 2024-09-18T11:30:51.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7643e-05 2024-09-18T11:30:51.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164553+0000", 2024-09-18T11:30:51.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0581000000000001e-05 2024-09-18T11:30:51.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164637+0000", 2024-09-18T11:30:51.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3825999999999993e-05 2024-09-18T11:30:51.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:51.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164644+0000", 2024-09-18T11:30:51.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5499999999999997e-06 2024-09-18T11:30:51.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.166446+0000", 2024-09-18T11:30:51.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018018789999999999 2024-09-18T11:30:51.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:51.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.166537+0000", 2024-09-18T11:30:51.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0325e-05 2024-09-18T11:30:51.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:51.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170252+0000", 2024-09-18T11:30:51.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037150820000000002 2024-09-18T11:30:51.193 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.193 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:51.193 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.194 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1001 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:51.194 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:30:51.194 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.004553854000001, 2024-09-18T11:30:51.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007461082, 2024-09-18T11:30:51.194 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:51.194 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:51.194 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:51.194 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:51.194 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.194 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1001 2024-09-18T11:30:51.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.194 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:51.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:51.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:30:51.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:51.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:30:51.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:51.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164570+0000", 2024-09-18T11:30:51.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:51.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:51.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164574+0000", 2024-09-18T11:30:51.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.822e-06 2024-09-18T11:30:51.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:51.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164582+0000", 2024-09-18T11:30:51.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3410000000000003e-06 2024-09-18T11:30:51.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164587+0000", 2024-09-18T11:30:51.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769999999999999e-06 2024-09-18T11:30:51.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164702+0000", 2024-09-18T11:30:51.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115705 2024-09-18T11:30:51.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:51.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164709+0000", 2024-09-18T11:30:51.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-09-18T11:30:51.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170296+0000", 2024-09-18T11:30:51.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055876390000000001 2024-09-18T11:30:51.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:51.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170418+0000", 2024-09-18T11:30:51.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121549 2024-09-18T11:30:51.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:51.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172032+0000", 2024-09-18T11:30:51.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00161407 2024-09-18T11:30:51.199 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.199 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:51.199 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.199 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:51.199 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:51.199 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.004527138, 2024-09-18T11:30:51.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:30:51.199 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:51.199 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:51.199 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:51.199 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:51.200 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.200 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:30:51.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.200 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:51.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:51.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:51.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:51.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:51.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:51.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:51.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:51.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:51.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:30:51.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:30:51.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:51.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:30:51.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:30:51.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:30:51.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:30:51.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:30:51.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:30:51.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:51.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:30:51.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:30:51.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:30:51.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:30:51.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:51.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:30:51.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:30:51.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:51.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:30:51.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:30:51.204 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.204 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:51.204 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.204 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:51.204 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:30:51.205 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.004508848999997, 2024-09-18T11:30:51.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:30:51.205 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:51.205 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:51.205 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:51.205 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:51.205 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.205 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:30:51.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.205 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:51.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:51.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:30:51.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:51.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:30:51.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:51.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:30:51.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:51.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:51.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:30:51.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:30:51.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:51.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:30:51.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:30:51.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:30:51.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:30:51.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:30:51.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:30:51.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:51.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:30:51.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:30:51.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:30:51.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:30:51.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:51.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:30:51.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:30:51.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:51.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:30:51.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:30:51.209 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.209 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:51.209 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.210 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:51.210 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:30:51.210 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.004448080000003, 2024-09-18T11:30:51.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:30:51.210 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:51.210 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:51.210 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:51.210 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:51.210 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.210 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:30:51.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.211 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:51.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:51.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:30:51.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:51.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:30:51.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:51.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:30:51.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:51.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:51.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:30:51.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:30:51.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:51.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:30:51.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:30:51.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:30:51.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:30:51.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:30:51.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:30:51.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:51.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:30:51.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:30:51.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:30:51.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:30:51.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:51.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:30:51.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:30:51.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:51.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:30:51.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:30:51.215 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.215 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:51.215 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.215 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:51.215 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:30:51.215 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.004425759, 2024-09-18T11:30:51.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:30:51.215 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:51.215 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:51.216 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:51.216 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:51.216 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.216 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:30:51.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.216 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:51.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:51.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:30:51.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:51.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:30:51.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:51.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:30:51.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:51.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:51.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:30:51.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:30:51.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:51.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:30:51.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:30:51.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:30:51.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:30:51.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:30:51.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:30:51.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:51.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:30:51.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:30:51.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:30:51.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:30:51.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:51.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:30:51.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:30:51.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:51.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:30:51.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:30:51.220 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.220 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:51.220 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.220 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1018 4.0 4:0aec5e1c:::smithi11939506-50:head [write 1622620~730352] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:51.221 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:30:51.221 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.488191203, 2024-09-18T11:30:51.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064405160000000003, 2024-09-18T11:30:51.221 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:51.221 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:51.221 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:51.221 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:51.221 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.221 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1018 2024-09-18T11:30:51.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.221 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:51.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:51.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:30:51.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:51.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:30:51.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:51.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680932+0000", 2024-09-18T11:30:51.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:51.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:51.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681334+0000", 2024-09-18T11:30:51.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000401803 2024-09-18T11:30:51.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:51.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681350+0000", 2024-09-18T11:30:51.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6218999999999998e-05 2024-09-18T11:30:51.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681374+0000", 2024-09-18T11:30:51.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4088e-05 2024-09-18T11:30:51.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681415+0000", 2024-09-18T11:30:51.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1307000000000003e-05 2024-09-18T11:30:51.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:51.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681507+0000", 2024-09-18T11:30:51.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1867999999999995e-05 2024-09-18T11:30:51.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:51.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681671+0000", 2024-09-18T11:30:51.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016406 2024-09-18T11:30:51.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:51.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.683856+0000", 2024-09-18T11:30:51.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002184624 2024-09-18T11:30:51.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:51.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687266+0000", 2024-09-18T11:30:51.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034105490000000001 2024-09-18T11:30:51.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:51.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687321+0000", 2024-09-18T11:30:51.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4960999999999998e-05 2024-09-18T11:30:51.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:51.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687374+0000", 2024-09-18T11:30:51.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2553000000000003e-05 2024-09-18T11:30:51.226 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.226 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:51.226 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.226 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1037 4.0 4:019b6e09:::smithi11939506-40:head [write 477332~766101] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:51.226 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.707508+0000", 2024-09-18T11:30:51.227 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.461616477, 2024-09-18T11:30:51.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050976160000000001, 2024-09-18T11:30:51.227 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:51.227 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:51.227 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:51.227 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:51.227 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.227 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1037 2024-09-18T11:30:51.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.227 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:51.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:51.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.707508+0000", 2024-09-18T11:30:51.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:51.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.707508+0000", 2024-09-18T11:30:51.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:51.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.707506+0000", 2024-09-18T11:30:51.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:51.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:51.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708363+0000", 2024-09-18T11:30:51.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00085727299999999998 2024-09-18T11:30:51.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:51.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708388+0000", 2024-09-18T11:30:51.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5031000000000001e-05 2024-09-18T11:30:51.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708396+0000", 2024-09-18T11:30:51.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0870000000000007e-06 2024-09-18T11:30:51.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708452+0000", 2024-09-18T11:30:51.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.533e-05 2024-09-18T11:30:51.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:51.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708563+0000", 2024-09-18T11:30:51.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011086600000000001 2024-09-18T11:30:51.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:51.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.708761+0000", 2024-09-18T11:30:51.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019803 2024-09-18T11:30:51.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:51.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.711546+0000", 2024-09-18T11:30:51.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002785689 2024-09-18T11:30:51.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:51.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712507+0000", 2024-09-18T11:30:51.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096094300000000002 2024-09-18T11:30:51.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:51.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712557+0000", 2024-09-18T11:30:51.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9701999999999999e-05 2024-09-18T11:30:51.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:51.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.712606+0000", 2024-09-18T11:30:51.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.888e-05 2024-09-18T11:30:51.232 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.232 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:51.232 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.232 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1050 4.0 4:0aec5e1c:::smithi11939506-50:head [write 510946~632952] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:51.232 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.185718+0000", 2024-09-18T11:30:51.232 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 34.983406231000004, 2024-09-18T11:30:51.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055213279999999998, 2024-09-18T11:30:51.233 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:51.233 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:51.233 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:51.233 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:51.233 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.233 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1050 2024-09-18T11:30:51.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.233 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:51.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:51.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185718+0000", 2024-09-18T11:30:51.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:51.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185718+0000", 2024-09-18T11:30:51.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:51.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185717+0000", 2024-09-18T11:30:51.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:51.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:51.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185850+0000", 2024-09-18T11:30:51.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000133616 2024-09-18T11:30:51.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:51.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185867+0000", 2024-09-18T11:30:51.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7047000000000001e-05 2024-09-18T11:30:51.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185874+0000", 2024-09-18T11:30:51.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4250000000000003e-06 2024-09-18T11:30:51.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185926+0000", 2024-09-18T11:30:51.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1867999999999999e-05 2024-09-18T11:30:51.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:51.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.185985+0000", 2024-09-18T11:30:51.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9769000000000003e-05 2024-09-18T11:30:51.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:51.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.186079+0000", 2024-09-18T11:30:51.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3109000000000006e-05 2024-09-18T11:30:51.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:51.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.187878+0000", 2024-09-18T11:30:51.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001799557 2024-09-18T11:30:51.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:51.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191165+0000", 2024-09-18T11:30:51.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003286892 2024-09-18T11:30:51.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:51.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191196+0000", 2024-09-18T11:30:51.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0607e-05 2024-09-18T11:30:51.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:51.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.191240+0000", 2024-09-18T11:30:51.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4078e-05 2024-09-18T11:30:51.238 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.238 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:51.238 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.238 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:51.238 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:30:51.238 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.223300876, 2024-09-18T11:30:51.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:30:51.238 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:51.239 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:51.239 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:51.239 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:51.239 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.239 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:30:51.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.239 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:51.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:51.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:30:51.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:51.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:30:51.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:51.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:30:51.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:30:51.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:51.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:30:51.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:30:51.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:51.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:30:51.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:30:51.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:30:51.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:30:51.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:30:51.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:30:51.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:51.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:30:51.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:30:51.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:51.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:30:51.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:30:51.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:51.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:30:51.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:30:51.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:51.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:30:51.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:30:51.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:51.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:30:51.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:30:51.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:51.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:30:51.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:30:51.244 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.244 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:51.244 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.244 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1121 4.f e38/22 4:f5d7df41:::smithi11939506-16:head v 38'34, mlcod=30'32)", 2024-09-18T11:30:51.244 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:30:51.244 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.203313616999999, 2024-09-18T11:30:51.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060953680000000003, 2024-09-18T11:30:51.244 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:51.244 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:51.244 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:51.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:51.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:30:51.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:51.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:30:51.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:51.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965810+0000", 2024-09-18T11:30:51.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:51.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:51.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966142+0000", 2024-09-18T11:30:51.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000331591 2024-09-18T11:30:51.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:51.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966154+0000", 2024-09-18T11:30:51.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.253e-05 2024-09-18T11:30:51.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966160+0000", 2024-09-18T11:30:51.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7130000000000002e-06 2024-09-18T11:30:51.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966196+0000", 2024-09-18T11:30:51.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6276999999999998e-05 2024-09-18T11:30:51.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:51.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966217+0000", 2024-09-18T11:30:51.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0335000000000001e-05 2024-09-18T11:30:51.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:51.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.971872+0000", 2024-09-18T11:30:51.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056549290000000004 2024-09-18T11:30:51.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:51.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.971906+0000", 2024-09-18T11:30:51.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4702000000000001e-05 2024-09-18T11:30:51.248 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.248 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:51.248 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.249 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:29 3.2 3:5fa8fcfa:::ec59520759824f0488c7a3ca135dacb324de2330e16b7f54053de5719c4e96b6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:51.249 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:30:51.249 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.488942959999999, 2024-09-18T11:30:51.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064126499999999998, 2024-09-18T11:30:51.249 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:51.249 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:51.249 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:51.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:51.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:30:51.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:51.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:30:51.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:51.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680181+0000", 2024-09-18T11:30:51.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:51.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:51.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680654+0000", 2024-09-18T11:30:51.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047334900000000002 2024-09-18T11:30:51.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:51.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680666+0000", 2024-09-18T11:30:51.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2232000000000001e-05 2024-09-18T11:30:51.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680672+0000", 2024-09-18T11:30:51.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:30:51.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681222+0000", 2024-09-18T11:30:51.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00054928699999999997 2024-09-18T11:30:51.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:51.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681332+0000", 2024-09-18T11:30:51.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110634 2024-09-18T11:30:51.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:51.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681552+0000", 2024-09-18T11:30:51.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021926899999999999 2024-09-18T11:30:51.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:51.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682950+0000", 2024-09-18T11:30:51.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013982669999999999 2024-09-18T11:30:51.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:51.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686503+0000", 2024-09-18T11:30:51.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035536130000000002 2024-09-18T11:30:51.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:51.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686553+0000", 2024-09-18T11:30:51.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9801e-05 2024-09-18T11:30:51.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:51.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686594+0000", 2024-09-18T11:30:51.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1022000000000001e-05 2024-09-18T11:30:51.254 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.254 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:51.254 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.254 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:30 3.2 3:49074e56:::7fc21ffee1ba17f2a9197c61ff60ef0b03804c121c8aeb83ea424b7ac77081e9:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:51.254 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.680698+0000", 2024-09-18T11:30:51.255 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.488426642, 2024-09-18T11:30:51.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060349130000000003, 2024-09-18T11:30:51.255 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:51.255 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:51.255 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:51.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:51.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680698+0000", 2024-09-18T11:30:51.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:51.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680698+0000", 2024-09-18T11:30:51.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:51.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680697+0000", 2024-09-18T11:30:51.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:51.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:51.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680750+0000", 2024-09-18T11:30:51.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3170000000000001e-05 2024-09-18T11:30:51.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:51.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680760+0000", 2024-09-18T11:30:51.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3379999999999994e-06 2024-09-18T11:30:51.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680766+0000", 2024-09-18T11:30:51.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8249999999999999e-06 2024-09-18T11:30:51.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682201+0000", 2024-09-18T11:30:51.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001434868 2024-09-18T11:30:51.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:51.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682294+0000", 2024-09-18T11:30:51.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2965000000000002e-05 2024-09-18T11:30:51.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:51.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682445+0000", 2024-09-18T11:30:51.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015103600000000001 2024-09-18T11:30:51.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:51.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.683524+0000", 2024-09-18T11:30:51.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0010789980000000001 2024-09-18T11:30:51.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:51.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686687+0000", 2024-09-18T11:30:51.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003163052 2024-09-18T11:30:51.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:51.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686714+0000", 2024-09-18T11:30:51.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6369000000000001e-05 2024-09-18T11:30:51.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:51.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686733+0000", 2024-09-18T11:30:51.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9117999999999999e-05 2024-09-18T11:30:51.260 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.260 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:51.260 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.260 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1199 4.0 4:0aec5e1c:::smithi11939506-50:head [write 763051~794393] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-18T11:30:51.260 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:30:51.260 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 29.972128836, 2024-09-18T11:30:51.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067768530000000002, 2024-09-18T11:30:51.261 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:51.261 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:51.261 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:51.261 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:51.261 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.261 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1199 2024-09-18T11:30:51.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.261 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:51.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:51.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:30:51.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:51.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:30:51.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:51.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196994+0000", 2024-09-18T11:30:51.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:51.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:51.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197355+0000", 2024-09-18T11:30:51.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036035199999999998 2024-09-18T11:30:51.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:51.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197366+0000", 2024-09-18T11:30:51.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.17e-05 2024-09-18T11:30:51.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197371+0000", 2024-09-18T11:30:51.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6099999999999999e-06 2024-09-18T11:30:51.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197416+0000", 2024-09-18T11:30:51.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5099000000000003e-05 2024-09-18T11:30:51.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:51.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197420+0000", 2024-09-18T11:30:51.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3189999999999996e-06 2024-09-18T11:30:51.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199392+0000", 2024-09-18T11:30:51.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019720139999999998 2024-09-18T11:30:51.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:51.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199452+0000", 2024-09-18T11:30:51.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9929999999999997e-05 2024-09-18T11:30:51.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:51.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199565+0000", 2024-09-18T11:30:51.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112811 2024-09-18T11:30:51.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:51.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.202108+0000", 2024-09-18T11:30:51.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025429060000000002 2024-09-18T11:30:51.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:51.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203714+0000", 2024-09-18T11:30:51.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001606335 2024-09-18T11:30:51.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:51.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203745+0000", 2024-09-18T11:30:51.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0837999999999998e-05 2024-09-18T11:30:51.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:51.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203773+0000", 2024-09-18T11:30:51.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7562000000000001e-05 2024-09-18T11:30:51.267 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.267 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:51.267 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.267 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1660 4.0 4:019b6e09:::smithi11939506-40:17 [read 0~3445402,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:51.267 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349874+0000", 2024-09-18T11:30:51.267 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.819250486, 2024-09-18T11:30:51.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057173650000000003, 2024-09-18T11:30:51.267 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:51.267 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:51.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:51.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:51.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.268 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1660 2024-09-18T11:30:51.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.268 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:51.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:51.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349874+0000", 2024-09-18T11:30:51.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:51.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349874+0000", 2024-09-18T11:30:51.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:51.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349873+0000", 2024-09-18T11:30:51.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:51.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:51.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349878+0000", 2024-09-18T11:30:51.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0860000000000004e-06 2024-09-18T11:30:51.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:51.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349886+0000", 2024-09-18T11:30:51.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4249999999999998e-06 2024-09-18T11:30:51.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349891+0000", 2024-09-18T11:30:51.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8119999999999998e-06 2024-09-18T11:30:51.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.353312+0000", 2024-09-18T11:30:51.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003420799 2024-09-18T11:30:51.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:51.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.353437+0000", 2024-09-18T11:30:51.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012563799999999999 2024-09-18T11:30:51.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:51.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355591+0000", 2024-09-18T11:30:51.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0021542369999999998 2024-09-18T11:30:51.272 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.272 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:51.272 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.272 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1661 4.0 4:019b6e09:::smithi11939506-40:17 [read 0~3445402,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:51.272 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:30:51.272 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.819228172999999, 2024-09-18T11:30:51.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075190880000000002, 2024-09-18T11:30:51.272 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:51.273 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:51.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:51.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:51.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.273 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1661 2024-09-18T11:30:51.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.273 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:51.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:51.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:30:51.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:51.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:30:51.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:51.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:30:51.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:51.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:51.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349899+0000", 2024-09-18T11:30:51.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9749999999999999e-06 2024-09-18T11:30:51.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:51.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349906+0000", 2024-09-18T11:30:51.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9290000000000001e-06 2024-09-18T11:30:51.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349912+0000", 2024-09-18T11:30:51.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9900000000000002e-06 2024-09-18T11:30:51.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355645+0000", 2024-09-18T11:30:51.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057329549999999996 2024-09-18T11:30:51.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:51.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355776+0000", 2024-09-18T11:30:51.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130975 2024-09-18T11:30:51.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:51.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357416+0000", 2024-09-18T11:30:51.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001639649 2024-09-18T11:30:51.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.276 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:51.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.277 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1761 4.0 4:0aec5e1c:::smithi11939506-50:head [write 769787~704463,stat] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:51.277 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:30:51.277 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.8667357590000009, 2024-09-18T11:30:51.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074082879999999999, 2024-09-18T11:30:51.277 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:51.277 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:51.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:51.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:51.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.277 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1761 2024-09-18T11:30:51.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.277 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:51.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:51.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:30:51.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:51.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:30:51.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:51.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:51.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302386+0000", 2024-09-18T11:30:51.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:51.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:51.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302889+0000", 2024-09-18T11:30:51.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050318499999999996 2024-09-18T11:30:51.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:51.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302930+0000", 2024-09-18T11:30:51.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0198999999999999e-05 2024-09-18T11:30:51.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302965+0000", 2024-09-18T11:30:51.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5292e-05 2024-09-18T11:30:51.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303046+0000", 2024-09-18T11:30:51.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1254999999999994e-05 2024-09-18T11:30:51.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:51.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303054+0000", 2024-09-18T11:30:51.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2800000000000003e-06 2024-09-18T11:30:51.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:51.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304658+0000", 2024-09-18T11:30:51.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016039520000000001 2024-09-18T11:30:51.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:51.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304724+0000", 2024-09-18T11:30:51.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5888000000000003e-05 2024-09-18T11:30:51.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:51.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304853+0000", 2024-09-18T11:30:51.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012912700000000001 2024-09-18T11:30:51.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:51.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309084+0000", 2024-09-18T11:30:51.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042311229999999998 2024-09-18T11:30:51.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:51.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309738+0000", 2024-09-18T11:30:51.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00065401399999999996 2024-09-18T11:30:51.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:51.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309770+0000", 2024-09-18T11:30:51.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1652000000000001e-05 2024-09-18T11:30:51.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:51.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:51.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:51.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309797+0000", 2024-09-18T11:30:51.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7052000000000001e-05 2024-09-18T11:30:51.283 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.283 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:51.283 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.283 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:51.283 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:51.283 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:51.285 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:30:51.286 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op export --pgid 3.1 --file /var/log/ceph/exp.3.1.0 2024-09-18T11:30:51.286 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:51.283+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:51.339 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is set 2024-09-18T11:30:51.340 INFO:tasks.rados.rados.0.smithi119.stdout:1017: done (3 left) 2024-09-18T11:30:51.340 INFO:tasks.rados.rados.0.smithi119.stdout:1018: read oid 20 snap -1 2024-09-18T11:30:51.340 INFO:tasks.rados.rados.0.smithi119.stdout:1018: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:30:51.340 INFO:tasks.rados.rados.0.smithi119.stdout:1019: read oid 40 snap 10 2024-09-18T11:30:51.341 INFO:tasks.rados.rados.0.smithi119.stdout:1019: expect (ObjNum 86 snap 5 seq_num 86) 2024-09-18T11:30:51.341 INFO:tasks.rados.rados.0.smithi119.stdout:1020: read oid 8 snap -1 2024-09-18T11:30:51.341 INFO:tasks.rados.rados.0.smithi119.stdout:1020: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-18T11:30:51.341 INFO:tasks.rados.rados.0.smithi119.stdout:1021: tier_evict oid 4 2024-09-18T11:30:51.341 INFO:tasks.rados.rados.0.smithi119.stdout:1021: tier_evict oid 4 snap -1 2024-09-18T11:30:51.341 INFO:tasks.rados.rados.0.smithi119.stdout:1022: write oid 30 current snap is 20 2024-09-18T11:30:51.341 INFO:tasks.rados.rados.0.smithi119.stdout:1022: seq_num 161 ranges {705927=558502,2004895=422707} 2024-09-18T11:30:51.347 INFO:tasks.rados.rados.0.smithi119.stdout:1022: writing smithi11939506-30 from 705927 to 1264429 tid 1 2024-09-18T11:30:51.358 INFO:tasks.rados.rados.0.smithi119.stdout:1022: writing smithi11939506-30 from 2004895 to 2427602 tid 2 2024-09-18T11:30:51.358 INFO:tasks.rados.rados.0.smithi119.stdout:1023: write (excl) oid 47 current snap is 20 2024-09-18T11:30:51.359 INFO:tasks.rados.rados.0.smithi119.stdout:1023: seq_num 162 ranges {722734=633751,2063377=659357} 2024-09-18T11:30:51.366 INFO:tasks.rados.rados.0.smithi119.stdout:1023: writing smithi11939506-47 from 722734 to 1356485 tid 1 2024-09-18T11:30:51.375 INFO:tasks.rados.rados.0.smithi119.stdout:1023: writing smithi11939506-47 from 2063377 to 2722734 tid 2 2024-09-18T11:30:51.375 INFO:tasks.rados.rados.0.smithi119.stdout:1024: rollback oid 36 current snap is 20 2024-09-18T11:30:51.375 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 36 to 19 2024-09-18T11:30:51.375 INFO:tasks.rados.rados.0.smithi119.stdout:1015: got (16) Device or resource busy 2024-09-18T11:30:51.375 INFO:tasks.rados.rados.0.smithi119.stdout:check_oldest_snap_flushed oid 12 v 37 (ObjNum 140 snap 16 seq_num 140) unflushed 16 2024-09-18T11:30:51.375 INFO:tasks.rados.rados.0.smithi119.stdout:1014: got (0) Success 2024-09-18T11:30:51.375 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 41 v 121 (ObjNum 273 snap 0 seq_num 0) clean dne 2024-09-18T11:30:51.375 INFO:tasks.rados.rados.0.smithi119.stdout:1016: finishing rollback tid 0 to smithi11939506-1 2024-09-18T11:30:51.375 INFO:tasks.rados.rados.0.smithi119.stdout:1014: done (9 left) 2024-09-18T11:30:51.375 INFO:tasks.rados.rados.0.smithi119.stdout:1015: done (8 left) 2024-09-18T11:30:51.376 INFO:tasks.rados.rados.0.smithi119.stdout:1025: read oid 18 snap 14 2024-09-18T11:30:51.376 INFO:tasks.rados.rados.0.smithi119.stdout:1025: expect (ObjNum 116 snap 11 seq_num 116) 2024-09-18T11:30:51.376 INFO:tasks.rados.rados.0.smithi119.stdout:1026: write oid 2 current snap is 20 2024-09-18T11:30:51.376 INFO:tasks.rados.rados.0.smithi119.stdout:1016: finishing rollback tid 1 to smithi11939506-1 2024-09-18T11:30:51.376 INFO:tasks.rados.rados.0.smithi119.stdout:1016: finishing rollback tid 2 to smithi11939506-1 2024-09-18T11:30:51.376 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 79 (ObjNum 134 snap 16 seq_num 134) dirty exists 2024-09-18T11:30:51.376 INFO:tasks.rados.rados.0.smithi119.stdout:1021: got (22) Invalid argument 2024-09-18T11:30:51.376 INFO:tasks.rados.rados.0.smithi119.stdout:1019: expect (ObjNum 86 snap 5 seq_num 86) 2024-09-18T11:30:51.385 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:51.384+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:51.396 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:51.396 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:51.405 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:30:51.486 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:51.484+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:51.544 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:51.544 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:51.553 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:30:51.560 INFO:tasks.rados.rados.0.smithi119.stdout:1020: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-18T11:30:51.587 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:51.585+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:51.652 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.652 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1133 4.1 4:8b9a985d:::smithi11939506-23:a [read 0~3327753,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.178999+0000", 2024-09-18T11:30:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.471029465000001, 2024-09-18T11:30:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059312619999999996, 2024-09-18T11:30:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1133 2024-09-18T11:30:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178999+0000", 2024-09-18T11:30:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178999+0000", 2024-09-18T11:30:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.178997+0000", 2024-09-18T11:30:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:51.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179012+0000", 2024-09-18T11:30:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5166e-05 2024-09-18T11:30:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179043+0000", 2024-09-18T11:30:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0892999999999997e-05 2024-09-18T11:30:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179057+0000", 2024-09-18T11:30:51.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4212e-05 2024-09-18T11:30:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179217+0000", 2024-09-18T11:30:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016031699999999999 2024-09-18T11:30:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179235+0000", 2024-09-18T11:30:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7903000000000001e-05 2024-09-18T11:30:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.181282+0000", 2024-09-18T11:30:51.656 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002046614 2024-09-18T11:30:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.181350+0000", 2024-09-18T11:30:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8190999999999998e-05 2024-09-18T11:30:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184930+0000", 2024-09-18T11:30:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035805960000000001 2024-09-18T11:30:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:51.657 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1134 4.1 4:8b9a985d:::smithi11939506-23:a [sparse-read 0~1709094] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.470935684000001, 2024-09-18T11:30:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061866400000000002, 2024-09-18T11:30:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1134 2024-09-18T11:30:51.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:51.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179092+0000", 2024-09-18T11:30:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179104+0000", 2024-09-18T11:30:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2346e-05 2024-09-18T11:30:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179128+0000", 2024-09-18T11:30:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4046999999999999e-05 2024-09-18T11:30:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179141+0000", 2024-09-18T11:30:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3149000000000001e-05 2024-09-18T11:30:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179406+0000", 2024-09-18T11:30:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026496100000000003 2024-09-18T11:30:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179431+0000", 2024-09-18T11:30:51.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5242999999999999e-05 2024-09-18T11:30:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184981+0000", 2024-09-18T11:30:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055495969999999999 2024-09-18T11:30:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185055+0000", 2024-09-18T11:30:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4375999999999994e-05 2024-09-18T11:30:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.662 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185280+0000", 2024-09-18T11:30:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022433800000000001 2024-09-18T11:30:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1135 4.1 4:8b9a985d:::smithi11939506-23:a [sparse-read 0~1709094] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179201+0000", 2024-09-18T11:30:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.470827237000002, 2024-09-18T11:30:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063123149999999998, 2024-09-18T11:30:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:51.663 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1135 2024-09-18T11:30:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179201+0000", 2024-09-18T11:30:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179201+0000", 2024-09-18T11:30:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179200+0000", 2024-09-18T11:30:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179207+0000", 2024-09-18T11:30:51.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1960000000000002e-06 2024-09-18T11:30:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179229+0000", 2024-09-18T11:30:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1322999999999999e-05 2024-09-18T11:30:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179240+0000", 2024-09-18T11:30:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1793e-05 2024-09-18T11:30:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179460+0000", 2024-09-18T11:30:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000219407 2024-09-18T11:30:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179466+0000", 2024-09-18T11:30:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7490000000000004e-06 2024-09-18T11:30:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185300+0000", 2024-09-18T11:30:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058347490000000002 2024-09-18T11:30:51.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185345+0000", 2024-09-18T11:30:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4316000000000001e-05 2024-09-18T11:30:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185514+0000", 2024-09-18T11:30:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016905800000000001 2024-09-18T11:30:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1378 4.d 4:b81644f2:::smithi11939506-34:head [write 615156~555834,stat] snapc c=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-18T11:30:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:25.201447+0000", 2024-09-18T11:30:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.448581683, 2024-09-18T11:30:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060023309999999996, 2024-09-18T11:30:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1378 2024-09-18T11:30:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.669 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:51.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.670 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:51.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.201447+0000", 2024-09-18T11:30:51.670 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.201447+0000", 2024-09-18T11:30:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.201445+0000", 2024-09-18T11:30:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:51.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202027+0000", 2024-09-18T11:30:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00058162700000000003 2024-09-18T11:30:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202044+0000", 2024-09-18T11:30:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6685e-05 2024-09-18T11:30:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202051+0000", 2024-09-18T11:30:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0890000000000002e-06 2024-09-18T11:30:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202086+0000", 2024-09-18T11:30:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5593999999999998e-05 2024-09-18T11:30:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202153+0000", 2024-09-18T11:30:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7237999999999996e-05 2024-09-18T11:30:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.673 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.202277+0000", 2024-09-18T11:30:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012383899999999999 2024-09-18T11:30:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.205380+0000", 2024-09-18T11:30:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031026109999999999 2024-09-18T11:30:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.207341+0000", 2024-09-18T11:30:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019606419999999999 2024-09-18T11:30:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:51.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.207403+0000", 2024-09-18T11:30:51.675 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2921000000000003e-05 2024-09-18T11:30:51.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:51.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:25.207449+0000", 2024-09-18T11:30:51.675 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5824e-05 2024-09-18T11:30:51.675 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.675 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:51.675 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.676 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1421 4.d 4:b5a463bf:::smithi11939506-20:head [write 665577~781581] snapc 10=[10,e,c] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T11:30:51.676 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:29.216974+0000", 2024-09-18T11:30:51.676 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.433055053, 2024-09-18T11:30:51.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006459869, 2024-09-18T11:30:51.676 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:51.676 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:51.676 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:51.676 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:51.676 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.676 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1421 2024-09-18T11:30:51.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.676 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:51.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.676 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:51.677 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.216974+0000", 2024-09-18T11:30:51.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.677 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:51.677 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.216974+0000", 2024-09-18T11:30:51.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.677 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:51.677 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.216972+0000", 2024-09-18T11:30:51.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:51.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.677 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:51.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217766+0000", 2024-09-18T11:30:51.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079344699999999997 2024-09-18T11:30:51.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.678 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:51.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217780+0000", 2024-09-18T11:30:51.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.428e-05 2024-09-18T11:30:51.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.678 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217786+0000", 2024-09-18T11:30:51.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9050000000000002e-06 2024-09-18T11:30:51.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.679 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.679 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217828+0000", 2024-09-18T11:30:51.679 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1839999999999999e-05 2024-09-18T11:30:51.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.679 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:51.679 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217897+0000", 2024-09-18T11:30:51.679 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9663e-05 2024-09-18T11:30:51.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.679 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:51.679 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.218013+0000", 2024-09-18T11:30:51.679 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011509 2024-09-18T11:30:51.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.680 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:51.680 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.220250+0000", 2024-09-18T11:30:51.680 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022370150000000002 2024-09-18T11:30:51.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.680 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:51.680 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.223359+0000", 2024-09-18T11:30:51.680 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031092519999999998 2024-09-18T11:30:51.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.680 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:51.680 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.223396+0000", 2024-09-18T11:30:51.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7531999999999999e-05 2024-09-18T11:30:51.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.681 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:51.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.223433+0000", 2024-09-18T11:30:51.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7098999999999998e-05 2024-09-18T11:30:51.681 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.681 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:51.681 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.681 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1530 4.d 4:b81644f2:::smithi11939506-34:head [write 467261~717275] snapc 14=[14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:30:51.681 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:30:51.681 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.259344105, 2024-09-18T11:30:51.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075660950000000001, 2024-09-18T11:30:51.682 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:51.682 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:51.682 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:51.682 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:51.682 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.682 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1530 2024-09-18T11:30:51.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.682 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:51.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.682 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:51.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:30:51.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.683 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:51.683 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:30:51.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.683 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:51.683 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390682+0000", 2024-09-18T11:30:51.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:51.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.683 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:51.683 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391554+0000", 2024-09-18T11:30:51.684 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00087220600000000004 2024-09-18T11:30:51.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.684 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.684 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:51.684 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391570+0000", 2024-09-18T11:30:51.684 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6002999999999999e-05 2024-09-18T11:30:51.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.684 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.684 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.684 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391575+0000", 2024-09-18T11:30:51.684 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2549999999999997e-06 2024-09-18T11:30:51.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.685 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.685 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.685 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391631+0000", 2024-09-18T11:30:51.685 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5671000000000003e-05 2024-09-18T11:30:51.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.685 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.685 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:51.685 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391751+0000", 2024-09-18T11:30:51.685 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000119821 2024-09-18T11:30:51.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.685 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.685 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:51.685 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391974+0000", 2024-09-18T11:30:51.686 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022309900000000001 2024-09-18T11:30:51.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:51.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398136+0000", 2024-09-18T11:30:51.686 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006161573 2024-09-18T11:30:51.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:51.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398184+0000", 2024-09-18T11:30:51.686 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8458000000000003e-05 2024-09-18T11:30:51.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:51.687 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398219+0000", 2024-09-18T11:30:51.687 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4573999999999998e-05 2024-09-18T11:30:51.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.687 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:51.687 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398251+0000", 2024-09-18T11:30:51.687 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2085000000000002e-05 2024-09-18T11:30:51.687 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.687 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:51.687 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.687 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:30:51.687 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:30:51.688 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.233941780999999, 2024-09-18T11:30:51.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:30:51.688 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:51.688 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:51.688 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:51.688 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:51.688 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.688 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:30:51.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.688 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:51.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.688 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:51.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:30:51.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.689 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:51.689 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:30:51.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.689 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:51.689 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:30:51.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:51.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.690 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.690 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:51.690 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:30:51.690 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:30:51.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.690 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.690 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:51.690 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:30:51.690 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:30:51.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.690 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.690 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.690 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:30:51.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:30:51.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.691 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.691 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:30:51.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:30:51.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.691 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:51.691 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:30:51.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:30:51.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.691 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:51.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:30:51.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:30:51.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:51.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:30:51.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:30:51.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:51.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:30:51.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:30:51.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.693 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:51.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:30:51.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:30:51.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.693 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:51.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:30:51.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:30:51.693 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.694 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:51.694 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.694 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1615 4.e 4:73745217:::smithi11939506-6:head [sparse-read 0~1834156] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:51.694 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.294431+0000", 2024-09-18T11:30:51.694 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.355597738, 2024-09-18T11:30:51.694 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060124970000000003, 2024-09-18T11:30:51.694 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:51.694 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:51.694 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:51.694 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:51.694 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.694 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1615 2024-09-18T11:30:51.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.695 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:51.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:51.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294431+0000", 2024-09-18T11:30:51.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:51.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294431+0000", 2024-09-18T11:30:51.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:51.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294430+0000", 2024-09-18T11:30:51.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:51.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.696 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.696 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:51.696 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294439+0000", 2024-09-18T11:30:51.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9129999999999992e-06 2024-09-18T11:30:51.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.696 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.696 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:51.696 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294454+0000", 2024-09-18T11:30:51.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.524e-05 2024-09-18T11:30:51.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.697 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.697 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.294460+0000", 2024-09-18T11:30:51.697 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6810000000000003e-06 2024-09-18T11:30:51.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.697 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.697 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.300155+0000", 2024-09-18T11:30:51.697 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056949779999999998 2024-09-18T11:30:51.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.697 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:51.697 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.300230+0000", 2024-09-18T11:30:51.698 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5593000000000006e-05 2024-09-18T11:30:51.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.698 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.698 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:51.698 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.300443+0000", 2024-09-18T11:30:51.698 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021289599999999999 2024-09-18T11:30:51.698 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.698 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:51.698 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.698 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.698 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1628 3.3 3:f80aa31b:::77cdbf3abbada57d0636b2198046a030673818705cbf13bb77b30c3e68b24345:head [write 0~582144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:51.698 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:30:51.698 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.339051781, 2024-09-18T11:30:51.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081714639999999998, 2024-09-18T11:30:51.699 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:51.699 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:51.699 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:51.699 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:51.699 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.699 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1628 2024-09-18T11:30:51.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.699 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:51.699 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.699 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:51.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:30:51.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.700 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:51.700 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:30:51.700 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.700 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:51.700 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310975+0000", 2024-09-18T11:30:51.700 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:51.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.700 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:51.700 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312086+0000", 2024-09-18T11:30:51.700 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001111288 2024-09-18T11:30:51.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.701 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:51.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312104+0000", 2024-09-18T11:30:51.701 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7427000000000001e-05 2024-09-18T11:30:51.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.701 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312110+0000", 2024-09-18T11:30:51.701 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5320000000000003e-06 2024-09-18T11:30:51.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.701 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312179+0000", 2024-09-18T11:30:51.702 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9196000000000006e-05 2024-09-18T11:30:51.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.702 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.702 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:51.702 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312375+0000", 2024-09-18T11:30:51.702 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00019550899999999999 2024-09-18T11:30:51.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.702 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.702 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:51.702 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312558+0000", 2024-09-18T11:30:51.702 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000182566 2024-09-18T11:30:51.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.702 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.702 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:51.702 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.314286+0000", 2024-09-18T11:30:51.702 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00172831 2024-09-18T11:30:51.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.703 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.703 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:51.703 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319079+0000", 2024-09-18T11:30:51.703 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047929340000000004 2024-09-18T11:30:51.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.703 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.703 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:51.703 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319116+0000", 2024-09-18T11:30:51.703 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6993999999999999e-05 2024-09-18T11:30:51.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.703 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.703 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:51.703 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319148+0000", 2024-09-18T11:30:51.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2539999999999997e-05 2024-09-18T11:30:51.704 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.704 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:51.704 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.704 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.704 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1662 4.1 4:87078dfe:::smithi11939506-14:head [write 635331~786118,stat] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:51.704 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.360496+0000", 2024-09-18T11:30:51.704 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.289533036, 2024-09-18T11:30:51.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071849519999999997, 2024-09-18T11:30:51.704 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:51.704 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:51.704 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:51.705 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:51.705 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.705 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1662 2024-09-18T11:30:51.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.705 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:51.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.705 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:51.705 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.360496+0000", 2024-09-18T11:30:51.705 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.705 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:51.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.360496+0000", 2024-09-18T11:30:51.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:51.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.360493+0000", 2024-09-18T11:30:51.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:51.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:51.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361139+0000", 2024-09-18T11:30:51.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064615400000000002 2024-09-18T11:30:51.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:51.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361154+0000", 2024-09-18T11:30:51.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4970000000000001e-05 2024-09-18T11:30:51.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361161+0000", 2024-09-18T11:30:51.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4139999999999999e-06 2024-09-18T11:30:51.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361201+0000", 2024-09-18T11:30:51.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0385e-05 2024-09-18T11:30:51.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.708 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.708 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:51.708 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361273+0000", 2024-09-18T11:30:51.708 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2224999999999994e-05 2024-09-18T11:30:51.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.708 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.708 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:51.708 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361406+0000", 2024-09-18T11:30:51.708 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013257699999999999 2024-09-18T11:30:51.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.708 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.708 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:51.709 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367415+0000", 2024-09-18T11:30:51.709 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006009655 2024-09-18T11:30:51.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.709 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:51.709 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367586+0000", 2024-09-18T11:30:51.709 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017109800000000001 2024-09-18T11:30:51.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.709 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:51.709 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367636+0000", 2024-09-18T11:30:51.709 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9721e-05 2024-09-18T11:30:51.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.710 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:51.710 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367681+0000", 2024-09-18T11:30:51.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4382999999999998e-05 2024-09-18T11:30:51.710 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.710 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:51.710 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.710 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1663 4.1 4:87078dfe:::smithi11939506-14:head [write 1916458~318873] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:51.710 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:30:51.711 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.286105217999999, 2024-09-18T11:30:51.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079505749999999997, 2024-09-18T11:30:51.711 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:51.711 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:51.711 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:51.711 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:51.711 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.711 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1663 2024-09-18T11:30:51.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.711 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:51.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:51.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:30:51.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.712 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:51.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:30:51.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.712 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:51.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363922+0000", 2024-09-18T11:30:51.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:51.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.712 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:51.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364211+0000", 2024-09-18T11:30:51.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000288385 2024-09-18T11:30:51.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:51.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364223+0000", 2024-09-18T11:30:51.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2582000000000001e-05 2024-09-18T11:30:51.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364228+0000", 2024-09-18T11:30:51.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0000000000000004e-06 2024-09-18T11:30:51.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364268+0000", 2024-09-18T11:30:51.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0235999999999999e-05 2024-09-18T11:30:51.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.714 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:51.714 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364341+0000", 2024-09-18T11:30:51.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2266000000000002e-05 2024-09-18T11:30:51.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.714 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:51.714 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364444+0000", 2024-09-18T11:30:51.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103632 2024-09-18T11:30:51.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.714 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:51.714 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369461+0000", 2024-09-18T11:30:51.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005016848 2024-09-18T11:30:51.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.715 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:51.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371820+0000", 2024-09-18T11:30:51.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023586169999999999 2024-09-18T11:30:51.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.715 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:51.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371851+0000", 2024-09-18T11:30:51.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1297999999999997e-05 2024-09-18T11:30:51.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.715 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:51.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371874+0000", 2024-09-18T11:30:51.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2841e-05 2024-09-18T11:30:51.715 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.716 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:51.716 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.716 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1664 4.1 4:87078dfe:::smithi11939506-14:head [setxattr _header (57) in=7b,truncate 2235331] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:51.716 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:30:51.716 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.285791561, 2024-09-18T11:30:51.716 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010368235999999999, 2024-09-18T11:30:51.716 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:51.716 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:51.716 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:51.716 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:51.716 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.716 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1664 2024-09-18T11:30:51.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.717 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:51.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.717 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:51.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:30:51.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.717 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:51.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:30:51.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.717 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:51.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364236+0000", 2024-09-18T11:30:51.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:51.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:51.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364240+0000", 2024-09-18T11:30:51.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9829999999999998e-06 2024-09-18T11:30:51.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:51.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364247+0000", 2024-09-18T11:30:51.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2140000000000001e-06 2024-09-18T11:30:51.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364256+0000", 2024-09-18T11:30:51.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1509999999999993e-06 2024-09-18T11:30:51.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366529+0000", 2024-09-18T11:30:51.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022731409999999998 2024-09-18T11:30:51.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:51.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366621+0000", 2024-09-18T11:30:51.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2408e-05 2024-09-18T11:30:51.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.720 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:51.720 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366801+0000", 2024-09-18T11:30:51.720 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018002200000000001 2024-09-18T11:30:51.720 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.720 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.720 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:51.720 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372155+0000", 2024-09-18T11:30:51.720 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053531519999999999 2024-09-18T11:30:51.720 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.720 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.720 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:51.720 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374539+0000", 2024-09-18T11:30:51.720 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023842580000000002 2024-09-18T11:30:51.720 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.720 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.721 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:51.721 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374573+0000", 2024-09-18T11:30:51.721 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4205999999999997e-05 2024-09-18T11:30:51.721 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.721 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.721 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:51.721 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374605+0000", 2024-09-18T11:30:51.721 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2283999999999998e-05 2024-09-18T11:30:51.721 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.721 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:51.721 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.721 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.721 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.721 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1665 4.1 4:87078dfe:::smithi11939506-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:51.721 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:30:51.721 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.285767387, 2024-09-18T11:30:51.722 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010471897000000001, 2024-09-18T11:30:51.722 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:51.722 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:51.722 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:51.722 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:51.722 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.722 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1665 2024-09-18T11:30:51.722 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.722 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:51.722 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.722 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:51.722 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:30:51.722 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.722 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.722 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.723 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:51.723 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:30:51.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.723 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:51.723 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:30:51.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:51.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.723 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:51.723 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364263+0000", 2024-09-18T11:30:51.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6400000000000001e-06 2024-09-18T11:30:51.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:51.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364270+0000", 2024-09-18T11:30:51.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:30:51.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364274+0000", 2024-09-18T11:30:51.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.968e-06 2024-09-18T11:30:51.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367354+0000", 2024-09-18T11:30:51.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030803430000000001 2024-09-18T11:30:51.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:51.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367406+0000", 2024-09-18T11:30:51.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2330999999999998e-05 2024-09-18T11:30:51.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374625+0000", 2024-09-18T11:30:51.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072190689999999998 2024-09-18T11:30:51.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:51.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374673+0000", 2024-09-18T11:30:51.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7525999999999999e-05 2024-09-18T11:30:51.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.726 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:51.726 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374733+0000", 2024-09-18T11:30:51.726 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0359000000000003e-05 2024-09-18T11:30:51.726 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.726 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:51.726 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.726 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:101 4.5 e58/22 4:aa575018:::smithi11939506-11:17 v 58'67, mlcod=55'62)", 2024-09-18T11:30:51.726 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.273235+0000", 2024-09-18T11:30:51.726 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.376794095999999, 2024-09-18T11:30:51.726 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0067904050000000002, 2024-09-18T11:30:51.726 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:51.727 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:51.727 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:51.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.727 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:51.727 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273235+0000", 2024-09-18T11:30:51.727 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.727 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:51.727 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273235+0000", 2024-09-18T11:30:51.727 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.728 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:51.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273234+0000", 2024-09-18T11:30:51.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:51.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.728 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:51.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273239+0000", 2024-09-18T11:30:51.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7999999999999998e-06 2024-09-18T11:30:51.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.728 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:51.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273247+0000", 2024-09-18T11:30:51.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4700000000000002e-06 2024-09-18T11:30:51.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.729 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.729 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273251+0000", 2024-09-18T11:30:51.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7500000000000001e-06 2024-09-18T11:30:51.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.729 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.729 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.274599+0000", 2024-09-18T11:30:51.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013483900000000001 2024-09-18T11:30:51.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.729 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:51.730 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.274623+0000", 2024-09-18T11:30:51.730 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3499999999999999e-05 2024-09-18T11:30:51.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.730 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:51.730 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.279994+0000", 2024-09-18T11:30:51.730 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053713889999999999 2024-09-18T11:30:51.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.730 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:51.730 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.280025+0000", 2024-09-18T11:30:51.730 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0579e-05 2024-09-18T11:30:51.730 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.730 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:51.730 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.731 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1919 4.9 4:99feffe9:::smithi11939506-45:head [write 638699~541153] snapc 1e=[1e,1c,1b,19,18,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:30:51.731 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.342505+0000", 2024-09-18T11:30:51.731 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.3075241120000003, 2024-09-18T11:30:51.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064384309999999997, 2024-09-18T11:30:51.731 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:51.731 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:51.731 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:51.731 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:51.731 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.731 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1919 2024-09-18T11:30:51.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.732 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:51.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.732 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:51.732 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.342505+0000", 2024-09-18T11:30:51.732 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.732 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:51.732 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.342505+0000", 2024-09-18T11:30:51.732 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.732 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:51.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.342501+0000", 2024-09-18T11:30:51.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999962 2024-09-18T11:30:51.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.733 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:51.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.343053+0000", 2024-09-18T11:30:51.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00055221999999999997 2024-09-18T11:30:51.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.733 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:51.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.343069+0000", 2024-09-18T11:30:51.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5917999999999999e-05 2024-09-18T11:30:51.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.734 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.343075+0000", 2024-09-18T11:30:51.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3260000000000001e-06 2024-09-18T11:30:51.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.734 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.343153+0000", 2024-09-18T11:30:51.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8183999999999995e-05 2024-09-18T11:30:51.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.734 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:51.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.343261+0000", 2024-09-18T11:30:51.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0001076 2024-09-18T11:30:51.735 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.735 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.735 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:51.735 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.343469+0000", 2024-09-18T11:30:51.735 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020852199999999999 2024-09-18T11:30:51.735 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.735 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.735 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:51.735 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.346706+0000", 2024-09-18T11:30:51.735 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032370400000000001 2024-09-18T11:30:51.735 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.735 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.735 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:51.736 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.348871+0000", 2024-09-18T11:30:51.736 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021645200000000001 2024-09-18T11:30:51.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.736 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:51.736 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.348914+0000", 2024-09-18T11:30:51.736 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3031999999999997e-05 2024-09-18T11:30:51.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.736 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:51.736 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.348943+0000", 2024-09-18T11:30:51.736 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9039000000000001e-05 2024-09-18T11:30:51.736 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.736 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:51.737 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.737 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.737 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1946 4.d 4:b5a463bf:::smithi11939506-20:head [sparse-read 0~2642181,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:30:51.737 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339490+0000", 2024-09-18T11:30:51.737 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.31053863199999998, 2024-09-18T11:30:51.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060807769999999999, 2024-09-18T11:30:51.737 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:51.737 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:51.737 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:51.737 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:51.737 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.737 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1946 2024-09-18T11:30:51.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.738 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:51.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:51.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339490+0000", 2024-09-18T11:30:51.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:51.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339490+0000", 2024-09-18T11:30:51.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:51.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339488+0000", 2024-09-18T11:30:51.739 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:51.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.739 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.739 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:51.739 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339496+0000", 2024-09-18T11:30:51.739 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6190000000000002e-06 2024-09-18T11:30:51.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.739 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.739 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:51.739 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339515+0000", 2024-09-18T11:30:51.739 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8681999999999998e-05 2024-09-18T11:30:51.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.739 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.739 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339521+0000", 2024-09-18T11:30:51.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1940000000000003e-06 2024-09-18T11:30:51.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.740 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339603+0000", 2024-09-18T11:30:51.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2231000000000006e-05 2024-09-18T11:30:51.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.740 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:51.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339707+0000", 2024-09-18T11:30:51.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104374 2024-09-18T11:30:51.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.741 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:51.741 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345571+0000", 2024-09-18T11:30:51.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058633579999999999 2024-09-18T11:30:51.741 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.741 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:51.741 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.741 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1947 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865577,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:30:51.741 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:30:51.741 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.31048767700000002, 2024-09-18T11:30:51.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010043066999999999, 2024-09-18T11:30:51.741 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:51.741 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:51.742 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:51.742 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:51.742 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.742 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1947 2024-09-18T11:30:51.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.742 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:51.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:51.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:30:51.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:51.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:30:51.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:51.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339540+0000", 2024-09-18T11:30:51.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:51.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:51.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339545+0000", 2024-09-18T11:30:51.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6290000000000001e-06 2024-09-18T11:30:51.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:51.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339553+0000", 2024-09-18T11:30:51.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6299999999999998e-06 2024-09-18T11:30:51.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.744 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.744 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339557+0000", 2024-09-18T11:30:51.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3590000000000001e-06 2024-09-18T11:30:51.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.744 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.744 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345600+0000", 2024-09-18T11:30:51.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060434640000000001 2024-09-18T11:30:51.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.744 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:51.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345710+0000", 2024-09-18T11:30:51.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010924700000000001 2024-09-18T11:30:51.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:51.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349584+0000", 2024-09-18T11:30:51.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038743459999999999 2024-09-18T11:30:51.745 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.745 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:51.745 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.745 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1948 4.d 4:b5a463bf:::smithi11939506-20:head [sparse-read 0~2642181] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:30:51.745 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:30:51.746 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.31046713999999997, 2024-09-18T11:30:51.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010701952000000001, 2024-09-18T11:30:51.746 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:51.746 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:51.746 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:51.746 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:51.746 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:51.746 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1948 2024-09-18T11:30:51.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.746 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:51.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.746 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:51.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:30:51.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:51.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:30:51.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:51.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:30:51.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:51.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:51.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339563+0000", 2024-09-18T11:30:51.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1950000000000002e-06 2024-09-18T11:30:51.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:51.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339569+0000", 2024-09-18T11:30:51.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5459999999999999e-06 2024-09-18T11:30:51.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339572+0000", 2024-09-18T11:30:51.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.337e-06 2024-09-18T11:30:51.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349693+0000", 2024-09-18T11:30:51.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010120987999999999 2024-09-18T11:30:51.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.749 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:51.749 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349816+0000", 2024-09-18T11:30:51.749 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012241199999999999 2024-09-18T11:30:51.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.749 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:51.749 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350263+0000", 2024-09-18T11:30:51.749 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00044787199999999998 2024-09-18T11:30:51.749 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.749 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:51.749 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.750 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:129 4.5 e69/22 4:a087a566:::smithi11939506-46:19 v 69'95, mlcod=65'92)", 2024-09-18T11:30:51.750 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.343498+0000", 2024-09-18T11:30:51.750 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.30653049399999999, 2024-09-18T11:30:51.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068162960000000003, 2024-09-18T11:30:51.750 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:51.750 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:51.750 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:51.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:51.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343498+0000", 2024-09-18T11:30:51.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:51.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343498+0000", 2024-09-18T11:30:51.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.751 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:51.751 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343497+0000", 2024-09-18T11:30:51.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:51.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.751 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:51.751 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343504+0000", 2024-09-18T11:30:51.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5799999999999997e-06 2024-09-18T11:30:51.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.751 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:51.751 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343513+0000", 2024-09-18T11:30:51.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7940000000000008e-06 2024-09-18T11:30:51.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343518+0000", 2024-09-18T11:30:51.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2719999999999997e-06 2024-09-18T11:30:51.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345616+0000", 2024-09-18T11:30:51.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020977410000000002 2024-09-18T11:30:51.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:51.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345641+0000", 2024-09-18T11:30:51.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5539000000000001e-05 2024-09-18T11:30:51.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.753 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:51.753 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350279+0000", 2024-09-18T11:30:51.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046379560000000004 2024-09-18T11:30:51.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.753 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:51.753 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350314+0000", 2024-09-18T11:30:51.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5083999999999998e-05 2024-09-18T11:30:51.753 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.753 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:51.753 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.754 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:131 4.5 e69/22 4:aa575018:::smithi11939506-11:17 v 69'97, mlcod=65'92)", 2024-09-18T11:30:51.754 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.346049+0000", 2024-09-18T11:30:51.754 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.30398007999999999, 2024-09-18T11:30:51.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0077275240000000004, 2024-09-18T11:30:51.754 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:51.754 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:51.754 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:51.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.754 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:51.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346049+0000", 2024-09-18T11:30:51.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:51.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346049+0000", 2024-09-18T11:30:51.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:51.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:51.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346047+0000", 2024-09-18T11:30:51.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:51.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:51.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346058+0000", 2024-09-18T11:30:51.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0463e-05 2024-09-18T11:30:51.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:51.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346069+0000", 2024-09-18T11:30:51.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1708999999999999e-05 2024-09-18T11:30:51.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:51.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346076+0000", 2024-09-18T11:30:51.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2450000000000003e-06 2024-09-18T11:30:51.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:51.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349616+0000", 2024-09-18T11:30:51.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035406550000000002 2024-09-18T11:30:51.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:51.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349638+0000", 2024-09-18T11:30:51.757 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2098000000000001e-05 2024-09-18T11:30:51.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:51.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.353735+0000", 2024-09-18T11:30:51.757 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004096237 2024-09-18T11:30:51.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:51.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:51.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:51.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.353776+0000", 2024-09-18T11:30:51.757 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1530999999999999e-05 2024-09-18T11:30:51.758 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.758 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:51.758 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.758 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:51.758 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:51.758 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:51.759 INFO:tasks.rados.rados.0.smithi119.stdout:1018: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:30:51.760 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:30:51.760 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:51.758+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:51.860 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:51.858+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:51.898 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:51.898 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:51.898 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:51.898 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:51.907 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:30:51.929 INFO:tasks.rados.rados.0.smithi119.stdout:1022: finishing write tid 1 to smithi11939506-30 2024-09-18T11:30:51.930 INFO:tasks.rados.rados.0.smithi119.stdout:1022: finishing write tid 2 to smithi11939506-30 2024-09-18T11:30:51.930 INFO:tasks.rados.rados.0.smithi119.stdout:1022: finishing write tid 3 to smithi11939506-30 2024-09-18T11:30:51.930 INFO:tasks.rados.rados.0.smithi119.stdout:1022: finishing write tid 4 to smithi11939506-30 2024-09-18T11:30:51.930 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 30 v 39 (ObjNum 161 snap 20 seq_num 161) dirty exists 2024-09-18T11:30:51.930 INFO:tasks.rados.rados.0.smithi119.stdout:1022: left oid 30 (ObjNum 161 snap 20 seq_num 161) 2024-09-18T11:30:51.930 INFO:tasks.rados.rados.0.smithi119.stdout:1023: finishing write tid 1 to smithi11939506-47 2024-09-18T11:30:51.930 INFO:tasks.rados.rados.0.smithi119.stdout:1026: seq_num 163 ranges {739541=709000,2121859=426376} 2024-09-18T11:30:51.937 INFO:tasks.rados.rados.0.smithi119.stdout:1026: writing smithi11939506-2 from 739541 to 1448541 tid 1 2024-09-18T11:30:51.943 INFO:tasks.rados.rados.0.smithi119.stdout:1026: writing smithi11939506-2 from 2121859 to 2548235 tid 2 2024-09-18T11:30:51.944 INFO:tasks.rados.rados.0.smithi119.stdout:1016: done (9 left) 2024-09-18T11:30:51.944 INFO:tasks.rados.rados.0.smithi119.stdout:1018: done (8 left) 2024-09-18T11:30:51.944 INFO:tasks.rados.rados.0.smithi119.stdout:1019: done (7 left) 2024-09-18T11:30:51.944 INFO:tasks.rados.rados.0.smithi119.stdout:1020: done (6 left) 2024-09-18T11:30:51.944 INFO:tasks.rados.rados.0.smithi119.stdout:1021: done (5 left) 2024-09-18T11:30:51.944 INFO:tasks.rados.rados.0.smithi119.stdout:1022: done (4 left) 2024-09-18T11:30:51.944 INFO:tasks.rados.rados.0.smithi119.stdout:1027: set_chunk oid 4 target oid 2024-09-18T11:30:51.953 INFO:tasks.rados.rados.0.smithi119.stdout:1027: set_chunk oid 4 offset: 1977344 length: 1042432 target oid c9b84a24a133e2461d917d97907be9530ee74315ec353aa8a24522dbbe08ec71 offset: 0 snap 15 2024-09-18T11:30:51.953 INFO:tasks.rados.rados.0.smithi119.stdout:1028: write (excl) oid 5 current snap is 20 2024-09-18T11:30:51.953 INFO:tasks.rados.rados.0.smithi119.stdout:1023: finishing write tid 2 to smithi11939506-47 2024-09-18T11:30:51.953 INFO:tasks.rados.rados.0.smithi119.stdout:1023: finishing write tid 3 to smithi11939506-47 2024-09-18T11:30:51.953 INFO:tasks.rados.rados.0.smithi119.stdout:1028: seq_num 164 ranges {756348=784249,2063988=686387} 2024-09-18T11:30:51.961 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:51.958+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:51.961 INFO:tasks.rados.rados.0.smithi119.stdout:1028: writing smithi11939506-5 from 756348 to 1540597 tid 1 2024-09-18T11:30:51.968 INFO:tasks.rados.rados.0.smithi119.stdout:1028: writing smithi11939506-5 from 2063988 to 2750375 tid 2 2024-09-18T11:30:51.968 INFO:tasks.rados.rados.0.smithi119.stdout:1029: delete oid 9 current snap is 20 2024-09-18T11:30:51.968 INFO:tasks.rados.rados.0.smithi119.stdout:1023: finishing write tid 4 to smithi11939506-47 2024-09-18T11:30:51.968 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 59 (ObjNum 162 snap 20 seq_num 162) dirty exists 2024-09-18T11:30:51.968 INFO:tasks.rados.rados.0.smithi119.stdout:1023: left oid 47 (ObjNum 162 snap 20 seq_num 162) 2024-09-18T11:30:51.969 INFO:tasks.rados.rados.0.smithi119.stdout:1024: finishing rollback tid 0 to smithi11939506-36 2024-09-18T11:30:51.969 INFO:tasks.rados.rados.0.smithi119.stdout:1024: finishing rollback tid 1 to smithi11939506-36 2024-09-18T11:30:51.969 INFO:tasks.rados.rados.0.smithi119.stdout:1024: finishing rollback tid 2 to smithi11939506-36 2024-09-18T11:30:51.969 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 36 v 75 (ObjNum 100 snap 6 seq_num 100) dirty exists 2024-09-18T11:30:51.969 INFO:tasks.rados.rados.0.smithi119.stdout:1025: expect (ObjNum 116 snap 11 seq_num 116) 2024-09-18T11:30:52.045 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:52.045 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:52.045 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:52.045 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:52.045 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:52.053 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:30:52.098 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:52.097+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:52.153 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:52.153 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:52.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:52.153 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:52.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.153 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:30:52.153 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:30:52.153 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 62.124296307999998, 2024-09-18T11:30:52.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073601550000000002, 2024-09-18T11:30:52.153 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.153 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.153 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.154 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:52.154 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:52.154 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1 2024-09-18T11:30:52.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.154 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:30:52.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:30:52.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.155 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026524+0000", 2024-09-18T11:30:52.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999905 2024-09-18T11:30:52.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.155 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026541+0000", 2024-09-18T11:30:52.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7070000000000001e-05 2024-09-18T11:30:52.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.155 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026554+0000", 2024-09-18T11:30:52.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.288e-05 2024-09-18T11:30:52.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.156 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026564+0000", 2024-09-18T11:30:52.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0197000000000001e-05 2024-09-18T11:30:52.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.156 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026826+0000", 2024-09-18T11:30:52.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026212400000000003 2024-09-18T11:30:52.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.156 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:30:52.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026839+0000", 2024-09-18T11:30:52.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2831e-05 2024-09-18T11:30:52.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.157 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.157 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028605+0000", 2024-09-18T11:30:52.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001765794 2024-09-18T11:30:52.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.157 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.157 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028709+0000", 2024-09-18T11:30:52.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104348 2024-09-18T11:30:52.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.158 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:52.158 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028824+0000", 2024-09-18T11:30:52.158 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011527399999999999 2024-09-18T11:30:52.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.158 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:52.158 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030319+0000", 2024-09-18T11:30:52.158 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001494439 2024-09-18T11:30:52.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.158 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:52.158 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033805+0000", 2024-09-18T11:30:52.158 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034857820000000002 2024-09-18T11:30:52.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.159 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033856+0000", 2024-09-18T11:30:52.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1687999999999997e-05 2024-09-18T11:30:52.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.159 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033894+0000", 2024-09-18T11:30:52.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7481000000000002e-05 2024-09-18T11:30:52.159 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.159 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.159 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.160 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T11:30:52.160 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:30:52.160 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 60.105017877000002, 2024-09-18T11:30:52.160 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055107869999999996, 2024-09-18T11:30:52.160 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.160 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.160 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.160 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:30:52.160 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:30:52.160 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 7 2024-09-18T11:30:52.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.160 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:30:52.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:30:52.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045810+0000", 2024-09-18T11:30:52.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:52.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045820+0000", 2024-09-18T11:30:52.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0215e-05 2024-09-18T11:30:52.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045838+0000", 2024-09-18T11:30:52.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7849000000000001e-05 2024-09-18T11:30:52.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045845+0000", 2024-09-18T11:30:52.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4229999999999999e-06 2024-09-18T11:30:52.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.163 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.163 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045898+0000", 2024-09-18T11:30:52.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2537999999999997e-05 2024-09-18T11:30:52.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.163 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:52.163 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045904+0000", 2024-09-18T11:30:52.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4019999999999999e-06 2024-09-18T11:30:52.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.163 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.163 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047663+0000", 2024-09-18T11:30:52.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017584079999999999 2024-09-18T11:30:52.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.164 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047736+0000", 2024-09-18T11:30:52.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3509000000000004e-05 2024-09-18T11:30:52.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.164 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:52.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047857+0000", 2024-09-18T11:30:52.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012037800000000001 2024-09-18T11:30:52.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.164 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:52.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.050868+0000", 2024-09-18T11:30:52.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030111420000000001 2024-09-18T11:30:52.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:52.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051239+0000", 2024-09-18T11:30:52.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037141000000000002 2024-09-18T11:30:52.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051288+0000", 2024-09-18T11:30:52.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9051000000000002e-05 2024-09-18T11:30:52.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051323+0000", 2024-09-18T11:30:52.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4545000000000002e-05 2024-09-18T11:30:52.166 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.166 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.166 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.166 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:905 4.7 e31/22 4:e73ba68f:::smithi11939506-1:head v 31'32, mlcod=30'29)", 2024-09-18T11:30:52.166 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:30:52.166 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 41.009361665999997, 2024-09-18T11:30:52.166 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048351990000000001, 2024-09-18T11:30:52.166 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.166 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.166 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:30:52.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:30:52.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:30:52.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144912+0000", 2024-09-18T11:30:52.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003444 2024-09-18T11:30:52.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144921+0000", 2024-09-18T11:30:52.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1900000000000001e-06 2024-09-18T11:30:52.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144927+0000", 2024-09-18T11:30:52.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4940000000000003e-06 2024-09-18T11:30:52.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144952+0000", 2024-09-18T11:30:52.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5845000000000001e-05 2024-09-18T11:30:52.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144966+0000", 2024-09-18T11:30:52.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.396e-05 2024-09-18T11:30:52.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.146274+0000", 2024-09-18T11:30:52.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013073550000000001 2024-09-18T11:30:52.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.146303+0000", 2024-09-18T11:30:52.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9754e-05 2024-09-18T11:30:52.170 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.170 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.170 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.170 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:911 4.3 e31/22 4:cc46532a:::smithi11939506-33:head v 31'24, mlcod=30'21)", 2024-09-18T11:30:52.170 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.166801+0000", 2024-09-18T11:30:52.171 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.984029339999999, 2024-09-18T11:30:52.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048144349999999997, 2024-09-18T11:30:52.171 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.171 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.171 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.166801+0000", 2024-09-18T11:30:52.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.166801+0000", 2024-09-18T11:30:52.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.172 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.172 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.166799+0000", 2024-09-18T11:30:52.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:52.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.172 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.172 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167413+0000", 2024-09-18T11:30:52.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000613259 2024-09-18T11:30:52.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.173 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167431+0000", 2024-09-18T11:30:52.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7906000000000001e-05 2024-09-18T11:30:52.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.173 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167438+0000", 2024-09-18T11:30:52.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.883e-06 2024-09-18T11:30:52.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.174 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.174 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167468+0000", 2024-09-18T11:30:52.174 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0168e-05 2024-09-18T11:30:52.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.174 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.174 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167483+0000", 2024-09-18T11:30:52.174 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4997999999999999e-05 2024-09-18T11:30:52.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.174 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.174 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.171574+0000", 2024-09-18T11:30:52.174 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040909589999999999 2024-09-18T11:30:52.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.175 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.175 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.171615+0000", 2024-09-18T11:30:52.175 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1371000000000001e-05 2024-09-18T11:30:52.175 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.175 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.175 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.175 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:990 4.3 e35/22 4:ca2b2309:::smithi11939506-44:head v 35'31, mlcod=34'28)", 2024-09-18T11:30:52.175 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:15.158794+0000", 2024-09-18T11:30:52.176 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.992036292000002, 2024-09-18T11:30:52.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042163180000000001, 2024-09-18T11:30:52.176 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.176 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.176 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.158794+0000", 2024-09-18T11:30:52.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.158794+0000", 2024-09-18T11:30:52.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.158793+0000", 2024-09-18T11:30:52.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:52.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159307+0000", 2024-09-18T11:30:52.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00051453999999999996 2024-09-18T11:30:52.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159320+0000", 2024-09-18T11:30:52.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2639e-05 2024-09-18T11:30:52.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159327+0000", 2024-09-18T11:30:52.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3200000000000002e-06 2024-09-18T11:30:52.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159359+0000", 2024-09-18T11:30:52.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2147999999999998e-05 2024-09-18T11:30:52.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.179 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.179 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.179 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.159380+0000", 2024-09-18T11:30:52.179 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0625999999999999e-05 2024-09-18T11:30:52.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.179 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.179 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.179 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.162980+0000", 2024-09-18T11:30:52.179 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003600125 2024-09-18T11:30:52.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.179 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.179 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.179 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:15.163010+0000", 2024-09-18T11:30:52.180 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0020999999999999e-05 2024-09-18T11:30:52.180 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.180 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.180 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.180 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.180 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.180 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1118 3.2 e38/21 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head v 38'76, mlcod=38'74)", 2024-09-18T11:30:52.180 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:30:52.180 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.202886155999998, 2024-09-18T11:30:52.180 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006048688, 2024-09-18T11:30:52.180 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.180 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.180 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.180 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.180 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:30:52.181 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.181 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:30:52.181 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.181 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947941+0000", 2024-09-18T11:30:52.181 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:52.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.182 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.182 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.182 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949868+0000", 2024-09-18T11:30:52.182 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019267659999999999 2024-09-18T11:30:52.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.182 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.182 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.182 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949893+0000", 2024-09-18T11:30:52.182 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5239999999999999e-05 2024-09-18T11:30:52.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.182 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.182 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.182 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949901+0000", 2024-09-18T11:30:52.182 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8420000000000005e-06 2024-09-18T11:30:52.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.183 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.183 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.183 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949954+0000", 2024-09-18T11:30:52.183 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3464999999999997e-05 2024-09-18T11:30:52.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.183 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.183 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.183 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949974+0000", 2024-09-18T11:30:52.183 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9731000000000001e-05 2024-09-18T11:30:52.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.183 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.184 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.184 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953948+0000", 2024-09-18T11:30:52.184 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039735489999999998 2024-09-18T11:30:52.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.184 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.184 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.184 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953992+0000", 2024-09-18T11:30:52.184 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4700999999999998e-05 2024-09-18T11:30:52.184 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.184 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.184 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.185 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.185 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1165 4.7 e39/22 4:e73ba68f:::smithi11939506-1:head v 39'40, mlcod=34'37)", 2024-09-18T11:30:52.185 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.657838+0000", 2024-09-18T11:30:52.185 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.492991494000002, 2024-09-18T11:30:52.185 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042136450000000002, 2024-09-18T11:30:52.185 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.185 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.185 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.185 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.185 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.185 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.657838+0000", 2024-09-18T11:30:52.185 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.185 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.186 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.657838+0000", 2024-09-18T11:30:52.186 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.186 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.657837+0000", 2024-09-18T11:30:52.186 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:52.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.186 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659431+0000", 2024-09-18T11:30:52.186 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015942479999999999 2024-09-18T11:30:52.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659450+0000", 2024-09-18T11:30:52.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8909000000000001e-05 2024-09-18T11:30:52.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659457+0000", 2024-09-18T11:30:52.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8390000000000001e-06 2024-09-18T11:30:52.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659500+0000", 2024-09-18T11:30:52.188 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3101e-05 2024-09-18T11:30:52.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.188 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.188 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.659518+0000", 2024-09-18T11:30:52.188 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8020999999999999e-05 2024-09-18T11:30:52.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.188 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.188 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.662024+0000", 2024-09-18T11:30:52.188 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025059399999999999 2024-09-18T11:30:52.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.189 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.189 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.189 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.662052+0000", 2024-09-18T11:30:52.189 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8206999999999999e-05 2024-09-18T11:30:52.189 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.189 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.189 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.189 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.189 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.13:29 3.2 e39/21 3:5fa8fcfa:::ec59520759824f0488c7a3ca135dacb324de2330e16b7f54053de5719c4e96b6:head v 39'81, mlcod=38'76)", 2024-09-18T11:30:52.189 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:30:52.189 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.469185679999999, 2024-09-18T11:30:52.189 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004755858, 2024-09-18T11:30:52.190 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.190 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.190 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.190 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.190 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.190 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:30:52.190 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.190 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.190 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.190 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:30:52.190 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.191 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.191 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.191 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:30:52.191 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.191 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.191 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.191 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681961+0000", 2024-09-18T11:30:52.191 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00031696799999999999 2024-09-18T11:30:52.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.191 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.191 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.191 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681969+0000", 2024-09-18T11:30:52.192 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4800000000000001e-06 2024-09-18T11:30:52.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.192 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.192 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.192 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681973+0000", 2024-09-18T11:30:52.192 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4819999999999999e-06 2024-09-18T11:30:52.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.192 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.192 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.192 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682007+0000", 2024-09-18T11:30:52.192 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3961999999999998e-05 2024-09-18T11:30:52.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.193 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.193 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.193 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682033+0000", 2024-09-18T11:30:52.193 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6103999999999999e-05 2024-09-18T11:30:52.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.193 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.193 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.193 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686375+0000", 2024-09-18T11:30:52.193 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043422019999999999 2024-09-18T11:30:52.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.193 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.193 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.194 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686400+0000", 2024-09-18T11:30:52.194 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5265e-05 2024-09-18T11:30:52.194 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.194 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.194 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.194 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.194 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.194 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1623 4.3 e56/22 4:c70c3162:::smithi11939506-35:head v 56'43, mlcod=48'40)", 2024-09-18T11:30:52.194 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.302560+0000", 2024-09-18T11:30:52.194 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.848270342999999, 2024-09-18T11:30:52.194 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046159060000000003, 2024-09-18T11:30:52.195 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.195 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.195 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.195 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.195 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.195 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.302560+0000", 2024-09-18T11:30:52.195 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.195 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.195 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.195 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.302560+0000", 2024-09-18T11:30:52.196 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.196 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.196 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.196 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.196 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.302559+0000", 2024-09-18T11:30:52.196 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.196 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.196 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.196 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.196 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303837+0000", 2024-09-18T11:30:52.196 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012781089999999999 2024-09-18T11:30:52.196 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.196 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.196 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.197 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303853+0000", 2024-09-18T11:30:52.197 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5551999999999999e-05 2024-09-18T11:30:52.197 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.197 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.197 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.197 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303862+0000", 2024-09-18T11:30:52.197 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0489999999999993e-06 2024-09-18T11:30:52.197 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.197 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.197 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.197 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303914+0000", 2024-09-18T11:30:52.197 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1697999999999999e-05 2024-09-18T11:30:52.197 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.198 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.198 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.198 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303938+0000", 2024-09-18T11:30:52.198 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4369e-05 2024-09-18T11:30:52.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.198 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.198 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.198 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.307122+0000", 2024-09-18T11:30:52.198 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031841780000000002 2024-09-18T11:30:52.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.198 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.198 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.198 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.307176+0000", 2024-09-18T11:30:52.199 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3400999999999999e-05 2024-09-18T11:30:52.199 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.199 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.199 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.199 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.199 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.199 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1662 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'73, mlcod=51'69)", 2024-09-18T11:30:52.199 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:30:52.199 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.789281121, 2024-09-18T11:30:52.199 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059408539999999998, 2024-09-18T11:30:52.199 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.199 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.199 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.199 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.200 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.200 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:30:52.200 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.200 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.200 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.200 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.200 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:30:52.200 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.200 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.200 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.200 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.200 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:30:52.200 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.201 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.201 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.201 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361914+0000", 2024-09-18T11:30:52.201 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036569 2024-09-18T11:30:52.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.201 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.201 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.201 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361926+0000", 2024-09-18T11:30:52.201 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1960999999999999e-05 2024-09-18T11:30:52.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.201 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.201 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.201 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361937+0000", 2024-09-18T11:30:52.201 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0876999999999999e-05 2024-09-18T11:30:52.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.202 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.202 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.202 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361986+0000", 2024-09-18T11:30:52.202 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8652000000000001e-05 2024-09-18T11:30:52.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.202 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.202 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.202 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.362012+0000", 2024-09-18T11:30:52.202 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5854999999999999e-05 2024-09-18T11:30:52.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.202 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.202 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.202 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367460+0000", 2024-09-18T11:30:52.203 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054482100000000002 2024-09-18T11:30:52.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.203 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.203 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.203 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367490+0000", 2024-09-18T11:30:52.203 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9909000000000001e-05 2024-09-18T11:30:52.203 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.203 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.203 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.203 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.203 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1663 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'74, mlcod=51'69)", 2024-09-18T11:30:52.203 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364521+0000", 2024-09-18T11:30:52.203 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.786309123000001, 2024-09-18T11:30:52.203 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048541399999999998, 2024-09-18T11:30:52.204 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.204 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.204 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.204 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.204 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.204 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364521+0000", 2024-09-18T11:30:52.204 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.204 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.204 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.204 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364521+0000", 2024-09-18T11:30:52.204 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.205 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.205 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.205 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364520+0000", 2024-09-18T11:30:52.205 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.205 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.205 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.205 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.205 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364616+0000", 2024-09-18T11:30:52.205 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5569999999999995e-05 2024-09-18T11:30:52.205 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.205 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.205 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.205 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364625+0000", 2024-09-18T11:30:52.205 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3379999999999994e-06 2024-09-18T11:30:52.206 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.206 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.206 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.206 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364629+0000", 2024-09-18T11:30:52.206 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2490000000000003e-06 2024-09-18T11:30:52.206 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.206 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.206 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.206 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364664+0000", 2024-09-18T11:30:52.206 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5132999999999997e-05 2024-09-18T11:30:52.206 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.206 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.206 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.206 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364687+0000", 2024-09-18T11:30:52.207 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2691999999999999e-05 2024-09-18T11:30:52.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.207 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.207 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.207 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369349+0000", 2024-09-18T11:30:52.207 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046622950000000003 2024-09-18T11:30:52.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.207 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.207 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.207 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369375+0000", 2024-09-18T11:30:52.207 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5534e-05 2024-09-18T11:30:52.207 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.207 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.208 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.208 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.208 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.208 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1664 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'75, mlcod=51'69)", 2024-09-18T11:30:52.208 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:30:52.208 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.783942614000001, 2024-09-18T11:30:52.208 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051832229999999998, 2024-09-18T11:30:52.208 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.208 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.208 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.208 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.208 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.209 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:30:52.209 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.209 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.209 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.209 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:30:52.209 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.209 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.209 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.209 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:30:52.209 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.210 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.210 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.210 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366894+0000", 2024-09-18T11:30:52.210 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2219999999999996e-06 2024-09-18T11:30:52.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.210 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.210 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.211 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366906+0000", 2024-09-18T11:30:52.211 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2278e-05 2024-09-18T11:30:52.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.211 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.211 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.211 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366912+0000", 2024-09-18T11:30:52.211 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2430000000000002e-06 2024-09-18T11:30:52.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.211 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.211 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.211 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366952+0000", 2024-09-18T11:30:52.211 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9428999999999998e-05 2024-09-18T11:30:52.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.212 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.212 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.212 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366972+0000", 2024-09-18T11:30:52.212 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0554999999999999e-05 2024-09-18T11:30:52.212 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.212 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.212 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.212 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372045+0000", 2024-09-18T11:30:52.212 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050727319999999999 2024-09-18T11:30:52.212 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.212 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.212 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.212 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372071+0000", 2024-09-18T11:30:52.213 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5477e-05 2024-09-18T11:30:52.213 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.213 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.213 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.213 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.213 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.213 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1673 4.7 e57/22 4:e73ba68f:::smithi11939506-1:head v 57'70, mlcod=53'66)", 2024-09-18T11:30:52.213 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:38.836645+0000", 2024-09-18T11:30:52.213 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.314184600000001, 2024-09-18T11:30:52.213 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043162260000000003, 2024-09-18T11:30:52.213 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.213 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.213 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.213 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.214 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.214 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:38.836645+0000", 2024-09-18T11:30:52.214 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.214 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.214 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.214 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:38.836645+0000", 2024-09-18T11:30:52.214 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.214 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.214 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.214 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:38.836645+0000", 2024-09-18T11:30:52.214 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.215 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.215 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.215 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:38.836650+0000", 2024-09-18T11:30:52.215 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9559999999999996e-06 2024-09-18T11:30:52.215 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.215 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.215 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.215 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:38.836657+0000", 2024-09-18T11:30:52.215 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7490000000000001e-06 2024-09-18T11:30:52.215 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.215 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.215 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.215 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:38.836662+0000", 2024-09-18T11:30:52.216 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6980000000000003e-06 2024-09-18T11:30:52.216 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.216 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.216 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.216 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:38.837225+0000", 2024-09-18T11:30:52.216 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056309700000000003 2024-09-18T11:30:52.216 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.216 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.216 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.216 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:38.837248+0000", 2024-09-18T11:30:52.216 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2972e-05 2024-09-18T11:30:52.216 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.216 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.216 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.217 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:38.840914+0000", 2024-09-18T11:30:52.217 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036657859999999999 2024-09-18T11:30:52.217 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.217 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.217 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.217 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:38.840962+0000", 2024-09-18T11:30:52.217 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7499e-05 2024-09-18T11:30:52.217 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.217 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.217 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.217 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.217 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.217 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:269 4.7 e58/22 4:e73ba68f:::smithi11939506-1:13 v 58'72, mlcod=57'69)", 2024-09-18T11:30:52.218 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:30:52.218 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.885449224, 2024-09-18T11:30:52.218 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065512569999999996, 2024-09-18T11:30:52.218 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.218 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.218 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.218 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.218 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.218 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:30:52.218 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.218 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.218 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.219 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.219 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:30:52.219 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.219 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.219 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.219 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.219 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265380+0000", 2024-09-18T11:30:52.219 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.219 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.219 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.219 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.219 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265388+0000", 2024-09-18T11:30:52.220 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.174e-06 2024-09-18T11:30:52.220 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.220 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.220 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.220 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265398+0000", 2024-09-18T11:30:52.220 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2599999999999994e-06 2024-09-18T11:30:52.220 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.220 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.220 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.220 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265402+0000", 2024-09-18T11:30:52.220 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0289999999999999e-06 2024-09-18T11:30:52.220 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.220 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.221 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.221 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265434+0000", 2024-09-18T11:30:52.221 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2511000000000001e-05 2024-09-18T11:30:52.221 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.221 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.221 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.221 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265447+0000", 2024-09-18T11:30:52.221 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2947e-05 2024-09-18T11:30:52.221 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.221 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.221 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.221 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271892+0000", 2024-09-18T11:30:52.222 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064445930000000002 2024-09-18T11:30:52.222 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.222 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.222 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.222 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271932+0000", 2024-09-18T11:30:52.222 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0303999999999999e-05 2024-09-18T11:30:52.222 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.222 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.222 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.222 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.222 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.222 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1788 4.8 4:1898b15d:::smithi11939506-25:head [write 453822~797061] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:52.222 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:30:52.222 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.2350166409999996, 2024-09-18T11:30:52.223 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094305159999999999, 2024-09-18T11:30:52.223 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.223 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.223 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.223 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.223 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.223 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1788 2024-09-18T11:30:52.223 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.223 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.223 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.224 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.224 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:30:52.224 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.224 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.224 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.224 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.224 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:30:52.224 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.224 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.224 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.224 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.224 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915811+0000", 2024-09-18T11:30:52.225 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:52.225 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.225 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.225 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.225 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916608+0000", 2024-09-18T11:30:52.225 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079635000000000003 2024-09-18T11:30:52.225 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.225 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.225 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.225 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916624+0000", 2024-09-18T11:30:52.225 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6735000000000002e-05 2024-09-18T11:30:52.225 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.225 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.226 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.226 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916630+0000", 2024-09-18T11:30:52.226 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5709999999999997e-06 2024-09-18T11:30:52.226 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.226 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.226 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.226 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916674+0000", 2024-09-18T11:30:52.226 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4540999999999999e-05 2024-09-18T11:30:52.226 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.226 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.226 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.226 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916751+0000", 2024-09-18T11:30:52.226 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6695999999999999e-05 2024-09-18T11:30:52.227 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.227 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.227 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:52.227 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916872+0000", 2024-09-18T11:30:52.227 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120648 2024-09-18T11:30:52.227 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.227 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.227 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:52.227 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.919076+0000", 2024-09-18T11:30:52.227 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022046829999999998 2024-09-18T11:30:52.227 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.227 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.228 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:52.228 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925151+0000", 2024-09-18T11:30:52.228 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060740239999999999 2024-09-18T11:30:52.228 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.228 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.228 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.228 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925198+0000", 2024-09-18T11:30:52.228 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7896999999999999e-05 2024-09-18T11:30:52.228 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.228 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.228 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.228 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925244+0000", 2024-09-18T11:30:52.228 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5398999999999997e-05 2024-09-18T11:30:52.229 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.229 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.229 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.229 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.229 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.229 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1789 4.8 4:1898b15d:::smithi11939506-25:head [write 1760371~653366] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:52.229 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.921860+0000", 2024-09-18T11:30:52.229 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.2289699560000003, 2024-09-18T11:30:52.229 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055904329999999997, 2024-09-18T11:30:52.229 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.229 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.229 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.230 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.230 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.230 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1789 2024-09-18T11:30:52.230 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.230 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.230 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.230 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.230 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.921860+0000", 2024-09-18T11:30:52.230 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.230 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.230 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.230 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.230 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.921860+0000", 2024-09-18T11:30:52.231 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.231 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.231 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.231 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.231 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.921859+0000", 2024-09-18T11:30:52.231 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:52.231 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.231 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.231 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.231 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922436+0000", 2024-09-18T11:30:52.231 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00057708300000000004 2024-09-18T11:30:52.231 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.231 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.231 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.232 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922450+0000", 2024-09-18T11:30:52.232 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4575e-05 2024-09-18T11:30:52.232 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.232 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.232 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.232 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922458+0000", 2024-09-18T11:30:52.232 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:30:52.232 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.232 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.232 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.232 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922508+0000", 2024-09-18T11:30:52.232 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0102999999999998e-05 2024-09-18T11:30:52.232 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.233 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.233 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.233 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922607+0000", 2024-09-18T11:30:52.233 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8813000000000004e-05 2024-09-18T11:30:52.233 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.233 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.233 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:52.233 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922814+0000", 2024-09-18T11:30:52.233 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000207488 2024-09-18T11:30:52.233 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.233 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.233 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:52.234 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925504+0000", 2024-09-18T11:30:52.234 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026899200000000002 2024-09-18T11:30:52.234 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.234 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.234 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:52.234 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927350+0000", 2024-09-18T11:30:52.234 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00184575 2024-09-18T11:30:52.234 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.234 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.234 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.234 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927404+0000", 2024-09-18T11:30:52.234 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4147000000000002e-05 2024-09-18T11:30:52.234 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.234 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.235 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.235 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927450+0000", 2024-09-18T11:30:52.235 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6480000000000002e-05 2024-09-18T11:30:52.235 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.235 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.235 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.235 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.235 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.235 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1790 4.8 4:1898b15d:::smithi11939506-25:head [setxattr _header (57) in=7b,truncate 2453822] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:52.235 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.922467+0000", 2024-09-18T11:30:52.235 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.2283625350000005, 2024-09-18T11:30:52.235 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0052363219999999999, 2024-09-18T11:30:52.235 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.236 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.236 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.236 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.236 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.236 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1790 2024-09-18T11:30:52.236 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.236 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.236 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.236 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.236 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922467+0000", 2024-09-18T11:30:52.236 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.236 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.236 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.237 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.237 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922467+0000", 2024-09-18T11:30:52.237 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.237 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.237 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.237 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.237 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922467+0000", 2024-09-18T11:30:52.237 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.237 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.237 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.237 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.237 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922470+0000", 2024-09-18T11:30:52.237 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2459999999999999e-06 2024-09-18T11:30:52.238 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.238 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.238 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.238 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922474+0000", 2024-09-18T11:30:52.238 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:30:52.238 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.238 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.238 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.238 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922476+0000", 2024-09-18T11:30:52.238 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3410000000000001e-06 2024-09-18T11:30:52.238 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.238 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.238 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.239 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.924488+0000", 2024-09-18T11:30:52.239 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002012425 2024-09-18T11:30:52.239 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.239 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.239 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.239 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.924623+0000", 2024-09-18T11:30:52.239 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013435699999999999 2024-09-18T11:30:52.239 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.239 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.239 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:52.239 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.924870+0000", 2024-09-18T11:30:52.239 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00024748199999999999 2024-09-18T11:30:52.239 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.239 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.240 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:52.240 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927574+0000", 2024-09-18T11:30:52.240 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027032800000000002 2024-09-18T11:30:52.240 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.240 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.240 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:52.240 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927628+0000", 2024-09-18T11:30:52.240 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3909000000000002e-05 2024-09-18T11:30:52.240 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.240 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.240 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.240 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927661+0000", 2024-09-18T11:30:52.240 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3467999999999999e-05 2024-09-18T11:30:52.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.241 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.241 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.241 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927704+0000", 2024-09-18T11:30:52.241 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2749e-05 2024-09-18T11:30:52.241 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.241 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.241 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.241 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.241 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1791 4.8 4:1898b15d:::smithi11939506-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:52.241 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.922500+0000", 2024-09-18T11:30:52.241 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.2283298639999991, 2024-09-18T11:30:52.242 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005360259, 2024-09-18T11:30:52.242 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.242 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:52.242 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.242 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.242 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.242 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1791 2024-09-18T11:30:52.242 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.242 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.242 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.242 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.242 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922500+0000", 2024-09-18T11:30:52.242 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.243 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.243 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.243 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922500+0000", 2024-09-18T11:30:52.243 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.243 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.243 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.243 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922500+0000", 2024-09-18T11:30:52.243 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.243 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.244 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.244 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922503+0000", 2024-09-18T11:30:52.244 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7009999999999998e-06 2024-09-18T11:30:52.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.244 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.244 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.244 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922509+0000", 2024-09-18T11:30:52.244 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1710000000000001e-06 2024-09-18T11:30:52.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.244 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.244 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.244 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922511+0000", 2024-09-18T11:30:52.244 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:30:52.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.245 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.245 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.245 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925283+0000", 2024-09-18T11:30:52.245 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027722659999999998 2024-09-18T11:30:52.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.245 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.245 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:52.245 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925376+0000", 2024-09-18T11:30:52.245 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3018999999999998e-05 2024-09-18T11:30:52.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.245 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.245 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.245 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927736+0000", 2024-09-18T11:30:52.245 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023600990000000001 2024-09-18T11:30:52.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.246 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.246 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.246 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927798+0000", 2024-09-18T11:30:52.246 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1197000000000004e-05 2024-09-18T11:30:52.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.246 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.246 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.246 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927860+0000", 2024-09-18T11:30:52.246 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2725999999999995e-05 2024-09-18T11:30:52.246 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.246 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.246 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.247 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.247 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1804 4.1 e60/22 4:854aa293:::smithi11939506-26:head v 60'81, mlcod=58'77)", 2024-09-18T11:30:52.247 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.928460+0000", 2024-09-18T11:30:52.247 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.2223702260000007, 2024-09-18T11:30:52.247 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0044520979999999998, 2024-09-18T11:30:52.247 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.247 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.247 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.247 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.247 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.247 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.928460+0000", 2024-09-18T11:30:52.247 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.248 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.248 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.248 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.928460+0000", 2024-09-18T11:30:52.248 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.248 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.248 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.248 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.928459+0000", 2024-09-18T11:30:52.248 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.248 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.248 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.248 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.928571+0000", 2024-09-18T11:30:52.249 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011205300000000001 2024-09-18T11:30:52.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.249 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.249 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.928579+0000", 2024-09-18T11:30:52.249 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1019999999999997e-06 2024-09-18T11:30:52.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.249 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.249 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.928583+0000", 2024-09-18T11:30:52.249 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1930000000000003e-06 2024-09-18T11:30:52.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.249 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.249 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.928621+0000", 2024-09-18T11:30:52.250 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7954999999999998e-05 2024-09-18T11:30:52.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.250 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.250 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.928636+0000", 2024-09-18T11:30:52.250 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4994000000000001e-05 2024-09-18T11:30:52.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.250 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.250 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.932871+0000", 2024-09-18T11:30:52.250 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042342459999999997 2024-09-18T11:30:52.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.250 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.250 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.932912+0000", 2024-09-18T11:30:52.251 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1195000000000001e-05 2024-09-18T11:30:52.251 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.251 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.251 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.251 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.251 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1934 4.2 e65/22 4:425afe9c:::smithi11939506-29:head v 65'44, mlcod=60'40)", 2024-09-18T11:30:52.251 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:30:52.251 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.850837871, 2024-09-18T11:30:52.251 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064585119999999996, 2024-09-18T11:30:52.251 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.251 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.251 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.251 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.251 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.252 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:30:52.252 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.252 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.252 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.252 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:30:52.252 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.252 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.252 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.252 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299990+0000", 2024-09-18T11:30:52.252 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:52.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.252 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.253 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.253 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300789+0000", 2024-09-18T11:30:52.253 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079851499999999997 2024-09-18T11:30:52.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.253 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.253 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.253 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300806+0000", 2024-09-18T11:30:52.253 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7096999999999999e-05 2024-09-18T11:30:52.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.253 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.253 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.253 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300813+0000", 2024-09-18T11:30:52.253 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5079999999999999e-06 2024-09-18T11:30:52.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.254 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.254 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300870+0000", 2024-09-18T11:30:52.254 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7178e-05 2024-09-18T11:30:52.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.254 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.254 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300897+0000", 2024-09-18T11:30:52.254 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6284000000000001e-05 2024-09-18T11:30:52.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.254 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.254 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306410+0000", 2024-09-18T11:30:52.254 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055135210000000004 2024-09-18T11:30:52.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.255 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.255 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306451+0000", 2024-09-18T11:30:52.255 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0379000000000001e-05 2024-09-18T11:30:52.255 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.255 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.255 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.255 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.255 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.255 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:52.257 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:30:52.258 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:52.255+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:52.259 INFO:tasks.rados.rados.0.smithi119.stdout:1026: finishing write tid 1 to smithi11939506-2 2024-09-18T11:30:52.259 INFO:tasks.rados.rados.0.smithi119.stdout:1026: finishing write tid 2 to smithi11939506-2 2024-09-18T11:30:52.259 INFO:tasks.rados.rados.0.smithi119.stdout:1026: finishing write tid 3 to smithi11939506-2 2024-09-18T11:30:52.259 INFO:tasks.rados.rados.0.smithi119.stdout:1026: finishing write tid 4 to smithi11939506-2 2024-09-18T11:30:52.259 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 135 (ObjNum 163 snap 20 seq_num 163) dirty exists 2024-09-18T11:30:52.259 INFO:tasks.rados.rados.0.smithi119.stdout:1026: left oid 2 (ObjNum 163 snap 20 seq_num 163) 2024-09-18T11:30:52.259 INFO:tasks.rados.rados.0.smithi119.stdout:1027: finishing set_chunk to oid 4 2024-09-18T11:30:52.259 INFO:tasks.rados.rados.0.smithi119.stdout:1028: finishing write tid 1 to smithi11939506-5 2024-09-18T11:30:52.259 INFO:tasks.rados.rados.0.smithi119.stdout:1023: done (6 left) 2024-09-18T11:30:52.259 INFO:tasks.rados.rados.0.smithi119.stdout:1024: done (5 left) 2024-09-18T11:30:52.260 INFO:tasks.rados.rados.0.smithi119.stdout:1025: done (4 left) 2024-09-18T11:30:52.260 INFO:tasks.rados.rados.0.smithi119.stdout:1026: done (3 left) 2024-09-18T11:30:52.260 INFO:tasks.rados.rados.0.smithi119.stdout:1027: done (2 left) 2024-09-18T11:30:52.260 INFO:tasks.rados.rados.0.smithi119.stdout:1029: done (1 left) 2024-09-18T11:30:52.260 INFO:tasks.rados.rados.0.smithi119.stdout:1030: read oid 2 snap -1 2024-09-18T11:30:52.260 INFO:tasks.rados.rados.0.smithi119.stdout:1030: expect (ObjNum 163 snap 20 seq_num 163) 2024-09-18T11:30:52.260 INFO:tasks.rados.rados.0.smithi119.stdout:1031: read oid 11 snap -1 2024-09-18T11:30:52.260 INFO:tasks.rados.rados.0.smithi119.stdout:1031: expect (ObjNum 148 snap 18 seq_num 148) 2024-09-18T11:30:52.260 INFO:tasks.rados.rados.0.smithi119.stdout:1032: read oid 26 snap -1 2024-09-18T11:30:52.260 INFO:tasks.rados.rados.0.smithi119.stdout:1032: expect (ObjNum 147 snap 18 seq_num 147) 2024-09-18T11:30:52.260 INFO:tasks.rados.rados.0.smithi119.stdout:1033: read oid 20 snap -1 2024-09-18T11:30:52.260 INFO:tasks.rados.rados.0.smithi119.stdout:1033: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:30:52.260 INFO:tasks.rados.rados.0.smithi119.stdout:1034: set_chunk oid 18 target oid 2024-09-18T11:30:52.261 INFO:tasks.rados.rados.0.smithi119.stdout:1034: set_chunk oid 18 offset: 832000 length: 577536 target oid fe6aae3707ee6dfd44fc3a3fe0efa6b3e5364ad2f76a43a9eeb8b69a3c3bcab6 offset: 0 snap -1 2024-09-18T11:30:52.261 INFO:tasks.rados.rados.0.smithi119.stdout:1035: delete oid 21 current snap is 20 2024-09-18T11:30:52.261 INFO:tasks.rados.rados.0.smithi119.stdout:1028: finishing write tid 2 to smithi11939506-5 2024-09-18T11:30:52.261 INFO:tasks.rados.rados.0.smithi119.stdout:1028: finishing write tid 3 to smithi11939506-5 2024-09-18T11:30:52.261 INFO:tasks.rados.rados.0.smithi119.stdout:1028: finishing write tid 4 to smithi11939506-5 2024-09-18T11:30:52.261 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 75 (ObjNum 164 snap 20 seq_num 164) dirty exists 2024-09-18T11:30:52.261 INFO:tasks.rados.rados.0.smithi119.stdout:1028: left oid 5 (ObjNum 164 snap 20 seq_num 164) 2024-09-18T11:30:52.261 INFO:tasks.rados.rados.0.smithi119.stdout:1030: expect (ObjNum 163 snap 20 seq_num 163) 2024-09-18T11:30:52.357 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:52.356+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:52.366 INFO:teuthology.orchestra.run.smithi119.stderr:Exporting 3.1 info 3.1( v 62'101 (0'0,62'101] local-lis/les=21/22 n=25 ec=21/21 lis/c=21/21 les/c/f=22/22/0 sis=21) 2024-09-18T11:30:52.372 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:822c82fd:::16c9af8752ce95120a507ec000947859dae1395d7d6d118e597b9892dcd8465e:head# 2024-09-18T11:30:52.373 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:83e57124:::3697192fc171029dd257b30a7532fa2228ae20837ed34e739f2a2edcc34a61c7:head# 2024-09-18T11:30:52.373 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:84219092:::smithi11939506-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head# 2024-09-18T11:30:52.377 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:854f581f:::9fb14befb08f54939f2c71c890ba0ea89332fb5c1fd9e0b07290d878c4431425:head# 2024-09-18T11:30:52.377 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:8613c1ed:::2d6b732d315f6106b1a30740b78802c0014a2c0700a0c6eb7d53c5b00ebff73b:head# 2024-09-18T11:30:52.380 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:8abdb616:::smithi11939506-46 low_tier:head# 2024-09-18T11:30:52.383 INFO:tasks.rados.rados.0.smithi119.stdout:1031: expect (ObjNum 148 snap 18 seq_num 148) 2024-09-18T11:30:52.384 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:8b69ac22:::1cf2f7c6a3cb930317737d5f25c48971fd9e8dcf6ceeadb55a7d6ea9967ce068:head# 2024-09-18T11:30:52.384 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:8d296107:::smithi11939506-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head# 2024-09-18T11:30:52.388 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:9201bbaf:::ac47b8623647b4cd1851128a75d93065bd2a2122c8cd6774e7ae0a9937ae73a7:head# 2024-09-18T11:30:52.390 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:94708c4c:::888f815178b2a748104fffa559dcb609f143f519d8737c6aa160c9cce13a6441:head# 2024-09-18T11:30:52.390 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:9668061d:::smithi11939506-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head# 2024-09-18T11:30:52.393 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:97025964:::d85d35018f93ce704e0c86e65d6c262758d2b38883b05fc1a96898c5a961f23a:head# 2024-09-18T11:30:52.396 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:99e1d655:::d01ff5a07a68e5351ca985a6c57ddc5ef0d05d539368179784aa67dbeae8d271:head# 2024-09-18T11:30:52.397 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:52.398 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:52.398 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:52.398 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:52.398 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:9f1cb8cf:::smithi11939506-38 low_tier:head# 2024-09-18T11:30:52.402 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:a0497df5:::smithi11939506-14 low_tier:head# 2024-09-18T11:30:52.406 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:30:52.407 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:a17fb6fd:::be0721f3a30df04624eb43dc58fbbc00d7ea210014325fb73bbc8d75e46bcb7d:head# 2024-09-18T11:30:52.410 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:a3a1203d:::c2b3fc9c1ba640cc8bcddb60552d5559ebeb3b68ed444eb13795808d4ae6858d:head# 2024-09-18T11:30:52.412 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:a3da18a4:::smithi11939506-28 low_tier:head# 2024-09-18T11:30:52.415 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:a8c4fc97:::45274de591d4198ebdf7025d46b87dfab949a0114f7ca760f95c9bdc5f783864:head# 2024-09-18T11:30:52.418 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:abad5243:::554b36e1a8dfdc0f27ed2d1e3c014381bf16adc7985c58ddbe8e14169aa70c76:head# 2024-09-18T11:30:52.419 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:b03ed61d:::974d9437087c20cb4e3ff5922dab38768579714a2c9ce6904cfb47136071b991:head# 2024-09-18T11:30:52.419 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:b10e849c:::a705ccfdb1cdc6ef082e4405c7b5f3c792a2b27cdec018193178c38e5ec259a4:head# 2024-09-18T11:30:52.421 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:b6a90588:::smithi11939506-2 low_tier:head# 2024-09-18T11:30:52.423 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:b900d23f:::smithi11939506-24 low_tier:head# 2024-09-18T11:30:52.427 INFO:teuthology.orchestra.run.smithi119.stderr:Read #3:bf804683:::e58794d0a579007c3bd048992c5646e06b9775f9667464eee6a2a0508e23a0a6:head# 2024-09-18T11:30:52.428 INFO:teuthology.orchestra.run.smithi119.stderr:Export successful 2024-09-18T11:30:52.438 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:52.437+0000 7f6bf3052640 -1 osd.5 69 handle_osd_map injecting map crc failure 2024-09-18T11:30:52.457 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:52.455+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:52.467 INFO:tasks.rados.rados.0.smithi119.stdout:1033: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:30:52.536 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:52.536 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:52.536 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:52.536 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:52.536 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:52.544 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:30:52.589 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:52.588+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:52.645 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:52.645 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:52.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:52.645 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:52.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.645 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:52.645 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:52.645 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 41.526164938000001, 2024-09-18T11:30:52.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:30:52.645 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.645 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:52.646 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.646 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.646 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.646 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:30:52.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.646 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.646 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:52.646 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:52.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:30:52.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:52.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:30:52.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:30:52.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:30:52.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:30:52.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:30:52.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:30:52.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.649 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:30:52.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:30:52.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.649 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:52.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:30:52.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:30:52.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.649 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:30:52.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:30:52.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.650 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.650 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:30:52.650 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:30:52.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.650 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.650 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:30:52.650 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:30:52.650 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.650 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.650 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.651 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1132 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2605052,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:52.651 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179179+0000", 2024-09-18T11:30:52.651 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.463304934, 2024-09-18T11:30:52.651 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059816840000000001, 2024-09-18T11:30:52.651 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.651 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:52.651 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.651 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.651 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.651 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1132 2024-09-18T11:30:52.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.651 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179179+0000", 2024-09-18T11:30:52.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179179+0000", 2024-09-18T11:30:52.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179178+0000", 2024-09-18T11:30:52.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:52.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179193+0000", 2024-09-18T11:30:52.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5446000000000001e-05 2024-09-18T11:30:52.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179215+0000", 2024-09-18T11:30:52.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1253000000000001e-05 2024-09-18T11:30:52.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179227+0000", 2024-09-18T11:30:52.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2612999999999999e-05 2024-09-18T11:30:52.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179447+0000", 2024-09-18T11:30:52.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021972199999999999 2024-09-18T11:30:52.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:52.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179453+0000", 2024-09-18T11:30:52.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8830000000000003e-06 2024-09-18T11:30:52.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184113+0000", 2024-09-18T11:30:52.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046603180000000001 2024-09-18T11:30:52.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184229+0000", 2024-09-18T11:30:52.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000115516 2024-09-18T11:30:52.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185161+0000", 2024-09-18T11:30:52.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00093216000000000002 2024-09-18T11:30:52.655 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.655 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.656 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.656 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:52.656 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:30:52.656 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.973139537000002, 2024-09-18T11:30:52.656 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:30:52.656 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.656 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:52.656 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.656 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.656 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.656 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:30:52.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.657 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.657 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:30:52.657 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.657 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:30:52.657 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.657 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:30:52.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:52.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:30:52.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:30:52.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:30:52.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:30:52.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:30:52.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:30:52.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:30:52.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:30:52.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:30:52.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:30:52.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:30:52.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:30:52.660 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.660 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.660 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.660 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1185 4.5 4:af4ba75a:::smithi11939506-37:head [write 729437~527542] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:52.660 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.156648+0000", 2024-09-18T11:30:52.660 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.485835612999999, 2024-09-18T11:30:52.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057485540000000003, 2024-09-18T11:30:52.661 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.661 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.661 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.661 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.661 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.661 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1185 2024-09-18T11:30:52.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.661 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.156648+0000", 2024-09-18T11:30:52.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.662 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.662 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.156648+0000", 2024-09-18T11:30:52.662 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.662 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.662 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.156647+0000", 2024-09-18T11:30:52.662 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:52.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.663 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.663 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157166+0000", 2024-09-18T11:30:52.663 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00051975999999999997 2024-09-18T11:30:52.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.663 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.663 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.663 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157183+0000", 2024-09-18T11:30:52.663 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.611e-05 2024-09-18T11:30:52.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.663 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.663 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.663 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157188+0000", 2024-09-18T11:30:52.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.648e-06 2024-09-18T11:30:52.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157253+0000", 2024-09-18T11:30:52.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4503000000000002e-05 2024-09-18T11:30:52.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157322+0000", 2024-09-18T11:30:52.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9031999999999999e-05 2024-09-18T11:30:52.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.665 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:52.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157437+0000", 2024-09-18T11:30:52.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011543400000000001 2024-09-18T11:30:52.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.665 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:52.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162104+0000", 2024-09-18T11:30:52.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046668009999999999 2024-09-18T11:30:52.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.665 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:52.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162321+0000", 2024-09-18T11:30:52.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021706699999999999 2024-09-18T11:30:52.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162355+0000", 2024-09-18T11:30:52.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3670000000000001e-05 2024-09-18T11:30:52.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162397+0000", 2024-09-18T11:30:52.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2168e-05 2024-09-18T11:30:52.666 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.666 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.667 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.667 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.667 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1688 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 19=[19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-18T11:30:52.667 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:30:52.667 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.109984944000001, 2024-09-18T11:30:52.667 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071575789999999999, 2024-09-18T11:30:52.667 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.667 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.667 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.667 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.667 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.667 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1688 2024-09-18T11:30:52.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.668 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.668 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:30:52.668 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.668 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:30:52.668 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.668 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.669 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:30:52.669 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:52.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.669 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.669 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.669 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532500+0000", 2024-09-18T11:30:52.669 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5260000000000001e-06 2024-09-18T11:30:52.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.669 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.669 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.669 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532503+0000", 2024-09-18T11:30:52.669 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:30:52.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.670 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532505+0000", 2024-09-18T11:30:52.670 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.889e-06 2024-09-18T11:30:52.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.670 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.534722+0000", 2024-09-18T11:30:52.670 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002217119 2024-09-18T11:30:52.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.670 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.534827+0000", 2024-09-18T11:30:52.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0001049 2024-09-18T11:30:52.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.671 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:52.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.535007+0000", 2024-09-18T11:30:52.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017965599999999999 2024-09-18T11:30:52.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.671 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:52.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539463+0000", 2024-09-18T11:30:52.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004455713 2024-09-18T11:30:52.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:52.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539607+0000", 2024-09-18T11:30:52.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014440300000000001 2024-09-18T11:30:52.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539634+0000", 2024-09-18T11:30:52.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7398000000000001e-05 2024-09-18T11:30:52.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539657+0000", 2024-09-18T11:30:52.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2228000000000001e-05 2024-09-18T11:30:52.673 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.673 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.673 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.673 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.673 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1759 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2319366,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:52.673 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:30:52.673 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 10.348524734, 2024-09-18T11:30:52.673 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064323039999999998, 2024-09-18T11:30:52.673 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.673 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:52.673 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.674 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.674 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.674 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1759 2024-09-18T11:30:52.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.674 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.674 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.674 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:30:52.674 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:30:52.675 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:30:52.675 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293963+0000", 2024-09-18T11:30:52.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0810000000000004e-06 2024-09-18T11:30:52.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.676 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293970+0000", 2024-09-18T11:30:52.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5109999999999997e-06 2024-09-18T11:30:52.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.676 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293975+0000", 2024-09-18T11:30:52.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.33e-06 2024-09-18T11:30:52.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.677 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.677 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294078+0000", 2024-09-18T11:30:52.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102871 2024-09-18T11:30:52.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.677 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:52.677 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294084+0000", 2024-09-18T11:30:52.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5470000000000002e-06 2024-09-18T11:30:52.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.677 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.299550+0000", 2024-09-18T11:30:52.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054660799999999999 2024-09-18T11:30:52.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.678 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.299608+0000", 2024-09-18T11:30:52.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8307000000000003e-05 2024-09-18T11:30:52.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.678 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300392+0000", 2024-09-18T11:30:52.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078306999999999997 2024-09-18T11:30:52.678 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.679 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.679 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.679 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1760 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2319366,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:52.679 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:30:52.679 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 10.348490394000001, 2024-09-18T11:30:52.679 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070132529999999997, 2024-09-18T11:30:52.679 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.679 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:52.679 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.679 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.679 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.679 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1760 2024-09-18T11:30:52.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.680 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.680 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.680 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:30:52.680 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.680 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.680 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:30:52.680 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.680 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293993+0000", 2024-09-18T11:30:52.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.681 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293997+0000", 2024-09-18T11:30:52.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4999999999999999e-06 2024-09-18T11:30:52.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.681 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294004+0000", 2024-09-18T11:30:52.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3950000000000003e-06 2024-09-18T11:30:52.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.682 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294009+0000", 2024-09-18T11:30:52.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4989999999999996e-06 2024-09-18T11:30:52.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.682 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294103+0000", 2024-09-18T11:30:52.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4817999999999999e-05 2024-09-18T11:30:52.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.682 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:52.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294110+0000", 2024-09-18T11:30:52.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7240000000000003e-06 2024-09-18T11:30:52.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.683 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.683 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300406+0000", 2024-09-18T11:30:52.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062964470000000002 2024-09-18T11:30:52.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.683 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.683 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300457+0000", 2024-09-18T11:30:52.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0915000000000003e-05 2024-09-18T11:30:52.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.684 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.684 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.301007+0000", 2024-09-18T11:30:52.684 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00054942299999999995 2024-09-18T11:30:52.684 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.684 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.684 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.684 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.684 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1775 4.a 4:588debc2:::smithi11939506-18:head [write 1527054~766050] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:52.684 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:30:52.684 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.7461376529999999, 2024-09-18T11:30:52.684 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070939050000000002, 2024-09-18T11:30:52.684 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.685 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.685 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.685 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.685 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.685 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1775 2024-09-18T11:30:52.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.685 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.685 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.685 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.685 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:30:52.685 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.685 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:30:52.686 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896345+0000", 2024-09-18T11:30:52.686 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:52.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896993+0000", 2024-09-18T11:30:52.686 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064769999999999997 2024-09-18T11:30:52.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.687 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.687 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897011+0000", 2024-09-18T11:30:52.687 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8355999999999998e-05 2024-09-18T11:30:52.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.687 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.687 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897016+0000", 2024-09-18T11:30:52.687 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3890000000000004e-06 2024-09-18T11:30:52.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.687 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.687 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897054+0000", 2024-09-18T11:30:52.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7963000000000002e-05 2024-09-18T11:30:52.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.688 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897123+0000", 2024-09-18T11:30:52.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8128999999999998e-05 2024-09-18T11:30:52.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.688 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:52.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897224+0000", 2024-09-18T11:30:52.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010168800000000001 2024-09-18T11:30:52.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.689 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:52.689 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899515+0000", 2024-09-18T11:30:52.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022910280000000001 2024-09-18T11:30:52.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.689 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:52.689 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903333+0000", 2024-09-18T11:30:52.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003817459 2024-09-18T11:30:52.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.689 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.689 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903388+0000", 2024-09-18T11:30:52.690 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5651e-05 2024-09-18T11:30:52.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.690 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.690 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.690 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903440+0000", 2024-09-18T11:30:52.690 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1836999999999999e-05 2024-09-18T11:30:52.690 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.690 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.690 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.690 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.690 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1776 4.a 4:588debc2:::smithi11939506-18:head [setxattr _header (57) in=7b,truncate 2420208] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:52.690 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:30:52.691 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.7454598410000006, 2024-09-18T11:30:52.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065840400000000002, 2024-09-18T11:30:52.691 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.691 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.691 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.691 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.691 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.691 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1776 2024-09-18T11:30:52.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.691 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.691 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:30:52.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:30:52.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:30:52.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.693 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897026+0000", 2024-09-18T11:30:52.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9579999999999999e-06 2024-09-18T11:30:52.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.693 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897030+0000", 2024-09-18T11:30:52.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5370000000000002e-06 2024-09-18T11:30:52.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.693 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.694 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897032+0000", 2024-09-18T11:30:52.694 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.17e-06 2024-09-18T11:30:52.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.694 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.694 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898415+0000", 2024-09-18T11:30:52.694 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013823189999999999 2024-09-18T11:30:52.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.694 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.694 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898527+0000", 2024-09-18T11:30:52.694 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011236800000000001 2024-09-18T11:30:52.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:52.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898714+0000", 2024-09-18T11:30:52.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018703599999999999 2024-09-18T11:30:52.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:52.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899683+0000", 2024-09-18T11:30:52.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00096944700000000002 2024-09-18T11:30:52.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:52.696 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903475+0000", 2024-09-18T11:30:52.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037920129999999999 2024-09-18T11:30:52.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.696 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.696 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.696 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903535+0000", 2024-09-18T11:30:52.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9843999999999998e-05 2024-09-18T11:30:52.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.697 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.697 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903608+0000", 2024-09-18T11:30:52.697 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2946000000000003e-05 2024-09-18T11:30:52.697 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.697 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.697 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.697 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1777 4.a 4:588debc2:::smithi11939506-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:52.697 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:30:52.697 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.7454332010000009, 2024-09-18T11:30:52.698 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0069010809999999999, 2024-09-18T11:30:52.698 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.698 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:52.698 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.698 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.698 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.698 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1777 2024-09-18T11:30:52.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.698 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.698 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.698 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.698 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:30:52.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.699 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.699 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:30:52.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.699 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.699 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897050+0000", 2024-09-18T11:30:52.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.699 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.699 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.700 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897052+0000", 2024-09-18T11:30:52.700 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:30:52.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.700 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.700 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897057+0000", 2024-09-18T11:30:52.700 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9470000000000002e-06 2024-09-18T11:30:52.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.700 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.700 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897060+0000", 2024-09-18T11:30:52.700 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3410000000000001e-06 2024-09-18T11:30:52.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.701 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899417+0000", 2024-09-18T11:30:52.701 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023571170000000002 2024-09-18T11:30:52.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.701 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:52.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899501+0000", 2024-09-18T11:30:52.701 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4160999999999996e-05 2024-09-18T11:30:52.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.701 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903661+0000", 2024-09-18T11:30:52.701 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041599510000000003 2024-09-18T11:30:52.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.702 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.702 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.702 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903790+0000", 2024-09-18T11:30:52.702 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012941499999999999 2024-09-18T11:30:52.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.702 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.702 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.702 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903952+0000", 2024-09-18T11:30:52.702 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016146800000000001 2024-09-18T11:30:52.702 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.702 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.702 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.703 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.703 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1815 4.5 4:aa575018:::smithi11939506-11:head [write 1760982~657035] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:52.703 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.948313+0000", 2024-09-18T11:30:52.703 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.6941709490000001, 2024-09-18T11:30:52.703 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056498440000000002, 2024-09-18T11:30:52.703 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.703 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.703 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.703 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.703 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.703 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1815 2024-09-18T11:30:52.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.703 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.703 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.703 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.704 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.948313+0000", 2024-09-18T11:30:52.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.704 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.704 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.704 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.948313+0000", 2024-09-18T11:30:52.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.704 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.704 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.704 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.948301+0000", 2024-09-18T11:30:52.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999881 2024-09-18T11:30:52.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.705 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.705 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949192+0000", 2024-09-18T11:30:52.705 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089072299999999995 2024-09-18T11:30:52.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.705 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.705 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949207+0000", 2024-09-18T11:30:52.705 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4705999999999999e-05 2024-09-18T11:30:52.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.705 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.705 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949211+0000", 2024-09-18T11:30:52.705 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5020000000000003e-06 2024-09-18T11:30:52.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949274+0000", 2024-09-18T11:30:52.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2951000000000001e-05 2024-09-18T11:30:52.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949331+0000", 2024-09-18T11:30:52.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7343999999999998e-05 2024-09-18T11:30:52.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:52.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.949429+0000", 2024-09-18T11:30:52.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7899999999999994e-05 2024-09-18T11:30:52.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:52.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.953537+0000", 2024-09-18T11:30:52.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004108006 2024-09-18T11:30:52.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:52.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.953885+0000", 2024-09-18T11:30:52.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00034758 2024-09-18T11:30:52.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.708 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.708 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.953932+0000", 2024-09-18T11:30:52.708 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7345000000000002e-05 2024-09-18T11:30:52.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.708 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.708 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.708 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.953963+0000", 2024-09-18T11:30:52.708 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0716999999999997e-05 2024-09-18T11:30:52.708 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.708 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.708 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.709 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1895 4.5 4:a087a566:::smithi11939506-46:head [read 0~2756315,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:30:52.709 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:30:52.709 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.3341776960000002, 2024-09-18T11:30:52.709 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007540377, 2024-09-18T11:30:52.709 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.709 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:52.709 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.709 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.709 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.709 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1895 2024-09-18T11:30:52.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.709 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.710 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.710 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:30:52.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.710 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.710 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:30:52.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.710 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.710 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:30:52.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308310+0000", 2024-09-18T11:30:52.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0559999999999998e-06 2024-09-18T11:30:52.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308317+0000", 2024-09-18T11:30:52.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7209999999999997e-06 2024-09-18T11:30:52.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308320+0000", 2024-09-18T11:30:52.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.721e-06 2024-09-18T11:30:52.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.712 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308447+0000", 2024-09-18T11:30:52.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012645199999999999 2024-09-18T11:30:52.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.712 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:52.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308451+0000", 2024-09-18T11:30:52.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6800000000000001e-06 2024-09-18T11:30:52.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.312462+0000", 2024-09-18T11:30:52.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040102139999999998 2024-09-18T11:30:52.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.312585+0000", 2024-09-18T11:30:52.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012388899999999999 2024-09-18T11:30:52.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.315847+0000", 2024-09-18T11:30:52.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003261197 2024-09-18T11:30:52.714 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.714 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.714 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.714 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1924 4.5 4:af4ba75a:::smithi11939506-37:head [write 1829449~675380] snapc 1e=[1e,1c,1b,19,18,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:30:52.714 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.362228+0000", 2024-09-18T11:30:52.714 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.2802559970000003, 2024-09-18T11:30:52.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066182139999999999, 2024-09-18T11:30:52.714 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.714 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.714 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.714 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.714 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.715 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1924 2024-09-18T11:30:52.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.715 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.715 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362228+0000", 2024-09-18T11:30:52.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.715 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362228+0000", 2024-09-18T11:30:52.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.716 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.716 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362227+0000", 2024-09-18T11:30:52.716 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:52.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.716 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.716 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362749+0000", 2024-09-18T11:30:52.716 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00052262399999999996 2024-09-18T11:30:52.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.716 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.716 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362762+0000", 2024-09-18T11:30:52.716 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2821e-05 2024-09-18T11:30:52.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.717 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362767+0000", 2024-09-18T11:30:52.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5349999999999998e-06 2024-09-18T11:30:52.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.717 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362806+0000", 2024-09-18T11:30:52.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9029999999999997e-05 2024-09-18T11:30:52.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.717 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362863+0000", 2024-09-18T11:30:52.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7584999999999998e-05 2024-09-18T11:30:52.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:52.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362949+0000", 2024-09-18T11:30:52.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5655000000000006e-05 2024-09-18T11:30:52.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:52.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364314+0000", 2024-09-18T11:30:52.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001364961 2024-09-18T11:30:52.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:52.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.368771+0000", 2024-09-18T11:30:52.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004457244 2024-09-18T11:30:52.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.368809+0000", 2024-09-18T11:30:52.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8368999999999998e-05 2024-09-18T11:30:52.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.368846+0000", 2024-09-18T11:30:52.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6718000000000003e-05 2024-09-18T11:30:52.720 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.720 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.720 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.720 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.720 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.720 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1925 4.5 4:af4ba75a:::smithi11939506-37:head [setxattr _header (57) in=7b,truncate 2655506] snapc 1e=[1e,1c,1b,19,18,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:30:52.720 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:30:52.720 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.2797108450000003, 2024-09-18T11:30:52.720 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064745779999999999, 2024-09-18T11:30:52.720 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.720 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.720 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.720 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.721 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.721 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1925 2024-09-18T11:30:52.721 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.721 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.721 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.721 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.721 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:30:52.721 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.721 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.721 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.721 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.721 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:30:52.721 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.722 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.722 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.722 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.722 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:30:52.722 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.722 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.722 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.722 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.722 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362776+0000", 2024-09-18T11:30:52.722 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.208e-06 2024-09-18T11:30:52.722 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.722 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.722 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.722 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362780+0000", 2024-09-18T11:30:52.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.771e-06 2024-09-18T11:30:52.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.723 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.723 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362782+0000", 2024-09-18T11:30:52.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.277e-06 2024-09-18T11:30:52.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.723 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.723 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.363737+0000", 2024-09-18T11:30:52.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00095483999999999999 2024-09-18T11:30:52.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.363832+0000", 2024-09-18T11:30:52.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5594999999999995e-05 2024-09-18T11:30:52.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:52.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.363940+0000", 2024-09-18T11:30:52.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010738000000000001 2024-09-18T11:30:52.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:52.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364577+0000", 2024-09-18T11:30:52.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00063679600000000002 2024-09-18T11:30:52.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:52.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369126+0000", 2024-09-18T11:30:52.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0045496440000000003 2024-09-18T11:30:52.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369177+0000", 2024-09-18T11:30:52.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0958000000000002e-05 2024-09-18T11:30:52.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.726 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369248+0000", 2024-09-18T11:30:52.726 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0599000000000006e-05 2024-09-18T11:30:52.726 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.726 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.726 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.726 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1926 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:30:52.726 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:30:52.726 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.2796865889999998, 2024-09-18T11:30:52.726 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066190219999999996, 2024-09-18T11:30:52.726 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.726 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:52.727 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.727 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.727 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.727 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1926 2024-09-18T11:30:52.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.727 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.727 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.727 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:30:52.727 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.727 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:30:52.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.728 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:30:52.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.728 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362799+0000", 2024-09-18T11:30:52.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.029e-06 2024-09-18T11:30:52.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.729 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.729 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362804+0000", 2024-09-18T11:30:52.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3989999999999997e-06 2024-09-18T11:30:52.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.729 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.729 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362806+0000", 2024-09-18T11:30:52.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3020000000000002e-06 2024-09-18T11:30:52.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.729 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.729 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364105+0000", 2024-09-18T11:30:52.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001299478 2024-09-18T11:30:52.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.733 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:52.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364153+0000", 2024-09-18T11:30:52.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8059000000000002e-05 2024-09-18T11:30:52.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.734 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369273+0000", 2024-09-18T11:30:52.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051199540000000003 2024-09-18T11:30:52.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.734 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369343+0000", 2024-09-18T11:30:52.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0191999999999995e-05 2024-09-18T11:30:52.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.734 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369416+0000", 2024-09-18T11:30:52.735 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2925999999999999e-05 2024-09-18T11:30:52.735 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.735 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.735 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.735 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.735 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.735 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1964 4.a 4:5975a5ec:::smithi11939506-36:head [append 0~0] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:30:52.735 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373800+0000", 2024-09-18T11:30:52.735 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.268683682, 2024-09-18T11:30:52.735 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006226105, 2024-09-18T11:30:52.735 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.735 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.735 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.735 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.735 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.736 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1964 2024-09-18T11:30:52.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.736 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.736 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.736 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373800+0000", 2024-09-18T11:30:52.736 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.736 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.736 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373800+0000", 2024-09-18T11:30:52.736 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373799+0000", 2024-09-18T11:30:52.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:52.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.737 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373809+0000", 2024-09-18T11:30:52.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0777e-05 2024-09-18T11:30:52.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.737 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373824+0000", 2024-09-18T11:30:52.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4640999999999999e-05 2024-09-18T11:30:52.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373831+0000", 2024-09-18T11:30:52.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5899999999999996e-06 2024-09-18T11:30:52.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373888+0000", 2024-09-18T11:30:52.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7679000000000001e-05 2024-09-18T11:30:52.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373957+0000", 2024-09-18T11:30:52.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9006999999999998e-05 2024-09-18T11:30:52.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.739 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.739 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:52.739 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.374089+0000", 2024-09-18T11:30:52.739 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013161999999999999 2024-09-18T11:30:52.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.739 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.739 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:52.739 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.379939+0000", 2024-09-18T11:30:52.739 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058500080000000003 2024-09-18T11:30:52.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.739 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.739 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:52.739 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.379974+0000", 2024-09-18T11:30:52.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4764999999999997e-05 2024-09-18T11:30:52.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.740 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380002+0000", 2024-09-18T11:30:52.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8201999999999999e-05 2024-09-18T11:30:52.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.740 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380026+0000", 2024-09-18T11:30:52.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4477999999999998e-05 2024-09-18T11:30:52.740 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.740 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.740 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.741 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1965 4.a 4:5975a5ec:::smithi11939506-36:head [rollback 1e] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:30:52.741 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373840+0000", 2024-09-18T11:30:52.741 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.268644173, 2024-09-18T11:30:52.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062698229999999999, 2024-09-18T11:30:52.741 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.741 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.741 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.741 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.741 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.741 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1965 2024-09-18T11:30:52.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.741 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373840+0000", 2024-09-18T11:30:52.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373840+0000", 2024-09-18T11:30:52.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373839+0000", 2024-09-18T11:30:52.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373842+0000", 2024-09-18T11:30:52.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8770000000000002e-06 2024-09-18T11:30:52.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373849+0000", 2024-09-18T11:30:52.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3929999999999996e-06 2024-09-18T11:30:52.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.744 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373853+0000", 2024-09-18T11:30:52.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0099999999999997e-06 2024-09-18T11:30:52.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.744 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.744 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.375207+0000", 2024-09-18T11:30:52.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001354755 2024-09-18T11:30:52.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.744 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.744 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.375268+0000", 2024-09-18T11:30:52.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0983999999999998e-05 2024-09-18T11:30:52.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:52.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.375449+0000", 2024-09-18T11:30:52.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000181091 2024-09-18T11:30:52.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:52.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380058+0000", 2024-09-18T11:30:52.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046089390000000003 2024-09-18T11:30:52.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:52.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380065+0000", 2024-09-18T11:30:52.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4479999999999999e-06 2024-09-18T11:30:52.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.746 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380088+0000", 2024-09-18T11:30:52.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3337e-05 2024-09-18T11:30:52.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.746 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380110+0000", 2024-09-18T11:30:52.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.145e-05 2024-09-18T11:30:52.746 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.746 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.746 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.747 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1966 4.a 4:5975a5ec:::smithi11939506-36:head [append 0~0] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:30:52.747 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373876+0000", 2024-09-18T11:30:52.747 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.2686081490000001, 2024-09-18T11:30:52.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063095080000000001, 2024-09-18T11:30:52.747 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.747 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:52.747 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.747 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.747 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.747 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1966 2024-09-18T11:30:52.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.748 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373876+0000", 2024-09-18T11:30:52.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373876+0000", 2024-09-18T11:30:52.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.749 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373875+0000", 2024-09-18T11:30:52.749 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.749 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.749 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373879+0000", 2024-09-18T11:30:52.749 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3859999999999999e-06 2024-09-18T11:30:52.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.749 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.749 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373885+0000", 2024-09-18T11:30:52.749 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6030000000000001e-06 2024-09-18T11:30:52.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373889+0000", 2024-09-18T11:30:52.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9620000000000004e-06 2024-09-18T11:30:52.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.376620+0000", 2024-09-18T11:30:52.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027307920000000001 2024-09-18T11:30:52.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.376672+0000", 2024-09-18T11:30:52.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2244999999999999e-05 2024-09-18T11:30:52.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.751 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:52.751 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.376838+0000", 2024-09-18T11:30:52.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000165716 2024-09-18T11:30:52.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.751 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:52.751 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380116+0000", 2024-09-18T11:30:52.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003278352 2024-09-18T11:30:52.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.751 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:52.751 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380145+0000", 2024-09-18T11:30:52.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8379999999999999e-05 2024-09-18T11:30:52.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.751 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:52.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380166+0000", 2024-09-18T11:30:52.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1118e-05 2024-09-18T11:30:52.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380185+0000", 2024-09-18T11:30:52.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9519999999999999e-05 2024-09-18T11:30:52.752 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.752 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.752 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.752 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1968 4.a 4:588debc2:::smithi11939506-18:18 [read 0~3949612,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:30:52.752 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373915+0000", 2024-09-18T11:30:52.753 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.268568726, 2024-09-18T11:30:52.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060271159999999999, 2024-09-18T11:30:52.753 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.753 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:52.753 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.753 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.753 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.753 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1968 2024-09-18T11:30:52.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.753 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.753 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.753 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373915+0000", 2024-09-18T11:30:52.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.754 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373915+0000", 2024-09-18T11:30:52.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.754 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373915+0000", 2024-09-18T11:30:52.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.754 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373918+0000", 2024-09-18T11:30:52.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.619e-06 2024-09-18T11:30:52.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373925+0000", 2024-09-18T11:30:52.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3910000000000003e-06 2024-09-18T11:30:52.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373928+0000", 2024-09-18T11:30:52.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.478e-06 2024-09-18T11:30:52.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.378583+0000", 2024-09-18T11:30:52.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046544339999999998 2024-09-18T11:30:52.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.378640+0000", 2024-09-18T11:30:52.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7226999999999999e-05 2024-09-18T11:30:52.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.379942+0000", 2024-09-18T11:30:52.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001302465 2024-09-18T11:30:52.757 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.757 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.757 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.757 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1969 4.a 4:588debc2:::smithi11939506-18:18 [read 0~3949612,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:30:52.757 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:30:52.757 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.2684977, 2024-09-18T11:30:52.757 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075409190000000001, 2024-09-18T11:30:52.757 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:52.757 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:52.757 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:52.757 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:52.758 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:52.758 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1969 2024-09-18T11:30:52.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.758 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:52.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.758 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:52.758 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:30:52.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.758 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:52.758 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:30:52.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:52.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:52.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:30:52.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:52.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:52.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373990+0000", 2024-09-18T11:30:52.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1119999999999998e-06 2024-09-18T11:30:52.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:52.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373997+0000", 2024-09-18T11:30:52.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4200000000000001e-06 2024-09-18T11:30:52.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.760 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.760 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:52.760 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.374002+0000", 2024-09-18T11:30:52.760 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:30:52.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.760 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.760 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:52.760 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380200+0000", 2024-09-18T11:30:52.760 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006197302 2024-09-18T11:30:52.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.760 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.760 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:52.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380261+0000", 2024-09-18T11:30:52.761 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1389999999999993e-05 2024-09-18T11:30:52.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:52.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:52.761 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:52.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.381527+0000", 2024-09-18T11:30:52.761 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012660080000000001 2024-09-18T11:30:52.761 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.761 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.761 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.761 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:52.761 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:52.761 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:52.762 INFO:tasks.rados.rados.0.smithi119.stdout:1032: expect (ObjNum 147 snap 18 seq_num 147) 2024-09-18T11:30:52.763 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:30:52.764 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:52.761+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:52.849 INFO:tasks.rados.rados.0.smithi119.stdout:1034: finishing set_chunk to oid 18 2024-09-18T11:30:52.849 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 63 (ObjNum 144 snap 18 seq_num 144) dirty exists 2024-09-18T11:30:52.849 INFO:tasks.rados.rados.0.smithi119.stdout:1028: done (6 left) 2024-09-18T11:30:52.849 INFO:tasks.rados.rados.0.smithi119.stdout:1030: done (5 left) 2024-09-18T11:30:52.849 INFO:tasks.rados.rados.0.smithi119.stdout:1031: done (4 left) 2024-09-18T11:30:52.850 INFO:tasks.rados.rados.0.smithi119.stdout:1032: done (3 left) 2024-09-18T11:30:52.850 INFO:tasks.rados.rados.0.smithi119.stdout:1033: done (2 left) 2024-09-18T11:30:52.850 INFO:tasks.rados.rados.0.smithi119.stdout:1034: done (1 left) 2024-09-18T11:30:52.850 INFO:tasks.rados.rados.0.smithi119.stdout:1035: done (0 left) 2024-09-18T11:30:52.850 INFO:tasks.rados.rados.0.smithi119.stdout:1036: read oid 9 snap -1 2024-09-18T11:30:52.850 INFO:tasks.rados.rados.0.smithi119.stdout:1036: expect deleted 2024-09-18T11:30:52.850 INFO:tasks.rados.rados.0.smithi119.stdout:1037: write oid 26 current snap is 20 2024-09-18T11:30:52.850 INFO:tasks.rados.rados.0.smithi119.stdout:1037: seq_num 167 ranges {406769=609996,1439434=433714,2303522=503247} 2024-09-18T11:30:52.854 INFO:tasks.rados.rados.0.smithi119.stdout:1037: writing smithi11939506-26 from 406769 to 1016765 tid 1 2024-09-18T11:30:52.858 INFO:tasks.rados.rados.0.smithi119.stdout:1037: writing smithi11939506-26 from 1439434 to 1873148 tid 2 2024-09-18T11:30:52.863 INFO:tasks.rados.rados.0.smithi119.stdout:1037: writing smithi11939506-26 from 2303522 to 2806769 tid 3 2024-09-18T11:30:52.864 INFO:tasks.rados.rados.0.smithi119.stdout:1038: read oid 48 snap -1 2024-09-18T11:30:52.864 INFO:tasks.rados.rados.0.smithi119.stdout:1038: expect (ObjNum 47 snap 0 seq_num 47) 2024-09-18T11:30:52.864 INFO:tasks.rados.rados.0.smithi119.stdout:1039: delete oid 17 current snap is 20 2024-09-18T11:30:52.864 INFO:tasks.rados.rados.0.smithi119.stdout:1037: finishing write tid 1 to smithi11939506-26 2024-09-18T11:30:52.864 INFO:tasks.rados.rados.0.smithi119.stdout:1037: finishing write tid 2 to smithi11939506-26 2024-09-18T11:30:52.864 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:52.861+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:52.867 INFO:tasks.rados.rados.0.smithi119.stdout:1037: finishing write tid 3 to smithi11939506-26 2024-09-18T11:30:52.867 INFO:tasks.rados.rados.0.smithi119.stdout:1037: finishing write tid 4 to smithi11939506-26 2024-09-18T11:30:52.867 INFO:tasks.rados.rados.0.smithi119.stdout:1036: done (3 left) 2024-09-18T11:30:52.867 INFO:tasks.rados.rados.0.smithi119.stdout:1039: done (2 left) 2024-09-18T11:30:52.867 INFO:tasks.rados.rados.0.smithi119.stdout:1040: read oid 21 snap 12 2024-09-18T11:30:52.867 INFO:tasks.rados.rados.0.smithi119.stdout:1040: expect deleted 2024-09-18T11:30:52.867 INFO:tasks.rados.rados.0.smithi119.stdout:1041: set_chunk oid 46 target oid 2024-09-18T11:30:52.875 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:30:52.881 INFO:tasks.rados.rados.0.smithi119.stdout:1041: set_chunk oid 46 offset: 728064 length: 1263616 target oid dee72367d0ba12e168daefec0f1945a761042a8e9116a42c1e78d81d5bbf572e offset: 0 snap 17 2024-09-18T11:30:52.881 INFO:tasks.rados.rados.0.smithi119.stdout:1042: write oid 16 current snap is 20 2024-09-18T11:30:52.881 INFO:tasks.rados.rados.0.smithi119.stdout:1037: finishing write tid 5 to smithi11939506-26 2024-09-18T11:30:52.881 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 111 (ObjNum 167 snap 20 seq_num 167) dirty exists 2024-09-18T11:30:52.881 INFO:tasks.rados.rados.0.smithi119.stdout:1037: left oid 26 (ObjNum 167 snap 20 seq_num 167) 2024-09-18T11:30:52.881 INFO:tasks.rados.rados.0.smithi119.stdout:1042: seq_num 169 ranges {440383=476847,1556398=437383,2474721=365662} 2024-09-18T11:30:52.885 INFO:tasks.rados.rados.0.smithi119.stdout:1042: writing smithi11939506-16 from 440383 to 917230 tid 1 2024-09-18T11:30:52.891 INFO:tasks.rados.rados.0.smithi119.stdout:1042: writing smithi11939506-16 from 1556398 to 1993781 tid 2 2024-09-18T11:30:52.896 INFO:tasks.rados.rados.0.smithi119.stdout:1042: writing smithi11939506-16 from 2474721 to 2840383 tid 3 2024-09-18T11:30:52.896 INFO:tasks.rados.rados.0.smithi119.stdout:1037: done (4 left) 2024-09-18T11:30:52.896 INFO:tasks.rados.rados.0.smithi119.stdout:1043: tier_evict oid 26 2024-09-18T11:30:52.896 INFO:tasks.rados.rados.0.smithi119.stdout:1043: tier_evict oid 26 snap -1 2024-09-18T11:30:52.897 INFO:tasks.rados.rados.0.smithi119.stdout:1038: expect (ObjNum 47 snap 0 seq_num 47) 2024-09-18T11:30:52.907 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:52.907 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:52.907 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:52.907 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:52.915 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:30:52.953 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --force --op remove --pgid 3.1 2024-09-18T11:30:52.965 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:52.964+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:52.978 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:30:52.990 INFO:tasks.rados.rados.0.smithi119.stdout:1041: finishing set_chunk to oid 46 2024-09-18T11:30:52.990 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 46 set_chunk returned error code -95 offset: 728064 length: 1263616 2024-09-18T11:30:52.990 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 0 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-18T11:30:52.991 INFO:tasks.rados.rados.0.smithi119.stdout:1042: finishing write tid 1 to smithi11939506-16 2024-09-18T11:30:52.991 INFO:tasks.rados.rados.0.smithi119.stdout:1038: done (4 left) 2024-09-18T11:30:52.991 INFO:tasks.rados.rados.0.smithi119.stdout:1040: done (3 left) 2024-09-18T11:30:52.991 INFO:tasks.rados.rados.0.smithi119.stdout:1041: done (2 left) 2024-09-18T11:30:52.991 INFO:tasks.rados.rados.0.smithi119.stdout:1044: write (excl) oid 50 current snap is 20 2024-09-18T11:30:52.991 INFO:tasks.rados.rados.0.smithi119.stdout:1044: seq_num 170 ranges {457190=552096,1614880=581041,2785791=71399} 2024-09-18T11:30:52.994 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:52.994 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:52.996 INFO:tasks.rados.rados.0.smithi119.stdout:1044: writing smithi11939506-50 from 457190 to 1009286 tid 1 2024-09-18T11:30:52.998 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:52.996+0000 7fb6950ff640 -1 osd.2 70 handle_osd_map injecting map crc failure 2024-09-18T11:30:53.004 INFO:tasks.rados.rados.0.smithi119.stdout:1044: writing smithi11939506-50 from 1614880 to 2195921 tid 2 2024-09-18T11:30:53.009 INFO:tasks.rados.rados.0.smithi119.stdout:1044: writing smithi11939506-50 from 2785791 to 2857190 tid 3 2024-09-18T11:30:53.009 INFO:tasks.rados.rados.0.smithi119.stdout:1045: rollback oid 42 current snap is 20 2024-09-18T11:30:53.009 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 42 to 15 2024-09-18T11:30:53.009 INFO:tasks.rados.rados.0.smithi119.stdout:1042: finishing write tid 2 to smithi11939506-16 2024-09-18T11:30:53.009 INFO:tasks.rados.rados.0.smithi119.stdout:1043: got (22) Invalid argument 2024-09-18T11:30:53.009 INFO:tasks.rados.rados.0.smithi119.stdout:1042: finishing write tid 3 to smithi11939506-16 2024-09-18T11:30:53.009 INFO:tasks.rados.rados.0.smithi119.stdout:1042: finishing write tid 4 to smithi11939506-16 2024-09-18T11:30:53.009 INFO:tasks.rados.rados.0.smithi119.stdout:1042: finishing write tid 5 to smithi11939506-16 2024-09-18T11:30:53.009 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 100 (ObjNum 169 snap 20 seq_num 169) dirty exists 2024-09-18T11:30:53.010 INFO:tasks.rados.rados.0.smithi119.stdout:1042: left oid 16 (ObjNum 169 snap 20 seq_num 169) 2024-09-18T11:30:53.010 INFO:tasks.rados.rados.0.smithi119.stdout:1042: done (3 left) 2024-09-18T11:30:53.010 INFO:tasks.rados.rados.0.smithi119.stdout:1043: done (2 left) 2024-09-18T11:30:53.010 INFO:tasks.rados.rados.0.smithi119.stdout:1046: read oid 21 snap 17 2024-09-18T11:30:53.010 INFO:tasks.rados.rados.0.smithi119.stdout:1046: expect deleted 2024-09-18T11:30:53.010 INFO:tasks.rados.rados.0.smithi119.stdout:1047: write oid 46 current snap is 20 2024-09-18T11:30:53.010 INFO:tasks.rados.rados.0.smithi119.stdout:1047: seq_num 171 ranges {473997=627345,1673362=441052,2529567=344430} 2024-09-18T11:30:53.015 INFO:tasks.rados.rados.0.smithi119.stdout:1047: writing smithi11939506-46 from 473997 to 1101342 tid 1 2024-09-18T11:30:53.016 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:53.016 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:53.016 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:53.016 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:53.016 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:53.022 INFO:tasks.rados.rados.0.smithi119.stdout:1047: writing smithi11939506-46 from 1673362 to 2114414 tid 2 2024-09-18T11:30:53.022 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:53.023 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:53.027 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:30:53.028 INFO:tasks.rados.rados.0.smithi119.stdout:1047: writing smithi11939506-46 from 2529567 to 2873997 tid 3 2024-09-18T11:30:53.028 INFO:tasks.rados.rados.0.smithi119.stdout:1048: read oid 2 snap -1 2024-09-18T11:30:53.028 INFO:tasks.rados.rados.0.smithi119.stdout:1048: expect (ObjNum 163 snap 20 seq_num 163) 2024-09-18T11:30:53.028 INFO:tasks.rados.rados.0.smithi119.stdout:1049: read oid 38 snap -1 2024-09-18T11:30:53.028 INFO:tasks.rados.rados.0.smithi119.stdout:1049: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-18T11:30:53.028 INFO:tasks.rados.rados.0.smithi119.stdout:1050: snap_remove snap 10 2024-09-18T11:30:53.043 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:53.043 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:53.058 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:53.059 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:53.068 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:53.067+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:53.073 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:53.074 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:53.084 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:53.083+0000 7f6bf3052640 -1 osd.5 70 handle_osd_map injecting map crc failure 2024-09-18T11:30:53.089 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:53.090 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:53.105 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:53.105 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:53.108 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:30:53.108 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:30:53.166 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:53.165+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:53.167 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:53.167 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:53.175 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:30:53.267 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:53.266+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:53.274 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:53.274 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:53.283 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:30:53.365 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T11:30:53.369 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:53.367+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:53.400 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:53.400 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:53.414 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:30:53.498 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:53.496+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:53.554 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:53.555 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:53.563 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:30:53.604 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:53.602+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:53.672 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:53.670+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:53.701 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:53.701 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:53.709 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:30:53.793 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:53.791+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:53.853 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:53.853 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:53.862 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:30:53.873 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:53.872+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:53.974 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:53.973+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:54.000 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:53.998+0000 7fb6950ff640 -1 osd.2 71 handle_osd_map injecting map crc failure 2024-09-18T11:30:54.001 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:54.001 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:54.003 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is unset 2024-09-18T11:30:54.004 INFO:tasks.rados.rados.0.smithi119.stdout:1050: done (6 left) 2024-09-18T11:30:54.004 INFO:tasks.rados.rados.0.smithi119.stdout:1051: delete oid 40 current snap is 20 2024-09-18T11:30:54.004 INFO:tasks.rados.rados.0.smithi119.stdout:1044: finishing write tid 1 to smithi11939506-50 2024-09-18T11:30:54.004 INFO:tasks.rados.rados.0.smithi119.stdout:1044: finishing write tid 2 to smithi11939506-50 2024-09-18T11:30:54.004 INFO:tasks.rados.rados.0.smithi119.stdout:1044: finishing write tid 3 to smithi11939506-50 2024-09-18T11:30:54.004 INFO:tasks.rados.rados.0.smithi119.stdout:1044: finishing write tid 4 to smithi11939506-50 2024-09-18T11:30:54.004 INFO:tasks.rados.rados.0.smithi119.stdout:1044: finishing write tid 5 to smithi11939506-50 2024-09-18T11:30:54.004 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 79 (ObjNum 170 snap 20 seq_num 170) dirty exists 2024-09-18T11:30:54.004 INFO:tasks.rados.rados.0.smithi119.stdout:1044: left oid 50 (ObjNum 170 snap 20 seq_num 170) 2024-09-18T11:30:54.004 INFO:tasks.rados.rados.0.smithi119.stdout:1045: finishing rollback tid 0 to smithi11939506-42 2024-09-18T11:30:54.004 INFO:tasks.rados.rados.0.smithi119.stdout:1045: finishing rollback tid 1 to smithi11939506-42 2024-09-18T11:30:54.004 INFO:tasks.rados.rados.0.smithi119.stdout:1045: finishing rollback tid 2 to smithi11939506-42 2024-09-18T11:30:54.005 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 105 (ObjNum 99 snap 6 seq_num 99) dirty exists 2024-09-18T11:30:54.005 INFO:tasks.rados.rados.0.smithi119.stdout:1047: finishing write tid 1 to smithi11939506-46 2024-09-18T11:30:54.005 INFO:tasks.rados.rados.0.smithi119.stdout:1047: finishing write tid 2 to smithi11939506-46 2024-09-18T11:30:54.005 INFO:tasks.rados.rados.0.smithi119.stdout:1048: expect (ObjNum 163 snap 20 seq_num 163) 2024-09-18T11:30:54.015 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:30:54.075 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:54.074+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:54.111 INFO:teuthology.orchestra.run.smithi119.stdout: marking collection for removal 2024-09-18T11:30:54.111 INFO:teuthology.orchestra.run.smithi119.stdout:setting '_remove' omap key 2024-09-18T11:30:54.111 INFO:teuthology.orchestra.run.smithi119.stdout:finish_remove_pgs 3.1_head removing 3.1 2024-09-18T11:30:54.121 INFO:teuthology.orchestra.run.smithi119.stdout:Remove successful 2024-09-18T11:30:54.123 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:54.123 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:54.132 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:30:54.176 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:54.174+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:54.189 INFO:tasks.rados.rados.0.smithi119.stdout:1047: finishing write tid 3 to smithi11939506-46 2024-09-18T11:30:54.189 INFO:tasks.rados.rados.0.smithi119.stdout:1047: finishing write tid 4 to smithi11939506-46 2024-09-18T11:30:54.190 INFO:tasks.rados.rados.0.smithi119.stdout:1047: finishing write tid 5 to smithi11939506-46 2024-09-18T11:30:54.190 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 104 (ObjNum 171 snap 20 seq_num 171) dirty exists 2024-09-18T11:30:54.190 INFO:tasks.rados.rados.0.smithi119.stdout:1047: left oid 46 (ObjNum 171 snap 20 seq_num 171) 2024-09-18T11:30:54.190 INFO:tasks.rados.rados.0.smithi119.stdout:1049: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-18T11:30:54.231 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:54.231 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:54.240 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:30:54.277 INFO:tasks.rados.rados.0.smithi119.stdout:1044: done (6 left) 2024-09-18T11:30:54.277 INFO:tasks.rados.rados.0.smithi119.stdout:1045: done (5 left) 2024-09-18T11:30:54.277 INFO:tasks.rados.rados.0.smithi119.stdout:1046: done (4 left) 2024-09-18T11:30:54.277 INFO:tasks.rados.rados.0.smithi119.stdout:1047: done (3 left) 2024-09-18T11:30:54.277 INFO:tasks.rados.rados.0.smithi119.stdout:1048: done (2 left) 2024-09-18T11:30:54.277 INFO:tasks.rados.rados.0.smithi119.stdout:1049: done (1 left) 2024-09-18T11:30:54.277 INFO:tasks.rados.rados.0.smithi119.stdout:1051: done (0 left) 2024-09-18T11:30:54.277 INFO:tasks.rados.rados.0.smithi119.stdout:1052: snap_remove snap 8 2024-09-18T11:30:54.278 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:54.274+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:54.376 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:54.377 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:54.378 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:54.377+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:54.386 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:30:54.479 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:54.478+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:54.496 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:54.497 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:54.506 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:30:54.580 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:54.579+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:54.605 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:54.605 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:54.614 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:30:54.631 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.1.0 2024-09-18T11:30:54.712 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:54.712 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:54.716 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:54.714+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:54.721 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:30:54.782 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:54.781+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:54.818 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:54.818 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:54.826 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:30:54.883 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:54.881+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:54.963 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:54.963 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:54.971 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:30:54.983 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:54.981+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:55.009 INFO:tasks.rados.rados.0.smithi119.stdout:1052: done (0 left) 2024-09-18T11:30:55.009 INFO:tasks.rados.rados.0.smithi119.stdout:1053: delete oid 1 current snap is 20 2024-09-18T11:30:55.013 INFO:tasks.rados.rados.0.smithi119.stdout:1053: done (0 left) 2024-09-18T11:30:55.013 INFO:tasks.rados.rados.0.smithi119.stdout:1054: read oid 28 snap -1 2024-09-18T11:30:55.013 INFO:tasks.rados.rados.0.smithi119.stdout:1054: expect (ObjNum 122 snap 11 seq_num 122) 2024-09-18T11:30:55.013 INFO:tasks.rados.rados.0.smithi119.stdout:1055: set_chunk oid 43 target oid 2024-09-18T11:30:55.022 INFO:tasks.rados.rados.0.smithi119.stdout:1055: set_chunk oid 43 offset: 987648 length: 207360 target oid 4e932cf1932683e360c757d71fbacc2cc5f67bab4b0a5c1bd08568121339b4e3 offset: 0 snap -1 2024-09-18T11:30:55.022 INFO:tasks.rados.rados.0.smithi119.stdout:1056: read oid 18 snap -1 2024-09-18T11:30:55.022 INFO:tasks.rados.rados.0.smithi119.stdout:1056: expect (ObjNum 144 snap 18 seq_num 144) 2024-09-18T11:30:55.022 INFO:tasks.rados.rados.0.smithi119.stdout:1057: set_chunk oid 7 target oid 2024-09-18T11:30:55.036 INFO:tasks.rados.rados.0.smithi119.stdout:1057: set_chunk oid 7 offset: 1220096 length: 869376 target oid d39b9f4915792d73f89f55628b9b642f4ba57e20d3bd54920cfe090bd11c886f offset: 0 snap 19 2024-09-18T11:30:55.037 INFO:tasks.rados.rados.0.smithi119.stdout:1058: write (excl) oid 30 current snap is 20 2024-09-18T11:30:55.037 INFO:tasks.rados.rados.0.smithi119.stdout:1058: seq_num 174 ranges {524418=453092,1732455=588379,2779130=145288} 2024-09-18T11:30:55.042 INFO:tasks.rados.rados.0.smithi119.stdout:1058: writing smithi11939506-30 from 524418 to 977510 tid 1 2024-09-18T11:30:55.049 INFO:tasks.rados.rados.0.smithi119.stdout:1058: writing smithi11939506-30 from 1732455 to 2320834 tid 2 2024-09-18T11:30:55.053 INFO:tasks.rados.rados.0.smithi119.stdout:1058: writing smithi11939506-30 from 2779130 to 2924418 tid 3 2024-09-18T11:30:55.053 INFO:tasks.rados.rados.0.smithi119.stdout:1059: write (excl) oid 40 current snap is 20 2024-09-18T11:30:55.053 INFO:tasks.rados.rados.0.smithi119.stdout:1059: seq_num 175 ranges {541225=644694,1907290=448390} 2024-09-18T11:30:55.059 INFO:tasks.rados.rados.0.smithi119.stdout:1059: writing smithi11939506-40 from 541225 to 1185919 tid 1 2024-09-18T11:30:55.067 INFO:tasks.rados.rados.0.smithi119.stdout:1059: writing smithi11939506-40 from 1907290 to 2355680 tid 2 2024-09-18T11:30:55.067 INFO:tasks.rados.rados.0.smithi119.stdout:1060: tier_evict oid 14 2024-09-18T11:30:55.067 INFO:tasks.rados.rados.0.smithi119.stdout:1060: tier_evict oid 14 snap -1 2024-09-18T11:30:55.068 INFO:tasks.rados.rados.0.smithi119.stdout:1054: expect (ObjNum 122 snap 11 seq_num 122) 2024-09-18T11:30:55.080 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:55.080 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:55.084 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:55.083+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:55.089 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:30:55.108 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:30:55.151 INFO:tasks.rados.rados.0.smithi119.stdout:1055: finishing set_chunk to oid 43 2024-09-18T11:30:55.151 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 43 set_chunk returned error code -95 offset: 987648 length: 207360 2024-09-18T11:30:55.151 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-18T11:30:55.151 INFO:tasks.rados.rados.0.smithi119.stdout:1056: expect (ObjNum 144 snap 18 seq_num 144) 2024-09-18T11:30:55.185 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:55.183+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:55.209 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:30:55.224 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:55.224 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:55.228 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:55.229 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:55.237 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:30:55.240 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:55.240 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:55.254 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:55.254 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:55.270 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:55.270 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:55.285 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:55.285 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:55.286 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:55.284+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:55.300 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:55.300 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:55.301 INFO:tasks.rados.rados.0.smithi119.stdout:1057: finishing set_chunk to oid 7 2024-09-18T11:30:55.301 INFO:tasks.rados.rados.0.smithi119.stdout:1058: finishing write tid 1 to smithi11939506-30 2024-09-18T11:30:55.301 INFO:tasks.rados.rados.0.smithi119.stdout:1058: finishing write tid 2 to smithi11939506-30 2024-09-18T11:30:55.301 INFO:tasks.rados.rados.0.smithi119.stdout:1058: finishing write tid 3 to smithi11939506-30 2024-09-18T11:30:55.301 INFO:tasks.rados.rados.0.smithi119.stdout:1058: finishing write tid 4 to smithi11939506-30 2024-09-18T11:30:55.302 INFO:tasks.rados.rados.0.smithi119.stdout:1058: finishing write tid 5 to smithi11939506-30 2024-09-18T11:30:55.302 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 30 v 47 (ObjNum 174 snap 20 seq_num 174) dirty exists 2024-09-18T11:30:55.302 INFO:tasks.rados.rados.0.smithi119.stdout:1058: left oid 30 (ObjNum 174 snap 20 seq_num 174) 2024-09-18T11:30:55.302 INFO:tasks.rados.rados.0.smithi119.stdout:1054: done (6 left) 2024-09-18T11:30:55.302 INFO:tasks.rados.rados.0.smithi119.stdout:1055: done (5 left) 2024-09-18T11:30:55.302 INFO:tasks.rados.rados.0.smithi119.stdout:1056: done (4 left) 2024-09-18T11:30:55.302 INFO:tasks.rados.rados.0.smithi119.stdout:1057: done (3 left) 2024-09-18T11:30:55.302 INFO:tasks.rados.rados.0.smithi119.stdout:1058: done (2 left) 2024-09-18T11:30:55.302 INFO:tasks.rados.rados.0.smithi119.stdout:1061: write (excl) oid 47 current snap is 20 2024-09-18T11:30:55.302 INFO:tasks.rados.rados.0.smithi119.stdout:1061: seq_num 176 ranges {558032=719943,1849419=592048} 2024-09-18T11:30:55.307 INFO:tasks.rados.rados.0.smithi119.stdout:1061: writing smithi11939506-47 from 558032 to 1277975 tid 1 2024-09-18T11:30:55.314 INFO:tasks.rados.rados.0.smithi119.stdout:1061: writing smithi11939506-47 from 1849419 to 2441467 tid 2 2024-09-18T11:30:55.314 INFO:tasks.rados.rados.0.smithi119.stdout:1062: set_chunk oid 2 target oid 2024-09-18T11:30:55.315 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:55.315 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:55.319 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:30:55.319 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:30:55.329 INFO:tasks.rados.rados.0.smithi119.stdout:1062: set_chunk oid 2 offset: 251392 length: 2096640 target oid abf5d1c1c6ff48a895de7f094078ad489e0511876df30b6bd8853cd5ff4a6267 offset: 0 snap 12 2024-09-18T11:30:55.329 INFO:tasks.rados.rados.0.smithi119.stdout:1063: delete oid 25 current snap is 20 2024-09-18T11:30:55.329 INFO:tasks.rados.rados.0.smithi119.stdout:1059: finishing write tid 1 to smithi11939506-40 2024-09-18T11:30:55.329 INFO:tasks.rados.rados.0.smithi119.stdout:1060: got (22) Invalid argument 2024-09-18T11:30:55.329 INFO:tasks.rados.rados.0.smithi119.stdout:1059: finishing write tid 2 to smithi11939506-40 2024-09-18T11:30:55.329 INFO:tasks.rados.rados.0.smithi119.stdout:1059: finishing write tid 3 to smithi11939506-40 2024-09-18T11:30:55.330 INFO:tasks.rados.rados.0.smithi119.stdout:1059: finishing write tid 4 to smithi11939506-40 2024-09-18T11:30:55.330 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 90 (ObjNum 175 snap 20 seq_num 175) dirty exists 2024-09-18T11:30:55.330 INFO:tasks.rados.rados.0.smithi119.stdout:1059: left oid 40 (ObjNum 175 snap 20 seq_num 175) 2024-09-18T11:30:55.330 INFO:tasks.rados.rados.0.smithi119.stdout:1061: finishing write tid 1 to smithi11939506-47 2024-09-18T11:30:55.330 INFO:tasks.rados.rados.0.smithi119.stdout:1061: finishing write tid 2 to smithi11939506-47 2024-09-18T11:30:55.330 INFO:tasks.rados.rados.0.smithi119.stdout:1061: finishing write tid 3 to smithi11939506-47 2024-09-18T11:30:55.330 INFO:tasks.rados.rados.0.smithi119.stdout:1061: finishing write tid 4 to smithi11939506-47 2024-09-18T11:30:55.330 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 75 (ObjNum 176 snap 20 seq_num 176) dirty exists 2024-09-18T11:30:55.330 INFO:tasks.rados.rados.0.smithi119.stdout:1061: left oid 47 (ObjNum 176 snap 20 seq_num 176) 2024-09-18T11:30:55.330 INFO:tasks.rados.rados.0.smithi119.stdout:1062: finishing set_chunk to oid 2 2024-09-18T11:30:55.330 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 2 set_chunk returned error code -95 offset: 251392 length: 2096640 2024-09-18T11:30:55.330 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 0 (ObjNum 163 snap 20 seq_num 163) dirty exists 2024-09-18T11:30:55.331 INFO:tasks.rados.rados.0.smithi119.stdout:1059: done (4 left) 2024-09-18T11:30:55.332 INFO:tasks.rados.rados.0.smithi119.stdout:1060: done (3 left) 2024-09-18T11:30:55.332 INFO:tasks.rados.rados.0.smithi119.stdout:1061: done (2 left) 2024-09-18T11:30:55.332 INFO:tasks.rados.rados.0.smithi119.stdout:1062: done (1 left) 2024-09-18T11:30:55.332 INFO:tasks.rados.rados.0.smithi119.stdout:1063: done (0 left) 2024-09-18T11:30:55.332 INFO:tasks.rados.rados.0.smithi119.stdout:1064: write (excl) oid 26 current snap is 20 2024-09-18T11:30:55.332 INFO:tasks.rados.rados.0.smithi119.stdout:1064: seq_num 178 ranges {591646=470441,1566383=595717,2888822=102824} 2024-09-18T11:30:55.337 INFO:tasks.rados.rados.0.smithi119.stdout:1064: writing smithi11939506-26 from 591646 to 1062087 tid 1 2024-09-18T11:30:55.340 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:55.340 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:55.340 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:55.340 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:55.340 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:55.343 INFO:tasks.rados.rados.0.smithi119.stdout:1064: writing smithi11939506-26 from 1566383 to 2162100 tid 2 2024-09-18T11:30:55.347 INFO:tasks.rados.rados.0.smithi119.stdout:1064: writing smithi11939506-26 from 2888822 to 2991646 tid 3 2024-09-18T11:30:55.348 INFO:tasks.rados.rados.0.smithi119.stdout:1065: read oid 40 snap -1 2024-09-18T11:30:55.348 INFO:tasks.rados.rados.0.smithi119.stdout:1065: expect (ObjNum 175 snap 20 seq_num 175) 2024-09-18T11:30:55.348 INFO:tasks.rados.rados.0.smithi119.stdout:1066: write (excl) oid 17 current snap is 20 2024-09-18T11:30:55.348 INFO:tasks.rados.rados.0.smithi119.stdout:1066: seq_num 179 ranges {608453=545690,1624865=455728,2632598=375855} 2024-09-18T11:30:55.350 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:30:55.354 INFO:tasks.rados.rados.0.smithi119.stdout:1066: writing smithi11939506-17 from 608453 to 1154143 tid 1 2024-09-18T11:30:55.359 INFO:tasks.rados.rados.0.smithi119.stdout:1066: writing smithi11939506-17 from 1624865 to 2080593 tid 2 2024-09-18T11:30:55.364 INFO:tasks.rados.rados.0.smithi119.stdout:1066: writing smithi11939506-17 from 2632598 to 3008453 tid 3 2024-09-18T11:30:55.365 INFO:tasks.rados.rados.0.smithi119.stdout:1067: set_chunk oid 22 target oid 2024-09-18T11:30:55.370 INFO:tasks.rados.rados.0.smithi119.stdout:1067: set_chunk oid 22 offset: 730112 length: 838656 target oid 7058f5ab2978eaa1126ef9c5a465c8c279ea65da6bf00d30b77cf05442107133 offset: 0 snap -1 2024-09-18T11:30:55.370 INFO:tasks.rados.rados.0.smithi119.stdout:1068: read oid 8 snap -1 2024-09-18T11:30:55.370 INFO:tasks.rados.rados.0.smithi119.stdout:1068: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-18T11:30:55.370 INFO:tasks.rados.rados.0.smithi119.stdout:1064: finishing write tid 1 to smithi11939506-26 2024-09-18T11:30:55.371 INFO:tasks.rados.rados.0.smithi119.stdout:1069: read oid 15 snap -1 2024-09-18T11:30:55.371 INFO:tasks.rados.rados.0.smithi119.stdout:1069: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-18T11:30:55.371 INFO:tasks.rados.rados.0.smithi119.stdout:1070: read oid 49 snap 14 2024-09-18T11:30:55.371 INFO:tasks.rados.rados.0.smithi119.stdout:1070: expect deleted 2024-09-18T11:30:55.371 INFO:tasks.rados.rados.0.smithi119.stdout:1071: set_chunk oid 31 target oid 2024-09-18T11:30:55.378 INFO:tasks.rados.rados.0.smithi119.stdout:1071: set_chunk oid 31 offset: 1671680 length: 292352 target oid ec794cc04703e12690dffdaca549def93e8ba508f9768053de4db3c1a8b8fe9f offset: 0 snap -1 2024-09-18T11:30:55.378 INFO:tasks.rados.rados.0.smithi119.stdout:1072: read oid 21 snap 19 2024-09-18T11:30:55.378 INFO:tasks.rados.rados.0.smithi119.stdout:1072: expect deleted 2024-09-18T11:30:55.379 INFO:tasks.rados.rados.0.smithi119.stdout:1073: rollback oid 44 current snap is 20 2024-09-18T11:30:55.379 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 44 to 16 2024-09-18T11:30:55.379 INFO:tasks.rados.rados.0.smithi119.stdout:1064: finishing write tid 2 to smithi11939506-26 2024-09-18T11:30:55.379 INFO:tasks.rados.rados.0.smithi119.stdout:1064: finishing write tid 3 to smithi11939506-26 2024-09-18T11:30:55.379 INFO:tasks.rados.rados.0.smithi119.stdout:1064: finishing write tid 4 to smithi11939506-26 2024-09-18T11:30:55.379 INFO:tasks.rados.rados.0.smithi119.stdout:1064: finishing write tid 5 to smithi11939506-26 2024-09-18T11:30:55.379 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 127 (ObjNum 178 snap 20 seq_num 178) dirty exists 2024-09-18T11:30:55.379 INFO:tasks.rados.rados.0.smithi119.stdout:1064: left oid 26 (ObjNum 178 snap 20 seq_num 178) 2024-09-18T11:30:55.379 INFO:tasks.rados.rados.0.smithi119.stdout:1064: done (9 left) 2024-09-18T11:30:55.379 INFO:tasks.rados.rados.0.smithi119.stdout:1074: write oid 32 current snap is 20 2024-09-18T11:30:55.379 INFO:tasks.rados.rados.0.smithi119.stdout:1065: expect (ObjNum 175 snap 20 seq_num 175) 2024-09-18T11:30:55.393 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:55.391+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:55.476 INFO:tasks.rados.rados.0.smithi119.stdout:1066: finishing write tid 1 to smithi11939506-17 2024-09-18T11:30:55.476 INFO:tasks.rados.rados.0.smithi119.stdout:1066: finishing write tid 2 to smithi11939506-17 2024-09-18T11:30:55.476 INFO:tasks.rados.rados.0.smithi119.stdout:1066: finishing write tid 3 to smithi11939506-17 2024-09-18T11:30:55.476 INFO:tasks.rados.rados.0.smithi119.stdout:1066: finishing write tid 4 to smithi11939506-17 2024-09-18T11:30:55.476 INFO:tasks.rados.rados.0.smithi119.stdout:1066: finishing write tid 5 to smithi11939506-17 2024-09-18T11:30:55.476 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 163 (ObjNum 179 snap 20 seq_num 179) dirty exists 2024-09-18T11:30:55.477 INFO:tasks.rados.rados.0.smithi119.stdout:1066: left oid 17 (ObjNum 179 snap 20 seq_num 179) 2024-09-18T11:30:55.477 INFO:tasks.rados.rados.0.smithi119.stdout:1074: seq_num 180 ranges {625260=620939,1683347=599386,2943668=81592} 2024-09-18T11:30:55.482 INFO:tasks.rados.rados.0.smithi119.stdout:1074: writing smithi11939506-32 from 625260 to 1246199 tid 1 2024-09-18T11:30:55.487 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:55.486+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:55.488 INFO:tasks.rados.rados.0.smithi119.stdout:1074: writing smithi11939506-32 from 1683347 to 2282733 tid 2 2024-09-18T11:30:55.492 INFO:tasks.rados.rados.0.smithi119.stdout:1074: writing smithi11939506-32 from 2943668 to 3025260 tid 3 2024-09-18T11:30:55.493 INFO:tasks.rados.rados.0.smithi119.stdout:1065: done (9 left) 2024-09-18T11:30:55.493 INFO:tasks.rados.rados.0.smithi119.stdout:1066: done (8 left) 2024-09-18T11:30:55.493 INFO:tasks.rados.rados.0.smithi119.stdout:1075: tier_promote oid 46 2024-09-18T11:30:55.493 INFO:tasks.rados.rados.0.smithi119.stdout:1069: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-18T11:30:55.496 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:55.496 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:55.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:55.496 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:55.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.496 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:55.496 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:55.496 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 44.376344176000003, 2024-09-18T11:30:55.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:30:55.497 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:55.497 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:55.497 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:55.497 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:55.497 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:55.497 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:30:55.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.497 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:55.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:55.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:55.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:55.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:30:55.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:55.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:30:55.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:55.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:55.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:30:55.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:30:55.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:55.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:30:55.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:30:55.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:55.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:30:55.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:30:55.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:30:55.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:30:55.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:55.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:30:55.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:30:55.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:30:55.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:30:55.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:55.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:30:55.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:30:55.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:55.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:30:55.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:30:55.503 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.503 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:55.503 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.503 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1132 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2605052,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:55.504 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179179+0000", 2024-09-18T11:30:55.504 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.313484172000003, 2024-09-18T11:30:55.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059816840000000001, 2024-09-18T11:30:55.504 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:55.504 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:55.504 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:55.504 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:55.504 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:55.505 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1132 2024-09-18T11:30:55.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.505 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:55.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:55.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179179+0000", 2024-09-18T11:30:55.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:55.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179179+0000", 2024-09-18T11:30:55.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:55.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179178+0000", 2024-09-18T11:30:55.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:55.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:55.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179193+0000", 2024-09-18T11:30:55.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5446000000000001e-05 2024-09-18T11:30:55.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:55.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179215+0000", 2024-09-18T11:30:55.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1253000000000001e-05 2024-09-18T11:30:55.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:55.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179227+0000", 2024-09-18T11:30:55.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2612999999999999e-05 2024-09-18T11:30:55.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179447+0000", 2024-09-18T11:30:55.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021972199999999999 2024-09-18T11:30:55.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:55.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179453+0000", 2024-09-18T11:30:55.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8830000000000003e-06 2024-09-18T11:30:55.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184113+0000", 2024-09-18T11:30:55.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046603180000000001 2024-09-18T11:30:55.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:55.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184229+0000", 2024-09-18T11:30:55.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000115516 2024-09-18T11:30:55.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:55.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185161+0000", 2024-09-18T11:30:55.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00093216000000000002 2024-09-18T11:30:55.511 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.511 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:55.511 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.511 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:55.511 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:30:55.511 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 34.823318774999997, 2024-09-18T11:30:55.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:30:55.511 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:55.511 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:55.511 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:55.511 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:55.512 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:55.512 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:30:55.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.512 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:55.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:55.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:30:55.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:55.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:30:55.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:55.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:30:55.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:55.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:55.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:30:55.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:30:55.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:55.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:30:55.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:30:55.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:55.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:30:55.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:30:55.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:30:55.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:30:55.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:55.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:30:55.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:30:55.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:55.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:30:55.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:30:55.516 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.516 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:55.517 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.517 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1185 4.5 4:af4ba75a:::smithi11939506-37:head [write 729437~527542] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:55.517 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:21.156648+0000", 2024-09-18T11:30:55.517 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 34.336014851000002, 2024-09-18T11:30:55.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057485540000000003, 2024-09-18T11:30:55.517 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:55.517 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:55.517 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:55.518 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:55.518 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:55.518 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1185 2024-09-18T11:30:55.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.518 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:55.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:55.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.156648+0000", 2024-09-18T11:30:55.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:55.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.156648+0000", 2024-09-18T11:30:55.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.519 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:55.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.156647+0000", 2024-09-18T11:30:55.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:55.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.519 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:55.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157166+0000", 2024-09-18T11:30:55.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00051975999999999997 2024-09-18T11:30:55.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:55.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157183+0000", 2024-09-18T11:30:55.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.611e-05 2024-09-18T11:30:55.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:55.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157188+0000", 2024-09-18T11:30:55.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.648e-06 2024-09-18T11:30:55.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157253+0000", 2024-09-18T11:30:55.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4503000000000002e-05 2024-09-18T11:30:55.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:55.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157322+0000", 2024-09-18T11:30:55.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9031999999999999e-05 2024-09-18T11:30:55.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:55.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.157437+0000", 2024-09-18T11:30:55.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011543400000000001 2024-09-18T11:30:55.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:55.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162104+0000", 2024-09-18T11:30:55.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046668009999999999 2024-09-18T11:30:55.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:55.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162321+0000", 2024-09-18T11:30:55.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021706699999999999 2024-09-18T11:30:55.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:55.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162355+0000", 2024-09-18T11:30:55.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3670000000000001e-05 2024-09-18T11:30:55.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:55.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:21.162397+0000", 2024-09-18T11:30:55.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2168e-05 2024-09-18T11:30:55.523 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.523 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:55.523 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.524 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1688 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 19=[19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-18T11:30:55.524 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:30:55.524 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 15.960164182, 2024-09-18T11:30:55.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071575789999999999, 2024-09-18T11:30:55.524 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:55.524 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:55.524 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:55.524 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:55.525 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:55.525 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1688 2024-09-18T11:30:55.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.525 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:55.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:55.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:30:55.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:55.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:30:55.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:55.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:30:55.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:55.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:55.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532500+0000", 2024-09-18T11:30:55.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5260000000000001e-06 2024-09-18T11:30:55.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:55.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532503+0000", 2024-09-18T11:30:55.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:30:55.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:55.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532505+0000", 2024-09-18T11:30:55.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.889e-06 2024-09-18T11:30:55.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.534722+0000", 2024-09-18T11:30:55.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002217119 2024-09-18T11:30:55.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:55.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.534827+0000", 2024-09-18T11:30:55.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0001049 2024-09-18T11:30:55.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.530 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:55.530 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.535007+0000", 2024-09-18T11:30:55.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017965599999999999 2024-09-18T11:30:55.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.530 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:55.530 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539463+0000", 2024-09-18T11:30:55.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004455713 2024-09-18T11:30:55.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.530 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:55.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539607+0000", 2024-09-18T11:30:55.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014440300000000001 2024-09-18T11:30:55.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:55.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539634+0000", 2024-09-18T11:30:55.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7398000000000001e-05 2024-09-18T11:30:55.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:55.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539657+0000", 2024-09-18T11:30:55.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2228000000000001e-05 2024-09-18T11:30:55.531 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.531 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:55.531 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.532 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1759 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2319366,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:55.532 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:30:55.532 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.198703972000001, 2024-09-18T11:30:55.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064323039999999998, 2024-09-18T11:30:55.532 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:55.532 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:55.532 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:55.533 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:55.533 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:55.533 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1759 2024-09-18T11:30:55.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.533 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:55.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:55.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:30:55.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:55.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:30:55.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:55.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:30:55.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:55.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:55.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293963+0000", 2024-09-18T11:30:55.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0810000000000004e-06 2024-09-18T11:30:55.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.535 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:55.535 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293970+0000", 2024-09-18T11:30:55.535 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5109999999999997e-06 2024-09-18T11:30:55.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.535 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:55.535 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293975+0000", 2024-09-18T11:30:55.535 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.33e-06 2024-09-18T11:30:55.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.535 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.535 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294078+0000", 2024-09-18T11:30:55.535 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102871 2024-09-18T11:30:55.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.536 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:55.536 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294084+0000", 2024-09-18T11:30:55.536 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5470000000000002e-06 2024-09-18T11:30:55.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.536 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.536 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.299550+0000", 2024-09-18T11:30:55.536 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054660799999999999 2024-09-18T11:30:55.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.537 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:55.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.299608+0000", 2024-09-18T11:30:55.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8307000000000003e-05 2024-09-18T11:30:55.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.537 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:55.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300392+0000", 2024-09-18T11:30:55.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078306999999999997 2024-09-18T11:30:55.537 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.537 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:55.538 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.538 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1760 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2319366,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:55.538 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:30:55.538 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.198669632, 2024-09-18T11:30:55.538 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070132529999999997, 2024-09-18T11:30:55.538 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:55.538 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:55.538 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:55.538 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:55.538 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:55.538 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1760 2024-09-18T11:30:55.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.539 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:55.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:55.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:30:55.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:55.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:30:55.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.540 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:55.540 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293993+0000", 2024-09-18T11:30:55.540 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:55.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.540 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:55.540 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293997+0000", 2024-09-18T11:30:55.540 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4999999999999999e-06 2024-09-18T11:30:55.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:55.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294004+0000", 2024-09-18T11:30:55.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3950000000000003e-06 2024-09-18T11:30:55.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:55.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294009+0000", 2024-09-18T11:30:55.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4989999999999996e-06 2024-09-18T11:30:55.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.542 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294103+0000", 2024-09-18T11:30:55.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4817999999999999e-05 2024-09-18T11:30:55.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.542 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:55.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294110+0000", 2024-09-18T11:30:55.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7240000000000003e-06 2024-09-18T11:30:55.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.542 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300406+0000", 2024-09-18T11:30:55.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062964470000000002 2024-09-18T11:30:55.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:55.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300457+0000", 2024-09-18T11:30:55.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0915000000000003e-05 2024-09-18T11:30:55.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:55.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.301007+0000", 2024-09-18T11:30:55.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00054942299999999995 2024-09-18T11:30:55.543 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.543 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:55.544 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.544 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1775 4.a 4:588debc2:::smithi11939506-18:head [write 1527054~766050] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:55.544 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:30:55.544 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.596316891000001, 2024-09-18T11:30:55.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070939050000000002, 2024-09-18T11:30:55.544 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:55.544 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:55.544 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:55.544 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:55.544 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:55.545 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1775 2024-09-18T11:30:55.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.545 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:55.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.545 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:55.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:30:55.545 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.546 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:55.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:30:55.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.546 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:55.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896345+0000", 2024-09-18T11:30:55.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:55.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.547 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:55.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896993+0000", 2024-09-18T11:30:55.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064769999999999997 2024-09-18T11:30:55.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.547 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:55.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897011+0000", 2024-09-18T11:30:55.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8355999999999998e-05 2024-09-18T11:30:55.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:55.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897016+0000", 2024-09-18T11:30:55.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3890000000000004e-06 2024-09-18T11:30:55.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897054+0000", 2024-09-18T11:30:55.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7963000000000002e-05 2024-09-18T11:30:55.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:55.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897123+0000", 2024-09-18T11:30:55.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8128999999999998e-05 2024-09-18T11:30:55.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:55.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897224+0000", 2024-09-18T11:30:55.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010168800000000001 2024-09-18T11:30:55.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:55.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899515+0000", 2024-09-18T11:30:55.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022910280000000001 2024-09-18T11:30:55.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:55.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903333+0000", 2024-09-18T11:30:55.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003817459 2024-09-18T11:30:55.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:55.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903388+0000", 2024-09-18T11:30:55.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5651e-05 2024-09-18T11:30:55.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.551 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:55.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903440+0000", 2024-09-18T11:30:55.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1836999999999999e-05 2024-09-18T11:30:55.551 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.551 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:55.551 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.552 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1776 4.a 4:588debc2:::smithi11939506-18:head [setxattr _header (57) in=7b,truncate 2420208] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:55.552 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:30:55.552 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.595639079, 2024-09-18T11:30:55.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065840400000000002, 2024-09-18T11:30:55.552 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:55.552 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:55.552 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:55.552 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:55.552 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:55.552 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1776 2024-09-18T11:30:55.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.552 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:55.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:55.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:30:55.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:55.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:30:55.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:55.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:30:55.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:55.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.554 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:55.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897026+0000", 2024-09-18T11:30:55.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9579999999999999e-06 2024-09-18T11:30:55.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:55.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897030+0000", 2024-09-18T11:30:55.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5370000000000002e-06 2024-09-18T11:30:55.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:55.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897032+0000", 2024-09-18T11:30:55.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.17e-06 2024-09-18T11:30:55.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898415+0000", 2024-09-18T11:30:55.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013823189999999999 2024-09-18T11:30:55.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.556 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:55.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898527+0000", 2024-09-18T11:30:55.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011236800000000001 2024-09-18T11:30:55.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.556 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:55.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898714+0000", 2024-09-18T11:30:55.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018703599999999999 2024-09-18T11:30:55.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.556 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:55.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899683+0000", 2024-09-18T11:30:55.557 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00096944700000000002 2024-09-18T11:30:55.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.557 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:55.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903475+0000", 2024-09-18T11:30:55.557 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037920129999999999 2024-09-18T11:30:55.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.557 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:55.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903535+0000", 2024-09-18T11:30:55.557 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9843999999999998e-05 2024-09-18T11:30:55.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.558 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:55.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903608+0000", 2024-09-18T11:30:55.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2946000000000003e-05 2024-09-18T11:30:55.558 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.558 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:55.558 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.558 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1777 4.a 4:588debc2:::smithi11939506-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:55.558 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:30:55.558 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.595612439, 2024-09-18T11:30:55.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0069010809999999999, 2024-09-18T11:30:55.558 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:55.559 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:55.559 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:55.559 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:55.559 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:55.559 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1777 2024-09-18T11:30:55.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.559 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:55.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:55.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:30:55.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:55.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:30:55.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:55.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897050+0000", 2024-09-18T11:30:55.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:55.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.561 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:55.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897052+0000", 2024-09-18T11:30:55.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:30:55.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.561 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:55.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897057+0000", 2024-09-18T11:30:55.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9470000000000002e-06 2024-09-18T11:30:55.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.561 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:55.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897060+0000", 2024-09-18T11:30:55.562 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3410000000000001e-06 2024-09-18T11:30:55.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.562 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899417+0000", 2024-09-18T11:30:55.562 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023571170000000002 2024-09-18T11:30:55.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.562 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:55.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899501+0000", 2024-09-18T11:30:55.562 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4160999999999996e-05 2024-09-18T11:30:55.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.563 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.563 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.563 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903661+0000", 2024-09-18T11:30:55.563 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041599510000000003 2024-09-18T11:30:55.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.563 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.563 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:55.563 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903790+0000", 2024-09-18T11:30:55.563 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012941499999999999 2024-09-18T11:30:55.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.563 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.563 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:55.564 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903952+0000", 2024-09-18T11:30:55.564 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016146800000000001 2024-09-18T11:30:55.564 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.564 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:55.564 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.564 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.564 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1895 4.5 4:a087a566:::smithi11939506-46:head [read 0~2756315,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:30:55.564 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:30:55.564 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.1843569340000002, 2024-09-18T11:30:55.564 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007540377, 2024-09-18T11:30:55.564 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:55.564 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:55.565 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:55.565 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:55.565 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:55.565 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1895 2024-09-18T11:30:55.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.565 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:55.565 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.565 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:55.565 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:30:55.565 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.565 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.565 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:55.565 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:30:55.566 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.566 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:55.566 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:30:55.566 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:55.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.566 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:55.566 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308310+0000", 2024-09-18T11:30:55.566 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0559999999999998e-06 2024-09-18T11:30:55.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.566 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:55.567 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308317+0000", 2024-09-18T11:30:55.567 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7209999999999997e-06 2024-09-18T11:30:55.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.567 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.567 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:55.567 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308320+0000", 2024-09-18T11:30:55.567 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.721e-06 2024-09-18T11:30:55.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.567 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.567 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.567 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308447+0000", 2024-09-18T11:30:55.568 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012645199999999999 2024-09-18T11:30:55.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.568 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.568 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:55.568 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308451+0000", 2024-09-18T11:30:55.568 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6800000000000001e-06 2024-09-18T11:30:55.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.568 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.568 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.568 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.312462+0000", 2024-09-18T11:30:55.568 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040102139999999998 2024-09-18T11:30:55.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.569 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:55.569 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.312585+0000", 2024-09-18T11:30:55.569 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012388899999999999 2024-09-18T11:30:55.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.569 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:55.569 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.315847+0000", 2024-09-18T11:30:55.569 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003261197 2024-09-18T11:30:55.570 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.570 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:55.570 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.570 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.570 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1924 4.5 4:af4ba75a:::smithi11939506-37:head [write 1829449~675380] snapc 1e=[1e,1c,1b,19,18,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:30:55.570 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.362228+0000", 2024-09-18T11:30:55.570 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.1304352350000002, 2024-09-18T11:30:55.570 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066182139999999999, 2024-09-18T11:30:55.570 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:55.570 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:55.570 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:55.571 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:55.571 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:55.571 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1924 2024-09-18T11:30:55.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.571 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:55.571 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.571 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:55.571 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362228+0000", 2024-09-18T11:30:55.571 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.571 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.571 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:55.572 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362228+0000", 2024-09-18T11:30:55.572 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.572 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.572 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:55.572 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362227+0000", 2024-09-18T11:30:55.572 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:55.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.572 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.572 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:55.572 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362749+0000", 2024-09-18T11:30:55.573 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00052262399999999996 2024-09-18T11:30:55.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.573 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.573 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:55.573 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362762+0000", 2024-09-18T11:30:55.573 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2821e-05 2024-09-18T11:30:55.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.573 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.573 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:55.573 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362767+0000", 2024-09-18T11:30:55.573 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5349999999999998e-06 2024-09-18T11:30:55.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.574 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.574 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.574 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362806+0000", 2024-09-18T11:30:55.574 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9029999999999997e-05 2024-09-18T11:30:55.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.574 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.574 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:55.575 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362863+0000", 2024-09-18T11:30:55.575 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7584999999999998e-05 2024-09-18T11:30:55.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.575 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.575 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:55.575 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362949+0000", 2024-09-18T11:30:55.576 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5655000000000006e-05 2024-09-18T11:30:55.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.576 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:55.576 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364314+0000", 2024-09-18T11:30:55.576 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001364961 2024-09-18T11:30:55.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.576 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:55.576 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.368771+0000", 2024-09-18T11:30:55.577 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004457244 2024-09-18T11:30:55.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.577 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:55.577 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.368809+0000", 2024-09-18T11:30:55.577 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8368999999999998e-05 2024-09-18T11:30:55.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.577 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:55.577 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.368846+0000", 2024-09-18T11:30:55.577 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6718000000000003e-05 2024-09-18T11:30:55.577 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.577 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:55.577 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.578 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.578 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1925 4.5 4:af4ba75a:::smithi11939506-37:head [setxattr _header (57) in=7b,truncate 2655506] snapc 1e=[1e,1c,1b,19,18,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:30:55.578 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:30:55.578 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.1298900829999994, 2024-09-18T11:30:55.578 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064745779999999999, 2024-09-18T11:30:55.578 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:55.578 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:55.578 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:55.578 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:55.578 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:55.579 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1925 2024-09-18T11:30:55.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.579 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:55.579 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.579 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:55.579 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:30:55.579 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.580 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.580 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:55.580 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:30:55.580 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.580 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.580 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:55.580 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:30:55.580 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:55.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.580 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.580 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:55.580 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362776+0000", 2024-09-18T11:30:55.581 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.208e-06 2024-09-18T11:30:55.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.581 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.581 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:55.581 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362780+0000", 2024-09-18T11:30:55.581 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.771e-06 2024-09-18T11:30:55.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.581 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.581 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:55.581 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362782+0000", 2024-09-18T11:30:55.582 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.277e-06 2024-09-18T11:30:55.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.582 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.582 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.582 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.363737+0000", 2024-09-18T11:30:55.582 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00095483999999999999 2024-09-18T11:30:55.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.582 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.582 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:55.582 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.363832+0000", 2024-09-18T11:30:55.582 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5594999999999995e-05 2024-09-18T11:30:55.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.582 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.582 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:55.583 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.363940+0000", 2024-09-18T11:30:55.583 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010738000000000001 2024-09-18T11:30:55.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.583 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.583 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:55.583 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364577+0000", 2024-09-18T11:30:55.583 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00063679600000000002 2024-09-18T11:30:55.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.583 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.583 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:55.583 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369126+0000", 2024-09-18T11:30:55.583 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0045496440000000003 2024-09-18T11:30:55.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.584 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.584 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:55.584 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369177+0000", 2024-09-18T11:30:55.584 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0958000000000002e-05 2024-09-18T11:30:55.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.584 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.584 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:55.584 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369248+0000", 2024-09-18T11:30:55.585 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0599000000000006e-05 2024-09-18T11:30:55.585 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.585 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:55.586 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.586 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.586 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1926 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:30:55.586 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:30:55.586 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.1298658269999997, 2024-09-18T11:30:55.586 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066190219999999996, 2024-09-18T11:30:55.586 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:55.586 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:55.587 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:55.587 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:55.587 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:55.587 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1926 2024-09-18T11:30:55.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.587 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:55.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.587 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:55.587 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:30:55.587 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.588 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:55.588 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:30:55.588 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.588 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.588 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:55.588 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:30:55.588 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:55.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.588 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.588 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:55.589 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362799+0000", 2024-09-18T11:30:55.589 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.029e-06 2024-09-18T11:30:55.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.589 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:55.589 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362804+0000", 2024-09-18T11:30:55.589 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3989999999999997e-06 2024-09-18T11:30:55.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.590 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:55.590 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362806+0000", 2024-09-18T11:30:55.590 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3020000000000002e-06 2024-09-18T11:30:55.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.590 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.590 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364105+0000", 2024-09-18T11:30:55.590 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001299478 2024-09-18T11:30:55.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.591 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.591 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:30:55.591 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364153+0000", 2024-09-18T11:30:55.591 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8059000000000002e-05 2024-09-18T11:30:55.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.591 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.591 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.591 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369273+0000", 2024-09-18T11:30:55.591 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051199540000000003 2024-09-18T11:30:55.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.591 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.591 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:55.592 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369343+0000", 2024-09-18T11:30:55.592 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0191999999999995e-05 2024-09-18T11:30:55.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.592 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.592 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:55.592 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369416+0000", 2024-09-18T11:30:55.592 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2925999999999999e-05 2024-09-18T11:30:55.592 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.592 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:55.592 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.593 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.593 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1964 4.a 4:5975a5ec:::smithi11939506-36:head [append 0~0] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:30:55.593 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373800+0000", 2024-09-18T11:30:55.593 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.1188629199999998, 2024-09-18T11:30:55.593 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006226105, 2024-09-18T11:30:55.593 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:55.593 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:55.593 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:55.593 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:55.593 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:55.593 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1964 2024-09-18T11:30:55.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.594 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:55.594 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.594 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:55.594 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373800+0000", 2024-09-18T11:30:55.594 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.596 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.596 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:55.596 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373800+0000", 2024-09-18T11:30:55.596 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.597 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.597 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:55.597 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373799+0000", 2024-09-18T11:30:55.597 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:55.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.597 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.597 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:55.597 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373809+0000", 2024-09-18T11:30:55.597 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0777e-05 2024-09-18T11:30:55.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.597 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.597 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:55.598 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373824+0000", 2024-09-18T11:30:55.598 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4640999999999999e-05 2024-09-18T11:30:55.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.598 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.598 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:55.598 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373831+0000", 2024-09-18T11:30:55.598 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5899999999999996e-06 2024-09-18T11:30:55.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.598 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.598 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.598 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373888+0000", 2024-09-18T11:30:55.598 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7679000000000001e-05 2024-09-18T11:30:55.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.599 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.599 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:55.599 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373957+0000", 2024-09-18T11:30:55.599 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9006999999999998e-05 2024-09-18T11:30:55.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.599 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.599 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:55.599 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.374089+0000", 2024-09-18T11:30:55.599 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013161999999999999 2024-09-18T11:30:55.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.599 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.599 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:55.600 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.379939+0000", 2024-09-18T11:30:55.600 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058500080000000003 2024-09-18T11:30:55.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.600 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.600 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:55.600 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.379974+0000", 2024-09-18T11:30:55.600 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4764999999999997e-05 2024-09-18T11:30:55.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.600 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.600 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:55.601 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380002+0000", 2024-09-18T11:30:55.601 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8201999999999999e-05 2024-09-18T11:30:55.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.601 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.601 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:55.601 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380026+0000", 2024-09-18T11:30:55.601 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4477999999999998e-05 2024-09-18T11:30:55.601 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.601 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:55.601 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.601 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.601 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1965 4.a 4:5975a5ec:::smithi11939506-36:head [rollback 1e] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:30:55.602 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373840+0000", 2024-09-18T11:30:55.602 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.1188234110000002, 2024-09-18T11:30:55.602 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062698229999999999, 2024-09-18T11:30:55.602 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:55.602 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:55.602 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:55.602 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:55.602 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:55.602 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1965 2024-09-18T11:30:55.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.602 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:55.602 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.602 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:55.603 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373840+0000", 2024-09-18T11:30:55.603 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.603 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:55.603 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373840+0000", 2024-09-18T11:30:55.603 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.603 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:55.603 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373839+0000", 2024-09-18T11:30:55.603 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:55.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.603 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:55.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373842+0000", 2024-09-18T11:30:55.604 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8770000000000002e-06 2024-09-18T11:30:55.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.604 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.604 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:55.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373849+0000", 2024-09-18T11:30:55.604 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3929999999999996e-06 2024-09-18T11:30:55.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.604 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.605 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:55.605 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373853+0000", 2024-09-18T11:30:55.605 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0099999999999997e-06 2024-09-18T11:30:55.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.605 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.375207+0000", 2024-09-18T11:30:55.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001354755 2024-09-18T11:30:55.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.606 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:55.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.375268+0000", 2024-09-18T11:30:55.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0983999999999998e-05 2024-09-18T11:30:55.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.606 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:55.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.375449+0000", 2024-09-18T11:30:55.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000181091 2024-09-18T11:30:55.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.607 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:55.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380058+0000", 2024-09-18T11:30:55.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046089390000000003 2024-09-18T11:30:55.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.607 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:55.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380065+0000", 2024-09-18T11:30:55.608 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4479999999999999e-06 2024-09-18T11:30:55.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.608 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:55.608 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380088+0000", 2024-09-18T11:30:55.608 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3337e-05 2024-09-18T11:30:55.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.608 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:55.608 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380110+0000", 2024-09-18T11:30:55.608 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.145e-05 2024-09-18T11:30:55.608 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.608 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:55.609 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.609 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1966 4.a 4:5975a5ec:::smithi11939506-36:head [append 0~0] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:30:55.609 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373876+0000", 2024-09-18T11:30:55.609 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.1187873870000002, 2024-09-18T11:30:55.609 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063095080000000001, 2024-09-18T11:30:55.609 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:55.609 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:55.609 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:55.609 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:55.609 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:55.609 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1966 2024-09-18T11:30:55.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.610 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:55.610 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.610 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:55.610 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373876+0000", 2024-09-18T11:30:55.610 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.610 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.610 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:55.610 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373876+0000", 2024-09-18T11:30:55.610 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.610 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.610 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:55.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373875+0000", 2024-09-18T11:30:55.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:55.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:55.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373879+0000", 2024-09-18T11:30:55.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3859999999999999e-06 2024-09-18T11:30:55.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:55.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373885+0000", 2024-09-18T11:30:55.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6030000000000001e-06 2024-09-18T11:30:55.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.612 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:55.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373889+0000", 2024-09-18T11:30:55.612 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9620000000000004e-06 2024-09-18T11:30:55.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.612 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.376620+0000", 2024-09-18T11:30:55.612 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027307920000000001 2024-09-18T11:30:55.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.612 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:55.613 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.376672+0000", 2024-09-18T11:30:55.613 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2244999999999999e-05 2024-09-18T11:30:55.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.613 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:30:55.613 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.376838+0000", 2024-09-18T11:30:55.613 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000165716 2024-09-18T11:30:55.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.614 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:55.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380116+0000", 2024-09-18T11:30:55.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003278352 2024-09-18T11:30:55.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.614 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:55.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380145+0000", 2024-09-18T11:30:55.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8379999999999999e-05 2024-09-18T11:30:55.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.614 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:55.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380166+0000", 2024-09-18T11:30:55.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1118e-05 2024-09-18T11:30:55.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.615 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:55.615 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380185+0000", 2024-09-18T11:30:55.615 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9519999999999999e-05 2024-09-18T11:30:55.615 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.615 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:55.615 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.616 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1968 4.a 4:588debc2:::smithi11939506-18:18 [read 0~3949612,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:30:55.616 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373915+0000", 2024-09-18T11:30:55.616 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.1187479639999998, 2024-09-18T11:30:55.616 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060271159999999999, 2024-09-18T11:30:55.617 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:55.617 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:55.617 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:55.617 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:55.617 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:55.617 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1968 2024-09-18T11:30:55.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.618 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:55.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:55.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373915+0000", 2024-09-18T11:30:55.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:55.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373915+0000", 2024-09-18T11:30:55.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:55.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373915+0000", 2024-09-18T11:30:55.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:55.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.619 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:55.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373918+0000", 2024-09-18T11:30:55.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.619e-06 2024-09-18T11:30:55.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.620 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:55.620 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373925+0000", 2024-09-18T11:30:55.620 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3910000000000003e-06 2024-09-18T11:30:55.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.620 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:55.620 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373928+0000", 2024-09-18T11:30:55.620 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.478e-06 2024-09-18T11:30:55.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.378583+0000", 2024-09-18T11:30:55.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046544339999999998 2024-09-18T11:30:55.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:55.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.378640+0000", 2024-09-18T11:30:55.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7226999999999999e-05 2024-09-18T11:30:55.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:55.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.379942+0000", 2024-09-18T11:30:55.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001302465 2024-09-18T11:30:55.622 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.622 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:55.622 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.622 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1969 4.a 4:588debc2:::smithi11939506-18:18 [read 0~3949612,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:30:55.622 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:30:55.622 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.1186769380000001, 2024-09-18T11:30:55.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075409190000000001, 2024-09-18T11:30:55.623 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:55.623 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:30:55.623 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:55.623 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:55.623 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:55.623 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1969 2024-09-18T11:30:55.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.623 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:55.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.623 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:55.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:30:55.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.624 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:55.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:30:55.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.625 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:55.625 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:30:55.625 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:55.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.625 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:55.625 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373990+0000", 2024-09-18T11:30:55.625 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1119999999999998e-06 2024-09-18T11:30:55.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:55.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373997+0000", 2024-09-18T11:30:55.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4200000000000001e-06 2024-09-18T11:30:55.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:55.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.374002+0000", 2024-09-18T11:30:55.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:30:55.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380200+0000", 2024-09-18T11:30:55.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006197302 2024-09-18T11:30:55.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:55.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380261+0000", 2024-09-18T11:30:55.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1389999999999993e-05 2024-09-18T11:30:55.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:55.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.381527+0000", 2024-09-18T11:30:55.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012660080000000001 2024-09-18T11:30:55.628 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.628 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:55.628 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.628 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2033 4.5 4:a087a566:::smithi11939506-46:head [write 473997~627345] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T11:30:55.628 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:53.015661+0000", 2024-09-18T11:30:55.628 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 2.477002583, 2024-09-18T11:30:55.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059696810000000001, 2024-09-18T11:30:55.629 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:55.629 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:55.629 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:55.629 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:30:55.629 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:55.629 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2033 2024-09-18T11:30:55.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.629 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:55.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:55.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:53.015661+0000", 2024-09-18T11:30:55.629 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:55.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:53.015661+0000", 2024-09-18T11:30:55.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:55.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:55.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:53.015659+0000", 2024-09-18T11:30:55.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:55.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:55.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:53.016359+0000", 2024-09-18T11:30:55.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00070014600000000001 2024-09-18T11:30:55.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:55.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:53.016397+0000", 2024-09-18T11:30:55.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7673999999999999e-05 2024-09-18T11:30:55.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:55.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:53.016405+0000", 2024-09-18T11:30:55.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3510000000000002e-06 2024-09-18T11:30:55.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.632 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:53.016476+0000", 2024-09-18T11:30:55.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1441999999999998e-05 2024-09-18T11:30:55.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.632 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:30:55.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:53.016481+0000", 2024-09-18T11:30:55.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8369999999999996e-06 2024-09-18T11:30:55.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:55.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:53.018227+0000", 2024-09-18T11:30:55.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00174577 2024-09-18T11:30:55.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:55.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:53.018331+0000", 2024-09-18T11:30:55.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104429 2024-09-18T11:30:55.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:30:55.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:53.018516+0000", 2024-09-18T11:30:55.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000184103 2024-09-18T11:30:55.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:55.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:53.020461+0000", 2024-09-18T11:30:55.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019451869999999999 2024-09-18T11:30:55.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:55.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:53.021567+0000", 2024-09-18T11:30:55.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011057529999999999 2024-09-18T11:30:55.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.635 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:55.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:53.021601+0000", 2024-09-18T11:30:55.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4555999999999999e-05 2024-09-18T11:30:55.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:55.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:55.635 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:55.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:53.021630+0000", 2024-09-18T11:30:55.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9249000000000001e-05 2024-09-18T11:30:55.635 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.635 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:55.636 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.636 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:55.636 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:55.636 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:55.637 INFO:tasks.rados.rados.0.smithi119.stdout:1068: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-18T11:30:55.638 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:30:55.639 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:55.636+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:55.694 INFO:tasks.rados.rados.0.smithi119.stdout:1067: finishing set_chunk to oid 22 2024-09-18T11:30:55.694 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 53 (ObjNum 101 snap 6 seq_num 101) dirty exists 2024-09-18T11:30:55.694 INFO:tasks.rados.rados.0.smithi119.stdout:1067: done (8 left) 2024-09-18T11:30:55.694 INFO:tasks.rados.rados.0.smithi119.stdout:1068: done (7 left) 2024-09-18T11:30:55.694 INFO:tasks.rados.rados.0.smithi119.stdout:1069: done (6 left) 2024-09-18T11:30:55.695 INFO:tasks.rados.rados.0.smithi119.stdout:1070: done (5 left) 2024-09-18T11:30:55.695 INFO:tasks.rados.rados.0.smithi119.stdout:1072: done (4 left) 2024-09-18T11:30:55.695 INFO:tasks.rados.rados.0.smithi119.stdout:1076: tier_evict oid 14 2024-09-18T11:30:55.695 INFO:tasks.rados.rados.0.smithi119.stdout:1076: tier_evict oid 14 snap -1 2024-09-18T11:30:55.695 INFO:tasks.rados.rados.0.smithi119.stdout:1071: finishing set_chunk to oid 31 2024-09-18T11:30:55.695 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 66 (ObjNum 121 snap 11 seq_num 121) dirty exists 2024-09-18T11:30:55.695 INFO:tasks.rados.rados.0.smithi119.stdout:1073: finishing rollback tid 0 to smithi11939506-44 2024-09-18T11:30:55.695 INFO:tasks.rados.rados.0.smithi119.stdout:1073: finishing rollback tid 1 to smithi11939506-44 2024-09-18T11:30:55.695 INFO:tasks.rados.rados.0.smithi119.stdout:1073: finishing rollback tid 2 to smithi11939506-44 2024-09-18T11:30:55.695 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 44 v 65 (ObjNum 72 snap 3 seq_num 72) dirty exists 2024-09-18T11:30:55.695 INFO:tasks.rados.rados.0.smithi119.stdout:1074: finishing write tid 1 to smithi11939506-32 2024-09-18T11:30:55.695 INFO:tasks.rados.rados.0.smithi119.stdout:1074: finishing write tid 2 to smithi11939506-32 2024-09-18T11:30:55.695 INFO:tasks.rados.rados.0.smithi119.stdout:1074: finishing write tid 3 to smithi11939506-32 2024-09-18T11:30:55.696 INFO:tasks.rados.rados.0.smithi119.stdout:1074: finishing write tid 4 to smithi11939506-32 2024-09-18T11:30:55.696 INFO:tasks.rados.rados.0.smithi119.stdout:1071: done (4 left) 2024-09-18T11:30:55.696 INFO:tasks.rados.rados.0.smithi119.stdout:1073: done (3 left) 2024-09-18T11:30:55.696 INFO:tasks.rados.rados.0.smithi119.stdout:1077: write oid 22 current snap is 20 2024-09-18T11:30:55.696 INFO:tasks.rados.rados.0.smithi119.stdout:1075: got (0) Success 2024-09-18T11:30:55.696 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 104 (ObjNum 171 snap 20 seq_num 171) dirty exists 2024-09-18T11:30:55.696 INFO:tasks.rados.rados.0.smithi119.stdout:1074: finishing write tid 5 to smithi11939506-32 2024-09-18T11:30:55.696 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 94 (ObjNum 180 snap 20 seq_num 180) dirty exists 2024-09-18T11:30:55.696 INFO:tasks.rados.rados.0.smithi119.stdout:1074: left oid 32 (ObjNum 180 snap 20 seq_num 180) 2024-09-18T11:30:55.696 INFO:tasks.rados.rados.0.smithi119.stdout:1077: seq_num 181 ranges {642067=696188,2025476=459397,2971091=70976} 2024-09-18T11:30:55.700 INFO:tasks.rados.rados.0.smithi119.stdout:1077: writing smithi11939506-22 from 642067 to 1338255 tid 1 2024-09-18T11:30:55.707 INFO:tasks.rados.rados.0.smithi119.stdout:1077: writing smithi11939506-22 from 2025476 to 2484873 tid 2 2024-09-18T11:30:55.710 INFO:tasks.rados.rados.0.smithi119.stdout:1077: writing smithi11939506-22 from 2971091 to 3042067 tid 3 2024-09-18T11:30:55.710 INFO:tasks.rados.rados.0.smithi119.stdout:1074: done (3 left) 2024-09-18T11:30:55.710 INFO:tasks.rados.rados.0.smithi119.stdout:1075: done (2 left) 2024-09-18T11:30:55.711 INFO:tasks.rados.rados.0.smithi119.stdout:1078: read oid 17 snap 14 2024-09-18T11:30:55.711 INFO:tasks.rados.rados.0.smithi119.stdout:1078: expect (ObjNum 119 snap 11 seq_num 119) 2024-09-18T11:30:55.711 INFO:tasks.rados.rados.0.smithi119.stdout:1079: read oid 23 snap 12 2024-09-18T11:30:55.711 INFO:tasks.rados.rados.0.smithi119.stdout:1079: expect (ObjNum 90 snap 5 seq_num 90) 2024-09-18T11:30:55.711 INFO:tasks.rados.rados.0.smithi119.stdout:1080: delete oid 5 current snap is 20 2024-09-18T11:30:55.711 INFO:tasks.rados.rados.0.smithi119.stdout:1076: got (22) Invalid argument 2024-09-18T11:30:55.711 INFO:tasks.rados.rados.0.smithi119.stdout:1077: finishing write tid 1 to smithi11939506-22 2024-09-18T11:30:55.711 INFO:tasks.rados.rados.0.smithi119.stdout:1077: finishing write tid 2 to smithi11939506-22 2024-09-18T11:30:55.713 INFO:tasks.rados.rados.0.smithi119.stdout:1077: finishing write tid 3 to smithi11939506-22 2024-09-18T11:30:55.713 INFO:tasks.rados.rados.0.smithi119.stdout:1076: done (4 left) 2024-09-18T11:30:55.713 INFO:tasks.rados.rados.0.smithi119.stdout:1080: done (3 left) 2024-09-18T11:30:55.713 INFO:tasks.rados.rados.0.smithi119.stdout:1081: set_chunk oid 33 target oid 2024-09-18T11:30:55.713 INFO:tasks.rados.rados.0.smithi119.stdout:1081: set_chunk oid 33 offset: 977408 length: 1458176 target oid smithi11939506- offset: 977408 snap 15 2024-09-18T11:30:55.713 INFO:tasks.rados.rados.0.smithi119.stdout:1082: rollback oid 47 current snap is 20 2024-09-18T11:30:55.713 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 47 to 15 2024-09-18T11:30:55.714 INFO:tasks.rados.rados.0.smithi119.stdout:1077: finishing write tid 4 to smithi11939506-22 2024-09-18T11:30:55.714 INFO:tasks.rados.rados.0.smithi119.stdout:1083: write oid 50 current snap is 20 2024-09-18T11:30:55.714 INFO:tasks.rados.rados.0.smithi119.stdout:1083: seq_num 183 ranges {675681=563039,1858793=746713,3025937=49744} 2024-09-18T11:30:55.719 INFO:tasks.rados.rados.0.smithi119.stdout:1083: writing smithi11939506-50 from 675681 to 1238720 tid 1 2024-09-18T11:30:55.726 INFO:tasks.rados.rados.0.smithi119.stdout:1083: writing smithi11939506-50 from 1858793 to 2605506 tid 2 2024-09-18T11:30:55.729 INFO:tasks.rados.rados.0.smithi119.stdout:1083: writing smithi11939506-50 from 3025937 to 3075681 tid 3 2024-09-18T11:30:55.729 INFO:tasks.rados.rados.0.smithi119.stdout:1084: set_chunk oid 4 target oid 2024-09-18T11:30:55.738 INFO:tasks.rados.rados.0.smithi119.stdout:1084: set_chunk oid 4 offset: 2265600 length: 855552 target oid 95e762d743c69e6efaefe92d905ac94b659d7d90631d34d4368df67f56c10c79 offset: 0 snap -1 2024-09-18T11:30:55.738 INFO:tasks.rados.rados.0.smithi119.stdout:1085: read oid 8 snap -1 2024-09-18T11:30:55.738 INFO:tasks.rados.rados.0.smithi119.stdout:1085: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-18T11:30:55.738 INFO:tasks.rados.rados.0.smithi119.stdout:1086: write (excl) oid 3 current snap is 20 2024-09-18T11:30:55.738 INFO:tasks.rados.rados.0.smithi119.stdout:1077: finishing write tid 5 to smithi11939506-22 2024-09-18T11:30:55.738 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 92 (ObjNum 181 snap 20 seq_num 181) dirty exists 2024-09-18T11:30:55.738 INFO:tasks.rados.rados.0.smithi119.stdout:1077: left oid 22 (ObjNum 181 snap 20 seq_num 181) 2024-09-18T11:30:55.738 INFO:tasks.rados.rados.0.smithi119.stdout:1086: seq_num 184 ranges {692488=638288,1917275=606724} 2024-09-18T11:30:55.739 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:55.735+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:55.745 INFO:tasks.rados.rados.0.smithi119.stdout:1086: writing smithi11939506-3 from 692488 to 1330776 tid 1 2024-09-18T11:30:55.752 INFO:tasks.rados.rados.0.smithi119.stdout:1086: writing smithi11939506-3 from 1917275 to 2523999 tid 2 2024-09-18T11:30:55.752 INFO:tasks.rados.rados.0.smithi119.stdout:1077: done (8 left) 2024-09-18T11:30:55.752 INFO:tasks.rados.rados.0.smithi119.stdout:1087: read oid 46 snap -1 2024-09-18T11:30:55.752 INFO:tasks.rados.rados.0.smithi119.stdout:1087: expect (ObjNum 171 snap 20 seq_num 171) 2024-09-18T11:30:55.753 INFO:tasks.rados.rados.0.smithi119.stdout:1088: read oid 25 snap 12 2024-09-18T11:30:55.753 INFO:tasks.rados.rados.0.smithi119.stdout:1088: expect (ObjNum 24 snap 0 seq_num 24) 2024-09-18T11:30:55.753 INFO:tasks.rados.rados.0.smithi119.stdout:1089: write oid 49 current snap is 20 2024-09-18T11:30:55.753 INFO:tasks.rados.rados.0.smithi119.stdout:1081: finishing set_chunk to oid 33 2024-09-18T11:30:55.753 INFO:tasks.rados.rados.0.smithi119.stdout:1081: got expected ENOENT (src dne) 2024-09-18T11:30:55.753 INFO:tasks.rados.rados.0.smithi119.stdout:1078: expect (ObjNum 119 snap 11 seq_num 119) 2024-09-18T11:30:55.757 INFO:teuthology.orchestra.run.smithi119.stdout:get_pg_num_history pg_num_history pg_num_history(e65 pg_nums {1={14=1},2={15=8},3={21=4},4={22=16}} deleted_pools ) 2024-09-18T11:30:55.757 INFO:teuthology.orchestra.run.smithi119.stdout:Importing pgid 3.1 2024-09-18T11:30:55.760 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:822c82fd:::16c9af8752ce95120a507ec000947859dae1395d7d6d118e597b9892dcd8465e:head# 2024-09-18T11:30:55.761 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.762 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:83e57124:::3697192fc171029dd257b30a7532fa2228ae20837ed34e739f2a2edcc34a61c7:head# 2024-09-18T11:30:55.762 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.762 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:84219092:::smithi11939506-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head# 2024-09-18T11:30:55.763 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.765 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:854f581f:::9fb14befb08f54939f2c71c890ba0ea89332fb5c1fd9e0b07290d878c4431425:head# 2024-09-18T11:30:55.766 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.766 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:8613c1ed:::2d6b732d315f6106b1a30740b78802c0014a2c0700a0c6eb7d53c5b00ebff73b:head# 2024-09-18T11:30:55.766 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.768 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:8abdb616:::smithi11939506-46 low_tier:head# 2024-09-18T11:30:55.768 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.771 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:8b69ac22:::1cf2f7c6a3cb930317737d5f25c48971fd9e8dcf6ceeadb55a7d6ea9967ce068:head# 2024-09-18T11:30:55.771 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.771 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:8d296107:::smithi11939506-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head# 2024-09-18T11:30:55.772 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.776 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:9201bbaf:::ac47b8623647b4cd1851128a75d93065bd2a2122c8cd6774e7ae0a9937ae73a7:head# 2024-09-18T11:30:55.777 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.780 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:94708c4c:::888f815178b2a748104fffa559dcb609f143f519d8737c6aa160c9cce13a6441:head# 2024-09-18T11:30:55.780 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.780 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:9668061d:::smithi11939506-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head# 2024-09-18T11:30:55.780 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.782 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:97025964:::d85d35018f93ce704e0c86e65d6c262758d2b38883b05fc1a96898c5a961f23a:head# 2024-09-18T11:30:55.782 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.784 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:99e1d655:::d01ff5a07a68e5351ca985a6c57ddc5ef0d05d539368179784aa67dbeae8d271:head# 2024-09-18T11:30:55.784 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.784 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:55.784 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:55.784 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:30:55.785 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:55.785 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:9f1cb8cf:::smithi11939506-38 low_tier:head# 2024-09-18T11:30:55.786 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.787 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:a0497df5:::smithi11939506-14 low_tier:head# 2024-09-18T11:30:55.788 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.790 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:a17fb6fd:::be0721f3a30df04624eb43dc58fbbc00d7ea210014325fb73bbc8d75e46bcb7d:head# 2024-09-18T11:30:55.790 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.793 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:a3a1203d:::c2b3fc9c1ba640cc8bcddb60552d5559ebeb3b68ed444eb13795808d4ae6858d:head# 2024-09-18T11:30:55.793 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.793 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:30:55.794 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:a3da18a4:::smithi11939506-28 low_tier:head# 2024-09-18T11:30:55.794 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.796 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:a8c4fc97:::45274de591d4198ebdf7025d46b87dfab949a0114f7ca760f95c9bdc5f783864:head# 2024-09-18T11:30:55.796 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.801 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:abad5243:::554b36e1a8dfdc0f27ed2d1e3c014381bf16adc7985c58ddbe8e14169aa70c76:head# 2024-09-18T11:30:55.801 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.801 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:b03ed61d:::974d9437087c20cb4e3ff5922dab38768579714a2c9ce6904cfb47136071b991:head# 2024-09-18T11:30:55.801 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.801 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:b10e849c:::a705ccfdb1cdc6ef082e4405c7b5f3c792a2b27cdec018193178c38e5ec259a4:head# 2024-09-18T11:30:55.801 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.801 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:b6a90588:::smithi11939506-2 low_tier:head# 2024-09-18T11:30:55.801 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.802 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:b900d23f:::smithi11939506-24 low_tier:head# 2024-09-18T11:30:55.802 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.803 INFO:teuthology.orchestra.run.smithi119.stdout:Write #3:bf804683:::e58794d0a579007c3bd048992c5646e06b9775f9667464eee6a2a0508e23a0a6:head# 2024-09-18T11:30:55.803 INFO:teuthology.orchestra.run.smithi119.stdout:snapset 0=[]:{} 2024-09-18T11:30:55.805 INFO:teuthology.orchestra.run.smithi119.stdout:write_pg epoch 62 info 3.1( v 62'101 (0'0,62'101] local-lis/les=21/22 n=25 ec=21/21 lis/c=21/21 les/c/f=22/22/0 sis=21) 2024-09-18T11:30:55.806 INFO:teuthology.orchestra.run.smithi119.stdout:Import successful 2024-09-18T11:30:55.836 INFO:tasks.rados.rados.0.smithi119.stdout:1082: finishing rollback tid 0 to smithi11939506-47 2024-09-18T11:30:55.836 INFO:tasks.rados.rados.0.smithi119.stdout:1082: finishing rollback tid 1 to smithi11939506-47 2024-09-18T11:30:55.836 INFO:tasks.rados.rados.0.smithi119.stdout:1082: finishing rollback tid 2 to smithi11939506-47 2024-09-18T11:30:55.837 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 78 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-18T11:30:55.837 INFO:tasks.rados.rados.0.smithi119.stdout:1083: finishing write tid 1 to smithi11939506-50 2024-09-18T11:30:55.837 INFO:tasks.rados.rados.0.smithi119.stdout:1079: expect (ObjNum 90 snap 5 seq_num 90) 2024-09-18T11:30:55.838 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:55.836+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:55.891 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:55.891 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:30:55.891 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:30:55.892 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:30:55.892 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:55.900 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:30:55.943 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:55.942+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:55.999 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:30:56.000 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:30:56.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:30:56.000 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:30:56.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.000 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:56.000 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:56.000 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.839980534000006, 2024-09-18T11:30:56.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054864400000000004, 2024-09-18T11:30:56.000 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:56.000 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.000 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:56.000 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:56.000 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:56.001 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 42 2024-09-18T11:30:56.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.001 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:56.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:56.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:56.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:56.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:56.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:56.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:30:56.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:56.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157416+0000", 2024-09-18T11:30:56.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.721e-06 2024-09-18T11:30:56.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:56.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157423+0000", 2024-09-18T11:30:56.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4049999999999997e-06 2024-09-18T11:30:56.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157425+0000", 2024-09-18T11:30:56.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.481e-06 2024-09-18T11:30:56.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:56.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158649+0000", 2024-09-18T11:30:56.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001223453 2024-09-18T11:30:56.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:56.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158729+0000", 2024-09-18T11:30:56.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0205999999999997e-05 2024-09-18T11:30:56.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:56.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158875+0000", 2024-09-18T11:30:56.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014628800000000001 2024-09-18T11:30:56.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:56.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161518+0000", 2024-09-18T11:30:56.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026427949999999999 2024-09-18T11:30:56.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162821+0000", 2024-09-18T11:30:56.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013028099999999999 2024-09-18T11:30:56.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:56.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162864+0000", 2024-09-18T11:30:56.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2839000000000001e-05 2024-09-18T11:30:56.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:56.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162900+0000", 2024-09-18T11:30:56.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5948999999999998e-05 2024-09-18T11:30:56.009 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.009 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:56.009 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.009 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:56.009 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:56.009 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.839894169999994, 2024-09-18T11:30:56.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055400850000000001, 2024-09-18T11:30:56.009 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:56.009 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.009 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:56.009 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:56.010 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:56.010 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 43 2024-09-18T11:30:56.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.010 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:56.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:56.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:56.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:56.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:56.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:56.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:30:56.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:30:56.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:56.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157504+0000", 2024-09-18T11:30:56.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4519999999999996e-06 2024-09-18T11:30:56.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:56.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157534+0000", 2024-09-18T11:30:56.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.97e-05 2024-09-18T11:30:56.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157539+0000", 2024-09-18T11:30:56.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4360000000000001e-06 2024-09-18T11:30:56.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:56.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159428+0000", 2024-09-18T11:30:56.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018888220000000001 2024-09-18T11:30:56.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:56.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159479+0000", 2024-09-18T11:30:56.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0608000000000001e-05 2024-09-18T11:30:56.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:56.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159559+0000", 2024-09-18T11:30:56.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0667000000000005e-05 2024-09-18T11:30:56.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:56.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162779+0000", 2024-09-18T11:30:56.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032197459999999999 2024-09-18T11:30:56.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162950+0000", 2024-09-18T11:30:56.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017059599999999999 2024-09-18T11:30:56.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:56.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163001+0000", 2024-09-18T11:30:56.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0968999999999999e-05 2024-09-18T11:30:56.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:56.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163040+0000", 2024-09-18T11:30:56.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8843000000000001e-05 2024-09-18T11:30:56.015 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.015 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:56.015 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.015 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:56.015 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:56.015 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.839843016000003, 2024-09-18T11:30:56.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:30:56.015 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:56.015 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.015 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:56.016 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:56.016 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:56.016 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:30:56.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.016 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:56.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:56.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:56.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:56.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:30:56.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:56.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:30:56.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:56.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:30:56.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:30:56.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:56.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:30:56.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:30:56.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:30:56.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:30:56.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:56.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:30:56.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:30:56.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:56.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:30:56.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:30:56.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:56.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:30:56.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:30:56.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:56.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:30:56.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:30:56.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:30:56.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:30:56.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:56.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:30:56.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:30:56.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:56.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:30:56.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:30:56.021 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.021 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:56.021 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.021 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:56.021 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:56.021 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.839716753999994, 2024-09-18T11:30:56.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055717040000000002, 2024-09-18T11:30:56.021 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:56.021 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.022 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:56.022 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:56.022 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:56.022 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 45 2024-09-18T11:30:56.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.022 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:56.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.022 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:56.022 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:56.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.022 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:56.022 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:30:56.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:56.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157676+0000", 2024-09-18T11:30:56.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:56.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157694+0000", 2024-09-18T11:30:56.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7422999999999999e-05 2024-09-18T11:30:56.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:56.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157704+0000", 2024-09-18T11:30:56.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9000000000000001e-06 2024-09-18T11:30:56.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157718+0000", 2024-09-18T11:30:56.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4788e-05 2024-09-18T11:30:56.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:56.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160937+0000", 2024-09-18T11:30:56.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00321886 2024-09-18T11:30:56.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:56.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161023+0000", 2024-09-18T11:30:56.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6063999999999995e-05 2024-09-18T11:30:56.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:56.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161202+0000", 2024-09-18T11:30:56.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017912599999999999 2024-09-18T11:30:56.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:56.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163052+0000", 2024-09-18T11:30:56.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001849554 2024-09-18T11:30:56.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163204+0000", 2024-09-18T11:30:56.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015186900000000001 2024-09-18T11:30:56.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:56.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163229+0000", 2024-09-18T11:30:56.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:30:56.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:56.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163249+0000", 2024-09-18T11:30:56.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9992000000000001e-05 2024-09-18T11:30:56.027 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.027 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:56.027 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.027 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:56.027 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:56.027 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.839667641000005, 2024-09-18T11:30:56.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:30:56.027 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:56.027 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.027 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:56.028 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:56.028 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:56.028 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:30:56.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.028 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:56.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:56.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:56.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:56.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:56.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:56.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:30:56.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:56.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:30:56.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:30:56.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:56.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:30:56.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:30:56.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:30:56.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:30:56.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:56.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:30:56.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:30:56.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:56.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:30:56.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:30:56.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:56.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:30:56.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:30:56.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:56.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:30:56.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:30:56.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.032 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:30:56.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:30:56.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.032 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:56.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:30:56.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:30:56.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.032 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:56.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:30:56.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:30:56.033 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.033 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:56.033 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.033 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:56.033 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:56.033 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.83962511, 2024-09-18T11:30:56.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:30:56.033 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:56.033 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.033 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:56.034 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:56.034 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:56.034 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:30:56.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.034 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:56.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:56.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:56.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:56.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:30:56.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:56.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:30:56.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:56.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:30:56.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:30:56.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:56.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:30:56.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:30:56.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.036 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:30:56.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:30:56.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.036 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:56.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:30:56.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:30:56.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:56.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:30:56.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:30:56.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:56.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:30:56.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:30:56.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:30:56.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:30:56.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:56.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:30:56.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:30:56.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:56.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:30:56.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:30:56.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:56.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:30:56.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:30:56.039 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.039 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:56.039 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.039 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:56.039 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:56.039 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.839570179999996, 2024-09-18T11:30:56.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:30:56.039 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:56.039 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.039 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:56.040 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:56.040 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:56.040 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:30:56.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.040 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:56.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:56.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:56.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:56.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:56.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:56.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:30:56.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:56.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:30:56.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:30:56.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:56.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:30:56.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:30:56.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:30:56.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:30:56.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:56.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:30:56.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:30:56.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.043 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:56.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:30:56.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:30:56.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.043 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:56.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:30:56.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:30:56.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.043 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:56.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:30:56.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:30:56.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.044 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.044 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:30:56.044 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:30:56.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.044 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:56.044 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:30:56.044 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:30:56.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.044 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:56.044 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:30:56.045 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:30:56.045 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.045 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:56.045 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.045 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:56.045 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:56.045 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.839560199999994, 2024-09-18T11:30:56.045 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:30:56.045 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:56.045 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.045 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:56.045 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:56.046 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:56.046 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:30:56.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.046 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:56.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.046 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:56.046 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:56.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.046 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:56.046 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:56.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.047 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:56.047 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:30:56.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:56.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.047 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:56.047 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:30:56.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:30:56.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.047 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:56.047 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:30:56.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:30:56.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.048 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.048 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:30:56.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:30:56.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.048 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:56.048 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:30:56.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:30:56.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.049 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:56.049 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:30:56.049 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:30:56.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.049 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:56.049 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:30:56.049 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:30:56.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.049 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.049 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:30:56.049 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:30:56.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.050 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:56.050 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:30:56.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:30:56.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.050 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:56.050 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:30:56.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:30:56.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.050 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:56.050 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:30:56.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:30:56.050 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.051 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:56.051 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.051 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:56.051 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:56.051 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.839548601000004, 2024-09-18T11:30:56.051 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:30:56.051 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:56.051 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.051 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:56.051 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:56.051 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:56.051 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:30:56.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.052 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:56.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.052 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:56.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:56.052 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.052 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:56.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:56.052 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.052 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:56.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:30:56.053 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:56.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.053 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:56.053 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:30:56.053 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:30:56.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.053 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:56.053 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:30:56.053 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:30:56.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.053 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.054 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:30:56.054 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:30:56.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.054 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:56.054 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:30:56.054 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:30:56.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.054 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:56.054 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:30:56.054 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:30:56.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:56.055 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:30:56.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:30:56.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:56.055 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:30:56.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:30:56.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.055 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:30:56.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:30:56.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.056 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:56.056 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:30:56.056 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:30:56.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.056 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:56.056 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:30:56.056 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:30:56.056 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.056 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:56.056 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.057 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:56.057 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:56.057 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.839492120000003, 2024-09-18T11:30:56.057 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:30:56.057 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:56.057 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.057 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:56.057 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:56.057 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:56.057 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:30:56.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.058 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:56.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.058 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:56.058 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:56.058 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.058 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.058 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:56.058 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:30:56.058 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.058 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.058 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:56.058 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:30:56.059 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.059 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.059 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.059 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:56.059 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:30:56.059 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:30:56.059 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.059 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.059 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:56.059 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:30:56.059 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:30:56.059 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.059 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.060 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.060 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:30:56.060 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:30:56.060 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.060 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.060 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:56.060 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:30:56.060 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:30:56.060 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.060 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.060 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:56.060 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:30:56.060 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:30:56.061 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.061 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.061 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:56.061 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:30:56.061 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:30:56.061 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.061 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.061 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.061 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:30:56.061 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:30:56.061 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.061 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.061 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:56.061 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:30:56.062 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:30:56.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.062 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.062 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:56.062 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:30:56.062 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:30:56.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.062 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.062 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:56.062 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:30:56.062 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:30:56.062 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.062 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:56.062 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.063 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.063 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.063 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:56.063 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:56.063 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.839478295999996, 2024-09-18T11:30:56.063 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:30:56.063 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:56.063 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.063 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:56.063 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:56.063 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:56.063 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:30:56.063 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.064 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:56.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.064 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:56.064 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:56.064 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.064 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.064 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:56.064 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:56.064 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.064 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.064 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:56.064 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:30:56.065 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:56.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.065 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.065 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:56.065 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:30:56.065 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:30:56.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.065 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.065 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:56.065 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:30:56.065 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:30:56.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.065 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.066 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.066 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:30:56.066 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:30:56.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.066 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:56.066 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:30:56.066 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:30:56.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.066 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:56.066 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:30:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:30:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:30:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:30:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:30:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:30:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:30:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:30:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:30:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:30:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:30:56.068 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:30:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.423299388999993, 2024-09-18T11:30:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:30:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:56.069 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:30:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:30:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:30:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:30:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:30:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:30:56.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:30:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:30:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:30:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:30:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:30:56.072 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:30:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:30:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:30:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:30:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:30:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.073 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:30:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:30:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:30:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:30:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:30:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:30:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:56.074 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.423276088999998, 2024-09-18T11:30:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:30:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:30:56.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:30:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:56.076 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:30:56.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.077 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:56.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:30:56.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:30:56.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.077 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:56.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:30:56.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:30:56.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:30:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:30:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:30:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:30:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:30:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:30:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.079 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:56.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:30:56.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:30:56.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.079 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:56.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:30:56.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:30:56.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.079 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:30:56.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:30:56.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.079 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:56.080 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:30:56.080 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:30:56.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:56.080 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:30:56.080 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:30:56.080 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.080 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:56.080 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.080 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:56.080 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:56.081 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.423190480000002, 2024-09-18T11:30:56.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:30:56.081 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:56.081 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.081 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:56.081 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:56.081 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:56.081 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:30:56.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.081 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:56.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.081 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:56.081 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:56.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.082 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:56.082 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:56.082 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.082 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:56.082 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:30:56.082 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.082 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:56.082 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:30:56.082 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:30:56.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.083 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:56.083 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:30:56.083 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:30:56.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.083 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.083 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:30:56.083 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:30:56.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.083 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:56.083 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:30:56.084 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:30:56.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.084 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:56.084 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:30:56.084 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:30:56.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.084 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:56.084 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:30:56.084 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:30:56.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.084 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:56.084 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:30:56.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:30:56.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:30:56.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:30:56.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:56.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:30:56.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:30:56.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:56.086 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:30:56.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:30:56.086 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.086 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:56.086 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.086 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:56.086 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:56.086 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.423179515000001, 2024-09-18T11:30:56.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:30:56.086 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:56.086 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.086 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:56.087 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:56.087 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:56.087 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:30:56.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.087 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:56.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.087 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:56.087 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:56.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.088 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:56.088 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:56.088 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.088 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:56.088 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:30:56.088 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:56.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.088 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:56.088 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:30:56.089 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:30:56.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.089 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:56.089 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:30:56.089 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:30:56.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.089 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.089 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:30:56.089 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:30:56.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.089 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:56.089 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:30:56.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:30:56.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:56.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:30:56.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:30:56.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:56.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:30:56.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:30:56.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:56.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:30:56.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:30:56.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:30:56.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:30:56.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:56.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:30:56.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:30:56.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:56.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:30:56.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:30:56.092 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.092 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:56.092 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.092 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:56.092 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:56.092 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.423168347000001, 2024-09-18T11:30:56.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:30:56.092 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:56.092 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.092 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:56.092 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:56.092 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:56.093 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:30:56.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.093 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:56.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.093 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:56.093 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:56.093 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.093 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:56.093 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:56.093 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.094 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:56.094 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:30:56.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:30:56.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.094 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:56.094 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:30:56.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:30:56.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.094 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:56.094 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:30:56.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:30:56.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.094 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.095 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:30:56.095 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:30:56.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.095 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:56.095 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:30:56.095 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:30:56.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.095 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:56.095 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:30:56.095 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:30:56.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:56.096 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:30:56.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:30:56.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.096 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:30:56.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:30:56.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:56.096 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:30:56.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:30:56.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:56.097 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:30:56.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:30:56.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:56.097 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:30:56.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:30:56.097 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.097 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:56.097 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.097 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:56.098 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:56.098 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.423103018000006, 2024-09-18T11:30:56.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:30:56.098 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:56.098 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.098 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:56.098 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:56.098 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:56.098 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:30:56.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.098 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:56.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.098 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:56.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:56.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:56.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:30:56.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:56.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:30:56.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:56.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:30:56.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:30:56.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:56.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:30:56.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:30:56.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:30:56.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:30:56.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:56.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:30:56.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:30:56.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.101 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:56.101 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:30:56.101 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:30:56.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.101 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:56.101 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:30:56.101 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:30:56.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.101 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.101 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:30:56.101 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:30:56.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.102 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:56.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:30:56.102 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:30:56.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.102 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:56.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:30:56.102 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:30:56.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.102 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:56.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:30:56.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:30:56.103 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.103 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:56.103 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.103 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:56.103 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:56.103 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.412941040000007, 2024-09-18T11:30:56.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:30:56.103 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:56.103 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.103 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:56.103 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:56.103 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:56.104 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:30:56.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.104 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:56.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:56.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:56.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:56.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:30:56.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:56.105 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:30:56.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.105 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:56.105 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:30:56.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:30:56.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.105 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:56.105 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:30:56.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:30:56.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:30:56.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:30:56.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:56.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:30:56.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:30:56.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:56.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:30:56.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:30:56.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:56.107 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:30:56.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:30:56.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:56.107 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:30:56.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:30:56.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.107 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:30:56.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:30:56.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.108 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:56.108 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:30:56.108 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:30:56.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.108 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:56.108 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:30:56.108 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:30:56.108 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.108 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:56.108 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.108 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:30:56.109 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:56.109 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.412848561000004, 2024-09-18T11:30:56.109 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:30:56.109 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:56.109 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.109 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:30:56.109 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:30:56.109 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:30:56.109 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:30:56.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.109 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:56.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.109 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:56.109 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:56.109 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.110 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:56.110 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:56.110 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.110 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:56.110 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:30:56.110 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.110 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:56.110 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:30:56.111 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:30:56.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.111 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:56.111 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:30:56.111 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:30:56.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.111 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.111 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:30:56.111 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:30:56.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.112 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:56.112 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:30:56.112 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:30:56.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.112 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:56.112 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:30:56.112 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:30:56.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.112 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:30:56.112 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:30:56.112 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:30:56.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.113 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:30:56.113 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:30:56.113 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:30:56.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.113 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.113 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:30:56.113 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:30:56.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.113 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:56.114 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:30:56.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:30:56.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.114 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:56.114 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:30:56.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:30:56.114 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.114 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:56.114 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.114 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1788 4.8 e60/22 4:1898b15d:::smithi11939506-25:head v 60'12, mlcod=58'9)", 2024-09-18T11:30:56.114 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:30:56.115 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.080443072, 2024-09-18T11:30:56.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058348599999999999, 2024-09-18T11:30:56.115 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:30:56.115 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.115 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:30:56.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:30:56.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:30:56.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:30:56.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:30:56.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:30:56.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:30:56.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916949+0000", 2024-09-18T11:30:56.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:56.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:30:56.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917602+0000", 2024-09-18T11:30:56.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00065337100000000003 2024-09-18T11:30:56.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:30:56.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917621+0000", 2024-09-18T11:30:56.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9287e-05 2024-09-18T11:30:56.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917629+0000", 2024-09-18T11:30:56.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3780000000000003e-06 2024-09-18T11:30:56.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:30:56.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917677+0000", 2024-09-18T11:30:56.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8068000000000001e-05 2024-09-18T11:30:56.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:30:56.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917699+0000", 2024-09-18T11:30:56.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2149000000000001e-05 2024-09-18T11:30:56.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.118 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:30:56.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922747+0000", 2024-09-18T11:30:56.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050477609999999996 2024-09-18T11:30:56.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:30:56.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:30:56.118 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:30:56.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922785+0000", 2024-09-18T11:30:56.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8726000000000002e-05 2024-09-18T11:30:56.118 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.118 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:56.118 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.118 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:30:56.118 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:30:56.118 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:30:56.119 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T11:30:56.121 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:30:56.121 INFO:tasks.rados.rados.0.smithi119.stdout:1083: finishing write tid 2 to smithi11939506-50 2024-09-18T11:30:56.121 INFO:tasks.rados.rados.0.smithi119.stdout:1083: finishing write tid 3 to smithi11939506-50 2024-09-18T11:30:56.122 INFO:tasks.rados.rados.0.smithi119.stdout:1083: finishing write tid 4 to smithi11939506-50 2024-09-18T11:30:56.122 INFO:tasks.rados.rados.0.smithi119.stdout:1083: finishing write tid 5 to smithi11939506-50 2024-09-18T11:30:56.122 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 98 (ObjNum 183 snap 20 seq_num 183) dirty exists 2024-09-18T11:30:56.122 INFO:tasks.rados.rados.0.smithi119.stdout:1083: left oid 50 (ObjNum 183 snap 20 seq_num 183) 2024-09-18T11:30:56.122 INFO:tasks.rados.rados.0.smithi119.stdout:1089: seq_num 185 ranges {709295=713537,1859404=750382} 2024-09-18T11:30:56.122 INFO:tasks.rados.rados.0.smithi119.stdout:1089: writing smithi11939506-49 from 709295 to 1422832 tid 1 2024-09-18T11:30:56.122 INFO:tasks.rados.rados.0.smithi119.stdout:1089: writing smithi11939506-49 from 1859404 to 2609786 tid 2 2024-09-18T11:30:56.122 INFO:tasks.rados.rados.0.smithi119.stdout:1078: done (10 left) 2024-09-18T11:30:56.122 INFO:tasks.rados.rados.0.smithi119.stdout:1079: done (9 left) 2024-09-18T11:30:56.122 INFO:tasks.rados.rados.0.smithi119.stdout:1081: done (8 left) 2024-09-18T11:30:56.122 INFO:tasks.rados.rados.0.smithi119.stdout:1082: done (7 left) 2024-09-18T11:30:56.122 INFO:tasks.rados.rados.0.smithi119.stdout:1083: done (6 left) 2024-09-18T11:30:56.122 INFO:tasks.rados.rados.0.smithi119.stdout:1090: tier_flush oid 7 2024-09-18T11:30:56.122 INFO:tasks.rados.rados.0.smithi119.stdout:1090: tier_flush oid 7 snap -1 2024-09-18T11:30:56.122 INFO:tasks.rados.rados.0.smithi119.stdout:1084: finishing set_chunk to oid 4 2024-09-18T11:30:56.123 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 4 set_chunk returned error code -95 offset: 2265600 length: 855552 2024-09-18T11:30:56.123 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 4 v 0 (ObjNum 110 snap 8 seq_num 110) dirty exists 2024-09-18T11:30:56.123 INFO:tasks.rados.rados.0.smithi119.stdout:1085: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-18T11:30:56.123 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:56.118+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:56.185 INFO:tasks.rados.rados.0.smithi119.stdout:1086: finishing write tid 1 to smithi11939506-3 2024-09-18T11:30:56.185 INFO:tasks.rados.rados.0.smithi119.stdout:1086: finishing write tid 2 to smithi11939506-3 2024-09-18T11:30:56.185 INFO:tasks.rados.rados.0.smithi119.stdout:1086: finishing write tid 3 to smithi11939506-3 2024-09-18T11:30:56.185 INFO:tasks.rados.rados.0.smithi119.stdout:1086: finishing write tid 4 to smithi11939506-3 2024-09-18T11:30:56.185 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 88 (ObjNum 184 snap 20 seq_num 184) dirty exists 2024-09-18T11:30:56.185 INFO:tasks.rados.rados.0.smithi119.stdout:1086: left oid 3 (ObjNum 184 snap 20 seq_num 184) 2024-09-18T11:30:56.185 INFO:tasks.rados.rados.0.smithi119.stdout:1088: expect (ObjNum 24 snap 0 seq_num 24) 2024-09-18T11:30:56.222 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:56.219+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:56.223 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:56.224 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:56.224 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:56.224 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:56.233 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:30:56.263 INFO:tasks.rados.rados.0.smithi119.stdout:1087: expect (ObjNum 171 snap 20 seq_num 171) 2024-09-18T11:30:56.322 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f /var/log/ceph/exp.3.1.0 2024-09-18T11:30:56.323 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:56.321+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:56.353 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T11:30:56.365 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:56.365 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:56.365 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:56.365 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:56.365 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:56.374 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:30:56.391 INFO:tasks.rados.rados.0.smithi119.stdout:1089: finishing write tid 1 to smithi11939506-49 2024-09-18T11:30:56.392 INFO:tasks.rados.rados.0.smithi119.stdout:1090: got (16) Device or resource busy 2024-09-18T11:30:56.392 INFO:tasks.rados.rados.0.smithi119.stdout:check_oldest_snap_flushed oid 7 v 37 (ObjNum 145 snap 18 seq_num 145) unflushed 19 2024-09-18T11:30:56.392 INFO:tasks.rados.rados.0.smithi119.stdout:1089: finishing write tid 2 to smithi11939506-49 2024-09-18T11:30:56.392 INFO:tasks.rados.rados.0.smithi119.stdout:1084: done (6 left) 2024-09-18T11:30:56.392 INFO:tasks.rados.rados.0.smithi119.stdout:1085: done (5 left) 2024-09-18T11:30:56.392 INFO:tasks.rados.rados.0.smithi119.stdout:1086: done (4 left) 2024-09-18T11:30:56.392 INFO:tasks.rados.rados.0.smithi119.stdout:1087: done (3 left) 2024-09-18T11:30:56.392 INFO:tasks.rados.rados.0.smithi119.stdout:1088: done (2 left) 2024-09-18T11:30:56.392 INFO:tasks.rados.rados.0.smithi119.stdout:1090: done (1 left) 2024-09-18T11:30:56.392 INFO:tasks.rados.rados.0.smithi119.stdout:1091: read oid 2 snap 19 2024-09-18T11:30:56.393 INFO:tasks.rados.rados.0.smithi119.stdout:1091: expect (ObjNum 131 snap 15 seq_num 131) 2024-09-18T11:30:56.393 INFO:tasks.rados.rados.0.smithi119.stdout:1092: read oid 42 snap -1 2024-09-18T11:30:56.393 INFO:tasks.rados.rados.0.smithi119.stdout:1092: expect (ObjNum 99 snap 6 seq_num 99) 2024-09-18T11:30:56.393 INFO:tasks.rados.rados.0.smithi119.stdout:1093: set_chunk oid 48 target oid 2024-09-18T11:30:56.402 INFO:tasks.rados.rados.0.smithi119.stdout:1093: set_chunk oid 48 offset: 1233920 length: 930816 target oid 23f6869ebe66364e2b8d056858f8baa81868616d028945ea2249332d157a259c offset: 0 snap 17 2024-09-18T11:30:56.402 INFO:tasks.rados.rados.0.smithi119.stdout:1094: write oid 1 current snap is 20 2024-09-18T11:30:56.402 INFO:tasks.rados.rados.0.smithi119.stdout:1089: finishing write tid 3 to smithi11939506-49 2024-09-18T11:30:56.402 INFO:tasks.rados.rados.0.smithi119.stdout:1089: finishing write tid 4 to smithi11939506-49 2024-09-18T11:30:56.402 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 166 (ObjNum 185 snap 20 seq_num 185) dirty exists 2024-09-18T11:30:56.402 INFO:tasks.rados.rados.0.smithi119.stdout:1089: left oid 49 (ObjNum 185 snap 20 seq_num 185) 2024-09-18T11:30:56.402 INFO:tasks.rados.rados.0.smithi119.stdout:1094: seq_num 186 ranges {726102=788786,1917886=610393,3108206=17896} 2024-09-18T11:30:56.411 INFO:tasks.rados.rados.0.smithi119.stdout:1094: writing smithi11939506-1 from 726102 to 1514888 tid 1 2024-09-18T11:30:56.417 INFO:tasks.rados.rados.0.smithi119.stdout:1094: writing smithi11939506-1 from 1917886 to 2528279 tid 2 2024-09-18T11:30:56.420 INFO:tasks.rados.rados.0.smithi119.stdout:1094: writing smithi11939506-1 from 3108206 to 3126102 tid 3 2024-09-18T11:30:56.420 INFO:tasks.rados.rados.0.smithi119.stdout:1089: done (4 left) 2024-09-18T11:30:56.420 INFO:tasks.rados.rados.0.smithi119.stdout:1095: write (excl) oid 6 current snap is 20 2024-09-18T11:30:56.420 INFO:tasks.rados.rados.0.smithi119.stdout:1095: seq_num 187 ranges {742909=464035,1976368=754051} 2024-09-18T11:30:56.422 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:56.421+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:56.426 INFO:tasks.rados.rados.0.smithi119.stdout:1095: writing smithi11939506-6 from 742909 to 1206944 tid 1 2024-09-18T11:30:56.435 INFO:tasks.rados.rados.0.smithi119.stdout:1095: writing smithi11939506-6 from 1976368 to 2730419 tid 2 2024-09-18T11:30:56.435 INFO:tasks.rados.rados.0.smithi119.stdout:1096: read oid 38 snap 19 2024-09-18T11:30:56.435 INFO:tasks.rados.rados.0.smithi119.stdout:1096: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-18T11:30:56.435 INFO:tasks.rados.rados.0.smithi119.stdout:1097: write (excl) oid 37 current snap is 20 2024-09-18T11:30:56.435 INFO:tasks.rados.rados.0.smithi119.stdout:1097: seq_num 188 ranges {759716=539284,2034850=614062} 2024-09-18T11:30:56.442 INFO:tasks.rados.rados.0.smithi119.stdout:1097: writing smithi11939506-37 from 759716 to 1299000 tid 1 2024-09-18T11:30:56.449 INFO:tasks.rados.rados.0.smithi119.stdout:1097: writing smithi11939506-37 from 2034850 to 2648912 tid 2 2024-09-18T11:30:56.449 INFO:tasks.rados.rados.0.smithi119.stdout:1098: tier_evict oid 34 2024-09-18T11:30:56.449 INFO:tasks.rados.rados.0.smithi119.stdout:1098: tier_evict oid 34 snap -1 2024-09-18T11:30:56.449 INFO:tasks.rados.rados.0.smithi119.stdout:1091: expect (ObjNum 131 snap 15 seq_num 131) 2024-09-18T11:30:56.506 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:56.506 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:56.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:56.506 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:56.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.507 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:887 4.7 4:e0d95e10:::smithi11939506-22:head [read 0~2352947,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:56.507 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:30:56.507 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.386993203000003, 2024-09-18T11:30:56.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079115169999999999, 2024-09-18T11:30:56.507 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:56.507 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:56.507 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:56.507 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:56.507 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:56.507 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 887 2024-09-18T11:30:56.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.507 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:56.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:56.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:30:56.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:56.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:30:56.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:56.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116365+0000", 2024-09-18T11:30:56.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:56.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:56.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116376+0000", 2024-09-18T11:30:56.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1471000000000001e-05 2024-09-18T11:30:56.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:56.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116394+0000", 2024-09-18T11:30:56.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8088e-05 2024-09-18T11:30:56.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116402+0000", 2024-09-18T11:30:56.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2320000000000004e-06 2024-09-18T11:30:56.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116458+0000", 2024-09-18T11:30:56.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6536000000000002e-05 2024-09-18T11:30:56.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:56.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116466+0000", 2024-09-18T11:30:56.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0819999999999999e-06 2024-09-18T11:30:56.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.122509+0000", 2024-09-18T11:30:56.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060431879999999997 2024-09-18T11:30:56.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:56.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.122577+0000", 2024-09-18T11:30:56.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7646999999999998e-05 2024-09-18T11:30:56.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:56.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124278+0000", 2024-09-18T11:30:56.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017013200000000001 2024-09-18T11:30:56.512 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.512 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:56.512 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.512 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:888 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:56.512 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:56.512 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.386939740000003, 2024-09-18T11:30:56.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082282120000000004, 2024-09-18T11:30:56.512 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:56.512 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:56.513 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:56.513 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:56.513 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:56.513 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 888 2024-09-18T11:30:56.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.513 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:56.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:56.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:56.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:56.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:56.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:56.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:30:56.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:56.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116423+0000", 2024-09-18T11:30:56.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1e-06 2024-09-18T11:30:56.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:56.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:30:56.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5510000000000001e-06 2024-09-18T11:30:56.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116431+0000", 2024-09-18T11:30:56.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6109999999999998e-06 2024-09-18T11:30:56.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116527+0000", 2024-09-18T11:30:56.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6212999999999995e-05 2024-09-18T11:30:56.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:56.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116532+0000", 2024-09-18T11:30:56.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6140000000000002e-06 2024-09-18T11:30:56.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124314+0000", 2024-09-18T11:30:56.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077818929999999998 2024-09-18T11:30:56.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:56.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124411+0000", 2024-09-18T11:30:56.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6992999999999998e-05 2024-09-18T11:30:56.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:56.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124649+0000", 2024-09-18T11:30:56.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237729 2024-09-18T11:30:56.517 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.517 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:56.517 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.517 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:889 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:56.517 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:56.517 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.386915090000002, 2024-09-18T11:30:56.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085383349999999993, 2024-09-18T11:30:56.517 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:56.518 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:56.518 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:56.518 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:56.518 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:56.518 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 889 2024-09-18T11:30:56.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.518 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:56.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:56.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:56.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:56.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:56.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:56.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:30:56.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:56.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116447+0000", 2024-09-18T11:30:56.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.854e-06 2024-09-18T11:30:56.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:56.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116452+0000", 2024-09-18T11:30:56.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0250000000000002e-06 2024-09-18T11:30:56.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116455+0000", 2024-09-18T11:30:56.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8739999999999999e-06 2024-09-18T11:30:56.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116565+0000", 2024-09-18T11:30:56.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110741 2024-09-18T11:30:56.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:56.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116571+0000", 2024-09-18T11:30:56.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4990000000000002e-06 2024-09-18T11:30:56.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124691+0000", 2024-09-18T11:30:56.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081200339999999999 2024-09-18T11:30:56.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:56.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124784+0000", 2024-09-18T11:30:56.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2886999999999993e-05 2024-09-18T11:30:56.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:56.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124983+0000", 2024-09-18T11:30:56.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019969000000000001 2024-09-18T11:30:56.522 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.522 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:56.522 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.522 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:46 4.a e31/22 4:552523b6:::smithi11939506-38:head v 31'20, mlcod=30'18)", 2024-09-18T11:30:56.522 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:30:56.522 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.372963458000001, 2024-09-18T11:30:56.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080208420000000002, 2024-09-18T11:30:56.522 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:56.523 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.523 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:56.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:56.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:30:56.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:56.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:30:56.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:56.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130396+0000", 2024-09-18T11:30:56.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:56.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133245+0000", 2024-09-18T11:30:56.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028486929999999998 2024-09-18T11:30:56.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:56.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133253+0000", 2024-09-18T11:30:56.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1540000000000002e-06 2024-09-18T11:30:56.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133261+0000", 2024-09-18T11:30:56.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3620000000000006e-06 2024-09-18T11:30:56.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133319+0000", 2024-09-18T11:30:56.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7719000000000001e-05 2024-09-18T11:30:56.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:56.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133341+0000", 2024-09-18T11:30:56.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1946999999999998e-05 2024-09-18T11:30:56.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:56.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138368+0000", 2024-09-18T11:30:56.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005026712 2024-09-18T11:30:56.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:56.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138418+0000", 2024-09-18T11:30:56.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9864000000000002e-05 2024-09-18T11:30:56.527 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.527 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:56.527 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.527 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1126 3.0 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head [write 0~2520064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:56.527 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:30:56.527 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.521473112000002, 2024-09-18T11:30:56.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086197050000000001, 2024-09-18T11:30:56.527 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:56.527 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.527 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:56.527 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:56.527 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:56.528 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1126 2024-09-18T11:30:56.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.528 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:56.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:56.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:30:56.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:56.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:30:56.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:56.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981886+0000", 2024-09-18T11:30:56.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:56.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:56.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983017+0000", 2024-09-18T11:30:56.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00113131 2024-09-18T11:30:56.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:56.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983037+0000", 2024-09-18T11:30:56.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0231e-05 2024-09-18T11:30:56.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983044+0000", 2024-09-18T11:30:56.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9539999999999998e-06 2024-09-18T11:30:56.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983087+0000", 2024-09-18T11:30:56.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3040000000000001e-05 2024-09-18T11:30:56.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:56.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983193+0000", 2024-09-18T11:30:56.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105682 2024-09-18T11:30:56.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:56.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983287+0000", 2024-09-18T11:30:56.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4066000000000004e-05 2024-09-18T11:30:56.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:56.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.986918+0000", 2024-09-18T11:30:56.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036311970000000001 2024-09-18T11:30:56.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990442+0000", 2024-09-18T11:30:56.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035243119999999999 2024-09-18T11:30:56.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:56.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990476+0000", 2024-09-18T11:30:56.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4028e-05 2024-09-18T11:30:56.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:56.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990507+0000", 2024-09-18T11:30:56.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0284999999999999e-05 2024-09-18T11:30:56.533 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.533 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:56.533 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.533 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1138 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~1201068] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:56.533 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.179081+0000", 2024-09-18T11:30:56.533 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.324279498999999, 2024-09-18T11:30:56.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007692099, 2024-09-18T11:30:56.533 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:56.533 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:56.533 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:56.533 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:56.533 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:56.534 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1138 2024-09-18T11:30:56.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.534 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:56.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:56.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179081+0000", 2024-09-18T11:30:56.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:56.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179081+0000", 2024-09-18T11:30:56.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:56.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179080+0000", 2024-09-18T11:30:56.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:56.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179083+0000", 2024-09-18T11:30:56.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.266e-06 2024-09-18T11:30:56.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:56.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179089+0000", 2024-09-18T11:30:56.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3589999999999996e-06 2024-09-18T11:30:56.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:30:56.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1999999999999996e-06 2024-09-18T11:30:56.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179194+0000", 2024-09-18T11:30:56.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101093 2024-09-18T11:30:56.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:56.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.179199+0000", 2024-09-18T11:30:56.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8960000000000002e-06 2024-09-18T11:30:56.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186435+0000", 2024-09-18T11:30:56.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072360979999999998 2024-09-18T11:30:56.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:56.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186530+0000", 2024-09-18T11:30:56.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5006999999999993e-05 2024-09-18T11:30:56.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:56.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.186773+0000", 2024-09-18T11:30:56.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000242496 2024-09-18T11:30:56.537 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.537 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:56.538 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.538 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1507 4.f 4:fc192b81:::smithi11939506-31:head [write 433647~450424,stat] snapc 13=[13,12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T11:30:56.538 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:32.763861+0000", 2024-09-18T11:30:56.538 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.739499105, 2024-09-18T11:30:56.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075480879999999997, 2024-09-18T11:30:56.538 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:56.538 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.538 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:56.538 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:56.538 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:56.538 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1507 2024-09-18T11:30:56.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.539 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:56.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:56.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.763861+0000", 2024-09-18T11:30:56.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:56.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.763861+0000", 2024-09-18T11:30:56.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:56.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.763860+0000", 2024-09-18T11:30:56.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:56.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:56.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.764059+0000", 2024-09-18T11:30:56.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019900500000000001 2024-09-18T11:30:56.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:56.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.764076+0000", 2024-09-18T11:30:56.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7558e-05 2024-09-18T11:30:56.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.764085+0000", 2024-09-18T11:30:56.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3119999999999994e-06 2024-09-18T11:30:56.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.764144+0000", 2024-09-18T11:30:56.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9880999999999998e-05 2024-09-18T11:30:56.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:56.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.764260+0000", 2024-09-18T11:30:56.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011586099999999999 2024-09-18T11:30:56.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:30:56.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.764485+0000", 2024-09-18T11:30:56.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00022445200000000001 2024-09-18T11:30:56.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:56.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.771268+0000", 2024-09-18T11:30:56.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067833149999999998 2024-09-18T11:30:56.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.771322+0000", 2024-09-18T11:30:56.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4361999999999999e-05 2024-09-18T11:30:56.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:56.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.771354+0000", 2024-09-18T11:30:56.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1451000000000001e-05 2024-09-18T11:30:56.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:56.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.771409+0000", 2024-09-18T11:30:56.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5287000000000001e-05 2024-09-18T11:30:56.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.543 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:56.544 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.544 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:262 4.9 e53/22 4:99feffe9:::smithi11939506-45:13 v 53'17, mlcod=51'14)", 2024-09-18T11:30:56.544 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:30:56.544 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.504369297, 2024-09-18T11:30:56.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013878679, 2024-09-18T11:30:56.544 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:56.544 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.544 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:56.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:56.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:30:56.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:56.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:30:56.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:56.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:30:56.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:56.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998997+0000", 2024-09-18T11:30:56.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1099999999999999e-06 2024-09-18T11:30:56.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:56.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999008+0000", 2024-09-18T11:30:56.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1321000000000001e-05 2024-09-18T11:30:56.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999015+0000", 2024-09-18T11:30:56.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3000000000000004e-06 2024-09-18T11:30:56.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999055+0000", 2024-09-18T11:30:56.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0058000000000002e-05 2024-09-18T11:30:56.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:56.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999070+0000", 2024-09-18T11:30:56.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4328000000000001e-05 2024-09-18T11:30:56.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:56.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012465+0000", 2024-09-18T11:30:56.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013395638 2024-09-18T11:30:56.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:56.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012870+0000", 2024-09-18T11:30:56.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040423400000000001 2024-09-18T11:30:56.548 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.548 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:56.548 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.548 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:205 4.b e53/22 4:dba5bd39:::smithi11939506-17:13 v 53'77, mlcod=51'74)", 2024-09-18T11:30:56.548 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:30:56.548 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.504320035999999, 2024-09-18T11:30:56.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013359586, 2024-09-18T11:30:56.548 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:56.548 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.548 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:56.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:56.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:30:56.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:56.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:30:56.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:56.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:30:56.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:56.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999048+0000", 2024-09-18T11:30:56.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3550000000000005e-06 2024-09-18T11:30:56.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:56.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999056+0000", 2024-09-18T11:30:56.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3259999999999996e-06 2024-09-18T11:30:56.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999063+0000", 2024-09-18T11:30:56.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6409999999999996e-06 2024-09-18T11:30:56.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999098+0000", 2024-09-18T11:30:56.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5203999999999997e-05 2024-09-18T11:30:56.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:56.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999115+0000", 2024-09-18T11:30:56.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6478999999999999e-05 2024-09-18T11:30:56.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:56.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012362+0000", 2024-09-18T11:30:56.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01324768 2024-09-18T11:30:56.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:56.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012400+0000", 2024-09-18T11:30:56.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.731e-05 2024-09-18T11:30:56.552 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.552 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:56.552 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.553 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:88 4.a e53/22 4:588debc2:::smithi11939506-18:13 v 53'45, mlcod=51'42)", 2024-09-18T11:30:56.553 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:30:56.553 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.501537613, 2024-09-18T11:30:56.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011307667, 2024-09-18T11:30:56.553 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:56.553 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.553 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:56.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:56.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:30:56.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:56.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:30:56.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:56.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001822+0000", 2024-09-18T11:30:56.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:56.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001827+0000", 2024-09-18T11:30:56.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.378e-06 2024-09-18T11:30:56.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:56.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001839+0000", 2024-09-18T11:30:56.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1225e-05 2024-09-18T11:30:56.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001844+0000", 2024-09-18T11:30:56.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2299999999999999e-06 2024-09-18T11:30:56.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001891+0000", 2024-09-18T11:30:56.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6961e-05 2024-09-18T11:30:56.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:56.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001906+0000", 2024-09-18T11:30:56.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5158e-05 2024-09-18T11:30:56.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:56.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013103+0000", 2024-09-18T11:30:56.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011197314 2024-09-18T11:30:56.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:56.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013130+0000", 2024-09-18T11:30:56.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6900999999999999e-05 2024-09-18T11:30:56.557 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.557 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:56.557 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.557 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:89 4.a e53/22 4:552523b6:::smithi11939506-38:13 v 53'47, mlcod=51'42)", 2024-09-18T11:30:56.557 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:30:56.557 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.500978393, 2024-09-18T11:30:56.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011327614, 2024-09-18T11:30:56.557 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:56.558 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.558 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:56.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:56.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:30:56.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:56.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:30:56.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:56.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002381+0000", 2024-09-18T11:30:56.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:56.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002397+0000", 2024-09-18T11:30:56.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5688e-05 2024-09-18T11:30:56.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:56.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002406+0000", 2024-09-18T11:30:56.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0229999999999998e-06 2024-09-18T11:30:56.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002415+0000", 2024-09-18T11:30:56.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8959999999999992e-06 2024-09-18T11:30:56.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012386+0000", 2024-09-18T11:30:56.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0099711450000000007 2024-09-18T11:30:56.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:56.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012405+0000", 2024-09-18T11:30:56.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9389e-05 2024-09-18T11:30:56.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:56.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013676+0000", 2024-09-18T11:30:56.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012708649999999999 2024-09-18T11:30:56.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:56.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013709+0000", 2024-09-18T11:30:56.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3207000000000002e-05 2024-09-18T11:30:56.561 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.561 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:56.561 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.561 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1643 4.7 4:e73ba68f:::smithi11939506-1:head [read 0~3983226,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:56.561 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:30:56.561 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.169473217, 2024-09-18T11:30:56.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079617669999999998, 2024-09-18T11:30:56.562 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:56.562 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:56.562 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:56.562 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:56.562 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:56.562 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1643 2024-09-18T11:30:56.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.562 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:56.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:56.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:30:56.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:56.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:30:56.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:56.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:30:56.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:56.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333889+0000", 2024-09-18T11:30:56.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8890000000000002e-06 2024-09-18T11:30:56.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:56.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333896+0000", 2024-09-18T11:30:56.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.156e-06 2024-09-18T11:30:56.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333900+0000", 2024-09-18T11:30:56.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.138e-06 2024-09-18T11:30:56.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.336677+0000", 2024-09-18T11:30:56.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027774169999999999 2024-09-18T11:30:56.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:56.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.336768+0000", 2024-09-18T11:30:56.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0983999999999995e-05 2024-09-18T11:30:56.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:56.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341849+0000", 2024-09-18T11:30:56.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050805140000000004 2024-09-18T11:30:56.565 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.565 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:56.565 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.565 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1644 4.7 4:e73ba68f:::smithi11939506-1:head [sparse-read 0~2695582] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:56.565 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:30:56.565 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.169456095000001, 2024-09-18T11:30:56.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084764039999999999, 2024-09-18T11:30:56.566 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:56.566 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:56.566 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:56.566 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:56.566 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:56.566 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1644 2024-09-18T11:30:56.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.566 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:56.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:56.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:30:56.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:56.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:30:56.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:56.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:30:56.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:56.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333906+0000", 2024-09-18T11:30:56.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.243e-06 2024-09-18T11:30:56.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:56.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333911+0000", 2024-09-18T11:30:56.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2959999999999998e-06 2024-09-18T11:30:56.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333915+0000", 2024-09-18T11:30:56.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6720000000000002e-06 2024-09-18T11:30:56.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341879+0000", 2024-09-18T11:30:56.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079638699999999996 2024-09-18T11:30:56.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:56.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341978+0000", 2024-09-18T11:30:56.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9562000000000007e-05 2024-09-18T11:30:56.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:56.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342380+0000", 2024-09-18T11:30:56.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000402113 2024-09-18T11:30:56.569 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.569 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:56.569 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.570 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1645 4.7 4:e73ba68f:::smithi11939506-1:head [read 0~3983226,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:56.570 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:30:56.570 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.169441201000001, 2024-09-18T11:30:56.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010539468999999999, 2024-09-18T11:30:56.570 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:56.570 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:56.570 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:56.570 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:56.570 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:56.570 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1645 2024-09-18T11:30:56.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.570 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:56.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:56.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:30:56.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:56.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:30:56.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:56.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:30:56.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:56.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333921+0000", 2024-09-18T11:30:56.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8909999999999999e-06 2024-09-18T11:30:56.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:56.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333927+0000", 2024-09-18T11:30:56.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4229999999999996e-06 2024-09-18T11:30:56.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333932+0000", 2024-09-18T11:30:56.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8280000000000002e-06 2024-09-18T11:30:56.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342418+0000", 2024-09-18T11:30:56.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084862219999999999 2024-09-18T11:30:56.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:56.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342515+0000", 2024-09-18T11:30:56.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7287999999999994e-05 2024-09-18T11:30:56.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:56.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.344458+0000", 2024-09-18T11:30:56.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019431660000000001 2024-09-18T11:30:56.574 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.574 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:56.574 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.574 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1669 4.4 4:2d9c3f7f:::smithi11939506-5:head [read 0~3764735,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-18T11:30:56.574 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:30:56.574 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.680492303000001, 2024-09-18T11:30:56.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0083694990000000007, 2024-09-18T11:30:56.574 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:56.574 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:56.575 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:56.575 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:56.575 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:56.575 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1669 2024-09-18T11:30:56.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.575 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:56.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:56.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:30:56.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:56.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:30:56.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:56.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822867+0000", 2024-09-18T11:30:56.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:56.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822871+0000", 2024-09-18T11:30:56.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:30:56.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:56.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822877+0000", 2024-09-18T11:30:56.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8000000000000004e-06 2024-09-18T11:30:56.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822880+0000", 2024-09-18T11:30:56.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5669999999999999e-06 2024-09-18T11:30:56.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822974+0000", 2024-09-18T11:30:56.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4048999999999993e-05 2024-09-18T11:30:56.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:56.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822980+0000", 2024-09-18T11:30:56.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2789999999999998e-06 2024-09-18T11:30:56.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.829160+0000", 2024-09-18T11:30:56.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0061800179999999998 2024-09-18T11:30:56.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:56.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.829248+0000", 2024-09-18T11:30:56.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7707000000000006e-05 2024-09-18T11:30:56.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:56.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.831237+0000", 2024-09-18T11:30:56.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001989827 2024-09-18T11:30:56.579 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.579 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:56.579 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.579 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:100 4.a e58/22 4:5975a5ec:::smithi11939506-36:18 v 58'55, mlcod=56'50)", 2024-09-18T11:30:56.579 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:30:56.579 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.230211064999999, 2024-09-18T11:30:56.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0088255, 2024-09-18T11:30:56.579 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:56.580 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.580 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:56.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:56.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:30:56.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:56.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:30:56.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:56.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:30:56.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:56.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273155+0000", 2024-09-18T11:30:56.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.212e-06 2024-09-18T11:30:56.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:56.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273164+0000", 2024-09-18T11:30:56.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7609999999999996e-06 2024-09-18T11:30:56.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273168+0000", 2024-09-18T11:30:56.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2819999999999998e-06 2024-09-18T11:30:56.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274162+0000", 2024-09-18T11:30:56.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099398600000000009 2024-09-18T11:30:56.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274180+0000", 2024-09-18T11:30:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8236e-05 2024-09-18T11:30:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281943+0000", 2024-09-18T11:30:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007763308 2024-09-18T11:30:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281975+0000", 2024-09-18T11:30:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1202999999999999e-05 2024-09-18T11:30:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:236 4.b e58/22 4:dba5bd39:::smithi11939506-17:13 v 58'95, mlcod=58'91)", 2024-09-18T11:30:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.274449+0000", 2024-09-18T11:30:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.228911028999999, 2024-09-18T11:30:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075435490000000001, 2024-09-18T11:30:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274449+0000", 2024-09-18T11:30:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274449+0000", 2024-09-18T11:30:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274448+0000", 2024-09-18T11:30:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274455+0000", 2024-09-18T11:30:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8700000000000003e-06 2024-09-18T11:30:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274467+0000", 2024-09-18T11:30:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1871e-05 2024-09-18T11:30:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274474+0000", 2024-09-18T11:30:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0439999999999998e-06 2024-09-18T11:30:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274512+0000", 2024-09-18T11:30:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8160000000000001e-05 2024-09-18T11:30:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274526+0000", 2024-09-18T11:30:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3478e-05 2024-09-18T11:30:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281961+0000", 2024-09-18T11:30:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007435629 2024-09-18T11:30:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281993+0000", 2024-09-18T11:30:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1242999999999999e-05 2024-09-18T11:30:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1954 4.7 4:ef60946b:::smithi11939506-8:head [read 0~2890771,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:30:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:30:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.163460025, 2024-09-18T11:30:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091142180000000003, 2024-09-18T11:30:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1954 2024-09-18T11:30:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:30:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:30:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:30:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339903+0000", 2024-09-18T11:30:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7349999999999998e-06 2024-09-18T11:30:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339909+0000", 2024-09-18T11:30:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-09-18T11:30:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339913+0000", 2024-09-18T11:30:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7670000000000001e-06 2024-09-18T11:30:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339970+0000", 2024-09-18T11:30:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6988999999999999e-05 2024-09-18T11:30:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339975+0000", 2024-09-18T11:30:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7929999999999997e-06 2024-09-18T11:30:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346917+0000", 2024-09-18T11:30:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069426640000000003 2024-09-18T11:30:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:56.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.347026+0000", 2024-09-18T11:30:56.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108241 2024-09-18T11:30:56.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:56.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.349014+0000", 2024-09-18T11:30:56.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001988723 2024-09-18T11:30:56.593 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.593 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:56.593 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.593 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1956 4.6 4:67c1765a:::smithi11939506-30:head [write 705927~558502] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:30:56.594 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:30:56.594 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.1513510849999999, 2024-09-18T11:30:56.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011813239, 2024-09-18T11:30:56.594 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:56.594 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.594 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:56.594 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:56.594 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:56.594 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1956 2024-09-18T11:30:56.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.594 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:56.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:30:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:30:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352007+0000", 2024-09-18T11:30:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:30:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:56.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352229+0000", 2024-09-18T11:30:56.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222024 2024-09-18T11:30:56.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:56.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352251+0000", 2024-09-18T11:30:56.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2512000000000001e-05 2024-09-18T11:30:56.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352260+0000", 2024-09-18T11:30:56.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7339999999999997e-06 2024-09-18T11:30:56.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352307+0000", 2024-09-18T11:30:56.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7061999999999998e-05 2024-09-18T11:30:56.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:56.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352408+0000", 2024-09-18T11:30:56.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101216 2024-09-18T11:30:56.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:30:56.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352550+0000", 2024-09-18T11:30:56.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014136199999999999 2024-09-18T11:30:56.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:56.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357684+0000", 2024-09-18T11:30:56.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051342499999999999 2024-09-18T11:30:56.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363736+0000", 2024-09-18T11:30:56.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006052577 2024-09-18T11:30:56.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:56.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363778+0000", 2024-09-18T11:30:56.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1989000000000001e-05 2024-09-18T11:30:56.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:56.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363822+0000", 2024-09-18T11:30:56.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3894999999999999e-05 2024-09-18T11:30:56.599 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.599 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:56.599 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.599 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2093 4.7 4:e0d95e10:::smithi11939506-22:head [assert-version v53,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e73)", 2024-09-18T11:30:56.599 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:55.369779+0000", 2024-09-18T11:30:56.599 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.1335808700000001, 2024-09-18T11:30:56.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078628099999999996, 2024-09-18T11:30:56.600 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:56.600 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:56.600 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:56.600 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:56.600 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:56.600 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2093 2024-09-18T11:30:56.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.600 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:56.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:56.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369779+0000", 2024-09-18T11:30:56.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:56.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369779+0000", 2024-09-18T11:30:56.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:56.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:56.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369775+0000", 2024-09-18T11:30:56.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999957 2024-09-18T11:30:56.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:56.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369785+0000", 2024-09-18T11:30:56.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7280000000000007e-06 2024-09-18T11:30:56.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:56.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369798+0000", 2024-09-18T11:30:56.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3509000000000001e-05 2024-09-18T11:30:56.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:56.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369806+0000", 2024-09-18T11:30:56.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3749999999999997e-06 2024-09-18T11:30:56.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:56.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369837+0000", 2024-09-18T11:30:56.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1440999999999999e-05 2024-09-18T11:30:56.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:56.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369886+0000", 2024-09-18T11:30:56.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9103000000000001e-05 2024-09-18T11:30:56.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:56.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.376462+0000", 2024-09-18T11:30:56.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065761980000000001 2024-09-18T11:30:56.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:56.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.377077+0000", 2024-09-18T11:30:56.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00061434599999999999 2024-09-18T11:30:56.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:56.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.377555+0000", 2024-09-18T11:30:56.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047826200000000002 2024-09-18T11:30:56.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:56.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.377599+0000", 2024-09-18T11:30:56.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4366999999999997e-05 2024-09-18T11:30:56.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:56.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:56.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:56.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.377642+0000", 2024-09-18T11:30:56.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2824999999999997e-05 2024-09-18T11:30:56.605 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.605 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:56.605 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.605 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:56.605 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:56.605 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:56.607 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:30:56.607 INFO:tasks.rados.rados.0.smithi119.stdout:1092: expect (ObjNum 99 snap 6 seq_num 99) 2024-09-18T11:30:56.607 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:56.605+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:56.644 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:30:56.644 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":73,"fsid":"f4cca259-b87a-45cc-8995-7e9fca891a9f","created":"2024-09-18T11:29:38.205487+0000","modified":"2024-09-18T11:30:54.998914+0000","last_up_change":"2024-09-18T11:30:48.309074+0000","last_in_change":"2024-09-18T11:30:49.312763+0000","flags":"nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5804032,"flags_set":["nodeep-scrub","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T11:29:47.951262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T11:29:48.866238+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"low_tier","create_time":"2024-09-18T11:29:59.343964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}},{"pool":4,"pool_name":"unique_pool_0","create_time":"2024-09-18T11:30:00.523818+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":35,"snap_epoch":73,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"dedup_cdc_chunk_size":131072,"dedup_chunk_algorithm":"fastcdc","dedup_tier":3,"fingerprint_algorithm":"sha256"},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"12faced2-7888-4265-ad12-815efe36466c","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":33347},{"type":"v1","addr":"172.21.15.119:6801","nonce":33347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":33347},{"type":"v1","addr":"172.21.15.119:6803","nonce":33347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":33347},{"type":"v1","addr":"172.21.15.119:6807","nonce":33347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":33347},{"type":"v1","addr":"172.21.15.119:6805","nonce":33347}]},"public_addr":"172.21.15.119:6801/33347","cluster_addr":"172.21.15.119:6803/33347","heartbeat_back_addr":"172.21.15.119:6807/33347","heartbeat_front_addr":"172.21.15.119:6805/33347","state":["exists"]},{"osd":1,"uuid":"20fee29d-b047-46ae-aa60-88e897fbb0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":33350},{"type":"v1","addr":"172.21.15.119:6825","nonce":33350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":33350},{"type":"v1","addr":"172.21.15.119:6827","nonce":33350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":33350},{"type":"v1","addr":"172.21.15.119:6831","nonce":33350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":33350},{"type":"v1","addr":"172.21.15.119:6829","nonce":33350}]},"public_addr":"172.21.15.119:6825/33350","cluster_addr":"172.21.15.119:6827/33350","heartbeat_back_addr":"172.21.15.119:6831/33350","heartbeat_front_addr":"172.21.15.119:6829/33350","state":["exists","up"]},{"osd":2,"uuid":"21a296f3-b259-4841-9a17-4cd21b302b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":33351},{"type":"v1","addr":"172.21.15.119:6809","nonce":33351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":33351},{"type":"v1","addr":"172.21.15.119:6813","nonce":33351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6819","nonce":33351},{"type":"v1","addr":"172.21.15.119:6821","nonce":33351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":33351},{"type":"v1","addr":"172.21.15.119:6817","nonce":33351}]},"public_addr":"172.21.15.119:6809/33351","cluster_addr":"172.21.15.119:6813/33351","heartbeat_back_addr":"172.21.15.119:6821/33351","heartbeat_front_addr":"172.21.15.119:6817/33351","state":["exists","up"]},{"osd":3,"uuid":"59044f30-32e1-4942-9a8e-202c1def221a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":33349},{"type":"v1","addr":"172.21.15.119:6812","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":33349},{"type":"v1","addr":"172.21.15.119:6816","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":33349},{"type":"v1","addr":"172.21.15.119:6823","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":33349},{"type":"v1","addr":"172.21.15.119:6820","nonce":33349}]},"public_addr":"172.21.15.119:6812/33349","cluster_addr":"172.21.15.119:6816/33349","heartbeat_back_addr":"172.21.15.119:6823/33349","heartbeat_front_addr":"172.21.15.119:6820/33349","state":["exists","up"]},{"osd":4,"uuid":"7d1ae3d0-8d69-4d3c-9e51-24fdd77a22a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":33088},{"type":"v1","addr":"172.21.15.148:6809","nonce":33088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":33088},{"type":"v1","addr":"172.21.15.148:6811","nonce":33088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":33088},{"type":"v1","addr":"172.21.15.148:6815","nonce":33088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":33088},{"type":"v1","addr":"172.21.15.148:6813","nonce":33088}]},"public_addr":"172.21.15.148:6809/33088","cluster_addr":"172.21.15.148:6811/33088","heartbeat_back_addr":"172.21.15.148:6815/33088","heartbeat_front_addr":"172.21.15.148:6813/33088","state":["exists","up"]},{"osd":5,"uuid":"b0dd3d30-b273-428c-a4a9-4c238abf45fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":33089},{"type":"v1","addr":"172.21.15.148:6801","nonce":33089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":33089},{"type":"v1","addr":"172.21.15.148:6803","nonce":33089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":33089},{"type":"v1","addr":"172.21.15.148:6807","nonce":33089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":33089},{"type":"v1","addr":"172.21.15.148:6805","nonce":33089}]},"public_addr":"172.21.15.148:6801/33089","cluster_addr":"172.21.15.148:6803/33089","heartbeat_back_addr":"172.21.15.148:6807/33089","heartbeat_front_addr":"172.21.15.148:6805/33089","state":["exists","up"]},{"osd":6,"uuid":"40290cb3-2334-4c68-8e6c-59d0d5455171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":33107},{"type":"v1","addr":"172.21.15.148:6817","nonce":33107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":33107},{"type":"v1","addr":"172.21.15.148:6819","nonce":33107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":33107},{"type":"v1","addr":"172.21.15.148:6823","nonce":33107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":33107},{"type":"v1","addr":"172.21.15.148:6821","nonce":33107}]},"public_addr":"172.21.15.148:6817/33107","cluster_addr":"172.21.15.148:6819/33107","heartbeat_back_addr":"172.21.15.148:6823/33107","heartbeat_front_addr":"172.21.15.148:6821/33107","state":["exists","up"]},{"osd":7,"uuid":"54802abe-ea1a-448e-85af-d1f14a45be27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":33119},{"type":"v1","addr":"172.21.15.148:6825","nonce":33119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":33119},{"type":"v1","addr":"172.21.15.148:6827","nonce":33119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":33119},{"type":"v1","addr":"172.21.15.148:6831","nonce":33119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":33119},{"type":"v1","addr":"172.21.15.148:6829","nonce":33119}]},"public_addr":"172.21.15.148:6825/33119","cluster_addr":"172.21.15.148:6827/33119","heartbeat_back_addr":"172.21.15.148:6831/33119","heartbeat_front_addr":"172.21.15.148:6829/33119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-18T11:29:45.374987+0000","dead_epoch":66},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.452341+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.393769+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:30:51.381336+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.463411+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.467809+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.522054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:29:45.497313+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":4,"snaps":[{"begin":16,"length":1},{"begin":19,"length":2}]}],"new_removed_snaps":[{"pool":4,"snaps":[{"begin":16,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T11:30:56.657 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T11:29:47.951262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T11:29:48.866238+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'low_tier', 'create_time': '2024-09-18T11:29:59.343964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}}, {'pool': 4, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-18T11:30:00.523818+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 35, 'snap_epoch': 73, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'dedup_cdc_chunk_size': 131072, 'dedup_chunk_algorithm': 'fastcdc', 'dedup_tier': 3, 'fingerprint_algorithm': 'sha256'}, 'application_metadata': {'rados': {}}}] 2024-09-18T11:30:56.657 DEBUG:teuthology.orchestra.run.smithi119:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op apply-layout-settings --pool .mgr 2024-09-18T11:30:56.707 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:56.706+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:56.743 INFO:tasks.rados.rados.0.smithi119.stdout:1093: finishing set_chunk to oid 48 2024-09-18T11:30:56.743 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 48 set_chunk returned error code -95 offset: 1233920 length: 930816 2024-09-18T11:30:56.743 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 48 v 0 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-18T11:30:56.743 INFO:tasks.rados.rados.0.smithi119.stdout:1094: finishing write tid 1 to smithi11939506-1 2024-09-18T11:30:56.743 INFO:tasks.rados.rados.0.smithi119.stdout:1094: finishing write tid 2 to smithi11939506-1 2024-09-18T11:30:56.743 INFO:tasks.rados.rados.0.smithi119.stdout:1094: finishing write tid 3 to smithi11939506-1 2024-09-18T11:30:56.743 INFO:tasks.rados.rados.0.smithi119.stdout:1091: done (7 left) 2024-09-18T11:30:56.743 INFO:tasks.rados.rados.0.smithi119.stdout:1092: done (6 left) 2024-09-18T11:30:56.744 INFO:tasks.rados.rados.0.smithi119.stdout:1093: done (5 left) 2024-09-18T11:30:56.744 INFO:tasks.rados.rados.0.smithi119.stdout:1099: set_chunk oid 32 target oid 2024-09-18T11:30:56.744 INFO:tasks.rados.rados.0.smithi119.stdout:1099: set_chunk oid 32 offset: 2394624 length: 260096 target oid smithi11939506- offset: 2394624 snap 12 2024-09-18T11:30:56.744 INFO:tasks.rados.rados.0.smithi119.stdout:1100: snap_create 2024-09-18T11:30:56.744 INFO:tasks.rados.rados.0.smithi119.stdout:1094: finishing write tid 4 to smithi11939506-1 2024-09-18T11:30:56.744 INFO:tasks.rados.rados.0.smithi119.stdout:1094: finishing write tid 5 to smithi11939506-1 2024-09-18T11:30:56.744 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 96 (ObjNum 186 snap 20 seq_num 186) dirty exists 2024-09-18T11:30:56.744 INFO:tasks.rados.rados.0.smithi119.stdout:1094: left oid 1 (ObjNum 186 snap 20 seq_num 186) 2024-09-18T11:30:56.744 INFO:tasks.rados.rados.0.smithi119.stdout:1095: finishing write tid 1 to smithi11939506-6 2024-09-18T11:30:56.744 INFO:tasks.rados.rados.0.smithi119.stdout:1095: finishing write tid 2 to smithi11939506-6 2024-09-18T11:30:56.744 INFO:tasks.rados.rados.0.smithi119.stdout:1095: finishing write tid 3 to smithi11939506-6 2024-09-18T11:30:56.744 INFO:tasks.rados.rados.0.smithi119.stdout:1095: finishing write tid 4 to smithi11939506-6 2024-09-18T11:30:56.744 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 6 v 81 (ObjNum 187 snap 20 seq_num 187) dirty exists 2024-09-18T11:30:56.744 INFO:tasks.rados.rados.0.smithi119.stdout:1095: left oid 6 (ObjNum 187 snap 20 seq_num 187) 2024-09-18T11:30:56.745 INFO:tasks.rados.rados.0.smithi119.stdout:1096: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-18T11:30:56.745 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:56.746 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:56.746 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:56.746 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:56.755 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:30:56.807 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:56.805+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:56.831 INFO:tasks.rados.rados.0.smithi119.stdout:1097: finishing write tid 1 to smithi11939506-37 2024-09-18T11:30:56.831 INFO:tasks.rados.rados.0.smithi119.stdout:1098: got (22) Invalid argument 2024-09-18T11:30:56.831 INFO:tasks.rados.rados.0.smithi119.stdout:1097: finishing write tid 2 to smithi11939506-37 2024-09-18T11:30:56.831 INFO:tasks.rados.rados.0.smithi119.stdout:1097: finishing write tid 3 to smithi11939506-37 2024-09-18T11:30:56.831 INFO:tasks.rados.rados.0.smithi119.stdout:1097: finishing write tid 4 to smithi11939506-37 2024-09-18T11:30:56.831 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 121 (ObjNum 188 snap 20 seq_num 188) dirty exists 2024-09-18T11:30:56.831 INFO:tasks.rados.rados.0.smithi119.stdout:1097: left oid 37 (ObjNum 188 snap 20 seq_num 188) 2024-09-18T11:30:56.831 INFO:tasks.rados.rados.0.smithi119.stdout:1099: finishing set_chunk to oid 32 2024-09-18T11:30:56.831 INFO:tasks.rados.rados.0.smithi119.stdout:1099: got expected ENOENT (src dne) 2024-09-18T11:30:56.853 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:56.853 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:56.853 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:56.853 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:56.853 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:56.861 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:30:56.945 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:56.943+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:57.000 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:57.000 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:57.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:57.000 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:57.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.000 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1046 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'24, mlcod=30'21)", 2024-09-18T11:30:57.001 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:30:57.001 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.824278544999999, 2024-09-18T11:30:57.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042519100000000002, 2024-09-18T11:30:57.001 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.001 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.001 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:30:57.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:30:57.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173782+0000", 2024-09-18T11:30:57.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:57.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174207+0000", 2024-09-18T11:30:57.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00042479799999999999 2024-09-18T11:30:57.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174224+0000", 2024-09-18T11:30:57.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7513999999999999e-05 2024-09-18T11:30:57.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174232+0000", 2024-09-18T11:30:57.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0840000000000003e-06 2024-09-18T11:30:57.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174297+0000", 2024-09-18T11:30:57.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5215000000000005e-05 2024-09-18T11:30:57.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174336+0000", 2024-09-18T11:30:57.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9403000000000002e-05 2024-09-18T11:30:57.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.177990+0000", 2024-09-18T11:30:57.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036533709999999999 2024-09-18T11:30:57.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.178035+0000", 2024-09-18T11:30:57.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5495999999999999e-05 2024-09-18T11:30:57.005 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.005 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.005 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.005 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1047 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'25, mlcod=35'23)", 2024-09-18T11:30:57.005 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:30:57.005 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.818306347000004, 2024-09-18T11:30:57.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038129129999999998, 2024-09-18T11:30:57.005 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.005 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.006 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:30:57.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:30:57.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179754+0000", 2024-09-18T11:30:57.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:57.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180088+0000", 2024-09-18T11:30:57.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00033395000000000001 2024-09-18T11:30:57.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180105+0000", 2024-09-18T11:30:57.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7039000000000001e-05 2024-09-18T11:30:57.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180112+0000", 2024-09-18T11:30:57.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3710000000000002e-06 2024-09-18T11:30:57.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180176+0000", 2024-09-18T11:30:57.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3178999999999999e-05 2024-09-18T11:30:57.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180197+0000", 2024-09-18T11:30:57.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1687000000000001e-05 2024-09-18T11:30:57.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183542+0000", 2024-09-18T11:30:57.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003344877 2024-09-18T11:30:57.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183568+0000", 2024-09-18T11:30:57.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6146e-05 2024-09-18T11:30:57.010 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.010 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.010 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.010 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:30:57.010 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:30:57.010 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.014687678000001, 2024-09-18T11:30:57.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:30:57.010 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.010 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.011 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:30:57.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:30:57.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:30:57.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:57.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:30:57.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:30:57.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:30:57.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:30:57.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:30:57.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:30:57.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:30:57.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:30:57.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:30:57.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:30:57.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:30:57.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:30:57.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:30:57.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:30:57.015 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.015 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.015 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.015 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:30:57.015 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:30:57.015 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.779856724999998, 2024-09-18T11:30:57.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:30:57.015 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.015 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.015 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:30:57.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:30:57.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:30:57.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:30:57.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:30:57.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:30:57.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:30:57.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:30:57.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:30:57.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:30:57.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:30:57.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:30:57.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:30:57.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:30:57.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:30:57.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:30:57.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:30:57.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:30:57.019 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.019 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.019 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.019 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:244 4.d e49/22 4:b5a463bf:::smithi11939506-20:10 v 49'68, mlcod=48'65)", 2024-09-18T11:30:57.019 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:30.220263+0000", 2024-09-18T11:30:57.019 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.777799134999999, 2024-09-18T11:30:57.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032524960000000001, 2024-09-18T11:30:57.020 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.020 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.020 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220263+0000", 2024-09-18T11:30:57.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220263+0000", 2024-09-18T11:30:57.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220262+0000", 2024-09-18T11:30:57.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:30:57.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220266+0000", 2024-09-18T11:30:57.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.895e-06 2024-09-18T11:30:57.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220278+0000", 2024-09-18T11:30:57.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1962e-05 2024-09-18T11:30:57.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220284+0000", 2024-09-18T11:30:57.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1299999999999998e-06 2024-09-18T11:30:57.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220334+0000", 2024-09-18T11:30:57.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9789000000000001e-05 2024-09-18T11:30:57.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220356+0000", 2024-09-18T11:30:57.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1897000000000001e-05 2024-09-18T11:30:57.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.223464+0000", 2024-09-18T11:30:57.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031082789999999998 2024-09-18T11:30:57.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.223515+0000", 2024-09-18T11:30:57.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0748999999999998e-05 2024-09-18T11:30:57.023 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.023 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.023 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.024 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1512 4.4 e51/22 4:2c93196f:::smithi11939506-28:head v 51'51, mlcod=51'48)", 2024-09-18T11:30:57.024 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:32.782201+0000", 2024-09-18T11:30:57.024 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.215860770999999, 2024-09-18T11:30:57.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00363554, 2024-09-18T11:30:57.024 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.024 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.024 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782201+0000", 2024-09-18T11:30:57.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782201+0000", 2024-09-18T11:30:57.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782200+0000", 2024-09-18T11:30:57.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:57.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782564+0000", 2024-09-18T11:30:57.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000364612 2024-09-18T11:30:57.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782585+0000", 2024-09-18T11:30:57.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0418999999999999e-05 2024-09-18T11:30:57.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782594+0000", 2024-09-18T11:30:57.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9570000000000008e-06 2024-09-18T11:30:57.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782663+0000", 2024-09-18T11:30:57.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9330000000000002e-05 2024-09-18T11:30:57.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782688+0000", 2024-09-18T11:30:57.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5420000000000001e-05 2024-09-18T11:30:57.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.785790+0000", 2024-09-18T11:30:57.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031016160000000002 2024-09-18T11:30:57.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.785836+0000", 2024-09-18T11:30:57.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6360999999999999e-05 2024-09-18T11:30:57.030 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.030 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.030 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.030 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:30:57.030 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:30:57.030 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.605950440000001, 2024-09-18T11:30:57.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:30:57.030 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.030 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.030 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:30:57.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:30:57.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:30:57.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:57.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:30:57.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:30:57.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:30:57.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:30:57.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:30:57.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:30:57.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:30:57.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:30:57.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:30:57.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:30:57.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:30:57.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:30:57.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:30:57.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:30:57.034 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.034 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.034 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.034 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1532 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'74, mlcod=51'69)", 2024-09-18T11:30:57.034 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.397573+0000", 2024-09-18T11:30:57.035 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.600488351999999, 2024-09-18T11:30:57.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032675960000000002, 2024-09-18T11:30:57.035 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.035 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.035 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397573+0000", 2024-09-18T11:30:57.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397573+0000", 2024-09-18T11:30:57.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397573+0000", 2024-09-18T11:30:57.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:57.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397577+0000", 2024-09-18T11:30:57.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5499999999999996e-06 2024-09-18T11:30:57.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397585+0000", 2024-09-18T11:30:57.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1470000000000001e-06 2024-09-18T11:30:57.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397591+0000", 2024-09-18T11:30:57.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5369999999999997e-06 2024-09-18T11:30:57.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397635+0000", 2024-09-18T11:30:57.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4308000000000003e-05 2024-09-18T11:30:57.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397656+0000", 2024-09-18T11:30:57.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1138e-05 2024-09-18T11:30:57.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.400817+0000", 2024-09-18T11:30:57.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031607499999999999 2024-09-18T11:30:57.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.400841+0000", 2024-09-18T11:30:57.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3717e-05 2024-09-18T11:30:57.039 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.039 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.039 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.039 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.13:87 3.0 e58/21 3:099aa86f:::smithi11939506-4 low_tier:head v 58'69, mlcod=55'66)", 2024-09-18T11:30:57.039 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.270180+0000", 2024-09-18T11:30:57.039 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.727881914000001, 2024-09-18T11:30:57.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037855440000000001, 2024-09-18T11:30:57.039 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.039 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.039 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270180+0000", 2024-09-18T11:30:57.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270180+0000", 2024-09-18T11:30:57.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270179+0000", 2024-09-18T11:30:57.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:57.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270187+0000", 2024-09-18T11:30:57.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5610000000000001e-06 2024-09-18T11:30:57.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270195+0000", 2024-09-18T11:30:57.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5909999999999996e-06 2024-09-18T11:30:57.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270200+0000", 2024-09-18T11:30:57.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2599999999999999e-06 2024-09-18T11:30:57.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270252+0000", 2024-09-18T11:30:57.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1891000000000002e-05 2024-09-18T11:30:57.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270277+0000", 2024-09-18T11:30:57.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5134e-05 2024-09-18T11:30:57.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273928+0000", 2024-09-18T11:30:57.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00365132 2024-09-18T11:30:57.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273965+0000", 2024-09-18T11:30:57.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7277000000000002e-05 2024-09-18T11:30:57.043 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.043 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.043 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.044 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.13:88 3.0 e58/21 3:099aa86f:::smithi11939506-4 low_tier:head v 58'70, mlcod=55'66)", 2024-09-18T11:30:57.044 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.270488+0000", 2024-09-18T11:30:57.044 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.727574039, 2024-09-18T11:30:57.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034966020000000001, 2024-09-18T11:30:57.044 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.044 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.044 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270488+0000", 2024-09-18T11:30:57.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270488+0000", 2024-09-18T11:30:57.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270487+0000", 2024-09-18T11:30:57.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:57.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270493+0000", 2024-09-18T11:30:57.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9780000000000002e-06 2024-09-18T11:30:57.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270503+0000", 2024-09-18T11:30:57.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0183000000000001e-05 2024-09-18T11:30:57.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270507+0000", 2024-09-18T11:30:57.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9960000000000004e-06 2024-09-18T11:30:57.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270542+0000", 2024-09-18T11:30:57.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5077000000000003e-05 2024-09-18T11:30:57.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270562+0000", 2024-09-18T11:30:57.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9456000000000001e-05 2024-09-18T11:30:57.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273967+0000", 2024-09-18T11:30:57.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003405805 2024-09-18T11:30:57.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273984+0000", 2024-09-18T11:30:57.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6824e-05 2024-09-18T11:30:57.048 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.048 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.048 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.048 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:289 4.d e58/22 4:b5a463bf:::smithi11939506-20:10 v 58'78, mlcod=53'73)", 2024-09-18T11:30:57.048 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.275682+0000", 2024-09-18T11:30:57.048 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.722379549999999, 2024-09-18T11:30:57.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003584318, 2024-09-18T11:30:57.048 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.048 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.048 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275682+0000", 2024-09-18T11:30:57.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275682+0000", 2024-09-18T11:30:57.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275681+0000", 2024-09-18T11:30:57.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:57.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275689+0000", 2024-09-18T11:30:57.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-18T11:30:57.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275698+0000", 2024-09-18T11:30:57.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1120000000000002e-06 2024-09-18T11:30:57.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275702+0000", 2024-09-18T11:30:57.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1450000000000001e-06 2024-09-18T11:30:57.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275731+0000", 2024-09-18T11:30:57.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8782999999999999e-05 2024-09-18T11:30:57.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275743+0000", 2024-09-18T11:30:57.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1436e-05 2024-09-18T11:30:57.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.279231+0000", 2024-09-18T11:30:57.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034884339999999999 2024-09-18T11:30:57.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.279266+0000", 2024-09-18T11:30:57.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5488000000000003e-05 2024-09-18T11:30:57.052 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.052 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.052 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.052 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1931 3.0 e65/21 3:3c05e498:::866d65857c51f67d56ecbcba33930954810c64ced2a213ca9705911438b0dfb7:head v 65'72, mlcod=58'70)", 2024-09-18T11:30:57.053 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:30:57.053 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.6299404220000007, 2024-09-18T11:30:57.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040630859999999996, 2024-09-18T11:30:57.053 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.053 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.053 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:30:57.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:30:57.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368120+0000", 2024-09-18T11:30:57.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:57.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368427+0000", 2024-09-18T11:30:57.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030742600000000002 2024-09-18T11:30:57.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368446+0000", 2024-09-18T11:30:57.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8046e-05 2024-09-18T11:30:57.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368453+0000", 2024-09-18T11:30:57.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1380000000000001e-06 2024-09-18T11:30:57.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368519+0000", 2024-09-18T11:30:57.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6470000000000006e-05 2024-09-18T11:30:57.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368550+0000", 2024-09-18T11:30:57.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0707999999999998e-05 2024-09-18T11:30:57.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.372151+0000", 2024-09-18T11:30:57.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036015470000000001 2024-09-18T11:30:57.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.372184+0000", 2024-09-18T11:30:57.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2994999999999998e-05 2024-09-18T11:30:57.056 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.056 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.056 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.057 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:304 4.6 e69/67 4:67c1765a:::smithi11939506-30:17 v 69'35, mlcod=58'31)", 2024-09-18T11:30:57.057 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:30:57.057 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.6521093459999996, 2024-09-18T11:30:57.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006377611, 2024-09-18T11:30:57.057 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.057 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.057 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:30:57.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:30:57.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:30:57.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:57.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345961+0000", 2024-09-18T11:30:57.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2050000000000008e-06 2024-09-18T11:30:57.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345969+0000", 2024-09-18T11:30:57.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0339999999999997e-06 2024-09-18T11:30:57.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345976+0000", 2024-09-18T11:30:57.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-18T11:30:57.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346026+0000", 2024-09-18T11:30:57.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9286999999999998e-05 2024-09-18T11:30:57.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346050+0000", 2024-09-18T11:30:57.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4397e-05 2024-09-18T11:30:57.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352293+0000", 2024-09-18T11:30:57.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062427170000000001 2024-09-18T11:30:57.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352330+0000", 2024-09-18T11:30:57.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7348999999999998e-05 2024-09-18T11:30:57.061 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.061 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.061 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.061 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:345 4.d e69/22 4:b81644f2:::smithi11939506-34:14 v 69'97, mlcod=65'93)", 2024-09-18T11:30:57.061 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351361+0000", 2024-09-18T11:30:57.061 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.6467005730000004, 2024-09-18T11:30:57.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003416484, 2024-09-18T11:30:57.062 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.062 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.062 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351361+0000", 2024-09-18T11:30:57.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351361+0000", 2024-09-18T11:30:57.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351361+0000", 2024-09-18T11:30:57.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:57.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351369+0000", 2024-09-18T11:30:57.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9100000000000005e-06 2024-09-18T11:30:57.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351377+0000", 2024-09-18T11:30:57.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9869999999999999e-06 2024-09-18T11:30:57.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351383+0000", 2024-09-18T11:30:57.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9290000000000003e-06 2024-09-18T11:30:57.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351454+0000", 2024-09-18T11:30:57.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1180000000000006e-05 2024-09-18T11:30:57.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351484+0000", 2024-09-18T11:30:57.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0596999999999999e-05 2024-09-18T11:30:57.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.354730+0000", 2024-09-18T11:30:57.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032452840000000002 2024-09-18T11:30:57.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.354778+0000", 2024-09-18T11:30:57.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7942999999999998e-05 2024-09-18T11:30:57.065 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.065 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.065 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.066 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:347 4.d e69/22 4:b44229a1:::smithi11939506-42:19 v 69'99, mlcod=65'93)", 2024-09-18T11:30:57.066 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:30:57.066 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.6463761510000001, 2024-09-18T11:30:57.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056998760000000004, 2024-09-18T11:30:57.066 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.066 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.066 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:30:57.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:30:57.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351684+0000", 2024-09-18T11:30:57.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:57.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351693+0000", 2024-09-18T11:30:57.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-18T11:30:57.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351704+0000", 2024-09-18T11:30:57.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1077000000000001e-05 2024-09-18T11:30:57.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351709+0000", 2024-09-18T11:30:57.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5419999999999999e-06 2024-09-18T11:30:57.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351956+0000", 2024-09-18T11:30:57.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000247252 2024-09-18T11:30:57.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351975+0000", 2024-09-18T11:30:57.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9201000000000001e-05 2024-09-18T11:30:57.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357334+0000", 2024-09-18T11:30:57.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053588749999999999 2024-09-18T11:30:57.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357385+0000", 2024-09-18T11:30:57.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1406999999999998e-05 2024-09-18T11:30:57.070 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.070 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.070 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.070 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1956 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'37, mlcod=58'31)", 2024-09-18T11:30:57.070 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:30:57.070 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.6454312380000005, 2024-09-18T11:30:57.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010958842, 2024-09-18T11:30:57.070 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.070 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.071 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:30:57.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:30:57.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:30:57.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:57.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352996+0000", 2024-09-18T11:30:57.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036622 2024-09-18T11:30:57.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353011+0000", 2024-09-18T11:30:57.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4849e-05 2024-09-18T11:30:57.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353017+0000", 2024-09-18T11:30:57.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:30:57.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353068+0000", 2024-09-18T11:30:57.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0695000000000002e-05 2024-09-18T11:30:57.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353092+0000", 2024-09-18T11:30:57.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4260000000000002e-05 2024-09-18T11:30:57.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363551+0000", 2024-09-18T11:30:57.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010459154999999999 2024-09-18T11:30:57.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363589+0000", 2024-09-18T11:30:57.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8161000000000003e-05 2024-09-18T11:30:57.074 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.074 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.074 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.075 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1957 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'38, mlcod=65'33)", 2024-09-18T11:30:57.075 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:30:57.075 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.6390153710000002, 2024-09-18T11:30:57.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050499400000000002, 2024-09-18T11:30:57.075 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.075 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.075 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:30:57.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:30:57.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:30:57.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:57.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359214+0000", 2024-09-18T11:30:57.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000168047 2024-09-18T11:30:57.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359227+0000", 2024-09-18T11:30:57.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3124e-05 2024-09-18T11:30:57.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359235+0000", 2024-09-18T11:30:57.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0760000000000003e-06 2024-09-18T11:30:57.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359287+0000", 2024-09-18T11:30:57.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.151e-05 2024-09-18T11:30:57.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359310+0000", 2024-09-18T11:30:57.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-09-18T11:30:57.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364065+0000", 2024-09-18T11:30:57.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047549439999999997 2024-09-18T11:30:57.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364096+0000", 2024-09-18T11:30:57.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1423000000000001e-05 2024-09-18T11:30:57.079 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.079 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.079 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.079 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:349 4.d e69/22 4:b9bb642b:::smithi11939506-27:19 v 69'101, mlcod=69'97)", 2024-09-18T11:30:57.080 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.360028+0000", 2024-09-18T11:30:57.080 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.6380337540000003, 2024-09-18T11:30:57.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040307850000000003, 2024-09-18T11:30:57.080 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.080 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.080 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360028+0000", 2024-09-18T11:30:57.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360028+0000", 2024-09-18T11:30:57.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360027+0000", 2024-09-18T11:30:57.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:57.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360039+0000", 2024-09-18T11:30:57.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1654000000000001e-05 2024-09-18T11:30:57.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360052+0000", 2024-09-18T11:30:57.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.33e-05 2024-09-18T11:30:57.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360058+0000", 2024-09-18T11:30:57.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0440000000000001e-06 2024-09-18T11:30:57.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360094+0000", 2024-09-18T11:30:57.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6402999999999997e-05 2024-09-18T11:30:57.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360109+0000", 2024-09-18T11:30:57.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4105e-05 2024-09-18T11:30:57.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364025+0000", 2024-09-18T11:30:57.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039163189999999997 2024-09-18T11:30:57.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364059+0000", 2024-09-18T11:30:57.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3810999999999999e-05 2024-09-18T11:30:57.084 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.084 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.084 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.085 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1978 4.4 e69/22 4:2d9c3f7f:::smithi11939506-5:head v 69'74, mlcod=69'71)", 2024-09-18T11:30:57.085 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.967898+0000", 2024-09-18T11:30:57.085 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.0301633030000001, 2024-09-18T11:30:57.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038277960000000001, 2024-09-18T11:30:57.085 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.085 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.085 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.967898+0000", 2024-09-18T11:30:57.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.967898+0000", 2024-09-18T11:30:57.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.967898+0000", 2024-09-18T11:30:57.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:57.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.968050+0000", 2024-09-18T11:30:57.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015286799999999999 2024-09-18T11:30:57.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.968064+0000", 2024-09-18T11:30:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3964999999999999e-05 2024-09-18T11:30:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.968070+0000", 2024-09-18T11:30:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:30:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.968112+0000", 2024-09-18T11:30:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1112999999999998e-05 2024-09-18T11:30:57.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.968133+0000", 2024-09-18T11:30:57.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1715000000000001e-05 2024-09-18T11:30:57.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.971687+0000", 2024-09-18T11:30:57.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035536410000000002 2024-09-18T11:30:57.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.971726+0000", 2024-09-18T11:30:57.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.926e-05 2024-09-18T11:30:57.089 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.089 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.089 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.089 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2141 3.0 e73/21 3:1799f92c:::95e762d743c69e6efaefe92d905ac94b659d7d90631d34d4368df67f56c10c79:head v 73'79, mlcod=73'77)", 2024-09-18T11:30:57.089 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:55.733432+0000", 2024-09-18T11:30:57.089 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.2646293150000001, 2024-09-18T11:30:57.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035087809999999999, 2024-09-18T11:30:57.089 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.089 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.090 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.733432+0000", 2024-09-18T11:30:57.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.733432+0000", 2024-09-18T11:30:57.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.733431+0000", 2024-09-18T11:30:57.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:57.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.733841+0000", 2024-09-18T11:30:57.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040973799999999999 2024-09-18T11:30:57.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.733855+0000", 2024-09-18T11:30:57.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4809e-05 2024-09-18T11:30:57.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.733861+0000", 2024-09-18T11:30:57.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5949999999999998e-06 2024-09-18T11:30:57.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.733908+0000", 2024-09-18T11:30:57.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6854999999999998e-05 2024-09-18T11:30:57.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.733932+0000", 2024-09-18T11:30:57.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4236999999999999e-05 2024-09-18T11:30:57.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.736906+0000", 2024-09-18T11:30:57.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029739129999999999 2024-09-18T11:30:57.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.736941+0000", 2024-09-18T11:30:57.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5129999999999997e-05 2024-09-18T11:30:57.094 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.094 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.094 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.094 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.094 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.094 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:57.095 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:30:57.096 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:57.093+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:57.196 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:57.195+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:30:57.234 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:57.234 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:57.234 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:30:57.235 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:57.243 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:30:57.296 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:57.294+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:57.320 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:30:57.341 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:57.341 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:57.341 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:57.341 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:57.341 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:57.351 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:30:57.367 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is unset 2024-09-18T11:30:57.367 INFO:tasks.rados.rados.0.smithi119.stdout:1094: done (6 left) 2024-09-18T11:30:57.368 INFO:tasks.rados.rados.0.smithi119.stdout:1095: done (5 left) 2024-09-18T11:30:57.368 INFO:tasks.rados.rados.0.smithi119.stdout:1096: done (4 left) 2024-09-18T11:30:57.368 INFO:tasks.rados.rados.0.smithi119.stdout:1097: done (3 left) 2024-09-18T11:30:57.368 INFO:tasks.rados.rados.0.smithi119.stdout:1098: done (2 left) 2024-09-18T11:30:57.368 INFO:tasks.rados.rados.0.smithi119.stdout:1099: done (1 left) 2024-09-18T11:30:57.368 INFO:tasks.rados.rados.0.smithi119.stdout:1100: done (0 left) 2024-09-18T11:30:57.368 INFO:tasks.rados.rados.0.smithi119.stdout:1101: write (excl) oid 16 current snap is 21 2024-09-18T11:30:57.368 INFO:tasks.rados.rados.0.smithi119.stdout:1101: seq_num 189 ranges {776523=614533,1976979=757720} 2024-09-18T11:30:57.375 INFO:tasks.rados.rados.0.smithi119.stdout:1101: writing smithi11939506-16 from 776523 to 1391056 tid 1 2024-09-18T11:30:57.384 INFO:tasks.rados.rados.0.smithi119.stdout:1101: writing smithi11939506-16 from 1976979 to 2734699 tid 2 2024-09-18T11:30:57.384 INFO:tasks.rados.rados.0.smithi119.stdout:1102: set_chunk oid 39 target oid 2024-09-18T11:30:57.384 INFO:tasks.rados.rados.0.smithi119.stdout:1102: set_chunk oid 39 offset: 1910272 length: 329728 target oid smithi11939506- offset: 1910272 snap 19 2024-09-18T11:30:57.384 INFO:tasks.rados.rados.0.smithi119.stdout:1103: tier_evict oid 49 2024-09-18T11:30:57.384 INFO:tasks.rados.rados.0.smithi119.stdout:1103: tier_evict oid 49 snap -1 2024-09-18T11:30:57.384 INFO:tasks.rados.rados.0.smithi119.stdout:1104: set_chunk oid 17 target oid 2024-09-18T11:30:57.392 INFO:tasks.rados.rados.0.smithi119.stdout:1104: set_chunk oid 17 offset: 2091520 length: 709120 target oid d4b4fff22a71c591f3e871de3fede050f6721df41a8c639a614c0b94ba6762c2 offset: 0 snap -1 2024-09-18T11:30:57.393 INFO:tasks.rados.rados.0.smithi119.stdout:1105: write oid 21 current snap is 21 2024-09-18T11:30:57.393 INFO:tasks.rados.rados.0.smithi119.stdout:1101: finishing write tid 1 to smithi11939506-16 2024-09-18T11:30:57.393 INFO:tasks.rados.rados.0.smithi119.stdout:1105: seq_num 190 ranges {793330=689782,2035461=617731,3101545=91785} 2024-09-18T11:30:57.397 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:57.395+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:57.400 INFO:tasks.rados.rados.0.smithi119.stdout:1105: writing smithi11939506-21 from 793330 to 1483112 tid 1 2024-09-18T11:30:57.407 INFO:tasks.rados.rados.0.smithi119.stdout:1105: writing smithi11939506-21 from 2035461 to 2653192 tid 2 2024-09-18T11:30:57.411 INFO:tasks.rados.rados.0.smithi119.stdout:1105: writing smithi11939506-21 from 3101545 to 3193330 tid 3 2024-09-18T11:30:57.411 INFO:tasks.rados.rados.0.smithi119.stdout:1106: read oid 28 snap -1 2024-09-18T11:30:57.411 INFO:tasks.rados.rados.0.smithi119.stdout:1106: expect (ObjNum 122 snap 11 seq_num 122) 2024-09-18T11:30:57.411 INFO:tasks.rados.rados.0.smithi119.stdout:1107: write (excl) oid 48 current snap is 21 2024-09-18T11:30:57.411 INFO:tasks.rados.rados.0.smithi119.stdout:1107: seq_num 191 ranges {410137=481384,1410296=761389,2728968=481169} 2024-09-18T11:30:57.415 INFO:tasks.rados.rados.0.smithi119.stdout:1107: writing smithi11939506-48 from 410137 to 891521 tid 1 2024-09-18T11:30:57.422 INFO:tasks.rados.rados.0.smithi119.stdout:1107: writing smithi11939506-48 from 1410296 to 2171685 tid 2 2024-09-18T11:30:57.429 INFO:tasks.rados.rados.0.smithi119.stdout:1107: writing smithi11939506-48 from 2728968 to 3210137 tid 3 2024-09-18T11:30:57.429 INFO:tasks.rados.rados.0.smithi119.stdout:1108: read oid 5 snap -1 2024-09-18T11:30:57.429 INFO:tasks.rados.rados.0.smithi119.stdout:1108: expect deleted 2024-09-18T11:30:57.429 INFO:tasks.rados.rados.0.smithi119.stdout:1109: snap_create 2024-09-18T11:30:57.429 INFO:tasks.rados.rados.0.smithi119.stdout:1102: finishing set_chunk to oid 39 2024-09-18T11:30:57.429 INFO:tasks.rados.rados.0.smithi119.stdout:1102: got expected ENOENT (src dne) 2024-09-18T11:30:57.429 INFO:tasks.rados.rados.0.smithi119.stdout:1103: got (22) Invalid argument 2024-09-18T11:30:57.429 INFO:tasks.rados.rados.0.smithi119.stdout:1101: finishing write tid 2 to smithi11939506-16 2024-09-18T11:30:57.429 INFO:tasks.rados.rados.0.smithi119.stdout:1101: finishing write tid 3 to smithi11939506-16 2024-09-18T11:30:57.429 INFO:tasks.rados.rados.0.smithi119.stdout:1101: finishing write tid 4 to smithi11939506-16 2024-09-18T11:30:57.430 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 117 (ObjNum 189 snap 21 seq_num 189) dirty exists 2024-09-18T11:30:57.430 INFO:tasks.rados.rados.0.smithi119.stdout:1101: left oid 16 (ObjNum 189 snap 21 seq_num 189) 2024-09-18T11:30:57.430 INFO:tasks.rados.rados.0.smithi119.stdout:1104: finishing set_chunk to oid 17 2024-09-18T11:30:57.430 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 163 (ObjNum 179 snap 20 seq_num 179) dirty exists 2024-09-18T11:30:57.430 INFO:tasks.rados.rados.0.smithi119.stdout:1105: finishing write tid 1 to smithi11939506-21 2024-09-18T11:30:57.430 INFO:tasks.rados.rados.0.smithi119.stdout:1105: finishing write tid 2 to smithi11939506-21 2024-09-18T11:30:57.430 INFO:tasks.rados.rados.0.smithi119.stdout:1105: finishing write tid 3 to smithi11939506-21 2024-09-18T11:30:57.430 INFO:tasks.rados.rados.0.smithi119.stdout:1106: expect (ObjNum 122 snap 11 seq_num 122) 2024-09-18T11:30:57.438 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:30:57.454 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:57.454 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:57.469 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:57.470 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:57.483 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:57.483 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:57.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:57.483 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:57.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.484 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:900 4.0 4:0aec5e1c:::smithi11939506-50:head [read 0~2974806,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:57.484 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.133226+0000", 2024-09-18T11:30:57.484 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.347172571000002, 2024-09-18T11:30:57.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059578069999999999, 2024-09-18T11:30:57.484 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.484 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:57.484 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:57.484 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:57.484 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:57.484 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 900 2024-09-18T11:30:57.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.485 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133226+0000", 2024-09-18T11:30:57.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133226+0000", 2024-09-18T11:30:57.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133225+0000", 2024-09-18T11:30:57.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:57.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133234+0000", 2024-09-18T11:30:57.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5799999999999992e-06 2024-09-18T11:30:57.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133242+0000", 2024-09-18T11:30:57.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4039999999999995e-06 2024-09-18T11:30:57.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133256+0000", 2024-09-18T11:30:57.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3278000000000001e-05 2024-09-18T11:30:57.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133293+0000", 2024-09-18T11:30:57.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7799000000000002e-05 2024-09-18T11:30:57.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:57.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133298+0000", 2024-09-18T11:30:57.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9110000000000001e-06 2024-09-18T11:30:57.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.137554+0000", 2024-09-18T11:30:57.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042556099999999999 2024-09-18T11:30:57.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.137713+0000", 2024-09-18T11:30:57.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015937199999999999 2024-09-18T11:30:57.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139184+0000", 2024-09-18T11:30:57.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001470694 2024-09-18T11:30:57.489 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.489 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.489 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.489 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:901 4.0 4:0aec5e1c:::smithi11939506-50:head [sparse-read 0~1781150] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:57.489 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:57.489 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.347133360999997, 2024-09-18T11:30:57.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063828720000000004, 2024-09-18T11:30:57.489 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.490 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:57.490 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:57.490 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:57.490 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:57.490 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 901 2024-09-18T11:30:57.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.490 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:57.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:57.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:30:57.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:57.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133268+0000", 2024-09-18T11:30:57.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9610000000000001e-06 2024-09-18T11:30:57.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133280+0000", 2024-09-18T11:30:57.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2481e-05 2024-09-18T11:30:57.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133285+0000", 2024-09-18T11:30:57.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5469999999999998e-06 2024-09-18T11:30:57.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133317+0000", 2024-09-18T11:30:57.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1838999999999998e-05 2024-09-18T11:30:57.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:57.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133320+0000", 2024-09-18T11:30:57.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3009999999999998e-06 2024-09-18T11:30:57.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139225+0000", 2024-09-18T11:30:57.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059047919999999999 2024-09-18T11:30:57.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139330+0000", 2024-09-18T11:30:57.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104924 2024-09-18T11:30:57.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139648+0000", 2024-09-18T11:30:57.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00031854799999999999 2024-09-18T11:30:57.494 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.494 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.494 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.495 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:30:57.495 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:30:57.495 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.310979320999998, 2024-09-18T11:30:57.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:30:57.495 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.495 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:57.495 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:57.495 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:57.495 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:57.495 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:30:57.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.495 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:30:57.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:30:57.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:30:57.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:57.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:30:57.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:30:57.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:30:57.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:30:57.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:30:57.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:30:57.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:30:57.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:30:57.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:30:57.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:30:57.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:30:57.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:30:57.498 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.499 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.499 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.499 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1001 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:57.499 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:30:57.499 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.315828043000003, 2024-09-18T11:30:57.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007461082, 2024-09-18T11:30:57.499 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.499 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:57.499 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:57.499 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:57.499 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:57.500 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1001 2024-09-18T11:30:57.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.500 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:30:57.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:30:57.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164570+0000", 2024-09-18T11:30:57.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:57.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164574+0000", 2024-09-18T11:30:57.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.822e-06 2024-09-18T11:30:57.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164582+0000", 2024-09-18T11:30:57.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3410000000000003e-06 2024-09-18T11:30:57.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164587+0000", 2024-09-18T11:30:57.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769999999999999e-06 2024-09-18T11:30:57.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164702+0000", 2024-09-18T11:30:57.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115705 2024-09-18T11:30:57.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:57.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164709+0000", 2024-09-18T11:30:57.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-09-18T11:30:57.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170296+0000", 2024-09-18T11:30:57.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055876390000000001 2024-09-18T11:30:57.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170418+0000", 2024-09-18T11:30:57.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121549 2024-09-18T11:30:57.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172032+0000", 2024-09-18T11:30:57.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00161407 2024-09-18T11:30:57.504 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.504 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.504 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.504 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:57.504 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:57.504 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.315801327000003, 2024-09-18T11:30:57.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:30:57.505 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.505 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:57.505 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:57.505 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:57.505 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:57.505 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:30:57.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.505 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:57.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:57.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:30:57.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:57.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:30:57.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:30:57.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:30:57.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:30:57.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:30:57.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:30:57.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:30:57.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:30:57.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:57.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:30:57.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:30:57.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:30:57.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:30:57.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:30:57.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:30:57.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:30:57.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:30:57.509 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.509 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.509 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.509 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:57.510 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:30:57.510 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.315783037999999, 2024-09-18T11:30:57.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:30:57.510 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.510 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:57.510 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:57.510 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:57.510 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:57.510 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:30:57.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.510 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:30:57.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:30:57.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:30:57.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:57.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:30:57.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:30:57.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:30:57.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:30:57.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:30:57.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:30:57.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:30:57.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:30:57.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:57.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:30:57.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:30:57.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:30:57.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:30:57.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:30:57.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:30:57.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:30:57.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:30:57.515 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.515 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.515 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.515 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:57.515 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:30:57.515 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.315722268999998, 2024-09-18T11:30:57.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:30:57.515 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.515 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:57.515 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:57.516 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:57.516 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:57.516 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:30:57.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.516 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:30:57.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:30:57.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:30:57.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:57.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:30:57.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:30:57.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:30:57.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:30:57.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:30:57.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:30:57.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:30:57.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:30:57.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:57.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:30:57.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:30:57.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:30:57.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:30:57.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:30:57.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:30:57.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:30:57.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:30:57.520 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.520 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.520 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.520 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:57.520 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:30:57.521 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 42.315699948000002, 2024-09-18T11:30:57.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:30:57.521 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.521 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:57.521 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:57.521 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:57.521 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:57.521 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:30:57.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.521 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:30:57.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:30:57.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:30:57.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:57.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:30:57.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:30:57.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:30:57.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:30:57.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:30:57.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:30:57.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:30:57.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:30:57.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:57.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:30:57.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:30:57.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:30:57.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:30:57.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:30:57.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:30:57.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:30:57.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:30:57.526 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.526 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.526 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.526 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1018 4.0 4:0aec5e1c:::smithi11939506-50:head [write 1622620~730352] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:30:57.526 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:30:57.526 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.799465392000002, 2024-09-18T11:30:57.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064405160000000003, 2024-09-18T11:30:57.526 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.526 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.526 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:57.527 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:57.527 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:57.527 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1018 2024-09-18T11:30:57.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.527 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:30:57.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:30:57.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680932+0000", 2024-09-18T11:30:57.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:57.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681334+0000", 2024-09-18T11:30:57.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000401803 2024-09-18T11:30:57.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681350+0000", 2024-09-18T11:30:57.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6218999999999998e-05 2024-09-18T11:30:57.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681374+0000", 2024-09-18T11:30:57.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4088e-05 2024-09-18T11:30:57.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681415+0000", 2024-09-18T11:30:57.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1307000000000003e-05 2024-09-18T11:30:57.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681507+0000", 2024-09-18T11:30:57.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1867999999999995e-05 2024-09-18T11:30:57.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:57.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681671+0000", 2024-09-18T11:30:57.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016406 2024-09-18T11:30:57.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:57.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.683856+0000", 2024-09-18T11:30:57.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002184624 2024-09-18T11:30:57.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:57.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687266+0000", 2024-09-18T11:30:57.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034105490000000001 2024-09-18T11:30:57.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687321+0000", 2024-09-18T11:30:57.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4960999999999998e-05 2024-09-18T11:30:57.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687374+0000", 2024-09-18T11:30:57.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2553000000000003e-05 2024-09-18T11:30:57.532 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.532 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.532 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.532 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:30:57.532 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:30:57.533 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.534575064999999, 2024-09-18T11:30:57.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:30:57.533 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.533 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.533 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:57.533 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:57.533 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:57.533 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:30:57.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.533 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:30:57.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:30:57.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:30:57.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:30:57.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:30:57.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:30:57.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:30:57.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:30:57.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:30:57.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:30:57.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:30:57.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:30:57.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:30:57.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:30:57.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:57.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:30:57.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:30:57.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:57.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:30:57.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:30:57.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:57.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:30:57.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:30:57.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:30:57.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:30:57.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:30:57.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:30:57.538 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.538 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.538 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.539 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1121 4.f e38/22 4:f5d7df41:::smithi11939506-16:head v 38'34, mlcod=30'32)", 2024-09-18T11:30:57.539 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:30:57.539 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.514587806000002, 2024-09-18T11:30:57.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060953680000000003, 2024-09-18T11:30:57.539 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.539 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.539 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:30:57.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:30:57.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965810+0000", 2024-09-18T11:30:57.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:57.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966142+0000", 2024-09-18T11:30:57.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000331591 2024-09-18T11:30:57.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966154+0000", 2024-09-18T11:30:57.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.253e-05 2024-09-18T11:30:57.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966160+0000", 2024-09-18T11:30:57.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7130000000000002e-06 2024-09-18T11:30:57.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966196+0000", 2024-09-18T11:30:57.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6276999999999998e-05 2024-09-18T11:30:57.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966217+0000", 2024-09-18T11:30:57.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0335000000000001e-05 2024-09-18T11:30:57.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.971872+0000", 2024-09-18T11:30:57.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056549290000000004 2024-09-18T11:30:57.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.971906+0000", 2024-09-18T11:30:57.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4702000000000001e-05 2024-09-18T11:30:57.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.543 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.543 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:29 3.2 3:5fa8fcfa:::ec59520759824f0488c7a3ca135dacb324de2330e16b7f54053de5719c4e96b6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:57.544 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:30:57.544 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.800217148999998, 2024-09-18T11:30:57.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064126499999999998, 2024-09-18T11:30:57.544 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.544 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.544 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:30:57.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:30:57.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680181+0000", 2024-09-18T11:30:57.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:57.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680654+0000", 2024-09-18T11:30:57.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047334900000000002 2024-09-18T11:30:57.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680666+0000", 2024-09-18T11:30:57.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2232000000000001e-05 2024-09-18T11:30:57.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680672+0000", 2024-09-18T11:30:57.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:30:57.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681222+0000", 2024-09-18T11:30:57.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00054928699999999997 2024-09-18T11:30:57.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681332+0000", 2024-09-18T11:30:57.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110634 2024-09-18T11:30:57.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:57.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681552+0000", 2024-09-18T11:30:57.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021926899999999999 2024-09-18T11:30:57.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:57.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682950+0000", 2024-09-18T11:30:57.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013982669999999999 2024-09-18T11:30:57.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:57.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686503+0000", 2024-09-18T11:30:57.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035536130000000002 2024-09-18T11:30:57.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686553+0000", 2024-09-18T11:30:57.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9801e-05 2024-09-18T11:30:57.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686594+0000", 2024-09-18T11:30:57.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1022000000000001e-05 2024-09-18T11:30:57.549 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.549 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.549 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.549 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:30 3.2 3:49074e56:::7fc21ffee1ba17f2a9197c61ff60ef0b03804c121c8aeb83ea424b7ac77081e9:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:30:57.550 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.680698+0000", 2024-09-18T11:30:57.550 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.799700831000003, 2024-09-18T11:30:57.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060349130000000003, 2024-09-18T11:30:57.550 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.550 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.550 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680698+0000", 2024-09-18T11:30:57.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680698+0000", 2024-09-18T11:30:57.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680697+0000", 2024-09-18T11:30:57.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:57.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680750+0000", 2024-09-18T11:30:57.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3170000000000001e-05 2024-09-18T11:30:57.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680760+0000", 2024-09-18T11:30:57.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3379999999999994e-06 2024-09-18T11:30:57.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680766+0000", 2024-09-18T11:30:57.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8249999999999999e-06 2024-09-18T11:30:57.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682201+0000", 2024-09-18T11:30:57.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001434868 2024-09-18T11:30:57.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682294+0000", 2024-09-18T11:30:57.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2965000000000002e-05 2024-09-18T11:30:57.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:57.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682445+0000", 2024-09-18T11:30:57.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015103600000000001 2024-09-18T11:30:57.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:57.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.683524+0000", 2024-09-18T11:30:57.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0010789980000000001 2024-09-18T11:30:57.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:57.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686687+0000", 2024-09-18T11:30:57.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003163052 2024-09-18T11:30:57.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686714+0000", 2024-09-18T11:30:57.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6369000000000001e-05 2024-09-18T11:30:57.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686733+0000", 2024-09-18T11:30:57.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9117999999999999e-05 2024-09-18T11:30:57.555 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.555 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.555 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.555 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1199 4.0 4:0aec5e1c:::smithi11939506-50:head [write 763051~794393] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-18T11:30:57.555 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:30:57.556 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.283403024999998, 2024-09-18T11:30:57.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067768530000000002, 2024-09-18T11:30:57.556 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.556 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.556 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:57.556 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:57.556 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:57.556 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1199 2024-09-18T11:30:57.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.556 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:30:57.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:30:57.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196994+0000", 2024-09-18T11:30:57.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:57.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197355+0000", 2024-09-18T11:30:57.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036035199999999998 2024-09-18T11:30:57.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197366+0000", 2024-09-18T11:30:57.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.17e-05 2024-09-18T11:30:57.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197371+0000", 2024-09-18T11:30:57.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6099999999999999e-06 2024-09-18T11:30:57.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197416+0000", 2024-09-18T11:30:57.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5099000000000003e-05 2024-09-18T11:30:57.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:57.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197420+0000", 2024-09-18T11:30:57.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3189999999999996e-06 2024-09-18T11:30:57.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199392+0000", 2024-09-18T11:30:57.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019720139999999998 2024-09-18T11:30:57.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199452+0000", 2024-09-18T11:30:57.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9929999999999997e-05 2024-09-18T11:30:57.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:57.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199565+0000", 2024-09-18T11:30:57.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112811 2024-09-18T11:30:57.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:57.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.202108+0000", 2024-09-18T11:30:57.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025429060000000002 2024-09-18T11:30:57.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:57.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203714+0000", 2024-09-18T11:30:57.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001606335 2024-09-18T11:30:57.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203745+0000", 2024-09-18T11:30:57.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0837999999999998e-05 2024-09-18T11:30:57.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203773+0000", 2024-09-18T11:30:57.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7562000000000001e-05 2024-09-18T11:30:57.561 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.562 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.562 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.562 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1661 4.0 4:019b6e09:::smithi11939506-40:17 [read 0~3445402,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:30:57.562 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:30:57.562 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 20.130502362000001, 2024-09-18T11:30:57.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075190880000000002, 2024-09-18T11:30:57.562 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.562 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:57.562 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:57.562 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:57.563 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:57.563 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1661 2024-09-18T11:30:57.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.563 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:30:57.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:30:57.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:30:57.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:57.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349899+0000", 2024-09-18T11:30:57.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9749999999999999e-06 2024-09-18T11:30:57.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349906+0000", 2024-09-18T11:30:57.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9290000000000001e-06 2024-09-18T11:30:57.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349912+0000", 2024-09-18T11:30:57.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9900000000000002e-06 2024-09-18T11:30:57.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355645+0000", 2024-09-18T11:30:57.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057329549999999996 2024-09-18T11:30:57.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355776+0000", 2024-09-18T11:30:57.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130975 2024-09-18T11:30:57.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357416+0000", 2024-09-18T11:30:57.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001639649 2024-09-18T11:30:57.566 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.566 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.566 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.567 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1761 4.0 4:0aec5e1c:::smithi11939506-50:head [write 769787~704463,stat] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:30:57.567 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:30:57.567 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.178009948, 2024-09-18T11:30:57.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074082879999999999, 2024-09-18T11:30:57.567 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.567 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.567 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:57.567 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:57.567 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:57.567 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1761 2024-09-18T11:30:57.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.567 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:30:57.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:30:57.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302386+0000", 2024-09-18T11:30:57.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:57.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302889+0000", 2024-09-18T11:30:57.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050318499999999996 2024-09-18T11:30:57.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302930+0000", 2024-09-18T11:30:57.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0198999999999999e-05 2024-09-18T11:30:57.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302965+0000", 2024-09-18T11:30:57.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5292e-05 2024-09-18T11:30:57.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303046+0000", 2024-09-18T11:30:57.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1254999999999994e-05 2024-09-18T11:30:57.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:30:57.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303054+0000", 2024-09-18T11:30:57.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2800000000000003e-06 2024-09-18T11:30:57.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304658+0000", 2024-09-18T11:30:57.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016039520000000001 2024-09-18T11:30:57.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304724+0000", 2024-09-18T11:30:57.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5888000000000003e-05 2024-09-18T11:30:57.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:57.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304853+0000", 2024-09-18T11:30:57.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012912700000000001 2024-09-18T11:30:57.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:57.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309084+0000", 2024-09-18T11:30:57.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042311229999999998 2024-09-18T11:30:57.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:57.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309738+0000", 2024-09-18T11:30:57.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00065401399999999996 2024-09-18T11:30:57.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309770+0000", 2024-09-18T11:30:57.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1652000000000001e-05 2024-09-18T11:30:57.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309797+0000", 2024-09-18T11:30:57.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7052000000000001e-05 2024-09-18T11:30:57.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.573 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.573 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1951 4.0 4:019b6e09:::smithi11939506-40:13 [sparse-read 0~2626478] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:30:57.573 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.339659+0000", 2024-09-18T11:30:57.573 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.1407402419999997, 2024-09-18T11:30:57.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067895999999999998, 2024-09-18T11:30:57.574 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.574 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:30:57.574 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:57.574 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:57.574 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:57.574 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1951 2024-09-18T11:30:57.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.574 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339659+0000", 2024-09-18T11:30:57.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339659+0000", 2024-09-18T11:30:57.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339658+0000", 2024-09-18T11:30:57.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:57.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339661+0000", 2024-09-18T11:30:57.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3749999999999999e-06 2024-09-18T11:30:57.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339668+0000", 2024-09-18T11:30:57.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1380000000000001e-06 2024-09-18T11:30:57.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339675+0000", 2024-09-18T11:30:57.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7499999999999997e-06 2024-09-18T11:30:57.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339748+0000", 2024-09-18T11:30:57.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2885000000000004e-05 2024-09-18T11:30:57.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:30:57.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339751+0000", 2024-09-18T11:30:57.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.446e-06 2024-09-18T11:30:57.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345249+0000", 2024-09-18T11:30:57.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054983360000000004 2024-09-18T11:30:57.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345332+0000", 2024-09-18T11:30:57.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3245999999999995e-05 2024-09-18T11:30:57.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346448+0000", 2024-09-18T11:30:57.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0011160530000000001 2024-09-18T11:30:57.578 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.578 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.578 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.579 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:306 4.f e69/22 4:f5d7df41:::smithi11939506-16:17 v 69'94, mlcod=65'88)", 2024-09-18T11:30:57.579 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:30:57.579 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.1292832309999996, 2024-09-18T11:30:57.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068024720000000004, 2024-09-18T11:30:57.579 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.579 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.579 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:30:57.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:30:57.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351115+0000", 2024-09-18T11:30:57.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:57.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351121+0000", 2024-09-18T11:30:57.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-09-18T11:30:57.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351126+0000", 2024-09-18T11:30:57.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5969999999999999e-06 2024-09-18T11:30:57.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351147+0000", 2024-09-18T11:30:57.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1160000000000001e-05 2024-09-18T11:30:57.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351518+0000", 2024-09-18T11:30:57.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00037079300000000001 2024-09-18T11:30:57.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351536+0000", 2024-09-18T11:30:57.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8168999999999998e-05 2024-09-18T11:30:57.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357870+0000", 2024-09-18T11:30:57.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063332650000000002 2024-09-18T11:30:57.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357918+0000", 2024-09-18T11:30:57.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8347999999999999e-05 2024-09-18T11:30:57.583 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.583 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.583 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.583 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2022 4.0 4:0aec5e1c:::smithi11939506-50:head [write 457190~552096,stat] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-18T11:30:57.583 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:30:57.583 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.4842450940000003, 2024-09-18T11:30:57.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006803974, 2024-09-18T11:30:57.584 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.584 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.584 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:57.584 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:57.584 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:57.584 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2022 2024-09-18T11:30:57.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.584 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:30:57.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:30:57.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996151+0000", 2024-09-18T11:30:57.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:30:57.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996461+0000", 2024-09-18T11:30:57.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000309587 2024-09-18T11:30:57.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996481+0000", 2024-09-18T11:30:57.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9949000000000002e-05 2024-09-18T11:30:57.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996489+0000", 2024-09-18T11:30:57.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9129999999999992e-06 2024-09-18T11:30:57.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996562+0000", 2024-09-18T11:30:57.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2807999999999998e-05 2024-09-18T11:30:57.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996687+0000", 2024-09-18T11:30:57.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012458200000000001 2024-09-18T11:30:57.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:30:57.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996909+0000", 2024-09-18T11:30:57.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222124 2024-09-18T11:30:57.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:57.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.000741+0000", 2024-09-18T11:30:57.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038320720000000002 2024-09-18T11:30:57.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:57.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002854+0000", 2024-09-18T11:30:57.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002113228 2024-09-18T11:30:57.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002911+0000", 2024-09-18T11:30:57.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7192999999999999e-05 2024-09-18T11:30:57.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002958+0000", 2024-09-18T11:30:57.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6242000000000002e-05 2024-09-18T11:30:57.589 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.589 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.589 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.589 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2075 3.2 3:5e6b2dd0:::abf5d1c1c6ff48a895de7f094078ad489e0511876df30b6bd8853cd5ff4a6267:head [write 0~2096640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-18T11:30:57.589 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:55.321813+0000", 2024-09-18T11:30:57.590 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 2.1585854210000002, 2024-09-18T11:30:57.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0061775409999999999, 2024-09-18T11:30:57.590 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:57.590 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:57.590 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:30:57.590 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:30:57.590 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:30:57.590 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2075 2024-09-18T11:30:57.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.590 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:57.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:57.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.321813+0000", 2024-09-18T11:30:57.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:57.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.321813+0000", 2024-09-18T11:30:57.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:57.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:57.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.321811+0000", 2024-09-18T11:30:57.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:30:57.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:57.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.322777+0000", 2024-09-18T11:30:57.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096617299999999997 2024-09-18T11:30:57.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:57.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.322796+0000", 2024-09-18T11:30:57.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.861e-05 2024-09-18T11:30:57.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:57.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.322803+0000", 2024-09-18T11:30:57.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3669999999999999e-06 2024-09-18T11:30:57.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:57.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.322847+0000", 2024-09-18T11:30:57.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3355000000000001e-05 2024-09-18T11:30:57.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:57.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.322948+0000", 2024-09-18T11:30:57.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101563 2024-09-18T11:30:57.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:30:57.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.323049+0000", 2024-09-18T11:30:57.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100426 2024-09-18T11:30:57.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:30:57.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.327713+0000", 2024-09-18T11:30:57.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046642280000000003 2024-09-18T11:30:57.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:30:57.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.327905+0000", 2024-09-18T11:30:57.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000192098 2024-09-18T11:30:57.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:57.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.327953+0000", 2024-09-18T11:30:57.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7907000000000001e-05 2024-09-18T11:30:57.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:57.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:57.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:57.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.327991+0000", 2024-09-18T11:30:57.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7954999999999998e-05 2024-09-18T11:30:57.595 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.595 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.595 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.595 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:57.595 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:57.595 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:57.596 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:57.596 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:57.597 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:30:57.598 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:57.598 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:57.598 INFO:tasks.rados.rados.0.smithi119.stdout:1105: finishing write tid 4 to smithi11939506-21 2024-09-18T11:30:57.598 INFO:tasks.rados.rados.0.smithi119.stdout:1105: finishing write tid 5 to smithi11939506-21 2024-09-18T11:30:57.598 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 121 (ObjNum 190 snap 21 seq_num 190) dirty exists 2024-09-18T11:30:57.599 INFO:tasks.rados.rados.0.smithi119.stdout:1105: left oid 21 (ObjNum 190 snap 21 seq_num 190) 2024-09-18T11:30:57.599 INFO:tasks.rados.rados.0.smithi119.stdout:1107: finishing write tid 1 to smithi11939506-48 2024-09-18T11:30:57.599 INFO:tasks.rados.rados.0.smithi119.stdout:1107: finishing write tid 2 to smithi11939506-48 2024-09-18T11:30:57.599 INFO:tasks.rados.rados.0.smithi119.stdout:1107: finishing write tid 3 to smithi11939506-48 2024-09-18T11:30:57.599 INFO:tasks.rados.rados.0.smithi119.stdout:1107: finishing write tid 4 to smithi11939506-48 2024-09-18T11:30:57.599 INFO:tasks.rados.rados.0.smithi119.stdout:1107: finishing write tid 5 to smithi11939506-48 2024-09-18T11:30:57.599 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 48 v 126 (ObjNum 191 snap 21 seq_num 191) dirty exists 2024-09-18T11:30:57.599 INFO:tasks.rados.rados.0.smithi119.stdout:1107: left oid 48 (ObjNum 191 snap 21 seq_num 191) 2024-09-18T11:30:57.599 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:57.600 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:57.600 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:57.600 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:57.600 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:57.601 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:30:57.601 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:57.601 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:30:57.602 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:57.595+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:57.697 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:57.696+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:57.713 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:57.713 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:57.722 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:30:57.790 INFO:teuthology.orchestra.run.smithi119.stderr:Nothing to do for non-filestore backend 2024-09-18T11:30:57.797 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:57.795+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:57.861 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:57.861 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:57.869 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:30:57.913 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:57.912+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:57.967 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:57.968 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:57.976 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:30:57.999 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:57.997+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:58.100 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:58.099+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:58.115 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:58.115 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:58.123 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:30:58.201 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:58.200+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:58.222 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:58.222 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:58.230 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:30:58.302 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:58.300+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:58.306 DEBUG:teuthology.orchestra.run.smithi119:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op apply-layout-settings --pool rbd 2024-09-18T11:30:58.369 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:58.370 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:58.371 INFO:tasks.rados.rados.0.smithi119.stdout:1101: done (8 left) 2024-09-18T11:30:58.371 INFO:tasks.rados.rados.0.smithi119.stdout:1102: done (7 left) 2024-09-18T11:30:58.371 INFO:tasks.rados.rados.0.smithi119.stdout:1103: done (6 left) 2024-09-18T11:30:58.372 INFO:tasks.rados.rados.0.smithi119.stdout:1104: done (5 left) 2024-09-18T11:30:58.372 INFO:tasks.rados.rados.0.smithi119.stdout:1105: done (4 left) 2024-09-18T11:30:58.372 INFO:tasks.rados.rados.0.smithi119.stdout:1106: done (3 left) 2024-09-18T11:30:58.372 INFO:tasks.rados.rados.0.smithi119.stdout:1107: done (2 left) 2024-09-18T11:30:58.372 INFO:tasks.rados.rados.0.smithi119.stdout:1108: done (1 left) 2024-09-18T11:30:58.372 INFO:tasks.rados.rados.0.smithi119.stdout:1109: done (0 left) 2024-09-18T11:30:58.372 INFO:tasks.rados.rados.0.smithi119.stdout:1110: read oid 7 snap -1 2024-09-18T11:30:58.372 INFO:tasks.rados.rados.0.smithi119.stdout:1110: expect (ObjNum 145 snap 18 seq_num 145) 2024-09-18T11:30:58.372 INFO:tasks.rados.rados.0.smithi119.stdout:1111: set_chunk oid 26 target oid 2024-09-18T11:30:58.381 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:30:58.381 INFO:tasks.rados.rados.0.smithi119.stdout:1111: set_chunk oid 26 offset: 979968 length: 646656 target oid 6029205ac3ec9343eddce68b7a0f3804c0448752b2bac99bb526fd5f0bbc67b1 offset: 0 snap -1 2024-09-18T11:30:58.382 INFO:tasks.rados.rados.0.smithi119.stdout:1112: snap_create 2024-09-18T11:30:58.382 INFO:tasks.rados.rados.0.smithi119.stdout:1110: expect (ObjNum 145 snap 18 seq_num 145) 2024-09-18T11:30:58.425 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:58.424+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:58.479 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:58.480 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:58.488 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:30:58.526 INFO:tasks.rados.rados.0.smithi119.stdout:1111: finishing set_chunk to oid 26 2024-09-18T11:30:58.526 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 127 (ObjNum 178 snap 20 seq_num 178) dirty exists 2024-09-18T11:30:58.529 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:58.528+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:58.604 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:58.603+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:58.627 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:58.627 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:58.635 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:30:58.719 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:58.718+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:58.773 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:58.773 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:58.781 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:30:58.822 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:58.821+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:58.907 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:58.905+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:58.919 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:58.919 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:58.928 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:30:59.008 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:30:59.006+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:30:59.065 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:59.065 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:59.073 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:30:59.117 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:59.116+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:59.171 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:59.171 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:59.179 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:30:59.220 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:59.219+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:59.278 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:59.278 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:59.286 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:30:59.310 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:30:59.309+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:30:59.378 INFO:tasks.rados.rados.0.smithi119.stdout:1110: done (2 left) 2024-09-18T11:30:59.378 INFO:tasks.rados.rados.0.smithi119.stdout:1111: done (1 left) 2024-09-18T11:30:59.378 INFO:tasks.rados.rados.0.smithi119.stdout:1112: done (0 left) 2024-09-18T11:30:59.378 INFO:tasks.rados.rados.0.smithi119.stdout:1113: write (excl) oid 32 current snap is 23 2024-09-18T11:30:59.378 INFO:tasks.rados.rados.0.smithi119.stdout:1113: seq_num 192 ranges {426944=556633,1468778=621400,2872744=354200} 2024-09-18T11:30:59.383 INFO:tasks.rados.rados.0.smithi119.stdout:1113: writing smithi11939506-32 from 426944 to 983577 tid 1 2024-09-18T11:30:59.387 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:59.387 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:59.388 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T11:30:59.389 INFO:tasks.rados.rados.0.smithi119.stdout:1113: writing smithi11939506-32 from 1468778 to 2090178 tid 2 2024-09-18T11:30:59.396 INFO:tasks.rados.rados.0.smithi119.stdout:1113: writing smithi11939506-32 from 2872744 to 3226944 tid 3 2024-09-18T11:30:59.396 INFO:tasks.rados.rados.0.smithi119.stdout:1113: finishing write tid 1 to smithi11939506-32 2024-09-18T11:30:59.396 INFO:tasks.rados.rados.0.smithi119.stdout:1114: read oid 41 snap 16 2024-09-18T11:30:59.396 INFO:tasks.rados.rados.0.smithi119.stdout:1114: expect (ObjNum 104 snap 6 seq_num 104) 2024-09-18T11:30:59.396 INFO:tasks.rados.rados.0.smithi119.stdout:1115: write (excl) oid 45 current snap is 23 2024-09-18T11:30:59.396 INFO:tasks.rados.rados.0.smithi119.stdout:1115: seq_num 193 ranges {443751=631882,1810907=765058,3067461=176290} 2024-09-18T11:30:59.397 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:30:59.402 INFO:tasks.rados.rados.0.smithi119.stdout:1115: writing smithi11939506-45 from 443751 to 1075633 tid 1 2024-09-18T11:30:59.410 INFO:tasks.rados.rados.0.smithi119.stdout:1115: writing smithi11939506-45 from 1810907 to 2575965 tid 2 2024-09-18T11:30:59.411 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:59.409+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:59.415 INFO:tasks.rados.rados.0.smithi119.stdout:1115: writing smithi11939506-45 from 3067461 to 3243751 tid 3 2024-09-18T11:30:59.415 INFO:tasks.rados.rados.0.smithi119.stdout:1116: read oid 29 snap -1 2024-09-18T11:30:59.415 INFO:tasks.rados.rados.0.smithi119.stdout:1116: expect (ObjNum 159 snap 20 seq_num 159) 2024-09-18T11:30:59.415 INFO:tasks.rados.rados.0.smithi119.stdout:1117: tier_promote oid 34 2024-09-18T11:30:59.415 INFO:tasks.rados.rados.0.smithi119.stdout:1113: finishing write tid 2 to smithi11939506-32 2024-09-18T11:30:59.415 INFO:tasks.rados.rados.0.smithi119.stdout:1118: write (excl) oid 10 current snap is 23 2024-09-18T11:30:59.415 INFO:tasks.rados.rados.0.smithi119.stdout:1113: finishing write tid 3 to smithi11939506-32 2024-09-18T11:30:59.415 INFO:tasks.rados.rados.0.smithi119.stdout:1118: seq_num 194 ranges {460558=707131,1869389=508716,3094884=165674} 2024-09-18T11:30:59.422 INFO:tasks.rados.rados.0.smithi119.stdout:1118: writing smithi11939506-10 from 460558 to 1167689 tid 1 2024-09-18T11:30:59.423 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:59.422+0000 7f6bf3052640 -1 osd.5 75 handle_osd_map injecting map crc failure 2024-09-18T11:30:59.429 INFO:tasks.rados.rados.0.smithi119.stdout:1118: writing smithi11939506-10 from 1869389 to 2378105 tid 2 2024-09-18T11:30:59.434 INFO:tasks.rados.rados.0.smithi119.stdout:1118: writing smithi11939506-10 from 3094884 to 3260558 tid 3 2024-09-18T11:30:59.435 INFO:tasks.rados.rados.0.smithi119.stdout:1119: read oid 8 snap -1 2024-09-18T11:30:59.435 INFO:tasks.rados.rados.0.smithi119.stdout:1119: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-18T11:30:59.435 INFO:tasks.rados.rados.0.smithi119.stdout:1120: write (excl) oid 2 current snap is 23 2024-09-18T11:30:59.435 INFO:tasks.rados.rados.0.smithi119.stdout:1113: finishing write tid 4 to smithi11939506-32 2024-09-18T11:30:59.435 INFO:tasks.rados.rados.0.smithi119.stdout:1113: finishing write tid 5 to smithi11939506-32 2024-09-18T11:30:59.435 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 103 (ObjNum 192 snap 23 seq_num 192) dirty exists 2024-09-18T11:30:59.435 INFO:tasks.rados.rados.0.smithi119.stdout:1113: left oid 32 (ObjNum 192 snap 23 seq_num 192) 2024-09-18T11:30:59.435 INFO:tasks.rados.rados.0.smithi119.stdout:1114: expect (ObjNum 104 snap 6 seq_num 104) 2024-09-18T11:30:59.436 INFO:teuthology.orchestra.run.smithi119.stderr:Nothing to do for non-filestore backend 2024-09-18T11:30:59.511 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:59.511 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:59.512 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:30:59.509+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:30:59.520 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:30:59.602 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:30:59.613 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:30:59.611+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:30:59.661 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:30:59.661 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:30:59.669 INFO:tasks.rados.rados.0.smithi119.stdout:1115: finishing write tid 1 to smithi11939506-45 2024-09-18T11:30:59.669 INFO:tasks.rados.rados.0.smithi119.stdout:1120: seq_num 195 ranges {477365=782380,1927871=768727,3122307=155058} 2024-09-18T11:30:59.670 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:30:59.675 INFO:tasks.rados.rados.0.smithi119.stdout:1120: writing smithi11939506-2 from 477365 to 1259745 tid 1 2024-09-18T11:30:59.683 INFO:tasks.rados.rados.0.smithi119.stdout:1120: writing smithi11939506-2 from 1927871 to 2696598 tid 2 2024-09-18T11:30:59.687 INFO:tasks.rados.rados.0.smithi119.stdout:1120: writing smithi11939506-2 from 3122307 to 3277365 tid 3 2024-09-18T11:30:59.687 INFO:tasks.rados.rados.0.smithi119.stdout:1113: done (7 left) 2024-09-18T11:30:59.687 INFO:tasks.rados.rados.0.smithi119.stdout:1114: done (6 left) 2024-09-18T11:30:59.687 INFO:tasks.rados.rados.0.smithi119.stdout:1121: write (excl) oid 14 current snap is 23 2024-09-18T11:30:59.688 INFO:tasks.rados.rados.0.smithi119.stdout:1115: finishing write tid 2 to smithi11939506-45 2024-09-18T11:30:59.688 INFO:tasks.rados.rados.0.smithi119.stdout:1116: expect (ObjNum 159 snap 20 seq_num 159) 2024-09-18T11:30:59.708 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:30:59.713 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:30:59.711+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:30:59.725 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:30:59.725 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:30:59.741 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:30:59.741 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:30:59.756 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:30:59.756 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:30:59.772 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:30:59.772 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:30:59.774 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:59.774 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:30:59.775 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:30:59.775 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:30:59.775 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:59.783 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:30:59.787 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:30:59.788 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:30:59.804 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:30:59.805 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:30:59.814 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:30:59.813+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:30:59.820 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:30:59.820 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:30:59.824 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:30:59.824 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:30:59.888 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:30:59.889 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:30:59.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:30:59.889 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:30:59.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.889 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1046 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'24, mlcod=30'21)", 2024-09-18T11:30:59.889 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:30:59.889 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.712652169999998, 2024-09-18T11:30:59.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042519100000000002, 2024-09-18T11:30:59.889 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:59.889 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:59.889 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:59.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:59.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:30:59.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:59.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:30:59.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:59.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173782+0000", 2024-09-18T11:30:59.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:59.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:59.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174207+0000", 2024-09-18T11:30:59.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00042479799999999999 2024-09-18T11:30:59.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:59.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174224+0000", 2024-09-18T11:30:59.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7513999999999999e-05 2024-09-18T11:30:59.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:59.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174232+0000", 2024-09-18T11:30:59.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0840000000000003e-06 2024-09-18T11:30:59.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:59.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174297+0000", 2024-09-18T11:30:59.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5215000000000005e-05 2024-09-18T11:30:59.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:59.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174336+0000", 2024-09-18T11:30:59.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9403000000000002e-05 2024-09-18T11:30:59.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:59.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.177990+0000", 2024-09-18T11:30:59.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036533709999999999 2024-09-18T11:30:59.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:59.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.178035+0000", 2024-09-18T11:30:59.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5495999999999999e-05 2024-09-18T11:30:59.893 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.893 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:59.893 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.893 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1047 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'25, mlcod=35'23)", 2024-09-18T11:30:59.893 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:30:59.893 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.706679972000003, 2024-09-18T11:30:59.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038129129999999998, 2024-09-18T11:30:59.893 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:59.893 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:59.894 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:59.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:59.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:30:59.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:59.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:30:59.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:59.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179754+0000", 2024-09-18T11:30:59.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:59.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:59.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180088+0000", 2024-09-18T11:30:59.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00033395000000000001 2024-09-18T11:30:59.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:59.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180105+0000", 2024-09-18T11:30:59.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7039000000000001e-05 2024-09-18T11:30:59.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:59.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180112+0000", 2024-09-18T11:30:59.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3710000000000002e-06 2024-09-18T11:30:59.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:59.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180176+0000", 2024-09-18T11:30:59.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3178999999999999e-05 2024-09-18T11:30:59.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:59.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180197+0000", 2024-09-18T11:30:59.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1687000000000001e-05 2024-09-18T11:30:59.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:59.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183542+0000", 2024-09-18T11:30:59.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003344877 2024-09-18T11:30:59.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:59.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183568+0000", 2024-09-18T11:30:59.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6146e-05 2024-09-18T11:30:59.897 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.897 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:59.898 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.898 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:30:59.898 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:30:59.898 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.903061303000001, 2024-09-18T11:30:59.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:30:59.898 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:59.898 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:59.898 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:59.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:59.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:30:59.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:59.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:30:59.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:59.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:30:59.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:59.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:59.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:30:59.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:30:59.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:59.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:30:59.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:30:59.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:59.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:30:59.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:30:59.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:59.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:30:59.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:30:59.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:59.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:30:59.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:30:59.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:59.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:30:59.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:30:59.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:59.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:30:59.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:30:59.902 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.902 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:59.902 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.902 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:30:59.902 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:30:59.902 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.668230350000002, 2024-09-18T11:30:59.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:30:59.902 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:59.902 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:59.903 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:59.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:59.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:30:59.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:59.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:30:59.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:59.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:30:59.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:30:59.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:59.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:30:59.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:30:59.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:59.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:30:59.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:30:59.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:59.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:30:59.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:30:59.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:59.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:30:59.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:30:59.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:59.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:30:59.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:30:59.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:59.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:30:59.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:30:59.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:59.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:30:59.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:30:59.906 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.906 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:59.906 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.906 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:244 4.d e49/22 4:b5a463bf:::smithi11939506-20:10 v 49'68, mlcod=48'65)", 2024-09-18T11:30:59.906 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:30.220263+0000", 2024-09-18T11:30:59.907 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 29.666172759999998, 2024-09-18T11:30:59.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032524960000000001, 2024-09-18T11:30:59.907 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:59.907 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:59.907 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:59.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:59.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220263+0000", 2024-09-18T11:30:59.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:59.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220263+0000", 2024-09-18T11:30:59.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:59.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220262+0000", 2024-09-18T11:30:59.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:30:59.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:59.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220266+0000", 2024-09-18T11:30:59.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.895e-06 2024-09-18T11:30:59.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:59.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220278+0000", 2024-09-18T11:30:59.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1962e-05 2024-09-18T11:30:59.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:59.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220284+0000", 2024-09-18T11:30:59.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1299999999999998e-06 2024-09-18T11:30:59.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:59.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220334+0000", 2024-09-18T11:30:59.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9789000000000001e-05 2024-09-18T11:30:59.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:59.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.220356+0000", 2024-09-18T11:30:59.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1897000000000001e-05 2024-09-18T11:30:59.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:59.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.223464+0000", 2024-09-18T11:30:59.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031082789999999998 2024-09-18T11:30:59.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:59.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:30.223515+0000", 2024-09-18T11:30:59.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0748999999999998e-05 2024-09-18T11:30:59.910 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.910 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:59.911 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.911 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1512 4.4 e51/22 4:2c93196f:::smithi11939506-28:head v 51'51, mlcod=51'48)", 2024-09-18T11:30:59.911 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:32.782201+0000", 2024-09-18T11:30:59.911 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.104234395999999, 2024-09-18T11:30:59.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00363554, 2024-09-18T11:30:59.911 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:59.911 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:59.911 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:59.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:59.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782201+0000", 2024-09-18T11:30:59.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:59.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782201+0000", 2024-09-18T11:30:59.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:59.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782200+0000", 2024-09-18T11:30:59.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:59.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:59.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782564+0000", 2024-09-18T11:30:59.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000364612 2024-09-18T11:30:59.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:59.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782585+0000", 2024-09-18T11:30:59.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0418999999999999e-05 2024-09-18T11:30:59.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:59.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782594+0000", 2024-09-18T11:30:59.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9570000000000008e-06 2024-09-18T11:30:59.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:59.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782663+0000", 2024-09-18T11:30:59.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9330000000000002e-05 2024-09-18T11:30:59.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:59.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782688+0000", 2024-09-18T11:30:59.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5420000000000001e-05 2024-09-18T11:30:59.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:59.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.785790+0000", 2024-09-18T11:30:59.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031016160000000002 2024-09-18T11:30:59.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:59.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.785836+0000", 2024-09-18T11:30:59.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6360999999999999e-05 2024-09-18T11:30:59.915 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.915 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:59.915 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.915 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:30:59.915 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:30:59.915 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 25.494324065000001, 2024-09-18T11:30:59.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:30:59.915 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:59.915 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:59.915 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:59.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:59.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:30:59.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:59.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:30:59.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:59.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:30:59.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:59.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:59.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:30:59.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:30:59.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:59.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:30:59.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:30:59.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:59.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:30:59.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:30:59.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:59.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:30:59.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:30:59.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:59.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:30:59.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:30:59.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:59.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:30:59.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:30:59.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:59.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:30:59.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:30:59.919 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.919 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:59.919 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.919 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1532 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'74, mlcod=51'69)", 2024-09-18T11:30:59.919 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.397573+0000", 2024-09-18T11:30:59.919 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 25.488861976999999, 2024-09-18T11:30:59.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032675960000000002, 2024-09-18T11:30:59.920 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:59.920 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:59.920 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:59.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:59.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397573+0000", 2024-09-18T11:30:59.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:59.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397573+0000", 2024-09-18T11:30:59.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:59.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397573+0000", 2024-09-18T11:30:59.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:59.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:59.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397577+0000", 2024-09-18T11:30:59.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5499999999999996e-06 2024-09-18T11:30:59.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:59.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397585+0000", 2024-09-18T11:30:59.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1470000000000001e-06 2024-09-18T11:30:59.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:59.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397591+0000", 2024-09-18T11:30:59.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5369999999999997e-06 2024-09-18T11:30:59.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:59.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397635+0000", 2024-09-18T11:30:59.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4308000000000003e-05 2024-09-18T11:30:59.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:59.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397656+0000", 2024-09-18T11:30:59.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1138e-05 2024-09-18T11:30:59.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:59.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.400817+0000", 2024-09-18T11:30:59.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031607499999999999 2024-09-18T11:30:59.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:59.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.400841+0000", 2024-09-18T11:30:59.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3717e-05 2024-09-18T11:30:59.923 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.923 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:59.924 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.924 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.13:87 3.0 e58/21 3:099aa86f:::smithi11939506-4 low_tier:head v 58'69, mlcod=55'66)", 2024-09-18T11:30:59.924 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.270180+0000", 2024-09-18T11:30:59.924 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 20.616255539000001, 2024-09-18T11:30:59.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037855440000000001, 2024-09-18T11:30:59.924 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:59.924 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:59.924 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:59.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:59.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270180+0000", 2024-09-18T11:30:59.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:59.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270180+0000", 2024-09-18T11:30:59.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:59.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270179+0000", 2024-09-18T11:30:59.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:59.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:59.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270187+0000", 2024-09-18T11:30:59.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5610000000000001e-06 2024-09-18T11:30:59.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:59.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270195+0000", 2024-09-18T11:30:59.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5909999999999996e-06 2024-09-18T11:30:59.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:59.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270200+0000", 2024-09-18T11:30:59.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2599999999999999e-06 2024-09-18T11:30:59.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:59.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270252+0000", 2024-09-18T11:30:59.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1891000000000002e-05 2024-09-18T11:30:59.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:59.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270277+0000", 2024-09-18T11:30:59.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5134e-05 2024-09-18T11:30:59.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:59.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273928+0000", 2024-09-18T11:30:59.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00365132 2024-09-18T11:30:59.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:59.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273965+0000", 2024-09-18T11:30:59.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7277000000000002e-05 2024-09-18T11:30:59.928 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.928 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:59.928 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.928 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.13:88 3.0 e58/21 3:099aa86f:::smithi11939506-4 low_tier:head v 58'70, mlcod=55'66)", 2024-09-18T11:30:59.928 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.270488+0000", 2024-09-18T11:30:59.929 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 20.615947664, 2024-09-18T11:30:59.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034966020000000001, 2024-09-18T11:30:59.929 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:59.929 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:59.929 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:59.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:59.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270488+0000", 2024-09-18T11:30:59.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:59.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270488+0000", 2024-09-18T11:30:59.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:59.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270487+0000", 2024-09-18T11:30:59.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:59.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:59.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270493+0000", 2024-09-18T11:30:59.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9780000000000002e-06 2024-09-18T11:30:59.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:59.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270503+0000", 2024-09-18T11:30:59.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0183000000000001e-05 2024-09-18T11:30:59.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:59.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270507+0000", 2024-09-18T11:30:59.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9960000000000004e-06 2024-09-18T11:30:59.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:59.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270542+0000", 2024-09-18T11:30:59.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5077000000000003e-05 2024-09-18T11:30:59.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:59.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270562+0000", 2024-09-18T11:30:59.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9456000000000001e-05 2024-09-18T11:30:59.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:59.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273967+0000", 2024-09-18T11:30:59.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003405805 2024-09-18T11:30:59.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:59.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273984+0000", 2024-09-18T11:30:59.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6824e-05 2024-09-18T11:30:59.932 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.932 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:59.933 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.933 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:289 4.d e58/22 4:b5a463bf:::smithi11939506-20:10 v 58'78, mlcod=53'73)", 2024-09-18T11:30:59.933 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.275682+0000", 2024-09-18T11:30:59.933 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 20.610753174999999, 2024-09-18T11:30:59.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003584318, 2024-09-18T11:30:59.933 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:59.933 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:59.933 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:59.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:59.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275682+0000", 2024-09-18T11:30:59.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:59.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275682+0000", 2024-09-18T11:30:59.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:59.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275681+0000", 2024-09-18T11:30:59.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:59.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:59.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275689+0000", 2024-09-18T11:30:59.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-18T11:30:59.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:59.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275698+0000", 2024-09-18T11:30:59.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1120000000000002e-06 2024-09-18T11:30:59.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:59.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275702+0000", 2024-09-18T11:30:59.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1450000000000001e-06 2024-09-18T11:30:59.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:59.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275731+0000", 2024-09-18T11:30:59.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8782999999999999e-05 2024-09-18T11:30:59.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:59.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275743+0000", 2024-09-18T11:30:59.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1436e-05 2024-09-18T11:30:59.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:59.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.279231+0000", 2024-09-18T11:30:59.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034884339999999999 2024-09-18T11:30:59.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:59.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.279266+0000", 2024-09-18T11:30:59.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5488000000000003e-05 2024-09-18T11:30:59.937 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.937 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:59.937 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.937 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1931 3.0 e65/21 3:3c05e498:::866d65857c51f67d56ecbcba33930954810c64ced2a213ca9705911438b0dfb7:head v 65'72, mlcod=58'70)", 2024-09-18T11:30:59.937 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:30:59.937 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.518314047, 2024-09-18T11:30:59.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040630859999999996, 2024-09-18T11:30:59.937 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:59.938 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:59.938 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:59.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:59.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:30:59.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:59.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:30:59.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:59.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368120+0000", 2024-09-18T11:30:59.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:59.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:59.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368427+0000", 2024-09-18T11:30:59.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030742600000000002 2024-09-18T11:30:59.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:59.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368446+0000", 2024-09-18T11:30:59.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8046e-05 2024-09-18T11:30:59.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:59.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368453+0000", 2024-09-18T11:30:59.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1380000000000001e-06 2024-09-18T11:30:59.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:59.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368519+0000", 2024-09-18T11:30:59.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6470000000000006e-05 2024-09-18T11:30:59.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:59.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368550+0000", 2024-09-18T11:30:59.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0707999999999998e-05 2024-09-18T11:30:59.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:59.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.372151+0000", 2024-09-18T11:30:59.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036015470000000001 2024-09-18T11:30:59.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:59.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.372184+0000", 2024-09-18T11:30:59.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2994999999999998e-05 2024-09-18T11:30:59.941 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.941 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:59.942 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.942 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:304 4.6 e69/67 4:67c1765a:::smithi11939506-30:17 v 69'35, mlcod=58'31)", 2024-09-18T11:30:59.942 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:30:59.942 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.5404829709999994, 2024-09-18T11:30:59.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006377611, 2024-09-18T11:30:59.942 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:59.942 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:59.942 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:59.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:59.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:30:59.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:59.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:30:59.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:59.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:30:59.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:59.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:59.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345961+0000", 2024-09-18T11:30:59.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2050000000000008e-06 2024-09-18T11:30:59.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:59.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345969+0000", 2024-09-18T11:30:59.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0339999999999997e-06 2024-09-18T11:30:59.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:59.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345976+0000", 2024-09-18T11:30:59.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-18T11:30:59.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:59.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346026+0000", 2024-09-18T11:30:59.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9286999999999998e-05 2024-09-18T11:30:59.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:59.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346050+0000", 2024-09-18T11:30:59.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4397e-05 2024-09-18T11:30:59.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:59.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352293+0000", 2024-09-18T11:30:59.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062427170000000001 2024-09-18T11:30:59.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:59.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352330+0000", 2024-09-18T11:30:59.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7348999999999998e-05 2024-09-18T11:30:59.946 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.946 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:59.946 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.946 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:345 4.d e69/22 4:b81644f2:::smithi11939506-34:14 v 69'97, mlcod=65'93)", 2024-09-18T11:30:59.946 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351361+0000", 2024-09-18T11:30:59.947 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.5350741980000002, 2024-09-18T11:30:59.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003416484, 2024-09-18T11:30:59.947 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:59.947 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:59.947 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:59.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:59.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351361+0000", 2024-09-18T11:30:59.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:59.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351361+0000", 2024-09-18T11:30:59.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:59.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351361+0000", 2024-09-18T11:30:59.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:59.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:59.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351369+0000", 2024-09-18T11:30:59.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9100000000000005e-06 2024-09-18T11:30:59.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:59.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351377+0000", 2024-09-18T11:30:59.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9869999999999999e-06 2024-09-18T11:30:59.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:59.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351383+0000", 2024-09-18T11:30:59.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9290000000000003e-06 2024-09-18T11:30:59.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:59.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351454+0000", 2024-09-18T11:30:59.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1180000000000006e-05 2024-09-18T11:30:59.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:59.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351484+0000", 2024-09-18T11:30:59.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0596999999999999e-05 2024-09-18T11:30:59.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:59.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.354730+0000", 2024-09-18T11:30:59.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0032452840000000002 2024-09-18T11:30:59.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:59.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.354778+0000", 2024-09-18T11:30:59.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7942999999999998e-05 2024-09-18T11:30:59.950 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.951 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:59.951 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.951 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:347 4.d e69/22 4:b44229a1:::smithi11939506-42:19 v 69'99, mlcod=65'93)", 2024-09-18T11:30:59.951 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:30:59.951 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.534749776, 2024-09-18T11:30:59.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056998760000000004, 2024-09-18T11:30:59.951 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:59.951 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:59.951 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:59.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:59.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:30:59.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:59.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:30:59.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:59.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351684+0000", 2024-09-18T11:30:59.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:59.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:59.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351693+0000", 2024-09-18T11:30:59.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-18T11:30:59.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:59.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351704+0000", 2024-09-18T11:30:59.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1077000000000001e-05 2024-09-18T11:30:59.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:59.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351709+0000", 2024-09-18T11:30:59.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5419999999999999e-06 2024-09-18T11:30:59.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:59.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351956+0000", 2024-09-18T11:30:59.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000247252 2024-09-18T11:30:59.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:59.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351975+0000", 2024-09-18T11:30:59.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9201000000000001e-05 2024-09-18T11:30:59.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:59.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357334+0000", 2024-09-18T11:30:59.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053588749999999999 2024-09-18T11:30:59.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:59.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357385+0000", 2024-09-18T11:30:59.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1406999999999998e-05 2024-09-18T11:30:59.955 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.955 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:59.955 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.956 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1956 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'37, mlcod=58'31)", 2024-09-18T11:30:59.956 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:30:59.956 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.5338048630000003, 2024-09-18T11:30:59.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010958842, 2024-09-18T11:30:59.956 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:59.956 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:59.956 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:59.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:59.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:30:59.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:59.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:30:59.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:59.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:30:59.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:59.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:59.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352996+0000", 2024-09-18T11:30:59.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036622 2024-09-18T11:30:59.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:59.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353011+0000", 2024-09-18T11:30:59.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4849e-05 2024-09-18T11:30:59.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:59.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353017+0000", 2024-09-18T11:30:59.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:30:59.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:59.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353068+0000", 2024-09-18T11:30:59.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0695000000000002e-05 2024-09-18T11:30:59.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:59.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353092+0000", 2024-09-18T11:30:59.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4260000000000002e-05 2024-09-18T11:30:59.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:59.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363551+0000", 2024-09-18T11:30:59.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010459154999999999 2024-09-18T11:30:59.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:59.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363589+0000", 2024-09-18T11:30:59.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8161000000000003e-05 2024-09-18T11:30:59.960 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.960 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:59.960 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.960 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1957 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'38, mlcod=65'33)", 2024-09-18T11:30:59.960 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:30:59.960 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.5273889959999991, 2024-09-18T11:30:59.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050499400000000002, 2024-09-18T11:30:59.960 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:59.960 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:59.961 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:59.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:59.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:30:59.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:59.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:30:59.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:59.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:30:59.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:30:59.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:59.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359214+0000", 2024-09-18T11:30:59.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000168047 2024-09-18T11:30:59.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:59.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359227+0000", 2024-09-18T11:30:59.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3124e-05 2024-09-18T11:30:59.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:59.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359235+0000", 2024-09-18T11:30:59.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0760000000000003e-06 2024-09-18T11:30:59.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:59.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359287+0000", 2024-09-18T11:30:59.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.151e-05 2024-09-18T11:30:59.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:59.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359310+0000", 2024-09-18T11:30:59.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-09-18T11:30:59.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:59.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364065+0000", 2024-09-18T11:30:59.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047549439999999997 2024-09-18T11:30:59.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:59.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364096+0000", 2024-09-18T11:30:59.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1423000000000001e-05 2024-09-18T11:30:59.964 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.964 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:59.964 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.965 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:349 4.d e69/22 4:b9bb642b:::smithi11939506-27:19 v 69'101, mlcod=69'97)", 2024-09-18T11:30:59.965 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.360028+0000", 2024-09-18T11:30:59.965 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.5264073790000001, 2024-09-18T11:30:59.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040307850000000003, 2024-09-18T11:30:59.965 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:59.965 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:59.965 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:59.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:59.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360028+0000", 2024-09-18T11:30:59.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:59.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360028+0000", 2024-09-18T11:30:59.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:59.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360027+0000", 2024-09-18T11:30:59.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:59.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:59.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360039+0000", 2024-09-18T11:30:59.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1654000000000001e-05 2024-09-18T11:30:59.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:59.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360052+0000", 2024-09-18T11:30:59.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.33e-05 2024-09-18T11:30:59.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:59.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360058+0000", 2024-09-18T11:30:59.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0440000000000001e-06 2024-09-18T11:30:59.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:59.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360094+0000", 2024-09-18T11:30:59.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6402999999999997e-05 2024-09-18T11:30:59.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:59.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360109+0000", 2024-09-18T11:30:59.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4105e-05 2024-09-18T11:30:59.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:59.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364025+0000", 2024-09-18T11:30:59.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039163189999999997 2024-09-18T11:30:59.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:59.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364059+0000", 2024-09-18T11:30:59.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3810999999999999e-05 2024-09-18T11:30:59.969 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.969 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:59.969 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.970 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1978 4.4 e69/22 4:2d9c3f7f:::smithi11939506-5:head v 69'74, mlcod=69'71)", 2024-09-18T11:30:59.970 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.967898+0000", 2024-09-18T11:30:59.970 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.918536928, 2024-09-18T11:30:59.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038277960000000001, 2024-09-18T11:30:59.970 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:59.970 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:59.970 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:59.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:59.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.967898+0000", 2024-09-18T11:30:59.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:59.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.967898+0000", 2024-09-18T11:30:59.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:59.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.967898+0000", 2024-09-18T11:30:59.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:30:59.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:59.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.968050+0000", 2024-09-18T11:30:59.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015286799999999999 2024-09-18T11:30:59.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:59.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.968064+0000", 2024-09-18T11:30:59.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3964999999999999e-05 2024-09-18T11:30:59.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:59.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.968070+0000", 2024-09-18T11:30:59.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:30:59.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:59.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.968112+0000", 2024-09-18T11:30:59.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1112999999999998e-05 2024-09-18T11:30:59.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:59.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.968133+0000", 2024-09-18T11:30:59.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1715000000000001e-05 2024-09-18T11:30:59.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:59.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.971687+0000", 2024-09-18T11:30:59.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035536410000000002 2024-09-18T11:30:59.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:59.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.971726+0000", 2024-09-18T11:30:59.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.926e-05 2024-09-18T11:30:59.974 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.974 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:59.974 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.974 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2141 3.0 e73/21 3:1799f92c:::95e762d743c69e6efaefe92d905ac94b659d7d90631d34d4368df67f56c10c79:head v 73'79, mlcod=73'77)", 2024-09-18T11:30:59.974 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:55.733432+0000", 2024-09-18T11:30:59.974 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.1530029400000004, 2024-09-18T11:30:59.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035087809999999999, 2024-09-18T11:30:59.975 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:30:59.975 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:30:59.975 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:30:59.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:30:59.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.733432+0000", 2024-09-18T11:30:59.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:30:59.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.733432+0000", 2024-09-18T11:30:59.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:30:59.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:30:59.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.733431+0000", 2024-09-18T11:30:59.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:30:59.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:30:59.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.733841+0000", 2024-09-18T11:30:59.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040973799999999999 2024-09-18T11:30:59.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:30:59.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.733855+0000", 2024-09-18T11:30:59.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4809e-05 2024-09-18T11:30:59.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:30:59.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.733861+0000", 2024-09-18T11:30:59.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5949999999999998e-06 2024-09-18T11:30:59.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:30:59.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.733908+0000", 2024-09-18T11:30:59.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6854999999999998e-05 2024-09-18T11:30:59.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:30:59.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.733932+0000", 2024-09-18T11:30:59.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4236999999999999e-05 2024-09-18T11:30:59.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:30:59.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.736906+0000", 2024-09-18T11:30:59.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029739129999999999 2024-09-18T11:30:59.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:30:59.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:30:59.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:30:59.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.736941+0000", 2024-09-18T11:30:59.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5129999999999997e-05 2024-09-18T11:30:59.979 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.979 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:59.979 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.979 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:30:59.979 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:30:59.979 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:30:59.981 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:30:59.982 INFO:tasks.rados.rados.0.smithi119.stdout:1115: finishing write tid 3 to smithi11939506-45 2024-09-18T11:30:59.982 INFO:tasks.rados.rados.0.smithi119.stdout:1115: finishing write tid 4 to smithi11939506-45 2024-09-18T11:30:59.982 INFO:tasks.rados.rados.0.smithi119.stdout:1115: finishing write tid 5 to smithi11939506-45 2024-09-18T11:30:59.982 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 35 (ObjNum 193 snap 23 seq_num 193) dirty exists 2024-09-18T11:30:59.983 INFO:tasks.rados.rados.0.smithi119.stdout:1115: left oid 45 (ObjNum 193 snap 23 seq_num 193) 2024-09-18T11:30:59.983 INFO:tasks.rados.rados.0.smithi119.stdout:1117: got (0) Success 2024-09-18T11:30:59.983 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 74 (ObjNum 123 snap 12 seq_num 123) dirty exists 2024-09-18T11:30:59.983 INFO:tasks.rados.rados.0.smithi119.stdout:1118: finishing write tid 1 to smithi11939506-10 2024-09-18T11:30:59.983 INFO:tasks.rados.rados.0.smithi119.stdout:1121: seq_num 196 ranges {494172=457629,1586353=512385,2749730=544442} 2024-09-18T11:30:59.983 INFO:tasks.rados.rados.0.smithi119.stdout:1121: writing smithi11939506-14 from 494172 to 951801 tid 1 2024-09-18T11:30:59.983 INFO:tasks.rados.rados.0.smithi119.stdout:1121: writing smithi11939506-14 from 1586353 to 2098738 tid 2 2024-09-18T11:30:59.983 INFO:tasks.rados.rados.0.smithi119.stdout:1121: writing smithi11939506-14 from 2749730 to 3294172 tid 3 2024-09-18T11:30:59.983 INFO:tasks.rados.rados.0.smithi119.stdout:1115: done (6 left) 2024-09-18T11:30:59.983 INFO:tasks.rados.rados.0.smithi119.stdout:1116: done (5 left) 2024-09-18T11:30:59.983 INFO:tasks.rados.rados.0.smithi119.stdout:1117: done (4 left) 2024-09-18T11:30:59.983 INFO:tasks.rados.rados.0.smithi119.stdout:1122: read oid 5 snap -1 2024-09-18T11:30:59.983 INFO:tasks.rados.rados.0.smithi119.stdout:1122: expect deleted 2024-09-18T11:30:59.983 INFO:tasks.rados.rados.0.smithi119.stdout:1123: write oid 45 current snap is 23 2024-09-18T11:30:59.984 INFO:tasks.rados.rados.0.smithi119.stdout:1118: finishing write tid 2 to smithi11939506-10 2024-09-18T11:30:59.984 INFO:tasks.rados.rados.0.smithi119.stdout:1118: finishing write tid 3 to smithi11939506-10 2024-09-18T11:30:59.984 INFO:tasks.rados.rados.0.smithi119.stdout:1118: finishing write tid 4 to smithi11939506-10 2024-09-18T11:30:59.984 INFO:tasks.rados.rados.0.smithi119.stdout:1118: finishing write tid 5 to smithi11939506-10 2024-09-18T11:30:59.984 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 10 v 100 (ObjNum 194 snap 23 seq_num 194) dirty exists 2024-09-18T11:30:59.984 INFO:tasks.rados.rados.0.smithi119.stdout:1118: left oid 10 (ObjNum 194 snap 23 seq_num 194) 2024-09-18T11:30:59.984 INFO:tasks.rados.rados.0.smithi119.stdout:1119: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-18T11:30:59.985 DEBUG:teuthology.orchestra.run.smithi119:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op apply-layout-settings --pool low_tier 2024-09-18T11:30:59.986 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:30:59.979+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:00.048 INFO:tasks.rados.rados.0.smithi119.stdout:1120: finishing write tid 1 to smithi11939506-2 2024-09-18T11:31:00.048 INFO:tasks.rados.rados.0.smithi119.stdout:1120: finishing write tid 2 to smithi11939506-2 2024-09-18T11:31:00.048 INFO:tasks.rados.rados.0.smithi119.stdout:1123: seq_num 197 ranges {510979=532878,1528482=772396,3060800=250179} 2024-09-18T11:31:00.054 INFO:tasks.rados.rados.0.smithi119.stdout:1123: writing smithi11939506-45 from 510979 to 1043857 tid 1 2024-09-18T11:31:00.063 INFO:tasks.rados.rados.0.smithi119.stdout:1123: writing smithi11939506-45 from 1528482 to 2300878 tid 2 2024-09-18T11:31:00.069 INFO:tasks.rados.rados.0.smithi119.stdout:1123: writing smithi11939506-45 from 3060800 to 3310979 tid 3 2024-09-18T11:31:00.069 INFO:tasks.rados.rados.0.smithi119.stdout:1118: done (5 left) 2024-09-18T11:31:00.070 INFO:tasks.rados.rados.0.smithi119.stdout:1119: done (4 left) 2024-09-18T11:31:00.070 INFO:tasks.rados.rados.0.smithi119.stdout:1124: read oid 49 snap -1 2024-09-18T11:31:00.070 INFO:tasks.rados.rados.0.smithi119.stdout:1124: expect (ObjNum 185 snap 20 seq_num 185) 2024-09-18T11:31:00.070 INFO:tasks.rados.rados.0.smithi119.stdout:1125: read oid 46 snap 22 2024-09-18T11:31:00.070 INFO:tasks.rados.rados.0.smithi119.stdout:1125: expect (ObjNum 171 snap 20 seq_num 171) 2024-09-18T11:31:00.070 INFO:tasks.rados.rados.0.smithi119.stdout:1126: read oid 19 snap -1 2024-09-18T11:31:00.070 INFO:tasks.rados.rados.0.smithi119.stdout:1126: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-18T11:31:00.070 INFO:tasks.rados.rados.0.smithi119.stdout:1127: snap_remove snap 16 2024-09-18T11:31:00.081 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:00.079+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:00.130 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:00.130 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:00.130 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:00.131 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:00.139 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:31:00.182 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:00.181+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:00.277 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:00.277 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:00.277 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:00.277 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:00.277 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:00.282 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:00.280+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:00.286 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:31:00.383 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is set 2024-09-18T11:31:00.384 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:00.382+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:00.385 INFO:tasks.rados.rados.0.smithi119.stdout:1127: done (7 left) 2024-09-18T11:31:00.385 INFO:tasks.rados.rados.0.smithi119.stdout:1128: read oid 47 snap 19 2024-09-18T11:31:00.385 INFO:tasks.rados.rados.0.smithi119.stdout:1128: expect deleted 2024-09-18T11:31:00.385 INFO:tasks.rados.rados.0.smithi119.stdout:1129: delete oid 31 current snap is 23 2024-09-18T11:31:00.385 INFO:tasks.rados.rados.0.smithi119.stdout:1120: finishing write tid 3 to smithi11939506-2 2024-09-18T11:31:00.385 INFO:tasks.rados.rados.0.smithi119.stdout:1120: finishing write tid 4 to smithi11939506-2 2024-09-18T11:31:00.385 INFO:tasks.rados.rados.0.smithi119.stdout:1120: finishing write tid 5 to smithi11939506-2 2024-09-18T11:31:00.385 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 172 (ObjNum 195 snap 23 seq_num 195) dirty exists 2024-09-18T11:31:00.385 INFO:tasks.rados.rados.0.smithi119.stdout:1120: left oid 2 (ObjNum 195 snap 23 seq_num 195) 2024-09-18T11:31:00.385 INFO:tasks.rados.rados.0.smithi119.stdout:1121: finishing write tid 1 to smithi11939506-14 2024-09-18T11:31:00.385 INFO:tasks.rados.rados.0.smithi119.stdout:1121: finishing write tid 2 to smithi11939506-14 2024-09-18T11:31:00.385 INFO:tasks.rados.rados.0.smithi119.stdout:1121: finishing write tid 3 to smithi11939506-14 2024-09-18T11:31:00.386 INFO:tasks.rados.rados.0.smithi119.stdout:1121: finishing write tid 4 to smithi11939506-14 2024-09-18T11:31:00.386 INFO:tasks.rados.rados.0.smithi119.stdout:1121: finishing write tid 5 to smithi11939506-14 2024-09-18T11:31:00.386 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 133 (ObjNum 196 snap 23 seq_num 196) dirty exists 2024-09-18T11:31:00.386 INFO:tasks.rados.rados.0.smithi119.stdout:1121: left oid 14 (ObjNum 196 snap 23 seq_num 196) 2024-09-18T11:31:00.386 INFO:tasks.rados.rados.0.smithi119.stdout:1123: finishing write tid 1 to smithi11939506-45 2024-09-18T11:31:00.386 INFO:tasks.rados.rados.0.smithi119.stdout:1123: finishing write tid 2 to smithi11939506-45 2024-09-18T11:31:00.386 INFO:tasks.rados.rados.0.smithi119.stdout:1126: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-18T11:31:00.391 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:00.390+0000 7f6bf3052640 -1 osd.5 76 handle_osd_map injecting map crc failure 2024-09-18T11:31:00.459 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:00.459 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:00.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:00.459 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:00.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.459 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:900 4.0 4:0aec5e1c:::smithi11939506-50:head [read 0~2974806,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:00.459 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.133226+0000", 2024-09-18T11:31:00.459 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 49.322173745000001, 2024-09-18T11:31:00.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059578069999999999, 2024-09-18T11:31:00.459 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:00.460 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:00.460 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:00.460 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:00.460 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.460 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 900 2024-09-18T11:31:00.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.460 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:00.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:00.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133226+0000", 2024-09-18T11:31:00.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:00.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133226+0000", 2024-09-18T11:31:00.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:00.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133225+0000", 2024-09-18T11:31:00.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:00.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:00.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133234+0000", 2024-09-18T11:31:00.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5799999999999992e-06 2024-09-18T11:31:00.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:00.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133242+0000", 2024-09-18T11:31:00.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4039999999999995e-06 2024-09-18T11:31:00.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133256+0000", 2024-09-18T11:31:00.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3278000000000001e-05 2024-09-18T11:31:00.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133293+0000", 2024-09-18T11:31:00.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7799000000000002e-05 2024-09-18T11:31:00.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:00.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133298+0000", 2024-09-18T11:31:00.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9110000000000001e-06 2024-09-18T11:31:00.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.137554+0000", 2024-09-18T11:31:00.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042556099999999999 2024-09-18T11:31:00.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:00.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.137713+0000", 2024-09-18T11:31:00.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015937199999999999 2024-09-18T11:31:00.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:00.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139184+0000", 2024-09-18T11:31:00.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001470694 2024-09-18T11:31:00.465 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.465 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:00.465 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.465 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:901 4.0 4:0aec5e1c:::smithi11939506-50:head [sparse-read 0~1781150] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:00.466 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:31:00.466 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 49.322134535000004, 2024-09-18T11:31:00.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063828720000000004, 2024-09-18T11:31:00.466 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:00.466 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:00.466 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:00.466 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:00.466 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.466 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 901 2024-09-18T11:31:00.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.466 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:00.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:00.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:31:00.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:00.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:31:00.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:00.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:31:00.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:00.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:00.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133268+0000", 2024-09-18T11:31:00.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9610000000000001e-06 2024-09-18T11:31:00.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:00.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133280+0000", 2024-09-18T11:31:00.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2481e-05 2024-09-18T11:31:00.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133285+0000", 2024-09-18T11:31:00.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5469999999999998e-06 2024-09-18T11:31:00.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133317+0000", 2024-09-18T11:31:00.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1838999999999998e-05 2024-09-18T11:31:00.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:00.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133320+0000", 2024-09-18T11:31:00.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3009999999999998e-06 2024-09-18T11:31:00.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139225+0000", 2024-09-18T11:31:00.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059047919999999999 2024-09-18T11:31:00.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:00.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139330+0000", 2024-09-18T11:31:00.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104924 2024-09-18T11:31:00.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:00.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139648+0000", 2024-09-18T11:31:00.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00031854799999999999 2024-09-18T11:31:00.471 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.472 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:00.472 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.472 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:00.472 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:00.472 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 49.285980494999997, 2024-09-18T11:31:00.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:31:00.472 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:00.472 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:00.472 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:00.473 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:00.473 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.473 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:31:00.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.473 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:00.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:00.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:00.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:00.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:00.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:00.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:31:00.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:00.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:00.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:31:00.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:31:00.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:00.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:31:00.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:31:00.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:31:00.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:31:00.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:31:00.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:31:00.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:00.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:31:00.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:31:00.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:00.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:31:00.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:31:00.477 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.477 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:00.477 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.478 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1001 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:00.478 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:00.478 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.290829217000002, 2024-09-18T11:31:00.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007461082, 2024-09-18T11:31:00.478 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:00.478 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:00.478 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:00.478 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:00.478 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.478 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1001 2024-09-18T11:31:00.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.478 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:00.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:00.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:00.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:00.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:00.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:00.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164570+0000", 2024-09-18T11:31:00.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:00.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:00.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164574+0000", 2024-09-18T11:31:00.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.822e-06 2024-09-18T11:31:00.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:00.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164582+0000", 2024-09-18T11:31:00.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3410000000000003e-06 2024-09-18T11:31:00.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164587+0000", 2024-09-18T11:31:00.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769999999999999e-06 2024-09-18T11:31:00.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164702+0000", 2024-09-18T11:31:00.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115705 2024-09-18T11:31:00.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:00.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164709+0000", 2024-09-18T11:31:00.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-09-18T11:31:00.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170296+0000", 2024-09-18T11:31:00.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055876390000000001 2024-09-18T11:31:00.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:00.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170418+0000", 2024-09-18T11:31:00.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121549 2024-09-18T11:31:00.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:00.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172032+0000", 2024-09-18T11:31:00.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00161407 2024-09-18T11:31:00.483 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.483 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:00.483 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.484 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:00.484 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:00.484 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.290802501000002, 2024-09-18T11:31:00.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:31:00.484 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:00.484 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:00.484 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:00.484 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:00.484 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.484 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:31:00.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.485 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:00.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:00.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:00.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:00.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:00.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:00.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:00.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:00.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:00.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:31:00.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:31:00.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:00.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:31:00.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:31:00.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:31:00.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:31:00.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:31:00.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:31:00.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:00.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:31:00.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:31:00.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:31:00.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:31:00.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:00.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:31:00.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:31:00.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:00.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:31:00.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:31:00.490 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.490 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:00.490 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.490 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:00.490 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:00.490 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.290784211999998, 2024-09-18T11:31:00.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:31:00.490 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:00.490 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:00.490 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:00.490 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:00.491 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.491 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:31:00.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.491 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:00.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:00.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:00.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:00.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:00.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:00.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:31:00.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:00.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:00.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:31:00.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:31:00.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:00.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:31:00.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:31:00.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:31:00.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:31:00.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:31:00.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:31:00.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:00.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:31:00.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:31:00.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:31:00.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:31:00.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:00.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:31:00.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:31:00.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:00.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:31:00.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:31:00.495 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.495 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:00.496 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.496 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:00.496 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:00.496 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.290723442999997, 2024-09-18T11:31:00.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:31:00.496 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:00.496 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:00.496 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:00.496 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:00.496 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.497 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:31:00.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.497 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:00.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:00.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:00.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:00.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:00.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:00.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:31:00.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:00.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:00.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:31:00.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:31:00.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:00.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:31:00.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:31:00.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:31:00.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:31:00.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:31:00.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:31:00.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:00.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:31:00.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:31:00.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:31:00.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:31:00.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:00.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:31:00.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:31:00.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:00.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:31:00.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:31:00.502 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.502 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:00.502 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.502 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:00.502 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:00.502 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.290701122000002, 2024-09-18T11:31:00.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:31:00.503 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:00.503 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:00.503 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:00.503 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:00.503 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.503 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:31:00.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.503 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:00.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:00.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:00.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:00.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:00.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:00.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:31:00.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:00.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:00.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:31:00.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:31:00.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:00.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:31:00.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:31:00.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:31:00.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:31:00.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:31:00.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:31:00.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:00.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:31:00.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:31:00.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:31:00.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:31:00.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:00.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:31:00.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:31:00.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:00.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:31:00.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:31:00.508 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.508 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:00.508 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.508 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1018 4.0 4:0aec5e1c:::smithi11939506-50:head [write 1622620~730352] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:00.509 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:31:00.509 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 44.774466566000001, 2024-09-18T11:31:00.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064405160000000003, 2024-09-18T11:31:00.509 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:00.509 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:00.509 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:00.509 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:00.509 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.509 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1018 2024-09-18T11:31:00.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.509 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:00.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:00.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:31:00.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:00.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:31:00.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:00.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680932+0000", 2024-09-18T11:31:00.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:00.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:00.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681334+0000", 2024-09-18T11:31:00.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000401803 2024-09-18T11:31:00.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:00.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681350+0000", 2024-09-18T11:31:00.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6218999999999998e-05 2024-09-18T11:31:00.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681374+0000", 2024-09-18T11:31:00.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4088e-05 2024-09-18T11:31:00.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681415+0000", 2024-09-18T11:31:00.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1307000000000003e-05 2024-09-18T11:31:00.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:00.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681507+0000", 2024-09-18T11:31:00.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1867999999999995e-05 2024-09-18T11:31:00.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:00.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681671+0000", 2024-09-18T11:31:00.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016406 2024-09-18T11:31:00.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:00.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.683856+0000", 2024-09-18T11:31:00.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002184624 2024-09-18T11:31:00.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:00.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687266+0000", 2024-09-18T11:31:00.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034105490000000001 2024-09-18T11:31:00.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:00.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687321+0000", 2024-09-18T11:31:00.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4960999999999998e-05 2024-09-18T11:31:00.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:00.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687374+0000", 2024-09-18T11:31:00.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2553000000000003e-05 2024-09-18T11:31:00.515 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.515 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:00.515 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.516 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:31:00.516 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:00.516 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.509576238999998, 2024-09-18T11:31:00.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:31:00.516 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:00.516 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:00.516 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:00.516 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:00.516 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.517 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:31:00.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.517 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:00.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:00.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:00.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:00.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:00.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:00.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:31:00.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:31:00.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:00.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:31:00.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:31:00.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:00.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:31:00.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:31:00.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:31:00.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:31:00.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:31:00.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:31:00.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:00.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:31:00.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:31:00.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:00.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:31:00.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:31:00.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:00.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:31:00.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:31:00.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:00.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:31:00.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:31:00.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:00.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:31:00.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:31:00.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:00.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:31:00.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:31:00.522 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.523 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:00.523 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.523 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1121 4.f e38/22 4:f5d7df41:::smithi11939506-16:head v 38'34, mlcod=30'32)", 2024-09-18T11:31:00.523 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:31:00.523 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.489588980000001, 2024-09-18T11:31:00.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060953680000000003, 2024-09-18T11:31:00.523 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:00.523 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:00.523 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:00.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:00.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:31:00.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:00.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:31:00.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:00.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965810+0000", 2024-09-18T11:31:00.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:00.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:00.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966142+0000", 2024-09-18T11:31:00.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000331591 2024-09-18T11:31:00.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:00.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966154+0000", 2024-09-18T11:31:00.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.253e-05 2024-09-18T11:31:00.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966160+0000", 2024-09-18T11:31:00.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7130000000000002e-06 2024-09-18T11:31:00.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966196+0000", 2024-09-18T11:31:00.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6276999999999998e-05 2024-09-18T11:31:00.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:00.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966217+0000", 2024-09-18T11:31:00.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0335000000000001e-05 2024-09-18T11:31:00.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:00.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.971872+0000", 2024-09-18T11:31:00.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056549290000000004 2024-09-18T11:31:00.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:00.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.971906+0000", 2024-09-18T11:31:00.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4702000000000001e-05 2024-09-18T11:31:00.528 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.528 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:00.528 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.528 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:29 3.2 3:5fa8fcfa:::ec59520759824f0488c7a3ca135dacb324de2330e16b7f54053de5719c4e96b6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:31:00.529 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:31:00.529 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.775218322999997, 2024-09-18T11:31:00.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064126499999999998, 2024-09-18T11:31:00.529 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:00.529 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:00.529 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:00.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:00.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:31:00.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:00.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:31:00.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:00.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680181+0000", 2024-09-18T11:31:00.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:00.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:00.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680654+0000", 2024-09-18T11:31:00.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047334900000000002 2024-09-18T11:31:00.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:00.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680666+0000", 2024-09-18T11:31:00.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2232000000000001e-05 2024-09-18T11:31:00.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680672+0000", 2024-09-18T11:31:00.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:31:00.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681222+0000", 2024-09-18T11:31:00.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00054928699999999997 2024-09-18T11:31:00.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:00.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681332+0000", 2024-09-18T11:31:00.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110634 2024-09-18T11:31:00.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:00.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681552+0000", 2024-09-18T11:31:00.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021926899999999999 2024-09-18T11:31:00.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:00.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682950+0000", 2024-09-18T11:31:00.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013982669999999999 2024-09-18T11:31:00.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:00.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686503+0000", 2024-09-18T11:31:00.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035536130000000002 2024-09-18T11:31:00.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:00.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686553+0000", 2024-09-18T11:31:00.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9801e-05 2024-09-18T11:31:00.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:00.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686594+0000", 2024-09-18T11:31:00.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1022000000000001e-05 2024-09-18T11:31:00.535 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.535 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:00.535 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.535 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:30 3.2 3:49074e56:::7fc21ffee1ba17f2a9197c61ff60ef0b03804c121c8aeb83ea424b7ac77081e9:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:31:00.535 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.680698+0000", 2024-09-18T11:31:00.535 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.774702005000002, 2024-09-18T11:31:00.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060349130000000003, 2024-09-18T11:31:00.536 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:00.536 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:00.536 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:00.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:00.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680698+0000", 2024-09-18T11:31:00.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:00.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680698+0000", 2024-09-18T11:31:00.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:00.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680697+0000", 2024-09-18T11:31:00.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:00.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:00.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680750+0000", 2024-09-18T11:31:00.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3170000000000001e-05 2024-09-18T11:31:00.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:00.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680760+0000", 2024-09-18T11:31:00.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3379999999999994e-06 2024-09-18T11:31:00.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680766+0000", 2024-09-18T11:31:00.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8249999999999999e-06 2024-09-18T11:31:00.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682201+0000", 2024-09-18T11:31:00.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001434868 2024-09-18T11:31:00.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:00.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682294+0000", 2024-09-18T11:31:00.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2965000000000002e-05 2024-09-18T11:31:00.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:00.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682445+0000", 2024-09-18T11:31:00.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015103600000000001 2024-09-18T11:31:00.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:00.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.683524+0000", 2024-09-18T11:31:00.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0010789980000000001 2024-09-18T11:31:00.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:00.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686687+0000", 2024-09-18T11:31:00.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003163052 2024-09-18T11:31:00.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:00.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686714+0000", 2024-09-18T11:31:00.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6369000000000001e-05 2024-09-18T11:31:00.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:00.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686733+0000", 2024-09-18T11:31:00.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9117999999999999e-05 2024-09-18T11:31:00.541 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.542 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:00.542 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.542 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1199 4.0 4:0aec5e1c:::smithi11939506-50:head [write 763051~794393] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-18T11:31:00.542 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:31:00.542 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.258404198999997, 2024-09-18T11:31:00.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067768530000000002, 2024-09-18T11:31:00.542 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:00.542 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:00.542 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:00.542 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:00.543 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.543 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1199 2024-09-18T11:31:00.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.543 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:00.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:00.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:31:00.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:00.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:31:00.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:00.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196994+0000", 2024-09-18T11:31:00.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:00.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:00.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197355+0000", 2024-09-18T11:31:00.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036035199999999998 2024-09-18T11:31:00.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:00.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197366+0000", 2024-09-18T11:31:00.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.17e-05 2024-09-18T11:31:00.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197371+0000", 2024-09-18T11:31:00.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6099999999999999e-06 2024-09-18T11:31:00.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197416+0000", 2024-09-18T11:31:00.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5099000000000003e-05 2024-09-18T11:31:00.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:00.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197420+0000", 2024-09-18T11:31:00.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3189999999999996e-06 2024-09-18T11:31:00.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199392+0000", 2024-09-18T11:31:00.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019720139999999998 2024-09-18T11:31:00.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:00.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199452+0000", 2024-09-18T11:31:00.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9929999999999997e-05 2024-09-18T11:31:00.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:00.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199565+0000", 2024-09-18T11:31:00.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112811 2024-09-18T11:31:00.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:00.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.202108+0000", 2024-09-18T11:31:00.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025429060000000002 2024-09-18T11:31:00.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:00.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203714+0000", 2024-09-18T11:31:00.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001606335 2024-09-18T11:31:00.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:00.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203745+0000", 2024-09-18T11:31:00.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0837999999999998e-05 2024-09-18T11:31:00.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:00.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203773+0000", 2024-09-18T11:31:00.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7562000000000001e-05 2024-09-18T11:31:00.549 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.549 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:00.549 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.550 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1661 4.0 4:019b6e09:::smithi11939506-40:17 [read 0~3445402,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:00.550 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:00.550 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.105503536000001, 2024-09-18T11:31:00.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075190880000000002, 2024-09-18T11:31:00.550 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:00.550 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:00.550 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:00.550 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:00.550 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.550 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1661 2024-09-18T11:31:00.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.551 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:00.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:00.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:00.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:00.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:00.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:00.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:00.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:00.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:00.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349899+0000", 2024-09-18T11:31:00.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9749999999999999e-06 2024-09-18T11:31:00.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:00.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349906+0000", 2024-09-18T11:31:00.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9290000000000001e-06 2024-09-18T11:31:00.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349912+0000", 2024-09-18T11:31:00.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9900000000000002e-06 2024-09-18T11:31:00.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355645+0000", 2024-09-18T11:31:00.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057329549999999996 2024-09-18T11:31:00.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:00.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355776+0000", 2024-09-18T11:31:00.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130975 2024-09-18T11:31:00.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:00.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357416+0000", 2024-09-18T11:31:00.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001639649 2024-09-18T11:31:00.554 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.555 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:00.555 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.555 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1761 4.0 4:0aec5e1c:::smithi11939506-50:head [write 769787~704463,stat] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:00.555 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:00.555 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.153011121999999, 2024-09-18T11:31:00.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074082879999999999, 2024-09-18T11:31:00.555 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:00.555 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:00.555 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:00.555 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:00.556 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.556 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1761 2024-09-18T11:31:00.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.556 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:00.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:00.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:00.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:00.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:00.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:00.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302386+0000", 2024-09-18T11:31:00.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:00.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:00.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302889+0000", 2024-09-18T11:31:00.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050318499999999996 2024-09-18T11:31:00.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:00.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302930+0000", 2024-09-18T11:31:00.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0198999999999999e-05 2024-09-18T11:31:00.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302965+0000", 2024-09-18T11:31:00.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5292e-05 2024-09-18T11:31:00.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303046+0000", 2024-09-18T11:31:00.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1254999999999994e-05 2024-09-18T11:31:00.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:00.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303054+0000", 2024-09-18T11:31:00.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2800000000000003e-06 2024-09-18T11:31:00.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304658+0000", 2024-09-18T11:31:00.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016039520000000001 2024-09-18T11:31:00.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:00.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304724+0000", 2024-09-18T11:31:00.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5888000000000003e-05 2024-09-18T11:31:00.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:00.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304853+0000", 2024-09-18T11:31:00.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012912700000000001 2024-09-18T11:31:00.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:00.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309084+0000", 2024-09-18T11:31:00.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042311229999999998 2024-09-18T11:31:00.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:00.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309738+0000", 2024-09-18T11:31:00.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00065401399999999996 2024-09-18T11:31:00.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:00.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309770+0000", 2024-09-18T11:31:00.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1652000000000001e-05 2024-09-18T11:31:00.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:00.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309797+0000", 2024-09-18T11:31:00.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7052000000000001e-05 2024-09-18T11:31:00.563 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.563 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:00.563 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.563 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1951 4.0 4:019b6e09:::smithi11939506-40:13 [sparse-read 0~2626478] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:00.563 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.339659+0000", 2024-09-18T11:31:00.563 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.1157414160000005, 2024-09-18T11:31:00.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067895999999999998, 2024-09-18T11:31:00.564 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:00.564 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:00.564 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:00.564 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:00.564 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.564 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1951 2024-09-18T11:31:00.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.564 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:00.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:00.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339659+0000", 2024-09-18T11:31:00.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:00.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339659+0000", 2024-09-18T11:31:00.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:00.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339658+0000", 2024-09-18T11:31:00.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:00.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:00.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339661+0000", 2024-09-18T11:31:00.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3749999999999999e-06 2024-09-18T11:31:00.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:00.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339668+0000", 2024-09-18T11:31:00.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1380000000000001e-06 2024-09-18T11:31:00.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339675+0000", 2024-09-18T11:31:00.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7499999999999997e-06 2024-09-18T11:31:00.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339748+0000", 2024-09-18T11:31:00.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2885000000000004e-05 2024-09-18T11:31:00.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:00.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339751+0000", 2024-09-18T11:31:00.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.446e-06 2024-09-18T11:31:00.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345249+0000", 2024-09-18T11:31:00.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054983360000000004 2024-09-18T11:31:00.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:00.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345332+0000", 2024-09-18T11:31:00.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3245999999999995e-05 2024-09-18T11:31:00.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:00.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346448+0000", 2024-09-18T11:31:00.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0011160530000000001 2024-09-18T11:31:00.569 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.569 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:00.569 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.570 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:306 4.f e69/22 4:f5d7df41:::smithi11939506-16:17 v 69'94, mlcod=65'88)", 2024-09-18T11:31:00.570 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:00.570 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.1042844049999996, 2024-09-18T11:31:00.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068024720000000004, 2024-09-18T11:31:00.570 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:00.570 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:00.570 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:00.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:00.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:00.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:00.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:00.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:00.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351115+0000", 2024-09-18T11:31:00.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:00.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:00.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351121+0000", 2024-09-18T11:31:00.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-09-18T11:31:00.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:00.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351126+0000", 2024-09-18T11:31:00.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5969999999999999e-06 2024-09-18T11:31:00.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351147+0000", 2024-09-18T11:31:00.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1160000000000001e-05 2024-09-18T11:31:00.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351518+0000", 2024-09-18T11:31:00.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00037079300000000001 2024-09-18T11:31:00.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:00.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351536+0000", 2024-09-18T11:31:00.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8168999999999998e-05 2024-09-18T11:31:00.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:00.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357870+0000", 2024-09-18T11:31:00.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063332650000000002 2024-09-18T11:31:00.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:00.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357918+0000", 2024-09-18T11:31:00.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8347999999999999e-05 2024-09-18T11:31:00.574 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.575 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:00.575 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.575 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2022 4.0 4:0aec5e1c:::smithi11939506-50:head [write 457190~552096,stat] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-18T11:31:00.575 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:00.575 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.4592462680000002, 2024-09-18T11:31:00.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006803974, 2024-09-18T11:31:00.575 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:00.575 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:00.575 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:00.575 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:00.576 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.576 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2022 2024-09-18T11:31:00.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.576 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:00.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:00.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:00.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:00.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:00.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:00.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996151+0000", 2024-09-18T11:31:00.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:00.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:00.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996461+0000", 2024-09-18T11:31:00.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000309587 2024-09-18T11:31:00.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:00.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996481+0000", 2024-09-18T11:31:00.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9949000000000002e-05 2024-09-18T11:31:00.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996489+0000", 2024-09-18T11:31:00.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9129999999999992e-06 2024-09-18T11:31:00.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996562+0000", 2024-09-18T11:31:00.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2807999999999998e-05 2024-09-18T11:31:00.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:00.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996687+0000", 2024-09-18T11:31:00.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012458200000000001 2024-09-18T11:31:00.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:00.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996909+0000", 2024-09-18T11:31:00.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222124 2024-09-18T11:31:00.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:00.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.000741+0000", 2024-09-18T11:31:00.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038320720000000002 2024-09-18T11:31:00.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:00.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002854+0000", 2024-09-18T11:31:00.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002113228 2024-09-18T11:31:00.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:00.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002911+0000", 2024-09-18T11:31:00.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7192999999999999e-05 2024-09-18T11:31:00.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:00.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002958+0000", 2024-09-18T11:31:00.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6242000000000002e-05 2024-09-18T11:31:00.582 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.582 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:00.582 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.582 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2075 3.2 3:5e6b2dd0:::abf5d1c1c6ff48a895de7f094078ad489e0511876df30b6bd8853cd5ff4a6267:head [write 0~2096640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-18T11:31:00.582 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:55.321813+0000", 2024-09-18T11:31:00.582 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.1335865949999997, 2024-09-18T11:31:00.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0061775409999999999, 2024-09-18T11:31:00.582 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:00.583 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:00.583 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:00.583 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:00.583 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.583 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2075 2024-09-18T11:31:00.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.583 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:00.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:00.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.321813+0000", 2024-09-18T11:31:00.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:00.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.321813+0000", 2024-09-18T11:31:00.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:00.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:00.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.321811+0000", 2024-09-18T11:31:00.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:00.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:00.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.322777+0000", 2024-09-18T11:31:00.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096617299999999997 2024-09-18T11:31:00.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:00.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.322796+0000", 2024-09-18T11:31:00.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.861e-05 2024-09-18T11:31:00.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.322803+0000", 2024-09-18T11:31:00.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3669999999999999e-06 2024-09-18T11:31:00.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:00.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.322847+0000", 2024-09-18T11:31:00.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3355000000000001e-05 2024-09-18T11:31:00.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:00.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.322948+0000", 2024-09-18T11:31:00.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101563 2024-09-18T11:31:00.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:00.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.323049+0000", 2024-09-18T11:31:00.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100426 2024-09-18T11:31:00.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:00.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.327713+0000", 2024-09-18T11:31:00.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046642280000000003 2024-09-18T11:31:00.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:00.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.327905+0000", 2024-09-18T11:31:00.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000192098 2024-09-18T11:31:00.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:00.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.327953+0000", 2024-09-18T11:31:00.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7907000000000001e-05 2024-09-18T11:31:00.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:00.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:00.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:00.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.327991+0000", 2024-09-18T11:31:00.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7954999999999998e-05 2024-09-18T11:31:00.589 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.589 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:00.589 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.589 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:00.589 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:00.589 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:00.591 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:31:00.591 INFO:tasks.rados.rados.0.smithi119.stdout:1123: finishing write tid 3 to smithi11939506-45 2024-09-18T11:31:00.591 INFO:tasks.rados.rados.0.smithi119.stdout:1123: finishing write tid 4 to smithi11939506-45 2024-09-18T11:31:00.592 INFO:tasks.rados.rados.0.smithi119.stdout:1123: finishing write tid 5 to smithi11939506-45 2024-09-18T11:31:00.592 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 39 (ObjNum 197 snap 23 seq_num 197) dirty exists 2024-09-18T11:31:00.592 INFO:tasks.rados.rados.0.smithi119.stdout:1123: left oid 45 (ObjNum 197 snap 23 seq_num 197) 2024-09-18T11:31:00.592 INFO:tasks.rados.rados.0.smithi119.stdout:1124: expect (ObjNum 185 snap 20 seq_num 185) 2024-09-18T11:31:00.592 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:00.588+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:00.619 INFO:tasks.rados.rados.0.smithi119.stdout:1125: expect (ObjNum 171 snap 20 seq_num 171) 2024-09-18T11:31:00.691 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:00.689+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:00.693 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:00.693 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:00.694 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:00.694 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:00.702 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:31:00.739 INFO:tasks.rados.rados.0.smithi119.stdout:1120: done (8 left) 2024-09-18T11:31:00.739 INFO:tasks.rados.rados.0.smithi119.stdout:1121: done (7 left) 2024-09-18T11:31:00.739 INFO:tasks.rados.rados.0.smithi119.stdout:1122: done (6 left) 2024-09-18T11:31:00.739 INFO:tasks.rados.rados.0.smithi119.stdout:1123: done (5 left) 2024-09-18T11:31:00.739 INFO:tasks.rados.rados.0.smithi119.stdout:1124: done (4 left) 2024-09-18T11:31:00.739 INFO:tasks.rados.rados.0.smithi119.stdout:1125: done (3 left) 2024-09-18T11:31:00.739 INFO:tasks.rados.rados.0.smithi119.stdout:1126: done (2 left) 2024-09-18T11:31:00.739 INFO:tasks.rados.rados.0.smithi119.stdout:1128: done (1 left) 2024-09-18T11:31:00.739 INFO:tasks.rados.rados.0.smithi119.stdout:1129: done (0 left) 2024-09-18T11:31:00.739 INFO:tasks.rados.rados.0.smithi119.stdout:1130: read oid 20 snap -1 2024-09-18T11:31:00.740 INFO:tasks.rados.rados.0.smithi119.stdout:1130: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:31:00.740 INFO:tasks.rados.rados.0.smithi119.stdout:1131: set_chunk oid 6 target oid 2024-09-18T11:31:00.749 INFO:tasks.rados.rados.0.smithi119.stdout:1131: set_chunk oid 6 offset: 1133568 length: 459264 target oid ea92e9a89207f210bef8ed386ef582cc85bae990cf5d728396ff983e2b1c9585 offset: 0 snap 15 2024-09-18T11:31:00.750 INFO:tasks.rados.rados.0.smithi119.stdout:1132: write oid 9 current snap is 23 2024-09-18T11:31:00.750 INFO:tasks.rados.rados.0.smithi119.stdout:1132: seq_num 199 ranges {544593=799729,1761799=776065,2948352=396241} 2024-09-18T11:31:00.756 INFO:tasks.rados.rados.0.smithi119.stdout:1132: writing smithi11939506-9 from 544593 to 1344322 tid 1 2024-09-18T11:31:00.763 INFO:tasks.rados.rados.0.smithi119.stdout:1132: writing smithi11939506-9 from 1761799 to 2537864 tid 2 2024-09-18T11:31:00.768 INFO:tasks.rados.rados.0.smithi119.stdout:1132: writing smithi11939506-9 from 2948352 to 3344593 tid 3 2024-09-18T11:31:00.768 INFO:tasks.rados.rados.0.smithi119.stdout:1133: read oid 31 snap -1 2024-09-18T11:31:00.768 INFO:tasks.rados.rados.0.smithi119.stdout:1133: expect deleted 2024-09-18T11:31:00.768 INFO:tasks.rados.rados.0.smithi119.stdout:1134: read oid 33 snap -1 2024-09-18T11:31:00.768 INFO:tasks.rados.rados.0.smithi119.stdout:1134: expect deleted 2024-09-18T11:31:00.768 INFO:tasks.rados.rados.0.smithi119.stdout:1135: rollback oid 14 current snap is 23 2024-09-18T11:31:00.768 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 14 to 14 2024-09-18T11:31:00.768 INFO:tasks.rados.rados.0.smithi119.stdout:1130: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:31:00.791 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:00.789+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:00.805 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:00.805 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:00.805 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:00.805 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:00.805 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:00.814 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:31:00.893 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:00.892+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:00.913 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:00.914 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:00.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:00.914 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:00.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.914 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1134 4.1 4:8b9a985d:::smithi11939506-23:a [sparse-read 0~1709094] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:31:00.914 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:31:00.914 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.732317283, 2024-09-18T11:31:00.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061866400000000002, 2024-09-18T11:31:00.914 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:00.914 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:00.914 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:00.915 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:00.915 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.915 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1134 2024-09-18T11:31:00.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.915 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:00.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:00.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:31:00.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:00.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179093+0000", 2024-09-18T11:31:00.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:00.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179092+0000", 2024-09-18T11:31:00.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:00.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:00.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179104+0000", 2024-09-18T11:31:00.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2346e-05 2024-09-18T11:31:00.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:00.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179128+0000", 2024-09-18T11:31:00.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4046999999999999e-05 2024-09-18T11:31:00.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179141+0000", 2024-09-18T11:31:00.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3149000000000001e-05 2024-09-18T11:31:00.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:00.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179406+0000", 2024-09-18T11:31:00.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026496100000000003 2024-09-18T11:31:00.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:00.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179431+0000", 2024-09-18T11:31:00.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5242999999999999e-05 2024-09-18T11:31:00.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:00.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.184981+0000", 2024-09-18T11:31:00.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055495969999999999 2024-09-18T11:31:00.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:00.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185055+0000", 2024-09-18T11:31:00.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4375999999999994e-05 2024-09-18T11:31:00.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:00.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185280+0000", 2024-09-18T11:31:00.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022433800000000001 2024-09-18T11:31:00.919 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.919 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:00.919 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.919 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1135 4.1 4:8b9a985d:::smithi11939506-23:a [sparse-read 0~1709094] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:31:00.919 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.179201+0000", 2024-09-18T11:31:00.920 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.732208835999998, 2024-09-18T11:31:00.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063123149999999998, 2024-09-18T11:31:00.920 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:00.920 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:00.920 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:00.920 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:00.920 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.920 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1135 2024-09-18T11:31:00.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.920 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:00.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:00.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179201+0000", 2024-09-18T11:31:00.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:00.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179201+0000", 2024-09-18T11:31:00.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:00.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179200+0000", 2024-09-18T11:31:00.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:00.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:00.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179207+0000", 2024-09-18T11:31:00.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1960000000000002e-06 2024-09-18T11:31:00.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:00.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179229+0000", 2024-09-18T11:31:00.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1322999999999999e-05 2024-09-18T11:31:00.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179240+0000", 2024-09-18T11:31:00.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1793e-05 2024-09-18T11:31:00.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:00.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179460+0000", 2024-09-18T11:31:00.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000219407 2024-09-18T11:31:00.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:00.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.179466+0000", 2024-09-18T11:31:00.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7490000000000004e-06 2024-09-18T11:31:00.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:00.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185300+0000", 2024-09-18T11:31:00.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058347490000000002 2024-09-18T11:31:00.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:00.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185345+0000", 2024-09-18T11:31:00.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4316000000000001e-05 2024-09-18T11:31:00.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:00.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.185514+0000", 2024-09-18T11:31:00.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016905800000000001 2024-09-18T11:31:00.926 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.926 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:00.926 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.926 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1421 4.d 4:b5a463bf:::smithi11939506-20:head [write 665577~781581] snapc 10=[10,e,c] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T11:31:00.926 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:29.216974+0000", 2024-09-18T11:31:00.927 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.694436652, 2024-09-18T11:31:00.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006459869, 2024-09-18T11:31:00.927 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:00.927 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:00.927 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:00.927 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:00.927 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.927 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1421 2024-09-18T11:31:00.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.927 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:00.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:00.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.216974+0000", 2024-09-18T11:31:00.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:00.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.216974+0000", 2024-09-18T11:31:00.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:00.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.216972+0000", 2024-09-18T11:31:00.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:00.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:00.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217766+0000", 2024-09-18T11:31:00.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079344699999999997 2024-09-18T11:31:00.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:00.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217780+0000", 2024-09-18T11:31:00.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.428e-05 2024-09-18T11:31:00.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217786+0000", 2024-09-18T11:31:00.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9050000000000002e-06 2024-09-18T11:31:00.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:00.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217828+0000", 2024-09-18T11:31:00.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1839999999999999e-05 2024-09-18T11:31:00.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:00.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.217897+0000", 2024-09-18T11:31:00.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9663e-05 2024-09-18T11:31:00.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:00.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.218013+0000", 2024-09-18T11:31:00.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011509 2024-09-18T11:31:00.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:00.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.220250+0000", 2024-09-18T11:31:00.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022370150000000002 2024-09-18T11:31:00.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:00.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.223359+0000", 2024-09-18T11:31:00.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031092519999999998 2024-09-18T11:31:00.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:00.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.223396+0000", 2024-09-18T11:31:00.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7531999999999999e-05 2024-09-18T11:31:00.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:00.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:29.223433+0000", 2024-09-18T11:31:00.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7098999999999998e-05 2024-09-18T11:31:00.932 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.932 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:00.932 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.933 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1530 4.d 4:b81644f2:::smithi11939506-34:head [write 467261~717275] snapc 14=[14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:31:00.933 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:31:00.933 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.520725704, 2024-09-18T11:31:00.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075660950000000001, 2024-09-18T11:31:00.933 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:00.933 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:00.933 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:00.933 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:00.933 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.933 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1530 2024-09-18T11:31:00.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.933 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:00.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:00.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:31:00.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:00.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:31:00.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:00.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390682+0000", 2024-09-18T11:31:00.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:00.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:00.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391554+0000", 2024-09-18T11:31:00.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00087220600000000004 2024-09-18T11:31:00.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:00.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391570+0000", 2024-09-18T11:31:00.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6002999999999999e-05 2024-09-18T11:31:00.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391575+0000", 2024-09-18T11:31:00.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2549999999999997e-06 2024-09-18T11:31:00.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:00.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391631+0000", 2024-09-18T11:31:00.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5671000000000003e-05 2024-09-18T11:31:00.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:00.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391751+0000", 2024-09-18T11:31:00.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000119821 2024-09-18T11:31:00.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:00.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391974+0000", 2024-09-18T11:31:00.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022309900000000001 2024-09-18T11:31:00.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:00.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398136+0000", 2024-09-18T11:31:00.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006161573 2024-09-18T11:31:00.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:00.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398184+0000", 2024-09-18T11:31:00.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8458000000000003e-05 2024-09-18T11:31:00.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:00.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398219+0000", 2024-09-18T11:31:00.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4573999999999998e-05 2024-09-18T11:31:00.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:00.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398251+0000", 2024-09-18T11:31:00.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2085000000000002e-05 2024-09-18T11:31:00.938 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.938 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:00.939 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.939 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:31:00.939 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:00.939 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.495323379999999, 2024-09-18T11:31:00.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:31:00.939 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:00.939 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:00.939 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:00.939 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:00.939 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.939 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:31:00.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.940 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:00.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:00.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:00.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:00.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:00.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:00.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:31:00.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:00.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:00.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:31:00.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:31:00.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:00.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:31:00.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:31:00.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:31:00.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:31:00.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:00.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:31:00.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:31:00.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:00.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:31:00.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:31:00.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:00.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:31:00.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:31:00.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:00.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:31:00.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:31:00.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:00.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:31:00.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:31:00.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:00.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:31:00.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:31:00.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:00.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:31:00.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:31:00.945 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.945 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:00.945 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.945 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1628 3.3 3:f80aa31b:::77cdbf3abbada57d0636b2198046a030673818705cbf13bb77b30c3e68b24345:head [write 0~582144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:00.945 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:00.945 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.600433379999998, 2024-09-18T11:31:00.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081714639999999998, 2024-09-18T11:31:00.945 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:00.945 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:00.945 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:00.946 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:00.946 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.946 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1628 2024-09-18T11:31:00.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.946 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:00.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:00.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:00.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:00.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:00.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:00.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310975+0000", 2024-09-18T11:31:00.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:00.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:00.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312086+0000", 2024-09-18T11:31:00.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001111288 2024-09-18T11:31:00.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:00.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312104+0000", 2024-09-18T11:31:00.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7427000000000001e-05 2024-09-18T11:31:00.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312110+0000", 2024-09-18T11:31:00.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5320000000000003e-06 2024-09-18T11:31:00.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:00.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312179+0000", 2024-09-18T11:31:00.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9196000000000006e-05 2024-09-18T11:31:00.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:00.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312375+0000", 2024-09-18T11:31:00.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00019550899999999999 2024-09-18T11:31:00.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:00.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312558+0000", 2024-09-18T11:31:00.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000182566 2024-09-18T11:31:00.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:00.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.314286+0000", 2024-09-18T11:31:00.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00172831 2024-09-18T11:31:00.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:00.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319079+0000", 2024-09-18T11:31:00.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047929340000000004 2024-09-18T11:31:00.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:00.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319116+0000", 2024-09-18T11:31:00.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6993999999999999e-05 2024-09-18T11:31:00.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:00.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319148+0000", 2024-09-18T11:31:00.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2539999999999997e-05 2024-09-18T11:31:00.951 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.951 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:00.951 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.951 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1662 4.1 4:87078dfe:::smithi11939506-14:head [write 635331~786118,stat] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:00.951 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.360496+0000", 2024-09-18T11:31:00.951 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.550914635000002, 2024-09-18T11:31:00.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071849519999999997, 2024-09-18T11:31:00.951 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:00.951 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:00.951 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:00.951 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:00.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.952 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1662 2024-09-18T11:31:00.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.952 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:00.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:00.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.360496+0000", 2024-09-18T11:31:00.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:00.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.360496+0000", 2024-09-18T11:31:00.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:00.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.360493+0000", 2024-09-18T11:31:00.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:00.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:00.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361139+0000", 2024-09-18T11:31:00.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064615400000000002 2024-09-18T11:31:00.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:00.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361154+0000", 2024-09-18T11:31:00.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4970000000000001e-05 2024-09-18T11:31:00.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361161+0000", 2024-09-18T11:31:00.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4139999999999999e-06 2024-09-18T11:31:00.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:00.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361201+0000", 2024-09-18T11:31:00.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0385e-05 2024-09-18T11:31:00.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:00.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361273+0000", 2024-09-18T11:31:00.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2224999999999994e-05 2024-09-18T11:31:00.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:00.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361406+0000", 2024-09-18T11:31:00.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013257699999999999 2024-09-18T11:31:00.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:00.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367415+0000", 2024-09-18T11:31:00.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006009655 2024-09-18T11:31:00.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:00.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367586+0000", 2024-09-18T11:31:00.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017109800000000001 2024-09-18T11:31:00.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:00.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367636+0000", 2024-09-18T11:31:00.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9721e-05 2024-09-18T11:31:00.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:00.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367681+0000", 2024-09-18T11:31:00.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4382999999999998e-05 2024-09-18T11:31:00.957 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.957 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:00.957 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.957 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1663 4.1 4:87078dfe:::smithi11939506-14:head [write 1916458~318873] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:00.957 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:00.957 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.547486816999999, 2024-09-18T11:31:00.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079505749999999997, 2024-09-18T11:31:00.957 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:00.957 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:00.957 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:00.957 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:00.958 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.958 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1663 2024-09-18T11:31:00.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.958 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:00.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:00.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:00.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:00.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:00.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:00.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363922+0000", 2024-09-18T11:31:00.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:00.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:00.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364211+0000", 2024-09-18T11:31:00.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000288385 2024-09-18T11:31:00.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:00.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364223+0000", 2024-09-18T11:31:00.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2582000000000001e-05 2024-09-18T11:31:00.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364228+0000", 2024-09-18T11:31:00.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0000000000000004e-06 2024-09-18T11:31:00.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:00.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364268+0000", 2024-09-18T11:31:00.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0235999999999999e-05 2024-09-18T11:31:00.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:00.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364341+0000", 2024-09-18T11:31:00.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2266000000000002e-05 2024-09-18T11:31:00.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:00.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364444+0000", 2024-09-18T11:31:00.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103632 2024-09-18T11:31:00.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:00.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369461+0000", 2024-09-18T11:31:00.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005016848 2024-09-18T11:31:00.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:00.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371820+0000", 2024-09-18T11:31:00.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023586169999999999 2024-09-18T11:31:00.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:00.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371851+0000", 2024-09-18T11:31:00.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1297999999999997e-05 2024-09-18T11:31:00.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:00.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371874+0000", 2024-09-18T11:31:00.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2841e-05 2024-09-18T11:31:00.963 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.963 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:00.963 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.963 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1664 4.1 4:87078dfe:::smithi11939506-14:head [setxattr _header (57) in=7b,truncate 2235331] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:00.963 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:00.963 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.54717316, 2024-09-18T11:31:00.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010368235999999999, 2024-09-18T11:31:00.963 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:00.963 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:00.963 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:00.963 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:00.964 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.964 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1664 2024-09-18T11:31:00.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.964 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:00.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:00.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:00.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:00.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:00.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:00.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364236+0000", 2024-09-18T11:31:00.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:00.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:00.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364240+0000", 2024-09-18T11:31:00.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9829999999999998e-06 2024-09-18T11:31:00.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:00.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364247+0000", 2024-09-18T11:31:00.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2140000000000001e-06 2024-09-18T11:31:00.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364256+0000", 2024-09-18T11:31:00.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1509999999999993e-06 2024-09-18T11:31:00.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:00.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366529+0000", 2024-09-18T11:31:00.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022731409999999998 2024-09-18T11:31:00.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:00.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366621+0000", 2024-09-18T11:31:00.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2408e-05 2024-09-18T11:31:00.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:00.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366801+0000", 2024-09-18T11:31:00.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018002200000000001 2024-09-18T11:31:00.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:00.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372155+0000", 2024-09-18T11:31:00.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053531519999999999 2024-09-18T11:31:00.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:00.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374539+0000", 2024-09-18T11:31:00.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023842580000000002 2024-09-18T11:31:00.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:00.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374573+0000", 2024-09-18T11:31:00.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4205999999999997e-05 2024-09-18T11:31:00.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:00.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374605+0000", 2024-09-18T11:31:00.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2283999999999998e-05 2024-09-18T11:31:00.969 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.969 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:00.969 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.969 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1665 4.1 4:87078dfe:::smithi11939506-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:00.969 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:00.969 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.547148986, 2024-09-18T11:31:00.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010471897000000001, 2024-09-18T11:31:00.969 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:00.969 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:00.969 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:00.970 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:00.970 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.970 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1665 2024-09-18T11:31:00.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.970 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:00.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:00.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:00.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:00.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:00.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:00.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:00.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:00.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:00.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364263+0000", 2024-09-18T11:31:00.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6400000000000001e-06 2024-09-18T11:31:00.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:00.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364270+0000", 2024-09-18T11:31:00.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:31:00.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364274+0000", 2024-09-18T11:31:00.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.968e-06 2024-09-18T11:31:00.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:00.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367354+0000", 2024-09-18T11:31:00.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030803430000000001 2024-09-18T11:31:00.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:00.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367406+0000", 2024-09-18T11:31:00.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2330999999999998e-05 2024-09-18T11:31:00.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:00.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374625+0000", 2024-09-18T11:31:00.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072190689999999998 2024-09-18T11:31:00.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:00.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374673+0000", 2024-09-18T11:31:00.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7525999999999999e-05 2024-09-18T11:31:00.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:00.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374733+0000", 2024-09-18T11:31:00.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0359000000000003e-05 2024-09-18T11:31:00.974 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.974 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:00.974 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.975 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:101 4.5 e58/22 4:aa575018:::smithi11939506-11:17 v 58'67, mlcod=55'62)", 2024-09-18T11:31:00.975 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.273235+0000", 2024-09-18T11:31:00.975 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.638175695000001, 2024-09-18T11:31:00.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0067904050000000002, 2024-09-18T11:31:00.975 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:00.975 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:00.975 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:00.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:00.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273235+0000", 2024-09-18T11:31:00.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:00.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273235+0000", 2024-09-18T11:31:00.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:00.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273234+0000", 2024-09-18T11:31:00.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:00.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:00.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273239+0000", 2024-09-18T11:31:00.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7999999999999998e-06 2024-09-18T11:31:00.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:00.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273247+0000", 2024-09-18T11:31:00.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4700000000000002e-06 2024-09-18T11:31:00.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273251+0000", 2024-09-18T11:31:00.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7500000000000001e-06 2024-09-18T11:31:00.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:00.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.274599+0000", 2024-09-18T11:31:00.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013483900000000001 2024-09-18T11:31:00.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:00.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.274623+0000", 2024-09-18T11:31:00.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3499999999999999e-05 2024-09-18T11:31:00.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:00.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.279994+0000", 2024-09-18T11:31:00.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053713889999999999 2024-09-18T11:31:00.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:00.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.280025+0000", 2024-09-18T11:31:00.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0579e-05 2024-09-18T11:31:00.979 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.979 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:00.979 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.979 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1919 4.9 4:99feffe9:::smithi11939506-45:head [write 638699~541153] snapc 1e=[1e,1c,1b,19,18,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:00.979 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.342505+0000", 2024-09-18T11:31:00.979 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.568905710999999, 2024-09-18T11:31:00.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064384309999999997, 2024-09-18T11:31:00.980 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:00.980 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:00.980 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:00.980 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:00.980 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.980 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1919 2024-09-18T11:31:00.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.980 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:00.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:00.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.342505+0000", 2024-09-18T11:31:00.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:00.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.342505+0000", 2024-09-18T11:31:00.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:00.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.342501+0000", 2024-09-18T11:31:00.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999962 2024-09-18T11:31:00.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:00.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.343053+0000", 2024-09-18T11:31:00.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00055221999999999997 2024-09-18T11:31:00.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:00.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.343069+0000", 2024-09-18T11:31:00.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5917999999999999e-05 2024-09-18T11:31:00.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.343075+0000", 2024-09-18T11:31:00.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3260000000000001e-06 2024-09-18T11:31:00.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:00.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.343153+0000", 2024-09-18T11:31:00.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8183999999999995e-05 2024-09-18T11:31:00.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:00.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.343261+0000", 2024-09-18T11:31:00.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0001076 2024-09-18T11:31:00.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:00.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.343469+0000", 2024-09-18T11:31:00.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020852199999999999 2024-09-18T11:31:00.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:00.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.346706+0000", 2024-09-18T11:31:00.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032370400000000001 2024-09-18T11:31:00.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:00.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.348871+0000", 2024-09-18T11:31:00.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021645200000000001 2024-09-18T11:31:00.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:00.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.348914+0000", 2024-09-18T11:31:00.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3031999999999997e-05 2024-09-18T11:31:00.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:00.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.348943+0000", 2024-09-18T11:31:00.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9039000000000001e-05 2024-09-18T11:31:00.985 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.985 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:00.985 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.985 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1947 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865577,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:00.986 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:00.986 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.5718692759999993, 2024-09-18T11:31:00.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010043066999999999, 2024-09-18T11:31:00.986 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:00.986 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:00.986 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:00.986 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:00.986 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.986 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1947 2024-09-18T11:31:00.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.986 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:00.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:00.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:00.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:00.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:00.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:00.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339540+0000", 2024-09-18T11:31:00.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:00.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:00.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339545+0000", 2024-09-18T11:31:00.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6290000000000001e-06 2024-09-18T11:31:00.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:00.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339553+0000", 2024-09-18T11:31:00.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6299999999999998e-06 2024-09-18T11:31:00.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339557+0000", 2024-09-18T11:31:00.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3590000000000001e-06 2024-09-18T11:31:00.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:00.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345600+0000", 2024-09-18T11:31:00.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060434640000000001 2024-09-18T11:31:00.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:00.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345710+0000", 2024-09-18T11:31:00.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010924700000000001 2024-09-18T11:31:00.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:00.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349584+0000", 2024-09-18T11:31:00.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038743459999999999 2024-09-18T11:31:00.990 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.990 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:00.990 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.990 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1948 4.d 4:b5a463bf:::smithi11939506-20:head [sparse-read 0~2642181] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:00.990 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:00.990 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.571848739, 2024-09-18T11:31:00.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010701952000000001, 2024-09-18T11:31:00.990 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:00.990 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:00.990 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:00.991 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:00.991 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:00.991 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1948 2024-09-18T11:31:00.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.991 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:00.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:00.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:00.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:00.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:00.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:00.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:00.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:00.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:00.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339563+0000", 2024-09-18T11:31:00.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1950000000000002e-06 2024-09-18T11:31:00.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:00.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339569+0000", 2024-09-18T11:31:00.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5459999999999999e-06 2024-09-18T11:31:00.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339572+0000", 2024-09-18T11:31:00.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.337e-06 2024-09-18T11:31:00.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:00.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349693+0000", 2024-09-18T11:31:00.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010120987999999999 2024-09-18T11:31:00.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:00.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349816+0000", 2024-09-18T11:31:00.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012241199999999999 2024-09-18T11:31:00.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:00.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350263+0000", 2024-09-18T11:31:00.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00044787199999999998 2024-09-18T11:31:00.994 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.994 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:00.994 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.995 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:129 4.5 e69/22 4:a087a566:::smithi11939506-46:19 v 69'95, mlcod=65'92)", 2024-09-18T11:31:00.995 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.343498+0000", 2024-09-18T11:31:00.995 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.5679120930000003, 2024-09-18T11:31:00.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068162960000000003, 2024-09-18T11:31:00.995 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:00.995 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:00.995 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:00.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:00.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343498+0000", 2024-09-18T11:31:00.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:00.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343498+0000", 2024-09-18T11:31:00.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:00.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:00.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343497+0000", 2024-09-18T11:31:00.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:00.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:00.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343504+0000", 2024-09-18T11:31:00.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5799999999999997e-06 2024-09-18T11:31:00.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:00.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343513+0000", 2024-09-18T11:31:00.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7940000000000008e-06 2024-09-18T11:31:00.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:00.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343518+0000", 2024-09-18T11:31:00.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2719999999999997e-06 2024-09-18T11:31:00.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:00.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345616+0000", 2024-09-18T11:31:00.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020977410000000002 2024-09-18T11:31:00.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:00.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345641+0000", 2024-09-18T11:31:00.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5539000000000001e-05 2024-09-18T11:31:00.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:00.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350279+0000", 2024-09-18T11:31:00.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046379560000000004 2024-09-18T11:31:00.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:00.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350314+0000", 2024-09-18T11:31:00.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5083999999999998e-05 2024-09-18T11:31:00.999 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.999 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:00.999 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:00.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:00.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:00.999 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:131 4.5 e69/22 4:aa575018:::smithi11939506-11:17 v 69'97, mlcod=65'92)", 2024-09-18T11:31:00.999 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.346049+0000", 2024-09-18T11:31:00.999 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.5653616790000004, 2024-09-18T11:31:00.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0077275240000000004, 2024-09-18T11:31:00.999 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:00.999 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.000 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346049+0000", 2024-09-18T11:31:01.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346049+0000", 2024-09-18T11:31:01.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346047+0000", 2024-09-18T11:31:01.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:01.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346058+0000", 2024-09-18T11:31:01.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0463e-05 2024-09-18T11:31:01.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346069+0000", 2024-09-18T11:31:01.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1708999999999999e-05 2024-09-18T11:31:01.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346076+0000", 2024-09-18T11:31:01.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2450000000000003e-06 2024-09-18T11:31:01.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349616+0000", 2024-09-18T11:31:01.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035406550000000002 2024-09-18T11:31:01.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349638+0000", 2024-09-18T11:31:01.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2098000000000001e-05 2024-09-18T11:31:01.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.353735+0000", 2024-09-18T11:31:01.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004096237 2024-09-18T11:31:01.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.353776+0000", 2024-09-18T11:31:01.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1530999999999999e-05 2024-09-18T11:31:01.003 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.003 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.003 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.004 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1995 3.3 3:ef0abd1c:::fe6aae3707ee6dfd44fc3a3fe0efa6b3e5364ad2f76a43a9eeb8b69a3c3bcab6:head [write 0~577536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:01.004 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:52.187800+0000", 2024-09-18T11:31:01.004 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.7236101379999997, 2024-09-18T11:31:01.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066968339999999996, 2024-09-18T11:31:01.004 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.004 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.004 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.004 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.004 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.004 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1995 2024-09-18T11:31:01.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.004 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:52.187800+0000", 2024-09-18T11:31:01.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:52.187800+0000", 2024-09-18T11:31:01.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:52.187799+0000", 2024-09-18T11:31:01.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:01.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:52.189212+0000", 2024-09-18T11:31:01.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014134499999999999 2024-09-18T11:31:01.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:52.189225+0000", 2024-09-18T11:31:01.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3278000000000001e-05 2024-09-18T11:31:01.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:52.189234+0000", 2024-09-18T11:31:01.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4069999999999993e-06 2024-09-18T11:31:01.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:52.189276+0000", 2024-09-18T11:31:01.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2602000000000003e-05 2024-09-18T11:31:01.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:52.189380+0000", 2024-09-18T11:31:01.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103849 2024-09-18T11:31:01.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 6", 2024-09-18T11:31:01.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:52.189491+0000", 2024-09-18T11:31:01.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011082399999999999 2024-09-18T11:31:01.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:01.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:52.191791+0000", 2024-09-18T11:31:01.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022994579999999999 2024-09-18T11:31:01.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:01.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:52.194407+0000", 2024-09-18T11:31:01.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026161180000000002 2024-09-18T11:31:01.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:52.194457+0000", 2024-09-18T11:31:01.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0528000000000001e-05 2024-09-18T11:31:01.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:52.194497+0000", 2024-09-18T11:31:01.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9768999999999998e-05 2024-09-18T11:31:01.009 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.009 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.009 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.010 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2128 4.1 4:8b9a985d:::smithi11939506-23:16 [read 0~3512630,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T11:31:01.010 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:55.710141+0000", 2024-09-18T11:31:01.010 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.201268765, 2024-09-18T11:31:01.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068000389999999999, 2024-09-18T11:31:01.010 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.010 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:01.010 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.010 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.010 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.010 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2128 2024-09-18T11:31:01.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.010 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710141+0000", 2024-09-18T11:31:01.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710141+0000", 2024-09-18T11:31:01.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710141+0000", 2024-09-18T11:31:01.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710162+0000", 2024-09-18T11:31:01.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1492e-05 2024-09-18T11:31:01.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710169+0000", 2024-09-18T11:31:01.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0029999999999997e-06 2024-09-18T11:31:01.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710174+0000", 2024-09-18T11:31:01.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1250000000000003e-06 2024-09-18T11:31:01.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.715332+0000", 2024-09-18T11:31:01.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051581680000000003 2024-09-18T11:31:01.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.715431+0000", 2024-09-18T11:31:01.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9693999999999998e-05 2024-09-18T11:31:01.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.716942+0000", 2024-09-18T11:31:01.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001510033 2024-09-18T11:31:01.014 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.014 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.014 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.014 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2129 4.1 4:8b9a985d:::smithi11939506-23:16 [read 0~3512630,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T11:31:01.014 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:55.710202+0000", 2024-09-18T11:31:01.014 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.2012084180000002, 2024-09-18T11:31:01.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0078396100000000003, 2024-09-18T11:31:01.014 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.014 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:01.014 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.015 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.015 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.015 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2129 2024-09-18T11:31:01.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.015 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710202+0000", 2024-09-18T11:31:01.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710202+0000", 2024-09-18T11:31:01.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710202+0000", 2024-09-18T11:31:01.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710205+0000", 2024-09-18T11:31:01.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.382e-06 2024-09-18T11:31:01.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710212+0000", 2024-09-18T11:31:01.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770000000000004e-06 2024-09-18T11:31:01.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710217+0000", 2024-09-18T11:31:01.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1680000000000003e-06 2024-09-18T11:31:01.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.716963+0000", 2024-09-18T11:31:01.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0067462449999999997 2024-09-18T11:31:01.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.717033+0000", 2024-09-18T11:31:01.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9643999999999999e-05 2024-09-18T11:31:01.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.718041+0000", 2024-09-18T11:31:01.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0010086019999999999 2024-09-18T11:31:01.018 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.018 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.018 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.019 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2175 4.e 4:73745217:::smithi11939506-6:head [write 742909~464035,stat] snapc 1e=[1e,1c,1b,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-18T11:31:01.019 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:56.426542+0000", 2024-09-18T11:31:01.019 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.4848683769999997, 2024-09-18T11:31:01.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070437850000000003, 2024-09-18T11:31:01.019 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.019 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.019 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.019 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.019 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.019 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2175 2024-09-18T11:31:01.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.019 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.426542+0000", 2024-09-18T11:31:01.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.426542+0000", 2024-09-18T11:31:01.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.426540+0000", 2024-09-18T11:31:01.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:01.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.426987+0000", 2024-09-18T11:31:01.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00044653899999999997 2024-09-18T11:31:01.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.427005+0000", 2024-09-18T11:31:01.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8669e-05 2024-09-18T11:31:01.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.022 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.427017+0000", 2024-09-18T11:31:01.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1666e-05 2024-09-18T11:31:01.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.022 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.022 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.427065+0000", 2024-09-18T11:31:01.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8071000000000001e-05 2024-09-18T11:31:01.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.022 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.022 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.427134+0000", 2024-09-18T11:31:01.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8736000000000001e-05 2024-09-18T11:31:01.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 6", 2024-09-18T11:31:01.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.427252+0000", 2024-09-18T11:31:01.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000118402 2024-09-18T11:31:01.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:01.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.428537+0000", 2024-09-18T11:31:01.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001284456 2024-09-18T11:31:01.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:01.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.433512+0000", 2024-09-18T11:31:01.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049752499999999996 2024-09-18T11:31:01.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.433545+0000", 2024-09-18T11:31:01.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2529e-05 2024-09-18T11:31:01.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.433586+0000", 2024-09-18T11:31:01.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1087999999999997e-05 2024-09-18T11:31:01.024 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.024 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.024 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.024 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.025 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.025 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:01.027 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:31:01.028 INFO:tasks.rados.rados.0.smithi119.stdout:1131: finishing set_chunk to oid 6 2024-09-18T11:31:01.028 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 6 set_chunk returned error code -95 offset: 1133568 length: 459264 2024-09-18T11:31:01.029 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 6 v 0 (ObjNum 187 snap 20 seq_num 187) dirty exists 2024-09-18T11:31:01.029 INFO:tasks.rados.rados.0.smithi119.stdout:1132: finishing write tid 1 to smithi11939506-9 2024-09-18T11:31:01.029 INFO:tasks.rados.rados.0.smithi119.stdout:1130: done (5 left) 2024-09-18T11:31:01.029 INFO:tasks.rados.rados.0.smithi119.stdout:1131: done (4 left) 2024-09-18T11:31:01.029 INFO:tasks.rados.rados.0.smithi119.stdout:1136: tier_promote oid 37 2024-09-18T11:31:01.029 INFO:tasks.rados.rados.0.smithi119.stdout:1132: finishing write tid 2 to smithi11939506-9 2024-09-18T11:31:01.029 INFO:tasks.rados.rados.0.smithi119.stdout:1132: finishing write tid 3 to smithi11939506-9 2024-09-18T11:31:01.029 INFO:tasks.rados.rados.0.smithi119.stdout:1135: finishing rollback tid 0 to smithi11939506-14 2024-09-18T11:31:01.029 INFO:tasks.rados.rados.0.smithi119.stdout:1132: finishing write tid 4 to smithi11939506-9 2024-09-18T11:31:01.029 INFO:tasks.rados.rados.0.smithi119.stdout:1132: finishing write tid 5 to smithi11939506-9 2024-09-18T11:31:01.029 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 184 (ObjNum 199 snap 23 seq_num 199) dirty exists 2024-09-18T11:31:01.029 INFO:tasks.rados.rados.0.smithi119.stdout:1132: left oid 9 (ObjNum 199 snap 23 seq_num 199) 2024-09-18T11:31:01.029 INFO:tasks.rados.rados.0.smithi119.stdout:1135: finishing rollback tid 1 to smithi11939506-14 2024-09-18T11:31:01.030 INFO:tasks.rados.rados.0.smithi119.stdout:1135: finishing rollback tid 2 to smithi11939506-14 2024-09-18T11:31:01.030 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 142 (ObjNum 66 snap 0 seq_num 66) dirty exists 2024-09-18T11:31:01.030 INFO:tasks.rados.rados.0.smithi119.stdout:1132: done (4 left) 2024-09-18T11:31:01.030 INFO:tasks.rados.rados.0.smithi119.stdout:1133: done (3 left) 2024-09-18T11:31:01.030 INFO:tasks.rados.rados.0.smithi119.stdout:1134: done (2 left) 2024-09-18T11:31:01.030 INFO:tasks.rados.rados.0.smithi119.stdout:1135: done (1 left) 2024-09-18T11:31:01.030 INFO:tasks.rados.rados.0.smithi119.stdout:1137: write oid 28 current snap is 23 2024-09-18T11:31:01.030 INFO:tasks.rados.rados.0.smithi119.stdout:1137: seq_num 200 ranges {561400=474978,1820281=519723,2859422=457579} 2024-09-18T11:31:01.030 INFO:tasks.rados.rados.0.smithi119.stdout:1137: writing smithi11939506-28 from 561400 to 1036378 tid 1 2024-09-18T11:31:01.030 INFO:tasks.rados.rados.0.smithi119.stdout:1137: writing smithi11939506-28 from 1820281 to 2340004 tid 2 2024-09-18T11:31:01.030 INFO:tasks.rados.rados.0.smithi119.stdout:1137: writing smithi11939506-28 from 2859422 to 3317001 tid 3 2024-09-18T11:31:01.030 INFO:tasks.rados.rados.0.smithi119.stdout:1138: tier_promote oid 2 2024-09-18T11:31:01.030 INFO:tasks.rados.rados.0.smithi119.stdout:1139: tier_evict oid 17 2024-09-18T11:31:01.030 INFO:tasks.rados.rados.0.smithi119.stdout:1139: tier_evict oid 17 snap -1 2024-09-18T11:31:01.031 INFO:tasks.rados.rados.0.smithi119.stdout:1136: got (0) Success 2024-09-18T11:31:01.031 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 121 (ObjNum 188 snap 20 seq_num 188) dirty exists 2024-09-18T11:31:01.031 INFO:tasks.rados.rados.0.smithi119.stdout:1137: finishing write tid 1 to smithi11939506-28 2024-09-18T11:31:01.031 INFO:tasks.rados.rados.0.smithi119.stdout:1137: finishing write tid 2 to smithi11939506-28 2024-09-18T11:31:01.031 INFO:tasks.rados.rados.0.smithi119.stdout:1136: done (3 left) 2024-09-18T11:31:01.031 INFO:tasks.rados.rados.0.smithi119.stdout:1140: rollback oid 14 current snap is 23 2024-09-18T11:31:01.031 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 14 to 21 2024-09-18T11:31:01.031 INFO:tasks.rados.rados.0.smithi119.stdout:1141: read oid 7 snap -1 2024-09-18T11:31:01.031 INFO:tasks.rados.rados.0.smithi119.stdout:1141: expect (ObjNum 145 snap 18 seq_num 145) 2024-09-18T11:31:01.031 INFO:tasks.rados.rados.0.smithi119.stdout:1142: set_chunk oid 47 target oid 2024-09-18T11:31:01.031 INFO:tasks.rados.rados.0.smithi119.stdout:1142: set_chunk oid 47 offset: 1314816 length: 521216 target oid 40900f1ddc0ea877b9771dbac792ede9c7f71b94693b4b3c18b4dca4339a196d offset: 0 snap 20 2024-09-18T11:31:01.031 INFO:tasks.rados.rados.0.smithi119.stdout:1143: write oid 15 current snap is 23 2024-09-18T11:31:01.031 INFO:tasks.rados.rados.0.smithi119.stdout:1139: got (22) Invalid argument 2024-09-18T11:31:01.031 INFO:tasks.rados.rados.0.smithi119.stdout:1143: seq_num 201 ranges {578207=550227,1762410=779734,3286845=91362} 2024-09-18T11:31:01.032 INFO:tasks.rados.rados.0.smithi119.stdout:1143: writing smithi11939506-15 from 578207 to 1128434 tid 1 2024-09-18T11:31:01.032 INFO:tasks.rados.rados.0.smithi119.stdout:1143: writing smithi11939506-15 from 1762410 to 2542144 tid 2 2024-09-18T11:31:01.032 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:01.024+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:01.033 INFO:tasks.rados.rados.0.smithi119.stdout:1143: writing smithi11939506-15 from 3286845 to 3378207 tid 3 2024-09-18T11:31:01.033 INFO:tasks.rados.rados.0.smithi119.stdout:1139: done (6 left) 2024-09-18T11:31:01.033 INFO:tasks.rados.rados.0.smithi119.stdout:1144: write (excl) oid 29 current snap is 23 2024-09-18T11:31:01.033 INFO:tasks.rados.rados.0.smithi119.stdout:1137: finishing write tid 3 to smithi11939506-28 2024-09-18T11:31:01.033 INFO:tasks.rados.rados.0.smithi119.stdout:1137: finishing write tid 4 to smithi11939506-28 2024-09-18T11:31:01.033 INFO:tasks.rados.rados.0.smithi119.stdout:1137: finishing write tid 5 to smithi11939506-28 2024-09-18T11:31:01.033 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 28 v 97 (ObjNum 200 snap 23 seq_num 200) dirty exists 2024-09-18T11:31:01.033 INFO:tasks.rados.rados.0.smithi119.stdout:1137: left oid 28 (ObjNum 200 snap 23 seq_num 200) 2024-09-18T11:31:01.033 INFO:tasks.rados.rados.0.smithi119.stdout:1138: got (0) Success 2024-09-18T11:31:01.033 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 172 (ObjNum 195 snap 23 seq_num 195) dirty exists 2024-09-18T11:31:01.033 INFO:tasks.rados.rados.0.smithi119.stdout:1141: expect (ObjNum 145 snap 18 seq_num 145) 2024-09-18T11:31:01.118 INFO:teuthology.orchestra.run.smithi119.stderr:Nothing to do for non-filestore backend 2024-09-18T11:31:01.126 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:01.124+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:01.167 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:01.167 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:01.167 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:01.167 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:01.176 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:31:01.176 INFO:tasks.rados.rados.0.smithi119.stdout:1140: finishing rollback tid 0 to smithi11939506-14 2024-09-18T11:31:01.177 INFO:tasks.rados.rados.0.smithi119.stdout:1140: finishing rollback tid 1 to smithi11939506-14 2024-09-18T11:31:01.177 INFO:tasks.rados.rados.0.smithi119.stdout:1140: finishing rollback tid 2 to smithi11939506-14 2024-09-18T11:31:01.177 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 145 (ObjNum 153 snap 20 seq_num 153) dirty exists 2024-09-18T11:31:01.177 INFO:tasks.rados.rados.0.smithi119.stdout:1142: finishing set_chunk to oid 47 2024-09-18T11:31:01.177 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 47 set_chunk returned error code -95 offset: 1314816 length: 521216 2024-09-18T11:31:01.177 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 0 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-18T11:31:01.177 INFO:tasks.rados.rados.0.smithi119.stdout:1143: finishing write tid 1 to smithi11939506-15 2024-09-18T11:31:01.177 INFO:tasks.rados.rados.0.smithi119.stdout:1143: finishing write tid 2 to smithi11939506-15 2024-09-18T11:31:01.177 INFO:tasks.rados.rados.0.smithi119.stdout:1143: finishing write tid 3 to smithi11939506-15 2024-09-18T11:31:01.177 INFO:tasks.rados.rados.0.smithi119.stdout:1144: seq_num 202 ranges {595014=625476,1820892=523392,2797915=596476} 2024-09-18T11:31:01.182 INFO:tasks.rados.rados.0.smithi119.stdout:1144: writing smithi11939506-29 from 595014 to 1220490 tid 1 2024-09-18T11:31:01.188 INFO:tasks.rados.rados.0.smithi119.stdout:1144: writing smithi11939506-29 from 1820892 to 2344284 tid 2 2024-09-18T11:31:01.194 INFO:tasks.rados.rados.0.smithi119.stdout:1144: writing smithi11939506-29 from 2797915 to 3394391 tid 3 2024-09-18T11:31:01.194 INFO:tasks.rados.rados.0.smithi119.stdout:1137: done (6 left) 2024-09-18T11:31:01.194 INFO:tasks.rados.rados.0.smithi119.stdout:1138: done (5 left) 2024-09-18T11:31:01.195 INFO:tasks.rados.rados.0.smithi119.stdout:1140: done (4 left) 2024-09-18T11:31:01.195 INFO:tasks.rados.rados.0.smithi119.stdout:1141: done (3 left) 2024-09-18T11:31:01.195 INFO:tasks.rados.rados.0.smithi119.stdout:1142: done (2 left) 2024-09-18T11:31:01.195 INFO:tasks.rados.rados.0.smithi119.stdout:1145: read oid 46 snap -1 2024-09-18T11:31:01.195 INFO:tasks.rados.rados.0.smithi119.stdout:1145: expect (ObjNum 171 snap 20 seq_num 171) 2024-09-18T11:31:01.195 INFO:tasks.rados.rados.0.smithi119.stdout:1146: rollback oid 38 current snap is 23 2024-09-18T11:31:01.195 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 38 to 19 2024-09-18T11:31:01.195 INFO:tasks.rados.rados.0.smithi119.stdout:1143: finishing write tid 4 to smithi11939506-15 2024-09-18T11:31:01.195 INFO:tasks.rados.rados.0.smithi119.stdout:1143: finishing write tid 5 to smithi11939506-15 2024-09-18T11:31:01.195 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 54 (ObjNum 201 snap 23 seq_num 201) dirty exists 2024-09-18T11:31:01.195 INFO:tasks.rados.rados.0.smithi119.stdout:1143: left oid 15 (ObjNum 201 snap 23 seq_num 201) 2024-09-18T11:31:01.195 INFO:tasks.rados.rados.0.smithi119.stdout:1144: finishing write tid 1 to smithi11939506-29 2024-09-18T11:31:01.195 INFO:tasks.rados.rados.0.smithi119.stdout:1144: finishing write tid 2 to smithi11939506-29 2024-09-18T11:31:01.195 INFO:tasks.rados.rados.0.smithi119.stdout:1143: done (3 left) 2024-09-18T11:31:01.195 INFO:tasks.rados.rados.0.smithi119.stdout:1147: snap_remove snap 12 2024-09-18T11:31:01.227 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:01.225+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:01.278 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:01.278 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:01.278 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:01.278 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:01.278 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:01.286 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:31:01.329 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:01.328+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:01.387 INFO:tasks.rados.rados.0.smithi119.stdout:1147: done (3 left) 2024-09-18T11:31:01.388 INFO:tasks.rados.rados.0.smithi119.stdout:1148: read oid 33 snap 14 2024-09-18T11:31:01.388 INFO:tasks.rados.rados.0.smithi119.stdout:1148: expect deleted 2024-09-18T11:31:01.388 INFO:tasks.rados.rados.0.smithi119.stdout:1149: read oid 6 snap -1 2024-09-18T11:31:01.388 INFO:tasks.rados.rados.0.smithi119.stdout:1149: expect (ObjNum 187 snap 20 seq_num 187) 2024-09-18T11:31:01.388 INFO:tasks.rados.rados.0.smithi119.stdout:1150: read oid 11 snap -1 2024-09-18T11:31:01.388 INFO:tasks.rados.rados.0.smithi119.stdout:1150: expect (ObjNum 148 snap 18 seq_num 148) 2024-09-18T11:31:01.388 INFO:tasks.rados.rados.0.smithi119.stdout:1151: delete oid 21 current snap is 23 2024-09-18T11:31:01.388 INFO:tasks.rados.rados.0.smithi119.stdout:1144: finishing write tid 3 to smithi11939506-29 2024-09-18T11:31:01.388 INFO:tasks.rados.rados.0.smithi119.stdout:1144: finishing write tid 4 to smithi11939506-29 2024-09-18T11:31:01.388 INFO:tasks.rados.rados.0.smithi119.stdout:1144: finishing write tid 5 to smithi11939506-29 2024-09-18T11:31:01.388 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 59 (ObjNum 202 snap 23 seq_num 202) dirty exists 2024-09-18T11:31:01.388 INFO:tasks.rados.rados.0.smithi119.stdout:1144: left oid 29 (ObjNum 202 snap 23 seq_num 202) 2024-09-18T11:31:01.388 INFO:tasks.rados.rados.0.smithi119.stdout:1145: expect (ObjNum 171 snap 20 seq_num 171) 2024-09-18T11:31:01.389 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:01.390 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:01.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:01.390 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:01.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.390 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:31:01.390 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:01.390 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 71.359052528999996, 2024-09-18T11:31:01.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073601550000000002, 2024-09-18T11:31:01.390 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.390 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.390 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.390 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:31:01.390 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:31:01.390 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1 2024-09-18T11:31:01.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.391 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:01.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:01.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026524+0000", 2024-09-18T11:31:01.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999905 2024-09-18T11:31:01.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026541+0000", 2024-09-18T11:31:01.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7070000000000001e-05 2024-09-18T11:31:01.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026554+0000", 2024-09-18T11:31:01.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.288e-05 2024-09-18T11:31:01.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026564+0000", 2024-09-18T11:31:01.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0197000000000001e-05 2024-09-18T11:31:01.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026826+0000", 2024-09-18T11:31:01.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026212400000000003 2024-09-18T11:31:01.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:31:01.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026839+0000", 2024-09-18T11:31:01.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2831e-05 2024-09-18T11:31:01.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028605+0000", 2024-09-18T11:31:01.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001765794 2024-09-18T11:31:01.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028709+0000", 2024-09-18T11:31:01.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104348 2024-09-18T11:31:01.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:01.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028824+0000", 2024-09-18T11:31:01.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011527399999999999 2024-09-18T11:31:01.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:01.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030319+0000", 2024-09-18T11:31:01.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001494439 2024-09-18T11:31:01.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:01.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033805+0000", 2024-09-18T11:31:01.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034857820000000002 2024-09-18T11:31:01.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033856+0000", 2024-09-18T11:31:01.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1687999999999997e-05 2024-09-18T11:31:01.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033894+0000", 2024-09-18T11:31:01.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7481000000000002e-05 2024-09-18T11:31:01.396 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.396 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.397 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.397 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T11:31:01.397 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:31:01.397 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 69.339774098000007, 2024-09-18T11:31:01.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055107869999999996, 2024-09-18T11:31:01.397 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.397 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.397 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.397 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:31:01.397 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:31:01.397 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 7 2024-09-18T11:31:01.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.398 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:31:01.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:31:01.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045810+0000", 2024-09-18T11:31:01.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:01.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045820+0000", 2024-09-18T11:31:01.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0215e-05 2024-09-18T11:31:01.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045838+0000", 2024-09-18T11:31:01.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7849000000000001e-05 2024-09-18T11:31:01.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045845+0000", 2024-09-18T11:31:01.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4229999999999999e-06 2024-09-18T11:31:01.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045898+0000", 2024-09-18T11:31:01.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2537999999999997e-05 2024-09-18T11:31:01.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:01.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045904+0000", 2024-09-18T11:31:01.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4019999999999999e-06 2024-09-18T11:31:01.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047663+0000", 2024-09-18T11:31:01.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017584079999999999 2024-09-18T11:31:01.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047736+0000", 2024-09-18T11:31:01.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3509000000000004e-05 2024-09-18T11:31:01.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:01.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047857+0000", 2024-09-18T11:31:01.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012037800000000001 2024-09-18T11:31:01.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:01.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.050868+0000", 2024-09-18T11:31:01.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030111420000000001 2024-09-18T11:31:01.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:01.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051239+0000", 2024-09-18T11:31:01.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037141000000000002 2024-09-18T11:31:01.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051288+0000", 2024-09-18T11:31:01.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9051000000000002e-05 2024-09-18T11:31:01.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051323+0000", 2024-09-18T11:31:01.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4545000000000002e-05 2024-09-18T11:31:01.403 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.403 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.403 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.404 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:905 4.7 e31/22 4:e73ba68f:::smithi11939506-1:head v 31'32, mlcod=30'29)", 2024-09-18T11:31:01.404 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:31:01.404 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 50.244117887000002, 2024-09-18T11:31:01.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048351990000000001, 2024-09-18T11:31:01.404 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.404 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.404 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:31:01.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:31:01.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:31:01.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144912+0000", 2024-09-18T11:31:01.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003444 2024-09-18T11:31:01.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144921+0000", 2024-09-18T11:31:01.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1900000000000001e-06 2024-09-18T11:31:01.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144927+0000", 2024-09-18T11:31:01.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4940000000000003e-06 2024-09-18T11:31:01.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144952+0000", 2024-09-18T11:31:01.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5845000000000001e-05 2024-09-18T11:31:01.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144966+0000", 2024-09-18T11:31:01.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.396e-05 2024-09-18T11:31:01.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.146274+0000", 2024-09-18T11:31:01.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013073550000000001 2024-09-18T11:31:01.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.146303+0000", 2024-09-18T11:31:01.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9754e-05 2024-09-18T11:31:01.408 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.408 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.408 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.408 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:911 4.3 e31/22 4:cc46532a:::smithi11939506-33:head v 31'24, mlcod=30'21)", 2024-09-18T11:31:01.408 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.166801+0000", 2024-09-18T11:31:01.408 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 50.218785560999997, 2024-09-18T11:31:01.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048144349999999997, 2024-09-18T11:31:01.408 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.408 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.408 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.166801+0000", 2024-09-18T11:31:01.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.166801+0000", 2024-09-18T11:31:01.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.166799+0000", 2024-09-18T11:31:01.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:01.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167413+0000", 2024-09-18T11:31:01.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000613259 2024-09-18T11:31:01.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167431+0000", 2024-09-18T11:31:01.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7906000000000001e-05 2024-09-18T11:31:01.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167438+0000", 2024-09-18T11:31:01.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.883e-06 2024-09-18T11:31:01.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167468+0000", 2024-09-18T11:31:01.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0168e-05 2024-09-18T11:31:01.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167483+0000", 2024-09-18T11:31:01.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4997999999999999e-05 2024-09-18T11:31:01.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.171574+0000", 2024-09-18T11:31:01.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040909589999999999 2024-09-18T11:31:01.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.171615+0000", 2024-09-18T11:31:01.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1371000000000001e-05 2024-09-18T11:31:01.412 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.412 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.412 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.413 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1118 3.2 e38/21 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head v 38'76, mlcod=38'74)", 2024-09-18T11:31:01.413 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:01.413 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 41.437642377000003, 2024-09-18T11:31:01.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006048688, 2024-09-18T11:31:01.413 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.413 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.413 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:01.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:01.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947941+0000", 2024-09-18T11:31:01.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:01.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949868+0000", 2024-09-18T11:31:01.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019267659999999999 2024-09-18T11:31:01.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949893+0000", 2024-09-18T11:31:01.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5239999999999999e-05 2024-09-18T11:31:01.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949901+0000", 2024-09-18T11:31:01.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8420000000000005e-06 2024-09-18T11:31:01.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949954+0000", 2024-09-18T11:31:01.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3464999999999997e-05 2024-09-18T11:31:01.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949974+0000", 2024-09-18T11:31:01.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9731000000000001e-05 2024-09-18T11:31:01.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953948+0000", 2024-09-18T11:31:01.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039735489999999998 2024-09-18T11:31:01.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953992+0000", 2024-09-18T11:31:01.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4700999999999998e-05 2024-09-18T11:31:01.417 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.417 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.417 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.417 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.13:29 3.2 e39/21 3:5fa8fcfa:::ec59520759824f0488c7a3ca135dacb324de2330e16b7f54053de5719c4e96b6:head v 39'81, mlcod=38'76)", 2024-09-18T11:31:01.417 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:31:01.417 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.703941901, 2024-09-18T11:31:01.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004755858, 2024-09-18T11:31:01.417 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.417 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.418 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:31:01.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:31:01.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:31:01.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681961+0000", 2024-09-18T11:31:01.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00031696799999999999 2024-09-18T11:31:01.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681969+0000", 2024-09-18T11:31:01.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4800000000000001e-06 2024-09-18T11:31:01.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681973+0000", 2024-09-18T11:31:01.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4819999999999999e-06 2024-09-18T11:31:01.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682007+0000", 2024-09-18T11:31:01.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3961999999999998e-05 2024-09-18T11:31:01.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682033+0000", 2024-09-18T11:31:01.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6103999999999999e-05 2024-09-18T11:31:01.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686375+0000", 2024-09-18T11:31:01.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043422019999999999 2024-09-18T11:31:01.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686400+0000", 2024-09-18T11:31:01.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5265e-05 2024-09-18T11:31:01.421 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.421 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.421 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.421 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1623 4.3 e56/22 4:c70c3162:::smithi11939506-35:head v 56'43, mlcod=48'40)", 2024-09-18T11:31:01.421 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.302560+0000", 2024-09-18T11:31:01.421 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 24.083026564000001, 2024-09-18T11:31:01.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046159060000000003, 2024-09-18T11:31:01.422 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.422 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.422 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.302560+0000", 2024-09-18T11:31:01.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.302560+0000", 2024-09-18T11:31:01.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.302559+0000", 2024-09-18T11:31:01.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303837+0000", 2024-09-18T11:31:01.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012781089999999999 2024-09-18T11:31:01.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303853+0000", 2024-09-18T11:31:01.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5551999999999999e-05 2024-09-18T11:31:01.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303862+0000", 2024-09-18T11:31:01.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0489999999999993e-06 2024-09-18T11:31:01.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303914+0000", 2024-09-18T11:31:01.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1697999999999999e-05 2024-09-18T11:31:01.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303938+0000", 2024-09-18T11:31:01.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4369e-05 2024-09-18T11:31:01.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.307122+0000", 2024-09-18T11:31:01.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031841780000000002 2024-09-18T11:31:01.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.307176+0000", 2024-09-18T11:31:01.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3400999999999999e-05 2024-09-18T11:31:01.426 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.426 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.426 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.426 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1662 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'73, mlcod=51'69)", 2024-09-18T11:31:01.426 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:01.426 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 24.024037342, 2024-09-18T11:31:01.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059408539999999998, 2024-09-18T11:31:01.426 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.426 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.427 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:01.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:01.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:01.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361914+0000", 2024-09-18T11:31:01.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036569 2024-09-18T11:31:01.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361926+0000", 2024-09-18T11:31:01.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1960999999999999e-05 2024-09-18T11:31:01.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361937+0000", 2024-09-18T11:31:01.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0876999999999999e-05 2024-09-18T11:31:01.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361986+0000", 2024-09-18T11:31:01.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8652000000000001e-05 2024-09-18T11:31:01.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.362012+0000", 2024-09-18T11:31:01.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5854999999999999e-05 2024-09-18T11:31:01.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367460+0000", 2024-09-18T11:31:01.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054482100000000002 2024-09-18T11:31:01.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367490+0000", 2024-09-18T11:31:01.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9909000000000001e-05 2024-09-18T11:31:01.430 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.430 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.430 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.431 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1663 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'74, mlcod=51'69)", 2024-09-18T11:31:01.431 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364521+0000", 2024-09-18T11:31:01.431 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 24.021065344, 2024-09-18T11:31:01.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048541399999999998, 2024-09-18T11:31:01.431 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.431 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.431 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364521+0000", 2024-09-18T11:31:01.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364521+0000", 2024-09-18T11:31:01.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364520+0000", 2024-09-18T11:31:01.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364616+0000", 2024-09-18T11:31:01.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5569999999999995e-05 2024-09-18T11:31:01.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364625+0000", 2024-09-18T11:31:01.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3379999999999994e-06 2024-09-18T11:31:01.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364629+0000", 2024-09-18T11:31:01.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2490000000000003e-06 2024-09-18T11:31:01.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364664+0000", 2024-09-18T11:31:01.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5132999999999997e-05 2024-09-18T11:31:01.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364687+0000", 2024-09-18T11:31:01.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2691999999999999e-05 2024-09-18T11:31:01.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369349+0000", 2024-09-18T11:31:01.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046622950000000003 2024-09-18T11:31:01.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369375+0000", 2024-09-18T11:31:01.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5534e-05 2024-09-18T11:31:01.435 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.435 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.435 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.435 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1664 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'75, mlcod=51'69)", 2024-09-18T11:31:01.435 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:31:01.435 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 24.018698834999999, 2024-09-18T11:31:01.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051832229999999998, 2024-09-18T11:31:01.435 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.435 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.435 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:31:01.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:31:01.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:31:01.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366894+0000", 2024-09-18T11:31:01.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2219999999999996e-06 2024-09-18T11:31:01.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366906+0000", 2024-09-18T11:31:01.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2278e-05 2024-09-18T11:31:01.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366912+0000", 2024-09-18T11:31:01.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2430000000000002e-06 2024-09-18T11:31:01.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366952+0000", 2024-09-18T11:31:01.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9428999999999998e-05 2024-09-18T11:31:01.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366972+0000", 2024-09-18T11:31:01.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0554999999999999e-05 2024-09-18T11:31:01.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372045+0000", 2024-09-18T11:31:01.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050727319999999999 2024-09-18T11:31:01.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372071+0000", 2024-09-18T11:31:01.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5477e-05 2024-09-18T11:31:01.439 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.439 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.439 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.440 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:269 4.7 e58/22 4:e73ba68f:::smithi11939506-1:13 v 58'72, mlcod=57'69)", 2024-09-18T11:31:01.440 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:01.440 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.120205445, 2024-09-18T11:31:01.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065512569999999996, 2024-09-18T11:31:01.440 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.440 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.440 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:01.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:01.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265380+0000", 2024-09-18T11:31:01.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265388+0000", 2024-09-18T11:31:01.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.174e-06 2024-09-18T11:31:01.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265398+0000", 2024-09-18T11:31:01.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2599999999999994e-06 2024-09-18T11:31:01.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265402+0000", 2024-09-18T11:31:01.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0289999999999999e-06 2024-09-18T11:31:01.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265434+0000", 2024-09-18T11:31:01.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2511000000000001e-05 2024-09-18T11:31:01.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265447+0000", 2024-09-18T11:31:01.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2947e-05 2024-09-18T11:31:01.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271892+0000", 2024-09-18T11:31:01.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064445930000000002 2024-09-18T11:31:01.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271932+0000", 2024-09-18T11:31:01.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0303999999999999e-05 2024-09-18T11:31:01.444 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.444 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.444 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.444 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1788 4.8 4:1898b15d:::smithi11939506-25:head [write 453822~797061] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:01.444 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:01.444 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.469772861999999, 2024-09-18T11:31:01.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094305159999999999, 2024-09-18T11:31:01.444 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.444 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.444 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.445 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.445 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.445 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1788 2024-09-18T11:31:01.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.445 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:01.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:01.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915811+0000", 2024-09-18T11:31:01.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:01.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916608+0000", 2024-09-18T11:31:01.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079635000000000003 2024-09-18T11:31:01.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916624+0000", 2024-09-18T11:31:01.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6735000000000002e-05 2024-09-18T11:31:01.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916630+0000", 2024-09-18T11:31:01.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5709999999999997e-06 2024-09-18T11:31:01.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916674+0000", 2024-09-18T11:31:01.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4540999999999999e-05 2024-09-18T11:31:01.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916751+0000", 2024-09-18T11:31:01.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6695999999999999e-05 2024-09-18T11:31:01.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:01.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916872+0000", 2024-09-18T11:31:01.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120648 2024-09-18T11:31:01.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:01.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.919076+0000", 2024-09-18T11:31:01.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022046829999999998 2024-09-18T11:31:01.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:01.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925151+0000", 2024-09-18T11:31:01.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060740239999999999 2024-09-18T11:31:01.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925198+0000", 2024-09-18T11:31:01.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7896999999999999e-05 2024-09-18T11:31:01.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925244+0000", 2024-09-18T11:31:01.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5398999999999997e-05 2024-09-18T11:31:01.450 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.450 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.450 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.450 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1789 4.8 4:1898b15d:::smithi11939506-25:head [write 1760371~653366] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:01.450 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.921860+0000", 2024-09-18T11:31:01.450 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.463726177000002, 2024-09-18T11:31:01.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055904329999999997, 2024-09-18T11:31:01.450 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.450 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.450 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.451 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.451 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.451 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1789 2024-09-18T11:31:01.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.451 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.921860+0000", 2024-09-18T11:31:01.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.921860+0000", 2024-09-18T11:31:01.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.921859+0000", 2024-09-18T11:31:01.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:01.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922436+0000", 2024-09-18T11:31:01.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00057708300000000004 2024-09-18T11:31:01.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922450+0000", 2024-09-18T11:31:01.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4575e-05 2024-09-18T11:31:01.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922458+0000", 2024-09-18T11:31:01.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:31:01.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922508+0000", 2024-09-18T11:31:01.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0102999999999998e-05 2024-09-18T11:31:01.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922607+0000", 2024-09-18T11:31:01.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8813000000000004e-05 2024-09-18T11:31:01.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:01.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922814+0000", 2024-09-18T11:31:01.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000207488 2024-09-18T11:31:01.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:01.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925504+0000", 2024-09-18T11:31:01.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026899200000000002 2024-09-18T11:31:01.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:01.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927350+0000", 2024-09-18T11:31:01.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00184575 2024-09-18T11:31:01.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927404+0000", 2024-09-18T11:31:01.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4147000000000002e-05 2024-09-18T11:31:01.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927450+0000", 2024-09-18T11:31:01.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6480000000000002e-05 2024-09-18T11:31:01.456 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.456 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.456 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.456 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1790 4.8 4:1898b15d:::smithi11939506-25:head [setxattr _header (57) in=7b,truncate 2453822] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:01.456 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.922467+0000", 2024-09-18T11:31:01.456 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.463118756, 2024-09-18T11:31:01.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0052363219999999999, 2024-09-18T11:31:01.456 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.457 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.457 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.457 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.457 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.457 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1790 2024-09-18T11:31:01.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.457 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922467+0000", 2024-09-18T11:31:01.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922467+0000", 2024-09-18T11:31:01.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922467+0000", 2024-09-18T11:31:01.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922470+0000", 2024-09-18T11:31:01.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2459999999999999e-06 2024-09-18T11:31:01.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922474+0000", 2024-09-18T11:31:01.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:31:01.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922476+0000", 2024-09-18T11:31:01.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3410000000000001e-06 2024-09-18T11:31:01.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.924488+0000", 2024-09-18T11:31:01.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002012425 2024-09-18T11:31:01.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.924623+0000", 2024-09-18T11:31:01.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013435699999999999 2024-09-18T11:31:01.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:01.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.924870+0000", 2024-09-18T11:31:01.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00024748199999999999 2024-09-18T11:31:01.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:01.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927574+0000", 2024-09-18T11:31:01.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027032800000000002 2024-09-18T11:31:01.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:01.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927628+0000", 2024-09-18T11:31:01.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3909000000000002e-05 2024-09-18T11:31:01.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927661+0000", 2024-09-18T11:31:01.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3467999999999999e-05 2024-09-18T11:31:01.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927704+0000", 2024-09-18T11:31:01.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2749e-05 2024-09-18T11:31:01.462 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.462 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.462 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.462 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1791 4.8 4:1898b15d:::smithi11939506-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:01.462 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.922500+0000", 2024-09-18T11:31:01.462 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.463086085, 2024-09-18T11:31:01.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005360259, 2024-09-18T11:31:01.462 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.462 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:01.462 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.463 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.463 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.463 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1791 2024-09-18T11:31:01.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.463 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922500+0000", 2024-09-18T11:31:01.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922500+0000", 2024-09-18T11:31:01.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922500+0000", 2024-09-18T11:31:01.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922503+0000", 2024-09-18T11:31:01.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7009999999999998e-06 2024-09-18T11:31:01.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922509+0000", 2024-09-18T11:31:01.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1710000000000001e-06 2024-09-18T11:31:01.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922511+0000", 2024-09-18T11:31:01.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:31:01.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925283+0000", 2024-09-18T11:31:01.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027722659999999998 2024-09-18T11:31:01.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:01.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925376+0000", 2024-09-18T11:31:01.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3018999999999998e-05 2024-09-18T11:31:01.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927736+0000", 2024-09-18T11:31:01.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023600990000000001 2024-09-18T11:31:01.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927798+0000", 2024-09-18T11:31:01.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1197000000000004e-05 2024-09-18T11:31:01.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927860+0000", 2024-09-18T11:31:01.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2725999999999995e-05 2024-09-18T11:31:01.467 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.467 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.467 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.467 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1804 4.1 e60/22 4:854aa293:::smithi11939506-26:head v 60'81, mlcod=58'77)", 2024-09-18T11:31:01.467 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.928460+0000", 2024-09-18T11:31:01.467 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.457126447, 2024-09-18T11:31:01.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0044520979999999998, 2024-09-18T11:31:01.468 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.468 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.468 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.928460+0000", 2024-09-18T11:31:01.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.928460+0000", 2024-09-18T11:31:01.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.928459+0000", 2024-09-18T11:31:01.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.928571+0000", 2024-09-18T11:31:01.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011205300000000001 2024-09-18T11:31:01.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.928579+0000", 2024-09-18T11:31:01.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1019999999999997e-06 2024-09-18T11:31:01.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.928583+0000", 2024-09-18T11:31:01.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1930000000000003e-06 2024-09-18T11:31:01.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.928621+0000", 2024-09-18T11:31:01.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7954999999999998e-05 2024-09-18T11:31:01.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.928636+0000", 2024-09-18T11:31:01.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4994000000000001e-05 2024-09-18T11:31:01.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.932871+0000", 2024-09-18T11:31:01.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042342459999999997 2024-09-18T11:31:01.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.932912+0000", 2024-09-18T11:31:01.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1195000000000001e-05 2024-09-18T11:31:01.471 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.471 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.472 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.472 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1934 4.2 e65/22 4:425afe9c:::smithi11939506-29:head v 65'44, mlcod=60'40)", 2024-09-18T11:31:01.472 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:01.472 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.085594091999999, 2024-09-18T11:31:01.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064585119999999996, 2024-09-18T11:31:01.472 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.472 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.472 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:01.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:01.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299990+0000", 2024-09-18T11:31:01.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:01.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300789+0000", 2024-09-18T11:31:01.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079851499999999997 2024-09-18T11:31:01.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300806+0000", 2024-09-18T11:31:01.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7096999999999999e-05 2024-09-18T11:31:01.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300813+0000", 2024-09-18T11:31:01.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5079999999999999e-06 2024-09-18T11:31:01.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300870+0000", 2024-09-18T11:31:01.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7178e-05 2024-09-18T11:31:01.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300897+0000", 2024-09-18T11:31:01.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6284000000000001e-05 2024-09-18T11:31:01.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306410+0000", 2024-09-18T11:31:01.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055135210000000004 2024-09-18T11:31:01.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306451+0000", 2024-09-18T11:31:01.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0379000000000001e-05 2024-09-18T11:31:01.476 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.476 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.476 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.476 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2075 3.2 e73/21 3:5e6b2dd0:::abf5d1c1c6ff48a895de7f094078ad489e0511876df30b6bd8853cd5ff4a6267:head v 73'120, mlcod=73'118)", 2024-09-18T11:31:01.476 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:55.323181+0000", 2024-09-18T11:31:01.477 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 6.0624047450000003, 2024-09-18T11:31:01.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043812620000000004, 2024-09-18T11:31:01.477 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.477 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.477 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.323181+0000", 2024-09-18T11:31:01.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.323181+0000", 2024-09-18T11:31:01.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.323180+0000", 2024-09-18T11:31:01.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:01.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.325078+0000", 2024-09-18T11:31:01.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018978739999999999 2024-09-18T11:31:01.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.325098+0000", 2024-09-18T11:31:01.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9661e-05 2024-09-18T11:31:01.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.325122+0000", 2024-09-18T11:31:01.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4335e-05 2024-09-18T11:31:01.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.325161+0000", 2024-09-18T11:31:01.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8658000000000002e-05 2024-09-18T11:31:01.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.325177+0000", 2024-09-18T11:31:01.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6398000000000001e-05 2024-09-18T11:31:01.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.327533+0000", 2024-09-18T11:31:01.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023562269999999998 2024-09-18T11:31:01.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.327563+0000", 2024-09-18T11:31:01.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9224999999999999e-05 2024-09-18T11:31:01.481 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.481 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.481 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.481 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2111 4.3 4:ca2b2309:::smithi11939506-44:head [append 0~0] snapc 1e=[1e,1c,1b,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-18T11:31:01.481 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:55.377984+0000", 2024-09-18T11:31:01.481 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 6.0076019279999997, 2024-09-18T11:31:01.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043350790000000004, 2024-09-18T11:31:01.481 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.481 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.481 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.482 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.482 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.482 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2111 2024-09-18T11:31:01.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.482 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.377984+0000", 2024-09-18T11:31:01.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.377984+0000", 2024-09-18T11:31:01.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.377984+0000", 2024-09-18T11:31:01.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.377985+0000", 2024-09-18T11:31:01.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.491e-06 2024-09-18T11:31:01.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.377995+0000", 2024-09-18T11:31:01.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1810000000000007e-06 2024-09-18T11:31:01.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.377999+0000", 2024-09-18T11:31:01.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2459999999999997e-06 2024-09-18T11:31:01.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.381565+0000", 2024-09-18T11:31:01.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035661629999999998 2024-09-18T11:31:01.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.381611+0000", 2024-09-18T11:31:01.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5691e-05 2024-09-18T11:31:01.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:01.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.381694+0000", 2024-09-18T11:31:01.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2976e-05 2024-09-18T11:31:01.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:01.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.382244+0000", 2024-09-18T11:31:01.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00055042999999999997 2024-09-18T11:31:01.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:01.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.382272+0000", 2024-09-18T11:31:01.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.756e-05 2024-09-18T11:31:01.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.382298+0000", 2024-09-18T11:31:01.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.671e-05 2024-09-18T11:31:01.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.382319+0000", 2024-09-18T11:31:01.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0888999999999999e-05 2024-09-18T11:31:01.487 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.487 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.487 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.487 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2119 4.7 e73/22 4:e0d95e10:::smithi11939506-22:head v 73'89, mlcod=73'86)", 2024-09-18T11:31:01.487 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:55.701097+0000", 2024-09-18T11:31:01.487 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.6844894459999997, 2024-09-18T11:31:01.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056939520000000004, 2024-09-18T11:31:01.487 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.488 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.488 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701097+0000", 2024-09-18T11:31:01.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701097+0000", 2024-09-18T11:31:01.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701095+0000", 2024-09-18T11:31:01.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:01.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701919+0000", 2024-09-18T11:31:01.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000824181 2024-09-18T11:31:01.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701938+0000", 2024-09-18T11:31:01.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8614000000000002e-05 2024-09-18T11:31:01.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701947+0000", 2024-09-18T11:31:01.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0699999999999996e-06 2024-09-18T11:31:01.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701992+0000", 2024-09-18T11:31:01.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5614000000000001e-05 2024-09-18T11:31:01.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.702012+0000", 2024-09-18T11:31:01.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9639999999999998e-05 2024-09-18T11:31:01.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.706755+0000", 2024-09-18T11:31:01.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004742708 2024-09-18T11:31:01.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.706791+0000", 2024-09-18T11:31:01.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6106999999999998e-05 2024-09-18T11:31:01.491 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.491 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.491 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.491 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.491 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.492 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:01.493 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:31:01.494 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:01.491+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:01.513 INFO:tasks.rados.rados.0.smithi119.stdout:1146: finishing rollback tid 0 to smithi11939506-38 2024-09-18T11:31:01.513 INFO:tasks.rados.rados.0.smithi119.stdout:1146: finishing rollback tid 1 to smithi11939506-38 2024-09-18T11:31:01.513 INFO:tasks.rados.rados.0.smithi119.stdout:1146: finishing rollback tid 2 to smithi11939506-38 2024-09-18T11:31:01.513 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 96 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-18T11:31:01.513 INFO:tasks.rados.rados.0.smithi119.stdout:1144: done (6 left) 2024-09-18T11:31:01.513 INFO:tasks.rados.rados.0.smithi119.stdout:1145: done (5 left) 2024-09-18T11:31:01.513 INFO:tasks.rados.rados.0.smithi119.stdout:1146: done (4 left) 2024-09-18T11:31:01.513 INFO:tasks.rados.rados.0.smithi119.stdout:1151: done (3 left) 2024-09-18T11:31:01.513 INFO:tasks.rados.rados.0.smithi119.stdout:1152: write oid 13 current snap is 23 2024-09-18T11:31:01.513 INFO:tasks.rados.rados.0.smithi119.stdout:1152: seq_num 204 ranges {628628=775974,1937856=527061,3252761=175867} 2024-09-18T11:31:01.520 INFO:tasks.rados.rados.0.smithi119.stdout:1152: writing smithi11939506-13 from 628628 to 1404602 tid 1 2024-09-18T11:31:01.525 INFO:tasks.rados.rados.0.smithi119.stdout:1152: writing smithi11939506-13 from 1937856 to 2464917 tid 2 2024-09-18T11:31:01.531 INFO:tasks.rados.rados.0.smithi119.stdout:1152: writing smithi11939506-13 from 3252761 to 3428628 tid 3 2024-09-18T11:31:01.531 INFO:tasks.rados.rados.0.smithi119.stdout:1153: snap_remove snap 15 2024-09-18T11:31:01.594 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:01.592+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:01.631 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:01.631 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:01.631 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:01.631 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:01.635 DEBUG:teuthology.orchestra.run.smithi119:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op apply-layout-settings --pool unique_pool_0 2024-09-18T11:31:01.639 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:31:01.719 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:01.717+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:01.777 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:01.777 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:01.777 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:01.777 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:01.777 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:01.785 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:31:01.825 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:31:01.828 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:01.827+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:01.884 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:01.885 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:01.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:01.885 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:01.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.885 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:01.885 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:01.885 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 50.765885578000002, 2024-09-18T11:31:01.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:31:01.885 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.885 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:01.885 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.885 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.886 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.886 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:31:01.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.886 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:01.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:01.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:31:01.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:01.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:31:01.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:31:01.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:31:01.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:31:01.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:31:01.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:31:01.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:31:01.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:31:01.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:01.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:31:01.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:31:01.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:31:01.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:31:01.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:31:01.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:31:01.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:31:01.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:31:01.890 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.890 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.890 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.890 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:31:01.890 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:01.890 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 41.212860177000003, 2024-09-18T11:31:01.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:31:01.890 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.890 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:01.890 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.891 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.891 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.891 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:31:01.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.891 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:01.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:01.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:31:01.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:01.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:31:01.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:31:01.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:31:01.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:31:01.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:31:01.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:31:01.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:31:01.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:31:01.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:31:01.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:31:01.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:31:01.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:31:01.894 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.894 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.894 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.894 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1688 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 19=[19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-18T11:31:01.894 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.349705583999999, 2024-09-18T11:31:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071575789999999999, 2024-09-18T11:31:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1688 2024-09-18T11:31:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:01.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:01.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:01.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532500+0000", 2024-09-18T11:31:01.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5260000000000001e-06 2024-09-18T11:31:01.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532503+0000", 2024-09-18T11:31:01.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:31:01.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532505+0000", 2024-09-18T11:31:01.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.889e-06 2024-09-18T11:31:01.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.534722+0000", 2024-09-18T11:31:01.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002217119 2024-09-18T11:31:01.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.534827+0000", 2024-09-18T11:31:01.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0001049 2024-09-18T11:31:01.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:01.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.535007+0000", 2024-09-18T11:31:01.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017965599999999999 2024-09-18T11:31:01.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:01.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539463+0000", 2024-09-18T11:31:01.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004455713 2024-09-18T11:31:01.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:01.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539607+0000", 2024-09-18T11:31:01.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014440300000000001 2024-09-18T11:31:01.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539634+0000", 2024-09-18T11:31:01.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7398000000000001e-05 2024-09-18T11:31:01.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539657+0000", 2024-09-18T11:31:01.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2228000000000001e-05 2024-09-18T11:31:01.900 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.900 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.900 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.900 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1759 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2319366,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:01.900 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:31:01.900 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.588245374, 2024-09-18T11:31:01.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064323039999999998, 2024-09-18T11:31:01.901 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.901 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:01.901 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.901 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.901 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.901 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1759 2024-09-18T11:31:01.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.901 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:31:01.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:31:01.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:31:01.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293963+0000", 2024-09-18T11:31:01.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0810000000000004e-06 2024-09-18T11:31:01.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293970+0000", 2024-09-18T11:31:01.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5109999999999997e-06 2024-09-18T11:31:01.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293975+0000", 2024-09-18T11:31:01.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.33e-06 2024-09-18T11:31:01.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294078+0000", 2024-09-18T11:31:01.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102871 2024-09-18T11:31:01.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:01.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294084+0000", 2024-09-18T11:31:01.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5470000000000002e-06 2024-09-18T11:31:01.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.299550+0000", 2024-09-18T11:31:01.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054660799999999999 2024-09-18T11:31:01.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.299608+0000", 2024-09-18T11:31:01.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8307000000000003e-05 2024-09-18T11:31:01.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300392+0000", 2024-09-18T11:31:01.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078306999999999997 2024-09-18T11:31:01.905 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.905 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.905 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.906 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1760 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2319366,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:01.906 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:31:01.906 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.588211034, 2024-09-18T11:31:01.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070132529999999997, 2024-09-18T11:31:01.906 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.906 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:01.906 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.906 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.906 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.906 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1760 2024-09-18T11:31:01.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.907 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:31:01.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:31:01.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293993+0000", 2024-09-18T11:31:01.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293997+0000", 2024-09-18T11:31:01.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4999999999999999e-06 2024-09-18T11:31:01.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294004+0000", 2024-09-18T11:31:01.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3950000000000003e-06 2024-09-18T11:31:01.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294009+0000", 2024-09-18T11:31:01.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4989999999999996e-06 2024-09-18T11:31:01.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294103+0000", 2024-09-18T11:31:01.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4817999999999999e-05 2024-09-18T11:31:01.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:01.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294110+0000", 2024-09-18T11:31:01.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7240000000000003e-06 2024-09-18T11:31:01.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300406+0000", 2024-09-18T11:31:01.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062964470000000002 2024-09-18T11:31:01.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300457+0000", 2024-09-18T11:31:01.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0915000000000003e-05 2024-09-18T11:31:01.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.301007+0000", 2024-09-18T11:31:01.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00054942299999999995 2024-09-18T11:31:01.911 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.911 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.911 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.911 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1775 4.a 4:588debc2:::smithi11939506-18:head [write 1527054~766050] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:01.911 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:31:01.911 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.985858293, 2024-09-18T11:31:01.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070939050000000002, 2024-09-18T11:31:01.912 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.912 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.912 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.912 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.912 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.912 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1775 2024-09-18T11:31:01.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.912 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:31:01.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:31:01.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896345+0000", 2024-09-18T11:31:01.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:01.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896993+0000", 2024-09-18T11:31:01.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064769999999999997 2024-09-18T11:31:01.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897011+0000", 2024-09-18T11:31:01.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8355999999999998e-05 2024-09-18T11:31:01.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897016+0000", 2024-09-18T11:31:01.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3890000000000004e-06 2024-09-18T11:31:01.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897054+0000", 2024-09-18T11:31:01.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7963000000000002e-05 2024-09-18T11:31:01.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897123+0000", 2024-09-18T11:31:01.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8128999999999998e-05 2024-09-18T11:31:01.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:01.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897224+0000", 2024-09-18T11:31:01.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010168800000000001 2024-09-18T11:31:01.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:01.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899515+0000", 2024-09-18T11:31:01.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022910280000000001 2024-09-18T11:31:01.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:01.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903333+0000", 2024-09-18T11:31:01.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003817459 2024-09-18T11:31:01.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903388+0000", 2024-09-18T11:31:01.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5651e-05 2024-09-18T11:31:01.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903440+0000", 2024-09-18T11:31:01.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1836999999999999e-05 2024-09-18T11:31:01.917 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.917 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.917 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.918 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1776 4.a 4:588debc2:::smithi11939506-18:head [setxattr _header (57) in=7b,truncate 2420208] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:01.918 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:31:01.918 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.985180481, 2024-09-18T11:31:01.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065840400000000002, 2024-09-18T11:31:01.918 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.918 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.918 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.918 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.918 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.918 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1776 2024-09-18T11:31:01.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.918 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:31:01.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:31:01.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:31:01.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897026+0000", 2024-09-18T11:31:01.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9579999999999999e-06 2024-09-18T11:31:01.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897030+0000", 2024-09-18T11:31:01.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5370000000000002e-06 2024-09-18T11:31:01.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897032+0000", 2024-09-18T11:31:01.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.17e-06 2024-09-18T11:31:01.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898415+0000", 2024-09-18T11:31:01.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013823189999999999 2024-09-18T11:31:01.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898527+0000", 2024-09-18T11:31:01.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011236800000000001 2024-09-18T11:31:01.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:01.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898714+0000", 2024-09-18T11:31:01.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018703599999999999 2024-09-18T11:31:01.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:01.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899683+0000", 2024-09-18T11:31:01.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00096944700000000002 2024-09-18T11:31:01.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:01.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903475+0000", 2024-09-18T11:31:01.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037920129999999999 2024-09-18T11:31:01.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903535+0000", 2024-09-18T11:31:01.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9843999999999998e-05 2024-09-18T11:31:01.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903608+0000", 2024-09-18T11:31:01.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2946000000000003e-05 2024-09-18T11:31:01.923 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.923 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.924 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.924 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1777 4.a 4:588debc2:::smithi11939506-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:01.924 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:31:01.924 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.985153840999999, 2024-09-18T11:31:01.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0069010809999999999, 2024-09-18T11:31:01.924 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.924 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:01.924 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.924 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.924 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.925 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1777 2024-09-18T11:31:01.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.925 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:31:01.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:31:01.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897050+0000", 2024-09-18T11:31:01.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897052+0000", 2024-09-18T11:31:01.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:31:01.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897057+0000", 2024-09-18T11:31:01.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9470000000000002e-06 2024-09-18T11:31:01.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897060+0000", 2024-09-18T11:31:01.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3410000000000001e-06 2024-09-18T11:31:01.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899417+0000", 2024-09-18T11:31:01.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023571170000000002 2024-09-18T11:31:01.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:01.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899501+0000", 2024-09-18T11:31:01.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4160999999999996e-05 2024-09-18T11:31:01.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903661+0000", 2024-09-18T11:31:01.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041599510000000003 2024-09-18T11:31:01.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903790+0000", 2024-09-18T11:31:01.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012941499999999999 2024-09-18T11:31:01.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903952+0000", 2024-09-18T11:31:01.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016146800000000001 2024-09-18T11:31:01.929 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.929 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.929 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.929 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1895 4.5 4:a087a566:::smithi11939506-46:head [read 0~2756315,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:01.929 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:01.929 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.573898335999999, 2024-09-18T11:31:01.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007540377, 2024-09-18T11:31:01.930 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.930 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:01.930 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.930 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.930 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.930 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1895 2024-09-18T11:31:01.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.930 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:01.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:01.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:01.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308310+0000", 2024-09-18T11:31:01.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0559999999999998e-06 2024-09-18T11:31:01.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308317+0000", 2024-09-18T11:31:01.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7209999999999997e-06 2024-09-18T11:31:01.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308320+0000", 2024-09-18T11:31:01.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.721e-06 2024-09-18T11:31:01.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308447+0000", 2024-09-18T11:31:01.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012645199999999999 2024-09-18T11:31:01.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:01.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308451+0000", 2024-09-18T11:31:01.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6800000000000001e-06 2024-09-18T11:31:01.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.312462+0000", 2024-09-18T11:31:01.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040102139999999998 2024-09-18T11:31:01.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.312585+0000", 2024-09-18T11:31:01.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012388899999999999 2024-09-18T11:31:01.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.315847+0000", 2024-09-18T11:31:01.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003261197 2024-09-18T11:31:01.934 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.934 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.935 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.935 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1924 4.5 4:af4ba75a:::smithi11939506-37:head [write 1829449~675380] snapc 1e=[1e,1c,1b,19,18,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:01.935 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.362228+0000", 2024-09-18T11:31:01.935 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.519976636999999, 2024-09-18T11:31:01.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066182139999999999, 2024-09-18T11:31:01.935 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.935 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.935 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.935 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.935 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.936 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1924 2024-09-18T11:31:01.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.936 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362228+0000", 2024-09-18T11:31:01.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362228+0000", 2024-09-18T11:31:01.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362227+0000", 2024-09-18T11:31:01.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:01.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362749+0000", 2024-09-18T11:31:01.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00052262399999999996 2024-09-18T11:31:01.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362762+0000", 2024-09-18T11:31:01.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2821e-05 2024-09-18T11:31:01.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362767+0000", 2024-09-18T11:31:01.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5349999999999998e-06 2024-09-18T11:31:01.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362806+0000", 2024-09-18T11:31:01.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9029999999999997e-05 2024-09-18T11:31:01.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362863+0000", 2024-09-18T11:31:01.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7584999999999998e-05 2024-09-18T11:31:01.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:01.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362949+0000", 2024-09-18T11:31:01.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5655000000000006e-05 2024-09-18T11:31:01.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:01.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364314+0000", 2024-09-18T11:31:01.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001364961 2024-09-18T11:31:01.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:01.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.368771+0000", 2024-09-18T11:31:01.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004457244 2024-09-18T11:31:01.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.368809+0000", 2024-09-18T11:31:01.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8368999999999998e-05 2024-09-18T11:31:01.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.368846+0000", 2024-09-18T11:31:01.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6718000000000003e-05 2024-09-18T11:31:01.941 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.941 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.941 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.941 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1925 4.5 4:af4ba75a:::smithi11939506-37:head [setxattr _header (57) in=7b,truncate 2655506] snapc 1e=[1e,1c,1b,19,18,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:01.942 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:31:01.942 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.519431485, 2024-09-18T11:31:01.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064745779999999999, 2024-09-18T11:31:01.942 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.942 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.942 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.942 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.942 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.942 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1925 2024-09-18T11:31:01.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.942 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:31:01.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:31:01.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:31:01.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362776+0000", 2024-09-18T11:31:01.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.208e-06 2024-09-18T11:31:01.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362780+0000", 2024-09-18T11:31:01.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.771e-06 2024-09-18T11:31:01.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362782+0000", 2024-09-18T11:31:01.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.277e-06 2024-09-18T11:31:01.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.363737+0000", 2024-09-18T11:31:01.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00095483999999999999 2024-09-18T11:31:01.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.363832+0000", 2024-09-18T11:31:01.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5594999999999995e-05 2024-09-18T11:31:01.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:01.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.363940+0000", 2024-09-18T11:31:01.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010738000000000001 2024-09-18T11:31:01.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:01.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364577+0000", 2024-09-18T11:31:01.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00063679600000000002 2024-09-18T11:31:01.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:01.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369126+0000", 2024-09-18T11:31:01.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0045496440000000003 2024-09-18T11:31:01.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369177+0000", 2024-09-18T11:31:01.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0958000000000002e-05 2024-09-18T11:31:01.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369248+0000", 2024-09-18T11:31:01.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0599000000000006e-05 2024-09-18T11:31:01.947 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.947 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.947 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.948 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1926 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:01.948 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:01.948 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.519407229, 2024-09-18T11:31:01.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066190219999999996, 2024-09-18T11:31:01.948 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.948 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:01.948 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.948 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.948 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.948 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1926 2024-09-18T11:31:01.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.948 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:01.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:01.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:01.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362799+0000", 2024-09-18T11:31:01.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.029e-06 2024-09-18T11:31:01.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362804+0000", 2024-09-18T11:31:01.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3989999999999997e-06 2024-09-18T11:31:01.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362806+0000", 2024-09-18T11:31:01.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3020000000000002e-06 2024-09-18T11:31:01.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364105+0000", 2024-09-18T11:31:01.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001299478 2024-09-18T11:31:01.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:01.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364153+0000", 2024-09-18T11:31:01.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8059000000000002e-05 2024-09-18T11:31:01.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369273+0000", 2024-09-18T11:31:01.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051199540000000003 2024-09-18T11:31:01.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369343+0000", 2024-09-18T11:31:01.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0191999999999995e-05 2024-09-18T11:31:01.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369416+0000", 2024-09-18T11:31:01.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2925999999999999e-05 2024-09-18T11:31:01.955 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.955 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.955 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.955 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1965 4.a 4:5975a5ec:::smithi11939506-36:head [rollback 1e] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:01.955 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373840+0000", 2024-09-18T11:31:01.956 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 10.508364813, 2024-09-18T11:31:01.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062698229999999999, 2024-09-18T11:31:01.956 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.956 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.956 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.956 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.956 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.956 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1965 2024-09-18T11:31:01.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.956 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373840+0000", 2024-09-18T11:31:01.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373840+0000", 2024-09-18T11:31:01.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373839+0000", 2024-09-18T11:31:01.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373842+0000", 2024-09-18T11:31:01.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8770000000000002e-06 2024-09-18T11:31:01.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373849+0000", 2024-09-18T11:31:01.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3929999999999996e-06 2024-09-18T11:31:01.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373853+0000", 2024-09-18T11:31:01.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0099999999999997e-06 2024-09-18T11:31:01.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.375207+0000", 2024-09-18T11:31:01.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001354755 2024-09-18T11:31:01.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.375268+0000", 2024-09-18T11:31:01.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0983999999999998e-05 2024-09-18T11:31:01.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:01.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.375449+0000", 2024-09-18T11:31:01.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000181091 2024-09-18T11:31:01.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:01.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380058+0000", 2024-09-18T11:31:01.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046089390000000003 2024-09-18T11:31:01.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:01.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380065+0000", 2024-09-18T11:31:01.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4479999999999999e-06 2024-09-18T11:31:01.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380088+0000", 2024-09-18T11:31:01.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3337e-05 2024-09-18T11:31:01.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380110+0000", 2024-09-18T11:31:01.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.145e-05 2024-09-18T11:31:01.961 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.961 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.961 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.962 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1966 4.a 4:5975a5ec:::smithi11939506-36:head [append 0~0] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:01.962 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373876+0000", 2024-09-18T11:31:01.962 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 10.508328789, 2024-09-18T11:31:01.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063095080000000001, 2024-09-18T11:31:01.962 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.962 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.962 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.962 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.962 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.962 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1966 2024-09-18T11:31:01.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.962 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373876+0000", 2024-09-18T11:31:01.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373876+0000", 2024-09-18T11:31:01.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373875+0000", 2024-09-18T11:31:01.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373879+0000", 2024-09-18T11:31:01.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3859999999999999e-06 2024-09-18T11:31:01.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373885+0000", 2024-09-18T11:31:01.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6030000000000001e-06 2024-09-18T11:31:01.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373889+0000", 2024-09-18T11:31:01.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9620000000000004e-06 2024-09-18T11:31:01.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.376620+0000", 2024-09-18T11:31:01.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027307920000000001 2024-09-18T11:31:01.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.376672+0000", 2024-09-18T11:31:01.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2244999999999999e-05 2024-09-18T11:31:01.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:01.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.376838+0000", 2024-09-18T11:31:01.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000165716 2024-09-18T11:31:01.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:01.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380116+0000", 2024-09-18T11:31:01.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003278352 2024-09-18T11:31:01.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:01.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380145+0000", 2024-09-18T11:31:01.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8379999999999999e-05 2024-09-18T11:31:01.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380166+0000", 2024-09-18T11:31:01.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1118e-05 2024-09-18T11:31:01.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380185+0000", 2024-09-18T11:31:01.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9519999999999999e-05 2024-09-18T11:31:01.967 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.967 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.967 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.967 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1969 4.a 4:588debc2:::smithi11939506-18:18 [read 0~3949612,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:01.968 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:01.968 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 10.508218339999999, 2024-09-18T11:31:01.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075409190000000001, 2024-09-18T11:31:01.968 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.968 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:01.968 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.968 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.968 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.968 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1969 2024-09-18T11:31:01.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.968 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:01.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:01.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:01.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373990+0000", 2024-09-18T11:31:01.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1119999999999998e-06 2024-09-18T11:31:01.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373997+0000", 2024-09-18T11:31:01.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4200000000000001e-06 2024-09-18T11:31:01.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.374002+0000", 2024-09-18T11:31:01.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:31:01.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380200+0000", 2024-09-18T11:31:01.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006197302 2024-09-18T11:31:01.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380261+0000", 2024-09-18T11:31:01.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1389999999999993e-05 2024-09-18T11:31:01.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.381527+0000", 2024-09-18T11:31:01.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012660080000000001 2024-09-18T11:31:01.971 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.971 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.972 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.972 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2207 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1410296~761389] snapc 24=[24,1e,1c,1b,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T11:31:01.972 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:01.972 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.4597075139999998, 2024-09-18T11:31:01.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008410743, 2024-09-18T11:31:01.972 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.972 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.972 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.972 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.972 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.972 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2207 2024-09-18T11:31:01.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.973 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:01.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:01.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422496+0000", 2024-09-18T11:31:01.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:01.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423190+0000", 2024-09-18T11:31:01.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00069405499999999995 2024-09-18T11:31:01.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423204+0000", 2024-09-18T11:31:01.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:31:01.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423208+0000", 2024-09-18T11:31:01.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:31:01.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423293+0000", 2024-09-18T11:31:01.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4524999999999995e-05 2024-09-18T11:31:01.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423394+0000", 2024-09-18T11:31:01.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101395 2024-09-18T11:31:01.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:01.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423579+0000", 2024-09-18T11:31:01.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018461000000000001 2024-09-18T11:31:01.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:01.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.428574+0000", 2024-09-18T11:31:01.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049944960000000002 2024-09-18T11:31:01.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:01.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430809+0000", 2024-09-18T11:31:01.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022351820000000001 2024-09-18T11:31:01.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430862+0000", 2024-09-18T11:31:01.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3189999999999997e-05 2024-09-18T11:31:01.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430908+0000", 2024-09-18T11:31:01.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5973999999999997e-05 2024-09-18T11:31:01.977 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.978 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.978 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.978 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2331 4.a 4:552523b6:::smithi11939506-38:head [append 0~0] snapc 26=[26,25,24,1e,1c,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T11:31:01.978 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.194086+0000", 2024-09-18T11:31:01.978 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.68811854800000005, 2024-09-18T11:31:01.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062829319999999998, 2024-09-18T11:31:01.978 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.978 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:01.978 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.978 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.978 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.978 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2331 2024-09-18T11:31:01.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.979 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194086+0000", 2024-09-18T11:31:01.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194086+0000", 2024-09-18T11:31:01.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194086+0000", 2024-09-18T11:31:01.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194088+0000", 2024-09-18T11:31:01.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.367e-06 2024-09-18T11:31:01.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194094+0000", 2024-09-18T11:31:01.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8089999999999998e-06 2024-09-18T11:31:01.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194097+0000", 2024-09-18T11:31:01.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5920000000000001e-06 2024-09-18T11:31:01.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.196736+0000", 2024-09-18T11:31:01.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002638428 2024-09-18T11:31:01.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.196834+0000", 2024-09-18T11:31:01.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7907999999999998e-05 2024-09-18T11:31:01.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:01.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.197015+0000", 2024-09-18T11:31:01.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000180946 2024-09-18T11:31:01.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:01.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.199755+0000", 2024-09-18T11:31:01.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027399830000000001 2024-09-18T11:31:01.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:01.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.200306+0000", 2024-09-18T11:31:01.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00055154399999999995 2024-09-18T11:31:01.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:01.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.200341+0000", 2024-09-18T11:31:01.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4839999999999998e-05 2024-09-18T11:31:01.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.200369+0000", 2024-09-18T11:31:01.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7909999999999999e-05 2024-09-18T11:31:01.984 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.984 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.984 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.984 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2339 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:01.984 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:01.984 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.49540288500000002, 2024-09-18T11:31:01.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011809992, 2024-09-18T11:31:01.984 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.984 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:01.985 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.985 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.985 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.985 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2339 2024-09-18T11:31:01.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.985 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:01.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:01.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:31:01.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:01.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:31:01.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4710000000000001e-06 2024-09-18T11:31:01.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386824+0000", 2024-09-18T11:31:01.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6104e-05 2024-09-18T11:31:01.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386836+0000", 2024-09-18T11:31:01.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1827999999999999e-05 2024-09-18T11:31:01.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388390+0000", 2024-09-18T11:31:01.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015539499999999999 2024-09-18T11:31:01.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:01.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388443+0000", 2024-09-18T11:31:01.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3785000000000001e-05 2024-09-18T11:31:01.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398238+0000", 2024-09-18T11:31:01.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097947989999999999 2024-09-18T11:31:01.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398315+0000", 2024-09-18T11:31:01.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6909999999999994e-05 2024-09-18T11:31:01.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398612+0000", 2024-09-18T11:31:01.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029671999999999999 2024-09-18T11:31:01.989 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.989 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.989 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.989 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2340 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:01.989 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:01.989 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.495359573, 2024-09-18T11:31:01.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012086448, 2024-09-18T11:31:01.989 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.989 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:01.990 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.990 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.990 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.990 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2340 2024-09-18T11:31:01.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.990 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:01.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:01.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:01.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386848+0000", 2024-09-18T11:31:01.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3069999999999998e-06 2024-09-18T11:31:01.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386855+0000", 2024-09-18T11:31:01.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0979999999999996e-06 2024-09-18T11:31:01.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386859+0000", 2024-09-18T11:31:01.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9360000000000001e-06 2024-09-18T11:31:01.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388466+0000", 2024-09-18T11:31:01.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001607154 2024-09-18T11:31:01.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:01.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388507+0000", 2024-09-18T11:31:01.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1093000000000001e-05 2024-09-18T11:31:01.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398647+0000", 2024-09-18T11:31:01.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010139686 2024-09-18T11:31:01.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398718+0000", 2024-09-18T11:31:01.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1138999999999998e-05 2024-09-18T11:31:01.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398931+0000", 2024-09-18T11:31:01.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021356899999999999 2024-09-18T11:31:01.994 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.994 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.994 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.994 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2341 4.5 4:aa575018:::smithi11939506-11:head [read 0~2487436,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:01.994 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:01.994 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.49534135099999999, 2024-09-18T11:31:01.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013751648999999999, 2024-09-18T11:31:01.995 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:01.995 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:01.995 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:01.995 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:01.995 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:01.995 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2341 2024-09-18T11:31:01.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.995 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:01.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:01.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:01.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:01.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:01.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:01.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:01.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:01.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:01.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:01.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386866+0000", 2024-09-18T11:31:01.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4249999999999998e-06 2024-09-18T11:31:01.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:01.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386872+0000", 2024-09-18T11:31:01.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.003e-06 2024-09-18T11:31:01.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:01.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:31:01.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.507e-06 2024-09-18T11:31:01.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388520+0000", 2024-09-18T11:31:01.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001644146 2024-09-18T11:31:01.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:01.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388561+0000", 2024-09-18T11:31:01.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1486999999999998e-05 2024-09-18T11:31:01.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:01.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398963+0000", 2024-09-18T11:31:01.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010401353 2024-09-18T11:31:01.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:01.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.399046+0000", 2024-09-18T11:31:01.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3141999999999998e-05 2024-09-18T11:31:01.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:01.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:01.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:01.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.400615+0000", 2024-09-18T11:31:01.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015690960000000001 2024-09-18T11:31:01.999 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.999 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.999 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.999 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:01.999 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:01.999 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:02.000 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:02.000 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:02.001 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:02.001 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:02.002 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:31:02.002 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:01.999+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:02.002 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:02.003 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:02.003 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:02.003 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:02.004 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:02.004 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:02.004 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:02.018 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:02.018 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:02.034 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:02.034 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:02.037 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:02.038 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:02.101 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:02.099+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:02.139 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:02.139 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:02.147 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:31:02.202 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:02.201+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:02.245 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:02.246 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:02.254 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:31:02.303 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:02.301+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:02.352 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:02.353 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:02.361 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:31:02.399 INFO:tasks.rados.rados.0.smithi119.stdout:1153: done (4 left) 2024-09-18T11:31:02.399 INFO:tasks.rados.rados.0.smithi119.stdout:1154: write oid 18 current snap is 23 2024-09-18T11:31:02.399 INFO:tasks.rados.rados.0.smithi119.stdout:1154: seq_num 205 ranges {645435=451223,1879985=787072,3280184=165251} 2024-09-18T11:31:02.404 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:02.402+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:02.405 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T11:31:02.406 INFO:tasks.rados.rados.0.smithi119.stdout:1154: writing smithi11939506-18 from 645435 to 1096658 tid 1 2024-09-18T11:31:02.416 INFO:tasks.rados.rados.0.smithi119.stdout:1154: writing smithi11939506-18 from 1879985 to 2667057 tid 2 2024-09-18T11:31:02.421 INFO:tasks.rados.rados.0.smithi119.stdout:1154: writing smithi11939506-18 from 3280184 to 3445435 tid 3 2024-09-18T11:31:02.421 INFO:tasks.rados.rados.0.smithi119.stdout:1155: set_chunk oid 39 target oid 2024-09-18T11:31:02.421 INFO:tasks.rados.rados.0.smithi119.stdout:1155: set_chunk oid 39 offset: 0 length: 4000000 target oid smithi11939506- offset: 0 snap -1 2024-09-18T11:31:02.421 INFO:tasks.rados.rados.0.smithi119.stdout:1156: tier_evict oid 10 2024-09-18T11:31:02.421 INFO:tasks.rados.rados.0.smithi119.stdout:1156: tier_evict oid 10 snap -1 2024-09-18T11:31:02.421 INFO:tasks.rados.rados.0.smithi119.stdout:1149: expect (ObjNum 187 snap 20 seq_num 187) 2024-09-18T11:31:02.471 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:02.471 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:02.481 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:31:02.504 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:02.503+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:02.600 INFO:tasks.rados.rados.0.smithi119.stdout:1150: expect (ObjNum 148 snap 18 seq_num 148) 2024-09-18T11:31:02.605 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:02.604+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:02.620 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:02.620 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:02.628 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:31:02.683 INFO:tasks.rados.rados.0.smithi119.stdout:1152: finishing write tid 1 to smithi11939506-13 2024-09-18T11:31:02.683 INFO:tasks.rados.rados.0.smithi119.stdout:1152: finishing write tid 2 to smithi11939506-13 2024-09-18T11:31:02.683 INFO:tasks.rados.rados.0.smithi119.stdout:1152: finishing write tid 3 to smithi11939506-13 2024-09-18T11:31:02.683 INFO:tasks.rados.rados.0.smithi119.stdout:1152: finishing write tid 4 to smithi11939506-13 2024-09-18T11:31:02.683 INFO:tasks.rados.rados.0.smithi119.stdout:1152: finishing write tid 5 to smithi11939506-13 2024-09-18T11:31:02.684 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 13 v 201 (ObjNum 204 snap 23 seq_num 204) dirty exists 2024-09-18T11:31:02.684 INFO:tasks.rados.rados.0.smithi119.stdout:1152: left oid 13 (ObjNum 204 snap 23 seq_num 204) 2024-09-18T11:31:02.684 INFO:tasks.rados.rados.0.smithi119.stdout:1154: finishing write tid 1 to smithi11939506-18 2024-09-18T11:31:02.684 INFO:tasks.rados.rados.0.smithi119.stdout:1155: finishing set_chunk to oid 39 2024-09-18T11:31:02.684 INFO:tasks.rados.rados.0.smithi119.stdout:1155: got expected ENOENT (src dne) 2024-09-18T11:31:02.684 INFO:tasks.rados.rados.0.smithi119.stdout:1156: got (22) Invalid argument 2024-09-18T11:31:02.684 INFO:tasks.rados.rados.0.smithi119.stdout:1154: finishing write tid 2 to smithi11939506-18 2024-09-18T11:31:02.684 INFO:tasks.rados.rados.0.smithi119.stdout:1148: done (6 left) 2024-09-18T11:31:02.684 INFO:tasks.rados.rados.0.smithi119.stdout:1149: done (5 left) 2024-09-18T11:31:02.684 INFO:tasks.rados.rados.0.smithi119.stdout:1150: done (4 left) 2024-09-18T11:31:02.684 INFO:tasks.rados.rados.0.smithi119.stdout:1152: done (3 left) 2024-09-18T11:31:02.684 INFO:tasks.rados.rados.0.smithi119.stdout:1155: done (2 left) 2024-09-18T11:31:02.684 INFO:tasks.rados.rados.0.smithi119.stdout:1156: done (1 left) 2024-09-18T11:31:02.685 INFO:tasks.rados.rados.0.smithi119.stdout:1157: rollback oid 4 current snap is 23 2024-09-18T11:31:02.685 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 4 to 14 2024-09-18T11:31:02.685 INFO:tasks.rados.rados.0.smithi119.stdout:1154: finishing write tid 3 to smithi11939506-18 2024-09-18T11:31:02.685 INFO:tasks.rados.rados.0.smithi119.stdout:1154: finishing write tid 4 to smithi11939506-18 2024-09-18T11:31:02.685 INFO:tasks.rados.rados.0.smithi119.stdout:1154: finishing write tid 5 to smithi11939506-18 2024-09-18T11:31:02.685 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 113 (ObjNum 205 snap 23 seq_num 205) dirty exists 2024-09-18T11:31:02.685 INFO:tasks.rados.rados.0.smithi119.stdout:1154: left oid 18 (ObjNum 205 snap 23 seq_num 205) 2024-09-18T11:31:02.685 INFO:tasks.rados.rados.0.smithi119.stdout:1154: done (1 left) 2024-09-18T11:31:02.685 INFO:tasks.rados.rados.0.smithi119.stdout:1158: read oid 9 snap -1 2024-09-18T11:31:02.685 INFO:tasks.rados.rados.0.smithi119.stdout:1158: expect (ObjNum 199 snap 23 seq_num 199) 2024-09-18T11:31:02.685 INFO:tasks.rados.rados.0.smithi119.stdout:1159: read oid 6 snap -1 2024-09-18T11:31:02.685 INFO:tasks.rados.rados.0.smithi119.stdout:1159: expect (ObjNum 187 snap 20 seq_num 187) 2024-09-18T11:31:02.685 INFO:tasks.rados.rados.0.smithi119.stdout:1160: set_chunk oid 27 target oid 2024-09-18T11:31:02.696 INFO:tasks.rados.rados.0.smithi119.stdout:1160: set_chunk oid 27 offset: 30720 length: 264704 target oid 84fef4b9d56ae336567fbdaa2901afe7fd193c521d6b29bafbc12022dbbfabf5 offset: 0 snap -1 2024-09-18T11:31:02.696 INFO:tasks.rados.rados.0.smithi119.stdout:1161: read oid 33 snap -1 2024-09-18T11:31:02.696 INFO:tasks.rados.rados.0.smithi119.stdout:1161: expect deleted 2024-09-18T11:31:02.696 INFO:tasks.rados.rados.0.smithi119.stdout:1162: write (excl) oid 44 current snap is 23 2024-09-18T11:31:02.696 INFO:tasks.rados.rados.0.smithi119.stdout:1162: seq_num 206 ranges {662242=642825,2054820=530730,3307607=154635} 2024-09-18T11:31:02.702 INFO:tasks.rados.rados.0.smithi119.stdout:1162: writing smithi11939506-44 from 662242 to 1305067 tid 1 2024-09-18T11:31:02.706 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:02.705+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:02.709 INFO:tasks.rados.rados.0.smithi119.stdout:1162: writing smithi11939506-44 from 2054820 to 2585550 tid 2 2024-09-18T11:31:02.714 INFO:tasks.rados.rados.0.smithi119.stdout:1162: writing smithi11939506-44 from 3307607 to 3462242 tid 3 2024-09-18T11:31:02.714 INFO:tasks.rados.rados.0.smithi119.stdout:1163: write oid 13 current snap is 23 2024-09-18T11:31:02.714 INFO:tasks.rados.rados.0.smithi119.stdout:1163: seq_num 207 ranges {679049=718074,2113302=674388,3335030=144019} 2024-09-18T11:31:02.722 INFO:tasks.rados.rados.0.smithi119.stdout:1163: writing smithi11939506-13 from 679049 to 1397123 tid 1 2024-09-18T11:31:02.729 INFO:tasks.rados.rados.0.smithi119.stdout:1163: writing smithi11939506-13 from 2113302 to 2787690 tid 2 2024-09-18T11:31:02.733 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:02.733 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:02.734 INFO:tasks.rados.rados.0.smithi119.stdout:1163: writing smithi11939506-13 from 3335030 to 3479049 tid 3 2024-09-18T11:31:02.734 INFO:tasks.rados.rados.0.smithi119.stdout:1164: write oid 16 current snap is 23 2024-09-18T11:31:02.734 INFO:tasks.rados.rados.0.smithi119.stdout:1164: seq_num 208 ranges {695856=793323,2171784=534399,3362453=133403} 2024-09-18T11:31:02.741 INFO:tasks.rados.rados.0.smithi119.stdout:1164: writing smithi11939506-16 from 695856 to 1489179 tid 1 2024-09-18T11:31:02.742 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:31:02.748 INFO:tasks.rados.rados.0.smithi119.stdout:1164: writing smithi11939506-16 from 2171784 to 2706183 tid 2 2024-09-18T11:31:02.753 INFO:tasks.rados.rados.0.smithi119.stdout:1164: writing smithi11939506-16 from 3362453 to 3495856 tid 3 2024-09-18T11:31:02.753 INFO:tasks.rados.rados.0.smithi119.stdout:1165: read oid 30 snap -1 2024-09-18T11:31:02.753 INFO:tasks.rados.rados.0.smithi119.stdout:1165: expect (ObjNum 174 snap 20 seq_num 174) 2024-09-18T11:31:02.753 INFO:tasks.rados.rados.0.smithi119.stdout:1166: delete oid 34 current snap is 23 2024-09-18T11:31:02.753 INFO:tasks.rados.rados.0.smithi119.stdout:1158: expect (ObjNum 199 snap 23 seq_num 199) 2024-09-18T11:31:02.772 INFO:teuthology.orchestra.run.smithi119.stderr:Nothing to do for non-filestore backend 2024-09-18T11:31:02.807 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:02.806+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:02.847 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:02.847 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:02.856 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:31:02.860 INFO:tasks.rados.rados.0.smithi119.stdout:1157: finishing rollback tid 0 to smithi11939506-4 2024-09-18T11:31:02.860 INFO:tasks.rados.rados.0.smithi119.stdout:1159: expect (ObjNum 187 snap 20 seq_num 187) 2024-09-18T11:31:02.908 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:02.906+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:02.959 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:02.959 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:02.967 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:31:03.001 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:02.998+0000 7fb6950ff640 -1 osd.2 79 handle_osd_map injecting map crc failure 2024-09-18T11:31:03.003 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is set 2024-09-18T11:31:03.009 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:03.008+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:03.036 INFO:tasks.rados.rados.0.smithi119.stdout:1157: finishing rollback tid 1 to smithi11939506-4 2024-09-18T11:31:03.036 INFO:tasks.rados.rados.0.smithi119.stdout:1157: finishing rollback tid 2 to smithi11939506-4 2024-09-18T11:31:03.036 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 4 v 54 (ObjNum 110 snap 8 seq_num 110) dirty exists 2024-09-18T11:31:03.036 INFO:tasks.rados.rados.0.smithi119.stdout:1160: finishing set_chunk to oid 27 2024-09-18T11:31:03.036 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 27 set_chunk returned error code -95 offset: 30720 length: 264704 2024-09-18T11:31:03.036 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 0 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T11:31:03.036 INFO:tasks.rados.rados.0.smithi119.stdout:1157: done (9 left) 2024-09-18T11:31:03.036 INFO:tasks.rados.rados.0.smithi119.stdout:1158: done (8 left) 2024-09-18T11:31:03.036 INFO:tasks.rados.rados.0.smithi119.stdout:1159: done (7 left) 2024-09-18T11:31:03.037 INFO:tasks.rados.rados.0.smithi119.stdout:1160: done (6 left) 2024-09-18T11:31:03.037 INFO:tasks.rados.rados.0.smithi119.stdout:1166: done (5 left) 2024-09-18T11:31:03.037 INFO:tasks.rados.rados.0.smithi119.stdout:1167: read oid 24 snap -1 2024-09-18T11:31:03.037 INFO:tasks.rados.rados.0.smithi119.stdout:1167: expect (ObjNum 135 snap 16 seq_num 135) 2024-09-18T11:31:03.037 INFO:tasks.rados.rados.0.smithi119.stdout:1168: write (excl) oid 29 current snap is 23 2024-09-18T11:31:03.037 INFO:tasks.rados.rados.0.smithi119.stdout:1162: finishing write tid 1 to smithi11939506-44 2024-09-18T11:31:03.037 INFO:tasks.rados.rados.0.smithi119.stdout:1162: finishing write tid 2 to smithi11939506-44 2024-09-18T11:31:03.037 INFO:tasks.rados.rados.0.smithi119.stdout:1168: seq_num 210 ranges {729470=543821,1772395=538068,2900946=403005} 2024-09-18T11:31:03.042 INFO:tasks.rados.rados.0.smithi119.stdout:1168: writing smithi11939506-29 from 729470 to 1273291 tid 1 2024-09-18T11:31:03.048 INFO:tasks.rados.rados.0.smithi119.stdout:1168: writing smithi11939506-29 from 1772395 to 2310463 tid 2 2024-09-18T11:31:03.054 INFO:tasks.rados.rados.0.smithi119.stdout:1168: writing smithi11939506-29 from 2900946 to 3303951 tid 3 2024-09-18T11:31:03.054 INFO:tasks.rados.rados.0.smithi119.stdout:1161: done (6 left) 2024-09-18T11:31:03.054 INFO:tasks.rados.rados.0.smithi119.stdout:1169: write (excl) oid 14 current snap is 23 2024-09-18T11:31:03.054 INFO:tasks.rados.rados.0.smithi119.stdout:1162: finishing write tid 3 to smithi11939506-44 2024-09-18T11:31:03.054 INFO:tasks.rados.rados.0.smithi119.stdout:1162: finishing write tid 4 to smithi11939506-44 2024-09-18T11:31:03.054 INFO:tasks.rados.rados.0.smithi119.stdout:1162: finishing write tid 5 to smithi11939506-44 2024-09-18T11:31:03.054 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 44 v 82 (ObjNum 206 snap 23 seq_num 206) dirty exists 2024-09-18T11:31:03.055 INFO:tasks.rados.rados.0.smithi119.stdout:1162: left oid 44 (ObjNum 206 snap 23 seq_num 206) 2024-09-18T11:31:03.055 INFO:tasks.rados.rados.0.smithi119.stdout:1163: finishing write tid 1 to smithi11939506-13 2024-09-18T11:31:03.055 INFO:tasks.rados.rados.0.smithi119.stdout:1163: finishing write tid 2 to smithi11939506-13 2024-09-18T11:31:03.055 INFO:tasks.rados.rados.0.smithi119.stdout:1163: finishing write tid 3 to smithi11939506-13 2024-09-18T11:31:03.055 INFO:tasks.rados.rados.0.smithi119.stdout:1169: seq_num 211 ranges {746277=619070,1830877=681726,2928369=617908} 2024-09-18T11:31:03.061 INFO:tasks.rados.rados.0.smithi119.stdout:1169: writing smithi11939506-14 from 746277 to 1365347 tid 1 2024-09-18T11:31:03.067 INFO:tasks.rados.rados.0.smithi119.stdout:1169: writing smithi11939506-14 from 1830877 to 2512603 tid 2 2024-09-18T11:31:03.073 INFO:tasks.rados.rados.0.smithi119.stdout:1169: writing smithi11939506-14 from 2928369 to 3546277 tid 3 2024-09-18T11:31:03.073 INFO:tasks.rados.rados.0.smithi119.stdout:1162: done (6 left) 2024-09-18T11:31:03.073 INFO:tasks.rados.rados.0.smithi119.stdout:1170: set_chunk oid 33 target oid 2024-09-18T11:31:03.073 INFO:tasks.rados.rados.0.smithi119.stdout:1170: set_chunk oid 33 offset: 0 length: 4000000 target oid smithi11939506- offset: 0 snap -1 2024-09-18T11:31:03.074 INFO:tasks.rados.rados.0.smithi119.stdout:1171: write oid 25 current snap is 23 2024-09-18T11:31:03.074 INFO:tasks.rados.rados.0.smithi119.stdout:1171: seq_num 212 ranges {763084=694319,1889359=541737,2955792=541902} 2024-09-18T11:31:03.081 INFO:tasks.rados.rados.0.smithi119.stdout:1171: writing smithi11939506-25 from 763084 to 1457403 tid 1 2024-09-18T11:31:03.086 INFO:tasks.rados.rados.0.smithi119.stdout:1171: writing smithi11939506-25 from 1889359 to 2431096 tid 2 2024-09-18T11:31:03.093 INFO:tasks.rados.rados.0.smithi119.stdout:1171: writing smithi11939506-25 from 2955792 to 3497694 tid 3 2024-09-18T11:31:03.093 INFO:tasks.rados.rados.0.smithi119.stdout:1172: read oid 26 snap 22 2024-09-18T11:31:03.093 INFO:tasks.rados.rados.0.smithi119.stdout:1172: expect (ObjNum 178 snap 20 seq_num 178) 2024-09-18T11:31:03.093 INFO:tasks.rados.rados.0.smithi119.stdout:1173: snap_create 2024-09-18T11:31:03.093 INFO:tasks.rados.rados.0.smithi119.stdout:1163: finishing write tid 4 to smithi11939506-13 2024-09-18T11:31:03.093 INFO:tasks.rados.rados.0.smithi119.stdout:1163: finishing write tid 5 to smithi11939506-13 2024-09-18T11:31:03.093 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 13 v 215 (ObjNum 207 snap 23 seq_num 207) dirty exists 2024-09-18T11:31:03.093 INFO:tasks.rados.rados.0.smithi119.stdout:1163: left oid 13 (ObjNum 207 snap 23 seq_num 207) 2024-09-18T11:31:03.093 INFO:tasks.rados.rados.0.smithi119.stdout:1164: finishing write tid 1 to smithi11939506-16 2024-09-18T11:31:03.093 INFO:tasks.rados.rados.0.smithi119.stdout:1164: finishing write tid 2 to smithi11939506-16 2024-09-18T11:31:03.093 INFO:tasks.rados.rados.0.smithi119.stdout:1165: expect (ObjNum 174 snap 20 seq_num 174) 2024-09-18T11:31:03.102 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:03.102 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:03.110 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:03.109+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:03.111 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:31:03.211 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:03.210+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:03.250 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:03.250 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:03.259 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:31:03.293 INFO:tasks.rados.rados.0.smithi119.stdout:1164: finishing write tid 3 to smithi11939506-16 2024-09-18T11:31:03.293 INFO:tasks.rados.rados.0.smithi119.stdout:1164: finishing write tid 4 to smithi11939506-16 2024-09-18T11:31:03.293 INFO:tasks.rados.rados.0.smithi119.stdout:1164: finishing write tid 5 to smithi11939506-16 2024-09-18T11:31:03.293 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 144 (ObjNum 208 snap 23 seq_num 208) dirty exists 2024-09-18T11:31:03.293 INFO:tasks.rados.rados.0.smithi119.stdout:1164: left oid 16 (ObjNum 208 snap 23 seq_num 208) 2024-09-18T11:31:03.293 INFO:tasks.rados.rados.0.smithi119.stdout:1167: expect (ObjNum 135 snap 16 seq_num 135) 2024-09-18T11:31:03.312 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:03.309+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:03.317 INFO:tasks.thrashosds.thrasher:Waiting for clean again 2024-09-18T11:31:03.317 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T11:31:03.318 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:31:03.358 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:03.358 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:03.366 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:31:03.412 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:03.411+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:03.441 INFO:tasks.rados.rados.0.smithi119.stdout:1168: finishing write tid 1 to smithi11939506-29 2024-09-18T11:31:03.441 INFO:tasks.rados.rados.0.smithi119.stdout:1168: finishing write tid 2 to smithi11939506-29 2024-09-18T11:31:03.441 INFO:tasks.rados.rados.0.smithi119.stdout:1168: finishing write tid 3 to smithi11939506-29 2024-09-18T11:31:03.441 INFO:tasks.rados.rados.0.smithi119.stdout:1168: finishing write tid 4 to smithi11939506-29 2024-09-18T11:31:03.441 INFO:tasks.rados.rados.0.smithi119.stdout:1168: finishing write tid 5 to smithi11939506-29 2024-09-18T11:31:03.442 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 67 (ObjNum 210 snap 23 seq_num 210) dirty exists 2024-09-18T11:31:03.442 INFO:tasks.rados.rados.0.smithi119.stdout:1168: left oid 29 (ObjNum 210 snap 23 seq_num 210) 2024-09-18T11:31:03.442 INFO:tasks.rados.rados.0.smithi119.stdout:1169: finishing write tid 1 to smithi11939506-14 2024-09-18T11:31:03.442 INFO:tasks.rados.rados.0.smithi119.stdout:1169: finishing write tid 2 to smithi11939506-14 2024-09-18T11:31:03.442 INFO:tasks.rados.rados.0.smithi119.stdout:1170: finishing set_chunk to oid 33 2024-09-18T11:31:03.442 INFO:tasks.rados.rados.0.smithi119.stdout:1170: got expected ENOENT (src dne) 2024-09-18T11:31:03.442 INFO:tasks.rados.rados.0.smithi119.stdout:1169: finishing write tid 3 to smithi11939506-14 2024-09-18T11:31:03.442 INFO:tasks.rados.rados.0.smithi119.stdout:1169: finishing write tid 4 to smithi11939506-14 2024-09-18T11:31:03.442 INFO:tasks.rados.rados.0.smithi119.stdout:1169: finishing write tid 5 to smithi11939506-14 2024-09-18T11:31:03.442 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 161 (ObjNum 211 snap 23 seq_num 211) dirty exists 2024-09-18T11:31:03.442 INFO:tasks.rados.rados.0.smithi119.stdout:1169: left oid 14 (ObjNum 211 snap 23 seq_num 211) 2024-09-18T11:31:03.442 INFO:tasks.rados.rados.0.smithi119.stdout:1171: finishing write tid 1 to smithi11939506-25 2024-09-18T11:31:03.442 INFO:tasks.rados.rados.0.smithi119.stdout:1171: finishing write tid 2 to smithi11939506-25 2024-09-18T11:31:03.442 INFO:tasks.rados.rados.0.smithi119.stdout:1171: finishing write tid 3 to smithi11939506-25 2024-09-18T11:31:03.443 INFO:tasks.rados.rados.0.smithi119.stdout:1171: finishing write tid 4 to smithi11939506-25 2024-09-18T11:31:03.443 INFO:tasks.rados.rados.0.smithi119.stdout:1171: finishing write tid 5 to smithi11939506-25 2024-09-18T11:31:03.443 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 34 (ObjNum 212 snap 23 seq_num 212) dirty exists 2024-09-18T11:31:03.443 INFO:tasks.rados.rados.0.smithi119.stdout:1171: left oid 25 (ObjNum 212 snap 23 seq_num 212) 2024-09-18T11:31:03.443 INFO:tasks.rados.rados.0.smithi119.stdout:1172: expect (ObjNum 178 snap 20 seq_num 178) 2024-09-18T11:31:03.509 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:03.509 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:03.513 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:03.512+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:03.517 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:31:03.614 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:03.613+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:03.617 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:03.617 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:03.626 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:31:03.629 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:31:03.630 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:31:03.642 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-09-18T11:31:03.000480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":333679945,"num_objects":218,"num_object_clones":67,"num_object_copies":436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":216,"num_whiteouts":7,"num_read":2451,"num_read_kb":2219574,"num_write":1482,"num_write_kb":430095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":63687773,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":68,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2177,"ondisk_log_size":2177,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":2504988,"kb_used_data":412836,"kb_used_omap":11,"kb_used_meta":2089716,"kb_avail":658097892,"statfs":{"total":676457349120,"available":673892241408,"internally_reserved":0,"allocated":422744064,"data_stored":495512762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2139869544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12243013,"num_objects":14,"num_object_clones":11,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":2,"num_read":169,"num_read_kb":181260,"num_write":100,"num_write_kb":24581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.624830"},"pg_stats":[{"pgid":"4.f","version":"79'144","reported_seq":326,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.757606+0000","last_change":"2024-09-18T11:31:02.422200+0000","last_active":"2024-09-18T11:31:02.757606+0000","last_peered":"2024-09-18T11:31:02.757606+0000","last_clean":"2024-09-18T11:31:02.757606+0000","last_became_active":"2024-09-18T11:30:02.074391+0000","last_became_peered":"2024-09-18T11:30:02.074391+0000","last_unstale":"2024-09-18T11:31:02.757606+0000","last_undegraded":"2024-09-18T11:31:02.757606+0000","last_fullsized":"2024-09-18T11:31:02.757606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:11.123483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019127767,"stat_sum":{"num_bytes":15633962,"num_objects":10,"num_object_clones":6,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":155,"num_read_kb":161567,"num_write":86,"num_write_kb":23057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"4.e","version":"79'99","reported_seq":291,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.685797+0000","last_change":"2024-09-18T11:31:02.397939+0000","last_active":"2024-09-18T11:31:02.685797+0000","last_peered":"2024-09-18T11:31:02.685797+0000","last_clean":"2024-09-18T11:31:02.685797+0000","last_became_active":"2024-09-18T11:30:02.073185+0000","last_became_peered":"2024-09-18T11:30:02.073185+0000","last_unstale":"2024-09-18T11:31:02.685797+0000","last_undegraded":"2024-09-18T11:31:02.685797+0000","last_fullsized":"2024-09-18T11:31:02.685797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:12.655066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038824889999999998,"stat_sum":{"num_bytes":15730974,"num_objects":6,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":164,"num_read_kb":182460,"num_write":45,"num_write_kb":12076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":5,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"4.d","version":"79'133","reported_seq":341,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.683251+0000","last_change":"2024-09-18T11:31:02.405361+0000","last_active":"2024-09-18T11:31:02.683251+0000","last_peered":"2024-09-18T11:31:02.683251+0000","last_clean":"2024-09-18T11:31:02.683251+0000","last_became_active":"2024-09-18T11:30:02.334498+0000","last_became_peered":"2024-09-18T11:30:02.334498+0000","last_unstale":"2024-09-18T11:31:02.683251+0000","last_undegraded":"2024-09-18T11:31:02.683251+0000","last_fullsized":"2024-09-18T11:31:02.683251+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:10.245504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011210894000000001,"stat_sum":{"num_bytes":17897939,"num_objects":9,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":217,"num_read_kb":236686,"num_write":69,"num_write_kb":19979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":8,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"4.c","version":"79'54","reported_seq":217,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.687172+0000","last_change":"2024-09-18T11:31:02.399810+0000","last_active":"2024-09-18T11:31:02.687172+0000","last_peered":"2024-09-18T11:31:02.687172+0000","last_clean":"2024-09-18T11:31:02.687172+0000","last_became_active":"2024-09-18T11:30:02.073883+0000","last_became_peered":"2024-09-18T11:30:02.073883+0000","last_unstale":"2024-09-18T11:31:02.687172+0000","last_undegraded":"2024-09-18T11:31:02.687172+0000","last_fullsized":"2024-09-18T11:31:02.687172+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:20.359490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005805279,"stat_sum":{"num_bytes":8197975,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":67,"num_read_kb":69995,"num_write":25,"num_write_kb":6106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"4.b","version":"79'215","reported_seq":457,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.735216+0000","last_change":"2024-09-18T11:31:02.419554+0000","last_active":"2024-09-18T11:31:02.735216+0000","last_peered":"2024-09-18T11:31:02.735216+0000","last_clean":"2024-09-18T11:31:02.735216+0000","last_became_active":"2024-09-18T11:30:50.333230+0000","last_became_peered":"2024-09-18T11:30:50.333230+0000","last_unstale":"2024-09-18T11:31:02.735216+0000","last_undegraded":"2024-09-18T11:31:02.735216+0000","last_fullsized":"2024-09-18T11:31:02.735216+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:20.199941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01681738,"stat_sum":{"num_bytes":35361703,"num_objects":17,"num_object_clones":11,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":220,"num_read_kb":214577,"num_write":116,"num_write_kb":32298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":17763749,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":9,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"4.a","version":"79'113","reported_seq":299,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.427542+0000","last_change":"2024-09-18T11:31:02.405986+0000","last_active":"2024-09-18T11:31:02.427542+0000","last_peered":"2024-09-18T11:31:02.427542+0000","last_clean":"2024-09-18T11:31:02.427542+0000","last_became_active":"2024-09-18T11:30:02.075538+0000","last_became_peered":"2024-09-18T11:30:02.075538+0000","last_unstale":"2024-09-18T11:31:02.427542+0000","last_undegraded":"2024-09-18T11:31:02.427542+0000","last_fullsized":"2024-09-18T11:31:02.427542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:26.428990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009454166,"stat_sum":{"num_bytes":13565104,"num_objects":8,"num_object_clones":5,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":177,"num_read_kb":249814,"num_write":50,"num_write_kb":14990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":6,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"4.9","version":"79'45","reported_seq":197,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.413709+0000","last_change":"2024-09-18T11:31:02.413902+0000","last_active":"2024-09-18T11:31:02.413709+0000","last_peered":"2024-09-18T11:31:02.413709+0000","last_clean":"2024-09-18T11:31:02.413709+0000","last_became_active":"2024-09-18T11:30:02.074414+0000","last_became_peered":"2024-09-18T11:30:02.074414+0000","last_unstale":"2024-09-18T11:31:02.413709+0000","last_undegraded":"2024-09-18T11:31:02.413709+0000","last_fullsized":"2024-09-18T11:31:02.413709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019722076000000002,"stat_sum":{"num_bytes":7323711,"num_objects":3,"num_object_clones":2,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":43640,"num_write":27,"num_write_kb":8449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"4.8","version":"79'30","reported_seq":173,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.402126+0000","last_change":"2024-09-18T11:31:02.402318+0000","last_active":"2024-09-18T11:31:02.402126+0000","last_peered":"2024-09-18T11:31:02.402126+0000","last_clean":"2024-09-18T11:31:02.402126+0000","last_became_active":"2024-09-18T11:30:02.076172+0000","last_became_peered":"2024-09-18T11:30:02.076172+0000","last_unstale":"2024-09-18T11:31:02.402126+0000","last_undegraded":"2024-09-18T11:31:02.402126+0000","last_fullsized":"2024-09-18T11:31:02.402126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:29.097553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001762069,"stat_sum":{"num_bytes":4916668,"num_objects":4,"num_object_clones":3,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":23,"num_read_kb":15209,"num_write":13,"num_write_kb":3671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"4.5","version":"79'147","reported_seq":347,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.402115+0000","last_change":"2024-09-18T11:31:02.402316+0000","last_active":"2024-09-18T11:31:02.402115+0000","last_peered":"2024-09-18T11:31:02.402115+0000","last_clean":"2024-09-18T11:31:02.402115+0000","last_became_active":"2024-09-18T11:30:02.074697+0000","last_became_peered":"2024-09-18T11:30:02.074697+0000","last_unstale":"2024-09-18T11:31:02.402115+0000","last_undegraded":"2024-09-18T11:31:02.402115+0000","last_fullsized":"2024-09-18T11:31:02.402115+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.577548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056736490000000002,"stat_sum":{"num_bytes":17403447,"num_objects":9,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":194,"num_read_kb":211919,"num_write":79,"num_write_kb":20300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"1.0","version":"16'72","reported_seq":188,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.390551+0000","last_change":"2024-09-18T11:30:57.650616+0000","last_active":"2024-09-18T11:31:01.390551+0000","last_peered":"2024-09-18T11:31:01.390551+0000","last_clean":"2024-09-18T11:31:01.390551+0000","last_became_active":"2024-09-18T11:30:50.741897+0000","last_became_peered":"2024-09-18T11:30:50.741897+0000","last_unstale":"2024-09-18T11:31:01.390551+0000","last_undegraded":"2024-09-18T11:31:01.390551+0000","last_fullsized":"2024-09-18T11:31:01.390551+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'72","last_scrub_stamp":"2024-09-18T11:30:57.650532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_clean_scrub_stamp":"2024-09-18T11:30:57.650532+0000","objects_scrubbed":2,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:58.577847+0000","scrub_duration":0.0016372979999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":75,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.390629+0000","last_change":"2024-09-18T11:29:52.047782+0000","last_active":"2024-09-18T11:31:01.390629+0000","last_peered":"2024-09-18T11:31:01.390629+0000","last_clean":"2024-09-18T11:31:01.390629+0000","last_became_active":"2024-09-18T11:29:50.028407+0000","last_became_peered":"2024-09-18T11:29:50.028407+0000","last_unstale":"2024-09-18T11:31:01.390629+0000","last_undegraded":"2024-09-18T11:31:01.390629+0000","last_fullsized":"2024-09-18T11:31:01.390629+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.851663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"77'126","reported_seq":198,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.385209+0000","last_change":"2024-09-18T11:30:01.077948+0000","last_active":"2024-09-18T11:31:01.385209+0000","last_peered":"2024-09-18T11:31:01.385209+0000","last_clean":"2024-09-18T11:31:01.385209+0000","last_became_active":"2024-09-18T11:30:01.077612+0000","last_became_peered":"2024-09-18T11:30:01.077612+0000","last_unstale":"2024-09-18T11:31:01.385209+0000","last_undegraded":"2024-09-18T11:31:01.385209+0000","last_fullsized":"2024-09-18T11:31:01.385209+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:30:00.059457+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:11.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27212175,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":100,"num_read_kb":2402,"num_write":143,"num_write_kb":51344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"79'62","reported_seq":243,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.755687+0000","last_change":"2024-09-18T11:31:02.416619+0000","last_active":"2024-09-18T11:31:02.755687+0000","last_peered":"2024-09-18T11:31:02.755687+0000","last_clean":"2024-09-18T11:31:02.755687+0000","last_became_active":"2024-09-18T11:30:50.743510+0000","last_became_peered":"2024-09-18T11:30:50.743510+0000","last_unstale":"2024-09-18T11:31:02.755687+0000","last_undegraded":"2024-09-18T11:31:02.755687+0000","last_fullsized":"2024-09-18T11:31:02.755687+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.427753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013723536999999999,"stat_sum":{"num_bytes":8434171,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":80,"num_read_kb":66751,"num_write":33,"num_write_kb":7919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":2949092,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":75,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.390468+0000","last_change":"2024-09-18T11:29:52.420488+0000","last_active":"2024-09-18T11:31:01.390468+0000","last_peered":"2024-09-18T11:31:01.390468+0000","last_clean":"2024-09-18T11:31:01.390468+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:31:01.390468+0000","last_undegraded":"2024-09-18T11:31:01.390468+0000","last_fullsized":"2024-09-18T11:31:01.390468+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:02.759929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'114","reported_seq":275,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.393954+0000","last_change":"2024-09-18T11:30:50.488394+0000","last_active":"2024-09-18T11:31:02.393954+0000","last_peered":"2024-09-18T11:31:02.393954+0000","last_clean":"2024-09-18T11:31:02.393954+0000","last_became_active":"2024-09-18T11:30:50.329599+0000","last_became_peered":"2024-09-18T11:30:50.329599+0000","last_unstale":"2024-09-18T11:31:02.393954+0000","last_undegraded":"2024-09-18T11:31:02.393954+0000","last_fullsized":"2024-09-18T11:31:02.393954+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:30:00.059457+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:27.886849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33501848,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":102,"num_read_kb":7881,"num_write":123,"num_write_kb":53221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":20418189,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"79'112","reported_seq":327,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.425041+0000","last_change":"2024-09-18T11:31:02.425246+0000","last_active":"2024-09-18T11:31:02.425041+0000","last_peered":"2024-09-18T11:31:02.425041+0000","last_clean":"2024-09-18T11:31:02.425041+0000","last_became_active":"2024-09-18T11:30:02.076465+0000","last_became_peered":"2024-09-18T11:30:02.076465+0000","last_unstale":"2024-09-18T11:31:02.425041+0000","last_undegraded":"2024-09-18T11:31:02.425041+0000","last_fullsized":"2024-09-18T11:31:02.425041+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:15.876699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.022014636000000001,"stat_sum":{"num_bytes":21625816,"num_objects":8,"num_object_clones":3,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":217,"num_read_kb":229052,"num_write":72,"num_write_kb":19025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":76,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.387578+0000","last_change":"2024-09-18T11:29:52.348137+0000","last_active":"2024-09-18T11:31:01.387578+0000","last_peered":"2024-09-18T11:31:01.387578+0000","last_clean":"2024-09-18T11:31:01.387578+0000","last_became_active":"2024-09-18T11:29:50.180029+0000","last_became_peered":"2024-09-18T11:29:50.180029+0000","last_unstale":"2024-09-18T11:31:01.387578+0000","last_undegraded":"2024-09-18T11:31:01.387578+0000","last_fullsized":"2024-09-18T11:31:01.387578+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.269389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023063399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"78'83","reported_seq":150,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.424433+0000","last_change":"2024-09-18T11:30:01.071704+0000","last_active":"2024-09-18T11:31:01.424433+0000","last_peered":"2024-09-18T11:31:01.424433+0000","last_clean":"2024-09-18T11:31:01.424433+0000","last_became_active":"2024-09-18T11:30:01.071435+0000","last_became_peered":"2024-09-18T11:30:01.071435+0000","last_unstale":"2024-09-18T11:31:01.424433+0000","last_undegraded":"2024-09-18T11:31:01.424433+0000","last_fullsized":"2024-09-18T11:31:01.424433+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:30:00.059457+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14444376,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":66,"num_read_kb":140,"num_write":88,"num_write_kb":29433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"79'109","reported_seq":278,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.408380+0000","last_change":"2024-09-18T11:31:02.408609+0000","last_active":"2024-09-18T11:31:02.408380+0000","last_peered":"2024-09-18T11:31:02.408380+0000","last_clean":"2024-09-18T11:31:02.408380+0000","last_became_active":"2024-09-18T11:30:02.334751+0000","last_became_peered":"2024-09-18T11:30:02.334751+0000","last_unstale":"2024-09-18T11:31:02.408380+0000","last_undegraded":"2024-09-18T11:31:02.408380+0000","last_fullsized":"2024-09-18T11:31:02.408380+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:20.359490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060395900000000001,"stat_sum":{"num_bytes":13664458,"num_objects":7,"num_object_clones":4,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":111,"num_read_kb":119178,"num_write":62,"num_write_kb":17894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":76,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.390539+0000","last_change":"2024-09-18T11:29:52.048389+0000","last_active":"2024-09-18T11:31:01.390539+0000","last_peered":"2024-09-18T11:31:01.390539+0000","last_clean":"2024-09-18T11:31:01.390539+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:31:01.390539+0000","last_undegraded":"2024-09-18T11:31:01.390539+0000","last_fullsized":"2024-09-18T11:31:01.390539+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:13.358402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"77'99","reported_seq":249,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.383245+0000","last_change":"2024-09-18T11:30:50.864329+0000","last_active":"2024-09-18T11:31:01.383245+0000","last_peered":"2024-09-18T11:31:01.383245+0000","last_clean":"2024-09-18T11:31:01.383245+0000","last_became_active":"2024-09-18T11:30:50.741594+0000","last_became_peered":"2024-09-18T11:30:50.741594+0000","last_unstale":"2024-09-18T11:31:01.383245+0000","last_undegraded":"2024-09-18T11:31:01.383245+0000","last_fullsized":"2024-09-18T11:31:01.383245+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:30:00.059457+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.802711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26632899,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":80,"num_read_kb":1690,"num_write":107,"num_write_kb":42852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":16290641,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"79'63","reported_seq":215,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.413693+0000","last_change":"2024-09-18T11:31:02.413904+0000","last_active":"2024-09-18T11:31:02.413693+0000","last_peered":"2024-09-18T11:31:02.413693+0000","last_clean":"2024-09-18T11:31:02.413693+0000","last_became_active":"2024-09-18T11:30:02.076874+0000","last_became_peered":"2024-09-18T11:30:02.076874+0000","last_unstale":"2024-09-18T11:31:02.413693+0000","last_undegraded":"2024-09-18T11:31:02.413693+0000","last_fullsized":"2024-09-18T11:31:02.413693+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:06.825500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011070398,"stat_sum":{"num_bytes":6541447,"num_objects":4,"num_object_clones":3,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":49,"num_read_kb":47757,"num_write":40,"num_write_kb":11590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":86,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.390558+0000","last_change":"2024-09-18T11:30:50.331231+0000","last_active":"2024-09-18T11:31:01.390558+0000","last_peered":"2024-09-18T11:31:01.390558+0000","last_clean":"2024-09-18T11:31:01.390558+0000","last_became_active":"2024-09-18T11:30:50.331026+0000","last_became_peered":"2024-09-18T11:30:50.331026+0000","last_unstale":"2024-09-18T11:31:01.390558+0000","last_undegraded":"2024-09-18T11:31:01.390558+0000","last_fullsized":"2024-09-18T11:31:01.390558+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:09.742125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"79'82","reported_seq":261,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.720894+0000","last_change":"2024-09-18T11:31:02.405543+0000","last_active":"2024-09-18T11:31:02.720894+0000","last_peered":"2024-09-18T11:31:02.720894+0000","last_clean":"2024-09-18T11:31:02.720894+0000","last_became_active":"2024-09-18T11:30:50.333722+0000","last_became_peered":"2024-09-18T11:30:50.333722+0000","last_unstale":"2024-09-18T11:31:02.720894+0000","last_undegraded":"2024-09-18T11:31:02.720894+0000","last_fullsized":"2024-09-18T11:31:02.720894+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:25.884327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049286369999999996,"stat_sum":{"num_bytes":9320535,"num_objects":6,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":64,"num_read_kb":62957,"num_write":43,"num_write_kb":11223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":5737174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":5,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":86,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.390516+0000","last_change":"2024-09-18T11:30:50.327769+0000","last_active":"2024-09-18T11:31:01.390516+0000","last_peered":"2024-09-18T11:31:01.390516+0000","last_clean":"2024-09-18T11:31:01.390516+0000","last_became_active":"2024-09-18T11:30:50.327483+0000","last_became_peered":"2024-09-18T11:30:50.327483+0000","last_unstale":"2024-09-18T11:31:01.390516+0000","last_undegraded":"2024-09-18T11:31:01.390516+0000","last_fullsized":"2024-09-18T11:31:01.390516+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:13.119942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"79'115","reported_seq":286,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.408568+0000","last_change":"2024-09-18T11:31:02.408792+0000","last_active":"2024-09-18T11:31:02.408568+0000","last_peered":"2024-09-18T11:31:02.408568+0000","last_clean":"2024-09-18T11:31:02.408568+0000","last_became_active":"2024-09-18T11:30:02.072165+0000","last_became_peered":"2024-09-18T11:30:02.072165+0000","last_unstale":"2024-09-18T11:31:02.408568+0000","last_undegraded":"2024-09-18T11:31:02.408568+0000","last_fullsized":"2024-09-18T11:31:02.408568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:04.270484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006340844,"stat_sum":{"num_bytes":17555791,"num_objects":7,"num_object_clones":4,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":108,"num_read_kb":116510,"num_write":78,"num_write_kb":22821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":85,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.390536+0000","last_change":"2024-09-18T11:30:57.765542+0000","last_active":"2024-09-18T11:31:01.390536+0000","last_peered":"2024-09-18T11:31:01.390536+0000","last_clean":"2024-09-18T11:31:01.390536+0000","last_became_active":"2024-09-18T11:29:50.028674+0000","last_became_peered":"2024-09-18T11:29:50.028674+0000","last_unstale":"2024-09-18T11:31:01.390536+0000","last_undegraded":"2024-09-18T11:31:01.390536+0000","last_fullsized":"2024-09-18T11:31:01.390536+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:57.765475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:30:57.765475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:20.322582+0000","scrub_duration":0.001835167,"objects_trimmed":0,"snaptrim_duration":0.00031621200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"79'157","reported_seq":339,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.408451+0000","last_change":"2024-09-18T11:31:02.410962+0000","last_active":"2024-09-18T11:31:02.408451+0000","last_peered":"2024-09-18T11:31:02.408451+0000","last_clean":"2024-09-18T11:31:02.408451+0000","last_became_active":"2024-09-18T11:30:02.075529+0000","last_became_peered":"2024-09-18T11:30:02.075529+0000","last_unstale":"2024-09-18T11:31:02.408451+0000","last_undegraded":"2024-09-18T11:31:02.408451+0000","last_fullsized":"2024-09-18T11:31:02.408451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:21.261787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.016883939000000001,"stat_sum":{"num_bytes":18124559,"num_objects":9,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":154,"num_read_kb":179334,"num_write":80,"num_write_kb":20217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":7,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":85,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.385358+0000","last_change":"2024-09-18T11:30:57.053072+0000","last_active":"2024-09-18T11:31:01.385358+0000","last_peered":"2024-09-18T11:31:01.385358+0000","last_clean":"2024-09-18T11:31:01.385358+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:31:01.385358+0000","last_undegraded":"2024-09-18T11:31:01.385358+0000","last_fullsized":"2024-09-18T11:31:01.385358+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:57.052994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:30:57.052994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:58.065434+0000","scrub_duration":0.0012785050000000001,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":231298260,"num_objects":115,"num_object_clones":67,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":7,"num_read":2037,"num_read_kb":2207406,"num_write":918,"num_write_kb":251615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":26450015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":68,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":329928704,"data_stored":417097862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1680,"ondisk_log_size":1680,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":101791298,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":348,"num_read_kb":12113,"num_write":461,"num_write_kb":176850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":36708830,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":153198592,"data_stored":152833186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":422,"ondisk_log_size":422,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320088,"kb_used_data":22724,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94051752,"statfs":{"total":96636764160,"available":96308994048,"internally_reserved":0,"allocated":23269376,"data_stored":28544444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":1,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.40400000000000003}]},{"osd":2,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.39100000000000001}]},{"osd":3,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.41799999999999998}]},{"osd":4,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.314},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.32700000000000001}]},{"osd":5,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.28699999999999998}]},{"osd":6,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.372},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.30299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":339648,"kb_used_data":42284,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94032192,"statfs":{"total":96636764160,"available":96288964608,"internally_reserved":0,"allocated":43298816,"data_stored":54723931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Wed Sep 18 11:30:50 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.45200000000000001}]},{"osd":2,"last update":"Wed Sep 18 11:30:50 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.496},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.46999999999999997}]},{"osd":3,"last update":"Wed Sep 18 11:30:50 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.436}]},{"osd":4,"last update":"Wed Sep 18 11:30:50 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.53900000000000003}]},{"osd":5,"last update":"Wed Sep 18 11:30:50 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.51200000000000001}]},{"osd":7,"last update":"Wed Sep 18 11:30:50 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.57299999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":351916,"kb_used_data":54552,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94019924,"statfs":{"total":96636764160,"available":96276402176,"internally_reserved":0,"allocated":55861248,"data_stored":66302023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":1,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.375}]},{"osd":2,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.38500000000000001}]},{"osd":3,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.44500000000000001}]},{"osd":4,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.32300000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.30299999999999999}]},{"osd":7,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.40699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413460,"kb_used_data":110772,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93958380,"statfs":{"total":96636764160,"available":96213381120,"internally_reserved":0,"allocated":113430528,"data_stored":129913881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.42799999999999999}]},{"osd":1,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.55100000000000005}]},{"osd":3,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.59299999999999997}]},{"osd":5,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.35899999999999999}]},{"osd":6,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.438}]},{"osd":7,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.39800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":357792,"kb_used_data":60428,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94014048,"statfs":{"total":96636764160,"available":96270385152,"internally_reserved":0,"allocated":61878272,"data_stored":65229808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.318},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.34999999999999998}]},{"osd":1,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.214}]},{"osd":2,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.24399999999999999}]},{"osd":4,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.61299999999999999}]},{"osd":5,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.63600000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.65800000000000003}]},{"osd":7,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.55000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":330628,"kb_used_data":33268,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94041212,"statfs":{"total":96636764160,"available":96298201088,"internally_reserved":0,"allocated":34066432,"data_stored":40112417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.253}]},{"osd":1,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.29299999999999998}]},{"osd":3,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.23799999999999999}]},{"osd":4,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.376}]},{"osd":5,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.44},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.45400000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.32300000000000001}]},{"osd":7,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.42399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":391456,"kb_used_data":88808,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93980384,"statfs":{"total":96636764160,"available":96235913216,"internally_reserved":0,"allocated":90939392,"data_stored":110686258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.224}]},{"osd":2,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.17399999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.26600000000000001}]},{"osd":3,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.23699999999999999}]},{"osd":4,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.32300000000000001}]},{"osd":5,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.438},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.312}]},{"osd":6,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.39000000000000001}]},{"osd":7,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.379}]}]},{"osd":0,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36782080,"data_stored":36708830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8220672,"data_stored":8202102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8220672,"data_stored":8202102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33914880,"data_stored":33819019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":35041280,"data_stored":34963694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":14946304,"data_stored":14896382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16072704,"data_stored":16041057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25010176,"data_stored":27495860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81805312,"data_stored":102017381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24973312,"data_stored":31453239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28524544,"data_stored":35601225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":77852672,"data_stored":97009634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":41369600,"data_stored":53469581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":27222016,"data_stored":39976970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":23171072,"data_stored":30073972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:31:03.645 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:31:03.715 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:03.714+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:03.765 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:03.765 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:03.773 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:31:03.815 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:03.813+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:03.909 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:31:03.909 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:31:03.911 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:03.911 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:03.916 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:03.915+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:03.920 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:31:03.921 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-09-18T11:31:03.000480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":333679945,"num_objects":218,"num_object_clones":67,"num_object_copies":436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":216,"num_whiteouts":7,"num_read":2451,"num_read_kb":2219574,"num_write":1482,"num_write_kb":430095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":63687773,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":68,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2177,"ondisk_log_size":2177,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":2504988,"kb_used_data":412836,"kb_used_omap":11,"kb_used_meta":2089716,"kb_avail":658097892,"statfs":{"total":676457349120,"available":673892241408,"internally_reserved":0,"allocated":422744064,"data_stored":495512762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2139869544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12243013,"num_objects":14,"num_object_clones":11,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":2,"num_read":169,"num_read_kb":181260,"num_write":100,"num_write_kb":24581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.624830"},"pg_stats":[{"pgid":"4.f","version":"79'144","reported_seq":326,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.757606+0000","last_change":"2024-09-18T11:31:02.422200+0000","last_active":"2024-09-18T11:31:02.757606+0000","last_peered":"2024-09-18T11:31:02.757606+0000","last_clean":"2024-09-18T11:31:02.757606+0000","last_became_active":"2024-09-18T11:30:02.074391+0000","last_became_peered":"2024-09-18T11:30:02.074391+0000","last_unstale":"2024-09-18T11:31:02.757606+0000","last_undegraded":"2024-09-18T11:31:02.757606+0000","last_fullsized":"2024-09-18T11:31:02.757606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:11.123483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019127767,"stat_sum":{"num_bytes":15633962,"num_objects":10,"num_object_clones":6,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":155,"num_read_kb":161567,"num_write":86,"num_write_kb":23057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"4.e","version":"79'99","reported_seq":291,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.685797+0000","last_change":"2024-09-18T11:31:02.397939+0000","last_active":"2024-09-18T11:31:02.685797+0000","last_peered":"2024-09-18T11:31:02.685797+0000","last_clean":"2024-09-18T11:31:02.685797+0000","last_became_active":"2024-09-18T11:30:02.073185+0000","last_became_peered":"2024-09-18T11:30:02.073185+0000","last_unstale":"2024-09-18T11:31:02.685797+0000","last_undegraded":"2024-09-18T11:31:02.685797+0000","last_fullsized":"2024-09-18T11:31:02.685797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:12.655066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038824889999999998,"stat_sum":{"num_bytes":15730974,"num_objects":6,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":164,"num_read_kb":182460,"num_write":45,"num_write_kb":12076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":5,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"4.d","version":"79'133","reported_seq":341,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.683251+0000","last_change":"2024-09-18T11:31:02.405361+0000","last_active":"2024-09-18T11:31:02.683251+0000","last_peered":"2024-09-18T11:31:02.683251+0000","last_clean":"2024-09-18T11:31:02.683251+0000","last_became_active":"2024-09-18T11:30:02.334498+0000","last_became_peered":"2024-09-18T11:30:02.334498+0000","last_unstale":"2024-09-18T11:31:02.683251+0000","last_undegraded":"2024-09-18T11:31:02.683251+0000","last_fullsized":"2024-09-18T11:31:02.683251+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:10.245504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011210894000000001,"stat_sum":{"num_bytes":17897939,"num_objects":9,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":217,"num_read_kb":236686,"num_write":69,"num_write_kb":19979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":8,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"4.c","version":"79'54","reported_seq":217,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.687172+0000","last_change":"2024-09-18T11:31:02.399810+0000","last_active":"2024-09-18T11:31:02.687172+0000","last_peered":"2024-09-18T11:31:02.687172+0000","last_clean":"2024-09-18T11:31:02.687172+0000","last_became_active":"2024-09-18T11:30:02.073883+0000","last_became_peered":"2024-09-18T11:30:02.073883+0000","last_unstale":"2024-09-18T11:31:02.687172+0000","last_undegraded":"2024-09-18T11:31:02.687172+0000","last_fullsized":"2024-09-18T11:31:02.687172+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:20.359490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005805279,"stat_sum":{"num_bytes":8197975,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":67,"num_read_kb":69995,"num_write":25,"num_write_kb":6106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"4.b","version":"79'215","reported_seq":457,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.735216+0000","last_change":"2024-09-18T11:31:02.419554+0000","last_active":"2024-09-18T11:31:02.735216+0000","last_peered":"2024-09-18T11:31:02.735216+0000","last_clean":"2024-09-18T11:31:02.735216+0000","last_became_active":"2024-09-18T11:30:50.333230+0000","last_became_peered":"2024-09-18T11:30:50.333230+0000","last_unstale":"2024-09-18T11:31:02.735216+0000","last_undegraded":"2024-09-18T11:31:02.735216+0000","last_fullsized":"2024-09-18T11:31:02.735216+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:20.199941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01681738,"stat_sum":{"num_bytes":35361703,"num_objects":17,"num_object_clones":11,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":220,"num_read_kb":214577,"num_write":116,"num_write_kb":32298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":17763749,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":9,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"4.a","version":"79'113","reported_seq":299,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.427542+0000","last_change":"2024-09-18T11:31:02.405986+0000","last_active":"2024-09-18T11:31:02.427542+0000","last_peered":"2024-09-18T11:31:02.427542+0000","last_clean":"2024-09-18T11:31:02.427542+0000","last_became_active":"2024-09-18T11:30:02.075538+0000","last_became_peered":"2024-09-18T11:30:02.075538+0000","last_unstale":"2024-09-18T11:31:02.427542+0000","last_undegraded":"2024-09-18T11:31:02.427542+0000","last_fullsized":"2024-09-18T11:31:02.427542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:26.428990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009454166,"stat_sum":{"num_bytes":13565104,"num_objects":8,"num_object_clones":5,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":177,"num_read_kb":249814,"num_write":50,"num_write_kb":14990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":6,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"4.9","version":"79'45","reported_seq":197,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.413709+0000","last_change":"2024-09-18T11:31:02.413902+0000","last_active":"2024-09-18T11:31:02.413709+0000","last_peered":"2024-09-18T11:31:02.413709+0000","last_clean":"2024-09-18T11:31:02.413709+0000","last_became_active":"2024-09-18T11:30:02.074414+0000","last_became_peered":"2024-09-18T11:30:02.074414+0000","last_unstale":"2024-09-18T11:31:02.413709+0000","last_undegraded":"2024-09-18T11:31:02.413709+0000","last_fullsized":"2024-09-18T11:31:02.413709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019722076000000002,"stat_sum":{"num_bytes":7323711,"num_objects":3,"num_object_clones":2,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":43640,"num_write":27,"num_write_kb":8449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"4.8","version":"79'30","reported_seq":173,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.402126+0000","last_change":"2024-09-18T11:31:02.402318+0000","last_active":"2024-09-18T11:31:02.402126+0000","last_peered":"2024-09-18T11:31:02.402126+0000","last_clean":"2024-09-18T11:31:02.402126+0000","last_became_active":"2024-09-18T11:30:02.076172+0000","last_became_peered":"2024-09-18T11:30:02.076172+0000","last_unstale":"2024-09-18T11:31:02.402126+0000","last_undegraded":"2024-09-18T11:31:02.402126+0000","last_fullsized":"2024-09-18T11:31:02.402126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:29.097553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001762069,"stat_sum":{"num_bytes":4916668,"num_objects":4,"num_object_clones":3,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":23,"num_read_kb":15209,"num_write":13,"num_write_kb":3671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"4.5","version":"79'147","reported_seq":347,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.402115+0000","last_change":"2024-09-18T11:31:02.402316+0000","last_active":"2024-09-18T11:31:02.402115+0000","last_peered":"2024-09-18T11:31:02.402115+0000","last_clean":"2024-09-18T11:31:02.402115+0000","last_became_active":"2024-09-18T11:30:02.074697+0000","last_became_peered":"2024-09-18T11:30:02.074697+0000","last_unstale":"2024-09-18T11:31:02.402115+0000","last_undegraded":"2024-09-18T11:31:02.402115+0000","last_fullsized":"2024-09-18T11:31:02.402115+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.577548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056736490000000002,"stat_sum":{"num_bytes":17403447,"num_objects":9,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":194,"num_read_kb":211919,"num_write":79,"num_write_kb":20300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"1.0","version":"16'72","reported_seq":188,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.390551+0000","last_change":"2024-09-18T11:30:57.650616+0000","last_active":"2024-09-18T11:31:01.390551+0000","last_peered":"2024-09-18T11:31:01.390551+0000","last_clean":"2024-09-18T11:31:01.390551+0000","last_became_active":"2024-09-18T11:30:50.741897+0000","last_became_peered":"2024-09-18T11:30:50.741897+0000","last_unstale":"2024-09-18T11:31:01.390551+0000","last_undegraded":"2024-09-18T11:31:01.390551+0000","last_fullsized":"2024-09-18T11:31:01.390551+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'72","last_scrub_stamp":"2024-09-18T11:30:57.650532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_clean_scrub_stamp":"2024-09-18T11:30:57.650532+0000","objects_scrubbed":2,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:58.577847+0000","scrub_duration":0.0016372979999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":75,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.390629+0000","last_change":"2024-09-18T11:29:52.047782+0000","last_active":"2024-09-18T11:31:01.390629+0000","last_peered":"2024-09-18T11:31:01.390629+0000","last_clean":"2024-09-18T11:31:01.390629+0000","last_became_active":"2024-09-18T11:29:50.028407+0000","last_became_peered":"2024-09-18T11:29:50.028407+0000","last_unstale":"2024-09-18T11:31:01.390629+0000","last_undegraded":"2024-09-18T11:31:01.390629+0000","last_fullsized":"2024-09-18T11:31:01.390629+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.851663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"77'126","reported_seq":198,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.385209+0000","last_change":"2024-09-18T11:30:01.077948+0000","last_active":"2024-09-18T11:31:01.385209+0000","last_peered":"2024-09-18T11:31:01.385209+0000","last_clean":"2024-09-18T11:31:01.385209+0000","last_became_active":"2024-09-18T11:30:01.077612+0000","last_became_peered":"2024-09-18T11:30:01.077612+0000","last_unstale":"2024-09-18T11:31:01.385209+0000","last_undegraded":"2024-09-18T11:31:01.385209+0000","last_fullsized":"2024-09-18T11:31:01.385209+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:30:00.059457+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:11.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27212175,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":100,"num_read_kb":2402,"num_write":143,"num_write_kb":51344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"79'62","reported_seq":243,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.755687+0000","last_change":"2024-09-18T11:31:02.416619+0000","last_active":"2024-09-18T11:31:02.755687+0000","last_peered":"2024-09-18T11:31:02.755687+0000","last_clean":"2024-09-18T11:31:02.755687+0000","last_became_active":"2024-09-18T11:30:50.743510+0000","last_became_peered":"2024-09-18T11:30:50.743510+0000","last_unstale":"2024-09-18T11:31:02.755687+0000","last_undegraded":"2024-09-18T11:31:02.755687+0000","last_fullsized":"2024-09-18T11:31:02.755687+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.427753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013723536999999999,"stat_sum":{"num_bytes":8434171,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":80,"num_read_kb":66751,"num_write":33,"num_write_kb":7919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":2949092,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":75,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.390468+0000","last_change":"2024-09-18T11:29:52.420488+0000","last_active":"2024-09-18T11:31:01.390468+0000","last_peered":"2024-09-18T11:31:01.390468+0000","last_clean":"2024-09-18T11:31:01.390468+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:31:01.390468+0000","last_undegraded":"2024-09-18T11:31:01.390468+0000","last_fullsized":"2024-09-18T11:31:01.390468+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:02.759929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'114","reported_seq":275,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.393954+0000","last_change":"2024-09-18T11:30:50.488394+0000","last_active":"2024-09-18T11:31:02.393954+0000","last_peered":"2024-09-18T11:31:02.393954+0000","last_clean":"2024-09-18T11:31:02.393954+0000","last_became_active":"2024-09-18T11:30:50.329599+0000","last_became_peered":"2024-09-18T11:30:50.329599+0000","last_unstale":"2024-09-18T11:31:02.393954+0000","last_undegraded":"2024-09-18T11:31:02.393954+0000","last_fullsized":"2024-09-18T11:31:02.393954+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:30:00.059457+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:27.886849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33501848,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":102,"num_read_kb":7881,"num_write":123,"num_write_kb":53221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":20418189,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"79'112","reported_seq":327,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.425041+0000","last_change":"2024-09-18T11:31:02.425246+0000","last_active":"2024-09-18T11:31:02.425041+0000","last_peered":"2024-09-18T11:31:02.425041+0000","last_clean":"2024-09-18T11:31:02.425041+0000","last_became_active":"2024-09-18T11:30:02.076465+0000","last_became_peered":"2024-09-18T11:30:02.076465+0000","last_unstale":"2024-09-18T11:31:02.425041+0000","last_undegraded":"2024-09-18T11:31:02.425041+0000","last_fullsized":"2024-09-18T11:31:02.425041+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:15.876699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.022014636000000001,"stat_sum":{"num_bytes":21625816,"num_objects":8,"num_object_clones":3,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":217,"num_read_kb":229052,"num_write":72,"num_write_kb":19025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":76,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.387578+0000","last_change":"2024-09-18T11:29:52.348137+0000","last_active":"2024-09-18T11:31:01.387578+0000","last_peered":"2024-09-18T11:31:01.387578+0000","last_clean":"2024-09-18T11:31:01.387578+0000","last_became_active":"2024-09-18T11:29:50.180029+0000","last_became_peered":"2024-09-18T11:29:50.180029+0000","last_unstale":"2024-09-18T11:31:01.387578+0000","last_undegraded":"2024-09-18T11:31:01.387578+0000","last_fullsized":"2024-09-18T11:31:01.387578+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.269389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023063399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"78'83","reported_seq":150,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.424433+0000","last_change":"2024-09-18T11:30:01.071704+0000","last_active":"2024-09-18T11:31:01.424433+0000","last_peered":"2024-09-18T11:31:01.424433+0000","last_clean":"2024-09-18T11:31:01.424433+0000","last_became_active":"2024-09-18T11:30:01.071435+0000","last_became_peered":"2024-09-18T11:30:01.071435+0000","last_unstale":"2024-09-18T11:31:01.424433+0000","last_undegraded":"2024-09-18T11:31:01.424433+0000","last_fullsized":"2024-09-18T11:31:01.424433+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:30:00.059457+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14444376,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":66,"num_read_kb":140,"num_write":88,"num_write_kb":29433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"79'109","reported_seq":278,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.408380+0000","last_change":"2024-09-18T11:31:02.408609+0000","last_active":"2024-09-18T11:31:02.408380+0000","last_peered":"2024-09-18T11:31:02.408380+0000","last_clean":"2024-09-18T11:31:02.408380+0000","last_became_active":"2024-09-18T11:30:02.334751+0000","last_became_peered":"2024-09-18T11:30:02.334751+0000","last_unstale":"2024-09-18T11:31:02.408380+0000","last_undegraded":"2024-09-18T11:31:02.408380+0000","last_fullsized":"2024-09-18T11:31:02.408380+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:20.359490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060395900000000001,"stat_sum":{"num_bytes":13664458,"num_objects":7,"num_object_clones":4,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":111,"num_read_kb":119178,"num_write":62,"num_write_kb":17894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":76,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.390539+0000","last_change":"2024-09-18T11:29:52.048389+0000","last_active":"2024-09-18T11:31:01.390539+0000","last_peered":"2024-09-18T11:31:01.390539+0000","last_clean":"2024-09-18T11:31:01.390539+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:31:01.390539+0000","last_undegraded":"2024-09-18T11:31:01.390539+0000","last_fullsized":"2024-09-18T11:31:01.390539+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:13.358402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"77'99","reported_seq":249,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.383245+0000","last_change":"2024-09-18T11:30:50.864329+0000","last_active":"2024-09-18T11:31:01.383245+0000","last_peered":"2024-09-18T11:31:01.383245+0000","last_clean":"2024-09-18T11:31:01.383245+0000","last_became_active":"2024-09-18T11:30:50.741594+0000","last_became_peered":"2024-09-18T11:30:50.741594+0000","last_unstale":"2024-09-18T11:31:01.383245+0000","last_undegraded":"2024-09-18T11:31:01.383245+0000","last_fullsized":"2024-09-18T11:31:01.383245+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:30:00.059457+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:05.802711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26632899,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":80,"num_read_kb":1690,"num_write":107,"num_write_kb":42852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":16290641,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"79'63","reported_seq":215,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.413693+0000","last_change":"2024-09-18T11:31:02.413904+0000","last_active":"2024-09-18T11:31:02.413693+0000","last_peered":"2024-09-18T11:31:02.413693+0000","last_clean":"2024-09-18T11:31:02.413693+0000","last_became_active":"2024-09-18T11:30:02.076874+0000","last_became_peered":"2024-09-18T11:30:02.076874+0000","last_unstale":"2024-09-18T11:31:02.413693+0000","last_undegraded":"2024-09-18T11:31:02.413693+0000","last_fullsized":"2024-09-18T11:31:02.413693+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:06.825500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011070398,"stat_sum":{"num_bytes":6541447,"num_objects":4,"num_object_clones":3,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":49,"num_read_kb":47757,"num_write":40,"num_write_kb":11590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":86,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.390558+0000","last_change":"2024-09-18T11:30:50.331231+0000","last_active":"2024-09-18T11:31:01.390558+0000","last_peered":"2024-09-18T11:31:01.390558+0000","last_clean":"2024-09-18T11:31:01.390558+0000","last_became_active":"2024-09-18T11:30:50.331026+0000","last_became_peered":"2024-09-18T11:30:50.331026+0000","last_unstale":"2024-09-18T11:31:01.390558+0000","last_undegraded":"2024-09-18T11:31:01.390558+0000","last_fullsized":"2024-09-18T11:31:01.390558+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:09.742125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"79'82","reported_seq":261,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.720894+0000","last_change":"2024-09-18T11:31:02.405543+0000","last_active":"2024-09-18T11:31:02.720894+0000","last_peered":"2024-09-18T11:31:02.720894+0000","last_clean":"2024-09-18T11:31:02.720894+0000","last_became_active":"2024-09-18T11:30:50.333722+0000","last_became_peered":"2024-09-18T11:30:50.333722+0000","last_unstale":"2024-09-18T11:31:02.720894+0000","last_undegraded":"2024-09-18T11:31:02.720894+0000","last_fullsized":"2024-09-18T11:31:02.720894+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:25.884327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049286369999999996,"stat_sum":{"num_bytes":9320535,"num_objects":6,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":64,"num_read_kb":62957,"num_write":43,"num_write_kb":11223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":5737174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":5,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":86,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.390516+0000","last_change":"2024-09-18T11:30:50.327769+0000","last_active":"2024-09-18T11:31:01.390516+0000","last_peered":"2024-09-18T11:31:01.390516+0000","last_clean":"2024-09-18T11:31:01.390516+0000","last_became_active":"2024-09-18T11:30:50.327483+0000","last_became_peered":"2024-09-18T11:30:50.327483+0000","last_unstale":"2024-09-18T11:31:01.390516+0000","last_undegraded":"2024-09-18T11:31:01.390516+0000","last_fullsized":"2024-09-18T11:31:01.390516+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:29:48.952943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:13.119942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"79'115","reported_seq":286,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.408568+0000","last_change":"2024-09-18T11:31:02.408792+0000","last_active":"2024-09-18T11:31:02.408568+0000","last_peered":"2024-09-18T11:31:02.408568+0000","last_clean":"2024-09-18T11:31:02.408568+0000","last_became_active":"2024-09-18T11:30:02.072165+0000","last_became_peered":"2024-09-18T11:30:02.072165+0000","last_unstale":"2024-09-18T11:31:02.408568+0000","last_undegraded":"2024-09-18T11:31:02.408568+0000","last_fullsized":"2024-09-18T11:31:02.408568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:04.270484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006340844,"stat_sum":{"num_bytes":17555791,"num_objects":7,"num_object_clones":4,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":108,"num_read_kb":116510,"num_write":78,"num_write_kb":22821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":85,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.390536+0000","last_change":"2024-09-18T11:30:57.765542+0000","last_active":"2024-09-18T11:31:01.390536+0000","last_peered":"2024-09-18T11:31:01.390536+0000","last_clean":"2024-09-18T11:31:01.390536+0000","last_became_active":"2024-09-18T11:29:50.028674+0000","last_became_peered":"2024-09-18T11:29:50.028674+0000","last_unstale":"2024-09-18T11:31:01.390536+0000","last_undegraded":"2024-09-18T11:31:01.390536+0000","last_fullsized":"2024-09-18T11:31:01.390536+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:57.765475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:30:57.765475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:20.322582+0000","scrub_duration":0.001835167,"objects_trimmed":0,"snaptrim_duration":0.00031621200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"79'157","reported_seq":339,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-18T11:31:02.408451+0000","last_change":"2024-09-18T11:31:02.410962+0000","last_active":"2024-09-18T11:31:02.408451+0000","last_peered":"2024-09-18T11:31:02.408451+0000","last_clean":"2024-09-18T11:31:02.408451+0000","last_became_active":"2024-09-18T11:30:02.075529+0000","last_became_peered":"2024-09-18T11:30:02.075529+0000","last_unstale":"2024-09-18T11:31:02.408451+0000","last_undegraded":"2024-09-18T11:31:02.408451+0000","last_fullsized":"2024-09-18T11:31:02.408451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:30:01.063520+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:21.261787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.016883939000000001,"stat_sum":{"num_bytes":18124559,"num_objects":9,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":154,"num_read_kb":179334,"num_write":80,"num_write_kb":20217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":7,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"13","length":"2"},{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1b","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":85,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-18T11:31:01.385358+0000","last_change":"2024-09-18T11:30:57.053072+0000","last_active":"2024-09-18T11:31:01.385358+0000","last_peered":"2024-09-18T11:31:01.385358+0000","last_clean":"2024-09-18T11:31:01.385358+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:31:01.385358+0000","last_undegraded":"2024-09-18T11:31:01.385358+0000","last_fullsized":"2024-09-18T11:31:01.385358+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:57.052994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:30:57.052994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:58.065434+0000","scrub_duration":0.0012785050000000001,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":231298260,"num_objects":115,"num_object_clones":67,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":7,"num_read":2037,"num_read_kb":2207406,"num_write":918,"num_write_kb":251615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":26450015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":68,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":329928704,"data_stored":417097862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1680,"ondisk_log_size":1680,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":101791298,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":348,"num_read_kb":12113,"num_write":461,"num_write_kb":176850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":36708830,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":153198592,"data_stored":152833186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":422,"ondisk_log_size":422,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320088,"kb_used_data":22724,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94051752,"statfs":{"total":96636764160,"available":96308994048,"internally_reserved":0,"allocated":23269376,"data_stored":28544444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":1,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.40400000000000003}]},{"osd":2,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.39100000000000001}]},{"osd":3,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.41799999999999998}]},{"osd":4,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.314},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.32700000000000001}]},{"osd":5,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.28699999999999998}]},{"osd":6,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.372},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.30299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":339648,"kb_used_data":42284,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94032192,"statfs":{"total":96636764160,"available":96288964608,"internally_reserved":0,"allocated":43298816,"data_stored":54723931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Wed Sep 18 11:30:50 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.45200000000000001}]},{"osd":2,"last update":"Wed Sep 18 11:30:50 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.496},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.46999999999999997}]},{"osd":3,"last update":"Wed Sep 18 11:30:50 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.436}]},{"osd":4,"last update":"Wed Sep 18 11:30:50 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.53900000000000003}]},{"osd":5,"last update":"Wed Sep 18 11:30:50 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.51200000000000001}]},{"osd":7,"last update":"Wed Sep 18 11:30:50 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.57299999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":351916,"kb_used_data":54552,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94019924,"statfs":{"total":96636764160,"available":96276402176,"internally_reserved":0,"allocated":55861248,"data_stored":66302023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":1,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.375}]},{"osd":2,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.38500000000000001}]},{"osd":3,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.44500000000000001}]},{"osd":4,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.32300000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.30299999999999999}]},{"osd":7,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.40699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413460,"kb_used_data":110772,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93958380,"statfs":{"total":96636764160,"available":96213381120,"internally_reserved":0,"allocated":113430528,"data_stored":129913881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.42799999999999999}]},{"osd":1,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.55100000000000005}]},{"osd":3,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.59299999999999997}]},{"osd":5,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.35899999999999999}]},{"osd":6,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.438}]},{"osd":7,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.39800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":357792,"kb_used_data":60428,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94014048,"statfs":{"total":96636764160,"available":96270385152,"internally_reserved":0,"allocated":61878272,"data_stored":65229808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.318},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.34999999999999998}]},{"osd":1,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.214}]},{"osd":2,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.24399999999999999}]},{"osd":4,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.61299999999999999}]},{"osd":5,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.63600000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.65800000000000003}]},{"osd":7,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.55000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":330628,"kb_used_data":33268,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94041212,"statfs":{"total":96636764160,"available":96298201088,"internally_reserved":0,"allocated":34066432,"data_stored":40112417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.253}]},{"osd":1,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.29299999999999998}]},{"osd":3,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.23799999999999999}]},{"osd":4,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.376}]},{"osd":5,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.44},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.45400000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.32300000000000001}]},{"osd":7,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.42399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":391456,"kb_used_data":88808,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93980384,"statfs":{"total":96636764160,"available":96235913216,"internally_reserved":0,"allocated":90939392,"data_stored":110686258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.224}]},{"osd":2,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.17399999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.26600000000000001}]},{"osd":3,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.23699999999999999}]},{"osd":4,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.32300000000000001}]},{"osd":5,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.438},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.312}]},{"osd":6,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.39000000000000001}]},{"osd":7,"last update":"Wed Sep 18 11:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.379}]}]},{"osd":0,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36782080,"data_stored":36708830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8220672,"data_stored":8202102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8220672,"data_stored":8202102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33914880,"data_stored":33819019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":35041280,"data_stored":34963694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":14946304,"data_stored":14896382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16072704,"data_stored":16041057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25010176,"data_stored":27495860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81805312,"data_stored":102017381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24973312,"data_stored":31453239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28524544,"data_stored":35601225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":77852672,"data_stored":97009634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":41369600,"data_stored":53469581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":27222016,"data_stored":39976970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":23171072,"data_stored":30073972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:31:03.923 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T11:31:03.923 INFO:tasks.thrashosds.thrasher:Waiting for trim 2024-09-18T11:31:04.005 INFO:tasks.rados.rados.0.smithi119.stdout:1163: done (9 left) 2024-09-18T11:31:04.005 INFO:tasks.rados.rados.0.smithi119.stdout:1164: done (8 left) 2024-09-18T11:31:04.005 INFO:tasks.rados.rados.0.smithi119.stdout:1165: done (7 left) 2024-09-18T11:31:04.006 INFO:tasks.rados.rados.0.smithi119.stdout:1167: done (6 left) 2024-09-18T11:31:04.006 INFO:tasks.rados.rados.0.smithi119.stdout:1168: done (5 left) 2024-09-18T11:31:04.006 INFO:tasks.rados.rados.0.smithi119.stdout:1169: done (4 left) 2024-09-18T11:31:04.006 INFO:tasks.rados.rados.0.smithi119.stdout:1170: done (3 left) 2024-09-18T11:31:04.006 INFO:tasks.rados.rados.0.smithi119.stdout:1171: done (2 left) 2024-09-18T11:31:04.006 INFO:tasks.rados.rados.0.smithi119.stdout:1172: done (1 left) 2024-09-18T11:31:04.006 INFO:tasks.rados.rados.0.smithi119.stdout:1173: done (0 left) 2024-09-18T11:31:04.006 INFO:tasks.rados.rados.0.smithi119.stdout:1174: tier_evict oid 15 2024-09-18T11:31:04.006 INFO:tasks.rados.rados.0.smithi119.stdout:1174: tier_evict oid 15 snap -1 2024-09-18T11:31:04.006 INFO:tasks.rados.rados.0.smithi119.stdout:1175: write (excl) oid 21 current snap is 24 2024-09-18T11:31:04.006 INFO:tasks.rados.rados.0.smithi119.stdout:1175: seq_num 213 ranges {779891=485921,1947841=685395,3383215=196676} 2024-09-18T11:31:04.011 INFO:tasks.rados.rados.0.smithi119.stdout:1175: writing smithi11939506-21 from 779891 to 1265812 tid 1 2024-09-18T11:31:04.017 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:04.015+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:04.019 INFO:tasks.rados.rados.0.smithi119.stdout:1175: writing smithi11939506-21 from 1947841 to 2633236 tid 2 2024-09-18T11:31:04.025 INFO:tasks.rados.rados.0.smithi119.stdout:1175: writing smithi11939506-21 from 3383215 to 3579891 tid 3 2024-09-18T11:31:04.025 INFO:tasks.rados.rados.0.smithi119.stdout:1176: read oid 39 snap -1 2024-09-18T11:31:04.025 INFO:tasks.rados.rados.0.smithi119.stdout:1176: expect deleted 2024-09-18T11:31:04.025 INFO:tasks.rados.rados.0.smithi119.stdout:1177: read oid 30 snap -1 2024-09-18T11:31:04.025 INFO:tasks.rados.rados.0.smithi119.stdout:1177: expect (ObjNum 174 snap 20 seq_num 174) 2024-09-18T11:31:04.025 INFO:tasks.rados.rados.0.smithi119.stdout:1178: snap_create 2024-09-18T11:31:04.025 INFO:tasks.rados.rados.0.smithi119.stdout:1174: got (22) Invalid argument 2024-09-18T11:31:04.025 INFO:tasks.rados.rados.0.smithi119.stdout:1175: finishing write tid 1 to smithi11939506-21 2024-09-18T11:31:04.025 INFO:tasks.rados.rados.0.smithi119.stdout:1175: finishing write tid 2 to smithi11939506-21 2024-09-18T11:31:04.026 INFO:tasks.rados.rados.0.smithi119.stdout:1177: expect (ObjNum 174 snap 20 seq_num 174) 2024-09-18T11:31:04.039 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:31:04.060 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:04.060 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:04.068 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:31:04.117 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:04.115+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:04.138 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:04.154 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:04.154 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:04.169 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:04.169 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:04.183 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:04.201 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:04.201 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:04.207 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:04.207 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:04.214 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:04.214 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:04.216 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:31:04.218 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:04.217+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:04.226 INFO:tasks.rados.rados.0.smithi119.stdout:1175: finishing write tid 3 to smithi11939506-21 2024-09-18T11:31:04.227 INFO:tasks.rados.rados.0.smithi119.stdout:1175: finishing write tid 4 to smithi11939506-21 2024-09-18T11:31:04.227 INFO:tasks.rados.rados.0.smithi119.stdout:1175: finishing write tid 5 to smithi11939506-21 2024-09-18T11:31:04.227 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 148 (ObjNum 213 snap 24 seq_num 213) dirty exists 2024-09-18T11:31:04.227 INFO:tasks.rados.rados.0.smithi119.stdout:1175: left oid 21 (ObjNum 213 snap 24 seq_num 213) 2024-09-18T11:31:04.229 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:04.229 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:04.244 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:04.245 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:04.247 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:04.248 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:04.316 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:04.316 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:04.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:04.316 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:04.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.316 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:31:04.316 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:04.317 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 74.287328303999999, 2024-09-18T11:31:04.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073601550000000002, 2024-09-18T11:31:04.317 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.317 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.317 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.317 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:31:04.317 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:31:04.317 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1 2024-09-18T11:31:04.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.317 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:04.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:04.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026524+0000", 2024-09-18T11:31:04.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999905 2024-09-18T11:31:04.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026541+0000", 2024-09-18T11:31:04.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7070000000000001e-05 2024-09-18T11:31:04.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026554+0000", 2024-09-18T11:31:04.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.288e-05 2024-09-18T11:31:04.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026564+0000", 2024-09-18T11:31:04.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0197000000000001e-05 2024-09-18T11:31:04.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026826+0000", 2024-09-18T11:31:04.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026212400000000003 2024-09-18T11:31:04.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:31:04.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026839+0000", 2024-09-18T11:31:04.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2831e-05 2024-09-18T11:31:04.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028605+0000", 2024-09-18T11:31:04.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001765794 2024-09-18T11:31:04.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028709+0000", 2024-09-18T11:31:04.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104348 2024-09-18T11:31:04.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:04.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028824+0000", 2024-09-18T11:31:04.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011527399999999999 2024-09-18T11:31:04.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:04.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030319+0000", 2024-09-18T11:31:04.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001494439 2024-09-18T11:31:04.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:04.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033805+0000", 2024-09-18T11:31:04.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034857820000000002 2024-09-18T11:31:04.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033856+0000", 2024-09-18T11:31:04.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1687999999999997e-05 2024-09-18T11:31:04.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033894+0000", 2024-09-18T11:31:04.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7481000000000002e-05 2024-09-18T11:31:04.323 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.323 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.323 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.324 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T11:31:04.324 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:31:04.324 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 72.268049872999995, 2024-09-18T11:31:04.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055107869999999996, 2024-09-18T11:31:04.324 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.324 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.324 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.324 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:31:04.324 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:31:04.324 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 7 2024-09-18T11:31:04.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.324 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:31:04.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:31:04.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045810+0000", 2024-09-18T11:31:04.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:04.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045820+0000", 2024-09-18T11:31:04.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0215e-05 2024-09-18T11:31:04.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045838+0000", 2024-09-18T11:31:04.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7849000000000001e-05 2024-09-18T11:31:04.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045845+0000", 2024-09-18T11:31:04.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4229999999999999e-06 2024-09-18T11:31:04.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045898+0000", 2024-09-18T11:31:04.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2537999999999997e-05 2024-09-18T11:31:04.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:04.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045904+0000", 2024-09-18T11:31:04.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4019999999999999e-06 2024-09-18T11:31:04.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.328 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047663+0000", 2024-09-18T11:31:04.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017584079999999999 2024-09-18T11:31:04.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.328 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047736+0000", 2024-09-18T11:31:04.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3509000000000004e-05 2024-09-18T11:31:04.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.328 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:04.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047857+0000", 2024-09-18T11:31:04.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012037800000000001 2024-09-18T11:31:04.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:04.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.050868+0000", 2024-09-18T11:31:04.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030111420000000001 2024-09-18T11:31:04.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:04.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051239+0000", 2024-09-18T11:31:04.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037141000000000002 2024-09-18T11:31:04.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051288+0000", 2024-09-18T11:31:04.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9051000000000002e-05 2024-09-18T11:31:04.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051323+0000", 2024-09-18T11:31:04.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4545000000000002e-05 2024-09-18T11:31:04.330 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.330 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.330 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.330 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:905 4.7 e31/22 4:e73ba68f:::smithi11939506-1:head v 31'32, mlcod=30'29)", 2024-09-18T11:31:04.330 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:31:04.330 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.172393661999998, 2024-09-18T11:31:04.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048351990000000001, 2024-09-18T11:31:04.331 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.331 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.331 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:31:04.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:31:04.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.141468+0000", 2024-09-18T11:31:04.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144912+0000", 2024-09-18T11:31:04.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003444 2024-09-18T11:31:04.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144921+0000", 2024-09-18T11:31:04.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1900000000000001e-06 2024-09-18T11:31:04.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144927+0000", 2024-09-18T11:31:04.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4940000000000003e-06 2024-09-18T11:31:04.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144952+0000", 2024-09-18T11:31:04.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5845000000000001e-05 2024-09-18T11:31:04.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.144966+0000", 2024-09-18T11:31:04.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.396e-05 2024-09-18T11:31:04.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.146274+0000", 2024-09-18T11:31:04.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013073550000000001 2024-09-18T11:31:04.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.146303+0000", 2024-09-18T11:31:04.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9754e-05 2024-09-18T11:31:04.335 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.335 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.335 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.335 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:911 4.3 e31/22 4:cc46532a:::smithi11939506-33:head v 31'24, mlcod=30'21)", 2024-09-18T11:31:04.335 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.166801+0000", 2024-09-18T11:31:04.335 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.147061336, 2024-09-18T11:31:04.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048144349999999997, 2024-09-18T11:31:04.335 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.335 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.335 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.166801+0000", 2024-09-18T11:31:04.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.166801+0000", 2024-09-18T11:31:04.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.166799+0000", 2024-09-18T11:31:04.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:04.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167413+0000", 2024-09-18T11:31:04.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000613259 2024-09-18T11:31:04.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167431+0000", 2024-09-18T11:31:04.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7906000000000001e-05 2024-09-18T11:31:04.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167438+0000", 2024-09-18T11:31:04.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.883e-06 2024-09-18T11:31:04.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167468+0000", 2024-09-18T11:31:04.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0168e-05 2024-09-18T11:31:04.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.167483+0000", 2024-09-18T11:31:04.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4997999999999999e-05 2024-09-18T11:31:04.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.171574+0000", 2024-09-18T11:31:04.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040909589999999999 2024-09-18T11:31:04.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.171615+0000", 2024-09-18T11:31:04.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1371000000000001e-05 2024-09-18T11:31:04.339 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.339 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.339 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.339 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1118 3.2 e38/21 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head v 38'76, mlcod=38'74)", 2024-09-18T11:31:04.339 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:04.340 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 44.365918151999999, 2024-09-18T11:31:04.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006048688, 2024-09-18T11:31:04.340 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.340 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.340 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:04.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:04.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947941+0000", 2024-09-18T11:31:04.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:04.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949868+0000", 2024-09-18T11:31:04.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019267659999999999 2024-09-18T11:31:04.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949893+0000", 2024-09-18T11:31:04.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5239999999999999e-05 2024-09-18T11:31:04.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949901+0000", 2024-09-18T11:31:04.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8420000000000005e-06 2024-09-18T11:31:04.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949954+0000", 2024-09-18T11:31:04.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3464999999999997e-05 2024-09-18T11:31:04.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949974+0000", 2024-09-18T11:31:04.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9731000000000001e-05 2024-09-18T11:31:04.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953948+0000", 2024-09-18T11:31:04.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039735489999999998 2024-09-18T11:31:04.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953992+0000", 2024-09-18T11:31:04.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4700999999999998e-05 2024-09-18T11:31:04.344 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.344 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.344 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.344 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.13:29 3.2 e39/21 3:5fa8fcfa:::ec59520759824f0488c7a3ca135dacb324de2330e16b7f54053de5719c4e96b6:head v 39'81, mlcod=38'76)", 2024-09-18T11:31:04.344 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:31:04.344 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 43.632217676000003, 2024-09-18T11:31:04.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004755858, 2024-09-18T11:31:04.344 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.345 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.345 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:31:04.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:31:04.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681644+0000", 2024-09-18T11:31:04.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681961+0000", 2024-09-18T11:31:04.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00031696799999999999 2024-09-18T11:31:04.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681969+0000", 2024-09-18T11:31:04.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4800000000000001e-06 2024-09-18T11:31:04.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.681973+0000", 2024-09-18T11:31:04.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4819999999999999e-06 2024-09-18T11:31:04.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682007+0000", 2024-09-18T11:31:04.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3961999999999998e-05 2024-09-18T11:31:04.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.682033+0000", 2024-09-18T11:31:04.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6103999999999999e-05 2024-09-18T11:31:04.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686375+0000", 2024-09-18T11:31:04.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043422019999999999 2024-09-18T11:31:04.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.686400+0000", 2024-09-18T11:31:04.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5265e-05 2024-09-18T11:31:04.348 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.348 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.349 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.349 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1623 4.3 e56/22 4:c70c3162:::smithi11939506-35:head v 56'43, mlcod=48'40)", 2024-09-18T11:31:04.349 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.302560+0000", 2024-09-18T11:31:04.349 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.011302339, 2024-09-18T11:31:04.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046159060000000003, 2024-09-18T11:31:04.349 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.349 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.349 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.302560+0000", 2024-09-18T11:31:04.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.302560+0000", 2024-09-18T11:31:04.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.302559+0000", 2024-09-18T11:31:04.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303837+0000", 2024-09-18T11:31:04.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012781089999999999 2024-09-18T11:31:04.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303853+0000", 2024-09-18T11:31:04.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5551999999999999e-05 2024-09-18T11:31:04.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303862+0000", 2024-09-18T11:31:04.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0489999999999993e-06 2024-09-18T11:31:04.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303914+0000", 2024-09-18T11:31:04.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1697999999999999e-05 2024-09-18T11:31:04.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.303938+0000", 2024-09-18T11:31:04.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4369e-05 2024-09-18T11:31:04.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.307122+0000", 2024-09-18T11:31:04.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031841780000000002 2024-09-18T11:31:04.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.307176+0000", 2024-09-18T11:31:04.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3400999999999999e-05 2024-09-18T11:31:04.353 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.353 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.353 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.353 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1662 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'73, mlcod=51'69)", 2024-09-18T11:31:04.353 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:04.353 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.952313116999999, 2024-09-18T11:31:04.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059408539999999998, 2024-09-18T11:31:04.354 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.354 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.354 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:04.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:04.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:04.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361914+0000", 2024-09-18T11:31:04.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036569 2024-09-18T11:31:04.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361926+0000", 2024-09-18T11:31:04.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1960999999999999e-05 2024-09-18T11:31:04.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361937+0000", 2024-09-18T11:31:04.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0876999999999999e-05 2024-09-18T11:31:04.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361986+0000", 2024-09-18T11:31:04.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8652000000000001e-05 2024-09-18T11:31:04.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.362012+0000", 2024-09-18T11:31:04.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5854999999999999e-05 2024-09-18T11:31:04.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367460+0000", 2024-09-18T11:31:04.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054482100000000002 2024-09-18T11:31:04.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367490+0000", 2024-09-18T11:31:04.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9909000000000001e-05 2024-09-18T11:31:04.357 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.357 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.357 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.358 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1663 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'74, mlcod=51'69)", 2024-09-18T11:31:04.358 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364521+0000", 2024-09-18T11:31:04.358 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.949341119, 2024-09-18T11:31:04.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048541399999999998, 2024-09-18T11:31:04.358 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.358 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.358 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364521+0000", 2024-09-18T11:31:04.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364521+0000", 2024-09-18T11:31:04.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364520+0000", 2024-09-18T11:31:04.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364616+0000", 2024-09-18T11:31:04.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5569999999999995e-05 2024-09-18T11:31:04.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364625+0000", 2024-09-18T11:31:04.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3379999999999994e-06 2024-09-18T11:31:04.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364629+0000", 2024-09-18T11:31:04.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2490000000000003e-06 2024-09-18T11:31:04.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364664+0000", 2024-09-18T11:31:04.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5132999999999997e-05 2024-09-18T11:31:04.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364687+0000", 2024-09-18T11:31:04.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2691999999999999e-05 2024-09-18T11:31:04.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369349+0000", 2024-09-18T11:31:04.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046622950000000003 2024-09-18T11:31:04.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369375+0000", 2024-09-18T11:31:04.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5534e-05 2024-09-18T11:31:04.362 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.362 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.362 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.362 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1664 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'75, mlcod=51'69)", 2024-09-18T11:31:04.362 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:31:04.362 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.946974610000002, 2024-09-18T11:31:04.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051832229999999998, 2024-09-18T11:31:04.363 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.363 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.363 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:31:04.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:31:04.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:31:04.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366894+0000", 2024-09-18T11:31:04.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2219999999999996e-06 2024-09-18T11:31:04.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366906+0000", 2024-09-18T11:31:04.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2278e-05 2024-09-18T11:31:04.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366912+0000", 2024-09-18T11:31:04.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2430000000000002e-06 2024-09-18T11:31:04.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366952+0000", 2024-09-18T11:31:04.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9428999999999998e-05 2024-09-18T11:31:04.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366972+0000", 2024-09-18T11:31:04.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0554999999999999e-05 2024-09-18T11:31:04.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372045+0000", 2024-09-18T11:31:04.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050727319999999999 2024-09-18T11:31:04.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372071+0000", 2024-09-18T11:31:04.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5477e-05 2024-09-18T11:31:04.366 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.367 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.367 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.367 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:269 4.7 e58/22 4:e73ba68f:::smithi11939506-1:13 v 58'72, mlcod=57'69)", 2024-09-18T11:31:04.367 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:04.367 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.048481219999999, 2024-09-18T11:31:04.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065512569999999996, 2024-09-18T11:31:04.367 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.367 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.367 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:04.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:04.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265380+0000", 2024-09-18T11:31:04.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265388+0000", 2024-09-18T11:31:04.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.174e-06 2024-09-18T11:31:04.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265398+0000", 2024-09-18T11:31:04.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2599999999999994e-06 2024-09-18T11:31:04.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265402+0000", 2024-09-18T11:31:04.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0289999999999999e-06 2024-09-18T11:31:04.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265434+0000", 2024-09-18T11:31:04.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2511000000000001e-05 2024-09-18T11:31:04.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265447+0000", 2024-09-18T11:31:04.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2947e-05 2024-09-18T11:31:04.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271892+0000", 2024-09-18T11:31:04.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064445930000000002 2024-09-18T11:31:04.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271932+0000", 2024-09-18T11:31:04.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0303999999999999e-05 2024-09-18T11:31:04.371 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.371 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.372 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.372 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1788 4.8 4:1898b15d:::smithi11939506-25:head [write 453822~797061] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:04.372 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:04.372 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.398048636999999, 2024-09-18T11:31:04.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094305159999999999, 2024-09-18T11:31:04.372 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.372 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.372 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.372 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.372 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.372 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1788 2024-09-18T11:31:04.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.373 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:04.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:04.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915811+0000", 2024-09-18T11:31:04.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:04.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916608+0000", 2024-09-18T11:31:04.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079635000000000003 2024-09-18T11:31:04.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916624+0000", 2024-09-18T11:31:04.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6735000000000002e-05 2024-09-18T11:31:04.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916630+0000", 2024-09-18T11:31:04.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5709999999999997e-06 2024-09-18T11:31:04.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916674+0000", 2024-09-18T11:31:04.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4540999999999999e-05 2024-09-18T11:31:04.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916751+0000", 2024-09-18T11:31:04.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6695999999999999e-05 2024-09-18T11:31:04.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:04.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916872+0000", 2024-09-18T11:31:04.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120648 2024-09-18T11:31:04.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:04.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.919076+0000", 2024-09-18T11:31:04.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022046829999999998 2024-09-18T11:31:04.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:04.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925151+0000", 2024-09-18T11:31:04.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060740239999999999 2024-09-18T11:31:04.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925198+0000", 2024-09-18T11:31:04.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7896999999999999e-05 2024-09-18T11:31:04.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925244+0000", 2024-09-18T11:31:04.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5398999999999997e-05 2024-09-18T11:31:04.377 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.377 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.378 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.378 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1789 4.8 4:1898b15d:::smithi11939506-25:head [write 1760371~653366] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:04.378 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.921860+0000", 2024-09-18T11:31:04.378 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.392001952000001, 2024-09-18T11:31:04.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055904329999999997, 2024-09-18T11:31:04.378 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.378 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.378 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.378 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.378 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.378 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1789 2024-09-18T11:31:04.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.379 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.921860+0000", 2024-09-18T11:31:04.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.921860+0000", 2024-09-18T11:31:04.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.921859+0000", 2024-09-18T11:31:04.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:04.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922436+0000", 2024-09-18T11:31:04.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00057708300000000004 2024-09-18T11:31:04.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922450+0000", 2024-09-18T11:31:04.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4575e-05 2024-09-18T11:31:04.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922458+0000", 2024-09-18T11:31:04.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:31:04.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922508+0000", 2024-09-18T11:31:04.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0102999999999998e-05 2024-09-18T11:31:04.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922607+0000", 2024-09-18T11:31:04.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8813000000000004e-05 2024-09-18T11:31:04.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:04.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922814+0000", 2024-09-18T11:31:04.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000207488 2024-09-18T11:31:04.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:04.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925504+0000", 2024-09-18T11:31:04.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026899200000000002 2024-09-18T11:31:04.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:04.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927350+0000", 2024-09-18T11:31:04.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00184575 2024-09-18T11:31:04.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927404+0000", 2024-09-18T11:31:04.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4147000000000002e-05 2024-09-18T11:31:04.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927450+0000", 2024-09-18T11:31:04.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6480000000000002e-05 2024-09-18T11:31:04.383 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.383 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.384 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.384 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1790 4.8 4:1898b15d:::smithi11939506-25:head [setxattr _header (57) in=7b,truncate 2453822] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:04.384 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.922467+0000", 2024-09-18T11:31:04.384 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.391394531, 2024-09-18T11:31:04.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0052363219999999999, 2024-09-18T11:31:04.384 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.384 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.384 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.384 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.384 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.384 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1790 2024-09-18T11:31:04.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.385 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922467+0000", 2024-09-18T11:31:04.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922467+0000", 2024-09-18T11:31:04.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922467+0000", 2024-09-18T11:31:04.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922470+0000", 2024-09-18T11:31:04.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2459999999999999e-06 2024-09-18T11:31:04.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922474+0000", 2024-09-18T11:31:04.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:31:04.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922476+0000", 2024-09-18T11:31:04.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3410000000000001e-06 2024-09-18T11:31:04.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.924488+0000", 2024-09-18T11:31:04.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002012425 2024-09-18T11:31:04.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.924623+0000", 2024-09-18T11:31:04.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013435699999999999 2024-09-18T11:31:04.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:04.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.924870+0000", 2024-09-18T11:31:04.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00024748199999999999 2024-09-18T11:31:04.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:04.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927574+0000", 2024-09-18T11:31:04.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027032800000000002 2024-09-18T11:31:04.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:04.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927628+0000", 2024-09-18T11:31:04.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3909000000000002e-05 2024-09-18T11:31:04.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927661+0000", 2024-09-18T11:31:04.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3467999999999999e-05 2024-09-18T11:31:04.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927704+0000", 2024-09-18T11:31:04.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2749e-05 2024-09-18T11:31:04.389 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.389 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.390 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.390 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1791 4.8 4:1898b15d:::smithi11939506-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:04.390 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.922500+0000", 2024-09-18T11:31:04.390 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.39136186, 2024-09-18T11:31:04.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005360259, 2024-09-18T11:31:04.390 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.390 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:04.390 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.391 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.391 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.391 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1791 2024-09-18T11:31:04.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.391 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922500+0000", 2024-09-18T11:31:04.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922500+0000", 2024-09-18T11:31:04.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922500+0000", 2024-09-18T11:31:04.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922503+0000", 2024-09-18T11:31:04.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7009999999999998e-06 2024-09-18T11:31:04.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922509+0000", 2024-09-18T11:31:04.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1710000000000001e-06 2024-09-18T11:31:04.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922511+0000", 2024-09-18T11:31:04.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:31:04.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925283+0000", 2024-09-18T11:31:04.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027722659999999998 2024-09-18T11:31:04.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:04.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925376+0000", 2024-09-18T11:31:04.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3018999999999998e-05 2024-09-18T11:31:04.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927736+0000", 2024-09-18T11:31:04.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023600990000000001 2024-09-18T11:31:04.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927798+0000", 2024-09-18T11:31:04.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1197000000000004e-05 2024-09-18T11:31:04.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927860+0000", 2024-09-18T11:31:04.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2725999999999995e-05 2024-09-18T11:31:04.395 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.395 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.395 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.395 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1934 4.2 e65/22 4:425afe9c:::smithi11939506-29:head v 65'44, mlcod=60'40)", 2024-09-18T11:31:04.395 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:04.395 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 16.013869867, 2024-09-18T11:31:04.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064585119999999996, 2024-09-18T11:31:04.396 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.396 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.396 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:04.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:04.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299990+0000", 2024-09-18T11:31:04.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:04.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300789+0000", 2024-09-18T11:31:04.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079851499999999997 2024-09-18T11:31:04.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300806+0000", 2024-09-18T11:31:04.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7096999999999999e-05 2024-09-18T11:31:04.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300813+0000", 2024-09-18T11:31:04.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5079999999999999e-06 2024-09-18T11:31:04.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300870+0000", 2024-09-18T11:31:04.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7178e-05 2024-09-18T11:31:04.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300897+0000", 2024-09-18T11:31:04.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6284000000000001e-05 2024-09-18T11:31:04.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306410+0000", 2024-09-18T11:31:04.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055135210000000004 2024-09-18T11:31:04.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306451+0000", 2024-09-18T11:31:04.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0379000000000001e-05 2024-09-18T11:31:04.399 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.399 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.400 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.400 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2119 4.7 e73/22 4:e0d95e10:::smithi11939506-22:head v 73'89, mlcod=73'86)", 2024-09-18T11:31:04.400 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:55.701097+0000", 2024-09-18T11:31:04.400 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.6127652210000001, 2024-09-18T11:31:04.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056939520000000004, 2024-09-18T11:31:04.400 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.400 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.400 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701097+0000", 2024-09-18T11:31:04.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701097+0000", 2024-09-18T11:31:04.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701095+0000", 2024-09-18T11:31:04.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:04.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701919+0000", 2024-09-18T11:31:04.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000824181 2024-09-18T11:31:04.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701938+0000", 2024-09-18T11:31:04.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8614000000000002e-05 2024-09-18T11:31:04.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701947+0000", 2024-09-18T11:31:04.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0699999999999996e-06 2024-09-18T11:31:04.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701992+0000", 2024-09-18T11:31:04.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5614000000000001e-05 2024-09-18T11:31:04.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.702012+0000", 2024-09-18T11:31:04.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9639999999999998e-05 2024-09-18T11:31:04.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.706755+0000", 2024-09-18T11:31:04.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004742708 2024-09-18T11:31:04.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.706791+0000", 2024-09-18T11:31:04.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6106999999999998e-05 2024-09-18T11:31:04.407 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.407 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.407 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.408 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2323 4.2 e77/22 4:425afe9c:::smithi11939506-29:head v 77'58, mlcod=77'56)", 2024-09-18T11:31:04.408 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.194236+0000", 2024-09-18T11:31:04.408 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.1196259080000002, 2024-09-18T11:31:04.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054717079999999996, 2024-09-18T11:31:04.408 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.408 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.408 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194236+0000", 2024-09-18T11:31:04.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194236+0000", 2024-09-18T11:31:04.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194235+0000", 2024-09-18T11:31:04.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:04.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194777+0000", 2024-09-18T11:31:04.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00054227599999999996 2024-09-18T11:31:04.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194793+0000", 2024-09-18T11:31:04.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5662e-05 2024-09-18T11:31:04.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194800+0000", 2024-09-18T11:31:04.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9879999999999998e-06 2024-09-18T11:31:04.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194859+0000", 2024-09-18T11:31:04.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8876999999999999e-05 2024-09-18T11:31:04.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194889+0000", 2024-09-18T11:31:04.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0409999999999999e-05 2024-09-18T11:31:04.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.199659+0000", 2024-09-18T11:31:04.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047701330000000002 2024-09-18T11:31:04.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.199708+0000", 2024-09-18T11:31:04.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8242999999999999e-05 2024-09-18T11:31:04.412 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.412 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.412 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.412 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2324 4.2 e77/22 4:425afe9c:::smithi11939506-29:head v 77'59, mlcod=77'56)", 2024-09-18T11:31:04.412 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.195204+0000", 2024-09-18T11:31:04.412 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.1186575919999999, 2024-09-18T11:31:04.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047028310000000002, 2024-09-18T11:31:04.413 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.413 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.413 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.195204+0000", 2024-09-18T11:31:04.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.195204+0000", 2024-09-18T11:31:04.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.195203+0000", 2024-09-18T11:31:04.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:04.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.195211+0000", 2024-09-18T11:31:04.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0700000000000001e-06 2024-09-18T11:31:04.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.195221+0000", 2024-09-18T11:31:04.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0434999999999999e-05 2024-09-18T11:31:04.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.195226+0000", 2024-09-18T11:31:04.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3510000000000001e-06 2024-09-18T11:31:04.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.195881+0000", 2024-09-18T11:31:04.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00065441999999999996 2024-09-18T11:31:04.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.195904+0000", 2024-09-18T11:31:04.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3167999999999999e-05 2024-09-18T11:31:04.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.199869+0000", 2024-09-18T11:31:04.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039655419999999999 2024-09-18T11:31:04.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.199907+0000", 2024-09-18T11:31:04.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7753000000000003e-05 2024-09-18T11:31:04.417 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.417 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.417 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.417 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2334 4.3 4:cc46532a:::smithi11939506-33:18 [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:04.417 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386785+0000", 2024-09-18T11:31:04.417 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 2.9270765020000002, 2024-09-18T11:31:04.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046009909999999996, 2024-09-18T11:31:04.417 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.417 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "reached pg", 2024-09-18T11:31:04.417 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.418 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.418 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.418 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2334 2024-09-18T11:31:04.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.418 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386785+0000", 2024-09-18T11:31:04.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386785+0000", 2024-09-18T11:31:04.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386785+0000", 2024-09-18T11:31:04.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386788+0000", 2024-09-18T11:31:04.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4599999999999999e-06 2024-09-18T11:31:04.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386795+0000", 2024-09-18T11:31:04.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5050000000000004e-06 2024-09-18T11:31:04.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386799+0000", 2024-09-18T11:31:04.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3080000000000001e-06 2024-09-18T11:31:04.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386917+0000", 2024-09-18T11:31:04.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000118124 2024-09-18T11:31:04.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:04.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386921+0000", 2024-09-18T11:31:04.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9820000000000002e-06 2024-09-18T11:31:04.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.390790+0000", 2024-09-18T11:31:04.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038692140000000002 2024-09-18T11:31:04.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.391386+0000", 2024-09-18T11:31:04.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059604299999999999 2024-09-18T11:31:04.422 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.422 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.422 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.422 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.422 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.422 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:04.424 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:31:04.424 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:04.421+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:04.522 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:04.522 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:04.522 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:04.522 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:04.525 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:04.523+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:04.530 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:31:04.626 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:04.625+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:04.628 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:04.628 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:04.628 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:04.628 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:04.629 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:04.638 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:31:04.727 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:04.726+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:04.738 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:04.738 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:04.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:04.738 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:04.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.738 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:04.738 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:04.738 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.619449629999998, 2024-09-18T11:31:04.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:31:04.738 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.739 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:04.739 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.739 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.739 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.739 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:31:04.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.739 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.739 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.739 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.739 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:04.739 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.740 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:04.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.740 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:31:04.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:04.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.740 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:31:04.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:31:04.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.741 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.741 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:31:04.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:31:04.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.741 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.741 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:31:04.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:31:04.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:31:04.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:31:04.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:04.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:31:04.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:31:04.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:31:04.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:31:04.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:31:04.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:31:04.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:31:04.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:31:04.743 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.743 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.743 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.744 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:31:04.744 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:04.744 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 44.066424228999999, 2024-09-18T11:31:04.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:31:04.744 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.744 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:04.744 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.744 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.744 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.744 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:31:04.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.745 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:04.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:04.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:31:04.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:04.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.746 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:31:04.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:31:04.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.746 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:31:04.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:31:04.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:31:04.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:31:04.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:31:04.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:31:04.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:31:04.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:31:04.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:31:04.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:31:04.748 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.748 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.748 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.748 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1688 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 19=[19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-18T11:31:04.748 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:04.749 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.203269636000002, 2024-09-18T11:31:04.749 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071575789999999999, 2024-09-18T11:31:04.749 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.749 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.749 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.749 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.749 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.749 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1688 2024-09-18T11:31:04.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.749 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.749 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.749 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:04.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:04.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:04.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:04.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.751 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532500+0000", 2024-09-18T11:31:04.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5260000000000001e-06 2024-09-18T11:31:04.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.751 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.751 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532503+0000", 2024-09-18T11:31:04.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:31:04.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.751 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.751 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532505+0000", 2024-09-18T11:31:04.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.889e-06 2024-09-18T11:31:04.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.534722+0000", 2024-09-18T11:31:04.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002217119 2024-09-18T11:31:04.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.534827+0000", 2024-09-18T11:31:04.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0001049 2024-09-18T11:31:04.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:04.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.535007+0000", 2024-09-18T11:31:04.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017965599999999999 2024-09-18T11:31:04.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.753 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:04.753 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539463+0000", 2024-09-18T11:31:04.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004455713 2024-09-18T11:31:04.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.753 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:04.753 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539607+0000", 2024-09-18T11:31:04.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014440300000000001 2024-09-18T11:31:04.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.754 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539634+0000", 2024-09-18T11:31:04.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7398000000000001e-05 2024-09-18T11:31:04.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.754 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539657+0000", 2024-09-18T11:31:04.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2228000000000001e-05 2024-09-18T11:31:04.754 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.754 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.754 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.755 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1759 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2319366,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:04.755 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:31:04.755 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.441809425999999, 2024-09-18T11:31:04.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064323039999999998, 2024-09-18T11:31:04.755 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.755 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:04.755 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.755 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.755 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.755 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1759 2024-09-18T11:31:04.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.755 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:31:04.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:31:04.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:31:04.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293963+0000", 2024-09-18T11:31:04.757 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0810000000000004e-06 2024-09-18T11:31:04.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293970+0000", 2024-09-18T11:31:04.757 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5109999999999997e-06 2024-09-18T11:31:04.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293975+0000", 2024-09-18T11:31:04.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.33e-06 2024-09-18T11:31:04.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.758 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.758 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294078+0000", 2024-09-18T11:31:04.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102871 2024-09-18T11:31:04.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.758 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:04.758 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294084+0000", 2024-09-18T11:31:04.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5470000000000002e-06 2024-09-18T11:31:04.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.299550+0000", 2024-09-18T11:31:04.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054660799999999999 2024-09-18T11:31:04.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.299608+0000", 2024-09-18T11:31:04.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8307000000000003e-05 2024-09-18T11:31:04.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300392+0000", 2024-09-18T11:31:04.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078306999999999997 2024-09-18T11:31:04.760 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.760 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.760 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.760 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.760 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1760 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2319366,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:04.760 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:31:04.760 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.441775086, 2024-09-18T11:31:04.760 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070132529999999997, 2024-09-18T11:31:04.760 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.760 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:04.760 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.760 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.760 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.761 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1760 2024-09-18T11:31:04.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.761 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.761 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:31:04.761 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.761 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:31:04.761 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293993+0000", 2024-09-18T11:31:04.762 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293997+0000", 2024-09-18T11:31:04.762 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4999999999999999e-06 2024-09-18T11:31:04.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294004+0000", 2024-09-18T11:31:04.763 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3950000000000003e-06 2024-09-18T11:31:04.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.763 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.763 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294009+0000", 2024-09-18T11:31:04.763 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4989999999999996e-06 2024-09-18T11:31:04.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.763 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.763 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294103+0000", 2024-09-18T11:31:04.763 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4817999999999999e-05 2024-09-18T11:31:04.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.763 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:04.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294110+0000", 2024-09-18T11:31:04.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7240000000000003e-06 2024-09-18T11:31:04.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300406+0000", 2024-09-18T11:31:04.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062964470000000002 2024-09-18T11:31:04.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300457+0000", 2024-09-18T11:31:04.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0915000000000003e-05 2024-09-18T11:31:04.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.765 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.301007+0000", 2024-09-18T11:31:04.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00054942299999999995 2024-09-18T11:31:04.765 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.765 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.765 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.765 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1775 4.a 4:588debc2:::smithi11939506-18:head [write 1527054~766050] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:04.765 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:31:04.765 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.839422344999999, 2024-09-18T11:31:04.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070939050000000002, 2024-09-18T11:31:04.765 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.766 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.766 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.766 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.766 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.766 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1775 2024-09-18T11:31:04.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.766 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.766 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.766 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:31:04.766 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.766 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.767 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:31:04.767 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.767 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.767 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.767 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896345+0000", 2024-09-18T11:31:04.767 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:04.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.767 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.767 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.767 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896993+0000", 2024-09-18T11:31:04.767 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064769999999999997 2024-09-18T11:31:04.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.768 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.768 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897011+0000", 2024-09-18T11:31:04.768 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8355999999999998e-05 2024-09-18T11:31:04.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.768 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.768 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897016+0000", 2024-09-18T11:31:04.768 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3890000000000004e-06 2024-09-18T11:31:04.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.768 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.768 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897054+0000", 2024-09-18T11:31:04.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7963000000000002e-05 2024-09-18T11:31:04.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.769 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.769 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.769 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897123+0000", 2024-09-18T11:31:04.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8128999999999998e-05 2024-09-18T11:31:04.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.769 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.769 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:04.769 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897224+0000", 2024-09-18T11:31:04.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010168800000000001 2024-09-18T11:31:04.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.769 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.770 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:04.770 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899515+0000", 2024-09-18T11:31:04.770 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022910280000000001 2024-09-18T11:31:04.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.770 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:04.770 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903333+0000", 2024-09-18T11:31:04.770 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003817459 2024-09-18T11:31:04.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.770 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.770 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903388+0000", 2024-09-18T11:31:04.770 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5651e-05 2024-09-18T11:31:04.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903440+0000", 2024-09-18T11:31:04.771 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1836999999999999e-05 2024-09-18T11:31:04.771 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.771 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.771 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.771 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1776 4.a 4:588debc2:::smithi11939506-18:head [setxattr _header (57) in=7b,truncate 2420208] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:04.771 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:31:04.771 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.838744533, 2024-09-18T11:31:04.771 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065840400000000002, 2024-09-18T11:31:04.771 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.772 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.772 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.772 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.772 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.772 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1776 2024-09-18T11:31:04.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.772 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.772 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.772 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:31:04.773 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.773 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.773 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.773 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:31:04.773 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.773 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.773 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.773 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:31:04.773 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.773 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.774 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897026+0000", 2024-09-18T11:31:04.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9579999999999999e-06 2024-09-18T11:31:04.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.774 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897030+0000", 2024-09-18T11:31:04.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5370000000000002e-06 2024-09-18T11:31:04.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.774 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897032+0000", 2024-09-18T11:31:04.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.17e-06 2024-09-18T11:31:04.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.775 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.775 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898415+0000", 2024-09-18T11:31:04.775 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013823189999999999 2024-09-18T11:31:04.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.775 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.775 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898527+0000", 2024-09-18T11:31:04.775 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011236800000000001 2024-09-18T11:31:04.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.775 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:04.775 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898714+0000", 2024-09-18T11:31:04.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018703599999999999 2024-09-18T11:31:04.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.776 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:04.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899683+0000", 2024-09-18T11:31:04.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00096944700000000002 2024-09-18T11:31:04.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.776 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:04.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903475+0000", 2024-09-18T11:31:04.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037920129999999999 2024-09-18T11:31:04.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.777 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903535+0000", 2024-09-18T11:31:04.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9843999999999998e-05 2024-09-18T11:31:04.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.777 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903608+0000", 2024-09-18T11:31:04.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2946000000000003e-05 2024-09-18T11:31:04.777 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.777 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.777 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.777 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1777 4.a 4:588debc2:::smithi11939506-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:04.778 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:31:04.778 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.838717892999998, 2024-09-18T11:31:04.778 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0069010809999999999, 2024-09-18T11:31:04.778 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.778 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:04.778 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.778 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.778 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.778 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1777 2024-09-18T11:31:04.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.778 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.778 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.779 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:31:04.779 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.779 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.779 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.779 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:31:04.779 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.779 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.779 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.779 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897050+0000", 2024-09-18T11:31:04.779 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.779 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897052+0000", 2024-09-18T11:31:04.780 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:31:04.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897057+0000", 2024-09-18T11:31:04.780 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9470000000000002e-06 2024-09-18T11:31:04.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897060+0000", 2024-09-18T11:31:04.780 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3410000000000001e-06 2024-09-18T11:31:04.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.781 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899417+0000", 2024-09-18T11:31:04.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023571170000000002 2024-09-18T11:31:04.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.781 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:04.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899501+0000", 2024-09-18T11:31:04.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4160999999999996e-05 2024-09-18T11:31:04.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.781 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903661+0000", 2024-09-18T11:31:04.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041599510000000003 2024-09-18T11:31:04.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903790+0000", 2024-09-18T11:31:04.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012941499999999999 2024-09-18T11:31:04.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903952+0000", 2024-09-18T11:31:04.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016146800000000001 2024-09-18T11:31:04.782 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.782 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.782 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.783 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1895 4.5 4:a087a566:::smithi11939506-46:head [read 0~2756315,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:04.783 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:04.783 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.427462387999999, 2024-09-18T11:31:04.783 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007540377, 2024-09-18T11:31:04.783 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.783 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:04.783 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.783 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.783 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.783 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1895 2024-09-18T11:31:04.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.784 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.784 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.784 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:04.784 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.784 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.784 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:04.784 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.784 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.784 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:04.785 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.785 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.785 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308310+0000", 2024-09-18T11:31:04.785 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0559999999999998e-06 2024-09-18T11:31:04.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.785 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.785 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308317+0000", 2024-09-18T11:31:04.785 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7209999999999997e-06 2024-09-18T11:31:04.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308320+0000", 2024-09-18T11:31:04.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.721e-06 2024-09-18T11:31:04.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308447+0000", 2024-09-18T11:31:04.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012645199999999999 2024-09-18T11:31:04.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:04.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308451+0000", 2024-09-18T11:31:04.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6800000000000001e-06 2024-09-18T11:31:04.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.312462+0000", 2024-09-18T11:31:04.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040102139999999998 2024-09-18T11:31:04.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.312585+0000", 2024-09-18T11:31:04.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012388899999999999 2024-09-18T11:31:04.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.315847+0000", 2024-09-18T11:31:04.788 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003261197 2024-09-18T11:31:04.788 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.788 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.788 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.788 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1924 4.5 4:af4ba75a:::smithi11939506-37:head [write 1829449~675380] snapc 1e=[1e,1c,1b,19,18,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:04.788 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.362228+0000", 2024-09-18T11:31:04.788 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.373540688999999, 2024-09-18T11:31:04.788 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066182139999999999, 2024-09-18T11:31:04.788 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.788 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.788 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.788 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.789 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.789 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1924 2024-09-18T11:31:04.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.789 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.789 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.789 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362228+0000", 2024-09-18T11:31:04.789 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.789 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.789 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362228+0000", 2024-09-18T11:31:04.789 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.790 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.790 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.790 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362227+0000", 2024-09-18T11:31:04.790 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:04.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.790 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.790 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.790 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362749+0000", 2024-09-18T11:31:04.790 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00052262399999999996 2024-09-18T11:31:04.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.790 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.790 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.791 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362762+0000", 2024-09-18T11:31:04.791 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2821e-05 2024-09-18T11:31:04.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.791 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.791 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.791 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362767+0000", 2024-09-18T11:31:04.791 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5349999999999998e-06 2024-09-18T11:31:04.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.791 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.791 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.791 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362806+0000", 2024-09-18T11:31:04.791 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9029999999999997e-05 2024-09-18T11:31:04.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.792 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.792 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362863+0000", 2024-09-18T11:31:04.792 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7584999999999998e-05 2024-09-18T11:31:04.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.792 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:04.792 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362949+0000", 2024-09-18T11:31:04.792 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5655000000000006e-05 2024-09-18T11:31:04.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.792 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:04.792 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364314+0000", 2024-09-18T11:31:04.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001364961 2024-09-18T11:31:04.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.793 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:04.793 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.368771+0000", 2024-09-18T11:31:04.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004457244 2024-09-18T11:31:04.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.793 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.793 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.368809+0000", 2024-09-18T11:31:04.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8368999999999998e-05 2024-09-18T11:31:04.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.793 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.368846+0000", 2024-09-18T11:31:04.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6718000000000003e-05 2024-09-18T11:31:04.794 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.794 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.794 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.794 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1925 4.5 4:af4ba75a:::smithi11939506-37:head [setxattr _header (57) in=7b,truncate 2655506] snapc 1e=[1e,1c,1b,19,18,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:04.794 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:31:04.794 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.372995537000001, 2024-09-18T11:31:04.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064745779999999999, 2024-09-18T11:31:04.794 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.794 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.795 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.795 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.795 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.795 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1925 2024-09-18T11:31:04.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.795 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:31:04.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:31:04.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:31:04.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362776+0000", 2024-09-18T11:31:04.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.208e-06 2024-09-18T11:31:04.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.797 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.797 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362780+0000", 2024-09-18T11:31:04.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.771e-06 2024-09-18T11:31:04.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.797 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.797 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362782+0000", 2024-09-18T11:31:04.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.277e-06 2024-09-18T11:31:04.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.797 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.797 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.363737+0000", 2024-09-18T11:31:04.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00095483999999999999 2024-09-18T11:31:04.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.363832+0000", 2024-09-18T11:31:04.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5594999999999995e-05 2024-09-18T11:31:04.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:04.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.363940+0000", 2024-09-18T11:31:04.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010738000000000001 2024-09-18T11:31:04.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:04.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364577+0000", 2024-09-18T11:31:04.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00063679600000000002 2024-09-18T11:31:04.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:04.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369126+0000", 2024-09-18T11:31:04.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0045496440000000003 2024-09-18T11:31:04.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369177+0000", 2024-09-18T11:31:04.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0958000000000002e-05 2024-09-18T11:31:04.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369248+0000", 2024-09-18T11:31:04.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0599000000000006e-05 2024-09-18T11:31:04.800 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.800 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.800 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.800 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1926 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:04.800 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:04.800 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.372971281000002, 2024-09-18T11:31:04.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066190219999999996, 2024-09-18T11:31:04.800 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.800 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:04.800 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.801 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.801 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.801 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1926 2024-09-18T11:31:04.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.801 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:04.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:04.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:04.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362799+0000", 2024-09-18T11:31:04.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.029e-06 2024-09-18T11:31:04.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362804+0000", 2024-09-18T11:31:04.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3989999999999997e-06 2024-09-18T11:31:04.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362806+0000", 2024-09-18T11:31:04.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3020000000000002e-06 2024-09-18T11:31:04.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364105+0000", 2024-09-18T11:31:04.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001299478 2024-09-18T11:31:04.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.804 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:04.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364153+0000", 2024-09-18T11:31:04.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8059000000000002e-05 2024-09-18T11:31:04.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.804 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369273+0000", 2024-09-18T11:31:04.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051199540000000003 2024-09-18T11:31:04.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.804 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369343+0000", 2024-09-18T11:31:04.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0191999999999995e-05 2024-09-18T11:31:04.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369416+0000", 2024-09-18T11:31:04.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2925999999999999e-05 2024-09-18T11:31:04.805 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.805 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.805 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.805 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1965 4.a 4:5975a5ec:::smithi11939506-36:head [rollback 1e] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:04.805 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373840+0000", 2024-09-18T11:31:04.805 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.361928864999999, 2024-09-18T11:31:04.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062698229999999999, 2024-09-18T11:31:04.806 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.806 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.806 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.806 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.806 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.806 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1965 2024-09-18T11:31:04.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.806 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373840+0000", 2024-09-18T11:31:04.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373840+0000", 2024-09-18T11:31:04.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373839+0000", 2024-09-18T11:31:04.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373842+0000", 2024-09-18T11:31:04.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8770000000000002e-06 2024-09-18T11:31:04.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373849+0000", 2024-09-18T11:31:04.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3929999999999996e-06 2024-09-18T11:31:04.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373853+0000", 2024-09-18T11:31:04.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0099999999999997e-06 2024-09-18T11:31:04.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.375207+0000", 2024-09-18T11:31:04.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001354755 2024-09-18T11:31:04.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.375268+0000", 2024-09-18T11:31:04.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0983999999999998e-05 2024-09-18T11:31:04.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:04.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.375449+0000", 2024-09-18T11:31:04.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000181091 2024-09-18T11:31:04.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:04.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380058+0000", 2024-09-18T11:31:04.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046089390000000003 2024-09-18T11:31:04.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:04.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380065+0000", 2024-09-18T11:31:04.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4479999999999999e-06 2024-09-18T11:31:04.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380088+0000", 2024-09-18T11:31:04.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3337e-05 2024-09-18T11:31:04.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380110+0000", 2024-09-18T11:31:04.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.145e-05 2024-09-18T11:31:04.814 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.814 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.814 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.815 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1966 4.a 4:5975a5ec:::smithi11939506-36:head [append 0~0] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:04.815 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373876+0000", 2024-09-18T11:31:04.815 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.361892841, 2024-09-18T11:31:04.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063095080000000001, 2024-09-18T11:31:04.815 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.815 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.815 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.815 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.815 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.815 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1966 2024-09-18T11:31:04.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.815 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373876+0000", 2024-09-18T11:31:04.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373876+0000", 2024-09-18T11:31:04.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373875+0000", 2024-09-18T11:31:04.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373879+0000", 2024-09-18T11:31:04.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3859999999999999e-06 2024-09-18T11:31:04.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373885+0000", 2024-09-18T11:31:04.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6030000000000001e-06 2024-09-18T11:31:04.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373889+0000", 2024-09-18T11:31:04.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9620000000000004e-06 2024-09-18T11:31:04.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.376620+0000", 2024-09-18T11:31:04.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027307920000000001 2024-09-18T11:31:04.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.376672+0000", 2024-09-18T11:31:04.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2244999999999999e-05 2024-09-18T11:31:04.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:04.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.376838+0000", 2024-09-18T11:31:04.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000165716 2024-09-18T11:31:04.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:04.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380116+0000", 2024-09-18T11:31:04.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003278352 2024-09-18T11:31:04.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:04.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380145+0000", 2024-09-18T11:31:04.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8379999999999999e-05 2024-09-18T11:31:04.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380166+0000", 2024-09-18T11:31:04.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1118e-05 2024-09-18T11:31:04.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380185+0000", 2024-09-18T11:31:04.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9519999999999999e-05 2024-09-18T11:31:04.820 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.820 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.821 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.821 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1969 4.a 4:588debc2:::smithi11939506-18:18 [read 0~3949612,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:04.821 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:04.821 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.361782392, 2024-09-18T11:31:04.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075409190000000001, 2024-09-18T11:31:04.821 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.821 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:04.822 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.822 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.822 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.822 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1969 2024-09-18T11:31:04.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.822 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.822 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.822 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:04.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.822 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:04.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:04.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373990+0000", 2024-09-18T11:31:04.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1119999999999998e-06 2024-09-18T11:31:04.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373997+0000", 2024-09-18T11:31:04.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4200000000000001e-06 2024-09-18T11:31:04.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.374002+0000", 2024-09-18T11:31:04.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:31:04.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380200+0000", 2024-09-18T11:31:04.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006197302 2024-09-18T11:31:04.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380261+0000", 2024-09-18T11:31:04.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1389999999999993e-05 2024-09-18T11:31:04.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.381527+0000", 2024-09-18T11:31:04.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012660080000000001 2024-09-18T11:31:04.825 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.825 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.825 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.826 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2207 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1410296~761389] snapc 24=[24,1e,1c,1b,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T11:31:04.826 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:04.826 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.3132715660000001, 2024-09-18T11:31:04.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008410743, 2024-09-18T11:31:04.826 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.826 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.826 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.826 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.826 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.826 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2207 2024-09-18T11:31:04.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.826 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:04.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:04.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422496+0000", 2024-09-18T11:31:04.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:04.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423190+0000", 2024-09-18T11:31:04.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00069405499999999995 2024-09-18T11:31:04.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423204+0000", 2024-09-18T11:31:04.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:31:04.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423208+0000", 2024-09-18T11:31:04.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:31:04.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423293+0000", 2024-09-18T11:31:04.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4524999999999995e-05 2024-09-18T11:31:04.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423394+0000", 2024-09-18T11:31:04.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101395 2024-09-18T11:31:04.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:04.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423579+0000", 2024-09-18T11:31:04.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018461000000000001 2024-09-18T11:31:04.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:04.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.428574+0000", 2024-09-18T11:31:04.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049944960000000002 2024-09-18T11:31:04.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:04.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430809+0000", 2024-09-18T11:31:04.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022351820000000001 2024-09-18T11:31:04.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430862+0000", 2024-09-18T11:31:04.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3189999999999997e-05 2024-09-18T11:31:04.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430908+0000", 2024-09-18T11:31:04.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5973999999999997e-05 2024-09-18T11:31:04.831 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.831 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.832 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.832 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2331 4.a 4:552523b6:::smithi11939506-38:head [append 0~0] snapc 26=[26,25,24,1e,1c,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T11:31:04.832 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.194086+0000", 2024-09-18T11:31:04.832 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.5416826000000001, 2024-09-18T11:31:04.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062829319999999998, 2024-09-18T11:31:04.832 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.832 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:04.832 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.832 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.832 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.832 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2331 2024-09-18T11:31:04.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.833 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194086+0000", 2024-09-18T11:31:04.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194086+0000", 2024-09-18T11:31:04.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194086+0000", 2024-09-18T11:31:04.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194088+0000", 2024-09-18T11:31:04.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.367e-06 2024-09-18T11:31:04.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194094+0000", 2024-09-18T11:31:04.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8089999999999998e-06 2024-09-18T11:31:04.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194097+0000", 2024-09-18T11:31:04.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5920000000000001e-06 2024-09-18T11:31:04.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.196736+0000", 2024-09-18T11:31:04.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002638428 2024-09-18T11:31:04.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.196834+0000", 2024-09-18T11:31:04.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7907999999999998e-05 2024-09-18T11:31:04.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:04.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.197015+0000", 2024-09-18T11:31:04.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000180946 2024-09-18T11:31:04.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:04.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.199755+0000", 2024-09-18T11:31:04.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027399830000000001 2024-09-18T11:31:04.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:04.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.200306+0000", 2024-09-18T11:31:04.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00055154399999999995 2024-09-18T11:31:04.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:04.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.200341+0000", 2024-09-18T11:31:04.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4839999999999998e-05 2024-09-18T11:31:04.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.200369+0000", 2024-09-18T11:31:04.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7909999999999999e-05 2024-09-18T11:31:04.838 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.838 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.838 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.838 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2339 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:04.838 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:04.838 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.3489669370000001, 2024-09-18T11:31:04.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011809992, 2024-09-18T11:31:04.838 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.838 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:04.838 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.838 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.838 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.839 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2339 2024-09-18T11:31:04.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.839 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:04.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:04.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:31:04.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:04.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:31:04.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4710000000000001e-06 2024-09-18T11:31:04.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386824+0000", 2024-09-18T11:31:04.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6104e-05 2024-09-18T11:31:04.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386836+0000", 2024-09-18T11:31:04.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1827999999999999e-05 2024-09-18T11:31:04.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388390+0000", 2024-09-18T11:31:04.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015539499999999999 2024-09-18T11:31:04.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:04.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388443+0000", 2024-09-18T11:31:04.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3785000000000001e-05 2024-09-18T11:31:04.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398238+0000", 2024-09-18T11:31:04.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097947989999999999 2024-09-18T11:31:04.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398315+0000", 2024-09-18T11:31:04.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6909999999999994e-05 2024-09-18T11:31:04.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398612+0000", 2024-09-18T11:31:04.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029671999999999999 2024-09-18T11:31:04.843 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.843 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.843 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.843 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2340 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:04.843 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:04.843 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.3489236249999998, 2024-09-18T11:31:04.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012086448, 2024-09-18T11:31:04.844 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.844 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:04.844 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.844 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.844 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.844 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2340 2024-09-18T11:31:04.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.844 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:04.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:04.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:04.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386848+0000", 2024-09-18T11:31:04.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3069999999999998e-06 2024-09-18T11:31:04.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386855+0000", 2024-09-18T11:31:04.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0979999999999996e-06 2024-09-18T11:31:04.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386859+0000", 2024-09-18T11:31:04.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9360000000000001e-06 2024-09-18T11:31:04.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388466+0000", 2024-09-18T11:31:04.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001607154 2024-09-18T11:31:04.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:04.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388507+0000", 2024-09-18T11:31:04.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1093000000000001e-05 2024-09-18T11:31:04.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398647+0000", 2024-09-18T11:31:04.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010139686 2024-09-18T11:31:04.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398718+0000", 2024-09-18T11:31:04.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1138999999999998e-05 2024-09-18T11:31:04.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398931+0000", 2024-09-18T11:31:04.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021356899999999999 2024-09-18T11:31:04.848 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.848 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.848 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.849 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2341 4.5 4:aa575018:::smithi11939506-11:head [read 0~2487436,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:04.849 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:04.849 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.3489054029999998, 2024-09-18T11:31:04.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013751648999999999, 2024-09-18T11:31:04.849 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:04.849 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:04.849 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:04.849 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:04.849 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:04.849 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2341 2024-09-18T11:31:04.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.850 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:04.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:04.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:04.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:04.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:04.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:04.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:04.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:04.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:04.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:04.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386866+0000", 2024-09-18T11:31:04.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4249999999999998e-06 2024-09-18T11:31:04.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:04.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386872+0000", 2024-09-18T11:31:04.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.003e-06 2024-09-18T11:31:04.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:04.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:31:04.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.507e-06 2024-09-18T11:31:04.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388520+0000", 2024-09-18T11:31:04.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001644146 2024-09-18T11:31:04.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:04.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388561+0000", 2024-09-18T11:31:04.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1486999999999998e-05 2024-09-18T11:31:04.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:04.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398963+0000", 2024-09-18T11:31:04.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010401353 2024-09-18T11:31:04.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:04.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.399046+0000", 2024-09-18T11:31:04.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3141999999999998e-05 2024-09-18T11:31:04.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:04.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:04.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:04.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.400615+0000", 2024-09-18T11:31:04.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015690960000000001 2024-09-18T11:31:04.854 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.854 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.854 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.854 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:04.854 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:04.854 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:04.855 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:04.752+0000 7fb6930fb640 -1 osd.2 80 handle_osd_map injecting map crc failure 2024-09-18T11:31:04.856 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:31:04.856 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:04.854+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:04.956 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:04.955+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:04.994 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:04.994 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:04.994 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:04.994 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:05.003 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:31:05.008 INFO:tasks.rados.rados.0.smithi119.stdout:1174: done (4 left) 2024-09-18T11:31:05.008 INFO:tasks.rados.rados.0.smithi119.stdout:1175: done (3 left) 2024-09-18T11:31:05.008 INFO:tasks.rados.rados.0.smithi119.stdout:1176: done (2 left) 2024-09-18T11:31:05.008 INFO:tasks.rados.rados.0.smithi119.stdout:1177: done (1 left) 2024-09-18T11:31:05.008 INFO:tasks.rados.rados.0.smithi119.stdout:1178: done (0 left) 2024-09-18T11:31:05.008 INFO:tasks.rados.rados.0.smithi119.stdout:1179: rollback oid 27 current snap is 25 2024-09-18T11:31:05.008 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 27 to 19 2024-09-18T11:31:05.008 INFO:tasks.rados.rados.0.smithi119.stdout:1180: write (excl) oid 49 current snap is 25 2024-09-18T11:31:05.008 INFO:tasks.rados.rados.0.smithi119.stdout:1180: seq_num 214 ranges {796698=561170,2006323=545406,3126991=469707} 2024-09-18T11:31:05.015 INFO:tasks.rados.rados.0.smithi119.stdout:1180: writing smithi11939506-49 from 796698 to 1357868 tid 1 2024-09-18T11:31:05.015 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T11:31:05.021 INFO:tasks.rados.rados.0.smithi119.stdout:1180: writing smithi11939506-49 from 2006323 to 2551729 tid 2 2024-09-18T11:31:05.028 INFO:tasks.rados.rados.0.smithi119.stdout:1180: writing smithi11939506-49 from 3126991 to 3596698 tid 3 2024-09-18T11:31:05.028 INFO:tasks.rados.rados.0.smithi119.stdout:1181: read oid 42 snap -1 2024-09-18T11:31:05.028 INFO:tasks.rados.rados.0.smithi119.stdout:1181: expect (ObjNum 99 snap 6 seq_num 99) 2024-09-18T11:31:05.028 INFO:tasks.rados.rados.0.smithi119.stdout:1182: snap_create 2024-09-18T11:31:05.028 INFO:tasks.rados.rados.0.smithi119.stdout:1179: finishing rollback tid 0 to smithi11939506-27 2024-09-18T11:31:05.028 INFO:tasks.rados.rados.0.smithi119.stdout:1179: finishing rollback tid 1 to smithi11939506-27 2024-09-18T11:31:05.028 INFO:tasks.rados.rados.0.smithi119.stdout:1179: finishing rollback tid 2 to smithi11939506-27 2024-09-18T11:31:05.028 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 137 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T11:31:05.029 INFO:tasks.rados.rados.0.smithi119.stdout:1180: finishing write tid 1 to smithi11939506-49 2024-09-18T11:31:05.029 INFO:tasks.rados.rados.0.smithi119.stdout:1180: finishing write tid 2 to smithi11939506-49 2024-09-18T11:31:05.030 INFO:tasks.rados.rados.0.smithi119.stdout:1180: finishing write tid 3 to smithi11939506-49 2024-09-18T11:31:05.033 INFO:tasks.rados.rados.0.smithi119.stdout:1180: finishing write tid 4 to smithi11939506-49 2024-09-18T11:31:05.033 INFO:tasks.rados.rados.0.smithi119.stdout:1180: finishing write tid 5 to smithi11939506-49 2024-09-18T11:31:05.033 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 220 (ObjNum 214 snap 25 seq_num 214) dirty exists 2024-09-18T11:31:05.033 INFO:tasks.rados.rados.0.smithi119.stdout:1180: left oid 49 (ObjNum 214 snap 25 seq_num 214) 2024-09-18T11:31:05.035 INFO:tasks.rados.rados.0.smithi119.stdout:1181: expect (ObjNum 99 snap 6 seq_num 99) 2024-09-18T11:31:05.057 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:05.055+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:05.101 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:05.102 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:05.102 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:05.102 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:05.102 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:05.110 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:31:05.194 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:05.193+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:05.251 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:05.251 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:05.251 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:05.252 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:05.252 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.252 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:05.252 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:31:05.252 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 76.091416562000006, 2024-09-18T11:31:05.252 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054864400000000004, 2024-09-18T11:31:05.252 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:05.252 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.252 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:05.252 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:05.252 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:05.252 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 42 2024-09-18T11:31:05.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.253 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:05.253 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.253 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:05.253 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:31:05.253 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.253 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.253 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:05.253 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:31:05.254 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.254 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.254 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:05.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157413+0000", 2024-09-18T11:31:05.254 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.254 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.254 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:05.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157416+0000", 2024-09-18T11:31:05.254 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.721e-06 2024-09-18T11:31:05.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.254 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.255 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:05.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157423+0000", 2024-09-18T11:31:05.255 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4049999999999997e-06 2024-09-18T11:31:05.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.255 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.255 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157425+0000", 2024-09-18T11:31:05.255 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.481e-06 2024-09-18T11:31:05.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.255 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.255 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:05.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158649+0000", 2024-09-18T11:31:05.255 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001223453 2024-09-18T11:31:05.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.256 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.256 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:05.256 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158729+0000", 2024-09-18T11:31:05.256 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0205999999999997e-05 2024-09-18T11:31:05.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.256 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.256 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:05.256 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.158875+0000", 2024-09-18T11:31:05.256 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014628800000000001 2024-09-18T11:31:05.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.256 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.256 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:05.256 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161518+0000", 2024-09-18T11:31:05.257 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026427949999999999 2024-09-18T11:31:05.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.257 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.257 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.257 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162821+0000", 2024-09-18T11:31:05.257 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013028099999999999 2024-09-18T11:31:05.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.257 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.257 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:05.257 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162864+0000", 2024-09-18T11:31:05.257 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2839000000000001e-05 2024-09-18T11:31:05.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.257 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.258 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:05.258 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162900+0000", 2024-09-18T11:31:05.258 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5948999999999998e-05 2024-09-18T11:31:05.258 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.258 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:05.258 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.258 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.258 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:05.258 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:05.258 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 76.091330197999994, 2024-09-18T11:31:05.258 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055400850000000001, 2024-09-18T11:31:05.258 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:05.259 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.259 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:05.259 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:05.259 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:05.259 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 43 2024-09-18T11:31:05.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.259 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:05.259 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.259 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:05.259 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:05.259 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.260 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.260 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:05.260 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:05.260 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.260 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.260 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:05.260 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:05.261 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:05.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.261 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.261 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:05.261 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157504+0000", 2024-09-18T11:31:05.261 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4519999999999996e-06 2024-09-18T11:31:05.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.261 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.262 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:05.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157534+0000", 2024-09-18T11:31:05.262 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.97e-05 2024-09-18T11:31:05.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.262 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.262 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157539+0000", 2024-09-18T11:31:05.262 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4360000000000001e-06 2024-09-18T11:31:05.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.262 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.262 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:05.263 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159428+0000", 2024-09-18T11:31:05.263 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018888220000000001 2024-09-18T11:31:05.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.263 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.263 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:05.263 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159479+0000", 2024-09-18T11:31:05.263 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0608000000000001e-05 2024-09-18T11:31:05.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.264 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.264 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:05.264 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159559+0000", 2024-09-18T11:31:05.264 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0667000000000005e-05 2024-09-18T11:31:05.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.265 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.265 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:05.265 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162779+0000", 2024-09-18T11:31:05.265 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032197459999999999 2024-09-18T11:31:05.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.265 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.266 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.266 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162950+0000", 2024-09-18T11:31:05.266 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017059599999999999 2024-09-18T11:31:05.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.266 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.266 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:05.266 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163001+0000", 2024-09-18T11:31:05.266 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0968999999999999e-05 2024-09-18T11:31:05.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.267 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.267 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:05.267 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163040+0000", 2024-09-18T11:31:05.267 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8843000000000001e-05 2024-09-18T11:31:05.267 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.267 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:05.267 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.267 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.267 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:05.268 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:05.268 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 76.091279044000004, 2024-09-18T11:31:05.268 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:31:05.268 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:05.268 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.269 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:05.269 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:05.269 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:05.269 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:31:05.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.269 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:05.269 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.269 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:05.269 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:05.269 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.269 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.269 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:05.270 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:05.270 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.270 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.270 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:05.270 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:31:05.270 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.270 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.270 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:05.270 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:31:05.270 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:31:05.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.271 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.271 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:05.271 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:31:05.271 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:31:05.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.271 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.271 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.271 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:31:05.271 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:05.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.271 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.271 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:05.272 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:31:05.272 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:31:05.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.272 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.272 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:05.272 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:31:05.272 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:31:05.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.272 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.272 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:05.272 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:31:05.272 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:31:05.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.273 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.273 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:05.273 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:31:05.273 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:31:05.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.273 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.273 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.273 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:31:05.273 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:31:05.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.274 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.274 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:05.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:31:05.274 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:31:05.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.274 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.274 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:05.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:31:05.274 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:31:05.274 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.275 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:05.275 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.275 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.275 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:05.275 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:05.275 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 76.091152781999995, 2024-09-18T11:31:05.275 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055717040000000002, 2024-09-18T11:31:05.275 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:05.275 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.275 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:05.275 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:05.276 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:05.276 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 45 2024-09-18T11:31:05.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.276 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:05.276 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.276 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:05.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:05.276 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.276 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.276 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:05.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:05.277 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.277 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:05.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157676+0000", 2024-09-18T11:31:05.277 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.277 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:05.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157694+0000", 2024-09-18T11:31:05.277 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7422999999999999e-05 2024-09-18T11:31:05.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.278 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:05.278 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157704+0000", 2024-09-18T11:31:05.278 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9000000000000001e-06 2024-09-18T11:31:05.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.278 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.278 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.278 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157718+0000", 2024-09-18T11:31:05.278 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4788e-05 2024-09-18T11:31:05.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.278 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.278 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:05.278 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160937+0000", 2024-09-18T11:31:05.278 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00321886 2024-09-18T11:31:05.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.279 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.279 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:05.279 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161023+0000", 2024-09-18T11:31:05.279 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6063999999999995e-05 2024-09-18T11:31:05.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.279 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.279 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:05.279 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161202+0000", 2024-09-18T11:31:05.279 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017912599999999999 2024-09-18T11:31:05.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.279 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.280 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:05.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163052+0000", 2024-09-18T11:31:05.280 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001849554 2024-09-18T11:31:05.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.280 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163204+0000", 2024-09-18T11:31:05.280 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015186900000000001 2024-09-18T11:31:05.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.280 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:05.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163229+0000", 2024-09-18T11:31:05.280 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:31:05.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.281 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:05.281 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163249+0000", 2024-09-18T11:31:05.281 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9992000000000001e-05 2024-09-18T11:31:05.281 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.281 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:05.281 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.281 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:05.281 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:05.281 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 76.091103669000006, 2024-09-18T11:31:05.282 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:31:05.282 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:05.282 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.282 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:05.282 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:05.282 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:05.282 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:31:05.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.282 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:05.282 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.282 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:05.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:05.282 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.283 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:05.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:05.283 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.283 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:05.284 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:05.284 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.284 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.284 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:05.284 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:31:05.284 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:31:05.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.284 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.285 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:05.285 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:31:05.285 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:31:05.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.285 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.285 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:31:05.285 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:31:05.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.285 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:05.285 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:31:05.285 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:31:05.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.286 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.286 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:05.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:31:05.286 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:31:05.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.286 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.286 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:05.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:31:05.286 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:31:05.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.286 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.286 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:05.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:31:05.287 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:31:05.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.287 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.287 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:31:05.287 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:31:05.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.288 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:05.288 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:31:05.288 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:31:05.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.288 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.288 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:05.288 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:31:05.288 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:31:05.288 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.288 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:05.289 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.289 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.289 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:05.289 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:05.289 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 76.091061138000001, 2024-09-18T11:31:05.289 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:31:05.289 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:05.289 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.289 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:05.289 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:05.290 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:05.290 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:31:05.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.290 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:05.290 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.290 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:05.290 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:05.290 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.290 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.290 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:05.290 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:05.290 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.291 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:05.291 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:31:05.291 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.291 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:05.291 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:31:05.291 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:31:05.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.292 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:05.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:31:05.292 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:31:05.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.292 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.292 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:31:05.292 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:31:05.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.292 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.292 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:05.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:31:05.292 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:31:05.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.293 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:05.293 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:31:05.293 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:31:05.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.293 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:05.293 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:31:05.293 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:31:05.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.293 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:31:05.294 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:31:05.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.294 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:05.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:31:05.294 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:31:05.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.294 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:05.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:31:05.294 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:31:05.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.295 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:05.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:31:05.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:31:05.295 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.295 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:05.295 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.295 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:05.295 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:05.295 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 76.091006207999996, 2024-09-18T11:31:05.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:31:05.296 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:05.296 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.296 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:05.296 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:05.296 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:05.296 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:31:05.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.296 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:05.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.296 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:05.296 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:05.296 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.297 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:05.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:05.297 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.297 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:05.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:05.297 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.297 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:05.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:31:05.298 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:31:05.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.298 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:05.298 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:31:05.298 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:31:05.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.298 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.298 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:31:05.298 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:31:05.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.299 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:05.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:31:05.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:31:05.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.299 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:05.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:31:05.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:31:05.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.299 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:05.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:31:05.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:31:05.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:05.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:31:05.300 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:31:05.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:31:05.300 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:31:05.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:05.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:31:05.300 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:31:05.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.301 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:05.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:31:05.301 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:31:05.301 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.301 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:05.301 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.301 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:05.301 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:05.301 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 76.090996227999995, 2024-09-18T11:31:05.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:31:05.302 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:05.302 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.302 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:05.302 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:05.302 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:05.302 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:31:05.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.302 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:05.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.302 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:05.302 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:05.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.303 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:05.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:05.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.303 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:05.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:05.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:05.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.303 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:05.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:31:05.304 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:31:05.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.304 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:05.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:31:05.304 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:31:05.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:31:05.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:31:05.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:05.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:31:05.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:31:05.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:05.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:31:05.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:31:05.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:05.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:31:05.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:31:05.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:31:05.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:31:05.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:05.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:31:05.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:31:05.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:05.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:31:05.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:31:05.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:05.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:31:05.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:31:05.308 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.308 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:05.308 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.308 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.308 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:05.308 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:05.308 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 76.090984629000005, 2024-09-18T11:31:05.308 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:31:05.308 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:05.308 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.308 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:05.309 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:05.309 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:05.309 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:31:05.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.309 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:05.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.309 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:05.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:05.309 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.309 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:05.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:05.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:05.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:05.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:05.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:05.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:31:05.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:31:05.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:05.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:31:05.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:31:05.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:31:05.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:31:05.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:05.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:31:05.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:31:05.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.312 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:05.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:31:05.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:31:05.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.312 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:05.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:31:05.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:31:05.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.312 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:05.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:31:05.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:31:05.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.313 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:31:05.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:31:05.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.313 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:05.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:31:05.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:31:05.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.314 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:05.314 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:31:05.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:31:05.314 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.314 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:05.314 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.314 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:05.314 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:05.314 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 76.090928148000003, 2024-09-18T11:31:05.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:31:05.314 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:05.315 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.315 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:05.315 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:05.315 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:05.315 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:31:05.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.315 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:05.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:05.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:05.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:05.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:05.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:05.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:31:05.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:05.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:31:05.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:31:05.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:05.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:31:05.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:31:05.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:31:05.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:31:05.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:05.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:31:05.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:31:05.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:05.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:31:05.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:31:05.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:05.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:31:05.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:31:05.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:31:05.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:31:05.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:05.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:31:05.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:31:05.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:05.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:31:05.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:31:05.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:05.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:31:05.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:31:05.320 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.320 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:05.320 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.320 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:05.320 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:05.320 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 76.090914323999996, 2024-09-18T11:31:05.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:31:05.321 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:05.321 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.321 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:05.321 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:05.321 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:05.321 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:31:05.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.321 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:05.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:05.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:05.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:05.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:05.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:05.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:05.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:05.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:05.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:31:05.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:31:05.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:05.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:31:05.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:31:05.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:31:05.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:31:05.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:05.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:31:05.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:31:05.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:05.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:31:05.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:31:05.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:05.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:31:05.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:31:05.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:31:05.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:31:05.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:05.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:31:05.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:31:05.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:05.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:31:05.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:31:05.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:05.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:31:05.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:31:05.327 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.327 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:05.327 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.327 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:05.327 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:05.327 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 75.674735416999994, 2024-09-18T11:31:05.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:31:05.327 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:05.327 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.327 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:05.328 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:05.328 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:05.328 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:31:05.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.328 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:05.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.328 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:05.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:05.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.328 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:05.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:05.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:05.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:05.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:05.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:05.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:31:05.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:31:05.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:05.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:31:05.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:31:05.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:31:05.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:31:05.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:05.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:31:05.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:31:05.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:05.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:31:05.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:31:05.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:05.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:31:05.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:31:05.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:05.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:31:05.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:31:05.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:31:05.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:31:05.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:05.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:31:05.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:31:05.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:05.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:31:05.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:31:05.333 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.333 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:05.333 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.333 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:05.333 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:05.333 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 75.674712116999999, 2024-09-18T11:31:05.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:31:05.334 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:05.334 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.334 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:05.334 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:05.334 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:05.334 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:31:05.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.334 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:05.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:05.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:05.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:05.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:05.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:05.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:31:05.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:05.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:31:05.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:31:05.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:05.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:31:05.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:31:05.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:31:05.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:31:05.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:05.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:31:05.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:31:05.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:05.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:31:05.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:31:05.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:05.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:31:05.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:31:05.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:05.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:31:05.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:31:05.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:31:05.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:31:05.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:05.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:31:05.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:31:05.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:05.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:31:05.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:31:05.339 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.339 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:05.339 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.340 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:05.340 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:05.340 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 75.674626508000003, 2024-09-18T11:31:05.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:31:05.340 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:05.340 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.340 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:05.340 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:05.340 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:05.340 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:31:05.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.340 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:05.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:05.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:05.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:05.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:05.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:05.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:05.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:05.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:31:05.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:31:05.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:05.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:31:05.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:31:05.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:31:05.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:31:05.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:05.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:31:05.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:31:05.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:05.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:31:05.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:31:05.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:05.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:31:05.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:31:05.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:05.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:31:05.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:31:05.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:31:05.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:31:05.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:05.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:31:05.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:31:05.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:05.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:31:05.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:31:05.346 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.346 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:05.346 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.346 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:05.346 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:05.346 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 75.674615543000002, 2024-09-18T11:31:05.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:31:05.346 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:05.347 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.347 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:05.347 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:05.347 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:05.347 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:31:05.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.347 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:05.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:05.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:05.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:05.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:05.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:05.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:05.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:05.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:05.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:31:05.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:31:05.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:05.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:31:05.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:31:05.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:31:05.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:31:05.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:05.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:31:05.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:31:05.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:05.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:31:05.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:31:05.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:05.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:31:05.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:31:05.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:05.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:31:05.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:31:05.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:31:05.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:31:05.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:05.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:31:05.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:31:05.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:05.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:31:05.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:31:05.352 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.352 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:05.353 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.353 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:05.353 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:05.353 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 75.674604375000001, 2024-09-18T11:31:05.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:31:05.353 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:05.353 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.354 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:05.354 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:05.354 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:05.354 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:31:05.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.354 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:05.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:05.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:05.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:05.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:05.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:05.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:05.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:05.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:05.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:31:05.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:31:05.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:05.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:31:05.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:31:05.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:31:05.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:31:05.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:05.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:31:05.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:31:05.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:05.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:31:05.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:31:05.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:05.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:31:05.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:31:05.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:31:05.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:31:05.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:05.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:31:05.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:31:05.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:05.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:31:05.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:31:05.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:05.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:31:05.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:31:05.359 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.359 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:05.359 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.359 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:05.359 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:05.359 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 75.674539046000007, 2024-09-18T11:31:05.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:31:05.359 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:05.359 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.360 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:05.360 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:05.360 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:05.360 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:31:05.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.360 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:05.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:05.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:05.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:05.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:05.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:05.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:31:05.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:05.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:31:05.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:31:05.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:05.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:31:05.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:31:05.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:31:05.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:31:05.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:05.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:31:05.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:31:05.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:05.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:31:05.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:31:05.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:05.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:31:05.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:31:05.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:31:05.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:31:05.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:05.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:31:05.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:31:05.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:05.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:31:05.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:31:05.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:05.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:31:05.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:31:05.365 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.365 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:05.365 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.365 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:05.365 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:05.365 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 75.664377067999993, 2024-09-18T11:31:05.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:31:05.366 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:05.366 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.366 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:05.366 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:05.366 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:05.366 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:31:05.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.366 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:05.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:05.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:05.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:05.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:05.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:05.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:31:05.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:05.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:31:05.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:31:05.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:05.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:31:05.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:31:05.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:31:05.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:31:05.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:05.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:31:05.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:31:05.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:05.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:31:05.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:31:05.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:05.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:31:05.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:31:05.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:05.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:31:05.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:31:05.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:31:05.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:31:05.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:05.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:31:05.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:31:05.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:05.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:31:05.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:31:05.371 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.371 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:05.371 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.371 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:05.371 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:05.371 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 75.664284589000005, 2024-09-18T11:31:05.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:31:05.371 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:05.371 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.371 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:05.371 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:05.372 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:05.372 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:31:05.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.372 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:05.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:05.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:05.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:05.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:05.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:05.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:05.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:05.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:31:05.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:31:05.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:05.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:31:05.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:31:05.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:31:05.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:31:05.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:05.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:31:05.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:31:05.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:05.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:31:05.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:31:05.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:05.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:31:05.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:31:05.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:05.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:31:05.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:31:05.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:31:05.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:31:05.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:05.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:31:05.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:31:05.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:05.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:31:05.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:31:05.378 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.378 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:05.378 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.378 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1788 4.8 e60/22 4:1898b15d:::smithi11939506-25:head v 60'12, mlcod=58'9)", 2024-09-18T11:31:05.378 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:05.378 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.331879099999998, 2024-09-18T11:31:05.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058348599999999999, 2024-09-18T11:31:05.378 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:05.378 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.378 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:05.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:05.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:05.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:05.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:05.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:05.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:05.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916949+0000", 2024-09-18T11:31:05.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:05.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:05.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917602+0000", 2024-09-18T11:31:05.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00065337100000000003 2024-09-18T11:31:05.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:05.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917621+0000", 2024-09-18T11:31:05.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9287e-05 2024-09-18T11:31:05.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917629+0000", 2024-09-18T11:31:05.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3780000000000003e-06 2024-09-18T11:31:05.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:05.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917677+0000", 2024-09-18T11:31:05.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8068000000000001e-05 2024-09-18T11:31:05.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:05.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917699+0000", 2024-09-18T11:31:05.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2149000000000001e-05 2024-09-18T11:31:05.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:05.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922747+0000", 2024-09-18T11:31:05.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050477609999999996 2024-09-18T11:31:05.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:05.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:05.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:05.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922785+0000", 2024-09-18T11:31:05.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8726000000000002e-05 2024-09-18T11:31:05.382 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.382 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:05.383 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.383 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:05.383 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:05.383 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:05.384 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:05.383+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:05.384 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:31:05.485 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:05.483+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:05.485 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:05.485 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:05.485 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:05.486 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:05.494 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:31:05.586 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:05.585+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:05.631 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:05.631 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:05.631 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:05.631 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:05.631 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:05.639 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:31:05.687 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:05.686+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:05.739 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:05.739 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:05.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:05.739 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:05.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.739 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:887 4.7 4:e0d95e10:::smithi11939506-22:head [read 0~2352947,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:05.739 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:31:05.740 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.620492990000002, 2024-09-18T11:31:05.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079115169999999999, 2024-09-18T11:31:05.740 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:05.740 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:05.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:05.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:05.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:05.740 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 887 2024-09-18T11:31:05.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.740 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:05.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:05.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:31:05.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:05.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:31:05.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:05.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116365+0000", 2024-09-18T11:31:05.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:05.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:05.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116376+0000", 2024-09-18T11:31:05.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1471000000000001e-05 2024-09-18T11:31:05.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:05.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116394+0000", 2024-09-18T11:31:05.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8088e-05 2024-09-18T11:31:05.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116402+0000", 2024-09-18T11:31:05.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2320000000000004e-06 2024-09-18T11:31:05.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116458+0000", 2024-09-18T11:31:05.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6536000000000002e-05 2024-09-18T11:31:05.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:05.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116466+0000", 2024-09-18T11:31:05.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0819999999999999e-06 2024-09-18T11:31:05.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.122509+0000", 2024-09-18T11:31:05.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060431879999999997 2024-09-18T11:31:05.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:05.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.122577+0000", 2024-09-18T11:31:05.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7646999999999998e-05 2024-09-18T11:31:05.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:05.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124278+0000", 2024-09-18T11:31:05.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017013200000000001 2024-09-18T11:31:05.744 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.744 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:05.744 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.745 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:888 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:05.745 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:05.745 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.620439527000002, 2024-09-18T11:31:05.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082282120000000004, 2024-09-18T11:31:05.745 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:05.745 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:05.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:05.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:05.745 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:05.745 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 888 2024-09-18T11:31:05.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.746 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:05.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:05.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:05.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:05.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:05.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:05.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:05.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:05.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116423+0000", 2024-09-18T11:31:05.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1e-06 2024-09-18T11:31:05.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:05.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:31:05.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5510000000000001e-06 2024-09-18T11:31:05.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116431+0000", 2024-09-18T11:31:05.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6109999999999998e-06 2024-09-18T11:31:05.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116527+0000", 2024-09-18T11:31:05.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6212999999999995e-05 2024-09-18T11:31:05.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:05.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116532+0000", 2024-09-18T11:31:05.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6140000000000002e-06 2024-09-18T11:31:05.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124314+0000", 2024-09-18T11:31:05.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077818929999999998 2024-09-18T11:31:05.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:05.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124411+0000", 2024-09-18T11:31:05.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6992999999999998e-05 2024-09-18T11:31:05.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:05.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124649+0000", 2024-09-18T11:31:05.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237729 2024-09-18T11:31:05.750 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.750 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:05.750 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.750 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:889 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:05.750 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:05.750 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.620414877000002, 2024-09-18T11:31:05.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085383349999999993, 2024-09-18T11:31:05.751 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:05.751 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:05.751 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:05.751 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:05.751 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:05.751 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 889 2024-09-18T11:31:05.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.751 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:05.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:05.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:05.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:05.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:05.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:05.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:05.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:05.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116447+0000", 2024-09-18T11:31:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.854e-06 2024-09-18T11:31:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116452+0000", 2024-09-18T11:31:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0250000000000002e-06 2024-09-18T11:31:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116455+0000", 2024-09-18T11:31:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8739999999999999e-06 2024-09-18T11:31:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116565+0000", 2024-09-18T11:31:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110741 2024-09-18T11:31:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116571+0000", 2024-09-18T11:31:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4990000000000002e-06 2024-09-18T11:31:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124691+0000", 2024-09-18T11:31:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081200339999999999 2024-09-18T11:31:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:05.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124784+0000", 2024-09-18T11:31:05.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2886999999999993e-05 2024-09-18T11:31:05.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:05.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124983+0000", 2024-09-18T11:31:05.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019969000000000001 2024-09-18T11:31:05.755 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.755 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:05.755 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.755 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:46 4.a e31/22 4:552523b6:::smithi11939506-38:head v 31'20, mlcod=30'18)", 2024-09-18T11:31:05.756 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:31:05.756 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.606463245, 2024-09-18T11:31:05.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080208420000000002, 2024-09-18T11:31:05.756 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:05.756 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.756 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:05.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:05.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:31:05.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:05.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:31:05.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:05.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130396+0000", 2024-09-18T11:31:05.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:05.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133245+0000", 2024-09-18T11:31:05.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028486929999999998 2024-09-18T11:31:05.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:05.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133253+0000", 2024-09-18T11:31:05.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1540000000000002e-06 2024-09-18T11:31:05.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133261+0000", 2024-09-18T11:31:05.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3620000000000006e-06 2024-09-18T11:31:05.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133319+0000", 2024-09-18T11:31:05.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7719000000000001e-05 2024-09-18T11:31:05.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:05.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133341+0000", 2024-09-18T11:31:05.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1946999999999998e-05 2024-09-18T11:31:05.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:05.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138368+0000", 2024-09-18T11:31:05.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005026712 2024-09-18T11:31:05.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:05.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138418+0000", 2024-09-18T11:31:05.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9864000000000002e-05 2024-09-18T11:31:05.760 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.760 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:05.760 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.760 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1126 3.0 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head [write 0~2520064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:31:05.760 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:31:05.760 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.754972899000002, 2024-09-18T11:31:05.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086197050000000001, 2024-09-18T11:31:05.760 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:05.761 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.761 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:05.761 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:05.761 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:05.761 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1126 2024-09-18T11:31:05.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.761 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:05.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:05.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:31:05.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:05.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:31:05.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:05.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981886+0000", 2024-09-18T11:31:05.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:05.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:05.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983017+0000", 2024-09-18T11:31:05.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00113131 2024-09-18T11:31:05.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:05.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983037+0000", 2024-09-18T11:31:05.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0231e-05 2024-09-18T11:31:05.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983044+0000", 2024-09-18T11:31:05.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9539999999999998e-06 2024-09-18T11:31:05.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983087+0000", 2024-09-18T11:31:05.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3040000000000001e-05 2024-09-18T11:31:05.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:05.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983193+0000", 2024-09-18T11:31:05.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105682 2024-09-18T11:31:05.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:05.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983287+0000", 2024-09-18T11:31:05.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4066000000000004e-05 2024-09-18T11:31:05.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:05.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.986918+0000", 2024-09-18T11:31:05.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036311970000000001 2024-09-18T11:31:05.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990442+0000", 2024-09-18T11:31:05.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035243119999999999 2024-09-18T11:31:05.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:05.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990476+0000", 2024-09-18T11:31:05.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4028e-05 2024-09-18T11:31:05.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990507+0000", 2024-09-18T11:31:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0284999999999999e-05 2024-09-18T11:31:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:262 4.9 e53/22 4:99feffe9:::smithi11939506-45:13 v 53'17, mlcod=51'14)", 2024-09-18T11:31:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.737869084, 2024-09-18T11:31:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013878679, 2024-09-18T11:31:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998997+0000", 2024-09-18T11:31:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1099999999999999e-06 2024-09-18T11:31:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999008+0000", 2024-09-18T11:31:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1321000000000001e-05 2024-09-18T11:31:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999015+0000", 2024-09-18T11:31:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3000000000000004e-06 2024-09-18T11:31:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999055+0000", 2024-09-18T11:31:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0058000000000002e-05 2024-09-18T11:31:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999070+0000", 2024-09-18T11:31:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4328000000000001e-05 2024-09-18T11:31:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012465+0000", 2024-09-18T11:31:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013395638 2024-09-18T11:31:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012870+0000", 2024-09-18T11:31:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040423400000000001 2024-09-18T11:31:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:205 4.b e53/22 4:dba5bd39:::smithi11939506-17:13 v 53'77, mlcod=51'74)", 2024-09-18T11:31:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.737819822999999, 2024-09-18T11:31:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013359586, 2024-09-18T11:31:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999048+0000", 2024-09-18T11:31:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3550000000000005e-06 2024-09-18T11:31:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999056+0000", 2024-09-18T11:31:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3259999999999996e-06 2024-09-18T11:31:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999063+0000", 2024-09-18T11:31:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6409999999999996e-06 2024-09-18T11:31:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999098+0000", 2024-09-18T11:31:05.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5203999999999997e-05 2024-09-18T11:31:05.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:05.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999115+0000", 2024-09-18T11:31:05.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6478999999999999e-05 2024-09-18T11:31:05.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:05.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012362+0000", 2024-09-18T11:31:05.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01324768 2024-09-18T11:31:05.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:05.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012400+0000", 2024-09-18T11:31:05.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.731e-05 2024-09-18T11:31:05.775 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.775 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:05.775 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.775 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:88 4.a e53/22 4:588debc2:::smithi11939506-18:13 v 53'45, mlcod=51'42)", 2024-09-18T11:31:05.775 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:05.775 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.7350374, 2024-09-18T11:31:05.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011307667, 2024-09-18T11:31:05.775 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:05.775 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.776 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:05.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:05.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:05.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:05.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:05.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:05.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001822+0000", 2024-09-18T11:31:05.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:05.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001827+0000", 2024-09-18T11:31:05.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.378e-06 2024-09-18T11:31:05.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:05.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001839+0000", 2024-09-18T11:31:05.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1225e-05 2024-09-18T11:31:05.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001844+0000", 2024-09-18T11:31:05.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2299999999999999e-06 2024-09-18T11:31:05.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001891+0000", 2024-09-18T11:31:05.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6961e-05 2024-09-18T11:31:05.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:05.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001906+0000", 2024-09-18T11:31:05.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5158e-05 2024-09-18T11:31:05.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:05.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013103+0000", 2024-09-18T11:31:05.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011197314 2024-09-18T11:31:05.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:05.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013130+0000", 2024-09-18T11:31:05.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6900999999999999e-05 2024-09-18T11:31:05.780 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.780 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:05.780 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.780 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:89 4.a e53/22 4:552523b6:::smithi11939506-38:13 v 53'47, mlcod=51'42)", 2024-09-18T11:31:05.780 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:05.780 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.73447818, 2024-09-18T11:31:05.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011327614, 2024-09-18T11:31:05.780 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:05.780 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.780 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:05.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:05.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:05.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:05.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:05.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:05.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002381+0000", 2024-09-18T11:31:05.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:05.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002397+0000", 2024-09-18T11:31:05.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5688e-05 2024-09-18T11:31:05.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:05.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002406+0000", 2024-09-18T11:31:05.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0229999999999998e-06 2024-09-18T11:31:05.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002415+0000", 2024-09-18T11:31:05.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8959999999999992e-06 2024-09-18T11:31:05.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012386+0000", 2024-09-18T11:31:05.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0099711450000000007 2024-09-18T11:31:05.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:05.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012405+0000", 2024-09-18T11:31:05.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9389e-05 2024-09-18T11:31:05.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:05.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013676+0000", 2024-09-18T11:31:05.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012708649999999999 2024-09-18T11:31:05.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:05.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013709+0000", 2024-09-18T11:31:05.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3207000000000002e-05 2024-09-18T11:31:05.784 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.784 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:05.784 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.784 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1643 4.7 4:e73ba68f:::smithi11939506-1:head [read 0~3983226,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:05.784 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:31:05.785 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.402973004, 2024-09-18T11:31:05.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079617669999999998, 2024-09-18T11:31:05.785 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:05.785 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:05.785 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:05.785 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:05.785 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:05.785 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1643 2024-09-18T11:31:05.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.785 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:05.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:05.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:31:05.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:05.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:31:05.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:05.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:31:05.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:05.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333889+0000", 2024-09-18T11:31:05.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8890000000000002e-06 2024-09-18T11:31:05.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:05.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333896+0000", 2024-09-18T11:31:05.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.156e-06 2024-09-18T11:31:05.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333900+0000", 2024-09-18T11:31:05.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.138e-06 2024-09-18T11:31:05.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.336677+0000", 2024-09-18T11:31:05.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027774169999999999 2024-09-18T11:31:05.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:05.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.336768+0000", 2024-09-18T11:31:05.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0983999999999995e-05 2024-09-18T11:31:05.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:05.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341849+0000", 2024-09-18T11:31:05.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050805140000000004 2024-09-18T11:31:05.788 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.788 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:05.788 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.789 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1644 4.7 4:e73ba68f:::smithi11939506-1:head [sparse-read 0~2695582] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:05.789 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:05.789 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.402955882000001, 2024-09-18T11:31:05.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084764039999999999, 2024-09-18T11:31:05.789 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:05.789 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:05.789 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:05.789 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:05.789 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:05.789 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1644 2024-09-18T11:31:05.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.789 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:05.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:05.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:05.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:05.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:05.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:05.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:05.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:05.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333906+0000", 2024-09-18T11:31:05.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.243e-06 2024-09-18T11:31:05.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:05.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333911+0000", 2024-09-18T11:31:05.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2959999999999998e-06 2024-09-18T11:31:05.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333915+0000", 2024-09-18T11:31:05.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6720000000000002e-06 2024-09-18T11:31:05.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341879+0000", 2024-09-18T11:31:05.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079638699999999996 2024-09-18T11:31:05.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:05.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341978+0000", 2024-09-18T11:31:05.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9562000000000007e-05 2024-09-18T11:31:05.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:05.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342380+0000", 2024-09-18T11:31:05.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000402113 2024-09-18T11:31:05.793 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.793 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:05.793 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.793 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1645 4.7 4:e73ba68f:::smithi11939506-1:head [read 0~3983226,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:05.793 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:05.793 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.402940988000001, 2024-09-18T11:31:05.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010539468999999999, 2024-09-18T11:31:05.794 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:05.794 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:05.794 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:05.794 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:05.794 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:05.794 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1645 2024-09-18T11:31:05.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.794 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:05.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:05.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:05.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:05.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:05.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:05.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:05.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:05.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333921+0000", 2024-09-18T11:31:05.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8909999999999999e-06 2024-09-18T11:31:05.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:05.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333927+0000", 2024-09-18T11:31:05.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4229999999999996e-06 2024-09-18T11:31:05.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333932+0000", 2024-09-18T11:31:05.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8280000000000002e-06 2024-09-18T11:31:05.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342418+0000", 2024-09-18T11:31:05.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084862219999999999 2024-09-18T11:31:05.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:05.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342515+0000", 2024-09-18T11:31:05.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7287999999999994e-05 2024-09-18T11:31:05.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:05.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.344458+0000", 2024-09-18T11:31:05.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019431660000000001 2024-09-18T11:31:05.798 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.798 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:05.798 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.798 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1669 4.4 4:2d9c3f7f:::smithi11939506-5:head [read 0~3764735,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-18T11:31:05.798 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:31:05.798 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.913992090000001, 2024-09-18T11:31:05.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0083694990000000007, 2024-09-18T11:31:05.798 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:05.798 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:05.798 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:05.798 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:05.799 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:05.799 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1669 2024-09-18T11:31:05.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.799 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:05.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:05.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:31:05.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:05.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:31:05.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:05.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822867+0000", 2024-09-18T11:31:05.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:05.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822871+0000", 2024-09-18T11:31:05.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:31:05.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:05.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822877+0000", 2024-09-18T11:31:05.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8000000000000004e-06 2024-09-18T11:31:05.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822880+0000", 2024-09-18T11:31:05.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5669999999999999e-06 2024-09-18T11:31:05.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822974+0000", 2024-09-18T11:31:05.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4048999999999993e-05 2024-09-18T11:31:05.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:05.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822980+0000", 2024-09-18T11:31:05.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2789999999999998e-06 2024-09-18T11:31:05.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.829160+0000", 2024-09-18T11:31:05.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0061800179999999998 2024-09-18T11:31:05.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:05.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.829248+0000", 2024-09-18T11:31:05.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7707000000000006e-05 2024-09-18T11:31:05.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:05.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.831237+0000", 2024-09-18T11:31:05.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001989827 2024-09-18T11:31:05.803 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.803 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:05.803 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.803 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:100 4.a e58/22 4:5975a5ec:::smithi11939506-36:18 v 58'55, mlcod=56'50)", 2024-09-18T11:31:05.803 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:05.804 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.463710851999998, 2024-09-18T11:31:05.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0088255, 2024-09-18T11:31:05.804 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:05.804 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.804 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:05.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:05.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:05.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:05.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:05.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:05.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:05.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:05.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273155+0000", 2024-09-18T11:31:05.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.212e-06 2024-09-18T11:31:05.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:05.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273164+0000", 2024-09-18T11:31:05.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7609999999999996e-06 2024-09-18T11:31:05.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273168+0000", 2024-09-18T11:31:05.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2819999999999998e-06 2024-09-18T11:31:05.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274162+0000", 2024-09-18T11:31:05.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099398600000000009 2024-09-18T11:31:05.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:05.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274180+0000", 2024-09-18T11:31:05.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8236e-05 2024-09-18T11:31:05.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:05.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281943+0000", 2024-09-18T11:31:05.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007763308 2024-09-18T11:31:05.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:05.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281975+0000", 2024-09-18T11:31:05.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1202999999999999e-05 2024-09-18T11:31:05.808 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.808 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:05.808 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.808 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1954 4.7 4:ef60946b:::smithi11939506-8:head [read 0~2890771,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:05.808 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:05.808 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.396959812, 2024-09-18T11:31:05.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091142180000000003, 2024-09-18T11:31:05.808 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:05.808 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:05.808 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:05.808 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:05.809 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:05.809 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1954 2024-09-18T11:31:05.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.809 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:05.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:05.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:05.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:05.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:05.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:05.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:05.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:05.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:05.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339903+0000", 2024-09-18T11:31:05.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7349999999999998e-06 2024-09-18T11:31:05.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:05.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339909+0000", 2024-09-18T11:31:05.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-09-18T11:31:05.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339913+0000", 2024-09-18T11:31:05.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7670000000000001e-06 2024-09-18T11:31:05.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339970+0000", 2024-09-18T11:31:05.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6988999999999999e-05 2024-09-18T11:31:05.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:05.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339975+0000", 2024-09-18T11:31:05.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7929999999999997e-06 2024-09-18T11:31:05.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346917+0000", 2024-09-18T11:31:05.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069426640000000003 2024-09-18T11:31:05.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:05.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.347026+0000", 2024-09-18T11:31:05.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108241 2024-09-18T11:31:05.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:05.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.349014+0000", 2024-09-18T11:31:05.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001988723 2024-09-18T11:31:05.813 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.813 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:05.813 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.813 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1956 4.6 4:67c1765a:::smithi11939506-30:head [write 705927~558502] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:05.813 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:05.813 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.384850871999999, 2024-09-18T11:31:05.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011813239, 2024-09-18T11:31:05.813 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:05.813 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.814 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:05.814 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:05.814 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:05.814 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1956 2024-09-18T11:31:05.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.814 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:05.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:05.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:05.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:05.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:05.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:05.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352007+0000", 2024-09-18T11:31:05.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:05.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:05.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352229+0000", 2024-09-18T11:31:05.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222024 2024-09-18T11:31:05.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:05.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352251+0000", 2024-09-18T11:31:05.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2512000000000001e-05 2024-09-18T11:31:05.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352260+0000", 2024-09-18T11:31:05.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7339999999999997e-06 2024-09-18T11:31:05.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352307+0000", 2024-09-18T11:31:05.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7061999999999998e-05 2024-09-18T11:31:05.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:05.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352408+0000", 2024-09-18T11:31:05.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101216 2024-09-18T11:31:05.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:05.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352550+0000", 2024-09-18T11:31:05.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014136199999999999 2024-09-18T11:31:05.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:05.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357684+0000", 2024-09-18T11:31:05.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051342499999999999 2024-09-18T11:31:05.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363736+0000", 2024-09-18T11:31:05.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006052577 2024-09-18T11:31:05.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:05.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363778+0000", 2024-09-18T11:31:05.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1989000000000001e-05 2024-09-18T11:31:05.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:05.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363822+0000", 2024-09-18T11:31:05.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3894999999999999e-05 2024-09-18T11:31:05.819 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.819 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:05.819 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.819 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2093 4.7 4:e0d95e10:::smithi11939506-22:head [assert-version v53,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e73)", 2024-09-18T11:31:05.819 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:55.369779+0000", 2024-09-18T11:31:05.819 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 10.367080657000001, 2024-09-18T11:31:05.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078628099999999996, 2024-09-18T11:31:05.819 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:05.819 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.819 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:05.819 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:05.820 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:05.820 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2093 2024-09-18T11:31:05.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.820 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:05.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:05.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369779+0000", 2024-09-18T11:31:05.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:05.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369779+0000", 2024-09-18T11:31:05.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:05.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369775+0000", 2024-09-18T11:31:05.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999957 2024-09-18T11:31:05.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:05.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369785+0000", 2024-09-18T11:31:05.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7280000000000007e-06 2024-09-18T11:31:05.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:05.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369798+0000", 2024-09-18T11:31:05.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3509000000000001e-05 2024-09-18T11:31:05.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369806+0000", 2024-09-18T11:31:05.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3749999999999997e-06 2024-09-18T11:31:05.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369837+0000", 2024-09-18T11:31:05.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1440999999999999e-05 2024-09-18T11:31:05.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:05.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369886+0000", 2024-09-18T11:31:05.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9103000000000001e-05 2024-09-18T11:31:05.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:05.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.376462+0000", 2024-09-18T11:31:05.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065761980000000001 2024-09-18T11:31:05.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:05.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.377077+0000", 2024-09-18T11:31:05.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00061434599999999999 2024-09-18T11:31:05.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.377555+0000", 2024-09-18T11:31:05.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047826200000000002 2024-09-18T11:31:05.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:05.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.377599+0000", 2024-09-18T11:31:05.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4366999999999997e-05 2024-09-18T11:31:05.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:05.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.377642+0000", 2024-09-18T11:31:05.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2824999999999997e-05 2024-09-18T11:31:05.825 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.825 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:05.825 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.825 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2278 4.f 4:fc192b81:::smithi11939506-31:head [stat,delete] snapc 26=[26,25,24,1e,1c,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T11:31:05.825 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:05.825 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.3538136600000001, 2024-09-18T11:31:05.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01018694, 2024-09-18T11:31:05.825 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:05.825 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.825 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:05.825 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:05.825 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:05.826 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2278 2024-09-18T11:31:05.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.826 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:05.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:05.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:05.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:05.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:05.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:05.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383044+0000", 2024-09-18T11:31:05.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:05.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:05.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383054+0000", 2024-09-18T11:31:05.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5969999999999998e-06 2024-09-18T11:31:05.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:05.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383071+0000", 2024-09-18T11:31:05.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7353000000000001e-05 2024-09-18T11:31:05.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383081+0000", 2024-09-18T11:31:05.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9170000000000001e-06 2024-09-18T11:31:05.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383150+0000", 2024-09-18T11:31:05.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8503999999999993e-05 2024-09-18T11:31:05.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:05.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383158+0000", 2024-09-18T11:31:05.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5089999999999993e-06 2024-09-18T11:31:05.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385396+0000", 2024-09-18T11:31:05.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022381850000000002 2024-09-18T11:31:05.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:05.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385494+0000", 2024-09-18T11:31:05.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7751000000000007e-05 2024-09-18T11:31:05.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:05.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385697+0000", 2024-09-18T11:31:05.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020247400000000001 2024-09-18T11:31:05.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.390505+0000", 2024-09-18T11:31:05.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048084039999999996 2024-09-18T11:31:05.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:05.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393148+0000", 2024-09-18T11:31:05.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026432399999999998 2024-09-18T11:31:05.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:05.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393194+0000", 2024-09-18T11:31:05.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5259999999999997e-05 2024-09-18T11:31:05.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:05.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393233+0000", 2024-09-18T11:31:05.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9659000000000001e-05 2024-09-18T11:31:05.831 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.831 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:05.831 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.831 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2342 4.f 4:f3c51b3f:::smithi11939506-21:head [stat,delete] snapc 26=[26,25,24,1e,1c,19,18] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:05.832 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:05.832 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.3500576479999999, 2024-09-18T11:31:05.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011167565000000001, 2024-09-18T11:31:05.832 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:05.832 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.832 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:05.832 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:05.832 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:05.832 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2342 2024-09-18T11:31:05.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.832 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:05.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:05.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:05.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:05.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:05.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:05.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:31:05.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:05.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:05.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:31:05.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1439999999999997e-06 2024-09-18T11:31:05.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:05.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386820+0000", 2024-09-18T11:31:05.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2651e-05 2024-09-18T11:31:05.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386825+0000", 2024-09-18T11:31:05.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3150000000000002e-06 2024-09-18T11:31:05.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386868+0000", 2024-09-18T11:31:05.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3832999999999999e-05 2024-09-18T11:31:05.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:05.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:31:05.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0409999999999998e-06 2024-09-18T11:31:05.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390445+0000", 2024-09-18T11:31:05.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035689459999999999 2024-09-18T11:31:05.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:05.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390539+0000", 2024-09-18T11:31:05.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3791999999999999e-05 2024-09-18T11:31:05.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:05.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390748+0000", 2024-09-18T11:31:05.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000208863 2024-09-18T11:31:05.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397659+0000", 2024-09-18T11:31:05.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069110710000000004 2024-09-18T11:31:05.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:05.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397873+0000", 2024-09-18T11:31:05.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021343999999999999 2024-09-18T11:31:05.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:05.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397923+0000", 2024-09-18T11:31:05.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0300000000000003e-05 2024-09-18T11:31:05.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:05.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397970+0000", 2024-09-18T11:31:05.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6968999999999997e-05 2024-09-18T11:31:05.838 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.838 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:05.838 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.838 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2429 4.b 4:d8d14c58:::smithi11939506-49:head [write 796698~561170,stat] snapc 2b=[2b,2a,26,25,24,1e,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T11:31:05.838 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:05.838 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.72168847400000002, 2024-09-18T11:31:05.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097188989999999996, 2024-09-18T11:31:05.838 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:05.838 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:05.838 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:05.839 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:05.839 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:05.839 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2429 2024-09-18T11:31:05.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.839 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:05.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:05.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:05.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:05.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:05.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:05.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:05.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015170+0000", 2024-09-18T11:31:05.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:05.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:05.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015435+0000", 2024-09-18T11:31:05.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026533299999999998 2024-09-18T11:31:05.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:05.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015455+0000", 2024-09-18T11:31:05.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9623000000000002e-05 2024-09-18T11:31:05.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:05.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015461+0000", 2024-09-18T11:31:05.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1859999999999997e-06 2024-09-18T11:31:05.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015535+0000", 2024-09-18T11:31:05.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4555999999999996e-05 2024-09-18T11:31:05.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:05.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015543+0000", 2024-09-18T11:31:05.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9549999999999992e-06 2024-09-18T11:31:05.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:05.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017106+0000", 2024-09-18T11:31:05.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015630290000000001 2024-09-18T11:31:05.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:05.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017183+0000", 2024-09-18T11:31:05.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6953999999999995e-05 2024-09-18T11:31:05.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:05.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017301+0000", 2024-09-18T11:31:05.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011734 2024-09-18T11:31:05.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:05.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.020225+0000", 2024-09-18T11:31:05.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029244090000000002 2024-09-18T11:31:05.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:05.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024801+0000", 2024-09-18T11:31:05.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045764129999999997 2024-09-18T11:31:05.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:05.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024840+0000", 2024-09-18T11:31:05.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8325999999999999e-05 2024-09-18T11:31:05.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:05.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:05.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:05.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024890+0000", 2024-09-18T11:31:05.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0596999999999997e-05 2024-09-18T11:31:05.844 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.844 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:05.844 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.845 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:05.845 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:05.845 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:05.846 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:31:05.847 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:05.845+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:05.943 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:05.943 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:05.943 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:05.944 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:05.947 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:05.946+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:05.952 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:31:06.005 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:06.004+0000 7fcb682b1640 -1 osd.7 82 handle_osd_map injecting map crc failure 2024-09-18T11:31:06.007 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is unset 2024-09-18T11:31:06.008 INFO:tasks.rados.rados.0.smithi119.stdout:1179: done (3 left) 2024-09-18T11:31:06.008 INFO:tasks.rados.rados.0.smithi119.stdout:1180: done (2 left) 2024-09-18T11:31:06.008 INFO:tasks.rados.rados.0.smithi119.stdout:1181: done (1 left) 2024-09-18T11:31:06.008 INFO:tasks.rados.rados.0.smithi119.stdout:1182: done (0 left) 2024-09-18T11:31:06.008 INFO:tasks.rados.rados.0.smithi119.stdout:1183: read oid 23 snap -1 2024-09-18T11:31:06.008 INFO:tasks.rados.rados.0.smithi119.stdout:1183: expect deleted 2024-09-18T11:31:06.008 INFO:tasks.rados.rados.0.smithi119.stdout:1184: write oid 16 current snap is 26 2024-09-18T11:31:06.008 INFO:tasks.rados.rados.0.smithi119.stdout:1184: seq_num 215 ranges {413505=636419,1664805=689064,3038061=492071} 2024-09-18T11:31:06.013 INFO:tasks.rados.rados.0.smithi119.stdout:1184: writing smithi11939506-16 from 413505 to 1049924 tid 1 2024-09-18T11:31:06.021 INFO:tasks.rados.rados.0.smithi119.stdout:1184: writing smithi11939506-16 from 1664805 to 2353869 tid 2 2024-09-18T11:31:06.027 INFO:tasks.rados.rados.0.smithi119.stdout:1184: writing smithi11939506-16 from 3038061 to 3530132 tid 3 2024-09-18T11:31:06.028 INFO:tasks.rados.rados.0.smithi119.stdout:1185: set_chunk oid 21 target oid 2024-09-18T11:31:06.040 INFO:tasks.rados.rados.0.smithi119.stdout:1185: set_chunk oid 21 offset: 2655232 length: 568832 target oid 375fd160d013aaaf783aa2b97a970e891c2bd639e23a7f045ef259b9223c9ebe offset: 0 snap -1 2024-09-18T11:31:06.040 INFO:tasks.rados.rados.0.smithi119.stdout:1186: read oid 29 snap -1 2024-09-18T11:31:06.040 INFO:tasks.rados.rados.0.smithi119.stdout:1186: expect (ObjNum 210 snap 23 seq_num 210) 2024-09-18T11:31:06.040 INFO:tasks.rados.rados.0.smithi119.stdout:1187: read oid 44 snap -1 2024-09-18T11:31:06.040 INFO:tasks.rados.rados.0.smithi119.stdout:1187: expect (ObjNum 206 snap 23 seq_num 206) 2024-09-18T11:31:06.041 INFO:tasks.rados.rados.0.smithi119.stdout:1188: tier_evict oid 19 2024-09-18T11:31:06.041 INFO:tasks.rados.rados.0.smithi119.stdout:1188: tier_evict oid 19 snap -1 2024-09-18T11:31:06.041 INFO:tasks.rados.rados.0.smithi119.stdout:1183: done (5 left) 2024-09-18T11:31:06.041 INFO:tasks.rados.rados.0.smithi119.stdout:1189: write oid 49 current snap is 26 2024-09-18T11:31:06.041 INFO:tasks.rados.rados.0.smithi119.stdout:1189: seq_num 216 ranges {430312=711668,1723287=549075,2781837=703343} 2024-09-18T11:31:06.043 INFO:tasks.rados.rados.0.smithi119.stdout:1189: writing smithi11939506-49 from 430312 to 1141980 tid 1 2024-09-18T11:31:06.047 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:06.046+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:06.049 INFO:tasks.rados.rados.0.smithi119.stdout:1189: writing smithi11939506-49 from 1723287 to 2272362 tid 2 2024-09-18T11:31:06.056 INFO:tasks.rados.rados.0.smithi119.stdout:1189: writing smithi11939506-49 from 2781837 to 3485180 tid 3 2024-09-18T11:31:06.056 INFO:tasks.rados.rados.0.smithi119.stdout:1190: write (excl) oid 32 current snap is 26 2024-09-18T11:31:06.056 INFO:tasks.rados.rados.0.smithi119.stdout:1190: seq_num 217 ranges {447119=786917,1665416=692733,2976554=514615} 2024-09-18T11:31:06.063 INFO:tasks.rados.rados.0.smithi119.stdout:1190: writing smithi11939506-32 from 447119 to 1234036 tid 1 2024-09-18T11:31:06.069 INFO:tasks.rados.rados.0.smithi119.stdout:1190: writing smithi11939506-32 from 1665416 to 2358149 tid 2 2024-09-18T11:31:06.076 INFO:tasks.rados.rados.0.smithi119.stdout:1190: writing smithi11939506-32 from 2976554 to 3491169 tid 3 2024-09-18T11:31:06.076 INFO:tasks.rados.rados.0.smithi119.stdout:1191: delete oid 30 current snap is 26 2024-09-18T11:31:06.076 INFO:tasks.rados.rados.0.smithi119.stdout:1184: finishing write tid 1 to smithi11939506-16 2024-09-18T11:31:06.076 INFO:tasks.rados.rados.0.smithi119.stdout:1184: finishing write tid 2 to smithi11939506-16 2024-09-18T11:31:06.076 INFO:tasks.rados.rados.0.smithi119.stdout:1184: finishing write tid 3 to smithi11939506-16 2024-09-18T11:31:06.076 INFO:tasks.rados.rados.0.smithi119.stdout:1184: finishing write tid 4 to smithi11939506-16 2024-09-18T11:31:06.076 INFO:tasks.rados.rados.0.smithi119.stdout:1184: finishing write tid 5 to smithi11939506-16 2024-09-18T11:31:06.076 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 153 (ObjNum 215 snap 26 seq_num 215) dirty exists 2024-09-18T11:31:06.077 INFO:tasks.rados.rados.0.smithi119.stdout:1184: left oid 16 (ObjNum 215 snap 26 seq_num 215) 2024-09-18T11:31:06.077 INFO:tasks.rados.rados.0.smithi119.stdout:1188: got (22) Invalid argument 2024-09-18T11:31:06.077 INFO:tasks.rados.rados.0.smithi119.stdout:1185: finishing set_chunk to oid 21 2024-09-18T11:31:06.077 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 148 (ObjNum 213 snap 24 seq_num 213) dirty exists 2024-09-18T11:31:06.077 INFO:tasks.rados.rados.0.smithi119.stdout:1186: expect (ObjNum 210 snap 23 seq_num 210) 2024-09-18T11:31:06.105 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:06.105 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:06.105 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:06.105 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:06.105 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:06.114 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:31:06.155 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:06.154+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:06.249 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:31:06.250 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:06.248+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:06.254 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:06.254 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:06.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:06.254 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:06.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.255 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1046 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'24, mlcod=30'21)", 2024-09-18T11:31:06.255 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:31:06.255 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 50.078826847000002, 2024-09-18T11:31:06.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042519100000000002, 2024-09-18T11:31:06.255 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:06.255 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:06.255 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:06.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:31:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:31:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173782+0000", 2024-09-18T11:31:06.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174207+0000", 2024-09-18T11:31:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00042479799999999999 2024-09-18T11:31:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174224+0000", 2024-09-18T11:31:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7513999999999999e-05 2024-09-18T11:31:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:06.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174232+0000", 2024-09-18T11:31:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0840000000000003e-06 2024-09-18T11:31:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174297+0000", 2024-09-18T11:31:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5215000000000005e-05 2024-09-18T11:31:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:06.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174336+0000", 2024-09-18T11:31:06.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9403000000000002e-05 2024-09-18T11:31:06.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.177990+0000", 2024-09-18T11:31:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036533709999999999 2024-09-18T11:31:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.178035+0000", 2024-09-18T11:31:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5495999999999999e-05 2024-09-18T11:31:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.260 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1047 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'25, mlcod=35'23)", 2024-09-18T11:31:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:31:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 50.072854649, 2024-09-18T11:31:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038129129999999998, 2024-09-18T11:31:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:31:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:06.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:31:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179754+0000", 2024-09-18T11:31:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180088+0000", 2024-09-18T11:31:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00033395000000000001 2024-09-18T11:31:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180105+0000", 2024-09-18T11:31:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7039000000000001e-05 2024-09-18T11:31:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180112+0000", 2024-09-18T11:31:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3710000000000002e-06 2024-09-18T11:31:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180176+0000", 2024-09-18T11:31:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3178999999999999e-05 2024-09-18T11:31:06.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180197+0000", 2024-09-18T11:31:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1687000000000001e-05 2024-09-18T11:31:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183542+0000", 2024-09-18T11:31:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003344877 2024-09-18T11:31:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:06.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183568+0000", 2024-09-18T11:31:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6146e-05 2024-09-18T11:31:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:31:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:31:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.269235979999998, 2024-09-18T11:31:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:31:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:31:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:31:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:31:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:06.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:31:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:31:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:31:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:31:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:31:06.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:31:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:31:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:31:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:31:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:31:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:06.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:31:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:31:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:31:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:31:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:31:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:06.269 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.034405026999998, 2024-09-18T11:31:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:31:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:31:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:31:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:31:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:31:06.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:31:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:31:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:31:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:31:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:31:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:06.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:31:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:31:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:31:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:31:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:31:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:31:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:06.273 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1512 4.4 e51/22 4:2c93196f:::smithi11939506-28:head v 51'51, mlcod=51'48)", 2024-09-18T11:31:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:32.782201+0000", 2024-09-18T11:31:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.470409072999999, 2024-09-18T11:31:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00363554, 2024-09-18T11:31:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782201+0000", 2024-09-18T11:31:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782201+0000", 2024-09-18T11:31:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782200+0000", 2024-09-18T11:31:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782564+0000", 2024-09-18T11:31:06.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000364612 2024-09-18T11:31:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782585+0000", 2024-09-18T11:31:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0418999999999999e-05 2024-09-18T11:31:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782594+0000", 2024-09-18T11:31:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9570000000000008e-06 2024-09-18T11:31:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:06.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782663+0000", 2024-09-18T11:31:06.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9330000000000002e-05 2024-09-18T11:31:06.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:06.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782688+0000", 2024-09-18T11:31:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5420000000000001e-05 2024-09-18T11:31:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.785790+0000", 2024-09-18T11:31:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031016160000000002 2024-09-18T11:31:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.785836+0000", 2024-09-18T11:31:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6360999999999999e-05 2024-09-18T11:31:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:31:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:31:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.860498742000001, 2024-09-18T11:31:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:31:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:31:06.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:06.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:31:06.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:06.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:31:06.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:31:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:31:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:31:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:31:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:06.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:31:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:31:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:31:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:31:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:31:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:31:06.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:31:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:31:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:31:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:31:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.13:87 3.0 e58/21 3:099aa86f:::smithi11939506-4 low_tier:head v 58'69, mlcod=55'66)", 2024-09-18T11:31:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.270180+0000", 2024-09-18T11:31:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.982430216000001, 2024-09-18T11:31:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037855440000000001, 2024-09-18T11:31:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270180+0000", 2024-09-18T11:31:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270180+0000", 2024-09-18T11:31:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270179+0000", 2024-09-18T11:31:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270187+0000", 2024-09-18T11:31:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5610000000000001e-06 2024-09-18T11:31:06.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270195+0000", 2024-09-18T11:31:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5909999999999996e-06 2024-09-18T11:31:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270200+0000", 2024-09-18T11:31:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2599999999999999e-06 2024-09-18T11:31:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:06.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270252+0000", 2024-09-18T11:31:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1891000000000002e-05 2024-09-18T11:31:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270277+0000", 2024-09-18T11:31:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5134e-05 2024-09-18T11:31:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273928+0000", 2024-09-18T11:31:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00365132 2024-09-18T11:31:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273965+0000", 2024-09-18T11:31:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7277000000000002e-05 2024-09-18T11:31:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:289 4.d e58/22 4:b5a463bf:::smithi11939506-20:10 v 58'78, mlcod=53'73)", 2024-09-18T11:31:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.275682+0000", 2024-09-18T11:31:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.976927851999999, 2024-09-18T11:31:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003584318, 2024-09-18T11:31:06.289 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275682+0000", 2024-09-18T11:31:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275682+0000", 2024-09-18T11:31:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:06.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275681+0000", 2024-09-18T11:31:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275689+0000", 2024-09-18T11:31:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-18T11:31:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275698+0000", 2024-09-18T11:31:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1120000000000002e-06 2024-09-18T11:31:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275702+0000", 2024-09-18T11:31:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1450000000000001e-06 2024-09-18T11:31:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275731+0000", 2024-09-18T11:31:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8782999999999999e-05 2024-09-18T11:31:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275743+0000", 2024-09-18T11:31:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1436e-05 2024-09-18T11:31:06.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.279231+0000", 2024-09-18T11:31:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034884339999999999 2024-09-18T11:31:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.279266+0000", 2024-09-18T11:31:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5488000000000003e-05 2024-09-18T11:31:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1931 3.0 e65/21 3:3c05e498:::866d65857c51f67d56ecbcba33930954810c64ced2a213ca9705911438b0dfb7:head v 65'72, mlcod=58'70)", 2024-09-18T11:31:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:31:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.884488724000001, 2024-09-18T11:31:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040630859999999996, 2024-09-18T11:31:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:31:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:31:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368120+0000", 2024-09-18T11:31:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368427+0000", 2024-09-18T11:31:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030742600000000002 2024-09-18T11:31:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368446+0000", 2024-09-18T11:31:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8046e-05 2024-09-18T11:31:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368453+0000", 2024-09-18T11:31:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1380000000000001e-06 2024-09-18T11:31:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368519+0000", 2024-09-18T11:31:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6470000000000006e-05 2024-09-18T11:31:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:06.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368550+0000", 2024-09-18T11:31:06.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0707999999999998e-05 2024-09-18T11:31:06.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:06.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.372151+0000", 2024-09-18T11:31:06.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036015470000000001 2024-09-18T11:31:06.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:06.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.372184+0000", 2024-09-18T11:31:06.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2994999999999998e-05 2024-09-18T11:31:06.297 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.297 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:06.298 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.298 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:304 4.6 e69/67 4:67c1765a:::smithi11939506-30:17 v 69'35, mlcod=58'31)", 2024-09-18T11:31:06.298 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:06.298 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.906657647999999, 2024-09-18T11:31:06.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006377611, 2024-09-18T11:31:06.298 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:06.298 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:06.298 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:06.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:06.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:06.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:06.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:06.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:06.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:06.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:06.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:06.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345961+0000", 2024-09-18T11:31:06.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2050000000000008e-06 2024-09-18T11:31:06.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:06.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345969+0000", 2024-09-18T11:31:06.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0339999999999997e-06 2024-09-18T11:31:06.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:06.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345976+0000", 2024-09-18T11:31:06.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-18T11:31:06.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:06.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346026+0000", 2024-09-18T11:31:06.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9286999999999998e-05 2024-09-18T11:31:06.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:06.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346050+0000", 2024-09-18T11:31:06.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4397e-05 2024-09-18T11:31:06.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:06.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352293+0000", 2024-09-18T11:31:06.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062427170000000001 2024-09-18T11:31:06.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:06.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352330+0000", 2024-09-18T11:31:06.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7348999999999998e-05 2024-09-18T11:31:06.302 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.302 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:06.302 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.302 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:347 4.d e69/22 4:b44229a1:::smithi11939506-42:19 v 69'99, mlcod=65'93)", 2024-09-18T11:31:06.303 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:31:06.303 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.900924453, 2024-09-18T11:31:06.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056998760000000004, 2024-09-18T11:31:06.303 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:06.303 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:06.303 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:06.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:06.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:31:06.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:06.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:31:06.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:06.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351684+0000", 2024-09-18T11:31:06.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:06.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:06.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351693+0000", 2024-09-18T11:31:06.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-18T11:31:06.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:06.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351704+0000", 2024-09-18T11:31:06.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1077000000000001e-05 2024-09-18T11:31:06.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:06.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351709+0000", 2024-09-18T11:31:06.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5419999999999999e-06 2024-09-18T11:31:06.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:06.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351956+0000", 2024-09-18T11:31:06.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000247252 2024-09-18T11:31:06.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:06.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351975+0000", 2024-09-18T11:31:06.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9201000000000001e-05 2024-09-18T11:31:06.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:06.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357334+0000", 2024-09-18T11:31:06.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053588749999999999 2024-09-18T11:31:06.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:06.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357385+0000", 2024-09-18T11:31:06.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1406999999999998e-05 2024-09-18T11:31:06.307 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.307 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:06.307 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.307 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1956 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'37, mlcod=58'31)", 2024-09-18T11:31:06.308 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:06.308 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.89997954, 2024-09-18T11:31:06.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010958842, 2024-09-18T11:31:06.308 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:06.308 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:06.308 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:06.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:06.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:06.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:06.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:06.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:06.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:06.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:06.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:06.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352996+0000", 2024-09-18T11:31:06.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036622 2024-09-18T11:31:06.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:06.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353011+0000", 2024-09-18T11:31:06.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4849e-05 2024-09-18T11:31:06.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:06.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353017+0000", 2024-09-18T11:31:06.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:31:06.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:06.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353068+0000", 2024-09-18T11:31:06.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0695000000000002e-05 2024-09-18T11:31:06.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:06.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353092+0000", 2024-09-18T11:31:06.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4260000000000002e-05 2024-09-18T11:31:06.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:06.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363551+0000", 2024-09-18T11:31:06.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010459154999999999 2024-09-18T11:31:06.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:06.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363589+0000", 2024-09-18T11:31:06.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8161000000000003e-05 2024-09-18T11:31:06.312 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.312 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:06.312 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.312 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1957 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'38, mlcod=65'33)", 2024-09-18T11:31:06.312 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:06.312 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.893563672999999, 2024-09-18T11:31:06.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050499400000000002, 2024-09-18T11:31:06.312 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:06.312 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:06.312 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:06.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:06.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:06.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:06.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:06.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:06.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:06.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:06.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:06.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359214+0000", 2024-09-18T11:31:06.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000168047 2024-09-18T11:31:06.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:06.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359227+0000", 2024-09-18T11:31:06.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3124e-05 2024-09-18T11:31:06.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:06.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359235+0000", 2024-09-18T11:31:06.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0760000000000003e-06 2024-09-18T11:31:06.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:06.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359287+0000", 2024-09-18T11:31:06.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.151e-05 2024-09-18T11:31:06.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:06.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359310+0000", 2024-09-18T11:31:06.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-09-18T11:31:06.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:06.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364065+0000", 2024-09-18T11:31:06.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047549439999999997 2024-09-18T11:31:06.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:06.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364096+0000", 2024-09-18T11:31:06.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1423000000000001e-05 2024-09-18T11:31:06.316 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.316 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:06.316 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.316 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:349 4.d e69/22 4:b9bb642b:::smithi11939506-27:19 v 69'101, mlcod=69'97)", 2024-09-18T11:31:06.316 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.360028+0000", 2024-09-18T11:31:06.316 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.892582056, 2024-09-18T11:31:06.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040307850000000003, 2024-09-18T11:31:06.316 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:06.317 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:06.317 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:06.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:06.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360028+0000", 2024-09-18T11:31:06.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:06.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360028+0000", 2024-09-18T11:31:06.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:06.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360027+0000", 2024-09-18T11:31:06.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:06.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:06.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360039+0000", 2024-09-18T11:31:06.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1654000000000001e-05 2024-09-18T11:31:06.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:06.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360052+0000", 2024-09-18T11:31:06.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.33e-05 2024-09-18T11:31:06.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:06.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360058+0000", 2024-09-18T11:31:06.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0440000000000001e-06 2024-09-18T11:31:06.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:06.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360094+0000", 2024-09-18T11:31:06.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6402999999999997e-05 2024-09-18T11:31:06.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:06.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360109+0000", 2024-09-18T11:31:06.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4105e-05 2024-09-18T11:31:06.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:06.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364025+0000", 2024-09-18T11:31:06.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039163189999999997 2024-09-18T11:31:06.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:06.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364059+0000", 2024-09-18T11:31:06.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3810999999999999e-05 2024-09-18T11:31:06.320 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.320 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:06.320 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.320 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1978 4.4 e69/22 4:2d9c3f7f:::smithi11939506-5:head v 69'74, mlcod=69'71)", 2024-09-18T11:31:06.320 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.967898+0000", 2024-09-18T11:31:06.320 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.284711605, 2024-09-18T11:31:06.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038277960000000001, 2024-09-18T11:31:06.321 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:06.321 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:06.321 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:06.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:06.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.967898+0000", 2024-09-18T11:31:06.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:06.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.967898+0000", 2024-09-18T11:31:06.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:06.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.967898+0000", 2024-09-18T11:31:06.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:06.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:06.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.968050+0000", 2024-09-18T11:31:06.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015286799999999999 2024-09-18T11:31:06.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:06.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.968064+0000", 2024-09-18T11:31:06.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3964999999999999e-05 2024-09-18T11:31:06.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:06.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.968070+0000", 2024-09-18T11:31:06.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:31:06.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:06.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.968112+0000", 2024-09-18T11:31:06.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1112999999999998e-05 2024-09-18T11:31:06.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:06.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.968133+0000", 2024-09-18T11:31:06.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1715000000000001e-05 2024-09-18T11:31:06.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:06.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.971687+0000", 2024-09-18T11:31:06.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035536410000000002 2024-09-18T11:31:06.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:06.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.971726+0000", 2024-09-18T11:31:06.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.926e-05 2024-09-18T11:31:06.324 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.324 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:06.324 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.324 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:407 4.4 e77/22 4:2d9c3f7f:::smithi11939506-5:1d v 77'92, mlcod=73'87)", 2024-09-18T11:31:06.324 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.387293+0000", 2024-09-18T11:31:06.325 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.8653168490000001, 2024-09-18T11:31:06.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038531820000000001, 2024-09-18T11:31:06.325 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:06.325 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:06.325 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:06.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:06.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387293+0000", 2024-09-18T11:31:06.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:06.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387293+0000", 2024-09-18T11:31:06.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:06.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387293+0000", 2024-09-18T11:31:06.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:06.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:06.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387300+0000", 2024-09-18T11:31:06.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6799999999999993e-06 2024-09-18T11:31:06.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:06.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387311+0000", 2024-09-18T11:31:06.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0278999999999999e-05 2024-09-18T11:31:06.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:06.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387316+0000", 2024-09-18T11:31:06.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9269999999999996e-06 2024-09-18T11:31:06.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:06.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387357+0000", 2024-09-18T11:31:06.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1208000000000003e-05 2024-09-18T11:31:06.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:06.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387374+0000", 2024-09-18T11:31:06.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7499e-05 2024-09-18T11:31:06.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:06.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.391105+0000", 2024-09-18T11:31:06.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037302619999999998 2024-09-18T11:31:06.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:06.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.391146+0000", 2024-09-18T11:31:06.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1797e-05 2024-09-18T11:31:06.328 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.328 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:06.329 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.329 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2300 4.4 e77/22 4:2c93196f:::smithi11939506-28:head v 77'94, mlcod=77'90)", 2024-09-18T11:31:06.329 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:31:06.329 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2645014549999996, 2024-09-18T11:31:06.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050425380000000001, 2024-09-18T11:31:06.329 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:06.329 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:06.329 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:06.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:06.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:31:06.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:06.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:31:06.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:06.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988108+0000", 2024-09-18T11:31:06.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:06.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:06.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988388+0000", 2024-09-18T11:31:06.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028087700000000001 2024-09-18T11:31:06.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:06.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988405+0000", 2024-09-18T11:31:06.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6565000000000001e-05 2024-09-18T11:31:06.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:06.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988412+0000", 2024-09-18T11:31:06.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9709999999999998e-06 2024-09-18T11:31:06.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:06.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988464+0000", 2024-09-18T11:31:06.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1949e-05 2024-09-18T11:31:06.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:06.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988490+0000", 2024-09-18T11:31:06.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5780999999999999e-05 2024-09-18T11:31:06.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:06.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993112+0000", 2024-09-18T11:31:06.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046228399999999996 2024-09-18T11:31:06.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:06.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993151+0000", 2024-09-18T11:31:06.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8587999999999997e-05 2024-09-18T11:31:06.333 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.333 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:06.333 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.333 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:451 4.d e78/22 4:b44229a1:::smithi11939506-42:19 v 78'125, mlcod=76'121)", 2024-09-18T11:31:06.333 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.400699+0000", 2024-09-18T11:31:06.333 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.8519107290000001, 2024-09-18T11:31:06.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038090789999999999, 2024-09-18T11:31:06.333 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:06.334 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:06.334 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:06.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:06.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.400699+0000", 2024-09-18T11:31:06.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:06.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.400699+0000", 2024-09-18T11:31:06.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:06.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.400699+0000", 2024-09-18T11:31:06.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:06.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:06.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.400705+0000", 2024-09-18T11:31:06.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.789e-06 2024-09-18T11:31:06.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:06.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.400721+0000", 2024-09-18T11:31:06.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6412000000000001e-05 2024-09-18T11:31:06.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:06.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.400727+0000", 2024-09-18T11:31:06.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1410000000000002e-06 2024-09-18T11:31:06.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:06.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.400783+0000", 2024-09-18T11:31:06.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5588999999999999e-05 2024-09-18T11:31:06.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:06.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.400797+0000", 2024-09-18T11:31:06.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4255e-05 2024-09-18T11:31:06.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:06.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.404472+0000", 2024-09-18T11:31:06.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036744709999999999 2024-09-18T11:31:06.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:06.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.404508+0000", 2024-09-18T11:31:06.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6693999999999998e-05 2024-09-18T11:31:06.337 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.337 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:06.337 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.337 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.13:63 3.0 e78/21 3:24ebb740:::smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 78'80, mlcod=73'78)", 2024-09-18T11:31:06.337 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:31:06.337 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.8475120389999997, 2024-09-18T11:31:06.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052262230000000003, 2024-09-18T11:31:06.338 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:06.338 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:06.338 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:06.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:06.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:31:06.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:06.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:31:06.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:06.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405097+0000", 2024-09-18T11:31:06.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:06.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:06.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405103+0000", 2024-09-18T11:31:06.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8780000000000003e-06 2024-09-18T11:31:06.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:06.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405117+0000", 2024-09-18T11:31:06.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3441000000000001e-05 2024-09-18T11:31:06.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:06.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405121+0000", 2024-09-18T11:31:06.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7099999999999998e-06 2024-09-18T11:31:06.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:06.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405158+0000", 2024-09-18T11:31:06.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6477999999999999e-05 2024-09-18T11:31:06.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:06.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405173+0000", 2024-09-18T11:31:06.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4796e-05 2024-09-18T11:31:06.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:06.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410285+0000", 2024-09-18T11:31:06.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051118919999999998 2024-09-18T11:31:06.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:06.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410324+0000", 2024-09-18T11:31:06.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9616999999999997e-05 2024-09-18T11:31:06.341 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.341 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:06.341 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.341 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:451 4.6 e78/67 4:6f86ffcc:::smithi11939506-15:26 v 78'58, mlcod=77'53)", 2024-09-18T11:31:06.341 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.414259+0000", 2024-09-18T11:31:06.341 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.8383514559999998, 2024-09-18T11:31:06.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041821610000000002, 2024-09-18T11:31:06.342 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:06.342 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:06.342 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:06.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:06.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414259+0000", 2024-09-18T11:31:06.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:06.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414259+0000", 2024-09-18T11:31:06.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:06.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:06.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414258+0000", 2024-09-18T11:31:06.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:06.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:06.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414264+0000", 2024-09-18T11:31:06.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1850000000000001e-06 2024-09-18T11:31:06.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:06.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414273+0000", 2024-09-18T11:31:06.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7290000000000006e-06 2024-09-18T11:31:06.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:06.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414278+0000", 2024-09-18T11:31:06.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8470000000000003e-06 2024-09-18T11:31:06.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:06.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414315+0000", 2024-09-18T11:31:06.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7462000000000001e-05 2024-09-18T11:31:06.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:06.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414334+0000", 2024-09-18T11:31:06.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9080999999999999e-05 2024-09-18T11:31:06.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:06.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.418419+0000", 2024-09-18T11:31:06.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040846540000000001 2024-09-18T11:31:06.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:06.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:06.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:06.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.418441+0000", 2024-09-18T11:31:06.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1844999999999998e-05 2024-09-18T11:31:06.345 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.345 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:06.345 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.345 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:06.345 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:06.345 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:06.347 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:31:06.349 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:06.350 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:06.349+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:06.365 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:06.365 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:06.381 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:06.382 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:06.396 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:06.396 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:06.412 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:06.412 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:06.423 INFO:tasks.rados.rados.0.smithi119.stdout:1189: finishing write tid 1 to smithi11939506-49 2024-09-18T11:31:06.423 INFO:tasks.rados.rados.0.smithi119.stdout:1187: expect (ObjNum 206 snap 23 seq_num 206) 2024-09-18T11:31:06.441 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:06.442 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:06.451 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:06.450+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:06.472 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:06.472 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:06.473 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:06.473 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:06.485 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:31:06.493 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:06.494 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:06.497 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:06.497 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:06.552 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:06.551+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:06.591 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:06.591 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:06.600 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:31:06.624 INFO:tasks.rados.rados.0.smithi119.stdout:1189: finishing write tid 2 to smithi11939506-49 2024-09-18T11:31:06.624 INFO:tasks.rados.rados.0.smithi119.stdout:1189: finishing write tid 3 to smithi11939506-49 2024-09-18T11:31:06.624 INFO:tasks.rados.rados.0.smithi119.stdout:1189: finishing write tid 4 to smithi11939506-49 2024-09-18T11:31:06.624 INFO:tasks.rados.rados.0.smithi119.stdout:1189: finishing write tid 5 to smithi11939506-49 2024-09-18T11:31:06.624 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 225 (ObjNum 216 snap 26 seq_num 216) dirty exists 2024-09-18T11:31:06.625 INFO:tasks.rados.rados.0.smithi119.stdout:1189: left oid 49 (ObjNum 216 snap 26 seq_num 216) 2024-09-18T11:31:06.625 INFO:tasks.rados.rados.0.smithi119.stdout:1184: done (7 left) 2024-09-18T11:31:06.625 INFO:tasks.rados.rados.0.smithi119.stdout:1185: done (6 left) 2024-09-18T11:31:06.625 INFO:tasks.rados.rados.0.smithi119.stdout:1186: done (5 left) 2024-09-18T11:31:06.625 INFO:tasks.rados.rados.0.smithi119.stdout:1187: done (4 left) 2024-09-18T11:31:06.625 INFO:tasks.rados.rados.0.smithi119.stdout:1188: done (3 left) 2024-09-18T11:31:06.625 INFO:tasks.rados.rados.0.smithi119.stdout:1189: done (2 left) 2024-09-18T11:31:06.625 INFO:tasks.rados.rados.0.smithi119.stdout:1191: done (1 left) 2024-09-18T11:31:06.625 INFO:tasks.rados.rados.0.smithi119.stdout:1192: snap_remove snap 25 2024-09-18T11:31:06.670 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:06.669+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:06.723 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:06.723 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:06.731 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:31:06.776 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:06.775+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:06.837 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:06.837 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:06.845 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:31:06.887 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:06.885+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:06.958 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:06.957+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:06.984 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:06.984 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:06.993 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:31:07.060 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:07.059+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:07.131 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:07.131 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:07.139 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:31:07.182 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:07.181+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:07.237 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:07.237 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:07.245 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:31:07.286 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:07.285+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:07.362 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:07.361+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:07.383 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:07.383 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:07.391 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:31:07.421 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:07.419+0000 7f3157e90640 -1 osd.4 83 handle_osd_map injecting map crc failure 2024-09-18T11:31:07.422 INFO:tasks.rados.rados.0.smithi119.stdout:1192: done (1 left) 2024-09-18T11:31:07.422 INFO:tasks.rados.rados.0.smithi119.stdout:1193: tier_evict oid 24 2024-09-18T11:31:07.422 INFO:tasks.rados.rados.0.smithi119.stdout:1193: tier_evict oid 24 snap -1 2024-09-18T11:31:07.422 INFO:tasks.rados.rados.0.smithi119.stdout:1190: finishing write tid 1 to smithi11939506-32 2024-09-18T11:31:07.423 INFO:tasks.rados.rados.0.smithi119.stdout:1190: finishing write tid 2 to smithi11939506-32 2024-09-18T11:31:07.423 INFO:tasks.rados.rados.0.smithi119.stdout:1190: finishing write tid 3 to smithi11939506-32 2024-09-18T11:31:07.423 INFO:tasks.rados.rados.0.smithi119.stdout:1190: finishing write tid 4 to smithi11939506-32 2024-09-18T11:31:07.423 INFO:tasks.rados.rados.0.smithi119.stdout:1190: finishing write tid 5 to smithi11939506-32 2024-09-18T11:31:07.423 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 120 (ObjNum 217 snap 26 seq_num 217) dirty exists 2024-09-18T11:31:07.423 INFO:tasks.rados.rados.0.smithi119.stdout:1190: left oid 32 (ObjNum 217 snap 26 seq_num 217) 2024-09-18T11:31:07.423 INFO:tasks.rados.rados.0.smithi119.stdout:1190: done (1 left) 2024-09-18T11:31:07.423 INFO:tasks.rados.rados.0.smithi119.stdout:1194: write (excl) oid 33 current snap is 26 2024-09-18T11:31:07.423 INFO:tasks.rados.rados.0.smithi119.stdout:1194: seq_num 219 ranges {480733=537415,1782380=696402,3031400=537159} 2024-09-18T11:31:07.428 INFO:tasks.rados.rados.0.smithi119.stdout:1194: writing smithi11939506-33 from 480733 to 1018148 tid 1 2024-09-18T11:31:07.428 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:07.427+0000 7f3155e8c640 -1 osd.4 83 handle_osd_map injecting map crc failure 2024-09-18T11:31:07.437 INFO:tasks.rados.rados.0.smithi119.stdout:1194: writing smithi11939506-33 from 1782380 to 2478782 tid 2 2024-09-18T11:31:07.443 INFO:tasks.rados.rados.0.smithi119.stdout:1194: writing smithi11939506-33 from 3031400 to 3568559 tid 3 2024-09-18T11:31:07.443 INFO:tasks.rados.rados.0.smithi119.stdout:1193: got (22) Invalid argument 2024-09-18T11:31:07.443 INFO:tasks.rados.rados.0.smithi119.stdout:1194: finishing write tid 1 to smithi11939506-33 2024-09-18T11:31:07.443 INFO:tasks.rados.rados.0.smithi119.stdout:1195: read oid 20 snap -1 2024-09-18T11:31:07.443 INFO:tasks.rados.rados.0.smithi119.stdout:1195: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:31:07.443 INFO:tasks.rados.rados.0.smithi119.stdout:1193: done (2 left) 2024-09-18T11:31:07.443 INFO:tasks.rados.rados.0.smithi119.stdout:1196: set_chunk oid 29 target oid 2024-09-18T11:31:07.454 INFO:tasks.rados.rados.0.smithi119.stdout:1196: set_chunk oid 29 offset: 2153472 length: 47616 target oid cf7a086dafaa90bb4b2862c4eaaebb0a183a579f7044b248569f1da60a29c096 offset: 0 snap 20 2024-09-18T11:31:07.454 INFO:tasks.rados.rados.0.smithi119.stdout:1197: tier_flush oid 15 2024-09-18T11:31:07.454 INFO:tasks.rados.rados.0.smithi119.stdout:1197: tier_flush oid 15 snap -1 2024-09-18T11:31:07.454 INFO:tasks.rados.rados.0.smithi119.stdout:1194: finishing write tid 2 to smithi11939506-33 2024-09-18T11:31:07.454 INFO:tasks.rados.rados.0.smithi119.stdout:1194: finishing write tid 3 to smithi11939506-33 2024-09-18T11:31:07.454 INFO:tasks.rados.rados.0.smithi119.stdout:1198: read oid 26 snap -1 2024-09-18T11:31:07.454 INFO:tasks.rados.rados.0.smithi119.stdout:1198: expect (ObjNum 178 snap 20 seq_num 178) 2024-09-18T11:31:07.454 INFO:tasks.rados.rados.0.smithi119.stdout:1199: read oid 19 snap -1 2024-09-18T11:31:07.454 INFO:tasks.rados.rados.0.smithi119.stdout:1199: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-18T11:31:07.455 INFO:tasks.rados.rados.0.smithi119.stdout:1200: set_chunk oid 43 target oid 2024-09-18T11:31:07.463 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:07.461+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:07.465 INFO:tasks.rados.rados.0.smithi119.stdout:1200: set_chunk oid 43 offset: 1060864 length: 928768 target oid bbd8a56945da880432531848e1b12bd423a075aa9d5e513c0a391d3887259b43 offset: 0 snap 17 2024-09-18T11:31:07.465 INFO:tasks.rados.rados.0.smithi119.stdout:1201: set_chunk oid 37 target oid 2024-09-18T11:31:07.469 INFO:tasks.rados.rados.0.smithi119.stdout:1201: set_chunk oid 37 offset: 1315328 length: 212992 target oid 530c21dc215641f188486e84399daa1dd29913882fae387f603eeaeb0458b4a0 offset: 0 snap -1 2024-09-18T11:31:07.469 INFO:tasks.rados.rados.0.smithi119.stdout:1202: snap_remove snap 19 2024-09-18T11:31:07.500 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:07.501 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:07.509 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:31:07.564 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:07.563+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:07.648 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:07.649 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:07.657 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:31:07.701 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:07.699+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:07.754 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:07.755 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:07.763 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:31:07.804 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:07.803+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:07.866 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:07.865+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:07.902 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:07.902 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:07.910 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:31:07.968 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:07.967+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:08.023 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T11:31:08.049 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:08.049 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:08.057 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:31:08.100 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:08.099+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:08.156 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:08.157 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:08.165 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:31:08.206 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:08.205+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:08.270 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:08.269+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:08.303 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:08.303 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:08.311 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:31:08.372 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:08.371+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:08.409 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:08.409 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:08.418 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:31:08.429 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:08.427+0000 7fcb682b1640 -1 osd.7 84 handle_osd_map injecting map crc failure 2024-09-18T11:31:08.432 INFO:tasks.rados.rados.0.smithi119.stdout:1202: done (8 left) 2024-09-18T11:31:08.432 INFO:tasks.rados.rados.0.smithi119.stdout:1203: tier_promote oid 21 2024-09-18T11:31:08.432 INFO:tasks.rados.rados.0.smithi119.stdout:1194: finishing write tid 4 to smithi11939506-33 2024-09-18T11:31:08.432 INFO:tasks.rados.rados.0.smithi119.stdout:1194: finishing write tid 5 to smithi11939506-33 2024-09-18T11:31:08.433 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 33 v 86 (ObjNum 219 snap 26 seq_num 219) dirty exists 2024-09-18T11:31:08.433 INFO:tasks.rados.rados.0.smithi119.stdout:1194: left oid 33 (ObjNum 219 snap 26 seq_num 219) 2024-09-18T11:31:08.433 INFO:tasks.rados.rados.0.smithi119.stdout:1195: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:31:08.433 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is unset 2024-09-18T11:31:08.433 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:08.432+0000 7f697aead640 -1 osd.3 84 handle_osd_map injecting map crc failure 2024-09-18T11:31:08.495 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:08.494+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:08.497 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:31:08.525 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:08.525 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:08.534 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:31:08.573 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:08.572+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:08.598 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:08.614 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:08.614 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:08.630 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:08.630 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:08.646 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:08.647 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:08.659 INFO:tasks.rados.rados.0.smithi119.stdout:1197: got (0) Success 2024-09-18T11:31:08.660 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 54 (ObjNum 201 snap 23 seq_num 201) dirty exists 2024-09-18T11:31:08.660 INFO:tasks.rados.rados.0.smithi119.stdout:1196: finishing set_chunk to oid 29 2024-09-18T11:31:08.660 INFO:tasks.rados.rados.0.smithi119.stdout:1199: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-18T11:31:08.662 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:08.662 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:08.674 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:08.673+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:08.678 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:08.678 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:08.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:08.678 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:08.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.678 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:887 4.7 4:e0d95e10:::smithi11939506-22:head [read 0~2352947,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:08.679 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:31:08.679 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 57.559287282, 2024-09-18T11:31:08.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079115169999999999, 2024-09-18T11:31:08.679 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:08.679 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:08.679 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:08.679 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:08.679 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:08.679 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 887 2024-09-18T11:31:08.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.679 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:08.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:08.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:31:08.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:08.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:31:08.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:08.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116365+0000", 2024-09-18T11:31:08.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:08.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:08.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116376+0000", 2024-09-18T11:31:08.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1471000000000001e-05 2024-09-18T11:31:08.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:08.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116394+0000", 2024-09-18T11:31:08.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8088e-05 2024-09-18T11:31:08.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:08.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116402+0000", 2024-09-18T11:31:08.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2320000000000004e-06 2024-09-18T11:31:08.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116458+0000", 2024-09-18T11:31:08.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6536000000000002e-05 2024-09-18T11:31:08.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:08.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116466+0000", 2024-09-18T11:31:08.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0819999999999999e-06 2024-09-18T11:31:08.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.122509+0000", 2024-09-18T11:31:08.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060431879999999997 2024-09-18T11:31:08.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:08.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.122577+0000", 2024-09-18T11:31:08.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7646999999999998e-05 2024-09-18T11:31:08.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:08.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124278+0000", 2024-09-18T11:31:08.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017013200000000001 2024-09-18T11:31:08.684 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.684 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:08.684 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.684 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:888 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:08.684 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:08.684 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 57.559233818999999, 2024-09-18T11:31:08.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082282120000000004, 2024-09-18T11:31:08.684 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:08.684 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:08.684 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:08.684 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:08.685 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:08.685 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 888 2024-09-18T11:31:08.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.685 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:08.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:08.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:08.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:08.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:08.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:08.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:08.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:08.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:08.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116423+0000", 2024-09-18T11:31:08.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1e-06 2024-09-18T11:31:08.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:08.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:31:08.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5510000000000001e-06 2024-09-18T11:31:08.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:08.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116431+0000", 2024-09-18T11:31:08.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6109999999999998e-06 2024-09-18T11:31:08.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116527+0000", 2024-09-18T11:31:08.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6212999999999995e-05 2024-09-18T11:31:08.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:08.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116532+0000", 2024-09-18T11:31:08.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6140000000000002e-06 2024-09-18T11:31:08.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124314+0000", 2024-09-18T11:31:08.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077818929999999998 2024-09-18T11:31:08.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:08.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124411+0000", 2024-09-18T11:31:08.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6992999999999998e-05 2024-09-18T11:31:08.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:08.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124649+0000", 2024-09-18T11:31:08.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237729 2024-09-18T11:31:08.690 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.690 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:08.690 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.691 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:889 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:08.691 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:08.691 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 57.559209168999999, 2024-09-18T11:31:08.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085383349999999993, 2024-09-18T11:31:08.691 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:08.691 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:08.691 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:08.691 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:08.691 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:08.691 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 889 2024-09-18T11:31:08.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.691 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:08.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:08.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:08.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:08.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:08.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:08.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:08.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:08.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:08.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116447+0000", 2024-09-18T11:31:08.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.854e-06 2024-09-18T11:31:08.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:08.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116452+0000", 2024-09-18T11:31:08.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0250000000000002e-06 2024-09-18T11:31:08.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:08.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116455+0000", 2024-09-18T11:31:08.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8739999999999999e-06 2024-09-18T11:31:08.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116565+0000", 2024-09-18T11:31:08.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110741 2024-09-18T11:31:08.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:08.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116571+0000", 2024-09-18T11:31:08.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4990000000000002e-06 2024-09-18T11:31:08.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124691+0000", 2024-09-18T11:31:08.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081200339999999999 2024-09-18T11:31:08.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:08.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124784+0000", 2024-09-18T11:31:08.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2886999999999993e-05 2024-09-18T11:31:08.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:08.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124983+0000", 2024-09-18T11:31:08.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019969000000000001 2024-09-18T11:31:08.698 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.698 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:08.698 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.698 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:46 4.a e31/22 4:552523b6:::smithi11939506-38:head v 31'20, mlcod=30'18)", 2024-09-18T11:31:08.699 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:31:08.699 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 57.545257536999998, 2024-09-18T11:31:08.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080208420000000002, 2024-09-18T11:31:08.699 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:08.699 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:08.699 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:08.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:08.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:31:08.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:08.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:31:08.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:08.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130396+0000", 2024-09-18T11:31:08.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:08.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:08.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133245+0000", 2024-09-18T11:31:08.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028486929999999998 2024-09-18T11:31:08.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:08.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133253+0000", 2024-09-18T11:31:08.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1540000000000002e-06 2024-09-18T11:31:08.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:08.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133261+0000", 2024-09-18T11:31:08.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3620000000000006e-06 2024-09-18T11:31:08.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133319+0000", 2024-09-18T11:31:08.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7719000000000001e-05 2024-09-18T11:31:08.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:08.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133341+0000", 2024-09-18T11:31:08.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1946999999999998e-05 2024-09-18T11:31:08.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:08.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138368+0000", 2024-09-18T11:31:08.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005026712 2024-09-18T11:31:08.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:08.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138418+0000", 2024-09-18T11:31:08.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9864000000000002e-05 2024-09-18T11:31:08.703 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.703 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:08.703 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.704 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1126 3.0 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head [write 0~2520064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:31:08.704 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:31:08.704 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.693767190999999, 2024-09-18T11:31:08.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086197050000000001, 2024-09-18T11:31:08.704 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:08.704 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:08.704 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:08.704 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:08.704 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:08.704 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1126 2024-09-18T11:31:08.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.705 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:08.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:08.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:31:08.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:08.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:31:08.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:08.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981886+0000", 2024-09-18T11:31:08.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:08.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:08.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983017+0000", 2024-09-18T11:31:08.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00113131 2024-09-18T11:31:08.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:08.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983037+0000", 2024-09-18T11:31:08.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0231e-05 2024-09-18T11:31:08.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:08.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983044+0000", 2024-09-18T11:31:08.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9539999999999998e-06 2024-09-18T11:31:08.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983087+0000", 2024-09-18T11:31:08.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3040000000000001e-05 2024-09-18T11:31:08.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:08.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983193+0000", 2024-09-18T11:31:08.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105682 2024-09-18T11:31:08.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:08.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983287+0000", 2024-09-18T11:31:08.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4066000000000004e-05 2024-09-18T11:31:08.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:08.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.986918+0000", 2024-09-18T11:31:08.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036311970000000001 2024-09-18T11:31:08.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:08.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990442+0000", 2024-09-18T11:31:08.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035243119999999999 2024-09-18T11:31:08.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:08.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990476+0000", 2024-09-18T11:31:08.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4028e-05 2024-09-18T11:31:08.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:08.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990507+0000", 2024-09-18T11:31:08.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0284999999999999e-05 2024-09-18T11:31:08.710 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.710 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:08.710 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.710 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:262 4.9 e53/22 4:99feffe9:::smithi11939506-45:13 v 53'17, mlcod=51'14)", 2024-09-18T11:31:08.710 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:08.710 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 34.676663376, 2024-09-18T11:31:08.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013878679, 2024-09-18T11:31:08.710 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:08.710 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:08.711 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:08.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:08.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:08.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:08.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:08.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:08.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:08.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:08.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:08.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998997+0000", 2024-09-18T11:31:08.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1099999999999999e-06 2024-09-18T11:31:08.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:08.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999008+0000", 2024-09-18T11:31:08.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1321000000000001e-05 2024-09-18T11:31:08.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:08.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999015+0000", 2024-09-18T11:31:08.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3000000000000004e-06 2024-09-18T11:31:08.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999055+0000", 2024-09-18T11:31:08.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0058000000000002e-05 2024-09-18T11:31:08.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:08.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999070+0000", 2024-09-18T11:31:08.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4328000000000001e-05 2024-09-18T11:31:08.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:08.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012465+0000", 2024-09-18T11:31:08.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013395638 2024-09-18T11:31:08.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:08.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012870+0000", 2024-09-18T11:31:08.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040423400000000001 2024-09-18T11:31:08.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.715 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:08.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.715 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:205 4.b e53/22 4:dba5bd39:::smithi11939506-17:13 v 53'77, mlcod=51'74)", 2024-09-18T11:31:08.715 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:08.715 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 34.676614115, 2024-09-18T11:31:08.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013359586, 2024-09-18T11:31:08.715 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:08.715 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:08.715 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:08.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:08.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:08.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:08.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:08.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:08.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:08.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:08.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:08.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999048+0000", 2024-09-18T11:31:08.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3550000000000005e-06 2024-09-18T11:31:08.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:08.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999056+0000", 2024-09-18T11:31:08.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3259999999999996e-06 2024-09-18T11:31:08.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:08.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999063+0000", 2024-09-18T11:31:08.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6409999999999996e-06 2024-09-18T11:31:08.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999098+0000", 2024-09-18T11:31:08.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5203999999999997e-05 2024-09-18T11:31:08.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:08.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999115+0000", 2024-09-18T11:31:08.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6478999999999999e-05 2024-09-18T11:31:08.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:08.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012362+0000", 2024-09-18T11:31:08.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01324768 2024-09-18T11:31:08.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:08.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012400+0000", 2024-09-18T11:31:08.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.731e-05 2024-09-18T11:31:08.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.719 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:08.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.720 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:88 4.a e53/22 4:588debc2:::smithi11939506-18:13 v 53'45, mlcod=51'42)", 2024-09-18T11:31:08.720 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:08.720 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 34.673831692, 2024-09-18T11:31:08.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011307667, 2024-09-18T11:31:08.720 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:08.720 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:08.720 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:08.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:08.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:08.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:08.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:08.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:08.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001822+0000", 2024-09-18T11:31:08.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:08.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:08.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001827+0000", 2024-09-18T11:31:08.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.378e-06 2024-09-18T11:31:08.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:08.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001839+0000", 2024-09-18T11:31:08.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1225e-05 2024-09-18T11:31:08.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:08.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001844+0000", 2024-09-18T11:31:08.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2299999999999999e-06 2024-09-18T11:31:08.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001891+0000", 2024-09-18T11:31:08.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6961e-05 2024-09-18T11:31:08.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:08.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001906+0000", 2024-09-18T11:31:08.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5158e-05 2024-09-18T11:31:08.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:08.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013103+0000", 2024-09-18T11:31:08.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011197314 2024-09-18T11:31:08.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:08.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013130+0000", 2024-09-18T11:31:08.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6900999999999999e-05 2024-09-18T11:31:08.724 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.724 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:08.725 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.725 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:89 4.a e53/22 4:552523b6:::smithi11939506-38:13 v 53'47, mlcod=51'42)", 2024-09-18T11:31:08.725 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:08.725 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 34.673272472000001, 2024-09-18T11:31:08.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011327614, 2024-09-18T11:31:08.725 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:08.725 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:08.725 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:08.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:08.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:08.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:08.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:08.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:08.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002381+0000", 2024-09-18T11:31:08.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:08.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:08.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002397+0000", 2024-09-18T11:31:08.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5688e-05 2024-09-18T11:31:08.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:08.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002406+0000", 2024-09-18T11:31:08.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0229999999999998e-06 2024-09-18T11:31:08.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:08.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002415+0000", 2024-09-18T11:31:08.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8959999999999992e-06 2024-09-18T11:31:08.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012386+0000", 2024-09-18T11:31:08.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0099711450000000007 2024-09-18T11:31:08.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:08.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012405+0000", 2024-09-18T11:31:08.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9389e-05 2024-09-18T11:31:08.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:08.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013676+0000", 2024-09-18T11:31:08.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012708649999999999 2024-09-18T11:31:08.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:08.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013709+0000", 2024-09-18T11:31:08.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3207000000000002e-05 2024-09-18T11:31:08.729 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.729 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:08.729 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.730 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1643 4.7 4:e73ba68f:::smithi11939506-1:head [read 0~3983226,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:08.730 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:31:08.730 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.341767296, 2024-09-18T11:31:08.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079617669999999998, 2024-09-18T11:31:08.730 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:08.730 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:08.730 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:08.730 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:08.730 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:08.730 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1643 2024-09-18T11:31:08.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.730 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:08.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:08.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:31:08.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:08.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:31:08.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:08.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:31:08.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:08.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:08.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333889+0000", 2024-09-18T11:31:08.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8890000000000002e-06 2024-09-18T11:31:08.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:08.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333896+0000", 2024-09-18T11:31:08.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.156e-06 2024-09-18T11:31:08.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:08.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333900+0000", 2024-09-18T11:31:08.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.138e-06 2024-09-18T11:31:08.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.336677+0000", 2024-09-18T11:31:08.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027774169999999999 2024-09-18T11:31:08.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:08.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.336768+0000", 2024-09-18T11:31:08.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0983999999999995e-05 2024-09-18T11:31:08.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:08.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341849+0000", 2024-09-18T11:31:08.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050805140000000004 2024-09-18T11:31:08.734 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.734 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:08.734 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.734 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1644 4.7 4:e73ba68f:::smithi11939506-1:head [sparse-read 0~2695582] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:08.734 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:08.734 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.341750174000001, 2024-09-18T11:31:08.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084764039999999999, 2024-09-18T11:31:08.735 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:08.735 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:08.735 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:08.735 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:08.735 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:08.735 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1644 2024-09-18T11:31:08.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.735 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:08.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:08.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:08.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:08.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:08.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:08.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:08.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:08.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:08.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333906+0000", 2024-09-18T11:31:08.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.243e-06 2024-09-18T11:31:08.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:08.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333911+0000", 2024-09-18T11:31:08.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2959999999999998e-06 2024-09-18T11:31:08.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:08.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333915+0000", 2024-09-18T11:31:08.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6720000000000002e-06 2024-09-18T11:31:08.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341879+0000", 2024-09-18T11:31:08.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079638699999999996 2024-09-18T11:31:08.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:08.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341978+0000", 2024-09-18T11:31:08.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9562000000000007e-05 2024-09-18T11:31:08.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:08.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342380+0000", 2024-09-18T11:31:08.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000402113 2024-09-18T11:31:08.739 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.739 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:08.739 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.739 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1645 4.7 4:e73ba68f:::smithi11939506-1:head [read 0~3983226,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:08.739 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:08.739 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.341735280000002, 2024-09-18T11:31:08.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010539468999999999, 2024-09-18T11:31:08.739 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:08.739 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:08.739 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1645 2024-09-18T11:31:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:08.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:08.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:08.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:08.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:08.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333921+0000", 2024-09-18T11:31:08.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8909999999999999e-06 2024-09-18T11:31:08.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:08.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333927+0000", 2024-09-18T11:31:08.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4229999999999996e-06 2024-09-18T11:31:08.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:08.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333932+0000", 2024-09-18T11:31:08.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8280000000000002e-06 2024-09-18T11:31:08.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342418+0000", 2024-09-18T11:31:08.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084862219999999999 2024-09-18T11:31:08.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:08.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342515+0000", 2024-09-18T11:31:08.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7287999999999994e-05 2024-09-18T11:31:08.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:08.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.344458+0000", 2024-09-18T11:31:08.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019431660000000001 2024-09-18T11:31:08.743 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.743 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:08.743 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.744 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1669 4.4 4:2d9c3f7f:::smithi11939506-5:head [read 0~3764735,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-18T11:31:08.744 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:31:08.744 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 29.852786382000001, 2024-09-18T11:31:08.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0083694990000000007, 2024-09-18T11:31:08.744 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:08.744 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:08.744 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:08.744 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:08.744 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:08.744 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1669 2024-09-18T11:31:08.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.744 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:08.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:08.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:31:08.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:08.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:31:08.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:08.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822867+0000", 2024-09-18T11:31:08.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:08.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:08.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822871+0000", 2024-09-18T11:31:08.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:31:08.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:08.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822877+0000", 2024-09-18T11:31:08.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8000000000000004e-06 2024-09-18T11:31:08.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:08.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822880+0000", 2024-09-18T11:31:08.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5669999999999999e-06 2024-09-18T11:31:08.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822974+0000", 2024-09-18T11:31:08.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4048999999999993e-05 2024-09-18T11:31:08.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:08.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822980+0000", 2024-09-18T11:31:08.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2789999999999998e-06 2024-09-18T11:31:08.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.829160+0000", 2024-09-18T11:31:08.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0061800179999999998 2024-09-18T11:31:08.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:08.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.829248+0000", 2024-09-18T11:31:08.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7707000000000006e-05 2024-09-18T11:31:08.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:08.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.831237+0000", 2024-09-18T11:31:08.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001989827 2024-09-18T11:31:08.748 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.748 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:08.748 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.749 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:100 4.a e58/22 4:5975a5ec:::smithi11939506-36:18 v 58'55, mlcod=56'50)", 2024-09-18T11:31:08.749 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:08.749 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 29.402505143999999, 2024-09-18T11:31:08.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0088255, 2024-09-18T11:31:08.749 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:08.749 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:08.749 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:08.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:08.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:08.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:08.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:08.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:08.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:08.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:08.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:08.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273155+0000", 2024-09-18T11:31:08.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.212e-06 2024-09-18T11:31:08.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:08.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273164+0000", 2024-09-18T11:31:08.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7609999999999996e-06 2024-09-18T11:31:08.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:08.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273168+0000", 2024-09-18T11:31:08.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2819999999999998e-06 2024-09-18T11:31:08.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274162+0000", 2024-09-18T11:31:08.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099398600000000009 2024-09-18T11:31:08.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:08.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274180+0000", 2024-09-18T11:31:08.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8236e-05 2024-09-18T11:31:08.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:08.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281943+0000", 2024-09-18T11:31:08.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007763308 2024-09-18T11:31:08.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:08.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281975+0000", 2024-09-18T11:31:08.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1202999999999999e-05 2024-09-18T11:31:08.753 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.753 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:08.753 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.753 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1954 4.7 4:ef60946b:::smithi11939506-8:head [read 0~2890771,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:08.753 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:08.753 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.335754103999999, 2024-09-18T11:31:08.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091142180000000003, 2024-09-18T11:31:08.754 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:08.754 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:08.754 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:08.754 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:08.754 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:08.754 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1954 2024-09-18T11:31:08.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.754 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:08.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:08.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:08.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:08.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:08.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:08.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:08.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:08.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:08.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339903+0000", 2024-09-18T11:31:08.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7349999999999998e-06 2024-09-18T11:31:08.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:08.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339909+0000", 2024-09-18T11:31:08.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-09-18T11:31:08.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:08.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339913+0000", 2024-09-18T11:31:08.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7670000000000001e-06 2024-09-18T11:31:08.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339970+0000", 2024-09-18T11:31:08.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6988999999999999e-05 2024-09-18T11:31:08.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:08.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339975+0000", 2024-09-18T11:31:08.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7929999999999997e-06 2024-09-18T11:31:08.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346917+0000", 2024-09-18T11:31:08.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069426640000000003 2024-09-18T11:31:08.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:08.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.347026+0000", 2024-09-18T11:31:08.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108241 2024-09-18T11:31:08.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:08.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.349014+0000", 2024-09-18T11:31:08.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001988723 2024-09-18T11:31:08.759 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.759 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:08.759 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.759 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1956 4.6 4:67c1765a:::smithi11939506-30:head [write 705927~558502] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:08.759 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:08.759 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.323645163999998, 2024-09-18T11:31:08.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011813239, 2024-09-18T11:31:08.759 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:08.759 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:08.759 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:08.760 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:08.760 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:08.760 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1956 2024-09-18T11:31:08.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.760 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:08.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:08.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:08.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:08.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:08.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:08.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352007+0000", 2024-09-18T11:31:08.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:08.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:08.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352229+0000", 2024-09-18T11:31:08.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222024 2024-09-18T11:31:08.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:08.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352251+0000", 2024-09-18T11:31:08.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2512000000000001e-05 2024-09-18T11:31:08.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:08.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352260+0000", 2024-09-18T11:31:08.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7339999999999997e-06 2024-09-18T11:31:08.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352307+0000", 2024-09-18T11:31:08.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7061999999999998e-05 2024-09-18T11:31:08.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:08.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352408+0000", 2024-09-18T11:31:08.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101216 2024-09-18T11:31:08.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:08.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352550+0000", 2024-09-18T11:31:08.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014136199999999999 2024-09-18T11:31:08.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:08.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357684+0000", 2024-09-18T11:31:08.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051342499999999999 2024-09-18T11:31:08.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:08.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363736+0000", 2024-09-18T11:31:08.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006052577 2024-09-18T11:31:08.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:08.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363778+0000", 2024-09-18T11:31:08.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1989000000000001e-05 2024-09-18T11:31:08.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:08.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363822+0000", 2024-09-18T11:31:08.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3894999999999999e-05 2024-09-18T11:31:08.765 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.765 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:08.765 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.765 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2093 4.7 4:e0d95e10:::smithi11939506-22:head [assert-version v53,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e73)", 2024-09-18T11:31:08.765 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:55.369779+0000", 2024-09-18T11:31:08.765 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.305874949, 2024-09-18T11:31:08.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078628099999999996, 2024-09-18T11:31:08.766 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:08.766 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:08.766 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:08.766 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:08.766 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:08.766 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2093 2024-09-18T11:31:08.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.766 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:08.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:08.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369779+0000", 2024-09-18T11:31:08.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:08.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369779+0000", 2024-09-18T11:31:08.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:08.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369775+0000", 2024-09-18T11:31:08.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999957 2024-09-18T11:31:08.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:08.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369785+0000", 2024-09-18T11:31:08.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7280000000000007e-06 2024-09-18T11:31:08.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:08.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369798+0000", 2024-09-18T11:31:08.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3509000000000001e-05 2024-09-18T11:31:08.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:08.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369806+0000", 2024-09-18T11:31:08.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3749999999999997e-06 2024-09-18T11:31:08.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369837+0000", 2024-09-18T11:31:08.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1440999999999999e-05 2024-09-18T11:31:08.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:08.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.369886+0000", 2024-09-18T11:31:08.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9103000000000001e-05 2024-09-18T11:31:08.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:08.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.376462+0000", 2024-09-18T11:31:08.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065761980000000001 2024-09-18T11:31:08.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:08.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.377077+0000", 2024-09-18T11:31:08.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00061434599999999999 2024-09-18T11:31:08.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:08.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.377555+0000", 2024-09-18T11:31:08.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047826200000000002 2024-09-18T11:31:08.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:08.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.377599+0000", 2024-09-18T11:31:08.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4366999999999997e-05 2024-09-18T11:31:08.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:08.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.377642+0000", 2024-09-18T11:31:08.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2824999999999997e-05 2024-09-18T11:31:08.771 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.771 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:08.771 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.771 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2278 4.f 4:fc192b81:::smithi11939506-31:head [stat,delete] snapc 26=[26,25,24,1e,1c,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T11:31:08.771 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:08.771 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.2926079520000009, 2024-09-18T11:31:08.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01018694, 2024-09-18T11:31:08.772 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:08.772 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:08.772 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:08.772 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:08.772 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:08.772 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2278 2024-09-18T11:31:08.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.772 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:08.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:08.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:08.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:08.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:08.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:08.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383044+0000", 2024-09-18T11:31:08.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:08.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:08.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383054+0000", 2024-09-18T11:31:08.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5969999999999998e-06 2024-09-18T11:31:08.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:08.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383071+0000", 2024-09-18T11:31:08.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7353000000000001e-05 2024-09-18T11:31:08.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:08.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383081+0000", 2024-09-18T11:31:08.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9170000000000001e-06 2024-09-18T11:31:08.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383150+0000", 2024-09-18T11:31:08.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8503999999999993e-05 2024-09-18T11:31:08.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:08.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383158+0000", 2024-09-18T11:31:08.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5089999999999993e-06 2024-09-18T11:31:08.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385396+0000", 2024-09-18T11:31:08.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022381850000000002 2024-09-18T11:31:08.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:08.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385494+0000", 2024-09-18T11:31:08.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7751000000000007e-05 2024-09-18T11:31:08.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:08.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385697+0000", 2024-09-18T11:31:08.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020247400000000001 2024-09-18T11:31:08.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:08.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.390505+0000", 2024-09-18T11:31:08.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048084039999999996 2024-09-18T11:31:08.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:08.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393148+0000", 2024-09-18T11:31:08.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026432399999999998 2024-09-18T11:31:08.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:08.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393194+0000", 2024-09-18T11:31:08.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5259999999999997e-05 2024-09-18T11:31:08.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:08.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393233+0000", 2024-09-18T11:31:08.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9659000000000001e-05 2024-09-18T11:31:08.778 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.778 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:08.778 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.778 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2342 4.f 4:f3c51b3f:::smithi11939506-21:head [stat,delete] snapc 26=[26,25,24,1e,1c,19,18] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:08.778 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:08.778 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.2888519399999998, 2024-09-18T11:31:08.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011167565000000001, 2024-09-18T11:31:08.779 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:08.779 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:08.779 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:08.779 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:08.779 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:08.779 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2342 2024-09-18T11:31:08.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.779 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:08.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:08.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:08.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:08.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:08.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:08.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:31:08.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:08.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:08.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:31:08.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1439999999999997e-06 2024-09-18T11:31:08.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:08.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386820+0000", 2024-09-18T11:31:08.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2651e-05 2024-09-18T11:31:08.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:08.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386825+0000", 2024-09-18T11:31:08.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3150000000000002e-06 2024-09-18T11:31:08.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386868+0000", 2024-09-18T11:31:08.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3832999999999999e-05 2024-09-18T11:31:08.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:08.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:31:08.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0409999999999998e-06 2024-09-18T11:31:08.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390445+0000", 2024-09-18T11:31:08.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035689459999999999 2024-09-18T11:31:08.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:08.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390539+0000", 2024-09-18T11:31:08.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3791999999999999e-05 2024-09-18T11:31:08.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:08.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390748+0000", 2024-09-18T11:31:08.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000208863 2024-09-18T11:31:08.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:08.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397659+0000", 2024-09-18T11:31:08.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069110710000000004 2024-09-18T11:31:08.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:08.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397873+0000", 2024-09-18T11:31:08.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021343999999999999 2024-09-18T11:31:08.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:08.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397923+0000", 2024-09-18T11:31:08.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0300000000000003e-05 2024-09-18T11:31:08.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:08.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397970+0000", 2024-09-18T11:31:08.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6968999999999997e-05 2024-09-18T11:31:08.785 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.785 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:08.785 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.785 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2429 4.b 4:d8d14c58:::smithi11939506-49:head [write 796698~561170,stat] snapc 2b=[2b,2a,26,25,24,1e,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T11:31:08.785 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:08.785 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.6604827659999999, 2024-09-18T11:31:08.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097188989999999996, 2024-09-18T11:31:08.785 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:08.785 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:08.785 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:08.785 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:08.785 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:08.786 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2429 2024-09-18T11:31:08.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.786 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:08.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:08.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:08.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:08.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:08.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:08.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:08.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015170+0000", 2024-09-18T11:31:08.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:08.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:08.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015435+0000", 2024-09-18T11:31:08.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026533299999999998 2024-09-18T11:31:08.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:08.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015455+0000", 2024-09-18T11:31:08.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9623000000000002e-05 2024-09-18T11:31:08.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:08.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015461+0000", 2024-09-18T11:31:08.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1859999999999997e-06 2024-09-18T11:31:08.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015535+0000", 2024-09-18T11:31:08.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4555999999999996e-05 2024-09-18T11:31:08.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:08.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015543+0000", 2024-09-18T11:31:08.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9549999999999992e-06 2024-09-18T11:31:08.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:08.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017106+0000", 2024-09-18T11:31:08.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015630290000000001 2024-09-18T11:31:08.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:08.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017183+0000", 2024-09-18T11:31:08.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6953999999999995e-05 2024-09-18T11:31:08.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:08.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017301+0000", 2024-09-18T11:31:08.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011734 2024-09-18T11:31:08.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:08.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.020225+0000", 2024-09-18T11:31:08.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029244090000000002 2024-09-18T11:31:08.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:08.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024801+0000", 2024-09-18T11:31:08.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045764129999999997 2024-09-18T11:31:08.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:08.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024840+0000", 2024-09-18T11:31:08.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8325999999999999e-05 2024-09-18T11:31:08.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:08.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:08.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:08.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024890+0000", 2024-09-18T11:31:08.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0596999999999997e-05 2024-09-18T11:31:08.791 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.791 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:08.791 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.791 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:08.791 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:08.791 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:08.792 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:08.793 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:08.794 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:31:08.794 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:08.794 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:08.794 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:08.795 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:08.795 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:08.795 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:08.796 INFO:tasks.rados.rados.0.smithi119.stdout:1198: expect (ObjNum 178 snap 20 seq_num 178) 2024-09-18T11:31:08.796 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:08.792+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:08.894 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:08.893+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:08.932 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:08.932 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:08.933 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:08.933 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:08.941 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:31:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:1200: finishing set_chunk to oid 43 2024-09-18T11:31:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 43 set_chunk returned error code -95 offset: 1060864 length: 928768 2024-09-18T11:31:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-18T11:31:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:1201: finishing set_chunk to oid 37 2024-09-18T11:31:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 121 (ObjNum 188 snap 20 seq_num 188) dirty exists 2024-09-18T11:31:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:1203: got (0) Success 2024-09-18T11:31:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 148 (ObjNum 213 snap 24 seq_num 213) dirty exists 2024-09-18T11:31:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:1194: done (8 left) 2024-09-18T11:31:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:1195: done (7 left) 2024-09-18T11:31:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:1196: done (6 left) 2024-09-18T11:31:08.961 INFO:tasks.rados.rados.0.smithi119.stdout:1197: done (5 left) 2024-09-18T11:31:08.962 INFO:tasks.rados.rados.0.smithi119.stdout:1198: done (4 left) 2024-09-18T11:31:08.962 INFO:tasks.rados.rados.0.smithi119.stdout:1199: done (3 left) 2024-09-18T11:31:08.962 INFO:tasks.rados.rados.0.smithi119.stdout:1200: done (2 left) 2024-09-18T11:31:08.962 INFO:tasks.rados.rados.0.smithi119.stdout:1201: done (1 left) 2024-09-18T11:31:08.962 INFO:tasks.rados.rados.0.smithi119.stdout:1203: done (0 left) 2024-09-18T11:31:08.962 INFO:tasks.rados.rados.0.smithi119.stdout:1204: tier_promote oid 10 2024-09-18T11:31:08.962 INFO:tasks.rados.rados.0.smithi119.stdout:1205: set_chunk oid 37 target oid 2024-09-18T11:31:08.963 INFO:tasks.rados.rados.0.smithi119.stdout:1205: set_chunk oid 37 offset: 230400 length: 207872 target oid 62b68375792ff53ce6e2124fb3abe7260e646271175f80bf91d38dde142b1d6f offset: 0 snap -1 2024-09-18T11:31:08.963 INFO:tasks.rados.rados.0.smithi119.stdout:1206: read oid 42 snap -1 2024-09-18T11:31:08.963 INFO:tasks.rados.rados.0.smithi119.stdout:1206: expect (ObjNum 99 snap 6 seq_num 99) 2024-09-18T11:31:08.963 INFO:tasks.rados.rados.0.smithi119.stdout:1207: set_chunk oid 28 target oid 2024-09-18T11:31:08.986 INFO:tasks.rados.rados.0.smithi119.stdout:1207: set_chunk oid 28 offset: 476160 length: 2110976 target oid e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6 offset: 0 snap -1 2024-09-18T11:31:08.986 INFO:tasks.rados.rados.0.smithi119.stdout:1208: write (excl) oid 14 current snap is 26 2024-09-18T11:31:08.987 INFO:tasks.rados.rados.0.smithi119.stdout:1204: got (0) Success 2024-09-18T11:31:08.987 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 10 v 100 (ObjNum 194 snap 23 seq_num 194) dirty exists 2024-09-18T11:31:08.987 INFO:tasks.rados.rados.0.smithi119.stdout:1208: seq_num 220 ranges {497540=612664,1840862=440060,3058823=464784} 2024-09-18T11:31:08.992 INFO:tasks.rados.rados.0.smithi119.stdout:1208: writing smithi11939506-14 from 497540 to 1110204 tid 1 2024-09-18T11:31:08.994 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:08.993+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:08.999 INFO:tasks.rados.rados.0.smithi119.stdout:1208: writing smithi11939506-14 from 1840862 to 2280922 tid 2 2024-09-18T11:31:09.006 INFO:tasks.rados.rados.0.smithi119.stdout:1208: writing smithi11939506-14 from 3058823 to 3523607 tid 3 2024-09-18T11:31:09.007 INFO:tasks.rados.rados.0.smithi119.stdout:1204: done (4 left) 2024-09-18T11:31:09.007 INFO:tasks.rados.rados.0.smithi119.stdout:1209: set_chunk oid 41 target oid 2024-09-18T11:31:09.007 INFO:tasks.rados.rados.0.smithi119.stdout:1209: set_chunk oid 41 offset: 0 length: 4000000 target oid smithi11939506- offset: 0 snap -1 2024-09-18T11:31:09.007 INFO:tasks.rados.rados.0.smithi119.stdout:1205: finishing set_chunk to oid 37 2024-09-18T11:31:09.007 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 121 (ObjNum 188 snap 20 seq_num 188) dirty exists 2024-09-18T11:31:09.007 INFO:tasks.rados.rados.0.smithi119.stdout:1206: expect (ObjNum 99 snap 6 seq_num 99) 2024-09-18T11:31:09.063 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:09.063 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:09.064 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:09.064 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:09.064 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:09.072 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:31:09.095 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:09.095+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:09.174 INFO:tasks.rados.rados.0.smithi119.stdout:1207: finishing set_chunk to oid 28 2024-09-18T11:31:09.174 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 28 v 97 (ObjNum 200 snap 23 seq_num 200) dirty exists 2024-09-18T11:31:09.174 INFO:tasks.rados.rados.0.smithi119.stdout:1208: finishing write tid 1 to smithi11939506-14 2024-09-18T11:31:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:1210: read oid 11 snap -1 2024-09-18T11:31:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:1210: expect (ObjNum 148 snap 18 seq_num 148) 2024-09-18T11:31:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:1205: done (5 left) 2024-09-18T11:31:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:1206: done (4 left) 2024-09-18T11:31:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:1207: done (3 left) 2024-09-18T11:31:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:1211: read oid 35 snap -1 2024-09-18T11:31:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:1211: expect (ObjNum 155 snap 20 seq_num 155) 2024-09-18T11:31:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:1212: read oid 42 snap -1 2024-09-18T11:31:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:1212: expect (ObjNum 99 snap 6 seq_num 99) 2024-09-18T11:31:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:1208: finishing write tid 2 to smithi11939506-14 2024-09-18T11:31:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:1209: finishing set_chunk to oid 41 2024-09-18T11:31:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:1209: got expected ERANGE (src dne) 2024-09-18T11:31:09.175 INFO:tasks.rados.rados.0.smithi119.stdout:1208: finishing write tid 3 to smithi11939506-14 2024-09-18T11:31:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:1213: read oid 10 snap -1 2024-09-18T11:31:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:1213: expect (ObjNum 194 snap 23 seq_num 194) 2024-09-18T11:31:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:1209: done (5 left) 2024-09-18T11:31:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:1214: read oid 20 snap -1 2024-09-18T11:31:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:1214: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:31:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:1215: read oid 45 snap -1 2024-09-18T11:31:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:1215: expect (ObjNum 197 snap 23 seq_num 197) 2024-09-18T11:31:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:1216: rollback oid 37 current snap is 26 2024-09-18T11:31:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 37 to 21 2024-09-18T11:31:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:1208: finishing write tid 4 to smithi11939506-14 2024-09-18T11:31:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:1208: finishing write tid 5 to smithi11939506-14 2024-09-18T11:31:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 170 (ObjNum 220 snap 26 seq_num 220) dirty exists 2024-09-18T11:31:09.176 INFO:tasks.rados.rados.0.smithi119.stdout:1208: left oid 14 (ObjNum 220 snap 26 seq_num 220) 2024-09-18T11:31:09.177 INFO:tasks.rados.rados.0.smithi119.stdout:1208: done (7 left) 2024-09-18T11:31:09.177 INFO:tasks.rados.rados.0.smithi119.stdout:1217: set_chunk oid 1 target oid 2024-09-18T11:31:09.181 INFO:tasks.rados.rados.0.smithi119.stdout:1217: set_chunk oid 1 offset: 1474048 length: 160768 target oid a05045e91e01e7432fd564e9167fb4970a512c60eb0731e2e2d995a0e1507dc5 offset: 0 snap -1 2024-09-18T11:31:09.182 INFO:tasks.rados.rados.0.smithi119.stdout:1218: delete oid 5 current snap is 26 2024-09-18T11:31:09.182 INFO:tasks.rados.rados.0.smithi119.stdout:1213: expect (ObjNum 194 snap 23 seq_num 194) 2024-09-18T11:31:09.196 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:09.195+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:09.219 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:09.219 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:09.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:09.219 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:09.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.220 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1046 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'24, mlcod=30'21)", 2024-09-18T11:31:09.220 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:31:09.220 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.043513789999999, 2024-09-18T11:31:09.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042519100000000002, 2024-09-18T11:31:09.220 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.220 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.220 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:31:09.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:31:09.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173782+0000", 2024-09-18T11:31:09.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:09.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174207+0000", 2024-09-18T11:31:09.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00042479799999999999 2024-09-18T11:31:09.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174224+0000", 2024-09-18T11:31:09.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7513999999999999e-05 2024-09-18T11:31:09.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174232+0000", 2024-09-18T11:31:09.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0840000000000003e-06 2024-09-18T11:31:09.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174297+0000", 2024-09-18T11:31:09.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5215000000000005e-05 2024-09-18T11:31:09.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174336+0000", 2024-09-18T11:31:09.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9403000000000002e-05 2024-09-18T11:31:09.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.177990+0000", 2024-09-18T11:31:09.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036533709999999999 2024-09-18T11:31:09.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.178035+0000", 2024-09-18T11:31:09.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5495999999999999e-05 2024-09-18T11:31:09.225 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.225 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.225 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.225 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1047 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'25, mlcod=35'23)", 2024-09-18T11:31:09.225 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:31:09.225 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.037541591999997, 2024-09-18T11:31:09.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038129129999999998, 2024-09-18T11:31:09.226 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.226 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.226 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:31:09.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179755+0000", 2024-09-18T11:31:09.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.179754+0000", 2024-09-18T11:31:09.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:09.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180088+0000", 2024-09-18T11:31:09.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00033395000000000001 2024-09-18T11:31:09.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180105+0000", 2024-09-18T11:31:09.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7039000000000001e-05 2024-09-18T11:31:09.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180112+0000", 2024-09-18T11:31:09.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3710000000000002e-06 2024-09-18T11:31:09.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180176+0000", 2024-09-18T11:31:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3178999999999999e-05 2024-09-18T11:31:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.180197+0000", 2024-09-18T11:31:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1687000000000001e-05 2024-09-18T11:31:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183542+0000", 2024-09-18T11:31:09.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003344877 2024-09-18T11:31:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.183568+0000", 2024-09-18T11:31:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6146e-05 2024-09-18T11:31:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:31:09.230 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:31:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 49.233922923000002, 2024-09-18T11:31:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:31:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:31:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:31:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:31:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:31:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:31:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:31:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:31:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:31:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:31:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:31:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:31:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:31:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:31:09.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:31:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:31:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:31:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:31:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:31:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.999091970000002, 2024-09-18T11:31:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:31:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:09.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:31:09.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:31:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:31:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:31:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:31:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:31:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:31:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:31:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:31:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:31:09.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:31:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:31:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:31:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:31:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:31:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.240 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1512 4.4 e51/22 4:2c93196f:::smithi11939506-28:head v 51'51, mlcod=51'48)", 2024-09-18T11:31:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:32.782201+0000", 2024-09-18T11:31:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.435096016000003, 2024-09-18T11:31:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00363554, 2024-09-18T11:31:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782201+0000", 2024-09-18T11:31:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782201+0000", 2024-09-18T11:31:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782200+0000", 2024-09-18T11:31:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:09.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782564+0000", 2024-09-18T11:31:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000364612 2024-09-18T11:31:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782585+0000", 2024-09-18T11:31:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0418999999999999e-05 2024-09-18T11:31:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782594+0000", 2024-09-18T11:31:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9570000000000008e-06 2024-09-18T11:31:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782663+0000", 2024-09-18T11:31:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9330000000000002e-05 2024-09-18T11:31:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.782688+0000", 2024-09-18T11:31:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5420000000000001e-05 2024-09-18T11:31:09.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.785790+0000", 2024-09-18T11:31:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0031016160000000002 2024-09-18T11:31:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:32.785836+0000", 2024-09-18T11:31:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6360999999999999e-05 2024-09-18T11:31:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.245 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:31:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:31:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 34.825185685000001, 2024-09-18T11:31:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:31:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:31:09.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:31:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:31:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:31:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:31:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:31:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:31:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:31:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:31:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:31:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:31:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:31:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:31:09.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:31:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:31:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:31:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:31:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.13:87 3.0 e58/21 3:099aa86f:::smithi11939506-4 low_tier:head v 58'69, mlcod=55'66)", 2024-09-18T11:31:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.270180+0000", 2024-09-18T11:31:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 29.947117159000001, 2024-09-18T11:31:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037855440000000001, 2024-09-18T11:31:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270180+0000", 2024-09-18T11:31:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270180+0000", 2024-09-18T11:31:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270179+0000", 2024-09-18T11:31:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270187+0000", 2024-09-18T11:31:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5610000000000001e-06 2024-09-18T11:31:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270195+0000", 2024-09-18T11:31:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5909999999999996e-06 2024-09-18T11:31:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270200+0000", 2024-09-18T11:31:09.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2599999999999999e-06 2024-09-18T11:31:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270252+0000", 2024-09-18T11:31:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1891000000000002e-05 2024-09-18T11:31:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.270277+0000", 2024-09-18T11:31:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5134e-05 2024-09-18T11:31:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273928+0000", 2024-09-18T11:31:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00365132 2024-09-18T11:31:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273965+0000", 2024-09-18T11:31:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7277000000000002e-05 2024-09-18T11:31:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.255 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:289 4.d e58/22 4:b5a463bf:::smithi11939506-20:10 v 58'78, mlcod=53'73)", 2024-09-18T11:31:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.275682+0000", 2024-09-18T11:31:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 29.941614795, 2024-09-18T11:31:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003584318, 2024-09-18T11:31:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275682+0000", 2024-09-18T11:31:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275682+0000", 2024-09-18T11:31:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275681+0000", 2024-09-18T11:31:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275689+0000", 2024-09-18T11:31:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6429999999999995e-06 2024-09-18T11:31:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275698+0000", 2024-09-18T11:31:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1120000000000002e-06 2024-09-18T11:31:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275702+0000", 2024-09-18T11:31:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1450000000000001e-06 2024-09-18T11:31:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275731+0000", 2024-09-18T11:31:09.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8782999999999999e-05 2024-09-18T11:31:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.275743+0000", 2024-09-18T11:31:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1436e-05 2024-09-18T11:31:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.279231+0000", 2024-09-18T11:31:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034884339999999999 2024-09-18T11:31:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.279266+0000", 2024-09-18T11:31:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5488000000000003e-05 2024-09-18T11:31:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1931 3.0 e65/21 3:3c05e498:::866d65857c51f67d56ecbcba33930954810c64ced2a213ca9705911438b0dfb7:head v 65'72, mlcod=58'70)", 2024-09-18T11:31:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:31:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.849175667000001, 2024-09-18T11:31:09.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040630859999999996, 2024-09-18T11:31:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:31:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:31:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368120+0000", 2024-09-18T11:31:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368427+0000", 2024-09-18T11:31:09.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030742600000000002 2024-09-18T11:31:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368446+0000", 2024-09-18T11:31:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8046e-05 2024-09-18T11:31:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368453+0000", 2024-09-18T11:31:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1380000000000001e-06 2024-09-18T11:31:09.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368519+0000", 2024-09-18T11:31:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6470000000000006e-05 2024-09-18T11:31:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368550+0000", 2024-09-18T11:31:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0707999999999998e-05 2024-09-18T11:31:09.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.372151+0000", 2024-09-18T11:31:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036015470000000001 2024-09-18T11:31:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.372184+0000", 2024-09-18T11:31:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2994999999999998e-05 2024-09-18T11:31:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.266 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:304 4.6 e69/67 4:67c1765a:::smithi11939506-30:17 v 69'35, mlcod=58'31)", 2024-09-18T11:31:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.871344591, 2024-09-18T11:31:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006377611, 2024-09-18T11:31:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345961+0000", 2024-09-18T11:31:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2050000000000008e-06 2024-09-18T11:31:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345969+0000", 2024-09-18T11:31:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0339999999999997e-06 2024-09-18T11:31:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345976+0000", 2024-09-18T11:31:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-18T11:31:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346026+0000", 2024-09-18T11:31:09.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9286999999999998e-05 2024-09-18T11:31:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346050+0000", 2024-09-18T11:31:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4397e-05 2024-09-18T11:31:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352293+0000", 2024-09-18T11:31:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062427170000000001 2024-09-18T11:31:09.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352330+0000", 2024-09-18T11:31:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7348999999999998e-05 2024-09-18T11:31:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:347 4.d e69/22 4:b44229a1:::smithi11939506-42:19 v 69'99, mlcod=65'93)", 2024-09-18T11:31:09.272 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:31:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.865611395999998, 2024-09-18T11:31:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056998760000000004, 2024-09-18T11:31:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:31:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:31:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351684+0000", 2024-09-18T11:31:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351693+0000", 2024-09-18T11:31:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-18T11:31:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351704+0000", 2024-09-18T11:31:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1077000000000001e-05 2024-09-18T11:31:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351709+0000", 2024-09-18T11:31:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5419999999999999e-06 2024-09-18T11:31:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351956+0000", 2024-09-18T11:31:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000247252 2024-09-18T11:31:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351975+0000", 2024-09-18T11:31:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9201000000000001e-05 2024-09-18T11:31:09.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357334+0000", 2024-09-18T11:31:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053588749999999999 2024-09-18T11:31:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357385+0000", 2024-09-18T11:31:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1406999999999998e-05 2024-09-18T11:31:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.277 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1956 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'37, mlcod=58'31)", 2024-09-18T11:31:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.864666483000001, 2024-09-18T11:31:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010958842, 2024-09-18T11:31:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352996+0000", 2024-09-18T11:31:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036622 2024-09-18T11:31:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353011+0000", 2024-09-18T11:31:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4849e-05 2024-09-18T11:31:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353017+0000", 2024-09-18T11:31:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:31:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353068+0000", 2024-09-18T11:31:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0695000000000002e-05 2024-09-18T11:31:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353092+0000", 2024-09-18T11:31:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4260000000000002e-05 2024-09-18T11:31:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363551+0000", 2024-09-18T11:31:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010459154999999999 2024-09-18T11:31:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363589+0000", 2024-09-18T11:31:09.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8161000000000003e-05 2024-09-18T11:31:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1957 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'38, mlcod=65'33)", 2024-09-18T11:31:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.858250615999999, 2024-09-18T11:31:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050499400000000002, 2024-09-18T11:31:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359214+0000", 2024-09-18T11:31:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000168047 2024-09-18T11:31:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359227+0000", 2024-09-18T11:31:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3124e-05 2024-09-18T11:31:09.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359235+0000", 2024-09-18T11:31:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0760000000000003e-06 2024-09-18T11:31:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359287+0000", 2024-09-18T11:31:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.151e-05 2024-09-18T11:31:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359310+0000", 2024-09-18T11:31:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-09-18T11:31:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364065+0000", 2024-09-18T11:31:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047549439999999997 2024-09-18T11:31:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364096+0000", 2024-09-18T11:31:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1423000000000001e-05 2024-09-18T11:31:09.287 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:349 4.d e69/22 4:b9bb642b:::smithi11939506-27:19 v 69'101, mlcod=69'97)", 2024-09-18T11:31:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.360028+0000", 2024-09-18T11:31:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.857268998999999, 2024-09-18T11:31:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040307850000000003, 2024-09-18T11:31:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.288 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360028+0000", 2024-09-18T11:31:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360028+0000", 2024-09-18T11:31:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360027+0000", 2024-09-18T11:31:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360039+0000", 2024-09-18T11:31:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1654000000000001e-05 2024-09-18T11:31:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360052+0000", 2024-09-18T11:31:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.33e-05 2024-09-18T11:31:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360058+0000", 2024-09-18T11:31:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0440000000000001e-06 2024-09-18T11:31:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360094+0000", 2024-09-18T11:31:09.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6402999999999997e-05 2024-09-18T11:31:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360109+0000", 2024-09-18T11:31:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4105e-05 2024-09-18T11:31:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364025+0000", 2024-09-18T11:31:09.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039163189999999997 2024-09-18T11:31:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364059+0000", 2024-09-18T11:31:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3810999999999999e-05 2024-09-18T11:31:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.293 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1978 4.4 e69/22 4:2d9c3f7f:::smithi11939506-5:head v 69'74, mlcod=69'71)", 2024-09-18T11:31:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.967898+0000", 2024-09-18T11:31:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.249398547999998, 2024-09-18T11:31:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038277960000000001, 2024-09-18T11:31:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.967898+0000", 2024-09-18T11:31:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.967898+0000", 2024-09-18T11:31:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.967898+0000", 2024-09-18T11:31:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.968050+0000", 2024-09-18T11:31:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015286799999999999 2024-09-18T11:31:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.968064+0000", 2024-09-18T11:31:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3964999999999999e-05 2024-09-18T11:31:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.968070+0000", 2024-09-18T11:31:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:31:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.968112+0000", 2024-09-18T11:31:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1112999999999998e-05 2024-09-18T11:31:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.968133+0000", 2024-09-18T11:31:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1715000000000001e-05 2024-09-18T11:31:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.971687+0000", 2024-09-18T11:31:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035536410000000002 2024-09-18T11:31:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.971726+0000", 2024-09-18T11:31:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.926e-05 2024-09-18T11:31:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:407 4.4 e77/22 4:2d9c3f7f:::smithi11939506-5:1d v 77'92, mlcod=73'87)", 2024-09-18T11:31:09.299 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.387293+0000", 2024-09-18T11:31:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.8300037919999994, 2024-09-18T11:31:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038531820000000001, 2024-09-18T11:31:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387293+0000", 2024-09-18T11:31:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387293+0000", 2024-09-18T11:31:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387293+0000", 2024-09-18T11:31:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387300+0000", 2024-09-18T11:31:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6799999999999993e-06 2024-09-18T11:31:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387311+0000", 2024-09-18T11:31:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0278999999999999e-05 2024-09-18T11:31:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387316+0000", 2024-09-18T11:31:09.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9269999999999996e-06 2024-09-18T11:31:09.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387357+0000", 2024-09-18T11:31:09.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1208000000000003e-05 2024-09-18T11:31:09.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387374+0000", 2024-09-18T11:31:09.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7499e-05 2024-09-18T11:31:09.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.391105+0000", 2024-09-18T11:31:09.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037302619999999998 2024-09-18T11:31:09.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.391146+0000", 2024-09-18T11:31:09.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1797e-05 2024-09-18T11:31:09.305 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.305 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.305 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.305 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2300 4.4 e77/22 4:2c93196f:::smithi11939506-28:head v 77'94, mlcod=77'90)", 2024-09-18T11:31:09.305 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:31:09.305 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.2291883979999998, 2024-09-18T11:31:09.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050425380000000001, 2024-09-18T11:31:09.305 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.305 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.306 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:31:09.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:31:09.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988108+0000", 2024-09-18T11:31:09.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:09.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988388+0000", 2024-09-18T11:31:09.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028087700000000001 2024-09-18T11:31:09.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988405+0000", 2024-09-18T11:31:09.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6565000000000001e-05 2024-09-18T11:31:09.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988412+0000", 2024-09-18T11:31:09.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9709999999999998e-06 2024-09-18T11:31:09.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988464+0000", 2024-09-18T11:31:09.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1949e-05 2024-09-18T11:31:09.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988490+0000", 2024-09-18T11:31:09.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5780999999999999e-05 2024-09-18T11:31:09.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993112+0000", 2024-09-18T11:31:09.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046228399999999996 2024-09-18T11:31:09.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993151+0000", 2024-09-18T11:31:09.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8587999999999997e-05 2024-09-18T11:31:09.310 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.310 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.310 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.310 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:451 4.d e78/22 4:b44229a1:::smithi11939506-42:19 v 78'125, mlcod=76'121)", 2024-09-18T11:31:09.311 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.400699+0000", 2024-09-18T11:31:09.311 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.8165976720000003, 2024-09-18T11:31:09.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038090789999999999, 2024-09-18T11:31:09.311 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.311 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.311 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.400699+0000", 2024-09-18T11:31:09.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.400699+0000", 2024-09-18T11:31:09.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.400699+0000", 2024-09-18T11:31:09.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:09.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.400705+0000", 2024-09-18T11:31:09.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.789e-06 2024-09-18T11:31:09.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.400721+0000", 2024-09-18T11:31:09.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6412000000000001e-05 2024-09-18T11:31:09.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.400727+0000", 2024-09-18T11:31:09.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1410000000000002e-06 2024-09-18T11:31:09.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.400783+0000", 2024-09-18T11:31:09.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5588999999999999e-05 2024-09-18T11:31:09.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.400797+0000", 2024-09-18T11:31:09.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4255e-05 2024-09-18T11:31:09.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.404472+0000", 2024-09-18T11:31:09.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036744709999999999 2024-09-18T11:31:09.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.404508+0000", 2024-09-18T11:31:09.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6693999999999998e-05 2024-09-18T11:31:09.315 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.315 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.316 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.316 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.13:63 3.0 e78/21 3:24ebb740:::smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 78'80, mlcod=73'78)", 2024-09-18T11:31:09.316 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:31:09.316 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.812198982, 2024-09-18T11:31:09.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052262230000000003, 2024-09-18T11:31:09.316 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.316 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.316 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:31:09.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:31:09.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405097+0000", 2024-09-18T11:31:09.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:09.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405103+0000", 2024-09-18T11:31:09.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8780000000000003e-06 2024-09-18T11:31:09.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405117+0000", 2024-09-18T11:31:09.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3441000000000001e-05 2024-09-18T11:31:09.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405121+0000", 2024-09-18T11:31:09.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7099999999999998e-06 2024-09-18T11:31:09.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405158+0000", 2024-09-18T11:31:09.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6477999999999999e-05 2024-09-18T11:31:09.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405173+0000", 2024-09-18T11:31:09.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4796e-05 2024-09-18T11:31:09.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410285+0000", 2024-09-18T11:31:09.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051118919999999998 2024-09-18T11:31:09.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410324+0000", 2024-09-18T11:31:09.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9616999999999997e-05 2024-09-18T11:31:09.320 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.321 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.321 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.321 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:451 4.6 e78/67 4:6f86ffcc:::smithi11939506-15:26 v 78'58, mlcod=77'53)", 2024-09-18T11:31:09.321 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.414259+0000", 2024-09-18T11:31:09.321 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.8030383990000001, 2024-09-18T11:31:09.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041821610000000002, 2024-09-18T11:31:09.321 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.321 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.321 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414259+0000", 2024-09-18T11:31:09.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414259+0000", 2024-09-18T11:31:09.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414258+0000", 2024-09-18T11:31:09.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:09.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414264+0000", 2024-09-18T11:31:09.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1850000000000001e-06 2024-09-18T11:31:09.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414273+0000", 2024-09-18T11:31:09.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7290000000000006e-06 2024-09-18T11:31:09.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414278+0000", 2024-09-18T11:31:09.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8470000000000003e-06 2024-09-18T11:31:09.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414315+0000", 2024-09-18T11:31:09.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7462000000000001e-05 2024-09-18T11:31:09.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414334+0000", 2024-09-18T11:31:09.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9080999999999999e-05 2024-09-18T11:31:09.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.418419+0000", 2024-09-18T11:31:09.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040846540000000001 2024-09-18T11:31:09.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.418441+0000", 2024-09-18T11:31:09.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1844999999999998e-05 2024-09-18T11:31:09.325 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.325 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.326 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.326 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.326 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.326 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:09.328 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:31:09.328 INFO:tasks.rados.rados.0.smithi119.stdout:1210: expect (ObjNum 148 snap 18 seq_num 148) 2024-09-18T11:31:09.329 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:09.326+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:09.374 INFO:tasks.rados.rados.0.smithi119.stdout:1216: finishing rollback tid 0 to smithi11939506-37 2024-09-18T11:31:09.374 INFO:tasks.rados.rados.0.smithi119.stdout:1216: finishing rollback tid 1 to smithi11939506-37 2024-09-18T11:31:09.374 INFO:tasks.rados.rados.0.smithi119.stdout:1210: done (8 left) 2024-09-18T11:31:09.374 INFO:tasks.rados.rados.0.smithi119.stdout:1213: done (7 left) 2024-09-18T11:31:09.374 INFO:tasks.rados.rados.0.smithi119.stdout:1218: done (6 left) 2024-09-18T11:31:09.374 INFO:tasks.rados.rados.0.smithi119.stdout:1219: write oid 21 current snap is 26 2024-09-18T11:31:09.374 INFO:tasks.rados.rados.0.smithi119.stdout:1216: finishing rollback tid 2 to smithi11939506-37 2024-09-18T11:31:09.374 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 157 (ObjNum 188 snap 20 seq_num 188) dirty exists 2024-09-18T11:31:09.374 INFO:tasks.rados.rados.0.smithi119.stdout:1217: finishing set_chunk to oid 1 2024-09-18T11:31:09.375 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 96 (ObjNum 186 snap 20 seq_num 186) dirty exists 2024-09-18T11:31:09.375 INFO:tasks.rados.rados.0.smithi119.stdout:1212: expect (ObjNum 99 snap 6 seq_num 99) 2024-09-18T11:31:09.428 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:09.427+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:09.470 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:09.471 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:09.471 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:09.471 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:09.480 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:31:09.529 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:09.528+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:09.545 INFO:tasks.rados.rados.0.smithi119.stdout:1211: expect (ObjNum 155 snap 20 seq_num 155) 2024-09-18T11:31:09.620 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:09.620 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:09.620 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:09.620 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:09.620 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:09.628 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:31:09.629 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:09.628+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:09.691 INFO:tasks.rados.rados.0.smithi119.stdout:1215: expect (ObjNum 197 snap 23 seq_num 197) 2024-09-18T11:31:09.731 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:09.729+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:09.768 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:09.768 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:09.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:09.768 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:09.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.768 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:901 4.0 4:0aec5e1c:::smithi11939506-50:head [sparse-read 0~1781150] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:31:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.632810696999996, 2024-09-18T11:31:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063828720000000004, 2024-09-18T11:31:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 901 2024-09-18T11:31:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:31:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:31:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133265+0000", 2024-09-18T11:31:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133268+0000", 2024-09-18T11:31:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9610000000000001e-06 2024-09-18T11:31:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133280+0000", 2024-09-18T11:31:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2481e-05 2024-09-18T11:31:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133285+0000", 2024-09-18T11:31:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5469999999999998e-06 2024-09-18T11:31:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133317+0000", 2024-09-18T11:31:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1838999999999998e-05 2024-09-18T11:31:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133320+0000", 2024-09-18T11:31:09.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3009999999999998e-06 2024-09-18T11:31:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139225+0000", 2024-09-18T11:31:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059047919999999999 2024-09-18T11:31:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139330+0000", 2024-09-18T11:31:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104924 2024-09-18T11:31:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.139648+0000", 2024-09-18T11:31:09.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00031854799999999999 2024-09-18T11:31:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.596656656999997, 2024-09-18T11:31:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:31:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:09.774 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:31:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:31:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:31:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:31:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:31:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:31:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:31:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:31:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:31:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:31:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:31:09.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:31:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:31:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:31:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1001 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:09.778 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.601505379000002, 2024-09-18T11:31:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007461082, 2024-09-18T11:31:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1001 2024-09-18T11:31:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:09.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164570+0000", 2024-09-18T11:31:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164574+0000", 2024-09-18T11:31:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.822e-06 2024-09-18T11:31:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164582+0000", 2024-09-18T11:31:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3410000000000003e-06 2024-09-18T11:31:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164587+0000", 2024-09-18T11:31:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769999999999999e-06 2024-09-18T11:31:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164702+0000", 2024-09-18T11:31:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115705 2024-09-18T11:31:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164709+0000", 2024-09-18T11:31:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-09-18T11:31:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170296+0000", 2024-09-18T11:31:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055876390000000001 2024-09-18T11:31:09.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170418+0000", 2024-09-18T11:31:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121549 2024-09-18T11:31:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172032+0000", 2024-09-18T11:31:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00161407 2024-09-18T11:31:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.601478663000002, 2024-09-18T11:31:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:31:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:31:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:31:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:31:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:31:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:31:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:31:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:31:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:31:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:31:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:31:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:31:09.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:31:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:31:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:31:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:31:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:31:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:31:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.601460373999998, 2024-09-18T11:31:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:31:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.789 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:31:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:31:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:31:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:31:09.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:31:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:31:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:31:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:31:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:31:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:31:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:31:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:31:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:31:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:31:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:31:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:31:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:31:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:31:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:09.794 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.601399604999997, 2024-09-18T11:31:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:31:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:31:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:31:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:31:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:31:09.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:31:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:31:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:31:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:31:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:31:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:31:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:31:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:31:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:31:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:31:09.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:31:09.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:31:09.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:31:09.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:31:09.799 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.799 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.799 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.800 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:09.800 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:09.800 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.601377284000002, 2024-09-18T11:31:09.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:31:09.800 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.800 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:09.800 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:09.800 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:09.800 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:09.800 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:31:09.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.800 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:09.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:09.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:31:09.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:09.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:31:09.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:31:09.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:31:09.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:31:09.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:31:09.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:31:09.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:31:09.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:31:09.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:09.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:31:09.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:31:09.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:31:09.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:31:09.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:31:09.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:31:09.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:31:09.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:31:09.804 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.804 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.804 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.804 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1018 4.0 4:0aec5e1c:::smithi11939506-50:head [write 1622620~730352] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:09.805 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:31:09.805 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.085142728000001, 2024-09-18T11:31:09.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064405160000000003, 2024-09-18T11:31:09.805 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.805 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.805 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:09.805 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:09.805 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:09.805 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1018 2024-09-18T11:31:09.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.805 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:31:09.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:31:09.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680932+0000", 2024-09-18T11:31:09.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:09.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681334+0000", 2024-09-18T11:31:09.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000401803 2024-09-18T11:31:09.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681350+0000", 2024-09-18T11:31:09.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6218999999999998e-05 2024-09-18T11:31:09.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681374+0000", 2024-09-18T11:31:09.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4088e-05 2024-09-18T11:31:09.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681415+0000", 2024-09-18T11:31:09.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1307000000000003e-05 2024-09-18T11:31:09.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681507+0000", 2024-09-18T11:31:09.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1867999999999995e-05 2024-09-18T11:31:09.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:09.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681671+0000", 2024-09-18T11:31:09.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016406 2024-09-18T11:31:09.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:09.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.683856+0000", 2024-09-18T11:31:09.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002184624 2024-09-18T11:31:09.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:09.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687266+0000", 2024-09-18T11:31:09.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034105490000000001 2024-09-18T11:31:09.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687321+0000", 2024-09-18T11:31:09.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4960999999999998e-05 2024-09-18T11:31:09.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687374+0000", 2024-09-18T11:31:09.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2553000000000003e-05 2024-09-18T11:31:09.810 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.810 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.810 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.810 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:31:09.810 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:09.810 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 49.820252400999998, 2024-09-18T11:31:09.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:31:09.810 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.810 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.810 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:09.810 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:09.811 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:09.811 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:31:09.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.811 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:09.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:09.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:31:09.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:31:09.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:31:09.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:31:09.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:31:09.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:31:09.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:31:09.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:31:09.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:31:09.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:31:09.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:31:09.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:31:09.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:09.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:31:09.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:31:09.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:09.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:31:09.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:31:09.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:09.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:31:09.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:31:09.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:31:09.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:31:09.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:31:09.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:31:09.815 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.815 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.815 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.816 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1121 4.f e38/22 4:f5d7df41:::smithi11939506-16:head v 38'34, mlcod=30'32)", 2024-09-18T11:31:09.816 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:31:09.816 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 49.800265142000001, 2024-09-18T11:31:09.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060953680000000003, 2024-09-18T11:31:09.816 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.816 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.816 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:31:09.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965811+0000", 2024-09-18T11:31:09.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.965810+0000", 2024-09-18T11:31:09.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:09.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966142+0000", 2024-09-18T11:31:09.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000331591 2024-09-18T11:31:09.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966154+0000", 2024-09-18T11:31:09.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.253e-05 2024-09-18T11:31:09.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966160+0000", 2024-09-18T11:31:09.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7130000000000002e-06 2024-09-18T11:31:09.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966196+0000", 2024-09-18T11:31:09.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6276999999999998e-05 2024-09-18T11:31:09.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.966217+0000", 2024-09-18T11:31:09.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0335000000000001e-05 2024-09-18T11:31:09.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.971872+0000", 2024-09-18T11:31:09.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056549290000000004 2024-09-18T11:31:09.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.971906+0000", 2024-09-18T11:31:09.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4702000000000001e-05 2024-09-18T11:31:09.820 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.820 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.820 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.820 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:29 3.2 3:5fa8fcfa:::ec59520759824f0488c7a3ca135dacb324de2330e16b7f54053de5719c4e96b6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:31:09.820 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:31:09.820 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 49.085894484999997, 2024-09-18T11:31:09.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064126499999999998, 2024-09-18T11:31:09.820 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.820 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.820 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:31:09.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:31:09.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680181+0000", 2024-09-18T11:31:09.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:09.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680654+0000", 2024-09-18T11:31:09.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047334900000000002 2024-09-18T11:31:09.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680666+0000", 2024-09-18T11:31:09.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2232000000000001e-05 2024-09-18T11:31:09.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680672+0000", 2024-09-18T11:31:09.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:31:09.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681222+0000", 2024-09-18T11:31:09.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00054928699999999997 2024-09-18T11:31:09.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681332+0000", 2024-09-18T11:31:09.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110634 2024-09-18T11:31:09.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:09.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681552+0000", 2024-09-18T11:31:09.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021926899999999999 2024-09-18T11:31:09.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:09.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682950+0000", 2024-09-18T11:31:09.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013982669999999999 2024-09-18T11:31:09.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:09.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686503+0000", 2024-09-18T11:31:09.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035536130000000002 2024-09-18T11:31:09.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686553+0000", 2024-09-18T11:31:09.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9801e-05 2024-09-18T11:31:09.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686594+0000", 2024-09-18T11:31:09.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1022000000000001e-05 2024-09-18T11:31:09.825 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.825 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.825 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.826 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1199 4.0 4:0aec5e1c:::smithi11939506-50:head [write 763051~794393] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-18T11:31:09.826 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:31:09.826 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.569080360999997, 2024-09-18T11:31:09.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067768530000000002, 2024-09-18T11:31:09.826 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.826 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:09.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:09.826 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:09.826 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1199 2024-09-18T11:31:09.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.826 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:31:09.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:31:09.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196994+0000", 2024-09-18T11:31:09.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:09.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197355+0000", 2024-09-18T11:31:09.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036035199999999998 2024-09-18T11:31:09.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197366+0000", 2024-09-18T11:31:09.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.17e-05 2024-09-18T11:31:09.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197371+0000", 2024-09-18T11:31:09.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6099999999999999e-06 2024-09-18T11:31:09.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197416+0000", 2024-09-18T11:31:09.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5099000000000003e-05 2024-09-18T11:31:09.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:09.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197420+0000", 2024-09-18T11:31:09.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3189999999999996e-06 2024-09-18T11:31:09.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199392+0000", 2024-09-18T11:31:09.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019720139999999998 2024-09-18T11:31:09.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199452+0000", 2024-09-18T11:31:09.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9929999999999997e-05 2024-09-18T11:31:09.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:09.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199565+0000", 2024-09-18T11:31:09.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112811 2024-09-18T11:31:09.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:09.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.202108+0000", 2024-09-18T11:31:09.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025429060000000002 2024-09-18T11:31:09.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:09.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203714+0000", 2024-09-18T11:31:09.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001606335 2024-09-18T11:31:09.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203745+0000", 2024-09-18T11:31:09.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0837999999999998e-05 2024-09-18T11:31:09.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203773+0000", 2024-09-18T11:31:09.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7562000000000001e-05 2024-09-18T11:31:09.832 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.832 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.832 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.832 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1661 4.0 4:019b6e09:::smithi11939506-40:17 [read 0~3445402,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:09.832 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:09.832 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 32.416179698000001, 2024-09-18T11:31:09.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075190880000000002, 2024-09-18T11:31:09.832 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.832 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:09.832 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:09.833 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:09.833 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:09.833 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1661 2024-09-18T11:31:09.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.833 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:09.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:09.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:09.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:09.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349899+0000", 2024-09-18T11:31:09.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9749999999999999e-06 2024-09-18T11:31:09.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349906+0000", 2024-09-18T11:31:09.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9290000000000001e-06 2024-09-18T11:31:09.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349912+0000", 2024-09-18T11:31:09.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9900000000000002e-06 2024-09-18T11:31:09.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355645+0000", 2024-09-18T11:31:09.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057329549999999996 2024-09-18T11:31:09.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355776+0000", 2024-09-18T11:31:09.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130975 2024-09-18T11:31:09.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357416+0000", 2024-09-18T11:31:09.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001639649 2024-09-18T11:31:09.836 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.836 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.836 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.836 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1761 4.0 4:0aec5e1c:::smithi11939506-50:head [write 769787~704463,stat] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:09.836 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:09.836 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.463687283999999, 2024-09-18T11:31:09.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074082879999999999, 2024-09-18T11:31:09.836 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.836 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.837 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:09.837 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:09.837 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:09.837 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1761 2024-09-18T11:31:09.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.837 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:09.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:09.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302386+0000", 2024-09-18T11:31:09.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:09.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302889+0000", 2024-09-18T11:31:09.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050318499999999996 2024-09-18T11:31:09.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302930+0000", 2024-09-18T11:31:09.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0198999999999999e-05 2024-09-18T11:31:09.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302965+0000", 2024-09-18T11:31:09.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5292e-05 2024-09-18T11:31:09.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303046+0000", 2024-09-18T11:31:09.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1254999999999994e-05 2024-09-18T11:31:09.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:09.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303054+0000", 2024-09-18T11:31:09.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2800000000000003e-06 2024-09-18T11:31:09.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304658+0000", 2024-09-18T11:31:09.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016039520000000001 2024-09-18T11:31:09.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304724+0000", 2024-09-18T11:31:09.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5888000000000003e-05 2024-09-18T11:31:09.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:09.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304853+0000", 2024-09-18T11:31:09.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012912700000000001 2024-09-18T11:31:09.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:09.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309084+0000", 2024-09-18T11:31:09.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042311229999999998 2024-09-18T11:31:09.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:09.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309738+0000", 2024-09-18T11:31:09.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00065401399999999996 2024-09-18T11:31:09.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309770+0000", 2024-09-18T11:31:09.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1652000000000001e-05 2024-09-18T11:31:09.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309797+0000", 2024-09-18T11:31:09.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7052000000000001e-05 2024-09-18T11:31:09.843 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.843 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.843 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.843 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1951 4.0 4:019b6e09:::smithi11939506-40:13 [sparse-read 0~2626478] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:09.843 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.339659+0000", 2024-09-18T11:31:09.843 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.426417577999999, 2024-09-18T11:31:09.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067895999999999998, 2024-09-18T11:31:09.843 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.843 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:09.843 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:09.843 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:09.843 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:09.844 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1951 2024-09-18T11:31:09.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.844 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339659+0000", 2024-09-18T11:31:09.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339659+0000", 2024-09-18T11:31:09.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339658+0000", 2024-09-18T11:31:09.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:09.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339661+0000", 2024-09-18T11:31:09.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3749999999999999e-06 2024-09-18T11:31:09.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339668+0000", 2024-09-18T11:31:09.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1380000000000001e-06 2024-09-18T11:31:09.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339675+0000", 2024-09-18T11:31:09.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7499999999999997e-06 2024-09-18T11:31:09.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339748+0000", 2024-09-18T11:31:09.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2885000000000004e-05 2024-09-18T11:31:09.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:09.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339751+0000", 2024-09-18T11:31:09.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.446e-06 2024-09-18T11:31:09.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345249+0000", 2024-09-18T11:31:09.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054983360000000004 2024-09-18T11:31:09.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345332+0000", 2024-09-18T11:31:09.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3245999999999995e-05 2024-09-18T11:31:09.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346448+0000", 2024-09-18T11:31:09.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0011160530000000001 2024-09-18T11:31:09.848 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.848 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.848 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.848 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:306 4.f e69/22 4:f5d7df41:::smithi11939506-16:17 v 69'94, mlcod=65'88)", 2024-09-18T11:31:09.848 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:09.848 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.414960567000001, 2024-09-18T11:31:09.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068024720000000004, 2024-09-18T11:31:09.849 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.849 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.849 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:09.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:09.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351115+0000", 2024-09-18T11:31:09.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:09.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351121+0000", 2024-09-18T11:31:09.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-09-18T11:31:09.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351126+0000", 2024-09-18T11:31:09.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5969999999999999e-06 2024-09-18T11:31:09.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351147+0000", 2024-09-18T11:31:09.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1160000000000001e-05 2024-09-18T11:31:09.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351518+0000", 2024-09-18T11:31:09.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00037079300000000001 2024-09-18T11:31:09.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351536+0000", 2024-09-18T11:31:09.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8168999999999998e-05 2024-09-18T11:31:09.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357870+0000", 2024-09-18T11:31:09.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063332650000000002 2024-09-18T11:31:09.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357918+0000", 2024-09-18T11:31:09.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8347999999999999e-05 2024-09-18T11:31:09.852 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.852 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.852 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.852 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2022 4.0 4:0aec5e1c:::smithi11939506-50:head [write 457190~552096,stat] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-18T11:31:09.853 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:09.853 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 16.769922430000001, 2024-09-18T11:31:09.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006803974, 2024-09-18T11:31:09.853 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.853 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.853 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:09.853 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:09.853 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:09.853 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2022 2024-09-18T11:31:09.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.853 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:09.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:09.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996151+0000", 2024-09-18T11:31:09.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:09.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996461+0000", 2024-09-18T11:31:09.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000309587 2024-09-18T11:31:09.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996481+0000", 2024-09-18T11:31:09.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9949000000000002e-05 2024-09-18T11:31:09.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996489+0000", 2024-09-18T11:31:09.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9129999999999992e-06 2024-09-18T11:31:09.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996562+0000", 2024-09-18T11:31:09.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2807999999999998e-05 2024-09-18T11:31:09.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996687+0000", 2024-09-18T11:31:09.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012458200000000001 2024-09-18T11:31:09.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:09.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996909+0000", 2024-09-18T11:31:09.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222124 2024-09-18T11:31:09.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:09.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.000741+0000", 2024-09-18T11:31:09.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038320720000000002 2024-09-18T11:31:09.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:09.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002854+0000", 2024-09-18T11:31:09.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002113228 2024-09-18T11:31:09.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002911+0000", 2024-09-18T11:31:09.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7192999999999999e-05 2024-09-18T11:31:09.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002958+0000", 2024-09-18T11:31:09.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6242000000000002e-05 2024-09-18T11:31:09.858 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.858 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.858 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.858 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2075 3.2 3:5e6b2dd0:::abf5d1c1c6ff48a895de7f094078ad489e0511876df30b6bd8853cd5ff4a6267:head [write 0~2096640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-18T11:31:09.858 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:55.321813+0000", 2024-09-18T11:31:09.858 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.444262757000001, 2024-09-18T11:31:09.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0061775409999999999, 2024-09-18T11:31:09.859 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.859 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.859 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:09.859 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:09.859 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:09.859 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2075 2024-09-18T11:31:09.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.859 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.321813+0000", 2024-09-18T11:31:09.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.321813+0000", 2024-09-18T11:31:09.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.321811+0000", 2024-09-18T11:31:09.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:09.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.322777+0000", 2024-09-18T11:31:09.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096617299999999997 2024-09-18T11:31:09.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.322796+0000", 2024-09-18T11:31:09.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.861e-05 2024-09-18T11:31:09.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.322803+0000", 2024-09-18T11:31:09.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3669999999999999e-06 2024-09-18T11:31:09.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.322847+0000", 2024-09-18T11:31:09.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3355000000000001e-05 2024-09-18T11:31:09.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.322948+0000", 2024-09-18T11:31:09.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101563 2024-09-18T11:31:09.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:09.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.323049+0000", 2024-09-18T11:31:09.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100426 2024-09-18T11:31:09.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:09.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.327713+0000", 2024-09-18T11:31:09.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046642280000000003 2024-09-18T11:31:09.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:09.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.327905+0000", 2024-09-18T11:31:09.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000192098 2024-09-18T11:31:09.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.327953+0000", 2024-09-18T11:31:09.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7907000000000001e-05 2024-09-18T11:31:09.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:55.327991+0000", 2024-09-18T11:31:09.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7954999999999998e-05 2024-09-18T11:31:09.864 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.864 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.864 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.864 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2342 4.f e78/22 4:f3c51b3f:::smithi11939506-21:head v 78'127, mlcod=77'123)", 2024-09-18T11:31:09.865 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.390871+0000", 2024-09-18T11:31:09.865 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3752051089999995, 2024-09-18T11:31:09.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065260930000000002, 2024-09-18T11:31:09.865 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.865 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.865 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390871+0000", 2024-09-18T11:31:09.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390871+0000", 2024-09-18T11:31:09.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390871+0000", 2024-09-18T11:31:09.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:09.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390875+0000", 2024-09-18T11:31:09.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2390000000000004e-06 2024-09-18T11:31:09.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390887+0000", 2024-09-18T11:31:09.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2055999999999999e-05 2024-09-18T11:31:09.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390892+0000", 2024-09-18T11:31:09.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4179999999999996e-06 2024-09-18T11:31:09.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390937+0000", 2024-09-18T11:31:09.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4415000000000001e-05 2024-09-18T11:31:09.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390954+0000", 2024-09-18T11:31:09.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7062999999999999e-05 2024-09-18T11:31:09.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397345+0000", 2024-09-18T11:31:09.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063909609999999997 2024-09-18T11:31:09.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397397+0000", 2024-09-18T11:31:09.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2237999999999997e-05 2024-09-18T11:31:09.869 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.869 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.869 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.869 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2503 3.1 e85/67 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head v 85'117, mlcod=84'115)", 2024-09-18T11:31:09.870 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:31:09.870 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.78885371400000004, 2024-09-18T11:31:09.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086527619999999996, 2024-09-18T11:31:09.870 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:09.870 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:09.870 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:09.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:09.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:31:09.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:09.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:31:09.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:09.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:09.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977220+0000", 2024-09-18T11:31:09.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:09.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:09.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980771+0000", 2024-09-18T11:31:09.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035505200000000002 2024-09-18T11:31:09.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:09.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980797+0000", 2024-09-18T11:31:09.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.618e-05 2024-09-18T11:31:09.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:09.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980807+0000", 2024-09-18T11:31:09.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0433e-05 2024-09-18T11:31:09.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:09.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980861+0000", 2024-09-18T11:31:09.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3415999999999998e-05 2024-09-18T11:31:09.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:09.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980885+0000", 2024-09-18T11:31:09.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4212000000000001e-05 2024-09-18T11:31:09.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:09.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985826+0000", 2024-09-18T11:31:09.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049412450000000004 2024-09-18T11:31:09.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:09.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:09.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:09.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985875+0000", 2024-09-18T11:31:09.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9116000000000002e-05 2024-09-18T11:31:09.873 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.873 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.873 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.873 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:09.873 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:09.873 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:09.875 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:31:09.875 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:09.874+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:09.954 INFO:tasks.rados.rados.0.smithi119.stdout:1214: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:31:09.975 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:09.974+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:09.985 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:09.985 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:09.985 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:09.985 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:09.994 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:31:10.077 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:10.076+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:10.134 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:10.134 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:10.134 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:10.134 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:10.134 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:10.143 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:31:10.166 INFO:tasks.rados.rados.0.smithi119.stdout:1219: seq_num 222 ranges {531154=479515,1557826=443729,2713669=487328,3679295=51859} 2024-09-18T11:31:10.172 INFO:tasks.rados.rados.0.smithi119.stdout:1219: writing smithi11939506-21 from 531154 to 1010669 tid 1 2024-09-18T11:31:10.177 INFO:tasks.rados.rados.0.smithi119.stdout:1219: writing smithi11939506-21 from 1557826 to 2001555 tid 2 2024-09-18T11:31:10.184 INFO:tasks.rados.rados.0.smithi119.stdout:1219: writing smithi11939506-21 from 2713669 to 3200997 tid 3 2024-09-18T11:31:10.186 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:10.185+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:10.188 INFO:tasks.rados.rados.0.smithi119.stdout:1219: writing smithi11939506-21 from 3679295 to 3731154 tid 4 2024-09-18T11:31:10.188 INFO:tasks.rados.rados.0.smithi119.stdout:1211: done (6 left) 2024-09-18T11:31:10.188 INFO:tasks.rados.rados.0.smithi119.stdout:1212: done (5 left) 2024-09-18T11:31:10.188 INFO:tasks.rados.rados.0.smithi119.stdout:1214: done (4 left) 2024-09-18T11:31:10.188 INFO:tasks.rados.rados.0.smithi119.stdout:1215: done (3 left) 2024-09-18T11:31:10.188 INFO:tasks.rados.rados.0.smithi119.stdout:1216: done (2 left) 2024-09-18T11:31:10.188 INFO:tasks.rados.rados.0.smithi119.stdout:1217: done (1 left) 2024-09-18T11:31:10.188 INFO:tasks.rados.rados.0.smithi119.stdout:1220: read oid 15 snap -1 2024-09-18T11:31:10.188 INFO:tasks.rados.rados.0.smithi119.stdout:1220: expect (ObjNum 201 snap 23 seq_num 201) 2024-09-18T11:31:10.189 INFO:tasks.rados.rados.0.smithi119.stdout:1221: read oid 19 snap -1 2024-09-18T11:31:10.189 INFO:tasks.rados.rados.0.smithi119.stdout:1221: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-18T11:31:10.189 INFO:tasks.rados.rados.0.smithi119.stdout:1222: delete oid 25 current snap is 26 2024-09-18T11:31:10.189 INFO:tasks.rados.rados.0.smithi119.stdout:1219: finishing write tid 1 to smithi11939506-21 2024-09-18T11:31:10.189 INFO:tasks.rados.rados.0.smithi119.stdout:1219: finishing write tid 2 to smithi11939506-21 2024-09-18T11:31:10.189 INFO:tasks.rados.rados.0.smithi119.stdout:1219: finishing write tid 3 to smithi11939506-21 2024-09-18T11:31:10.191 INFO:tasks.rados.rados.0.smithi119.stdout:1219: finishing write tid 4 to smithi11939506-21 2024-09-18T11:31:10.191 INFO:tasks.rados.rados.0.smithi119.stdout:1219: finishing write tid 5 to smithi11939506-21 2024-09-18T11:31:10.191 INFO:tasks.rados.rados.0.smithi119.stdout:1222: done (3 left) 2024-09-18T11:31:10.191 INFO:tasks.rados.rados.0.smithi119.stdout:1223: tier_evict oid 3 2024-09-18T11:31:10.191 INFO:tasks.rados.rados.0.smithi119.stdout:1223: tier_evict oid 3 snap -1 2024-09-18T11:31:10.191 INFO:tasks.rados.rados.0.smithi119.stdout:1224: read oid 23 snap 17 2024-09-18T11:31:10.191 INFO:tasks.rados.rados.0.smithi119.stdout:1224: expect (ObjNum 90 snap 5 seq_num 90) 2024-09-18T11:31:10.191 INFO:tasks.rados.rados.0.smithi119.stdout:1225: write (excl) oid 44 current snap is 26 2024-09-18T11:31:10.191 INFO:tasks.rados.rados.0.smithi119.stdout:1225: seq_num 224 ranges {564768=630013,1674790=447398,2768515=626225} 2024-09-18T11:31:10.197 INFO:tasks.rados.rados.0.smithi119.stdout:1225: writing smithi11939506-44 from 564768 to 1194781 tid 1 2024-09-18T11:31:10.202 INFO:tasks.rados.rados.0.smithi119.stdout:1225: writing smithi11939506-44 from 1674790 to 2122188 tid 2 2024-09-18T11:31:10.209 INFO:tasks.rados.rados.0.smithi119.stdout:1225: writing smithi11939506-44 from 2768515 to 3394740 tid 3 2024-09-18T11:31:10.209 INFO:tasks.rados.rados.0.smithi119.stdout:1226: set_chunk oid 47 target oid 2024-09-18T11:31:10.213 INFO:tasks.rados.rados.0.smithi119.stdout:1226: set_chunk oid 47 offset: 640512 length: 488960 target oid 1d863b935b7eae129a692d71317d51fff17fe2746e871e9d72143bee1c1c148a offset: 0 snap 22 2024-09-18T11:31:10.213 INFO:tasks.rados.rados.0.smithi119.stdout:1227: set_chunk oid 34 target oid 2024-09-18T11:31:10.213 INFO:tasks.rados.rados.0.smithi119.stdout:1227: set_chunk oid 34 offset: 0 length: 4000000 target oid smithi11939506- offset: 0 snap -1 2024-09-18T11:31:10.213 INFO:tasks.rados.rados.0.smithi119.stdout:1228: delete oid 27 current snap is 26 2024-09-18T11:31:10.214 INFO:tasks.rados.rados.0.smithi119.stdout:1221: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-18T11:31:10.247 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:10.247 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:10.247 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:10.247 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:10.247 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.247 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1530 4.d 4:b81644f2:::smithi11939506-34:head [write 467261~717275] snapc 14=[14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:31:10.247 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:31:10.247 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.853895825999999, 2024-09-18T11:31:10.247 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075660950000000001, 2024-09-18T11:31:10.247 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.247 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.248 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.248 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.248 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.248 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1530 2024-09-18T11:31:10.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.248 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.248 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.248 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.248 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:31:10.248 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.248 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.248 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:31:10.249 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.249 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.249 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390682+0000", 2024-09-18T11:31:10.249 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:10.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.249 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.249 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391554+0000", 2024-09-18T11:31:10.249 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00087220600000000004 2024-09-18T11:31:10.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.250 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.250 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391570+0000", 2024-09-18T11:31:10.250 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6002999999999999e-05 2024-09-18T11:31:10.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.250 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.250 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391575+0000", 2024-09-18T11:31:10.250 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2549999999999997e-06 2024-09-18T11:31:10.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.250 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.250 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391631+0000", 2024-09-18T11:31:10.251 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5671000000000003e-05 2024-09-18T11:31:10.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.251 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.251 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.251 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391751+0000", 2024-09-18T11:31:10.251 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000119821 2024-09-18T11:31:10.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.251 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.251 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:10.251 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391974+0000", 2024-09-18T11:31:10.251 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022309900000000001 2024-09-18T11:31:10.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.251 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.252 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:10.252 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398136+0000", 2024-09-18T11:31:10.252 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006161573 2024-09-18T11:31:10.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.252 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.252 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:10.252 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398184+0000", 2024-09-18T11:31:10.252 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8458000000000003e-05 2024-09-18T11:31:10.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.252 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.252 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.252 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398219+0000", 2024-09-18T11:31:10.252 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4573999999999998e-05 2024-09-18T11:31:10.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.253 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.253 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.253 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398251+0000", 2024-09-18T11:31:10.253 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2085000000000002e-05 2024-09-18T11:31:10.253 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.253 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.253 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.253 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.253 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:31:10.253 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:10.253 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.828493502000001, 2024-09-18T11:31:10.253 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:31:10.254 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.254 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.254 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.254 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.254 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.254 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:31:10.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.254 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.254 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.254 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:10.254 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.254 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.254 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:10.255 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.255 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.255 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:31:10.255 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:10.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.255 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.255 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:31:10.255 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:31:10.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.256 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.256 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.256 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:31:10.256 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:31:10.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.256 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.256 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.256 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:31:10.256 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:31:10.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.256 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.256 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.256 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:31:10.256 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:31:10.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.257 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.257 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.257 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:31:10.257 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:31:10.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.257 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.257 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:10.257 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:31:10.257 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:31:10.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.257 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.257 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:10.257 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:31:10.257 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:31:10.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.258 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.258 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:10.258 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:31:10.258 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:31:10.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.258 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.258 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.258 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:31:10.258 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:31:10.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.258 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.258 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.259 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:31:10.259 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:31:10.259 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.259 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.259 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.259 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.259 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1628 3.3 3:f80aa31b:::77cdbf3abbada57d0636b2198046a030673818705cbf13bb77b30c3e68b24345:head [write 0~582144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:10.259 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:10.259 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.933603501999997, 2024-09-18T11:31:10.259 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081714639999999998, 2024-09-18T11:31:10.259 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.259 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.259 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.260 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.260 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.260 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1628 2024-09-18T11:31:10.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.260 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.260 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.260 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.260 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:10.260 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.260 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.260 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.260 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:10.261 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.261 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.261 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.261 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310975+0000", 2024-09-18T11:31:10.261 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:10.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.261 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.261 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.261 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312086+0000", 2024-09-18T11:31:10.261 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001111288 2024-09-18T11:31:10.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.261 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.261 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312104+0000", 2024-09-18T11:31:10.262 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7427000000000001e-05 2024-09-18T11:31:10.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.262 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.262 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312110+0000", 2024-09-18T11:31:10.262 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5320000000000003e-06 2024-09-18T11:31:10.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.262 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.262 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312179+0000", 2024-09-18T11:31:10.262 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9196000000000006e-05 2024-09-18T11:31:10.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.263 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.263 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.263 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312375+0000", 2024-09-18T11:31:10.263 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00019550899999999999 2024-09-18T11:31:10.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.263 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.263 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:10.263 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312558+0000", 2024-09-18T11:31:10.263 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000182566 2024-09-18T11:31:10.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.263 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.263 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:10.263 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.314286+0000", 2024-09-18T11:31:10.264 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00172831 2024-09-18T11:31:10.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.264 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.264 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:10.264 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319079+0000", 2024-09-18T11:31:10.264 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047929340000000004 2024-09-18T11:31:10.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.264 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.264 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.264 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319116+0000", 2024-09-18T11:31:10.264 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6993999999999999e-05 2024-09-18T11:31:10.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.264 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.265 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.265 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319148+0000", 2024-09-18T11:31:10.265 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2539999999999997e-05 2024-09-18T11:31:10.265 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.265 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.265 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.265 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.265 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1662 4.1 4:87078dfe:::smithi11939506-14:head [write 635331~786118,stat] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:10.265 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.360496+0000", 2024-09-18T11:31:10.265 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.884084756999997, 2024-09-18T11:31:10.265 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071849519999999997, 2024-09-18T11:31:10.265 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.266 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.266 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.266 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.266 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.266 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1662 2024-09-18T11:31:10.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.266 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.266 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.266 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.266 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.360496+0000", 2024-09-18T11:31:10.266 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.266 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.266 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.360496+0000", 2024-09-18T11:31:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.360493+0000", 2024-09-18T11:31:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361139+0000", 2024-09-18T11:31:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064615400000000002 2024-09-18T11:31:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.267 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.268 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.268 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361154+0000", 2024-09-18T11:31:10.268 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4970000000000001e-05 2024-09-18T11:31:10.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.268 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.268 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.268 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361161+0000", 2024-09-18T11:31:10.268 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4139999999999999e-06 2024-09-18T11:31:10.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.268 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.268 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.268 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361201+0000", 2024-09-18T11:31:10.268 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0385e-05 2024-09-18T11:31:10.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.269 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.269 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.269 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361273+0000", 2024-09-18T11:31:10.269 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2224999999999994e-05 2024-09-18T11:31:10.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.269 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.269 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:10.269 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361406+0000", 2024-09-18T11:31:10.269 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013257699999999999 2024-09-18T11:31:10.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.269 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.269 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:10.269 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367415+0000", 2024-09-18T11:31:10.270 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006009655 2024-09-18T11:31:10.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.270 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.270 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:10.270 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367586+0000", 2024-09-18T11:31:10.270 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017109800000000001 2024-09-18T11:31:10.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.270 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.270 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.270 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367636+0000", 2024-09-18T11:31:10.270 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9721e-05 2024-09-18T11:31:10.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.270 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.270 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.271 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367681+0000", 2024-09-18T11:31:10.271 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4382999999999998e-05 2024-09-18T11:31:10.271 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.271 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.271 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.271 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.271 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1663 4.1 4:87078dfe:::smithi11939506-14:head [write 1916458~318873] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:10.271 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:10.271 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.880656938999998, 2024-09-18T11:31:10.271 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079505749999999997, 2024-09-18T11:31:10.271 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.271 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.272 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.272 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.272 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.272 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1663 2024-09-18T11:31:10.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.272 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.272 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.272 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.272 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:10.272 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.272 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.272 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.273 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:10.273 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.273 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.273 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.273 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363922+0000", 2024-09-18T11:31:10.273 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:10.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.273 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.273 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.273 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364211+0000", 2024-09-18T11:31:10.273 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000288385 2024-09-18T11:31:10.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.274 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.274 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364223+0000", 2024-09-18T11:31:10.274 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2582000000000001e-05 2024-09-18T11:31:10.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.274 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.274 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364228+0000", 2024-09-18T11:31:10.274 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0000000000000004e-06 2024-09-18T11:31:10.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.274 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.274 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364268+0000", 2024-09-18T11:31:10.275 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0235999999999999e-05 2024-09-18T11:31:10.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.275 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.275 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.275 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364341+0000", 2024-09-18T11:31:10.275 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2266000000000002e-05 2024-09-18T11:31:10.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.275 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.275 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:10.275 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364444+0000", 2024-09-18T11:31:10.275 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103632 2024-09-18T11:31:10.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.275 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.276 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:10.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369461+0000", 2024-09-18T11:31:10.276 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005016848 2024-09-18T11:31:10.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.276 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.276 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:10.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371820+0000", 2024-09-18T11:31:10.276 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023586169999999999 2024-09-18T11:31:10.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.276 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.276 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371851+0000", 2024-09-18T11:31:10.277 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1297999999999997e-05 2024-09-18T11:31:10.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.277 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371874+0000", 2024-09-18T11:31:10.277 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2841e-05 2024-09-18T11:31:10.277 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.277 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.277 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.277 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1664 4.1 4:87078dfe:::smithi11939506-14:head [setxattr _header (57) in=7b,truncate 2235331] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:10.278 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:10.278 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.880343281999998, 2024-09-18T11:31:10.278 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010368235999999999, 2024-09-18T11:31:10.278 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.278 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.278 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.278 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.278 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.278 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1664 2024-09-18T11:31:10.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.278 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.278 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.278 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.278 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:10.279 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.279 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.279 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.279 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:10.279 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.279 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.279 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.279 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364236+0000", 2024-09-18T11:31:10.279 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:10.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.280 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364240+0000", 2024-09-18T11:31:10.280 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9829999999999998e-06 2024-09-18T11:31:10.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.280 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364247+0000", 2024-09-18T11:31:10.280 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2140000000000001e-06 2024-09-18T11:31:10.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.280 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364256+0000", 2024-09-18T11:31:10.280 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1509999999999993e-06 2024-09-18T11:31:10.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.281 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.281 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366529+0000", 2024-09-18T11:31:10.281 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022731409999999998 2024-09-18T11:31:10.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.281 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.281 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366621+0000", 2024-09-18T11:31:10.281 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2408e-05 2024-09-18T11:31:10.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.281 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:10.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366801+0000", 2024-09-18T11:31:10.282 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018002200000000001 2024-09-18T11:31:10.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.282 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.282 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:10.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372155+0000", 2024-09-18T11:31:10.282 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053531519999999999 2024-09-18T11:31:10.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.282 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.282 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:10.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374539+0000", 2024-09-18T11:31:10.282 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023842580000000002 2024-09-18T11:31:10.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.283 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374573+0000", 2024-09-18T11:31:10.283 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4205999999999997e-05 2024-09-18T11:31:10.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.283 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374605+0000", 2024-09-18T11:31:10.283 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2283999999999998e-05 2024-09-18T11:31:10.283 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.283 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.283 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.284 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1665 4.1 4:87078dfe:::smithi11939506-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:10.284 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:10.284 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.880319108000002, 2024-09-18T11:31:10.284 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010471897000000001, 2024-09-18T11:31:10.284 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.284 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:10.284 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.284 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.284 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.284 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1665 2024-09-18T11:31:10.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.284 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.284 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.284 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.284 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:10.285 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.285 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.285 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:10.285 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.285 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.285 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:10.285 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:10.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.285 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364263+0000", 2024-09-18T11:31:10.286 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6400000000000001e-06 2024-09-18T11:31:10.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.286 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.286 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364270+0000", 2024-09-18T11:31:10.286 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:31:10.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.286 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.286 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364274+0000", 2024-09-18T11:31:10.286 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.968e-06 2024-09-18T11:31:10.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.287 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.287 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367354+0000", 2024-09-18T11:31:10.287 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030803430000000001 2024-09-18T11:31:10.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.287 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:10.287 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367406+0000", 2024-09-18T11:31:10.287 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2330999999999998e-05 2024-09-18T11:31:10.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.287 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.287 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374625+0000", 2024-09-18T11:31:10.287 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072190689999999998 2024-09-18T11:31:10.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.288 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.288 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.288 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374673+0000", 2024-09-18T11:31:10.288 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7525999999999999e-05 2024-09-18T11:31:10.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.288 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.288 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.288 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374733+0000", 2024-09-18T11:31:10.288 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0359000000000003e-05 2024-09-18T11:31:10.288 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.288 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.288 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.289 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.289 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:101 4.5 e58/22 4:aa575018:::smithi11939506-11:17 v 58'67, mlcod=55'62)", 2024-09-18T11:31:10.289 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.273235+0000", 2024-09-18T11:31:10.289 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 30.971345817, 2024-09-18T11:31:10.289 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0067904050000000002, 2024-09-18T11:31:10.289 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.289 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.289 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.289 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.289 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.289 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273235+0000", 2024-09-18T11:31:10.289 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.290 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.290 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.290 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273235+0000", 2024-09-18T11:31:10.290 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.290 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.290 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.290 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273234+0000", 2024-09-18T11:31:10.290 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:10.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.290 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.290 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.290 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273239+0000", 2024-09-18T11:31:10.290 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7999999999999998e-06 2024-09-18T11:31:10.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.291 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.291 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273247+0000", 2024-09-18T11:31:10.291 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4700000000000002e-06 2024-09-18T11:31:10.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.291 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.291 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273251+0000", 2024-09-18T11:31:10.291 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7500000000000001e-06 2024-09-18T11:31:10.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.291 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.291 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.274599+0000", 2024-09-18T11:31:10.291 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013483900000000001 2024-09-18T11:31:10.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.292 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.292 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.274623+0000", 2024-09-18T11:31:10.292 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3499999999999999e-05 2024-09-18T11:31:10.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.292 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.292 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.279994+0000", 2024-09-18T11:31:10.292 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053713889999999999 2024-09-18T11:31:10.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.292 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.292 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.293 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.280025+0000", 2024-09-18T11:31:10.293 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0579e-05 2024-09-18T11:31:10.293 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.293 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.293 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.293 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1947 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865577,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:10.293 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:10.293 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.905039398, 2024-09-18T11:31:10.293 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010043066999999999, 2024-09-18T11:31:10.294 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.294 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:10.294 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.294 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.294 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.294 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1947 2024-09-18T11:31:10.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.294 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.294 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:10.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.295 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:10.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.295 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339540+0000", 2024-09-18T11:31:10.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:10.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.295 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.296 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339545+0000", 2024-09-18T11:31:10.296 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6290000000000001e-06 2024-09-18T11:31:10.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.296 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.296 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339553+0000", 2024-09-18T11:31:10.296 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6299999999999998e-06 2024-09-18T11:31:10.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.296 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.296 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339557+0000", 2024-09-18T11:31:10.296 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3590000000000001e-06 2024-09-18T11:31:10.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.297 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345600+0000", 2024-09-18T11:31:10.297 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060434640000000001 2024-09-18T11:31:10.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.297 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345710+0000", 2024-09-18T11:31:10.297 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010924700000000001 2024-09-18T11:31:10.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.297 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.298 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349584+0000", 2024-09-18T11:31:10.298 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038743459999999999 2024-09-18T11:31:10.298 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.298 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.298 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.298 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1948 4.d 4:b5a463bf:::smithi11939506-20:head [sparse-read 0~2642181] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:10.298 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:10.298 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.905018860999999, 2024-09-18T11:31:10.298 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010701952000000001, 2024-09-18T11:31:10.298 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.298 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:10.299 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.299 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.299 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.299 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1948 2024-09-18T11:31:10.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.299 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.299 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:10.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.299 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:10.300 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:10.300 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:10.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339563+0000", 2024-09-18T11:31:10.300 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1950000000000002e-06 2024-09-18T11:31:10.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.301 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339569+0000", 2024-09-18T11:31:10.301 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5459999999999999e-06 2024-09-18T11:31:10.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.301 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339572+0000", 2024-09-18T11:31:10.301 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.337e-06 2024-09-18T11:31:10.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.301 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349693+0000", 2024-09-18T11:31:10.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010120987999999999 2024-09-18T11:31:10.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.302 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.302 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349816+0000", 2024-09-18T11:31:10.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012241199999999999 2024-09-18T11:31:10.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.302 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.302 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350263+0000", 2024-09-18T11:31:10.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00044787199999999998 2024-09-18T11:31:10.303 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.303 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.303 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.303 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:129 4.5 e69/22 4:a087a566:::smithi11939506-46:19 v 69'95, mlcod=65'92)", 2024-09-18T11:31:10.303 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.343498+0000", 2024-09-18T11:31:10.303 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.901082214999999, 2024-09-18T11:31:10.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068162960000000003, 2024-09-18T11:31:10.303 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.303 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.304 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.304 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343498+0000", 2024-09-18T11:31:10.304 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.304 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343498+0000", 2024-09-18T11:31:10.304 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.304 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343497+0000", 2024-09-18T11:31:10.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:10.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343504+0000", 2024-09-18T11:31:10.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5799999999999997e-06 2024-09-18T11:31:10.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343513+0000", 2024-09-18T11:31:10.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7940000000000008e-06 2024-09-18T11:31:10.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343518+0000", 2024-09-18T11:31:10.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2719999999999997e-06 2024-09-18T11:31:10.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345616+0000", 2024-09-18T11:31:10.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020977410000000002 2024-09-18T11:31:10.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345641+0000", 2024-09-18T11:31:10.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5539000000000001e-05 2024-09-18T11:31:10.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350279+0000", 2024-09-18T11:31:10.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046379560000000004 2024-09-18T11:31:10.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350314+0000", 2024-09-18T11:31:10.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5083999999999998e-05 2024-09-18T11:31:10.307 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.307 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.308 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.308 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.308 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:131 4.5 e69/22 4:aa575018:::smithi11939506-11:17 v 69'97, mlcod=65'92)", 2024-09-18T11:31:10.308 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.346049+0000", 2024-09-18T11:31:10.308 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.898531801000001, 2024-09-18T11:31:10.308 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0077275240000000004, 2024-09-18T11:31:10.308 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.308 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.308 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.308 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.308 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346049+0000", 2024-09-18T11:31:10.309 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.309 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346049+0000", 2024-09-18T11:31:10.309 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.309 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346047+0000", 2024-09-18T11:31:10.309 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:10.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346058+0000", 2024-09-18T11:31:10.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0463e-05 2024-09-18T11:31:10.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346069+0000", 2024-09-18T11:31:10.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1708999999999999e-05 2024-09-18T11:31:10.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346076+0000", 2024-09-18T11:31:10.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2450000000000003e-06 2024-09-18T11:31:10.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349616+0000", 2024-09-18T11:31:10.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035406550000000002 2024-09-18T11:31:10.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349638+0000", 2024-09-18T11:31:10.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2098000000000001e-05 2024-09-18T11:31:10.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.353735+0000", 2024-09-18T11:31:10.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004096237 2024-09-18T11:31:10.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.312 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.353776+0000", 2024-09-18T11:31:10.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1530999999999999e-05 2024-09-18T11:31:10.312 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.312 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.312 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.313 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2128 4.1 4:8b9a985d:::smithi11939506-23:16 [read 0~3512630,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T11:31:10.313 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:55.710141+0000", 2024-09-18T11:31:10.313 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.534438887, 2024-09-18T11:31:10.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068000389999999999, 2024-09-18T11:31:10.313 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.313 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:10.313 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.313 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.313 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.313 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2128 2024-09-18T11:31:10.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.313 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.314 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.314 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710141+0000", 2024-09-18T11:31:10.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.314 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.314 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710141+0000", 2024-09-18T11:31:10.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.314 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.314 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710141+0000", 2024-09-18T11:31:10.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:10.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710162+0000", 2024-09-18T11:31:10.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1492e-05 2024-09-18T11:31:10.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710169+0000", 2024-09-18T11:31:10.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0029999999999997e-06 2024-09-18T11:31:10.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710174+0000", 2024-09-18T11:31:10.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1250000000000003e-06 2024-09-18T11:31:10.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.715332+0000", 2024-09-18T11:31:10.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051581680000000003 2024-09-18T11:31:10.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.715431+0000", 2024-09-18T11:31:10.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9693999999999998e-05 2024-09-18T11:31:10.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.716942+0000", 2024-09-18T11:31:10.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001510033 2024-09-18T11:31:10.317 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.317 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.317 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.317 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2129 4.1 4:8b9a985d:::smithi11939506-23:16 [read 0~3512630,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T11:31:10.317 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:55.710202+0000", 2024-09-18T11:31:10.317 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.534378540000001, 2024-09-18T11:31:10.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0078396100000000003, 2024-09-18T11:31:10.317 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.317 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:10.318 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.318 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.318 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.318 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2129 2024-09-18T11:31:10.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.318 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710202+0000", 2024-09-18T11:31:10.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710202+0000", 2024-09-18T11:31:10.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710202+0000", 2024-09-18T11:31:10.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:10.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710205+0000", 2024-09-18T11:31:10.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.382e-06 2024-09-18T11:31:10.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710212+0000", 2024-09-18T11:31:10.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770000000000004e-06 2024-09-18T11:31:10.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710217+0000", 2024-09-18T11:31:10.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1680000000000003e-06 2024-09-18T11:31:10.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.716963+0000", 2024-09-18T11:31:10.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0067462449999999997 2024-09-18T11:31:10.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.717033+0000", 2024-09-18T11:31:10.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9643999999999999e-05 2024-09-18T11:31:10.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.718041+0000", 2024-09-18T11:31:10.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0010086019999999999 2024-09-18T11:31:10.321 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.321 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.321 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.322 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2175 4.e 4:73745217:::smithi11939506-6:head [write 742909~464035,stat] snapc 1e=[1e,1c,1b,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-18T11:31:10.322 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:56.426542+0000", 2024-09-18T11:31:10.322 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.818038499, 2024-09-18T11:31:10.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070437850000000003, 2024-09-18T11:31:10.322 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.322 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.322 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.322 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.322 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.322 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2175 2024-09-18T11:31:10.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.323 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.426542+0000", 2024-09-18T11:31:10.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.426542+0000", 2024-09-18T11:31:10.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.426540+0000", 2024-09-18T11:31:10.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:10.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.426987+0000", 2024-09-18T11:31:10.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00044653899999999997 2024-09-18T11:31:10.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.427005+0000", 2024-09-18T11:31:10.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8669e-05 2024-09-18T11:31:10.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.427017+0000", 2024-09-18T11:31:10.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1666e-05 2024-09-18T11:31:10.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.427065+0000", 2024-09-18T11:31:10.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8071000000000001e-05 2024-09-18T11:31:10.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.427134+0000", 2024-09-18T11:31:10.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8736000000000001e-05 2024-09-18T11:31:10.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 6", 2024-09-18T11:31:10.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.427252+0000", 2024-09-18T11:31:10.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000118402 2024-09-18T11:31:10.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:10.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.428537+0000", 2024-09-18T11:31:10.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001284456 2024-09-18T11:31:10.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:10.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.433512+0000", 2024-09-18T11:31:10.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049752499999999996 2024-09-18T11:31:10.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.433545+0000", 2024-09-18T11:31:10.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2529e-05 2024-09-18T11:31:10.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.328 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.433586+0000", 2024-09-18T11:31:10.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1087999999999997e-05 2024-09-18T11:31:10.328 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.328 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.328 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.328 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:172 4.5 e78/22 4:a087a566:::smithi11939506-46:19 v 78'135, mlcod=77'132)", 2024-09-18T11:31:10.328 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:10.328 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.8587568480000005, 2024-09-18T11:31:10.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010938658, 2024-09-18T11:31:10.329 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.329 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.329 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:10.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:10.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385823+0000", 2024-09-18T11:31:10.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:10.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385830+0000", 2024-09-18T11:31:10.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2790000000000001e-06 2024-09-18T11:31:10.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385843+0000", 2024-09-18T11:31:10.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2785e-05 2024-09-18T11:31:10.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385848+0000", 2024-09-18T11:31:10.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5570000000000003e-06 2024-09-18T11:31:10.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385914+0000", 2024-09-18T11:31:10.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5168999999999999e-05 2024-09-18T11:31:10.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385930+0000", 2024-09-18T11:31:10.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6884999999999998e-05 2024-09-18T11:31:10.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396721+0000", 2024-09-18T11:31:10.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010790137 2024-09-18T11:31:10.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396762+0000", 2024-09-18T11:31:10.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1635999999999999e-05 2024-09-18T11:31:10.333 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.333 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.333 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.333 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:31:10.333 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:10.333 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.8582640959999992, 2024-09-18T11:31:10.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:31:10.333 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.333 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.333 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:10.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:10.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:10.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:10.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:31:10.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:31:10.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:31:10.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:31:10.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:31:10.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:31:10.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:31:10.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:31:10.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:31:10.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:31:10.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:31:10.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:31:10.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:31:10.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:31:10.337 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.337 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.337 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.337 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:10.338 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:10.338 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.2721514899999999, 2024-09-18T11:31:10.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:31:10.338 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.338 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.338 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.338 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.338 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.338 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:31:10.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.338 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:10.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:10.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:31:10.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:10.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:31:10.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:31:10.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:31:10.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:31:10.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:31:10.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:31:10.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:31:10.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:31:10.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:31:10.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:31:10.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:10.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:31:10.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:31:10.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:10.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:31:10.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:31:10.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:10.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:31:10.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:31:10.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:31:10.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:31:10.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:31:10.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:31:10.343 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.343 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.343 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.344 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2505 4.1 4:87078dfe:::smithi11939506-14:head [write 497540~612664,stat] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:10.344 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.992775+0000", 2024-09-18T11:31:10.344 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.251805504, 2024-09-18T11:31:10.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076090139999999999, 2024-09-18T11:31:10.344 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.344 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.344 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.344 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.344 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.344 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2505 2024-09-18T11:31:10.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.345 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.992775+0000", 2024-09-18T11:31:10.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.992775+0000", 2024-09-18T11:31:10.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.992773+0000", 2024-09-18T11:31:10.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:10.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993364+0000", 2024-09-18T11:31:10.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059123300000000004 2024-09-18T11:31:10.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993377+0000", 2024-09-18T11:31:10.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3464e-05 2024-09-18T11:31:10.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993382+0000", 2024-09-18T11:31:10.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2440000000000004e-06 2024-09-18T11:31:10.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993460+0000", 2024-09-18T11:31:10.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8584000000000005e-05 2024-09-18T11:31:10.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993530+0000", 2024-09-18T11:31:10.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0060000000000003e-05 2024-09-18T11:31:10.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:10.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993650+0000", 2024-09-18T11:31:10.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120079 2024-09-18T11:31:10.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:10.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.998802+0000", 2024-09-18T11:31:10.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005152106 2024-09-18T11:31:10.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:10.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.000318+0000", 2024-09-18T11:31:10.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015154719999999999 2024-09-18T11:31:10.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.000352+0000", 2024-09-18T11:31:10.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4418000000000001e-05 2024-09-18T11:31:10.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.000384+0000", 2024-09-18T11:31:10.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1594000000000003e-05 2024-09-18T11:31:10.350 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.350 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.350 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.350 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2525 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865577,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:10.350 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173466+0000", 2024-09-18T11:31:10.350 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.071114798, 2024-09-18T11:31:10.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075147870000000002, 2024-09-18T11:31:10.350 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.350 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:10.350 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.350 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.350 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.350 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2525 2024-09-18T11:31:10.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.351 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173466+0000", 2024-09-18T11:31:10.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173466+0000", 2024-09-18T11:31:10.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173465+0000", 2024-09-18T11:31:10.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:10.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173469+0000", 2024-09-18T11:31:10.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7299999999999999e-06 2024-09-18T11:31:10.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173476+0000", 2024-09-18T11:31:10.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7750000000000004e-06 2024-09-18T11:31:10.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173480+0000", 2024-09-18T11:31:10.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1810000000000003e-06 2024-09-18T11:31:10.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.180013+0000", 2024-09-18T11:31:10.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065332419999999999 2024-09-18T11:31:10.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.180115+0000", 2024-09-18T11:31:10.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102111 2024-09-18T11:31:10.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.180980+0000", 2024-09-18T11:31:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00086527100000000003 2024-09-18T11:31:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.354 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:10.355 INFO:tasks.rados.rados.0.smithi119.stdout:1219: finishing write tid 6 to smithi11939506-21 2024-09-18T11:31:10.355 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 168 (ObjNum 222 snap 26 seq_num 222) dirty exists 2024-09-18T11:31:10.355 INFO:tasks.rados.rados.0.smithi119.stdout:1219: left oid 21 (ObjNum 222 snap 26 seq_num 222) 2024-09-18T11:31:10.355 INFO:tasks.rados.rados.0.smithi119.stdout:1223: got (22) Invalid argument 2024-09-18T11:31:10.355 INFO:tasks.rados.rados.0.smithi119.stdout:1220: expect (ObjNum 201 snap 23 seq_num 201) 2024-09-18T11:31:10.356 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:31:10.357 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:10.354+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:10.430 INFO:tasks.rados.rados.0.smithi119.stdout:1224: expect (ObjNum 90 snap 5 seq_num 90) 2024-09-18T11:31:10.454 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T11:31:10.455 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:10.454+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:10.498 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:10.498 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:10.498 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:10.498 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:10.506 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:31:10.590 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:10.589+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:10.639 INFO:tasks.rados.rados.0.smithi119.stdout:1225: finishing write tid 1 to smithi11939506-44 2024-09-18T11:31:10.640 INFO:tasks.rados.rados.0.smithi119.stdout:1225: finishing write tid 2 to smithi11939506-44 2024-09-18T11:31:10.640 INFO:tasks.rados.rados.0.smithi119.stdout:1219: done (8 left) 2024-09-18T11:31:10.640 INFO:tasks.rados.rados.0.smithi119.stdout:1220: done (7 left) 2024-09-18T11:31:10.640 INFO:tasks.rados.rados.0.smithi119.stdout:1221: done (6 left) 2024-09-18T11:31:10.640 INFO:tasks.rados.rados.0.smithi119.stdout:1223: done (5 left) 2024-09-18T11:31:10.640 INFO:tasks.rados.rados.0.smithi119.stdout:1224: done (4 left) 2024-09-18T11:31:10.640 INFO:tasks.rados.rados.0.smithi119.stdout:1228: done (3 left) 2024-09-18T11:31:10.640 INFO:tasks.rados.rados.0.smithi119.stdout:1229: read oid 13 snap 22 2024-09-18T11:31:10.640 INFO:tasks.rados.rados.0.smithi119.stdout:1229: expect (ObjNum 136 snap 16 seq_num 136) 2024-09-18T11:31:10.640 INFO:tasks.rados.rados.0.smithi119.stdout:1225: finishing write tid 3 to smithi11939506-44 2024-09-18T11:31:10.640 INFO:tasks.rados.rados.0.smithi119.stdout:1225: finishing write tid 4 to smithi11939506-44 2024-09-18T11:31:10.640 INFO:tasks.rados.rados.0.smithi119.stdout:1225: finishing write tid 5 to smithi11939506-44 2024-09-18T11:31:10.640 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 44 v 95 (ObjNum 224 snap 26 seq_num 224) dirty exists 2024-09-18T11:31:10.641 INFO:tasks.rados.rados.0.smithi119.stdout:1225: left oid 44 (ObjNum 224 snap 26 seq_num 224) 2024-09-18T11:31:10.641 INFO:tasks.rados.rados.0.smithi119.stdout:1226: finishing set_chunk to oid 47 2024-09-18T11:31:10.641 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 47 set_chunk returned error code -95 offset: 640512 length: 488960 2024-09-18T11:31:10.641 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 0 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-18T11:31:10.641 INFO:tasks.rados.rados.0.smithi119.stdout:1227: finishing set_chunk to oid 34 2024-09-18T11:31:10.641 INFO:tasks.rados.rados.0.smithi119.stdout:1227: got expected ERANGE (src dne) 2024-09-18T11:31:10.641 INFO:tasks.rados.rados.0.smithi119.stdout:1230: read oid 20 snap -1 2024-09-18T11:31:10.641 INFO:tasks.rados.rados.0.smithi119.stdout:1230: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:31:10.641 INFO:tasks.rados.rados.0.smithi119.stdout:1225: done (4 left) 2024-09-18T11:31:10.641 INFO:tasks.rados.rados.0.smithi119.stdout:1226: done (3 left) 2024-09-18T11:31:10.641 INFO:tasks.rados.rados.0.smithi119.stdout:1227: done (2 left) 2024-09-18T11:31:10.641 INFO:tasks.rados.rados.0.smithi119.stdout:1231: read oid 42 snap -1 2024-09-18T11:31:10.641 INFO:tasks.rados.rados.0.smithi119.stdout:1231: expect (ObjNum 99 snap 6 seq_num 99) 2024-09-18T11:31:10.641 INFO:tasks.rados.rados.0.smithi119.stdout:1232: delete oid 6 current snap is 26 2024-09-18T11:31:10.644 INFO:tasks.rados.rados.0.smithi119.stdout:1229: expect (ObjNum 136 snap 16 seq_num 136) 2024-09-18T11:31:10.646 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:10.646 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:10.646 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:10.646 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:10.647 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:10.657 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:31:10.658 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:10.657+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:10.742 INFO:tasks.rados.rados.0.smithi119.stdout:1230: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:31:10.758 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:10.757+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:10.796 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:31:10.797 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:10.798 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:10.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:10.798 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:10.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.798 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:31:10.798 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:10.798 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 80.768261202999994, 2024-09-18T11:31:10.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073601550000000002, 2024-09-18T11:31:10.798 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.798 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.798 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.799 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:31:10.799 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:31:10.799 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1 2024-09-18T11:31:10.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.799 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:10.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:10.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026524+0000", 2024-09-18T11:31:10.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999905 2024-09-18T11:31:10.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026541+0000", 2024-09-18T11:31:10.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7070000000000001e-05 2024-09-18T11:31:10.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026554+0000", 2024-09-18T11:31:10.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.288e-05 2024-09-18T11:31:10.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026564+0000", 2024-09-18T11:31:10.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0197000000000001e-05 2024-09-18T11:31:10.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026826+0000", 2024-09-18T11:31:10.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026212400000000003 2024-09-18T11:31:10.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:31:10.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026839+0000", 2024-09-18T11:31:10.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2831e-05 2024-09-18T11:31:10.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028605+0000", 2024-09-18T11:31:10.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001765794 2024-09-18T11:31:10.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028709+0000", 2024-09-18T11:31:10.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104348 2024-09-18T11:31:10.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:10.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028824+0000", 2024-09-18T11:31:10.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011527399999999999 2024-09-18T11:31:10.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:10.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030319+0000", 2024-09-18T11:31:10.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001494439 2024-09-18T11:31:10.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.804 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:10.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033805+0000", 2024-09-18T11:31:10.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034857820000000002 2024-09-18T11:31:10.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.804 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033856+0000", 2024-09-18T11:31:10.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1687999999999997e-05 2024-09-18T11:31:10.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033894+0000", 2024-09-18T11:31:10.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7481000000000002e-05 2024-09-18T11:31:10.805 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.805 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.805 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.805 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T11:31:10.805 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:31:10.805 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 78.748982772000005, 2024-09-18T11:31:10.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055107869999999996, 2024-09-18T11:31:10.805 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.805 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.806 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.806 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:31:10.806 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:31:10.806 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 7 2024-09-18T11:31:10.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.806 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:31:10.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:31:10.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045810+0000", 2024-09-18T11:31:10.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:10.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045820+0000", 2024-09-18T11:31:10.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0215e-05 2024-09-18T11:31:10.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045838+0000", 2024-09-18T11:31:10.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7849000000000001e-05 2024-09-18T11:31:10.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045845+0000", 2024-09-18T11:31:10.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4229999999999999e-06 2024-09-18T11:31:10.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045898+0000", 2024-09-18T11:31:10.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2537999999999997e-05 2024-09-18T11:31:10.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:10.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045904+0000", 2024-09-18T11:31:10.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4019999999999999e-06 2024-09-18T11:31:10.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047663+0000", 2024-09-18T11:31:10.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017584079999999999 2024-09-18T11:31:10.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047736+0000", 2024-09-18T11:31:10.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3509000000000004e-05 2024-09-18T11:31:10.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.810 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:10.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047857+0000", 2024-09-18T11:31:10.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012037800000000001 2024-09-18T11:31:10.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.810 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:10.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.050868+0000", 2024-09-18T11:31:10.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030111420000000001 2024-09-18T11:31:10.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:10.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051239+0000", 2024-09-18T11:31:10.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037141000000000002 2024-09-18T11:31:10.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051288+0000", 2024-09-18T11:31:10.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9051000000000002e-05 2024-09-18T11:31:10.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051323+0000", 2024-09-18T11:31:10.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4545000000000002e-05 2024-09-18T11:31:10.812 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.812 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.812 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.812 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1118 3.2 e38/21 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head v 38'76, mlcod=38'74)", 2024-09-18T11:31:10.812 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:10.812 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 50.846851051000002, 2024-09-18T11:31:10.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006048688, 2024-09-18T11:31:10.812 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.812 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.812 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:10.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:10.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947941+0000", 2024-09-18T11:31:10.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:10.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949868+0000", 2024-09-18T11:31:10.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019267659999999999 2024-09-18T11:31:10.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949893+0000", 2024-09-18T11:31:10.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5239999999999999e-05 2024-09-18T11:31:10.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949901+0000", 2024-09-18T11:31:10.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8420000000000005e-06 2024-09-18T11:31:10.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.815 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949954+0000", 2024-09-18T11:31:10.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3464999999999997e-05 2024-09-18T11:31:10.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.815 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949974+0000", 2024-09-18T11:31:10.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9731000000000001e-05 2024-09-18T11:31:10.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953948+0000", 2024-09-18T11:31:10.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039735489999999998 2024-09-18T11:31:10.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953992+0000", 2024-09-18T11:31:10.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4700999999999998e-05 2024-09-18T11:31:10.816 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.816 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.816 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.817 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1662 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'73, mlcod=51'69)", 2024-09-18T11:31:10.817 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:10.817 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 33.433246015999998, 2024-09-18T11:31:10.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059408539999999998, 2024-09-18T11:31:10.817 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.817 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.817 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:10.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:10.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:10.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:10.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361914+0000", 2024-09-18T11:31:10.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036569 2024-09-18T11:31:10.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361926+0000", 2024-09-18T11:31:10.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1960999999999999e-05 2024-09-18T11:31:10.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361937+0000", 2024-09-18T11:31:10.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0876999999999999e-05 2024-09-18T11:31:10.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361986+0000", 2024-09-18T11:31:10.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8652000000000001e-05 2024-09-18T11:31:10.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.362012+0000", 2024-09-18T11:31:10.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5854999999999999e-05 2024-09-18T11:31:10.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367460+0000", 2024-09-18T11:31:10.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054482100000000002 2024-09-18T11:31:10.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367490+0000", 2024-09-18T11:31:10.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9909000000000001e-05 2024-09-18T11:31:10.821 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.821 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.821 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.821 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1663 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'74, mlcod=51'69)", 2024-09-18T11:31:10.822 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364521+0000", 2024-09-18T11:31:10.822 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 33.430274017999999, 2024-09-18T11:31:10.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048541399999999998, 2024-09-18T11:31:10.822 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.822 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.822 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.822 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.822 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364521+0000", 2024-09-18T11:31:10.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.822 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.822 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364521+0000", 2024-09-18T11:31:10.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364520+0000", 2024-09-18T11:31:10.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:10.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364616+0000", 2024-09-18T11:31:10.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5569999999999995e-05 2024-09-18T11:31:10.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364625+0000", 2024-09-18T11:31:10.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3379999999999994e-06 2024-09-18T11:31:10.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364629+0000", 2024-09-18T11:31:10.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2490000000000003e-06 2024-09-18T11:31:10.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364664+0000", 2024-09-18T11:31:10.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5132999999999997e-05 2024-09-18T11:31:10.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364687+0000", 2024-09-18T11:31:10.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2691999999999999e-05 2024-09-18T11:31:10.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369349+0000", 2024-09-18T11:31:10.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046622950000000003 2024-09-18T11:31:10.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369375+0000", 2024-09-18T11:31:10.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5534e-05 2024-09-18T11:31:10.826 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.826 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.826 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.826 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1664 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'75, mlcod=51'69)", 2024-09-18T11:31:10.826 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:31:10.826 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 33.427907509000001, 2024-09-18T11:31:10.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051832229999999998, 2024-09-18T11:31:10.826 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.826 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.826 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:31:10.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:31:10.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:31:10.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:10.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366894+0000", 2024-09-18T11:31:10.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2219999999999996e-06 2024-09-18T11:31:10.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366906+0000", 2024-09-18T11:31:10.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2278e-05 2024-09-18T11:31:10.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366912+0000", 2024-09-18T11:31:10.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2430000000000002e-06 2024-09-18T11:31:10.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366952+0000", 2024-09-18T11:31:10.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9428999999999998e-05 2024-09-18T11:31:10.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366972+0000", 2024-09-18T11:31:10.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0554999999999999e-05 2024-09-18T11:31:10.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372045+0000", 2024-09-18T11:31:10.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050727319999999999 2024-09-18T11:31:10.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372071+0000", 2024-09-18T11:31:10.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5477e-05 2024-09-18T11:31:10.830 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.830 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.830 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.830 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:269 4.7 e58/22 4:e73ba68f:::smithi11939506-1:13 v 58'72, mlcod=57'69)", 2024-09-18T11:31:10.830 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:10.831 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.529414118999998, 2024-09-18T11:31:10.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065512569999999996, 2024-09-18T11:31:10.831 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.831 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.831 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:10.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:10.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265380+0000", 2024-09-18T11:31:10.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:10.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265388+0000", 2024-09-18T11:31:10.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.174e-06 2024-09-18T11:31:10.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265398+0000", 2024-09-18T11:31:10.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2599999999999994e-06 2024-09-18T11:31:10.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265402+0000", 2024-09-18T11:31:10.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0289999999999999e-06 2024-09-18T11:31:10.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265434+0000", 2024-09-18T11:31:10.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2511000000000001e-05 2024-09-18T11:31:10.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265447+0000", 2024-09-18T11:31:10.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2947e-05 2024-09-18T11:31:10.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271892+0000", 2024-09-18T11:31:10.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064445930000000002 2024-09-18T11:31:10.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271932+0000", 2024-09-18T11:31:10.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0303999999999999e-05 2024-09-18T11:31:10.835 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.835 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.835 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.835 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1788 4.8 4:1898b15d:::smithi11939506-25:head [write 453822~797061] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:10.835 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:10.835 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.878981536000001, 2024-09-18T11:31:10.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094305159999999999, 2024-09-18T11:31:10.835 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.835 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.836 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.836 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.836 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.836 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1788 2024-09-18T11:31:10.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.836 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:10.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:10.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915811+0000", 2024-09-18T11:31:10.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:10.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916608+0000", 2024-09-18T11:31:10.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079635000000000003 2024-09-18T11:31:10.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916624+0000", 2024-09-18T11:31:10.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6735000000000002e-05 2024-09-18T11:31:10.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916630+0000", 2024-09-18T11:31:10.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5709999999999997e-06 2024-09-18T11:31:10.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916674+0000", 2024-09-18T11:31:10.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4540999999999999e-05 2024-09-18T11:31:10.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916751+0000", 2024-09-18T11:31:10.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6695999999999999e-05 2024-09-18T11:31:10.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:10.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916872+0000", 2024-09-18T11:31:10.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120648 2024-09-18T11:31:10.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:10.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.919076+0000", 2024-09-18T11:31:10.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022046829999999998 2024-09-18T11:31:10.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:10.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925151+0000", 2024-09-18T11:31:10.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060740239999999999 2024-09-18T11:31:10.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925198+0000", 2024-09-18T11:31:10.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7896999999999999e-05 2024-09-18T11:31:10.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925244+0000", 2024-09-18T11:31:10.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5398999999999997e-05 2024-09-18T11:31:10.841 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.841 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.841 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.841 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1789 4.8 4:1898b15d:::smithi11939506-25:head [write 1760371~653366] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:10.841 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.921860+0000", 2024-09-18T11:31:10.841 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.872934851, 2024-09-18T11:31:10.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055904329999999997, 2024-09-18T11:31:10.842 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.842 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.842 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.842 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.842 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.842 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1789 2024-09-18T11:31:10.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.842 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.921860+0000", 2024-09-18T11:31:10.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.921860+0000", 2024-09-18T11:31:10.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.921859+0000", 2024-09-18T11:31:10.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:10.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922436+0000", 2024-09-18T11:31:10.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00057708300000000004 2024-09-18T11:31:10.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922450+0000", 2024-09-18T11:31:10.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4575e-05 2024-09-18T11:31:10.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922458+0000", 2024-09-18T11:31:10.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:31:10.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922508+0000", 2024-09-18T11:31:10.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0102999999999998e-05 2024-09-18T11:31:10.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922607+0000", 2024-09-18T11:31:10.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8813000000000004e-05 2024-09-18T11:31:10.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:10.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922814+0000", 2024-09-18T11:31:10.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000207488 2024-09-18T11:31:10.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:10.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925504+0000", 2024-09-18T11:31:10.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026899200000000002 2024-09-18T11:31:10.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:10.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927350+0000", 2024-09-18T11:31:10.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00184575 2024-09-18T11:31:10.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927404+0000", 2024-09-18T11:31:10.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4147000000000002e-05 2024-09-18T11:31:10.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927450+0000", 2024-09-18T11:31:10.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6480000000000002e-05 2024-09-18T11:31:10.847 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.847 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.847 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.847 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1790 4.8 4:1898b15d:::smithi11939506-25:head [setxattr _header (57) in=7b,truncate 2453822] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:10.847 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.922467+0000", 2024-09-18T11:31:10.847 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.872327429999999, 2024-09-18T11:31:10.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0052363219999999999, 2024-09-18T11:31:10.848 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.848 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.848 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.848 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.848 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.848 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1790 2024-09-18T11:31:10.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.848 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922467+0000", 2024-09-18T11:31:10.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922467+0000", 2024-09-18T11:31:10.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922467+0000", 2024-09-18T11:31:10.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:10.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922470+0000", 2024-09-18T11:31:10.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2459999999999999e-06 2024-09-18T11:31:10.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922474+0000", 2024-09-18T11:31:10.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:31:10.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922476+0000", 2024-09-18T11:31:10.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3410000000000001e-06 2024-09-18T11:31:10.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.924488+0000", 2024-09-18T11:31:10.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002012425 2024-09-18T11:31:10.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.924623+0000", 2024-09-18T11:31:10.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013435699999999999 2024-09-18T11:31:10.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:10.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.924870+0000", 2024-09-18T11:31:10.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00024748199999999999 2024-09-18T11:31:10.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:10.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927574+0000", 2024-09-18T11:31:10.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027032800000000002 2024-09-18T11:31:10.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:10.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927628+0000", 2024-09-18T11:31:10.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3909000000000002e-05 2024-09-18T11:31:10.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927661+0000", 2024-09-18T11:31:10.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3467999999999999e-05 2024-09-18T11:31:10.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927704+0000", 2024-09-18T11:31:10.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2749e-05 2024-09-18T11:31:10.853 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.853 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.853 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.853 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1791 4.8 4:1898b15d:::smithi11939506-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:10.853 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.922500+0000", 2024-09-18T11:31:10.853 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.872294758999999, 2024-09-18T11:31:10.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005360259, 2024-09-18T11:31:10.853 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.853 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:10.854 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.854 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.854 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.854 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1791 2024-09-18T11:31:10.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.854 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922500+0000", 2024-09-18T11:31:10.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922500+0000", 2024-09-18T11:31:10.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922500+0000", 2024-09-18T11:31:10.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:10.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922503+0000", 2024-09-18T11:31:10.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7009999999999998e-06 2024-09-18T11:31:10.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922509+0000", 2024-09-18T11:31:10.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1710000000000001e-06 2024-09-18T11:31:10.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922511+0000", 2024-09-18T11:31:10.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:31:10.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925283+0000", 2024-09-18T11:31:10.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027722659999999998 2024-09-18T11:31:10.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:10.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925376+0000", 2024-09-18T11:31:10.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3018999999999998e-05 2024-09-18T11:31:10.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927736+0000", 2024-09-18T11:31:10.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023600990000000001 2024-09-18T11:31:10.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927798+0000", 2024-09-18T11:31:10.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1197000000000004e-05 2024-09-18T11:31:10.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927860+0000", 2024-09-18T11:31:10.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2725999999999995e-05 2024-09-18T11:31:10.858 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.858 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.858 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.858 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1934 4.2 e65/22 4:425afe9c:::smithi11939506-29:head v 65'44, mlcod=60'40)", 2024-09-18T11:31:10.858 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:10.859 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.494802765999999, 2024-09-18T11:31:10.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064585119999999996, 2024-09-18T11:31:10.859 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.859 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.859 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:10.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:10.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299990+0000", 2024-09-18T11:31:10.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:10.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300789+0000", 2024-09-18T11:31:10.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079851499999999997 2024-09-18T11:31:10.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300806+0000", 2024-09-18T11:31:10.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7096999999999999e-05 2024-09-18T11:31:10.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300813+0000", 2024-09-18T11:31:10.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5079999999999999e-06 2024-09-18T11:31:10.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300870+0000", 2024-09-18T11:31:10.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7178e-05 2024-09-18T11:31:10.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300897+0000", 2024-09-18T11:31:10.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6284000000000001e-05 2024-09-18T11:31:10.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306410+0000", 2024-09-18T11:31:10.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055135210000000004 2024-09-18T11:31:10.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306451+0000", 2024-09-18T11:31:10.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0379000000000001e-05 2024-09-18T11:31:10.863 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.863 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.863 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.863 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2119 4.7 e73/22 4:e0d95e10:::smithi11939506-22:head v 73'89, mlcod=73'86)", 2024-09-18T11:31:10.863 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:55.701097+0000", 2024-09-18T11:31:10.863 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 15.093698119999999, 2024-09-18T11:31:10.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056939520000000004, 2024-09-18T11:31:10.863 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.863 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.864 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701097+0000", 2024-09-18T11:31:10.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701097+0000", 2024-09-18T11:31:10.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701095+0000", 2024-09-18T11:31:10.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:10.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701919+0000", 2024-09-18T11:31:10.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000824181 2024-09-18T11:31:10.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701938+0000", 2024-09-18T11:31:10.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8614000000000002e-05 2024-09-18T11:31:10.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701947+0000", 2024-09-18T11:31:10.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0699999999999996e-06 2024-09-18T11:31:10.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701992+0000", 2024-09-18T11:31:10.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5614000000000001e-05 2024-09-18T11:31:10.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.702012+0000", 2024-09-18T11:31:10.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9639999999999998e-05 2024-09-18T11:31:10.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.706755+0000", 2024-09-18T11:31:10.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004742708 2024-09-18T11:31:10.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.706791+0000", 2024-09-18T11:31:10.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6106999999999998e-05 2024-09-18T11:31:10.867 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.867 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.867 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.868 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2323 4.2 e77/22 4:425afe9c:::smithi11939506-29:head v 77'58, mlcod=77'56)", 2024-09-18T11:31:10.868 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.194236+0000", 2024-09-18T11:31:10.868 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.6005588070000005, 2024-09-18T11:31:10.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054717079999999996, 2024-09-18T11:31:10.868 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.868 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.868 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194236+0000", 2024-09-18T11:31:10.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194236+0000", 2024-09-18T11:31:10.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194235+0000", 2024-09-18T11:31:10.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:10.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194777+0000", 2024-09-18T11:31:10.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00054227599999999996 2024-09-18T11:31:10.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194793+0000", 2024-09-18T11:31:10.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5662e-05 2024-09-18T11:31:10.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194800+0000", 2024-09-18T11:31:10.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9879999999999998e-06 2024-09-18T11:31:10.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194859+0000", 2024-09-18T11:31:10.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8876999999999999e-05 2024-09-18T11:31:10.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194889+0000", 2024-09-18T11:31:10.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0409999999999999e-05 2024-09-18T11:31:10.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.199659+0000", 2024-09-18T11:31:10.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047701330000000002 2024-09-18T11:31:10.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.199708+0000", 2024-09-18T11:31:10.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8242999999999999e-05 2024-09-18T11:31:10.872 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.872 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.872 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.872 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2452 4.3 4:ca2b2309:::smithi11939506-44:head [read 0~3462242,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:31:10.872 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:31:10.872 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.7580274090000003, 2024-09-18T11:31:10.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060906450000000004, 2024-09-18T11:31:10.872 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.872 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:10.872 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.873 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.873 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.873 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2452 2024-09-18T11:31:10.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.873 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:31:10.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:31:10.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036765+0000", 2024-09-18T11:31:10.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:10.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036777+0000", 2024-09-18T11:31:10.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1422e-05 2024-09-18T11:31:10.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036793+0000", 2024-09-18T11:31:10.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6575999999999999e-05 2024-09-18T11:31:10.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036800+0000", 2024-09-18T11:31:10.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9940000000000003e-06 2024-09-18T11:31:10.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:10.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5847999999999999e-05 2024-09-18T11:31:10.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:10.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036853+0000", 2024-09-18T11:31:10.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0119999999999999e-06 2024-09-18T11:31:10.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038527+0000", 2024-09-18T11:31:10.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016732730000000001 2024-09-18T11:31:10.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038580+0000", 2024-09-18T11:31:10.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3443999999999999e-05 2024-09-18T11:31:10.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042858+0000", 2024-09-18T11:31:10.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042781049999999999 2024-09-18T11:31:10.877 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.877 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.877 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.878 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2453 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:31:10.878 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:31:10.878 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.7579851770000001, 2024-09-18T11:31:10.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063798429999999996, 2024-09-18T11:31:10.878 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.878 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:10.878 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.878 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.878 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.878 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2453 2024-09-18T11:31:10.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.878 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:31:10.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:31:10.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036809+0000", 2024-09-18T11:31:10.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:10.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036812+0000", 2024-09-18T11:31:10.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8339999999999999e-06 2024-09-18T11:31:10.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036819+0000", 2024-09-18T11:31:10.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9890000000000003e-06 2024-09-18T11:31:10.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036823+0000", 2024-09-18T11:31:10.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8099999999999999e-06 2024-09-18T11:31:10.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036912+0000", 2024-09-18T11:31:10.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9083000000000001e-05 2024-09-18T11:31:10.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:10.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036932+0000", 2024-09-18T11:31:10.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0222000000000001e-05 2024-09-18T11:31:10.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042896+0000", 2024-09-18T11:31:10.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059637240000000001 2024-09-18T11:31:10.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042952+0000", 2024-09-18T11:31:10.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5877000000000001e-05 2024-09-18T11:31:10.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043190+0000", 2024-09-18T11:31:10.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00023774500000000001 2024-09-18T11:31:10.882 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.882 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.882 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.883 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2454 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:31:10.883 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:10.883 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.7579492209999996, 2024-09-18T11:31:10.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066404469999999998, 2024-09-18T11:31:10.883 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.883 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:10.883 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.883 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.883 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.883 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2454 2024-09-18T11:31:10.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.883 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:10.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:10.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036845+0000", 2024-09-18T11:31:10.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:10.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036856+0000", 2024-09-18T11:31:10.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1338000000000001e-05 2024-09-18T11:31:10.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036863+0000", 2024-09-18T11:31:10.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4749999999999998e-06 2024-09-18T11:31:10.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036867+0000", 2024-09-18T11:31:10.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3839999999999999e-06 2024-09-18T11:31:10.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036946+0000", 2024-09-18T11:31:10.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9054000000000006e-05 2024-09-18T11:31:10.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:10.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036949+0000", 2024-09-18T11:31:10.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9390000000000002e-06 2024-09-18T11:31:10.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043213+0000", 2024-09-18T11:31:10.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062640090000000001 2024-09-18T11:31:10.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043261+0000", 2024-09-18T11:31:10.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7750999999999997e-05 2024-09-18T11:31:10.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043486+0000", 2024-09-18T11:31:10.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000224985 2024-09-18T11:31:10.887 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.887 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.887 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.887 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2505 4.1 e85/22 4:87078dfe:::smithi11939506-14:head v 85'167, mlcod=85'163)", 2024-09-18T11:31:10.888 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.993736+0000", 2024-09-18T11:31:10.888 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.8010587770000002, 2024-09-18T11:31:10.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056183630000000003, 2024-09-18T11:31:10.888 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.888 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.888 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993736+0000", 2024-09-18T11:31:10.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993736+0000", 2024-09-18T11:31:10.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993735+0000", 2024-09-18T11:31:10.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:10.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.994296+0000", 2024-09-18T11:31:10.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056099399999999999 2024-09-18T11:31:10.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.994313+0000", 2024-09-18T11:31:10.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6982000000000001e-05 2024-09-18T11:31:10.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.994319+0000", 2024-09-18T11:31:10.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0279999999999997e-06 2024-09-18T11:31:10.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.994374+0000", 2024-09-18T11:31:10.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5019999999999998e-05 2024-09-18T11:31:10.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.994401+0000", 2024-09-18T11:31:10.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7104e-05 2024-09-18T11:31:10.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.999314+0000", 2024-09-18T11:31:10.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049127390000000002 2024-09-18T11:31:10.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.999354+0000", 2024-09-18T11:31:10.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0763000000000003e-05 2024-09-18T11:31:10.892 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.892 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.892 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.892 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2516 4.3 4:c70c3162:::smithi11939506-35:head [read 0~2605085,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:10.892 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173302+0000", 2024-09-18T11:31:10.892 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.6214924580000001, 2024-09-18T11:31:10.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050184130000000002, 2024-09-18T11:31:10.892 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.892 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:10.892 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.892 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.893 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.893 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2516 2024-09-18T11:31:10.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.893 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173302+0000", 2024-09-18T11:31:10.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173302+0000", 2024-09-18T11:31:10.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173302+0000", 2024-09-18T11:31:10.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:10.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173307+0000", 2024-09-18T11:31:10.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7539999999999997e-06 2024-09-18T11:31:10.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173317+0000", 2024-09-18T11:31:10.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0013000000000001e-05 2024-09-18T11:31:10.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173334+0000", 2024-09-18T11:31:10.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7430000000000001e-05 2024-09-18T11:31:10.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176876+0000", 2024-09-18T11:31:10.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035424010000000001 2024-09-18T11:31:10.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.177062+0000", 2024-09-18T11:31:10.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000185475 2024-09-18T11:31:10.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178321+0000", 2024-09-18T11:31:10.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001258962 2024-09-18T11:31:10.896 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.896 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.896 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.896 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2553 4.3 4:ca2b2309:::smithi11939506-44:head [write 564768~630013,stat] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-18T11:31:10.896 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:31:10.896 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.59767158799999998, 2024-09-18T11:31:10.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065686879999999996, 2024-09-18T11:31:10.896 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:10.897 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:10.897 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:10.897 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:10.897 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:10.897 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2553 2024-09-18T11:31:10.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.897 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:10.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:10.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:31:10.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:10.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:31:10.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:10.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:10.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197120+0000", 2024-09-18T11:31:10.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:10.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:10.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199000+0000", 2024-09-18T11:31:10.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018792080000000001 2024-09-18T11:31:10.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:10.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199044+0000", 2024-09-18T11:31:10.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4048999999999998e-05 2024-09-18T11:31:10.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:10.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199071+0000", 2024-09-18T11:31:10.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7288e-05 2024-09-18T11:31:10.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:10.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199119+0000", 2024-09-18T11:31:10.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8000999999999997e-05 2024-09-18T11:31:10.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:10.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199208+0000", 2024-09-18T11:31:10.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9209000000000006e-05 2024-09-18T11:31:10.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:10.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199385+0000", 2024-09-18T11:31:10.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000177095 2024-09-18T11:31:10.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:10.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.202282+0000", 2024-09-18T11:31:10.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028970469999999998 2024-09-18T11:31:10.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:10.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203578+0000", 2024-09-18T11:31:10.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012960490000000001 2024-09-18T11:31:10.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:10.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203638+0000", 2024-09-18T11:31:10.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9599000000000003e-05 2024-09-18T11:31:10.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:10.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:10.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:10.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203692+0000", 2024-09-18T11:31:10.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3908e-05 2024-09-18T11:31:10.902 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.902 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.902 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.902 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:10.902 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:10.902 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:10.904 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:31:10.904 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:10.902+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:10.958 INFO:tasks.rados.rados.0.smithi119.stdout:1231: expect (ObjNum 99 snap 6 seq_num 99) 2024-09-18T11:31:11.002 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:11.004 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:11.003+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:11.018 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:11.018 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:11.033 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:11.033 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:11.045 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:11.046 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:11.046 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:11.046 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:11.048 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:11.048 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:11.054 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:31:11.063 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:11.063 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:11.079 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:11.079 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:11.094 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:11.094 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:11.106 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:11.104+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:11.109 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:11.109 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:11.112 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:11.113 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:11.127 INFO:tasks.rados.rados.0.smithi119.stdout:1229: done (3 left) 2024-09-18T11:31:11.127 INFO:tasks.rados.rados.0.smithi119.stdout:1230: done (2 left) 2024-09-18T11:31:11.127 INFO:tasks.rados.rados.0.smithi119.stdout:1231: done (1 left) 2024-09-18T11:31:11.127 INFO:tasks.rados.rados.0.smithi119.stdout:1232: done (0 left) 2024-09-18T11:31:11.127 INFO:tasks.rados.rados.0.smithi119.stdout:1233: write oid 28 current snap is 26 2024-09-18T11:31:11.127 INFO:tasks.rados.rados.0.smithi119.stdout:1233: seq_num 227 ranges {615189=455760,1733883=711078,3134431=460041} 2024-09-18T11:31:11.132 INFO:tasks.rados.rados.0.smithi119.stdout:1233: writing smithi11939506-28 from 615189 to 1070949 tid 1 2024-09-18T11:31:11.140 INFO:tasks.rados.rados.0.smithi119.stdout:1233: writing smithi11939506-28 from 1733883 to 2444961 tid 2 2024-09-18T11:31:11.147 INFO:tasks.rados.rados.0.smithi119.stdout:1233: writing smithi11939506-28 from 3134431 to 3594472 tid 3 2024-09-18T11:31:11.147 INFO:tasks.rados.rados.0.smithi119.stdout:1234: delete oid 17 current snap is 26 2024-09-18T11:31:11.147 INFO:tasks.rados.rados.0.smithi119.stdout:1233: finishing write tid 1 to smithi11939506-28 2024-09-18T11:31:11.147 INFO:tasks.rados.rados.0.smithi119.stdout:1233: finishing write tid 2 to smithi11939506-28 2024-09-18T11:31:11.153 INFO:tasks.rados.rados.0.smithi119.stdout:1233: finishing write tid 3 to smithi11939506-28 2024-09-18T11:31:11.153 INFO:tasks.rados.rados.0.smithi119.stdout:1233: finishing write tid 4 to smithi11939506-28 2024-09-18T11:31:11.153 INFO:tasks.rados.rados.0.smithi119.stdout:1233: finishing write tid 5 to smithi11939506-28 2024-09-18T11:31:11.154 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 28 v 122 (ObjNum 227 snap 26 seq_num 227) dirty exists 2024-09-18T11:31:11.154 INFO:tasks.rados.rados.0.smithi119.stdout:1233: left oid 28 (ObjNum 227 snap 26 seq_num 227) 2024-09-18T11:31:11.154 INFO:tasks.rados.rados.0.smithi119.stdout:1233: done (1 left) 2024-09-18T11:31:11.154 INFO:tasks.rados.rados.0.smithi119.stdout:1234: done (0 left) 2024-09-18T11:31:11.154 INFO:tasks.rados.rados.0.smithi119.stdout:1235: write oid 39 current snap is 26 2024-09-18T11:31:11.154 INFO:tasks.rados.rados.0.smithi119.stdout:1235: seq_num 229 ranges {648803=722611,1967200=714747,3421983=426820} 2024-09-18T11:31:11.154 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:11.155 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:11.159 INFO:tasks.rados.rados.0.smithi119.stdout:1235: writing smithi11939506-39 from 648803 to 1371414 tid 1 2024-09-18T11:31:11.163 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:31:11.166 INFO:tasks.rados.rados.0.smithi119.stdout:1235: writing smithi11939506-39 from 1967200 to 2681947 tid 2 2024-09-18T11:31:11.173 INFO:tasks.rados.rados.0.smithi119.stdout:1235: writing smithi11939506-39 from 3421983 to 3848803 tid 3 2024-09-18T11:31:11.173 INFO:tasks.rados.rados.0.smithi119.stdout:1236: rollback oid 27 current snap is 26 2024-09-18T11:31:11.173 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 27 to 23 2024-09-18T11:31:11.173 INFO:tasks.rados.rados.0.smithi119.stdout:1235: finishing write tid 1 to smithi11939506-39 2024-09-18T11:31:11.173 INFO:tasks.rados.rados.0.smithi119.stdout:1235: finishing write tid 2 to smithi11939506-39 2024-09-18T11:31:11.173 INFO:tasks.rados.rados.0.smithi119.stdout:1237: read oid 8 snap -1 2024-09-18T11:31:11.173 INFO:tasks.rados.rados.0.smithi119.stdout:1237: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-18T11:31:11.173 INFO:tasks.rados.rados.0.smithi119.stdout:1238: read oid 48 snap -1 2024-09-18T11:31:11.174 INFO:tasks.rados.rados.0.smithi119.stdout:1238: expect (ObjNum 191 snap 21 seq_num 191) 2024-09-18T11:31:11.174 INFO:tasks.rados.rados.0.smithi119.stdout:1239: read oid 17 snap -1 2024-09-18T11:31:11.174 INFO:tasks.rados.rados.0.smithi119.stdout:1239: expect deleted 2024-09-18T11:31:11.174 INFO:tasks.rados.rados.0.smithi119.stdout:1240: write (excl) oid 3 current snap is 26 2024-09-18T11:31:11.174 INFO:tasks.rados.rados.0.smithi119.stdout:1240: seq_num 230 ranges {665610=797860,1909329=458405,2816700=410210} 2024-09-18T11:31:11.180 INFO:tasks.rados.rados.0.smithi119.stdout:1240: writing smithi11939506-3 from 665610 to 1463470 tid 1 2024-09-18T11:31:11.186 INFO:tasks.rados.rados.0.smithi119.stdout:1240: writing smithi11939506-3 from 1909329 to 2367734 tid 2 2024-09-18T11:31:11.191 INFO:tasks.rados.rados.0.smithi119.stdout:1240: writing smithi11939506-3 from 2816700 to 3226910 tid 3 2024-09-18T11:31:11.192 INFO:tasks.rados.rados.0.smithi119.stdout:1241: read oid 43 snap -1 2024-09-18T11:31:11.192 INFO:tasks.rados.rados.0.smithi119.stdout:1241: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:31:11.192 INFO:tasks.rados.rados.0.smithi119.stdout:1242: snap_create 2024-09-18T11:31:11.192 INFO:tasks.rados.rados.0.smithi119.stdout:1236: finishing rollback tid 1 to smithi11939506-27 2024-09-18T11:31:11.192 INFO:tasks.rados.rados.0.smithi119.stdout:1236: finishing rollback tid 2 to smithi11939506-27 2024-09-18T11:31:11.192 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 146 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-18T11:31:11.192 INFO:tasks.rados.rados.0.smithi119.stdout:1237: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-18T11:31:11.206 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:11.205+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:11.302 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:11.302 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:11.307 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:11.306+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:11.310 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:31:11.324 INFO:tasks.rados.rados.0.smithi119.stdout:1235: finishing write tid 3 to smithi11939506-39 2024-09-18T11:31:11.324 INFO:tasks.rados.rados.0.smithi119.stdout:1235: finishing write tid 4 to smithi11939506-39 2024-09-18T11:31:11.324 INFO:tasks.rados.rados.0.smithi119.stdout:1235: finishing write tid 5 to smithi11939506-39 2024-09-18T11:31:11.324 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 39 v 174 (ObjNum 229 snap 26 seq_num 229) dirty exists 2024-09-18T11:31:11.324 INFO:tasks.rados.rados.0.smithi119.stdout:1235: left oid 39 (ObjNum 229 snap 26 seq_num 229) 2024-09-18T11:31:11.325 INFO:tasks.rados.rados.0.smithi119.stdout:1238: expect (ObjNum 191 snap 21 seq_num 191) 2024-09-18T11:31:11.408 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:11.408 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:11.408 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:11.407+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:11.416 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:31:11.450 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is set 2024-09-18T11:31:11.453 INFO:tasks.rados.rados.0.smithi119.stdout:1240: finishing write tid 1 to smithi11939506-3 2024-09-18T11:31:11.453 INFO:tasks.rados.rados.0.smithi119.stdout:1240: finishing write tid 2 to smithi11939506-3 2024-09-18T11:31:11.454 INFO:tasks.rados.rados.0.smithi119.stdout:1240: finishing write tid 3 to smithi11939506-3 2024-09-18T11:31:11.454 INFO:tasks.rados.rados.0.smithi119.stdout:1240: finishing write tid 4 to smithi11939506-3 2024-09-18T11:31:11.454 INFO:tasks.rados.rados.0.smithi119.stdout:1240: finishing write tid 5 to smithi11939506-3 2024-09-18T11:31:11.454 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 127 (ObjNum 230 snap 26 seq_num 230) dirty exists 2024-09-18T11:31:11.454 INFO:tasks.rados.rados.0.smithi119.stdout:1240: left oid 3 (ObjNum 230 snap 26 seq_num 230) 2024-09-18T11:31:11.454 INFO:tasks.rados.rados.0.smithi119.stdout:1241: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:31:11.508 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:11.507+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:11.545 INFO:tasks.rados.rados.0.smithi119.stdout:1235: done (7 left) 2024-09-18T11:31:11.545 INFO:tasks.rados.rados.0.smithi119.stdout:1236: done (6 left) 2024-09-18T11:31:11.545 INFO:tasks.rados.rados.0.smithi119.stdout:1237: done (5 left) 2024-09-18T11:31:11.545 INFO:tasks.rados.rados.0.smithi119.stdout:1238: done (4 left) 2024-09-18T11:31:11.545 INFO:tasks.rados.rados.0.smithi119.stdout:1239: done (3 left) 2024-09-18T11:31:11.545 INFO:tasks.rados.rados.0.smithi119.stdout:1240: done (2 left) 2024-09-18T11:31:11.545 INFO:tasks.rados.rados.0.smithi119.stdout:1241: done (1 left) 2024-09-18T11:31:11.546 INFO:tasks.rados.rados.0.smithi119.stdout:1242: done (0 left) 2024-09-18T11:31:11.546 INFO:tasks.rados.rados.0.smithi119.stdout:1243: write (excl) oid 38 current snap is 27 2024-09-18T11:31:11.546 INFO:tasks.rados.rados.0.smithi119.stdout:1243: seq_num 231 ranges {682417=473109,1567811=602063,2844123=621482} 2024-09-18T11:31:11.551 INFO:tasks.rados.rados.0.smithi119.stdout:1243: writing smithi11939506-38 from 682417 to 1155526 tid 1 2024-09-18T11:31:11.556 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:11.556 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:11.557 INFO:tasks.rados.rados.0.smithi119.stdout:1243: writing smithi11939506-38 from 1567811 to 2169874 tid 2 2024-09-18T11:31:11.564 INFO:tasks.rados.rados.0.smithi119.stdout:1243: writing smithi11939506-38 from 2844123 to 3465605 tid 3 2024-09-18T11:31:11.565 INFO:tasks.rados.rados.0.smithi119.stdout:1244: read oid 14 snap -1 2024-09-18T11:31:11.565 INFO:tasks.rados.rados.0.smithi119.stdout:1244: expect (ObjNum 220 snap 26 seq_num 220) 2024-09-18T11:31:11.565 INFO:tasks.rados.rados.0.smithi119.stdout:1245: write (excl) oid 45 current snap is 27 2024-09-18T11:31:11.565 INFO:tasks.rados.rados.0.smithi119.stdout:1243: finishing write tid 1 to smithi11939506-38 2024-09-18T11:31:11.565 INFO:tasks.rados.rados.0.smithi119.stdout:1243: finishing write tid 2 to smithi11939506-38 2024-09-18T11:31:11.565 INFO:tasks.rados.rados.0.smithi119.stdout:1245: seq_num 232 ranges {699224=548358,2026293=462074,3271546=432754} 2024-09-18T11:31:11.566 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:31:11.571 INFO:tasks.rados.rados.0.smithi119.stdout:1245: writing smithi11939506-45 from 699224 to 1247582 tid 1 2024-09-18T11:31:11.578 INFO:tasks.rados.rados.0.smithi119.stdout:1245: writing smithi11939506-45 from 2026293 to 2488367 tid 2 2024-09-18T11:31:11.585 INFO:tasks.rados.rados.0.smithi119.stdout:1245: writing smithi11939506-45 from 3271546 to 3704300 tid 3 2024-09-18T11:31:11.585 INFO:tasks.rados.rados.0.smithi119.stdout:1246: delete oid 28 current snap is 27 2024-09-18T11:31:11.585 INFO:tasks.rados.rados.0.smithi119.stdout:1243: finishing write tid 3 to smithi11939506-38 2024-09-18T11:31:11.585 INFO:tasks.rados.rados.0.smithi119.stdout:1243: finishing write tid 4 to smithi11939506-38 2024-09-18T11:31:11.585 INFO:tasks.rados.rados.0.smithi119.stdout:1243: finishing write tid 5 to smithi11939506-38 2024-09-18T11:31:11.586 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 124 (ObjNum 231 snap 27 seq_num 231) dirty exists 2024-09-18T11:31:11.586 INFO:tasks.rados.rados.0.smithi119.stdout:1243: left oid 38 (ObjNum 231 snap 27 seq_num 231) 2024-09-18T11:31:11.586 INFO:tasks.rados.rados.0.smithi119.stdout:1244: expect (ObjNum 220 snap 26 seq_num 220) 2024-09-18T11:31:11.609 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:11.608+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:11.705 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:11.705 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:11.710 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:11.709+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:11.713 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:31:11.811 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:11.810+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:11.812 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:11.813 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:11.821 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:31:11.859 INFO:tasks.rados.rados.0.smithi119.stdout:1245: finishing write tid 1 to smithi11939506-45 2024-09-18T11:31:11.859 INFO:tasks.rados.rados.0.smithi119.stdout:1245: finishing write tid 2 to smithi11939506-45 2024-09-18T11:31:11.859 INFO:tasks.rados.rados.0.smithi119.stdout:1245: finishing write tid 3 to smithi11939506-45 2024-09-18T11:31:11.859 INFO:tasks.rados.rados.0.smithi119.stdout:1245: finishing write tid 4 to smithi11939506-45 2024-09-18T11:31:11.859 INFO:tasks.rados.rados.0.smithi119.stdout:1245: finishing write tid 5 to smithi11939506-45 2024-09-18T11:31:11.859 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 52 (ObjNum 232 snap 27 seq_num 232) dirty exists 2024-09-18T11:31:11.859 INFO:tasks.rados.rados.0.smithi119.stdout:1245: left oid 45 (ObjNum 232 snap 27 seq_num 232) 2024-09-18T11:31:11.860 INFO:tasks.rados.rados.0.smithi119.stdout:1243: done (3 left) 2024-09-18T11:31:11.860 INFO:tasks.rados.rados.0.smithi119.stdout:1244: done (2 left) 2024-09-18T11:31:11.860 INFO:tasks.rados.rados.0.smithi119.stdout:1245: done (1 left) 2024-09-18T11:31:11.860 INFO:tasks.rados.rados.0.smithi119.stdout:1246: done (0 left) 2024-09-18T11:31:11.860 INFO:tasks.rados.rados.0.smithi119.stdout:1247: snap_create 2024-09-18T11:31:11.912 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:11.911+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:11.958 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:11.958 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:11.966 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:31:12.050 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:12.049+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:12.104 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:12.104 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:12.113 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:31:12.154 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:12.153+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:12.215 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:12.214+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:12.252 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:12.252 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:12.260 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:31:12.344 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:12.343+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:12.397 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:12.397 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:12.405 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:31:12.417 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:12.416+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:12.463 INFO:tasks.rados.rados.0.smithi119.stdout:1247: done (0 left) 2024-09-18T11:31:12.464 INFO:tasks.rados.rados.0.smithi119.stdout:1248: read oid 45 snap -1 2024-09-18T11:31:12.464 INFO:tasks.rados.rados.0.smithi119.stdout:1248: expect (ObjNum 232 snap 27 seq_num 232) 2024-09-18T11:31:12.464 INFO:tasks.rados.rados.0.smithi119.stdout:1249: delete oid 27 current snap is 28 2024-09-18T11:31:12.467 INFO:tasks.rados.rados.0.smithi119.stdout:1249: done (1 left) 2024-09-18T11:31:12.467 INFO:tasks.rados.rados.0.smithi119.stdout:1250: snap_remove snap 22 2024-09-18T11:31:12.517 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:12.517+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:12.544 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:12.544 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:12.553 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:31:12.637 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:12.636+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:12.692 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:12.692 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:12.701 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:31:12.719 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:12.718+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:12.820 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:12.819+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:12.838 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:12.838 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:12.846 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:31:12.921 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:12.920+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:12.945 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:12.945 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:12.954 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:31:13.006 INFO:tasks.rados.rados.0.smithi119.stdout:1250: done (1 left) 2024-09-18T11:31:13.007 INFO:tasks.rados.rados.0.smithi119.stdout:1251: set_chunk oid 50 target oid 2024-09-18T11:31:13.018 INFO:tasks.rados.rados.0.smithi119.stdout:1251: set_chunk oid 50 offset: 193024 length: 425984 target oid e8f0f164d73e243c608360599009d09aab72bb3975d6e5062dc6984eabd48d8b offset: 0 snap -1 2024-09-18T11:31:13.018 INFO:tasks.rados.rados.0.smithi119.stdout:1252: write oid 39 current snap is 28 2024-09-18T11:31:13.018 INFO:tasks.rados.rados.0.smithi119.stdout:1252: seq_num 235 ranges {749645=774105,2085386=609401,3237462=712183} 2024-09-18T11:31:13.021 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:13.020+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:13.026 INFO:tasks.rados.rados.0.smithi119.stdout:1252: writing smithi11939506-39 from 749645 to 1523750 tid 1 2024-09-18T11:31:13.032 INFO:tasks.rados.rados.0.smithi119.stdout:1252: writing smithi11939506-39 from 2085386 to 2694787 tid 2 2024-09-18T11:31:13.039 INFO:tasks.rados.rados.0.smithi119.stdout:1252: writing smithi11939506-39 from 3237462 to 3949645 tid 3 2024-09-18T11:31:13.040 INFO:tasks.rados.rados.0.smithi119.stdout:1253: read oid 3 snap -1 2024-09-18T11:31:13.040 INFO:tasks.rados.rados.0.smithi119.stdout:1253: expect (ObjNum 230 snap 26 seq_num 230) 2024-09-18T11:31:13.040 INFO:tasks.rados.rados.0.smithi119.stdout:1254: set_chunk oid 17 target oid 2024-09-18T11:31:13.040 INFO:tasks.rados.rados.0.smithi119.stdout:1254: set_chunk oid 17 offset: 0 length: 4000000 target oid smithi11939506- offset: 0 snap -1 2024-09-18T11:31:13.040 INFO:tasks.rados.rados.0.smithi119.stdout:1255: read oid 30 snap 21 2024-09-18T11:31:13.040 INFO:tasks.rados.rados.0.smithi119.stdout:1255: expect (ObjNum 174 snap 20 seq_num 174) 2024-09-18T11:31:13.040 INFO:tasks.rados.rados.0.smithi119.stdout:1256: write (excl) oid 31 current snap is 28 2024-09-18T11:31:13.040 INFO:tasks.rados.rados.0.smithi119.stdout:1256: seq_num 236 ranges {766452=565707,1860221=469412,2981238=594195} 2024-09-18T11:31:13.046 INFO:tasks.rados.rados.0.smithi119.stdout:1256: writing smithi11939506-31 from 766452 to 1332159 tid 1 2024-09-18T11:31:13.052 INFO:tasks.rados.rados.0.smithi119.stdout:1256: writing smithi11939506-31 from 1860221 to 2329633 tid 2 2024-09-18T11:31:13.059 INFO:tasks.rados.rados.0.smithi119.stdout:1256: writing smithi11939506-31 from 2981238 to 3575433 tid 3 2024-09-18T11:31:13.059 INFO:tasks.rados.rados.0.smithi119.stdout:1257: snap_remove snap 14 2024-09-18T11:31:13.073 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:13.073 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:13.082 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:31:13.113 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:31:13.126 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:13.125+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:13.181 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:13.181 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:13.189 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:31:13.212 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:13.228 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:13.228 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:13.230 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:13.229+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:13.243 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:13.243 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:13.258 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:13.258 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:13.273 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:13.273 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:13.288 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:13.288 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:13.303 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:13.303 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:13.318 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:13.319 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:13.321 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:13.322 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:13.323 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:13.322+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:13.330 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:13.330 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:13.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:13.330 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:13.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.330 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1530 4.d 4:b81644f2:::smithi11939506-34:head [write 467261~717275] snapc 14=[14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:31:13.330 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:31:13.330 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 38.937185538999998, 2024-09-18T11:31:13.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075660950000000001, 2024-09-18T11:31:13.331 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.331 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.331 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.331 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.331 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.331 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1530 2024-09-18T11:31:13.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.331 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:31:13.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:31:13.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390682+0000", 2024-09-18T11:31:13.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:13.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391554+0000", 2024-09-18T11:31:13.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00087220600000000004 2024-09-18T11:31:13.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391570+0000", 2024-09-18T11:31:13.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6002999999999999e-05 2024-09-18T11:31:13.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391575+0000", 2024-09-18T11:31:13.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2549999999999997e-06 2024-09-18T11:31:13.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391631+0000", 2024-09-18T11:31:13.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5671000000000003e-05 2024-09-18T11:31:13.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391751+0000", 2024-09-18T11:31:13.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000119821 2024-09-18T11:31:13.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:13.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391974+0000", 2024-09-18T11:31:13.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022309900000000001 2024-09-18T11:31:13.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:13.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398136+0000", 2024-09-18T11:31:13.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006161573 2024-09-18T11:31:13.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:13.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398184+0000", 2024-09-18T11:31:13.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8458000000000003e-05 2024-09-18T11:31:13.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398219+0000", 2024-09-18T11:31:13.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4573999999999998e-05 2024-09-18T11:31:13.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398251+0000", 2024-09-18T11:31:13.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2085000000000002e-05 2024-09-18T11:31:13.336 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.336 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.336 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.336 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:31:13.336 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:13.337 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 38.911783215, 2024-09-18T11:31:13.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:31:13.337 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.337 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.337 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.337 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.337 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.337 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:31:13.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.337 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:13.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:13.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:31:13.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:13.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:31:13.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:31:13.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:31:13.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:31:13.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:31:13.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:31:13.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:31:13.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:31:13.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:31:13.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:31:13.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:13.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:31:13.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:31:13.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:13.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:31:13.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:31:13.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:13.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:31:13.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:31:13.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:31:13.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:31:13.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:31:13.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:31:13.342 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.342 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.342 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.343 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1628 3.3 3:f80aa31b:::77cdbf3abbada57d0636b2198046a030673818705cbf13bb77b30c3e68b24345:head [write 0~582144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:13.343 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:13.343 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.016893215000003, 2024-09-18T11:31:13.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081714639999999998, 2024-09-18T11:31:13.343 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.343 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.343 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.343 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.343 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.343 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1628 2024-09-18T11:31:13.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.344 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:13.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:13.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310975+0000", 2024-09-18T11:31:13.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:13.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312086+0000", 2024-09-18T11:31:13.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001111288 2024-09-18T11:31:13.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312104+0000", 2024-09-18T11:31:13.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7427000000000001e-05 2024-09-18T11:31:13.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312110+0000", 2024-09-18T11:31:13.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5320000000000003e-06 2024-09-18T11:31:13.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312179+0000", 2024-09-18T11:31:13.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9196000000000006e-05 2024-09-18T11:31:13.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312375+0000", 2024-09-18T11:31:13.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00019550899999999999 2024-09-18T11:31:13.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:13.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312558+0000", 2024-09-18T11:31:13.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000182566 2024-09-18T11:31:13.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:13.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.314286+0000", 2024-09-18T11:31:13.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00172831 2024-09-18T11:31:13.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:13.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319079+0000", 2024-09-18T11:31:13.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047929340000000004 2024-09-18T11:31:13.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319116+0000", 2024-09-18T11:31:13.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6993999999999999e-05 2024-09-18T11:31:13.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319148+0000", 2024-09-18T11:31:13.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2539999999999997e-05 2024-09-18T11:31:13.349 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.349 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.349 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.349 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1662 4.1 4:87078dfe:::smithi11939506-14:head [write 635331~786118,stat] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:13.349 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.360496+0000", 2024-09-18T11:31:13.349 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.967374470000003, 2024-09-18T11:31:13.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071849519999999997, 2024-09-18T11:31:13.349 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.349 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.349 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.349 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.349 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.350 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1662 2024-09-18T11:31:13.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.350 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.360496+0000", 2024-09-18T11:31:13.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.360496+0000", 2024-09-18T11:31:13.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.360493+0000", 2024-09-18T11:31:13.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:13.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361139+0000", 2024-09-18T11:31:13.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064615400000000002 2024-09-18T11:31:13.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361154+0000", 2024-09-18T11:31:13.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4970000000000001e-05 2024-09-18T11:31:13.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361161+0000", 2024-09-18T11:31:13.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4139999999999999e-06 2024-09-18T11:31:13.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361201+0000", 2024-09-18T11:31:13.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0385e-05 2024-09-18T11:31:13.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361273+0000", 2024-09-18T11:31:13.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2224999999999994e-05 2024-09-18T11:31:13.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:13.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361406+0000", 2024-09-18T11:31:13.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013257699999999999 2024-09-18T11:31:13.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:13.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367415+0000", 2024-09-18T11:31:13.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006009655 2024-09-18T11:31:13.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:13.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367586+0000", 2024-09-18T11:31:13.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017109800000000001 2024-09-18T11:31:13.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367636+0000", 2024-09-18T11:31:13.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9721e-05 2024-09-18T11:31:13.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367681+0000", 2024-09-18T11:31:13.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4382999999999998e-05 2024-09-18T11:31:13.355 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.355 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.355 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.355 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1663 4.1 4:87078dfe:::smithi11939506-14:head [write 1916458~318873] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:13.355 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:13.355 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.963946651999997, 2024-09-18T11:31:13.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079505749999999997, 2024-09-18T11:31:13.356 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.356 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.356 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.356 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.356 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.356 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1663 2024-09-18T11:31:13.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.356 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:13.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:13.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363922+0000", 2024-09-18T11:31:13.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:13.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364211+0000", 2024-09-18T11:31:13.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000288385 2024-09-18T11:31:13.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364223+0000", 2024-09-18T11:31:13.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2582000000000001e-05 2024-09-18T11:31:13.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364228+0000", 2024-09-18T11:31:13.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0000000000000004e-06 2024-09-18T11:31:13.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364268+0000", 2024-09-18T11:31:13.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0235999999999999e-05 2024-09-18T11:31:13.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364341+0000", 2024-09-18T11:31:13.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2266000000000002e-05 2024-09-18T11:31:13.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:13.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364444+0000", 2024-09-18T11:31:13.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103632 2024-09-18T11:31:13.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369461+0000", 2024-09-18T11:31:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005016848 2024-09-18T11:31:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371820+0000", 2024-09-18T11:31:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023586169999999999 2024-09-18T11:31:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371851+0000", 2024-09-18T11:31:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1297999999999997e-05 2024-09-18T11:31:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371874+0000", 2024-09-18T11:31:13.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2841e-05 2024-09-18T11:31:13.361 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.361 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.361 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.361 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1664 4.1 4:87078dfe:::smithi11939506-14:head [setxattr _header (57) in=7b,truncate 2235331] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:13.361 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:13.361 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.963632994999998, 2024-09-18T11:31:13.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010368235999999999, 2024-09-18T11:31:13.361 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.361 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.362 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.362 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.362 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.362 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1664 2024-09-18T11:31:13.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.362 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:13.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:13.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364236+0000", 2024-09-18T11:31:13.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:13.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364240+0000", 2024-09-18T11:31:13.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9829999999999998e-06 2024-09-18T11:31:13.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364247+0000", 2024-09-18T11:31:13.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2140000000000001e-06 2024-09-18T11:31:13.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364256+0000", 2024-09-18T11:31:13.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1509999999999993e-06 2024-09-18T11:31:13.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366529+0000", 2024-09-18T11:31:13.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022731409999999998 2024-09-18T11:31:13.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366621+0000", 2024-09-18T11:31:13.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2408e-05 2024-09-18T11:31:13.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:13.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366801+0000", 2024-09-18T11:31:13.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018002200000000001 2024-09-18T11:31:13.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:13.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372155+0000", 2024-09-18T11:31:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053531519999999999 2024-09-18T11:31:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374539+0000", 2024-09-18T11:31:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023842580000000002 2024-09-18T11:31:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374573+0000", 2024-09-18T11:31:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4205999999999997e-05 2024-09-18T11:31:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374605+0000", 2024-09-18T11:31:13.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2283999999999998e-05 2024-09-18T11:31:13.367 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.367 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.367 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.367 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1665 4.1 4:87078dfe:::smithi11939506-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:13.367 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:13.367 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.963608821000001, 2024-09-18T11:31:13.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010471897000000001, 2024-09-18T11:31:13.367 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.367 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:13.368 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.368 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.368 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.368 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1665 2024-09-18T11:31:13.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.368 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:13.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:13.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:13.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:13.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364263+0000", 2024-09-18T11:31:13.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6400000000000001e-06 2024-09-18T11:31:13.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364270+0000", 2024-09-18T11:31:13.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:31:13.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364274+0000", 2024-09-18T11:31:13.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.968e-06 2024-09-18T11:31:13.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367354+0000", 2024-09-18T11:31:13.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030803430000000001 2024-09-18T11:31:13.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:13.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367406+0000", 2024-09-18T11:31:13.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2330999999999998e-05 2024-09-18T11:31:13.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374625+0000", 2024-09-18T11:31:13.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072190689999999998 2024-09-18T11:31:13.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374673+0000", 2024-09-18T11:31:13.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7525999999999999e-05 2024-09-18T11:31:13.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374733+0000", 2024-09-18T11:31:13.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0359000000000003e-05 2024-09-18T11:31:13.372 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.372 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.372 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.372 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:101 4.5 e58/22 4:aa575018:::smithi11939506-11:17 v 58'67, mlcod=55'62)", 2024-09-18T11:31:13.372 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.273235+0000", 2024-09-18T11:31:13.372 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 34.054635529999999, 2024-09-18T11:31:13.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0067904050000000002, 2024-09-18T11:31:13.372 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.372 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.373 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273235+0000", 2024-09-18T11:31:13.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273235+0000", 2024-09-18T11:31:13.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273234+0000", 2024-09-18T11:31:13.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:13.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273239+0000", 2024-09-18T11:31:13.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7999999999999998e-06 2024-09-18T11:31:13.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273247+0000", 2024-09-18T11:31:13.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4700000000000002e-06 2024-09-18T11:31:13.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.273251+0000", 2024-09-18T11:31:13.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7500000000000001e-06 2024-09-18T11:31:13.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.274599+0000", 2024-09-18T11:31:13.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013483900000000001 2024-09-18T11:31:13.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.274623+0000", 2024-09-18T11:31:13.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3499999999999999e-05 2024-09-18T11:31:13.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.279994+0000", 2024-09-18T11:31:13.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053713889999999999 2024-09-18T11:31:13.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.280025+0000", 2024-09-18T11:31:13.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0579e-05 2024-09-18T11:31:13.376 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.376 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.377 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.377 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1947 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865577,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:13.377 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:13.377 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.988329110999999, 2024-09-18T11:31:13.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010043066999999999, 2024-09-18T11:31:13.377 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.377 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:13.377 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.377 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.377 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.377 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1947 2024-09-18T11:31:13.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.378 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:13.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:13.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339540+0000", 2024-09-18T11:31:13.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:13.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339545+0000", 2024-09-18T11:31:13.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6290000000000001e-06 2024-09-18T11:31:13.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339553+0000", 2024-09-18T11:31:13.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6299999999999998e-06 2024-09-18T11:31:13.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339557+0000", 2024-09-18T11:31:13.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3590000000000001e-06 2024-09-18T11:31:13.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345600+0000", 2024-09-18T11:31:13.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060434640000000001 2024-09-18T11:31:13.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345710+0000", 2024-09-18T11:31:13.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010924700000000001 2024-09-18T11:31:13.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349584+0000", 2024-09-18T11:31:13.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038743459999999999 2024-09-18T11:31:13.381 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.381 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.381 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.381 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1948 4.d 4:b5a463bf:::smithi11939506-20:head [sparse-read 0~2642181] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:13.381 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:13.381 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.988308574000001, 2024-09-18T11:31:13.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010701952000000001, 2024-09-18T11:31:13.381 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.381 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:13.381 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.382 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.382 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.382 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1948 2024-09-18T11:31:13.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.382 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:13.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:13.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:13.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:13.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339563+0000", 2024-09-18T11:31:13.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1950000000000002e-06 2024-09-18T11:31:13.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339569+0000", 2024-09-18T11:31:13.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5459999999999999e-06 2024-09-18T11:31:13.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339572+0000", 2024-09-18T11:31:13.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.337e-06 2024-09-18T11:31:13.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349693+0000", 2024-09-18T11:31:13.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010120987999999999 2024-09-18T11:31:13.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349816+0000", 2024-09-18T11:31:13.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012241199999999999 2024-09-18T11:31:13.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350263+0000", 2024-09-18T11:31:13.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00044787199999999998 2024-09-18T11:31:13.385 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.385 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.385 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.385 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:129 4.5 e69/22 4:a087a566:::smithi11939506-46:19 v 69'95, mlcod=65'92)", 2024-09-18T11:31:13.385 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.343498+0000", 2024-09-18T11:31:13.385 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.984371928000002, 2024-09-18T11:31:13.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068162960000000003, 2024-09-18T11:31:13.386 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.386 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.386 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343498+0000", 2024-09-18T11:31:13.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343498+0000", 2024-09-18T11:31:13.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343497+0000", 2024-09-18T11:31:13.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:13.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343504+0000", 2024-09-18T11:31:13.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5799999999999997e-06 2024-09-18T11:31:13.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343513+0000", 2024-09-18T11:31:13.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7940000000000008e-06 2024-09-18T11:31:13.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.343518+0000", 2024-09-18T11:31:13.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2719999999999997e-06 2024-09-18T11:31:13.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345616+0000", 2024-09-18T11:31:13.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020977410000000002 2024-09-18T11:31:13.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345641+0000", 2024-09-18T11:31:13.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5539000000000001e-05 2024-09-18T11:31:13.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350279+0000", 2024-09-18T11:31:13.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046379560000000004 2024-09-18T11:31:13.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350314+0000", 2024-09-18T11:31:13.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5083999999999998e-05 2024-09-18T11:31:13.389 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.389 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.390 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.390 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:131 4.5 e69/22 4:aa575018:::smithi11939506-11:17 v 69'97, mlcod=65'92)", 2024-09-18T11:31:13.390 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.346049+0000", 2024-09-18T11:31:13.390 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.981821514, 2024-09-18T11:31:13.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0077275240000000004, 2024-09-18T11:31:13.390 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.390 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.390 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346049+0000", 2024-09-18T11:31:13.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346049+0000", 2024-09-18T11:31:13.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346047+0000", 2024-09-18T11:31:13.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:13.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346058+0000", 2024-09-18T11:31:13.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0463e-05 2024-09-18T11:31:13.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346069+0000", 2024-09-18T11:31:13.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1708999999999999e-05 2024-09-18T11:31:13.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346076+0000", 2024-09-18T11:31:13.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2450000000000003e-06 2024-09-18T11:31:13.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349616+0000", 2024-09-18T11:31:13.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035406550000000002 2024-09-18T11:31:13.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349638+0000", 2024-09-18T11:31:13.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2098000000000001e-05 2024-09-18T11:31:13.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.353735+0000", 2024-09-18T11:31:13.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004096237 2024-09-18T11:31:13.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.353776+0000", 2024-09-18T11:31:13.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1530999999999999e-05 2024-09-18T11:31:13.397 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.397 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.397 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.397 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2128 4.1 4:8b9a985d:::smithi11939506-23:16 [read 0~3512630,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T11:31:13.397 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:55.710141+0000", 2024-09-18T11:31:13.397 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.6177286, 2024-09-18T11:31:13.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068000389999999999, 2024-09-18T11:31:13.397 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.397 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:13.398 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.398 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.398 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.398 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2128 2024-09-18T11:31:13.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.398 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710141+0000", 2024-09-18T11:31:13.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710141+0000", 2024-09-18T11:31:13.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710141+0000", 2024-09-18T11:31:13.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:13.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710162+0000", 2024-09-18T11:31:13.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1492e-05 2024-09-18T11:31:13.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710169+0000", 2024-09-18T11:31:13.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0029999999999997e-06 2024-09-18T11:31:13.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710174+0000", 2024-09-18T11:31:13.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1250000000000003e-06 2024-09-18T11:31:13.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.715332+0000", 2024-09-18T11:31:13.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051581680000000003 2024-09-18T11:31:13.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.715431+0000", 2024-09-18T11:31:13.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9693999999999998e-05 2024-09-18T11:31:13.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.716942+0000", 2024-09-18T11:31:13.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001510033 2024-09-18T11:31:13.401 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.401 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.402 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.402 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2129 4.1 4:8b9a985d:::smithi11939506-23:16 [read 0~3512630,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T11:31:13.402 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:55.710202+0000", 2024-09-18T11:31:13.402 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.617668253000001, 2024-09-18T11:31:13.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0078396100000000003, 2024-09-18T11:31:13.402 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.402 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:13.402 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.402 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.402 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.402 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2129 2024-09-18T11:31:13.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.403 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710202+0000", 2024-09-18T11:31:13.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710202+0000", 2024-09-18T11:31:13.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710202+0000", 2024-09-18T11:31:13.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:13.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710205+0000", 2024-09-18T11:31:13.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.382e-06 2024-09-18T11:31:13.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710212+0000", 2024-09-18T11:31:13.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770000000000004e-06 2024-09-18T11:31:13.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710217+0000", 2024-09-18T11:31:13.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1680000000000003e-06 2024-09-18T11:31:13.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.716963+0000", 2024-09-18T11:31:13.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0067462449999999997 2024-09-18T11:31:13.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.717033+0000", 2024-09-18T11:31:13.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9643999999999999e-05 2024-09-18T11:31:13.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.718041+0000", 2024-09-18T11:31:13.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0010086019999999999 2024-09-18T11:31:13.406 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.406 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.406 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.407 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2175 4.e 4:73745217:::smithi11939506-6:head [write 742909~464035,stat] snapc 1e=[1e,1c,1b,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-18T11:31:13.407 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:56.426542+0000", 2024-09-18T11:31:13.407 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 16.901328211999999, 2024-09-18T11:31:13.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070437850000000003, 2024-09-18T11:31:13.407 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.407 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.407 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.407 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.407 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.407 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2175 2024-09-18T11:31:13.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.407 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.426542+0000", 2024-09-18T11:31:13.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.426542+0000", 2024-09-18T11:31:13.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.426540+0000", 2024-09-18T11:31:13.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:13.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.426987+0000", 2024-09-18T11:31:13.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00044653899999999997 2024-09-18T11:31:13.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.427005+0000", 2024-09-18T11:31:13.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8669e-05 2024-09-18T11:31:13.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.427017+0000", 2024-09-18T11:31:13.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1666e-05 2024-09-18T11:31:13.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.427065+0000", 2024-09-18T11:31:13.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8071000000000001e-05 2024-09-18T11:31:13.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.427134+0000", 2024-09-18T11:31:13.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8736000000000001e-05 2024-09-18T11:31:13.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 6", 2024-09-18T11:31:13.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.427252+0000", 2024-09-18T11:31:13.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000118402 2024-09-18T11:31:13.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:13.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.428537+0000", 2024-09-18T11:31:13.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001284456 2024-09-18T11:31:13.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:13.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.433512+0000", 2024-09-18T11:31:13.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049752499999999996 2024-09-18T11:31:13.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.433545+0000", 2024-09-18T11:31:13.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2529e-05 2024-09-18T11:31:13.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:56.433586+0000", 2024-09-18T11:31:13.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1087999999999997e-05 2024-09-18T11:31:13.412 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.412 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.412 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.412 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:172 4.5 e78/22 4:a087a566:::smithi11939506-46:19 v 78'135, mlcod=77'132)", 2024-09-18T11:31:13.412 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:13.412 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 11.942046561, 2024-09-18T11:31:13.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010938658, 2024-09-18T11:31:13.412 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.412 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.413 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:13.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:13.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385823+0000", 2024-09-18T11:31:13.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:13.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385830+0000", 2024-09-18T11:31:13.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2790000000000001e-06 2024-09-18T11:31:13.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385843+0000", 2024-09-18T11:31:13.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2785e-05 2024-09-18T11:31:13.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385848+0000", 2024-09-18T11:31:13.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5570000000000003e-06 2024-09-18T11:31:13.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385914+0000", 2024-09-18T11:31:13.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5168999999999999e-05 2024-09-18T11:31:13.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385930+0000", 2024-09-18T11:31:13.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6884999999999998e-05 2024-09-18T11:31:13.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396721+0000", 2024-09-18T11:31:13.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010790137 2024-09-18T11:31:13.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396762+0000", 2024-09-18T11:31:13.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1635999999999999e-05 2024-09-18T11:31:13.416 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.416 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.416 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.417 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:31:13.417 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:13.417 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 11.941553809, 2024-09-18T11:31:13.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:31:13.417 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.417 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.417 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:13.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:13.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:13.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:13.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:31:13.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:31:13.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:31:13.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:31:13.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:31:13.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:31:13.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:31:13.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:31:13.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:31:13.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:31:13.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:31:13.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:31:13.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:31:13.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:31:13.421 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.421 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.421 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.421 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:13.421 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:13.421 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.3554412029999998, 2024-09-18T11:31:13.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:31:13.422 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.422 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.422 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.422 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.422 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.422 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:31:13.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.422 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:13.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:13.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:31:13.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:13.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:31:13.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:31:13.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:31:13.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:31:13.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:31:13.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:31:13.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:31:13.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:31:13.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:31:13.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:31:13.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:13.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:31:13.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:31:13.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:13.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:31:13.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:31:13.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:13.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:31:13.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:31:13.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:31:13.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:31:13.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:31:13.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:31:13.427 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.427 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.427 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.427 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2505 4.1 4:87078dfe:::smithi11939506-14:head [write 497540~612664,stat] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:13.427 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.992775+0000", 2024-09-18T11:31:13.427 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.3350952170000001, 2024-09-18T11:31:13.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076090139999999999, 2024-09-18T11:31:13.427 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.428 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.428 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.428 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.428 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.428 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2505 2024-09-18T11:31:13.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.428 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.992775+0000", 2024-09-18T11:31:13.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.992775+0000", 2024-09-18T11:31:13.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.992773+0000", 2024-09-18T11:31:13.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:13.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993364+0000", 2024-09-18T11:31:13.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059123300000000004 2024-09-18T11:31:13.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993377+0000", 2024-09-18T11:31:13.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3464e-05 2024-09-18T11:31:13.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993382+0000", 2024-09-18T11:31:13.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2440000000000004e-06 2024-09-18T11:31:13.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993460+0000", 2024-09-18T11:31:13.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8584000000000005e-05 2024-09-18T11:31:13.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993530+0000", 2024-09-18T11:31:13.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0060000000000003e-05 2024-09-18T11:31:13.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:13.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993650+0000", 2024-09-18T11:31:13.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120079 2024-09-18T11:31:13.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:13.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.998802+0000", 2024-09-18T11:31:13.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005152106 2024-09-18T11:31:13.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:13.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.000318+0000", 2024-09-18T11:31:13.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015154719999999999 2024-09-18T11:31:13.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.000352+0000", 2024-09-18T11:31:13.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4418000000000001e-05 2024-09-18T11:31:13.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.000384+0000", 2024-09-18T11:31:13.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1594000000000003e-05 2024-09-18T11:31:13.433 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.433 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.433 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.433 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2525 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865577,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:13.434 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173466+0000", 2024-09-18T11:31:13.434 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.1544045110000001, 2024-09-18T11:31:13.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075147870000000002, 2024-09-18T11:31:13.434 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.434 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:13.434 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.434 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.434 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.434 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2525 2024-09-18T11:31:13.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.434 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173466+0000", 2024-09-18T11:31:13.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173466+0000", 2024-09-18T11:31:13.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173465+0000", 2024-09-18T11:31:13.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:13.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173469+0000", 2024-09-18T11:31:13.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7299999999999999e-06 2024-09-18T11:31:13.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173476+0000", 2024-09-18T11:31:13.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7750000000000004e-06 2024-09-18T11:31:13.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173480+0000", 2024-09-18T11:31:13.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1810000000000003e-06 2024-09-18T11:31:13.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.180013+0000", 2024-09-18T11:31:13.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065332419999999999 2024-09-18T11:31:13.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.180115+0000", 2024-09-18T11:31:13.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102111 2024-09-18T11:31:13.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.180980+0000", 2024-09-18T11:31:13.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00086527100000000003 2024-09-18T11:31:13.438 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.438 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.438 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.438 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.438 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.438 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:13.440 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:31:13.440 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:13.438+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:13.467 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T11:31:13.538 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:13.538 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:13.538 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:13.538 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:13.540 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:13.539+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:13.546 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:31:13.641 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:13.640+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:13.685 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:13.685 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:13.686 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:13.686 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:13.686 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:13.694 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:31:13.742 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:13.741+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:13.793 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:13.793 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:13.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:13.793 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:13.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.794 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:31:13.794 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:13.794 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 83.764477038999999, 2024-09-18T11:31:13.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073601550000000002, 2024-09-18T11:31:13.794 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.794 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.794 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.794 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:31:13.794 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:31:13.794 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1 2024-09-18T11:31:13.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.795 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:13.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:13.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026524+0000", 2024-09-18T11:31:13.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999905 2024-09-18T11:31:13.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026541+0000", 2024-09-18T11:31:13.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7070000000000001e-05 2024-09-18T11:31:13.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026554+0000", 2024-09-18T11:31:13.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.288e-05 2024-09-18T11:31:13.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.797 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.797 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026564+0000", 2024-09-18T11:31:13.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0197000000000001e-05 2024-09-18T11:31:13.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.797 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.797 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026826+0000", 2024-09-18T11:31:13.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026212400000000003 2024-09-18T11:31:13.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:31:13.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026839+0000", 2024-09-18T11:31:13.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2831e-05 2024-09-18T11:31:13.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028605+0000", 2024-09-18T11:31:13.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001765794 2024-09-18T11:31:13.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028709+0000", 2024-09-18T11:31:13.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104348 2024-09-18T11:31:13.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:13.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028824+0000", 2024-09-18T11:31:13.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011527399999999999 2024-09-18T11:31:13.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:13.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030319+0000", 2024-09-18T11:31:13.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001494439 2024-09-18T11:31:13.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:13.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033805+0000", 2024-09-18T11:31:13.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034857820000000002 2024-09-18T11:31:13.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033856+0000", 2024-09-18T11:31:13.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1687999999999997e-05 2024-09-18T11:31:13.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033894+0000", 2024-09-18T11:31:13.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7481000000000002e-05 2024-09-18T11:31:13.800 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.800 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.800 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.801 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T11:31:13.801 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:31:13.801 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 81.745198607999995, 2024-09-18T11:31:13.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055107869999999996, 2024-09-18T11:31:13.801 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.801 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.801 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.801 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:31:13.801 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:31:13.801 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 7 2024-09-18T11:31:13.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.801 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:31:13.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:31:13.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045810+0000", 2024-09-18T11:31:13.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:13.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045820+0000", 2024-09-18T11:31:13.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0215e-05 2024-09-18T11:31:13.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045838+0000", 2024-09-18T11:31:13.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7849000000000001e-05 2024-09-18T11:31:13.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.804 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045845+0000", 2024-09-18T11:31:13.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4229999999999999e-06 2024-09-18T11:31:13.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.804 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045898+0000", 2024-09-18T11:31:13.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2537999999999997e-05 2024-09-18T11:31:13.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:13.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045904+0000", 2024-09-18T11:31:13.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4019999999999999e-06 2024-09-18T11:31:13.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047663+0000", 2024-09-18T11:31:13.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017584079999999999 2024-09-18T11:31:13.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047736+0000", 2024-09-18T11:31:13.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3509000000000004e-05 2024-09-18T11:31:13.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:13.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047857+0000", 2024-09-18T11:31:13.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012037800000000001 2024-09-18T11:31:13.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:13.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.050868+0000", 2024-09-18T11:31:13.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030111420000000001 2024-09-18T11:31:13.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:13.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051239+0000", 2024-09-18T11:31:13.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037141000000000002 2024-09-18T11:31:13.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051288+0000", 2024-09-18T11:31:13.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9051000000000002e-05 2024-09-18T11:31:13.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051323+0000", 2024-09-18T11:31:13.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4545000000000002e-05 2024-09-18T11:31:13.808 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.808 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.808 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.808 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1118 3.2 e38/21 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head v 38'76, mlcod=38'74)", 2024-09-18T11:31:13.808 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:13.808 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.843066886999999, 2024-09-18T11:31:13.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006048688, 2024-09-18T11:31:13.809 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.809 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.809 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:13.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:13.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.810 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947941+0000", 2024-09-18T11:31:13.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:13.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.810 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949868+0000", 2024-09-18T11:31:13.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019267659999999999 2024-09-18T11:31:13.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.810 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949893+0000", 2024-09-18T11:31:13.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5239999999999999e-05 2024-09-18T11:31:13.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949901+0000", 2024-09-18T11:31:13.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8420000000000005e-06 2024-09-18T11:31:13.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949954+0000", 2024-09-18T11:31:13.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3464999999999997e-05 2024-09-18T11:31:13.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949974+0000", 2024-09-18T11:31:13.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9731000000000001e-05 2024-09-18T11:31:13.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.812 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953948+0000", 2024-09-18T11:31:13.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039735489999999998 2024-09-18T11:31:13.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.812 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953992+0000", 2024-09-18T11:31:13.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4700999999999998e-05 2024-09-18T11:31:13.812 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.813 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.813 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.813 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1662 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'73, mlcod=51'69)", 2024-09-18T11:31:13.813 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:13.813 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.429461852000003, 2024-09-18T11:31:13.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059408539999999998, 2024-09-18T11:31:13.813 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.813 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.813 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:13.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:13.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:13.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:13.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.815 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361914+0000", 2024-09-18T11:31:13.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036569 2024-09-18T11:31:13.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.815 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361926+0000", 2024-09-18T11:31:13.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1960999999999999e-05 2024-09-18T11:31:13.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.815 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361937+0000", 2024-09-18T11:31:13.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0876999999999999e-05 2024-09-18T11:31:13.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361986+0000", 2024-09-18T11:31:13.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8652000000000001e-05 2024-09-18T11:31:13.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.362012+0000", 2024-09-18T11:31:13.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5854999999999999e-05 2024-09-18T11:31:13.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367460+0000", 2024-09-18T11:31:13.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054482100000000002 2024-09-18T11:31:13.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367490+0000", 2024-09-18T11:31:13.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9909000000000001e-05 2024-09-18T11:31:13.817 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.817 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.817 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.817 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1663 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'74, mlcod=51'69)", 2024-09-18T11:31:13.818 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364521+0000", 2024-09-18T11:31:13.818 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.426489854000003, 2024-09-18T11:31:13.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048541399999999998, 2024-09-18T11:31:13.818 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.818 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.818 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364521+0000", 2024-09-18T11:31:13.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364521+0000", 2024-09-18T11:31:13.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364520+0000", 2024-09-18T11:31:13.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:13.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364616+0000", 2024-09-18T11:31:13.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5569999999999995e-05 2024-09-18T11:31:13.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364625+0000", 2024-09-18T11:31:13.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3379999999999994e-06 2024-09-18T11:31:13.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364629+0000", 2024-09-18T11:31:13.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2490000000000003e-06 2024-09-18T11:31:13.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364664+0000", 2024-09-18T11:31:13.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5132999999999997e-05 2024-09-18T11:31:13.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364687+0000", 2024-09-18T11:31:13.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2691999999999999e-05 2024-09-18T11:31:13.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369349+0000", 2024-09-18T11:31:13.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046622950000000003 2024-09-18T11:31:13.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369375+0000", 2024-09-18T11:31:13.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5534e-05 2024-09-18T11:31:13.822 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.822 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.822 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.822 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1664 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'75, mlcod=51'69)", 2024-09-18T11:31:13.822 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:31:13.822 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.424123344999998, 2024-09-18T11:31:13.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051832229999999998, 2024-09-18T11:31:13.822 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.822 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.822 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:31:13.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:31:13.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366887+0000", 2024-09-18T11:31:13.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:13.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366894+0000", 2024-09-18T11:31:13.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2219999999999996e-06 2024-09-18T11:31:13.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366906+0000", 2024-09-18T11:31:13.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2278e-05 2024-09-18T11:31:13.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366912+0000", 2024-09-18T11:31:13.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2430000000000002e-06 2024-09-18T11:31:13.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366952+0000", 2024-09-18T11:31:13.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9428999999999998e-05 2024-09-18T11:31:13.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366972+0000", 2024-09-18T11:31:13.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0554999999999999e-05 2024-09-18T11:31:13.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372045+0000", 2024-09-18T11:31:13.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050727319999999999 2024-09-18T11:31:13.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372071+0000", 2024-09-18T11:31:13.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5477e-05 2024-09-18T11:31:13.826 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.826 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.826 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.827 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:269 4.7 e58/22 4:e73ba68f:::smithi11939506-1:13 v 58'72, mlcod=57'69)", 2024-09-18T11:31:13.827 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:13.827 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 34.525629954999999, 2024-09-18T11:31:13.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065512569999999996, 2024-09-18T11:31:13.827 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.827 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.827 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:13.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:13.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265380+0000", 2024-09-18T11:31:13.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:13.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265388+0000", 2024-09-18T11:31:13.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.174e-06 2024-09-18T11:31:13.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265398+0000", 2024-09-18T11:31:13.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2599999999999994e-06 2024-09-18T11:31:13.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265402+0000", 2024-09-18T11:31:13.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0289999999999999e-06 2024-09-18T11:31:13.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265434+0000", 2024-09-18T11:31:13.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2511000000000001e-05 2024-09-18T11:31:13.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265447+0000", 2024-09-18T11:31:13.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2947e-05 2024-09-18T11:31:13.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271892+0000", 2024-09-18T11:31:13.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064445930000000002 2024-09-18T11:31:13.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271932+0000", 2024-09-18T11:31:13.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0303999999999999e-05 2024-09-18T11:31:13.831 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.831 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.831 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.831 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1788 4.8 4:1898b15d:::smithi11939506-25:head [write 453822~797061] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:13.831 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:13.831 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 30.875197371999999, 2024-09-18T11:31:13.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094305159999999999, 2024-09-18T11:31:13.831 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.832 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.832 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.832 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.832 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.832 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1788 2024-09-18T11:31:13.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.832 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:13.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:13.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915811+0000", 2024-09-18T11:31:13.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:13.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916608+0000", 2024-09-18T11:31:13.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079635000000000003 2024-09-18T11:31:13.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916624+0000", 2024-09-18T11:31:13.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6735000000000002e-05 2024-09-18T11:31:13.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916630+0000", 2024-09-18T11:31:13.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5709999999999997e-06 2024-09-18T11:31:13.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916674+0000", 2024-09-18T11:31:13.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4540999999999999e-05 2024-09-18T11:31:13.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916751+0000", 2024-09-18T11:31:13.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6695999999999999e-05 2024-09-18T11:31:13.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:13.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916872+0000", 2024-09-18T11:31:13.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120648 2024-09-18T11:31:13.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:13.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.919076+0000", 2024-09-18T11:31:13.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022046829999999998 2024-09-18T11:31:13.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:13.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925151+0000", 2024-09-18T11:31:13.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060740239999999999 2024-09-18T11:31:13.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925198+0000", 2024-09-18T11:31:13.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7896999999999999e-05 2024-09-18T11:31:13.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925244+0000", 2024-09-18T11:31:13.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5398999999999997e-05 2024-09-18T11:31:13.837 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.837 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.837 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.837 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1789 4.8 4:1898b15d:::smithi11939506-25:head [write 1760371~653366] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:13.837 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.921860+0000", 2024-09-18T11:31:13.837 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 30.869150687000001, 2024-09-18T11:31:13.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055904329999999997, 2024-09-18T11:31:13.838 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.838 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.838 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.838 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.838 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.838 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1789 2024-09-18T11:31:13.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.838 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.921860+0000", 2024-09-18T11:31:13.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.921860+0000", 2024-09-18T11:31:13.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.921859+0000", 2024-09-18T11:31:13.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:13.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922436+0000", 2024-09-18T11:31:13.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00057708300000000004 2024-09-18T11:31:13.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922450+0000", 2024-09-18T11:31:13.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4575e-05 2024-09-18T11:31:13.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922458+0000", 2024-09-18T11:31:13.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:31:13.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922508+0000", 2024-09-18T11:31:13.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0102999999999998e-05 2024-09-18T11:31:13.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922607+0000", 2024-09-18T11:31:13.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8813000000000004e-05 2024-09-18T11:31:13.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:13.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922814+0000", 2024-09-18T11:31:13.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000207488 2024-09-18T11:31:13.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:13.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925504+0000", 2024-09-18T11:31:13.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026899200000000002 2024-09-18T11:31:13.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:13.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927350+0000", 2024-09-18T11:31:13.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00184575 2024-09-18T11:31:13.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927404+0000", 2024-09-18T11:31:13.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4147000000000002e-05 2024-09-18T11:31:13.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927450+0000", 2024-09-18T11:31:13.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6480000000000002e-05 2024-09-18T11:31:13.843 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.843 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.843 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.843 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1790 4.8 4:1898b15d:::smithi11939506-25:head [setxattr _header (57) in=7b,truncate 2453822] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:13.843 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.922467+0000", 2024-09-18T11:31:13.843 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 30.868543266, 2024-09-18T11:31:13.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0052363219999999999, 2024-09-18T11:31:13.844 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.844 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.844 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.844 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.844 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.844 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1790 2024-09-18T11:31:13.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.844 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922467+0000", 2024-09-18T11:31:13.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922467+0000", 2024-09-18T11:31:13.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922467+0000", 2024-09-18T11:31:13.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:13.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922470+0000", 2024-09-18T11:31:13.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2459999999999999e-06 2024-09-18T11:31:13.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922474+0000", 2024-09-18T11:31:13.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:31:13.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922476+0000", 2024-09-18T11:31:13.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3410000000000001e-06 2024-09-18T11:31:13.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.924488+0000", 2024-09-18T11:31:13.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002012425 2024-09-18T11:31:13.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.924623+0000", 2024-09-18T11:31:13.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013435699999999999 2024-09-18T11:31:13.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:13.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.924870+0000", 2024-09-18T11:31:13.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00024748199999999999 2024-09-18T11:31:13.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:13.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927574+0000", 2024-09-18T11:31:13.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027032800000000002 2024-09-18T11:31:13.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:13.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927628+0000", 2024-09-18T11:31:13.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3909000000000002e-05 2024-09-18T11:31:13.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927661+0000", 2024-09-18T11:31:13.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3467999999999999e-05 2024-09-18T11:31:13.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927704+0000", 2024-09-18T11:31:13.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2749e-05 2024-09-18T11:31:13.849 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.849 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.849 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.849 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1791 4.8 4:1898b15d:::smithi11939506-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:13.849 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.922500+0000", 2024-09-18T11:31:13.849 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 30.868510595, 2024-09-18T11:31:13.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005360259, 2024-09-18T11:31:13.849 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.849 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:13.849 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.850 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.850 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.850 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1791 2024-09-18T11:31:13.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.850 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922500+0000", 2024-09-18T11:31:13.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922500+0000", 2024-09-18T11:31:13.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922500+0000", 2024-09-18T11:31:13.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:13.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922503+0000", 2024-09-18T11:31:13.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7009999999999998e-06 2024-09-18T11:31:13.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922509+0000", 2024-09-18T11:31:13.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1710000000000001e-06 2024-09-18T11:31:13.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922511+0000", 2024-09-18T11:31:13.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:31:13.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925283+0000", 2024-09-18T11:31:13.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027722659999999998 2024-09-18T11:31:13.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:13.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925376+0000", 2024-09-18T11:31:13.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3018999999999998e-05 2024-09-18T11:31:13.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927736+0000", 2024-09-18T11:31:13.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023600990000000001 2024-09-18T11:31:13.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927798+0000", 2024-09-18T11:31:13.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1197000000000004e-05 2024-09-18T11:31:13.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927860+0000", 2024-09-18T11:31:13.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2725999999999995e-05 2024-09-18T11:31:13.854 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.854 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.854 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.854 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1934 4.2 e65/22 4:425afe9c:::smithi11939506-29:head v 65'44, mlcod=60'40)", 2024-09-18T11:31:13.854 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:13.854 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.491018602, 2024-09-18T11:31:13.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064585119999999996, 2024-09-18T11:31:13.855 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.855 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.855 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:13.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:13.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299990+0000", 2024-09-18T11:31:13.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:13.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300789+0000", 2024-09-18T11:31:13.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079851499999999997 2024-09-18T11:31:13.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300806+0000", 2024-09-18T11:31:13.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7096999999999999e-05 2024-09-18T11:31:13.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300813+0000", 2024-09-18T11:31:13.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5079999999999999e-06 2024-09-18T11:31:13.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300870+0000", 2024-09-18T11:31:13.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7178e-05 2024-09-18T11:31:13.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300897+0000", 2024-09-18T11:31:13.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6284000000000001e-05 2024-09-18T11:31:13.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306410+0000", 2024-09-18T11:31:13.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055135210000000004 2024-09-18T11:31:13.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306451+0000", 2024-09-18T11:31:13.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0379000000000001e-05 2024-09-18T11:31:13.859 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.859 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.859 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.859 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2119 4.7 e73/22 4:e0d95e10:::smithi11939506-22:head v 73'89, mlcod=73'86)", 2024-09-18T11:31:13.859 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:55.701097+0000", 2024-09-18T11:31:13.859 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.089913956, 2024-09-18T11:31:13.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056939520000000004, 2024-09-18T11:31:13.859 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.859 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.859 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701097+0000", 2024-09-18T11:31:13.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701097+0000", 2024-09-18T11:31:13.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701095+0000", 2024-09-18T11:31:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701919+0000", 2024-09-18T11:31:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000824181 2024-09-18T11:31:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701938+0000", 2024-09-18T11:31:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8614000000000002e-05 2024-09-18T11:31:13.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701947+0000", 2024-09-18T11:31:13.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0699999999999996e-06 2024-09-18T11:31:13.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701992+0000", 2024-09-18T11:31:13.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5614000000000001e-05 2024-09-18T11:31:13.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.702012+0000", 2024-09-18T11:31:13.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9639999999999998e-05 2024-09-18T11:31:13.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.706755+0000", 2024-09-18T11:31:13.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004742708 2024-09-18T11:31:13.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.706791+0000", 2024-09-18T11:31:13.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6106999999999998e-05 2024-09-18T11:31:13.863 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.863 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.864 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.864 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2323 4.2 e77/22 4:425afe9c:::smithi11939506-29:head v 77'58, mlcod=77'56)", 2024-09-18T11:31:13.864 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.194236+0000", 2024-09-18T11:31:13.864 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.596774643, 2024-09-18T11:31:13.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054717079999999996, 2024-09-18T11:31:13.864 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.864 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.864 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194236+0000", 2024-09-18T11:31:13.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194236+0000", 2024-09-18T11:31:13.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194235+0000", 2024-09-18T11:31:13.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:13.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194777+0000", 2024-09-18T11:31:13.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00054227599999999996 2024-09-18T11:31:13.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194793+0000", 2024-09-18T11:31:13.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5662e-05 2024-09-18T11:31:13.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194800+0000", 2024-09-18T11:31:13.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9879999999999998e-06 2024-09-18T11:31:13.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194859+0000", 2024-09-18T11:31:13.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8876999999999999e-05 2024-09-18T11:31:13.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.194889+0000", 2024-09-18T11:31:13.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0409999999999999e-05 2024-09-18T11:31:13.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.199659+0000", 2024-09-18T11:31:13.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047701330000000002 2024-09-18T11:31:13.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.199708+0000", 2024-09-18T11:31:13.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8242999999999999e-05 2024-09-18T11:31:13.868 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.868 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.868 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.868 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2452 4.3 4:ca2b2309:::smithi11939506-44:head [read 0~3462242,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:31:13.868 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:31:13.868 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.7542432449999996, 2024-09-18T11:31:13.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060906450000000004, 2024-09-18T11:31:13.868 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.869 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:13.869 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.869 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.869 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.869 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2452 2024-09-18T11:31:13.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.869 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:31:13.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:31:13.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036765+0000", 2024-09-18T11:31:13.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:13.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036777+0000", 2024-09-18T11:31:13.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1422e-05 2024-09-18T11:31:13.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036793+0000", 2024-09-18T11:31:13.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6575999999999999e-05 2024-09-18T11:31:13.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036800+0000", 2024-09-18T11:31:13.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9940000000000003e-06 2024-09-18T11:31:13.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:13.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5847999999999999e-05 2024-09-18T11:31:13.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:13.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036853+0000", 2024-09-18T11:31:13.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0119999999999999e-06 2024-09-18T11:31:13.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038527+0000", 2024-09-18T11:31:13.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016732730000000001 2024-09-18T11:31:13.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038580+0000", 2024-09-18T11:31:13.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3443999999999999e-05 2024-09-18T11:31:13.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042858+0000", 2024-09-18T11:31:13.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042781049999999999 2024-09-18T11:31:13.873 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.873 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.873 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.873 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2453 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:31:13.873 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:31:13.874 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.7542010130000003, 2024-09-18T11:31:13.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063798429999999996, 2024-09-18T11:31:13.874 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.874 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:13.874 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.874 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.874 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.874 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2453 2024-09-18T11:31:13.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.874 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:31:13.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:31:13.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036809+0000", 2024-09-18T11:31:13.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:13.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036812+0000", 2024-09-18T11:31:13.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8339999999999999e-06 2024-09-18T11:31:13.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036819+0000", 2024-09-18T11:31:13.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9890000000000003e-06 2024-09-18T11:31:13.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036823+0000", 2024-09-18T11:31:13.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8099999999999999e-06 2024-09-18T11:31:13.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036912+0000", 2024-09-18T11:31:13.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9083000000000001e-05 2024-09-18T11:31:13.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:13.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036932+0000", 2024-09-18T11:31:13.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0222000000000001e-05 2024-09-18T11:31:13.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042896+0000", 2024-09-18T11:31:13.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059637240000000001 2024-09-18T11:31:13.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042952+0000", 2024-09-18T11:31:13.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5877000000000001e-05 2024-09-18T11:31:13.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043190+0000", 2024-09-18T11:31:13.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00023774500000000001 2024-09-18T11:31:13.878 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.879 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.879 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.879 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2454 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:31:13.879 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:13.879 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.7541650569999998, 2024-09-18T11:31:13.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066404469999999998, 2024-09-18T11:31:13.879 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.879 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:13.879 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.879 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.879 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.879 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2454 2024-09-18T11:31:13.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.880 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:13.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:13.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036845+0000", 2024-09-18T11:31:13.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:13.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036856+0000", 2024-09-18T11:31:13.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1338000000000001e-05 2024-09-18T11:31:13.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036863+0000", 2024-09-18T11:31:13.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4749999999999998e-06 2024-09-18T11:31:13.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036867+0000", 2024-09-18T11:31:13.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3839999999999999e-06 2024-09-18T11:31:13.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036946+0000", 2024-09-18T11:31:13.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9054000000000006e-05 2024-09-18T11:31:13.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:13.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036949+0000", 2024-09-18T11:31:13.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9390000000000002e-06 2024-09-18T11:31:13.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043213+0000", 2024-09-18T11:31:13.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062640090000000001 2024-09-18T11:31:13.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043261+0000", 2024-09-18T11:31:13.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7750999999999997e-05 2024-09-18T11:31:13.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043486+0000", 2024-09-18T11:31:13.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000224985 2024-09-18T11:31:13.884 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.884 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.884 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.884 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2505 4.1 e85/22 4:87078dfe:::smithi11939506-14:head v 85'167, mlcod=85'163)", 2024-09-18T11:31:13.884 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.993736+0000", 2024-09-18T11:31:13.884 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.7972746129999999, 2024-09-18T11:31:13.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056183630000000003, 2024-09-18T11:31:13.884 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.884 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.884 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993736+0000", 2024-09-18T11:31:13.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993736+0000", 2024-09-18T11:31:13.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993735+0000", 2024-09-18T11:31:13.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:13.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.994296+0000", 2024-09-18T11:31:13.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056099399999999999 2024-09-18T11:31:13.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.994313+0000", 2024-09-18T11:31:13.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6982000000000001e-05 2024-09-18T11:31:13.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.994319+0000", 2024-09-18T11:31:13.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0279999999999997e-06 2024-09-18T11:31:13.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.994374+0000", 2024-09-18T11:31:13.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5019999999999998e-05 2024-09-18T11:31:13.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.994401+0000", 2024-09-18T11:31:13.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7104e-05 2024-09-18T11:31:13.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.999314+0000", 2024-09-18T11:31:13.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049127390000000002 2024-09-18T11:31:13.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.999354+0000", 2024-09-18T11:31:13.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0763000000000003e-05 2024-09-18T11:31:13.888 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.888 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.888 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.889 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2516 4.3 4:c70c3162:::smithi11939506-35:head [read 0~2605085,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:13.889 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173302+0000", 2024-09-18T11:31:13.889 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.6177082939999998, 2024-09-18T11:31:13.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050184130000000002, 2024-09-18T11:31:13.889 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.889 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:13.889 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.889 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.889 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.889 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2516 2024-09-18T11:31:13.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.890 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173302+0000", 2024-09-18T11:31:13.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173302+0000", 2024-09-18T11:31:13.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173302+0000", 2024-09-18T11:31:13.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:13.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173307+0000", 2024-09-18T11:31:13.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7539999999999997e-06 2024-09-18T11:31:13.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173317+0000", 2024-09-18T11:31:13.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0013000000000001e-05 2024-09-18T11:31:13.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173334+0000", 2024-09-18T11:31:13.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7430000000000001e-05 2024-09-18T11:31:13.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176876+0000", 2024-09-18T11:31:13.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035424010000000001 2024-09-18T11:31:13.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.177062+0000", 2024-09-18T11:31:13.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000185475 2024-09-18T11:31:13.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178321+0000", 2024-09-18T11:31:13.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001258962 2024-09-18T11:31:13.893 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.893 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.893 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.893 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2553 4.3 4:ca2b2309:::smithi11939506-44:head [write 564768~630013,stat] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-18T11:31:13.893 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:31:13.893 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.5938874240000001, 2024-09-18T11:31:13.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065686879999999996, 2024-09-18T11:31:13.893 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:13.894 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:13.894 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:13.894 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:13.894 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:13.894 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2553 2024-09-18T11:31:13.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.894 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:13.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:13.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:31:13.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:13.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:31:13.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:13.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:13.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197120+0000", 2024-09-18T11:31:13.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:13.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:13.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199000+0000", 2024-09-18T11:31:13.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018792080000000001 2024-09-18T11:31:13.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:13.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199044+0000", 2024-09-18T11:31:13.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4048999999999998e-05 2024-09-18T11:31:13.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:13.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199071+0000", 2024-09-18T11:31:13.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7288e-05 2024-09-18T11:31:13.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:13.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199119+0000", 2024-09-18T11:31:13.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8000999999999997e-05 2024-09-18T11:31:13.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:13.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199208+0000", 2024-09-18T11:31:13.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9209000000000006e-05 2024-09-18T11:31:13.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:13.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199385+0000", 2024-09-18T11:31:13.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000177095 2024-09-18T11:31:13.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:13.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.202282+0000", 2024-09-18T11:31:13.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028970469999999998 2024-09-18T11:31:13.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:13.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203578+0000", 2024-09-18T11:31:13.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012960490000000001 2024-09-18T11:31:13.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:13.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203638+0000", 2024-09-18T11:31:13.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9599000000000003e-05 2024-09-18T11:31:13.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:13.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:13.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:13.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203692+0000", 2024-09-18T11:31:13.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3908e-05 2024-09-18T11:31:13.899 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.899 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.899 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.899 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:13.899 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:13.899 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:13.901 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:31:13.901 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:13.899+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:14.001 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:14.000+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:14.006 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:14.005+0000 7f697aead640 -1 osd.3 90 handle_osd_map injecting map crc failure 2024-09-18T11:31:14.009 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is set 2024-09-18T11:31:14.010 INFO:tasks.rados.rados.0.smithi119.stdout:1257: done (7 left) 2024-09-18T11:31:14.010 INFO:tasks.rados.rados.0.smithi119.stdout:1258: rollback oid 7 current snap is 28 2024-09-18T11:31:14.010 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 7 to 26 2024-09-18T11:31:14.010 INFO:tasks.rados.rados.0.smithi119.stdout:1248: expect (ObjNum 232 snap 27 seq_num 232) 2024-09-18T11:31:14.049 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:14.049 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:14.049 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:14.049 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:14.058 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:31:14.102 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:14.101+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:14.197 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:14.198 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:14.198 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:14.198 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:14.198 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:14.203 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:14.202+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:14.206 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:31:14.304 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:14.303+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:14.332 INFO:tasks.rados.rados.0.smithi119.stdout:1251: finishing set_chunk to oid 50 2024-09-18T11:31:14.332 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 50 set_chunk returned error code -95 offset: 193024 length: 425984 2024-09-18T11:31:14.332 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 0 (ObjNum 183 snap 20 seq_num 183) dirty exists 2024-09-18T11:31:14.332 INFO:tasks.rados.rados.0.smithi119.stdout:1252: finishing write tid 1 to smithi11939506-39 2024-09-18T11:31:14.332 INFO:tasks.rados.rados.0.smithi119.stdout:1252: finishing write tid 2 to smithi11939506-39 2024-09-18T11:31:14.332 INFO:tasks.rados.rados.0.smithi119.stdout:1248: done (7 left) 2024-09-18T11:31:14.332 INFO:tasks.rados.rados.0.smithi119.stdout:1251: done (6 left) 2024-09-18T11:31:14.333 INFO:tasks.rados.rados.0.smithi119.stdout:1259: read oid 32 snap -1 2024-09-18T11:31:14.333 INFO:tasks.rados.rados.0.smithi119.stdout:1259: expect (ObjNum 217 snap 26 seq_num 217) 2024-09-18T11:31:14.333 INFO:tasks.rados.rados.0.smithi119.stdout:1260: read oid 16 snap -1 2024-09-18T11:31:14.333 INFO:tasks.rados.rados.0.smithi119.stdout:1260: expect (ObjNum 215 snap 26 seq_num 215) 2024-09-18T11:31:14.333 INFO:tasks.rados.rados.0.smithi119.stdout:1261: read oid 43 snap 21 2024-09-18T11:31:14.333 INFO:tasks.rados.rados.0.smithi119.stdout:1261: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:31:14.333 INFO:tasks.rados.rados.0.smithi119.stdout:1262: read oid 9 snap 24 2024-09-18T11:31:14.333 INFO:tasks.rados.rados.0.smithi119.stdout:1262: expect (ObjNum 199 snap 23 seq_num 199) 2024-09-18T11:31:14.333 INFO:tasks.rados.rados.0.smithi119.stdout:1263: read oid 28 snap 27 2024-09-18T11:31:14.333 INFO:tasks.rados.rados.0.smithi119.stdout:1263: expect deleted 2024-09-18T11:31:14.333 INFO:tasks.rados.rados.0.smithi119.stdout:1264: set_chunk oid 24 target oid 2024-09-18T11:31:14.345 INFO:tasks.rados.rados.0.smithi119.stdout:1264: set_chunk oid 24 offset: 1410560 length: 168448 target oid a5380f2e11cc7e1b90b0e57b793fcc195d82840d04b333bb015323b024763061 offset: 0 snap -1 2024-09-18T11:31:14.346 INFO:tasks.rados.rados.0.smithi119.stdout:1265: read oid 42 snap -1 2024-09-18T11:31:14.346 INFO:tasks.rados.rados.0.smithi119.stdout:1265: expect (ObjNum 99 snap 6 seq_num 99) 2024-09-18T11:31:14.346 INFO:tasks.rados.rados.0.smithi119.stdout:1266: write (excl) oid 41 current snap is 28 2024-09-18T11:31:14.346 INFO:tasks.rados.rados.0.smithi119.stdout:1254: finishing set_chunk to oid 17 2024-09-18T11:31:14.346 INFO:tasks.rados.rados.0.smithi119.stdout:1254: got expected ERANGE (src dne) 2024-09-18T11:31:14.346 INFO:tasks.rados.rados.0.smithi119.stdout:1252: finishing write tid 3 to smithi11939506-39 2024-09-18T11:31:14.346 INFO:tasks.rados.rados.0.smithi119.stdout:1252: finishing write tid 4 to smithi11939506-39 2024-09-18T11:31:14.346 INFO:tasks.rados.rados.0.smithi119.stdout:1252: finishing write tid 5 to smithi11939506-39 2024-09-18T11:31:14.346 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 39 v 181 (ObjNum 235 snap 28 seq_num 235) dirty exists 2024-09-18T11:31:14.346 INFO:tasks.rados.rados.0.smithi119.stdout:1252: left oid 39 (ObjNum 235 snap 28 seq_num 235) 2024-09-18T11:31:14.346 INFO:tasks.rados.rados.0.smithi119.stdout:1255: expect (ObjNum 174 snap 20 seq_num 174) 2024-09-18T11:31:14.347 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:14.347 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:14.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:14.348 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:14.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.348 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:14.348 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:14.348 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 63.228683445999998, 2024-09-18T11:31:14.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:31:14.348 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.348 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:14.348 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.348 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:14.348 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:14.349 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:31:14.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.349 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:14.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:14.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:31:14.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:14.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:31:14.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:31:14.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:31:14.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:31:14.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:31:14.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:31:14.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:31:14.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:31:14.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:14.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:31:14.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:31:14.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:31:14.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:31:14.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:31:14.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:31:14.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:31:14.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:31:14.353 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.353 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.353 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.353 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:31:14.353 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:14.353 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.675658044999999, 2024-09-18T11:31:14.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:31:14.354 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.354 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:14.354 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.354 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:14.354 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:14.354 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:31:14.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.354 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:14.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:14.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:31:14.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:14.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:31:14.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:31:14.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:31:14.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:31:14.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:31:14.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:31:14.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:31:14.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:31:14.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:31:14.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:31:14.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:31:14.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:31:14.358 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.358 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.358 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.358 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1688 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 19=[19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-18T11:31:14.358 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:14.358 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 34.812503452000001, 2024-09-18T11:31:14.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071575789999999999, 2024-09-18T11:31:14.359 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.359 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.359 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.359 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:14.359 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:14.359 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1688 2024-09-18T11:31:14.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.359 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:14.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:14.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:14.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:14.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532500+0000", 2024-09-18T11:31:14.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5260000000000001e-06 2024-09-18T11:31:14.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532503+0000", 2024-09-18T11:31:14.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:31:14.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532505+0000", 2024-09-18T11:31:14.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.889e-06 2024-09-18T11:31:14.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.534722+0000", 2024-09-18T11:31:14.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002217119 2024-09-18T11:31:14.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.534827+0000", 2024-09-18T11:31:14.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0001049 2024-09-18T11:31:14.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:14.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.535007+0000", 2024-09-18T11:31:14.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017965599999999999 2024-09-18T11:31:14.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539463+0000", 2024-09-18T11:31:14.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004455713 2024-09-18T11:31:14.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539607+0000", 2024-09-18T11:31:14.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014440300000000001 2024-09-18T11:31:14.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539634+0000", 2024-09-18T11:31:14.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7398000000000001e-05 2024-09-18T11:31:14.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539657+0000", 2024-09-18T11:31:14.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2228000000000001e-05 2024-09-18T11:31:14.364 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.364 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.364 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.364 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1759 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2319366,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:14.364 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:31:14.364 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.051043241999999, 2024-09-18T11:31:14.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064323039999999998, 2024-09-18T11:31:14.364 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.365 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:14.365 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.365 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:14.365 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:14.365 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1759 2024-09-18T11:31:14.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.365 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:31:14.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:31:14.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293959+0000", 2024-09-18T11:31:14.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293963+0000", 2024-09-18T11:31:14.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0810000000000004e-06 2024-09-18T11:31:14.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293970+0000", 2024-09-18T11:31:14.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5109999999999997e-06 2024-09-18T11:31:14.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293975+0000", 2024-09-18T11:31:14.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.33e-06 2024-09-18T11:31:14.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294078+0000", 2024-09-18T11:31:14.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102871 2024-09-18T11:31:14.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:14.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294084+0000", 2024-09-18T11:31:14.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5470000000000002e-06 2024-09-18T11:31:14.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.299550+0000", 2024-09-18T11:31:14.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054660799999999999 2024-09-18T11:31:14.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.299608+0000", 2024-09-18T11:31:14.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8307000000000003e-05 2024-09-18T11:31:14.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300392+0000", 2024-09-18T11:31:14.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078306999999999997 2024-09-18T11:31:14.369 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.369 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.369 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.369 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1760 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2319366,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:14.369 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:31:14.370 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.051008902, 2024-09-18T11:31:14.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070132529999999997, 2024-09-18T11:31:14.370 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.370 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:14.370 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.370 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:14.370 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:14.370 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1760 2024-09-18T11:31:14.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.370 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:31:14.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:31:14.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293993+0000", 2024-09-18T11:31:14.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293997+0000", 2024-09-18T11:31:14.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4999999999999999e-06 2024-09-18T11:31:14.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294004+0000", 2024-09-18T11:31:14.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3950000000000003e-06 2024-09-18T11:31:14.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294009+0000", 2024-09-18T11:31:14.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4989999999999996e-06 2024-09-18T11:31:14.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294103+0000", 2024-09-18T11:31:14.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4817999999999999e-05 2024-09-18T11:31:14.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:14.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294110+0000", 2024-09-18T11:31:14.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7240000000000003e-06 2024-09-18T11:31:14.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300406+0000", 2024-09-18T11:31:14.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062964470000000002 2024-09-18T11:31:14.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300457+0000", 2024-09-18T11:31:14.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0915000000000003e-05 2024-09-18T11:31:14.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.301007+0000", 2024-09-18T11:31:14.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00054942299999999995 2024-09-18T11:31:14.374 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.374 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.375 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.375 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1775 4.a 4:588debc2:::smithi11939506-18:head [write 1527054~766050] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:14.375 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:31:14.375 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.448656160999999, 2024-09-18T11:31:14.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070939050000000002, 2024-09-18T11:31:14.375 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.375 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.375 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.375 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:14.375 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:14.375 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1775 2024-09-18T11:31:14.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.376 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:31:14.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:31:14.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896345+0000", 2024-09-18T11:31:14.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:14.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896993+0000", 2024-09-18T11:31:14.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064769999999999997 2024-09-18T11:31:14.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897011+0000", 2024-09-18T11:31:14.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8355999999999998e-05 2024-09-18T11:31:14.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897016+0000", 2024-09-18T11:31:14.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3890000000000004e-06 2024-09-18T11:31:14.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897054+0000", 2024-09-18T11:31:14.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7963000000000002e-05 2024-09-18T11:31:14.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897123+0000", 2024-09-18T11:31:14.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8128999999999998e-05 2024-09-18T11:31:14.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:14.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897224+0000", 2024-09-18T11:31:14.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010168800000000001 2024-09-18T11:31:14.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899515+0000", 2024-09-18T11:31:14.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022910280000000001 2024-09-18T11:31:14.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903333+0000", 2024-09-18T11:31:14.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003817459 2024-09-18T11:31:14.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903388+0000", 2024-09-18T11:31:14.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5651e-05 2024-09-18T11:31:14.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903440+0000", 2024-09-18T11:31:14.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1836999999999999e-05 2024-09-18T11:31:14.380 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.380 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.380 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.381 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1776 4.a 4:588debc2:::smithi11939506-18:head [setxattr _header (57) in=7b,truncate 2420208] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:14.381 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:31:14.381 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.447978349, 2024-09-18T11:31:14.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065840400000000002, 2024-09-18T11:31:14.381 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.381 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.381 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.381 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:14.381 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:14.381 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1776 2024-09-18T11:31:14.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.382 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:31:14.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:31:14.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:31:14.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897026+0000", 2024-09-18T11:31:14.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9579999999999999e-06 2024-09-18T11:31:14.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897030+0000", 2024-09-18T11:31:14.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5370000000000002e-06 2024-09-18T11:31:14.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897032+0000", 2024-09-18T11:31:14.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.17e-06 2024-09-18T11:31:14.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898415+0000", 2024-09-18T11:31:14.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013823189999999999 2024-09-18T11:31:14.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898527+0000", 2024-09-18T11:31:14.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011236800000000001 2024-09-18T11:31:14.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:14.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898714+0000", 2024-09-18T11:31:14.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018703599999999999 2024-09-18T11:31:14.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899683+0000", 2024-09-18T11:31:14.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00096944700000000002 2024-09-18T11:31:14.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903475+0000", 2024-09-18T11:31:14.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037920129999999999 2024-09-18T11:31:14.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903535+0000", 2024-09-18T11:31:14.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9843999999999998e-05 2024-09-18T11:31:14.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903608+0000", 2024-09-18T11:31:14.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2946000000000003e-05 2024-09-18T11:31:14.386 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.386 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.386 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.387 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1777 4.a 4:588debc2:::smithi11939506-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:14.387 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:31:14.387 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.447951709000002, 2024-09-18T11:31:14.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0069010809999999999, 2024-09-18T11:31:14.387 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.387 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:14.387 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.387 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:14.387 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:14.387 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1777 2024-09-18T11:31:14.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.387 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:31:14.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:31:14.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897050+0000", 2024-09-18T11:31:14.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897052+0000", 2024-09-18T11:31:14.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:31:14.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897057+0000", 2024-09-18T11:31:14.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9470000000000002e-06 2024-09-18T11:31:14.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897060+0000", 2024-09-18T11:31:14.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3410000000000001e-06 2024-09-18T11:31:14.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899417+0000", 2024-09-18T11:31:14.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023571170000000002 2024-09-18T11:31:14.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:14.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899501+0000", 2024-09-18T11:31:14.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4160999999999996e-05 2024-09-18T11:31:14.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903661+0000", 2024-09-18T11:31:14.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041599510000000003 2024-09-18T11:31:14.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903790+0000", 2024-09-18T11:31:14.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012941499999999999 2024-09-18T11:31:14.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903952+0000", 2024-09-18T11:31:14.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016146800000000001 2024-09-18T11:31:14.391 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.391 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.392 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.392 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1895 4.5 4:a087a566:::smithi11939506-46:head [read 0~2756315,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:14.392 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:14.392 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.036696203999998, 2024-09-18T11:31:14.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007540377, 2024-09-18T11:31:14.392 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.392 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:14.392 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.392 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:14.392 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:14.392 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1895 2024-09-18T11:31:14.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.393 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:14.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:14.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:14.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308310+0000", 2024-09-18T11:31:14.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0559999999999998e-06 2024-09-18T11:31:14.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308317+0000", 2024-09-18T11:31:14.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7209999999999997e-06 2024-09-18T11:31:14.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308320+0000", 2024-09-18T11:31:14.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.721e-06 2024-09-18T11:31:14.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308447+0000", 2024-09-18T11:31:14.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012645199999999999 2024-09-18T11:31:14.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:14.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308451+0000", 2024-09-18T11:31:14.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6800000000000001e-06 2024-09-18T11:31:14.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.312462+0000", 2024-09-18T11:31:14.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040102139999999998 2024-09-18T11:31:14.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.312585+0000", 2024-09-18T11:31:14.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012388899999999999 2024-09-18T11:31:14.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.315847+0000", 2024-09-18T11:31:14.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003261197 2024-09-18T11:31:14.397 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.397 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.397 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.397 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1924 4.5 4:af4ba75a:::smithi11939506-37:head [write 1829449~675380] snapc 1e=[1e,1c,1b,19,18,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:14.397 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.362228+0000", 2024-09-18T11:31:14.397 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.982774504999998, 2024-09-18T11:31:14.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066182139999999999, 2024-09-18T11:31:14.397 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.397 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.398 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.398 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:14.398 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:14.398 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1924 2024-09-18T11:31:14.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.398 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362228+0000", 2024-09-18T11:31:14.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362228+0000", 2024-09-18T11:31:14.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362227+0000", 2024-09-18T11:31:14.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:14.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362749+0000", 2024-09-18T11:31:14.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00052262399999999996 2024-09-18T11:31:14.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362762+0000", 2024-09-18T11:31:14.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2821e-05 2024-09-18T11:31:14.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362767+0000", 2024-09-18T11:31:14.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5349999999999998e-06 2024-09-18T11:31:14.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362806+0000", 2024-09-18T11:31:14.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9029999999999997e-05 2024-09-18T11:31:14.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362863+0000", 2024-09-18T11:31:14.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7584999999999998e-05 2024-09-18T11:31:14.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:14.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362949+0000", 2024-09-18T11:31:14.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5655000000000006e-05 2024-09-18T11:31:14.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364314+0000", 2024-09-18T11:31:14.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001364961 2024-09-18T11:31:14.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.368771+0000", 2024-09-18T11:31:14.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004457244 2024-09-18T11:31:14.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.368809+0000", 2024-09-18T11:31:14.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8368999999999998e-05 2024-09-18T11:31:14.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.368846+0000", 2024-09-18T11:31:14.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6718000000000003e-05 2024-09-18T11:31:14.403 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.403 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.403 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.403 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1925 4.5 4:af4ba75a:::smithi11939506-37:head [setxattr _header (57) in=7b,truncate 2655506] snapc 1e=[1e,1c,1b,19,18,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:14.403 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:31:14.403 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.982229353000001, 2024-09-18T11:31:14.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064745779999999999, 2024-09-18T11:31:14.403 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.403 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.403 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.403 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:14.403 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:14.403 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1925 2024-09-18T11:31:14.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.404 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:31:14.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:31:14.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:31:14.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362776+0000", 2024-09-18T11:31:14.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.208e-06 2024-09-18T11:31:14.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362780+0000", 2024-09-18T11:31:14.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.771e-06 2024-09-18T11:31:14.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362782+0000", 2024-09-18T11:31:14.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.277e-06 2024-09-18T11:31:14.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.363737+0000", 2024-09-18T11:31:14.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00095483999999999999 2024-09-18T11:31:14.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.363832+0000", 2024-09-18T11:31:14.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5594999999999995e-05 2024-09-18T11:31:14.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:14.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.363940+0000", 2024-09-18T11:31:14.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010738000000000001 2024-09-18T11:31:14.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364577+0000", 2024-09-18T11:31:14.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00063679600000000002 2024-09-18T11:31:14.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369126+0000", 2024-09-18T11:31:14.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0045496440000000003 2024-09-18T11:31:14.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369177+0000", 2024-09-18T11:31:14.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0958000000000002e-05 2024-09-18T11:31:14.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369248+0000", 2024-09-18T11:31:14.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0599000000000006e-05 2024-09-18T11:31:14.408 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.409 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.409 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.409 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1926 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:14.409 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:14.409 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.982205097000001, 2024-09-18T11:31:14.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066190219999999996, 2024-09-18T11:31:14.409 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.409 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:14.409 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.409 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:14.409 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:14.410 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1926 2024-09-18T11:31:14.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.410 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:14.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:14.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:14.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362799+0000", 2024-09-18T11:31:14.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.029e-06 2024-09-18T11:31:14.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362804+0000", 2024-09-18T11:31:14.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3989999999999997e-06 2024-09-18T11:31:14.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362806+0000", 2024-09-18T11:31:14.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3020000000000002e-06 2024-09-18T11:31:14.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364105+0000", 2024-09-18T11:31:14.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001299478 2024-09-18T11:31:14.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:14.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364153+0000", 2024-09-18T11:31:14.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8059000000000002e-05 2024-09-18T11:31:14.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369273+0000", 2024-09-18T11:31:14.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051199540000000003 2024-09-18T11:31:14.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369343+0000", 2024-09-18T11:31:14.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0191999999999995e-05 2024-09-18T11:31:14.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369416+0000", 2024-09-18T11:31:14.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2925999999999999e-05 2024-09-18T11:31:14.414 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.414 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.414 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.414 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1969 4.a 4:588debc2:::smithi11939506-18:18 [read 0~3949612,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:14.414 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:14.415 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.971016208000002, 2024-09-18T11:31:14.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075409190000000001, 2024-09-18T11:31:14.415 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.415 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:14.415 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.415 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:14.415 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:14.415 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1969 2024-09-18T11:31:14.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.415 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:14.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:14.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:14.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373990+0000", 2024-09-18T11:31:14.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1119999999999998e-06 2024-09-18T11:31:14.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373997+0000", 2024-09-18T11:31:14.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4200000000000001e-06 2024-09-18T11:31:14.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.374002+0000", 2024-09-18T11:31:14.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:31:14.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380200+0000", 2024-09-18T11:31:14.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006197302 2024-09-18T11:31:14.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380261+0000", 2024-09-18T11:31:14.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1389999999999993e-05 2024-09-18T11:31:14.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.381527+0000", 2024-09-18T11:31:14.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012660080000000001 2024-09-18T11:31:14.418 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.418 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.419 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.419 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2207 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1410296~761389] snapc 24=[24,1e,1c,1b,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T11:31:14.419 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:14.419 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 16.922505382000001, 2024-09-18T11:31:14.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008410743, 2024-09-18T11:31:14.419 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.419 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.419 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.419 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:14.419 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:14.419 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2207 2024-09-18T11:31:14.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.420 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:14.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:14.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422496+0000", 2024-09-18T11:31:14.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:14.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423190+0000", 2024-09-18T11:31:14.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00069405499999999995 2024-09-18T11:31:14.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423204+0000", 2024-09-18T11:31:14.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:31:14.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423208+0000", 2024-09-18T11:31:14.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:31:14.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423293+0000", 2024-09-18T11:31:14.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4524999999999995e-05 2024-09-18T11:31:14.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423394+0000", 2024-09-18T11:31:14.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101395 2024-09-18T11:31:14.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:14.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423579+0000", 2024-09-18T11:31:14.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018461000000000001 2024-09-18T11:31:14.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.428574+0000", 2024-09-18T11:31:14.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049944960000000002 2024-09-18T11:31:14.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430809+0000", 2024-09-18T11:31:14.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022351820000000001 2024-09-18T11:31:14.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430862+0000", 2024-09-18T11:31:14.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3189999999999997e-05 2024-09-18T11:31:14.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430908+0000", 2024-09-18T11:31:14.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5973999999999997e-05 2024-09-18T11:31:14.424 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.424 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.424 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.425 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2339 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:14.425 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:14.425 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.958200753, 2024-09-18T11:31:14.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011809992, 2024-09-18T11:31:14.425 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.425 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:14.425 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.425 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:14.425 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:14.425 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2339 2024-09-18T11:31:14.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.425 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:14.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:14.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:31:14.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:14.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:31:14.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4710000000000001e-06 2024-09-18T11:31:14.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386824+0000", 2024-09-18T11:31:14.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6104e-05 2024-09-18T11:31:14.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386836+0000", 2024-09-18T11:31:14.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1827999999999999e-05 2024-09-18T11:31:14.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388390+0000", 2024-09-18T11:31:14.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015539499999999999 2024-09-18T11:31:14.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:14.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388443+0000", 2024-09-18T11:31:14.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3785000000000001e-05 2024-09-18T11:31:14.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398238+0000", 2024-09-18T11:31:14.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097947989999999999 2024-09-18T11:31:14.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398315+0000", 2024-09-18T11:31:14.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6909999999999994e-05 2024-09-18T11:31:14.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398612+0000", 2024-09-18T11:31:14.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029671999999999999 2024-09-18T11:31:14.429 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.430 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.430 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.430 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2340 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:14.430 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:14.430 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.958157441000001, 2024-09-18T11:31:14.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012086448, 2024-09-18T11:31:14.430 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.430 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:14.430 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.430 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:14.430 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:14.430 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2340 2024-09-18T11:31:14.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.431 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:14.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:14.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:14.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386848+0000", 2024-09-18T11:31:14.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3069999999999998e-06 2024-09-18T11:31:14.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386855+0000", 2024-09-18T11:31:14.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0979999999999996e-06 2024-09-18T11:31:14.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386859+0000", 2024-09-18T11:31:14.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9360000000000001e-06 2024-09-18T11:31:14.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388466+0000", 2024-09-18T11:31:14.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001607154 2024-09-18T11:31:14.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:14.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388507+0000", 2024-09-18T11:31:14.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1093000000000001e-05 2024-09-18T11:31:14.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398647+0000", 2024-09-18T11:31:14.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010139686 2024-09-18T11:31:14.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398718+0000", 2024-09-18T11:31:14.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1138999999999998e-05 2024-09-18T11:31:14.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398931+0000", 2024-09-18T11:31:14.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021356899999999999 2024-09-18T11:31:14.435 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.435 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.435 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.435 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2341 4.5 4:aa575018:::smithi11939506-11:head [read 0~2487436,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:14.435 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:14.435 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.958139219, 2024-09-18T11:31:14.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013751648999999999, 2024-09-18T11:31:14.435 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.435 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:14.435 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.436 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:14.436 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:14.436 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2341 2024-09-18T11:31:14.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.436 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:14.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:14.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:14.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386866+0000", 2024-09-18T11:31:14.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4249999999999998e-06 2024-09-18T11:31:14.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386872+0000", 2024-09-18T11:31:14.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.003e-06 2024-09-18T11:31:14.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:31:14.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.507e-06 2024-09-18T11:31:14.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388520+0000", 2024-09-18T11:31:14.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001644146 2024-09-18T11:31:14.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:14.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388561+0000", 2024-09-18T11:31:14.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1486999999999998e-05 2024-09-18T11:31:14.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398963+0000", 2024-09-18T11:31:14.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010401353 2024-09-18T11:31:14.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.399046+0000", 2024-09-18T11:31:14.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3141999999999998e-05 2024-09-18T11:31:14.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.400615+0000", 2024-09-18T11:31:14.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015690960000000001 2024-09-18T11:31:14.440 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.440 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.440 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.440 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2529 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:14.440 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:31:14.440 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.1716659549999999, 2024-09-18T11:31:14.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079268120000000001, 2024-09-18T11:31:14.440 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.441 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.441 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.441 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:14.441 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:14.441 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2529 2024-09-18T11:31:14.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.441 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:31:14.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:31:14.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173336+0000", 2024-09-18T11:31:14.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173339+0000", 2024-09-18T11:31:14.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9019999999999999e-06 2024-09-18T11:31:14.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173381+0000", 2024-09-18T11:31:14.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1836999999999999e-05 2024-09-18T11:31:14.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173390+0000", 2024-09-18T11:31:14.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.302e-06 2024-09-18T11:31:14.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174507+0000", 2024-09-18T11:31:14.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001116868 2024-09-18T11:31:14.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174603+0000", 2024-09-18T11:31:14.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5598000000000002e-05 2024-09-18T11:31:14.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:14.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174759+0000", 2024-09-18T11:31:14.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015616299999999999 2024-09-18T11:31:14.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178943+0000", 2024-09-18T11:31:14.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041846180000000002 2024-09-18T11:31:14.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181175+0000", 2024-09-18T11:31:14.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00223215 2024-09-18T11:31:14.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181229+0000", 2024-09-18T11:31:14.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3585999999999998e-05 2024-09-18T11:31:14.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181263+0000", 2024-09-18T11:31:14.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4292999999999998e-05 2024-09-18T11:31:14.446 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.446 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.446 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.446 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2530 4.5 4:af4ba75a:::smithi11939506-37:head [rollback 25] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:14.446 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:31:14.446 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.1716036489999997, 2024-09-18T11:31:14.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080198800000000001, 2024-09-18T11:31:14.446 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.446 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.446 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.447 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:14.447 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:14.447 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2530 2024-09-18T11:31:14.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.447 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:31:14.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:31:14.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173398+0000", 2024-09-18T11:31:14.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:14.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173403+0000", 2024-09-18T11:31:14.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4389999999999999e-06 2024-09-18T11:31:14.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173414+0000", 2024-09-18T11:31:14.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0411000000000001e-05 2024-09-18T11:31:14.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173420+0000", 2024-09-18T11:31:14.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9610000000000002e-06 2024-09-18T11:31:14.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176390+0000", 2024-09-18T11:31:14.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029706289999999998 2024-09-18T11:31:14.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176501+0000", 2024-09-18T11:31:14.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011090300000000001 2024-09-18T11:31:14.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:14.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176827+0000", 2024-09-18T11:31:14.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00032526899999999999 2024-09-18T11:31:14.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178958+0000", 2024-09-18T11:31:14.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021314860000000001 2024-09-18T11:31:14.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181318+0000", 2024-09-18T11:31:14.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002359571 2024-09-18T11:31:14.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181372+0000", 2024-09-18T11:31:14.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3884000000000001e-05 2024-09-18T11:31:14.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181419+0000", 2024-09-18T11:31:14.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7148000000000003e-05 2024-09-18T11:31:14.452 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.452 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.452 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.452 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2531 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:14.452 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:14.452 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.1715775260000001, 2024-09-18T11:31:14.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080687239999999993, 2024-09-18T11:31:14.452 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.452 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.453 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.453 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:14.453 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:14.453 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2531 2024-09-18T11:31:14.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.453 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:14.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:14.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:14.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173429+0000", 2024-09-18T11:31:14.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2989999999999998e-06 2024-09-18T11:31:14.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173436+0000", 2024-09-18T11:31:14.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1799999999999999e-06 2024-09-18T11:31:14.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173441+0000", 2024-09-18T11:31:14.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8810000000000004e-06 2024-09-18T11:31:14.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178267+0000", 2024-09-18T11:31:14.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048257969999999997 2024-09-18T11:31:14.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178388+0000", 2024-09-18T11:31:14.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121779 2024-09-18T11:31:14.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:14.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178599+0000", 2024-09-18T11:31:14.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000210094 2024-09-18T11:31:14.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.179339+0000", 2024-09-18T11:31:14.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00074063799999999995 2024-09-18T11:31:14.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181444+0000", 2024-09-18T11:31:14.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021044100000000001 2024-09-18T11:31:14.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181470+0000", 2024-09-18T11:31:14.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6415999999999999e-05 2024-09-18T11:31:14.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181494+0000", 2024-09-18T11:31:14.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3672e-05 2024-09-18T11:31:14.458 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.458 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.458 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.458 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.458 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.458 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:14.460 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:31:14.460 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:14.458+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:14.539 INFO:tasks.rados.rados.0.smithi119.stdout:1253: expect (ObjNum 230 snap 26 seq_num 230) 2024-09-18T11:31:14.561 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:14.560+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:14.598 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:14.598 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:14.598 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:14.598 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:14.607 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:31:14.691 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:14.690+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:14.745 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:14.745 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:14.745 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:14.745 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:14.746 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:14.754 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:31:14.795 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:14.794+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:14.813 INFO:tasks.rados.rados.0.smithi119.stdout:1256: finishing write tid 1 to smithi11939506-31 2024-09-18T11:31:14.813 INFO:tasks.rados.rados.0.smithi119.stdout:1256: finishing write tid 2 to smithi11939506-31 2024-09-18T11:31:14.813 INFO:tasks.rados.rados.0.smithi119.stdout:1256: finishing write tid 3 to smithi11939506-31 2024-09-18T11:31:14.813 INFO:tasks.rados.rados.0.smithi119.stdout:1256: finishing write tid 4 to smithi11939506-31 2024-09-18T11:31:14.813 INFO:tasks.rados.rados.0.smithi119.stdout:1256: finishing write tid 5 to smithi11939506-31 2024-09-18T11:31:14.813 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 178 (ObjNum 236 snap 28 seq_num 236) dirty exists 2024-09-18T11:31:14.813 INFO:tasks.rados.rados.0.smithi119.stdout:1256: left oid 31 (ObjNum 236 snap 28 seq_num 236) 2024-09-18T11:31:14.814 INFO:tasks.rados.rados.0.smithi119.stdout:1258: finishing rollback tid 0 to smithi11939506-7 2024-09-18T11:31:14.814 INFO:tasks.rados.rados.0.smithi119.stdout:1258: finishing rollback tid 1 to smithi11939506-7 2024-09-18T11:31:14.814 INFO:tasks.rados.rados.0.smithi119.stdout:1258: finishing rollback tid 2 to smithi11939506-7 2024-09-18T11:31:14.814 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 7 v 113 (ObjNum 145 snap 18 seq_num 145) dirty exists 2024-09-18T11:31:14.814 INFO:tasks.rados.rados.0.smithi119.stdout:1262: expect (ObjNum 199 snap 23 seq_num 199) 2024-09-18T11:31:14.863 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:14.862+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:14.893 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:14.893 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:14.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:14.893 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:14.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.893 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:14.893 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:14.893 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.732975503999995, 2024-09-18T11:31:14.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055400850000000001, 2024-09-18T11:31:14.894 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.894 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.894 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.894 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:14.894 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:14.894 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 43 2024-09-18T11:31:14.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.894 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:14.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:14.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:14.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:14.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157504+0000", 2024-09-18T11:31:14.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4519999999999996e-06 2024-09-18T11:31:14.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157534+0000", 2024-09-18T11:31:14.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.97e-05 2024-09-18T11:31:14.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157539+0000", 2024-09-18T11:31:14.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4360000000000001e-06 2024-09-18T11:31:14.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159428+0000", 2024-09-18T11:31:14.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018888220000000001 2024-09-18T11:31:14.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159479+0000", 2024-09-18T11:31:14.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0608000000000001e-05 2024-09-18T11:31:14.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:14.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159559+0000", 2024-09-18T11:31:14.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0667000000000005e-05 2024-09-18T11:31:14.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162779+0000", 2024-09-18T11:31:14.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032197459999999999 2024-09-18T11:31:14.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162950+0000", 2024-09-18T11:31:14.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017059599999999999 2024-09-18T11:31:14.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163001+0000", 2024-09-18T11:31:14.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0968999999999999e-05 2024-09-18T11:31:14.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163040+0000", 2024-09-18T11:31:14.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8843000000000001e-05 2024-09-18T11:31:14.899 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.899 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.899 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.899 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:14.899 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:14.899 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.732924350000005, 2024-09-18T11:31:14.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:31:14.900 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.900 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.900 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.900 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:14.900 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:14.900 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:31:14.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.900 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:14.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:14.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:31:14.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:31:14.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:31:14.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:31:14.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:31:14.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:31:14.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:14.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:31:14.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:31:14.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:31:14.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:31:14.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:14.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:31:14.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:31:14.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:31:14.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:31:14.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:31:14.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:31:14.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:31:14.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:31:14.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:31:14.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:31:14.905 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.905 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.905 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.905 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:14.905 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:14.905 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.732798087999996, 2024-09-18T11:31:14.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055717040000000002, 2024-09-18T11:31:14.905 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.905 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.906 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.906 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:14.906 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:14.906 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 45 2024-09-18T11:31:14.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.906 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:14.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:14.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157676+0000", 2024-09-18T11:31:14.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157694+0000", 2024-09-18T11:31:14.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7422999999999999e-05 2024-09-18T11:31:14.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157704+0000", 2024-09-18T11:31:14.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9000000000000001e-06 2024-09-18T11:31:14.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157718+0000", 2024-09-18T11:31:14.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4788e-05 2024-09-18T11:31:14.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160937+0000", 2024-09-18T11:31:14.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00321886 2024-09-18T11:31:14.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161023+0000", 2024-09-18T11:31:14.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6063999999999995e-05 2024-09-18T11:31:14.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:14.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161202+0000", 2024-09-18T11:31:14.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017912599999999999 2024-09-18T11:31:14.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163052+0000", 2024-09-18T11:31:14.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001849554 2024-09-18T11:31:14.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163204+0000", 2024-09-18T11:31:14.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015186900000000001 2024-09-18T11:31:14.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163229+0000", 2024-09-18T11:31:14.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:31:14.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163249+0000", 2024-09-18T11:31:14.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9992000000000001e-05 2024-09-18T11:31:14.911 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.911 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.911 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.911 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:14.911 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:14.912 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.732748975000007, 2024-09-18T11:31:14.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:31:14.912 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.912 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.912 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.912 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:14.912 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:14.912 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:31:14.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.912 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:14.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:14.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:14.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:31:14.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:31:14.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:31:14.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:31:14.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:31:14.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:31:14.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:31:14.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:31:14.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:31:14.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:31:14.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:14.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:31:14.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:31:14.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:31:14.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:31:14.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:31:14.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:31:14.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:31:14.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:31:14.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:31:14.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:31:14.917 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.917 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.917 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.917 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:14.917 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:14.917 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.732706444000002, 2024-09-18T11:31:14.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:31:14.918 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.918 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.918 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.918 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:14.918 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:14.918 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:31:14.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.918 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:14.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:14.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:31:14.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:31:14.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:31:14.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:31:14.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:31:14.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:31:14.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:31:14.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:31:14.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:31:14.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:31:14.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:31:14.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:14.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:31:14.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:31:14.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:31:14.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:31:14.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:31:14.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:31:14.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:31:14.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:31:14.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:31:14.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:31:14.923 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.923 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.923 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.923 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:14.923 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:14.923 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.732651513999997, 2024-09-18T11:31:14.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:31:14.924 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.924 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.924 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.924 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:14.924 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:14.924 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:31:14.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.924 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:14.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:14.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:14.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:31:14.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:31:14.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:31:14.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:31:14.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:31:14.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:31:14.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:31:14.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:31:14.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:31:14.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:31:14.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:14.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:31:14.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:31:14.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:31:14.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:31:14.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:31:14.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:31:14.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:31:14.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:31:14.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:31:14.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:31:14.929 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.929 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.929 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.929 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:14.929 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:14.929 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.732641533999995, 2024-09-18T11:31:14.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:31:14.929 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.929 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.930 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.930 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:14.930 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:14.930 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:31:14.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.930 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:14.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:14.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:14.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:14.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:31:14.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:31:14.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:31:14.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:31:14.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:31:14.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:31:14.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:31:14.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:31:14.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:31:14.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:31:14.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:14.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:31:14.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:31:14.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:31:14.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:31:14.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:31:14.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:31:14.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:31:14.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:31:14.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:31:14.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:31:14.935 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.935 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.935 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.935 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:14.935 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:14.935 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.732629935000006, 2024-09-18T11:31:14.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:31:14.935 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.935 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.935 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.935 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:14.936 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:14.936 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:31:14.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.936 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:14.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:14.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:14.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:14.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:31:14.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:31:14.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:31:14.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:31:14.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:31:14.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:31:14.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:31:14.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:31:14.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:31:14.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:31:14.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:14.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:31:14.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:31:14.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:31:14.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:31:14.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:31:14.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:31:14.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:31:14.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:31:14.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:31:14.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:31:14.941 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.941 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.941 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.941 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:14.941 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:14.941 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.732573454000004, 2024-09-18T11:31:14.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:31:14.941 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.941 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.941 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.941 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:14.942 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:14.942 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:31:14.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.942 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:14.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:14.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:31:14.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:31:14.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:31:14.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:31:14.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:31:14.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:31:14.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:31:14.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:31:14.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:31:14.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:31:14.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:31:14.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:14.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:31:14.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:31:14.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:31:14.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:31:14.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:31:14.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:31:14.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:31:14.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:31:14.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:31:14.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:31:14.947 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.947 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.947 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.947 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:14.947 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:14.947 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.732559629999997, 2024-09-18T11:31:14.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:31:14.947 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.947 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.947 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.947 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:14.948 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:14.948 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:31:14.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.948 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:14.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:14.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:14.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:14.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:31:14.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:31:14.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:31:14.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:31:14.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:31:14.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:31:14.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:31:14.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:31:14.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:31:14.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:31:14.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:14.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:31:14.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:31:14.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:31:14.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:31:14.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:31:14.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:31:14.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:31:14.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:31:14.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:31:14.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:31:14.953 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.953 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.953 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.953 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:14.953 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:14.953 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.316380722999995, 2024-09-18T11:31:14.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:31:14.953 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.953 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.953 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.953 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:14.953 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:14.954 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:31:14.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.954 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:14.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:14.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:14.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:14.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:31:14.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:31:14.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:31:14.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:31:14.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:31:14.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:31:14.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:31:14.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:31:14.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:31:14.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:31:14.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:14.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:31:14.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:31:14.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:31:14.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:31:14.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:31:14.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:31:14.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:31:14.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:31:14.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:31:14.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:31:14.958 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.958 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.959 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.959 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:14.959 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:14.959 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.316357422999999, 2024-09-18T11:31:14.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:31:14.959 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.959 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.959 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.960 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:14.960 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:14.960 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:31:14.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.960 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:14.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:14.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:31:14.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:31:14.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:31:14.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:31:14.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:31:14.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:31:14.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:31:14.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:31:14.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:31:14.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:31:14.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:31:14.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:14.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:31:14.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:31:14.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:31:14.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:31:14.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:31:14.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:31:14.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:31:14.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:31:14.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:31:14.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:31:14.964 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.965 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.965 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.965 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:14.965 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:14.965 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.316271814000004, 2024-09-18T11:31:14.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:31:14.965 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.965 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.965 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.965 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:14.965 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:14.966 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:31:14.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.966 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:14.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:14.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:14.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:31:14.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:31:14.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:31:14.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:31:14.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:31:14.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:31:14.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:31:14.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:31:14.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:31:14.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:31:14.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:14.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:31:14.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:31:14.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:31:14.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:31:14.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:31:14.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:31:14.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:31:14.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:31:14.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:31:14.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:31:14.970 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.970 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.971 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.971 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:14.971 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:14.971 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.316260849000003, 2024-09-18T11:31:14.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:31:14.971 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.971 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.971 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.971 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:14.971 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:14.971 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:31:14.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.971 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:14.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:14.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:14.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:14.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:31:14.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:31:14.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:31:14.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:31:14.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:31:14.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:31:14.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:31:14.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:31:14.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:31:14.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:31:14.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:14.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:31:14.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:31:14.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:31:14.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:31:14.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:31:14.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:31:14.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:31:14.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:31:14.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:31:14.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:31:14.976 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.976 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.976 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.976 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:14.977 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:14.977 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.316249681000002, 2024-09-18T11:31:14.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:31:14.977 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.977 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.977 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.977 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:14.977 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:14.977 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:31:14.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.977 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:14.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:14.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:14.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:14.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:31:14.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:31:14.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:31:14.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:31:14.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:31:14.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:31:14.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:31:14.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:31:14.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:31:14.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:31:14.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:14.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:31:14.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:31:14.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:31:14.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:31:14.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:31:14.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:31:14.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:31:14.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:31:14.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:31:14.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:31:14.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.982 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.982 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:14.982 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:14.982 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.316184351999993, 2024-09-18T11:31:14.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:31:14.983 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.983 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.983 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.983 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:14.983 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:14.983 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:31:14.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.983 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:14.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:14.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:31:14.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:31:14.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:31:14.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:31:14.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:31:14.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:31:14.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:31:14.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:31:14.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:31:14.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:31:14.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:31:14.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:14.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:31:14.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:31:14.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:31:14.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:31:14.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:31:14.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:31:14.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:31:14.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:31:14.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:31:14.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:31:14.988 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.988 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.988 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.988 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:14.988 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:14.988 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.306022373999994, 2024-09-18T11:31:14.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:31:14.988 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.988 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.988 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.989 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:14.989 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:14.989 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:31:14.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.989 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:14.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:14.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:31:14.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:31:14.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:31:14.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:31:14.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:31:14.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:31:14.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:31:14.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:31:14.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:31:14.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:31:14.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:31:14.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:14.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:31:14.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:31:14.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:31:14.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:31:14.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:31:14.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:31:14.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:31:14.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:31:14.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:31:14.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:31:14.993 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.994 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.994 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.994 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:14.994 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:14.994 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.305929895000006, 2024-09-18T11:31:14.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:31:14.994 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:14.994 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:14.994 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:14.994 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:14.994 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:14.994 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:31:14.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.995 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:14.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:14.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:14.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:14.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:14.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:14.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:14.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:14.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:14.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:14.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:31:14.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:31:14.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:14.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:31:14.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:31:14.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:14.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:31:14.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:31:14.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:14.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:31:14.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:31:14.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:14.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:31:14.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:31:14.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:14.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:31:14.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:31:14.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:14.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:31:14.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:31:14.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:14.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:31:14.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:31:14.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:14.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:31:14.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:31:14.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:14.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:14.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:14.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:31:14.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:31:14.999 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:14.999 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:14.999 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:15.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:15.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.000 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1788 4.8 e60/22 4:1898b15d:::smithi11939506-25:head v 60'12, mlcod=58'9)", 2024-09-18T11:31:15.000 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:15.000 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.973524405999999, 2024-09-18T11:31:15.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058348599999999999, 2024-09-18T11:31:15.000 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:15.000 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:15.000 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:15.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:15.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:15.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:15.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:15.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:15.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:15.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:15.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:15.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:15.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916949+0000", 2024-09-18T11:31:15.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:15.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:15.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:15.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917602+0000", 2024-09-18T11:31:15.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00065337100000000003 2024-09-18T11:31:15.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:15.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:15.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917621+0000", 2024-09-18T11:31:15.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9287e-05 2024-09-18T11:31:15.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:15.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917629+0000", 2024-09-18T11:31:15.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3780000000000003e-06 2024-09-18T11:31:15.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:15.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:15.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917677+0000", 2024-09-18T11:31:15.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8068000000000001e-05 2024-09-18T11:31:15.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:15.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:15.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917699+0000", 2024-09-18T11:31:15.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2149000000000001e-05 2024-09-18T11:31:15.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:15.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:15.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922747+0000", 2024-09-18T11:31:15.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050477609999999996 2024-09-18T11:31:15.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:15.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:15.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922785+0000", 2024-09-18T11:31:15.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8726000000000002e-05 2024-09-18T11:31:15.004 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:15.004 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:15.004 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:15.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:15.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.004 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:58 4.3 e91/67 4:c70c3162:::smithi11939506-35:18 v 91'101, mlcod=86'95)", 2024-09-18T11:31:15.004 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:15.004 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.87486618299999996, 2024-09-18T11:31:15.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055647500000000003, 2024-09-18T11:31:15.004 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:15.004 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:15.004 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:15.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:15.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:15.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:15.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:15.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:15.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:15.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:15.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:15.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:15.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015608+0000", 2024-09-18T11:31:15.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:15.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:15.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:15.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015614+0000", 2024-09-18T11:31:15.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5310000000000001e-06 2024-09-18T11:31:15.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:15.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:15.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015622+0000", 2024-09-18T11:31:15.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0809999999999994e-06 2024-09-18T11:31:15.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:15.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015627+0000", 2024-09-18T11:31:15.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8439999999999997e-06 2024-09-18T11:31:15.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:15.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:15.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016003+0000", 2024-09-18T11:31:15.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037652999999999997 2024-09-18T11:31:15.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:15.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:15.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016024+0000", 2024-09-18T11:31:15.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0780000000000001e-05 2024-09-18T11:31:15.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:15.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:15.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021141+0000", 2024-09-18T11:31:15.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051166639999999999 2024-09-18T11:31:15.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:15.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:15.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:15.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021174+0000", 2024-09-18T11:31:15.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2839000000000002e-05 2024-09-18T11:31:15.008 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:15.008 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:15.008 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:15.008 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:15.008 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:15.008 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:15.009 INFO:tasks.rados.rados.0.smithi119.stdout:1261: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:31:15.010 INFO:tasks.rados.rados.0.smithi119.stdout:1259: expect (ObjNum 217 snap 26 seq_num 217) 2024-09-18T11:31:15.010 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:15.008+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:15.010 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:31:15.110 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:15.110 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:15.110 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:15.110 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:15.110 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:15.109+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:15.118 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:31:15.190 INFO:tasks.rados.rados.0.smithi119.stdout:1260: expect (ObjNum 215 snap 26 seq_num 215) 2024-09-18T11:31:15.211 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:15.209+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:15.256 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:15.256 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:15.257 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:15.257 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:15.257 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:15.265 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:31:15.323 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:31:15.349 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:15.348+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:15.410 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:15.410 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:15.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:15.410 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:15.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.410 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:887 4.7 4:e0d95e10:::smithi11939506-22:head [read 0~2352947,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:15.410 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:31:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.290487588000005, 2024-09-18T11:31:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079115169999999999, 2024-09-18T11:31:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 887 2024-09-18T11:31:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:31:15.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:31:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116365+0000", 2024-09-18T11:31:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116376+0000", 2024-09-18T11:31:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1471000000000001e-05 2024-09-18T11:31:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116394+0000", 2024-09-18T11:31:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8088e-05 2024-09-18T11:31:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116402+0000", 2024-09-18T11:31:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2320000000000004e-06 2024-09-18T11:31:15.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116458+0000", 2024-09-18T11:31:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6536000000000002e-05 2024-09-18T11:31:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116466+0000", 2024-09-18T11:31:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0819999999999999e-06 2024-09-18T11:31:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.122509+0000", 2024-09-18T11:31:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060431879999999997 2024-09-18T11:31:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.122577+0000", 2024-09-18T11:31:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7646999999999998e-05 2024-09-18T11:31:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124278+0000", 2024-09-18T11:31:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017013200000000001 2024-09-18T11:31:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:15.415 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:888 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.290434125000004, 2024-09-18T11:31:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082282120000000004, 2024-09-18T11:31:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 888 2024-09-18T11:31:15.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:15.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116423+0000", 2024-09-18T11:31:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1e-06 2024-09-18T11:31:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:31:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5510000000000001e-06 2024-09-18T11:31:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116431+0000", 2024-09-18T11:31:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6109999999999998e-06 2024-09-18T11:31:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116527+0000", 2024-09-18T11:31:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6212999999999995e-05 2024-09-18T11:31:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116532+0000", 2024-09-18T11:31:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6140000000000002e-06 2024-09-18T11:31:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124314+0000", 2024-09-18T11:31:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077818929999999998 2024-09-18T11:31:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124411+0000", 2024-09-18T11:31:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6992999999999998e-05 2024-09-18T11:31:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124649+0000", 2024-09-18T11:31:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237729 2024-09-18T11:31:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:889 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.290409475000004, 2024-09-18T11:31:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085383349999999993, 2024-09-18T11:31:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:15.421 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 889 2024-09-18T11:31:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116447+0000", 2024-09-18T11:31:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.854e-06 2024-09-18T11:31:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116452+0000", 2024-09-18T11:31:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0250000000000002e-06 2024-09-18T11:31:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116455+0000", 2024-09-18T11:31:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8739999999999999e-06 2024-09-18T11:31:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116565+0000", 2024-09-18T11:31:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110741 2024-09-18T11:31:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116571+0000", 2024-09-18T11:31:15.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4990000000000002e-06 2024-09-18T11:31:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124691+0000", 2024-09-18T11:31:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081200339999999999 2024-09-18T11:31:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124784+0000", 2024-09-18T11:31:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2886999999999993e-05 2024-09-18T11:31:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124983+0000", 2024-09-18T11:31:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019969000000000001 2024-09-18T11:31:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:46 4.a e31/22 4:552523b6:::smithi11939506-38:head v 31'20, mlcod=30'18)", 2024-09-18T11:31:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:31:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.276457843000003, 2024-09-18T11:31:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080208420000000002, 2024-09-18T11:31:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:15.426 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:31:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:31:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130396+0000", 2024-09-18T11:31:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133245+0000", 2024-09-18T11:31:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028486929999999998 2024-09-18T11:31:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133253+0000", 2024-09-18T11:31:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1540000000000002e-06 2024-09-18T11:31:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133261+0000", 2024-09-18T11:31:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3620000000000006e-06 2024-09-18T11:31:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133319+0000", 2024-09-18T11:31:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7719000000000001e-05 2024-09-18T11:31:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133341+0000", 2024-09-18T11:31:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1946999999999998e-05 2024-09-18T11:31:15.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138368+0000", 2024-09-18T11:31:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005026712 2024-09-18T11:31:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138418+0000", 2024-09-18T11:31:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9864000000000002e-05 2024-09-18T11:31:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1126 3.0 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head [write 0~2520064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:31:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:31:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.424967496999997, 2024-09-18T11:31:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086197050000000001, 2024-09-18T11:31:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1126 2024-09-18T11:31:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:31:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:31:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981886+0000", 2024-09-18T11:31:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:15.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983017+0000", 2024-09-18T11:31:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00113131 2024-09-18T11:31:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983037+0000", 2024-09-18T11:31:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0231e-05 2024-09-18T11:31:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983044+0000", 2024-09-18T11:31:15.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9539999999999998e-06 2024-09-18T11:31:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983087+0000", 2024-09-18T11:31:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3040000000000001e-05 2024-09-18T11:31:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983193+0000", 2024-09-18T11:31:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105682 2024-09-18T11:31:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983287+0000", 2024-09-18T11:31:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4066000000000004e-05 2024-09-18T11:31:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.986918+0000", 2024-09-18T11:31:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036311970000000001 2024-09-18T11:31:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990442+0000", 2024-09-18T11:31:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035243119999999999 2024-09-18T11:31:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990476+0000", 2024-09-18T11:31:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4028e-05 2024-09-18T11:31:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990507+0000", 2024-09-18T11:31:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0284999999999999e-05 2024-09-18T11:31:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:262 4.9 e53/22 4:99feffe9:::smithi11939506-45:13 v 53'17, mlcod=51'14)", 2024-09-18T11:31:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.407863681999999, 2024-09-18T11:31:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013878679, 2024-09-18T11:31:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998997+0000", 2024-09-18T11:31:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1099999999999999e-06 2024-09-18T11:31:15.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999008+0000", 2024-09-18T11:31:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1321000000000001e-05 2024-09-18T11:31:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999015+0000", 2024-09-18T11:31:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3000000000000004e-06 2024-09-18T11:31:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999055+0000", 2024-09-18T11:31:15.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0058000000000002e-05 2024-09-18T11:31:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999070+0000", 2024-09-18T11:31:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4328000000000001e-05 2024-09-18T11:31:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012465+0000", 2024-09-18T11:31:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013395638 2024-09-18T11:31:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012870+0000", 2024-09-18T11:31:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040423400000000001 2024-09-18T11:31:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:205 4.b e53/22 4:dba5bd39:::smithi11939506-17:13 v 53'77, mlcod=51'74)", 2024-09-18T11:31:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.407814420999998, 2024-09-18T11:31:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013359586, 2024-09-18T11:31:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:15.441 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:15.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999048+0000", 2024-09-18T11:31:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3550000000000005e-06 2024-09-18T11:31:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999056+0000", 2024-09-18T11:31:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3259999999999996e-06 2024-09-18T11:31:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999063+0000", 2024-09-18T11:31:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6409999999999996e-06 2024-09-18T11:31:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999098+0000", 2024-09-18T11:31:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5203999999999997e-05 2024-09-18T11:31:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999115+0000", 2024-09-18T11:31:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6478999999999999e-05 2024-09-18T11:31:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012362+0000", 2024-09-18T11:31:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01324768 2024-09-18T11:31:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012400+0000", 2024-09-18T11:31:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.731e-05 2024-09-18T11:31:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.445 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:88 4.a e53/22 4:588debc2:::smithi11939506-18:13 v 53'45, mlcod=51'42)", 2024-09-18T11:31:15.446 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:15.446 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.405031997999998, 2024-09-18T11:31:15.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011307667, 2024-09-18T11:31:15.446 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:15.446 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:15.446 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:15.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:15.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:15.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001822+0000", 2024-09-18T11:31:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001827+0000", 2024-09-18T11:31:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.378e-06 2024-09-18T11:31:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001839+0000", 2024-09-18T11:31:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1225e-05 2024-09-18T11:31:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001844+0000", 2024-09-18T11:31:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2299999999999999e-06 2024-09-18T11:31:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001891+0000", 2024-09-18T11:31:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6961e-05 2024-09-18T11:31:15.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001906+0000", 2024-09-18T11:31:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5158e-05 2024-09-18T11:31:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013103+0000", 2024-09-18T11:31:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011197314 2024-09-18T11:31:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013130+0000", 2024-09-18T11:31:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6900999999999999e-05 2024-09-18T11:31:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:89 4.a e53/22 4:552523b6:::smithi11939506-38:13 v 53'47, mlcod=51'42)", 2024-09-18T11:31:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.404472777999999, 2024-09-18T11:31:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011327614, 2024-09-18T11:31:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:15.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002381+0000", 2024-09-18T11:31:15.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002397+0000", 2024-09-18T11:31:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5688e-05 2024-09-18T11:31:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002406+0000", 2024-09-18T11:31:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0229999999999998e-06 2024-09-18T11:31:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002415+0000", 2024-09-18T11:31:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8959999999999992e-06 2024-09-18T11:31:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012386+0000", 2024-09-18T11:31:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0099711450000000007 2024-09-18T11:31:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012405+0000", 2024-09-18T11:31:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9389e-05 2024-09-18T11:31:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013676+0000", 2024-09-18T11:31:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012708649999999999 2024-09-18T11:31:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013709+0000", 2024-09-18T11:31:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3207000000000002e-05 2024-09-18T11:31:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.454 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1643 4.7 4:e73ba68f:::smithi11939506-1:head [read 0~3983226,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:31:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.072967601999999, 2024-09-18T11:31:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079617669999999998, 2024-09-18T11:31:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1643 2024-09-18T11:31:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:15.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:31:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:31:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:31:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333889+0000", 2024-09-18T11:31:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8890000000000002e-06 2024-09-18T11:31:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333896+0000", 2024-09-18T11:31:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.156e-06 2024-09-18T11:31:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333900+0000", 2024-09-18T11:31:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.138e-06 2024-09-18T11:31:15.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.336677+0000", 2024-09-18T11:31:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027774169999999999 2024-09-18T11:31:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.336768+0000", 2024-09-18T11:31:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0983999999999995e-05 2024-09-18T11:31:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341849+0000", 2024-09-18T11:31:15.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050805140000000004 2024-09-18T11:31:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1644 4.7 4:e73ba68f:::smithi11939506-1:head [sparse-read 0~2695582] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.072950480000003, 2024-09-18T11:31:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084764039999999999, 2024-09-18T11:31:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:15.459 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1644 2024-09-18T11:31:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:15.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333906+0000", 2024-09-18T11:31:15.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.243e-06 2024-09-18T11:31:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333911+0000", 2024-09-18T11:31:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2959999999999998e-06 2024-09-18T11:31:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333915+0000", 2024-09-18T11:31:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6720000000000002e-06 2024-09-18T11:31:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341879+0000", 2024-09-18T11:31:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079638699999999996 2024-09-18T11:31:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341978+0000", 2024-09-18T11:31:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9562000000000007e-05 2024-09-18T11:31:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342380+0000", 2024-09-18T11:31:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000402113 2024-09-18T11:31:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1645 4.7 4:e73ba68f:::smithi11939506-1:head [read 0~3983226,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.072935586, 2024-09-18T11:31:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010539468999999999, 2024-09-18T11:31:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1645 2024-09-18T11:31:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.464 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:15.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333921+0000", 2024-09-18T11:31:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8909999999999999e-06 2024-09-18T11:31:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333927+0000", 2024-09-18T11:31:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4229999999999996e-06 2024-09-18T11:31:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333932+0000", 2024-09-18T11:31:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8280000000000002e-06 2024-09-18T11:31:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342418+0000", 2024-09-18T11:31:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084862219999999999 2024-09-18T11:31:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342515+0000", 2024-09-18T11:31:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7287999999999994e-05 2024-09-18T11:31:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.344458+0000", 2024-09-18T11:31:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019431660000000001 2024-09-18T11:31:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1669 4.4 4:2d9c3f7f:::smithi11939506-5:head [read 0~3764735,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-18T11:31:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:31:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.583986688000003, 2024-09-18T11:31:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0083694990000000007, 2024-09-18T11:31:15.468 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1669 2024-09-18T11:31:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:31:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:31:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822867+0000", 2024-09-18T11:31:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822871+0000", 2024-09-18T11:31:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:31:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822877+0000", 2024-09-18T11:31:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8000000000000004e-06 2024-09-18T11:31:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822880+0000", 2024-09-18T11:31:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5669999999999999e-06 2024-09-18T11:31:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822974+0000", 2024-09-18T11:31:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4048999999999993e-05 2024-09-18T11:31:15.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822980+0000", 2024-09-18T11:31:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2789999999999998e-06 2024-09-18T11:31:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.829160+0000", 2024-09-18T11:31:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0061800179999999998 2024-09-18T11:31:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:15.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.829248+0000", 2024-09-18T11:31:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7707000000000006e-05 2024-09-18T11:31:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.831237+0000", 2024-09-18T11:31:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001989827 2024-09-18T11:31:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:100 4.a e58/22 4:5975a5ec:::smithi11939506-36:18 v 58'55, mlcod=56'50)", 2024-09-18T11:31:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:15.473 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.133705450000001, 2024-09-18T11:31:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0088255, 2024-09-18T11:31:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273155+0000", 2024-09-18T11:31:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.212e-06 2024-09-18T11:31:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273164+0000", 2024-09-18T11:31:15.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7609999999999996e-06 2024-09-18T11:31:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273168+0000", 2024-09-18T11:31:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2819999999999998e-06 2024-09-18T11:31:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274162+0000", 2024-09-18T11:31:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099398600000000009 2024-09-18T11:31:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274180+0000", 2024-09-18T11:31:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8236e-05 2024-09-18T11:31:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281943+0000", 2024-09-18T11:31:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007763308 2024-09-18T11:31:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281975+0000", 2024-09-18T11:31:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1202999999999999e-05 2024-09-18T11:31:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.477 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1954 4.7 4:ef60946b:::smithi11939506-8:head [read 0~2890771,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.066954410000001, 2024-09-18T11:31:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091142180000000003, 2024-09-18T11:31:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1954 2024-09-18T11:31:15.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339903+0000", 2024-09-18T11:31:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7349999999999998e-06 2024-09-18T11:31:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339909+0000", 2024-09-18T11:31:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-09-18T11:31:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339913+0000", 2024-09-18T11:31:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7670000000000001e-06 2024-09-18T11:31:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339970+0000", 2024-09-18T11:31:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6988999999999999e-05 2024-09-18T11:31:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339975+0000", 2024-09-18T11:31:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7929999999999997e-06 2024-09-18T11:31:15.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346917+0000", 2024-09-18T11:31:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069426640000000003 2024-09-18T11:31:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.347026+0000", 2024-09-18T11:31:15.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108241 2024-09-18T11:31:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.349014+0000", 2024-09-18T11:31:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001988723 2024-09-18T11:31:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1956 4.6 4:67c1765a:::smithi11939506-30:head [write 705927~558502] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:15.483 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.05484547, 2024-09-18T11:31:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011813239, 2024-09-18T11:31:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1956 2024-09-18T11:31:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352007+0000", 2024-09-18T11:31:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:15.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352229+0000", 2024-09-18T11:31:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222024 2024-09-18T11:31:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352251+0000", 2024-09-18T11:31:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2512000000000001e-05 2024-09-18T11:31:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352260+0000", 2024-09-18T11:31:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7339999999999997e-06 2024-09-18T11:31:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352307+0000", 2024-09-18T11:31:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7061999999999998e-05 2024-09-18T11:31:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352408+0000", 2024-09-18T11:31:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101216 2024-09-18T11:31:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352550+0000", 2024-09-18T11:31:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014136199999999999 2024-09-18T11:31:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:15.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357684+0000", 2024-09-18T11:31:15.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051342499999999999 2024-09-18T11:31:15.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:15.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363736+0000", 2024-09-18T11:31:15.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006052577 2024-09-18T11:31:15.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:15.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363778+0000", 2024-09-18T11:31:15.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1989000000000001e-05 2024-09-18T11:31:15.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:15.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363822+0000", 2024-09-18T11:31:15.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3894999999999999e-05 2024-09-18T11:31:15.489 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.489 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:15.489 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.489 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2278 4.f 4:fc192b81:::smithi11939506-31:head [stat,delete] snapc 26=[26,25,24,1e,1c,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T11:31:15.489 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:15.489 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.023808258000001, 2024-09-18T11:31:15.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01018694, 2024-09-18T11:31:15.490 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:15.490 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:15.490 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:15.490 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:15.490 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:15.490 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2278 2024-09-18T11:31:15.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.490 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:15.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:15.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:15.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:15.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:15.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:15.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383044+0000", 2024-09-18T11:31:15.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:15.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:15.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383054+0000", 2024-09-18T11:31:15.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5969999999999998e-06 2024-09-18T11:31:15.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:15.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383071+0000", 2024-09-18T11:31:15.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7353000000000001e-05 2024-09-18T11:31:15.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383081+0000", 2024-09-18T11:31:15.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9170000000000001e-06 2024-09-18T11:31:15.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383150+0000", 2024-09-18T11:31:15.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8503999999999993e-05 2024-09-18T11:31:15.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:15.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383158+0000", 2024-09-18T11:31:15.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5089999999999993e-06 2024-09-18T11:31:15.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385396+0000", 2024-09-18T11:31:15.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022381850000000002 2024-09-18T11:31:15.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:15.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385494+0000", 2024-09-18T11:31:15.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7751000000000007e-05 2024-09-18T11:31:15.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:15.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385697+0000", 2024-09-18T11:31:15.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020247400000000001 2024-09-18T11:31:15.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:15.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.390505+0000", 2024-09-18T11:31:15.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048084039999999996 2024-09-18T11:31:15.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:15.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393148+0000", 2024-09-18T11:31:15.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026432399999999998 2024-09-18T11:31:15.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:15.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393194+0000", 2024-09-18T11:31:15.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5259999999999997e-05 2024-09-18T11:31:15.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:15.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393233+0000", 2024-09-18T11:31:15.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9659000000000001e-05 2024-09-18T11:31:15.495 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.496 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:15.496 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.496 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2342 4.f 4:f3c51b3f:::smithi11939506-21:head [stat,delete] snapc 26=[26,25,24,1e,1c,19,18] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:15.496 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:15.496 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.020052246000001, 2024-09-18T11:31:15.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011167565000000001, 2024-09-18T11:31:15.496 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:15.496 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:15.496 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:15.496 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:15.497 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:15.497 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2342 2024-09-18T11:31:15.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.497 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:15.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:15.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:15.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:15.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:15.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:15.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:31:15.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:15.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:15.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:31:15.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1439999999999997e-06 2024-09-18T11:31:15.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:15.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386820+0000", 2024-09-18T11:31:15.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2651e-05 2024-09-18T11:31:15.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386825+0000", 2024-09-18T11:31:15.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3150000000000002e-06 2024-09-18T11:31:15.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386868+0000", 2024-09-18T11:31:15.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3832999999999999e-05 2024-09-18T11:31:15.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:15.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:31:15.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0409999999999998e-06 2024-09-18T11:31:15.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390445+0000", 2024-09-18T11:31:15.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035689459999999999 2024-09-18T11:31:15.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:15.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390539+0000", 2024-09-18T11:31:15.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3791999999999999e-05 2024-09-18T11:31:15.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:15.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390748+0000", 2024-09-18T11:31:15.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000208863 2024-09-18T11:31:15.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:15.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397659+0000", 2024-09-18T11:31:15.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069110710000000004 2024-09-18T11:31:15.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:15.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397873+0000", 2024-09-18T11:31:15.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021343999999999999 2024-09-18T11:31:15.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:15.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397923+0000", 2024-09-18T11:31:15.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0300000000000003e-05 2024-09-18T11:31:15.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:15.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397970+0000", 2024-09-18T11:31:15.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6968999999999997e-05 2024-09-18T11:31:15.502 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.502 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:15.503 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.503 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2429 4.b 4:d8d14c58:::smithi11939506-49:head [write 796698~561170,stat] snapc 2b=[2b,2a,26,25,24,1e,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T11:31:15.503 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:15.503 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 10.391683071999999, 2024-09-18T11:31:15.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097188989999999996, 2024-09-18T11:31:15.503 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:15.503 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:15.503 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:15.503 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:15.503 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:15.503 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2429 2024-09-18T11:31:15.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.504 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:15.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:15.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:15.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:15.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:15.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:15.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015170+0000", 2024-09-18T11:31:15.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:15.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:15.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015435+0000", 2024-09-18T11:31:15.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026533299999999998 2024-09-18T11:31:15.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:15.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015455+0000", 2024-09-18T11:31:15.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9623000000000002e-05 2024-09-18T11:31:15.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015461+0000", 2024-09-18T11:31:15.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1859999999999997e-06 2024-09-18T11:31:15.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015535+0000", 2024-09-18T11:31:15.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4555999999999996e-05 2024-09-18T11:31:15.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:15.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015543+0000", 2024-09-18T11:31:15.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9549999999999992e-06 2024-09-18T11:31:15.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017106+0000", 2024-09-18T11:31:15.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015630290000000001 2024-09-18T11:31:15.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:15.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017183+0000", 2024-09-18T11:31:15.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6953999999999995e-05 2024-09-18T11:31:15.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:15.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017301+0000", 2024-09-18T11:31:15.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011734 2024-09-18T11:31:15.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:15.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.020225+0000", 2024-09-18T11:31:15.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029244090000000002 2024-09-18T11:31:15.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:15.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024801+0000", 2024-09-18T11:31:15.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045764129999999997 2024-09-18T11:31:15.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:15.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024840+0000", 2024-09-18T11:31:15.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8325999999999999e-05 2024-09-18T11:31:15.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:15.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024890+0000", 2024-09-18T11:31:15.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0596999999999997e-05 2024-09-18T11:31:15.509 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.509 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:15.509 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.509 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:207 4.a e91/22 4:5975a5ec:::smithi11939506-36:18 v 91'132, mlcod=90'126)", 2024-09-18T11:31:15.510 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.010357+0000", 2024-09-18T11:31:15.510 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.3964976069999999, 2024-09-18T11:31:15.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079598080000000005, 2024-09-18T11:31:15.510 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:15.510 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:15.510 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:15.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:15.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.010357+0000", 2024-09-18T11:31:15.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:15.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.010357+0000", 2024-09-18T11:31:15.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:15.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:15.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.010356+0000", 2024-09-18T11:31:15.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:15.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:15.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.010362+0000", 2024-09-18T11:31:15.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1720000000000004e-06 2024-09-18T11:31:15.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:15.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.010375+0000", 2024-09-18T11:31:15.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3053e-05 2024-09-18T11:31:15.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:15.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.010381+0000", 2024-09-18T11:31:15.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8019999999999997e-06 2024-09-18T11:31:15.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:15.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.010522+0000", 2024-09-18T11:31:15.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000140488 2024-09-18T11:31:15.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:15.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.010542+0000", 2024-09-18T11:31:15.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0007e-05 2024-09-18T11:31:15.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:15.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.018285+0000", 2024-09-18T11:31:15.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077432289999999999 2024-09-18T11:31:15.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:15.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:15.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:15.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.018317+0000", 2024-09-18T11:31:15.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1705000000000002e-05 2024-09-18T11:31:15.514 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.514 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:15.514 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.514 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:15.514 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:15.514 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:15.515 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:31:15.516 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:15.516 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:15.516 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:15.517 INFO:tasks.rados.rados.0.smithi119.stdout:1264: finishing set_chunk to oid 24 2024-09-18T11:31:15.517 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 86 (ObjNum 135 snap 16 seq_num 135) dirty exists 2024-09-18T11:31:15.517 INFO:tasks.rados.rados.0.smithi119.stdout:1266: seq_num 237 ranges {783259=640956,1918703=613070,3008661=405467,3821939=161320} 2024-09-18T11:31:15.517 INFO:tasks.rados.rados.0.smithi119.stdout:1266: writing smithi11939506-41 from 783259 to 1424215 tid 1 2024-09-18T11:31:15.517 INFO:tasks.rados.rados.0.smithi119.stdout:1266: writing smithi11939506-41 from 1918703 to 2531773 tid 2 2024-09-18T11:31:15.518 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:15.518 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:15.519 INFO:tasks.rados.rados.0.smithi119.stdout:1266: writing smithi11939506-41 from 3008661 to 3414128 tid 3 2024-09-18T11:31:15.519 INFO:tasks.rados.rados.0.smithi119.stdout:1266: writing smithi11939506-41 from 3821939 to 3983259 tid 4 2024-09-18T11:31:15.519 INFO:tasks.rados.rados.0.smithi119.stdout:1252: done (13 left) 2024-09-18T11:31:15.519 INFO:tasks.rados.rados.0.smithi119.stdout:1253: done (12 left) 2024-09-18T11:31:15.519 INFO:tasks.rados.rados.0.smithi119.stdout:1254: done (11 left) 2024-09-18T11:31:15.519 INFO:tasks.rados.rados.0.smithi119.stdout:1255: done (10 left) 2024-09-18T11:31:15.519 INFO:tasks.rados.rados.0.smithi119.stdout:1256: done (9 left) 2024-09-18T11:31:15.519 INFO:tasks.rados.rados.0.smithi119.stdout:1258: done (8 left) 2024-09-18T11:31:15.519 INFO:tasks.rados.rados.0.smithi119.stdout:1259: done (7 left) 2024-09-18T11:31:15.520 INFO:tasks.rados.rados.0.smithi119.stdout:1260: done (6 left) 2024-09-18T11:31:15.520 INFO:tasks.rados.rados.0.smithi119.stdout:1261: done (5 left) 2024-09-18T11:31:15.520 INFO:tasks.rados.rados.0.smithi119.stdout:1262: done (4 left) 2024-09-18T11:31:15.520 INFO:tasks.rados.rados.0.smithi119.stdout:1263: done (3 left) 2024-09-18T11:31:15.520 INFO:tasks.rados.rados.0.smithi119.stdout:1264: done (2 left) 2024-09-18T11:31:15.520 INFO:tasks.rados.rados.0.smithi119.stdout:1267: write oid 22 current snap is 28 2024-09-18T11:31:15.520 INFO:tasks.rados.rados.0.smithi119.stdout:1265: expect (ObjNum 99 snap 6 seq_num 99) 2024-09-18T11:31:15.520 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:15.520 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:15.521 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:15.521 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:15.522 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:15.513+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:15.529 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:15.529 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:15.545 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:15.545 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:15.560 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:15.561 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:15.563 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:15.564 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:15.616 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:15.614+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:15.620 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:15.620 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:15.628 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:31:15.659 INFO:tasks.rados.rados.0.smithi119.stdout:1266: finishing write tid 1 to smithi11939506-41 2024-09-18T11:31:15.660 INFO:tasks.rados.rados.0.smithi119.stdout:1266: finishing write tid 2 to smithi11939506-41 2024-09-18T11:31:15.660 INFO:tasks.rados.rados.0.smithi119.stdout:1266: finishing write tid 3 to smithi11939506-41 2024-09-18T11:31:15.660 INFO:tasks.rados.rados.0.smithi119.stdout:1266: finishing write tid 4 to smithi11939506-41 2024-09-18T11:31:15.660 INFO:tasks.rados.rados.0.smithi119.stdout:1266: finishing write tid 5 to smithi11939506-41 2024-09-18T11:31:15.660 INFO:tasks.rados.rados.0.smithi119.stdout:1266: finishing write tid 6 to smithi11939506-41 2024-09-18T11:31:15.660 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 41 v 262 (ObjNum 237 snap 28 seq_num 237) dirty exists 2024-09-18T11:31:15.660 INFO:tasks.rados.rados.0.smithi119.stdout:1266: left oid 41 (ObjNum 237 snap 28 seq_num 237) 2024-09-18T11:31:15.660 INFO:tasks.rados.rados.0.smithi119.stdout:1267: seq_num 238 ranges {400066=716205,1860832=139234} 2024-09-18T11:31:15.665 INFO:tasks.rados.rados.0.smithi119.stdout:1267: writing smithi11939506-22 from 400066 to 1116271 tid 1 2024-09-18T11:31:15.670 INFO:tasks.rados.rados.0.smithi119.stdout:1267: writing smithi11939506-22 from 1860832 to 2000066 tid 2 2024-09-18T11:31:15.670 INFO:tasks.rados.rados.0.smithi119.stdout:1265: done (2 left) 2024-09-18T11:31:15.670 INFO:tasks.rados.rados.0.smithi119.stdout:1266: done (1 left) 2024-09-18T11:31:15.671 INFO:tasks.rados.rados.0.smithi119.stdout:1268: read oid 21 snap -1 2024-09-18T11:31:15.671 INFO:tasks.rados.rados.0.smithi119.stdout:1268: expect (ObjNum 222 snap 26 seq_num 222) 2024-09-18T11:31:15.671 INFO:tasks.rados.rados.0.smithi119.stdout:1269: set_chunk oid 47 target oid 2024-09-18T11:31:15.686 INFO:tasks.rados.rados.0.smithi119.stdout:1269: set_chunk oid 47 offset: 434176 length: 2087424 target oid e0162aefbd7ccebf38676fc7fc4dd7c7dc72b14c0f9f1c978ce0b5a49ff8e2a6 offset: 0 snap 27 2024-09-18T11:31:15.686 INFO:tasks.rados.rados.0.smithi119.stdout:1270: set_chunk oid 38 target oid 2024-09-18T11:31:15.691 INFO:tasks.rados.rados.0.smithi119.stdout:1270: set_chunk oid 38 offset: 1856512 length: 431104 target oid 8972a10aed27f8d0a6b0bb9626194d488374d181167228b3d8e199098a51d586 offset: 0 snap -1 2024-09-18T11:31:15.691 INFO:tasks.rados.rados.0.smithi119.stdout:1271: write (excl) oid 27 current snap is 28 2024-09-18T11:31:15.692 INFO:tasks.rados.rados.0.smithi119.stdout:1267: finishing write tid 1 to smithi11939506-22 2024-09-18T11:31:15.692 INFO:tasks.rados.rados.0.smithi119.stdout:1271: seq_num 239 ranges {416873=791454,1919314=97559} 2024-09-18T11:31:15.698 INFO:tasks.rados.rados.0.smithi119.stdout:1271: writing smithi11939506-27 from 416873 to 1208327 tid 1 2024-09-18T11:31:15.703 INFO:tasks.rados.rados.0.smithi119.stdout:1271: writing smithi11939506-27 from 1919314 to 2016873 tid 2 2024-09-18T11:31:15.703 INFO:tasks.rados.rados.0.smithi119.stdout:1272: tier_promote oid 8 2024-09-18T11:31:15.703 INFO:tasks.rados.rados.0.smithi119.stdout:1267: finishing write tid 2 to smithi11939506-22 2024-09-18T11:31:15.703 INFO:tasks.rados.rados.0.smithi119.stdout:1267: finishing write tid 3 to smithi11939506-22 2024-09-18T11:31:15.703 INFO:tasks.rados.rados.0.smithi119.stdout:1273: write (excl) oid 10 current snap is 28 2024-09-18T11:31:15.703 INFO:tasks.rados.rados.0.smithi119.stdout:1267: finishing write tid 4 to smithi11939506-22 2024-09-18T11:31:15.703 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 126 (ObjNum 238 snap 28 seq_num 238) dirty exists 2024-09-18T11:31:15.703 INFO:tasks.rados.rados.0.smithi119.stdout:1267: left oid 22 (ObjNum 238 snap 28 seq_num 238) 2024-09-18T11:31:15.703 INFO:tasks.rados.rados.0.smithi119.stdout:1268: expect (ObjNum 222 snap 26 seq_num 222) 2024-09-18T11:31:15.717 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:15.715+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:15.732 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:15.733 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:15.741 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:31:15.825 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:15.824+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:15.875 INFO:tasks.rados.rados.0.smithi119.stdout:1269: finishing set_chunk to oid 47 2024-09-18T11:31:15.875 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 47 set_chunk returned error code -95 offset: 434176 length: 2087424 2024-09-18T11:31:15.875 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 0 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-18T11:31:15.875 INFO:tasks.rados.rados.0.smithi119.stdout:1270: finishing set_chunk to oid 38 2024-09-18T11:31:15.875 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 124 (ObjNum 231 snap 27 seq_num 231) dirty exists 2024-09-18T11:31:15.875 INFO:tasks.rados.rados.0.smithi119.stdout:1271: finishing write tid 1 to smithi11939506-27 2024-09-18T11:31:15.875 INFO:tasks.rados.rados.0.smithi119.stdout:1271: finishing write tid 2 to smithi11939506-27 2024-09-18T11:31:15.875 INFO:tasks.rados.rados.0.smithi119.stdout:1271: finishing write tid 3 to smithi11939506-27 2024-09-18T11:31:15.875 INFO:tasks.rados.rados.0.smithi119.stdout:1271: finishing write tid 4 to smithi11939506-27 2024-09-18T11:31:15.876 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 161 (ObjNum 239 snap 28 seq_num 239) dirty exists 2024-09-18T11:31:15.876 INFO:tasks.rados.rados.0.smithi119.stdout:1271: left oid 27 (ObjNum 239 snap 28 seq_num 239) 2024-09-18T11:31:15.876 INFO:tasks.rados.rados.0.smithi119.stdout:1272: got (0) Success 2024-09-18T11:31:15.876 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 8 v 30 (ObjNum 53 snap 0 seq_num 53) dirty exists 2024-09-18T11:31:15.876 INFO:tasks.rados.rados.0.smithi119.stdout:1273: seq_num 240 ranges {433680=466703,1577796=455884} 2024-09-18T11:31:15.879 INFO:tasks.rados.rados.0.smithi119.stdout:1273: writing smithi11939506-10 from 433680 to 900383 tid 1 2024-09-18T11:31:15.880 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:15.880 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:15.886 INFO:tasks.rados.rados.0.smithi119.stdout:1273: writing smithi11939506-10 from 1577796 to 2033680 tid 2 2024-09-18T11:31:15.886 INFO:tasks.rados.rados.0.smithi119.stdout:1267: done (6 left) 2024-09-18T11:31:15.887 INFO:tasks.rados.rados.0.smithi119.stdout:1268: done (5 left) 2024-09-18T11:31:15.887 INFO:tasks.rados.rados.0.smithi119.stdout:1269: done (4 left) 2024-09-18T11:31:15.887 INFO:tasks.rados.rados.0.smithi119.stdout:1270: done (3 left) 2024-09-18T11:31:15.887 INFO:tasks.rados.rados.0.smithi119.stdout:1271: done (2 left) 2024-09-18T11:31:15.887 INFO:tasks.rados.rados.0.smithi119.stdout:1272: done (1 left) 2024-09-18T11:31:15.887 INFO:tasks.rados.rados.0.smithi119.stdout:1274: tier_promote oid 40 2024-09-18T11:31:15.887 INFO:tasks.rados.rados.0.smithi119.stdout:1273: finishing write tid 1 to smithi11939506-10 2024-09-18T11:31:15.887 INFO:tasks.rados.rados.0.smithi119.stdout:1275: set_chunk oid 28 target oid 2024-09-18T11:31:15.887 INFO:tasks.rados.rados.0.smithi119.stdout:1275: set_chunk oid 28 offset: 749056 length: 1760768 target oid smithi11939506- offset: 749056 snap 27 2024-09-18T11:31:15.887 INFO:tasks.rados.rados.0.smithi119.stdout:1276: read oid 6 snap -1 2024-09-18T11:31:15.887 INFO:tasks.rados.rados.0.smithi119.stdout:1276: expect deleted 2024-09-18T11:31:15.887 INFO:tasks.rados.rados.0.smithi119.stdout:1277: read oid 44 snap -1 2024-09-18T11:31:15.887 INFO:tasks.rados.rados.0.smithi119.stdout:1277: expect (ObjNum 224 snap 26 seq_num 224) 2024-09-18T11:31:15.888 INFO:tasks.rados.rados.0.smithi119.stdout:1278: tier_evict oid 43 2024-09-18T11:31:15.888 INFO:tasks.rados.rados.0.smithi119.stdout:1278: tier_evict oid 43 snap -1 2024-09-18T11:31:15.888 INFO:tasks.rados.rados.0.smithi119.stdout:1279: set_chunk oid 19 target oid 2024-09-18T11:31:15.890 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:31:15.900 INFO:tasks.rados.rados.0.smithi119.stdout:1279: set_chunk oid 19 offset: 172032 length: 1969664 target oid e2b4292d18ddb83f4e7983af11b940ab108014a3ea18db25ef9d028068b53b45 offset: 0 snap -1 2024-09-18T11:31:15.900 INFO:tasks.rados.rados.0.smithi119.stdout:1280: snap_remove snap 17 2024-09-18T11:31:15.943 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:15.942+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:15.997 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:15.997 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:16.005 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:31:16.020 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:16.018+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:16.026 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T11:31:16.121 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:16.119+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:16.146 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:16.146 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:16.155 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:31:16.240 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:16.239+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:16.296 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:16.296 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:16.304 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:31:16.346 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:16.345+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:16.415 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:16.415 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:16.424 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:16.422+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:16.425 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:31:16.473 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is unset 2024-09-18T11:31:16.474 INFO:tasks.rados.rados.0.smithi119.stdout:1280: done (7 left) 2024-09-18T11:31:16.474 INFO:tasks.rados.rados.0.smithi119.stdout:1281: read oid 8 snap -1 2024-09-18T11:31:16.474 INFO:tasks.rados.rados.0.smithi119.stdout:1281: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-18T11:31:16.474 INFO:tasks.rados.rados.0.smithi119.stdout:1282: read oid 16 snap -1 2024-09-18T11:31:16.474 INFO:tasks.rados.rados.0.smithi119.stdout:1282: expect (ObjNum 215 snap 26 seq_num 215) 2024-09-18T11:31:16.474 INFO:tasks.rados.rados.0.smithi119.stdout:1283: read oid 9 snap -1 2024-09-18T11:31:16.474 INFO:tasks.rados.rados.0.smithi119.stdout:1283: expect (ObjNum 199 snap 23 seq_num 199) 2024-09-18T11:31:16.474 INFO:tasks.rados.rados.0.smithi119.stdout:1284: set_chunk oid 21 target oid 2024-09-18T11:31:16.491 INFO:tasks.rados.rados.0.smithi119.stdout:1284: set_chunk oid 21 offset: 111616 length: 288256 target oid be3608e547e9a7268a63d1d086aa58756904f4bd13e4dddfacc46c62eb9fd92d offset: 0 snap -1 2024-09-18T11:31:16.491 INFO:tasks.rados.rados.0.smithi119.stdout:1285: write (excl) oid 49 current snap is 28 2024-09-18T11:31:16.491 INFO:tasks.rados.rados.0.smithi119.stdout:1285: seq_num 241 ranges {450487=541952,1636278=414209} 2024-09-18T11:31:16.497 INFO:tasks.rados.rados.0.smithi119.stdout:1285: writing smithi11939506-49 from 450487 to 992439 tid 1 2024-09-18T11:31:16.504 INFO:tasks.rados.rados.0.smithi119.stdout:1285: writing smithi11939506-49 from 1636278 to 2050487 tid 2 2024-09-18T11:31:16.504 INFO:tasks.rados.rados.0.smithi119.stdout:1286: read oid 18 snap -1 2024-09-18T11:31:16.504 INFO:tasks.rados.rados.0.smithi119.stdout:1286: expect (ObjNum 205 snap 23 seq_num 205) 2024-09-18T11:31:16.504 INFO:tasks.rados.rados.0.smithi119.stdout:1287: read oid 17 snap -1 2024-09-18T11:31:16.504 INFO:tasks.rados.rados.0.smithi119.stdout:1287: expect deleted 2024-09-18T11:31:16.504 INFO:tasks.rados.rados.0.smithi119.stdout:1288: read oid 22 snap 24 2024-09-18T11:31:16.505 INFO:tasks.rados.rados.0.smithi119.stdout:1288: expect (ObjNum 181 snap 20 seq_num 181) 2024-09-18T11:31:16.505 INFO:tasks.rados.rados.0.smithi119.stdout:1289: read oid 2 snap -1 2024-09-18T11:31:16.505 INFO:tasks.rados.rados.0.smithi119.stdout:1289: expect (ObjNum 195 snap 23 seq_num 195) 2024-09-18T11:31:16.505 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:31:16.505 INFO:tasks.rados.rados.0.smithi119.stdout:1275: finishing set_chunk to oid 28 2024-09-18T11:31:16.505 INFO:tasks.rados.rados.0.smithi119.stdout:1275: got expected ERANGE (src dne) 2024-09-18T11:31:16.505 INFO:tasks.rados.rados.0.smithi119.stdout:1278: got (22) Invalid argument 2024-09-18T11:31:16.505 INFO:tasks.rados.rados.0.smithi119.stdout:1274: got (0) Success 2024-09-18T11:31:16.505 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 90 (ObjNum 175 snap 20 seq_num 175) dirty exists 2024-09-18T11:31:16.505 INFO:tasks.rados.rados.0.smithi119.stdout:1273: finishing write tid 2 to smithi11939506-10 2024-09-18T11:31:16.505 INFO:tasks.rados.rados.0.smithi119.stdout:1273: finishing write tid 3 to smithi11939506-10 2024-09-18T11:31:16.506 INFO:tasks.rados.rados.0.smithi119.stdout:1273: finishing write tid 4 to smithi11939506-10 2024-09-18T11:31:16.506 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 10 v 132 (ObjNum 240 snap 28 seq_num 240) dirty exists 2024-09-18T11:31:16.506 INFO:tasks.rados.rados.0.smithi119.stdout:1273: left oid 10 (ObjNum 240 snap 28 seq_num 240) 2024-09-18T11:31:16.506 INFO:tasks.rados.rados.0.smithi119.stdout:1277: expect (ObjNum 224 snap 26 seq_num 224) 2024-09-18T11:31:16.524 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:16.523+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:16.538 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:16.538 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:16.547 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:31:16.625 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:16.623+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:16.645 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:16.645 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:16.653 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:31:16.738 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:16.736+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:16.739 INFO:tasks.rados.rados.0.smithi119.stdout:1279: finishing set_chunk to oid 19 2024-09-18T11:31:16.739 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 19 set_chunk returned error code -95 offset: 172032 length: 1969664 2024-09-18T11:31:16.739 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 0 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-18T11:31:16.739 INFO:tasks.rados.rados.0.smithi119.stdout:1281: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-18T11:31:16.793 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:16.794 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:16.801 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:31:16.842 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:16.840+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:16.869 INFO:tasks.rados.rados.0.smithi119.stdout:1283: expect (ObjNum 199 snap 23 seq_num 199) 2024-09-18T11:31:16.928 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:16.926+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:16.938 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:16.939 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:16.947 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:31:16.967 INFO:tasks.rados.rados.0.smithi119.stdout:1282: expect (ObjNum 215 snap 26 seq_num 215) 2024-09-18T11:31:17.029 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:17.028+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:17.085 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:17.086 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:17.094 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:31:17.130 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:17.129+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:17.192 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:17.192 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:17.201 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:31:17.231 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:17.230+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:17.234 INFO:tasks.rados.rados.0.smithi119.stdout:1284: finishing set_chunk to oid 21 2024-09-18T11:31:17.234 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 168 (ObjNum 222 snap 26 seq_num 222) dirty exists 2024-09-18T11:31:17.234 INFO:tasks.rados.rados.0.smithi119.stdout:1288: expect (ObjNum 181 snap 20 seq_num 181) 2024-09-18T11:31:17.298 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:17.298 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:17.307 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:31:17.351 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:17.349+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:17.406 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:17.406 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:17.414 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:31:17.443 INFO:tasks.rados.rados.0.smithi119.stdout:1285: finishing write tid 1 to smithi11939506-49 2024-09-18T11:31:17.443 INFO:tasks.rados.rados.0.smithi119.stdout:1285: finishing write tid 2 to smithi11939506-49 2024-09-18T11:31:17.443 INFO:tasks.rados.rados.0.smithi119.stdout:1285: finishing write tid 3 to smithi11939506-49 2024-09-18T11:31:17.443 INFO:tasks.rados.rados.0.smithi119.stdout:1285: finishing write tid 4 to smithi11939506-49 2024-09-18T11:31:17.443 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 274 (ObjNum 241 snap 28 seq_num 241) dirty exists 2024-09-18T11:31:17.443 INFO:tasks.rados.rados.0.smithi119.stdout:1285: left oid 49 (ObjNum 241 snap 28 seq_num 241) 2024-09-18T11:31:17.443 INFO:tasks.rados.rados.0.smithi119.stdout:1289: expect (ObjNum 195 snap 23 seq_num 195) 2024-09-18T11:31:17.455 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:17.453+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:17.534 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:17.532+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:17.554 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:17.554 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:17.563 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:31:17.565 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:31:17.648 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:17.646+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:17.665 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:17.680 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:17.680 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:17.682 INFO:tasks.rados.rados.0.smithi119.stdout:1286: expect (ObjNum 205 snap 23 seq_num 205) 2024-09-18T11:31:17.696 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:17.696 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:17.702 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:17.702 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:17.711 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:31:17.711 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:17.712 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:17.726 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:17.726 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:17.742 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:17.742 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:17.752 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:17.750+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:17.758 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:17.758 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:17.773 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:17.774 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:17.777 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:17.777 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:17.837 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:17.834+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:17.851 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:17.852 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:17.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:17.852 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:17.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.852 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:17.852 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:17.852 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.691865239999998, 2024-09-18T11:31:17.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055400850000000001, 2024-09-18T11:31:17.852 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:17.852 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:17.853 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:17.853 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:17.853 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:17.853 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 43 2024-09-18T11:31:17.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.853 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:17.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:17.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:17.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:17.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:17.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:17.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:17.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:17.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:17.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157504+0000", 2024-09-18T11:31:17.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4519999999999996e-06 2024-09-18T11:31:17.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:17.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157534+0000", 2024-09-18T11:31:17.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.97e-05 2024-09-18T11:31:17.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:17.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157539+0000", 2024-09-18T11:31:17.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4360000000000001e-06 2024-09-18T11:31:17.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:17.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159428+0000", 2024-09-18T11:31:17.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018888220000000001 2024-09-18T11:31:17.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:17.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159479+0000", 2024-09-18T11:31:17.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0608000000000001e-05 2024-09-18T11:31:17.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:17.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159559+0000", 2024-09-18T11:31:17.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0667000000000005e-05 2024-09-18T11:31:17.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:17.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162779+0000", 2024-09-18T11:31:17.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032197459999999999 2024-09-18T11:31:17.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:17.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162950+0000", 2024-09-18T11:31:17.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017059599999999999 2024-09-18T11:31:17.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:17.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163001+0000", 2024-09-18T11:31:17.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0968999999999999e-05 2024-09-18T11:31:17.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:17.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163040+0000", 2024-09-18T11:31:17.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8843000000000001e-05 2024-09-18T11:31:17.858 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.858 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:17.859 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.859 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:17.859 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:17.859 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.691814085999994, 2024-09-18T11:31:17.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:31:17.859 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:17.859 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:17.859 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:17.859 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:17.859 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:17.860 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:31:17.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.860 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:17.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:17.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:17.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:17.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:17.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:17.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:31:17.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:17.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:17.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:31:17.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:31:17.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:17.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:31:17.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:31:17.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:17.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:31:17.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:17.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:17.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:31:17.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:31:17.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:17.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:31:17.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:31:17.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:17.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:31:17.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:31:17.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:17.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:31:17.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:31:17.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:17.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:31:17.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:31:17.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:17.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:31:17.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:31:17.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:17.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:31:17.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:31:17.865 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.865 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:17.865 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.866 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:17.866 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:17.866 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.691687823999999, 2024-09-18T11:31:17.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055717040000000002, 2024-09-18T11:31:17.866 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:17.866 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:17.866 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:17.866 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:17.866 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:17.866 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 45 2024-09-18T11:31:17.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.867 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:17.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:17.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:17.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:17.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:17.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:17.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157676+0000", 2024-09-18T11:31:17.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:17.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:17.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157694+0000", 2024-09-18T11:31:17.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7422999999999999e-05 2024-09-18T11:31:17.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:17.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157704+0000", 2024-09-18T11:31:17.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9000000000000001e-06 2024-09-18T11:31:17.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:17.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157718+0000", 2024-09-18T11:31:17.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4788e-05 2024-09-18T11:31:17.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:17.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160937+0000", 2024-09-18T11:31:17.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00321886 2024-09-18T11:31:17.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:17.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161023+0000", 2024-09-18T11:31:17.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6063999999999995e-05 2024-09-18T11:31:17.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:17.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161202+0000", 2024-09-18T11:31:17.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017912599999999999 2024-09-18T11:31:17.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:17.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163052+0000", 2024-09-18T11:31:17.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001849554 2024-09-18T11:31:17.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:17.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163204+0000", 2024-09-18T11:31:17.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015186900000000001 2024-09-18T11:31:17.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:17.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163229+0000", 2024-09-18T11:31:17.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:31:17.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:17.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163249+0000", 2024-09-18T11:31:17.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9992000000000001e-05 2024-09-18T11:31:17.872 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.872 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:17.872 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.873 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:17.873 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:17.873 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.691638710999996, 2024-09-18T11:31:17.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:31:17.873 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:17.873 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:17.873 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:17.873 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:17.873 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:17.873 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:31:17.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.874 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:17.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:17.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:17.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:17.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:17.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:17.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:17.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:17.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:17.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:31:17.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:31:17.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:17.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:31:17.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:31:17.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:17.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:31:17.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:31:17.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:17.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:31:17.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:31:17.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:17.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:31:17.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:31:17.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:17.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:31:17.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:31:17.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:17.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:31:17.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:31:17.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:17.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:31:17.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:31:17.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:17.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:31:17.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:31:17.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:17.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:31:17.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:31:17.879 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.879 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:17.879 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.879 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:17.879 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:17.879 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.691596180000005, 2024-09-18T11:31:17.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:31:17.879 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:17.880 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:17.880 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:17.880 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:17.880 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:17.880 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:31:17.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.880 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:17.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:17.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:17.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:17.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:17.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:17.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:31:17.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:17.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:17.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:31:17.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:31:17.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:17.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:31:17.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:31:17.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:17.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:31:17.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:31:17.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:17.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:31:17.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:31:17.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:17.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:31:17.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:31:17.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:17.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:31:17.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:31:17.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:17.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:31:17.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:31:17.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:17.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:31:17.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:31:17.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:17.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:31:17.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:31:17.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:17.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:31:17.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:31:17.885 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.885 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:17.885 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.885 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:17.885 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:17.885 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.69154125, 2024-09-18T11:31:17.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:31:17.885 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:17.886 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:17.886 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:17.886 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:17.886 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:17.886 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:31:17.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.886 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:17.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:17.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:17.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:17.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:17.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:17.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:17.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:17.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:17.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:31:17.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:31:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:31:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:31:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:31:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:31:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:31:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:31:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:31:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:31:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:31:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:31:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:31:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:31:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:31:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:31:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:31:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:31:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:31:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:31:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.691531269999999, 2024-09-18T11:31:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:31:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:31:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:31:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:31:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:31:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:31:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:31:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:31:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:31:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:31:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:31:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:31:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:31:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:31:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:31:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:31:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:31:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:31:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:31:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:31:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:31:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:31:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.691519670999995, 2024-09-18T11:31:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:31:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:31:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:31:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:31:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:31:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:31:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:31:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:31:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:31:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:31:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:31:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:31:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:31:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:31:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:31:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:31:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:31:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:31:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:31:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:31:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:31:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:31:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.691463189999993, 2024-09-18T11:31:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:31:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:31:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:31:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:31:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:31:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:31:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:31:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:31:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:31:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:31:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:31:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:31:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:31:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:31:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:31:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:31:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:31:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:31:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:31:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:31:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:31:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:31:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:31:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.691449366000001, 2024-09-18T11:31:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:31:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:31:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:17.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:31:17.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:31:17.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:17.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:31:17.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:31:17.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:17.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:31:17.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:31:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:31:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:31:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:31:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:31:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:31:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:31:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:31:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:31:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:31:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:31:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:31:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:31:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:31:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:31:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.275270458999998, 2024-09-18T11:31:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:31:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:31:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:17.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:17.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:17.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:17.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:17.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:17.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:17.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:17.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:31:17.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:31:17.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:17.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:31:17.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:31:17.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:17.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:31:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:31:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:31:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:31:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:31:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:31:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:31:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:31:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:31:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:31:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:31:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:31:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:31:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:31:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:31:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:31:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.275247159000003, 2024-09-18T11:31:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:31:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:31:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:17.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:17.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:17.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:17.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:17.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:31:17.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:17.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:31:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:31:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:31:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:31:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:31:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:31:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:31:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:31:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:31:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:31:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:31:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:31:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:31:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:31:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:31:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:31:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:31:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:31:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:31:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:31:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:17.928 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:17.928 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.275161550000007, 2024-09-18T11:31:17.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:31:17.928 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:17.928 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:17.928 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:17.928 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:17.928 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:17.928 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:31:17.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.928 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:17.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:31:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:31:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:31:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:31:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:31:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:31:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:31:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:31:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:31:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:31:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:31:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:31:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:31:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:31:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:31:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:31:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:31:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:31:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:31:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:31:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.275150585000006, 2024-09-18T11:31:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:31:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:31:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:31:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:31:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:31:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:31:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:31:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:31:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:31:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:31:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:31:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:31:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:31:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:31:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:31:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:31:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:31:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:31:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:31:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:31:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:31:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:31:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.275139417000005, 2024-09-18T11:31:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:31:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:31:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:31:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:31:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:31:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:31:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:31:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:31:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:31:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:31:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:31:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:31:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:31:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:31:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:31:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:31:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:31:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:31:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:31:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:31:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:31:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:31:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.275074087999997, 2024-09-18T11:31:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:31:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:31:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:31:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:31:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:31:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:31:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:31:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:31:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:31:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:31:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:31:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:31:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:31:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:31:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:31:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:31:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:31:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:31:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:31:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:31:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:31:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:31:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:31:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.264912109999997, 2024-09-18T11:31:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:31:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:31:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:31:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:31:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:31:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:31:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:31:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:31:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:31:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:31:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:31:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:31:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:31:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:31:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:31:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:31:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:31:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:31:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:31:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:31:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:31:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:31:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:31:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.264819630999995, 2024-09-18T11:31:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:31:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:31:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:31:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:31:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:31:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:31:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:31:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:31:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:31:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:31:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:31:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:31:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:31:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:31:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:31:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:31:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:31:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:31:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:31:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:31:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:31:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:31:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1788 4.8 e60/22 4:1898b15d:::smithi11939506-25:head v 60'12, mlcod=58'9)", 2024-09-18T11:31:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 34.932414141999999, 2024-09-18T11:31:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058348599999999999, 2024-09-18T11:31:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916949+0000", 2024-09-18T11:31:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917602+0000", 2024-09-18T11:31:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00065337100000000003 2024-09-18T11:31:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917621+0000", 2024-09-18T11:31:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9287e-05 2024-09-18T11:31:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917629+0000", 2024-09-18T11:31:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3780000000000003e-06 2024-09-18T11:31:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917677+0000", 2024-09-18T11:31:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8068000000000001e-05 2024-09-18T11:31:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917699+0000", 2024-09-18T11:31:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2149000000000001e-05 2024-09-18T11:31:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922747+0000", 2024-09-18T11:31:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050477609999999996 2024-09-18T11:31:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922785+0000", 2024-09-18T11:31:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8726000000000002e-05 2024-09-18T11:31:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:58 4.3 e91/67 4:c70c3162:::smithi11939506-35:18 v 91'101, mlcod=86'95)", 2024-09-18T11:31:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.8337559190000001, 2024-09-18T11:31:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055647500000000003, 2024-09-18T11:31:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015608+0000", 2024-09-18T11:31:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015614+0000", 2024-09-18T11:31:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5310000000000001e-06 2024-09-18T11:31:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015622+0000", 2024-09-18T11:31:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0809999999999994e-06 2024-09-18T11:31:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015627+0000", 2024-09-18T11:31:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8439999999999997e-06 2024-09-18T11:31:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016003+0000", 2024-09-18T11:31:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037652999999999997 2024-09-18T11:31:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016024+0000", 2024-09-18T11:31:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0780000000000001e-05 2024-09-18T11:31:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021141+0000", 2024-09-18T11:31:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051166639999999999 2024-09-18T11:31:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021174+0000", 2024-09-18T11:31:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2839000000000002e-05 2024-09-18T11:31:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:17.972 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:17.975 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:31:17.976 INFO:tasks.rados.rados.0.smithi119.stdout:1273: done (15 left) 2024-09-18T11:31:17.976 INFO:tasks.rados.rados.0.smithi119.stdout:1274: done (14 left) 2024-09-18T11:31:17.976 INFO:tasks.rados.rados.0.smithi119.stdout:1275: done (13 left) 2024-09-18T11:31:17.976 INFO:tasks.rados.rados.0.smithi119.stdout:1276: done (12 left) 2024-09-18T11:31:17.976 INFO:tasks.rados.rados.0.smithi119.stdout:1277: done (11 left) 2024-09-18T11:31:17.977 INFO:tasks.rados.rados.0.smithi119.stdout:1278: done (10 left) 2024-09-18T11:31:17.977 INFO:tasks.rados.rados.0.smithi119.stdout:1279: done (9 left) 2024-09-18T11:31:17.977 INFO:tasks.rados.rados.0.smithi119.stdout:1281: done (8 left) 2024-09-18T11:31:17.977 INFO:tasks.rados.rados.0.smithi119.stdout:1282: done (7 left) 2024-09-18T11:31:17.977 INFO:tasks.rados.rados.0.smithi119.stdout:1283: done (6 left) 2024-09-18T11:31:17.977 INFO:tasks.rados.rados.0.smithi119.stdout:1284: done (5 left) 2024-09-18T11:31:17.977 INFO:tasks.rados.rados.0.smithi119.stdout:1285: done (4 left) 2024-09-18T11:31:17.977 INFO:tasks.rados.rados.0.smithi119.stdout:1286: done (3 left) 2024-09-18T11:31:17.977 INFO:tasks.rados.rados.0.smithi119.stdout:1287: done (2 left) 2024-09-18T11:31:17.977 INFO:tasks.rados.rados.0.smithi119.stdout:1288: done (1 left) 2024-09-18T11:31:17.977 INFO:tasks.rados.rados.0.smithi119.stdout:1289: done (0 left) 2024-09-18T11:31:17.977 INFO:tasks.rados.rados.0.smithi119.stdout:1290: read oid 2 snap 20 2024-09-18T11:31:17.977 INFO:tasks.rados.rados.0.smithi119.stdout:1290: expect (ObjNum 163 snap 20 seq_num 163) 2024-09-18T11:31:17.978 INFO:tasks.rados.rados.0.smithi119.stdout:1291: set_chunk oid 9 target oid 2024-09-18T11:31:17.978 INFO:tasks.rados.rados.0.smithi119.stdout:1291: set_chunk oid 9 offset: 862208 length: 1963008 target oid d1f5dedcb3022b9197e6a56ae9e1b2a5faa666f6ebf8e29a923c1fa409f4377f offset: 0 snap -1 2024-09-18T11:31:17.978 INFO:tasks.rados.rados.0.smithi119.stdout:1292: read oid 42 snap -1 2024-09-18T11:31:17.978 INFO:tasks.rados.rados.0.smithi119.stdout:1292: expect (ObjNum 99 snap 6 seq_num 99) 2024-09-18T11:31:17.978 INFO:tasks.rados.rados.0.smithi119.stdout:1293: read oid 29 snap -1 2024-09-18T11:31:17.978 INFO:tasks.rados.rados.0.smithi119.stdout:1293: expect (ObjNum 210 snap 23 seq_num 210) 2024-09-18T11:31:17.978 INFO:tasks.rados.rados.0.smithi119.stdout:1294: read oid 44 snap 21 2024-09-18T11:31:17.978 INFO:tasks.rados.rados.0.smithi119.stdout:1294: expect (ObjNum 72 snap 3 seq_num 72) 2024-09-18T11:31:17.978 INFO:tasks.rados.rados.0.smithi119.stdout:1295: tier_evict oid 27 2024-09-18T11:31:17.978 INFO:tasks.rados.rados.0.smithi119.stdout:1295: tier_evict oid 27 snap -1 2024-09-18T11:31:17.978 INFO:tasks.rados.rados.0.smithi119.stdout:1296: rollback oid 3 current snap is 28 2024-09-18T11:31:17.978 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 3 to 24 2024-09-18T11:31:17.978 INFO:tasks.rados.rados.0.smithi119.stdout:1290: expect (ObjNum 163 snap 20 seq_num 163) 2024-09-18T11:31:17.979 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:17.973+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:18.067 INFO:tasks.rados.rados.0.smithi119.stdout:1293: expect (ObjNum 210 snap 23 seq_num 210) 2024-09-18T11:31:18.076 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:18.074+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:18.079 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:18.080 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:18.080 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:18.080 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:18.088 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:31:18.176 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:18.175+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:18.185 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:18.185 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:18.186 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:18.186 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:18.186 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:18.195 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:31:18.276 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:18.275+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:18.336 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:18.336 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:18.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:18.336 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:18.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.336 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:887 4.7 4:e0d95e10:::smithi11939506-22:head [read 0~2352947,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:18.336 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:31:18.337 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 67.217458261000004, 2024-09-18T11:31:18.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079115169999999999, 2024-09-18T11:31:18.337 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.337 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:18.337 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:18.337 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:18.337 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:18.337 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 887 2024-09-18T11:31:18.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.337 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:31:18.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116367+0000", 2024-09-18T11:31:18.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116365+0000", 2024-09-18T11:31:18.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:18.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116376+0000", 2024-09-18T11:31:18.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1471000000000001e-05 2024-09-18T11:31:18.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116394+0000", 2024-09-18T11:31:18.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8088e-05 2024-09-18T11:31:18.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116402+0000", 2024-09-18T11:31:18.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2320000000000004e-06 2024-09-18T11:31:18.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116458+0000", 2024-09-18T11:31:18.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6536000000000002e-05 2024-09-18T11:31:18.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:18.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116466+0000", 2024-09-18T11:31:18.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0819999999999999e-06 2024-09-18T11:31:18.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.122509+0000", 2024-09-18T11:31:18.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060431879999999997 2024-09-18T11:31:18.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.122577+0000", 2024-09-18T11:31:18.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7646999999999998e-05 2024-09-18T11:31:18.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124278+0000", 2024-09-18T11:31:18.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0017013200000000001 2024-09-18T11:31:18.341 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.341 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.341 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.342 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:888 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:18.342 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:18.342 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 67.217404798000004, 2024-09-18T11:31:18.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082282120000000004, 2024-09-18T11:31:18.342 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.342 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:18.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:18.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:18.342 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:18.342 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 888 2024-09-18T11:31:18.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.342 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:18.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:18.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:18.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116423+0000", 2024-09-18T11:31:18.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1e-06 2024-09-18T11:31:18.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:31:18.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5510000000000001e-06 2024-09-18T11:31:18.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116431+0000", 2024-09-18T11:31:18.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6109999999999998e-06 2024-09-18T11:31:18.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116527+0000", 2024-09-18T11:31:18.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6212999999999995e-05 2024-09-18T11:31:18.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:18.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116532+0000", 2024-09-18T11:31:18.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6140000000000002e-06 2024-09-18T11:31:18.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124314+0000", 2024-09-18T11:31:18.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077818929999999998 2024-09-18T11:31:18.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124411+0000", 2024-09-18T11:31:18.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6992999999999998e-05 2024-09-18T11:31:18.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124649+0000", 2024-09-18T11:31:18.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237729 2024-09-18T11:31:18.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.346 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.347 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.347 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:889 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:18.347 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:18.347 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 67.217380148000004, 2024-09-18T11:31:18.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085383349999999993, 2024-09-18T11:31:18.347 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.347 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:18.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:18.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:18.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:18.347 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 889 2024-09-18T11:31:18.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.348 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:18.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:18.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:18.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116447+0000", 2024-09-18T11:31:18.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.854e-06 2024-09-18T11:31:18.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116452+0000", 2024-09-18T11:31:18.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0250000000000002e-06 2024-09-18T11:31:18.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116455+0000", 2024-09-18T11:31:18.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8739999999999999e-06 2024-09-18T11:31:18.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116565+0000", 2024-09-18T11:31:18.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110741 2024-09-18T11:31:18.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:18.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116571+0000", 2024-09-18T11:31:18.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4990000000000002e-06 2024-09-18T11:31:18.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124691+0000", 2024-09-18T11:31:18.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081200339999999999 2024-09-18T11:31:18.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124784+0000", 2024-09-18T11:31:18.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2886999999999993e-05 2024-09-18T11:31:18.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124983+0000", 2024-09-18T11:31:18.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019969000000000001 2024-09-18T11:31:18.351 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.352 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.352 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.352 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:46 4.a e31/22 4:552523b6:::smithi11939506-38:head v 31'20, mlcod=30'18)", 2024-09-18T11:31:18.352 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:31:18.352 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 67.203428516000002, 2024-09-18T11:31:18.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080208420000000002, 2024-09-18T11:31:18.352 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.352 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.352 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:31:18.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130397+0000", 2024-09-18T11:31:18.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.130396+0000", 2024-09-18T11:31:18.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133245+0000", 2024-09-18T11:31:18.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028486929999999998 2024-09-18T11:31:18.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133253+0000", 2024-09-18T11:31:18.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1540000000000002e-06 2024-09-18T11:31:18.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133261+0000", 2024-09-18T11:31:18.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3620000000000006e-06 2024-09-18T11:31:18.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133319+0000", 2024-09-18T11:31:18.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7719000000000001e-05 2024-09-18T11:31:18.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.133341+0000", 2024-09-18T11:31:18.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1946999999999998e-05 2024-09-18T11:31:18.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138368+0000", 2024-09-18T11:31:18.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005026712 2024-09-18T11:31:18.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.138418+0000", 2024-09-18T11:31:18.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9864000000000002e-05 2024-09-18T11:31:18.356 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.356 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.356 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.356 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1126 3.0 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head [write 0~2520064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:31:18.356 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:31:18.356 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.351938169999997, 2024-09-18T11:31:18.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086197050000000001, 2024-09-18T11:31:18.357 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.357 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.357 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:18.357 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:18.357 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:18.357 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1126 2024-09-18T11:31:18.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.357 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:31:18.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:31:18.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981886+0000", 2024-09-18T11:31:18.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:18.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983017+0000", 2024-09-18T11:31:18.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00113131 2024-09-18T11:31:18.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983037+0000", 2024-09-18T11:31:18.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0231e-05 2024-09-18T11:31:18.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983044+0000", 2024-09-18T11:31:18.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9539999999999998e-06 2024-09-18T11:31:18.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983087+0000", 2024-09-18T11:31:18.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3040000000000001e-05 2024-09-18T11:31:18.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983193+0000", 2024-09-18T11:31:18.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105682 2024-09-18T11:31:18.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:18.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983287+0000", 2024-09-18T11:31:18.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4066000000000004e-05 2024-09-18T11:31:18.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:18.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.986918+0000", 2024-09-18T11:31:18.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036311970000000001 2024-09-18T11:31:18.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:18.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990442+0000", 2024-09-18T11:31:18.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035243119999999999 2024-09-18T11:31:18.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990476+0000", 2024-09-18T11:31:18.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4028e-05 2024-09-18T11:31:18.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990507+0000", 2024-09-18T11:31:18.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0284999999999999e-05 2024-09-18T11:31:18.362 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.362 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.362 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.362 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:262 4.9 e53/22 4:99feffe9:::smithi11939506-45:13 v 53'17, mlcod=51'14)", 2024-09-18T11:31:18.362 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:18.362 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 44.334834354999998, 2024-09-18T11:31:18.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013878679, 2024-09-18T11:31:18.362 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.363 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.363 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:18.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:18.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:18.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998997+0000", 2024-09-18T11:31:18.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1099999999999999e-06 2024-09-18T11:31:18.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999008+0000", 2024-09-18T11:31:18.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1321000000000001e-05 2024-09-18T11:31:18.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999015+0000", 2024-09-18T11:31:18.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3000000000000004e-06 2024-09-18T11:31:18.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999055+0000", 2024-09-18T11:31:18.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0058000000000002e-05 2024-09-18T11:31:18.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999070+0000", 2024-09-18T11:31:18.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4328000000000001e-05 2024-09-18T11:31:18.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012465+0000", 2024-09-18T11:31:18.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013395638 2024-09-18T11:31:18.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012870+0000", 2024-09-18T11:31:18.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040423400000000001 2024-09-18T11:31:18.366 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.366 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.366 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.367 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:205 4.b e53/22 4:dba5bd39:::smithi11939506-17:13 v 53'77, mlcod=51'74)", 2024-09-18T11:31:18.367 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:18.367 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 44.334785093999997, 2024-09-18T11:31:18.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013359586, 2024-09-18T11:31:18.367 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.367 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.367 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:18.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:18.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:18.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999048+0000", 2024-09-18T11:31:18.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3550000000000005e-06 2024-09-18T11:31:18.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999056+0000", 2024-09-18T11:31:18.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3259999999999996e-06 2024-09-18T11:31:18.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999063+0000", 2024-09-18T11:31:18.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6409999999999996e-06 2024-09-18T11:31:18.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999098+0000", 2024-09-18T11:31:18.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5203999999999997e-05 2024-09-18T11:31:18.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999115+0000", 2024-09-18T11:31:18.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6478999999999999e-05 2024-09-18T11:31:18.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012362+0000", 2024-09-18T11:31:18.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01324768 2024-09-18T11:31:18.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012400+0000", 2024-09-18T11:31:18.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.731e-05 2024-09-18T11:31:18.371 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.371 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.371 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.371 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:88 4.a e53/22 4:588debc2:::smithi11939506-18:13 v 53'45, mlcod=51'42)", 2024-09-18T11:31:18.371 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:18.371 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 44.332002670999998, 2024-09-18T11:31:18.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011307667, 2024-09-18T11:31:18.371 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.371 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.371 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:18.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:18.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001822+0000", 2024-09-18T11:31:18.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001827+0000", 2024-09-18T11:31:18.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.378e-06 2024-09-18T11:31:18.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001839+0000", 2024-09-18T11:31:18.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1225e-05 2024-09-18T11:31:18.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001844+0000", 2024-09-18T11:31:18.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2299999999999999e-06 2024-09-18T11:31:18.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001891+0000", 2024-09-18T11:31:18.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6961e-05 2024-09-18T11:31:18.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001906+0000", 2024-09-18T11:31:18.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5158e-05 2024-09-18T11:31:18.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013103+0000", 2024-09-18T11:31:18.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011197314 2024-09-18T11:31:18.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013130+0000", 2024-09-18T11:31:18.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6900999999999999e-05 2024-09-18T11:31:18.375 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.375 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.375 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.376 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:89 4.a e53/22 4:552523b6:::smithi11939506-38:13 v 53'47, mlcod=51'42)", 2024-09-18T11:31:18.376 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:18.376 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 44.331443450999998, 2024-09-18T11:31:18.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011327614, 2024-09-18T11:31:18.376 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.376 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.376 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:18.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:18.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002381+0000", 2024-09-18T11:31:18.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002397+0000", 2024-09-18T11:31:18.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5688e-05 2024-09-18T11:31:18.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002406+0000", 2024-09-18T11:31:18.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0229999999999998e-06 2024-09-18T11:31:18.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002415+0000", 2024-09-18T11:31:18.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8959999999999992e-06 2024-09-18T11:31:18.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012386+0000", 2024-09-18T11:31:18.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0099711450000000007 2024-09-18T11:31:18.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012405+0000", 2024-09-18T11:31:18.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9389e-05 2024-09-18T11:31:18.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013676+0000", 2024-09-18T11:31:18.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012708649999999999 2024-09-18T11:31:18.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013709+0000", 2024-09-18T11:31:18.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3207000000000002e-05 2024-09-18T11:31:18.380 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.380 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.380 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.380 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1643 4.7 4:e73ba68f:::smithi11939506-1:head [read 0~3983226,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:18.380 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:31:18.380 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.999938274999998, 2024-09-18T11:31:18.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079617669999999998, 2024-09-18T11:31:18.380 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.380 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:18.380 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:18.381 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:18.381 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:18.381 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1643 2024-09-18T11:31:18.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.381 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:31:18.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:31:18.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333887+0000", 2024-09-18T11:31:18.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333889+0000", 2024-09-18T11:31:18.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8890000000000002e-06 2024-09-18T11:31:18.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333896+0000", 2024-09-18T11:31:18.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.156e-06 2024-09-18T11:31:18.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333900+0000", 2024-09-18T11:31:18.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.138e-06 2024-09-18T11:31:18.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.336677+0000", 2024-09-18T11:31:18.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0027774169999999999 2024-09-18T11:31:18.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.336768+0000", 2024-09-18T11:31:18.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0983999999999995e-05 2024-09-18T11:31:18.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341849+0000", 2024-09-18T11:31:18.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050805140000000004 2024-09-18T11:31:18.384 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.384 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.384 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.384 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1644 4.7 4:e73ba68f:::smithi11939506-1:head [sparse-read 0~2695582] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:18.385 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:18.385 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.999921153000003, 2024-09-18T11:31:18.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084764039999999999, 2024-09-18T11:31:18.385 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.385 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:18.385 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:18.385 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:18.385 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:18.385 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1644 2024-09-18T11:31:18.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.385 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:18.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:18.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:18.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333906+0000", 2024-09-18T11:31:18.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.243e-06 2024-09-18T11:31:18.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333911+0000", 2024-09-18T11:31:18.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2959999999999998e-06 2024-09-18T11:31:18.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333915+0000", 2024-09-18T11:31:18.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6720000000000002e-06 2024-09-18T11:31:18.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341879+0000", 2024-09-18T11:31:18.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079638699999999996 2024-09-18T11:31:18.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341978+0000", 2024-09-18T11:31:18.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9562000000000007e-05 2024-09-18T11:31:18.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342380+0000", 2024-09-18T11:31:18.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000402113 2024-09-18T11:31:18.388 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.389 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.389 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.389 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1645 4.7 4:e73ba68f:::smithi11939506-1:head [read 0~3983226,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:18.389 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:18.389 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.999906258999999, 2024-09-18T11:31:18.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010539468999999999, 2024-09-18T11:31:18.389 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.389 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:18.389 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:18.389 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:18.389 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:18.389 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1645 2024-09-18T11:31:18.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.390 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:18.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:18.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:18.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333921+0000", 2024-09-18T11:31:18.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8909999999999999e-06 2024-09-18T11:31:18.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333927+0000", 2024-09-18T11:31:18.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4229999999999996e-06 2024-09-18T11:31:18.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333932+0000", 2024-09-18T11:31:18.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8280000000000002e-06 2024-09-18T11:31:18.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342418+0000", 2024-09-18T11:31:18.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084862219999999999 2024-09-18T11:31:18.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342515+0000", 2024-09-18T11:31:18.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7287999999999994e-05 2024-09-18T11:31:18.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.344458+0000", 2024-09-18T11:31:18.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019431660000000001 2024-09-18T11:31:18.393 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.393 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.393 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.393 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1669 4.4 4:2d9c3f7f:::smithi11939506-5:head [read 0~3764735,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-18T11:31:18.393 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:31:18.393 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.510957361000003, 2024-09-18T11:31:18.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0083694990000000007, 2024-09-18T11:31:18.394 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.394 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:18.394 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:18.394 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:18.394 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:18.394 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1669 2024-09-18T11:31:18.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.394 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:31:18.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:31:18.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822867+0000", 2024-09-18T11:31:18.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822871+0000", 2024-09-18T11:31:18.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:31:18.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822877+0000", 2024-09-18T11:31:18.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8000000000000004e-06 2024-09-18T11:31:18.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822880+0000", 2024-09-18T11:31:18.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5669999999999999e-06 2024-09-18T11:31:18.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822974+0000", 2024-09-18T11:31:18.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4048999999999993e-05 2024-09-18T11:31:18.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:18.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822980+0000", 2024-09-18T11:31:18.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2789999999999998e-06 2024-09-18T11:31:18.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.829160+0000", 2024-09-18T11:31:18.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0061800179999999998 2024-09-18T11:31:18.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.829248+0000", 2024-09-18T11:31:18.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7707000000000006e-05 2024-09-18T11:31:18.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.831237+0000", 2024-09-18T11:31:18.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001989827 2024-09-18T11:31:18.398 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.398 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.398 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.398 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:100 4.a e58/22 4:5975a5ec:::smithi11939506-36:18 v 58'55, mlcod=56'50)", 2024-09-18T11:31:18.398 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:18.399 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.060676123, 2024-09-18T11:31:18.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0088255, 2024-09-18T11:31:18.399 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.399 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.399 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:18.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:18.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:18.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273155+0000", 2024-09-18T11:31:18.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.212e-06 2024-09-18T11:31:18.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273164+0000", 2024-09-18T11:31:18.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7609999999999996e-06 2024-09-18T11:31:18.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273168+0000", 2024-09-18T11:31:18.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2819999999999998e-06 2024-09-18T11:31:18.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274162+0000", 2024-09-18T11:31:18.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099398600000000009 2024-09-18T11:31:18.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274180+0000", 2024-09-18T11:31:18.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8236e-05 2024-09-18T11:31:18.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281943+0000", 2024-09-18T11:31:18.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007763308 2024-09-18T11:31:18.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281975+0000", 2024-09-18T11:31:18.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1202999999999999e-05 2024-09-18T11:31:18.403 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.403 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.403 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.403 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1954 4.7 4:ef60946b:::smithi11939506-8:head [read 0~2890771,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:18.403 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:18.403 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.993925083000001, 2024-09-18T11:31:18.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091142180000000003, 2024-09-18T11:31:18.403 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.403 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:18.403 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:18.403 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:18.403 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:18.404 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1954 2024-09-18T11:31:18.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.404 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:18.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:18.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:18.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339903+0000", 2024-09-18T11:31:18.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7349999999999998e-06 2024-09-18T11:31:18.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339909+0000", 2024-09-18T11:31:18.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-09-18T11:31:18.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339913+0000", 2024-09-18T11:31:18.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7670000000000001e-06 2024-09-18T11:31:18.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339970+0000", 2024-09-18T11:31:18.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6988999999999999e-05 2024-09-18T11:31:18.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:18.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339975+0000", 2024-09-18T11:31:18.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7929999999999997e-06 2024-09-18T11:31:18.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346917+0000", 2024-09-18T11:31:18.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069426640000000003 2024-09-18T11:31:18.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.347026+0000", 2024-09-18T11:31:18.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108241 2024-09-18T11:31:18.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.349014+0000", 2024-09-18T11:31:18.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001988723 2024-09-18T11:31:18.408 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.408 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.408 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.408 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1956 4.6 4:67c1765a:::smithi11939506-30:head [write 705927~558502] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:18.408 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:18.408 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.981816143, 2024-09-18T11:31:18.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011813239, 2024-09-18T11:31:18.408 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.408 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.408 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:18.409 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:18.409 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:18.409 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1956 2024-09-18T11:31:18.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.409 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:18.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:18.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352007+0000", 2024-09-18T11:31:18.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:18.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352229+0000", 2024-09-18T11:31:18.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222024 2024-09-18T11:31:18.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352251+0000", 2024-09-18T11:31:18.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2512000000000001e-05 2024-09-18T11:31:18.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352260+0000", 2024-09-18T11:31:18.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7339999999999997e-06 2024-09-18T11:31:18.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352307+0000", 2024-09-18T11:31:18.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7061999999999998e-05 2024-09-18T11:31:18.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352408+0000", 2024-09-18T11:31:18.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101216 2024-09-18T11:31:18.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:18.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352550+0000", 2024-09-18T11:31:18.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014136199999999999 2024-09-18T11:31:18.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:18.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357684+0000", 2024-09-18T11:31:18.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051342499999999999 2024-09-18T11:31:18.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:18.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363736+0000", 2024-09-18T11:31:18.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006052577 2024-09-18T11:31:18.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363778+0000", 2024-09-18T11:31:18.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1989000000000001e-05 2024-09-18T11:31:18.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363822+0000", 2024-09-18T11:31:18.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3894999999999999e-05 2024-09-18T11:31:18.413 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.414 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.414 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.414 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2278 4.f 4:fc192b81:::smithi11939506-31:head [stat,delete] snapc 26=[26,25,24,1e,1c,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T11:31:18.414 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:18.414 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.950778930999999, 2024-09-18T11:31:18.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01018694, 2024-09-18T11:31:18.414 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.414 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:18.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:18.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:18.414 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2278 2024-09-18T11:31:18.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.415 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:18.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:18.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383044+0000", 2024-09-18T11:31:18.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:18.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383054+0000", 2024-09-18T11:31:18.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5969999999999998e-06 2024-09-18T11:31:18.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383071+0000", 2024-09-18T11:31:18.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7353000000000001e-05 2024-09-18T11:31:18.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383081+0000", 2024-09-18T11:31:18.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9170000000000001e-06 2024-09-18T11:31:18.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383150+0000", 2024-09-18T11:31:18.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8503999999999993e-05 2024-09-18T11:31:18.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:18.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383158+0000", 2024-09-18T11:31:18.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5089999999999993e-06 2024-09-18T11:31:18.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385396+0000", 2024-09-18T11:31:18.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022381850000000002 2024-09-18T11:31:18.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385494+0000", 2024-09-18T11:31:18.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7751000000000007e-05 2024-09-18T11:31:18.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:18.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385697+0000", 2024-09-18T11:31:18.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020247400000000001 2024-09-18T11:31:18.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:18.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.390505+0000", 2024-09-18T11:31:18.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048084039999999996 2024-09-18T11:31:18.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:18.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393148+0000", 2024-09-18T11:31:18.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026432399999999998 2024-09-18T11:31:18.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393194+0000", 2024-09-18T11:31:18.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5259999999999997e-05 2024-09-18T11:31:18.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393233+0000", 2024-09-18T11:31:18.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9659000000000001e-05 2024-09-18T11:31:18.420 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.420 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.420 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.420 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2342 4.f 4:f3c51b3f:::smithi11939506-21:head [stat,delete] snapc 26=[26,25,24,1e,1c,19,18] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:18.420 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:18.420 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 16.947022918999998, 2024-09-18T11:31:18.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011167565000000001, 2024-09-18T11:31:18.421 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.421 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.421 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:18.421 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:18.421 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:18.421 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2342 2024-09-18T11:31:18.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.421 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:18.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:18.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:31:18.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:18.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:31:18.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1439999999999997e-06 2024-09-18T11:31:18.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386820+0000", 2024-09-18T11:31:18.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2651e-05 2024-09-18T11:31:18.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386825+0000", 2024-09-18T11:31:18.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3150000000000002e-06 2024-09-18T11:31:18.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386868+0000", 2024-09-18T11:31:18.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3832999999999999e-05 2024-09-18T11:31:18.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:18.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:31:18.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0409999999999998e-06 2024-09-18T11:31:18.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390445+0000", 2024-09-18T11:31:18.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035689459999999999 2024-09-18T11:31:18.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390539+0000", 2024-09-18T11:31:18.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3791999999999999e-05 2024-09-18T11:31:18.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:18.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390748+0000", 2024-09-18T11:31:18.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000208863 2024-09-18T11:31:18.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:18.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397659+0000", 2024-09-18T11:31:18.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069110710000000004 2024-09-18T11:31:18.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:18.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397873+0000", 2024-09-18T11:31:18.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021343999999999999 2024-09-18T11:31:18.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397923+0000", 2024-09-18T11:31:18.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0300000000000003e-05 2024-09-18T11:31:18.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397970+0000", 2024-09-18T11:31:18.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6968999999999997e-05 2024-09-18T11:31:18.427 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.427 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.427 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.427 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2429 4.b 4:d8d14c58:::smithi11939506-49:head [write 796698~561170,stat] snapc 2b=[2b,2a,26,25,24,1e,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T11:31:18.427 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:18.427 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.318653745000001, 2024-09-18T11:31:18.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097188989999999996, 2024-09-18T11:31:18.427 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.427 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.428 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:18.428 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:18.428 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:18.428 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2429 2024-09-18T11:31:18.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.428 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:18.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:18.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015170+0000", 2024-09-18T11:31:18.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:18.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015435+0000", 2024-09-18T11:31:18.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026533299999999998 2024-09-18T11:31:18.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015455+0000", 2024-09-18T11:31:18.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9623000000000002e-05 2024-09-18T11:31:18.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015461+0000", 2024-09-18T11:31:18.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1859999999999997e-06 2024-09-18T11:31:18.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015535+0000", 2024-09-18T11:31:18.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4555999999999996e-05 2024-09-18T11:31:18.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:18.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015543+0000", 2024-09-18T11:31:18.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9549999999999992e-06 2024-09-18T11:31:18.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017106+0000", 2024-09-18T11:31:18.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015630290000000001 2024-09-18T11:31:18.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017183+0000", 2024-09-18T11:31:18.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6953999999999995e-05 2024-09-18T11:31:18.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:18.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017301+0000", 2024-09-18T11:31:18.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011734 2024-09-18T11:31:18.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:18.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.020225+0000", 2024-09-18T11:31:18.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029244090000000002 2024-09-18T11:31:18.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:18.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024801+0000", 2024-09-18T11:31:18.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045764129999999997 2024-09-18T11:31:18.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024840+0000", 2024-09-18T11:31:18.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8325999999999999e-05 2024-09-18T11:31:18.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024890+0000", 2024-09-18T11:31:18.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0596999999999997e-05 2024-09-18T11:31:18.433 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.433 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.433 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.434 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:207 4.a e91/22 4:5975a5ec:::smithi11939506-36:18 v 91'132, mlcod=90'126)", 2024-09-18T11:31:18.434 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.010357+0000", 2024-09-18T11:31:18.434 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.3234682800000002, 2024-09-18T11:31:18.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079598080000000005, 2024-09-18T11:31:18.434 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.434 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.434 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.010357+0000", 2024-09-18T11:31:18.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.010357+0000", 2024-09-18T11:31:18.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.010356+0000", 2024-09-18T11:31:18.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.010362+0000", 2024-09-18T11:31:18.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1720000000000004e-06 2024-09-18T11:31:18.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.010375+0000", 2024-09-18T11:31:18.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3053e-05 2024-09-18T11:31:18.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.010381+0000", 2024-09-18T11:31:18.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8019999999999997e-06 2024-09-18T11:31:18.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.010522+0000", 2024-09-18T11:31:18.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000140488 2024-09-18T11:31:18.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.010542+0000", 2024-09-18T11:31:18.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0007e-05 2024-09-18T11:31:18.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.018285+0000", 2024-09-18T11:31:18.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077432289999999999 2024-09-18T11:31:18.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.018317+0000", 2024-09-18T11:31:18.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1705000000000002e-05 2024-09-18T11:31:18.438 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.438 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.438 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.438 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.438 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.438 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:18.439 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:31:18.440 INFO:tasks.rados.rados.0.smithi119.stdout:1296: finishing rollback tid 0 to smithi11939506-3 2024-09-18T11:31:18.440 INFO:tasks.rados.rados.0.smithi119.stdout:1296: finishing rollback tid 1 to smithi11939506-3 2024-09-18T11:31:18.440 INFO:tasks.rados.rados.0.smithi119.stdout:1296: finishing rollback tid 2 to smithi11939506-3 2024-09-18T11:31:18.440 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 149 (ObjNum 184 snap 20 seq_num 184) dirty exists 2024-09-18T11:31:18.440 INFO:tasks.rados.rados.0.smithi119.stdout:1291: finishing set_chunk to oid 9 2024-09-18T11:31:18.440 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 184 (ObjNum 199 snap 23 seq_num 199) dirty exists 2024-09-18T11:31:18.440 INFO:tasks.rados.rados.0.smithi119.stdout:1294: expect (ObjNum 72 snap 3 seq_num 72) 2024-09-18T11:31:18.441 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:18.437+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:18.496 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T11:31:18.540 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:18.539+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:18.543 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:18.543 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:18.543 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:18.543 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:18.550 INFO:tasks.rados.rados.0.smithi119.stdout:1292: expect (ObjNum 99 snap 6 seq_num 99) 2024-09-18T11:31:18.552 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:31:18.640 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:18.639+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:18.693 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:18.693 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:18.701 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:31:18.722 INFO:tasks.rados.rados.0.smithi119.stdout:1295: got (22) Invalid argument 2024-09-18T11:31:18.722 INFO:tasks.rados.rados.0.smithi119.stdout:1290: done (6 left) 2024-09-18T11:31:18.722 INFO:tasks.rados.rados.0.smithi119.stdout:1291: done (5 left) 2024-09-18T11:31:18.722 INFO:tasks.rados.rados.0.smithi119.stdout:1292: done (4 left) 2024-09-18T11:31:18.722 INFO:tasks.rados.rados.0.smithi119.stdout:1293: done (3 left) 2024-09-18T11:31:18.722 INFO:tasks.rados.rados.0.smithi119.stdout:1294: done (2 left) 2024-09-18T11:31:18.722 INFO:tasks.rados.rados.0.smithi119.stdout:1295: done (1 left) 2024-09-18T11:31:18.722 INFO:tasks.rados.rados.0.smithi119.stdout:1296: done (0 left) 2024-09-18T11:31:18.723 INFO:tasks.rados.rados.0.smithi119.stdout:1297: set_chunk oid 34 target oid 2024-09-18T11:31:18.723 INFO:tasks.rados.rados.0.smithi119.stdout:1297: set_chunk oid 34 offset: 0 length: 4000000 target oid smithi11939506- offset: 0 snap -1 2024-09-18T11:31:18.723 INFO:tasks.rados.rados.0.smithi119.stdout:1298: write (excl) oid 1 current snap is 28 2024-09-18T11:31:18.723 INFO:tasks.rados.rados.0.smithi119.stdout:1298: seq_num 242 ranges {467294=617201,1578407=488887} 2024-09-18T11:31:18.729 INFO:tasks.rados.rados.0.smithi119.stdout:1298: writing smithi11939506-1 from 467294 to 1084495 tid 1 2024-09-18T11:31:18.736 INFO:tasks.rados.rados.0.smithi119.stdout:1298: writing smithi11939506-1 from 1578407 to 2067294 tid 2 2024-09-18T11:31:18.736 INFO:tasks.rados.rados.0.smithi119.stdout:1297: finishing set_chunk to oid 34 2024-09-18T11:31:18.736 INFO:tasks.rados.rados.0.smithi119.stdout:1297: got expected ERANGE (src dne) 2024-09-18T11:31:18.736 INFO:tasks.rados.rados.0.smithi119.stdout:1298: finishing write tid 1 to smithi11939506-1 2024-09-18T11:31:18.736 INFO:tasks.rados.rados.0.smithi119.stdout:1299: read oid 13 snap 21 2024-09-18T11:31:18.736 INFO:tasks.rados.rados.0.smithi119.stdout:1299: expect (ObjNum 136 snap 16 seq_num 136) 2024-09-18T11:31:18.736 INFO:tasks.rados.rados.0.smithi119.stdout:1297: done (2 left) 2024-09-18T11:31:18.736 INFO:tasks.rados.rados.0.smithi119.stdout:1300: read oid 17 snap -1 2024-09-18T11:31:18.736 INFO:tasks.rados.rados.0.smithi119.stdout:1300: expect deleted 2024-09-18T11:31:18.737 INFO:tasks.rados.rados.0.smithi119.stdout:1301: tier_promote oid 42 2024-09-18T11:31:18.737 INFO:tasks.rados.rados.0.smithi119.stdout:1302: write oid 3 current snap is 28 2024-09-18T11:31:18.737 INFO:tasks.rados.rados.0.smithi119.stdout:1302: seq_num 243 ranges {484101=692450,1636889=447212} 2024-09-18T11:31:18.742 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:18.741+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:18.743 INFO:tasks.rados.rados.0.smithi119.stdout:1302: writing smithi11939506-3 from 484101 to 1176551 tid 1 2024-09-18T11:31:18.750 INFO:tasks.rados.rados.0.smithi119.stdout:1302: writing smithi11939506-3 from 1636889 to 2084101 tid 2 2024-09-18T11:31:18.750 INFO:tasks.rados.rados.0.smithi119.stdout:1303: tier_promote oid 18 2024-09-18T11:31:18.750 INFO:tasks.rados.rados.0.smithi119.stdout:1301: got (0) Success 2024-09-18T11:31:18.750 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 105 (ObjNum 99 snap 6 seq_num 99) dirty exists 2024-09-18T11:31:18.750 INFO:tasks.rados.rados.0.smithi119.stdout:1299: expect (ObjNum 136 snap 16 seq_num 136) 2024-09-18T11:31:18.808 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:18.808 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:18.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:18.808 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:18.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.808 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1046 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'24, mlcod=30'21)", 2024-09-18T11:31:18.808 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:31:18.808 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 62.632137600999997, 2024-09-18T11:31:18.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042519100000000002, 2024-09-18T11:31:18.809 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.809 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.809 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:31:18.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:31:18.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173782+0000", 2024-09-18T11:31:18.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:18.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174207+0000", 2024-09-18T11:31:18.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00042479799999999999 2024-09-18T11:31:18.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174224+0000", 2024-09-18T11:31:18.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7513999999999999e-05 2024-09-18T11:31:18.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174232+0000", 2024-09-18T11:31:18.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0840000000000003e-06 2024-09-18T11:31:18.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174297+0000", 2024-09-18T11:31:18.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5215000000000005e-05 2024-09-18T11:31:18.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174336+0000", 2024-09-18T11:31:18.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9403000000000002e-05 2024-09-18T11:31:18.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.177990+0000", 2024-09-18T11:31:18.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036533709999999999 2024-09-18T11:31:18.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.178035+0000", 2024-09-18T11:31:18.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5495999999999999e-05 2024-09-18T11:31:18.812 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.813 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.813 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.813 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:31:18.813 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:31:18.813 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.822546733999999, 2024-09-18T11:31:18.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:31:18.813 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.813 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.813 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:31:18.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:31:18.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:31:18.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:18.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:31:18.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:31:18.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:31:18.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:31:18.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:31:18.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:31:18.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:31:18.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:31:18.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:31:18.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:31:18.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:31:18.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:31:18.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:31:18.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:31:18.817 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.817 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.817 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.817 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:31:18.817 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:18.817 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 49.587715781, 2024-09-18T11:31:18.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:31:18.818 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.818 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.818 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:18.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:18.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:18.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:31:18.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:31:18.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:31:18.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:31:18.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:31:18.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:31:18.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:31:18.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:31:18.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:31:18.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:31:18.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:31:18.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:31:18.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:31:18.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:31:18.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:31:18.821 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.821 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.822 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.822 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:31:18.822 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:31:18.822 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 44.413809495999999, 2024-09-18T11:31:18.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:31:18.822 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.822 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.822 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:31:18.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:31:18.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:31:18.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:18.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:31:18.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:31:18.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:31:18.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:31:18.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:31:18.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:31:18.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:31:18.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:31:18.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:31:18.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:31:18.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:31:18.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:31:18.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:31:18.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:31:18.826 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.826 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.826 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.826 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1931 3.0 e65/21 3:3c05e498:::866d65857c51f67d56ecbcba33930954810c64ced2a213ca9705911438b0dfb7:head v 65'72, mlcod=58'70)", 2024-09-18T11:31:18.827 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:31:18.827 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.437799477999999, 2024-09-18T11:31:18.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040630859999999996, 2024-09-18T11:31:18.827 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.827 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.827 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:31:18.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:31:18.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368120+0000", 2024-09-18T11:31:18.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:18.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368427+0000", 2024-09-18T11:31:18.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030742600000000002 2024-09-18T11:31:18.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368446+0000", 2024-09-18T11:31:18.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8046e-05 2024-09-18T11:31:18.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368453+0000", 2024-09-18T11:31:18.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1380000000000001e-06 2024-09-18T11:31:18.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368519+0000", 2024-09-18T11:31:18.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6470000000000006e-05 2024-09-18T11:31:18.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368550+0000", 2024-09-18T11:31:18.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0707999999999998e-05 2024-09-18T11:31:18.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.372151+0000", 2024-09-18T11:31:18.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036015470000000001 2024-09-18T11:31:18.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.372184+0000", 2024-09-18T11:31:18.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2994999999999998e-05 2024-09-18T11:31:18.831 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.831 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.831 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.831 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:304 4.6 e69/67 4:67c1765a:::smithi11939506-30:17 v 69'35, mlcod=58'31)", 2024-09-18T11:31:18.831 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:18.832 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.459968402000001, 2024-09-18T11:31:18.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006377611, 2024-09-18T11:31:18.832 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.832 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.832 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:18.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:18.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:18.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345961+0000", 2024-09-18T11:31:18.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2050000000000008e-06 2024-09-18T11:31:18.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345969+0000", 2024-09-18T11:31:18.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0339999999999997e-06 2024-09-18T11:31:18.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345976+0000", 2024-09-18T11:31:18.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-18T11:31:18.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346026+0000", 2024-09-18T11:31:18.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9286999999999998e-05 2024-09-18T11:31:18.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346050+0000", 2024-09-18T11:31:18.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4397e-05 2024-09-18T11:31:18.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352293+0000", 2024-09-18T11:31:18.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062427170000000001 2024-09-18T11:31:18.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352330+0000", 2024-09-18T11:31:18.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7348999999999998e-05 2024-09-18T11:31:18.835 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.836 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.836 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.836 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:347 4.d e69/22 4:b44229a1:::smithi11939506-42:19 v 69'99, mlcod=65'93)", 2024-09-18T11:31:18.836 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:31:18.836 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.454235207, 2024-09-18T11:31:18.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056998760000000004, 2024-09-18T11:31:18.836 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.836 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.836 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:31:18.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:31:18.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351684+0000", 2024-09-18T11:31:18.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:18.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351693+0000", 2024-09-18T11:31:18.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-18T11:31:18.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351704+0000", 2024-09-18T11:31:18.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1077000000000001e-05 2024-09-18T11:31:18.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351709+0000", 2024-09-18T11:31:18.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5419999999999999e-06 2024-09-18T11:31:18.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351956+0000", 2024-09-18T11:31:18.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000247252 2024-09-18T11:31:18.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351975+0000", 2024-09-18T11:31:18.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9201000000000001e-05 2024-09-18T11:31:18.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357334+0000", 2024-09-18T11:31:18.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053588749999999999 2024-09-18T11:31:18.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357385+0000", 2024-09-18T11:31:18.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1406999999999998e-05 2024-09-18T11:31:18.840 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.840 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.840 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.840 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1956 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'37, mlcod=58'31)", 2024-09-18T11:31:18.841 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:18.841 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.453290293999999, 2024-09-18T11:31:18.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010958842, 2024-09-18T11:31:18.841 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.841 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.841 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:18.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:18.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:18.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:18.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352996+0000", 2024-09-18T11:31:18.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036622 2024-09-18T11:31:18.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353011+0000", 2024-09-18T11:31:18.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4849e-05 2024-09-18T11:31:18.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353017+0000", 2024-09-18T11:31:18.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:31:18.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353068+0000", 2024-09-18T11:31:18.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0695000000000002e-05 2024-09-18T11:31:18.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353092+0000", 2024-09-18T11:31:18.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4260000000000002e-05 2024-09-18T11:31:18.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363551+0000", 2024-09-18T11:31:18.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010459154999999999 2024-09-18T11:31:18.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363589+0000", 2024-09-18T11:31:18.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8161000000000003e-05 2024-09-18T11:31:18.845 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.845 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.845 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.845 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1957 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'38, mlcod=65'33)", 2024-09-18T11:31:18.845 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:18.845 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.446874427000001, 2024-09-18T11:31:18.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050499400000000002, 2024-09-18T11:31:18.846 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.846 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.846 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:18.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:18.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:18.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359214+0000", 2024-09-18T11:31:18.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000168047 2024-09-18T11:31:18.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359227+0000", 2024-09-18T11:31:18.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3124e-05 2024-09-18T11:31:18.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359235+0000", 2024-09-18T11:31:18.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0760000000000003e-06 2024-09-18T11:31:18.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359287+0000", 2024-09-18T11:31:18.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.151e-05 2024-09-18T11:31:18.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359310+0000", 2024-09-18T11:31:18.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-09-18T11:31:18.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364065+0000", 2024-09-18T11:31:18.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047549439999999997 2024-09-18T11:31:18.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364096+0000", 2024-09-18T11:31:18.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1423000000000001e-05 2024-09-18T11:31:18.850 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.850 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.850 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.850 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:349 4.d e69/22 4:b9bb642b:::smithi11939506-27:19 v 69'101, mlcod=69'97)", 2024-09-18T11:31:18.850 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.360028+0000", 2024-09-18T11:31:18.850 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.44589281, 2024-09-18T11:31:18.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040307850000000003, 2024-09-18T11:31:18.850 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.850 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.850 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360028+0000", 2024-09-18T11:31:18.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360028+0000", 2024-09-18T11:31:18.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360027+0000", 2024-09-18T11:31:18.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:18.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360039+0000", 2024-09-18T11:31:18.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1654000000000001e-05 2024-09-18T11:31:18.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360052+0000", 2024-09-18T11:31:18.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.33e-05 2024-09-18T11:31:18.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360058+0000", 2024-09-18T11:31:18.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0440000000000001e-06 2024-09-18T11:31:18.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360094+0000", 2024-09-18T11:31:18.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6402999999999997e-05 2024-09-18T11:31:18.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.360109+0000", 2024-09-18T11:31:18.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4105e-05 2024-09-18T11:31:18.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364025+0000", 2024-09-18T11:31:18.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039163189999999997 2024-09-18T11:31:18.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364059+0000", 2024-09-18T11:31:18.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3810999999999999e-05 2024-09-18T11:31:18.854 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.854 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.854 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.854 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:407 4.4 e77/22 4:2d9c3f7f:::smithi11939506-5:1d v 77'92, mlcod=73'87)", 2024-09-18T11:31:18.855 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.387293+0000", 2024-09-18T11:31:18.855 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.418627603000001, 2024-09-18T11:31:18.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038531820000000001, 2024-09-18T11:31:18.855 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.855 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.855 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387293+0000", 2024-09-18T11:31:18.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387293+0000", 2024-09-18T11:31:18.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387293+0000", 2024-09-18T11:31:18.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387300+0000", 2024-09-18T11:31:18.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6799999999999993e-06 2024-09-18T11:31:18.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387311+0000", 2024-09-18T11:31:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0278999999999999e-05 2024-09-18T11:31:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387316+0000", 2024-09-18T11:31:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9269999999999996e-06 2024-09-18T11:31:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387357+0000", 2024-09-18T11:31:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1208000000000003e-05 2024-09-18T11:31:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.387374+0000", 2024-09-18T11:31:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7499e-05 2024-09-18T11:31:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.391105+0000", 2024-09-18T11:31:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037302619999999998 2024-09-18T11:31:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.391146+0000", 2024-09-18T11:31:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1797e-05 2024-09-18T11:31:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2300 4.4 e77/22 4:2c93196f:::smithi11939506-28:head v 77'94, mlcod=77'90)", 2024-09-18T11:31:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:31:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.817812209, 2024-09-18T11:31:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050425380000000001, 2024-09-18T11:31:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:31:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:31:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988108+0000", 2024-09-18T11:31:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988388+0000", 2024-09-18T11:31:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028087700000000001 2024-09-18T11:31:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988405+0000", 2024-09-18T11:31:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6565000000000001e-05 2024-09-18T11:31:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988412+0000", 2024-09-18T11:31:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9709999999999998e-06 2024-09-18T11:31:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988464+0000", 2024-09-18T11:31:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1949e-05 2024-09-18T11:31:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988490+0000", 2024-09-18T11:31:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5780999999999999e-05 2024-09-18T11:31:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993112+0000", 2024-09-18T11:31:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046228399999999996 2024-09-18T11:31:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993151+0000", 2024-09-18T11:31:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8587999999999997e-05 2024-09-18T11:31:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.13:63 3.0 e78/21 3:24ebb740:::smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 78'80, mlcod=73'78)", 2024-09-18T11:31:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:31:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.400822793, 2024-09-18T11:31:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052262230000000003, 2024-09-18T11:31:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:31:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:31:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405097+0000", 2024-09-18T11:31:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405103+0000", 2024-09-18T11:31:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8780000000000003e-06 2024-09-18T11:31:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405117+0000", 2024-09-18T11:31:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3441000000000001e-05 2024-09-18T11:31:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405121+0000", 2024-09-18T11:31:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7099999999999998e-06 2024-09-18T11:31:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405158+0000", 2024-09-18T11:31:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6477999999999999e-05 2024-09-18T11:31:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405173+0000", 2024-09-18T11:31:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4796e-05 2024-09-18T11:31:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410285+0000", 2024-09-18T11:31:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051118919999999998 2024-09-18T11:31:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410324+0000", 2024-09-18T11:31:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9616999999999997e-05 2024-09-18T11:31:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:451 4.6 e78/67 4:6f86ffcc:::smithi11939506-15:26 v 78'58, mlcod=77'53)", 2024-09-18T11:31:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.414259+0000", 2024-09-18T11:31:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.39166221, 2024-09-18T11:31:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041821610000000002, 2024-09-18T11:31:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414259+0000", 2024-09-18T11:31:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414259+0000", 2024-09-18T11:31:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414258+0000", 2024-09-18T11:31:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414264+0000", 2024-09-18T11:31:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1850000000000001e-06 2024-09-18T11:31:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414273+0000", 2024-09-18T11:31:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7290000000000006e-06 2024-09-18T11:31:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414278+0000", 2024-09-18T11:31:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8470000000000003e-06 2024-09-18T11:31:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414315+0000", 2024-09-18T11:31:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7462000000000001e-05 2024-09-18T11:31:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414334+0000", 2024-09-18T11:31:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9080999999999999e-05 2024-09-18T11:31:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.418419+0000", 2024-09-18T11:31:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040846540000000001 2024-09-18T11:31:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.418441+0000", 2024-09-18T11:31:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1844999999999998e-05 2024-09-18T11:31:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.13:18 3.0 e91/21 3:35e3c617:::2e64dd273da7619bd64b294f52905c1b9405a1ced2a8fd3c0b5bc481bc9732a6:head v 91'89, mlcod=85'87)", 2024-09-18T11:31:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:31:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.7840473819999998, 2024-09-18T11:31:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048558539999999997, 2024-09-18T11:31:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:31:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:31:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:31:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021880+0000", 2024-09-18T11:31:18.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5860000000000001e-06 2024-09-18T11:31:18.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021890+0000", 2024-09-18T11:31:18.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8290000000000003e-06 2024-09-18T11:31:18.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021895+0000", 2024-09-18T11:31:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2480000000000004e-06 2024-09-18T11:31:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022166+0000", 2024-09-18T11:31:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027073499999999998 2024-09-18T11:31:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022184+0000", 2024-09-18T11:31:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8082000000000001e-05 2024-09-18T11:31:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026693+0000", 2024-09-18T11:31:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045093479999999998 2024-09-18T11:31:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026729+0000", 2024-09-18T11:31:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6282999999999998e-05 2024-09-18T11:31:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:573 4.4 e91/22 4:2c93196f:::smithi11939506-28:26 v 91'135, mlcod=90'131)", 2024-09-18T11:31:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:31:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.7815807929999998, 2024-09-18T11:31:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050722739999999999, 2024-09-18T11:31:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:31:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:31:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024339+0000", 2024-09-18T11:31:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024349+0000", 2024-09-18T11:31:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6339999999999996e-06 2024-09-18T11:31:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024362+0000", 2024-09-18T11:31:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-18T11:31:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024368+0000", 2024-09-18T11:31:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8649999999999998e-06 2024-09-18T11:31:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024431+0000", 2024-09-18T11:31:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2780999999999994e-05 2024-09-18T11:31:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024448+0000", 2024-09-18T11:31:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7388999999999999e-05 2024-09-18T11:31:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029385+0000", 2024-09-18T11:31:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004936523 2024-09-18T11:31:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029412+0000", 2024-09-18T11:31:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7702999999999999e-05 2024-09-18T11:31:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:574 4.4 e91/22 4:2cb3e6a5:::smithi11939506-3:1e v 91'137, mlcod=90'131)", 2024-09-18T11:31:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:31:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.7811427499999999, 2024-09-18T11:31:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070802850000000004, 2024-09-18T11:31:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:31:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:31:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024777+0000", 2024-09-18T11:31:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024785+0000", 2024-09-18T11:31:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8099999999999998e-06 2024-09-18T11:31:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024794+0000", 2024-09-18T11:31:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5839999999999995e-06 2024-09-18T11:31:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024798+0000", 2024-09-18T11:31:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.031e-06 2024-09-18T11:31:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024830+0000", 2024-09-18T11:31:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2268999999999999e-05 2024-09-18T11:31:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024847+0000", 2024-09-18T11:31:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7306999999999999e-05 2024-09-18T11:31:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031834+0000", 2024-09-18T11:31:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069868380000000004 2024-09-18T11:31:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031858+0000", 2024-09-18T11:31:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4082000000000001e-05 2024-09-18T11:31:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:541 4.d e91/22 4:b9bb642b:::smithi11939506-27:19 v 91'158, mlcod=91'154)", 2024-09-18T11:31:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:31:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.773762058, 2024-09-18T11:31:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052185, 2024-09-18T11:31:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:31:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:31:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032158+0000", 2024-09-18T11:31:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032165+0000", 2024-09-18T11:31:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-09-18T11:31:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032176+0000", 2024-09-18T11:31:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1460999999999999e-05 2024-09-18T11:31:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032182+0000", 2024-09-18T11:31:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0870000000000003e-06 2024-09-18T11:31:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032224+0000", 2024-09-18T11:31:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1372999999999998e-05 2024-09-18T11:31:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032246+0000", 2024-09-18T11:31:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2158e-05 2024-09-18T11:31:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037348+0000", 2024-09-18T11:31:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051025319999999999 2024-09-18T11:31:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037377+0000", 2024-09-18T11:31:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8937000000000001e-05 2024-09-18T11:31:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:582 4.6 e91/67 4:6f86ffcc:::smithi11939506-15:26 v 91'78, mlcod=90'72)", 2024-09-18T11:31:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.043567+0000", 2024-09-18T11:31:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.7623541080000003, 2024-09-18T11:31:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004448446, 2024-09-18T11:31:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043567+0000", 2024-09-18T11:31:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043567+0000", 2024-09-18T11:31:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043566+0000", 2024-09-18T11:31:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043571+0000", 2024-09-18T11:31:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.181e-06 2024-09-18T11:31:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043579+0000", 2024-09-18T11:31:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9559999999999997e-06 2024-09-18T11:31:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043583+0000", 2024-09-18T11:31:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0770000000000001e-06 2024-09-18T11:31:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043610+0000", 2024-09-18T11:31:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6778e-05 2024-09-18T11:31:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043633+0000", 2024-09-18T11:31:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2904e-05 2024-09-18T11:31:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.047991+0000", 2024-09-18T11:31:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043582910000000002 2024-09-18T11:31:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.048015+0000", 2024-09-18T11:31:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3813000000000001e-05 2024-09-18T11:31:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2754 4.4 e92/22 4:2cb3e6a5:::smithi11939506-3:head v 92'147, mlcod=92'143)", 2024-09-18T11:31:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:17.887302+0000", 2024-09-18T11:31:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.91861868000000002, 2024-09-18T11:31:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044319160000000002, 2024-09-18T11:31:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887302+0000", 2024-09-18T11:31:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887302+0000", 2024-09-18T11:31:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887301+0000", 2024-09-18T11:31:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887313+0000", 2024-09-18T11:31:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2468e-05 2024-09-18T11:31:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887331+0000", 2024-09-18T11:31:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7362000000000001e-05 2024-09-18T11:31:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887338+0000", 2024-09-18T11:31:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7980000000000006e-06 2024-09-18T11:31:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887416+0000", 2024-09-18T11:31:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7577999999999995e-05 2024-09-18T11:31:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887441+0000", 2024-09-18T11:31:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.531e-05 2024-09-18T11:31:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.891671+0000", 2024-09-18T11:31:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00422989 2024-09-18T11:31:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.891734+0000", 2024-09-18T11:31:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2885999999999994e-05 2024-09-18T11:31:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:18.900 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:18.901 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:31:18.903 INFO:tasks.rados.rados.0.smithi119.stdout:1298: finishing write tid 2 to smithi11939506-1 2024-09-18T11:31:18.903 INFO:tasks.rados.rados.0.smithi119.stdout:1298: finishing write tid 3 to smithi11939506-1 2024-09-18T11:31:18.903 INFO:tasks.rados.rados.0.smithi119.stdout:1298: finishing write tid 4 to smithi11939506-1 2024-09-18T11:31:18.903 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 140 (ObjNum 242 snap 28 seq_num 242) dirty exists 2024-09-18T11:31:18.903 INFO:tasks.rados.rados.0.smithi119.stdout:1298: left oid 1 (ObjNum 242 snap 28 seq_num 242) 2024-09-18T11:31:18.903 INFO:tasks.rados.rados.0.smithi119.stdout:1298: done (5 left) 2024-09-18T11:31:18.903 INFO:tasks.rados.rados.0.smithi119.stdout:1299: done (4 left) 2024-09-18T11:31:18.903 INFO:tasks.rados.rados.0.smithi119.stdout:1300: done (3 left) 2024-09-18T11:31:18.903 INFO:tasks.rados.rados.0.smithi119.stdout:1301: done (2 left) 2024-09-18T11:31:18.903 INFO:tasks.rados.rados.0.smithi119.stdout:1304: read oid 45 snap 27 2024-09-18T11:31:18.903 INFO:tasks.rados.rados.0.smithi119.stdout:1304: expect (ObjNum 232 snap 27 seq_num 232) 2024-09-18T11:31:18.903 INFO:tasks.rados.rados.0.smithi119.stdout:1305: set_chunk oid 2 target oid 2024-09-18T11:31:18.904 INFO:tasks.rados.rados.0.smithi119.stdout:1305: set_chunk oid 2 offset: 124416 length: 1735680 target oid b8456d9a736fc972059ad1f74bf78736a34fe4288aba823b2ba5bd7abb89128c offset: 0 snap -1 2024-09-18T11:31:18.904 INFO:tasks.rados.rados.0.smithi119.stdout:1306: write oid 27 current snap is 28 2024-09-18T11:31:18.904 INFO:tasks.rados.rados.0.smithi119.stdout:1303: got (0) Success 2024-09-18T11:31:18.904 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 113 (ObjNum 205 snap 23 seq_num 205) dirty exists 2024-09-18T11:31:18.904 INFO:tasks.rados.rados.0.smithi119.stdout:1302: finishing write tid 1 to smithi11939506-3 2024-09-18T11:31:18.904 INFO:tasks.rados.rados.0.smithi119.stdout:1302: finishing write tid 2 to smithi11939506-3 2024-09-18T11:31:18.904 INFO:tasks.rados.rados.0.smithi119.stdout:1306: seq_num 244 ranges {500908=484052,1411724=689184} 2024-09-18T11:31:18.904 INFO:tasks.rados.rados.0.smithi119.stdout:1306: writing smithi11939506-27 from 500908 to 984960 tid 1 2024-09-18T11:31:18.904 INFO:tasks.rados.rados.0.smithi119.stdout:1306: writing smithi11939506-27 from 1411724 to 2100908 tid 2 2024-09-18T11:31:18.904 INFO:tasks.rados.rados.0.smithi119.stdout:1303: done (4 left) 2024-09-18T11:31:18.904 INFO:tasks.rados.rados.0.smithi119.stdout:1307: set_chunk oid 17 target oid 2024-09-18T11:31:18.904 INFO:tasks.rados.rados.0.smithi119.stdout:1302: finishing write tid 3 to smithi11939506-3 2024-09-18T11:31:18.904 INFO:tasks.rados.rados.0.smithi119.stdout:1302: finishing write tid 4 to smithi11939506-3 2024-09-18T11:31:18.905 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 152 (ObjNum 243 snap 28 seq_num 243) dirty exists 2024-09-18T11:31:18.905 INFO:tasks.rados.rados.0.smithi119.stdout:1302: left oid 3 (ObjNum 243 snap 28 seq_num 243) 2024-09-18T11:31:18.905 INFO:tasks.rados.rados.0.smithi119.stdout:1307: set_chunk oid 17 offset: 1119744 length: 429056 target oid 7f31751f088e4f7577561c03b406e8a8bc48ebfdd310b40206061655307aebe3 offset: 0 snap 21 2024-09-18T11:31:18.905 INFO:tasks.rados.rados.0.smithi119.stdout:1302: done (4 left) 2024-09-18T11:31:18.905 INFO:tasks.rados.rados.0.smithi119.stdout:1308: set_chunk oid 5 target oid 2024-09-18T11:31:18.905 INFO:tasks.rados.rados.0.smithi119.stdout:1308: set_chunk oid 5 offset: 0 length: 4000000 target oid smithi11939506- offset: 0 snap -1 2024-09-18T11:31:18.905 INFO:tasks.rados.rados.0.smithi119.stdout:1309: set_chunk oid 21 target oid 2024-09-18T11:31:18.905 INFO:tasks.rados.rados.0.smithi119.stdout:1309: set_chunk oid 21 offset: 1464320 length: 2442240 target oid smithi11939506- offset: 1464320 snap 23 2024-09-18T11:31:18.905 INFO:tasks.rados.rados.0.smithi119.stdout:1310: write oid 44 current snap is 28 2024-09-18T11:31:18.905 INFO:tasks.rados.rados.0.smithi119.stdout:1304: expect (ObjNum 232 snap 27 seq_num 232) 2024-09-18T11:31:18.905 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:18.899+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:19.002 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:19.000+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:19.039 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:19.039 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:19.039 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:19.040 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:19.048 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:31:19.103 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:19.101+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:19.147 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:19.147 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:19.147 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:19.147 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:19.147 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:19.156 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:31:19.204 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:19.202+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:19.205 INFO:tasks.rados.rados.0.smithi119.stdout:1305: finishing set_chunk to oid 2 2024-09-18T11:31:19.205 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 2 set_chunk returned error code -95 offset: 124416 length: 1735680 2024-09-18T11:31:19.206 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 0 (ObjNum 195 snap 23 seq_num 195) dirty exists 2024-09-18T11:31:19.206 INFO:tasks.rados.rados.0.smithi119.stdout:1306: finishing write tid 1 to smithi11939506-27 2024-09-18T11:31:19.206 INFO:tasks.rados.rados.0.smithi119.stdout:1310: seq_num 245 ranges {517715=559301,1870206=247509} 2024-09-18T11:31:19.211 INFO:tasks.rados.rados.0.smithi119.stdout:1310: writing smithi11939506-44 from 517715 to 1077016 tid 1 2024-09-18T11:31:19.218 INFO:tasks.rados.rados.0.smithi119.stdout:1310: writing smithi11939506-44 from 1870206 to 2117715 tid 2 2024-09-18T11:31:19.218 INFO:tasks.rados.rados.0.smithi119.stdout:1304: done (6 left) 2024-09-18T11:31:19.218 INFO:tasks.rados.rados.0.smithi119.stdout:1305: done (5 left) 2024-09-18T11:31:19.218 INFO:tasks.rados.rados.0.smithi119.stdout:1311: write oid 14 current snap is 28 2024-09-18T11:31:19.218 INFO:tasks.rados.rados.0.smithi119.stdout:1306: finishing write tid 2 to smithi11939506-27 2024-09-18T11:31:19.218 INFO:tasks.rados.rados.0.smithi119.stdout:1306: finishing write tid 3 to smithi11939506-27 2024-09-18T11:31:19.218 INFO:tasks.rados.rados.0.smithi119.stdout:1306: finishing write tid 4 to smithi11939506-27 2024-09-18T11:31:19.218 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 172 (ObjNum 244 snap 28 seq_num 244) dirty exists 2024-09-18T11:31:19.218 INFO:tasks.rados.rados.0.smithi119.stdout:1306: left oid 27 (ObjNum 244 snap 28 seq_num 244) 2024-09-18T11:31:19.218 INFO:tasks.rados.rados.0.smithi119.stdout:1308: finishing set_chunk to oid 5 2024-09-18T11:31:19.218 INFO:tasks.rados.rados.0.smithi119.stdout:1308: got expected ENOENT (src dne) 2024-09-18T11:31:19.218 INFO:tasks.rados.rados.0.smithi119.stdout:1309: finishing set_chunk to oid 21 2024-09-18T11:31:19.219 INFO:tasks.rados.rados.0.smithi119.stdout:1309: got expected ENOENT (src dne) 2024-09-18T11:31:19.219 INFO:tasks.rados.rados.0.smithi119.stdout:1307: finishing set_chunk to oid 17 2024-09-18T11:31:19.219 INFO:tasks.rados.rados.0.smithi119.stdout:1310: finishing write tid 1 to smithi11939506-44 2024-09-18T11:31:19.219 INFO:tasks.rados.rados.0.smithi119.stdout:1311: seq_num 246 ranges {534522=634550,1812335=322187} 2024-09-18T11:31:19.223 INFO:tasks.rados.rados.0.smithi119.stdout:1311: writing smithi11939506-14 from 534522 to 1169072 tid 1 2024-09-18T11:31:19.229 INFO:tasks.rados.rados.0.smithi119.stdout:1311: writing smithi11939506-14 from 1812335 to 2134522 tid 2 2024-09-18T11:31:19.229 INFO:tasks.rados.rados.0.smithi119.stdout:1306: done (5 left) 2024-09-18T11:31:19.229 INFO:tasks.rados.rados.0.smithi119.stdout:1307: done (4 left) 2024-09-18T11:31:19.229 INFO:tasks.rados.rados.0.smithi119.stdout:1308: done (3 left) 2024-09-18T11:31:19.229 INFO:tasks.rados.rados.0.smithi119.stdout:1309: done (2 left) 2024-09-18T11:31:19.229 INFO:tasks.rados.rados.0.smithi119.stdout:1312: set_chunk oid 24 target oid 2024-09-18T11:31:19.234 INFO:tasks.rados.rados.0.smithi119.stdout:1312: set_chunk oid 24 offset: 1268224 length: 3584 target oid 6cf1b57d59e7111bc218dfb01dda93ac0f776715599a1c69f89035bd20c16a10 offset: 0 snap -1 2024-09-18T11:31:19.234 INFO:tasks.rados.rados.0.smithi119.stdout:1313: write oid 17 current snap is 28 2024-09-18T11:31:19.234 INFO:tasks.rados.rados.0.smithi119.stdout:1310: finishing write tid 2 to smithi11939506-44 2024-09-18T11:31:19.234 INFO:tasks.rados.rados.0.smithi119.stdout:1310: finishing write tid 3 to smithi11939506-44 2024-09-18T11:31:19.235 INFO:tasks.rados.rados.0.smithi119.stdout:1313: seq_num 247 ranges {551329=709799,1870817=280512} 2024-09-18T11:31:19.241 INFO:tasks.rados.rados.0.smithi119.stdout:1313: writing smithi11939506-17 from 551329 to 1261128 tid 1 2024-09-18T11:31:19.246 INFO:tasks.rados.rados.0.smithi119.stdout:1313: writing smithi11939506-17 from 1870817 to 2151329 tid 2 2024-09-18T11:31:19.246 INFO:tasks.rados.rados.0.smithi119.stdout:1314: write (excl) oid 33 current snap is 28 2024-09-18T11:31:19.246 INFO:tasks.rados.rados.0.smithi119.stdout:1310: finishing write tid 4 to smithi11939506-44 2024-09-18T11:31:19.246 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 44 v 109 (ObjNum 245 snap 28 seq_num 245) dirty exists 2024-09-18T11:31:19.247 INFO:tasks.rados.rados.0.smithi119.stdout:1310: left oid 44 (ObjNum 245 snap 28 seq_num 245) 2024-09-18T11:31:19.247 INFO:tasks.rados.rados.0.smithi119.stdout:1311: finishing write tid 1 to smithi11939506-14 2024-09-18T11:31:19.247 INFO:tasks.rados.rados.0.smithi119.stdout:1311: finishing write tid 2 to smithi11939506-14 2024-09-18T11:31:19.247 INFO:tasks.rados.rados.0.smithi119.stdout:1314: seq_num 248 ranges {568136=785048,1929299=238837} 2024-09-18T11:31:19.253 INFO:tasks.rados.rados.0.smithi119.stdout:1314: writing smithi11939506-33 from 568136 to 1353184 tid 1 2024-09-18T11:31:19.258 INFO:tasks.rados.rados.0.smithi119.stdout:1314: writing smithi11939506-33 from 1929299 to 2168136 tid 2 2024-09-18T11:31:19.258 INFO:tasks.rados.rados.0.smithi119.stdout:1310: done (4 left) 2024-09-18T11:31:19.259 INFO:tasks.rados.rados.0.smithi119.stdout:1315: read oid 23 snap -1 2024-09-18T11:31:19.259 INFO:tasks.rados.rados.0.smithi119.stdout:1315: expect deleted 2024-09-18T11:31:19.259 INFO:tasks.rados.rados.0.smithi119.stdout:1316: snap_create 2024-09-18T11:31:19.259 INFO:tasks.rados.rados.0.smithi119.stdout:1311: finishing write tid 3 to smithi11939506-14 2024-09-18T11:31:19.259 INFO:tasks.rados.rados.0.smithi119.stdout:1311: finishing write tid 4 to smithi11939506-14 2024-09-18T11:31:19.259 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 197 (ObjNum 246 snap 28 seq_num 246) dirty exists 2024-09-18T11:31:19.259 INFO:tasks.rados.rados.0.smithi119.stdout:1311: left oid 14 (ObjNum 246 snap 28 seq_num 246) 2024-09-18T11:31:19.259 INFO:tasks.rados.rados.0.smithi119.stdout:1312: finishing set_chunk to oid 24 2024-09-18T11:31:19.259 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 86 (ObjNum 135 snap 16 seq_num 135) dirty exists 2024-09-18T11:31:19.259 INFO:tasks.rados.rados.0.smithi119.stdout:1313: finishing write tid 1 to smithi11939506-17 2024-09-18T11:31:19.259 INFO:tasks.rados.rados.0.smithi119.stdout:1313: finishing write tid 2 to smithi11939506-17 2024-09-18T11:31:19.259 INFO:tasks.rados.rados.0.smithi119.stdout:1313: finishing write tid 3 to smithi11939506-17 2024-09-18T11:31:19.259 INFO:tasks.rados.rados.0.smithi119.stdout:1313: finishing write tid 4 to smithi11939506-17 2024-09-18T11:31:19.259 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 281 (ObjNum 247 snap 28 seq_num 247) dirty exists 2024-09-18T11:31:19.260 INFO:tasks.rados.rados.0.smithi119.stdout:1313: left oid 17 (ObjNum 247 snap 28 seq_num 247) 2024-09-18T11:31:19.260 INFO:tasks.rados.rados.0.smithi119.stdout:1314: finishing write tid 1 to smithi11939506-33 2024-09-18T11:31:19.261 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:19.261 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:19.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:19.261 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:19.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.261 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:19.261 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:19.261 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.087975567000001, 2024-09-18T11:31:19.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:31:19.261 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:19.261 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:19.261 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:19.262 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:19.262 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.262 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:31:19.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.262 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:19.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:19.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:19.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:19.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:19.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:19.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:31:19.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:19.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:19.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:31:19.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:31:19.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:19.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:31:19.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:31:19.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:31:19.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:31:19.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:31:19.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:31:19.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:19.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:31:19.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:31:19.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:19.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:31:19.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:31:19.265 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.265 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:19.265 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.266 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1001 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:19.266 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:19.266 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.092824289000006, 2024-09-18T11:31:19.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007461082, 2024-09-18T11:31:19.266 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:19.266 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:19.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:19.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:19.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.266 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1001 2024-09-18T11:31:19.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.266 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:19.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:19.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:19.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:19.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:19.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:19.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164570+0000", 2024-09-18T11:31:19.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:19.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:19.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164574+0000", 2024-09-18T11:31:19.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.822e-06 2024-09-18T11:31:19.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:19.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164582+0000", 2024-09-18T11:31:19.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3410000000000003e-06 2024-09-18T11:31:19.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164587+0000", 2024-09-18T11:31:19.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769999999999999e-06 2024-09-18T11:31:19.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164702+0000", 2024-09-18T11:31:19.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115705 2024-09-18T11:31:19.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:19.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164709+0000", 2024-09-18T11:31:19.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-09-18T11:31:19.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170296+0000", 2024-09-18T11:31:19.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055876390000000001 2024-09-18T11:31:19.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:19.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170418+0000", 2024-09-18T11:31:19.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121549 2024-09-18T11:31:19.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:19.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172032+0000", 2024-09-18T11:31:19.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00161407 2024-09-18T11:31:19.270 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.270 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:19.271 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.271 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:19.271 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:19.271 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.092797572999999, 2024-09-18T11:31:19.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:31:19.271 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:19.271 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:19.271 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:19.271 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:19.271 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.271 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:31:19.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.272 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:19.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:19.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:19.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:19.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:19.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:19.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:19.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:19.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:19.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:31:19.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:31:19.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:19.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:31:19.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:31:19.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:31:19.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:31:19.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:31:19.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:31:19.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:19.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:31:19.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:31:19.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:31:19.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:31:19.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:19.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:31:19.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:31:19.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:19.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:31:19.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:31:19.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.276 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:19.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.276 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:19.276 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:19.276 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.092779284000002, 2024-09-18T11:31:19.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:31:19.276 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:19.276 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:19.276 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:19.276 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:19.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.277 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:31:19.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.277 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:19.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:19.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:19.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:19.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:19.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:19.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:31:19.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:19.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:19.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:31:19.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:31:19.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:19.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:31:19.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:31:19.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:31:19.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:31:19.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:31:19.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:31:19.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:19.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:31:19.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:31:19.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:31:19.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:31:19.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:19.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:31:19.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:31:19.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:19.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:31:19.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:31:19.281 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.281 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:19.281 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.281 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:19.281 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:19.281 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.092718515000001, 2024-09-18T11:31:19.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:31:19.281 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:19.282 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:19.282 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:19.282 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:19.282 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.282 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:31:19.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.282 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:19.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:19.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:19.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:19.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:19.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:19.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:31:19.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:19.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:19.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:31:19.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:31:19.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:19.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:31:19.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:31:19.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:31:19.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:31:19.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:31:19.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:31:19.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:19.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:31:19.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:31:19.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:31:19.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:31:19.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:19.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:31:19.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:31:19.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:19.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:31:19.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:31:19.286 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.286 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:19.286 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.286 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:19.286 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:19.286 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.092696193999998, 2024-09-18T11:31:19.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:31:19.287 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:19.287 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:19.287 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:19.287 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:19.287 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.287 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:31:19.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.287 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:19.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:19.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:19.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:19.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:19.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:19.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:31:19.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:19.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:19.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:31:19.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:31:19.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:19.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:31:19.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:31:19.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:31:19.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:31:19.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:31:19.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:31:19.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:19.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:31:19.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:31:19.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:31:19.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:31:19.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:19.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:31:19.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:31:19.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:19.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:31:19.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:31:19.291 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.291 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:19.291 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.291 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1018 4.0 4:0aec5e1c:::smithi11939506-50:head [write 1622620~730352] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:19.291 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:31:19.292 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.576461637999998, 2024-09-18T11:31:19.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064405160000000003, 2024-09-18T11:31:19.292 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:19.292 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.292 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:19.292 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:19.292 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.292 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1018 2024-09-18T11:31:19.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.292 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:19.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:19.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:31:19.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:19.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680933+0000", 2024-09-18T11:31:19.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:19.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.680932+0000", 2024-09-18T11:31:19.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:19.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:19.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681334+0000", 2024-09-18T11:31:19.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000401803 2024-09-18T11:31:19.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:19.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681350+0000", 2024-09-18T11:31:19.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6218999999999998e-05 2024-09-18T11:31:19.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681374+0000", 2024-09-18T11:31:19.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4088e-05 2024-09-18T11:31:19.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681415+0000", 2024-09-18T11:31:19.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1307000000000003e-05 2024-09-18T11:31:19.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:19.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681507+0000", 2024-09-18T11:31:19.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1867999999999995e-05 2024-09-18T11:31:19.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:19.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.681671+0000", 2024-09-18T11:31:19.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016406 2024-09-18T11:31:19.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:19.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.683856+0000", 2024-09-18T11:31:19.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002184624 2024-09-18T11:31:19.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:19.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687266+0000", 2024-09-18T11:31:19.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034105490000000001 2024-09-18T11:31:19.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:19.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687321+0000", 2024-09-18T11:31:19.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4960999999999998e-05 2024-09-18T11:31:19.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:19.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.687374+0000", 2024-09-18T11:31:19.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2553000000000003e-05 2024-09-18T11:31:19.297 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.297 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:19.297 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.297 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:31:19.297 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:19.298 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.311571311000002, 2024-09-18T11:31:19.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:31:19.298 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:19.298 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:19.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:19.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.298 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:31:19.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.298 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:19.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:19.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:19.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:19.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:19.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:19.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:31:19.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:31:19.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:19.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:31:19.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:31:19.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:19.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:31:19.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:31:19.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:31:19.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:31:19.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:31:19.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:31:19.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:19.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:31:19.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:31:19.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:19.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:31:19.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:31:19.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:19.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:31:19.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:31:19.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:19.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:31:19.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:31:19.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:19.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:31:19.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:31:19.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:19.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:31:19.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:31:19.303 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.303 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:19.303 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.303 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:29 3.2 3:5fa8fcfa:::ec59520759824f0488c7a3ca135dacb324de2330e16b7f54053de5719c4e96b6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:31:19.303 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:31:19.303 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.577213395000001, 2024-09-18T11:31:19.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0064126499999999998, 2024-09-18T11:31:19.303 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:19.304 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.304 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:19.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:19.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:31:19.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:19.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680182+0000", 2024-09-18T11:31:19.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:19.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680181+0000", 2024-09-18T11:31:19.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:19.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:19.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680654+0000", 2024-09-18T11:31:19.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00047334900000000002 2024-09-18T11:31:19.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:19.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680666+0000", 2024-09-18T11:31:19.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2232000000000001e-05 2024-09-18T11:31:19.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.680672+0000", 2024-09-18T11:31:19.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:31:19.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681222+0000", 2024-09-18T11:31:19.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00054928699999999997 2024-09-18T11:31:19.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:19.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681332+0000", 2024-09-18T11:31:19.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110634 2024-09-18T11:31:19.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:19.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.681552+0000", 2024-09-18T11:31:19.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021926899999999999 2024-09-18T11:31:19.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:19.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.682950+0000", 2024-09-18T11:31:19.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013982669999999999 2024-09-18T11:31:19.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:19.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686503+0000", 2024-09-18T11:31:19.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035536130000000002 2024-09-18T11:31:19.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:19.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686553+0000", 2024-09-18T11:31:19.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9801e-05 2024-09-18T11:31:19.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:19.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:20.686594+0000", 2024-09-18T11:31:19.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1022000000000001e-05 2024-09-18T11:31:19.308 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.309 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:19.309 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.309 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1199 4.0 4:0aec5e1c:::smithi11939506-50:head [write 763051~794393] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-18T11:31:19.309 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:31:19.309 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.060399271000001, 2024-09-18T11:31:19.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067768530000000002, 2024-09-18T11:31:19.309 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:19.309 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.309 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:19.309 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:19.309 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.309 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1199 2024-09-18T11:31:19.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.310 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:19.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:19.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:31:19.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:19.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196996+0000", 2024-09-18T11:31:19.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:19.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.196994+0000", 2024-09-18T11:31:19.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:19.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:19.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197355+0000", 2024-09-18T11:31:19.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036035199999999998 2024-09-18T11:31:19.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:19.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197366+0000", 2024-09-18T11:31:19.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.17e-05 2024-09-18T11:31:19.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197371+0000", 2024-09-18T11:31:19.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6099999999999999e-06 2024-09-18T11:31:19.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197416+0000", 2024-09-18T11:31:19.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5099000000000003e-05 2024-09-18T11:31:19.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:19.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.197420+0000", 2024-09-18T11:31:19.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3189999999999996e-06 2024-09-18T11:31:19.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199392+0000", 2024-09-18T11:31:19.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019720139999999998 2024-09-18T11:31:19.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:19.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199452+0000", 2024-09-18T11:31:19.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9929999999999997e-05 2024-09-18T11:31:19.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:19.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.199565+0000", 2024-09-18T11:31:19.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112811 2024-09-18T11:31:19.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:19.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.202108+0000", 2024-09-18T11:31:19.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025429060000000002 2024-09-18T11:31:19.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:19.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203714+0000", 2024-09-18T11:31:19.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001606335 2024-09-18T11:31:19.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:19.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203745+0000", 2024-09-18T11:31:19.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0837999999999998e-05 2024-09-18T11:31:19.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:19.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:21.203773+0000", 2024-09-18T11:31:19.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7562000000000001e-05 2024-09-18T11:31:19.315 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.315 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:19.315 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.315 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1661 4.0 4:019b6e09:::smithi11939506-40:17 [read 0~3445402,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:19.315 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:19.315 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.907498607999997, 2024-09-18T11:31:19.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075190880000000002, 2024-09-18T11:31:19.316 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:19.316 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:19.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:19.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:19.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.316 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1661 2024-09-18T11:31:19.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.316 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:19.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:19.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:19.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:19.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:19.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:19.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:19.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:19.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:19.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349899+0000", 2024-09-18T11:31:19.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9749999999999999e-06 2024-09-18T11:31:19.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:19.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349906+0000", 2024-09-18T11:31:19.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9290000000000001e-06 2024-09-18T11:31:19.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349912+0000", 2024-09-18T11:31:19.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9900000000000002e-06 2024-09-18T11:31:19.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355645+0000", 2024-09-18T11:31:19.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057329549999999996 2024-09-18T11:31:19.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:19.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355776+0000", 2024-09-18T11:31:19.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130975 2024-09-18T11:31:19.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:19.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357416+0000", 2024-09-18T11:31:19.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001639649 2024-09-18T11:31:19.319 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.319 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:19.320 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.320 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1761 4.0 4:0aec5e1c:::smithi11939506-50:head [write 769787~704463,stat] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:19.320 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:19.320 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.955006193999999, 2024-09-18T11:31:19.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074082879999999999, 2024-09-18T11:31:19.320 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:19.320 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:19.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:19.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.320 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1761 2024-09-18T11:31:19.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.321 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:19.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:19.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:19.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:19.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:19.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:19.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302386+0000", 2024-09-18T11:31:19.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:19.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:19.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302889+0000", 2024-09-18T11:31:19.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050318499999999996 2024-09-18T11:31:19.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:19.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302930+0000", 2024-09-18T11:31:19.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0198999999999999e-05 2024-09-18T11:31:19.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302965+0000", 2024-09-18T11:31:19.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5292e-05 2024-09-18T11:31:19.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303046+0000", 2024-09-18T11:31:19.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1254999999999994e-05 2024-09-18T11:31:19.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:19.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303054+0000", 2024-09-18T11:31:19.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2800000000000003e-06 2024-09-18T11:31:19.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304658+0000", 2024-09-18T11:31:19.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016039520000000001 2024-09-18T11:31:19.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:19.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304724+0000", 2024-09-18T11:31:19.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5888000000000003e-05 2024-09-18T11:31:19.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:19.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304853+0000", 2024-09-18T11:31:19.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012912700000000001 2024-09-18T11:31:19.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:19.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309084+0000", 2024-09-18T11:31:19.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042311229999999998 2024-09-18T11:31:19.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:19.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309738+0000", 2024-09-18T11:31:19.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00065401399999999996 2024-09-18T11:31:19.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:19.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309770+0000", 2024-09-18T11:31:19.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1652000000000001e-05 2024-09-18T11:31:19.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:19.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309797+0000", 2024-09-18T11:31:19.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7052000000000001e-05 2024-09-18T11:31:19.326 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.326 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:19.326 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.326 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1951 4.0 4:019b6e09:::smithi11939506-40:13 [sparse-read 0~2626478] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:19.326 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.339659+0000", 2024-09-18T11:31:19.327 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.917736487999999, 2024-09-18T11:31:19.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067895999999999998, 2024-09-18T11:31:19.327 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:19.327 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:19.327 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:19.327 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:19.327 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.327 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1951 2024-09-18T11:31:19.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.327 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:19.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:19.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339659+0000", 2024-09-18T11:31:19.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:19.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339659+0000", 2024-09-18T11:31:19.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:19.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339658+0000", 2024-09-18T11:31:19.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:19.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:19.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339661+0000", 2024-09-18T11:31:19.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3749999999999999e-06 2024-09-18T11:31:19.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:19.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339668+0000", 2024-09-18T11:31:19.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1380000000000001e-06 2024-09-18T11:31:19.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339675+0000", 2024-09-18T11:31:19.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7499999999999997e-06 2024-09-18T11:31:19.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339748+0000", 2024-09-18T11:31:19.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2885000000000004e-05 2024-09-18T11:31:19.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:19.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339751+0000", 2024-09-18T11:31:19.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.446e-06 2024-09-18T11:31:19.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345249+0000", 2024-09-18T11:31:19.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054983360000000004 2024-09-18T11:31:19.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:19.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345332+0000", 2024-09-18T11:31:19.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3245999999999995e-05 2024-09-18T11:31:19.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:19.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346448+0000", 2024-09-18T11:31:19.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0011160530000000001 2024-09-18T11:31:19.331 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.331 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:19.331 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.331 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:306 4.f e69/22 4:f5d7df41:::smithi11939506-16:17 v 69'94, mlcod=65'88)", 2024-09-18T11:31:19.332 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:19.332 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.906279476999998, 2024-09-18T11:31:19.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068024720000000004, 2024-09-18T11:31:19.332 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:19.332 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.332 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:19.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:19.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:19.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:19.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:19.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:19.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351115+0000", 2024-09-18T11:31:19.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:19.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:19.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351121+0000", 2024-09-18T11:31:19.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-09-18T11:31:19.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:19.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351126+0000", 2024-09-18T11:31:19.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5969999999999999e-06 2024-09-18T11:31:19.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351147+0000", 2024-09-18T11:31:19.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1160000000000001e-05 2024-09-18T11:31:19.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351518+0000", 2024-09-18T11:31:19.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00037079300000000001 2024-09-18T11:31:19.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:19.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351536+0000", 2024-09-18T11:31:19.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8168999999999998e-05 2024-09-18T11:31:19.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:19.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357870+0000", 2024-09-18T11:31:19.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063332650000000002 2024-09-18T11:31:19.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:19.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357918+0000", 2024-09-18T11:31:19.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8347999999999999e-05 2024-09-18T11:31:19.336 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.336 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:19.336 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.336 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2022 4.0 4:0aec5e1c:::smithi11939506-50:head [write 457190~552096,stat] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-18T11:31:19.336 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:19.336 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.261241340000002, 2024-09-18T11:31:19.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006803974, 2024-09-18T11:31:19.336 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:19.336 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.336 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:19.336 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:19.336 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.337 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2022 2024-09-18T11:31:19.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.337 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:19.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:19.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:19.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:19.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:19.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:19.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996151+0000", 2024-09-18T11:31:19.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:19.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:19.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996461+0000", 2024-09-18T11:31:19.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000309587 2024-09-18T11:31:19.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:19.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996481+0000", 2024-09-18T11:31:19.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9949000000000002e-05 2024-09-18T11:31:19.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996489+0000", 2024-09-18T11:31:19.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9129999999999992e-06 2024-09-18T11:31:19.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996562+0000", 2024-09-18T11:31:19.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2807999999999998e-05 2024-09-18T11:31:19.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:19.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996687+0000", 2024-09-18T11:31:19.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012458200000000001 2024-09-18T11:31:19.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:19.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996909+0000", 2024-09-18T11:31:19.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222124 2024-09-18T11:31:19.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:19.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.000741+0000", 2024-09-18T11:31:19.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038320720000000002 2024-09-18T11:31:19.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:19.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002854+0000", 2024-09-18T11:31:19.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002113228 2024-09-18T11:31:19.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:19.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002911+0000", 2024-09-18T11:31:19.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7192999999999999e-05 2024-09-18T11:31:19.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:19.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002958+0000", 2024-09-18T11:31:19.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6242000000000002e-05 2024-09-18T11:31:19.341 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.341 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:19.342 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.342 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2342 4.f e78/22 4:f3c51b3f:::smithi11939506-21:head v 78'127, mlcod=77'123)", 2024-09-18T11:31:19.342 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.390871+0000", 2024-09-18T11:31:19.342 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.866524019, 2024-09-18T11:31:19.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0065260930000000002, 2024-09-18T11:31:19.342 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:19.342 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.342 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:19.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:19.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390871+0000", 2024-09-18T11:31:19.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:19.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390871+0000", 2024-09-18T11:31:19.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:19.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390871+0000", 2024-09-18T11:31:19.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:19.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:19.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390875+0000", 2024-09-18T11:31:19.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2390000000000004e-06 2024-09-18T11:31:19.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:19.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390887+0000", 2024-09-18T11:31:19.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2055999999999999e-05 2024-09-18T11:31:19.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390892+0000", 2024-09-18T11:31:19.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4179999999999996e-06 2024-09-18T11:31:19.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390937+0000", 2024-09-18T11:31:19.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4415000000000001e-05 2024-09-18T11:31:19.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:19.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390954+0000", 2024-09-18T11:31:19.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7062999999999999e-05 2024-09-18T11:31:19.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:19.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397345+0000", 2024-09-18T11:31:19.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063909609999999997 2024-09-18T11:31:19.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:19.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397397+0000", 2024-09-18T11:31:19.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2237999999999997e-05 2024-09-18T11:31:19.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.346 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:19.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.346 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2503 3.1 e85/67 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head v 85'117, mlcod=84'115)", 2024-09-18T11:31:19.346 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:31:19.346 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 10.280172624, 2024-09-18T11:31:19.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086527619999999996, 2024-09-18T11:31:19.346 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:19.346 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.347 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:19.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:19.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:31:19.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:19.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:31:19.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:19.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977220+0000", 2024-09-18T11:31:19.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:19.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:19.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980771+0000", 2024-09-18T11:31:19.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035505200000000002 2024-09-18T11:31:19.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:19.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980797+0000", 2024-09-18T11:31:19.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.618e-05 2024-09-18T11:31:19.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980807+0000", 2024-09-18T11:31:19.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0433e-05 2024-09-18T11:31:19.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980861+0000", 2024-09-18T11:31:19.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3415999999999998e-05 2024-09-18T11:31:19.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:19.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980885+0000", 2024-09-18T11:31:19.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4212000000000001e-05 2024-09-18T11:31:19.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:19.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985826+0000", 2024-09-18T11:31:19.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049412450000000004 2024-09-18T11:31:19.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:19.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985875+0000", 2024-09-18T11:31:19.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9116000000000002e-05 2024-09-18T11:31:19.350 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.350 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:19.350 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.351 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2648 4.0 4:059d8d95:::smithi11939506-32:head [read 0~3647119,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:19.351 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:31:19.351 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9260842540000001, 2024-09-18T11:31:19.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0076286949999999996, 2024-09-18T11:31:19.351 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:19.351 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:19.351 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:19.351 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:19.351 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.351 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2648 2024-09-18T11:31:19.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.351 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:19.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:19.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:31:19.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:19.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:31:19.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:19.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331309+0000", 2024-09-18T11:31:19.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:19.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:19.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331321+0000", 2024-09-18T11:31:19.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:31:19.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:19.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331340+0000", 2024-09-18T11:31:19.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8227e-05 2024-09-18T11:31:19.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331348+0000", 2024-09-18T11:31:19.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4640000000000006e-06 2024-09-18T11:31:19.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331406+0000", 2024-09-18T11:31:19.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7445000000000003e-05 2024-09-18T11:31:19.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:19.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331521+0000", 2024-09-18T11:31:19.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115945 2024-09-18T11:31:19.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:19.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338939+0000", 2024-09-18T11:31:19.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074179859999999997 2024-09-18T11:31:19.354 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.355 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:19.355 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.355 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2649 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:19.355 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:31:19.355 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.9260292549999996, 2024-09-18T11:31:19.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566240000000007, 2024-09-18T11:31:19.355 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:19.355 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:19.355 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:19.355 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:19.355 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.355 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2649 2024-09-18T11:31:19.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.356 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:19.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:19.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:31:19.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:19.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:31:19.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:19.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331365+0000", 2024-09-18T11:31:19.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:19.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:19.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331369+0000", 2024-09-18T11:31:19.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.551e-06 2024-09-18T11:31:19.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:19.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331377+0000", 2024-09-18T11:31:19.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8769999999999993e-06 2024-09-18T11:31:19.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331381+0000", 2024-09-18T11:31:19.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.16e-06 2024-09-18T11:31:19.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338973+0000", 2024-09-18T11:31:19.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075918460000000002 2024-09-18T11:31:19.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:19.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339048+0000", 2024-09-18T11:31:19.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5813000000000001e-05 2024-09-18T11:31:19.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:19.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339322+0000", 2024-09-18T11:31:19.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027399899999999998 2024-09-18T11:31:19.359 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.359 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:19.359 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.359 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2650 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:19.359 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:19.359 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.926009037, 2024-09-18T11:31:19.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084392180000000001, 2024-09-18T11:31:19.360 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:19.360 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:19.360 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:19.360 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:19.360 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.360 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2650 2024-09-18T11:31:19.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.360 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:19.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:19.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:19.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:19.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:19.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:19.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:19.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:19.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:19.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:19.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331388+0000", 2024-09-18T11:31:19.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3429999999999998e-06 2024-09-18T11:31:19.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:19.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331394+0000", 2024-09-18T11:31:19.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.19e-06 2024-09-18T11:31:19.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331398+0000", 2024-09-18T11:31:19.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.495e-06 2024-09-18T11:31:19.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:19.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339362+0000", 2024-09-18T11:31:19.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079641799999999995 2024-09-18T11:31:19.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:19.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339460+0000", 2024-09-18T11:31:19.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8329e-05 2024-09-18T11:31:19.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:19.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:19.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:19.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339825+0000", 2024-09-18T11:31:19.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036510500000000001 2024-09-18T11:31:19.363 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.363 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:19.363 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.363 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:19.363 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:19.364 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:19.364 INFO:tasks.rados.rados.0.smithi119.stdout:1314: finishing write tid 2 to smithi11939506-33 2024-09-18T11:31:19.364 INFO:tasks.rados.rados.0.smithi119.stdout:1314: finishing write tid 3 to smithi11939506-33 2024-09-18T11:31:19.364 INFO:tasks.rados.rados.0.smithi119.stdout:1314: finishing write tid 4 to smithi11939506-33 2024-09-18T11:31:19.364 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 33 v 113 (ObjNum 248 snap 28 seq_num 248) dirty exists 2024-09-18T11:31:19.365 INFO:tasks.rados.rados.0.smithi119.stdout:1314: left oid 33 (ObjNum 248 snap 28 seq_num 248) 2024-09-18T11:31:19.365 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:31:19.367 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:19.365+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:19.468 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:19.465+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:19.471 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:19.472 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:19.472 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:19.472 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:19.482 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:31:19.483 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is unset 2024-09-18T11:31:19.483 INFO:tasks.rados.rados.0.smithi119.stdout:1311: done (5 left) 2024-09-18T11:31:19.483 INFO:tasks.rados.rados.0.smithi119.stdout:1312: done (4 left) 2024-09-18T11:31:19.483 INFO:tasks.rados.rados.0.smithi119.stdout:1313: done (3 left) 2024-09-18T11:31:19.483 INFO:tasks.rados.rados.0.smithi119.stdout:1314: done (2 left) 2024-09-18T11:31:19.483 INFO:tasks.rados.rados.0.smithi119.stdout:1315: done (1 left) 2024-09-18T11:31:19.483 INFO:tasks.rados.rados.0.smithi119.stdout:1316: done (0 left) 2024-09-18T11:31:19.483 INFO:tasks.rados.rados.0.smithi119.stdout:1317: snap_create 2024-09-18T11:31:19.569 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:19.568+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:19.620 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:19.620 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:19.620 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:19.621 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:19.621 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:19.629 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:31:19.672 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:19.670+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:19.727 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:19.728 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:19.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:19.728 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:19.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.728 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1530 4.d 4:b81644f2:::smithi11939506-34:head [write 467261~717275] snapc 14=[14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:31:19.728 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:31:19.728 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.334886826000002, 2024-09-18T11:31:19.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075660950000000001, 2024-09-18T11:31:19.728 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:19.728 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.728 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:19.728 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:19.729 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.729 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1530 2024-09-18T11:31:19.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.729 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:19.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.729 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:19.729 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:31:19.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.729 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:19.729 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:31:19.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.730 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:19.730 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390682+0000", 2024-09-18T11:31:19.730 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:19.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.730 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:19.730 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391554+0000", 2024-09-18T11:31:19.730 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00087220600000000004 2024-09-18T11:31:19.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.730 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:19.730 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391570+0000", 2024-09-18T11:31:19.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6002999999999999e-05 2024-09-18T11:31:19.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.731 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391575+0000", 2024-09-18T11:31:19.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2549999999999997e-06 2024-09-18T11:31:19.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.731 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:19.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391631+0000", 2024-09-18T11:31:19.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5671000000000003e-05 2024-09-18T11:31:19.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.732 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:19.732 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391751+0000", 2024-09-18T11:31:19.732 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000119821 2024-09-18T11:31:19.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.732 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:19.732 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391974+0000", 2024-09-18T11:31:19.732 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022309900000000001 2024-09-18T11:31:19.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.733 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:19.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398136+0000", 2024-09-18T11:31:19.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006161573 2024-09-18T11:31:19.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.733 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:19.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398184+0000", 2024-09-18T11:31:19.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8458000000000003e-05 2024-09-18T11:31:19.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.733 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:19.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398219+0000", 2024-09-18T11:31:19.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4573999999999998e-05 2024-09-18T11:31:19.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.734 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:19.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398251+0000", 2024-09-18T11:31:19.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2085000000000002e-05 2024-09-18T11:31:19.735 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.735 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:19.735 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.736 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:31:19.736 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:19.736 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.309484501999997, 2024-09-18T11:31:19.736 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:31:19.736 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:19.736 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.736 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:19.736 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:19.736 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.736 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:31:19.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.737 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:19.737 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:19.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:19.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.737 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:19.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:19.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.737 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:19.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:31:19.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:19.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:19.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:31:19.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:31:19.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:19.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:31:19.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:31:19.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.739 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.739 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:31:19.739 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:31:19.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.739 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.739 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:19.739 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:31:19.739 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:31:19.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.739 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.739 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:19.739 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:31:19.739 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:31:19.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.740 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:19.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:31:19.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:31:19.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.740 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:19.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:31:19.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:31:19.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.740 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:19.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:31:19.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:31:19.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.741 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:19.741 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:31:19.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:31:19.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.741 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:19.741 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:31:19.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:31:19.741 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.741 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:19.741 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.742 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1628 3.3 3:f80aa31b:::77cdbf3abbada57d0636b2198046a030673818705cbf13bb77b30c3e68b24345:head [write 0~582144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:19.742 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:19.742 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 42.414594502, 2024-09-18T11:31:19.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081714639999999998, 2024-09-18T11:31:19.742 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:19.742 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.742 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:19.742 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:19.742 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.742 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1628 2024-09-18T11:31:19.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.742 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:19.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:19.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:19.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:19.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:19.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.744 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:19.744 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310975+0000", 2024-09-18T11:31:19.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:19.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.744 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:19.744 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312086+0000", 2024-09-18T11:31:19.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001111288 2024-09-18T11:31:19.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.744 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:19.744 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312104+0000", 2024-09-18T11:31:19.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7427000000000001e-05 2024-09-18T11:31:19.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312110+0000", 2024-09-18T11:31:19.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5320000000000003e-06 2024-09-18T11:31:19.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:19.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312179+0000", 2024-09-18T11:31:19.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9196000000000006e-05 2024-09-18T11:31:19.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:19.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312375+0000", 2024-09-18T11:31:19.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00019550899999999999 2024-09-18T11:31:19.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.746 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:19.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312558+0000", 2024-09-18T11:31:19.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000182566 2024-09-18T11:31:19.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.746 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:19.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.314286+0000", 2024-09-18T11:31:19.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00172831 2024-09-18T11:31:19.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:19.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319079+0000", 2024-09-18T11:31:19.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047929340000000004 2024-09-18T11:31:19.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:19.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319116+0000", 2024-09-18T11:31:19.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6993999999999999e-05 2024-09-18T11:31:19.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:19.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319148+0000", 2024-09-18T11:31:19.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2539999999999997e-05 2024-09-18T11:31:19.747 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.748 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:19.748 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.748 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1663 4.1 4:87078dfe:::smithi11939506-14:head [write 1916458~318873] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:19.748 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:19.748 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 42.361647939000001, 2024-09-18T11:31:19.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079505749999999997, 2024-09-18T11:31:19.748 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:19.748 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.748 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:19.748 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:19.748 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.749 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1663 2024-09-18T11:31:19.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.749 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:19.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.749 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:19.749 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:19.749 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.749 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:19.749 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:19.749 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:19.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363922+0000", 2024-09-18T11:31:19.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:19.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:19.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364211+0000", 2024-09-18T11:31:19.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000288385 2024-09-18T11:31:19.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:19.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364223+0000", 2024-09-18T11:31:19.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2582000000000001e-05 2024-09-18T11:31:19.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.751 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.751 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364228+0000", 2024-09-18T11:31:19.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0000000000000004e-06 2024-09-18T11:31:19.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.751 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:19.751 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364268+0000", 2024-09-18T11:31:19.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0235999999999999e-05 2024-09-18T11:31:19.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.751 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:19.751 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364341+0000", 2024-09-18T11:31:19.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2266000000000002e-05 2024-09-18T11:31:19.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:19.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364444+0000", 2024-09-18T11:31:19.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103632 2024-09-18T11:31:19.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:19.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369461+0000", 2024-09-18T11:31:19.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005016848 2024-09-18T11:31:19.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.753 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:19.753 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371820+0000", 2024-09-18T11:31:19.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023586169999999999 2024-09-18T11:31:19.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.753 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:19.753 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371851+0000", 2024-09-18T11:31:19.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1297999999999997e-05 2024-09-18T11:31:19.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.754 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:19.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371874+0000", 2024-09-18T11:31:19.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2841e-05 2024-09-18T11:31:19.754 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.754 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:19.754 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.754 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1664 4.1 4:87078dfe:::smithi11939506-14:head [setxattr _header (57) in=7b,truncate 2235331] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:19.754 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:19.754 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 42.361334282000001, 2024-09-18T11:31:19.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010368235999999999, 2024-09-18T11:31:19.754 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:19.755 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.755 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:19.755 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:19.755 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.755 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1664 2024-09-18T11:31:19.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.755 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:19.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:19.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:19.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:19.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:19.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:19.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364236+0000", 2024-09-18T11:31:19.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:19.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:19.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364240+0000", 2024-09-18T11:31:19.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9829999999999998e-06 2024-09-18T11:31:19.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:19.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364247+0000", 2024-09-18T11:31:19.757 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2140000000000001e-06 2024-09-18T11:31:19.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364256+0000", 2024-09-18T11:31:19.757 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1509999999999993e-06 2024-09-18T11:31:19.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:19.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366529+0000", 2024-09-18T11:31:19.757 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022731409999999998 2024-09-18T11:31:19.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.758 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:19.758 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366621+0000", 2024-09-18T11:31:19.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2408e-05 2024-09-18T11:31:19.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.758 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:19.758 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366801+0000", 2024-09-18T11:31:19.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018002200000000001 2024-09-18T11:31:19.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.758 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:19.758 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372155+0000", 2024-09-18T11:31:19.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053531519999999999 2024-09-18T11:31:19.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:19.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374539+0000", 2024-09-18T11:31:19.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023842580000000002 2024-09-18T11:31:19.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:19.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374573+0000", 2024-09-18T11:31:19.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4205999999999997e-05 2024-09-18T11:31:19.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:19.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374605+0000", 2024-09-18T11:31:19.760 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2283999999999998e-05 2024-09-18T11:31:19.760 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.760 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:19.760 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.760 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.760 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1665 4.1 4:87078dfe:::smithi11939506-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:19.760 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:19.760 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 42.361310107999998, 2024-09-18T11:31:19.760 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010471897000000001, 2024-09-18T11:31:19.760 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:19.760 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:19.760 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:19.760 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:19.761 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.761 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1665 2024-09-18T11:31:19.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.761 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:19.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.761 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:19.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:19.761 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.761 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:19.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:19.761 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:19.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:19.762 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:19.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:19.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364263+0000", 2024-09-18T11:31:19.762 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6400000000000001e-06 2024-09-18T11:31:19.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:19.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364270+0000", 2024-09-18T11:31:19.762 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:31:19.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.763 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.763 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364274+0000", 2024-09-18T11:31:19.763 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.968e-06 2024-09-18T11:31:19.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.763 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:19.763 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367354+0000", 2024-09-18T11:31:19.763 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030803430000000001 2024-09-18T11:31:19.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.763 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:19.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367406+0000", 2024-09-18T11:31:19.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2330999999999998e-05 2024-09-18T11:31:19.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:19.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374625+0000", 2024-09-18T11:31:19.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072190689999999998 2024-09-18T11:31:19.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:19.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374673+0000", 2024-09-18T11:31:19.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7525999999999999e-05 2024-09-18T11:31:19.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.765 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:19.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374733+0000", 2024-09-18T11:31:19.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0359000000000003e-05 2024-09-18T11:31:19.765 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.765 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:19.765 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.765 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1947 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865577,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:19.765 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:19.765 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 28.386030397999999, 2024-09-18T11:31:19.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010043066999999999, 2024-09-18T11:31:19.765 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:19.765 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:19.766 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:19.766 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:19.766 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.766 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1947 2024-09-18T11:31:19.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.766 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:19.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.766 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:19.766 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:19.766 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.766 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:19.767 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:19.767 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.767 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.767 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:19.767 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339540+0000", 2024-09-18T11:31:19.767 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:19.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.767 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.767 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:19.767 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339545+0000", 2024-09-18T11:31:19.767 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6290000000000001e-06 2024-09-18T11:31:19.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.768 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:19.768 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339553+0000", 2024-09-18T11:31:19.768 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6299999999999998e-06 2024-09-18T11:31:19.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.768 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.768 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339557+0000", 2024-09-18T11:31:19.768 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3590000000000001e-06 2024-09-18T11:31:19.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.768 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:19.768 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345600+0000", 2024-09-18T11:31:19.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060434640000000001 2024-09-18T11:31:19.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.769 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.769 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:19.769 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345710+0000", 2024-09-18T11:31:19.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010924700000000001 2024-09-18T11:31:19.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.769 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.769 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:19.769 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349584+0000", 2024-09-18T11:31:19.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038743459999999999 2024-09-18T11:31:19.769 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.769 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:19.769 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.770 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1948 4.d 4:b5a463bf:::smithi11939506-20:head [sparse-read 0~2642181] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:19.770 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:19.770 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 28.386009861000002, 2024-09-18T11:31:19.770 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010701952000000001, 2024-09-18T11:31:19.770 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:19.770 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:19.770 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:19.770 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:19.770 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.770 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1948 2024-09-18T11:31:19.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.771 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:19.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:19.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:19.771 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:19.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:19.771 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.772 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:19.772 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:19.772 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:19.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.772 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:19.772 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339563+0000", 2024-09-18T11:31:19.772 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1950000000000002e-06 2024-09-18T11:31:19.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.773 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.773 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:19.773 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339569+0000", 2024-09-18T11:31:19.773 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5459999999999999e-06 2024-09-18T11:31:19.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.773 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.773 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.773 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339572+0000", 2024-09-18T11:31:19.773 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.337e-06 2024-09-18T11:31:19.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.773 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.773 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:19.773 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349693+0000", 2024-09-18T11:31:19.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010120987999999999 2024-09-18T11:31:19.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.774 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:19.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349816+0000", 2024-09-18T11:31:19.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012241199999999999 2024-09-18T11:31:19.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.774 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:19.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350263+0000", 2024-09-18T11:31:19.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00044787199999999998 2024-09-18T11:31:19.774 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.774 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:19.774 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.775 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:131 4.5 e69/22 4:aa575018:::smithi11939506-11:17 v 69'97, mlcod=65'92)", 2024-09-18T11:31:19.775 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.346049+0000", 2024-09-18T11:31:19.775 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 28.379522801, 2024-09-18T11:31:19.775 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0077275240000000004, 2024-09-18T11:31:19.775 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:19.775 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.775 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:19.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.775 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:19.775 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346049+0000", 2024-09-18T11:31:19.775 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.776 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:19.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346049+0000", 2024-09-18T11:31:19.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.776 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:19.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346047+0000", 2024-09-18T11:31:19.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:19.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.776 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:19.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346058+0000", 2024-09-18T11:31:19.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0463e-05 2024-09-18T11:31:19.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:19.777 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346069+0000", 2024-09-18T11:31:19.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1708999999999999e-05 2024-09-18T11:31:19.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.777 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346076+0000", 2024-09-18T11:31:19.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2450000000000003e-06 2024-09-18T11:31:19.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:19.778 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349616+0000", 2024-09-18T11:31:19.778 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035406550000000002 2024-09-18T11:31:19.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.778 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:19.778 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349638+0000", 2024-09-18T11:31:19.778 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2098000000000001e-05 2024-09-18T11:31:19.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.778 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:19.778 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.353735+0000", 2024-09-18T11:31:19.778 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004096237 2024-09-18T11:31:19.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.779 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:19.779 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.353776+0000", 2024-09-18T11:31:19.779 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1530999999999999e-05 2024-09-18T11:31:19.779 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.779 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:19.779 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.779 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.779 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2129 4.1 4:8b9a985d:::smithi11939506-23:16 [read 0~3512630,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T11:31:19.779 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:55.710202+0000", 2024-09-18T11:31:19.779 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 24.015369539999998, 2024-09-18T11:31:19.779 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0078396100000000003, 2024-09-18T11:31:19.779 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:19.780 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:19.780 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:19.780 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:19.780 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.780 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2129 2024-09-18T11:31:19.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.780 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:19.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:19.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710202+0000", 2024-09-18T11:31:19.780 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:19.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710202+0000", 2024-09-18T11:31:19.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.781 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:19.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710202+0000", 2024-09-18T11:31:19.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:19.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.781 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:19.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710205+0000", 2024-09-18T11:31:19.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.382e-06 2024-09-18T11:31:19.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:19.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710212+0000", 2024-09-18T11:31:19.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770000000000004e-06 2024-09-18T11:31:19.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710217+0000", 2024-09-18T11:31:19.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1680000000000003e-06 2024-09-18T11:31:19.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:19.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.716963+0000", 2024-09-18T11:31:19.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0067462449999999997 2024-09-18T11:31:19.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.783 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:19.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.717033+0000", 2024-09-18T11:31:19.783 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9643999999999999e-05 2024-09-18T11:31:19.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.783 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:19.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.718041+0000", 2024-09-18T11:31:19.783 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0010086019999999999 2024-09-18T11:31:19.783 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.783 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:19.783 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.784 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:172 4.5 e78/22 4:a087a566:::smithi11939506-46:19 v 78'135, mlcod=77'132)", 2024-09-18T11:31:19.784 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:19.784 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.339747848000002, 2024-09-18T11:31:19.784 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010938658, 2024-09-18T11:31:19.784 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:19.784 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.784 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:19.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.784 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:19.784 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:19.784 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.785 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:19.785 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:19.785 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.785 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:19.785 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385823+0000", 2024-09-18T11:31:19.785 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:19.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.785 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:19.785 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385830+0000", 2024-09-18T11:31:19.785 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2790000000000001e-06 2024-09-18T11:31:19.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:19.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385843+0000", 2024-09-18T11:31:19.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2785e-05 2024-09-18T11:31:19.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385848+0000", 2024-09-18T11:31:19.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5570000000000003e-06 2024-09-18T11:31:19.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:19.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385914+0000", 2024-09-18T11:31:19.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5168999999999999e-05 2024-09-18T11:31:19.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:19.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385930+0000", 2024-09-18T11:31:19.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6884999999999998e-05 2024-09-18T11:31:19.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:19.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396721+0000", 2024-09-18T11:31:19.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010790137 2024-09-18T11:31:19.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:19.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396762+0000", 2024-09-18T11:31:19.788 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1635999999999999e-05 2024-09-18T11:31:19.788 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.788 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:19.788 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.788 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:31:19.788 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:19.788 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.339255095999999, 2024-09-18T11:31:19.788 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:31:19.788 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:19.788 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.788 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:19.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.789 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:19.789 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:19.789 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.789 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:19.789 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:19.789 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.789 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:19.789 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:19.789 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:19.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.790 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.790 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:19.790 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:31:19.790 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:31:19.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.790 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.790 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:19.790 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:31:19.790 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:31:19.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.790 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.790 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.790 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:31:19.791 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:31:19.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.791 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.791 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:19.791 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:31:19.791 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:31:19.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.791 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.791 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:19.791 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:31:19.791 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:31:19.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.792 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:19.792 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:31:19.792 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:31:19.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.792 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:19.792 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:31:19.792 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:31:19.792 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.792 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:19.792 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.793 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:19.793 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:19.793 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 10.75314249, 2024-09-18T11:31:19.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:31:19.793 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:19.793 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.793 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:19.793 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:19.793 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.793 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:31:19.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.793 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:19.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:19.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:19.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:19.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:19.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:19.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:31:19.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:19.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:19.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:31:19.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:31:19.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:19.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:31:19.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:31:19.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:31:19.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:31:19.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:19.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:31:19.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:31:19.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:19.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:31:19.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:31:19.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:19.797 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:31:19.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:31:19.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.797 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:19.797 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:31:19.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:31:19.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.797 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:19.797 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:31:19.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:31:19.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:19.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:31:19.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:31:19.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:19.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:31:19.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:31:19.798 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.798 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:19.798 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.798 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2505 4.1 4:87078dfe:::smithi11939506-14:head [write 497540~612664,stat] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:19.799 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.992775+0000", 2024-09-18T11:31:19.799 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 10.732796504, 2024-09-18T11:31:19.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076090139999999999, 2024-09-18T11:31:19.799 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:19.799 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.799 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:19.799 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:19.799 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.799 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2505 2024-09-18T11:31:19.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.799 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:19.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:19.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.992775+0000", 2024-09-18T11:31:19.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:19.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.992775+0000", 2024-09-18T11:31:19.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:19.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.992773+0000", 2024-09-18T11:31:19.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:19.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:19.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993364+0000", 2024-09-18T11:31:19.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059123300000000004 2024-09-18T11:31:19.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:19.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993377+0000", 2024-09-18T11:31:19.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3464e-05 2024-09-18T11:31:19.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993382+0000", 2024-09-18T11:31:19.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2440000000000004e-06 2024-09-18T11:31:19.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:19.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993460+0000", 2024-09-18T11:31:19.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8584000000000005e-05 2024-09-18T11:31:19.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:19.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993530+0000", 2024-09-18T11:31:19.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0060000000000003e-05 2024-09-18T11:31:19.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:19.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993650+0000", 2024-09-18T11:31:19.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120079 2024-09-18T11:31:19.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:19.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.998802+0000", 2024-09-18T11:31:19.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005152106 2024-09-18T11:31:19.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:19.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.000318+0000", 2024-09-18T11:31:19.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015154719999999999 2024-09-18T11:31:19.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:19.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.000352+0000", 2024-09-18T11:31:19.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4418000000000001e-05 2024-09-18T11:31:19.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.804 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:19.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.000384+0000", 2024-09-18T11:31:19.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1594000000000003e-05 2024-09-18T11:31:19.804 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.804 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:19.804 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.804 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2525 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865577,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:19.804 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173466+0000", 2024-09-18T11:31:19.805 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 10.552105797999999, 2024-09-18T11:31:19.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075147870000000002, 2024-09-18T11:31:19.805 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:19.805 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:19.805 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:19.805 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:19.805 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.805 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2525 2024-09-18T11:31:19.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.805 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:19.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:19.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173466+0000", 2024-09-18T11:31:19.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:19.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173466+0000", 2024-09-18T11:31:19.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:19.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173465+0000", 2024-09-18T11:31:19.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:19.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:19.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173469+0000", 2024-09-18T11:31:19.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7299999999999999e-06 2024-09-18T11:31:19.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:19.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173476+0000", 2024-09-18T11:31:19.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7750000000000004e-06 2024-09-18T11:31:19.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173480+0000", 2024-09-18T11:31:19.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1810000000000003e-06 2024-09-18T11:31:19.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:19.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.180013+0000", 2024-09-18T11:31:19.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065332419999999999 2024-09-18T11:31:19.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:19.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.180115+0000", 2024-09-18T11:31:19.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102111 2024-09-18T11:31:19.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:19.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.180980+0000", 2024-09-18T11:31:19.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00086527100000000003 2024-09-18T11:31:19.809 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.809 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:19.809 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.809 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2647 4.e 4:78eefc4b:::smithi11939506-7:head [append 0~0] snapc 30=[30,2f,2b,2a,25,24,1c] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:19.809 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:19.809 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.7157278099999997, 2024-09-18T11:31:19.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096752739999999993, 2024-09-18T11:31:19.809 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:19.809 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.809 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:19.809 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:19.810 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.810 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2647 2024-09-18T11:31:19.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.810 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:19.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.810 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:19.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:19.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.810 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:19.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:19.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:19.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009843+0000", 2024-09-18T11:31:19.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:19.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:19.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009846+0000", 2024-09-18T11:31:19.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3870000000000002e-06 2024-09-18T11:31:19.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:19.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009853+0000", 2024-09-18T11:31:19.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.238e-06 2024-09-18T11:31:19.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.812 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009860+0000", 2024-09-18T11:31:19.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1999999999999997e-06 2024-09-18T11:31:19.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.812 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:19.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.013798+0000", 2024-09-18T11:31:19.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003938441 2024-09-18T11:31:19.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:19.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.013889+0000", 2024-09-18T11:31:19.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0332000000000002e-05 2024-09-18T11:31:19.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 6", 2024-09-18T11:31:19.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.014066+0000", 2024-09-18T11:31:19.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017728700000000001 2024-09-18T11:31:19.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:19.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015813+0000", 2024-09-18T11:31:19.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001746804 2024-09-18T11:31:19.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:19.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019419+0000", 2024-09-18T11:31:19.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036058399999999999 2024-09-18T11:31:19.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:19.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019478+0000", 2024-09-18T11:31:19.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8860999999999998e-05 2024-09-18T11:31:19.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:19.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019519+0000", 2024-09-18T11:31:19.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1208000000000003e-05 2024-09-18T11:31:19.815 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.815 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:19.815 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.815 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:209 4.5 e91/22 4:af4ba75a:::smithi11939506-37:19 v 91'165, mlcod=91'161)", 2024-09-18T11:31:19.815 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:19.815 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.7076510819999999, 2024-09-18T11:31:19.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0091546930000000002, 2024-09-18T11:31:19.815 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:19.815 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.815 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:19.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:19.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:19.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:19.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:19.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:19.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:19.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:19.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:19.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017925+0000", 2024-09-18T11:31:19.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0819999999999998e-06 2024-09-18T11:31:19.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:19.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017935+0000", 2024-09-18T11:31:19.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0597999999999999e-05 2024-09-18T11:31:19.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017941+0000", 2024-09-18T11:31:19.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2920000000000003e-06 2024-09-18T11:31:19.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:19.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017978+0000", 2024-09-18T11:31:19.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7667999999999999e-05 2024-09-18T11:31:19.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:19.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018018+0000", 2024-09-18T11:31:19.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9700999999999998e-05 2024-09-18T11:31:19.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:19.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.027039+0000", 2024-09-18T11:31:19.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090214079999999999 2024-09-18T11:31:19.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:19.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.027075+0000", 2024-09-18T11:31:19.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5605000000000002e-05 2024-09-18T11:31:19.819 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.819 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:19.819 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.820 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:570 4.b e91/67 4:ddcb7ff4:::smithi11939506-9:18 v 91'251, mlcod=90'245)", 2024-09-18T11:31:19.820 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:19.820 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.7076153099999996, 2024-09-18T11:31:19.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089212549999999995, 2024-09-18T11:31:19.820 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:19.820 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.820 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:19.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:19.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:19.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:19.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:19.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:19.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:19.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:19.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:19.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017960+0000", 2024-09-18T11:31:19.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.777e-06 2024-09-18T11:31:19.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.822 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:19.822 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017966+0000", 2024-09-18T11:31:19.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4389999999999997e-06 2024-09-18T11:31:19.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.822 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.822 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017970+0000", 2024-09-18T11:31:19.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5549999999999998e-06 2024-09-18T11:31:19.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.822 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:19.822 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018032+0000", 2024-09-18T11:31:19.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2976000000000001e-05 2024-09-18T11:31:19.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:19.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018053+0000", 2024-09-18T11:31:19.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0322e-05 2024-09-18T11:31:19.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:19.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026844+0000", 2024-09-18T11:31:19.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087915790000000008 2024-09-18T11:31:19.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:19.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026877+0000", 2024-09-18T11:31:19.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2916000000000001e-05 2024-09-18T11:31:19.824 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.824 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:19.824 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.824 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.5.13:15 3.1 3:bf804683:::e58794d0a579007c3bd048992c5646e06b9775f9667464eee6a2a0508e23a0a6:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:19.824 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:19.824 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.7074604630000003, 2024-09-18T11:31:19.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008816908, 2024-09-18T11:31:19.824 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:19.824 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.824 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:19.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:19.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:19.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:19.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:19.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:19.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018110+0000", 2024-09-18T11:31:19.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:19.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:19.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018114+0000", 2024-09-18T11:31:19.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0339999999999998e-06 2024-09-18T11:31:19.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:19.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018121+0000", 2024-09-18T11:31:19.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0199999999999997e-06 2024-09-18T11:31:19.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018125+0000", 2024-09-18T11:31:19.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0409999999999999e-06 2024-09-18T11:31:19.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:19.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018793+0000", 2024-09-18T11:31:19.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00066779199999999995 2024-09-18T11:31:19.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:19.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018868+0000", 2024-09-18T11:31:19.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4814999999999994e-05 2024-09-18T11:31:19.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:19.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019008+0000", 2024-09-18T11:31:19.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014031800000000001 2024-09-18T11:31:19.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:19.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.023494+0000", 2024-09-18T11:31:19.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0044861720000000001 2024-09-18T11:31:19.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:19.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026796+0000", 2024-09-18T11:31:19.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0033010969999999998 2024-09-18T11:31:19.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:19.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026891+0000", 2024-09-18T11:31:19.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5828000000000005e-05 2024-09-18T11:31:19.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:19.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026928+0000", 2024-09-18T11:31:19.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6393999999999997e-05 2024-09-18T11:31:19.829 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.829 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:19.829 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.829 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2793 4.1 4:87078dfe:::smithi11939506-14:head [write 534522~634550] snapc 30=[30,2f,2b,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:31:19.830 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:19.830 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.501887835, 2024-09-18T11:31:19.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085134289999999994, 2024-09-18T11:31:19.830 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:19.830 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:19.830 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:19.830 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:19.830 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:19.830 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2793 2024-09-18T11:31:19.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.830 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:19.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:19.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:19.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:19.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:19.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:19.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:19.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223681+0000", 2024-09-18T11:31:19.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:19.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:19.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224251+0000", 2024-09-18T11:31:19.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056940699999999996 2024-09-18T11:31:19.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:19.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224267+0000", 2024-09-18T11:31:19.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6829000000000001e-05 2024-09-18T11:31:19.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:19.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224292+0000", 2024-09-18T11:31:19.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4198000000000001e-05 2024-09-18T11:31:19.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:19.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224352+0000", 2024-09-18T11:31:19.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0608e-05 2024-09-18T11:31:19.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:19.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224456+0000", 2024-09-18T11:31:19.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010412300000000001 2024-09-18T11:31:19.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:19.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224662+0000", 2024-09-18T11:31:19.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020561399999999999 2024-09-18T11:31:19.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:19.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231800+0000", 2024-09-18T11:31:19.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071384530000000003 2024-09-18T11:31:19.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:19.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232126+0000", 2024-09-18T11:31:19.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00032601400000000001 2024-09-18T11:31:19.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:19.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232158+0000", 2024-09-18T11:31:19.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1472e-05 2024-09-18T11:31:19.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:19.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:19.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:19.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232197+0000", 2024-09-18T11:31:19.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9011999999999999e-05 2024-09-18T11:31:19.835 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.835 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:19.835 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.835 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:19.835 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:19.835 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:19.836 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:31:19.838 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:31:19.838 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:19.835+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:19.937 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:19.940 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:19.936+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:19.952 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:19.952 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:19.967 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:19.967 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:19.976 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:19.976 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:19.977 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:19.977 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:19.982 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:19.982 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:19.984 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:31:19.997 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:19.997 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:20.012 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:20.012 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:20.027 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:20.027 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:20.038 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:20.037+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:20.042 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:20.042 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:20.045 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:20.046 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:20.083 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:20.083 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:20.091 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:31:20.139 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:20.137+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:20.189 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:20.189 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:20.197 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:31:20.239 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:20.238+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:20.337 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:20.337 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:20.340 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:20.338+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:20.345 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:31:20.441 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:20.440+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:20.483 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:20.483 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:20.488 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:20.487+0000 7fb6950ff640 -1 osd.2 92 handle_osd_map injecting map crc failure 2024-09-18T11:31:20.493 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:31:20.494 INFO:tasks.rados.rados.0.smithi119.stdout:1317: done (0 left) 2024-09-18T11:31:20.494 INFO:tasks.rados.rados.0.smithi119.stdout:1318: read oid 44 snap -1 2024-09-18T11:31:20.494 INFO:tasks.rados.rados.0.smithi119.stdout:1318: expect (ObjNum 245 snap 28 seq_num 245) 2024-09-18T11:31:20.494 INFO:tasks.rados.rados.0.smithi119.stdout:1319: read oid 30 snap -1 2024-09-18T11:31:20.495 INFO:tasks.rados.rados.0.smithi119.stdout:1319: expect deleted 2024-09-18T11:31:20.495 INFO:tasks.rados.rados.0.smithi119.stdout:1320: snap_remove snap 24 2024-09-18T11:31:20.542 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:20.539+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:20.593 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:20.593 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:20.601 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:31:20.643 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:20.640+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:20.740 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:20.740 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:20.743 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:20.741+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:20.749 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:31:20.845 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:20.844+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:20.888 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:20.888 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:20.896 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:31:20.945 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:20.944+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:20.994 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:20.994 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:21.002 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:31:21.047 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:21.044+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:21.112 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:21.112 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:21.121 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:31:21.165 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:21.164+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:21.221 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:21.221 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:21.229 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:31:21.270 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:21.269+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:21.349 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:21.347+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:21.367 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:21.367 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:21.376 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:31:21.450 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:21.448+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:21.474 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:21.475 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:21.483 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:31:21.497 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T11:31:21.498 INFO:tasks.rados.rados.0.smithi119.stdout:1318: expect (ObjNum 245 snap 28 seq_num 245) 2024-09-18T11:31:21.550 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:21.549+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:21.600 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:21.601 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:21.610 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:31:21.651 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:21.649+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:21.678 INFO:tasks.rados.rados.0.smithi119.stdout:1318: done (2 left) 2024-09-18T11:31:21.678 INFO:tasks.rados.rados.0.smithi119.stdout:1319: done (1 left) 2024-09-18T11:31:21.678 INFO:tasks.rados.rados.0.smithi119.stdout:1320: done (0 left) 2024-09-18T11:31:21.678 INFO:tasks.rados.rados.0.smithi119.stdout:1321: write oid 11 current snap is 30 2024-09-18T11:31:21.678 INFO:tasks.rados.rados.0.smithi119.stdout:1321: seq_num 249 ranges {584943=460297,1587781=597162} 2024-09-18T11:31:21.683 INFO:tasks.rados.rados.0.smithi119.stdout:1321: writing smithi11939506-11 from 584943 to 1045240 tid 1 2024-09-18T11:31:21.690 INFO:tasks.rados.rados.0.smithi119.stdout:1321: writing smithi11939506-11 from 1587781 to 2184943 tid 2 2024-09-18T11:31:21.690 INFO:tasks.rados.rados.0.smithi119.stdout:1322: read oid 1 snap 21 2024-09-18T11:31:21.690 INFO:tasks.rados.rados.0.smithi119.stdout:1322: expect (ObjNum 186 snap 20 seq_num 186) 2024-09-18T11:31:21.691 INFO:tasks.rados.rados.0.smithi119.stdout:1321: finishing write tid 1 to smithi11939506-11 2024-09-18T11:31:21.691 INFO:tasks.rados.rados.0.smithi119.stdout:1323: read oid 20 snap -1 2024-09-18T11:31:21.691 INFO:tasks.rados.rados.0.smithi119.stdout:1323: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:31:21.691 INFO:tasks.rados.rados.0.smithi119.stdout:1324: read oid 26 snap -1 2024-09-18T11:31:21.691 INFO:tasks.rados.rados.0.smithi119.stdout:1324: expect (ObjNum 178 snap 20 seq_num 178) 2024-09-18T11:31:21.691 INFO:tasks.rados.rados.0.smithi119.stdout:1325: write (excl) oid 39 current snap is 30 2024-09-18T11:31:21.691 INFO:tasks.rados.rados.0.smithi119.stdout:1325: seq_num 250 ranges {601750=535546,1929910=271840} 2024-09-18T11:31:21.697 INFO:tasks.rados.rados.0.smithi119.stdout:1325: writing smithi11939506-39 from 601750 to 1137296 tid 1 2024-09-18T11:31:21.703 INFO:tasks.rados.rados.0.smithi119.stdout:1325: writing smithi11939506-39 from 1929910 to 2201750 tid 2 2024-09-18T11:31:21.703 INFO:tasks.rados.rados.0.smithi119.stdout:1326: read oid 28 snap -1 2024-09-18T11:31:21.704 INFO:tasks.rados.rados.0.smithi119.stdout:1326: expect deleted 2024-09-18T11:31:21.704 INFO:tasks.rados.rados.0.smithi119.stdout:1327: write (excl) oid 42 current snap is 30 2024-09-18T11:31:21.704 INFO:tasks.rados.rados.0.smithi119.stdout:1327: seq_num 251 ranges {618557=727148,2104745=113812} 2024-09-18T11:31:21.711 INFO:tasks.rados.rados.0.smithi119.stdout:1327: writing smithi11939506-42 from 618557 to 1345705 tid 1 2024-09-18T11:31:21.715 INFO:tasks.rados.rados.0.smithi119.stdout:1327: writing smithi11939506-42 from 2104745 to 2218557 tid 2 2024-09-18T11:31:21.716 INFO:tasks.rados.rados.0.smithi119.stdout:1328: write (excl) oid 44 current snap is 30 2024-09-18T11:31:21.716 INFO:tasks.rados.rados.0.smithi119.stdout:1328: seq_num 252 ranges {635364=402397,1763227=472137} 2024-09-18T11:31:21.721 INFO:tasks.rados.rados.0.smithi119.stdout:1328: writing smithi11939506-44 from 635364 to 1037761 tid 1 2024-09-18T11:31:21.728 INFO:tasks.rados.rados.0.smithi119.stdout:1328: writing smithi11939506-44 from 1763227 to 2235364 tid 2 2024-09-18T11:31:21.728 INFO:tasks.rados.rados.0.smithi119.stdout:1329: set_chunk oid 27 target oid 2024-09-18T11:31:21.734 INFO:tasks.rados.rados.0.smithi119.stdout:1329: set_chunk oid 27 offset: 226816 length: 564224 target oid 45488605274bee7141d9b545aeb674a57d953993cd9ffe528f0a760ebb51086f offset: 0 snap -1 2024-09-18T11:31:21.734 INFO:tasks.rados.rados.0.smithi119.stdout:1330: write (excl) oid 21 current snap is 30 2024-09-18T11:31:21.734 INFO:tasks.rados.rados.0.smithi119.stdout:1321: finishing write tid 2 to smithi11939506-11 2024-09-18T11:31:21.734 INFO:tasks.rados.rados.0.smithi119.stdout:1321: finishing write tid 3 to smithi11939506-11 2024-09-18T11:31:21.734 INFO:tasks.rados.rados.0.smithi119.stdout:1321: finishing write tid 4 to smithi11939506-11 2024-09-18T11:31:21.734 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 11 v 177 (ObjNum 249 snap 30 seq_num 249) dirty exists 2024-09-18T11:31:21.734 INFO:tasks.rados.rados.0.smithi119.stdout:1321: left oid 11 (ObjNum 249 snap 30 seq_num 249) 2024-09-18T11:31:21.734 INFO:tasks.rados.rados.0.smithi119.stdout:1330: seq_num 253 ranges {652171=477646,1821709=430462} 2024-09-18T11:31:21.739 INFO:tasks.rados.rados.0.smithi119.stdout:1330: writing smithi11939506-21 from 652171 to 1129817 tid 1 2024-09-18T11:31:21.746 INFO:tasks.rados.rados.0.smithi119.stdout:1330: writing smithi11939506-21 from 1821709 to 2252171 tid 2 2024-09-18T11:31:21.746 INFO:tasks.rados.rados.0.smithi119.stdout:1321: done (9 left) 2024-09-18T11:31:21.746 INFO:tasks.rados.rados.0.smithi119.stdout:1331: write (excl) oid 45 current snap is 30 2024-09-18T11:31:21.746 INFO:tasks.rados.rados.0.smithi119.stdout:1331: seq_num 254 ranges {668978=552895,1763838=505140} 2024-09-18T11:31:21.751 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:21.750+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:21.753 INFO:tasks.rados.rados.0.smithi119.stdout:1331: writing smithi11939506-45 from 668978 to 1221873 tid 1 2024-09-18T11:31:21.759 INFO:tasks.rados.rados.0.smithi119.stdout:1331: writing smithi11939506-45 from 1763838 to 2268978 tid 2 2024-09-18T11:31:21.759 INFO:tasks.rados.rados.0.smithi119.stdout:1332: write oid 12 current snap is 30 2024-09-18T11:31:21.759 INFO:tasks.rados.rados.0.smithi119.stdout:1322: expect (ObjNum 186 snap 20 seq_num 186) 2024-09-18T11:31:21.760 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:21.761 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:21.768 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:31:21.852 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:21.851+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:21.869 INFO:tasks.rados.rados.0.smithi119.stdout:1323: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:31:21.908 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:21.908 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:21.917 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:31:21.953 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:21.951+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:22.047 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:31:22.053 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:22.052+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:22.055 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:22.056 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:22.065 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:31:22.091 INFO:tasks.rados.rados.0.smithi119.stdout:1324: expect (ObjNum 178 snap 20 seq_num 178) 2024-09-18T11:31:22.148 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:22.154 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:22.152+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:22.177 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:22.177 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:22.194 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:22.194 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:22.204 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:22.204 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:22.209 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:22.209 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:22.213 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:31:22.225 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:22.225 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:22.240 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:22.241 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:22.255 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:22.254+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:22.255 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:22.256 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:22.272 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:22.272 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:22.275 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:22.275 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:22.315 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:22.315 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:22.315 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:22.315 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:22.320 INFO:tasks.rados.rados.0.smithi119.stdout:1325: finishing write tid 1 to smithi11939506-39 2024-09-18T11:31:22.320 INFO:tasks.rados.rados.0.smithi119.stdout:1325: finishing write tid 2 to smithi11939506-39 2024-09-18T11:31:22.320 INFO:tasks.rados.rados.0.smithi119.stdout:1325: finishing write tid 3 to smithi11939506-39 2024-09-18T11:31:22.320 INFO:tasks.rados.rados.0.smithi119.stdout:1325: finishing write tid 4 to smithi11939506-39 2024-09-18T11:31:22.320 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 39 v 203 (ObjNum 250 snap 30 seq_num 250) dirty exists 2024-09-18T11:31:22.320 INFO:tasks.rados.rados.0.smithi119.stdout:1325: left oid 39 (ObjNum 250 snap 30 seq_num 250) 2024-09-18T11:31:22.320 INFO:tasks.rados.rados.0.smithi119.stdout:1327: finishing write tid 1 to smithi11939506-42 2024-09-18T11:31:22.320 INFO:tasks.rados.rados.0.smithi119.stdout:1327: finishing write tid 2 to smithi11939506-42 2024-09-18T11:31:22.321 INFO:tasks.rados.rados.0.smithi119.stdout:1332: seq_num 255 ranges {685785=628144,1822320=463465} 2024-09-18T11:31:22.323 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:31:22.326 INFO:tasks.rados.rados.0.smithi119.stdout:1332: writing smithi11939506-12 from 685785 to 1313929 tid 1 2024-09-18T11:31:22.333 INFO:tasks.rados.rados.0.smithi119.stdout:1332: writing smithi11939506-12 from 1822320 to 2285785 tid 2 2024-09-18T11:31:22.333 INFO:tasks.rados.rados.0.smithi119.stdout:1322: done (10 left) 2024-09-18T11:31:22.333 INFO:tasks.rados.rados.0.smithi119.stdout:1323: done (9 left) 2024-09-18T11:31:22.333 INFO:tasks.rados.rados.0.smithi119.stdout:1324: done (8 left) 2024-09-18T11:31:22.333 INFO:tasks.rados.rados.0.smithi119.stdout:1325: done (7 left) 2024-09-18T11:31:22.333 INFO:tasks.rados.rados.0.smithi119.stdout:1326: done (6 left) 2024-09-18T11:31:22.333 INFO:tasks.rados.rados.0.smithi119.stdout:1333: write oid 26 current snap is 30 2024-09-18T11:31:22.333 INFO:tasks.rados.rados.0.smithi119.stdout:1333: seq_num 256 ranges {702592=703393,1880802=421790} 2024-09-18T11:31:22.340 INFO:tasks.rados.rados.0.smithi119.stdout:1333: writing smithi11939506-26 from 702592 to 1405985 tid 1 2024-09-18T11:31:22.347 INFO:tasks.rados.rados.0.smithi119.stdout:1333: writing smithi11939506-26 from 1880802 to 2302592 tid 2 2024-09-18T11:31:22.347 INFO:tasks.rados.rados.0.smithi119.stdout:1334: read oid 43 snap 27 2024-09-18T11:31:22.347 INFO:tasks.rados.rados.0.smithi119.stdout:1334: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:31:22.347 INFO:tasks.rados.rados.0.smithi119.stdout:1335: tier_promote oid 18 2024-09-18T11:31:22.347 INFO:tasks.rados.rados.0.smithi119.stdout:1327: finishing write tid 3 to smithi11939506-42 2024-09-18T11:31:22.347 INFO:tasks.rados.rados.0.smithi119.stdout:1327: finishing write tid 4 to smithi11939506-42 2024-09-18T11:31:22.347 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 178 (ObjNum 251 snap 30 seq_num 251) dirty exists 2024-09-18T11:31:22.347 INFO:tasks.rados.rados.0.smithi119.stdout:1327: left oid 42 (ObjNum 251 snap 30 seq_num 251) 2024-09-18T11:31:22.347 INFO:tasks.rados.rados.0.smithi119.stdout:1328: finishing write tid 1 to smithi11939506-44 2024-09-18T11:31:22.347 INFO:tasks.rados.rados.0.smithi119.stdout:1327: done (8 left) 2024-09-18T11:31:22.347 INFO:tasks.rados.rados.0.smithi119.stdout:1336: snap_create 2024-09-18T11:31:22.347 INFO:tasks.rados.rados.0.smithi119.stdout:1328: finishing write tid 2 to smithi11939506-44 2024-09-18T11:31:22.348 INFO:tasks.rados.rados.0.smithi119.stdout:1328: finishing write tid 3 to smithi11939506-44 2024-09-18T11:31:22.348 INFO:tasks.rados.rados.0.smithi119.stdout:1328: finishing write tid 4 to smithi11939506-44 2024-09-18T11:31:22.348 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 44 v 119 (ObjNum 252 snap 30 seq_num 252) dirty exists 2024-09-18T11:31:22.348 INFO:tasks.rados.rados.0.smithi119.stdout:1328: left oid 44 (ObjNum 252 snap 30 seq_num 252) 2024-09-18T11:31:22.348 INFO:tasks.rados.rados.0.smithi119.stdout:1329: finishing set_chunk to oid 27 2024-09-18T11:31:22.348 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 172 (ObjNum 244 snap 28 seq_num 244) dirty exists 2024-09-18T11:31:22.348 INFO:tasks.rados.rados.0.smithi119.stdout:1330: finishing write tid 1 to smithi11939506-21 2024-09-18T11:31:22.348 INFO:tasks.rados.rados.0.smithi119.stdout:1330: finishing write tid 2 to smithi11939506-21 2024-09-18T11:31:22.348 INFO:tasks.rados.rados.0.smithi119.stdout:1330: finishing write tid 3 to smithi11939506-21 2024-09-18T11:31:22.348 INFO:tasks.rados.rados.0.smithi119.stdout:1330: finishing write tid 4 to smithi11939506-21 2024-09-18T11:31:22.348 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 199 (ObjNum 253 snap 30 seq_num 253) dirty exists 2024-09-18T11:31:22.348 INFO:tasks.rados.rados.0.smithi119.stdout:1330: left oid 21 (ObjNum 253 snap 30 seq_num 253) 2024-09-18T11:31:22.348 INFO:tasks.rados.rados.0.smithi119.stdout:1331: finishing write tid 1 to smithi11939506-45 2024-09-18T11:31:22.349 INFO:tasks.rados.rados.0.smithi119.stdout:1331: finishing write tid 2 to smithi11939506-45 2024-09-18T11:31:22.349 INFO:tasks.rados.rados.0.smithi119.stdout:1331: finishing write tid 3 to smithi11939506-45 2024-09-18T11:31:22.349 INFO:tasks.rados.rados.0.smithi119.stdout:1331: finishing write tid 4 to smithi11939506-45 2024-09-18T11:31:22.349 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 64 (ObjNum 254 snap 30 seq_num 254) dirty exists 2024-09-18T11:31:22.349 INFO:tasks.rados.rados.0.smithi119.stdout:1331: left oid 45 (ObjNum 254 snap 30 seq_num 254) 2024-09-18T11:31:22.349 INFO:tasks.rados.rados.0.smithi119.stdout:1332: finishing write tid 1 to smithi11939506-12 2024-09-18T11:31:22.349 INFO:tasks.rados.rados.0.smithi119.stdout:1332: finishing write tid 2 to smithi11939506-12 2024-09-18T11:31:22.349 INFO:tasks.rados.rados.0.smithi119.stdout:1332: finishing write tid 3 to smithi11939506-12 2024-09-18T11:31:22.349 INFO:tasks.rados.rados.0.smithi119.stdout:1332: finishing write tid 4 to smithi11939506-12 2024-09-18T11:31:22.349 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 12 v 68 (ObjNum 255 snap 30 seq_num 255) dirty exists 2024-09-18T11:31:22.349 INFO:tasks.rados.rados.0.smithi119.stdout:1332: left oid 12 (ObjNum 255 snap 30 seq_num 255) 2024-09-18T11:31:22.349 INFO:tasks.rados.rados.0.smithi119.stdout:1333: finishing write tid 1 to smithi11939506-26 2024-09-18T11:31:22.349 INFO:tasks.rados.rados.0.smithi119.stdout:1335: got (0) Success 2024-09-18T11:31:22.350 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 113 (ObjNum 205 snap 23 seq_num 205) dirty exists 2024-09-18T11:31:22.353 INFO:tasks.rados.rados.0.smithi119.stdout:1334: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:31:22.356 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:22.355+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:22.425 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:22.425 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:22.425 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:22.425 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:22.425 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:22.433 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:31:22.441 INFO:tasks.rados.rados.0.smithi119.stdout:1333: finishing write tid 2 to smithi11939506-26 2024-09-18T11:31:22.441 INFO:tasks.rados.rados.0.smithi119.stdout:1333: finishing write tid 3 to smithi11939506-26 2024-09-18T11:31:22.441 INFO:tasks.rados.rados.0.smithi119.stdout:1333: finishing write tid 4 to smithi11939506-26 2024-09-18T11:31:22.441 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 207 (ObjNum 256 snap 30 seq_num 256) dirty exists 2024-09-18T11:31:22.441 INFO:tasks.rados.rados.0.smithi119.stdout:1333: left oid 26 (ObjNum 256 snap 30 seq_num 256) 2024-09-18T11:31:22.477 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:22.475+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:22.506 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is set 2024-09-18T11:31:22.507 INFO:tasks.rados.rados.0.smithi119.stdout:1328: done (8 left) 2024-09-18T11:31:22.507 INFO:tasks.rados.rados.0.smithi119.stdout:1329: done (7 left) 2024-09-18T11:31:22.507 INFO:tasks.rados.rados.0.smithi119.stdout:1330: done (6 left) 2024-09-18T11:31:22.507 INFO:tasks.rados.rados.0.smithi119.stdout:1331: done (5 left) 2024-09-18T11:31:22.508 INFO:tasks.rados.rados.0.smithi119.stdout:1332: done (4 left) 2024-09-18T11:31:22.508 INFO:tasks.rados.rados.0.smithi119.stdout:1333: done (3 left) 2024-09-18T11:31:22.508 INFO:tasks.rados.rados.0.smithi119.stdout:1334: done (2 left) 2024-09-18T11:31:22.508 INFO:tasks.rados.rados.0.smithi119.stdout:1335: done (1 left) 2024-09-18T11:31:22.508 INFO:tasks.rados.rados.0.smithi119.stdout:1336: done (0 left) 2024-09-18T11:31:22.508 INFO:tasks.rados.rados.0.smithi119.stdout:1337: read oid 41 snap 30 2024-09-18T11:31:22.508 INFO:tasks.rados.rados.0.smithi119.stdout:1337: expect (ObjNum 237 snap 28 seq_num 237) 2024-09-18T11:31:22.508 INFO:tasks.rados.rados.0.smithi119.stdout:1338: rollback oid 5 current snap is 31 2024-09-18T11:31:22.508 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 5 to 23 2024-09-18T11:31:22.508 INFO:tasks.rados.rados.0.smithi119.stdout:1339: read oid 33 snap 26 2024-09-18T11:31:22.509 INFO:tasks.rados.rados.0.smithi119.stdout:1339: expect (ObjNum 219 snap 26 seq_num 219) 2024-09-18T11:31:22.509 INFO:tasks.rados.rados.0.smithi119.stdout:1340: set_chunk oid 7 target oid 2024-09-18T11:31:22.513 INFO:tasks.rados.rados.0.smithi119.stdout:1340: set_chunk oid 7 offset: 159232 length: 528384 target oid 12eb21d28e22e9cc131b6c4765d3368b558442ab6a939ebbfbe2c5ad2f7df0c0 offset: 0 snap -1 2024-09-18T11:31:22.513 INFO:tasks.rados.rados.0.smithi119.stdout:1341: write oid 31 current snap is 31 2024-09-18T11:31:22.513 INFO:tasks.rados.rados.0.smithi119.stdout:1338: finishing rollback tid 1 to smithi11939506-5 2024-09-18T11:31:22.513 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 0 (ObjNum 1254071520 snap 32633 seq_num 304) dirty dne 2024-09-18T11:31:22.513 INFO:tasks.rados.rados.0.smithi119.stdout:1341: seq_num 257 ranges {719399=778642,1939284=380115} 2024-09-18T11:31:22.522 INFO:tasks.rados.rados.0.smithi119.stdout:1341: writing smithi11939506-31 from 719399 to 1498041 tid 1 2024-09-18T11:31:22.527 INFO:tasks.rados.rados.0.smithi119.stdout:1341: writing smithi11939506-31 from 1939284 to 2319399 tid 2 2024-09-18T11:31:22.527 INFO:tasks.rados.rados.0.smithi119.stdout:1338: done (4 left) 2024-09-18T11:31:22.527 INFO:tasks.rados.rados.0.smithi119.stdout:1342: set_chunk oid 37 target oid 2024-09-18T11:31:22.539 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:22.539 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:22.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:22.539 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:22.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.539 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1530 4.d 4:b81644f2:::smithi11939506-34:head [write 467261~717275] snapc 14=[14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:31:22.540 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:31:22.540 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 48.146371664999997, 2024-09-18T11:31:22.540 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075660950000000001, 2024-09-18T11:31:22.540 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:22.540 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:22.540 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:22.540 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:22.540 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:22.540 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1530 2024-09-18T11:31:22.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.540 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:22.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.540 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:22.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:31:22.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:22.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390685+0000", 2024-09-18T11:31:22.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:22.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.390682+0000", 2024-09-18T11:31:22.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:22.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.542 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:22.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391554+0000", 2024-09-18T11:31:22.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00087220600000000004 2024-09-18T11:31:22.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.542 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:22.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391570+0000", 2024-09-18T11:31:22.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6002999999999999e-05 2024-09-18T11:31:22.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.542 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:22.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391575+0000", 2024-09-18T11:31:22.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2549999999999997e-06 2024-09-18T11:31:22.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:22.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391631+0000", 2024-09-18T11:31:22.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5671000000000003e-05 2024-09-18T11:31:22.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:22.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391751+0000", 2024-09-18T11:31:22.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000119821 2024-09-18T11:31:22.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:22.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.391974+0000", 2024-09-18T11:31:22.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022309900000000001 2024-09-18T11:31:22.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:22.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398136+0000", 2024-09-18T11:31:22.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006161573 2024-09-18T11:31:22.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:22.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398184+0000", 2024-09-18T11:31:22.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8458000000000003e-05 2024-09-18T11:31:22.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.545 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:22.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398219+0000", 2024-09-18T11:31:22.545 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4573999999999998e-05 2024-09-18T11:31:22.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.545 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:22.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.398251+0000", 2024-09-18T11:31:22.545 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2085000000000002e-05 2024-09-18T11:31:22.545 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.545 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:22.545 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.545 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:31:22.545 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:22.546 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 48.120969340999999, 2024-09-18T11:31:22.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:31:22.546 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:22.546 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:22.546 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:22.546 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:22.546 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:22.546 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:31:22.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.546 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:22.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.546 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:22.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:22.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.547 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:22.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:22.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.547 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:22.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:31:22.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:22.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:22.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:31:22.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:31:22.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:22.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:31:22.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:31:22.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:22.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:31:22.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:31:22.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:22.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:31:22.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:31:22.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:22.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:31:22.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:31:22.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:22.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:31:22.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:31:22.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:22.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:31:22.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:31:22.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:22.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:31:22.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:31:22.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:22.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:31:22.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:31:22.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.551 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:22.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:31:22.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:31:22.551 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.551 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:22.551 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.551 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1628 3.3 3:f80aa31b:::77cdbf3abbada57d0636b2198046a030673818705cbf13bb77b30c3e68b24345:head [write 0~582144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:22.551 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:22.551 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.226079341000002, 2024-09-18T11:31:22.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081714639999999998, 2024-09-18T11:31:22.552 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:22.552 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:22.552 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:22.552 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:22.552 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:22.552 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1628 2024-09-18T11:31:22.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.552 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:22.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.552 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:22.552 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:22.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:22.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:22.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:22.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310975+0000", 2024-09-18T11:31:22.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:22.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.554 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:22.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312086+0000", 2024-09-18T11:31:22.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001111288 2024-09-18T11:31:22.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.554 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:22.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312104+0000", 2024-09-18T11:31:22.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7427000000000001e-05 2024-09-18T11:31:22.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.554 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:22.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312110+0000", 2024-09-18T11:31:22.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5320000000000003e-06 2024-09-18T11:31:22.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:22.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312179+0000", 2024-09-18T11:31:22.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9196000000000006e-05 2024-09-18T11:31:22.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:22.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312375+0000", 2024-09-18T11:31:22.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00019550899999999999 2024-09-18T11:31:22.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:22.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312558+0000", 2024-09-18T11:31:22.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000182566 2024-09-18T11:31:22.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.556 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:22.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.314286+0000", 2024-09-18T11:31:22.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00172831 2024-09-18T11:31:22.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.556 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:22.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319079+0000", 2024-09-18T11:31:22.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047929340000000004 2024-09-18T11:31:22.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.556 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:22.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319116+0000", 2024-09-18T11:31:22.557 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6993999999999999e-05 2024-09-18T11:31:22.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.557 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:22.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319148+0000", 2024-09-18T11:31:22.557 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2539999999999997e-05 2024-09-18T11:31:22.557 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.557 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:22.557 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.558 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1663 4.1 4:87078dfe:::smithi11939506-14:head [write 1916458~318873] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:22.558 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:22.558 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.173132778000003, 2024-09-18T11:31:22.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079505749999999997, 2024-09-18T11:31:22.558 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:22.558 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:22.558 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:22.558 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:22.558 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:22.558 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1663 2024-09-18T11:31:22.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.558 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:22.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.558 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:22.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:22.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:22.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:22.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:22.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363922+0000", 2024-09-18T11:31:22.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:22.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:22.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364211+0000", 2024-09-18T11:31:22.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000288385 2024-09-18T11:31:22.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:22.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364223+0000", 2024-09-18T11:31:22.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2582000000000001e-05 2024-09-18T11:31:22.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:22.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364228+0000", 2024-09-18T11:31:22.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0000000000000004e-06 2024-09-18T11:31:22.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.561 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:22.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364268+0000", 2024-09-18T11:31:22.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0235999999999999e-05 2024-09-18T11:31:22.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.561 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:22.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364341+0000", 2024-09-18T11:31:22.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2266000000000002e-05 2024-09-18T11:31:22.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.562 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:22.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364444+0000", 2024-09-18T11:31:22.562 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103632 2024-09-18T11:31:22.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.562 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:22.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369461+0000", 2024-09-18T11:31:22.562 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005016848 2024-09-18T11:31:22.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.562 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:22.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371820+0000", 2024-09-18T11:31:22.563 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023586169999999999 2024-09-18T11:31:22.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.563 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.563 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:22.563 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371851+0000", 2024-09-18T11:31:22.563 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1297999999999997e-05 2024-09-18T11:31:22.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.563 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.563 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:22.563 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371874+0000", 2024-09-18T11:31:22.563 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2841e-05 2024-09-18T11:31:22.563 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.563 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:22.564 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.564 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.564 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1664 4.1 4:87078dfe:::smithi11939506-14:head [setxattr _header (57) in=7b,truncate 2235331] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:22.564 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:22.564 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.172819121000003, 2024-09-18T11:31:22.564 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010368235999999999, 2024-09-18T11:31:22.564 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:22.564 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:22.564 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:22.564 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:22.564 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:22.564 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1664 2024-09-18T11:31:22.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.565 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:22.565 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.565 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:22.565 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:22.565 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.565 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.565 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:22.565 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:22.565 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.565 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.565 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:22.566 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364236+0000", 2024-09-18T11:31:22.566 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:22.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.566 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:22.566 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364240+0000", 2024-09-18T11:31:22.566 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9829999999999998e-06 2024-09-18T11:31:22.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.566 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:22.566 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364247+0000", 2024-09-18T11:31:22.566 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2140000000000001e-06 2024-09-18T11:31:22.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.567 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:22.567 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364256+0000", 2024-09-18T11:31:22.567 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1509999999999993e-06 2024-09-18T11:31:22.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.567 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.567 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:22.567 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366529+0000", 2024-09-18T11:31:22.567 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022731409999999998 2024-09-18T11:31:22.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.567 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.567 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:22.567 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366621+0000", 2024-09-18T11:31:22.568 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2408e-05 2024-09-18T11:31:22.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.568 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.568 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:22.568 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366801+0000", 2024-09-18T11:31:22.568 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018002200000000001 2024-09-18T11:31:22.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.568 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.568 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:22.568 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372155+0000", 2024-09-18T11:31:22.568 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053531519999999999 2024-09-18T11:31:22.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.568 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.569 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:22.569 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374539+0000", 2024-09-18T11:31:22.569 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023842580000000002 2024-09-18T11:31:22.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.569 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:22.569 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374573+0000", 2024-09-18T11:31:22.569 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4205999999999997e-05 2024-09-18T11:31:22.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.569 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:22.569 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374605+0000", 2024-09-18T11:31:22.569 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2283999999999998e-05 2024-09-18T11:31:22.570 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.570 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:22.570 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.570 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.570 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1665 4.1 4:87078dfe:::smithi11939506-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:22.570 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:22.570 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.172794947, 2024-09-18T11:31:22.570 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010471897000000001, 2024-09-18T11:31:22.570 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:22.570 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:22.570 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:22.570 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:22.571 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:22.571 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1665 2024-09-18T11:31:22.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.571 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:22.571 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.571 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:22.571 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:22.571 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.571 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.571 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:22.571 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:22.571 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.572 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.572 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:22.572 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:22.572 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:22.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.572 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.572 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:22.572 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364263+0000", 2024-09-18T11:31:22.572 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6400000000000001e-06 2024-09-18T11:31:22.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.572 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.572 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:22.573 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364270+0000", 2024-09-18T11:31:22.573 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:31:22.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.573 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.573 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:22.573 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364274+0000", 2024-09-18T11:31:22.573 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.968e-06 2024-09-18T11:31:22.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.573 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.573 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:22.573 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367354+0000", 2024-09-18T11:31:22.573 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030803430000000001 2024-09-18T11:31:22.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.574 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.574 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:22.574 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367406+0000", 2024-09-18T11:31:22.574 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2330999999999998e-05 2024-09-18T11:31:22.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.574 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.574 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:22.574 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374625+0000", 2024-09-18T11:31:22.574 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072190689999999998 2024-09-18T11:31:22.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.574 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.574 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:22.574 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374673+0000", 2024-09-18T11:31:22.574 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7525999999999999e-05 2024-09-18T11:31:22.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.575 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.575 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:22.575 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374733+0000", 2024-09-18T11:31:22.575 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0359000000000003e-05 2024-09-18T11:31:22.575 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.575 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:22.575 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.575 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.575 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1947 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865577,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:22.575 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:22.575 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.197515237000001, 2024-09-18T11:31:22.575 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010043066999999999, 2024-09-18T11:31:22.575 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:22.575 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:22.576 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:22.576 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:22.576 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:22.576 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1947 2024-09-18T11:31:22.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.576 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:22.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.576 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:22.576 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:22.576 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.576 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:22.576 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:22.576 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.577 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:22.577 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339540+0000", 2024-09-18T11:31:22.577 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:22.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.577 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:22.577 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339545+0000", 2024-09-18T11:31:22.577 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6290000000000001e-06 2024-09-18T11:31:22.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.577 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:22.577 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339553+0000", 2024-09-18T11:31:22.578 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6299999999999998e-06 2024-09-18T11:31:22.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.578 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.578 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:22.578 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339557+0000", 2024-09-18T11:31:22.578 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3590000000000001e-06 2024-09-18T11:31:22.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.578 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.578 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:22.578 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345600+0000", 2024-09-18T11:31:22.578 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060434640000000001 2024-09-18T11:31:22.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.578 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.578 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:22.579 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345710+0000", 2024-09-18T11:31:22.579 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010924700000000001 2024-09-18T11:31:22.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.579 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.579 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:22.579 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349584+0000", 2024-09-18T11:31:22.579 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038743459999999999 2024-09-18T11:31:22.579 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.579 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:22.579 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.579 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.579 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1948 4.d 4:b5a463bf:::smithi11939506-20:head [sparse-read 0~2642181] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:22.579 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:22.579 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.1974947, 2024-09-18T11:31:22.580 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010701952000000001, 2024-09-18T11:31:22.580 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:22.580 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:22.580 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:22.580 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:22.580 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:22.580 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1948 2024-09-18T11:31:22.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.580 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:22.580 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.580 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:22.580 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:22.580 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.580 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.581 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:22.581 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:22.581 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.581 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.581 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:22.581 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:22.581 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:22.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.581 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.581 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:22.581 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339563+0000", 2024-09-18T11:31:22.581 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1950000000000002e-06 2024-09-18T11:31:22.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.581 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.582 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:22.582 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339569+0000", 2024-09-18T11:31:22.582 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5459999999999999e-06 2024-09-18T11:31:22.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.582 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.582 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:22.582 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339572+0000", 2024-09-18T11:31:22.582 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.337e-06 2024-09-18T11:31:22.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.582 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.582 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:22.582 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349693+0000", 2024-09-18T11:31:22.582 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010120987999999999 2024-09-18T11:31:22.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.583 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.583 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:22.583 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349816+0000", 2024-09-18T11:31:22.583 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012241199999999999 2024-09-18T11:31:22.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.583 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.583 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:22.583 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350263+0000", 2024-09-18T11:31:22.583 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00044787199999999998 2024-09-18T11:31:22.583 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.583 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:22.583 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.583 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.583 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:131 4.5 e69/22 4:aa575018:::smithi11939506-11:17 v 69'97, mlcod=65'92)", 2024-09-18T11:31:22.584 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.346049+0000", 2024-09-18T11:31:22.584 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.191007639999999, 2024-09-18T11:31:22.584 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0077275240000000004, 2024-09-18T11:31:22.584 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:22.584 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:22.584 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:22.584 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.584 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:22.584 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346049+0000", 2024-09-18T11:31:22.584 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.584 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.584 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:22.584 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346049+0000", 2024-09-18T11:31:22.585 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.585 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.585 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:22.585 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346047+0000", 2024-09-18T11:31:22.585 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:22.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.585 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.585 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:22.585 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346058+0000", 2024-09-18T11:31:22.585 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0463e-05 2024-09-18T11:31:22.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.585 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.585 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:22.586 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346069+0000", 2024-09-18T11:31:22.586 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1708999999999999e-05 2024-09-18T11:31:22.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.586 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.586 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:22.586 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.346076+0000", 2024-09-18T11:31:22.586 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2450000000000003e-06 2024-09-18T11:31:22.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.586 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.586 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:22.586 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349616+0000", 2024-09-18T11:31:22.586 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035406550000000002 2024-09-18T11:31:22.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.586 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.587 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:22.587 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349638+0000", 2024-09-18T11:31:22.587 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2098000000000001e-05 2024-09-18T11:31:22.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.587 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:22.587 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.353735+0000", 2024-09-18T11:31:22.587 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004096237 2024-09-18T11:31:22.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.587 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:22.587 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.353776+0000", 2024-09-18T11:31:22.587 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1530999999999999e-05 2024-09-18T11:31:22.587 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.587 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:22.588 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.588 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.588 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2129 4.1 4:8b9a985d:::smithi11939506-23:16 [read 0~3512630,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T11:31:22.588 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:55.710202+0000", 2024-09-18T11:31:22.588 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.826854379, 2024-09-18T11:31:22.588 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0078396100000000003, 2024-09-18T11:31:22.588 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:22.588 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:22.588 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:22.588 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:22.588 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:22.588 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2129 2024-09-18T11:31:22.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.589 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:22.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.589 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:22.589 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710202+0000", 2024-09-18T11:31:22.589 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.589 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:22.589 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710202+0000", 2024-09-18T11:31:22.589 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.590 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:22.590 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710202+0000", 2024-09-18T11:31:22.590 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:22.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.590 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:22.590 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710205+0000", 2024-09-18T11:31:22.590 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.382e-06 2024-09-18T11:31:22.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.591 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:22.591 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710212+0000", 2024-09-18T11:31:22.591 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770000000000004e-06 2024-09-18T11:31:22.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.591 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.591 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:22.591 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.710217+0000", 2024-09-18T11:31:22.591 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1680000000000003e-06 2024-09-18T11:31:22.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.591 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.591 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:22.591 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.716963+0000", 2024-09-18T11:31:22.591 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0067462449999999997 2024-09-18T11:31:22.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.591 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.592 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:22.592 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.717033+0000", 2024-09-18T11:31:22.592 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9643999999999999e-05 2024-09-18T11:31:22.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.592 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.592 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:22.592 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.718041+0000", 2024-09-18T11:31:22.592 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0010086019999999999 2024-09-18T11:31:22.592 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.592 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:22.592 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.592 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.592 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:172 4.5 e78/22 4:a087a566:::smithi11939506-46:19 v 78'135, mlcod=77'132)", 2024-09-18T11:31:22.593 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:22.593 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.151232687, 2024-09-18T11:31:22.593 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010938658, 2024-09-18T11:31:22.593 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:22.593 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:22.593 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:22.593 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.593 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:22.593 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:22.593 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.593 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.593 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:22.593 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:22.594 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.594 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.594 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:22.594 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385823+0000", 2024-09-18T11:31:22.594 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:22.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.594 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.594 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:22.594 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385830+0000", 2024-09-18T11:31:22.594 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2790000000000001e-06 2024-09-18T11:31:22.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.594 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.594 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:22.594 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385843+0000", 2024-09-18T11:31:22.595 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2785e-05 2024-09-18T11:31:22.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.595 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.595 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:22.595 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385848+0000", 2024-09-18T11:31:22.595 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5570000000000003e-06 2024-09-18T11:31:22.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.595 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.595 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:22.595 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385914+0000", 2024-09-18T11:31:22.595 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5168999999999999e-05 2024-09-18T11:31:22.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.595 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.595 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:22.596 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385930+0000", 2024-09-18T11:31:22.596 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6884999999999998e-05 2024-09-18T11:31:22.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.596 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.596 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:22.596 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396721+0000", 2024-09-18T11:31:22.596 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010790137 2024-09-18T11:31:22.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.596 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.596 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:22.596 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396762+0000", 2024-09-18T11:31:22.596 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1635999999999999e-05 2024-09-18T11:31:22.596 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.597 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:22.597 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.597 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.597 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:31:22.597 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:22.597 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.150739935000001, 2024-09-18T11:31:22.597 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:31:22.597 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:22.597 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:22.597 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:22.597 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.597 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:22.597 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:22.598 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.598 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.598 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:22.598 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:22.598 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.598 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.598 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:22.600 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:22.600 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:22.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.600 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.600 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:22.600 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:31:22.600 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:31:22.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.600 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.600 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:22.600 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:31:22.600 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:31:22.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.601 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.601 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:22.601 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:31:22.601 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:31:22.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.601 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.601 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:22.601 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:31:22.601 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:31:22.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.601 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.601 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:22.602 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:31:22.602 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:31:22.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.602 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.602 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:22.602 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:31:22.602 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:31:22.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.602 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.602 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:22.602 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:31:22.602 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:31:22.602 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.603 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:22.603 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.603 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:22.603 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:22.603 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.564627329, 2024-09-18T11:31:22.603 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:31:22.603 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:22.603 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:22.603 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:22.603 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:22.603 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:22.603 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:31:22.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.604 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:22.604 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.604 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:22.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:22.604 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.604 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.604 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:22.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:22.604 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.604 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.605 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:22.605 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:31:22.605 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:22.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.605 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:22.605 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:31:22.605 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:31:22.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.605 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:22.605 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:31:22.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:31:22.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.606 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:22.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:31:22.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:31:22.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.606 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:22.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:31:22.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:31:22.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.606 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:22.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:31:22.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:31:22.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.607 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:22.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:31:22.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:31:22.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.607 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:22.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:31:22.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:31:22.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.607 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:22.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:31:22.608 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:31:22.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.608 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:22.608 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:31:22.608 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:31:22.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.609 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:22.609 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:31:22.609 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:31:22.609 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.609 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:22.609 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.610 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2505 4.1 4:87078dfe:::smithi11939506-14:head [write 497540~612664,stat] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:22.610 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.992775+0000", 2024-09-18T11:31:22.610 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.544281343, 2024-09-18T11:31:22.610 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076090139999999999, 2024-09-18T11:31:22.610 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:22.610 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:22.610 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:22.610 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:22.610 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:22.610 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2505 2024-09-18T11:31:22.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.610 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:22.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:22.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.992775+0000", 2024-09-18T11:31:22.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:22.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.992775+0000", 2024-09-18T11:31:22.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:22.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.992773+0000", 2024-09-18T11:31:22.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:22.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.612 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:22.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993364+0000", 2024-09-18T11:31:22.612 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059123300000000004 2024-09-18T11:31:22.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.612 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:22.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993377+0000", 2024-09-18T11:31:22.612 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3464e-05 2024-09-18T11:31:22.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.612 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:22.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993382+0000", 2024-09-18T11:31:22.613 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2440000000000004e-06 2024-09-18T11:31:22.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.613 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:22.613 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993460+0000", 2024-09-18T11:31:22.613 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8584000000000005e-05 2024-09-18T11:31:22.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.613 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:22.613 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993530+0000", 2024-09-18T11:31:22.613 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0060000000000003e-05 2024-09-18T11:31:22.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.613 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:22.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993650+0000", 2024-09-18T11:31:22.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120079 2024-09-18T11:31:22.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.614 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:22.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.998802+0000", 2024-09-18T11:31:22.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005152106 2024-09-18T11:31:22.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.614 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:22.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.000318+0000", 2024-09-18T11:31:22.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015154719999999999 2024-09-18T11:31:22.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.615 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:22.615 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.000352+0000", 2024-09-18T11:31:22.615 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4418000000000001e-05 2024-09-18T11:31:22.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.615 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:22.615 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.000384+0000", 2024-09-18T11:31:22.615 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1594000000000003e-05 2024-09-18T11:31:22.615 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.615 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:22.615 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.616 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2525 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865577,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:22.616 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173466+0000", 2024-09-18T11:31:22.616 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.363590637, 2024-09-18T11:31:22.616 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075147870000000002, 2024-09-18T11:31:22.616 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:22.616 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:22.616 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:22.616 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:22.616 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:22.616 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2525 2024-09-18T11:31:22.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.616 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:22.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.617 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:22.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173466+0000", 2024-09-18T11:31:22.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.617 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:22.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173466+0000", 2024-09-18T11:31:22.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.617 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:22.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173465+0000", 2024-09-18T11:31:22.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:22.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:22.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173469+0000", 2024-09-18T11:31:22.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7299999999999999e-06 2024-09-18T11:31:22.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:22.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173476+0000", 2024-09-18T11:31:22.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7750000000000004e-06 2024-09-18T11:31:22.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:22.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173480+0000", 2024-09-18T11:31:22.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1810000000000003e-06 2024-09-18T11:31:22.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.619 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:22.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.180013+0000", 2024-09-18T11:31:22.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065332419999999999 2024-09-18T11:31:22.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.619 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:22.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.180115+0000", 2024-09-18T11:31:22.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102111 2024-09-18T11:31:22.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.619 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:22.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.180980+0000", 2024-09-18T11:31:22.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00086527100000000003 2024-09-18T11:31:22.619 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.619 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:22.620 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.620 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2647 4.e 4:78eefc4b:::smithi11939506-7:head [append 0~0] snapc 30=[30,2f,2b,2a,25,24,1c] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:22.620 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:22.620 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.5272126489999991, 2024-09-18T11:31:22.620 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096752739999999993, 2024-09-18T11:31:22.620 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:22.620 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:22.620 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:22.620 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:22.620 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:22.620 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2647 2024-09-18T11:31:22.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.621 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:22.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:22.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:22.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:22.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:22.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:22.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009843+0000", 2024-09-18T11:31:22.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:22.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:22.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009846+0000", 2024-09-18T11:31:22.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3870000000000002e-06 2024-09-18T11:31:22.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:22.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009853+0000", 2024-09-18T11:31:22.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.238e-06 2024-09-18T11:31:22.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:22.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009860+0000", 2024-09-18T11:31:22.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1999999999999997e-06 2024-09-18T11:31:22.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.623 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:22.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.013798+0000", 2024-09-18T11:31:22.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003938441 2024-09-18T11:31:22.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.623 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:22.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.013889+0000", 2024-09-18T11:31:22.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0332000000000002e-05 2024-09-18T11:31:22.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.623 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 6", 2024-09-18T11:31:22.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.014066+0000", 2024-09-18T11:31:22.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017728700000000001 2024-09-18T11:31:22.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.624 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:22.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015813+0000", 2024-09-18T11:31:22.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001746804 2024-09-18T11:31:22.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.624 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:22.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019419+0000", 2024-09-18T11:31:22.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036058399999999999 2024-09-18T11:31:22.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.625 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:22.625 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019478+0000", 2024-09-18T11:31:22.625 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8860999999999998e-05 2024-09-18T11:31:22.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.625 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:22.625 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019519+0000", 2024-09-18T11:31:22.625 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1208000000000003e-05 2024-09-18T11:31:22.625 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.625 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:22.625 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.625 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:209 4.5 e91/22 4:af4ba75a:::smithi11939506-37:19 v 91'165, mlcod=91'161)", 2024-09-18T11:31:22.626 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:22.626 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.5191359210000002, 2024-09-18T11:31:22.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0091546930000000002, 2024-09-18T11:31:22.626 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:22.626 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:22.626 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:22.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:22.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:22.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:22.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:22.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:22.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:22.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:22.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:22.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017925+0000", 2024-09-18T11:31:22.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0819999999999998e-06 2024-09-18T11:31:22.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.628 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:22.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017935+0000", 2024-09-18T11:31:22.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0597999999999999e-05 2024-09-18T11:31:22.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.628 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:22.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017941+0000", 2024-09-18T11:31:22.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2920000000000003e-06 2024-09-18T11:31:22.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.628 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:22.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017978+0000", 2024-09-18T11:31:22.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7667999999999999e-05 2024-09-18T11:31:22.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:22.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018018+0000", 2024-09-18T11:31:22.629 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9700999999999998e-05 2024-09-18T11:31:22.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:22.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.027039+0000", 2024-09-18T11:31:22.629 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090214079999999999 2024-09-18T11:31:22.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:22.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.027075+0000", 2024-09-18T11:31:22.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5605000000000002e-05 2024-09-18T11:31:22.630 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.630 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:22.630 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.630 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:570 4.b e91/67 4:ddcb7ff4:::smithi11939506-9:18 v 91'251, mlcod=90'245)", 2024-09-18T11:31:22.630 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:22.630 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.5191001489999998, 2024-09-18T11:31:22.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089212549999999995, 2024-09-18T11:31:22.630 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:22.630 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:22.631 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:22.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:22.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:22.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:22.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:22.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:22.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:22.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:22.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.632 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:22.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017960+0000", 2024-09-18T11:31:22.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.777e-06 2024-09-18T11:31:22.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.632 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:22.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017966+0000", 2024-09-18T11:31:22.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4389999999999997e-06 2024-09-18T11:31:22.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:22.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017970+0000", 2024-09-18T11:31:22.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5549999999999998e-06 2024-09-18T11:31:22.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:22.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018032+0000", 2024-09-18T11:31:22.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2976000000000001e-05 2024-09-18T11:31:22.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:22.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018053+0000", 2024-09-18T11:31:22.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0322e-05 2024-09-18T11:31:22.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:22.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026844+0000", 2024-09-18T11:31:22.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087915790000000008 2024-09-18T11:31:22.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:22.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026877+0000", 2024-09-18T11:31:22.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2916000000000001e-05 2024-09-18T11:31:22.634 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.634 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:22.634 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.635 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.5.13:15 3.1 3:bf804683:::e58794d0a579007c3bd048992c5646e06b9775f9667464eee6a2a0508e23a0a6:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:22.635 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:22.635 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.5189453020000006, 2024-09-18T11:31:22.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008816908, 2024-09-18T11:31:22.635 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:22.635 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:22.635 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:22.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.635 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:22.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:22.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.635 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:22.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:22.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.636 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:22.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018110+0000", 2024-09-18T11:31:22.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:22.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.636 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:22.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018114+0000", 2024-09-18T11:31:22.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0339999999999998e-06 2024-09-18T11:31:22.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.636 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:22.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018121+0000", 2024-09-18T11:31:22.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0199999999999997e-06 2024-09-18T11:31:22.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:22.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018125+0000", 2024-09-18T11:31:22.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0409999999999999e-06 2024-09-18T11:31:22.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:22.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018793+0000", 2024-09-18T11:31:22.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00066779199999999995 2024-09-18T11:31:22.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:22.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018868+0000", 2024-09-18T11:31:22.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4814999999999994e-05 2024-09-18T11:31:22.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:22.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019008+0000", 2024-09-18T11:31:22.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014031800000000001 2024-09-18T11:31:22.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:22.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.023494+0000", 2024-09-18T11:31:22.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0044861720000000001 2024-09-18T11:31:22.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:22.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026796+0000", 2024-09-18T11:31:22.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0033010969999999998 2024-09-18T11:31:22.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.639 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:22.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026891+0000", 2024-09-18T11:31:22.640 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5828000000000005e-05 2024-09-18T11:31:22.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.640 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:22.640 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026928+0000", 2024-09-18T11:31:22.640 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6393999999999997e-05 2024-09-18T11:31:22.640 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.640 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:22.640 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.640 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2793 4.1 4:87078dfe:::smithi11939506-14:head [write 534522~634550] snapc 30=[30,2f,2b,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:31:22.640 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:22.641 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.313372674, 2024-09-18T11:31:22.641 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085134289999999994, 2024-09-18T11:31:22.641 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:22.641 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:22.641 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:22.641 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:22.641 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:22.641 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2793 2024-09-18T11:31:22.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.641 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:22.641 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.641 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:22.641 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:22.641 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:22.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:22.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:22.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:22.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223681+0000", 2024-09-18T11:31:22.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:22.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:22.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224251+0000", 2024-09-18T11:31:22.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056940699999999996 2024-09-18T11:31:22.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:22.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224267+0000", 2024-09-18T11:31:22.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6829000000000001e-05 2024-09-18T11:31:22.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:22.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224292+0000", 2024-09-18T11:31:22.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4198000000000001e-05 2024-09-18T11:31:22.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:22.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224352+0000", 2024-09-18T11:31:22.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0608e-05 2024-09-18T11:31:22.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.644 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:22.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224456+0000", 2024-09-18T11:31:22.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010412300000000001 2024-09-18T11:31:22.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.644 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:22.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224662+0000", 2024-09-18T11:31:22.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020561399999999999 2024-09-18T11:31:22.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.645 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:22.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231800+0000", 2024-09-18T11:31:22.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071384530000000003 2024-09-18T11:31:22.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.645 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:22.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232126+0000", 2024-09-18T11:31:22.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00032601400000000001 2024-09-18T11:31:22.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.645 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:22.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232158+0000", 2024-09-18T11:31:22.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1472e-05 2024-09-18T11:31:22.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:22.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:22.646 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:22.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232197+0000", 2024-09-18T11:31:22.646 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9011999999999999e-05 2024-09-18T11:31:22.646 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.646 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:22.646 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.646 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:22.646 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:22.646 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:22.647 INFO:tasks.rados.rados.0.smithi119.stdout:1342: set_chunk oid 37 offset: 995840 length: 1164288 target oid fcdb49fcf0a243d0d7a3606e3bca63304b77089121363f625219a619714495be offset: 0 snap -1 2024-09-18T11:31:22.648 INFO:tasks.rados.rados.0.smithi119.stdout:1343: tier_promote oid 22 2024-09-18T11:31:22.648 INFO:tasks.rados.rados.0.smithi119.stdout:1337: expect (ObjNum 237 snap 28 seq_num 237) 2024-09-18T11:31:22.648 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:22.646+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:22.648 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:31:22.666 INFO:tasks.rados.rados.0.smithi119.stdout:1340: finishing set_chunk to oid 7 2024-09-18T11:31:22.667 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 7 v 113 (ObjNum 145 snap 18 seq_num 145) dirty exists 2024-09-18T11:31:22.667 INFO:tasks.rados.rados.0.smithi119.stdout:1339: expect (ObjNum 219 snap 26 seq_num 219) 2024-09-18T11:31:22.748 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:22.747+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:22.783 INFO:tasks.rados.rados.0.smithi119.stdout:1341: finishing write tid 1 to smithi11939506-31 2024-09-18T11:31:22.784 INFO:tasks.rados.rados.0.smithi119.stdout:1341: finishing write tid 2 to smithi11939506-31 2024-09-18T11:31:22.784 INFO:tasks.rados.rados.0.smithi119.stdout:1341: finishing write tid 3 to smithi11939506-31 2024-09-18T11:31:22.784 INFO:tasks.rados.rados.0.smithi119.stdout:1341: finishing write tid 4 to smithi11939506-31 2024-09-18T11:31:22.784 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 203 (ObjNum 257 snap 31 seq_num 257) dirty exists 2024-09-18T11:31:22.784 INFO:tasks.rados.rados.0.smithi119.stdout:1341: left oid 31 (ObjNum 257 snap 31 seq_num 257) 2024-09-18T11:31:22.784 INFO:tasks.rados.rados.0.smithi119.stdout:1342: finishing set_chunk to oid 37 2024-09-18T11:31:22.784 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 37 set_chunk returned error code -95 offset: 995840 length: 1164288 2024-09-18T11:31:22.784 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 0 (ObjNum 188 snap 20 seq_num 188) dirty exists 2024-09-18T11:31:22.784 INFO:tasks.rados.rados.0.smithi119.stdout:1343: got (0) Success 2024-09-18T11:31:22.784 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 126 (ObjNum 238 snap 28 seq_num 238) dirty exists 2024-09-18T11:31:22.784 INFO:tasks.rados.rados.0.smithi119.stdout:1337: done (5 left) 2024-09-18T11:31:22.784 INFO:tasks.rados.rados.0.smithi119.stdout:1339: done (4 left) 2024-09-18T11:31:22.785 INFO:tasks.rados.rados.0.smithi119.stdout:1340: done (3 left) 2024-09-18T11:31:22.785 INFO:tasks.rados.rados.0.smithi119.stdout:1341: done (2 left) 2024-09-18T11:31:22.785 INFO:tasks.rados.rados.0.smithi119.stdout:1342: done (1 left) 2024-09-18T11:31:22.785 INFO:tasks.rados.rados.0.smithi119.stdout:1343: done (0 left) 2024-09-18T11:31:22.785 INFO:tasks.rados.rados.0.smithi119.stdout:1344: read oid 1 snap 27 2024-09-18T11:31:22.785 INFO:tasks.rados.rados.0.smithi119.stdout:1344: expect (ObjNum 186 snap 20 seq_num 186) 2024-09-18T11:31:22.785 INFO:tasks.rados.rados.0.smithi119.stdout:1345: write (excl) oid 24 current snap is 31 2024-09-18T11:31:22.785 INFO:tasks.rados.rados.0.smithi119.stdout:1345: seq_num 258 ranges {736206=453891,1881413=454793} 2024-09-18T11:31:22.787 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:22.787 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:22.787 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:22.787 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:22.789 INFO:tasks.rados.rados.0.smithi119.stdout:1345: writing smithi11939506-24 from 736206 to 1190097 tid 1 2024-09-18T11:31:22.796 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:31:22.797 INFO:tasks.rados.rados.0.smithi119.stdout:1345: writing smithi11939506-24 from 1881413 to 2336206 tid 2 2024-09-18T11:31:22.797 INFO:tasks.rados.rados.0.smithi119.stdout:1346: read oid 19 snap -1 2024-09-18T11:31:22.797 INFO:tasks.rados.rados.0.smithi119.stdout:1346: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-18T11:31:22.797 INFO:tasks.rados.rados.0.smithi119.stdout:1347: write oid 39 current snap is 31 2024-09-18T11:31:22.797 INFO:tasks.rados.rados.0.smithi119.stdout:1347: seq_num 259 ranges {753013=645493,2056248=296765} 2024-09-18T11:31:22.803 INFO:tasks.rados.rados.0.smithi119.stdout:1347: writing smithi11939506-39 from 753013 to 1398506 tid 1 2024-09-18T11:31:22.809 INFO:tasks.rados.rados.0.smithi119.stdout:1347: writing smithi11939506-39 from 2056248 to 2353013 tid 2 2024-09-18T11:31:22.809 INFO:tasks.rados.rados.0.smithi119.stdout:1348: write oid 10 current snap is 31 2024-09-18T11:31:22.809 INFO:tasks.rados.rados.0.smithi119.stdout:1348: seq_num 260 ranges {769820=720742,2114730=255090} 2024-09-18T11:31:22.816 INFO:tasks.rados.rados.0.smithi119.stdout:1348: writing smithi11939506-10 from 769820 to 1490562 tid 1 2024-09-18T11:31:22.822 INFO:tasks.rados.rados.0.smithi119.stdout:1348: writing smithi11939506-10 from 2114730 to 2369820 tid 2 2024-09-18T11:31:22.822 INFO:tasks.rados.rados.0.smithi119.stdout:1349: rollback oid 28 current snap is 31 2024-09-18T11:31:22.822 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 28 to 23 2024-09-18T11:31:22.822 INFO:tasks.rados.rados.0.smithi119.stdout:1344: expect (ObjNum 186 snap 20 seq_num 186) 2024-09-18T11:31:22.849 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:22.847+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:22.895 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:22.895 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:22.895 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:22.895 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:22.895 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:22.904 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:31:22.922 INFO:tasks.rados.rados.0.smithi119.stdout:1345: finishing write tid 1 to smithi11939506-24 2024-09-18T11:31:22.922 INFO:tasks.rados.rados.0.smithi119.stdout:1345: finishing write tid 2 to smithi11939506-24 2024-09-18T11:31:22.922 INFO:tasks.rados.rados.0.smithi119.stdout:1345: finishing write tid 3 to smithi11939506-24 2024-09-18T11:31:22.922 INFO:tasks.rados.rados.0.smithi119.stdout:1345: finishing write tid 4 to smithi11939506-24 2024-09-18T11:31:22.922 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 207 (ObjNum 258 snap 31 seq_num 258) dirty exists 2024-09-18T11:31:22.923 INFO:tasks.rados.rados.0.smithi119.stdout:1345: left oid 24 (ObjNum 258 snap 31 seq_num 258) 2024-09-18T11:31:22.923 INFO:tasks.rados.rados.0.smithi119.stdout:1346: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-18T11:31:22.950 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:22.947+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:22.995 INFO:tasks.rados.rados.0.smithi119.stdout:1347: finishing write tid 1 to smithi11939506-39 2024-09-18T11:31:22.995 INFO:tasks.rados.rados.0.smithi119.stdout:1347: finishing write tid 2 to smithi11939506-39 2024-09-18T11:31:22.995 INFO:tasks.rados.rados.0.smithi119.stdout:1344: done (5 left) 2024-09-18T11:31:22.995 INFO:tasks.rados.rados.0.smithi119.stdout:1345: done (4 left) 2024-09-18T11:31:22.996 INFO:tasks.rados.rados.0.smithi119.stdout:1346: done (3 left) 2024-09-18T11:31:22.996 INFO:tasks.rados.rados.0.smithi119.stdout:1350: write (excl) oid 37 current snap is 31 2024-09-18T11:31:22.996 INFO:tasks.rados.rados.0.smithi119.stdout:1347: finishing write tid 3 to smithi11939506-39 2024-09-18T11:31:22.996 INFO:tasks.rados.rados.0.smithi119.stdout:1347: finishing write tid 4 to smithi11939506-39 2024-09-18T11:31:22.996 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 39 v 211 (ObjNum 259 snap 31 seq_num 259) dirty exists 2024-09-18T11:31:22.996 INFO:tasks.rados.rados.0.smithi119.stdout:1347: left oid 39 (ObjNum 259 snap 31 seq_num 259) 2024-09-18T11:31:22.996 INFO:tasks.rados.rados.0.smithi119.stdout:1348: finishing write tid 1 to smithi11939506-10 2024-09-18T11:31:22.996 INFO:tasks.rados.rados.0.smithi119.stdout:1348: finishing write tid 2 to smithi11939506-10 2024-09-18T11:31:22.996 INFO:tasks.rados.rados.0.smithi119.stdout:1348: finishing write tid 3 to smithi11939506-10 2024-09-18T11:31:22.996 INFO:tasks.rados.rados.0.smithi119.stdout:1348: finishing write tid 4 to smithi11939506-10 2024-09-18T11:31:22.996 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 10 v 151 (ObjNum 260 snap 31 seq_num 260) dirty exists 2024-09-18T11:31:22.996 INFO:tasks.rados.rados.0.smithi119.stdout:1348: left oid 10 (ObjNum 260 snap 31 seq_num 260) 2024-09-18T11:31:22.996 INFO:tasks.rados.rados.0.smithi119.stdout:1349: finishing rollback tid 1 to smithi11939506-28 2024-09-18T11:31:22.997 INFO:tasks.rados.rados.0.smithi119.stdout:1349: finishing rollback tid 2 to smithi11939506-28 2024-09-18T11:31:22.997 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 28 v 158 (ObjNum 200 snap 23 seq_num 200) dirty exists 2024-09-18T11:31:22.997 INFO:tasks.rados.rados.0.smithi119.stdout:1350: seq_num 261 ranges {786627=795991,2173212=213415} 2024-09-18T11:31:23.003 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:23.003 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:31:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 92.975063828000003, 2024-09-18T11:31:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073601550000000002, 2024-09-18T11:31:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:31:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:31:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1 2024-09-18T11:31:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026524+0000", 2024-09-18T11:31:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999905 2024-09-18T11:31:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026541+0000", 2024-09-18T11:31:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7070000000000001e-05 2024-09-18T11:31:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026554+0000", 2024-09-18T11:31:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.288e-05 2024-09-18T11:31:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026564+0000", 2024-09-18T11:31:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0197000000000001e-05 2024-09-18T11:31:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026826+0000", 2024-09-18T11:31:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026212400000000003 2024-09-18T11:31:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:31:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026839+0000", 2024-09-18T11:31:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2831e-05 2024-09-18T11:31:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028605+0000", 2024-09-18T11:31:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001765794 2024-09-18T11:31:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028709+0000", 2024-09-18T11:31:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104348 2024-09-18T11:31:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028824+0000", 2024-09-18T11:31:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011527399999999999 2024-09-18T11:31:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030319+0000", 2024-09-18T11:31:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001494439 2024-09-18T11:31:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033805+0000", 2024-09-18T11:31:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034857820000000002 2024-09-18T11:31:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033856+0000", 2024-09-18T11:31:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1687999999999997e-05 2024-09-18T11:31:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033894+0000", 2024-09-18T11:31:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7481000000000002e-05 2024-09-18T11:31:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T11:31:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:31:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 90.955785397, 2024-09-18T11:31:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055107869999999996, 2024-09-18T11:31:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:31:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:31:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 7 2024-09-18T11:31:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:31:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045812+0000", 2024-09-18T11:31:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045810+0000", 2024-09-18T11:31:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045820+0000", 2024-09-18T11:31:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0215e-05 2024-09-18T11:31:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045838+0000", 2024-09-18T11:31:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7849000000000001e-05 2024-09-18T11:31:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045845+0000", 2024-09-18T11:31:23.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4229999999999999e-06 2024-09-18T11:31:23.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045898+0000", 2024-09-18T11:31:23.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2537999999999997e-05 2024-09-18T11:31:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.045904+0000", 2024-09-18T11:31:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4019999999999999e-06 2024-09-18T11:31:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047663+0000", 2024-09-18T11:31:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017584079999999999 2024-09-18T11:31:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047736+0000", 2024-09-18T11:31:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3509000000000004e-05 2024-09-18T11:31:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.047857+0000", 2024-09-18T11:31:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012037800000000001 2024-09-18T11:31:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.050868+0000", 2024-09-18T11:31:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030111420000000001 2024-09-18T11:31:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051239+0000", 2024-09-18T11:31:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037141000000000002 2024-09-18T11:31:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051288+0000", 2024-09-18T11:31:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9051000000000002e-05 2024-09-18T11:31:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:52.051323+0000", 2024-09-18T11:31:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4545000000000002e-05 2024-09-18T11:31:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1118 3.2 e38/21 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head v 38'76, mlcod=38'74)", 2024-09-18T11:31:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 63.053653676000003, 2024-09-18T11:31:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006048688, 2024-09-18T11:31:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947941+0000", 2024-09-18T11:31:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949868+0000", 2024-09-18T11:31:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019267659999999999 2024-09-18T11:31:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949893+0000", 2024-09-18T11:31:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5239999999999999e-05 2024-09-18T11:31:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949901+0000", 2024-09-18T11:31:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8420000000000005e-06 2024-09-18T11:31:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949954+0000", 2024-09-18T11:31:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3464999999999997e-05 2024-09-18T11:31:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949974+0000", 2024-09-18T11:31:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9731000000000001e-05 2024-09-18T11:31:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953948+0000", 2024-09-18T11:31:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039735489999999998 2024-09-18T11:31:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953992+0000", 2024-09-18T11:31:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4700999999999998e-05 2024-09-18T11:31:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1662 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'73, mlcod=51'69)", 2024-09-18T11:31:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.640048641, 2024-09-18T11:31:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059408539999999998, 2024-09-18T11:31:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361914+0000", 2024-09-18T11:31:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036569 2024-09-18T11:31:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361926+0000", 2024-09-18T11:31:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1960999999999999e-05 2024-09-18T11:31:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361937+0000", 2024-09-18T11:31:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0876999999999999e-05 2024-09-18T11:31:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361986+0000", 2024-09-18T11:31:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8652000000000001e-05 2024-09-18T11:31:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.362012+0000", 2024-09-18T11:31:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5854999999999999e-05 2024-09-18T11:31:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367460+0000", 2024-09-18T11:31:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054482100000000002 2024-09-18T11:31:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367490+0000", 2024-09-18T11:31:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9909000000000001e-05 2024-09-18T11:31:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:269 4.7 e58/22 4:e73ba68f:::smithi11939506-1:13 v 58'72, mlcod=57'69)", 2024-09-18T11:31:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 43.736216744000004, 2024-09-18T11:31:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065512569999999996, 2024-09-18T11:31:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265380+0000", 2024-09-18T11:31:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265388+0000", 2024-09-18T11:31:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.174e-06 2024-09-18T11:31:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265398+0000", 2024-09-18T11:31:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2599999999999994e-06 2024-09-18T11:31:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265402+0000", 2024-09-18T11:31:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0289999999999999e-06 2024-09-18T11:31:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265434+0000", 2024-09-18T11:31:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2511000000000001e-05 2024-09-18T11:31:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265447+0000", 2024-09-18T11:31:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2947e-05 2024-09-18T11:31:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271892+0000", 2024-09-18T11:31:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064445930000000002 2024-09-18T11:31:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271932+0000", 2024-09-18T11:31:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0303999999999999e-05 2024-09-18T11:31:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1788 4.8 4:1898b15d:::smithi11939506-25:head [write 453822~797061] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.085784160999999, 2024-09-18T11:31:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094305159999999999, 2024-09-18T11:31:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.033 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.033 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.033 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1788 2024-09-18T11:31:23.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.033 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:23.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:23.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915811+0000", 2024-09-18T11:31:23.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:23.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916608+0000", 2024-09-18T11:31:23.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079635000000000003 2024-09-18T11:31:23.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916624+0000", 2024-09-18T11:31:23.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6735000000000002e-05 2024-09-18T11:31:23.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916630+0000", 2024-09-18T11:31:23.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5709999999999997e-06 2024-09-18T11:31:23.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916674+0000", 2024-09-18T11:31:23.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4540999999999999e-05 2024-09-18T11:31:23.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916751+0000", 2024-09-18T11:31:23.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6695999999999999e-05 2024-09-18T11:31:23.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.036 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:23.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916872+0000", 2024-09-18T11:31:23.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120648 2024-09-18T11:31:23.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.036 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:23.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.919076+0000", 2024-09-18T11:31:23.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022046829999999998 2024-09-18T11:31:23.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:23.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925151+0000", 2024-09-18T11:31:23.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060740239999999999 2024-09-18T11:31:23.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925198+0000", 2024-09-18T11:31:23.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7896999999999999e-05 2024-09-18T11:31:23.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925244+0000", 2024-09-18T11:31:23.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5398999999999997e-05 2024-09-18T11:31:23.037 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.038 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.038 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.038 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1789 4.8 4:1898b15d:::smithi11939506-25:head [write 1760371~653366] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:23.038 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.921860+0000", 2024-09-18T11:31:23.038 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.079737475999998, 2024-09-18T11:31:23.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055904329999999997, 2024-09-18T11:31:23.038 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.038 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.038 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.038 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.038 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.038 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1789 2024-09-18T11:31:23.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.039 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.921860+0000", 2024-09-18T11:31:23.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.921860+0000", 2024-09-18T11:31:23.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.921859+0000", 2024-09-18T11:31:23.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:23.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922436+0000", 2024-09-18T11:31:23.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00057708300000000004 2024-09-18T11:31:23.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922450+0000", 2024-09-18T11:31:23.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4575e-05 2024-09-18T11:31:23.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922458+0000", 2024-09-18T11:31:23.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:31:23.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922508+0000", 2024-09-18T11:31:23.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0102999999999998e-05 2024-09-18T11:31:23.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922607+0000", 2024-09-18T11:31:23.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8813000000000004e-05 2024-09-18T11:31:23.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:23.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922814+0000", 2024-09-18T11:31:23.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000207488 2024-09-18T11:31:23.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:23.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925504+0000", 2024-09-18T11:31:23.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026899200000000002 2024-09-18T11:31:23.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:23.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927350+0000", 2024-09-18T11:31:23.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00184575 2024-09-18T11:31:23.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.043 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927404+0000", 2024-09-18T11:31:23.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4147000000000002e-05 2024-09-18T11:31:23.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.043 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.927450+0000", 2024-09-18T11:31:23.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6480000000000002e-05 2024-09-18T11:31:23.043 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.043 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.044 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.044 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1934 4.2 e65/22 4:425afe9c:::smithi11939506-29:head v 65'44, mlcod=60'40)", 2024-09-18T11:31:23.044 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:23.044 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 34.701605391000001, 2024-09-18T11:31:23.044 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064585119999999996, 2024-09-18T11:31:23.044 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.044 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.044 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.044 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.044 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:23.044 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.045 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.045 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:23.045 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.045 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.045 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299990+0000", 2024-09-18T11:31:23.045 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:23.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.045 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.045 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300789+0000", 2024-09-18T11:31:23.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079851499999999997 2024-09-18T11:31:23.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.046 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.046 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300806+0000", 2024-09-18T11:31:23.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7096999999999999e-05 2024-09-18T11:31:23.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.046 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.046 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300813+0000", 2024-09-18T11:31:23.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5079999999999999e-06 2024-09-18T11:31:23.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.046 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.047 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300870+0000", 2024-09-18T11:31:23.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7178e-05 2024-09-18T11:31:23.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.047 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.047 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300897+0000", 2024-09-18T11:31:23.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6284000000000001e-05 2024-09-18T11:31:23.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.047 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.047 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306410+0000", 2024-09-18T11:31:23.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055135210000000004 2024-09-18T11:31:23.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.048 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.048 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306451+0000", 2024-09-18T11:31:23.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0379000000000001e-05 2024-09-18T11:31:23.048 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.048 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.048 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.048 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2119 4.7 e73/22 4:e0d95e10:::smithi11939506-22:head v 73'89, mlcod=73'86)", 2024-09-18T11:31:23.048 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:55.701097+0000", 2024-09-18T11:31:23.048 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.300500745000001, 2024-09-18T11:31:23.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056939520000000004, 2024-09-18T11:31:23.048 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.049 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.049 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.049 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.049 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701097+0000", 2024-09-18T11:31:23.049 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.049 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.049 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701097+0000", 2024-09-18T11:31:23.049 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.049 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.050 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701095+0000", 2024-09-18T11:31:23.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:23.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.050 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.050 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701919+0000", 2024-09-18T11:31:23.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000824181 2024-09-18T11:31:23.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.050 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.050 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701938+0000", 2024-09-18T11:31:23.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8614000000000002e-05 2024-09-18T11:31:23.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.051 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.051 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701947+0000", 2024-09-18T11:31:23.051 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0699999999999996e-06 2024-09-18T11:31:23.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.051 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.051 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701992+0000", 2024-09-18T11:31:23.051 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5614000000000001e-05 2024-09-18T11:31:23.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.051 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.051 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.702012+0000", 2024-09-18T11:31:23.051 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9639999999999998e-05 2024-09-18T11:31:23.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.052 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.706755+0000", 2024-09-18T11:31:23.052 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004742708 2024-09-18T11:31:23.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.052 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.706791+0000", 2024-09-18T11:31:23.052 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6106999999999998e-05 2024-09-18T11:31:23.052 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.052 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.052 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.053 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2452 4.3 4:ca2b2309:::smithi11939506-44:head [read 0~3462242,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:31:23.053 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:31:23.053 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 16.964830033999998, 2024-09-18T11:31:23.053 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060906450000000004, 2024-09-18T11:31:23.053 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.053 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:23.053 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.053 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.053 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.053 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2452 2024-09-18T11:31:23.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.053 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.054 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.054 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:31:23.054 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.054 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.054 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:31:23.054 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.054 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.054 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036765+0000", 2024-09-18T11:31:23.054 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:23.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.055 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036777+0000", 2024-09-18T11:31:23.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1422e-05 2024-09-18T11:31:23.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.055 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036793+0000", 2024-09-18T11:31:23.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6575999999999999e-05 2024-09-18T11:31:23.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.055 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036800+0000", 2024-09-18T11:31:23.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9940000000000003e-06 2024-09-18T11:31:23.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.056 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.056 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:23.056 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5847999999999999e-05 2024-09-18T11:31:23.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.056 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:23.056 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036853+0000", 2024-09-18T11:31:23.056 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0119999999999999e-06 2024-09-18T11:31:23.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.056 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.057 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038527+0000", 2024-09-18T11:31:23.057 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016732730000000001 2024-09-18T11:31:23.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.057 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.057 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038580+0000", 2024-09-18T11:31:23.057 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3443999999999999e-05 2024-09-18T11:31:23.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.057 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.057 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042858+0000", 2024-09-18T11:31:23.057 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042781049999999999 2024-09-18T11:31:23.057 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.058 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.058 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.058 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.058 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2453 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:31:23.058 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:31:23.058 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 16.964787802, 2024-09-18T11:31:23.058 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063798429999999996, 2024-09-18T11:31:23.058 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.058 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:23.058 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.058 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.058 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.058 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2453 2024-09-18T11:31:23.059 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.059 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.059 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.059 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.059 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:31:23.059 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.059 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.059 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.059 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.059 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:31:23.059 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.059 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.059 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.059 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.059 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036809+0000", 2024-09-18T11:31:23.060 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:23.060 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.060 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.060 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.060 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036812+0000", 2024-09-18T11:31:23.060 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8339999999999999e-06 2024-09-18T11:31:23.060 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.060 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.060 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.060 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036819+0000", 2024-09-18T11:31:23.060 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9890000000000003e-06 2024-09-18T11:31:23.060 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.060 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.061 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.061 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036823+0000", 2024-09-18T11:31:23.061 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8099999999999999e-06 2024-09-18T11:31:23.061 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.061 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.061 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.061 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036912+0000", 2024-09-18T11:31:23.061 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9083000000000001e-05 2024-09-18T11:31:23.061 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.061 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.061 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:23.062 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036932+0000", 2024-09-18T11:31:23.062 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0222000000000001e-05 2024-09-18T11:31:23.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.062 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.062 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.062 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042896+0000", 2024-09-18T11:31:23.062 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059637240000000001 2024-09-18T11:31:23.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.062 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.062 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.062 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042952+0000", 2024-09-18T11:31:23.062 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5877000000000001e-05 2024-09-18T11:31:23.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.063 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.063 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.063 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043190+0000", 2024-09-18T11:31:23.063 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00023774500000000001 2024-09-18T11:31:23.063 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.063 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.063 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.063 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.063 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.063 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2454 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:31:23.063 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:23.063 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 16.964751845999999, 2024-09-18T11:31:23.063 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066404469999999998, 2024-09-18T11:31:23.064 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.064 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:23.064 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.064 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.064 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.064 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2454 2024-09-18T11:31:23.064 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.064 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.064 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.064 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:23.064 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.064 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.065 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.065 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:23.065 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.065 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.065 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.065 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036845+0000", 2024-09-18T11:31:23.065 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:23.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.065 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.065 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.065 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036856+0000", 2024-09-18T11:31:23.065 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1338000000000001e-05 2024-09-18T11:31:23.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.066 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.066 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036863+0000", 2024-09-18T11:31:23.066 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4749999999999998e-06 2024-09-18T11:31:23.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.066 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.066 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036867+0000", 2024-09-18T11:31:23.066 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3839999999999999e-06 2024-09-18T11:31:23.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.066 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.066 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036946+0000", 2024-09-18T11:31:23.067 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9054000000000006e-05 2024-09-18T11:31:23.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.067 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:23.067 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036949+0000", 2024-09-18T11:31:23.067 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9390000000000002e-06 2024-09-18T11:31:23.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.067 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.067 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043213+0000", 2024-09-18T11:31:23.067 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062640090000000001 2024-09-18T11:31:23.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.067 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.068 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043261+0000", 2024-09-18T11:31:23.068 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7750999999999997e-05 2024-09-18T11:31:23.068 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.068 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.068 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.068 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043486+0000", 2024-09-18T11:31:23.068 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000224985 2024-09-18T11:31:23.068 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.068 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.068 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.068 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.068 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.068 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2505 4.1 e85/22 4:87078dfe:::smithi11939506-14:head v 85'167, mlcod=85'163)", 2024-09-18T11:31:23.069 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.993736+0000", 2024-09-18T11:31:23.069 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.007861402, 2024-09-18T11:31:23.069 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056183630000000003, 2024-09-18T11:31:23.069 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.069 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.069 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.069 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.069 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.069 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993736+0000", 2024-09-18T11:31:23.069 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.070 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.070 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993736+0000", 2024-09-18T11:31:23.070 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.070 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.070 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993735+0000", 2024-09-18T11:31:23.070 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:23.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.071 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.071 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.994296+0000", 2024-09-18T11:31:23.071 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056099399999999999 2024-09-18T11:31:23.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.071 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.071 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.071 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.994313+0000", 2024-09-18T11:31:23.071 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6982000000000001e-05 2024-09-18T11:31:23.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.071 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.071 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.071 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.994319+0000", 2024-09-18T11:31:23.072 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0279999999999997e-06 2024-09-18T11:31:23.072 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.072 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.072 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.072 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.994374+0000", 2024-09-18T11:31:23.072 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5019999999999998e-05 2024-09-18T11:31:23.072 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.072 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.072 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.072 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.994401+0000", 2024-09-18T11:31:23.072 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7104e-05 2024-09-18T11:31:23.072 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.072 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.073 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.073 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.999314+0000", 2024-09-18T11:31:23.073 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049127390000000002 2024-09-18T11:31:23.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.073 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.073 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.073 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.999354+0000", 2024-09-18T11:31:23.073 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0763000000000003e-05 2024-09-18T11:31:23.073 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.073 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.073 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.073 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.073 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2553 4.3 4:ca2b2309:::smithi11939506-44:head [write 564768~630013,stat] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-18T11:31:23.074 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:31:23.074 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.804474213000001, 2024-09-18T11:31:23.074 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065686879999999996, 2024-09-18T11:31:23.074 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.074 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.074 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.074 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.074 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.074 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2553 2024-09-18T11:31:23.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.074 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.074 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.074 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:31:23.075 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.075 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.075 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:31:23.075 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.075 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.075 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197120+0000", 2024-09-18T11:31:23.075 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:23.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.075 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.075 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199000+0000", 2024-09-18T11:31:23.076 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018792080000000001 2024-09-18T11:31:23.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.076 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.076 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199044+0000", 2024-09-18T11:31:23.076 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4048999999999998e-05 2024-09-18T11:31:23.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.076 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.076 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199071+0000", 2024-09-18T11:31:23.076 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7288e-05 2024-09-18T11:31:23.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.077 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199119+0000", 2024-09-18T11:31:23.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8000999999999997e-05 2024-09-18T11:31:23.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.077 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199208+0000", 2024-09-18T11:31:23.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9209000000000006e-05 2024-09-18T11:31:23.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.077 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:23.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199385+0000", 2024-09-18T11:31:23.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000177095 2024-09-18T11:31:23.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:23.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.202282+0000", 2024-09-18T11:31:23.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028970469999999998 2024-09-18T11:31:23.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:23.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203578+0000", 2024-09-18T11:31:23.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012960490000000001 2024-09-18T11:31:23.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203638+0000", 2024-09-18T11:31:23.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9599000000000003e-05 2024-09-18T11:31:23.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.079 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203692+0000", 2024-09-18T11:31:23.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3908e-05 2024-09-18T11:31:23.079 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.079 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.079 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.079 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2751 4.3 4:ca2b2309:::smithi11939506-44:25 [read 0~3210104,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:31:23.079 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:17.886856+0000", 2024-09-18T11:31:23.080 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.1147410720000002, 2024-09-18T11:31:23.080 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057395420000000003, 2024-09-18T11:31:23.080 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.080 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:23.080 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.080 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.080 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.080 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2751 2024-09-18T11:31:23.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.080 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.080 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886856+0000", 2024-09-18T11:31:23.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.081 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.081 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886856+0000", 2024-09-18T11:31:23.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.081 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.081 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886856+0000", 2024-09-18T11:31:23.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:23.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.081 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.082 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886859+0000", 2024-09-18T11:31:23.082 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.886e-06 2024-09-18T11:31:23.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.082 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.082 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886871+0000", 2024-09-18T11:31:23.082 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2249000000000001e-05 2024-09-18T11:31:23.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.082 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.082 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886874+0000", 2024-09-18T11:31:23.082 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2160000000000002e-06 2024-09-18T11:31:23.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.083 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.083 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.890901+0000", 2024-09-18T11:31:23.083 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040264860000000001 2024-09-18T11:31:23.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.083 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.083 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.891032+0000", 2024-09-18T11:31:23.083 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131438 2024-09-18T11:31:23.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.083 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.083 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.892596+0000", 2024-09-18T11:31:23.083 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015636350000000001 2024-09-18T11:31:23.083 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.084 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.084 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.084 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2752 4.3 4:ca2b2309:::smithi11939506-44:25 [read 0~3210104,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:31:23.084 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:31:23.084 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.1147050529999998, 2024-09-18T11:31:23.084 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068715460000000001, 2024-09-18T11:31:23.084 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.084 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:23.084 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.084 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.084 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.084 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2752 2024-09-18T11:31:23.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.085 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:31:23.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:31:23.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.086 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:31:23.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:23.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.086 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.086 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886895+0000", 2024-09-18T11:31:23.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.993e-06 2024-09-18T11:31:23.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.086 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.086 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886901+0000", 2024-09-18T11:31:23.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1859999999999997e-06 2024-09-18T11:31:23.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.087 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.087 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886904+0000", 2024-09-18T11:31:23.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1920000000000001e-06 2024-09-18T11:31:23.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.087 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.087 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.892622+0000", 2024-09-18T11:31:23.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057175660000000003 2024-09-18T11:31:23.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.087 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.087 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.892693+0000", 2024-09-18T11:31:23.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1265000000000003e-05 2024-09-18T11:31:23.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.088 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.088 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.893764+0000", 2024-09-18T11:31:23.088 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001070753 2024-09-18T11:31:23.088 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.088 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.088 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.088 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2793 4.1 e92/22 4:87078dfe:::smithi11939506-14:head v 92'195, mlcod=92'191)", 2024-09-18T11:31:23.088 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:31:23.088 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.7768050190000002, 2024-09-18T11:31:23.088 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062761989999999997, 2024-09-18T11:31:23.089 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.089 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.089 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.089 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.089 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:31:23.089 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.089 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.089 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:31:23.089 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:31:23.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:23.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224994+0000", 2024-09-18T11:31:23.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020197600000000001 2024-09-18T11:31:23.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225007+0000", 2024-09-18T11:31:23.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3791999999999999e-05 2024-09-18T11:31:23.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225013+0000", 2024-09-18T11:31:23.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3630000000000002e-06 2024-09-18T11:31:23.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225062+0000", 2024-09-18T11:31:23.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8925000000000003e-05 2024-09-18T11:31:23.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.092 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225086+0000", 2024-09-18T11:31:23.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4737999999999999e-05 2024-09-18T11:31:23.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.092 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.092 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231022+0000", 2024-09-18T11:31:23.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059360690000000004 2024-09-18T11:31:23.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.092 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.092 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231069+0000", 2024-09-18T11:31:23.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6252999999999999e-05 2024-09-18T11:31:23.092 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.093 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.093 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.093 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2876 4.3 4:cc46532a:::smithi11939506-33:2f [read 0~3680733,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:23.093 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:31:23.093 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.49473611899999997, 2024-09-18T11:31:23.093 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068391210000000001, 2024-09-18T11:31:23.093 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.093 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:23.093 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.093 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.094 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.094 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2876 2024-09-18T11:31:23.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.094 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.094 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.094 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:31:23.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.094 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.094 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:31:23.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.095 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.095 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506860+0000", 2024-09-18T11:31:23.095 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:23.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.095 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.095 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506869+0000", 2024-09-18T11:31:23.095 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5539999999999998e-06 2024-09-18T11:31:23.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.095 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.095 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506880+0000", 2024-09-18T11:31:23.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1552999999999999e-05 2024-09-18T11:31:23.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.096 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506887+0000", 2024-09-18T11:31:23.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7329999999999997e-06 2024-09-18T11:31:23.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.096 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506937+0000", 2024-09-18T11:31:23.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0317e-05 2024-09-18T11:31:23.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:23.097 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506945+0000", 2024-09-18T11:31:23.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1550000000000007e-06 2024-09-18T11:31:23.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.097 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.509021+0000", 2024-09-18T11:31:23.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020759870000000001 2024-09-18T11:31:23.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.097 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.509123+0000", 2024-09-18T11:31:23.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010132900000000001 2024-09-18T11:31:23.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.098 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513700+0000", 2024-09-18T11:31:23.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0045777320000000002 2024-09-18T11:31:23.098 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.098 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.098 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.098 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2877 4.3 4:cc46532a:::smithi11939506-33:2f [read 0~3680733,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:23.098 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:31:23.098 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.49470170499999999, 2024-09-18T11:31:23.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085715170000000007, 2024-09-18T11:31:23.098 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.098 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:23.098 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.099 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.099 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.099 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2877 2024-09-18T11:31:23.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.099 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:31:23.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:31:23.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:31:23.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:23.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506898+0000", 2024-09-18T11:31:23.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4760000000000001e-06 2024-09-18T11:31:23.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506903+0000", 2024-09-18T11:31:23.101 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:23.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.101 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.101 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506906+0000", 2024-09-18T11:31:23.101 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1769999999999998e-06 2024-09-18T11:31:23.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.101 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.101 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507007+0000", 2024-09-18T11:31:23.101 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101479 2024-09-18T11:31:23.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.101 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:23.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507015+0000", 2024-09-18T11:31:23.102 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8490000000000006e-06 2024-09-18T11:31:23.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.102 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513757+0000", 2024-09-18T11:31:23.102 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0067417800000000002 2024-09-18T11:31:23.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.102 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513867+0000", 2024-09-18T11:31:23.102 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000109987 2024-09-18T11:31:23.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.103 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.103 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515467+0000", 2024-09-18T11:31:23.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00160037 2024-09-18T11:31:23.103 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.103 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.103 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.103 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2878 4.3 4:cc46532a:::smithi11939506-33:2f [sparse-read 0~1770976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:23.103 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:31:23.103 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.49467125000000001, 2024-09-18T11:31:23.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090602640000000002, 2024-09-18T11:31:23.103 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.103 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:23.104 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.104 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.104 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.104 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2878 2024-09-18T11:31:23.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.104 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:31:23.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:31:23.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.105 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.105 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:31:23.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:23.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.105 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.105 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506929+0000", 2024-09-18T11:31:23.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.163e-06 2024-09-18T11:31:23.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.105 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506936+0000", 2024-09-18T11:31:23.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5519999999999999e-06 2024-09-18T11:31:23.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506941+0000", 2024-09-18T11:31:23.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:23.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507036+0000", 2024-09-18T11:31:23.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5070999999999998e-05 2024-09-18T11:31:23.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:23.107 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507042+0000", 2024-09-18T11:31:23.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3959999999999997e-06 2024-09-18T11:31:23.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.107 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515504+0000", 2024-09-18T11:31:23.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084620300000000006 2024-09-18T11:31:23.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.107 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515625+0000", 2024-09-18T11:31:23.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012150999999999999 2024-09-18T11:31:23.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.108 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.108 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515986+0000", 2024-09-18T11:31:23.108 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036140100000000002 2024-09-18T11:31:23.108 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.108 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.108 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.108 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.108 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.108 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:23.109 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:23.007+0000 7fd27659f640 -1 osd.6 96 handle_osd_map injecting map crc failure 2024-09-18T11:31:23.111 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:31:23.111 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:23.016+0000 7f6bf3052640 -1 osd.5 96 handle_osd_map injecting map crc failure 2024-09-18T11:31:23.111 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:23.108+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:23.113 INFO:tasks.rados.rados.0.smithi119.stdout:1350: writing smithi11939506-37 from 786627 to 1582618 tid 1 2024-09-18T11:31:23.113 INFO:tasks.rados.rados.0.smithi119.stdout:1350: writing smithi11939506-37 from 2173212 to 2386627 tid 2 2024-09-18T11:31:23.114 INFO:tasks.rados.rados.0.smithi119.stdout:1347: done (3 left) 2024-09-18T11:31:23.114 INFO:tasks.rados.rados.0.smithi119.stdout:1348: done (2 left) 2024-09-18T11:31:23.114 INFO:tasks.rados.rados.0.smithi119.stdout:1349: done (1 left) 2024-09-18T11:31:23.114 INFO:tasks.rados.rados.0.smithi119.stdout:1351: tier_flush oid 1 2024-09-18T11:31:23.114 INFO:tasks.rados.rados.0.smithi119.stdout:1351: tier_flush oid 1 snap -1 2024-09-18T11:31:23.114 INFO:tasks.rados.rados.0.smithi119.stdout:1352: read oid 28 snap -1 2024-09-18T11:31:23.114 INFO:tasks.rados.rados.0.smithi119.stdout:1352: expect (ObjNum 200 snap 23 seq_num 200) 2024-09-18T11:31:23.114 INFO:tasks.rados.rados.0.smithi119.stdout:1353: delete oid 33 current snap is 31 2024-09-18T11:31:23.114 INFO:tasks.rados.rados.0.smithi119.stdout:1351: got (0) Success 2024-09-18T11:31:23.114 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 140 (ObjNum 242 snap 28 seq_num 242) dirty exists 2024-09-18T11:31:23.114 INFO:tasks.rados.rados.0.smithi119.stdout:1351: done (3 left) 2024-09-18T11:31:23.114 INFO:tasks.rados.rados.0.smithi119.stdout:1353: done (2 left) 2024-09-18T11:31:23.114 INFO:tasks.rados.rados.0.smithi119.stdout:1354: tier_promote oid 48 2024-09-18T11:31:23.114 INFO:tasks.rados.rados.0.smithi119.stdout:1355: read oid 29 snap 21 2024-09-18T11:31:23.114 INFO:tasks.rados.rados.0.smithi119.stdout:1355: expect (ObjNum 159 snap 20 seq_num 159) 2024-09-18T11:31:23.115 INFO:tasks.rados.rados.0.smithi119.stdout:1356: write oid 22 current snap is 31 2024-09-18T11:31:23.115 INFO:tasks.rados.rados.0.smithi119.stdout:1356: seq_num 263 ranges {420241=546489,1373823=544414} 2024-09-18T11:31:23.115 INFO:tasks.rados.rados.0.smithi119.stdout:1356: writing smithi11939506-22 from 420241 to 966730 tid 1 2024-09-18T11:31:23.115 INFO:tasks.rados.rados.0.smithi119.stdout:1356: writing smithi11939506-22 from 1373823 to 1918237 tid 2 2024-09-18T11:31:23.115 INFO:tasks.rados.rados.0.smithi119.stdout:1357: write oid 34 current snap is 31 2024-09-18T11:31:23.115 INFO:tasks.rados.rados.0.smithi119.stdout:1350: finishing write tid 1 to smithi11939506-37 2024-09-18T11:31:23.115 INFO:tasks.rados.rados.0.smithi119.stdout:1357: seq_num 264 ranges {437048=621738,1832305=404425} 2024-09-18T11:31:23.115 INFO:tasks.rados.rados.0.smithi119.stdout:1357: writing smithi11939506-34 from 437048 to 1058786 tid 1 2024-09-18T11:31:23.115 INFO:tasks.rados.rados.0.smithi119.stdout:1357: writing smithi11939506-34 from 1832305 to 2236730 tid 2 2024-09-18T11:31:23.115 INFO:tasks.rados.rados.0.smithi119.stdout:1358: read oid 7 snap 21 2024-09-18T11:31:23.115 INFO:tasks.rados.rados.0.smithi119.stdout:1358: expect (ObjNum 145 snap 18 seq_num 145) 2024-09-18T11:31:23.115 INFO:tasks.rados.rados.0.smithi119.stdout:1359: read oid 13 snap -1 2024-09-18T11:31:23.115 INFO:tasks.rados.rados.0.smithi119.stdout:1359: expect (ObjNum 207 snap 23 seq_num 207) 2024-09-18T11:31:23.115 INFO:tasks.rados.rados.0.smithi119.stdout:1360: tier_flush oid 19 2024-09-18T11:31:23.116 INFO:tasks.rados.rados.0.smithi119.stdout:1360: tier_flush oid 19 snap -1 2024-09-18T11:31:23.116 INFO:tasks.rados.rados.0.smithi119.stdout:1361: read oid 24 snap -1 2024-09-18T11:31:23.116 INFO:tasks.rados.rados.0.smithi119.stdout:1361: expect (ObjNum 258 snap 31 seq_num 258) 2024-09-18T11:31:23.116 INFO:tasks.rados.rados.0.smithi119.stdout:1362: read oid 43 snap 23 2024-09-18T11:31:23.116 INFO:tasks.rados.rados.0.smithi119.stdout:1362: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:31:23.116 INFO:tasks.rados.rados.0.smithi119.stdout:1350: finishing write tid 2 to smithi11939506-37 2024-09-18T11:31:23.116 INFO:tasks.rados.rados.0.smithi119.stdout:1350: finishing write tid 3 to smithi11939506-37 2024-09-18T11:31:23.116 INFO:tasks.rados.rados.0.smithi119.stdout:1355: expect (ObjNum 159 snap 20 seq_num 159) 2024-09-18T11:31:23.210 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:23.208+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:23.238 INFO:tasks.rados.rados.0.smithi119.stdout:1350: finishing write tid 4 to smithi11939506-37 2024-09-18T11:31:23.238 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 181 (ObjNum 261 snap 31 seq_num 261) dirty exists 2024-09-18T11:31:23.239 INFO:tasks.rados.rados.0.smithi119.stdout:1350: left oid 37 (ObjNum 261 snap 31 seq_num 261) 2024-09-18T11:31:23.239 INFO:tasks.rados.rados.0.smithi119.stdout:1354: got (0) Success 2024-09-18T11:31:23.239 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 48 v 126 (ObjNum 191 snap 21 seq_num 191) dirty exists 2024-09-18T11:31:23.239 INFO:tasks.rados.rados.0.smithi119.stdout:1363: read oid 49 snap -1 2024-09-18T11:31:23.239 INFO:tasks.rados.rados.0.smithi119.stdout:1363: expect (ObjNum 241 snap 28 seq_num 241) 2024-09-18T11:31:23.239 INFO:tasks.rados.rados.0.smithi119.stdout:1350: done (11 left) 2024-09-18T11:31:23.239 INFO:tasks.rados.rados.0.smithi119.stdout:1354: done (10 left) 2024-09-18T11:31:23.239 INFO:tasks.rados.rados.0.smithi119.stdout:1355: done (9 left) 2024-09-18T11:31:23.239 INFO:tasks.rados.rados.0.smithi119.stdout:1364: snap_remove snap 26 2024-09-18T11:31:23.249 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:23.249 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:23.249 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:23.249 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:23.258 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:31:23.342 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:23.340+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:23.396 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:23.396 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:23.396 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:23.397 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:23.397 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:23.405 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:31:23.446 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:23.444+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:23.512 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:23.511+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:23.546 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:23.546 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:23.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:23.546 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:23.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.546 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:23.546 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:23.546 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 72.427265394000003, 2024-09-18T11:31:23.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:31:23.547 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.547 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:23.547 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.547 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.547 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.547 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:31:23.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.547 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.547 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:23.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:23.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:31:23.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:23.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:31:23.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:31:23.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:31:23.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:31:23.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:31:23.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:31:23.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:31:23.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:31:23.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:23.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:31:23.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:31:23.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:31:23.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:31:23.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.551 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:31:23.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:31:23.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.551 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:31:23.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:31:23.551 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.551 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.551 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.552 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:31:23.552 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:23.552 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 62.874239993000003, 2024-09-18T11:31:23.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:31:23.552 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.552 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:23.552 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.552 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.552 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.552 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:31:23.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.552 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:23.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:23.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:31:23.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:23.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.554 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:31:23.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:31:23.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.554 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:31:23.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:31:23.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.554 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:31:23.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:31:23.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:31:23.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:31:23.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:31:23.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:31:23.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.556 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:31:23.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:31:23.556 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.556 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.556 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.557 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1688 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 19=[19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-18T11:31:23.557 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:23.557 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 44.011085399999999, 2024-09-18T11:31:23.557 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071575789999999999, 2024-09-18T11:31:23.557 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.557 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.557 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.557 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.557 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.557 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1688 2024-09-18T11:31:23.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.557 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.558 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:23.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.558 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:23.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.558 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:23.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:23.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532500+0000", 2024-09-18T11:31:23.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5260000000000001e-06 2024-09-18T11:31:23.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532503+0000", 2024-09-18T11:31:23.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:31:23.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532505+0000", 2024-09-18T11:31:23.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.889e-06 2024-09-18T11:31:23.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.534722+0000", 2024-09-18T11:31:23.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002217119 2024-09-18T11:31:23.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.534827+0000", 2024-09-18T11:31:23.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0001049 2024-09-18T11:31:23.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.561 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:23.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.535007+0000", 2024-09-18T11:31:23.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017965599999999999 2024-09-18T11:31:23.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.561 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:23.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539463+0000", 2024-09-18T11:31:23.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004455713 2024-09-18T11:31:23.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.561 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:23.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539607+0000", 2024-09-18T11:31:23.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014440300000000001 2024-09-18T11:31:23.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.562 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539634+0000", 2024-09-18T11:31:23.562 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7398000000000001e-05 2024-09-18T11:31:23.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.562 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539657+0000", 2024-09-18T11:31:23.562 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2228000000000001e-05 2024-09-18T11:31:23.562 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.562 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.563 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.563 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.563 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1760 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2319366,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:23.563 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:31:23.563 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 41.249590849999997, 2024-09-18T11:31:23.563 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070132529999999997, 2024-09-18T11:31:23.563 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.563 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:23.563 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.563 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.563 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.563 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1760 2024-09-18T11:31:23.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.564 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.564 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.564 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.564 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:31:23.564 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.564 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.564 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.564 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:31:23.564 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.564 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.564 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.565 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293993+0000", 2024-09-18T11:31:23.565 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:23.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.565 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.565 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.565 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293997+0000", 2024-09-18T11:31:23.565 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4999999999999999e-06 2024-09-18T11:31:23.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.565 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.565 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.565 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294004+0000", 2024-09-18T11:31:23.565 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3950000000000003e-06 2024-09-18T11:31:23.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.566 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.566 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294009+0000", 2024-09-18T11:31:23.566 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4989999999999996e-06 2024-09-18T11:31:23.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.566 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.566 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294103+0000", 2024-09-18T11:31:23.566 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4817999999999999e-05 2024-09-18T11:31:23.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.566 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:23.566 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294110+0000", 2024-09-18T11:31:23.566 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7240000000000003e-06 2024-09-18T11:31:23.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.567 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.567 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.567 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300406+0000", 2024-09-18T11:31:23.567 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062964470000000002 2024-09-18T11:31:23.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.567 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.567 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.567 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300457+0000", 2024-09-18T11:31:23.567 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0915000000000003e-05 2024-09-18T11:31:23.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.567 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.567 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.568 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.301007+0000", 2024-09-18T11:31:23.568 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00054942299999999995 2024-09-18T11:31:23.568 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.568 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.568 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.568 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.568 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1775 4.a 4:588debc2:::smithi11939506-18:head [write 1527054~766050] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:23.568 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:31:23.568 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.647238109, 2024-09-18T11:31:23.568 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070939050000000002, 2024-09-18T11:31:23.568 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.568 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.569 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.569 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.569 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.569 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1775 2024-09-18T11:31:23.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.569 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.569 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.569 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:31:23.569 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.569 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.570 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:31:23.570 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.570 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.570 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.570 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896345+0000", 2024-09-18T11:31:23.570 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:23.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.570 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.570 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.570 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896993+0000", 2024-09-18T11:31:23.570 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064769999999999997 2024-09-18T11:31:23.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.570 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.571 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.571 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897011+0000", 2024-09-18T11:31:23.571 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8355999999999998e-05 2024-09-18T11:31:23.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.571 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.571 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.571 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897016+0000", 2024-09-18T11:31:23.571 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3890000000000004e-06 2024-09-18T11:31:23.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.571 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.571 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.571 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897054+0000", 2024-09-18T11:31:23.571 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7963000000000002e-05 2024-09-18T11:31:23.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.572 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.572 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.572 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897123+0000", 2024-09-18T11:31:23.572 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8128999999999998e-05 2024-09-18T11:31:23.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.572 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.572 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:23.572 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897224+0000", 2024-09-18T11:31:23.572 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010168800000000001 2024-09-18T11:31:23.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.572 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.572 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:23.573 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899515+0000", 2024-09-18T11:31:23.573 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022910280000000001 2024-09-18T11:31:23.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.573 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.573 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:23.573 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903333+0000", 2024-09-18T11:31:23.573 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003817459 2024-09-18T11:31:23.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.573 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.573 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.573 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903388+0000", 2024-09-18T11:31:23.573 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5651e-05 2024-09-18T11:31:23.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.574 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.574 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.574 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903440+0000", 2024-09-18T11:31:23.574 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1836999999999999e-05 2024-09-18T11:31:23.574 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.574 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.574 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.574 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.574 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1776 4.a 4:588debc2:::smithi11939506-18:head [setxattr _header (57) in=7b,truncate 2420208] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:23.574 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:31:23.574 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.646560297000001, 2024-09-18T11:31:23.574 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065840400000000002, 2024-09-18T11:31:23.575 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.575 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.575 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.575 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.575 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.575 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1776 2024-09-18T11:31:23.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.575 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.575 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.575 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.575 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:31:23.575 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.575 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.576 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.576 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:31:23.576 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.576 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.576 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:31:23.576 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:23.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.576 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.576 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897026+0000", 2024-09-18T11:31:23.576 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9579999999999999e-06 2024-09-18T11:31:23.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.577 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.577 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897030+0000", 2024-09-18T11:31:23.577 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5370000000000002e-06 2024-09-18T11:31:23.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.577 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.577 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897032+0000", 2024-09-18T11:31:23.577 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.17e-06 2024-09-18T11:31:23.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.577 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.578 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898415+0000", 2024-09-18T11:31:23.578 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013823189999999999 2024-09-18T11:31:23.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.578 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.578 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.578 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898527+0000", 2024-09-18T11:31:23.578 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011236800000000001 2024-09-18T11:31:23.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.578 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.578 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:23.578 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898714+0000", 2024-09-18T11:31:23.578 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018703599999999999 2024-09-18T11:31:23.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.578 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.579 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:23.579 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899683+0000", 2024-09-18T11:31:23.579 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00096944700000000002 2024-09-18T11:31:23.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.579 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.579 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:23.579 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903475+0000", 2024-09-18T11:31:23.579 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037920129999999999 2024-09-18T11:31:23.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.579 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.579 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.579 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903535+0000", 2024-09-18T11:31:23.579 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9843999999999998e-05 2024-09-18T11:31:23.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.580 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.580 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.580 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903608+0000", 2024-09-18T11:31:23.580 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2946000000000003e-05 2024-09-18T11:31:23.580 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.580 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.580 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.580 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.580 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1777 4.a 4:588debc2:::smithi11939506-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:23.580 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:31:23.581 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.646533656999999, 2024-09-18T11:31:23.581 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0069010809999999999, 2024-09-18T11:31:23.581 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.581 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:23.581 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.581 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.581 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.581 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1777 2024-09-18T11:31:23.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.581 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.581 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.581 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.581 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:31:23.581 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.582 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.582 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.582 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:31:23.582 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.582 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.582 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.582 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897050+0000", 2024-09-18T11:31:23.582 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:23.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.582 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.582 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.583 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897052+0000", 2024-09-18T11:31:23.583 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:31:23.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.583 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.583 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.583 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897057+0000", 2024-09-18T11:31:23.583 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9470000000000002e-06 2024-09-18T11:31:23.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.583 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.583 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.583 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897060+0000", 2024-09-18T11:31:23.583 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3410000000000001e-06 2024-09-18T11:31:23.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.583 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.584 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.584 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899417+0000", 2024-09-18T11:31:23.584 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023571170000000002 2024-09-18T11:31:23.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.584 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.584 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:23.584 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899501+0000", 2024-09-18T11:31:23.584 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4160999999999996e-05 2024-09-18T11:31:23.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.584 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.584 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.585 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903661+0000", 2024-09-18T11:31:23.585 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041599510000000003 2024-09-18T11:31:23.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.585 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.585 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.585 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903790+0000", 2024-09-18T11:31:23.585 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012941499999999999 2024-09-18T11:31:23.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.585 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.585 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.585 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903952+0000", 2024-09-18T11:31:23.585 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016146800000000001 2024-09-18T11:31:23.585 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.585 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.586 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.586 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.586 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1895 4.5 4:a087a566:::smithi11939506-46:head [read 0~2756315,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:23.586 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:23.586 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.235278151999999, 2024-09-18T11:31:23.586 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007540377, 2024-09-18T11:31:23.586 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.586 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:23.586 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.586 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.586 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.586 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1895 2024-09-18T11:31:23.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.587 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.587 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.587 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:23.587 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.587 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.587 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:23.587 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.587 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.588 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:23.588 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:23.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.588 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.588 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.588 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308310+0000", 2024-09-18T11:31:23.588 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0559999999999998e-06 2024-09-18T11:31:23.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.588 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.588 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.588 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308317+0000", 2024-09-18T11:31:23.588 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7209999999999997e-06 2024-09-18T11:31:23.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.589 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.589 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308320+0000", 2024-09-18T11:31:23.589 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.721e-06 2024-09-18T11:31:23.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.589 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.589 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308447+0000", 2024-09-18T11:31:23.589 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012645199999999999 2024-09-18T11:31:23.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.589 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:23.589 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308451+0000", 2024-09-18T11:31:23.589 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6800000000000001e-06 2024-09-18T11:31:23.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.590 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.590 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.312462+0000", 2024-09-18T11:31:23.590 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040102139999999998 2024-09-18T11:31:23.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.590 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.590 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.312585+0000", 2024-09-18T11:31:23.590 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012388899999999999 2024-09-18T11:31:23.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.590 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.590 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.315847+0000", 2024-09-18T11:31:23.590 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003261197 2024-09-18T11:31:23.591 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.591 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.591 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.591 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.591 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1924 4.5 4:af4ba75a:::smithi11939506-37:head [write 1829449~675380] snapc 1e=[1e,1c,1b,19,18,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:23.591 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.362228+0000", 2024-09-18T11:31:23.591 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.181356452999999, 2024-09-18T11:31:23.591 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066182139999999999, 2024-09-18T11:31:23.591 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.591 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.591 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.591 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.591 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.592 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1924 2024-09-18T11:31:23.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.592 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.592 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.592 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.592 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362228+0000", 2024-09-18T11:31:23.592 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.592 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.592 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.592 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362228+0000", 2024-09-18T11:31:23.592 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.593 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.593 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.593 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362227+0000", 2024-09-18T11:31:23.593 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:23.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.593 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.593 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.593 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362749+0000", 2024-09-18T11:31:23.593 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00052262399999999996 2024-09-18T11:31:23.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.593 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.593 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.594 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362762+0000", 2024-09-18T11:31:23.594 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2821e-05 2024-09-18T11:31:23.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.594 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.594 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.594 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362767+0000", 2024-09-18T11:31:23.594 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5349999999999998e-06 2024-09-18T11:31:23.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.594 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.594 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.594 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362806+0000", 2024-09-18T11:31:23.594 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9029999999999997e-05 2024-09-18T11:31:23.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.595 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.595 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.595 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362863+0000", 2024-09-18T11:31:23.595 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7584999999999998e-05 2024-09-18T11:31:23.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.595 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.595 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:23.595 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362949+0000", 2024-09-18T11:31:23.595 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5655000000000006e-05 2024-09-18T11:31:23.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.595 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.595 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:23.595 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364314+0000", 2024-09-18T11:31:23.595 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001364961 2024-09-18T11:31:23.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.596 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.596 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:23.596 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.368771+0000", 2024-09-18T11:31:23.596 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004457244 2024-09-18T11:31:23.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.596 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.596 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.596 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.368809+0000", 2024-09-18T11:31:23.596 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8368999999999998e-05 2024-09-18T11:31:23.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.596 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.597 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.597 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.368846+0000", 2024-09-18T11:31:23.597 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6718000000000003e-05 2024-09-18T11:31:23.597 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.597 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.597 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.597 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.597 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1925 4.5 4:af4ba75a:::smithi11939506-37:head [setxattr _header (57) in=7b,truncate 2655506] snapc 1e=[1e,1c,1b,19,18,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:23.597 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:31:23.597 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.180811300999999, 2024-09-18T11:31:23.597 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064745779999999999, 2024-09-18T11:31:23.598 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.598 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.598 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.598 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.598 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.598 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1925 2024-09-18T11:31:23.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.598 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.598 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.598 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.598 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:31:23.598 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.598 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.599 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.599 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:31:23.599 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.599 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.599 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.599 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362773+0000", 2024-09-18T11:31:23.599 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:23.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.599 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.599 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.599 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362776+0000", 2024-09-18T11:31:23.599 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.208e-06 2024-09-18T11:31:23.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.600 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.600 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.600 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362780+0000", 2024-09-18T11:31:23.600 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.771e-06 2024-09-18T11:31:23.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.600 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.600 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.600 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362782+0000", 2024-09-18T11:31:23.600 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.277e-06 2024-09-18T11:31:23.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.600 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.600 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.601 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.363737+0000", 2024-09-18T11:31:23.601 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00095483999999999999 2024-09-18T11:31:23.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.601 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.601 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.601 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.363832+0000", 2024-09-18T11:31:23.601 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5594999999999995e-05 2024-09-18T11:31:23.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.601 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.601 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:23.601 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.363940+0000", 2024-09-18T11:31:23.601 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010738000000000001 2024-09-18T11:31:23.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.601 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.602 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:23.602 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364577+0000", 2024-09-18T11:31:23.602 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00063679600000000002 2024-09-18T11:31:23.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.602 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.602 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:23.602 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369126+0000", 2024-09-18T11:31:23.602 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0045496440000000003 2024-09-18T11:31:23.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.602 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.602 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.602 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369177+0000", 2024-09-18T11:31:23.602 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0958000000000002e-05 2024-09-18T11:31:23.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.603 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.603 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369248+0000", 2024-09-18T11:31:23.603 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0599000000000006e-05 2024-09-18T11:31:23.603 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.603 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.603 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.603 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1926 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:23.603 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:23.603 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.180787045000002, 2024-09-18T11:31:23.603 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066190219999999996, 2024-09-18T11:31:23.604 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.604 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:23.604 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.604 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.604 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.604 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1926 2024-09-18T11:31:23.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.604 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.604 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.604 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:23.604 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.605 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.605 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:23.605 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.605 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.605 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:23.605 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:23.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.606 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362799+0000", 2024-09-18T11:31:23.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.029e-06 2024-09-18T11:31:23.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.606 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362804+0000", 2024-09-18T11:31:23.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3989999999999997e-06 2024-09-18T11:31:23.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.606 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362806+0000", 2024-09-18T11:31:23.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3020000000000002e-06 2024-09-18T11:31:23.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.607 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364105+0000", 2024-09-18T11:31:23.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001299478 2024-09-18T11:31:23.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.607 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:23.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364153+0000", 2024-09-18T11:31:23.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8059000000000002e-05 2024-09-18T11:31:23.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.608 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.608 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369273+0000", 2024-09-18T11:31:23.608 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051199540000000003 2024-09-18T11:31:23.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.608 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.608 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369343+0000", 2024-09-18T11:31:23.608 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0191999999999995e-05 2024-09-18T11:31:23.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.608 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.608 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369416+0000", 2024-09-18T11:31:23.608 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2925999999999999e-05 2024-09-18T11:31:23.609 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.609 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.609 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.609 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1969 4.a 4:588debc2:::smithi11939506-18:18 [read 0~3949612,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:23.609 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:23.609 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.169598155999999, 2024-09-18T11:31:23.609 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075409190000000001, 2024-09-18T11:31:23.609 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.609 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:23.609 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.609 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.610 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.610 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1969 2024-09-18T11:31:23.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.610 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.610 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.610 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.610 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:23.610 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.610 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.610 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.610 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:23.610 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:23.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:23.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373990+0000", 2024-09-18T11:31:23.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1119999999999998e-06 2024-09-18T11:31:23.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373997+0000", 2024-09-18T11:31:23.612 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4200000000000001e-06 2024-09-18T11:31:23.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.612 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.374002+0000", 2024-09-18T11:31:23.612 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:31:23.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.612 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380200+0000", 2024-09-18T11:31:23.612 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006197302 2024-09-18T11:31:23.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.613 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.613 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380261+0000", 2024-09-18T11:31:23.613 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1389999999999993e-05 2024-09-18T11:31:23.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.613 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.613 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.381527+0000", 2024-09-18T11:31:23.613 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012660080000000001 2024-09-18T11:31:23.613 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.613 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.613 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.614 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2207 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1410296~761389] snapc 24=[24,1e,1c,1b,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T11:31:23.614 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:23.614 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.121087330000002, 2024-09-18T11:31:23.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008410743, 2024-09-18T11:31:23.614 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.614 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.614 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.614 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.614 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.614 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2207 2024-09-18T11:31:23.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.614 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.614 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.615 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:23.615 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.615 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.615 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:23.615 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.615 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.615 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422496+0000", 2024-09-18T11:31:23.615 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:23.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.616 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423190+0000", 2024-09-18T11:31:23.616 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00069405499999999995 2024-09-18T11:31:23.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.616 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423204+0000", 2024-09-18T11:31:23.616 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:31:23.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.616 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423208+0000", 2024-09-18T11:31:23.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:31:23.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.617 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423293+0000", 2024-09-18T11:31:23.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4524999999999995e-05 2024-09-18T11:31:23.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.617 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423394+0000", 2024-09-18T11:31:23.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101395 2024-09-18T11:31:23.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.617 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:23.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423579+0000", 2024-09-18T11:31:23.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018461000000000001 2024-09-18T11:31:23.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:23.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.428574+0000", 2024-09-18T11:31:23.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049944960000000002 2024-09-18T11:31:23.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:23.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430809+0000", 2024-09-18T11:31:23.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022351820000000001 2024-09-18T11:31:23.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430862+0000", 2024-09-18T11:31:23.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3189999999999997e-05 2024-09-18T11:31:23.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.619 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430908+0000", 2024-09-18T11:31:23.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5973999999999997e-05 2024-09-18T11:31:23.619 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.619 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.619 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.619 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2339 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:23.620 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:23.620 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.156782701000001, 2024-09-18T11:31:23.620 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011809992, 2024-09-18T11:31:23.620 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.620 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:23.620 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.620 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.620 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.620 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2339 2024-09-18T11:31:23.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.620 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:23.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:23.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:31:23.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:23.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:31:23.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4710000000000001e-06 2024-09-18T11:31:23.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386824+0000", 2024-09-18T11:31:23.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6104e-05 2024-09-18T11:31:23.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.623 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386836+0000", 2024-09-18T11:31:23.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1827999999999999e-05 2024-09-18T11:31:23.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.623 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388390+0000", 2024-09-18T11:31:23.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015539499999999999 2024-09-18T11:31:23.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.624 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:23.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388443+0000", 2024-09-18T11:31:23.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3785000000000001e-05 2024-09-18T11:31:23.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.624 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398238+0000", 2024-09-18T11:31:23.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097947989999999999 2024-09-18T11:31:23.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.624 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398315+0000", 2024-09-18T11:31:23.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6909999999999994e-05 2024-09-18T11:31:23.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.625 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.625 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398612+0000", 2024-09-18T11:31:23.625 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029671999999999999 2024-09-18T11:31:23.625 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.625 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.625 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.625 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2340 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:23.625 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:23.625 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.156739388999998, 2024-09-18T11:31:23.625 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012086448, 2024-09-18T11:31:23.626 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.626 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:23.626 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.626 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.626 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.626 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2340 2024-09-18T11:31:23.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.626 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:23.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.628 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:23.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.628 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:23.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:23.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386848+0000", 2024-09-18T11:31:23.629 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3069999999999998e-06 2024-09-18T11:31:23.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386855+0000", 2024-09-18T11:31:23.629 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0979999999999996e-06 2024-09-18T11:31:23.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386859+0000", 2024-09-18T11:31:23.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9360000000000001e-06 2024-09-18T11:31:23.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388466+0000", 2024-09-18T11:31:23.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001607154 2024-09-18T11:31:23.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:23.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388507+0000", 2024-09-18T11:31:23.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1093000000000001e-05 2024-09-18T11:31:23.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398647+0000", 2024-09-18T11:31:23.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010139686 2024-09-18T11:31:23.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398718+0000", 2024-09-18T11:31:23.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1138999999999998e-05 2024-09-18T11:31:23.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398931+0000", 2024-09-18T11:31:23.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021356899999999999 2024-09-18T11:31:23.631 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2341 4.5 4:aa575018:::smithi11939506-11:head [read 0~2487436,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.156721167000001, 2024-09-18T11:31:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013751648999999999, 2024-09-18T11:31:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.632 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.633 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2341 2024-09-18T11:31:23.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.633 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:23.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:23.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:23.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:23.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386866+0000", 2024-09-18T11:31:23.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4249999999999998e-06 2024-09-18T11:31:23.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386872+0000", 2024-09-18T11:31:23.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.003e-06 2024-09-18T11:31:23.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.635 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:31:23.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.507e-06 2024-09-18T11:31:23.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.635 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388520+0000", 2024-09-18T11:31:23.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001644146 2024-09-18T11:31:23.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.636 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:23.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388561+0000", 2024-09-18T11:31:23.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1486999999999998e-05 2024-09-18T11:31:23.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.636 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398963+0000", 2024-09-18T11:31:23.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010401353 2024-09-18T11:31:23.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.636 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.399046+0000", 2024-09-18T11:31:23.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3141999999999998e-05 2024-09-18T11:31:23.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.400615+0000", 2024-09-18T11:31:23.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015690960000000001 2024-09-18T11:31:23.637 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.637 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.637 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.637 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2529 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:23.637 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:31:23.637 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.370247902999999, 2024-09-18T11:31:23.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079268120000000001, 2024-09-18T11:31:23.638 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.638 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.638 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.638 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.638 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.638 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2529 2024-09-18T11:31:23.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.638 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:31:23.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.639 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:31:23.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.639 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173336+0000", 2024-09-18T11:31:23.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:23.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.639 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173339+0000", 2024-09-18T11:31:23.640 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9019999999999999e-06 2024-09-18T11:31:23.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.640 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.640 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173381+0000", 2024-09-18T11:31:23.640 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1836999999999999e-05 2024-09-18T11:31:23.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.640 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.640 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173390+0000", 2024-09-18T11:31:23.640 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.302e-06 2024-09-18T11:31:23.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.641 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.641 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174507+0000", 2024-09-18T11:31:23.641 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001116868 2024-09-18T11:31:23.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.641 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.641 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.641 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174603+0000", 2024-09-18T11:31:23.641 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5598000000000002e-05 2024-09-18T11:31:23.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.641 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.641 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:23.641 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174759+0000", 2024-09-18T11:31:23.641 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015616299999999999 2024-09-18T11:31:23.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:23.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178943+0000", 2024-09-18T11:31:23.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041846180000000002 2024-09-18T11:31:23.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:23.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181175+0000", 2024-09-18T11:31:23.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00223215 2024-09-18T11:31:23.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181229+0000", 2024-09-18T11:31:23.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3585999999999998e-05 2024-09-18T11:31:23.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181263+0000", 2024-09-18T11:31:23.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4292999999999998e-05 2024-09-18T11:31:23.643 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.643 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.643 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.643 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2530 4.5 4:af4ba75a:::smithi11939506-37:head [rollback 25] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:23.643 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:31:23.644 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.370185597000001, 2024-09-18T11:31:23.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080198800000000001, 2024-09-18T11:31:23.644 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.644 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.644 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.644 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.644 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.644 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2530 2024-09-18T11:31:23.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.644 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.644 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:31:23.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.645 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:31:23.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.645 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173398+0000", 2024-09-18T11:31:23.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:23.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.646 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173403+0000", 2024-09-18T11:31:23.646 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4389999999999999e-06 2024-09-18T11:31:23.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.646 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173414+0000", 2024-09-18T11:31:23.646 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0411000000000001e-05 2024-09-18T11:31:23.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.646 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173420+0000", 2024-09-18T11:31:23.646 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9610000000000002e-06 2024-09-18T11:31:23.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176390+0000", 2024-09-18T11:31:23.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029706289999999998 2024-09-18T11:31:23.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176501+0000", 2024-09-18T11:31:23.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011090300000000001 2024-09-18T11:31:23.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:23.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176827+0000", 2024-09-18T11:31:23.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00032526899999999999 2024-09-18T11:31:23.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:23.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178958+0000", 2024-09-18T11:31:23.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021314860000000001 2024-09-18T11:31:23.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:23.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181318+0000", 2024-09-18T11:31:23.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002359571 2024-09-18T11:31:23.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181372+0000", 2024-09-18T11:31:23.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3884000000000001e-05 2024-09-18T11:31:23.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.649 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181419+0000", 2024-09-18T11:31:23.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7148000000000003e-05 2024-09-18T11:31:23.649 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.649 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.649 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.650 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2531 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:23.650 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:23.650 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.370159473999999, 2024-09-18T11:31:23.650 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080687239999999993, 2024-09-18T11:31:23.650 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.650 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.650 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.650 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.650 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.650 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2531 2024-09-18T11:31:23.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.650 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.650 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.651 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:23.651 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.651 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.651 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:23.651 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.651 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.651 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:23.651 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:23.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173429+0000", 2024-09-18T11:31:23.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2989999999999998e-06 2024-09-18T11:31:23.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173436+0000", 2024-09-18T11:31:23.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1799999999999999e-06 2024-09-18T11:31:23.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173441+0000", 2024-09-18T11:31:23.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8810000000000004e-06 2024-09-18T11:31:23.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178267+0000", 2024-09-18T11:31:23.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048257969999999997 2024-09-18T11:31:23.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178388+0000", 2024-09-18T11:31:23.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121779 2024-09-18T11:31:23.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:23.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178599+0000", 2024-09-18T11:31:23.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000210094 2024-09-18T11:31:23.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:23.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.179339+0000", 2024-09-18T11:31:23.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00074063799999999995 2024-09-18T11:31:23.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:23.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181444+0000", 2024-09-18T11:31:23.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021044100000000001 2024-09-18T11:31:23.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181470+0000", 2024-09-18T11:31:23.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6415999999999999e-05 2024-09-18T11:31:23.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181494+0000", 2024-09-18T11:31:23.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3672e-05 2024-09-18T11:31:23.655 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.655 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.656 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.656 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2910 4.5 4:af4ba75a:::smithi11939506-37:head [write 786627~795991,stat] snapc 37=[37,35,34,30,2f,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:23.656 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:31:23.656 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.54010164500000002, 2024-09-18T11:31:23.656 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084093710000000006, 2024-09-18T11:31:23.656 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:23.656 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:23.656 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:23.656 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:23.656 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:23.657 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2910 2024-09-18T11:31:23.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.657 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:23.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.657 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:23.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:31:23.657 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.657 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:23.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:31:23.657 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:23.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:23.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003481+0000", 2024-09-18T11:31:23.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:23.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:23.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004300+0000", 2024-09-18T11:31:23.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081934200000000003 2024-09-18T11:31:23.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:23.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004324+0000", 2024-09-18T11:31:23.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4618999999999999e-05 2024-09-18T11:31:23.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:23.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004335+0000", 2024-09-18T11:31:23.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.092e-05 2024-09-18T11:31:23.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:23.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004385+0000", 2024-09-18T11:31:23.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9709999999999997e-05 2024-09-18T11:31:23.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:23.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004470+0000", 2024-09-18T11:31:23.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4864000000000006e-05 2024-09-18T11:31:23.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:23.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004612+0000", 2024-09-18T11:31:23.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014241500000000001 2024-09-18T11:31:23.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:23.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011193+0000", 2024-09-18T11:31:23.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006580654 2024-09-18T11:31:23.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:23.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011788+0000", 2024-09-18T11:31:23.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059533600000000002 2024-09-18T11:31:23.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:23.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011842+0000", 2024-09-18T11:31:23.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3180000000000002e-05 2024-09-18T11:31:23.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:23.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:23.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:23.662 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011892+0000", 2024-09-18T11:31:23.662 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0618000000000002e-05 2024-09-18T11:31:23.662 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.662 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.662 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.662 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:23.662 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:23.662 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:23.664 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:31:23.664 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:23.662+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:23.764 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:23.762+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:23.803 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:23.803 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:23.803 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:23.803 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:23.811 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:31:23.865 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:23.864+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:23.950 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:23.950 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:23.950 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:23.950 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:23.950 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:23.958 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:31:23.966 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:23.965+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:24.007 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:24.005+0000 7f3157e90640 -1 osd.4 97 handle_osd_map injecting map crc failure 2024-09-18T11:31:24.008 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:24.007+0000 7f2d449e8640 -1 osd.1 97 handle_osd_map injecting map crc failure 2024-09-18T11:31:24.010 INFO:tasks.rados.rados.0.smithi119.stdout:1364: done (9 left) 2024-09-18T11:31:24.010 INFO:tasks.rados.rados.0.smithi119.stdout:1365: read oid 21 snap -1 2024-09-18T11:31:24.010 INFO:tasks.rados.rados.0.smithi119.stdout:1365: expect (ObjNum 253 snap 30 seq_num 253) 2024-09-18T11:31:24.010 INFO:tasks.rados.rados.0.smithi119.stdout:1366: set_chunk oid 18 target oid 2024-09-18T11:31:24.028 INFO:tasks.rados.rados.0.smithi119.stdout:1366: set_chunk oid 18 offset: 1750528 length: 1191936 target oid fd10397097f756bcac69a1c11efb1ed33e24224b2fb2c2d68353117e80840ef3 offset: 0 snap -1 2024-09-18T11:31:24.028 INFO:tasks.rados.rados.0.smithi119.stdout:1367: tier_promote oid 15 2024-09-18T11:31:24.028 INFO:tasks.rados.rados.0.smithi119.stdout:1368: read oid 27 snap -1 2024-09-18T11:31:24.028 INFO:tasks.rados.rados.0.smithi119.stdout:1368: expect (ObjNum 244 snap 28 seq_num 244) 2024-09-18T11:31:24.028 INFO:tasks.rados.rados.0.smithi119.stdout:1369: read oid 50 snap -1 2024-09-18T11:31:24.028 INFO:tasks.rados.rados.0.smithi119.stdout:1369: expect (ObjNum 183 snap 20 seq_num 183) 2024-09-18T11:31:24.028 INFO:tasks.rados.rados.0.smithi119.stdout:1370: read oid 40 snap -1 2024-09-18T11:31:24.028 INFO:tasks.rados.rados.0.smithi119.stdout:1370: expect (ObjNum 175 snap 20 seq_num 175) 2024-09-18T11:31:24.028 INFO:tasks.rados.rados.0.smithi119.stdout:1371: write (excl) oid 1 current snap is 31 2024-09-18T11:31:24.029 INFO:tasks.rados.rados.0.smithi119.stdout:1371: seq_num 265 ranges {453855=696987,1890787=548083} 2024-09-18T11:31:24.033 INFO:tasks.rados.rados.0.smithi119.stdout:1371: writing smithi11939506-1 from 453855 to 1150842 tid 1 2024-09-18T11:31:24.041 INFO:tasks.rados.rados.0.smithi119.stdout:1371: writing smithi11939506-1 from 1890787 to 2438870 tid 2 2024-09-18T11:31:24.041 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:31:24.041 INFO:tasks.rados.rados.0.smithi119.stdout:1352: expect (ObjNum 200 snap 23 seq_num 200) 2024-09-18T11:31:24.067 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:24.068 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:24.068 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:24.068 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:24.068 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.068 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:24.068 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:24.068 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.907396042000002, 2024-09-18T11:31:24.068 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055400850000000001, 2024-09-18T11:31:24.068 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:24.068 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:24.068 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:24.068 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:24.069 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:24.069 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 43 2024-09-18T11:31:24.069 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.069 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:24.069 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.069 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:24.069 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:24.069 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.069 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.069 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.069 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:24.069 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:24.069 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157504+0000", 2024-09-18T11:31:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4519999999999996e-06 2024-09-18T11:31:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:24.070 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157534+0000", 2024-09-18T11:31:24.071 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.97e-05 2024-09-18T11:31:24.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.071 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.071 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:24.071 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157539+0000", 2024-09-18T11:31:24.071 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4360000000000001e-06 2024-09-18T11:31:24.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.071 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.071 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:24.071 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159428+0000", 2024-09-18T11:31:24.071 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018888220000000001 2024-09-18T11:31:24.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.071 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.071 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:24.072 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159479+0000", 2024-09-18T11:31:24.072 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0608000000000001e-05 2024-09-18T11:31:24.072 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.072 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.072 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:24.072 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159559+0000", 2024-09-18T11:31:24.072 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0667000000000005e-05 2024-09-18T11:31:24.072 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.072 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.072 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:24.073 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162779+0000", 2024-09-18T11:31:24.073 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032197459999999999 2024-09-18T11:31:24.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.073 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.073 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:24.073 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162950+0000", 2024-09-18T11:31:24.073 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017059599999999999 2024-09-18T11:31:24.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.074 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:24.074 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163001+0000", 2024-09-18T11:31:24.074 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0968999999999999e-05 2024-09-18T11:31:24.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.074 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:24.075 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163040+0000", 2024-09-18T11:31:24.075 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8843000000000001e-05 2024-09-18T11:31:24.075 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.075 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:24.075 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.075 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:24.075 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:24.075 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.907344887999997, 2024-09-18T11:31:24.075 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:31:24.075 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:24.075 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:24.076 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:24.076 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:24.076 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:24.076 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:31:24.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.076 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:24.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.076 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:24.076 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:24.076 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.076 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:24.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:24.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.077 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:24.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:31:24.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:24.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.077 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:24.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:31:24.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:31:24.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:24.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:31:24.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:31:24.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:24.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:31:24.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:24.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:24.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:31:24.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:31:24.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.079 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:24.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:31:24.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:31:24.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.079 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:24.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:31:24.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:31:24.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:24.080 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:31:24.080 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:31:24.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:24.080 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:31:24.080 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:31:24.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:24.080 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:31:24.080 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:31:24.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.081 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:24.081 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:31:24.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:31:24.081 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.081 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:24.081 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.081 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:24.081 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:24.081 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.907218626000002, 2024-09-18T11:31:24.082 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055717040000000002, 2024-09-18T11:31:24.082 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:24.082 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:24.082 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:24.082 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:24.082 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:24.082 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 45 2024-09-18T11:31:24.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.082 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:24.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.082 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:24.082 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:24.082 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.083 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:24.083 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:24.083 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.083 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:24.083 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157676+0000", 2024-09-18T11:31:24.083 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:24.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.084 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:24.084 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157694+0000", 2024-09-18T11:31:24.084 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7422999999999999e-05 2024-09-18T11:31:24.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.084 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:24.084 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157704+0000", 2024-09-18T11:31:24.084 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9000000000000001e-06 2024-09-18T11:31:24.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:24.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157718+0000", 2024-09-18T11:31:24.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4788e-05 2024-09-18T11:31:24.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:24.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160937+0000", 2024-09-18T11:31:24.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00321886 2024-09-18T11:31:24.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:24.086 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161023+0000", 2024-09-18T11:31:24.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6063999999999995e-05 2024-09-18T11:31:24.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.086 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:24.086 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161202+0000", 2024-09-18T11:31:24.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017912599999999999 2024-09-18T11:31:24.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.086 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:24.086 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163052+0000", 2024-09-18T11:31:24.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001849554 2024-09-18T11:31:24.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.087 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:24.087 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163204+0000", 2024-09-18T11:31:24.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015186900000000001 2024-09-18T11:31:24.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.087 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:24.087 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163229+0000", 2024-09-18T11:31:24.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:31:24.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.087 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:24.087 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163249+0000", 2024-09-18T11:31:24.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9992000000000001e-05 2024-09-18T11:31:24.088 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.088 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:24.088 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.088 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:24.088 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:24.088 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.907169512999999, 2024-09-18T11:31:24.088 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:31:24.088 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:24.088 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:24.088 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:24.088 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:24.089 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:24.089 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:31:24.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.089 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:24.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.089 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:24.089 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:24.089 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.089 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:24.089 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:24.089 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:24.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:24.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:24.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:24.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:31:24.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:31:24.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:24.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:31:24.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:31:24.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:24.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:31:24.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:31:24.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:24.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:31:24.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:31:24.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:24.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:31:24.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:31:24.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.092 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:24.092 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:31:24.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:31:24.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.092 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:24.092 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:31:24.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:31:24.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.093 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:24.093 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:31:24.093 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:31:24.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.093 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:24.093 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:31:24.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:31:24.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.094 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:24.094 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:31:24.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:31:24.094 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.094 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:24.094 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.095 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:24.095 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:24.095 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.907126981999994, 2024-09-18T11:31:24.095 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:31:24.095 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:24.095 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:24.095 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:24.095 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:24.095 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:24.096 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:31:24.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.096 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:24.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:24.096 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:24.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:24.096 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:24.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:24.097 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:31:24.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:24.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:24.097 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:31:24.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:31:24.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:24.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:31:24.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:31:24.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.098 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:24.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:31:24.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:31:24.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.098 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:24.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:31:24.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:31:24.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:24.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:31:24.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:31:24.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:24.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:31:24.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:31:24.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:24.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:31:24.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:31:24.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:24.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:31:24.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:31:24.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:24.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:31:24.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:31:24.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:24.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:31:24.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:31:24.101 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.101 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:24.101 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.101 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:24.101 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:24.101 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.907072052000004, 2024-09-18T11:31:24.101 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:31:24.101 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:24.101 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:24.101 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:24.101 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:24.101 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:24.102 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:31:24.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.102 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:24.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.102 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:24.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:24.102 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.102 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:24.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:24.102 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.103 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:24.103 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:24.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:24.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.103 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:24.103 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:31:24.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:31:24.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.103 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:24.103 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:31:24.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:31:24.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:24.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:31:24.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:31:24.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:24.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:31:24.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:31:24.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.105 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:24.105 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:31:24.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:31:24.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.105 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:24.105 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:31:24.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:31:24.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:24.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:31:24.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:31:24.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:24.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:31:24.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:31:24.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:24.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:31:24.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:31:24.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:24.107 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:31:24.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:31:24.107 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.107 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:24.107 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.107 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:24.107 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:24.107 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.907062072000002, 2024-09-18T11:31:24.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:31:24.108 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:24.108 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:24.108 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:24.108 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:24.108 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:24.108 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:31:24.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.108 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:24.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.108 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:24.108 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:24.108 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.109 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:24.109 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:24.109 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.109 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:24.109 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:24.109 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:24.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.109 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:24.110 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:31:24.110 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:31:24.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.110 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:24.110 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:31:24.110 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:31:24.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.110 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:24.110 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:31:24.110 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:31:24.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.110 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:24.111 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:31:24.111 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:31:24.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.111 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:24.111 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:31:24.111 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:31:24.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.111 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:24.111 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:31:24.111 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:31:24.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.112 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:24.112 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:31:24.112 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:31:24.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.112 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:24.112 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:31:24.112 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:31:24.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.113 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:24.113 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:31:24.113 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:31:24.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.113 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:24.113 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:31:24.113 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:31:24.113 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.113 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:24.113 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.114 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:24.114 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:24.114 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.907050472999998, 2024-09-18T11:31:24.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:31:24.114 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:24.114 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:24.114 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:24.114 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:24.115 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:24.115 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:31:24.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.115 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:24.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:24.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:24.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:24.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:24.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:24.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:24.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:24.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:24.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:31:24.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:31:24.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:24.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:31:24.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:31:24.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:24.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:31:24.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:31:24.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.118 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:24.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:31:24.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:31:24.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.118 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:24.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:31:24.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:31:24.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.119 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:24.119 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:31:24.119 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:31:24.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.119 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:24.119 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:31:24.119 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:31:24.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.119 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:24.119 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:31:24.119 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:31:24.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:24.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:31:24.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:31:24.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:24.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:31:24.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:31:24.120 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.120 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:24.120 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.121 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:24.121 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:24.121 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.906993991999997, 2024-09-18T11:31:24.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:31:24.121 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:24.121 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:24.121 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:24.121 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:24.121 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:24.121 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:31:24.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.121 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:24.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.122 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:24.122 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:24.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.122 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:24.122 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:24.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.122 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:24.122 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:31:24.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:24.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:24.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:31:24.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:31:24.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:24.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:31:24.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:31:24.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:24.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:31:24.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:31:24.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.124 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:24.124 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:31:24.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:31:24.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.124 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:24.124 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:31:24.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:31:24.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.124 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:24.125 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:31:24.125 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:31:24.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.125 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:24.125 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:31:24.125 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:31:24.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.126 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:24.126 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:31:24.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:31:24.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.126 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:24.126 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:31:24.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:31:24.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.126 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:24.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:31:24.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:31:24.127 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.127 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:24.127 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.127 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:24.127 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:24.127 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.906980168000004, 2024-09-18T11:31:24.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:31:24.127 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:24.127 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:24.127 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:24.128 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:24.128 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:24.128 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:31:24.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.128 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:24.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:24.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:24.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:24.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:24.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:24.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:24.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:24.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:24.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:31:24.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:31:24.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:24.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:31:24.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:31:24.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.130 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:24.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:31:24.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:31:24.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.130 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:24.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:31:24.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:31:24.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.130 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:24.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:31:24.131 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:31:24.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.131 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:24.131 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:31:24.131 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:31:24.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.131 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:24.131 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:31:24.131 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:31:24.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.131 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:24.131 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:31:24.132 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:31:24.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.132 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:24.132 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:31:24.132 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:31:24.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.132 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:24.132 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:31:24.132 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:31:24.132 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.132 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:24.132 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.133 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.133 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:24.133 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:24.133 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.490801261000001, 2024-09-18T11:31:24.133 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:31:24.133 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:24.133 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:24.133 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:24.133 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:24.133 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:24.133 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:31:24.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.134 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:24.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:24.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:24.134 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:24.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:24.134 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:24.135 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:24.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:24.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.135 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:24.135 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:31:24.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:31:24.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.135 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:24.135 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:31:24.136 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:31:24.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.136 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:24.136 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:31:24.136 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:31:24.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.136 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:24.136 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:31:24.137 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:31:24.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.137 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.137 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:24.137 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:31:24.137 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:31:24.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.137 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.137 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:24.137 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:31:24.137 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:31:24.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.138 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:24.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:31:24.138 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:31:24.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.138 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:24.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:31:24.138 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:31:24.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.138 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:24.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:31:24.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:31:24.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.139 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:24.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:31:24.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:31:24.139 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.139 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:24.139 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.139 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:24.139 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:24.139 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.490777961000006, 2024-09-18T11:31:24.140 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:31:24.140 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:24.140 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:24.140 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:24.140 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:24.140 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:24.140 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:31:24.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.140 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:24.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.140 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:24.140 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:24.140 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.141 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:24.141 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:24.141 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.141 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:24.141 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:31:24.141 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:24.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.141 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:24.141 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:31:24.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:31:24.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.142 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:24.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:31:24.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:31:24.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.142 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:24.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:31:24.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:31:24.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.142 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:24.143 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:31:24.143 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:31:24.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.143 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:24.143 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:31:24.143 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:31:24.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.143 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:24.143 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:31:24.143 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:31:24.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.144 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:24.144 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:31:24.144 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:31:24.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.144 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:24.144 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:31:24.144 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:31:24.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.144 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:24.144 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:31:24.145 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:31:24.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.145 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:24.145 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:31:24.145 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:31:24.145 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.145 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:24.145 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.145 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:24.145 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:24.146 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.490692351999996, 2024-09-18T11:31:24.146 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:31:24.146 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:24.146 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:24.146 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:24.146 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:24.146 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:24.146 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:31:24.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.146 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:24.146 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.146 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:24.146 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:24.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.147 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:24.147 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:24.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.147 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:24.147 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:24.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:24.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:24.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:31:24.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:31:24.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:24.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:31:24.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:31:24.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:24.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:31:24.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:31:24.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.149 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:24.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:31:24.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:31:24.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.149 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:24.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:31:24.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:31:24.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.149 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:24.150 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:31:24.150 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:31:24.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.150 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.150 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:24.150 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:31:24.150 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:31:24.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.150 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.150 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:24.150 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:31:24.150 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:31:24.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.150 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.150 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:24.151 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:31:24.151 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:31:24.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.151 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.151 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:24.151 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:31:24.151 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:31:24.151 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.151 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:24.151 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.151 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.151 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:24.151 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:24.151 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.490681386999995, 2024-09-18T11:31:24.152 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:31:24.152 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:24.152 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:24.152 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:24.152 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:24.152 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:24.152 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:31:24.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.152 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:24.152 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.152 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:24.152 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:24.152 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.153 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:24.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:24.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.153 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:24.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:24.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:24.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.153 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:24.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:31:24.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:31:24.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:24.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:31:24.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:31:24.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:24.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:31:24.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:31:24.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.155 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:24.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:31:24.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:31:24.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.155 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:24.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:31:24.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:31:24.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.155 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:24.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:31:24.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:31:24.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.156 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:24.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:31:24.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:31:24.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.156 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:24.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:31:24.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:31:24.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.156 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:24.157 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:31:24.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:31:24.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.157 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:24.157 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:31:24.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:31:24.157 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.157 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:24.157 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.157 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:24.158 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:24.158 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.490670218999995, 2024-09-18T11:31:24.158 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:31:24.158 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:24.158 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:24.158 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:24.158 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:24.158 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:24.158 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:31:24.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.158 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:24.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.158 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:24.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:24.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.159 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:24.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:24.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.159 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:24.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:24.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:24.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.160 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:24.160 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:31:24.160 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:31:24.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.160 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:24.160 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:31:24.160 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:31:24.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.160 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:24.160 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:31:24.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:31:24.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:24.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:31:24.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:31:24.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:24.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:31:24.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:31:24.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:24.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:31:24.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:31:24.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:24.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:31:24.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:31:24.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:24.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:31:24.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:31:24.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.163 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:24.163 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:31:24.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:31:24.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.163 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:24.163 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:31:24.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:31:24.163 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.163 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:24.163 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.164 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:24.164 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:24.164 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.49060489, 2024-09-18T11:31:24.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:31:24.164 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:24.164 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:24.164 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:24.164 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:24.164 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:24.164 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:31:24.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.164 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:24.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:24.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:24.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:24.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:24.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:24.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:31:24.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:24.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:24.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:31:24.166 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:31:24.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:24.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:31:24.166 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:31:24.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:24.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:31:24.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:31:24.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:24.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:31:24.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:31:24.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:24.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:31:24.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:31:24.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:24.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:31:24.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:31:24.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:24.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:31:24.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:31:24.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:24.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:31:24.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:31:24.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:24.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:31:24.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:31:24.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:24.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:31:24.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:31:24.169 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.169 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:24.169 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.170 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:24.170 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:24.170 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.480442912000001, 2024-09-18T11:31:24.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:31:24.170 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:24.170 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:24.170 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:24.170 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:24.170 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:24.170 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:31:24.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.170 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:24.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:24.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:24.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:24.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:24.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:24.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:31:24.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:24.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:24.172 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:31:24.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:31:24.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.172 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:24.172 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:31:24.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:31:24.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.172 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:24.172 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:31:24.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:31:24.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.173 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:24.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:31:24.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:31:24.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.173 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:24.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:31:24.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:31:24.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.173 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:24.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:31:24.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:31:24.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.174 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:24.174 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:31:24.174 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:31:24.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.174 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:24.174 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:31:24.174 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:31:24.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.175 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:24.175 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:31:24.175 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:31:24.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.175 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:24.175 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:31:24.175 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:31:24.175 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.175 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:24.175 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.175 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:24.176 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:24.176 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.480350432999998, 2024-09-18T11:31:24.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:31:24.176 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:24.176 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:24.176 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:24.176 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:24.176 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:24.176 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:31:24.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.176 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:24.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:24.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:24.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:24.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:24.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:24.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:24.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:24.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:24.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:31:24.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:31:24.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:24.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:31:24.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:31:24.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:24.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:31:24.179 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:31:24.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.179 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.179 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:24.179 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:31:24.179 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:31:24.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.179 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.179 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:24.179 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:31:24.179 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:31:24.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.179 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.179 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:24.180 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:31:24.180 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:31:24.180 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.180 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.180 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:24.180 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:31:24.180 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:31:24.180 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.180 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.180 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:24.180 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:31:24.180 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:31:24.180 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.181 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:24.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:31:24.181 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:31:24.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.181 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:24.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:31:24.181 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:31:24.181 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.181 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:24.181 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.182 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.182 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1788 4.8 e60/22 4:1898b15d:::smithi11939506-25:head v 60'12, mlcod=58'9)", 2024-09-18T11:31:24.182 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:24.182 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 41.147944944000002, 2024-09-18T11:31:24.182 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058348599999999999, 2024-09-18T11:31:24.182 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:24.182 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:24.182 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:24.182 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.182 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:24.182 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:24.182 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.183 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.183 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:24.183 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:24.183 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.183 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.183 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:24.183 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916949+0000", 2024-09-18T11:31:24.183 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:24.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.183 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.183 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:24.183 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917602+0000", 2024-09-18T11:31:24.184 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00065337100000000003 2024-09-18T11:31:24.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.184 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.184 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:24.184 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917621+0000", 2024-09-18T11:31:24.184 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9287e-05 2024-09-18T11:31:24.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.184 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.184 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:24.184 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917629+0000", 2024-09-18T11:31:24.184 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3780000000000003e-06 2024-09-18T11:31:24.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.184 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.185 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:24.185 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917677+0000", 2024-09-18T11:31:24.185 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8068000000000001e-05 2024-09-18T11:31:24.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.185 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.185 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:24.185 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917699+0000", 2024-09-18T11:31:24.185 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2149000000000001e-05 2024-09-18T11:31:24.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.185 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.185 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:24.185 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922747+0000", 2024-09-18T11:31:24.185 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050477609999999996 2024-09-18T11:31:24.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.186 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:24.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922785+0000", 2024-09-18T11:31:24.186 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8726000000000002e-05 2024-09-18T11:31:24.186 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.186 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:24.186 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.186 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:58 4.3 e91/67 4:c70c3162:::smithi11939506-35:18 v 91'101, mlcod=86'95)", 2024-09-18T11:31:24.186 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:24.186 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 10.049286721, 2024-09-18T11:31:24.186 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055647500000000003, 2024-09-18T11:31:24.186 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:24.187 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:24.187 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:24.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:24.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:24.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:24.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:24.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:24.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.188 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:24.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015608+0000", 2024-09-18T11:31:24.188 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:24.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.188 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.188 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:24.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015614+0000", 2024-09-18T11:31:24.188 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5310000000000001e-06 2024-09-18T11:31:24.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.188 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.188 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:24.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015622+0000", 2024-09-18T11:31:24.188 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0809999999999994e-06 2024-09-18T11:31:24.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.189 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.189 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:24.189 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015627+0000", 2024-09-18T11:31:24.189 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8439999999999997e-06 2024-09-18T11:31:24.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.189 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.189 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:24.189 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016003+0000", 2024-09-18T11:31:24.189 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037652999999999997 2024-09-18T11:31:24.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.189 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.189 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:24.189 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016024+0000", 2024-09-18T11:31:24.190 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0780000000000001e-05 2024-09-18T11:31:24.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.190 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.190 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:24.190 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021141+0000", 2024-09-18T11:31:24.190 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051166639999999999 2024-09-18T11:31:24.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:24.190 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:24.190 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:24.190 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021174+0000", 2024-09-18T11:31:24.190 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2839000000000002e-05 2024-09-18T11:31:24.190 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.190 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:24.190 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.191 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:24.191 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:24.191 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:24.191 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:24.065+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:24.192 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:31:24.214 INFO:tasks.rados.rados.0.smithi119.stdout:1356: finishing write tid 1 to smithi11939506-22 2024-09-18T11:31:24.214 INFO:tasks.rados.rados.0.smithi119.stdout:1352: done (15 left) 2024-09-18T11:31:24.214 INFO:tasks.rados.rados.0.smithi119.stdout:1372: read oid 23 snap -1 2024-09-18T11:31:24.214 INFO:tasks.rados.rados.0.smithi119.stdout:1372: expect deleted 2024-09-18T11:31:24.214 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:31:24.214 INFO:tasks.rados.rados.0.smithi119.stdout:1356: finishing write tid 2 to smithi11939506-22 2024-09-18T11:31:24.215 INFO:tasks.rados.rados.0.smithi119.stdout:1356: finishing write tid 3 to smithi11939506-22 2024-09-18T11:31:24.215 INFO:tasks.rados.rados.0.smithi119.stdout:1356: finishing write tid 4 to smithi11939506-22 2024-09-18T11:31:24.215 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 155 (ObjNum 263 snap 31 seq_num 263) dirty exists 2024-09-18T11:31:24.215 INFO:tasks.rados.rados.0.smithi119.stdout:1356: left oid 22 (ObjNum 263 snap 31 seq_num 263) 2024-09-18T11:31:24.215 INFO:tasks.rados.rados.0.smithi119.stdout:1357: finishing write tid 1 to smithi11939506-34 2024-09-18T11:31:24.215 INFO:tasks.rados.rados.0.smithi119.stdout:1356: done (15 left) 2024-09-18T11:31:24.215 INFO:tasks.rados.rados.0.smithi119.stdout:1373: snap_remove snap 20 2024-09-18T11:31:24.233 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:24.231+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:24.276 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:31:24.291 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:24.291 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:24.291 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:24.291 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:24.294 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:24.293+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:24.300 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:31:24.377 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:24.392 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:24.392 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:24.394 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:24.394+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:24.409 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:24.409 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:24.423 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:24.423 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:24.439 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:24.439 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:24.441 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:24.441 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:24.450 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:31:24.455 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:24.471 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:24.471 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:24.486 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:24.486 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:24.489 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:24.490 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:24.526 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T11:31:24.531 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:24.530+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:24.555 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:24.555 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:24.564 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:31:24.605 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:24.604+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:24.697 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:24.696+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:24.704 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:24.704 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:24.713 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:31:24.798 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:24.796+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:24.850 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:24.850 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:24.859 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:31:24.898 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:24.897+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:24.956 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:24.956 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:24.964 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:31:25.008 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:25.007+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:25.015 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:25.012+0000 7f6bf3052640 -1 osd.5 98 handle_osd_map injecting map crc failure 2024-09-18T11:31:25.016 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is set 2024-09-18T11:31:25.016 INFO:tasks.rados.rados.0.smithi119.stdout:1373: done (15 left) 2024-09-18T11:31:25.017 INFO:tasks.rados.rados.0.smithi119.stdout:1374: read oid 36 snap -1 2024-09-18T11:31:25.017 INFO:tasks.rados.rados.0.smithi119.stdout:1374: expect (ObjNum 100 snap 6 seq_num 100) 2024-09-18T11:31:25.017 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:31:25.017 INFO:tasks.rados.rados.0.smithi119.stdout:1359: expect (ObjNum 207 snap 23 seq_num 207) 2024-09-18T11:31:25.072 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:25.073 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:25.081 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:31:25.101 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:25.099+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:25.201 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:25.200+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:25.220 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:25.220 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:25.228 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:31:25.256 INFO:tasks.rados.rados.0.smithi119.stdout:1361: expect (ObjNum 258 snap 31 seq_num 258) 2024-09-18T11:31:25.302 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:25.301+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:25.354 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:25.354 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:25.363 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:31:25.406 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:25.404+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:25.425 INFO:tasks.rados.rados.0.smithi119.stdout:1358: expect (ObjNum 145 snap 18 seq_num 145) 2024-09-18T11:31:25.464 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:25.464 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:25.473 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:31:25.504 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:25.503+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:25.567 INFO:tasks.rados.rados.0.smithi119.stdout:1362: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:31:25.605 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:25.603+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:25.610 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:25.611 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:25.619 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:31:25.653 INFO:tasks.rados.rados.0.smithi119.stdout:1357: finishing write tid 2 to smithi11939506-34 2024-09-18T11:31:25.653 INFO:tasks.rados.rados.0.smithi119.stdout:1357: finishing write tid 3 to smithi11939506-34 2024-09-18T11:31:25.653 INFO:tasks.rados.rados.0.smithi119.stdout:1357: finishing write tid 4 to smithi11939506-34 2024-09-18T11:31:25.653 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 182 (ObjNum 264 snap 31 seq_num 264) dirty exists 2024-09-18T11:31:25.653 INFO:tasks.rados.rados.0.smithi119.stdout:1357: left oid 34 (ObjNum 264 snap 31 seq_num 264) 2024-09-18T11:31:25.653 INFO:tasks.rados.rados.0.smithi119.stdout:1360: got (0) Success 2024-09-18T11:31:25.653 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 0 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-18T11:31:25.653 INFO:tasks.rados.rados.0.smithi119.stdout:1363: expect (ObjNum 241 snap 28 seq_num 241) 2024-09-18T11:31:25.706 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:25.705+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:25.757 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:25.757 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:25.766 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:31:25.773 INFO:tasks.rados.rados.0.smithi119.stdout:1365: expect (ObjNum 253 snap 30 seq_num 253) 2024-09-18T11:31:25.809 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:25.808+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:25.864 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:25.864 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:25.872 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:31:25.889 INFO:tasks.rados.rados.0.smithi119.stdout:1367: got (0) Success 2024-09-18T11:31:25.889 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 54 (ObjNum 201 snap 23 seq_num 201) dirty exists 2024-09-18T11:31:25.889 INFO:tasks.rados.rados.0.smithi119.stdout:1369: expect (ObjNum 183 snap 20 seq_num 183) 2024-09-18T11:31:25.913 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:25.912+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:26.009 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:26.008+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:26.009 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:26.010 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:26.018 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:31:26.094 INFO:tasks.rados.rados.0.smithi119.stdout:1366: finishing set_chunk to oid 18 2024-09-18T11:31:26.094 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 113 (ObjNum 205 snap 23 seq_num 205) dirty exists 2024-09-18T11:31:26.094 INFO:tasks.rados.rados.0.smithi119.stdout:1370: expect (ObjNum 175 snap 20 seq_num 175) 2024-09-18T11:31:26.109 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:26.108+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:26.115 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:26.115 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:26.124 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:31:26.191 INFO:tasks.rados.rados.0.smithi119.stdout:1371: finishing write tid 1 to smithi11939506-1 2024-09-18T11:31:26.273 INFO:tasks.rados.rados.0.smithi119.stdout:1368: expect (ObjNum 244 snap 28 seq_num 244) 2024-09-18T11:31:26.275 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:26.276 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:26.276 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:31:26.276 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:26.273+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:26.277 INFO:tasks.rados.rados.0.smithi119.stdout:1371: finishing write tid 2 to smithi11939506-1 2024-09-18T11:31:26.277 INFO:tasks.rados.rados.0.smithi119.stdout:1371: finishing write tid 3 to smithi11939506-1 2024-09-18T11:31:26.277 INFO:tasks.rados.rados.0.smithi119.stdout:1371: finishing write tid 4 to smithi11939506-1 2024-09-18T11:31:26.277 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 166 (ObjNum 265 snap 31 seq_num 265) dirty exists 2024-09-18T11:31:26.278 INFO:tasks.rados.rados.0.smithi119.stdout:1371: left oid 1 (ObjNum 265 snap 31 seq_num 265) 2024-09-18T11:31:26.278 INFO:tasks.rados.rados.0.smithi119.stdout:1357: done (15 left) 2024-09-18T11:31:26.278 INFO:tasks.rados.rados.0.smithi119.stdout:1358: done (14 left) 2024-09-18T11:31:26.278 INFO:tasks.rados.rados.0.smithi119.stdout:1359: done (13 left) 2024-09-18T11:31:26.278 INFO:tasks.rados.rados.0.smithi119.stdout:1360: done (12 left) 2024-09-18T11:31:26.278 INFO:tasks.rados.rados.0.smithi119.stdout:1361: done (11 left) 2024-09-18T11:31:26.278 INFO:tasks.rados.rados.0.smithi119.stdout:1362: done (10 left) 2024-09-18T11:31:26.278 INFO:tasks.rados.rados.0.smithi119.stdout:1363: done (9 left) 2024-09-18T11:31:26.278 INFO:tasks.rados.rados.0.smithi119.stdout:1365: done (8 left) 2024-09-18T11:31:26.278 INFO:tasks.rados.rados.0.smithi119.stdout:1366: done (7 left) 2024-09-18T11:31:26.278 INFO:tasks.rados.rados.0.smithi119.stdout:1367: done (6 left) 2024-09-18T11:31:26.278 INFO:tasks.rados.rados.0.smithi119.stdout:1368: done (5 left) 2024-09-18T11:31:26.278 INFO:tasks.rados.rados.0.smithi119.stdout:1369: done (4 left) 2024-09-18T11:31:26.278 INFO:tasks.rados.rados.0.smithi119.stdout:1370: done (3 left) 2024-09-18T11:31:26.279 INFO:tasks.rados.rados.0.smithi119.stdout:1371: done (2 left) 2024-09-18T11:31:26.279 INFO:tasks.rados.rados.0.smithi119.stdout:1375: tier_promote oid 35 2024-09-18T11:31:26.279 INFO:tasks.rados.rados.0.smithi119.stdout:1374: expect (ObjNum 100 snap 6 seq_num 100) 2024-09-18T11:31:26.376 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:26.375+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:26.414 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:26.414 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:26.422 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:31:26.436 INFO:tasks.rados.rados.0.smithi119.stdout:1375: got (0) Success 2024-09-18T11:31:26.436 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 51 (ObjNum 155 snap 20 seq_num 155) dirty exists 2024-09-18T11:31:26.436 INFO:tasks.rados.rados.0.smithi119.stdout:1372: done (2 left) 2024-09-18T11:31:26.436 INFO:tasks.rados.rados.0.smithi119.stdout:1374: done (1 left) 2024-09-18T11:31:26.436 INFO:tasks.rados.rados.0.smithi119.stdout:1375: done (0 left) 2024-09-18T11:31:26.436 INFO:tasks.rados.rados.0.smithi119.stdout:1376: read oid 35 snap 30 2024-09-18T11:31:26.436 INFO:tasks.rados.rados.0.smithi119.stdout:1376: expect (ObjNum 155 snap 20 seq_num 155) 2024-09-18T11:31:26.436 INFO:tasks.rados.rados.0.smithi119.stdout:1377: read oid 49 snap 29 2024-09-18T11:31:26.436 INFO:tasks.rados.rados.0.smithi119.stdout:1377: expect (ObjNum 241 snap 28 seq_num 241) 2024-09-18T11:31:26.437 INFO:tasks.rados.rados.0.smithi119.stdout:1378: write (excl) oid 15 current snap is 31 2024-09-18T11:31:26.437 INFO:tasks.rados.rados.0.smithi119.stdout:1378: seq_num 266 ranges {470662=772236,1832916=408094} 2024-09-18T11:31:26.442 INFO:tasks.rados.rados.0.smithi119.stdout:1378: writing smithi11939506-15 from 470662 to 1242898 tid 1 2024-09-18T11:31:26.448 INFO:tasks.rados.rados.0.smithi119.stdout:1378: writing smithi11939506-15 from 1832916 to 2241010 tid 2 2024-09-18T11:31:26.448 INFO:tasks.rados.rados.0.smithi119.stdout:1379: read oid 7 snap 27 2024-09-18T11:31:26.448 INFO:tasks.rados.rados.0.smithi119.stdout:1379: expect (ObjNum 145 snap 18 seq_num 145) 2024-09-18T11:31:26.449 INFO:tasks.rados.rados.0.smithi119.stdout:1380: read oid 47 snap 29 2024-09-18T11:31:26.449 INFO:tasks.rados.rados.0.smithi119.stdout:1380: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T11:31:26.449 INFO:tasks.rados.rados.0.smithi119.stdout:1381: read oid 43 snap 30 2024-09-18T11:31:26.449 INFO:tasks.rados.rados.0.smithi119.stdout:1381: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:31:26.449 INFO:tasks.rados.rados.0.smithi119.stdout:1382: set_chunk oid 18 target oid 2024-09-18T11:31:26.464 INFO:tasks.rados.rados.0.smithi119.stdout:1382: set_chunk oid 18 offset: 1060352 length: 1745920 target oid 904f6d957a26256ba3cabe3c68688590c74f3b70ec1926d3916271912b2bea19 offset: 0 snap 28 2024-09-18T11:31:26.464 INFO:tasks.rados.rados.0.smithi119.stdout:1383: set_chunk oid 30 target oid 2024-09-18T11:31:26.464 INFO:tasks.rados.rados.0.smithi119.stdout:1383: set_chunk oid 30 offset: 0 length: 4000000 target oid smithi11939506- offset: 0 snap -1 2024-09-18T11:31:26.464 INFO:tasks.rados.rados.0.smithi119.stdout:1384: delete oid 27 current snap is 31 2024-09-18T11:31:26.464 INFO:tasks.rados.rados.0.smithi119.stdout:1377: expect (ObjNum 241 snap 28 seq_num 241) 2024-09-18T11:31:26.477 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:26.476+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:26.490 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:31:26.559 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:26.560 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:26.568 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:31:26.589 INFO:tasks.rados.rados.0.smithi119.stdout:1376: expect (ObjNum 155 snap 20 seq_num 155) 2024-09-18T11:31:26.590 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:26.605 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:26.605 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:26.612 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:26.611+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:26.620 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:26.620 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:26.635 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:26.635 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:26.650 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:26.650 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:26.665 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:26.665 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:26.667 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:26.667 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:26.676 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:31:26.681 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:26.681 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:26.695 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:26.696 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:26.698 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:26.699 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:26.717 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:26.716+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:26.735 INFO:tasks.rados.rados.0.smithi119.stdout:1378: finishing write tid 1 to smithi11939506-15 2024-09-18T11:31:26.735 INFO:tasks.rados.rados.0.smithi119.stdout:1378: finishing write tid 2 to smithi11939506-15 2024-09-18T11:31:26.736 INFO:tasks.rados.rados.0.smithi119.stdout:1378: finishing write tid 3 to smithi11939506-15 2024-09-18T11:31:26.736 INFO:tasks.rados.rados.0.smithi119.stdout:1378: finishing write tid 4 to smithi11939506-15 2024-09-18T11:31:26.736 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 93 (ObjNum 266 snap 31 seq_num 266) dirty exists 2024-09-18T11:31:26.736 INFO:tasks.rados.rados.0.smithi119.stdout:1378: left oid 15 (ObjNum 266 snap 31 seq_num 266) 2024-09-18T11:31:26.736 INFO:tasks.rados.rados.0.smithi119.stdout:1379: expect (ObjNum 145 snap 18 seq_num 145) 2024-09-18T11:31:26.779 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:26.778+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:26.813 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:26.813 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:26.813 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:26.813 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:26.813 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:26.823 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:31:26.878 INFO:tasks.rados.rados.0.smithi119.stdout:1381: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:31:26.880 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:26.879+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:26.922 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:26.922 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:26.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:26.922 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:26.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.922 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:26.922 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:26.922 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 97.762051677000002, 2024-09-18T11:31:26.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055400850000000001, 2024-09-18T11:31:26.923 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:26.923 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:26.923 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:26.923 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:26.923 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:26.923 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 43 2024-09-18T11:31:26.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.923 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:26.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:26.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:26.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:26.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:26.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:26.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:26.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:26.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:26.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157504+0000", 2024-09-18T11:31:26.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4519999999999996e-06 2024-09-18T11:31:26.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:26.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157534+0000", 2024-09-18T11:31:26.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.97e-05 2024-09-18T11:31:26.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:26.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157539+0000", 2024-09-18T11:31:26.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4360000000000001e-06 2024-09-18T11:31:26.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:26.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159428+0000", 2024-09-18T11:31:26.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018888220000000001 2024-09-18T11:31:26.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:26.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159479+0000", 2024-09-18T11:31:26.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0608000000000001e-05 2024-09-18T11:31:26.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:26.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159559+0000", 2024-09-18T11:31:26.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0667000000000005e-05 2024-09-18T11:31:26.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:26.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162779+0000", 2024-09-18T11:31:26.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032197459999999999 2024-09-18T11:31:26.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:26.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162950+0000", 2024-09-18T11:31:26.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017059599999999999 2024-09-18T11:31:26.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:26.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163001+0000", 2024-09-18T11:31:26.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0968999999999999e-05 2024-09-18T11:31:26.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:26.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163040+0000", 2024-09-18T11:31:26.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8843000000000001e-05 2024-09-18T11:31:26.928 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.928 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:26.928 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.928 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:26.929 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:26.929 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 97.762000522999998, 2024-09-18T11:31:26.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:31:26.929 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:26.929 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:26.929 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:26.929 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:26.929 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:26.929 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:31:26.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.930 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:26.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:26.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:26.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:26.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:26.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:26.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:31:26.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:26.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:26.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:31:26.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:31:26.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:26.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:31:26.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:31:26.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:26.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:31:26.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:26.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:26.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:31:26.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:31:26.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:26.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:31:26.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:31:26.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:26.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:31:26.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:31:26.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:26.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:31:26.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:31:26.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:26.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:31:26.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:31:26.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:26.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:31:26.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:31:26.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:26.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:31:26.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:31:26.934 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.934 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:26.935 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.935 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:26.935 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:26.935 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 97.761874261000003, 2024-09-18T11:31:26.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055717040000000002, 2024-09-18T11:31:26.935 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:26.935 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:26.935 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:26.935 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:26.935 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:26.935 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 45 2024-09-18T11:31:26.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.935 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:26.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:26.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:26.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:26.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:26.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:26.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157676+0000", 2024-09-18T11:31:26.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:26.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:26.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157694+0000", 2024-09-18T11:31:26.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7422999999999999e-05 2024-09-18T11:31:26.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:26.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157704+0000", 2024-09-18T11:31:26.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9000000000000001e-06 2024-09-18T11:31:26.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:26.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157718+0000", 2024-09-18T11:31:26.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4788e-05 2024-09-18T11:31:26.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:26.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160937+0000", 2024-09-18T11:31:26.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00321886 2024-09-18T11:31:26.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:26.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161023+0000", 2024-09-18T11:31:26.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6063999999999995e-05 2024-09-18T11:31:26.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:26.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161202+0000", 2024-09-18T11:31:26.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017912599999999999 2024-09-18T11:31:26.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:26.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163052+0000", 2024-09-18T11:31:26.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001849554 2024-09-18T11:31:26.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:26.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163204+0000", 2024-09-18T11:31:26.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015186900000000001 2024-09-18T11:31:26.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:26.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163229+0000", 2024-09-18T11:31:26.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:31:26.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:26.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163249+0000", 2024-09-18T11:31:26.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9992000000000001e-05 2024-09-18T11:31:26.940 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.940 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:26.940 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.941 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:26.941 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:26.941 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 97.761825148, 2024-09-18T11:31:26.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:31:26.941 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:26.941 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:26.941 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:26.941 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:26.941 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:26.941 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:31:26.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.941 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:26.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:26.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:26.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:26.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:26.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:26.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:26.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:26.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:26.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:31:26.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:31:26.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:26.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:31:26.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:31:26.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:26.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:31:26.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:31:26.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:26.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:31:26.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:31:26.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:26.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:31:26.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:31:26.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:26.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:31:26.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:31:26.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:26.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:31:26.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:31:26.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:26.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:31:26.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:31:26.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:26.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:31:26.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:31:26.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:26.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:31:26.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:31:26.946 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.946 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:26.947 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.947 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:26.947 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:26.947 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 97.761782616999994, 2024-09-18T11:31:26.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:31:26.947 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:26.947 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:26.947 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:26.947 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:26.947 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:26.947 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:31:26.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.948 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:26.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:26.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:26.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:26.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:26.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:26.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:31:26.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:26.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:26.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:31:26.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:31:26.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:26.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:31:26.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:31:26.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:26.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:31:26.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:31:26.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:26.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:31:26.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:31:26.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:26.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:31:26.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:31:26.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:26.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:31:26.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:31:26.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:26.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:31:26.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:31:26.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:26.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:31:26.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:31:26.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:26.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:31:26.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:31:26.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:26.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:31:26.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:31:26.952 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.952 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:26.952 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.953 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:26.953 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:26.953 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 97.761727687000004, 2024-09-18T11:31:26.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:31:26.953 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:26.953 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:26.953 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:26.953 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:26.953 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:26.953 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:31:26.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.953 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:26.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:26.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:26.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:26.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:26.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:26.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:26.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:26.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:26.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:31:26.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:31:26.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:26.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:31:26.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:31:26.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:26.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:31:26.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:31:26.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:26.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:31:26.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:31:26.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:26.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:31:26.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:31:26.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:26.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:31:26.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:31:26.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:26.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:31:26.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:31:26.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:26.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:31:26.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:31:26.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:26.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:31:26.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:31:26.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:26.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:31:26.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:31:26.958 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.958 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:26.958 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.958 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:26.959 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:26.959 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 97.761717707000003, 2024-09-18T11:31:26.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:31:26.959 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:26.959 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:26.959 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:26.959 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:26.959 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:26.959 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:31:26.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.959 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:26.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:26.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:26.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:26.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:26.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:26.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:26.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:26.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:26.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:31:26.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:31:26.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:26.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:31:26.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:31:26.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:26.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:31:26.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:31:26.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:26.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:31:26.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:31:26.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:26.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:31:26.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:31:26.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:26.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:31:26.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:31:26.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:26.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:31:26.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:31:26.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:26.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:31:26.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:31:26.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:26.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:31:26.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:31:26.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:26.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:31:26.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:31:26.964 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.964 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:26.965 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.965 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:26.965 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:26.965 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 97.761706107999998, 2024-09-18T11:31:26.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:31:26.965 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:26.965 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:26.965 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:26.965 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:26.965 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:26.965 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:31:26.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.966 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:26.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:26.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:26.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:26.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:26.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:26.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:26.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:26.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:26.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:31:26.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:31:26.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:26.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:31:26.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:31:26.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:26.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:31:26.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:31:26.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:26.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:31:26.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:31:26.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:26.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:31:26.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:31:26.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:26.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:31:26.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:31:26.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:26.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:31:26.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:31:26.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:26.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:31:26.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:31:26.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:26.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:31:26.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:31:26.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:26.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:31:26.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:31:26.970 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.971 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:26.971 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.971 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:26.971 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:26.971 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 97.761649626999997, 2024-09-18T11:31:26.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:31:26.971 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:26.971 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:26.971 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:26.971 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:26.971 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:26.972 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:31:26.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.972 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:26.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:26.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:26.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:26.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:26.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:26.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:31:26.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:26.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:26.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:31:26.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:31:26.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:26.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:31:26.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:31:26.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:26.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:31:26.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:31:26.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:26.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:31:26.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:31:26.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:26.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:31:26.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:31:26.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:26.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:31:26.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:31:26.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:26.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:31:26.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:31:26.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:26.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:31:26.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:31:26.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:26.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:31:26.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:31:26.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:26.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:31:26.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:31:26.977 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.977 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:26.977 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.977 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:26.977 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:26.977 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 97.761635803000004, 2024-09-18T11:31:26.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:31:26.977 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:26.977 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:26.977 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:26.977 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:26.978 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:26.978 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:31:26.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.978 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:26.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:26.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:26.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:26.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:26.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:26.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:26.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:26.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:26.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:31:26.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:31:26.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:26.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:31:26.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:31:26.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:26.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:31:26.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:31:26.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:26.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:31:26.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:31:26.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:26.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:31:26.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:31:26.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:26.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:31:26.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:31:26.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:26.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:31:26.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:31:26.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:26.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:31:26.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:31:26.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:26.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:31:26.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:31:26.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:26.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:31:26.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:31:26.983 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.983 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:26.983 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.983 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:26.983 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:26.983 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 97.345456896000002, 2024-09-18T11:31:26.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:31:26.983 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:26.983 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:26.983 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:26.983 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:26.984 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:26.984 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:31:26.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.984 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:26.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:26.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:26.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:26.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:26.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:26.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:26.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:26.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:26.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:31:26.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:31:26.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:26.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:31:26.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:31:26.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:26.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:31:26.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:31:26.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:26.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:31:26.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:31:26.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:26.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:31:26.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:31:26.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:26.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:31:26.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:31:26.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:26.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:31:26.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:31:26.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:26.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:31:26.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:31:26.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:26.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:31:26.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:31:26.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:26.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:31:26.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:31:26.989 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.989 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:26.989 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.989 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:26.989 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:26.989 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 97.345433596000007, 2024-09-18T11:31:26.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:31:26.989 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:26.989 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:26.989 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:26.989 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:26.989 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:26.989 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:31:26.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.990 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:26.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:26.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:26.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:26.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:26.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:26.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:31:26.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:26.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:26.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:31:26.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:31:26.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:26.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:31:26.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:31:26.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:26.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:31:26.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:31:26.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:26.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:31:26.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:31:26.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:26.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:31:26.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:31:26.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:26.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:31:26.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:31:26.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:26.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:31:26.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:31:26.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:26.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:31:26.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:31:26.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:26.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:31:26.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:31:26.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:26.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:31:26.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:31:26.995 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.995 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:26.995 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:26.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.995 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:26.995 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:26.995 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 97.345347986999997, 2024-09-18T11:31:26.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:31:26.995 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:26.995 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:26.996 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:26.996 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:26.996 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:26.996 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:31:26.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.996 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:26.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:26.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:26.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:26.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:26.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:26.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:26.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:26.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:26.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:26.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:31:26.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:31:26.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:26.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:31:26.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:31:26.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:26.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:31:26.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:31:26.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:26.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:31:26.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:31:26.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:26.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:31:26.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:31:26.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:26.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:26.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:26.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:31:27.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:31:27.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:27.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:31:27.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:31:27.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:27.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:31:27.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:31:27.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:27.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:31:27.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:31:27.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:27.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:31:27.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:31:27.001 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:27.001 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:27.001 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:27.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.002 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:27.002 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:27.002 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 97.345337021999995, 2024-09-18T11:31:27.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:31:27.002 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:27.002 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.002 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:27.002 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:27.002 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:27.002 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:31:27.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.002 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:27.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:27.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:27.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:27.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:27.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:27.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:27.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:27.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:27.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:27.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:27.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:31:27.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:31:27.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:27.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:31:27.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:31:27.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:31:27.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:31:27.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:27.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:31:27.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:31:27.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:27.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:31:27.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:31:27.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:27.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:31:27.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:31:27.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:27.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:31:27.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:31:27.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:27.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:31:27.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:31:27.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:27.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:31:27.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:31:27.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:27.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:31:27.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:31:27.007 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:27.007 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:27.007 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:27.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.007 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:27.007 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:27.007 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 97.345325853999995, 2024-09-18T11:31:27.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:31:27.007 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:27.008 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.008 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:27.008 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:27.008 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:27.008 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:31:27.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.008 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:27.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:27.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:27.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:27.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:27.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:27.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:27.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:27.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:27.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:27.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:27.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:31:27.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:31:27.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:27.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:31:27.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:31:27.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:31:27.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:31:27.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:27.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:31:27.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:31:27.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:27.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:31:27.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:31:27.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:27.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:31:27.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:31:27.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:27.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:31:27.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:31:27.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:27.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:31:27.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:31:27.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:27.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:31:27.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:31:27.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:27.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:31:27.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:31:27.013 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:27.013 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:27.013 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:27.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.013 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:27.013 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:27.014 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 97.345260525, 2024-09-18T11:31:27.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:31:27.014 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:27.014 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.014 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:27.014 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:27.014 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:27.014 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:31:27.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.014 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:27.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:27.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:27.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:27.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:27.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:27.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:27.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:27.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:31:27.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:27.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:31:27.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:31:27.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:27.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:31:27.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:31:27.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:31:27.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:31:27.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:27.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:31:27.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:31:27.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:27.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:31:27.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:31:27.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:27.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:31:27.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:31:27.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:27.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:31:27.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:31:27.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:27.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:31:27.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:31:27.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:27.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:31:27.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:31:27.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:27.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:31:27.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:31:27.019 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:27.019 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:27.019 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:27.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.019 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:27.019 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:27.019 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 97.335098547000001, 2024-09-18T11:31:27.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:31:27.020 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:27.020 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.020 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:27.020 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:27.020 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:27.020 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:31:27.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.020 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:27.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:27.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:27.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:27.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:27.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:27.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:27.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:27.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:31:27.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:27.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:31:27.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:31:27.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.022 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:27.022 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:31:27.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:31:27.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.022 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.022 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:31:27.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:31:27.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:27.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:31:27.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:31:27.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:27.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:31:27.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:31:27.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:27.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:31:27.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:31:27.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:27.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:31:27.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:31:27.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:27.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:31:27.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:31:27.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:27.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:31:27.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:31:27.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:27.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:31:27.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:31:27.025 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:27.025 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:27.025 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:27.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.025 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:27.025 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:27.026 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 97.335006067999998, 2024-09-18T11:31:27.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:31:27.026 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:27.026 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.026 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:27.026 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:27.026 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:27.026 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:31:27.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.026 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:27.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:27.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:27.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:27.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.027 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:27.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:27.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:27.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.027 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:27.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:27.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.027 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:27.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:31:27.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:31:27.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:27.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:31:27.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:31:27.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:31:27.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:31:27.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:27.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:31:27.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:31:27.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:27.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:31:27.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:31:27.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:27.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:31:27.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:31:27.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:27.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:31:27.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:31:27.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:27.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:31:27.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:31:27.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:27.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:31:27.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:31:27.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:27.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:31:27.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:31:27.031 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:27.031 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:27.031 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:27.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.031 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1788 4.8 e60/22 4:1898b15d:::smithi11939506-25:head v 60'12, mlcod=58'9)", 2024-09-18T11:31:27.031 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:27.032 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 44.002600579000003, 2024-09-18T11:31:27.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058348599999999999, 2024-09-18T11:31:27.032 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:27.032 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.032 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:27.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.032 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:27.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:27.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:27.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.032 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:27.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:27.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:27.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:27.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916949+0000", 2024-09-18T11:31:27.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:27.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:27.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917602+0000", 2024-09-18T11:31:27.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00065337100000000003 2024-09-18T11:31:27.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:27.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917621+0000", 2024-09-18T11:31:27.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9287e-05 2024-09-18T11:31:27.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917629+0000", 2024-09-18T11:31:27.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3780000000000003e-06 2024-09-18T11:31:27.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:27.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917677+0000", 2024-09-18T11:31:27.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8068000000000001e-05 2024-09-18T11:31:27.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:27.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917699+0000", 2024-09-18T11:31:27.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2149000000000001e-05 2024-09-18T11:31:27.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:27.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922747+0000", 2024-09-18T11:31:27.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050477609999999996 2024-09-18T11:31:27.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:27.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922785+0000", 2024-09-18T11:31:27.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8726000000000002e-05 2024-09-18T11:31:27.036 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:27.036 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:27.036 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:27.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.036 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:58 4.3 e91/67 4:c70c3162:::smithi11939506-35:18 v 91'101, mlcod=86'95)", 2024-09-18T11:31:27.036 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:27.036 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.903942356, 2024-09-18T11:31:27.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055647500000000003, 2024-09-18T11:31:27.036 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:27.037 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.037 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:27.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:27.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:27.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:27.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:27.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:27.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:27.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:27.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015608+0000", 2024-09-18T11:31:27.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:27.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015614+0000", 2024-09-18T11:31:27.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5310000000000001e-06 2024-09-18T11:31:27.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:27.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015622+0000", 2024-09-18T11:31:27.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0809999999999994e-06 2024-09-18T11:31:27.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015627+0000", 2024-09-18T11:31:27.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8439999999999997e-06 2024-09-18T11:31:27.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:27.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016003+0000", 2024-09-18T11:31:27.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037652999999999997 2024-09-18T11:31:27.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:27.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016024+0000", 2024-09-18T11:31:27.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0780000000000001e-05 2024-09-18T11:31:27.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:27.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021141+0000", 2024-09-18T11:31:27.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051166639999999999 2024-09-18T11:31:27.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:27.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:27.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:27.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021174+0000", 2024-09-18T11:31:27.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2839000000000002e-05 2024-09-18T11:31:27.040 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:27.040 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:27.040 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:27.040 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:27.041 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:27.041 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:27.041 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T11:31:27.042 INFO:tasks.rados.rados.0.smithi119.stdout:1383: finishing set_chunk to oid 30 2024-09-18T11:31:27.042 INFO:tasks.rados.rados.0.smithi119.stdout:1383: got expected ERANGE (src dne) 2024-09-18T11:31:27.042 INFO:tasks.rados.rados.0.smithi119.stdout:1382: finishing set_chunk to oid 18 2024-09-18T11:31:27.042 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 18 set_chunk returned error code -95 offset: 1060352 length: 1745920 2024-09-18T11:31:27.042 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 0 (ObjNum 205 snap 23 seq_num 205) dirty exists 2024-09-18T11:31:27.042 INFO:tasks.rados.rados.0.smithi119.stdout:1380: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T11:31:27.043 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:27.041+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:27.044 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:31:27.055 INFO:tasks.rados.rados.0.smithi119.stdout:1376: done (8 left) 2024-09-18T11:31:27.055 INFO:tasks.rados.rados.0.smithi119.stdout:1377: done (7 left) 2024-09-18T11:31:27.055 INFO:tasks.rados.rados.0.smithi119.stdout:1378: done (6 left) 2024-09-18T11:31:27.055 INFO:tasks.rados.rados.0.smithi119.stdout:1379: done (5 left) 2024-09-18T11:31:27.055 INFO:tasks.rados.rados.0.smithi119.stdout:1380: done (4 left) 2024-09-18T11:31:27.055 INFO:tasks.rados.rados.0.smithi119.stdout:1381: done (3 left) 2024-09-18T11:31:27.055 INFO:tasks.rados.rados.0.smithi119.stdout:1382: done (2 left) 2024-09-18T11:31:27.055 INFO:tasks.rados.rados.0.smithi119.stdout:1383: done (1 left) 2024-09-18T11:31:27.055 INFO:tasks.rados.rados.0.smithi119.stdout:1384: done (0 left) 2024-09-18T11:31:27.055 INFO:tasks.rados.rados.0.smithi119.stdout:1385: set_chunk oid 11 target oid 2024-09-18T11:31:27.061 INFO:tasks.rados.rados.0.smithi119.stdout:1385: set_chunk oid 11 offset: 1678336 length: 368128 target oid b8f3417a63b50d92e54068fbdceeb292a7e424e11cea7e4f7220b8a7babff25e offset: 0 snap -1 2024-09-18T11:31:27.061 INFO:tasks.rados.rados.0.smithi119.stdout:1386: tier_promote oid 39 2024-09-18T11:31:27.061 INFO:tasks.rados.rados.0.smithi119.stdout:1387: read oid 20 snap 27 2024-09-18T11:31:27.061 INFO:tasks.rados.rados.0.smithi119.stdout:1387: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:31:27.061 INFO:tasks.rados.rados.0.smithi119.stdout:1388: read oid 14 snap -1 2024-09-18T11:31:27.061 INFO:tasks.rados.rados.0.smithi119.stdout:1388: expect (ObjNum 246 snap 28 seq_num 246) 2024-09-18T11:31:27.061 INFO:tasks.rados.rados.0.smithi119.stdout:1389: tier_promote oid 46 2024-09-18T11:31:27.061 INFO:tasks.rados.rados.0.smithi119.stdout:1390: tier_promote oid 37 2024-09-18T11:31:27.061 INFO:tasks.rados.rados.0.smithi119.stdout:1391: read oid 30 snap -1 2024-09-18T11:31:27.061 INFO:tasks.rados.rados.0.smithi119.stdout:1391: expect deleted 2024-09-18T11:31:27.062 INFO:tasks.rados.rados.0.smithi119.stdout:1392: read oid 22 snap 29 2024-09-18T11:31:27.062 INFO:tasks.rados.rados.0.smithi119.stdout:1392: expect (ObjNum 238 snap 28 seq_num 238) 2024-09-18T11:31:27.062 INFO:tasks.rados.rados.0.smithi119.stdout:1393: set_chunk oid 29 target oid 2024-09-18T11:31:27.074 INFO:tasks.rados.rados.0.smithi119.stdout:1393: set_chunk oid 29 offset: 1909760 length: 1523200 target oid a7c8becaa8141f0df3bc5b4a2708951e6d3e91ba1fb7de4f9dc69f582167cd9d offset: 0 snap 29 2024-09-18T11:31:27.074 INFO:tasks.rados.rados.0.smithi119.stdout:1394: read oid 33 snap -1 2024-09-18T11:31:27.074 INFO:tasks.rados.rados.0.smithi119.stdout:1394: expect deleted 2024-09-18T11:31:27.074 INFO:tasks.rados.rados.0.smithi119.stdout:1395: read oid 10 snap 27 2024-09-18T11:31:27.074 INFO:tasks.rados.rados.0.smithi119.stdout:1395: expect (ObjNum 194 snap 23 seq_num 194) 2024-09-18T11:31:27.074 INFO:tasks.rados.rados.0.smithi119.stdout:1396: rollback oid 49 current snap is 31 2024-09-18T11:31:27.074 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 49 to 27 2024-09-18T11:31:27.074 INFO:tasks.rados.rados.0.smithi119.stdout:1397: read oid 40 snap -1 2024-09-18T11:31:27.074 INFO:tasks.rados.rados.0.smithi119.stdout:1397: expect (ObjNum 175 snap 20 seq_num 175) 2024-09-18T11:31:27.074 INFO:tasks.rados.rados.0.smithi119.stdout:1398: rollback oid 19 current snap is 31 2024-09-18T11:31:27.074 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 19 to 23 2024-09-18T11:31:27.074 INFO:tasks.rados.rados.0.smithi119.stdout:1392: expect (ObjNum 238 snap 28 seq_num 238) 2024-09-18T11:31:27.143 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:27.142+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:27.152 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:27.153 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:27.153 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:27.153 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:27.164 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:31:27.214 INFO:tasks.rados.rados.0.smithi119.stdout:1390: got (0) Success 2024-09-18T11:31:27.214 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 181 (ObjNum 261 snap 31 seq_num 261) dirty exists 2024-09-18T11:31:27.214 INFO:tasks.rados.rados.0.smithi119.stdout:1385: finishing set_chunk to oid 11 2024-09-18T11:31:27.214 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 11 v 177 (ObjNum 249 snap 30 seq_num 249) dirty exists 2024-09-18T11:31:27.214 INFO:tasks.rados.rados.0.smithi119.stdout:1389: got (0) Success 2024-09-18T11:31:27.214 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 104 (ObjNum 171 snap 20 seq_num 171) dirty exists 2024-09-18T11:31:27.214 INFO:tasks.rados.rados.0.smithi119.stdout:1385: done (13 left) 2024-09-18T11:31:27.215 INFO:tasks.rados.rados.0.smithi119.stdout:1389: done (12 left) 2024-09-18T11:31:27.215 INFO:tasks.rados.rados.0.smithi119.stdout:1390: done (11 left) 2024-09-18T11:31:27.215 INFO:tasks.rados.rados.0.smithi119.stdout:1391: done (10 left) 2024-09-18T11:31:27.215 INFO:tasks.rados.rados.0.smithi119.stdout:1392: done (9 left) 2024-09-18T11:31:27.215 INFO:tasks.rados.rados.0.smithi119.stdout:1399: set_chunk oid 47 target oid 2024-09-18T11:31:27.218 INFO:tasks.rados.rados.0.smithi119.stdout:1399: set_chunk oid 47 offset: 1945600 length: 318464 target oid 9f61f8401cb1cc4eec836f4d2922ed8550bd5b26bc5bebb3bcf280a47dd1db8d offset: 0 snap 27 2024-09-18T11:31:27.218 INFO:tasks.rados.rados.0.smithi119.stdout:1400: rollback oid 13 current snap is 31 2024-09-18T11:31:27.218 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 13 to 21 2024-09-18T11:31:27.219 INFO:tasks.rados.rados.0.smithi119.stdout:1387: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:31:27.244 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:27.243+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:27.304 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:27.305 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:27.305 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:27.305 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:27.305 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:27.313 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:31:27.345 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:27.344+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:27.439 INFO:tasks.rados.rados.0.smithi119.stdout:1387: done (10 left) 2024-09-18T11:31:27.439 INFO:tasks.rados.rados.0.smithi119.stdout:1394: done (9 left) 2024-09-18T11:31:27.439 INFO:tasks.rados.rados.0.smithi119.stdout:1401: write oid 16 current snap is 31 2024-09-18T11:31:27.439 INFO:tasks.rados.rados.0.smithi119.stdout:1388: expect (ObjNum 246 snap 28 seq_num 246) 2024-09-18T11:31:27.445 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:27.444+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:27.452 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:27.452 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:27.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:27.453 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:27.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.453 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:888 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:27.453 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:27.453 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 76.333600207000003, 2024-09-18T11:31:27.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082282120000000004, 2024-09-18T11:31:27.453 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.453 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:27.453 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:27.453 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:27.453 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:27.453 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 888 2024-09-18T11:31:27.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.454 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:27.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:27.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:27.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116423+0000", 2024-09-18T11:31:27.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1e-06 2024-09-18T11:31:27.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:31:27.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5510000000000001e-06 2024-09-18T11:31:27.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116431+0000", 2024-09-18T11:31:27.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6109999999999998e-06 2024-09-18T11:31:27.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116527+0000", 2024-09-18T11:31:27.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6212999999999995e-05 2024-09-18T11:31:27.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:27.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116532+0000", 2024-09-18T11:31:27.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6140000000000002e-06 2024-09-18T11:31:27.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124314+0000", 2024-09-18T11:31:27.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077818929999999998 2024-09-18T11:31:27.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124411+0000", 2024-09-18T11:31:27.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6992999999999998e-05 2024-09-18T11:31:27.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124649+0000", 2024-09-18T11:31:27.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237729 2024-09-18T11:31:27.458 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.458 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.458 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.458 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:889 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:27.458 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:27.458 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 76.333575557000003, 2024-09-18T11:31:27.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085383349999999993, 2024-09-18T11:31:27.458 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.458 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:27.458 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:27.458 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:27.458 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:27.459 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 889 2024-09-18T11:31:27.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.459 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:27.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:27.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:27.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116447+0000", 2024-09-18T11:31:27.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.854e-06 2024-09-18T11:31:27.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116452+0000", 2024-09-18T11:31:27.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0250000000000002e-06 2024-09-18T11:31:27.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116455+0000", 2024-09-18T11:31:27.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8739999999999999e-06 2024-09-18T11:31:27.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116565+0000", 2024-09-18T11:31:27.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110741 2024-09-18T11:31:27.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:27.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116571+0000", 2024-09-18T11:31:27.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4990000000000002e-06 2024-09-18T11:31:27.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124691+0000", 2024-09-18T11:31:27.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081200339999999999 2024-09-18T11:31:27.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124784+0000", 2024-09-18T11:31:27.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2886999999999993e-05 2024-09-18T11:31:27.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124983+0000", 2024-09-18T11:31:27.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019969000000000001 2024-09-18T11:31:27.463 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.463 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.463 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.463 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1126 3.0 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head [write 0~2520064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:31:27.463 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:31:27.463 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 67.468133578999996, 2024-09-18T11:31:27.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086197050000000001, 2024-09-18T11:31:27.464 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.464 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.464 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:27.464 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:27.464 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:27.464 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1126 2024-09-18T11:31:27.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.464 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:31:27.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:31:27.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981886+0000", 2024-09-18T11:31:27.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:27.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983017+0000", 2024-09-18T11:31:27.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00113131 2024-09-18T11:31:27.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983037+0000", 2024-09-18T11:31:27.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0231e-05 2024-09-18T11:31:27.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983044+0000", 2024-09-18T11:31:27.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9539999999999998e-06 2024-09-18T11:31:27.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983087+0000", 2024-09-18T11:31:27.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3040000000000001e-05 2024-09-18T11:31:27.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983193+0000", 2024-09-18T11:31:27.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105682 2024-09-18T11:31:27.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:27.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983287+0000", 2024-09-18T11:31:27.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4066000000000004e-05 2024-09-18T11:31:27.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:27.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.986918+0000", 2024-09-18T11:31:27.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036311970000000001 2024-09-18T11:31:27.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:27.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990442+0000", 2024-09-18T11:31:27.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035243119999999999 2024-09-18T11:31:27.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990476+0000", 2024-09-18T11:31:27.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4028e-05 2024-09-18T11:31:27.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990507+0000", 2024-09-18T11:31:27.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0284999999999999e-05 2024-09-18T11:31:27.470 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.470 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.470 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.470 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:262 4.9 e53/22 4:99feffe9:::smithi11939506-45:13 v 53'17, mlcod=51'14)", 2024-09-18T11:31:27.471 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:27.471 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.451029763999998, 2024-09-18T11:31:27.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013878679, 2024-09-18T11:31:27.471 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.471 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.471 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:27.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:27.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:27.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998997+0000", 2024-09-18T11:31:27.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1099999999999999e-06 2024-09-18T11:31:27.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999008+0000", 2024-09-18T11:31:27.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1321000000000001e-05 2024-09-18T11:31:27.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999015+0000", 2024-09-18T11:31:27.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3000000000000004e-06 2024-09-18T11:31:27.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999055+0000", 2024-09-18T11:31:27.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0058000000000002e-05 2024-09-18T11:31:27.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999070+0000", 2024-09-18T11:31:27.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4328000000000001e-05 2024-09-18T11:31:27.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012465+0000", 2024-09-18T11:31:27.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013395638 2024-09-18T11:31:27.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012870+0000", 2024-09-18T11:31:27.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040423400000000001 2024-09-18T11:31:27.475 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.475 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.475 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.475 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:205 4.b e53/22 4:dba5bd39:::smithi11939506-17:13 v 53'77, mlcod=51'74)", 2024-09-18T11:31:27.476 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:27.476 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.450980502999997, 2024-09-18T11:31:27.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013359586, 2024-09-18T11:31:27.476 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.476 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.476 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:27.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:27.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:27.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999048+0000", 2024-09-18T11:31:27.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3550000000000005e-06 2024-09-18T11:31:27.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999056+0000", 2024-09-18T11:31:27.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3259999999999996e-06 2024-09-18T11:31:27.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999063+0000", 2024-09-18T11:31:27.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6409999999999996e-06 2024-09-18T11:31:27.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999098+0000", 2024-09-18T11:31:27.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5203999999999997e-05 2024-09-18T11:31:27.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999115+0000", 2024-09-18T11:31:27.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6478999999999999e-05 2024-09-18T11:31:27.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012362+0000", 2024-09-18T11:31:27.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01324768 2024-09-18T11:31:27.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012400+0000", 2024-09-18T11:31:27.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.731e-05 2024-09-18T11:31:27.480 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.480 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.480 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.480 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:88 4.a e53/22 4:588debc2:::smithi11939506-18:13 v 53'45, mlcod=51'42)", 2024-09-18T11:31:27.480 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:27.480 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.448198079999997, 2024-09-18T11:31:27.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011307667, 2024-09-18T11:31:27.481 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.481 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.481 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:27.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:27.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001822+0000", 2024-09-18T11:31:27.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001827+0000", 2024-09-18T11:31:27.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.378e-06 2024-09-18T11:31:27.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001839+0000", 2024-09-18T11:31:27.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1225e-05 2024-09-18T11:31:27.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001844+0000", 2024-09-18T11:31:27.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2299999999999999e-06 2024-09-18T11:31:27.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001891+0000", 2024-09-18T11:31:27.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6961e-05 2024-09-18T11:31:27.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001906+0000", 2024-09-18T11:31:27.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5158e-05 2024-09-18T11:31:27.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013103+0000", 2024-09-18T11:31:27.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011197314 2024-09-18T11:31:27.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013130+0000", 2024-09-18T11:31:27.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6900999999999999e-05 2024-09-18T11:31:27.484 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.485 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.485 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.485 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:89 4.a e53/22 4:552523b6:::smithi11939506-38:13 v 53'47, mlcod=51'42)", 2024-09-18T11:31:27.485 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:27.485 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.447638859999998, 2024-09-18T11:31:27.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011327614, 2024-09-18T11:31:27.485 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.485 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.485 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:27.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:27.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002381+0000", 2024-09-18T11:31:27.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002397+0000", 2024-09-18T11:31:27.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5688e-05 2024-09-18T11:31:27.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002406+0000", 2024-09-18T11:31:27.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0229999999999998e-06 2024-09-18T11:31:27.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002415+0000", 2024-09-18T11:31:27.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8959999999999992e-06 2024-09-18T11:31:27.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012386+0000", 2024-09-18T11:31:27.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0099711450000000007 2024-09-18T11:31:27.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012405+0000", 2024-09-18T11:31:27.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9389e-05 2024-09-18T11:31:27.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013676+0000", 2024-09-18T11:31:27.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012708649999999999 2024-09-18T11:31:27.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013709+0000", 2024-09-18T11:31:27.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3207000000000002e-05 2024-09-18T11:31:27.490 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.490 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.490 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.490 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1644 4.7 4:e73ba68f:::smithi11939506-1:head [sparse-read 0~2695582] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:27.490 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:27.490 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 50.116116562000002, 2024-09-18T11:31:27.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084764039999999999, 2024-09-18T11:31:27.490 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.490 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:27.490 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:27.491 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:27.491 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:27.491 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1644 2024-09-18T11:31:27.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.491 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:27.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:27.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:27.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333906+0000", 2024-09-18T11:31:27.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.243e-06 2024-09-18T11:31:27.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333911+0000", 2024-09-18T11:31:27.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2959999999999998e-06 2024-09-18T11:31:27.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333915+0000", 2024-09-18T11:31:27.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6720000000000002e-06 2024-09-18T11:31:27.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341879+0000", 2024-09-18T11:31:27.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079638699999999996 2024-09-18T11:31:27.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341978+0000", 2024-09-18T11:31:27.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9562000000000007e-05 2024-09-18T11:31:27.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342380+0000", 2024-09-18T11:31:27.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000402113 2024-09-18T11:31:27.495 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.495 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.495 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.495 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1645 4.7 4:e73ba68f:::smithi11939506-1:head [read 0~3983226,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:27.495 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:27.495 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 50.116101667999999, 2024-09-18T11:31:27.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010539468999999999, 2024-09-18T11:31:27.496 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.496 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:27.496 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:27.496 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:27.496 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:27.496 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1645 2024-09-18T11:31:27.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.496 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:27.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:27.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:27.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333921+0000", 2024-09-18T11:31:27.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8909999999999999e-06 2024-09-18T11:31:27.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333927+0000", 2024-09-18T11:31:27.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4229999999999996e-06 2024-09-18T11:31:27.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333932+0000", 2024-09-18T11:31:27.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8280000000000002e-06 2024-09-18T11:31:27.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342418+0000", 2024-09-18T11:31:27.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084862219999999999 2024-09-18T11:31:27.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342515+0000", 2024-09-18T11:31:27.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7287999999999994e-05 2024-09-18T11:31:27.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.344458+0000", 2024-09-18T11:31:27.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019431660000000001 2024-09-18T11:31:27.499 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.499 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.499 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.500 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1669 4.4 4:2d9c3f7f:::smithi11939506-5:head [read 0~3764735,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-18T11:31:27.500 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:31:27.500 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.627152770000002, 2024-09-18T11:31:27.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0083694990000000007, 2024-09-18T11:31:27.500 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.500 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:27.500 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:27.500 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:27.500 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:27.500 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1669 2024-09-18T11:31:27.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.501 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:31:27.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:31:27.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822867+0000", 2024-09-18T11:31:27.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822871+0000", 2024-09-18T11:31:27.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:31:27.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822877+0000", 2024-09-18T11:31:27.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8000000000000004e-06 2024-09-18T11:31:27.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822880+0000", 2024-09-18T11:31:27.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5669999999999999e-06 2024-09-18T11:31:27.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822974+0000", 2024-09-18T11:31:27.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4048999999999993e-05 2024-09-18T11:31:27.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:27.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822980+0000", 2024-09-18T11:31:27.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2789999999999998e-06 2024-09-18T11:31:27.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.829160+0000", 2024-09-18T11:31:27.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0061800179999999998 2024-09-18T11:31:27.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.829248+0000", 2024-09-18T11:31:27.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7707000000000006e-05 2024-09-18T11:31:27.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.831237+0000", 2024-09-18T11:31:27.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001989827 2024-09-18T11:31:27.505 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.505 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.505 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.505 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:100 4.a e58/22 4:5975a5ec:::smithi11939506-36:18 v 58'55, mlcod=56'50)", 2024-09-18T11:31:27.505 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:27.505 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.176871532, 2024-09-18T11:31:27.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0088255, 2024-09-18T11:31:27.505 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.505 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.506 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:27.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:27.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:27.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273155+0000", 2024-09-18T11:31:27.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.212e-06 2024-09-18T11:31:27.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273164+0000", 2024-09-18T11:31:27.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7609999999999996e-06 2024-09-18T11:31:27.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273168+0000", 2024-09-18T11:31:27.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2819999999999998e-06 2024-09-18T11:31:27.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274162+0000", 2024-09-18T11:31:27.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099398600000000009 2024-09-18T11:31:27.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274180+0000", 2024-09-18T11:31:27.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8236e-05 2024-09-18T11:31:27.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281943+0000", 2024-09-18T11:31:27.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007763308 2024-09-18T11:31:27.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281975+0000", 2024-09-18T11:31:27.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1202999999999999e-05 2024-09-18T11:31:27.510 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.510 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.510 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.511 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1954 4.7 4:ef60946b:::smithi11939506-8:head [read 0~2890771,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:27.511 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:27.511 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.110120492, 2024-09-18T11:31:27.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091142180000000003, 2024-09-18T11:31:27.511 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.511 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:27.511 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:27.511 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:27.511 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:27.511 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1954 2024-09-18T11:31:27.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.511 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:27.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:27.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:27.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339903+0000", 2024-09-18T11:31:27.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7349999999999998e-06 2024-09-18T11:31:27.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339909+0000", 2024-09-18T11:31:27.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-09-18T11:31:27.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339913+0000", 2024-09-18T11:31:27.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7670000000000001e-06 2024-09-18T11:31:27.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339970+0000", 2024-09-18T11:31:27.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6988999999999999e-05 2024-09-18T11:31:27.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:27.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339975+0000", 2024-09-18T11:31:27.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7929999999999997e-06 2024-09-18T11:31:27.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346917+0000", 2024-09-18T11:31:27.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069426640000000003 2024-09-18T11:31:27.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.347026+0000", 2024-09-18T11:31:27.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108241 2024-09-18T11:31:27.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.349014+0000", 2024-09-18T11:31:27.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001988723 2024-09-18T11:31:27.516 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.516 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.516 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.516 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1956 4.6 4:67c1765a:::smithi11939506-30:head [write 705927~558502] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:27.516 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:27.516 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.098011552000003, 2024-09-18T11:31:27.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011813239, 2024-09-18T11:31:27.516 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.516 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.516 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:27.517 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:27.517 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:27.517 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1956 2024-09-18T11:31:27.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.517 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:27.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:27.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352007+0000", 2024-09-18T11:31:27.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:27.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352229+0000", 2024-09-18T11:31:27.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222024 2024-09-18T11:31:27.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352251+0000", 2024-09-18T11:31:27.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2512000000000001e-05 2024-09-18T11:31:27.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352260+0000", 2024-09-18T11:31:27.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7339999999999997e-06 2024-09-18T11:31:27.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352307+0000", 2024-09-18T11:31:27.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7061999999999998e-05 2024-09-18T11:31:27.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352408+0000", 2024-09-18T11:31:27.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101216 2024-09-18T11:31:27.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:27.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352550+0000", 2024-09-18T11:31:27.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014136199999999999 2024-09-18T11:31:27.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:27.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357684+0000", 2024-09-18T11:31:27.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051342499999999999 2024-09-18T11:31:27.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:27.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363736+0000", 2024-09-18T11:31:27.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006052577 2024-09-18T11:31:27.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363778+0000", 2024-09-18T11:31:27.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1989000000000001e-05 2024-09-18T11:31:27.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363822+0000", 2024-09-18T11:31:27.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3894999999999999e-05 2024-09-18T11:31:27.522 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.522 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.522 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.522 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2278 4.f 4:fc192b81:::smithi11939506-31:head [stat,delete] snapc 26=[26,25,24,1e,1c,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T11:31:27.523 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:27.523 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.066974340000002, 2024-09-18T11:31:27.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01018694, 2024-09-18T11:31:27.523 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.523 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.523 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:27.523 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:27.523 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:27.523 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2278 2024-09-18T11:31:27.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.523 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:27.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:27.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383044+0000", 2024-09-18T11:31:27.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:27.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383054+0000", 2024-09-18T11:31:27.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5969999999999998e-06 2024-09-18T11:31:27.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383071+0000", 2024-09-18T11:31:27.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7353000000000001e-05 2024-09-18T11:31:27.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383081+0000", 2024-09-18T11:31:27.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9170000000000001e-06 2024-09-18T11:31:27.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383150+0000", 2024-09-18T11:31:27.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8503999999999993e-05 2024-09-18T11:31:27.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:27.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383158+0000", 2024-09-18T11:31:27.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5089999999999993e-06 2024-09-18T11:31:27.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385396+0000", 2024-09-18T11:31:27.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022381850000000002 2024-09-18T11:31:27.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385494+0000", 2024-09-18T11:31:27.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7751000000000007e-05 2024-09-18T11:31:27.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:27.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385697+0000", 2024-09-18T11:31:27.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020247400000000001 2024-09-18T11:31:27.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:27.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.390505+0000", 2024-09-18T11:31:27.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048084039999999996 2024-09-18T11:31:27.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:27.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393148+0000", 2024-09-18T11:31:27.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026432399999999998 2024-09-18T11:31:27.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393194+0000", 2024-09-18T11:31:27.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5259999999999997e-05 2024-09-18T11:31:27.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393233+0000", 2024-09-18T11:31:27.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9659000000000001e-05 2024-09-18T11:31:27.531 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.531 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.531 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.531 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2342 4.f 4:f3c51b3f:::smithi11939506-21:head [stat,delete] snapc 26=[26,25,24,1e,1c,19,18] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:27.531 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:27.531 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.063218328000001, 2024-09-18T11:31:27.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011167565000000001, 2024-09-18T11:31:27.531 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.531 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.532 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:27.532 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:27.532 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:27.532 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2342 2024-09-18T11:31:27.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.532 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:27.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:27.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:31:27.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:27.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:31:27.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1439999999999997e-06 2024-09-18T11:31:27.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386820+0000", 2024-09-18T11:31:27.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2651e-05 2024-09-18T11:31:27.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386825+0000", 2024-09-18T11:31:27.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3150000000000002e-06 2024-09-18T11:31:27.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386868+0000", 2024-09-18T11:31:27.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3832999999999999e-05 2024-09-18T11:31:27.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:27.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:31:27.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0409999999999998e-06 2024-09-18T11:31:27.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390445+0000", 2024-09-18T11:31:27.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035689459999999999 2024-09-18T11:31:27.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390539+0000", 2024-09-18T11:31:27.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3791999999999999e-05 2024-09-18T11:31:27.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:27.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390748+0000", 2024-09-18T11:31:27.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000208863 2024-09-18T11:31:27.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:27.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397659+0000", 2024-09-18T11:31:27.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069110710000000004 2024-09-18T11:31:27.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:27.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397873+0000", 2024-09-18T11:31:27.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021343999999999999 2024-09-18T11:31:27.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397923+0000", 2024-09-18T11:31:27.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0300000000000003e-05 2024-09-18T11:31:27.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397970+0000", 2024-09-18T11:31:27.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6968999999999997e-05 2024-09-18T11:31:27.539 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.539 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.539 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.539 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2429 4.b 4:d8d14c58:::smithi11939506-49:head [write 796698~561170,stat] snapc 2b=[2b,2a,26,25,24,1e,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T11:31:27.539 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:27.539 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.434849153999998, 2024-09-18T11:31:27.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097188989999999996, 2024-09-18T11:31:27.539 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.540 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.540 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:27.540 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:27.540 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:27.540 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2429 2024-09-18T11:31:27.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.540 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:27.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:27.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015170+0000", 2024-09-18T11:31:27.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:27.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015435+0000", 2024-09-18T11:31:27.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026533299999999998 2024-09-18T11:31:27.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015455+0000", 2024-09-18T11:31:27.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9623000000000002e-05 2024-09-18T11:31:27.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015461+0000", 2024-09-18T11:31:27.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1859999999999997e-06 2024-09-18T11:31:27.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015535+0000", 2024-09-18T11:31:27.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4555999999999996e-05 2024-09-18T11:31:27.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:27.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015543+0000", 2024-09-18T11:31:27.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9549999999999992e-06 2024-09-18T11:31:27.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017106+0000", 2024-09-18T11:31:27.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015630290000000001 2024-09-18T11:31:27.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017183+0000", 2024-09-18T11:31:27.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6953999999999995e-05 2024-09-18T11:31:27.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:27.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017301+0000", 2024-09-18T11:31:27.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011734 2024-09-18T11:31:27.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:27.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.020225+0000", 2024-09-18T11:31:27.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029244090000000002 2024-09-18T11:31:27.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:27.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024801+0000", 2024-09-18T11:31:27.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045764129999999997 2024-09-18T11:31:27.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024840+0000", 2024-09-18T11:31:27.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8325999999999999e-05 2024-09-18T11:31:27.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024890+0000", 2024-09-18T11:31:27.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0596999999999997e-05 2024-09-18T11:31:27.546 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.547 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.547 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.547 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2769 4.4 4:2cb3e6a5:::smithi11939506-3:head [write 484101~692450] snapc 30=[30,2f,2b,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:31:27.547 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:18.743766+0000", 2024-09-18T11:31:27.547 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.7062549609999991, 2024-09-18T11:31:27.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086202359999999999, 2024-09-18T11:31:27.547 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.547 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.547 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:27.547 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:27.547 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:27.548 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2769 2024-09-18T11:31:27.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.548 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.743766+0000", 2024-09-18T11:31:27.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.743766+0000", 2024-09-18T11:31:27.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.743764+0000", 2024-09-18T11:31:27.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:27.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744376+0000", 2024-09-18T11:31:27.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00061202299999999995 2024-09-18T11:31:27.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744393+0000", 2024-09-18T11:31:27.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6679000000000001e-05 2024-09-18T11:31:27.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744399+0000", 2024-09-18T11:31:27.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4840000000000001e-06 2024-09-18T11:31:27.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744452+0000", 2024-09-18T11:31:27.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2321999999999998e-05 2024-09-18T11:31:27.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744552+0000", 2024-09-18T11:31:27.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100536 2024-09-18T11:31:27.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:27.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744738+0000", 2024-09-18T11:31:27.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000185434 2024-09-18T11:31:27.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:27.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.749297+0000", 2024-09-18T11:31:27.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004559802 2024-09-18T11:31:27.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:27.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.752260+0000", 2024-09-18T11:31:27.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029623000000000002 2024-09-18T11:31:27.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.752338+0000", 2024-09-18T11:31:27.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8330000000000004e-05 2024-09-18T11:31:27.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.752386+0000", 2024-09-18T11:31:27.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:31:27.555 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.555 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.555 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.555 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2917 4.4 4:2c93196f:::smithi11939506-28:head [sparse-read 0~2075838] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:27.556 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.008117+0000", 2024-09-18T11:31:27.556 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.4419034259999997, 2024-09-18T11:31:27.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084318729999999995, 2024-09-18T11:31:27.556 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.556 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:27.556 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:27.557 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:27.557 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:27.557 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2917 2024-09-18T11:31:27.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.557 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.008117+0000", 2024-09-18T11:31:27.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.008117+0000", 2024-09-18T11:31:27.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.008117+0000", 2024-09-18T11:31:27.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.008120+0000", 2024-09-18T11:31:27.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9100000000000001e-06 2024-09-18T11:31:27.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.008126+0000", 2024-09-18T11:31:27.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9699999999999996e-06 2024-09-18T11:31:27.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.008129+0000", 2024-09-18T11:31:27.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5190000000000001e-06 2024-09-18T11:31:27.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.015988+0000", 2024-09-18T11:31:27.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078590699999999993 2024-09-18T11:31:27.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.016095+0000", 2024-09-18T11:31:27.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107357 2024-09-18T11:31:27.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.016549+0000", 2024-09-18T11:31:27.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00045354999999999999 2024-09-18T11:31:27.562 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.562 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.562 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.562 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2937 4.7 4:e301aefd:::smithi11939506-19:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:27.562 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:31:27.563 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.4151801900000001, 2024-09-18T11:31:27.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019287667000000001, 2024-09-18T11:31:27.563 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.563 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:27.563 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:27.563 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:27.563 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:27.563 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2937 2024-09-18T11:31:27.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.563 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:31:27.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:31:27.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:31:27.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:31:27.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034842+0000", 2024-09-18T11:31:27.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.372e-06 2024-09-18T11:31:27.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034846+0000", 2024-09-18T11:31:27.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3919999999999996e-06 2024-09-18T11:31:27.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034848+0000", 2024-09-18T11:31:27.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3539999999999998e-06 2024-09-18T11:31:27.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034893+0000", 2024-09-18T11:31:27.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4861000000000003e-05 2024-09-18T11:31:27.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034965+0000", 2024-09-18T11:31:27.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1860999999999995e-05 2024-09-18T11:31:27.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.054128+0000", 2024-09-18T11:31:27.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019163039 2024-09-18T11:31:27.567 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.567 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.567 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.567 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3020 4.2 4:425afe9c:::smithi11939506-29:35 [set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e99)", 2024-09-18T11:31:27.567 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:27.073196+0000", 2024-09-18T11:31:27.567 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.376824624, 2024-09-18T11:31:27.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085408519999999998, 2024-09-18T11:31:27.567 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.568 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:27.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:27.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:27.568 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3020 2024-09-18T11:31:27.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.568 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073196+0000", 2024-09-18T11:31:27.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073196+0000", 2024-09-18T11:31:27.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073194+0000", 2024-09-18T11:31:27.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:27.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073207+0000", 2024-09-18T11:31:27.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3244000000000001e-05 2024-09-18T11:31:27.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073230+0000", 2024-09-18T11:31:27.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2878999999999999e-05 2024-09-18T11:31:27.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073239+0000", 2024-09-18T11:31:27.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5589999999999994e-06 2024-09-18T11:31:27.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073305+0000", 2024-09-18T11:31:27.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5438000000000006e-05 2024-09-18T11:31:27.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073414+0000", 2024-09-18T11:31:27.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109473 2024-09-18T11:31:27.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:27.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.079319+0000", 2024-09-18T11:31:27.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059050320000000002 2024-09-18T11:31:27.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:27.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.080078+0000", 2024-09-18T11:31:27.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00075877799999999999 2024-09-18T11:31:27.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:27.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.081605+0000", 2024-09-18T11:31:27.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015268549999999999 2024-09-18T11:31:27.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.081681+0000", 2024-09-18T11:31:27.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6396999999999994e-05 2024-09-18T11:31:27.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.081737+0000", 2024-09-18T11:31:27.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5658999999999997e-05 2024-09-18T11:31:27.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.573 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.574 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.574 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:27.575 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:31:27.575 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:27.574+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:27.606 INFO:tasks.rados.rados.0.smithi119.stdout:1386: got (0) Success 2024-09-18T11:31:27.607 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 39 v 211 (ObjNum 259 snap 31 seq_num 259) dirty exists 2024-09-18T11:31:27.607 INFO:tasks.rados.rados.0.smithi119.stdout:1397: expect (ObjNum 175 snap 20 seq_num 175) 2024-09-18T11:31:27.676 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:27.675+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:27.706 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:27.706 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:27.706 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:27.706 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:27.707 INFO:tasks.rados.rados.0.smithi119.stdout:1396: finishing rollback tid 0 to smithi11939506-49 2024-09-18T11:31:27.708 INFO:tasks.rados.rados.0.smithi119.stdout:1396: finishing rollback tid 1 to smithi11939506-49 2024-09-18T11:31:27.708 INFO:tasks.rados.rados.0.smithi119.stdout:1395: expect (ObjNum 194 snap 23 seq_num 194) 2024-09-18T11:31:27.714 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:31:27.777 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:27.776+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:27.813 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:27.813 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:27.813 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:27.813 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:27.813 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:27.818 INFO:tasks.rados.rados.0.smithi119.stdout:1396: finishing rollback tid 2 to smithi11939506-49 2024-09-18T11:31:27.818 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 300 (ObjNum 216 snap 26 seq_num 216) dirty exists 2024-09-18T11:31:27.818 INFO:tasks.rados.rados.0.smithi119.stdout:1393: finishing set_chunk to oid 29 2024-09-18T11:31:27.818 INFO:tasks.rados.rados.0.smithi119.stdout:1398: finishing rollback tid 0 to smithi11939506-19 2024-09-18T11:31:27.818 INFO:tasks.rados.rados.0.smithi119.stdout:1398: finishing rollback tid 1 to smithi11939506-19 2024-09-18T11:31:27.818 INFO:tasks.rados.rados.0.smithi119.stdout:1398: finishing rollback tid 2 to smithi11939506-19 2024-09-18T11:31:27.818 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 174 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-18T11:31:27.818 INFO:tasks.rados.rados.0.smithi119.stdout:1401: seq_num 268 ranges {504276=639087,1666233=695410} 2024-09-18T11:31:27.821 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:31:27.824 INFO:tasks.rados.rados.0.smithi119.stdout:1401: writing smithi11939506-16 from 504276 to 1143363 tid 1 2024-09-18T11:31:27.831 INFO:tasks.rados.rados.0.smithi119.stdout:1401: writing smithi11939506-16 from 1666233 to 2361643 tid 2 2024-09-18T11:31:27.832 INFO:tasks.rados.rados.0.smithi119.stdout:1386: done (9 left) 2024-09-18T11:31:27.832 INFO:tasks.rados.rados.0.smithi119.stdout:1388: done (8 left) 2024-09-18T11:31:27.832 INFO:tasks.rados.rados.0.smithi119.stdout:1393: done (7 left) 2024-09-18T11:31:27.832 INFO:tasks.rados.rados.0.smithi119.stdout:1395: done (6 left) 2024-09-18T11:31:27.832 INFO:tasks.rados.rados.0.smithi119.stdout:1396: done (5 left) 2024-09-18T11:31:27.832 INFO:tasks.rados.rados.0.smithi119.stdout:1397: done (4 left) 2024-09-18T11:31:27.832 INFO:tasks.rados.rados.0.smithi119.stdout:1398: done (3 left) 2024-09-18T11:31:27.832 INFO:tasks.rados.rados.0.smithi119.stdout:1402: read oid 24 snap -1 2024-09-18T11:31:27.832 INFO:tasks.rados.rados.0.smithi119.stdout:1402: expect (ObjNum 258 snap 31 seq_num 258) 2024-09-18T11:31:27.832 INFO:tasks.rados.rados.0.smithi119.stdout:1403: read oid 35 snap -1 2024-09-18T11:31:27.832 INFO:tasks.rados.rados.0.smithi119.stdout:1403: expect (ObjNum 155 snap 20 seq_num 155) 2024-09-18T11:31:27.832 INFO:tasks.rados.rados.0.smithi119.stdout:1404: read oid 21 snap -1 2024-09-18T11:31:27.832 INFO:tasks.rados.rados.0.smithi119.stdout:1404: expect (ObjNum 253 snap 30 seq_num 253) 2024-09-18T11:31:27.832 INFO:tasks.rados.rados.0.smithi119.stdout:1405: write (excl) oid 5 current snap is 31 2024-09-18T11:31:27.832 INFO:tasks.rados.rados.0.smithi119.stdout:1405: seq_num 269 ranges {521083=714336,1724715=555421} 2024-09-18T11:31:27.839 INFO:tasks.rados.rados.0.smithi119.stdout:1405: writing smithi11939506-5 from 521083 to 1235419 tid 1 2024-09-18T11:31:27.845 INFO:tasks.rados.rados.0.smithi119.stdout:1405: writing smithi11939506-5 from 1724715 to 2280136 tid 2 2024-09-18T11:31:27.846 INFO:tasks.rados.rados.0.smithi119.stdout:1406: tier_promote oid 40 2024-09-18T11:31:27.846 INFO:tasks.rados.rados.0.smithi119.stdout:1399: finishing set_chunk to oid 47 2024-09-18T11:31:27.846 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 47 set_chunk returned error code -95 offset: 1945600 length: 318464 2024-09-18T11:31:27.846 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 0 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-18T11:31:27.846 INFO:tasks.rados.rados.0.smithi119.stdout:1400: finishing rollback tid 0 to smithi11939506-13 2024-09-18T11:31:27.846 INFO:tasks.rados.rados.0.smithi119.stdout:1400: finishing rollback tid 1 to smithi11939506-13 2024-09-18T11:31:27.846 INFO:tasks.rados.rados.0.smithi119.stdout:1400: finishing rollback tid 2 to smithi11939506-13 2024-09-18T11:31:27.846 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 13 v 304 (ObjNum 136 snap 16 seq_num 136) dirty exists 2024-09-18T11:31:27.846 INFO:tasks.rados.rados.0.smithi119.stdout:1399: done (7 left) 2024-09-18T11:31:27.846 INFO:tasks.rados.rados.0.smithi119.stdout:1400: done (6 left) 2024-09-18T11:31:27.846 INFO:tasks.rados.rados.0.smithi119.stdout:1407: tier_evict oid 29 2024-09-18T11:31:27.846 INFO:tasks.rados.rados.0.smithi119.stdout:1407: tier_evict oid 29 snap -1 2024-09-18T11:31:27.847 INFO:tasks.rados.rados.0.smithi119.stdout:1401: finishing write tid 1 to smithi11939506-16 2024-09-18T11:31:27.847 INFO:tasks.rados.rados.0.smithi119.stdout:1403: expect (ObjNum 155 snap 20 seq_num 155) 2024-09-18T11:31:27.878 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:27.877+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:27.929 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:27.929 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:27.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:27.930 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:27.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.930 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1046 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'24, mlcod=30'21)", 2024-09-18T11:31:27.930 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:31:27.930 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 71.753554922999996, 2024-09-18T11:31:27.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042519100000000002, 2024-09-18T11:31:27.930 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.930 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.930 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:31:27.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:31:27.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173782+0000", 2024-09-18T11:31:27.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:27.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174207+0000", 2024-09-18T11:31:27.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00042479799999999999 2024-09-18T11:31:27.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174224+0000", 2024-09-18T11:31:27.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7513999999999999e-05 2024-09-18T11:31:27.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174232+0000", 2024-09-18T11:31:27.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0840000000000003e-06 2024-09-18T11:31:27.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174297+0000", 2024-09-18T11:31:27.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5215000000000005e-05 2024-09-18T11:31:27.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174336+0000", 2024-09-18T11:31:27.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9403000000000002e-05 2024-09-18T11:31:27.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.177990+0000", 2024-09-18T11:31:27.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036533709999999999 2024-09-18T11:31:27.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.178035+0000", 2024-09-18T11:31:27.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5495999999999999e-05 2024-09-18T11:31:27.934 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.934 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.934 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.934 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:31:27.934 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:31:27.934 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 67.943964055999999, 2024-09-18T11:31:27.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:31:27.935 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.935 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.935 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:31:27.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:31:27.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:31:27.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:27.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:31:27.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:31:27.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:31:27.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:31:27.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:31:27.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:31:27.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:31:27.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:31:27.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:31:27.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:31:27.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:31:27.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:31:27.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:31:27.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:31:27.939 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.939 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.939 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.939 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:31:27.939 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:27.939 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.709133102999999, 2024-09-18T11:31:27.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:31:27.939 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.939 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.939 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:27.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:27.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:27.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:31:27.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:31:27.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:31:27.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:31:27.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:31:27.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:31:27.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:31:27.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:31:27.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:31:27.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:31:27.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:31:27.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:31:27.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:31:27.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:31:27.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:31:27.943 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.943 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.943 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.944 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:31:27.944 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:31:27.944 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.535226817999998, 2024-09-18T11:31:27.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:31:27.944 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.944 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.944 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:31:27.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:31:27.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:31:27.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:27.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:31:27.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:31:27.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:31:27.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:31:27.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:31:27.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:31:27.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:31:27.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:31:27.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:31:27.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:31:27.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:31:27.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:31:27.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:31:27.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:31:27.948 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.948 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.948 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.948 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1931 3.0 e65/21 3:3c05e498:::866d65857c51f67d56ecbcba33930954810c64ced2a213ca9705911438b0dfb7:head v 65'72, mlcod=58'70)", 2024-09-18T11:31:27.948 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:31:27.948 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.559216800000002, 2024-09-18T11:31:27.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040630859999999996, 2024-09-18T11:31:27.948 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.948 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.948 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:31:27.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:31:27.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368120+0000", 2024-09-18T11:31:27.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:27.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368427+0000", 2024-09-18T11:31:27.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030742600000000002 2024-09-18T11:31:27.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368446+0000", 2024-09-18T11:31:27.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8046e-05 2024-09-18T11:31:27.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368453+0000", 2024-09-18T11:31:27.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1380000000000001e-06 2024-09-18T11:31:27.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368519+0000", 2024-09-18T11:31:27.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6470000000000006e-05 2024-09-18T11:31:27.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368550+0000", 2024-09-18T11:31:27.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0707999999999998e-05 2024-09-18T11:31:27.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.372151+0000", 2024-09-18T11:31:27.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036015470000000001 2024-09-18T11:31:27.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.372184+0000", 2024-09-18T11:31:27.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2994999999999998e-05 2024-09-18T11:31:27.952 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.952 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.952 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.952 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:304 4.6 e69/67 4:67c1765a:::smithi11939506-30:17 v 69'35, mlcod=58'31)", 2024-09-18T11:31:27.952 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:27.953 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.581385724, 2024-09-18T11:31:27.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006377611, 2024-09-18T11:31:27.953 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.953 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.953 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:27.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:27.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:27.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345961+0000", 2024-09-18T11:31:27.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2050000000000008e-06 2024-09-18T11:31:27.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345969+0000", 2024-09-18T11:31:27.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0339999999999997e-06 2024-09-18T11:31:27.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345976+0000", 2024-09-18T11:31:27.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-18T11:31:27.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346026+0000", 2024-09-18T11:31:27.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9286999999999998e-05 2024-09-18T11:31:27.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346050+0000", 2024-09-18T11:31:27.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4397e-05 2024-09-18T11:31:27.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352293+0000", 2024-09-18T11:31:27.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062427170000000001 2024-09-18T11:31:27.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352330+0000", 2024-09-18T11:31:27.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7348999999999998e-05 2024-09-18T11:31:27.956 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.957 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.957 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.957 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:347 4.d e69/22 4:b44229a1:::smithi11939506-42:19 v 69'99, mlcod=65'93)", 2024-09-18T11:31:27.957 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:31:27.957 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.575652529000003, 2024-09-18T11:31:27.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056998760000000004, 2024-09-18T11:31:27.957 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.957 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.957 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:31:27.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:31:27.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351684+0000", 2024-09-18T11:31:27.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:27.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351693+0000", 2024-09-18T11:31:27.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-18T11:31:27.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351704+0000", 2024-09-18T11:31:27.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1077000000000001e-05 2024-09-18T11:31:27.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351709+0000", 2024-09-18T11:31:27.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5419999999999999e-06 2024-09-18T11:31:27.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351956+0000", 2024-09-18T11:31:27.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000247252 2024-09-18T11:31:27.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351975+0000", 2024-09-18T11:31:27.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9201000000000001e-05 2024-09-18T11:31:27.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357334+0000", 2024-09-18T11:31:27.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053588749999999999 2024-09-18T11:31:27.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357385+0000", 2024-09-18T11:31:27.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1406999999999998e-05 2024-09-18T11:31:27.961 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.961 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.962 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.962 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1956 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'37, mlcod=58'31)", 2024-09-18T11:31:27.962 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:27.962 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.574707615999998, 2024-09-18T11:31:27.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010958842, 2024-09-18T11:31:27.962 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.962 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.962 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:27.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:27.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:27.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:27.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352996+0000", 2024-09-18T11:31:27.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036622 2024-09-18T11:31:27.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353011+0000", 2024-09-18T11:31:27.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4849e-05 2024-09-18T11:31:27.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353017+0000", 2024-09-18T11:31:27.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:31:27.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353068+0000", 2024-09-18T11:31:27.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0695000000000002e-05 2024-09-18T11:31:27.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353092+0000", 2024-09-18T11:31:27.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4260000000000002e-05 2024-09-18T11:31:27.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363551+0000", 2024-09-18T11:31:27.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010459154999999999 2024-09-18T11:31:27.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363589+0000", 2024-09-18T11:31:27.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8161000000000003e-05 2024-09-18T11:31:27.966 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.966 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.966 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.966 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1957 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'38, mlcod=65'33)", 2024-09-18T11:31:27.966 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:27.966 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.568291748999997, 2024-09-18T11:31:27.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050499400000000002, 2024-09-18T11:31:27.967 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.967 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.967 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:27.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:27.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:27.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359214+0000", 2024-09-18T11:31:27.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000168047 2024-09-18T11:31:27.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359227+0000", 2024-09-18T11:31:27.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3124e-05 2024-09-18T11:31:27.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359235+0000", 2024-09-18T11:31:27.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0760000000000003e-06 2024-09-18T11:31:27.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359287+0000", 2024-09-18T11:31:27.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.151e-05 2024-09-18T11:31:27.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359310+0000", 2024-09-18T11:31:27.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-09-18T11:31:27.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364065+0000", 2024-09-18T11:31:27.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047549439999999997 2024-09-18T11:31:27.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364096+0000", 2024-09-18T11:31:27.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1423000000000001e-05 2024-09-18T11:31:27.970 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.970 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.971 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.971 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2300 4.4 e77/22 4:2c93196f:::smithi11939506-28:head v 77'94, mlcod=77'90)", 2024-09-18T11:31:27.971 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:31:27.971 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.939229530999999, 2024-09-18T11:31:27.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050425380000000001, 2024-09-18T11:31:27.971 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.971 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.971 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:31:27.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:31:27.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988108+0000", 2024-09-18T11:31:27.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:27.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988388+0000", 2024-09-18T11:31:27.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028087700000000001 2024-09-18T11:31:27.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988405+0000", 2024-09-18T11:31:27.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6565000000000001e-05 2024-09-18T11:31:27.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988412+0000", 2024-09-18T11:31:27.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9709999999999998e-06 2024-09-18T11:31:27.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988464+0000", 2024-09-18T11:31:27.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1949e-05 2024-09-18T11:31:27.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988490+0000", 2024-09-18T11:31:27.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5780999999999999e-05 2024-09-18T11:31:27.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993112+0000", 2024-09-18T11:31:27.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046228399999999996 2024-09-18T11:31:27.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993151+0000", 2024-09-18T11:31:27.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8587999999999997e-05 2024-09-18T11:31:27.975 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.975 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.975 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.975 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.13:63 3.0 e78/21 3:24ebb740:::smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 78'80, mlcod=73'78)", 2024-09-18T11:31:27.975 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:31:27.975 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.522240114999999, 2024-09-18T11:31:27.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052262230000000003, 2024-09-18T11:31:27.976 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.976 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.976 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:31:27.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:31:27.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405097+0000", 2024-09-18T11:31:27.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405103+0000", 2024-09-18T11:31:27.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8780000000000003e-06 2024-09-18T11:31:27.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405117+0000", 2024-09-18T11:31:27.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3441000000000001e-05 2024-09-18T11:31:27.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405121+0000", 2024-09-18T11:31:27.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7099999999999998e-06 2024-09-18T11:31:27.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405158+0000", 2024-09-18T11:31:27.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6477999999999999e-05 2024-09-18T11:31:27.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405173+0000", 2024-09-18T11:31:27.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4796e-05 2024-09-18T11:31:27.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410285+0000", 2024-09-18T11:31:27.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051118919999999998 2024-09-18T11:31:27.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410324+0000", 2024-09-18T11:31:27.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9616999999999997e-05 2024-09-18T11:31:27.980 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.980 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.980 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.980 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:451 4.6 e78/67 4:6f86ffcc:::smithi11939506-15:26 v 78'58, mlcod=77'53)", 2024-09-18T11:31:27.980 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.414259+0000", 2024-09-18T11:31:27.980 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.513079531999999, 2024-09-18T11:31:27.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041821610000000002, 2024-09-18T11:31:27.980 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.980 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.980 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414259+0000", 2024-09-18T11:31:27.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414259+0000", 2024-09-18T11:31:27.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414258+0000", 2024-09-18T11:31:27.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414264+0000", 2024-09-18T11:31:27.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1850000000000001e-06 2024-09-18T11:31:27.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414273+0000", 2024-09-18T11:31:27.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7290000000000006e-06 2024-09-18T11:31:27.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414278+0000", 2024-09-18T11:31:27.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8470000000000003e-06 2024-09-18T11:31:27.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414315+0000", 2024-09-18T11:31:27.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7462000000000001e-05 2024-09-18T11:31:27.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414334+0000", 2024-09-18T11:31:27.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9080999999999999e-05 2024-09-18T11:31:27.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.418419+0000", 2024-09-18T11:31:27.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040846540000000001 2024-09-18T11:31:27.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.418441+0000", 2024-09-18T11:31:27.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1844999999999998e-05 2024-09-18T11:31:27.984 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.984 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.984 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.984 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.13:18 3.0 e91/21 3:35e3c617:::2e64dd273da7619bd64b294f52905c1b9405a1ced2a8fd3c0b5bc481bc9732a6:head v 91'89, mlcod=85'87)", 2024-09-18T11:31:27.984 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:31:27.984 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.905464704, 2024-09-18T11:31:27.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048558539999999997, 2024-09-18T11:31:27.985 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.985 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.985 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:31:27.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:31:27.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:31:27.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021880+0000", 2024-09-18T11:31:27.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5860000000000001e-06 2024-09-18T11:31:27.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021890+0000", 2024-09-18T11:31:27.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8290000000000003e-06 2024-09-18T11:31:27.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021895+0000", 2024-09-18T11:31:27.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2480000000000004e-06 2024-09-18T11:31:27.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022166+0000", 2024-09-18T11:31:27.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027073499999999998 2024-09-18T11:31:27.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022184+0000", 2024-09-18T11:31:27.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8082000000000001e-05 2024-09-18T11:31:27.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026693+0000", 2024-09-18T11:31:27.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045093479999999998 2024-09-18T11:31:27.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026729+0000", 2024-09-18T11:31:27.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6282999999999998e-05 2024-09-18T11:31:27.988 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.988 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.988 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.989 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:573 4.4 e91/22 4:2c93196f:::smithi11939506-28:26 v 91'135, mlcod=90'131)", 2024-09-18T11:31:27.989 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:31:27.989 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.902998115000001, 2024-09-18T11:31:27.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050722739999999999, 2024-09-18T11:31:27.989 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.989 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.989 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:31:27.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:31:27.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024339+0000", 2024-09-18T11:31:27.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:27.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024349+0000", 2024-09-18T11:31:27.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6339999999999996e-06 2024-09-18T11:31:27.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024362+0000", 2024-09-18T11:31:27.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-18T11:31:27.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024368+0000", 2024-09-18T11:31:27.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8649999999999998e-06 2024-09-18T11:31:27.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024431+0000", 2024-09-18T11:31:27.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2780999999999994e-05 2024-09-18T11:31:27.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024448+0000", 2024-09-18T11:31:27.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7388999999999999e-05 2024-09-18T11:31:27.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029385+0000", 2024-09-18T11:31:27.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004936523 2024-09-18T11:31:27.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029412+0000", 2024-09-18T11:31:27.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7702999999999999e-05 2024-09-18T11:31:27.993 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.993 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.993 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.993 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:574 4.4 e91/22 4:2cb3e6a5:::smithi11939506-3:1e v 91'137, mlcod=90'131)", 2024-09-18T11:31:27.993 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:31:27.993 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.902560072, 2024-09-18T11:31:27.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070802850000000004, 2024-09-18T11:31:27.993 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.994 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.994 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:31:27.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:31:27.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024777+0000", 2024-09-18T11:31:27.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:27.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:27.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024785+0000", 2024-09-18T11:31:27.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8099999999999998e-06 2024-09-18T11:31:27.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:27.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024794+0000", 2024-09-18T11:31:27.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5839999999999995e-06 2024-09-18T11:31:27.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:27.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024798+0000", 2024-09-18T11:31:27.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.031e-06 2024-09-18T11:31:27.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:27.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024830+0000", 2024-09-18T11:31:27.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2268999999999999e-05 2024-09-18T11:31:27.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:27.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024847+0000", 2024-09-18T11:31:27.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7306999999999999e-05 2024-09-18T11:31:27.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:27.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031834+0000", 2024-09-18T11:31:27.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069868380000000004 2024-09-18T11:31:27.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:27.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031858+0000", 2024-09-18T11:31:27.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4082000000000001e-05 2024-09-18T11:31:27.997 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.998 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:27.998 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:27.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.998 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:541 4.d e91/22 4:b9bb642b:::smithi11939506-27:19 v 91'158, mlcod=91'154)", 2024-09-18T11:31:27.998 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:31:27.998 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.89517938, 2024-09-18T11:31:27.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052185, 2024-09-18T11:31:27.998 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:27.998 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:27.998 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:27.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:27.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:31:27.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:27.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:31:27.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:27.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:27.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032158+0000", 2024-09-18T11:31:27.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:27.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:27.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:27.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032165+0000", 2024-09-18T11:31:28.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-09-18T11:31:28.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032176+0000", 2024-09-18T11:31:28.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1460999999999999e-05 2024-09-18T11:31:28.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032182+0000", 2024-09-18T11:31:28.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0870000000000003e-06 2024-09-18T11:31:28.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032224+0000", 2024-09-18T11:31:28.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1372999999999998e-05 2024-09-18T11:31:28.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032246+0000", 2024-09-18T11:31:28.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2158e-05 2024-09-18T11:31:28.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:28.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037348+0000", 2024-09-18T11:31:28.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051025319999999999 2024-09-18T11:31:28.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037377+0000", 2024-09-18T11:31:28.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8937000000000001e-05 2024-09-18T11:31:28.002 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.002 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.002 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.002 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:582 4.6 e91/67 4:6f86ffcc:::smithi11939506-15:26 v 91'78, mlcod=90'72)", 2024-09-18T11:31:28.002 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.043567+0000", 2024-09-18T11:31:28.003 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.883771429999999, 2024-09-18T11:31:28.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004448446, 2024-09-18T11:31:28.003 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.003 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.003 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043567+0000", 2024-09-18T11:31:28.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043567+0000", 2024-09-18T11:31:28.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043566+0000", 2024-09-18T11:31:28.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043571+0000", 2024-09-18T11:31:28.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.181e-06 2024-09-18T11:31:28.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043579+0000", 2024-09-18T11:31:28.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9559999999999997e-06 2024-09-18T11:31:28.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043583+0000", 2024-09-18T11:31:28.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0770000000000001e-06 2024-09-18T11:31:28.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043610+0000", 2024-09-18T11:31:28.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6778e-05 2024-09-18T11:31:28.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043633+0000", 2024-09-18T11:31:28.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2904e-05 2024-09-18T11:31:28.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:28.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.047991+0000", 2024-09-18T11:31:28.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043582910000000002 2024-09-18T11:31:28.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.048015+0000", 2024-09-18T11:31:28.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3813000000000001e-05 2024-09-18T11:31:28.007 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.007 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.007 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.007 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2754 4.4 e92/22 4:2cb3e6a5:::smithi11939506-3:head v 92'147, mlcod=92'143)", 2024-09-18T11:31:28.007 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:17.887302+0000", 2024-09-18T11:31:28.007 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 10.040036002000001, 2024-09-18T11:31:28.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044319160000000002, 2024-09-18T11:31:28.007 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.008 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.008 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887302+0000", 2024-09-18T11:31:28.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887302+0000", 2024-09-18T11:31:28.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887301+0000", 2024-09-18T11:31:28.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:28.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887313+0000", 2024-09-18T11:31:28.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2468e-05 2024-09-18T11:31:28.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887331+0000", 2024-09-18T11:31:28.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7362000000000001e-05 2024-09-18T11:31:28.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887338+0000", 2024-09-18T11:31:28.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7980000000000006e-06 2024-09-18T11:31:28.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887416+0000", 2024-09-18T11:31:28.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7577999999999995e-05 2024-09-18T11:31:28.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887441+0000", 2024-09-18T11:31:28.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.531e-05 2024-09-18T11:31:28.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:28.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.891671+0000", 2024-09-18T11:31:28.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00422989 2024-09-18T11:31:28.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.891734+0000", 2024-09-18T11:31:28.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2885999999999994e-05 2024-09-18T11:31:28.011 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.011 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.011 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.012 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2769 4.4 e92/22 4:2cb3e6a5:::smithi11939506-3:head v 92'150, mlcod=92'148)", 2024-09-18T11:31:28.012 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:18.744829+0000", 2024-09-18T11:31:28.012 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.1825086720000009, 2024-09-18T11:31:28.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043433719999999999, 2024-09-18T11:31:28.012 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.012 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.012 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744829+0000", 2024-09-18T11:31:28.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744829+0000", 2024-09-18T11:31:28.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744828+0000", 2024-09-18T11:31:28.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:28.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.745480+0000", 2024-09-18T11:31:28.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00065170400000000002 2024-09-18T11:31:28.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.745499+0000", 2024-09-18T11:31:28.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9218000000000001e-05 2024-09-18T11:31:28.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.745507+0000", 2024-09-18T11:31:28.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1950000000000003e-06 2024-09-18T11:31:28.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.745567+0000", 2024-09-18T11:31:28.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0040000000000001e-05 2024-09-18T11:31:28.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.745594+0000", 2024-09-18T11:31:28.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6803000000000001e-05 2024-09-18T11:31:28.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:28.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.749126+0000", 2024-09-18T11:31:28.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003532622 2024-09-18T11:31:28.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.749173+0000", 2024-09-18T11:31:28.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6334000000000001e-05 2024-09-18T11:31:28.016 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.016 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.016 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.016 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2928 4.d e97/22 4:b81644f2:::smithi11939506-34:head v 97'181, mlcod=95'179)", 2024-09-18T11:31:28.016 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.035301+0000", 2024-09-18T11:31:28.016 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.8920368779999999, 2024-09-18T11:31:28.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040500099999999997, 2024-09-18T11:31:28.017 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.017 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.017 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.035301+0000", 2024-09-18T11:31:28.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.035301+0000", 2024-09-18T11:31:28.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.035301+0000", 2024-09-18T11:31:28.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.035695+0000", 2024-09-18T11:31:28.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000394101 2024-09-18T11:31:28.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.035708+0000", 2024-09-18T11:31:28.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3545e-05 2024-09-18T11:31:28.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.035713+0000", 2024-09-18T11:31:28.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6070000000000001e-06 2024-09-18T11:31:28.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.035760+0000", 2024-09-18T11:31:28.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7117000000000003e-05 2024-09-18T11:31:28.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.035782+0000", 2024-09-18T11:31:28.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1834000000000001e-05 2024-09-18T11:31:28.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:28.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.039297+0000", 2024-09-18T11:31:28.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035154560000000001 2024-09-18T11:31:28.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.039351+0000", 2024-09-18T11:31:28.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4005999999999998e-05 2024-09-18T11:31:28.020 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.020 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.021 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.021 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.021 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.021 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:28.022 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:31:28.023 INFO:tasks.rados.rados.0.smithi119.stdout:1402: expect (ObjNum 258 snap 31 seq_num 258) 2024-09-18T11:31:28.023 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:28.021+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:28.044 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is unset 2024-09-18T11:31:28.123 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:28.122+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:28.143 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:28.143 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:28.143 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:28.143 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:28.151 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:31:28.163 INFO:tasks.rados.rados.0.smithi119.stdout:1401: finishing write tid 2 to smithi11939506-16 2024-09-18T11:31:28.163 INFO:tasks.rados.rados.0.smithi119.stdout:1401: finishing write tid 3 to smithi11939506-16 2024-09-18T11:31:28.163 INFO:tasks.rados.rados.0.smithi119.stdout:1401: finishing write tid 4 to smithi11939506-16 2024-09-18T11:31:28.163 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 221 (ObjNum 268 snap 31 seq_num 268) dirty exists 2024-09-18T11:31:28.163 INFO:tasks.rados.rados.0.smithi119.stdout:1401: left oid 16 (ObjNum 268 snap 31 seq_num 268) 2024-09-18T11:31:28.163 INFO:tasks.rados.rados.0.smithi119.stdout:1404: expect (ObjNum 253 snap 30 seq_num 253) 2024-09-18T11:31:28.224 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:28.222+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:28.250 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:28.250 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:28.250 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:28.250 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:28.250 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:28.259 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:31:28.283 INFO:tasks.rados.rados.0.smithi119.stdout:1405: finishing write tid 1 to smithi11939506-5 2024-09-18T11:31:28.284 INFO:tasks.rados.rados.0.smithi119.stdout:1407: got (22) Invalid argument 2024-09-18T11:31:28.284 INFO:tasks.rados.rados.0.smithi119.stdout:1406: got (0) Success 2024-09-18T11:31:28.284 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 90 (ObjNum 175 snap 20 seq_num 175) dirty exists 2024-09-18T11:31:28.284 INFO:tasks.rados.rados.0.smithi119.stdout:1405: finishing write tid 2 to smithi11939506-5 2024-09-18T11:31:28.284 INFO:tasks.rados.rados.0.smithi119.stdout:1401: done (6 left) 2024-09-18T11:31:28.284 INFO:tasks.rados.rados.0.smithi119.stdout:1402: done (5 left) 2024-09-18T11:31:28.284 INFO:tasks.rados.rados.0.smithi119.stdout:1403: done (4 left) 2024-09-18T11:31:28.284 INFO:tasks.rados.rados.0.smithi119.stdout:1404: done (3 left) 2024-09-18T11:31:28.284 INFO:tasks.rados.rados.0.smithi119.stdout:1406: done (2 left) 2024-09-18T11:31:28.284 INFO:tasks.rados.rados.0.smithi119.stdout:1407: done (1 left) 2024-09-18T11:31:28.284 INFO:tasks.rados.rados.0.smithi119.stdout:1408: snap_remove snap 28 2024-09-18T11:31:28.327 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:28.326+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:28.383 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:28.383 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:28.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:28.383 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:28.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.383 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:28.383 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:28.383 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.211225166999995, 2024-09-18T11:31:28.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:31:28.383 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.383 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:28.383 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:28.384 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:28.384 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.384 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:31:28.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.384 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:28.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:28.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:31:28.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:28.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:31:28.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:31:28.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:31:28.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:31:28.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:31:28.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:31:28.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:31:28.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:31:28.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:31:28.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:31:28.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:31:28.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:31:28.387 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.387 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.387 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.388 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1001 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:28.388 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:28.388 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.216073889, 2024-09-18T11:31:28.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007461082, 2024-09-18T11:31:28.388 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.388 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:28.388 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:28.388 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:28.388 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.388 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1001 2024-09-18T11:31:28.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.388 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:28.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:28.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164570+0000", 2024-09-18T11:31:28.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164574+0000", 2024-09-18T11:31:28.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.822e-06 2024-09-18T11:31:28.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164582+0000", 2024-09-18T11:31:28.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3410000000000003e-06 2024-09-18T11:31:28.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164587+0000", 2024-09-18T11:31:28.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769999999999999e-06 2024-09-18T11:31:28.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164702+0000", 2024-09-18T11:31:28.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115705 2024-09-18T11:31:28.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:28.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164709+0000", 2024-09-18T11:31:28.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-09-18T11:31:28.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170296+0000", 2024-09-18T11:31:28.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055876390000000001 2024-09-18T11:31:28.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170418+0000", 2024-09-18T11:31:28.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121549 2024-09-18T11:31:28.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172032+0000", 2024-09-18T11:31:28.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00161407 2024-09-18T11:31:28.393 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.393 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.393 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.393 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:28.393 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:28.393 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.216047173000007, 2024-09-18T11:31:28.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:31:28.393 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.394 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:28.394 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:28.394 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:28.394 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.394 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:31:28.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.394 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:28.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:28.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:28.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:31:28.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:31:28.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:31:28.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:31:28.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:31:28.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:31:28.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:31:28.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:31:28.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:28.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:31:28.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:31:28.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:31:28.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:31:28.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:31:28.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:31:28.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:31:28.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:31:28.398 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.398 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.398 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.399 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:28.399 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:28.399 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.216028883999996, 2024-09-18T11:31:28.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:31:28.399 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.399 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:28.399 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:28.399 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:28.399 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.399 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:31:28.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.399 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:28.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:28.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:31:28.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:31:28.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:31:28.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:31:28.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:31:28.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:31:28.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:31:28.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:31:28.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:31:28.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:28.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:31:28.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:31:28.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:31:28.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:31:28.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:31:28.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:31:28.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:31:28.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:31:28.404 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.404 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.404 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.404 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:28.404 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:28.404 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.215968114999995, 2024-09-18T11:31:28.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:31:28.404 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.404 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:28.404 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:28.405 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:28.405 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.405 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:31:28.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.405 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:28.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:28.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:31:28.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:31:28.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:31:28.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:31:28.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:31:28.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:31:28.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:31:28.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:31:28.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:31:28.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:28.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:31:28.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:31:28.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:31:28.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:31:28.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:31:28.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:31:28.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:31:28.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:31:28.409 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.409 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.409 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.409 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:28.409 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:28.410 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.215945794000007, 2024-09-18T11:31:28.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:31:28.410 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.410 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:28.410 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:28.410 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:28.410 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.410 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:31:28.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.410 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:28.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:28.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:31:28.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:31:28.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:31:28.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:31:28.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:31:28.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:31:28.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:31:28.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:31:28.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:31:28.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:28.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:31:28.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:31:28.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:31:28.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:31:28.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:31:28.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:31:28.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:31:28.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:31:28.414 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.414 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.415 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.415 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:31:28.415 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:28.415 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.434820911000003, 2024-09-18T11:31:28.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:31:28.415 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.415 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.415 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:28.415 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:28.415 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.415 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:31:28.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.416 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:28.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:28.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:31:28.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:31:28.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:31:28.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:31:28.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:31:28.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:31:28.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:31:28.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:31:28.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:31:28.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:31:28.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:31:28.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:31:28.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:28.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:31:28.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:31:28.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:28.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:31:28.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:31:28.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:28.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:31:28.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:31:28.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:28.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:31:28.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:31:28.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:31:28.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:31:28.420 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.420 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.420 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.421 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1661 4.0 4:019b6e09:::smithi11939506-40:17 [read 0~3445402,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:28.421 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:28.421 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.030748207999999, 2024-09-18T11:31:28.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075190880000000002, 2024-09-18T11:31:28.421 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.421 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:28.421 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:28.421 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:28.421 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.421 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1661 2024-09-18T11:31:28.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.421 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:28.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:28.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:28.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349899+0000", 2024-09-18T11:31:28.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9749999999999999e-06 2024-09-18T11:31:28.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349906+0000", 2024-09-18T11:31:28.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9290000000000001e-06 2024-09-18T11:31:28.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349912+0000", 2024-09-18T11:31:28.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9900000000000002e-06 2024-09-18T11:31:28.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355645+0000", 2024-09-18T11:31:28.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057329549999999996 2024-09-18T11:31:28.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355776+0000", 2024-09-18T11:31:28.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130975 2024-09-18T11:31:28.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357416+0000", 2024-09-18T11:31:28.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001639649 2024-09-18T11:31:28.425 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.425 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.425 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.425 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1761 4.0 4:0aec5e1c:::smithi11939506-50:head [write 769787~704463,stat] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:28.425 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:28.425 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.078255794, 2024-09-18T11:31:28.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074082879999999999, 2024-09-18T11:31:28.426 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.426 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.426 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:28.426 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:28.426 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.426 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1761 2024-09-18T11:31:28.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.426 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:28.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:28.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302386+0000", 2024-09-18T11:31:28.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:28.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302889+0000", 2024-09-18T11:31:28.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050318499999999996 2024-09-18T11:31:28.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302930+0000", 2024-09-18T11:31:28.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0198999999999999e-05 2024-09-18T11:31:28.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302965+0000", 2024-09-18T11:31:28.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5292e-05 2024-09-18T11:31:28.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303046+0000", 2024-09-18T11:31:28.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1254999999999994e-05 2024-09-18T11:31:28.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:28.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303054+0000", 2024-09-18T11:31:28.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2800000000000003e-06 2024-09-18T11:31:28.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304658+0000", 2024-09-18T11:31:28.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016039520000000001 2024-09-18T11:31:28.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304724+0000", 2024-09-18T11:31:28.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5888000000000003e-05 2024-09-18T11:31:28.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:28.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304853+0000", 2024-09-18T11:31:28.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012912700000000001 2024-09-18T11:31:28.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:28.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309084+0000", 2024-09-18T11:31:28.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042311229999999998 2024-09-18T11:31:28.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:28.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309738+0000", 2024-09-18T11:31:28.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00065401399999999996 2024-09-18T11:31:28.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:28.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309770+0000", 2024-09-18T11:31:28.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1652000000000001e-05 2024-09-18T11:31:28.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309797+0000", 2024-09-18T11:31:28.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7052000000000001e-05 2024-09-18T11:31:28.432 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.432 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.432 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.432 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1951 4.0 4:019b6e09:::smithi11939506-40:13 [sparse-read 0~2626478] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:28.432 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.339659+0000", 2024-09-18T11:31:28.433 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.040986087999997, 2024-09-18T11:31:28.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067895999999999998, 2024-09-18T11:31:28.433 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.433 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:28.433 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:28.433 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:28.433 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.433 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1951 2024-09-18T11:31:28.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.433 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339659+0000", 2024-09-18T11:31:28.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339659+0000", 2024-09-18T11:31:28.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339658+0000", 2024-09-18T11:31:28.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339661+0000", 2024-09-18T11:31:28.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3749999999999999e-06 2024-09-18T11:31:28.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339668+0000", 2024-09-18T11:31:28.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1380000000000001e-06 2024-09-18T11:31:28.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339675+0000", 2024-09-18T11:31:28.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7499999999999997e-06 2024-09-18T11:31:28.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339748+0000", 2024-09-18T11:31:28.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2885000000000004e-05 2024-09-18T11:31:28.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:28.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339751+0000", 2024-09-18T11:31:28.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.446e-06 2024-09-18T11:31:28.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345249+0000", 2024-09-18T11:31:28.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054983360000000004 2024-09-18T11:31:28.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345332+0000", 2024-09-18T11:31:28.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3245999999999995e-05 2024-09-18T11:31:28.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346448+0000", 2024-09-18T11:31:28.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0011160530000000001 2024-09-18T11:31:28.437 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.437 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.437 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.438 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:306 4.f e69/22 4:f5d7df41:::smithi11939506-16:17 v 69'94, mlcod=65'88)", 2024-09-18T11:31:28.438 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:28.438 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.029529076999999, 2024-09-18T11:31:28.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068024720000000004, 2024-09-18T11:31:28.438 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.438 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.438 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:28.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:28.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351115+0000", 2024-09-18T11:31:28.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351121+0000", 2024-09-18T11:31:28.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-09-18T11:31:28.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351126+0000", 2024-09-18T11:31:28.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5969999999999999e-06 2024-09-18T11:31:28.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351147+0000", 2024-09-18T11:31:28.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1160000000000001e-05 2024-09-18T11:31:28.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351518+0000", 2024-09-18T11:31:28.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00037079300000000001 2024-09-18T11:31:28.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351536+0000", 2024-09-18T11:31:28.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8168999999999998e-05 2024-09-18T11:31:28.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:28.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357870+0000", 2024-09-18T11:31:28.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063332650000000002 2024-09-18T11:31:28.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357918+0000", 2024-09-18T11:31:28.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8347999999999999e-05 2024-09-18T11:31:28.442 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.442 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.442 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.442 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2022 4.0 4:0aec5e1c:::smithi11939506-50:head [write 457190~552096,stat] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-18T11:31:28.442 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:28.442 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.384490939999999, 2024-09-18T11:31:28.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006803974, 2024-09-18T11:31:28.442 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.442 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.442 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:28.442 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:28.442 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.443 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2022 2024-09-18T11:31:28.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.443 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:28.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:28.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996151+0000", 2024-09-18T11:31:28.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:28.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996461+0000", 2024-09-18T11:31:28.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000309587 2024-09-18T11:31:28.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996481+0000", 2024-09-18T11:31:28.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9949000000000002e-05 2024-09-18T11:31:28.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996489+0000", 2024-09-18T11:31:28.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9129999999999992e-06 2024-09-18T11:31:28.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996562+0000", 2024-09-18T11:31:28.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2807999999999998e-05 2024-09-18T11:31:28.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996687+0000", 2024-09-18T11:31:28.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012458200000000001 2024-09-18T11:31:28.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:28.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996909+0000", 2024-09-18T11:31:28.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222124 2024-09-18T11:31:28.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:28.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.000741+0000", 2024-09-18T11:31:28.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038320720000000002 2024-09-18T11:31:28.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:28.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002854+0000", 2024-09-18T11:31:28.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002113228 2024-09-18T11:31:28.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:28.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002911+0000", 2024-09-18T11:31:28.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7192999999999999e-05 2024-09-18T11:31:28.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002958+0000", 2024-09-18T11:31:28.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6242000000000002e-05 2024-09-18T11:31:28.447 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.448 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.448 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.448 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2503 3.1 e85/67 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head v 85'117, mlcod=84'115)", 2024-09-18T11:31:28.448 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:31:28.448 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.403422224, 2024-09-18T11:31:28.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086527619999999996, 2024-09-18T11:31:28.448 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.448 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.448 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:31:28.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:31:28.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977220+0000", 2024-09-18T11:31:28.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:28.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980771+0000", 2024-09-18T11:31:28.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035505200000000002 2024-09-18T11:31:28.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980797+0000", 2024-09-18T11:31:28.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.618e-05 2024-09-18T11:31:28.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980807+0000", 2024-09-18T11:31:28.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0433e-05 2024-09-18T11:31:28.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980861+0000", 2024-09-18T11:31:28.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3415999999999998e-05 2024-09-18T11:31:28.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980885+0000", 2024-09-18T11:31:28.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4212000000000001e-05 2024-09-18T11:31:28.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:28.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985826+0000", 2024-09-18T11:31:28.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049412450000000004 2024-09-18T11:31:28.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985875+0000", 2024-09-18T11:31:28.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9116000000000002e-05 2024-09-18T11:31:28.452 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.452 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.452 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.452 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2648 4.0 4:059d8d95:::smithi11939506-32:head [read 0~3647119,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:28.452 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:31:28.452 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.049333854, 2024-09-18T11:31:28.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0076286949999999996, 2024-09-18T11:31:28.453 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.453 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:28.453 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:28.453 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:28.453 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.453 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2648 2024-09-18T11:31:28.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.453 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:31:28.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:31:28.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331309+0000", 2024-09-18T11:31:28.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:28.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331321+0000", 2024-09-18T11:31:28.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:31:28.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331340+0000", 2024-09-18T11:31:28.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8227e-05 2024-09-18T11:31:28.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331348+0000", 2024-09-18T11:31:28.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4640000000000006e-06 2024-09-18T11:31:28.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331406+0000", 2024-09-18T11:31:28.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7445000000000003e-05 2024-09-18T11:31:28.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331521+0000", 2024-09-18T11:31:28.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115945 2024-09-18T11:31:28.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338939+0000", 2024-09-18T11:31:28.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074179859999999997 2024-09-18T11:31:28.456 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.457 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.457 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.457 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2649 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:28.457 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:31:28.457 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.049278855000001, 2024-09-18T11:31:28.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566240000000007, 2024-09-18T11:31:28.457 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.457 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:28.457 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:28.457 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:28.457 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.458 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2649 2024-09-18T11:31:28.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.458 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:31:28.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:31:28.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331365+0000", 2024-09-18T11:31:28.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331369+0000", 2024-09-18T11:31:28.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.551e-06 2024-09-18T11:31:28.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331377+0000", 2024-09-18T11:31:28.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8769999999999993e-06 2024-09-18T11:31:28.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331381+0000", 2024-09-18T11:31:28.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.16e-06 2024-09-18T11:31:28.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338973+0000", 2024-09-18T11:31:28.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075918460000000002 2024-09-18T11:31:28.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339048+0000", 2024-09-18T11:31:28.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5813000000000001e-05 2024-09-18T11:31:28.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339322+0000", 2024-09-18T11:31:28.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027399899999999998 2024-09-18T11:31:28.461 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.461 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.461 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.461 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2650 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:28.461 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:28.462 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.049258636999999, 2024-09-18T11:31:28.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084392180000000001, 2024-09-18T11:31:28.462 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.462 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:28.462 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:28.462 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:28.462 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.462 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2650 2024-09-18T11:31:28.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.462 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:28.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:28.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:28.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331388+0000", 2024-09-18T11:31:28.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3429999999999998e-06 2024-09-18T11:31:28.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331394+0000", 2024-09-18T11:31:28.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.19e-06 2024-09-18T11:31:28.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331398+0000", 2024-09-18T11:31:28.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.495e-06 2024-09-18T11:31:28.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339362+0000", 2024-09-18T11:31:28.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079641799999999995 2024-09-18T11:31:28.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339460+0000", 2024-09-18T11:31:28.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8329e-05 2024-09-18T11:31:28.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339825+0000", 2024-09-18T11:31:28.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036510500000000001 2024-09-18T11:31:28.466 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.466 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.466 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.466 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:119 3.2 3:5595a68d:::07854d2fef297a06ba81685e660c332de36d5d18d546927d30daad6d7fda1541:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:28.466 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:31:28.466 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.3371068800000003, 2024-09-18T11:31:28.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074145979999999997, 2024-09-18T11:31:28.466 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.466 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.466 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:31:28.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:31:28.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043535+0000", 2024-09-18T11:31:28.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:28.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043970+0000", 2024-09-18T11:31:28.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000435148 2024-09-18T11:31:28.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043993+0000", 2024-09-18T11:31:28.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3113999999999999e-05 2024-09-18T11:31:28.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044003+0000", 2024-09-18T11:31:28.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5310000000000007e-06 2024-09-18T11:31:28.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044062+0000", 2024-09-18T11:31:28.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9487000000000002e-05 2024-09-18T11:31:28.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:28.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044071+0000", 2024-09-18T11:31:28.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1610000000000003e-06 2024-09-18T11:31:28.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046423+0000", 2024-09-18T11:31:28.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023523680000000001 2024-09-18T11:31:28.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046502+0000", 2024-09-18T11:31:28.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9339999999999996e-05 2024-09-18T11:31:28.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:28.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046664+0000", 2024-09-18T11:31:28.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016185700000000001 2024-09-18T11:31:28.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:28.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050870+0000", 2024-09-18T11:31:28.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042060070000000003 2024-09-18T11:31:28.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:28.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050878+0000", 2024-09-18T11:31:28.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8960000000000003e-06 2024-09-18T11:31:28.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:28.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050918+0000", 2024-09-18T11:31:28.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0367000000000002e-05 2024-09-18T11:31:28.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050952+0000", 2024-09-18T11:31:28.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3989999999999998e-05 2024-09-18T11:31:28.472 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.472 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.472 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.472 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:124 3.2 3:4326e841:::a6567eb8c0be0365cdfccf2ece66d60995301bc908261a0c1daff039d2080a5f:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:28.472 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044017+0000", 2024-09-18T11:31:28.473 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.3366272700000001, 2024-09-18T11:31:28.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070611140000000003, 2024-09-18T11:31:28.473 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.473 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.473 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044017+0000", 2024-09-18T11:31:28.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044017+0000", 2024-09-18T11:31:28.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044016+0000", 2024-09-18T11:31:28.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:28.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044102+0000", 2024-09-18T11:31:28.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5756999999999999e-05 2024-09-18T11:31:28.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044114+0000", 2024-09-18T11:31:28.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1505000000000001e-05 2024-09-18T11:31:28.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044120+0000", 2024-09-18T11:31:28.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8549999999999998e-06 2024-09-18T11:31:28.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044154+0000", 2024-09-18T11:31:28.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4773000000000001e-05 2024-09-18T11:31:28.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:28.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044159+0000", 2024-09-18T11:31:28.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5310000000000003e-06 2024-09-18T11:31:28.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046975+0000", 2024-09-18T11:31:28.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028162700000000001 2024-09-18T11:31:28.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.047116+0000", 2024-09-18T11:31:28.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014086400000000001 2024-09-18T11:31:28.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:28.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.047336+0000", 2024-09-18T11:31:28.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021975299999999999 2024-09-18T11:31:28.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:28.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050964+0000", 2024-09-18T11:31:28.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036283000000000001 2024-09-18T11:31:28.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:28.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.051005+0000", 2024-09-18T11:31:28.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0763999999999998e-05 2024-09-18T11:31:28.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:28.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.051050+0000", 2024-09-18T11:31:28.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4993000000000001e-05 2024-09-18T11:31:28.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.051078+0000", 2024-09-18T11:31:28.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8694999999999999e-05 2024-09-18T11:31:28.478 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.478 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.478 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.479 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:127 3.2 3:4d3b277a:::3266c7aaa53a7d2c10c3c54d852d6cbbed49fb041bc0df41a3ac83b56e2b0615:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:28.479 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:31:28.479 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.3362436979999996, 2024-09-18T11:31:28.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093148009999999993, 2024-09-18T11:31:28.479 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.479 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.479 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:31:28.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:31:28.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044400+0000", 2024-09-18T11:31:28.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:28.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044442+0000", 2024-09-18T11:31:28.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2247999999999999e-05 2024-09-18T11:31:28.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044453+0000", 2024-09-18T11:31:28.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0617e-05 2024-09-18T11:31:28.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044457+0000", 2024-09-18T11:31:28.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2520000000000001e-06 2024-09-18T11:31:28.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044495+0000", 2024-09-18T11:31:28.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7710000000000003e-05 2024-09-18T11:31:28.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:28.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044499+0000", 2024-09-18T11:31:28.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6269999999999999e-06 2024-09-18T11:31:28.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048239+0000", 2024-09-18T11:31:28.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037396719999999999 2024-09-18T11:31:28.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048351+0000", 2024-09-18T11:31:28.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112342 2024-09-18T11:31:28.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:28.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048545+0000", 2024-09-18T11:31:28.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000193881 2024-09-18T11:31:28.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:28.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.052689+0000", 2024-09-18T11:31:28.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041435780000000002 2024-09-18T11:31:28.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:28.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053654+0000", 2024-09-18T11:31:28.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096558199999999999 2024-09-18T11:31:28.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:28.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053683+0000", 2024-09-18T11:31:28.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8850999999999999e-05 2024-09-18T11:31:28.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053716+0000", 2024-09-18T11:31:28.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2468000000000002e-05 2024-09-18T11:31:28.485 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.485 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.485 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.485 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2963 4.0 4:019b6e09:::smithi11939506-40:head [sparse-read 0~1093084] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:28.485 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:31:28.485 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.3532349000000004, 2024-09-18T11:31:28.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072292880000000004, 2024-09-18T11:31:28.485 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:28.485 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:28.486 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:28.486 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:28.486 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.486 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2963 2024-09-18T11:31:28.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.486 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:28.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:28.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:31:28.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:28.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:31:28.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:28.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:28.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027409+0000", 2024-09-18T11:31:28.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:28.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027412+0000", 2024-09-18T11:31:28.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.751e-06 2024-09-18T11:31:28.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:28.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027418+0000", 2024-09-18T11:31:28.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.404e-06 2024-09-18T11:31:28.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027423+0000", 2024-09-18T11:31:28.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7119999999999999e-06 2024-09-18T11:31:28.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:28.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034156+0000", 2024-09-18T11:31:28.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067331639999999998 2024-09-18T11:31:28.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:28.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034255+0000", 2024-09-18T11:31:28.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8701000000000002e-05 2024-09-18T11:31:28.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:28.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:28.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:28.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034639+0000", 2024-09-18T11:31:28.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000384006 2024-09-18T11:31:28.489 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.489 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.489 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.490 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:28.490 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:28.490 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:28.491 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:31:28.492 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:28.490+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:28.592 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:28.591+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:28.601 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:28.601 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:28.602 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:28.602 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:28.610 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:31:28.693 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:28.692+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:28.700 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:31:28.752 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:28.752 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:28.752 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:28.752 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:28.753 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:28.759 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:31:28.801 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:28.803 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:28.802+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:28.816 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:28.816 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:28.832 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:28.832 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:28.847 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:28.847 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:28.859 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:28.859 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:28.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:28.860 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:28.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.860 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:31:28.860 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:28.860 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.441070989000004, 2024-09-18T11:31:28.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:31:28.860 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:28.860 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.860 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:28.860 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:28.860 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.861 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:31:28.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.861 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:28.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:28.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:28.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:28.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:28.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:28.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:31:28.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:28.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:28.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:31:28.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:31:28.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:28.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:31:28.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:31:28.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:31:28.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:31:28.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:31:28.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:31:28.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:28.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:31:28.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:31:28.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:28.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:31:28.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:31:28.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:28.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:31:28.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:31:28.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:28.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:31:28.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:31:28.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:28.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:31:28.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:31:28.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:28.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:31:28.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:31:28.866 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.866 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:28.866 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.867 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1628 3.3 3:f80aa31b:::77cdbf3abbada57d0636b2198046a030673818705cbf13bb77b30c3e68b24345:head [write 0~582144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:28.867 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:28.867 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 51.546180989, 2024-09-18T11:31:28.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081714639999999998, 2024-09-18T11:31:28.867 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:28.867 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.867 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:28.867 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:28.867 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.867 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1628 2024-09-18T11:31:28.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.868 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:28.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:28.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:28.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:28.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:28.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:28.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310975+0000", 2024-09-18T11:31:28.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:28.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:28.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312086+0000", 2024-09-18T11:31:28.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001111288 2024-09-18T11:31:28.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:28.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312104+0000", 2024-09-18T11:31:28.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7427000000000001e-05 2024-09-18T11:31:28.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312110+0000", 2024-09-18T11:31:28.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5320000000000003e-06 2024-09-18T11:31:28.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312179+0000", 2024-09-18T11:31:28.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9196000000000006e-05 2024-09-18T11:31:28.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:28.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312375+0000", 2024-09-18T11:31:28.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00019550899999999999 2024-09-18T11:31:28.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:28.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312558+0000", 2024-09-18T11:31:28.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000182566 2024-09-18T11:31:28.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:28.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.314286+0000", 2024-09-18T11:31:28.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00172831 2024-09-18T11:31:28.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:28.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319079+0000", 2024-09-18T11:31:28.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047929340000000004 2024-09-18T11:31:28.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:28.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319116+0000", 2024-09-18T11:31:28.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6993999999999999e-05 2024-09-18T11:31:28.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:28.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319148+0000", 2024-09-18T11:31:28.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2539999999999997e-05 2024-09-18T11:31:28.873 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.873 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:28.873 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.873 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1663 4.1 4:87078dfe:::smithi11939506-14:head [write 1916458~318873] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:28.874 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:28.874 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 51.493234426000001, 2024-09-18T11:31:28.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079505749999999997, 2024-09-18T11:31:28.874 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:28.874 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.874 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:28.874 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:28.874 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.874 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1663 2024-09-18T11:31:28.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.874 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:28.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:28.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:28.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:28.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:28.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:28.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363922+0000", 2024-09-18T11:31:28.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:28.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:28.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364211+0000", 2024-09-18T11:31:28.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000288385 2024-09-18T11:31:28.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:28.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364223+0000", 2024-09-18T11:31:28.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2582000000000001e-05 2024-09-18T11:31:28.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364228+0000", 2024-09-18T11:31:28.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0000000000000004e-06 2024-09-18T11:31:28.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364268+0000", 2024-09-18T11:31:28.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0235999999999999e-05 2024-09-18T11:31:28.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:28.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364341+0000", 2024-09-18T11:31:28.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2266000000000002e-05 2024-09-18T11:31:28.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:28.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364444+0000", 2024-09-18T11:31:28.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103632 2024-09-18T11:31:28.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:28.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369461+0000", 2024-09-18T11:31:28.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005016848 2024-09-18T11:31:28.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:28.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371820+0000", 2024-09-18T11:31:28.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023586169999999999 2024-09-18T11:31:28.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:28.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371851+0000", 2024-09-18T11:31:28.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1297999999999997e-05 2024-09-18T11:31:28.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:28.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371874+0000", 2024-09-18T11:31:28.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2841e-05 2024-09-18T11:31:28.880 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.880 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:28.880 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.880 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1664 4.1 4:87078dfe:::smithi11939506-14:head [setxattr _header (57) in=7b,truncate 2235331] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:28.880 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:28.881 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 51.492920769000001, 2024-09-18T11:31:28.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010368235999999999, 2024-09-18T11:31:28.881 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:28.881 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.881 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:28.881 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:28.881 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.881 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1664 2024-09-18T11:31:28.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.881 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:28.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:28.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:28.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:28.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:28.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:28.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364236+0000", 2024-09-18T11:31:28.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:28.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364240+0000", 2024-09-18T11:31:28.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9829999999999998e-06 2024-09-18T11:31:28.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:28.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364247+0000", 2024-09-18T11:31:28.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2140000000000001e-06 2024-09-18T11:31:28.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364256+0000", 2024-09-18T11:31:28.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1509999999999993e-06 2024-09-18T11:31:28.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366529+0000", 2024-09-18T11:31:28.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022731409999999998 2024-09-18T11:31:28.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:28.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366621+0000", 2024-09-18T11:31:28.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2408e-05 2024-09-18T11:31:28.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:28.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366801+0000", 2024-09-18T11:31:28.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018002200000000001 2024-09-18T11:31:28.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:28.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372155+0000", 2024-09-18T11:31:28.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053531519999999999 2024-09-18T11:31:28.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:28.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374539+0000", 2024-09-18T11:31:28.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023842580000000002 2024-09-18T11:31:28.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:28.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374573+0000", 2024-09-18T11:31:28.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4205999999999997e-05 2024-09-18T11:31:28.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:28.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374605+0000", 2024-09-18T11:31:28.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2283999999999998e-05 2024-09-18T11:31:28.887 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.887 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:28.887 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.887 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1665 4.1 4:87078dfe:::smithi11939506-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:28.887 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:28.888 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 51.492896594999998, 2024-09-18T11:31:28.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010471897000000001, 2024-09-18T11:31:28.888 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:28.888 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:28.888 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:28.888 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:28.888 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.888 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1665 2024-09-18T11:31:28.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.888 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:28.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:28.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:28.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:28.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:28.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:28.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:28.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:28.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364263+0000", 2024-09-18T11:31:28.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6400000000000001e-06 2024-09-18T11:31:28.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:28.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364270+0000", 2024-09-18T11:31:28.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:31:28.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364274+0000", 2024-09-18T11:31:28.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.968e-06 2024-09-18T11:31:28.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367354+0000", 2024-09-18T11:31:28.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030803430000000001 2024-09-18T11:31:28.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:28.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367406+0000", 2024-09-18T11:31:28.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2330999999999998e-05 2024-09-18T11:31:28.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374625+0000", 2024-09-18T11:31:28.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072190689999999998 2024-09-18T11:31:28.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:28.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374673+0000", 2024-09-18T11:31:28.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7525999999999999e-05 2024-09-18T11:31:28.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:28.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374733+0000", 2024-09-18T11:31:28.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0359000000000003e-05 2024-09-18T11:31:28.893 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.893 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:28.893 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.893 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1947 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865577,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:28.893 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:28.893 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 37.517616885000002, 2024-09-18T11:31:28.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010043066999999999, 2024-09-18T11:31:28.894 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:28.894 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:28.894 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:28.894 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:28.894 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.894 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1947 2024-09-18T11:31:28.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.894 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:28.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:28.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:28.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:28.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:28.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:28.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339540+0000", 2024-09-18T11:31:28.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:28.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339545+0000", 2024-09-18T11:31:28.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6290000000000001e-06 2024-09-18T11:31:28.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:28.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339553+0000", 2024-09-18T11:31:28.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6299999999999998e-06 2024-09-18T11:31:28.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339557+0000", 2024-09-18T11:31:28.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3590000000000001e-06 2024-09-18T11:31:28.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345600+0000", 2024-09-18T11:31:28.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060434640000000001 2024-09-18T11:31:28.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:28.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345710+0000", 2024-09-18T11:31:28.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010924700000000001 2024-09-18T11:31:28.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:28.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349584+0000", 2024-09-18T11:31:28.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038743459999999999 2024-09-18T11:31:28.898 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.898 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:28.898 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.899 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1948 4.d 4:b5a463bf:::smithi11939506-20:head [sparse-read 0~2642181] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:28.899 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:28.899 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 37.517596347999998, 2024-09-18T11:31:28.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010701952000000001, 2024-09-18T11:31:28.899 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:28.899 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:28.899 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:28.899 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:28.899 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.899 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1948 2024-09-18T11:31:28.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.900 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:28.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:28.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:28.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:28.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:28.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:28.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:28.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:28.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339563+0000", 2024-09-18T11:31:28.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1950000000000002e-06 2024-09-18T11:31:28.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:28.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339569+0000", 2024-09-18T11:31:28.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5459999999999999e-06 2024-09-18T11:31:28.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339572+0000", 2024-09-18T11:31:28.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.337e-06 2024-09-18T11:31:28.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349693+0000", 2024-09-18T11:31:28.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010120987999999999 2024-09-18T11:31:28.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:28.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349816+0000", 2024-09-18T11:31:28.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012241199999999999 2024-09-18T11:31:28.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:28.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350263+0000", 2024-09-18T11:31:28.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00044787199999999998 2024-09-18T11:31:28.904 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.904 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:28.904 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.904 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:172 4.5 e78/22 4:a087a566:::smithi11939506-46:19 v 78'135, mlcod=77'132)", 2024-09-18T11:31:28.904 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:28.904 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.471334335000002, 2024-09-18T11:31:28.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010938658, 2024-09-18T11:31:28.905 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:28.905 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.905 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:28.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:28.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:28.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:28.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:28.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:28.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385823+0000", 2024-09-18T11:31:28.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:28.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:28.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385830+0000", 2024-09-18T11:31:28.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2790000000000001e-06 2024-09-18T11:31:28.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:28.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385843+0000", 2024-09-18T11:31:28.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2785e-05 2024-09-18T11:31:28.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385848+0000", 2024-09-18T11:31:28.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5570000000000003e-06 2024-09-18T11:31:28.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385914+0000", 2024-09-18T11:31:28.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5168999999999999e-05 2024-09-18T11:31:28.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:28.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385930+0000", 2024-09-18T11:31:28.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6884999999999998e-05 2024-09-18T11:31:28.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:28.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396721+0000", 2024-09-18T11:31:28.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010790137 2024-09-18T11:31:28.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:28.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396762+0000", 2024-09-18T11:31:28.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1635999999999999e-05 2024-09-18T11:31:28.909 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.910 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:28.910 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.910 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:31:28.910 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:28.911 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.470841582999999, 2024-09-18T11:31:28.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:31:28.911 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:28.911 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.911 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:28.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:28.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:28.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:28.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:28.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:28.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:28.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:28.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:28.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:31:28.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:31:28.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:28.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:31:28.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:31:28.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:31:28.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:31:28.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:31:28.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:31:28.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:28.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:31:28.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:31:28.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:28.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:31:28.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:31:28.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:28.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:31:28.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:31:28.918 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.918 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:28.918 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.918 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:28.919 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:28.919 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.884728977000002, 2024-09-18T11:31:28.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:31:28.919 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:28.919 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.919 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:28.919 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:28.919 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.919 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:31:28.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.919 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:28.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:28.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:28.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:28.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:28.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:28.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:31:28.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:28.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:28.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:31:28.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:31:28.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:28.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:31:28.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:31:28.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:31:28.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:31:28.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:31:28.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:31:28.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:28.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:31:28.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:31:28.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:28.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:31:28.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:31:28.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:28.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:31:28.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:31:28.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:28.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:31:28.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:31:28.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:28.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:31:28.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:31:28.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:28.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:31:28.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:31:28.924 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.924 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:28.924 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.924 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2647 4.e 4:78eefc4b:::smithi11939506-7:head [append 0~0] snapc 30=[30,2f,2b,2a,25,24,1c] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:28.925 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:28.925 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.847314297, 2024-09-18T11:31:28.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096752739999999993, 2024-09-18T11:31:28.925 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:28.925 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.925 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:28.925 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:28.925 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.925 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2647 2024-09-18T11:31:28.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.925 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:28.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:28.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:28.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:28.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:28.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:28.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009843+0000", 2024-09-18T11:31:28.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:28.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009846+0000", 2024-09-18T11:31:28.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3870000000000002e-06 2024-09-18T11:31:28.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:28.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009853+0000", 2024-09-18T11:31:28.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.238e-06 2024-09-18T11:31:28.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009860+0000", 2024-09-18T11:31:28.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1999999999999997e-06 2024-09-18T11:31:28.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.013798+0000", 2024-09-18T11:31:28.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003938441 2024-09-18T11:31:28.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:28.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.013889+0000", 2024-09-18T11:31:28.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0332000000000002e-05 2024-09-18T11:31:28.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 6", 2024-09-18T11:31:28.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.014066+0000", 2024-09-18T11:31:28.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017728700000000001 2024-09-18T11:31:28.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:28.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015813+0000", 2024-09-18T11:31:28.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001746804 2024-09-18T11:31:28.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:28.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019419+0000", 2024-09-18T11:31:28.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036058399999999999 2024-09-18T11:31:28.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:28.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019478+0000", 2024-09-18T11:31:28.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8860999999999998e-05 2024-09-18T11:31:28.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:28.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019519+0000", 2024-09-18T11:31:28.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1208000000000003e-05 2024-09-18T11:31:28.931 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.931 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:28.931 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.931 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:209 4.5 e91/22 4:af4ba75a:::smithi11939506-37:19 v 91'165, mlcod=91'161)", 2024-09-18T11:31:28.931 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:28.932 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.839237569, 2024-09-18T11:31:28.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0091546930000000002, 2024-09-18T11:31:28.932 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:28.932 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.932 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:28.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:28.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:28.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:28.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:28.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:28.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:28.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:28.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017925+0000", 2024-09-18T11:31:28.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0819999999999998e-06 2024-09-18T11:31:28.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:28.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017935+0000", 2024-09-18T11:31:28.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0597999999999999e-05 2024-09-18T11:31:28.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017941+0000", 2024-09-18T11:31:28.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2920000000000003e-06 2024-09-18T11:31:28.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017978+0000", 2024-09-18T11:31:28.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7667999999999999e-05 2024-09-18T11:31:28.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:28.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018018+0000", 2024-09-18T11:31:28.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9700999999999998e-05 2024-09-18T11:31:28.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:28.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.027039+0000", 2024-09-18T11:31:28.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090214079999999999 2024-09-18T11:31:28.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:28.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.027075+0000", 2024-09-18T11:31:28.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5605000000000002e-05 2024-09-18T11:31:28.936 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.936 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:28.936 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.937 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:570 4.b e91/67 4:ddcb7ff4:::smithi11939506-9:18 v 91'251, mlcod=90'245)", 2024-09-18T11:31:28.937 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:28.937 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.839201796999999, 2024-09-18T11:31:28.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089212549999999995, 2024-09-18T11:31:28.937 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:28.937 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.937 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:28.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:28.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:28.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:28.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:28.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:28.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:28.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:28.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017960+0000", 2024-09-18T11:31:28.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.777e-06 2024-09-18T11:31:28.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:28.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017966+0000", 2024-09-18T11:31:28.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4389999999999997e-06 2024-09-18T11:31:28.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017970+0000", 2024-09-18T11:31:28.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5549999999999998e-06 2024-09-18T11:31:28.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018032+0000", 2024-09-18T11:31:28.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2976000000000001e-05 2024-09-18T11:31:28.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:28.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018053+0000", 2024-09-18T11:31:28.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0322e-05 2024-09-18T11:31:28.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:28.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026844+0000", 2024-09-18T11:31:28.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087915790000000008 2024-09-18T11:31:28.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:28.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026877+0000", 2024-09-18T11:31:28.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2916000000000001e-05 2024-09-18T11:31:28.941 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.942 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:28.942 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.942 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.5.13:15 3.1 3:bf804683:::e58794d0a579007c3bd048992c5646e06b9775f9667464eee6a2a0508e23a0a6:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:28.942 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:28.942 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.83904695, 2024-09-18T11:31:28.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008816908, 2024-09-18T11:31:28.942 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:28.943 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.943 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:28.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:28.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:28.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:28.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:28.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018110+0000", 2024-09-18T11:31:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018114+0000", 2024-09-18T11:31:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0339999999999998e-06 2024-09-18T11:31:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018121+0000", 2024-09-18T11:31:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0199999999999997e-06 2024-09-18T11:31:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018125+0000", 2024-09-18T11:31:28.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0409999999999999e-06 2024-09-18T11:31:28.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018793+0000", 2024-09-18T11:31:28.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00066779199999999995 2024-09-18T11:31:28.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:28.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018868+0000", 2024-09-18T11:31:28.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4814999999999994e-05 2024-09-18T11:31:28.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:28.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019008+0000", 2024-09-18T11:31:28.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014031800000000001 2024-09-18T11:31:28.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:28.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.023494+0000", 2024-09-18T11:31:28.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0044861720000000001 2024-09-18T11:31:28.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:28.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026796+0000", 2024-09-18T11:31:28.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0033010969999999998 2024-09-18T11:31:28.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:28.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026891+0000", 2024-09-18T11:31:28.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5828000000000005e-05 2024-09-18T11:31:28.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:28.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026928+0000", 2024-09-18T11:31:28.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6393999999999997e-05 2024-09-18T11:31:28.948 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.948 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:28.948 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.949 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2793 4.1 4:87078dfe:::smithi11939506-14:head [write 534522~634550] snapc 30=[30,2f,2b,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:31:28.949 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:28.949 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.6334743219999996, 2024-09-18T11:31:28.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085134289999999994, 2024-09-18T11:31:28.949 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:28.949 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.949 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:28.949 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:28.949 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.949 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2793 2024-09-18T11:31:28.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.949 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:28.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:28.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:28.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:28.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:28.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:28.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223681+0000", 2024-09-18T11:31:28.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:28.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:28.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224251+0000", 2024-09-18T11:31:28.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056940699999999996 2024-09-18T11:31:28.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:28.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224267+0000", 2024-09-18T11:31:28.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6829000000000001e-05 2024-09-18T11:31:28.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224292+0000", 2024-09-18T11:31:28.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4198000000000001e-05 2024-09-18T11:31:28.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224352+0000", 2024-09-18T11:31:28.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0608e-05 2024-09-18T11:31:28.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:28.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224456+0000", 2024-09-18T11:31:28.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010412300000000001 2024-09-18T11:31:28.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:28.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224662+0000", 2024-09-18T11:31:28.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020561399999999999 2024-09-18T11:31:28.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:28.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231800+0000", 2024-09-18T11:31:28.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071384530000000003 2024-09-18T11:31:28.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:28.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232126+0000", 2024-09-18T11:31:28.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00032601400000000001 2024-09-18T11:31:28.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:28.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232158+0000", 2024-09-18T11:31:28.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1472e-05 2024-09-18T11:31:28.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:28.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232197+0000", 2024-09-18T11:31:28.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9011999999999999e-05 2024-09-18T11:31:28.957 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.957 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:28.957 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.957 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:123 3.1 3:bae7b855:::93307fb660774b1d286e3149e1a75006840a12081178bf31d320df6af68ad704:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:28.957 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:23.045517+0000", 2024-09-18T11:31:28.957 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.8116411259999996, 2024-09-18T11:31:28.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081745810000000002, 2024-09-18T11:31:28.957 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:28.957 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.957 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:28.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:28.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.045517+0000", 2024-09-18T11:31:28.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:28.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.045517+0000", 2024-09-18T11:31:28.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:28.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.045516+0000", 2024-09-18T11:31:28.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:28.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:28.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046503+0000", 2024-09-18T11:31:28.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00098663900000000009 2024-09-18T11:31:28.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:28.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046516+0000", 2024-09-18T11:31:28.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2971e-05 2024-09-18T11:31:28.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046522+0000", 2024-09-18T11:31:28.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.618e-06 2024-09-18T11:31:28.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046562+0000", 2024-09-18T11:31:28.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9400999999999998e-05 2024-09-18T11:31:28.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:28.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046639+0000", 2024-09-18T11:31:28.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7037000000000001e-05 2024-09-18T11:31:28.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:28.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046809+0000", 2024-09-18T11:31:28.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017013799999999999 2024-09-18T11:31:28.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:28.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.049963+0000", 2024-09-18T11:31:28.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031541389999999998 2024-09-18T11:31:28.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:28.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.053637+0000", 2024-09-18T11:31:28.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036736550000000001 2024-09-18T11:31:28.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:28.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.053665+0000", 2024-09-18T11:31:28.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8759999999999999e-05 2024-09-18T11:31:28.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:28.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.053691+0000", 2024-09-18T11:31:28.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5978000000000001e-05 2024-09-18T11:31:28.964 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.964 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:28.964 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.964 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:231 4.5 e98/22 4:af4ba75a:::smithi11939506-37:37 v 98'186, mlcod=96'181)", 2024-09-18T11:31:28.964 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.013478+0000", 2024-09-18T11:31:28.964 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.8436799759999998, 2024-09-18T11:31:28.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079316330000000004, 2024-09-18T11:31:28.965 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:28.965 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:28.965 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:28.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:28.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.013478+0000", 2024-09-18T11:31:28.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:28.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.013478+0000", 2024-09-18T11:31:28.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:28.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.013477+0000", 2024-09-18T11:31:28.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:28.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:28.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.013484+0000", 2024-09-18T11:31:28.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6470000000000001e-06 2024-09-18T11:31:28.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:28.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.013495+0000", 2024-09-18T11:31:28.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1146e-05 2024-09-18T11:31:28.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.013500+0000", 2024-09-18T11:31:28.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6480000000000002e-06 2024-09-18T11:31:28.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.014509+0000", 2024-09-18T11:31:28.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001009923 2024-09-18T11:31:28.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:28.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.014528+0000", 2024-09-18T11:31:28.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8476999999999999e-05 2024-09-18T11:31:28.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:28.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.021379+0000", 2024-09-18T11:31:28.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068506729999999998 2024-09-18T11:31:28.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:28.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.021409+0000", 2024-09-18T11:31:28.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0902000000000003e-05 2024-09-18T11:31:28.969 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.969 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:28.969 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.969 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2955 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:28.969 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:31:28.969 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.8299665479999998, 2024-09-18T11:31:28.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010869531, 2024-09-18T11:31:28.969 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:28.969 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:28.970 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:28.970 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:28.970 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.970 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2955 2024-09-18T11:31:28.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.970 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:28.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:28.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:31:28.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:28.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:31:28.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:28.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027190+0000", 2024-09-18T11:31:28.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:28.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:28.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027196+0000", 2024-09-18T11:31:28.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5590000000000002e-06 2024-09-18T11:31:28.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:28.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027208+0000", 2024-09-18T11:31:28.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1816e-05 2024-09-18T11:31:28.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027215+0000", 2024-09-18T11:31:28.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4509999999999997e-06 2024-09-18T11:31:28.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027725+0000", 2024-09-18T11:31:28.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000510053 2024-09-18T11:31:28.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:28.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027803+0000", 2024-09-18T11:31:28.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8750000000000003e-05 2024-09-18T11:31:28.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032412+0000", 2024-09-18T11:31:28.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046089030000000001 2024-09-18T11:31:28.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:28.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032499+0000", 2024-09-18T11:31:28.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6316000000000005e-05 2024-09-18T11:31:28.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:28.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038061+0000", 2024-09-18T11:31:28.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005562259 2024-09-18T11:31:28.974 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.974 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:28.974 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.975 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2956 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~2100908,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:28.975 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:28.975 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.8299327270000001, 2024-09-18T11:31:28.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011863831, 2024-09-18T11:31:28.975 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:28.975 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:28.975 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:28.975 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:28.975 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.975 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2956 2024-09-18T11:31:28.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.975 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:28.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:28.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:28.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:28.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:28.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:28.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:28.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:28.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027228+0000", 2024-09-18T11:31:28.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:31:28.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:28.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027235+0000", 2024-09-18T11:31:28.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:31:28.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027239+0000", 2024-09-18T11:31:28.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.535e-06 2024-09-18T11:31:28.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027831+0000", 2024-09-18T11:31:28.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059244699999999996 2024-09-18T11:31:28.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:28.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027879+0000", 2024-09-18T11:31:28.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:31:28.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038098+0000", 2024-09-18T11:31:28.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010219137 2024-09-18T11:31:28.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:28.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038207+0000", 2024-09-18T11:31:28.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010951999999999999 2024-09-18T11:31:28.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:28.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039089+0000", 2024-09-18T11:31:28.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00088151699999999996 2024-09-18T11:31:28.980 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.980 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:28.980 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.981 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2957 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:28.981 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:28.981 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.8299149430000003, 2024-09-18T11:31:28.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012041979, 2024-09-18T11:31:28.981 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:28.981 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:28.981 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:28.981 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:28.981 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:28.981 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2957 2024-09-18T11:31:28.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.981 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:28.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:28.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:28.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:28.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:28.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:28.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:28.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:28.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:28.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:28.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027245+0000", 2024-09-18T11:31:28.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:31:28.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:28.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027251+0000", 2024-09-18T11:31:28.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0090000000000004e-06 2024-09-18T11:31:28.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:28.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027255+0000", 2024-09-18T11:31:28.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:31:28.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027895+0000", 2024-09-18T11:31:28.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064076400000000005 2024-09-18T11:31:28.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:28.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027930+0000", 2024-09-18T11:31:28.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4334e-05 2024-09-18T11:31:28.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:28.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039111+0000", 2024-09-18T11:31:28.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181455999999999 2024-09-18T11:31:28.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:28.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039156+0000", 2024-09-18T11:31:28.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5256000000000002e-05 2024-09-18T11:31:28.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:28.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:28.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:28.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039285+0000", 2024-09-18T11:31:28.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012839200000000001 2024-09-18T11:31:28.986 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.986 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:28.986 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.986 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:28.986 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:28.986 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:28.987 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:28.987 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:28.988 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:28.988 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:28.988 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:31:28.989 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:28.989 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:28.990 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:28.990 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:28.990 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:28.991 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:28.991 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:28.986+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:29.043 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:29.042+0000 7fb6950ff640 -1 osd.2 100 handle_osd_map injecting map crc failure 2024-09-18T11:31:29.044 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:29.043+0000 7f6bf5056640 -1 osd.5 100 handle_osd_map injecting map crc failure 2024-09-18T11:31:29.047 INFO:tasks.rados.rados.0.smithi119.stdout:1408: done (1 left) 2024-09-18T11:31:29.047 INFO:tasks.rados.rados.0.smithi119.stdout:1409: snap_create 2024-09-18T11:31:29.047 INFO:tasks.rados.rados.0.smithi119.stdout:1405: finishing write tid 3 to smithi11939506-5 2024-09-18T11:31:29.047 INFO:tasks.rados.rados.0.smithi119.stdout:1405: finishing write tid 4 to smithi11939506-5 2024-09-18T11:31:29.047 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 169 (ObjNum 269 snap 31 seq_num 269) dirty exists 2024-09-18T11:31:29.047 INFO:tasks.rados.rados.0.smithi119.stdout:1405: left oid 5 (ObjNum 269 snap 31 seq_num 269) 2024-09-18T11:31:29.087 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:29.086+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:29.130 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:29.130 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:29.138 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:31:29.222 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:29.221+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:29.275 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:29.275 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:29.283 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:31:29.289 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:29.288+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:29.390 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:29.389+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:29.422 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:29.422 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:29.430 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:31:29.491 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:29.490+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:29.528 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:29.528 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:29.536 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:31:29.592 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:29.591+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:29.633 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:29.633 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:29.641 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:31:29.693 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:29.692+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:29.739 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:29.739 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:29.748 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:31:29.794 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:29.792+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:29.847 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:29.847 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:29.855 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:31:29.896 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:29.895+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:29.993 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:29.993 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:29.996 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:29.994+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:30.001 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:31:30.058 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T11:31:30.059 INFO:tasks.rados.rados.0.smithi119.stdout:1405: done (1 left) 2024-09-18T11:31:30.059 INFO:tasks.rados.rados.0.smithi119.stdout:1409: done (0 left) 2024-09-18T11:31:30.059 INFO:tasks.rados.rados.0.smithi119.stdout:1410: set_chunk oid 31 target oid 2024-09-18T11:31:30.064 INFO:tasks.rados.rados.0.smithi119.stdout:1410: set_chunk oid 31 offset: 227328 length: 233472 target oid 14fb988ddcc6e847861492ee7ce3661feedab1cc83c5c9806d9fa20092b3071f offset: 0 snap -1 2024-09-18T11:31:30.064 INFO:tasks.rados.rados.0.smithi119.stdout:1411: set_chunk oid 1 target oid 2024-09-18T11:31:30.072 INFO:tasks.rados.rados.0.smithi119.stdout:1411: set_chunk oid 1 offset: 1233920 length: 544768 target oid 2eedd4e14ca3a6fb1434c500ba9aafe089a6e7552db0f353be004f324bdf0d79 offset: 0 snap -1 2024-09-18T11:31:30.072 INFO:tasks.rados.rados.0.smithi119.stdout:1412: snap_remove snap 21 2024-09-18T11:31:30.096 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:30.095+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:30.103 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:30.103 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:30.112 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:31:30.198 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:30.196+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:30.219 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:30.219 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:30.227 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:31:30.299 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:30.298+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:30.326 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:30.327 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:30.335 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:31:30.400 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:30.399+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:30.432 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:30.432 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:30.440 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:31:30.501 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:30.500+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:30.538 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:30.539 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:30.547 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:31:30.602 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:30.601+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:30.644 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:30.644 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:30.652 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:31:30.703 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:30.702+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:30.750 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:30.750 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:30.759 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:31:30.804 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:30.803+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:30.857 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:30.857 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:30.865 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:31:30.905 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:30.904+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:30.962 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:30.962 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:30.971 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:31:30.991 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:31:31.015 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:31.013+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:31.073 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is unset 2024-09-18T11:31:31.075 INFO:tasks.rados.rados.0.smithi119.stdout:1412: done (2 left) 2024-09-18T11:31:31.075 INFO:tasks.rados.rados.0.smithi119.stdout:1413: rollback oid 25 current snap is 32 2024-09-18T11:31:31.075 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 25 to 23 2024-09-18T11:31:31.075 INFO:tasks.rados.rados.0.smithi119.stdout:1414: rollback oid 29 current snap is 32 2024-09-18T11:31:31.075 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 29 to 23 2024-09-18T11:31:31.075 INFO:tasks.rados.rados.0.smithi119.stdout:1415: delete oid 36 current snap is 32 2024-09-18T11:31:31.075 INFO:tasks.rados.rados.0.smithi119.stdout:1410: finishing set_chunk to oid 31 2024-09-18T11:31:31.075 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 203 (ObjNum 257 snap 31 seq_num 257) dirty exists 2024-09-18T11:31:31.076 INFO:tasks.rados.rados.0.smithi119.stdout:1411: finishing set_chunk to oid 1 2024-09-18T11:31:31.076 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 166 (ObjNum 265 snap 31 seq_num 265) dirty exists 2024-09-18T11:31:31.077 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:31.077 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:31.084 INFO:tasks.rados.rados.0.smithi119.stdout:1413: finishing rollback tid 1 to smithi11939506-25 2024-09-18T11:31:31.084 INFO:tasks.rados.rados.0.smithi119.stdout:1413: finishing rollback tid 2 to smithi11939506-25 2024-09-18T11:31:31.084 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 46 (ObjNum 212 snap 23 seq_num 212) dirty exists 2024-09-18T11:31:31.086 INFO:tasks.rados.rados.0.smithi119.stdout:1414: finishing rollback tid 0 to smithi11939506-29 2024-09-18T11:31:31.086 INFO:tasks.rados.rados.0.smithi119.stdout:1414: finishing rollback tid 1 to smithi11939506-29 2024-09-18T11:31:31.086 INFO:tasks.rados.rados.0.smithi119.stdout:1414: finishing rollback tid 2 to smithi11939506-29 2024-09-18T11:31:31.086 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 83 (ObjNum 210 snap 23 seq_num 210) dirty exists 2024-09-18T11:31:31.086 INFO:tasks.rados.rados.0.smithi119.stdout:1410: done (4 left) 2024-09-18T11:31:31.086 INFO:tasks.rados.rados.0.smithi119.stdout:1411: done (3 left) 2024-09-18T11:31:31.086 INFO:tasks.rados.rados.0.smithi119.stdout:1413: done (2 left) 2024-09-18T11:31:31.086 INFO:tasks.rados.rados.0.smithi119.stdout:1414: done (1 left) 2024-09-18T11:31:31.086 INFO:tasks.rados.rados.0.smithi119.stdout:1415: done (0 left) 2024-09-18T11:31:31.087 INFO:tasks.rados.rados.0.smithi119.stdout:1416: tier_evict oid 44 2024-09-18T11:31:31.087 INFO:tasks.rados.rados.0.smithi119.stdout:1416: tier_evict oid 44 snap -1 2024-09-18T11:31:31.087 INFO:tasks.rados.rados.0.smithi119.stdout:1417: read oid 4 snap 27 2024-09-18T11:31:31.087 INFO:tasks.rados.rados.0.smithi119.stdout:1417: expect (ObjNum 110 snap 8 seq_num 110) 2024-09-18T11:31:31.087 INFO:tasks.rados.rados.0.smithi119.stdout:1418: read oid 46 snap -1 2024-09-18T11:31:31.087 INFO:tasks.rados.rados.0.smithi119.stdout:1418: expect (ObjNum 171 snap 20 seq_num 171) 2024-09-18T11:31:31.087 INFO:tasks.rados.rados.0.smithi119.stdout:1419: set_chunk oid 38 target oid 2024-09-18T11:31:31.091 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:31:31.117 INFO:tasks.rados.rados.0.smithi119.stdout:1419: set_chunk oid 38 offset: 2204672 length: 1552896 target oid b4b6c1711812458f3f644b2afc9edeb763d5d103111de2afdade5918cfc9a6d3 offset: 0 snap -1 2024-09-18T11:31:31.117 INFO:tasks.rados.rados.0.smithi119.stdout:1420: set_chunk oid 33 target oid 2024-09-18T11:31:31.117 INFO:tasks.rados.rados.0.smithi119.stdout:1420: set_chunk oid 33 offset: 2056192 length: 356352 target oid smithi11939506- offset: 2056192 snap 31 2024-09-18T11:31:31.117 INFO:tasks.rados.rados.0.smithi119.stdout:1421: set_chunk oid 1 target oid 2024-09-18T11:31:31.117 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:31.115+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:31.121 INFO:tasks.rados.rados.0.smithi119.stdout:1421: set_chunk oid 1 offset: 1114112 length: 335872 target oid dee9dbfd26a32ef95b5b96d0dd4763122c0dcc985bf58dc3e98b7b37ea9c233a offset: 0 snap 23 2024-09-18T11:31:31.121 INFO:tasks.rados.rados.0.smithi119.stdout:1422: read oid 48 snap 29 2024-09-18T11:31:31.121 INFO:tasks.rados.rados.0.smithi119.stdout:1422: expect (ObjNum 191 snap 21 seq_num 191) 2024-09-18T11:31:31.121 INFO:tasks.rados.rados.0.smithi119.stdout:1423: read oid 19 snap 27 2024-09-18T11:31:31.121 INFO:tasks.rados.rados.0.smithi119.stdout:1423: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-18T11:31:31.121 INFO:tasks.rados.rados.0.smithi119.stdout:1424: snap_create 2024-09-18T11:31:31.121 INFO:tasks.rados.rados.0.smithi119.stdout:1416: got (22) Invalid argument 2024-09-18T11:31:31.121 INFO:tasks.rados.rados.0.smithi119.stdout:1418: expect (ObjNum 171 snap 20 seq_num 171) 2024-09-18T11:31:31.134 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:31.150 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:31.151 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:31.166 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:31.166 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:31.181 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:31.181 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:31.196 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:31.196 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:31.197 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:31.197 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:31.197 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:31.197 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:31.205 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:31:31.206 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:31.205+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:31.211 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:31.211 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:31.226 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:31.227 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:31.242 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:31.242 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:31.246 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:31.246 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:31.249 INFO:tasks.rados.rados.0.smithi119.stdout:1417: expect (ObjNum 110 snap 8 seq_num 110) 2024-09-18T11:31:31.304 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:31.304 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:31.304 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:31.304 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:31.304 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:31.307 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:31.306+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:31.312 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:31:31.408 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:31.406+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:31.426 INFO:tasks.rados.rados.0.smithi119.stdout:1420: finishing set_chunk to oid 33 2024-09-18T11:31:31.426 INFO:tasks.rados.rados.0.smithi119.stdout:1420: got expected ERANGE (src dne) 2024-09-18T11:31:31.426 INFO:tasks.rados.rados.0.smithi119.stdout:1419: finishing set_chunk to oid 38 2024-09-18T11:31:31.426 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 38 set_chunk returned error code -95 offset: 2204672 length: 1552896 2024-09-18T11:31:31.427 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 0 (ObjNum 231 snap 27 seq_num 231) dirty exists 2024-09-18T11:31:31.427 INFO:tasks.rados.rados.0.smithi119.stdout:1423: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-18T11:31:31.452 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:31.452 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:31.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:31.453 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:31.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.453 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:31:31.453 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:31.453 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.034192326000003, 2024-09-18T11:31:31.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:31:31.453 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.453 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.453 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.453 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.453 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.454 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:31:31.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.454 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:31.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:31.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:31:31.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:31.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:31:31.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:31:31.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:31:31.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:31:31.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:31:31.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:31:31.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:31:31.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:31:31.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:31:31.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:31:31.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:31.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:31:31.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:31:31.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:31.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:31:31.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:31:31.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:31.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:31:31.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:31:31.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:31:31.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:31:31.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:31:31.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:31:31.459 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.459 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.459 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.459 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1628 3.3 3:f80aa31b:::77cdbf3abbada57d0636b2198046a030673818705cbf13bb77b30c3e68b24345:head [write 0~582144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:31.459 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:31.459 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.139302325999999, 2024-09-18T11:31:31.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081714639999999998, 2024-09-18T11:31:31.459 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.459 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.460 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.460 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.460 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.460 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1628 2024-09-18T11:31:31.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.460 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:31.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:31.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310975+0000", 2024-09-18T11:31:31.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:31.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312086+0000", 2024-09-18T11:31:31.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001111288 2024-09-18T11:31:31.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312104+0000", 2024-09-18T11:31:31.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7427000000000001e-05 2024-09-18T11:31:31.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312110+0000", 2024-09-18T11:31:31.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5320000000000003e-06 2024-09-18T11:31:31.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312179+0000", 2024-09-18T11:31:31.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9196000000000006e-05 2024-09-18T11:31:31.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312375+0000", 2024-09-18T11:31:31.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00019550899999999999 2024-09-18T11:31:31.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:31.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312558+0000", 2024-09-18T11:31:31.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000182566 2024-09-18T11:31:31.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:31.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.314286+0000", 2024-09-18T11:31:31.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00172831 2024-09-18T11:31:31.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:31.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319079+0000", 2024-09-18T11:31:31.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047929340000000004 2024-09-18T11:31:31.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319116+0000", 2024-09-18T11:31:31.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6993999999999999e-05 2024-09-18T11:31:31.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319148+0000", 2024-09-18T11:31:31.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2539999999999997e-05 2024-09-18T11:31:31.465 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.465 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.465 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.465 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1663 4.1 4:87078dfe:::smithi11939506-14:head [write 1916458~318873] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:31.465 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:31.465 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.086355763, 2024-09-18T11:31:31.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079505749999999997, 2024-09-18T11:31:31.466 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.466 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.466 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.466 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.466 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.466 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1663 2024-09-18T11:31:31.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.466 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:31.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363922+0000", 2024-09-18T11:31:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364211+0000", 2024-09-18T11:31:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000288385 2024-09-18T11:31:31.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364223+0000", 2024-09-18T11:31:31.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2582000000000001e-05 2024-09-18T11:31:31.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364228+0000", 2024-09-18T11:31:31.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0000000000000004e-06 2024-09-18T11:31:31.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364268+0000", 2024-09-18T11:31:31.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0235999999999999e-05 2024-09-18T11:31:31.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364341+0000", 2024-09-18T11:31:31.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2266000000000002e-05 2024-09-18T11:31:31.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:31.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364444+0000", 2024-09-18T11:31:31.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103632 2024-09-18T11:31:31.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:31.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369461+0000", 2024-09-18T11:31:31.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005016848 2024-09-18T11:31:31.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:31.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371820+0000", 2024-09-18T11:31:31.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023586169999999999 2024-09-18T11:31:31.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371851+0000", 2024-09-18T11:31:31.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1297999999999997e-05 2024-09-18T11:31:31.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371874+0000", 2024-09-18T11:31:31.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2841e-05 2024-09-18T11:31:31.471 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.471 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.471 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.471 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1664 4.1 4:87078dfe:::smithi11939506-14:head [setxattr _header (57) in=7b,truncate 2235331] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:31.471 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:31.471 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.086042106000001, 2024-09-18T11:31:31.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010368235999999999, 2024-09-18T11:31:31.471 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.471 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.472 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.472 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.472 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.472 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1664 2024-09-18T11:31:31.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.472 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:31.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:31.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364236+0000", 2024-09-18T11:31:31.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:31.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364240+0000", 2024-09-18T11:31:31.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9829999999999998e-06 2024-09-18T11:31:31.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364247+0000", 2024-09-18T11:31:31.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2140000000000001e-06 2024-09-18T11:31:31.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364256+0000", 2024-09-18T11:31:31.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1509999999999993e-06 2024-09-18T11:31:31.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366529+0000", 2024-09-18T11:31:31.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022731409999999998 2024-09-18T11:31:31.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366621+0000", 2024-09-18T11:31:31.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2408e-05 2024-09-18T11:31:31.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:31.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366801+0000", 2024-09-18T11:31:31.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018002200000000001 2024-09-18T11:31:31.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:31.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372155+0000", 2024-09-18T11:31:31.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053531519999999999 2024-09-18T11:31:31.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:31.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374539+0000", 2024-09-18T11:31:31.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023842580000000002 2024-09-18T11:31:31.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374573+0000", 2024-09-18T11:31:31.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4205999999999997e-05 2024-09-18T11:31:31.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374605+0000", 2024-09-18T11:31:31.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2283999999999998e-05 2024-09-18T11:31:31.477 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.477 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.477 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.477 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1665 4.1 4:87078dfe:::smithi11939506-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:31.477 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:31.477 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.086017931999997, 2024-09-18T11:31:31.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010471897000000001, 2024-09-18T11:31:31.478 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.478 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:31.478 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.478 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.478 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.478 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1665 2024-09-18T11:31:31.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.478 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:31.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:31.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:31.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:31.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364263+0000", 2024-09-18T11:31:31.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6400000000000001e-06 2024-09-18T11:31:31.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364270+0000", 2024-09-18T11:31:31.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:31:31.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364274+0000", 2024-09-18T11:31:31.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.968e-06 2024-09-18T11:31:31.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367354+0000", 2024-09-18T11:31:31.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030803430000000001 2024-09-18T11:31:31.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:31.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367406+0000", 2024-09-18T11:31:31.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2330999999999998e-05 2024-09-18T11:31:31.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374625+0000", 2024-09-18T11:31:31.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072190689999999998 2024-09-18T11:31:31.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374673+0000", 2024-09-18T11:31:31.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7525999999999999e-05 2024-09-18T11:31:31.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374733+0000", 2024-09-18T11:31:31.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0359000000000003e-05 2024-09-18T11:31:31.482 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.482 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.482 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.483 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1947 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865577,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:31.483 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:31.483 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.110738222000002, 2024-09-18T11:31:31.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010043066999999999, 2024-09-18T11:31:31.483 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.483 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:31.483 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.483 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.483 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.483 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1947 2024-09-18T11:31:31.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.483 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:31.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:31.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339540+0000", 2024-09-18T11:31:31.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:31.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339545+0000", 2024-09-18T11:31:31.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6290000000000001e-06 2024-09-18T11:31:31.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339553+0000", 2024-09-18T11:31:31.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6299999999999998e-06 2024-09-18T11:31:31.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339557+0000", 2024-09-18T11:31:31.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3590000000000001e-06 2024-09-18T11:31:31.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345600+0000", 2024-09-18T11:31:31.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060434640000000001 2024-09-18T11:31:31.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345710+0000", 2024-09-18T11:31:31.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010924700000000001 2024-09-18T11:31:31.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349584+0000", 2024-09-18T11:31:31.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038743459999999999 2024-09-18T11:31:31.488 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.488 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.488 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.488 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1948 4.d 4:b5a463bf:::smithi11939506-20:head [sparse-read 0~2642181] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:31.488 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:31.488 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.110717684999997, 2024-09-18T11:31:31.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010701952000000001, 2024-09-18T11:31:31.488 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.488 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:31.488 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.489 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.489 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.489 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1948 2024-09-18T11:31:31.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.489 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:31.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:31.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:31.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:31.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339563+0000", 2024-09-18T11:31:31.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1950000000000002e-06 2024-09-18T11:31:31.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339569+0000", 2024-09-18T11:31:31.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5459999999999999e-06 2024-09-18T11:31:31.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339572+0000", 2024-09-18T11:31:31.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.337e-06 2024-09-18T11:31:31.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349693+0000", 2024-09-18T11:31:31.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010120987999999999 2024-09-18T11:31:31.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349816+0000", 2024-09-18T11:31:31.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012241199999999999 2024-09-18T11:31:31.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350263+0000", 2024-09-18T11:31:31.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00044787199999999998 2024-09-18T11:31:31.492 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.493 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.493 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.493 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:172 4.5 e78/22 4:a087a566:::smithi11939506-46:19 v 78'135, mlcod=77'132)", 2024-09-18T11:31:31.493 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:31.493 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 30.064455672000001, 2024-09-18T11:31:31.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010938658, 2024-09-18T11:31:31.493 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.493 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.493 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.493 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:31.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:31.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385823+0000", 2024-09-18T11:31:31.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:31.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385830+0000", 2024-09-18T11:31:31.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2790000000000001e-06 2024-09-18T11:31:31.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385843+0000", 2024-09-18T11:31:31.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2785e-05 2024-09-18T11:31:31.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385848+0000", 2024-09-18T11:31:31.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5570000000000003e-06 2024-09-18T11:31:31.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385914+0000", 2024-09-18T11:31:31.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5168999999999999e-05 2024-09-18T11:31:31.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385930+0000", 2024-09-18T11:31:31.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6884999999999998e-05 2024-09-18T11:31:31.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396721+0000", 2024-09-18T11:31:31.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010790137 2024-09-18T11:31:31.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396762+0000", 2024-09-18T11:31:31.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1635999999999999e-05 2024-09-18T11:31:31.497 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.497 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.497 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.498 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:31:31.498 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:31.498 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 30.063962920000002, 2024-09-18T11:31:31.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:31:31.498 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.498 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.498 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:31.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:31.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:31.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:31.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:31:31.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:31:31.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:31:31.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:31:31.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:31:31.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:31:31.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:31:31.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:31:31.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:31:31.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:31:31.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:31:31.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:31:31.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:31:31.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:31:31.502 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.502 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.502 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.502 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:31.502 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:31.502 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.477850314000001, 2024-09-18T11:31:31.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:31:31.502 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.502 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.503 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.503 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.503 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.503 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:31:31.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.503 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:31.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:31.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.504 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:31:31.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:31.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.504 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:31:31.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:31:31.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:31:31.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:31:31.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:31:31.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:31:31.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:31:31.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:31:31.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:31:31.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:31:31.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:31.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:31:31.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:31:31.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:31.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:31:31.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:31:31.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:31.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:31:31.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:31:31.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:31:31.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:31:31.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:31:31.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:31:31.508 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.508 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.508 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.508 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2647 4.e 4:78eefc4b:::smithi11939506-7:head [append 0~0] snapc 30=[30,2f,2b,2a,25,24,1c] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:31.509 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:31.509 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.440435634, 2024-09-18T11:31:31.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096752739999999993, 2024-09-18T11:31:31.509 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.509 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.509 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.509 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.509 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.509 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2647 2024-09-18T11:31:31.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.509 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:31.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:31.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009843+0000", 2024-09-18T11:31:31.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:31.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009846+0000", 2024-09-18T11:31:31.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3870000000000002e-06 2024-09-18T11:31:31.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009853+0000", 2024-09-18T11:31:31.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.238e-06 2024-09-18T11:31:31.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009860+0000", 2024-09-18T11:31:31.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1999999999999997e-06 2024-09-18T11:31:31.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.013798+0000", 2024-09-18T11:31:31.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003938441 2024-09-18T11:31:31.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.013889+0000", 2024-09-18T11:31:31.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0332000000000002e-05 2024-09-18T11:31:31.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 6", 2024-09-18T11:31:31.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.014066+0000", 2024-09-18T11:31:31.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017728700000000001 2024-09-18T11:31:31.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:31.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015813+0000", 2024-09-18T11:31:31.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001746804 2024-09-18T11:31:31.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:31.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019419+0000", 2024-09-18T11:31:31.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036058399999999999 2024-09-18T11:31:31.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019478+0000", 2024-09-18T11:31:31.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8860999999999998e-05 2024-09-18T11:31:31.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019519+0000", 2024-09-18T11:31:31.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1208000000000003e-05 2024-09-18T11:31:31.514 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.514 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.514 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.515 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:209 4.5 e91/22 4:af4ba75a:::smithi11939506-37:19 v 91'165, mlcod=91'161)", 2024-09-18T11:31:31.515 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:31.515 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.432358906000001, 2024-09-18T11:31:31.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0091546930000000002, 2024-09-18T11:31:31.515 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.515 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.515 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:31.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:31.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:31.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:31.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017925+0000", 2024-09-18T11:31:31.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0819999999999998e-06 2024-09-18T11:31:31.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017935+0000", 2024-09-18T11:31:31.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0597999999999999e-05 2024-09-18T11:31:31.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017941+0000", 2024-09-18T11:31:31.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2920000000000003e-06 2024-09-18T11:31:31.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017978+0000", 2024-09-18T11:31:31.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7667999999999999e-05 2024-09-18T11:31:31.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018018+0000", 2024-09-18T11:31:31.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9700999999999998e-05 2024-09-18T11:31:31.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.027039+0000", 2024-09-18T11:31:31.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090214079999999999 2024-09-18T11:31:31.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.519 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.027075+0000", 2024-09-18T11:31:31.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5605000000000002e-05 2024-09-18T11:31:31.519 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.519 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.519 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.519 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:570 4.b e91/67 4:ddcb7ff4:::smithi11939506-9:18 v 91'251, mlcod=90'245)", 2024-09-18T11:31:31.519 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:31.519 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.432323134000001, 2024-09-18T11:31:31.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089212549999999995, 2024-09-18T11:31:31.519 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.519 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.520 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:31.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:31.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:31.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:31.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017960+0000", 2024-09-18T11:31:31.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.777e-06 2024-09-18T11:31:31.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017966+0000", 2024-09-18T11:31:31.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4389999999999997e-06 2024-09-18T11:31:31.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017970+0000", 2024-09-18T11:31:31.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5549999999999998e-06 2024-09-18T11:31:31.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018032+0000", 2024-09-18T11:31:31.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2976000000000001e-05 2024-09-18T11:31:31.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018053+0000", 2024-09-18T11:31:31.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0322e-05 2024-09-18T11:31:31.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026844+0000", 2024-09-18T11:31:31.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087915790000000008 2024-09-18T11:31:31.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026877+0000", 2024-09-18T11:31:31.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2916000000000001e-05 2024-09-18T11:31:31.523 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.523 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.523 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.524 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.5.13:15 3.1 3:bf804683:::e58794d0a579007c3bd048992c5646e06b9775f9667464eee6a2a0508e23a0a6:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:31.524 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:31.524 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.432168287, 2024-09-18T11:31:31.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008816908, 2024-09-18T11:31:31.524 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.524 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.524 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.524 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:31.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:31.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018110+0000", 2024-09-18T11:31:31.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:31.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018114+0000", 2024-09-18T11:31:31.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0339999999999998e-06 2024-09-18T11:31:31.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018121+0000", 2024-09-18T11:31:31.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0199999999999997e-06 2024-09-18T11:31:31.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018125+0000", 2024-09-18T11:31:31.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0409999999999999e-06 2024-09-18T11:31:31.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018793+0000", 2024-09-18T11:31:31.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00066779199999999995 2024-09-18T11:31:31.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018868+0000", 2024-09-18T11:31:31.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4814999999999994e-05 2024-09-18T11:31:31.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:31.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019008+0000", 2024-09-18T11:31:31.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014031800000000001 2024-09-18T11:31:31.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:31.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.023494+0000", 2024-09-18T11:31:31.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0044861720000000001 2024-09-18T11:31:31.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:31.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026796+0000", 2024-09-18T11:31:31.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0033010969999999998 2024-09-18T11:31:31.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026891+0000", 2024-09-18T11:31:31.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5828000000000005e-05 2024-09-18T11:31:31.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026928+0000", 2024-09-18T11:31:31.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6393999999999997e-05 2024-09-18T11:31:31.529 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.529 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.529 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.530 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2793 4.1 4:87078dfe:::smithi11939506-14:head [write 534522~634550] snapc 30=[30,2f,2b,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:31:31.530 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:31.530 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.226595659000001, 2024-09-18T11:31:31.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085134289999999994, 2024-09-18T11:31:31.530 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.530 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.530 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.530 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.530 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.530 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2793 2024-09-18T11:31:31.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.530 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:31.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:31.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223681+0000", 2024-09-18T11:31:31.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:31.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224251+0000", 2024-09-18T11:31:31.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056940699999999996 2024-09-18T11:31:31.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224267+0000", 2024-09-18T11:31:31.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6829000000000001e-05 2024-09-18T11:31:31.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224292+0000", 2024-09-18T11:31:31.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4198000000000001e-05 2024-09-18T11:31:31.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224352+0000", 2024-09-18T11:31:31.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0608e-05 2024-09-18T11:31:31.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224456+0000", 2024-09-18T11:31:31.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010412300000000001 2024-09-18T11:31:31.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:31.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224662+0000", 2024-09-18T11:31:31.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020561399999999999 2024-09-18T11:31:31.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:31.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231800+0000", 2024-09-18T11:31:31.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071384530000000003 2024-09-18T11:31:31.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:31.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232126+0000", 2024-09-18T11:31:31.535 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00032601400000000001 2024-09-18T11:31:31.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.535 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.535 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232158+0000", 2024-09-18T11:31:31.535 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1472e-05 2024-09-18T11:31:31.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.535 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.535 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232197+0000", 2024-09-18T11:31:31.535 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9011999999999999e-05 2024-09-18T11:31:31.535 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.536 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.536 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.536 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:123 3.1 3:bae7b855:::93307fb660774b1d286e3149e1a75006840a12081178bf31d320df6af68ad704:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:31.536 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:23.045517+0000", 2024-09-18T11:31:31.536 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.4047624630000008, 2024-09-18T11:31:31.536 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081745810000000002, 2024-09-18T11:31:31.536 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.537 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.537 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.537 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.045517+0000", 2024-09-18T11:31:31.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.537 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.045517+0000", 2024-09-18T11:31:31.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.538 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.538 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.045516+0000", 2024-09-18T11:31:31.538 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:31.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.538 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.538 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046503+0000", 2024-09-18T11:31:31.538 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00098663900000000009 2024-09-18T11:31:31.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046516+0000", 2024-09-18T11:31:31.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2971e-05 2024-09-18T11:31:31.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046522+0000", 2024-09-18T11:31:31.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.618e-06 2024-09-18T11:31:31.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046562+0000", 2024-09-18T11:31:31.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9400999999999998e-05 2024-09-18T11:31:31.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.540 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.540 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046639+0000", 2024-09-18T11:31:31.540 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7037000000000001e-05 2024-09-18T11:31:31.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.540 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:31.540 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046809+0000", 2024-09-18T11:31:31.540 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017013799999999999 2024-09-18T11:31:31.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.540 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:31.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.049963+0000", 2024-09-18T11:31:31.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031541389999999998 2024-09-18T11:31:31.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:31.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.053637+0000", 2024-09-18T11:31:31.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036736550000000001 2024-09-18T11:31:31.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.053665+0000", 2024-09-18T11:31:31.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8759999999999999e-05 2024-09-18T11:31:31.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.542 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.053691+0000", 2024-09-18T11:31:31.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5978000000000001e-05 2024-09-18T11:31:31.542 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.542 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.542 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.542 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:231 4.5 e98/22 4:af4ba75a:::smithi11939506-37:37 v 98'186, mlcod=96'181)", 2024-09-18T11:31:31.542 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.013478+0000", 2024-09-18T11:31:31.542 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.4368013130000001, 2024-09-18T11:31:31.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079316330000000004, 2024-09-18T11:31:31.542 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.543 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.543 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.013478+0000", 2024-09-18T11:31:31.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.013478+0000", 2024-09-18T11:31:31.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.013477+0000", 2024-09-18T11:31:31.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:31.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.013484+0000", 2024-09-18T11:31:31.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6470000000000001e-06 2024-09-18T11:31:31.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.013495+0000", 2024-09-18T11:31:31.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1146e-05 2024-09-18T11:31:31.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.545 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.013500+0000", 2024-09-18T11:31:31.545 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6480000000000002e-06 2024-09-18T11:31:31.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.545 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.014509+0000", 2024-09-18T11:31:31.545 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001009923 2024-09-18T11:31:31.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.545 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.014528+0000", 2024-09-18T11:31:31.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8476999999999999e-05 2024-09-18T11:31:31.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.546 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.021379+0000", 2024-09-18T11:31:31.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068506729999999998 2024-09-18T11:31:31.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.546 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.021409+0000", 2024-09-18T11:31:31.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0902000000000003e-05 2024-09-18T11:31:31.547 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.547 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.547 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.547 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2955 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:31.547 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:31:31.547 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.4230878850000002, 2024-09-18T11:31:31.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010869531, 2024-09-18T11:31:31.547 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.547 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:31.547 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.547 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.547 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.548 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2955 2024-09-18T11:31:31.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.548 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:31:31.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:31:31.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027190+0000", 2024-09-18T11:31:31.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:31.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027196+0000", 2024-09-18T11:31:31.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5590000000000002e-06 2024-09-18T11:31:31.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027208+0000", 2024-09-18T11:31:31.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1816e-05 2024-09-18T11:31:31.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027215+0000", 2024-09-18T11:31:31.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4509999999999997e-06 2024-09-18T11:31:31.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027725+0000", 2024-09-18T11:31:31.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000510053 2024-09-18T11:31:31.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:31.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027803+0000", 2024-09-18T11:31:31.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8750000000000003e-05 2024-09-18T11:31:31.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.551 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032412+0000", 2024-09-18T11:31:31.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046089030000000001 2024-09-18T11:31:31.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.551 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032499+0000", 2024-09-18T11:31:31.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6316000000000005e-05 2024-09-18T11:31:31.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.552 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.552 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038061+0000", 2024-09-18T11:31:31.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005562259 2024-09-18T11:31:31.552 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.552 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.552 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.552 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2956 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~2100908,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:31.552 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:31.552 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.4230540640000005, 2024-09-18T11:31:31.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011863831, 2024-09-18T11:31:31.552 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.553 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:31.553 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.553 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.553 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.553 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2956 2024-09-18T11:31:31.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.553 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:31.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:31.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.554 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:31.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:31.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.554 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027228+0000", 2024-09-18T11:31:31.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:31:31.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027235+0000", 2024-09-18T11:31:31.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:31:31.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027239+0000", 2024-09-18T11:31:31.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.535e-06 2024-09-18T11:31:31.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027831+0000", 2024-09-18T11:31:31.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059244699999999996 2024-09-18T11:31:31.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.556 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:31.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027879+0000", 2024-09-18T11:31:31.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:31:31.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.556 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038098+0000", 2024-09-18T11:31:31.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010219137 2024-09-18T11:31:31.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.557 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038207+0000", 2024-09-18T11:31:31.557 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010951999999999999 2024-09-18T11:31:31.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.557 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039089+0000", 2024-09-18T11:31:31.557 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00088151699999999996 2024-09-18T11:31:31.557 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.557 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.558 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.558 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2957 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:31.558 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:31.558 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.4230362799999998, 2024-09-18T11:31:31.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012041979, 2024-09-18T11:31:31.558 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.558 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:31.558 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.558 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.558 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.558 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2957 2024-09-18T11:31:31.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.559 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:31.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:31.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:31.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:31.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027245+0000", 2024-09-18T11:31:31.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:31:31.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027251+0000", 2024-09-18T11:31:31.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0090000000000004e-06 2024-09-18T11:31:31.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.561 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027255+0000", 2024-09-18T11:31:31.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:31:31.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.561 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027895+0000", 2024-09-18T11:31:31.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064076400000000005 2024-09-18T11:31:31.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.561 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:31.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027930+0000", 2024-09-18T11:31:31.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4334e-05 2024-09-18T11:31:31.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.562 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039111+0000", 2024-09-18T11:31:31.562 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181455999999999 2024-09-18T11:31:31.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.562 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039156+0000", 2024-09-18T11:31:31.562 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5256000000000002e-05 2024-09-18T11:31:31.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.562 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039285+0000", 2024-09-18T11:31:31.563 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012839200000000001 2024-09-18T11:31:31.563 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.563 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.563 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.563 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.563 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.563 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:31.564 INFO:tasks.rados.rados.0.smithi119.stdout:1421: finishing set_chunk to oid 1 2024-09-18T11:31:31.564 INFO:tasks.rados.rados.0.smithi119.stdout:1422: expect (ObjNum 191 snap 21 seq_num 191) 2024-09-18T11:31:31.565 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:31:31.565 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:31.562+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:31.662 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:31.663 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:31.663 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:31.663 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:31.665 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:31.663+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:31.671 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:31:31.766 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:31.765+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:31.767 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:31.767 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:31.767 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:31.767 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:31.767 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:31.775 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:31:31.867 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:31.866+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:31.915 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:31.915 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:31.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:31.916 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:31.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.916 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:31:31.916 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:31.916 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 101.88659215600001, 2024-09-18T11:31:31.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073601550000000002, 2024-09-18T11:31:31.916 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.916 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.916 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.916 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:31:31.916 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:31:31.916 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1 2024-09-18T11:31:31.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.917 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:31.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:31.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026524+0000", 2024-09-18T11:31:31.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999905 2024-09-18T11:31:31.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026541+0000", 2024-09-18T11:31:31.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7070000000000001e-05 2024-09-18T11:31:31.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026554+0000", 2024-09-18T11:31:31.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.288e-05 2024-09-18T11:31:31.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026564+0000", 2024-09-18T11:31:31.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0197000000000001e-05 2024-09-18T11:31:31.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026826+0000", 2024-09-18T11:31:31.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026212400000000003 2024-09-18T11:31:31.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:31:31.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026839+0000", 2024-09-18T11:31:31.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2831e-05 2024-09-18T11:31:31.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028605+0000", 2024-09-18T11:31:31.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001765794 2024-09-18T11:31:31.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028709+0000", 2024-09-18T11:31:31.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104348 2024-09-18T11:31:31.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:31.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028824+0000", 2024-09-18T11:31:31.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011527399999999999 2024-09-18T11:31:31.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:31.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030319+0000", 2024-09-18T11:31:31.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001494439 2024-09-18T11:31:31.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:31.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033805+0000", 2024-09-18T11:31:31.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034857820000000002 2024-09-18T11:31:31.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033856+0000", 2024-09-18T11:31:31.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1687999999999997e-05 2024-09-18T11:31:31.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033894+0000", 2024-09-18T11:31:31.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7481000000000002e-05 2024-09-18T11:31:31.922 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.922 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.922 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.923 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1118 3.2 e38/21 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head v 38'76, mlcod=38'74)", 2024-09-18T11:31:31.923 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:31.923 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 71.965182003999999, 2024-09-18T11:31:31.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006048688, 2024-09-18T11:31:31.923 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.923 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.923 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:31.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:31.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947941+0000", 2024-09-18T11:31:31.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:31.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949868+0000", 2024-09-18T11:31:31.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019267659999999999 2024-09-18T11:31:31.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949893+0000", 2024-09-18T11:31:31.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5239999999999999e-05 2024-09-18T11:31:31.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949901+0000", 2024-09-18T11:31:31.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8420000000000005e-06 2024-09-18T11:31:31.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949954+0000", 2024-09-18T11:31:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3464999999999997e-05 2024-09-18T11:31:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949974+0000", 2024-09-18T11:31:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9731000000000001e-05 2024-09-18T11:31:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953948+0000", 2024-09-18T11:31:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039735489999999998 2024-09-18T11:31:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953992+0000", 2024-09-18T11:31:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4700999999999998e-05 2024-09-18T11:31:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1662 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'73, mlcod=51'69)", 2024-09-18T11:31:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.551576969000003, 2024-09-18T11:31:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059408539999999998, 2024-09-18T11:31:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361914+0000", 2024-09-18T11:31:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036569 2024-09-18T11:31:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361926+0000", 2024-09-18T11:31:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1960999999999999e-05 2024-09-18T11:31:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361937+0000", 2024-09-18T11:31:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0876999999999999e-05 2024-09-18T11:31:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361986+0000", 2024-09-18T11:31:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8652000000000001e-05 2024-09-18T11:31:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.362012+0000", 2024-09-18T11:31:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5854999999999999e-05 2024-09-18T11:31:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367460+0000", 2024-09-18T11:31:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054482100000000002 2024-09-18T11:31:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367490+0000", 2024-09-18T11:31:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9909000000000001e-05 2024-09-18T11:31:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:269 4.7 e58/22 4:e73ba68f:::smithi11939506-1:13 v 58'72, mlcod=57'69)", 2024-09-18T11:31:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 52.647745071999999, 2024-09-18T11:31:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065512569999999996, 2024-09-18T11:31:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265380+0000", 2024-09-18T11:31:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265388+0000", 2024-09-18T11:31:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.174e-06 2024-09-18T11:31:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265398+0000", 2024-09-18T11:31:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2599999999999994e-06 2024-09-18T11:31:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265402+0000", 2024-09-18T11:31:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0289999999999999e-06 2024-09-18T11:31:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265434+0000", 2024-09-18T11:31:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2511000000000001e-05 2024-09-18T11:31:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265447+0000", 2024-09-18T11:31:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2947e-05 2024-09-18T11:31:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271892+0000", 2024-09-18T11:31:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064445930000000002 2024-09-18T11:31:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271932+0000", 2024-09-18T11:31:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0303999999999999e-05 2024-09-18T11:31:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1788 4.8 4:1898b15d:::smithi11939506-25:head [write 453822~797061] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 48.997312489000002, 2024-09-18T11:31:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094305159999999999, 2024-09-18T11:31:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1788 2024-09-18T11:31:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:31.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915811+0000", 2024-09-18T11:31:31.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:31.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916608+0000", 2024-09-18T11:31:31.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079635000000000003 2024-09-18T11:31:31.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916624+0000", 2024-09-18T11:31:31.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6735000000000002e-05 2024-09-18T11:31:31.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916630+0000", 2024-09-18T11:31:31.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5709999999999997e-06 2024-09-18T11:31:31.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916674+0000", 2024-09-18T11:31:31.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4540999999999999e-05 2024-09-18T11:31:31.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916751+0000", 2024-09-18T11:31:31.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6695999999999999e-05 2024-09-18T11:31:31.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:31.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916872+0000", 2024-09-18T11:31:31.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120648 2024-09-18T11:31:31.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:31.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.919076+0000", 2024-09-18T11:31:31.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022046829999999998 2024-09-18T11:31:31.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:31.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925151+0000", 2024-09-18T11:31:31.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060740239999999999 2024-09-18T11:31:31.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925198+0000", 2024-09-18T11:31:31.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7896999999999999e-05 2024-09-18T11:31:31.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925244+0000", 2024-09-18T11:31:31.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5398999999999997e-05 2024-09-18T11:31:31.942 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.942 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.942 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.942 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1934 4.2 e65/22 4:425afe9c:::smithi11939506-29:head v 65'44, mlcod=60'40)", 2024-09-18T11:31:31.942 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:31.942 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 43.613133718999997, 2024-09-18T11:31:31.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064585119999999996, 2024-09-18T11:31:31.943 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.943 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.943 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:31.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:31.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299990+0000", 2024-09-18T11:31:31.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:31.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300789+0000", 2024-09-18T11:31:31.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079851499999999997 2024-09-18T11:31:31.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300806+0000", 2024-09-18T11:31:31.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7096999999999999e-05 2024-09-18T11:31:31.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300813+0000", 2024-09-18T11:31:31.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5079999999999999e-06 2024-09-18T11:31:31.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300870+0000", 2024-09-18T11:31:31.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7178e-05 2024-09-18T11:31:31.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300897+0000", 2024-09-18T11:31:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6284000000000001e-05 2024-09-18T11:31:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306410+0000", 2024-09-18T11:31:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055135210000000004 2024-09-18T11:31:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306451+0000", 2024-09-18T11:31:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0379000000000001e-05 2024-09-18T11:31:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2119 4.7 e73/22 4:e0d95e10:::smithi11939506-22:head v 73'89, mlcod=73'86)", 2024-09-18T11:31:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:55.701097+0000", 2024-09-18T11:31:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.212029072999997, 2024-09-18T11:31:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056939520000000004, 2024-09-18T11:31:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701097+0000", 2024-09-18T11:31:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701097+0000", 2024-09-18T11:31:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701095+0000", 2024-09-18T11:31:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701919+0000", 2024-09-18T11:31:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000824181 2024-09-18T11:31:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701938+0000", 2024-09-18T11:31:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8614000000000002e-05 2024-09-18T11:31:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701947+0000", 2024-09-18T11:31:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0699999999999996e-06 2024-09-18T11:31:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701992+0000", 2024-09-18T11:31:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5614000000000001e-05 2024-09-18T11:31:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.702012+0000", 2024-09-18T11:31:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9639999999999998e-05 2024-09-18T11:31:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.706755+0000", 2024-09-18T11:31:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004742708 2024-09-18T11:31:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.706791+0000", 2024-09-18T11:31:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6106999999999998e-05 2024-09-18T11:31:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2452 4.3 4:ca2b2309:::smithi11939506-44:head [read 0~3462242,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:31:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:31:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.876358362000001, 2024-09-18T11:31:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060906450000000004, 2024-09-18T11:31:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2452 2024-09-18T11:31:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:31:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:31:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036765+0000", 2024-09-18T11:31:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036777+0000", 2024-09-18T11:31:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1422e-05 2024-09-18T11:31:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036793+0000", 2024-09-18T11:31:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6575999999999999e-05 2024-09-18T11:31:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036800+0000", 2024-09-18T11:31:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9940000000000003e-06 2024-09-18T11:31:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5847999999999999e-05 2024-09-18T11:31:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036853+0000", 2024-09-18T11:31:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0119999999999999e-06 2024-09-18T11:31:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038527+0000", 2024-09-18T11:31:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016732730000000001 2024-09-18T11:31:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038580+0000", 2024-09-18T11:31:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3443999999999999e-05 2024-09-18T11:31:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042858+0000", 2024-09-18T11:31:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042781049999999999 2024-09-18T11:31:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2453 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:31:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:31:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.876316129999999, 2024-09-18T11:31:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063798429999999996, 2024-09-18T11:31:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2453 2024-09-18T11:31:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:31:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:31:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036809+0000", 2024-09-18T11:31:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036812+0000", 2024-09-18T11:31:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8339999999999999e-06 2024-09-18T11:31:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036819+0000", 2024-09-18T11:31:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9890000000000003e-06 2024-09-18T11:31:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036823+0000", 2024-09-18T11:31:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8099999999999999e-06 2024-09-18T11:31:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036912+0000", 2024-09-18T11:31:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9083000000000001e-05 2024-09-18T11:31:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036932+0000", 2024-09-18T11:31:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0222000000000001e-05 2024-09-18T11:31:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042896+0000", 2024-09-18T11:31:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059637240000000001 2024-09-18T11:31:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042952+0000", 2024-09-18T11:31:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5877000000000001e-05 2024-09-18T11:31:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043190+0000", 2024-09-18T11:31:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00023774500000000001 2024-09-18T11:31:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2454 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:31:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.876280174000001, 2024-09-18T11:31:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066404469999999998, 2024-09-18T11:31:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2454 2024-09-18T11:31:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036845+0000", 2024-09-18T11:31:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036856+0000", 2024-09-18T11:31:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1338000000000001e-05 2024-09-18T11:31:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036863+0000", 2024-09-18T11:31:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4749999999999998e-06 2024-09-18T11:31:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036867+0000", 2024-09-18T11:31:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3839999999999999e-06 2024-09-18T11:31:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036946+0000", 2024-09-18T11:31:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9054000000000006e-05 2024-09-18T11:31:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036949+0000", 2024-09-18T11:31:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9390000000000002e-06 2024-09-18T11:31:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043213+0000", 2024-09-18T11:31:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062640090000000001 2024-09-18T11:31:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043261+0000", 2024-09-18T11:31:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7750999999999997e-05 2024-09-18T11:31:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043486+0000", 2024-09-18T11:31:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000224985 2024-09-18T11:31:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2505 4.1 e85/22 4:87078dfe:::smithi11939506-14:head v 85'167, mlcod=85'163)", 2024-09-18T11:31:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.993736+0000", 2024-09-18T11:31:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.919389729999999, 2024-09-18T11:31:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056183630000000003, 2024-09-18T11:31:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993736+0000", 2024-09-18T11:31:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993736+0000", 2024-09-18T11:31:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.993735+0000", 2024-09-18T11:31:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.994296+0000", 2024-09-18T11:31:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056099399999999999 2024-09-18T11:31:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.994313+0000", 2024-09-18T11:31:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6982000000000001e-05 2024-09-18T11:31:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.994319+0000", 2024-09-18T11:31:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0279999999999997e-06 2024-09-18T11:31:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.994374+0000", 2024-09-18T11:31:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5019999999999998e-05 2024-09-18T11:31:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.994401+0000", 2024-09-18T11:31:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7104e-05 2024-09-18T11:31:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.999314+0000", 2024-09-18T11:31:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049127390000000002 2024-09-18T11:31:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.999354+0000", 2024-09-18T11:31:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0763000000000003e-05 2024-09-18T11:31:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2553 4.3 4:ca2b2309:::smithi11939506-44:head [write 564768~630013,stat] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-18T11:31:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:31:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.716002541000002, 2024-09-18T11:31:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065686879999999996, 2024-09-18T11:31:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2553 2024-09-18T11:31:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:31:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:31:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197120+0000", 2024-09-18T11:31:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199000+0000", 2024-09-18T11:31:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018792080000000001 2024-09-18T11:31:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199044+0000", 2024-09-18T11:31:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4048999999999998e-05 2024-09-18T11:31:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199071+0000", 2024-09-18T11:31:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7288e-05 2024-09-18T11:31:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199119+0000", 2024-09-18T11:31:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8000999999999997e-05 2024-09-18T11:31:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199208+0000", 2024-09-18T11:31:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9209000000000006e-05 2024-09-18T11:31:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199385+0000", 2024-09-18T11:31:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000177095 2024-09-18T11:31:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.202282+0000", 2024-09-18T11:31:31.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028970469999999998 2024-09-18T11:31:31.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:31.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203578+0000", 2024-09-18T11:31:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012960490000000001 2024-09-18T11:31:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203638+0000", 2024-09-18T11:31:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9599000000000003e-05 2024-09-18T11:31:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203692+0000", 2024-09-18T11:31:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3908e-05 2024-09-18T11:31:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2751 4.3 4:ca2b2309:::smithi11939506-44:25 [read 0~3210104,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:31:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:17.886856+0000", 2024-09-18T11:31:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.0262694, 2024-09-18T11:31:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057395420000000003, 2024-09-18T11:31:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2751 2024-09-18T11:31:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886856+0000", 2024-09-18T11:31:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886856+0000", 2024-09-18T11:31:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886856+0000", 2024-09-18T11:31:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886859+0000", 2024-09-18T11:31:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.886e-06 2024-09-18T11:31:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886871+0000", 2024-09-18T11:31:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2249000000000001e-05 2024-09-18T11:31:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886874+0000", 2024-09-18T11:31:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2160000000000002e-06 2024-09-18T11:31:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.890901+0000", 2024-09-18T11:31:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040264860000000001 2024-09-18T11:31:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.891032+0000", 2024-09-18T11:31:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131438 2024-09-18T11:31:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.892596+0000", 2024-09-18T11:31:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015636350000000001 2024-09-18T11:31:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2752 4.3 4:ca2b2309:::smithi11939506-44:25 [read 0~3210104,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:31:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:31:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.026233381000001, 2024-09-18T11:31:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068715460000000001, 2024-09-18T11:31:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2752 2024-09-18T11:31:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:31:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:31:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:31:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886895+0000", 2024-09-18T11:31:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.993e-06 2024-09-18T11:31:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886901+0000", 2024-09-18T11:31:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1859999999999997e-06 2024-09-18T11:31:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886904+0000", 2024-09-18T11:31:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1920000000000001e-06 2024-09-18T11:31:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.892622+0000", 2024-09-18T11:31:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057175660000000003 2024-09-18T11:31:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.892693+0000", 2024-09-18T11:31:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1265000000000003e-05 2024-09-18T11:31:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.893764+0000", 2024-09-18T11:31:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001070753 2024-09-18T11:31:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2793 4.1 e92/22 4:87078dfe:::smithi11939506-14:head v 92'195, mlcod=92'191)", 2024-09-18T11:31:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:31:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.688333347, 2024-09-18T11:31:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062761989999999997, 2024-09-18T11:31:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:31:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:31:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:31:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224994+0000", 2024-09-18T11:31:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020197600000000001 2024-09-18T11:31:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225007+0000", 2024-09-18T11:31:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3791999999999999e-05 2024-09-18T11:31:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225013+0000", 2024-09-18T11:31:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3630000000000002e-06 2024-09-18T11:31:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225062+0000", 2024-09-18T11:31:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8925000000000003e-05 2024-09-18T11:31:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225086+0000", 2024-09-18T11:31:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4737999999999999e-05 2024-09-18T11:31:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231022+0000", 2024-09-18T11:31:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059360690000000004 2024-09-18T11:31:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231069+0000", 2024-09-18T11:31:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6252999999999999e-05 2024-09-18T11:31:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2876 4.3 4:cc46532a:::smithi11939506-33:2f [read 0~3680733,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:31:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.4062644469999999, 2024-09-18T11:31:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068391210000000001, 2024-09-18T11:31:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2876 2024-09-18T11:31:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:31:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:31:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506860+0000", 2024-09-18T11:31:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506869+0000", 2024-09-18T11:31:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5539999999999998e-06 2024-09-18T11:31:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506880+0000", 2024-09-18T11:31:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1552999999999999e-05 2024-09-18T11:31:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506887+0000", 2024-09-18T11:31:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7329999999999997e-06 2024-09-18T11:31:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506937+0000", 2024-09-18T11:31:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0317e-05 2024-09-18T11:31:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506945+0000", 2024-09-18T11:31:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1550000000000007e-06 2024-09-18T11:31:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.509021+0000", 2024-09-18T11:31:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020759870000000001 2024-09-18T11:31:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.509123+0000", 2024-09-18T11:31:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010132900000000001 2024-09-18T11:31:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513700+0000", 2024-09-18T11:31:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0045777320000000002 2024-09-18T11:31:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2877 4.3 4:cc46532a:::smithi11939506-33:2f [read 0~3680733,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:31:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.4062300329999999, 2024-09-18T11:31:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085715170000000007, 2024-09-18T11:31:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2877 2024-09-18T11:31:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:31:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:31:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:31:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506898+0000", 2024-09-18T11:31:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4760000000000001e-06 2024-09-18T11:31:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506903+0000", 2024-09-18T11:31:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506906+0000", 2024-09-18T11:31:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1769999999999998e-06 2024-09-18T11:31:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507007+0000", 2024-09-18T11:31:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101479 2024-09-18T11:31:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507015+0000", 2024-09-18T11:31:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8490000000000006e-06 2024-09-18T11:31:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513757+0000", 2024-09-18T11:31:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0067417800000000002 2024-09-18T11:31:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513867+0000", 2024-09-18T11:31:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000109987 2024-09-18T11:31:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515467+0000", 2024-09-18T11:31:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00160037 2024-09-18T11:31:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2878 4.3 4:cc46532a:::smithi11939506-33:2f [sparse-read 0~1770976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:31:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.4061995780000007, 2024-09-18T11:31:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090602640000000002, 2024-09-18T11:31:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2878 2024-09-18T11:31:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:31:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:31:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:31:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506929+0000", 2024-09-18T11:31:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.163e-06 2024-09-18T11:31:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506936+0000", 2024-09-18T11:31:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5519999999999999e-06 2024-09-18T11:31:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506941+0000", 2024-09-18T11:31:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507036+0000", 2024-09-18T11:31:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5070999999999998e-05 2024-09-18T11:31:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507042+0000", 2024-09-18T11:31:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3959999999999997e-06 2024-09-18T11:31:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515504+0000", 2024-09-18T11:31:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084620300000000006 2024-09-18T11:31:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515625+0000", 2024-09-18T11:31:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012150999999999999 2024-09-18T11:31:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515986+0000", 2024-09-18T11:31:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036140100000000002 2024-09-18T11:31:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:686 4.7 e98/22 4:e5707c18:::smithi11939506-10:30 v 98'160, mlcod=97'155)", 2024-09-18T11:31:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:31:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.894357888, 2024-09-18T11:31:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074641239999999999, 2024-09-18T11:31:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:31:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:31:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018767+0000", 2024-09-18T11:31:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018773+0000", 2024-09-18T11:31:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1460000000000004e-06 2024-09-18T11:31:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018781+0000", 2024-09-18T11:31:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9780000000000005e-06 2024-09-18T11:31:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018785+0000", 2024-09-18T11:31:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0029999999999996e-06 2024-09-18T11:31:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018826+0000", 2024-09-18T11:31:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1616999999999998e-05 2024-09-18T11:31:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018843+0000", 2024-09-18T11:31:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6753999999999999e-05 2024-09-18T11:31:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026202+0000", 2024-09-18T11:31:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073592620000000001 2024-09-18T11:31:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026232+0000", 2024-09-18T11:31:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9779000000000001e-05 2024-09-18T11:31:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.4.0:621 4.1 e98/22 4:854aa293:::smithi11939506-26:35 v 98'215, mlcod=96'210)", 2024-09-18T11:31:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:31:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.8934744490000002, 2024-09-18T11:31:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007014752, 2024-09-18T11:31:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:31:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:31:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:31:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019659+0000", 2024-09-18T11:31:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8229999999999995e-06 2024-09-18T11:31:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019671+0000", 2024-09-18T11:31:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2636000000000001e-05 2024-09-18T11:31:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019677+0000", 2024-09-18T11:31:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5470000000000004e-06 2024-09-18T11:31:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019710+0000", 2024-09-18T11:31:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3191999999999997e-05 2024-09-18T11:31:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019722+0000", 2024-09-18T11:31:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2357e-05 2024-09-18T11:31:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026639+0000", 2024-09-18T11:31:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0069167630000000003 2024-09-18T11:31:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026666+0000", 2024-09-18T11:31:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7087999999999999e-05 2024-09-18T11:31:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.017 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:32.019 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:31:32.019 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:32.017+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:32.069 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:32.067+0000 7f2d449e8640 -1 osd.1 103 handle_osd_map injecting map crc failure 2024-09-18T11:31:32.071 INFO:tasks.rados.rados.0.smithi119.stdout:1416: done (8 left) 2024-09-18T11:31:32.071 INFO:tasks.rados.rados.0.smithi119.stdout:1417: done (7 left) 2024-09-18T11:31:32.071 INFO:tasks.rados.rados.0.smithi119.stdout:1418: done (6 left) 2024-09-18T11:31:32.071 INFO:tasks.rados.rados.0.smithi119.stdout:1419: done (5 left) 2024-09-18T11:31:32.071 INFO:tasks.rados.rados.0.smithi119.stdout:1420: done (4 left) 2024-09-18T11:31:32.071 INFO:tasks.rados.rados.0.smithi119.stdout:1421: done (3 left) 2024-09-18T11:31:32.072 INFO:tasks.rados.rados.0.smithi119.stdout:1422: done (2 left) 2024-09-18T11:31:32.072 INFO:tasks.rados.rados.0.smithi119.stdout:1423: done (1 left) 2024-09-18T11:31:32.072 INFO:tasks.rados.rados.0.smithi119.stdout:1424: done (0 left) 2024-09-18T11:31:32.072 INFO:tasks.rados.rados.0.smithi119.stdout:1425: snap_create 2024-09-18T11:31:32.119 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:32.117+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:32.158 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:32.158 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:32.158 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:32.158 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:32.166 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:31:32.220 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:32.219+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:32.265 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:32.266 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:32.266 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:32.266 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:32.266 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:32.274 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:31:32.358 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:32.357+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:32.415 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:32.415 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:32.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:32.415 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:32.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.415 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:32.415 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:32.415 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 81.296316998999998, 2024-09-18T11:31:32.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:31:32.416 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.416 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:32.416 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.416 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:32.416 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:32.416 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:31:32.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.416 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:32.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:32.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:31:32.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:32.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:31:32.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:31:32.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:31:32.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:31:32.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:31:32.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:31:32.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:31:32.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:31:32.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:32.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:31:32.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:31:32.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:31:32.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:31:32.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:31:32.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:31:32.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:31:32.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:31:32.420 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.420 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.420 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.421 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:31:32.421 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:32.421 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 71.743291597999999, 2024-09-18T11:31:32.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:31:32.421 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.421 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:32.421 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.421 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:32.421 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:32.421 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:31:32.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.421 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:32.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:32.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:31:32.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:32.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:31:32.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:31:32.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:31:32.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:31:32.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:31:32.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:31:32.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:31:32.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:31:32.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:31:32.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:31:32.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:31:32.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:31:32.425 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.425 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.425 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.425 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1688 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 19=[19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-18T11:31:32.425 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:32.425 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 52.880137005000002, 2024-09-18T11:31:32.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071575789999999999, 2024-09-18T11:31:32.425 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.425 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.425 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.426 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:32.426 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:32.426 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1688 2024-09-18T11:31:32.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.426 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:32.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:32.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:32.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:32.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532500+0000", 2024-09-18T11:31:32.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5260000000000001e-06 2024-09-18T11:31:32.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532503+0000", 2024-09-18T11:31:32.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:31:32.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532505+0000", 2024-09-18T11:31:32.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.889e-06 2024-09-18T11:31:32.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.534722+0000", 2024-09-18T11:31:32.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002217119 2024-09-18T11:31:32.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.534827+0000", 2024-09-18T11:31:32.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0001049 2024-09-18T11:31:32.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:32.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.535007+0000", 2024-09-18T11:31:32.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017965599999999999 2024-09-18T11:31:32.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539463+0000", 2024-09-18T11:31:32.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004455713 2024-09-18T11:31:32.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539607+0000", 2024-09-18T11:31:32.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014440300000000001 2024-09-18T11:31:32.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539634+0000", 2024-09-18T11:31:32.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7398000000000001e-05 2024-09-18T11:31:32.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539657+0000", 2024-09-18T11:31:32.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2228000000000001e-05 2024-09-18T11:31:32.431 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.431 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.431 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.431 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1760 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2319366,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:32.431 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:31:32.431 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 50.118642455, 2024-09-18T11:31:32.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070132529999999997, 2024-09-18T11:31:32.431 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.431 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:32.432 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.432 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:32.432 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:32.432 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1760 2024-09-18T11:31:32.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.432 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:31:32.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:31:32.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293993+0000", 2024-09-18T11:31:32.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293997+0000", 2024-09-18T11:31:32.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4999999999999999e-06 2024-09-18T11:31:32.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294004+0000", 2024-09-18T11:31:32.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3950000000000003e-06 2024-09-18T11:31:32.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294009+0000", 2024-09-18T11:31:32.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4989999999999996e-06 2024-09-18T11:31:32.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294103+0000", 2024-09-18T11:31:32.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4817999999999999e-05 2024-09-18T11:31:32.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:32.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294110+0000", 2024-09-18T11:31:32.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7240000000000003e-06 2024-09-18T11:31:32.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300406+0000", 2024-09-18T11:31:32.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062964470000000002 2024-09-18T11:31:32.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300457+0000", 2024-09-18T11:31:32.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0915000000000003e-05 2024-09-18T11:31:32.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.301007+0000", 2024-09-18T11:31:32.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00054942299999999995 2024-09-18T11:31:32.436 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.436 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.436 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.436 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1775 4.a 4:588debc2:::smithi11939506-18:head [write 1527054~766050] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:32.437 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:31:32.437 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 49.516289714000003, 2024-09-18T11:31:32.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070939050000000002, 2024-09-18T11:31:32.437 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.437 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.437 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.437 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:32.437 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:32.437 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1775 2024-09-18T11:31:32.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.437 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:31:32.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:31:32.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896345+0000", 2024-09-18T11:31:32.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:32.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896993+0000", 2024-09-18T11:31:32.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064769999999999997 2024-09-18T11:31:32.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897011+0000", 2024-09-18T11:31:32.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8355999999999998e-05 2024-09-18T11:31:32.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897016+0000", 2024-09-18T11:31:32.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3890000000000004e-06 2024-09-18T11:31:32.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897054+0000", 2024-09-18T11:31:32.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7963000000000002e-05 2024-09-18T11:31:32.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897123+0000", 2024-09-18T11:31:32.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8128999999999998e-05 2024-09-18T11:31:32.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:32.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897224+0000", 2024-09-18T11:31:32.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010168800000000001 2024-09-18T11:31:32.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899515+0000", 2024-09-18T11:31:32.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022910280000000001 2024-09-18T11:31:32.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903333+0000", 2024-09-18T11:31:32.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003817459 2024-09-18T11:31:32.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903388+0000", 2024-09-18T11:31:32.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5651e-05 2024-09-18T11:31:32.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903440+0000", 2024-09-18T11:31:32.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1836999999999999e-05 2024-09-18T11:31:32.442 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.443 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.443 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.444 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1776 4.a 4:588debc2:::smithi11939506-18:head [setxattr _header (57) in=7b,truncate 2420208] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:32.444 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:31:32.444 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 49.515611902000003, 2024-09-18T11:31:32.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065840400000000002, 2024-09-18T11:31:32.444 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.444 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.444 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.444 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:32.444 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:32.444 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1776 2024-09-18T11:31:32.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.445 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:31:32.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:31:32.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897024+0000", 2024-09-18T11:31:32.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897026+0000", 2024-09-18T11:31:32.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9579999999999999e-06 2024-09-18T11:31:32.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897030+0000", 2024-09-18T11:31:32.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5370000000000002e-06 2024-09-18T11:31:32.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897032+0000", 2024-09-18T11:31:32.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.17e-06 2024-09-18T11:31:32.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898415+0000", 2024-09-18T11:31:32.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013823189999999999 2024-09-18T11:31:32.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898527+0000", 2024-09-18T11:31:32.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011236800000000001 2024-09-18T11:31:32.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:32.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.898714+0000", 2024-09-18T11:31:32.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018703599999999999 2024-09-18T11:31:32.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899683+0000", 2024-09-18T11:31:32.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00096944700000000002 2024-09-18T11:31:32.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903475+0000", 2024-09-18T11:31:32.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037920129999999999 2024-09-18T11:31:32.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903535+0000", 2024-09-18T11:31:32.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9843999999999998e-05 2024-09-18T11:31:32.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903608+0000", 2024-09-18T11:31:32.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2946000000000003e-05 2024-09-18T11:31:32.450 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.450 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.450 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.450 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1777 4.a 4:588debc2:::smithi11939506-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:32.450 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:31:32.450 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 49.515585262000002, 2024-09-18T11:31:32.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0069010809999999999, 2024-09-18T11:31:32.450 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.451 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:32.451 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.451 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:32.451 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:32.451 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1777 2024-09-18T11:31:32.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.451 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:31:32.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:31:32.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897050+0000", 2024-09-18T11:31:32.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897052+0000", 2024-09-18T11:31:32.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:31:32.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897057+0000", 2024-09-18T11:31:32.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9470000000000002e-06 2024-09-18T11:31:32.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897060+0000", 2024-09-18T11:31:32.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3410000000000001e-06 2024-09-18T11:31:32.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899417+0000", 2024-09-18T11:31:32.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023571170000000002 2024-09-18T11:31:32.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:32.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899501+0000", 2024-09-18T11:31:32.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4160999999999996e-05 2024-09-18T11:31:32.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903661+0000", 2024-09-18T11:31:32.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041599510000000003 2024-09-18T11:31:32.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903790+0000", 2024-09-18T11:31:32.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012941499999999999 2024-09-18T11:31:32.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903952+0000", 2024-09-18T11:31:32.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016146800000000001 2024-09-18T11:31:32.455 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.455 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.455 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.455 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1895 4.5 4:a087a566:::smithi11939506-46:head [read 0~2756315,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:32.456 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:32.456 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.104329757000002, 2024-09-18T11:31:32.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007540377, 2024-09-18T11:31:32.456 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.456 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:32.456 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.456 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:32.456 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:32.456 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1895 2024-09-18T11:31:32.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.456 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:32.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:32.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:32.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308310+0000", 2024-09-18T11:31:32.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0559999999999998e-06 2024-09-18T11:31:32.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308317+0000", 2024-09-18T11:31:32.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7209999999999997e-06 2024-09-18T11:31:32.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308320+0000", 2024-09-18T11:31:32.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.721e-06 2024-09-18T11:31:32.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308447+0000", 2024-09-18T11:31:32.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012645199999999999 2024-09-18T11:31:32.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:32.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308451+0000", 2024-09-18T11:31:32.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6800000000000001e-06 2024-09-18T11:31:32.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.312462+0000", 2024-09-18T11:31:32.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040102139999999998 2024-09-18T11:31:32.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.312585+0000", 2024-09-18T11:31:32.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012388899999999999 2024-09-18T11:31:32.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.315847+0000", 2024-09-18T11:31:32.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003261197 2024-09-18T11:31:32.460 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.460 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.461 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.461 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1924 4.5 4:af4ba75a:::smithi11939506-37:head [write 1829449~675380] snapc 1e=[1e,1c,1b,19,18,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:32.461 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.362228+0000", 2024-09-18T11:31:32.461 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.050408058000002, 2024-09-18T11:31:32.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066182139999999999, 2024-09-18T11:31:32.461 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.461 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.461 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.461 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:32.462 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:32.462 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1924 2024-09-18T11:31:32.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.462 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362228+0000", 2024-09-18T11:31:32.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362228+0000", 2024-09-18T11:31:32.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362227+0000", 2024-09-18T11:31:32.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:32.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362749+0000", 2024-09-18T11:31:32.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00052262399999999996 2024-09-18T11:31:32.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362762+0000", 2024-09-18T11:31:32.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2821e-05 2024-09-18T11:31:32.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362767+0000", 2024-09-18T11:31:32.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5349999999999998e-06 2024-09-18T11:31:32.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362806+0000", 2024-09-18T11:31:32.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9029999999999997e-05 2024-09-18T11:31:32.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362863+0000", 2024-09-18T11:31:32.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7584999999999998e-05 2024-09-18T11:31:32.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:32.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362949+0000", 2024-09-18T11:31:32.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5655000000000006e-05 2024-09-18T11:31:32.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364314+0000", 2024-09-18T11:31:32.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001364961 2024-09-18T11:31:32.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.368771+0000", 2024-09-18T11:31:32.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004457244 2024-09-18T11:31:32.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.368809+0000", 2024-09-18T11:31:32.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8368999999999998e-05 2024-09-18T11:31:32.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.368846+0000", 2024-09-18T11:31:32.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6718000000000003e-05 2024-09-18T11:31:32.467 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.467 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.467 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.467 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1926 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:32.467 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:32.467 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.049838649999998, 2024-09-18T11:31:32.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066190219999999996, 2024-09-18T11:31:32.467 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.467 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:32.467 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.468 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:32.468 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:32.468 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1926 2024-09-18T11:31:32.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.468 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:32.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:32.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:32.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362799+0000", 2024-09-18T11:31:32.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.029e-06 2024-09-18T11:31:32.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362804+0000", 2024-09-18T11:31:32.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3989999999999997e-06 2024-09-18T11:31:32.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362806+0000", 2024-09-18T11:31:32.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3020000000000002e-06 2024-09-18T11:31:32.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364105+0000", 2024-09-18T11:31:32.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001299478 2024-09-18T11:31:32.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:32.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364153+0000", 2024-09-18T11:31:32.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8059000000000002e-05 2024-09-18T11:31:32.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369273+0000", 2024-09-18T11:31:32.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051199540000000003 2024-09-18T11:31:32.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369343+0000", 2024-09-18T11:31:32.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0191999999999995e-05 2024-09-18T11:31:32.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369416+0000", 2024-09-18T11:31:32.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2925999999999999e-05 2024-09-18T11:31:32.472 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.472 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.472 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.472 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1969 4.a 4:588debc2:::smithi11939506-18:18 [read 0~3949612,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:32.472 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:32.472 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 41.038649761000002, 2024-09-18T11:31:32.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075409190000000001, 2024-09-18T11:31:32.473 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.473 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:32.473 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.473 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:32.473 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:32.473 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1969 2024-09-18T11:31:32.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.473 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:32.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:32.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:32.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373990+0000", 2024-09-18T11:31:32.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1119999999999998e-06 2024-09-18T11:31:32.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373997+0000", 2024-09-18T11:31:32.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4200000000000001e-06 2024-09-18T11:31:32.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.374002+0000", 2024-09-18T11:31:32.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:31:32.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380200+0000", 2024-09-18T11:31:32.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006197302 2024-09-18T11:31:32.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380261+0000", 2024-09-18T11:31:32.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1389999999999993e-05 2024-09-18T11:31:32.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.381527+0000", 2024-09-18T11:31:32.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012660080000000001 2024-09-18T11:31:32.477 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.477 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.477 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.477 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2207 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1410296~761389] snapc 24=[24,1e,1c,1b,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T11:31:32.477 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:32.477 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 34.990138934999997, 2024-09-18T11:31:32.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008410743, 2024-09-18T11:31:32.477 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.477 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.477 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.477 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:32.477 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:32.478 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2207 2024-09-18T11:31:32.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.478 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:32.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:32.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422496+0000", 2024-09-18T11:31:32.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:32.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423190+0000", 2024-09-18T11:31:32.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00069405499999999995 2024-09-18T11:31:32.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423204+0000", 2024-09-18T11:31:32.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:31:32.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423208+0000", 2024-09-18T11:31:32.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:31:32.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423293+0000", 2024-09-18T11:31:32.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4524999999999995e-05 2024-09-18T11:31:32.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423394+0000", 2024-09-18T11:31:32.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101395 2024-09-18T11:31:32.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:32.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423579+0000", 2024-09-18T11:31:32.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018461000000000001 2024-09-18T11:31:32.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.428574+0000", 2024-09-18T11:31:32.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049944960000000002 2024-09-18T11:31:32.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430809+0000", 2024-09-18T11:31:32.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022351820000000001 2024-09-18T11:31:32.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430862+0000", 2024-09-18T11:31:32.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3189999999999997e-05 2024-09-18T11:31:32.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430908+0000", 2024-09-18T11:31:32.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5973999999999997e-05 2024-09-18T11:31:32.483 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.483 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.483 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.483 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2339 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:32.483 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:32.483 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.025834306, 2024-09-18T11:31:32.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011809992, 2024-09-18T11:31:32.483 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.483 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:32.483 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.483 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:32.483 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:32.484 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2339 2024-09-18T11:31:32.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.484 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:32.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:32.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:31:32.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:32.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:31:32.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4710000000000001e-06 2024-09-18T11:31:32.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386824+0000", 2024-09-18T11:31:32.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6104e-05 2024-09-18T11:31:32.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386836+0000", 2024-09-18T11:31:32.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1827999999999999e-05 2024-09-18T11:31:32.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388390+0000", 2024-09-18T11:31:32.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015539499999999999 2024-09-18T11:31:32.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:32.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388443+0000", 2024-09-18T11:31:32.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3785000000000001e-05 2024-09-18T11:31:32.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398238+0000", 2024-09-18T11:31:32.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097947989999999999 2024-09-18T11:31:32.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398315+0000", 2024-09-18T11:31:32.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6909999999999994e-05 2024-09-18T11:31:32.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398612+0000", 2024-09-18T11:31:32.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029671999999999999 2024-09-18T11:31:32.488 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.488 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.488 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.488 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2340 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:32.489 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:32.489 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.025790994000001, 2024-09-18T11:31:32.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012086448, 2024-09-18T11:31:32.489 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.489 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:32.489 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.489 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:32.489 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:32.489 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2340 2024-09-18T11:31:32.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.489 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:32.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:32.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:32.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386848+0000", 2024-09-18T11:31:32.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3069999999999998e-06 2024-09-18T11:31:32.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386855+0000", 2024-09-18T11:31:32.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0979999999999996e-06 2024-09-18T11:31:32.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386859+0000", 2024-09-18T11:31:32.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9360000000000001e-06 2024-09-18T11:31:32.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388466+0000", 2024-09-18T11:31:32.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001607154 2024-09-18T11:31:32.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:32.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388507+0000", 2024-09-18T11:31:32.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1093000000000001e-05 2024-09-18T11:31:32.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398647+0000", 2024-09-18T11:31:32.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010139686 2024-09-18T11:31:32.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.493 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398718+0000", 2024-09-18T11:31:32.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1138999999999998e-05 2024-09-18T11:31:32.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.493 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398931+0000", 2024-09-18T11:31:32.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021356899999999999 2024-09-18T11:31:32.493 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.493 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.493 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.494 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2341 4.5 4:aa575018:::smithi11939506-11:head [read 0~2487436,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:32.494 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:32.494 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.025772772, 2024-09-18T11:31:32.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013751648999999999, 2024-09-18T11:31:32.494 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.494 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:32.494 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.494 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:32.495 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:32.495 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2341 2024-09-18T11:31:32.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.495 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:32.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:32.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:32.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386866+0000", 2024-09-18T11:31:32.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4249999999999998e-06 2024-09-18T11:31:32.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386872+0000", 2024-09-18T11:31:32.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.003e-06 2024-09-18T11:31:32.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:31:32.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.507e-06 2024-09-18T11:31:32.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388520+0000", 2024-09-18T11:31:32.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001644146 2024-09-18T11:31:32.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:32.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388561+0000", 2024-09-18T11:31:32.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1486999999999998e-05 2024-09-18T11:31:32.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398963+0000", 2024-09-18T11:31:32.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010401353 2024-09-18T11:31:32.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.399046+0000", 2024-09-18T11:31:32.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3141999999999998e-05 2024-09-18T11:31:32.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.400615+0000", 2024-09-18T11:31:32.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015690960000000001 2024-09-18T11:31:32.499 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.499 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.499 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.499 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2529 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:32.499 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:31:32.499 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.239299507999998, 2024-09-18T11:31:32.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079268120000000001, 2024-09-18T11:31:32.499 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.499 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.500 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.500 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:32.500 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:32.500 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2529 2024-09-18T11:31:32.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.500 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:31:32.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:31:32.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173336+0000", 2024-09-18T11:31:32.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173339+0000", 2024-09-18T11:31:32.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9019999999999999e-06 2024-09-18T11:31:32.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173381+0000", 2024-09-18T11:31:32.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1836999999999999e-05 2024-09-18T11:31:32.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173390+0000", 2024-09-18T11:31:32.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.302e-06 2024-09-18T11:31:32.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174507+0000", 2024-09-18T11:31:32.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001116868 2024-09-18T11:31:32.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174603+0000", 2024-09-18T11:31:32.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5598000000000002e-05 2024-09-18T11:31:32.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:32.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174759+0000", 2024-09-18T11:31:32.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015616299999999999 2024-09-18T11:31:32.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178943+0000", 2024-09-18T11:31:32.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041846180000000002 2024-09-18T11:31:32.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.504 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181175+0000", 2024-09-18T11:31:32.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00223215 2024-09-18T11:31:32.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.504 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181229+0000", 2024-09-18T11:31:32.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3585999999999998e-05 2024-09-18T11:31:32.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.504 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181263+0000", 2024-09-18T11:31:32.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4292999999999998e-05 2024-09-18T11:31:32.505 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.505 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.505 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.505 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2530 4.5 4:af4ba75a:::smithi11939506-37:head [rollback 25] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:32.505 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:31:32.505 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.239237202000002, 2024-09-18T11:31:32.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080198800000000001, 2024-09-18T11:31:32.505 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.505 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.505 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.506 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:32.506 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:32.506 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2530 2024-09-18T11:31:32.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.506 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:31:32.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:31:32.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173398+0000", 2024-09-18T11:31:32.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:32.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173403+0000", 2024-09-18T11:31:32.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4389999999999999e-06 2024-09-18T11:31:32.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173414+0000", 2024-09-18T11:31:32.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0411000000000001e-05 2024-09-18T11:31:32.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173420+0000", 2024-09-18T11:31:32.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9610000000000002e-06 2024-09-18T11:31:32.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176390+0000", 2024-09-18T11:31:32.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029706289999999998 2024-09-18T11:31:32.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176501+0000", 2024-09-18T11:31:32.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011090300000000001 2024-09-18T11:31:32.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:32.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176827+0000", 2024-09-18T11:31:32.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00032526899999999999 2024-09-18T11:31:32.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178958+0000", 2024-09-18T11:31:32.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021314860000000001 2024-09-18T11:31:32.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181318+0000", 2024-09-18T11:31:32.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002359571 2024-09-18T11:31:32.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181372+0000", 2024-09-18T11:31:32.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3884000000000001e-05 2024-09-18T11:31:32.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181419+0000", 2024-09-18T11:31:32.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7148000000000003e-05 2024-09-18T11:31:32.511 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.511 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.511 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.511 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2531 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:32.511 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:32.511 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.239211079, 2024-09-18T11:31:32.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080687239999999993, 2024-09-18T11:31:32.511 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.511 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.511 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.512 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:32.512 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:32.512 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2531 2024-09-18T11:31:32.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.512 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:32.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:32.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:32.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173429+0000", 2024-09-18T11:31:32.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2989999999999998e-06 2024-09-18T11:31:32.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173436+0000", 2024-09-18T11:31:32.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1799999999999999e-06 2024-09-18T11:31:32.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173441+0000", 2024-09-18T11:31:32.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8810000000000004e-06 2024-09-18T11:31:32.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178267+0000", 2024-09-18T11:31:32.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048257969999999997 2024-09-18T11:31:32.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178388+0000", 2024-09-18T11:31:32.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121779 2024-09-18T11:31:32.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:32.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178599+0000", 2024-09-18T11:31:32.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000210094 2024-09-18T11:31:32.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.179339+0000", 2024-09-18T11:31:32.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00074063799999999995 2024-09-18T11:31:32.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181444+0000", 2024-09-18T11:31:32.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021044100000000001 2024-09-18T11:31:32.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181470+0000", 2024-09-18T11:31:32.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6415999999999999e-05 2024-09-18T11:31:32.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181494+0000", 2024-09-18T11:31:32.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3672e-05 2024-09-18T11:31:32.517 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.517 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.517 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.517 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2910 4.5 4:af4ba75a:::smithi11939506-37:head [write 786627~795991,stat] snapc 37=[37,35,34,30,2f,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:32.517 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:31:32.517 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.4091532499999992, 2024-09-18T11:31:32.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084093710000000006, 2024-09-18T11:31:32.517 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.517 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.517 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.517 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:32.518 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:32.518 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2910 2024-09-18T11:31:32.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.518 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:31:32.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:31:32.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.519 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003481+0000", 2024-09-18T11:31:32.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:32.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.519 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004300+0000", 2024-09-18T11:31:32.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081934200000000003 2024-09-18T11:31:32.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.519 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004324+0000", 2024-09-18T11:31:32.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4618999999999999e-05 2024-09-18T11:31:32.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004335+0000", 2024-09-18T11:31:32.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.092e-05 2024-09-18T11:31:32.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004385+0000", 2024-09-18T11:31:32.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9709999999999997e-05 2024-09-18T11:31:32.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004470+0000", 2024-09-18T11:31:32.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4864000000000006e-05 2024-09-18T11:31:32.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:32.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004612+0000", 2024-09-18T11:31:32.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014241500000000001 2024-09-18T11:31:32.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011193+0000", 2024-09-18T11:31:32.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006580654 2024-09-18T11:31:32.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011788+0000", 2024-09-18T11:31:32.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059533600000000002 2024-09-18T11:31:32.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011842+0000", 2024-09-18T11:31:32.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3180000000000002e-05 2024-09-18T11:31:32.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011892+0000", 2024-09-18T11:31:32.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0618000000000002e-05 2024-09-18T11:31:32.522 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.523 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.523 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.523 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2953 4.a 4:588debc2:::smithi11939506-18:head [assert-version v113,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:32.523 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:31:32.523 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.3858377320000006, 2024-09-18T11:31:32.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083931279999999997, 2024-09-18T11:31:32.523 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.523 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.523 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.523 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:32.523 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:32.524 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2953 2024-09-18T11:31:32.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.524 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.524 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:31:32.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.524 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:31:32.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026796+0000", 2024-09-18T11:31:32.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:32.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026806+0000", 2024-09-18T11:31:32.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9529999999999994e-06 2024-09-18T11:31:32.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026824+0000", 2024-09-18T11:31:32.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7901999999999999e-05 2024-09-18T11:31:32.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026832+0000", 2024-09-18T11:31:32.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8450000000000003e-06 2024-09-18T11:31:32.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026883+0000", 2024-09-18T11:31:32.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0883000000000001e-05 2024-09-18T11:31:32.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026980+0000", 2024-09-18T11:31:32.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7526000000000001e-05 2024-09-18T11:31:32.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:32.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032722+0000", 2024-09-18T11:31:32.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057414689999999999 2024-09-18T11:31:32.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.034230+0000", 2024-09-18T11:31:32.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001508035 2024-09-18T11:31:32.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035090+0000", 2024-09-18T11:31:32.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00086047300000000005 2024-09-18T11:31:32.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035147+0000", 2024-09-18T11:31:32.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6598000000000003e-05 2024-09-18T11:31:32.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035191+0000", 2024-09-18T11:31:32.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4747999999999999e-05 2024-09-18T11:31:32.528 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.528 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.529 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.529 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.529 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.529 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:32.530 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:31:32.531 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:32.529+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:32.631 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:32.629+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:32.669 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:32.669 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:32.669 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:32.669 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:32.678 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:31:32.732 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:32.731+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:32.775 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:32.775 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:32.776 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:32.776 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:32.776 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:32.783 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:31:32.834 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:32.832+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:32.883 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:32.883 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:32.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:32.883 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:32.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.883 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.723070726, 2024-09-18T11:31:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055400850000000001, 2024-09-18T11:31:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 43 2024-09-18T11:31:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:32.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157504+0000", 2024-09-18T11:31:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4519999999999996e-06 2024-09-18T11:31:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157534+0000", 2024-09-18T11:31:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.97e-05 2024-09-18T11:31:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157539+0000", 2024-09-18T11:31:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4360000000000001e-06 2024-09-18T11:31:32.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159428+0000", 2024-09-18T11:31:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018888220000000001 2024-09-18T11:31:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159479+0000", 2024-09-18T11:31:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0608000000000001e-05 2024-09-18T11:31:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159559+0000", 2024-09-18T11:31:32.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0667000000000005e-05 2024-09-18T11:31:32.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162779+0000", 2024-09-18T11:31:32.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032197459999999999 2024-09-18T11:31:32.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162950+0000", 2024-09-18T11:31:32.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017059599999999999 2024-09-18T11:31:32.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163001+0000", 2024-09-18T11:31:32.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0968999999999999e-05 2024-09-18T11:31:32.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163040+0000", 2024-09-18T11:31:32.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8843000000000001e-05 2024-09-18T11:31:32.890 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.890 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.890 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.890 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:32.890 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:32.890 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.723019572, 2024-09-18T11:31:32.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:31:32.891 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.891 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.891 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.891 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:32.891 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:32.891 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:31:32.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.891 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:32.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:32.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:31:32.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:31:32.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:31:32.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:31:32.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:31:32.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:31:32.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:32.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:31:32.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:31:32.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:31:32.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:31:32.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:32.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:31:32.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:31:32.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:31:32.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:31:32.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:31:32.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:31:32.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:31:32.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:31:32.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:31:32.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:31:32.896 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.896 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.896 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.896 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:32.896 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:32.896 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.72289331, 2024-09-18T11:31:32.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055717040000000002, 2024-09-18T11:31:32.896 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.896 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.896 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.896 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:32.897 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:32.897 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 45 2024-09-18T11:31:32.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.897 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:32.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:32.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157676+0000", 2024-09-18T11:31:32.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157694+0000", 2024-09-18T11:31:32.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7422999999999999e-05 2024-09-18T11:31:32.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157704+0000", 2024-09-18T11:31:32.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9000000000000001e-06 2024-09-18T11:31:32.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157718+0000", 2024-09-18T11:31:32.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4788e-05 2024-09-18T11:31:32.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160937+0000", 2024-09-18T11:31:32.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00321886 2024-09-18T11:31:32.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161023+0000", 2024-09-18T11:31:32.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6063999999999995e-05 2024-09-18T11:31:32.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:32.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161202+0000", 2024-09-18T11:31:32.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017912599999999999 2024-09-18T11:31:32.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163052+0000", 2024-09-18T11:31:32.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001849554 2024-09-18T11:31:32.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163204+0000", 2024-09-18T11:31:32.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015186900000000001 2024-09-18T11:31:32.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163229+0000", 2024-09-18T11:31:32.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:31:32.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163249+0000", 2024-09-18T11:31:32.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9992000000000001e-05 2024-09-18T11:31:32.901 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.902 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.902 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.902 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:32.902 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:32.902 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.722844197, 2024-09-18T11:31:32.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:31:32.902 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.902 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.902 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.903 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:32.903 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:32.903 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:31:32.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.903 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:32.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:32.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:32.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:31:32.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:31:32.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:31:32.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:31:32.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:31:32.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:31:32.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:31:32.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:31:32.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:31:32.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:31:32.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:32.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:31:32.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:31:32.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:31:32.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:31:32.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:31:32.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:31:32.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:31:32.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:31:32.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:31:32.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:31:32.907 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.907 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.907 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.908 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:32.908 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:32.908 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.722801666, 2024-09-18T11:31:32.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:31:32.908 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.908 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.908 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.908 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:32.908 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:32.908 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:31:32.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.908 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:32.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:32.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:31:32.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:31:32.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:31:32.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:31:32.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:31:32.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:31:32.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:31:32.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:31:32.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:31:32.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:31:32.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:31:32.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:32.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:31:32.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:31:32.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:31:32.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:31:32.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:31:32.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:31:32.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:31:32.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:31:32.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:31:32.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:31:32.914 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.914 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.914 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.914 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:32.914 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:32.914 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.722746736, 2024-09-18T11:31:32.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:31:32.914 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.914 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.914 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.914 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:32.915 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:32.915 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:31:32.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.915 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:32.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:32.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:32.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:31:32.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:31:32.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:31:32.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:31:32.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:31:32.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:31:32.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:31:32.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:31:32.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:31:32.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:31:32.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:32.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:31:32.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:31:32.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:31:32.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:31:32.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:31:32.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:31:32.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:31:32.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:31:32.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:31:32.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:31:32.920 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.920 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.920 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.920 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:32.920 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:32.920 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.722736756, 2024-09-18T11:31:32.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:31:32.920 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.920 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.920 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.920 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:32.920 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:32.920 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:31:32.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.921 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:32.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:32.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:32.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:32.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:31:32.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:31:32.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:31:32.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:31:32.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:31:32.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:31:32.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:31:32.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:31:32.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:31:32.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:31:32.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:32.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:31:32.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:31:32.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:31:32.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:31:32.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:31:32.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:31:32.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:31:32.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:31:32.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:31:32.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:31:32.925 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.926 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.926 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.926 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:32.926 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:32.926 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.722725157, 2024-09-18T11:31:32.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:31:32.926 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.926 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.926 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.926 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:32.926 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:32.926 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:31:32.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.927 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:32.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:32.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:32.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:32.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:31:32.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:31:32.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:31:32.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:31:32.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:31:32.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:31:32.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:31:32.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:31:32.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:31:32.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:31:32.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:32.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:31:32.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:31:32.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:31:32.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:31:32.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:31:32.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:31:32.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:31:32.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:31:32.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:31:32.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:31:32.931 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.931 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.931 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.932 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:32.932 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:32.932 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.722668676, 2024-09-18T11:31:32.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:31:32.932 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.932 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.932 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.932 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:32.933 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:32.933 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:31:32.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.933 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:32.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:32.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:31:32.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:31:32.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:31:32.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:31:32.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:31:32.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:31:32.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:31:32.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:31:32.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:31:32.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:31:32.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:31:32.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:32.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:31:32.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:31:32.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:31:32.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:31:32.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:31:32.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:31:32.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:31:32.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:31:32.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:31:32.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:31:32.938 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.938 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.938 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.938 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:32.938 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:32.938 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.72265485200001, 2024-09-18T11:31:32.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:31:32.938 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.938 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.939 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.939 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:32.939 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:32.939 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:31:32.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.939 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:32.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:31:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:31:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:31:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:31:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:31:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:31:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:31:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:31:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:31:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:31:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:32.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:31:32.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:31:32.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:31:32.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:31:32.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:31:32.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:31:32.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:31:32.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:31:32.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:31:32.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:31:32.945 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.945 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.945 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.945 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:32.945 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:32.945 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.306475945, 2024-09-18T11:31:32.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:31:32.945 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.945 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.945 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.945 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:32.945 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:32.946 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:31:32.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.946 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:32.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:32.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:32.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:32.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:31:32.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:31:32.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:31:32.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:31:32.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:31:32.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:31:32.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:31:32.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:31:32.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:31:32.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:31:32.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:32.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:31:32.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:31:32.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:31:32.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:31:32.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:31:32.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:31:32.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:31:32.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:31:32.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:31:32.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:31:32.951 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.951 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.952 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.952 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:32.952 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:32.952 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.30645264499999, 2024-09-18T11:31:32.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:31:32.952 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.952 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:32.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:32.952 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:31:32.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.952 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:32.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:32.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:31:32.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:31:32.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:31:32.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:31:32.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:31:32.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:31:32.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:31:32.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:31:32.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:31:32.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:31:32.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:31:32.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:32.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:31:32.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:31:32.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:31:32.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:31:32.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:31:32.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:31:32.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:31:32.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:31:32.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:31:32.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:31:32.958 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.958 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.958 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.958 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:32.958 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:32.958 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.306367036, 2024-09-18T11:31:32.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:31:32.958 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.958 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.958 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.958 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:32.958 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:32.958 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:31:32.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.959 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:32.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:32.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:32.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:31:32.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:31:32.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:31:32.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:31:32.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:31:32.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:31:32.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:31:32.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:31:32.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:31:32.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:31:32.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:32.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:31:32.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:31:32.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:31:32.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:31:32.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:31:32.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:31:32.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:31:32.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:31:32.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:31:32.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:31:32.963 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.963 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.963 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.964 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:32.964 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:32.964 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.306356071, 2024-09-18T11:31:32.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:31:32.964 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.964 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.964 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.964 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:32.964 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:32.964 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:31:32.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.965 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:32.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:32.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:32.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:32.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:31:32.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:31:32.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:31:32.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:31:32.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:31:32.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:31:32.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:31:32.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:31:32.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:31:32.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:31:32.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:32.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:31:32.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:31:32.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:31:32.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:31:32.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:31:32.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:31:32.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:31:32.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:31:32.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:31:32.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:31:32.969 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.969 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.969 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.969 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:32.969 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:32.969 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.306344903, 2024-09-18T11:31:32.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:31:32.970 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.970 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.970 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.970 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:32.970 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:32.970 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:31:32.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.970 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:32.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:32.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:32.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:32.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:31:32.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:31:32.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:31:32.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:31:32.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:31:32.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:31:32.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:31:32.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:31:32.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:31:32.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:31:32.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:32.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:31:32.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:31:32.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:31:32.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:31:32.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:31:32.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:31:32.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:31:32.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:31:32.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:31:32.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:31:32.974 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.975 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.975 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.975 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:32.975 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:32.975 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.306279574, 2024-09-18T11:31:32.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:31:32.975 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.975 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.975 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.975 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:32.975 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:32.975 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:31:32.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.976 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:32.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:32.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:31:32.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:31:32.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:31:32.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:31:32.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:31:32.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:31:32.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:31:32.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:31:32.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:31:32.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:31:32.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:31:32.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:32.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:31:32.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:31:32.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:31:32.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:31:32.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:31:32.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:31:32.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:31:32.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:31:32.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:31:32.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:31:32.981 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.981 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.981 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.981 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:32.981 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:32.981 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.296117596, 2024-09-18T11:31:32.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:31:32.981 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.981 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.981 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.981 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:32.982 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:32.982 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:31:32.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.982 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:32.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:32.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:31:32.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:31:32.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:31:32.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:31:32.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:31:32.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:31:32.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:31:32.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:31:32.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:31:32.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:31:32.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:31:32.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:32.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:31:32.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:31:32.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:31:32.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:31:32.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:31:32.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:31:32.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:31:32.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:31:32.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:31:32.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:31:32.986 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.986 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.986 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.986 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:32.986 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:32.987 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.296025117, 2024-09-18T11:31:32.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:31:32.987 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.987 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.987 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:32.987 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:32.987 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:32.987 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:31:32.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.987 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:32.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:32.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:32.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:31:32.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:31:32.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:31:32.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:31:32.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:31:32.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:31:32.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:31:32.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:31:32.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:31:32.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:31:32.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:32.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:31:32.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:31:32.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:32.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:31:32.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:31:32.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:32.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:31:32.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:31:32.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:31:32.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:31:32.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:31:32.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:31:32.992 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.992 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.992 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.992 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1788 4.8 e60/22 4:1898b15d:::smithi11939506-25:head v 60'12, mlcod=58'9)", 2024-09-18T11:31:32.992 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:32.992 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 49.963619628000004, 2024-09-18T11:31:32.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058348599999999999, 2024-09-18T11:31:32.992 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.992 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.992 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:32.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:32.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916949+0000", 2024-09-18T11:31:32.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:32.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917602+0000", 2024-09-18T11:31:32.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00065337100000000003 2024-09-18T11:31:32.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917621+0000", 2024-09-18T11:31:32.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9287e-05 2024-09-18T11:31:32.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917629+0000", 2024-09-18T11:31:32.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3780000000000003e-06 2024-09-18T11:31:32.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917677+0000", 2024-09-18T11:31:32.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8068000000000001e-05 2024-09-18T11:31:32.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917699+0000", 2024-09-18T11:31:32.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2149000000000001e-05 2024-09-18T11:31:32.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922747+0000", 2024-09-18T11:31:32.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050477609999999996 2024-09-18T11:31:32.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:32.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922785+0000", 2024-09-18T11:31:32.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8726000000000002e-05 2024-09-18T11:31:32.996 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.996 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:32.996 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:32.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.996 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:58 4.3 e91/67 4:c70c3162:::smithi11939506-35:18 v 91'101, mlcod=86'95)", 2024-09-18T11:31:32.996 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:32.996 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.864961404999999, 2024-09-18T11:31:32.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055647500000000003, 2024-09-18T11:31:32.996 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:32.996 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:32.996 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:32.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:32.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:32.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:32.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:32.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:32.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:32.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015608+0000", 2024-09-18T11:31:32.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:32.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:32.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015614+0000", 2024-09-18T11:31:32.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5310000000000001e-06 2024-09-18T11:31:32.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:32.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015622+0000", 2024-09-18T11:31:32.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0809999999999994e-06 2024-09-18T11:31:32.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:32.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015627+0000", 2024-09-18T11:31:32.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8439999999999997e-06 2024-09-18T11:31:32.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:32.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016003+0000", 2024-09-18T11:31:32.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037652999999999997 2024-09-18T11:31:32.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:32.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016024+0000", 2024-09-18T11:31:32.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0780000000000001e-05 2024-09-18T11:31:32.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:32.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:32.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:32.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021141+0000", 2024-09-18T11:31:32.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051166639999999999 2024-09-18T11:31:33.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:33.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:33.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:33.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021174+0000", 2024-09-18T11:31:33.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2839000000000002e-05 2024-09-18T11:31:33.000 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:33.000 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:33.000 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:33.000 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:33.000 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:33.000 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:33.002 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:33.000+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:33.002 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:31:33.012 INFO:tasks.rados.rados.0.smithi119.stdout:1425: done (0 left) 2024-09-18T11:31:33.012 INFO:tasks.rados.rados.0.smithi119.stdout:1426: tier_evict oid 28 2024-09-18T11:31:33.012 INFO:tasks.rados.rados.0.smithi119.stdout:1426: tier_evict oid 28 snap -1 2024-09-18T11:31:33.012 INFO:tasks.rados.rados.0.smithi119.stdout:1427: set_chunk oid 49 target oid 2024-09-18T11:31:33.019 INFO:tasks.rados.rados.0.smithi119.stdout:1427: set_chunk oid 49 offset: 1051136 length: 600576 target oid 1fa8eaf134ee041b41707be87a9928e02ffb7c7f2d814425473d38f8454b47bb offset: 0 snap 30 2024-09-18T11:31:33.019 INFO:tasks.rados.rados.0.smithi119.stdout:1428: read oid 37 snap 32 2024-09-18T11:31:33.019 INFO:tasks.rados.rados.0.smithi119.stdout:1428: expect (ObjNum 261 snap 31 seq_num 261) 2024-09-18T11:31:33.019 INFO:tasks.rados.rados.0.smithi119.stdout:1429: tier_promote oid 38 2024-09-18T11:31:33.019 INFO:tasks.rados.rados.0.smithi119.stdout:1426: got (22) Invalid argument 2024-09-18T11:31:33.019 INFO:tasks.rados.rados.0.smithi119.stdout:1426: done (3 left) 2024-09-18T11:31:33.019 INFO:tasks.rados.rados.0.smithi119.stdout:1430: read oid 43 snap 30 2024-09-18T11:31:33.019 INFO:tasks.rados.rados.0.smithi119.stdout:1430: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:31:33.020 INFO:tasks.rados.rados.0.smithi119.stdout:1431: read oid 14 snap -1 2024-09-18T11:31:33.020 INFO:tasks.rados.rados.0.smithi119.stdout:1431: expect (ObjNum 246 snap 28 seq_num 246) 2024-09-18T11:31:33.020 INFO:tasks.rados.rados.0.smithi119.stdout:1432: read oid 24 snap -1 2024-09-18T11:31:33.020 INFO:tasks.rados.rados.0.smithi119.stdout:1432: expect (ObjNum 258 snap 31 seq_num 258) 2024-09-18T11:31:33.020 INFO:tasks.rados.rados.0.smithi119.stdout:1433: set_chunk oid 6 target oid 2024-09-18T11:31:33.020 INFO:tasks.rados.rados.0.smithi119.stdout:1433: set_chunk oid 6 offset: 1420800 length: 215552 target oid smithi11939506- offset: 1420800 snap 31 2024-09-18T11:31:33.020 INFO:tasks.rados.rados.0.smithi119.stdout:1434: delete oid 2 current snap is 34 2024-09-18T11:31:33.020 INFO:tasks.rados.rados.0.smithi119.stdout:1433: finishing set_chunk to oid 6 2024-09-18T11:31:33.020 INFO:tasks.rados.rados.0.smithi119.stdout:1433: got expected ERANGE (src dne) 2024-09-18T11:31:33.023 INFO:tasks.rados.rados.0.smithi119.stdout:1432: expect (ObjNum 258 snap 31 seq_num 258) 2024-09-18T11:31:33.094 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T11:31:33.115 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:33.115 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:33.115 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:33.115 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:33.116 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:33.113+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:33.124 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:31:33.197 INFO:tasks.rados.rados.0.smithi119.stdout:1432: done (7 left) 2024-09-18T11:31:33.197 INFO:tasks.rados.rados.0.smithi119.stdout:1433: done (6 left) 2024-09-18T11:31:33.197 INFO:tasks.rados.rados.0.smithi119.stdout:1434: done (5 left) 2024-09-18T11:31:33.197 INFO:tasks.rados.rados.0.smithi119.stdout:1435: write oid 5 current snap is 34 2024-09-18T11:31:33.197 INFO:tasks.rados.rados.0.smithi119.stdout:1427: finishing set_chunk to oid 49 2024-09-18T11:31:33.197 INFO:tasks.rados.rados.0.smithi119.stdout:1435: seq_num 272 ranges {571504=540083,1783808=702748} 2024-09-18T11:31:33.203 INFO:tasks.rados.rados.0.smithi119.stdout:1435: writing smithi11939506-5 from 571504 to 1111587 tid 1 2024-09-18T11:31:33.203 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:33.201+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:33.211 INFO:tasks.rados.rados.0.smithi119.stdout:1435: writing smithi11939506-5 from 1783808 to 2486556 tid 2 2024-09-18T11:31:33.211 INFO:tasks.rados.rados.0.smithi119.stdout:1427: done (5 left) 2024-09-18T11:31:33.211 INFO:tasks.rados.rados.0.smithi119.stdout:1436: read oid 36 snap -1 2024-09-18T11:31:33.211 INFO:tasks.rados.rados.0.smithi119.stdout:1436: expect deleted 2024-09-18T11:31:33.211 INFO:tasks.rados.rados.0.smithi119.stdout:1437: tier_flush oid 30 2024-09-18T11:31:33.211 INFO:tasks.rados.rados.0.smithi119.stdout:1437: tier_flush oid 30 snap -1 2024-09-18T11:31:33.211 INFO:tasks.rados.rados.0.smithi119.stdout:1430: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:31:33.247 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:31:33.268 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:33.268 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:33.269 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:33.269 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:33.269 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:33.278 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:31:33.303 INFO:tasks.rados.rados.0.smithi119.stdout:1428: expect (ObjNum 261 snap 31 seq_num 261) 2024-09-18T11:31:33.304 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:33.303+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:33.350 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:33.365 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:33.366 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:33.382 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:33.382 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:33.398 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:33.398 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:33.404 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:33.403+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:33.407 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:33.407 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:33.414 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:33.414 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:33.415 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:31:33.429 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:33.430 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:33.445 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:33.445 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:33.447 INFO:tasks.rados.rados.0.smithi119.stdout:1429: got (0) Success 2024-09-18T11:31:33.447 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 124 (ObjNum 231 snap 27 seq_num 231) dirty exists 2024-09-18T11:31:33.448 INFO:tasks.rados.rados.0.smithi119.stdout:1431: expect (ObjNum 246 snap 28 seq_num 246) 2024-09-18T11:31:33.460 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:33.460 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:33.463 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:33.464 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:33.505 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:33.503+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:33.515 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:33.515 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:33.523 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:31:33.607 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:33.605+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:33.622 INFO:tasks.rados.rados.0.smithi119.stdout:1435: finishing write tid 1 to smithi11939506-5 2024-09-18T11:31:33.622 INFO:tasks.rados.rados.0.smithi119.stdout:1437: got (0) Success 2024-09-18T11:31:33.622 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 30 v 64 (ObjNum 129 snap 0 seq_num 384106944) clean dne 2024-09-18T11:31:33.623 INFO:tasks.rados.rados.0.smithi119.stdout:1435: finishing write tid 2 to smithi11939506-5 2024-09-18T11:31:33.623 INFO:tasks.rados.rados.0.smithi119.stdout:1435: finishing write tid 3 to smithi11939506-5 2024-09-18T11:31:33.623 INFO:tasks.rados.rados.0.smithi119.stdout:1435: finishing write tid 4 to smithi11939506-5 2024-09-18T11:31:33.623 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 177 (ObjNum 272 snap 34 seq_num 272) dirty exists 2024-09-18T11:31:33.623 INFO:tasks.rados.rados.0.smithi119.stdout:1435: left oid 5 (ObjNum 272 snap 34 seq_num 272) 2024-09-18T11:31:33.623 INFO:tasks.rados.rados.0.smithi119.stdout:1428: done (6 left) 2024-09-18T11:31:33.624 INFO:tasks.rados.rados.0.smithi119.stdout:1429: done (5 left) 2024-09-18T11:31:33.624 INFO:tasks.rados.rados.0.smithi119.stdout:1430: done (4 left) 2024-09-18T11:31:33.624 INFO:tasks.rados.rados.0.smithi119.stdout:1431: done (3 left) 2024-09-18T11:31:33.624 INFO:tasks.rados.rados.0.smithi119.stdout:1435: done (2 left) 2024-09-18T11:31:33.624 INFO:tasks.rados.rados.0.smithi119.stdout:1436: done (1 left) 2024-09-18T11:31:33.624 INFO:tasks.rados.rados.0.smithi119.stdout:1437: done (0 left) 2024-09-18T11:31:33.625 INFO:tasks.rados.rados.0.smithi119.stdout:1438: set_chunk oid 17 target oid 2024-09-18T11:31:33.625 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:33.622+0000 7f6bf3052640 -1 osd.5 104 handle_osd_map injecting map crc failure 2024-09-18T11:31:33.626 INFO:tasks.rados.rados.0.smithi119.stdout:1438: set_chunk oid 17 offset: 1510912 length: 38400 target oid 5c9b37300155c1f3338bc88f17d69c51fb248b0de0155630962ca6504d789bba offset: 0 snap -1 2024-09-18T11:31:33.626 INFO:tasks.rados.rados.0.smithi119.stdout:1439: set_chunk oid 8 target oid 2024-09-18T11:31:33.633 INFO:tasks.rados.rados.0.smithi119.stdout:1439: set_chunk oid 8 offset: 1349632 length: 1416192 target oid 474603b115e2445e2dfa75e884c997a735f8204dff6c6ae784eb0e406a45ecbb offset: 0 snap -1 2024-09-18T11:31:33.633 INFO:tasks.rados.rados.0.smithi119.stdout:1440: read oid 37 snap -1 2024-09-18T11:31:33.633 INFO:tasks.rados.rados.0.smithi119.stdout:1440: expect (ObjNum 261 snap 31 seq_num 261) 2024-09-18T11:31:33.633 INFO:tasks.rados.rados.0.smithi119.stdout:1441: set_chunk oid 32 target oid 2024-09-18T11:31:33.638 INFO:tasks.rados.rados.0.smithi119.stdout:1441: set_chunk oid 32 offset: 1252352 length: 513536 target oid 5e209eef37073a2e8daef1a0c4441faa1fb7ce47d59403514df302883c5dfdb2 offset: 0 snap -1 2024-09-18T11:31:33.638 INFO:tasks.rados.rados.0.smithi119.stdout:1442: read oid 21 snap 29 2024-09-18T11:31:33.638 INFO:tasks.rados.rados.0.smithi119.stdout:1442: expect (ObjNum 222 snap 26 seq_num 222) 2024-09-18T11:31:33.639 INFO:tasks.rados.rados.0.smithi119.stdout:1438: finishing set_chunk to oid 17 2024-09-18T11:31:33.639 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 281 (ObjNum 247 snap 28 seq_num 247) dirty exists 2024-09-18T11:31:33.639 INFO:tasks.rados.rados.0.smithi119.stdout:1443: read oid 3 snap -1 2024-09-18T11:31:33.639 INFO:tasks.rados.rados.0.smithi119.stdout:1443: expect (ObjNum 243 snap 28 seq_num 243) 2024-09-18T11:31:33.639 INFO:tasks.rados.rados.0.smithi119.stdout:1438: done (5 left) 2024-09-18T11:31:33.640 INFO:tasks.rados.rados.0.smithi119.stdout:1444: snap_create 2024-09-18T11:31:33.640 INFO:tasks.rados.rados.0.smithi119.stdout:1439: finishing set_chunk to oid 8 2024-09-18T11:31:33.640 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 8 set_chunk returned error code -95 offset: 1349632 length: 1416192 2024-09-18T11:31:33.640 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 8 v 0 (ObjNum 53 snap 0 seq_num 53) dirty exists 2024-09-18T11:31:33.641 INFO:tasks.rados.rados.0.smithi119.stdout:1442: expect (ObjNum 222 snap 26 seq_num 222) 2024-09-18T11:31:33.666 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:33.666 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:33.675 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:31:33.706 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:33.705+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:33.807 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:33.805+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:33.812 INFO:tasks.rados.rados.0.smithi119.stdout:1441: finishing set_chunk to oid 32 2024-09-18T11:31:33.812 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 120 (ObjNum 217 snap 26 seq_num 217) dirty exists 2024-09-18T11:31:33.812 INFO:tasks.rados.rados.0.smithi119.stdout:1443: expect (ObjNum 243 snap 28 seq_num 243) 2024-09-18T11:31:33.813 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:33.813 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:33.822 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:31:33.908 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:33.907+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:33.929 INFO:tasks.rados.rados.0.smithi119.stdout:1440: expect (ObjNum 261 snap 31 seq_num 261) 2024-09-18T11:31:33.962 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:33.962 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:33.970 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:31:34.009 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:34.008+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:34.068 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:34.068 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:34.078 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is set 2024-09-18T11:31:34.079 INFO:tasks.rados.rados.0.smithi119.stdout:1439: done (5 left) 2024-09-18T11:31:34.079 INFO:tasks.rados.rados.0.smithi119.stdout:1440: done (4 left) 2024-09-18T11:31:34.079 INFO:tasks.rados.rados.0.smithi119.stdout:1441: done (3 left) 2024-09-18T11:31:34.079 INFO:tasks.rados.rados.0.smithi119.stdout:1442: done (2 left) 2024-09-18T11:31:34.079 INFO:tasks.rados.rados.0.smithi119.stdout:1443: done (1 left) 2024-09-18T11:31:34.079 INFO:tasks.rados.rados.0.smithi119.stdout:1444: done (0 left) 2024-09-18T11:31:34.079 INFO:tasks.rados.rados.0.smithi119.stdout:1445: tier_promote oid 32 2024-09-18T11:31:34.079 INFO:tasks.rados.rados.0.smithi119.stdout:1446: rollback oid 19 current snap is 35 2024-09-18T11:31:34.079 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 19 to 23 2024-09-18T11:31:34.079 INFO:tasks.rados.rados.0.smithi119.stdout:1447: read oid 14 snap 30 2024-09-18T11:31:34.079 INFO:tasks.rados.rados.0.smithi119.stdout:1447: expect (ObjNum 246 snap 28 seq_num 246) 2024-09-18T11:31:34.080 INFO:tasks.rados.rados.0.smithi119.stdout:1448: set_chunk oid 35 target oid 2024-09-18T11:31:34.080 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:31:34.086 INFO:tasks.rados.rados.0.smithi119.stdout:1448: set_chunk oid 35 offset: 1644032 length: 428032 target oid 9be4be36b656a282792ff5d0c6a3706cf48e224b1601d674853cfc025fe56000 offset: 0 snap -1 2024-09-18T11:31:34.086 INFO:tasks.rados.rados.0.smithi119.stdout:1449: read oid 26 snap -1 2024-09-18T11:31:34.086 INFO:tasks.rados.rados.0.smithi119.stdout:1449: expect (ObjNum 256 snap 30 seq_num 256) 2024-09-18T11:31:34.086 INFO:tasks.rados.rados.0.smithi119.stdout:1450: set_chunk oid 46 target oid 2024-09-18T11:31:34.095 INFO:tasks.rados.rados.0.smithi119.stdout:1450: set_chunk oid 46 offset: 11264 length: 779264 target oid ea50c052465acf5a55c8c73196028c3f6644667deca7c68967a7c47850ba9a7e offset: 0 snap -1 2024-09-18T11:31:34.095 INFO:tasks.rados.rados.0.smithi119.stdout:1451: read oid 7 snap -1 2024-09-18T11:31:34.096 INFO:tasks.rados.rados.0.smithi119.stdout:1451: expect (ObjNum 145 snap 18 seq_num 145) 2024-09-18T11:31:34.096 INFO:tasks.rados.rados.0.smithi119.stdout:1452: read oid 10 snap 29 2024-09-18T11:31:34.096 INFO:tasks.rados.rados.0.smithi119.stdout:1452: expect (ObjNum 240 snap 28 seq_num 240) 2024-09-18T11:31:34.096 INFO:tasks.rados.rados.0.smithi119.stdout:1453: write (excl) oid 48 current snap is 35 2024-09-18T11:31:34.096 INFO:tasks.rados.rados.0.smithi119.stdout:1446: finishing rollback tid 0 to smithi11939506-19 2024-09-18T11:31:34.096 INFO:tasks.rados.rados.0.smithi119.stdout:1453: seq_num 273 ranges {588311=615332,1842290=562759} 2024-09-18T11:31:34.102 INFO:tasks.rados.rados.0.smithi119.stdout:1453: writing smithi11939506-48 from 588311 to 1203643 tid 1 2024-09-18T11:31:34.109 INFO:tasks.rados.rados.0.smithi119.stdout:1453: writing smithi11939506-48 from 1842290 to 2405049 tid 2 2024-09-18T11:31:34.109 INFO:tasks.rados.rados.0.smithi119.stdout:1454: read oid 42 snap -1 2024-09-18T11:31:34.109 INFO:tasks.rados.rados.0.smithi119.stdout:1454: expect (ObjNum 251 snap 30 seq_num 251) 2024-09-18T11:31:34.109 INFO:tasks.rados.rados.0.smithi119.stdout:1455: write (excl) oid 37 current snap is 35 2024-09-18T11:31:34.110 INFO:tasks.rados.rados.0.smithi119.stdout:1446: finishing rollback tid 1 to smithi11939506-19 2024-09-18T11:31:34.110 INFO:tasks.rados.rados.0.smithi119.stdout:1446: finishing rollback tid 2 to smithi11939506-19 2024-09-18T11:31:34.110 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 194 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-18T11:31:34.110 INFO:tasks.rados.rados.0.smithi119.stdout:1445: got (0) Success 2024-09-18T11:31:34.110 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 120 (ObjNum 217 snap 26 seq_num 217) dirty exists 2024-09-18T11:31:34.110 INFO:tasks.rados.rados.0.smithi119.stdout:1447: expect (ObjNum 246 snap 28 seq_num 246) 2024-09-18T11:31:34.110 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:34.108+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:34.186 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:34.186 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:34.195 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:31:34.238 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:34.237+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:34.280 INFO:tasks.rados.rados.0.smithi119.stdout:1448: finishing set_chunk to oid 35 2024-09-18T11:31:34.280 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 51 (ObjNum 155 snap 20 seq_num 155) dirty exists 2024-09-18T11:31:34.281 INFO:tasks.rados.rados.0.smithi119.stdout:1455: seq_num 274 ranges {605118=406934,1617125=706417} 2024-09-18T11:31:34.285 INFO:tasks.rados.rados.0.smithi119.stdout:1455: writing smithi11939506-37 from 605118 to 1012052 tid 1 2024-09-18T11:31:34.292 INFO:tasks.rados.rados.0.smithi119.stdout:1455: writing smithi11939506-37 from 1617125 to 2323542 tid 2 2024-09-18T11:31:34.292 INFO:tasks.rados.rados.0.smithi119.stdout:1445: done (10 left) 2024-09-18T11:31:34.292 INFO:tasks.rados.rados.0.smithi119.stdout:1446: done (9 left) 2024-09-18T11:31:34.293 INFO:tasks.rados.rados.0.smithi119.stdout:1447: done (8 left) 2024-09-18T11:31:34.293 INFO:tasks.rados.rados.0.smithi119.stdout:1448: done (7 left) 2024-09-18T11:31:34.293 INFO:tasks.rados.rados.0.smithi119.stdout:1456: set_chunk oid 5 target oid 2024-09-18T11:31:34.296 INFO:tasks.rados.rados.0.smithi119.stdout:1456: set_chunk oid 5 offset: 1699840 length: 105472 target oid 9b7c6e16d96061ea3ac2e82c9baaa6feb962ccaa6c51ee8fc802a81494794f96 offset: 0 snap -1 2024-09-18T11:31:34.296 INFO:tasks.rados.rados.0.smithi119.stdout:1457: read oid 40 snap -1 2024-09-18T11:31:34.296 INFO:tasks.rados.rados.0.smithi119.stdout:1457: expect (ObjNum 175 snap 20 seq_num 175) 2024-09-18T11:31:34.296 INFO:tasks.rados.rados.0.smithi119.stdout:1458: tier_evict oid 49 2024-09-18T11:31:34.296 INFO:tasks.rados.rados.0.smithi119.stdout:1458: tier_evict oid 49 snap -1 2024-09-18T11:31:34.296 INFO:tasks.rados.rados.0.smithi119.stdout:1450: finishing set_chunk to oid 46 2024-09-18T11:31:34.296 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 46 set_chunk returned error code -95 offset: 11264 length: 779264 2024-09-18T11:31:34.296 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 0 (ObjNum 171 snap 20 seq_num 171) dirty exists 2024-09-18T11:31:34.296 INFO:tasks.rados.rados.0.smithi119.stdout:1449: expect (ObjNum 256 snap 30 seq_num 256) 2024-09-18T11:31:34.297 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:34.297 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:34.305 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:31:34.346 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:34.345+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:34.411 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:34.410+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:34.443 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:34.443 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:34.452 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:31:34.457 INFO:tasks.rados.rados.0.smithi119.stdout:1452: expect (ObjNum 240 snap 28 seq_num 240) 2024-09-18T11:31:34.536 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:34.535+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:34.537 INFO:tasks.rados.rados.0.smithi119.stdout:1451: expect (ObjNum 145 snap 18 seq_num 145) 2024-09-18T11:31:34.590 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:34.590 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:34.598 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:31:34.613 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:34.611+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:34.679 INFO:tasks.rados.rados.0.smithi119.stdout:1453: finishing write tid 1 to smithi11939506-48 2024-09-18T11:31:34.679 INFO:tasks.rados.rados.0.smithi119.stdout:1453: finishing write tid 2 to smithi11939506-48 2024-09-18T11:31:34.679 INFO:tasks.rados.rados.0.smithi119.stdout:1453: finishing write tid 3 to smithi11939506-48 2024-09-18T11:31:34.679 INFO:tasks.rados.rados.0.smithi119.stdout:1453: finishing write tid 4 to smithi11939506-48 2024-09-18T11:31:34.679 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 48 v 209 (ObjNum 273 snap 35 seq_num 273) dirty exists 2024-09-18T11:31:34.679 INFO:tasks.rados.rados.0.smithi119.stdout:1453: left oid 48 (ObjNum 273 snap 35 seq_num 273) 2024-09-18T11:31:34.679 INFO:tasks.rados.rados.0.smithi119.stdout:1454: expect (ObjNum 251 snap 30 seq_num 251) 2024-09-18T11:31:34.714 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:34.712+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:34.737 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:34.737 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:34.746 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:31:34.776 INFO:tasks.rados.rados.0.smithi119.stdout:1455: finishing write tid 1 to smithi11939506-37 2024-09-18T11:31:34.776 INFO:tasks.rados.rados.0.smithi119.stdout:1458: got (22) Invalid argument 2024-09-18T11:31:34.776 INFO:tasks.rados.rados.0.smithi119.stdout:1456: finishing set_chunk to oid 5 2024-09-18T11:31:34.776 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 177 (ObjNum 272 snap 34 seq_num 272) dirty exists 2024-09-18T11:31:34.776 INFO:tasks.rados.rados.0.smithi119.stdout:1457: expect (ObjNum 175 snap 20 seq_num 175) 2024-09-18T11:31:34.815 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:34.813+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:34.842 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:34.843 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:34.851 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:31:34.873 INFO:tasks.rados.rados.0.smithi119.stdout:1455: finishing write tid 2 to smithi11939506-37 2024-09-18T11:31:34.873 INFO:tasks.rados.rados.0.smithi119.stdout:1455: finishing write tid 3 to smithi11939506-37 2024-09-18T11:31:34.873 INFO:tasks.rados.rados.0.smithi119.stdout:1455: finishing write tid 4 to smithi11939506-37 2024-09-18T11:31:34.873 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 213 (ObjNum 274 snap 35 seq_num 274) dirty exists 2024-09-18T11:31:34.873 INFO:tasks.rados.rados.0.smithi119.stdout:1455: left oid 37 (ObjNum 274 snap 35 seq_num 274) 2024-09-18T11:31:34.873 INFO:tasks.rados.rados.0.smithi119.stdout:1449: done (9 left) 2024-09-18T11:31:34.873 INFO:tasks.rados.rados.0.smithi119.stdout:1450: done (8 left) 2024-09-18T11:31:34.874 INFO:tasks.rados.rados.0.smithi119.stdout:1451: done (7 left) 2024-09-18T11:31:34.874 INFO:tasks.rados.rados.0.smithi119.stdout:1452: done (6 left) 2024-09-18T11:31:34.874 INFO:tasks.rados.rados.0.smithi119.stdout:1453: done (5 left) 2024-09-18T11:31:34.874 INFO:tasks.rados.rados.0.smithi119.stdout:1454: done (4 left) 2024-09-18T11:31:34.874 INFO:tasks.rados.rados.0.smithi119.stdout:1455: done (3 left) 2024-09-18T11:31:34.874 INFO:tasks.rados.rados.0.smithi119.stdout:1456: done (2 left) 2024-09-18T11:31:34.874 INFO:tasks.rados.rados.0.smithi119.stdout:1457: done (1 left) 2024-09-18T11:31:34.874 INFO:tasks.rados.rados.0.smithi119.stdout:1458: done (0 left) 2024-09-18T11:31:34.874 INFO:tasks.rados.rados.0.smithi119.stdout:1459: set_chunk oid 42 target oid 2024-09-18T11:31:34.881 INFO:tasks.rados.rados.0.smithi119.stdout:1459: set_chunk oid 42 offset: 922624 length: 960000 target oid 7075474d775ef62b433667d4d17a39e9dcef78733f5cfc7a2cb8f44c44efc0b7 offset: 0 snap -1 2024-09-18T11:31:34.881 INFO:tasks.rados.rados.0.smithi119.stdout:1460: set_chunk oid 13 target oid 2024-09-18T11:31:34.884 INFO:tasks.rados.rados.0.smithi119.stdout:1460: set_chunk oid 13 offset: 1336320 length: 231936 target oid 41de80ba7eac8415f51c3aa26173359f06f3551c8cd054133d1b78224fce1ac0 offset: 0 snap 27 2024-09-18T11:31:34.884 INFO:tasks.rados.rados.0.smithi119.stdout:1461: read oid 41 snap -1 2024-09-18T11:31:34.885 INFO:tasks.rados.rados.0.smithi119.stdout:1461: expect (ObjNum 237 snap 28 seq_num 237) 2024-09-18T11:31:34.885 INFO:tasks.rados.rados.0.smithi119.stdout:1462: read oid 35 snap -1 2024-09-18T11:31:34.885 INFO:tasks.rados.rados.0.smithi119.stdout:1462: expect (ObjNum 155 snap 20 seq_num 155) 2024-09-18T11:31:34.885 INFO:tasks.rados.rados.0.smithi119.stdout:1463: read oid 50 snap -1 2024-09-18T11:31:34.885 INFO:tasks.rados.rados.0.smithi119.stdout:1463: expect (ObjNum 183 snap 20 seq_num 183) 2024-09-18T11:31:34.885 INFO:tasks.rados.rados.0.smithi119.stdout:1464: read oid 45 snap -1 2024-09-18T11:31:34.885 INFO:tasks.rados.rados.0.smithi119.stdout:1464: expect (ObjNum 254 snap 30 seq_num 254) 2024-09-18T11:31:34.885 INFO:tasks.rados.rados.0.smithi119.stdout:1465: snap_remove snap 30 2024-09-18T11:31:34.935 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:34.934+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:34.989 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:34.989 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:34.998 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:31:35.039 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:35.038+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:35.085 INFO:tasks.rados.rados.0.smithi119.stdout:1465: done (6 left) 2024-09-18T11:31:35.085 INFO:tasks.rados.rados.0.smithi119.stdout:1466: read oid 6 snap 27 2024-09-18T11:31:35.085 INFO:tasks.rados.rados.0.smithi119.stdout:1466: expect deleted 2024-09-18T11:31:35.086 INFO:tasks.rados.rados.0.smithi119.stdout:1467: snap_create 2024-09-18T11:31:35.086 INFO:tasks.rados.rados.0.smithi119.stdout:1459: finishing set_chunk to oid 42 2024-09-18T11:31:35.086 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 178 (ObjNum 251 snap 30 seq_num 251) dirty exists 2024-09-18T11:31:35.086 INFO:tasks.rados.rados.0.smithi119.stdout:1463: expect (ObjNum 183 snap 20 seq_num 183) 2024-09-18T11:31:35.120 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:35.116+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:35.142 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:35.142 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:35.150 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:31:35.219 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:35.218+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:35.248 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:35.248 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:35.257 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:31:35.290 INFO:tasks.rados.rados.0.smithi119.stdout:1461: expect (ObjNum 237 snap 28 seq_num 237) 2024-09-18T11:31:35.320 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:35.318+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:35.397 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:35.397 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:35.405 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:31:35.417 INFO:tasks.rados.rados.0.smithi119.stdout:1460: finishing set_chunk to oid 13 2024-09-18T11:31:35.417 INFO:tasks.rados.rados.0.smithi119.stdout:1462: expect (ObjNum 155 snap 20 seq_num 155) 2024-09-18T11:31:35.421 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:35.419+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:35.464 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:31:35.503 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:35.503 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:35.512 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:31:35.556 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:35.555+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:35.561 INFO:tasks.rados.rados.0.smithi119.stdout:1464: expect (ObjNum 254 snap 30 seq_num 254) 2024-09-18T11:31:35.564 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:35.579 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:35.579 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:35.595 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:35.595 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:35.609 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:35.609 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:35.610 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:35.610 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:35.619 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:31:35.624 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:35.624 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:35.639 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:35.639 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:35.654 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:35.655 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:35.660 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:35.659+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:35.669 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:35.670 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:35.673 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:35.674 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:35.723 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:35.721+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:35.760 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:35.760 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:35.760 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:35.760 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:35.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.761 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:35.761 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:35.761 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 106.600117809, 2024-09-18T11:31:35.761 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055400850000000001, 2024-09-18T11:31:35.761 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:35.761 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:35.761 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:35.761 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:35.761 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:35.761 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 43 2024-09-18T11:31:35.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.761 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:35.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:35.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:35.762 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:35.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:35.762 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:35.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157499+0000", 2024-09-18T11:31:35.763 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:35.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.763 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:35.763 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157504+0000", 2024-09-18T11:31:35.763 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4519999999999996e-06 2024-09-18T11:31:35.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.763 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:35.763 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157534+0000", 2024-09-18T11:31:35.763 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.97e-05 2024-09-18T11:31:35.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.763 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:35.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157539+0000", 2024-09-18T11:31:35.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4360000000000001e-06 2024-09-18T11:31:35.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:35.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159428+0000", 2024-09-18T11:31:35.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018888220000000001 2024-09-18T11:31:35.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:35.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159479+0000", 2024-09-18T11:31:35.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0608000000000001e-05 2024-09-18T11:31:35.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.765 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:35.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.159559+0000", 2024-09-18T11:31:35.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0667000000000005e-05 2024-09-18T11:31:35.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.765 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:35.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162779+0000", 2024-09-18T11:31:35.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032197459999999999 2024-09-18T11:31:35.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.765 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:35.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162950+0000", 2024-09-18T11:31:35.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017059599999999999 2024-09-18T11:31:35.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.766 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:35.766 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163001+0000", 2024-09-18T11:31:35.766 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0968999999999999e-05 2024-09-18T11:31:35.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.766 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:35.766 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163040+0000", 2024-09-18T11:31:35.766 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8843000000000001e-05 2024-09-18T11:31:35.766 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.766 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:35.766 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.767 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.767 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:35.767 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:35.767 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 106.60006665500001, 2024-09-18T11:31:35.767 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:31:35.767 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:35.767 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:35.767 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:35.767 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:35.767 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:35.767 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:31:35.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.768 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:35.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.768 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:35.768 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:35.768 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.768 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:35.768 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:35.768 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.768 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:35.768 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:31:35.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:35.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.769 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.769 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:35.769 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:31:35.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:31:35.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.769 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.769 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:35.769 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:31:35.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:31:35.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.769 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.769 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:35.769 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:31:35.770 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:35.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.770 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:35.770 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:31:35.770 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:31:35.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.770 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:35.770 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:31:35.770 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:31:35.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.770 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:35.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:31:35.771 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:31:35.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:35.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:31:35.771 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:31:35.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:35.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:31:35.771 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:31:35.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:35.772 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:31:35.772 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:31:35.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.772 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:35.772 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:31:35.772 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:31:35.772 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.772 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:35.772 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.772 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:35.772 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:35.773 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 106.599940393, 2024-09-18T11:31:35.773 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055717040000000002, 2024-09-18T11:31:35.773 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:35.773 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:35.773 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:35.773 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:35.773 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:35.773 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 45 2024-09-18T11:31:35.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.773 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:35.773 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.773 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:35.773 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:35.773 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.774 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:35.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:35.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.774 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:35.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157676+0000", 2024-09-18T11:31:35.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:35.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.774 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:35.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157694+0000", 2024-09-18T11:31:35.775 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7422999999999999e-05 2024-09-18T11:31:35.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.775 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:35.775 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157704+0000", 2024-09-18T11:31:35.775 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9000000000000001e-06 2024-09-18T11:31:35.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.775 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:35.775 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157718+0000", 2024-09-18T11:31:35.775 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4788e-05 2024-09-18T11:31:35.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.775 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:35.775 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160937+0000", 2024-09-18T11:31:35.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00321886 2024-09-18T11:31:35.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.776 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:35.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161023+0000", 2024-09-18T11:31:35.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6063999999999995e-05 2024-09-18T11:31:35.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.776 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:35.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161202+0000", 2024-09-18T11:31:35.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017912599999999999 2024-09-18T11:31:35.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:35.777 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163052+0000", 2024-09-18T11:31:35.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001849554 2024-09-18T11:31:35.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:35.777 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163204+0000", 2024-09-18T11:31:35.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015186900000000001 2024-09-18T11:31:35.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:35.777 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163229+0000", 2024-09-18T11:31:35.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:31:35.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.778 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:35.778 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163249+0000", 2024-09-18T11:31:35.778 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9992000000000001e-05 2024-09-18T11:31:35.778 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.778 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:35.778 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.778 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:35.778 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:35.778 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 106.59989127999999, 2024-09-18T11:31:35.779 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:31:35.779 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:35.779 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:35.779 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:35.779 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:35.779 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:35.779 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:31:35.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.779 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:35.779 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.779 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:35.779 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:35.779 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:35.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:35.780 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:35.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:35.780 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:35.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:35.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:31:35.780 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:31:35.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.781 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:35.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:31:35.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:31:35.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.781 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:35.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:31:35.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:31:35.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.781 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:35.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:31:35.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:31:35.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:35.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:31:35.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:31:35.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:35.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:31:35.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:31:35.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:35.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:31:35.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:31:35.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.783 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:35.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:31:35.783 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:31:35.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.783 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:35.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:31:35.783 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:31:35.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.783 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:35.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:31:35.783 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:31:35.783 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.784 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:35.784 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.784 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:35.784 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:35.784 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 106.599848749, 2024-09-18T11:31:35.784 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:31:35.784 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:35.784 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:35.784 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:35.784 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:35.784 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:35.784 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:31:35.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.785 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:35.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.785 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:35.785 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:35.785 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.785 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:35.785 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:35.785 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:35.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:31:35.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:35.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:35.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:31:35.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:31:35.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:35.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:31:35.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:31:35.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:35.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:31:35.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:31:35.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:35.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:31:35.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:31:35.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:35.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:31:35.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:31:35.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.788 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:35.788 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:31:35.788 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:31:35.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.788 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:35.788 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:31:35.788 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:31:35.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.788 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:35.789 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:31:35.789 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:31:35.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.789 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:35.789 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:31:35.789 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:31:35.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.789 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:35.789 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:31:35.789 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:31:35.789 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.789 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:35.790 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.790 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.790 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:35.790 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:35.790 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 106.599793819, 2024-09-18T11:31:35.790 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:31:35.790 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:35.790 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:35.790 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:35.790 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:35.790 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:35.790 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:31:35.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.791 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:35.791 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.791 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:35.791 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:35.791 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.791 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.791 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:35.791 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:35.791 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.792 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:35.792 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:35.792 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:35.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.792 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:35.792 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:31:35.792 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:31:35.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.792 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:35.793 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:31:35.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:31:35.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.793 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:35.793 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:31:35.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:31:35.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.793 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:35.793 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:31:35.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:31:35.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:35.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:31:35.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:31:35.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:35.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:31:35.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:31:35.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:35.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:31:35.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:31:35.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:35.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:31:35.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:31:35.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:35.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:31:35.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:31:35.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:35.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:31:35.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:31:35.796 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.796 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:35.796 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.796 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:35.796 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:35.796 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 106.599783839, 2024-09-18T11:31:35.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:31:35.796 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:35.796 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:35.796 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:35.796 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:35.796 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:35.796 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:31:35.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.797 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:35.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.797 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:35.797 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:35.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.797 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:35.797 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:35.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.797 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:35.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:35.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:35.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:35.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:31:35.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:31:35.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:35.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:31:35.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:31:35.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:35.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:31:35.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:31:35.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:35.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:31:35.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:31:35.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:35.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:31:35.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:31:35.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:35.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:31:35.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:31:35.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:35.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:31:35.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:31:35.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:35.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:31:35.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:31:35.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:35.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:31:35.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:31:35.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:35.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:31:35.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:31:35.801 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.801 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:35.801 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.801 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:35.801 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:35.802 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 106.59977223999999, 2024-09-18T11:31:35.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:31:35.802 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:35.802 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:35.802 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:35.802 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:35.802 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:35.802 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:31:35.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.802 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:35.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:35.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:35.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:35.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:35.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:35.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:35.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:35.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:35.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:31:35.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:31:35.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.804 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:35.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:31:35.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:31:35.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.804 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:35.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:31:35.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:31:35.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:35.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:31:35.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:31:35.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:35.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:31:35.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:31:35.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:35.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:31:35.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:31:35.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:35.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:31:35.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:31:35.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:35.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:31:35.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:31:35.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:35.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:31:35.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:31:35.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:35.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:31:35.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:31:35.807 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.807 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:35.807 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.807 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:35.807 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:35.807 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 106.59971575900001, 2024-09-18T11:31:35.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:31:35.807 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:35.808 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:35.808 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:35.808 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:35.808 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:35.808 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:31:35.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.808 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:35.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:35.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:35.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:35.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:35.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:35.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:31:35.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:35.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:35.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:31:35.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:31:35.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.810 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:35.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:31:35.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:31:35.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.810 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:35.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:31:35.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:31:35.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.810 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:35.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:31:35.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:31:35.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:35.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:31:35.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:31:35.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:35.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:31:35.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:31:35.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:35.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:31:35.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:31:35.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.812 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:35.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:31:35.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:31:35.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.812 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:35.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:31:35.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:31:35.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.812 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:35.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:31:35.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:31:35.813 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.813 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:35.813 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.813 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:35.813 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:35.813 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 106.599701935, 2024-09-18T11:31:35.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:31:35.813 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:35.813 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:35.813 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:35.813 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:35.813 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:35.814 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:31:35.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.814 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:35.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:35.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:35.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:35.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:35.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:35.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:35.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:35.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.815 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:35.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:31:35.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:31:35.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.815 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:35.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:31:35.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:31:35.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.815 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:35.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:31:35.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:31:35.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:35.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:31:35.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:31:35.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:35.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:31:35.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:31:35.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:35.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:31:35.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:31:35.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:35.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:31:35.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:31:35.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:35.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:31:35.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:31:35.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:35.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:31:35.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:31:35.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:35.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:31:35.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:31:35.818 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.818 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:35.818 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.818 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:35.819 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:35.819 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 106.183523028, 2024-09-18T11:31:35.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:31:35.819 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:35.819 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:35.819 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:35.819 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:35.819 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:35.819 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:31:35.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.819 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:35.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:35.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:35.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:35.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:35.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:35.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:35.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:35.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:35.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:31:35.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:31:35.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:35.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:31:35.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:31:35.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:35.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:31:35.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:31:35.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.822 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:35.822 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:31:35.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:31:35.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.822 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:35.822 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:31:35.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:31:35.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.822 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:35.822 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:31:35.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:31:35.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:35.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:31:35.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:31:35.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:35.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:31:35.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:31:35.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:35.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:31:35.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:31:35.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:35.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:31:35.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:31:35.824 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.824 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:35.824 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.824 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:35.824 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:35.824 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 106.183499728, 2024-09-18T11:31:35.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:31:35.825 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:35.825 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:35.825 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:35.825 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:35.825 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:35.825 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:31:35.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.825 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:35.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:35.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:35.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:35.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:35.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:35.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:31:35.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:35.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:35.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:31:35.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:31:35.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:35.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:31:35.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:31:35.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:35.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:31:35.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:31:35.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:35.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:31:35.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:31:35.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:35.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:31:35.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:31:35.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:35.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:31:35.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:31:35.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:35.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:31:35.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:31:35.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:35.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:31:35.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:31:35.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:35.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:31:35.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:31:35.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:35.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:31:35.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:31:35.830 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.830 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:35.830 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.830 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:35.830 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:35.830 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 106.18341411900001, 2024-09-18T11:31:35.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:31:35.831 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:35.831 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:35.831 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:35.831 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:35.831 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:35.831 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:31:35.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.831 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:35.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:35.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:35.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:35.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:35.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:35.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:35.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:35.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:35.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:31:35.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:31:35.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:35.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:31:35.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:31:35.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:35.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:31:35.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:31:35.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:35.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:31:35.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:31:35.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:35.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:31:35.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:31:35.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:35.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:31:35.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:31:35.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:35.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:31:35.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:31:35.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:35.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:31:35.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:31:35.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:35.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:31:35.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:31:35.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:35.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:31:35.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:31:35.836 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.836 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:35.836 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.836 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:35.836 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:35.836 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 106.183403154, 2024-09-18T11:31:35.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:31:35.837 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:35.837 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:35.837 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:35.837 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:35.837 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:35.837 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:31:35.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.837 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:35.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:35.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:35.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:35.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:35.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:35.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:35.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:35.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:35.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:31:35.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:31:35.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:35.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:31:35.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:31:35.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:35.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:31:35.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:31:35.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:35.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:31:35.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:31:35.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:35.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:31:35.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:31:35.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:35.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:31:35.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:31:35.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:35.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:31:35.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:31:35.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:35.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:31:35.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:31:35.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:35.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:31:35.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:31:35.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:35.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:31:35.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:31:35.843 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.843 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:35.843 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.843 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:35.843 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:35.844 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 106.183391986, 2024-09-18T11:31:35.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:31:35.844 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:35.844 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:35.844 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:35.844 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:35.844 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:35.844 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:31:35.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.844 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:35.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:35.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:35.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:35.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:35.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:35.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:35.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:35.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:35.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:31:35.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:31:35.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:35.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:31:35.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:31:35.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:35.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:31:35.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:31:35.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:35.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:31:35.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:31:35.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:35.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:31:35.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:31:35.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:35.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:31:35.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:31:35.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:35.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:31:35.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:31:35.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:35.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:31:35.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:31:35.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:35.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:31:35.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:31:35.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:35.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:31:35.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:31:35.850 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.850 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:35.850 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.850 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:35.850 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:35.851 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 106.18332665699999, 2024-09-18T11:31:35.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:31:35.851 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:35.851 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:35.851 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:35.851 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:35.851 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:35.851 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:31:35.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.851 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:35.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:35.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:35.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:35.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:35.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:35.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:31:35.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:35.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:35.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:31:35.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:31:35.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:35.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:31:35.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:31:35.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:35.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:31:35.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:31:35.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:35.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:31:35.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:31:35.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:35.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:31:35.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:31:35.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:35.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:31:35.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:31:35.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:35.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:31:35.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:31:35.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:35.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:31:35.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:31:35.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:35.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:31:35.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:31:35.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:35.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:31:35.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:31:35.857 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.857 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:35.857 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.857 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:35.857 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:35.857 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 106.173164679, 2024-09-18T11:31:35.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:31:35.857 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:35.858 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:35.858 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:35.858 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:35.858 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:35.858 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:31:35.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.858 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:35.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:35.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:35.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:35.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:35.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:35.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:31:35.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:35.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:35.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:31:35.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:31:35.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:35.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:31:35.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:31:35.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:35.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:31:35.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:31:35.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:35.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:31:35.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:31:35.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:35.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:31:35.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:31:35.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:35.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:31:35.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:31:35.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:35.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:31:35.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:31:35.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:35.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:31:35.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:31:35.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:35.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:31:35.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:31:35.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:35.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:31:35.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:31:35.863 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.864 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:35.864 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.864 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:35.864 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:35.864 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 106.17307220000001, 2024-09-18T11:31:35.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:31:35.864 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:35.864 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:35.864 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:35.865 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:35.865 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:35.865 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:31:35.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.865 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:35.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:35.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:35.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:35.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:35.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:35.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:35.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:35.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:35.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:31:35.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:31:35.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:35.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:31:35.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:31:35.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:35.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:31:35.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:31:35.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:35.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:31:35.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:31:35.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:35.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:31:35.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:31:35.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:35.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:31:35.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:31:35.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:35.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:31:35.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:31:35.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:35.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:31:35.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:31:35.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:35.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:31:35.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:31:35.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:35.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:31:35.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:31:35.871 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.871 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:35.871 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.871 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1788 4.8 e60/22 4:1898b15d:::smithi11939506-25:head v 60'12, mlcod=58'9)", 2024-09-18T11:31:35.872 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:35.872 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 52.840666710999997, 2024-09-18T11:31:35.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058348599999999999, 2024-09-18T11:31:35.872 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:35.872 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:35.872 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:35.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:35.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:35.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:35.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:35.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:35.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916949+0000", 2024-09-18T11:31:35.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:35.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:35.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917602+0000", 2024-09-18T11:31:35.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00065337100000000003 2024-09-18T11:31:35.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:35.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917621+0000", 2024-09-18T11:31:35.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9287e-05 2024-09-18T11:31:35.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:35.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917629+0000", 2024-09-18T11:31:35.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3780000000000003e-06 2024-09-18T11:31:35.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:35.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917677+0000", 2024-09-18T11:31:35.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8068000000000001e-05 2024-09-18T11:31:35.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:35.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917699+0000", 2024-09-18T11:31:35.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2149000000000001e-05 2024-09-18T11:31:35.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:35.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922747+0000", 2024-09-18T11:31:35.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050477609999999996 2024-09-18T11:31:35.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:35.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922785+0000", 2024-09-18T11:31:35.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8726000000000002e-05 2024-09-18T11:31:35.876 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.876 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:35.876 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.877 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:58 4.3 e91/67 4:c70c3162:::smithi11939506-35:18 v 91'101, mlcod=86'95)", 2024-09-18T11:31:35.877 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:35.877 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.742008488, 2024-09-18T11:31:35.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055647500000000003, 2024-09-18T11:31:35.877 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:35.877 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:35.877 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:35.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:35.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:35.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:35.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:35.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:35.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:35.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015608+0000", 2024-09-18T11:31:35.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:35.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:35.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015614+0000", 2024-09-18T11:31:35.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5310000000000001e-06 2024-09-18T11:31:35.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:35.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015622+0000", 2024-09-18T11:31:35.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0809999999999994e-06 2024-09-18T11:31:35.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:35.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015627+0000", 2024-09-18T11:31:35.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8439999999999997e-06 2024-09-18T11:31:35.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:35.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016003+0000", 2024-09-18T11:31:35.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037652999999999997 2024-09-18T11:31:35.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:35.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016024+0000", 2024-09-18T11:31:35.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0780000000000001e-05 2024-09-18T11:31:35.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:35.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021141+0000", 2024-09-18T11:31:35.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051166639999999999 2024-09-18T11:31:35.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:35.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:35.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:35.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021174+0000", 2024-09-18T11:31:35.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2839000000000002e-05 2024-09-18T11:31:35.881 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.881 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:35.881 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.881 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:35.881 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:35.881 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:35.883 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:31:35.884 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:35.881+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:35.982 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:35.982 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:35.982 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:35.983 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:35.984 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:35.982+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:35.990 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:31:36.083 INFO:tasks.rados.rados.0.smithi119.stdout:1459: done (7 left) 2024-09-18T11:31:36.083 INFO:tasks.rados.rados.0.smithi119.stdout:1460: done (6 left) 2024-09-18T11:31:36.083 INFO:tasks.rados.rados.0.smithi119.stdout:1461: done (5 left) 2024-09-18T11:31:36.083 INFO:tasks.rados.rados.0.smithi119.stdout:1462: done (4 left) 2024-09-18T11:31:36.083 INFO:tasks.rados.rados.0.smithi119.stdout:1463: done (3 left) 2024-09-18T11:31:36.084 INFO:tasks.rados.rados.0.smithi119.stdout:1464: done (2 left) 2024-09-18T11:31:36.084 INFO:tasks.rados.rados.0.smithi119.stdout:1466: done (1 left) 2024-09-18T11:31:36.084 INFO:tasks.rados.rados.0.smithi119.stdout:1467: done (0 left) 2024-09-18T11:31:36.084 INFO:tasks.rados.rados.0.smithi119.stdout:1468: delete oid 8 current snap is 36 2024-09-18T11:31:36.085 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:36.083+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:36.085 INFO:tasks.rados.rados.0.smithi119.stdout:1468: done (0 left) 2024-09-18T11:31:36.085 INFO:tasks.rados.rados.0.smithi119.stdout:1469: snap_create 2024-09-18T11:31:36.098 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T11:31:36.132 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:36.133 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:36.133 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:36.133 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:36.133 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:36.141 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:31:36.185 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:36.183+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:36.284 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:36.284 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:36.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:36.285 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:36.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.285 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:888 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:36.285 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:36.285 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.165599309000001, 2024-09-18T11:31:36.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082282120000000004, 2024-09-18T11:31:36.285 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.285 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:36.285 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:36.285 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:36.285 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:36.285 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 888 2024-09-18T11:31:36.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.286 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:36.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:36.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:36.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:36.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116423+0000", 2024-09-18T11:31:36.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1e-06 2024-09-18T11:31:36.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:31:36.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5510000000000001e-06 2024-09-18T11:31:36.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116431+0000", 2024-09-18T11:31:36.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6109999999999998e-06 2024-09-18T11:31:36.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116527+0000", 2024-09-18T11:31:36.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6212999999999995e-05 2024-09-18T11:31:36.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:36.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116532+0000", 2024-09-18T11:31:36.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6140000000000002e-06 2024-09-18T11:31:36.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124314+0000", 2024-09-18T11:31:36.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077818929999999998 2024-09-18T11:31:36.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124411+0000", 2024-09-18T11:31:36.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6992999999999998e-05 2024-09-18T11:31:36.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124649+0000", 2024-09-18T11:31:36.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237729 2024-09-18T11:31:36.291 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.291 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.291 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.291 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:889 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:36.291 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:36.291 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.165574659000001, 2024-09-18T11:31:36.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085383349999999993, 2024-09-18T11:31:36.291 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.291 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:36.291 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:36.292 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:36.292 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:36.292 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 889 2024-09-18T11:31:36.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.292 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:36.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:36.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:36.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:36.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116447+0000", 2024-09-18T11:31:36.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.854e-06 2024-09-18T11:31:36.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116452+0000", 2024-09-18T11:31:36.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0250000000000002e-06 2024-09-18T11:31:36.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116455+0000", 2024-09-18T11:31:36.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8739999999999999e-06 2024-09-18T11:31:36.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116565+0000", 2024-09-18T11:31:36.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110741 2024-09-18T11:31:36.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:36.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116571+0000", 2024-09-18T11:31:36.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4990000000000002e-06 2024-09-18T11:31:36.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124691+0000", 2024-09-18T11:31:36.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081200339999999999 2024-09-18T11:31:36.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124784+0000", 2024-09-18T11:31:36.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2886999999999993e-05 2024-09-18T11:31:36.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124983+0000", 2024-09-18T11:31:36.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019969000000000001 2024-09-18T11:31:36.296 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.296 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.296 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.297 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1126 3.0 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head [write 0~2520064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:31:36.297 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:31:36.297 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 76.300132680999994, 2024-09-18T11:31:36.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086197050000000001, 2024-09-18T11:31:36.297 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.297 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.297 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:36.297 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:36.297 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:36.297 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1126 2024-09-18T11:31:36.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.297 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:31:36.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:31:36.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981886+0000", 2024-09-18T11:31:36.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:36.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983017+0000", 2024-09-18T11:31:36.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00113131 2024-09-18T11:31:36.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983037+0000", 2024-09-18T11:31:36.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0231e-05 2024-09-18T11:31:36.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983044+0000", 2024-09-18T11:31:36.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9539999999999998e-06 2024-09-18T11:31:36.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983087+0000", 2024-09-18T11:31:36.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3040000000000001e-05 2024-09-18T11:31:36.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983193+0000", 2024-09-18T11:31:36.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105682 2024-09-18T11:31:36.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:36.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983287+0000", 2024-09-18T11:31:36.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4066000000000004e-05 2024-09-18T11:31:36.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:36.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.986918+0000", 2024-09-18T11:31:36.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036311970000000001 2024-09-18T11:31:36.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:36.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990442+0000", 2024-09-18T11:31:36.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035243119999999999 2024-09-18T11:31:36.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990476+0000", 2024-09-18T11:31:36.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4028e-05 2024-09-18T11:31:36.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990507+0000", 2024-09-18T11:31:36.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0284999999999999e-05 2024-09-18T11:31:36.303 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.303 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.303 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.303 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:262 4.9 e53/22 4:99feffe9:::smithi11939506-45:13 v 53'17, mlcod=51'14)", 2024-09-18T11:31:36.303 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:36.303 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 62.283028866000002, 2024-09-18T11:31:36.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013878679, 2024-09-18T11:31:36.303 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.303 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.303 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:36.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:36.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:36.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:36.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998997+0000", 2024-09-18T11:31:36.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1099999999999999e-06 2024-09-18T11:31:36.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999008+0000", 2024-09-18T11:31:36.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1321000000000001e-05 2024-09-18T11:31:36.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999015+0000", 2024-09-18T11:31:36.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3000000000000004e-06 2024-09-18T11:31:36.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999055+0000", 2024-09-18T11:31:36.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0058000000000002e-05 2024-09-18T11:31:36.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999070+0000", 2024-09-18T11:31:36.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4328000000000001e-05 2024-09-18T11:31:36.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012465+0000", 2024-09-18T11:31:36.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013395638 2024-09-18T11:31:36.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012870+0000", 2024-09-18T11:31:36.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040423400000000001 2024-09-18T11:31:36.308 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.308 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.308 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.308 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:205 4.b e53/22 4:dba5bd39:::smithi11939506-17:13 v 53'77, mlcod=51'74)", 2024-09-18T11:31:36.308 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:36.308 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 62.282979605000001, 2024-09-18T11:31:36.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013359586, 2024-09-18T11:31:36.308 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.308 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.309 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:36.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:36.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:36.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:36.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999048+0000", 2024-09-18T11:31:36.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3550000000000005e-06 2024-09-18T11:31:36.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999056+0000", 2024-09-18T11:31:36.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3259999999999996e-06 2024-09-18T11:31:36.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999063+0000", 2024-09-18T11:31:36.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6409999999999996e-06 2024-09-18T11:31:36.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999098+0000", 2024-09-18T11:31:36.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5203999999999997e-05 2024-09-18T11:31:36.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999115+0000", 2024-09-18T11:31:36.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6478999999999999e-05 2024-09-18T11:31:36.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012362+0000", 2024-09-18T11:31:36.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01324768 2024-09-18T11:31:36.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012400+0000", 2024-09-18T11:31:36.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.731e-05 2024-09-18T11:31:36.312 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.312 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.313 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.313 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:88 4.a e53/22 4:588debc2:::smithi11939506-18:13 v 53'45, mlcod=51'42)", 2024-09-18T11:31:36.313 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:36.313 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 62.280197182000002, 2024-09-18T11:31:36.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011307667, 2024-09-18T11:31:36.313 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.313 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.313 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:36.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:36.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001822+0000", 2024-09-18T11:31:36.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:36.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001827+0000", 2024-09-18T11:31:36.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.378e-06 2024-09-18T11:31:36.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001839+0000", 2024-09-18T11:31:36.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1225e-05 2024-09-18T11:31:36.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001844+0000", 2024-09-18T11:31:36.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2299999999999999e-06 2024-09-18T11:31:36.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001891+0000", 2024-09-18T11:31:36.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6961e-05 2024-09-18T11:31:36.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001906+0000", 2024-09-18T11:31:36.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5158e-05 2024-09-18T11:31:36.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013103+0000", 2024-09-18T11:31:36.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011197314 2024-09-18T11:31:36.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013130+0000", 2024-09-18T11:31:36.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6900999999999999e-05 2024-09-18T11:31:36.317 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.317 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.317 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.317 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:89 4.a e53/22 4:552523b6:::smithi11939506-38:13 v 53'47, mlcod=51'42)", 2024-09-18T11:31:36.317 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:36.317 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 62.279637962000002, 2024-09-18T11:31:36.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011327614, 2024-09-18T11:31:36.317 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.317 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.317 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:36.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:36.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002381+0000", 2024-09-18T11:31:36.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:36.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002397+0000", 2024-09-18T11:31:36.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5688e-05 2024-09-18T11:31:36.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002406+0000", 2024-09-18T11:31:36.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0229999999999998e-06 2024-09-18T11:31:36.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002415+0000", 2024-09-18T11:31:36.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8959999999999992e-06 2024-09-18T11:31:36.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012386+0000", 2024-09-18T11:31:36.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0099711450000000007 2024-09-18T11:31:36.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012405+0000", 2024-09-18T11:31:36.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9389e-05 2024-09-18T11:31:36.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013676+0000", 2024-09-18T11:31:36.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012708649999999999 2024-09-18T11:31:36.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013709+0000", 2024-09-18T11:31:36.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3207000000000002e-05 2024-09-18T11:31:36.321 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.321 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.322 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.322 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1644 4.7 4:e73ba68f:::smithi11939506-1:head [sparse-read 0~2695582] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:36.322 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:36.322 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.948115663999999, 2024-09-18T11:31:36.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084764039999999999, 2024-09-18T11:31:36.322 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.322 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:36.322 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:36.322 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:36.322 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:36.322 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1644 2024-09-18T11:31:36.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.323 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:36.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:36.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:36.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:36.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333906+0000", 2024-09-18T11:31:36.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.243e-06 2024-09-18T11:31:36.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333911+0000", 2024-09-18T11:31:36.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2959999999999998e-06 2024-09-18T11:31:36.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333915+0000", 2024-09-18T11:31:36.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6720000000000002e-06 2024-09-18T11:31:36.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.325 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341879+0000", 2024-09-18T11:31:36.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079638699999999996 2024-09-18T11:31:36.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341978+0000", 2024-09-18T11:31:36.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9562000000000007e-05 2024-09-18T11:31:36.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342380+0000", 2024-09-18T11:31:36.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000402113 2024-09-18T11:31:36.326 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.326 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.326 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.326 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1645 4.7 4:e73ba68f:::smithi11939506-1:head [read 0~3983226,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:36.327 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:36.327 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.948100770000003, 2024-09-18T11:31:36.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010539468999999999, 2024-09-18T11:31:36.327 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.327 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:36.327 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:36.327 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:36.327 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:36.327 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1645 2024-09-18T11:31:36.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.327 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:36.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:36.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:36.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:36.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333921+0000", 2024-09-18T11:31:36.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8909999999999999e-06 2024-09-18T11:31:36.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333927+0000", 2024-09-18T11:31:36.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4229999999999996e-06 2024-09-18T11:31:36.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333932+0000", 2024-09-18T11:31:36.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8280000000000002e-06 2024-09-18T11:31:36.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342418+0000", 2024-09-18T11:31:36.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084862219999999999 2024-09-18T11:31:36.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342515+0000", 2024-09-18T11:31:36.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7287999999999994e-05 2024-09-18T11:31:36.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.344458+0000", 2024-09-18T11:31:36.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019431660000000001 2024-09-18T11:31:36.331 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.331 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.331 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.331 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1669 4.4 4:2d9c3f7f:::smithi11939506-5:head [read 0~3764735,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-18T11:31:36.331 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:31:36.331 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 57.459151872, 2024-09-18T11:31:36.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0083694990000000007, 2024-09-18T11:31:36.331 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.331 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:36.332 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:36.332 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:36.332 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:36.332 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1669 2024-09-18T11:31:36.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.332 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:31:36.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:31:36.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822867+0000", 2024-09-18T11:31:36.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:36.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822871+0000", 2024-09-18T11:31:36.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:31:36.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822877+0000", 2024-09-18T11:31:36.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8000000000000004e-06 2024-09-18T11:31:36.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822880+0000", 2024-09-18T11:31:36.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5669999999999999e-06 2024-09-18T11:31:36.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822974+0000", 2024-09-18T11:31:36.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4048999999999993e-05 2024-09-18T11:31:36.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:36.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822980+0000", 2024-09-18T11:31:36.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2789999999999998e-06 2024-09-18T11:31:36.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.829160+0000", 2024-09-18T11:31:36.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0061800179999999998 2024-09-18T11:31:36.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.829248+0000", 2024-09-18T11:31:36.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7707000000000006e-05 2024-09-18T11:31:36.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.831237+0000", 2024-09-18T11:31:36.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001989827 2024-09-18T11:31:36.336 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.336 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.336 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.336 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:100 4.a e58/22 4:5975a5ec:::smithi11939506-36:18 v 58'55, mlcod=56'50)", 2024-09-18T11:31:36.336 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:36.336 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 57.008870633999997, 2024-09-18T11:31:36.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0088255, 2024-09-18T11:31:36.336 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.337 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.337 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:36.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:36.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:36.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:36.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273155+0000", 2024-09-18T11:31:36.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.212e-06 2024-09-18T11:31:36.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273164+0000", 2024-09-18T11:31:36.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7609999999999996e-06 2024-09-18T11:31:36.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273168+0000", 2024-09-18T11:31:36.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2819999999999998e-06 2024-09-18T11:31:36.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274162+0000", 2024-09-18T11:31:36.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099398600000000009 2024-09-18T11:31:36.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274180+0000", 2024-09-18T11:31:36.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8236e-05 2024-09-18T11:31:36.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281943+0000", 2024-09-18T11:31:36.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007763308 2024-09-18T11:31:36.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281975+0000", 2024-09-18T11:31:36.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1202999999999999e-05 2024-09-18T11:31:36.340 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.340 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.341 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.341 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1954 4.7 4:ef60946b:::smithi11939506-8:head [read 0~2890771,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:36.341 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:36.341 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 44.942119593999998, 2024-09-18T11:31:36.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091142180000000003, 2024-09-18T11:31:36.341 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.341 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:36.341 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:36.341 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:36.341 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:36.341 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1954 2024-09-18T11:31:36.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.342 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:36.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:36.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:36.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:36.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339903+0000", 2024-09-18T11:31:36.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7349999999999998e-06 2024-09-18T11:31:36.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339909+0000", 2024-09-18T11:31:36.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-09-18T11:31:36.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339913+0000", 2024-09-18T11:31:36.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7670000000000001e-06 2024-09-18T11:31:36.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339970+0000", 2024-09-18T11:31:36.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6988999999999999e-05 2024-09-18T11:31:36.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:36.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339975+0000", 2024-09-18T11:31:36.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7929999999999997e-06 2024-09-18T11:31:36.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346917+0000", 2024-09-18T11:31:36.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069426640000000003 2024-09-18T11:31:36.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.347026+0000", 2024-09-18T11:31:36.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108241 2024-09-18T11:31:36.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.349014+0000", 2024-09-18T11:31:36.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001988723 2024-09-18T11:31:36.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.346 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.346 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1956 4.6 4:67c1765a:::smithi11939506-30:head [write 705927~558502] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:36.346 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:36.346 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 44.930010654, 2024-09-18T11:31:36.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011813239, 2024-09-18T11:31:36.346 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.347 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:36.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:36.347 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:36.347 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1956 2024-09-18T11:31:36.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.347 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:36.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:36.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352007+0000", 2024-09-18T11:31:36.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:36.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352229+0000", 2024-09-18T11:31:36.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222024 2024-09-18T11:31:36.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352251+0000", 2024-09-18T11:31:36.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2512000000000001e-05 2024-09-18T11:31:36.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352260+0000", 2024-09-18T11:31:36.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7339999999999997e-06 2024-09-18T11:31:36.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352307+0000", 2024-09-18T11:31:36.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7061999999999998e-05 2024-09-18T11:31:36.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352408+0000", 2024-09-18T11:31:36.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101216 2024-09-18T11:31:36.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:36.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352550+0000", 2024-09-18T11:31:36.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014136199999999999 2024-09-18T11:31:36.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:36.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357684+0000", 2024-09-18T11:31:36.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051342499999999999 2024-09-18T11:31:36.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:36.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363736+0000", 2024-09-18T11:31:36.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006052577 2024-09-18T11:31:36.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363778+0000", 2024-09-18T11:31:36.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1989000000000001e-05 2024-09-18T11:31:36.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363822+0000", 2024-09-18T11:31:36.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3894999999999999e-05 2024-09-18T11:31:36.352 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.352 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.352 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.352 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2278 4.f 4:fc192b81:::smithi11939506-31:head [stat,delete] snapc 26=[26,25,24,1e,1c,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T11:31:36.352 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:36.353 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.898973441999999, 2024-09-18T11:31:36.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01018694, 2024-09-18T11:31:36.353 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.353 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.353 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:36.353 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:36.353 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:36.353 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2278 2024-09-18T11:31:36.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.353 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:36.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:36.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383044+0000", 2024-09-18T11:31:36.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:36.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383054+0000", 2024-09-18T11:31:36.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5969999999999998e-06 2024-09-18T11:31:36.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383071+0000", 2024-09-18T11:31:36.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7353000000000001e-05 2024-09-18T11:31:36.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383081+0000", 2024-09-18T11:31:36.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9170000000000001e-06 2024-09-18T11:31:36.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383150+0000", 2024-09-18T11:31:36.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8503999999999993e-05 2024-09-18T11:31:36.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:36.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383158+0000", 2024-09-18T11:31:36.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5089999999999993e-06 2024-09-18T11:31:36.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385396+0000", 2024-09-18T11:31:36.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022381850000000002 2024-09-18T11:31:36.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385494+0000", 2024-09-18T11:31:36.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7751000000000007e-05 2024-09-18T11:31:36.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:36.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385697+0000", 2024-09-18T11:31:36.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020247400000000001 2024-09-18T11:31:36.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:36.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.390505+0000", 2024-09-18T11:31:36.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048084039999999996 2024-09-18T11:31:36.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:36.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393148+0000", 2024-09-18T11:31:36.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026432399999999998 2024-09-18T11:31:36.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393194+0000", 2024-09-18T11:31:36.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5259999999999997e-05 2024-09-18T11:31:36.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393233+0000", 2024-09-18T11:31:36.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9659000000000001e-05 2024-09-18T11:31:36.359 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.359 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.359 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.359 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2342 4.f 4:f3c51b3f:::smithi11939506-21:head [stat,delete] snapc 26=[26,25,24,1e,1c,19,18] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:36.360 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:36.360 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 34.895217430000002, 2024-09-18T11:31:36.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011167565000000001, 2024-09-18T11:31:36.360 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.360 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.360 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:36.360 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:36.360 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:36.360 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2342 2024-09-18T11:31:36.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.360 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:36.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:36.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:31:36.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:36.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:31:36.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1439999999999997e-06 2024-09-18T11:31:36.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386820+0000", 2024-09-18T11:31:36.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2651e-05 2024-09-18T11:31:36.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386825+0000", 2024-09-18T11:31:36.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3150000000000002e-06 2024-09-18T11:31:36.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386868+0000", 2024-09-18T11:31:36.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3832999999999999e-05 2024-09-18T11:31:36.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:36.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:31:36.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0409999999999998e-06 2024-09-18T11:31:36.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390445+0000", 2024-09-18T11:31:36.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035689459999999999 2024-09-18T11:31:36.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390539+0000", 2024-09-18T11:31:36.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3791999999999999e-05 2024-09-18T11:31:36.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:36.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390748+0000", 2024-09-18T11:31:36.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000208863 2024-09-18T11:31:36.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:36.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397659+0000", 2024-09-18T11:31:36.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069110710000000004 2024-09-18T11:31:36.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:36.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397873+0000", 2024-09-18T11:31:36.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021343999999999999 2024-09-18T11:31:36.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397923+0000", 2024-09-18T11:31:36.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0300000000000003e-05 2024-09-18T11:31:36.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397970+0000", 2024-09-18T11:31:36.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6968999999999997e-05 2024-09-18T11:31:36.367 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.367 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.367 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.367 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2429 4.b 4:d8d14c58:::smithi11939506-49:head [write 796698~561170,stat] snapc 2b=[2b,2a,26,25,24,1e,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T11:31:36.367 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:36.367 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.266848255999999, 2024-09-18T11:31:36.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097188989999999996, 2024-09-18T11:31:36.367 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.367 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.368 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:36.368 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:36.368 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:36.368 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2429 2024-09-18T11:31:36.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.368 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:36.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:36.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015170+0000", 2024-09-18T11:31:36.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:36.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015435+0000", 2024-09-18T11:31:36.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026533299999999998 2024-09-18T11:31:36.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015455+0000", 2024-09-18T11:31:36.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9623000000000002e-05 2024-09-18T11:31:36.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015461+0000", 2024-09-18T11:31:36.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1859999999999997e-06 2024-09-18T11:31:36.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015535+0000", 2024-09-18T11:31:36.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4555999999999996e-05 2024-09-18T11:31:36.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:36.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015543+0000", 2024-09-18T11:31:36.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9549999999999992e-06 2024-09-18T11:31:36.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017106+0000", 2024-09-18T11:31:36.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015630290000000001 2024-09-18T11:31:36.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017183+0000", 2024-09-18T11:31:36.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6953999999999995e-05 2024-09-18T11:31:36.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:36.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017301+0000", 2024-09-18T11:31:36.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011734 2024-09-18T11:31:36.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:36.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.020225+0000", 2024-09-18T11:31:36.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029244090000000002 2024-09-18T11:31:36.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:36.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024801+0000", 2024-09-18T11:31:36.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045764129999999997 2024-09-18T11:31:36.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024840+0000", 2024-09-18T11:31:36.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8325999999999999e-05 2024-09-18T11:31:36.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024890+0000", 2024-09-18T11:31:36.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0596999999999997e-05 2024-09-18T11:31:36.374 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.374 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.374 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.374 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2769 4.4 4:2cb3e6a5:::smithi11939506-3:head [write 484101~692450] snapc 30=[30,2f,2b,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:31:36.374 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:18.743766+0000", 2024-09-18T11:31:36.374 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.538254063, 2024-09-18T11:31:36.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086202359999999999, 2024-09-18T11:31:36.374 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.374 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.374 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:36.375 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:36.375 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:36.375 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2769 2024-09-18T11:31:36.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.375 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.743766+0000", 2024-09-18T11:31:36.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.743766+0000", 2024-09-18T11:31:36.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.743764+0000", 2024-09-18T11:31:36.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:36.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744376+0000", 2024-09-18T11:31:36.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00061202299999999995 2024-09-18T11:31:36.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744393+0000", 2024-09-18T11:31:36.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6679000000000001e-05 2024-09-18T11:31:36.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744399+0000", 2024-09-18T11:31:36.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4840000000000001e-06 2024-09-18T11:31:36.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744452+0000", 2024-09-18T11:31:36.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2321999999999998e-05 2024-09-18T11:31:36.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744552+0000", 2024-09-18T11:31:36.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100536 2024-09-18T11:31:36.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:36.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744738+0000", 2024-09-18T11:31:36.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000185434 2024-09-18T11:31:36.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:36.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.749297+0000", 2024-09-18T11:31:36.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004559802 2024-09-18T11:31:36.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:36.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.752260+0000", 2024-09-18T11:31:36.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029623000000000002 2024-09-18T11:31:36.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.752338+0000", 2024-09-18T11:31:36.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8330000000000004e-05 2024-09-18T11:31:36.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.752386+0000", 2024-09-18T11:31:36.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:31:36.380 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.380 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.380 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.380 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2917 4.4 4:2c93196f:::smithi11939506-28:head [sparse-read 0~2075838] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:36.381 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.008117+0000", 2024-09-18T11:31:36.381 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.273902528000001, 2024-09-18T11:31:36.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084318729999999995, 2024-09-18T11:31:36.381 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.381 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:36.381 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:36.381 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:36.381 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:36.381 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2917 2024-09-18T11:31:36.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.381 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.008117+0000", 2024-09-18T11:31:36.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.008117+0000", 2024-09-18T11:31:36.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.008117+0000", 2024-09-18T11:31:36.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:36.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.008120+0000", 2024-09-18T11:31:36.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9100000000000001e-06 2024-09-18T11:31:36.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.008126+0000", 2024-09-18T11:31:36.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9699999999999996e-06 2024-09-18T11:31:36.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.008129+0000", 2024-09-18T11:31:36.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5190000000000001e-06 2024-09-18T11:31:36.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.015988+0000", 2024-09-18T11:31:36.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078590699999999993 2024-09-18T11:31:36.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.016095+0000", 2024-09-18T11:31:36.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107357 2024-09-18T11:31:36.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.016549+0000", 2024-09-18T11:31:36.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00045354999999999999 2024-09-18T11:31:36.386 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.386 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.386 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.386 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2937 4.7 4:e301aefd:::smithi11939506-19:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:36.386 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:31:36.386 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.247179292, 2024-09-18T11:31:36.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019287667000000001, 2024-09-18T11:31:36.386 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.387 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:36.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:36.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:36.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:36.387 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2937 2024-09-18T11:31:36.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.387 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:31:36.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:31:36.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:31:36.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:31:36.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034842+0000", 2024-09-18T11:31:36.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.372e-06 2024-09-18T11:31:36.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034846+0000", 2024-09-18T11:31:36.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3919999999999996e-06 2024-09-18T11:31:36.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034848+0000", 2024-09-18T11:31:36.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3539999999999998e-06 2024-09-18T11:31:36.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034893+0000", 2024-09-18T11:31:36.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4861000000000003e-05 2024-09-18T11:31:36.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034965+0000", 2024-09-18T11:31:36.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1860999999999995e-05 2024-09-18T11:31:36.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.054128+0000", 2024-09-18T11:31:36.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019163039 2024-09-18T11:31:36.391 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.391 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.391 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.391 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3020 4.2 4:425afe9c:::smithi11939506-29:35 [set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e99)", 2024-09-18T11:31:36.391 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:27.073196+0000", 2024-09-18T11:31:36.391 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.2088237260000003, 2024-09-18T11:31:36.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085408519999999998, 2024-09-18T11:31:36.391 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.392 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.392 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:36.392 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:36.392 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:36.392 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3020 2024-09-18T11:31:36.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.392 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073196+0000", 2024-09-18T11:31:36.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073196+0000", 2024-09-18T11:31:36.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073194+0000", 2024-09-18T11:31:36.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:36.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073207+0000", 2024-09-18T11:31:36.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3244000000000001e-05 2024-09-18T11:31:36.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073230+0000", 2024-09-18T11:31:36.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2878999999999999e-05 2024-09-18T11:31:36.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073239+0000", 2024-09-18T11:31:36.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5589999999999994e-06 2024-09-18T11:31:36.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073305+0000", 2024-09-18T11:31:36.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5438000000000006e-05 2024-09-18T11:31:36.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073414+0000", 2024-09-18T11:31:36.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109473 2024-09-18T11:31:36.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:36.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.079319+0000", 2024-09-18T11:31:36.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059050320000000002 2024-09-18T11:31:36.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:36.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.080078+0000", 2024-09-18T11:31:36.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00075877799999999999 2024-09-18T11:31:36.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:36.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.081605+0000", 2024-09-18T11:31:36.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015268549999999999 2024-09-18T11:31:36.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.081681+0000", 2024-09-18T11:31:36.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6396999999999994e-05 2024-09-18T11:31:36.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.081737+0000", 2024-09-18T11:31:36.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5658999999999997e-05 2024-09-18T11:31:36.397 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.397 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.398 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.398 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.398 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.398 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:36.399 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:36.398+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:36.400 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:31:36.499 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:36.499 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:36.499 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:36.500 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:36.500 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:36.498+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:36.508 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:31:36.600 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:36.598+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:36.604 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:36.604 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:36.605 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:36.605 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:36.605 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:36.613 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:31:36.643 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:36.642+0000 7fcb662ad640 -1 osd.7 107 handle_osd_map injecting map crc failure 2024-09-18T11:31:36.702 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:36.701+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:36.752 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:36.752 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:36.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:36.752 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:36.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.752 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1046 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'24, mlcod=30'21)", 2024-09-18T11:31:36.752 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:31:36.752 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 80.576018063999996, 2024-09-18T11:31:36.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042519100000000002, 2024-09-18T11:31:36.752 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.752 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.753 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:31:36.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:31:36.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173782+0000", 2024-09-18T11:31:36.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:36.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174207+0000", 2024-09-18T11:31:36.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00042479799999999999 2024-09-18T11:31:36.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174224+0000", 2024-09-18T11:31:36.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7513999999999999e-05 2024-09-18T11:31:36.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174232+0000", 2024-09-18T11:31:36.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0840000000000003e-06 2024-09-18T11:31:36.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174297+0000", 2024-09-18T11:31:36.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5215000000000005e-05 2024-09-18T11:31:36.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174336+0000", 2024-09-18T11:31:36.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9403000000000002e-05 2024-09-18T11:31:36.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.177990+0000", 2024-09-18T11:31:36.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036533709999999999 2024-09-18T11:31:36.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.178035+0000", 2024-09-18T11:31:36.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5495999999999999e-05 2024-09-18T11:31:36.756 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.757 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.757 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.757 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:31:36.757 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:31:36.757 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 76.766427196999999, 2024-09-18T11:31:36.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:31:36.757 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.757 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.757 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:31:36.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:31:36.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:31:36.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:36.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:31:36.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:31:36.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:31:36.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:31:36.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:31:36.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:31:36.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:31:36.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:31:36.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:31:36.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:31:36.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:31:36.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:31:36.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:31:36.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:31:36.763 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.763 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.763 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.763 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:31:36.763 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:36.763 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 67.531596243999999, 2024-09-18T11:31:36.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:31:36.764 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.764 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.764 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:36.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:36.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:36.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:31:36.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:31:36.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:31:36.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:31:36.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:31:36.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:31:36.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:31:36.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:31:36.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:31:36.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:31:36.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:31:36.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:31:36.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:31:36.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:31:36.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:31:36.767 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.767 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.768 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.768 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:31:36.768 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:31:36.768 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 62.357689958999998, 2024-09-18T11:31:36.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:31:36.768 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.768 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.768 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:31:36.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:31:36.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:31:36.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:36.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:31:36.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:31:36.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:31:36.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:31:36.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:31:36.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:31:36.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:31:36.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:31:36.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:31:36.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:31:36.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:31:36.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:31:36.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:31:36.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:31:36.772 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.772 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.772 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.772 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1931 3.0 e65/21 3:3c05e498:::866d65857c51f67d56ecbcba33930954810c64ced2a213ca9705911438b0dfb7:head v 65'72, mlcod=58'70)", 2024-09-18T11:31:36.772 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:31:36.773 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 49.381679941000002, 2024-09-18T11:31:36.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040630859999999996, 2024-09-18T11:31:36.773 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.773 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.773 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:31:36.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368121+0000", 2024-09-18T11:31:36.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368120+0000", 2024-09-18T11:31:36.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:36.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368427+0000", 2024-09-18T11:31:36.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030742600000000002 2024-09-18T11:31:36.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368446+0000", 2024-09-18T11:31:36.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8046e-05 2024-09-18T11:31:36.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368453+0000", 2024-09-18T11:31:36.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1380000000000001e-06 2024-09-18T11:31:36.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368519+0000", 2024-09-18T11:31:36.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6470000000000006e-05 2024-09-18T11:31:36.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.368550+0000", 2024-09-18T11:31:36.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0707999999999998e-05 2024-09-18T11:31:36.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.372151+0000", 2024-09-18T11:31:36.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036015470000000001 2024-09-18T11:31:36.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:47.372184+0000", 2024-09-18T11:31:36.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2994999999999998e-05 2024-09-18T11:31:36.777 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.777 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.777 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.777 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:304 4.6 e69/67 4:67c1765a:::smithi11939506-30:17 v 69'35, mlcod=58'31)", 2024-09-18T11:31:36.777 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:36.777 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.403848865000001, 2024-09-18T11:31:36.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006377611, 2024-09-18T11:31:36.777 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.777 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.777 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:36.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:36.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:36.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:36.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345961+0000", 2024-09-18T11:31:36.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2050000000000008e-06 2024-09-18T11:31:36.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345969+0000", 2024-09-18T11:31:36.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0339999999999997e-06 2024-09-18T11:31:36.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345976+0000", 2024-09-18T11:31:36.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-18T11:31:36.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346026+0000", 2024-09-18T11:31:36.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9286999999999998e-05 2024-09-18T11:31:36.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346050+0000", 2024-09-18T11:31:36.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4397e-05 2024-09-18T11:31:36.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352293+0000", 2024-09-18T11:31:36.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062427170000000001 2024-09-18T11:31:36.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352330+0000", 2024-09-18T11:31:36.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7348999999999998e-05 2024-09-18T11:31:36.781 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.781 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.782 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.782 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:347 4.d e69/22 4:b44229a1:::smithi11939506-42:19 v 69'99, mlcod=65'93)", 2024-09-18T11:31:36.782 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:31:36.782 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.398115670000003, 2024-09-18T11:31:36.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056998760000000004, 2024-09-18T11:31:36.782 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.782 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.782 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:31:36.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:31:36.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351684+0000", 2024-09-18T11:31:36.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:36.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351693+0000", 2024-09-18T11:31:36.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-18T11:31:36.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351704+0000", 2024-09-18T11:31:36.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1077000000000001e-05 2024-09-18T11:31:36.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351709+0000", 2024-09-18T11:31:36.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5419999999999999e-06 2024-09-18T11:31:36.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351956+0000", 2024-09-18T11:31:36.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000247252 2024-09-18T11:31:36.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351975+0000", 2024-09-18T11:31:36.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9201000000000001e-05 2024-09-18T11:31:36.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357334+0000", 2024-09-18T11:31:36.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053588749999999999 2024-09-18T11:31:36.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357385+0000", 2024-09-18T11:31:36.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1406999999999998e-05 2024-09-18T11:31:36.786 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.786 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.786 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.786 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1956 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'37, mlcod=58'31)", 2024-09-18T11:31:36.786 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:36.786 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.397170756999998, 2024-09-18T11:31:36.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010958842, 2024-09-18T11:31:36.787 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.787 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.787 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:36.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:36.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:36.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:36.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352996+0000", 2024-09-18T11:31:36.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036622 2024-09-18T11:31:36.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353011+0000", 2024-09-18T11:31:36.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4849e-05 2024-09-18T11:31:36.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353017+0000", 2024-09-18T11:31:36.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:31:36.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353068+0000", 2024-09-18T11:31:36.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0695000000000002e-05 2024-09-18T11:31:36.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353092+0000", 2024-09-18T11:31:36.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4260000000000002e-05 2024-09-18T11:31:36.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363551+0000", 2024-09-18T11:31:36.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010459154999999999 2024-09-18T11:31:36.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363589+0000", 2024-09-18T11:31:36.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8161000000000003e-05 2024-09-18T11:31:36.790 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.790 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.791 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.791 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1957 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'38, mlcod=65'33)", 2024-09-18T11:31:36.791 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:36.791 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.390754889999997, 2024-09-18T11:31:36.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050499400000000002, 2024-09-18T11:31:36.791 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.791 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.791 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:36.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:36.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.792 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.792 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:36.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:36.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359214+0000", 2024-09-18T11:31:36.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000168047 2024-09-18T11:31:36.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359227+0000", 2024-09-18T11:31:36.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3124e-05 2024-09-18T11:31:36.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359235+0000", 2024-09-18T11:31:36.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0760000000000003e-06 2024-09-18T11:31:36.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359287+0000", 2024-09-18T11:31:36.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.151e-05 2024-09-18T11:31:36.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359310+0000", 2024-09-18T11:31:36.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-09-18T11:31:36.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364065+0000", 2024-09-18T11:31:36.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047549439999999997 2024-09-18T11:31:36.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364096+0000", 2024-09-18T11:31:36.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1423000000000001e-05 2024-09-18T11:31:36.795 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.795 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.795 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.796 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2300 4.4 e77/22 4:2c93196f:::smithi11939506-28:head v 77'94, mlcod=77'90)", 2024-09-18T11:31:36.796 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:31:36.796 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.761692672000002, 2024-09-18T11:31:36.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050425380000000001, 2024-09-18T11:31:36.796 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.796 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.796 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:31:36.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:31:36.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988108+0000", 2024-09-18T11:31:36.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:36.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988388+0000", 2024-09-18T11:31:36.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028087700000000001 2024-09-18T11:31:36.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988405+0000", 2024-09-18T11:31:36.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6565000000000001e-05 2024-09-18T11:31:36.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.798 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.798 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988412+0000", 2024-09-18T11:31:36.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9709999999999998e-06 2024-09-18T11:31:36.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988464+0000", 2024-09-18T11:31:36.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1949e-05 2024-09-18T11:31:36.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988490+0000", 2024-09-18T11:31:36.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5780999999999999e-05 2024-09-18T11:31:36.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993112+0000", 2024-09-18T11:31:36.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046228399999999996 2024-09-18T11:31:36.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993151+0000", 2024-09-18T11:31:36.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8587999999999997e-05 2024-09-18T11:31:36.800 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.800 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.800 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.800 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.13:63 3.0 e78/21 3:24ebb740:::smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 78'80, mlcod=73'78)", 2024-09-18T11:31:36.800 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:31:36.800 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.344703256000003, 2024-09-18T11:31:36.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052262230000000003, 2024-09-18T11:31:36.801 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.801 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.801 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:31:36.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:31:36.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405097+0000", 2024-09-18T11:31:36.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:36.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405103+0000", 2024-09-18T11:31:36.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8780000000000003e-06 2024-09-18T11:31:36.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405117+0000", 2024-09-18T11:31:36.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3441000000000001e-05 2024-09-18T11:31:36.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405121+0000", 2024-09-18T11:31:36.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7099999999999998e-06 2024-09-18T11:31:36.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405158+0000", 2024-09-18T11:31:36.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6477999999999999e-05 2024-09-18T11:31:36.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405173+0000", 2024-09-18T11:31:36.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4796e-05 2024-09-18T11:31:36.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410285+0000", 2024-09-18T11:31:36.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051118919999999998 2024-09-18T11:31:36.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410324+0000", 2024-09-18T11:31:36.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9616999999999997e-05 2024-09-18T11:31:36.804 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.804 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.805 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.805 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:451 4.6 e78/67 4:6f86ffcc:::smithi11939506-15:26 v 78'58, mlcod=77'53)", 2024-09-18T11:31:36.805 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.414259+0000", 2024-09-18T11:31:36.805 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.335542672999999, 2024-09-18T11:31:36.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041821610000000002, 2024-09-18T11:31:36.805 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.805 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.805 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.805 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414259+0000", 2024-09-18T11:31:36.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414259+0000", 2024-09-18T11:31:36.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414258+0000", 2024-09-18T11:31:36.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:36.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414264+0000", 2024-09-18T11:31:36.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1850000000000001e-06 2024-09-18T11:31:36.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414273+0000", 2024-09-18T11:31:36.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7290000000000006e-06 2024-09-18T11:31:36.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414278+0000", 2024-09-18T11:31:36.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8470000000000003e-06 2024-09-18T11:31:36.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414315+0000", 2024-09-18T11:31:36.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7462000000000001e-05 2024-09-18T11:31:36.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414334+0000", 2024-09-18T11:31:36.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9080999999999999e-05 2024-09-18T11:31:36.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.418419+0000", 2024-09-18T11:31:36.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040846540000000001 2024-09-18T11:31:36.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.418441+0000", 2024-09-18T11:31:36.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1844999999999998e-05 2024-09-18T11:31:36.809 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.809 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.809 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.809 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.13:18 3.0 e91/21 3:35e3c617:::2e64dd273da7619bd64b294f52905c1b9405a1ced2a8fd3c0b5bc481bc9732a6:head v 91'89, mlcod=85'87)", 2024-09-18T11:31:36.810 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:31:36.810 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.727927845, 2024-09-18T11:31:36.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048558539999999997, 2024-09-18T11:31:36.810 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.810 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.810 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:31:36.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:31:36.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:31:36.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:36.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021880+0000", 2024-09-18T11:31:36.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5860000000000001e-06 2024-09-18T11:31:36.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021890+0000", 2024-09-18T11:31:36.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8290000000000003e-06 2024-09-18T11:31:36.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021895+0000", 2024-09-18T11:31:36.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2480000000000004e-06 2024-09-18T11:31:36.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022166+0000", 2024-09-18T11:31:36.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027073499999999998 2024-09-18T11:31:36.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022184+0000", 2024-09-18T11:31:36.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8082000000000001e-05 2024-09-18T11:31:36.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026693+0000", 2024-09-18T11:31:36.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045093479999999998 2024-09-18T11:31:36.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026729+0000", 2024-09-18T11:31:36.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6282999999999998e-05 2024-09-18T11:31:36.814 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.814 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.814 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.814 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:573 4.4 e91/22 4:2c93196f:::smithi11939506-28:26 v 91'135, mlcod=90'131)", 2024-09-18T11:31:36.814 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:31:36.814 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.725461255999999, 2024-09-18T11:31:36.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050722739999999999, 2024-09-18T11:31:36.814 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.815 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.815 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:31:36.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:31:36.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024339+0000", 2024-09-18T11:31:36.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:36.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024349+0000", 2024-09-18T11:31:36.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6339999999999996e-06 2024-09-18T11:31:36.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024362+0000", 2024-09-18T11:31:36.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-18T11:31:36.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024368+0000", 2024-09-18T11:31:36.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8649999999999998e-06 2024-09-18T11:31:36.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024431+0000", 2024-09-18T11:31:36.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2780999999999994e-05 2024-09-18T11:31:36.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024448+0000", 2024-09-18T11:31:36.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7388999999999999e-05 2024-09-18T11:31:36.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029385+0000", 2024-09-18T11:31:36.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004936523 2024-09-18T11:31:36.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029412+0000", 2024-09-18T11:31:36.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7702999999999999e-05 2024-09-18T11:31:36.818 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.818 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.818 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.819 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:574 4.4 e91/22 4:2cb3e6a5:::smithi11939506-3:1e v 91'137, mlcod=90'131)", 2024-09-18T11:31:36.819 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:31:36.819 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.725023213, 2024-09-18T11:31:36.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070802850000000004, 2024-09-18T11:31:36.819 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.819 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.819 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:31:36.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:31:36.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024777+0000", 2024-09-18T11:31:36.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:36.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024785+0000", 2024-09-18T11:31:36.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8099999999999998e-06 2024-09-18T11:31:36.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024794+0000", 2024-09-18T11:31:36.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5839999999999995e-06 2024-09-18T11:31:36.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024798+0000", 2024-09-18T11:31:36.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.031e-06 2024-09-18T11:31:36.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024830+0000", 2024-09-18T11:31:36.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2268999999999999e-05 2024-09-18T11:31:36.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024847+0000", 2024-09-18T11:31:36.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7306999999999999e-05 2024-09-18T11:31:36.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031834+0000", 2024-09-18T11:31:36.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069868380000000004 2024-09-18T11:31:36.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031858+0000", 2024-09-18T11:31:36.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4082000000000001e-05 2024-09-18T11:31:36.823 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.823 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.823 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.823 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:541 4.d e91/22 4:b9bb642b:::smithi11939506-27:19 v 91'158, mlcod=91'154)", 2024-09-18T11:31:36.823 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:31:36.823 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.717642520999998, 2024-09-18T11:31:36.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052185, 2024-09-18T11:31:36.824 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.824 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.824 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:31:36.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:31:36.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032158+0000", 2024-09-18T11:31:36.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:36.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032165+0000", 2024-09-18T11:31:36.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-09-18T11:31:36.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032176+0000", 2024-09-18T11:31:36.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1460999999999999e-05 2024-09-18T11:31:36.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032182+0000", 2024-09-18T11:31:36.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0870000000000003e-06 2024-09-18T11:31:36.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032224+0000", 2024-09-18T11:31:36.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1372999999999998e-05 2024-09-18T11:31:36.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032246+0000", 2024-09-18T11:31:36.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2158e-05 2024-09-18T11:31:36.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037348+0000", 2024-09-18T11:31:36.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051025319999999999 2024-09-18T11:31:36.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037377+0000", 2024-09-18T11:31:36.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8937000000000001e-05 2024-09-18T11:31:36.828 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.828 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.828 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.828 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:582 4.6 e91/67 4:6f86ffcc:::smithi11939506-15:26 v 91'78, mlcod=90'72)", 2024-09-18T11:31:36.828 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.043567+0000", 2024-09-18T11:31:36.828 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.706234571, 2024-09-18T11:31:36.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004448446, 2024-09-18T11:31:36.828 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.828 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.828 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043567+0000", 2024-09-18T11:31:36.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043567+0000", 2024-09-18T11:31:36.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.829 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.829 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043566+0000", 2024-09-18T11:31:36.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:36.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043571+0000", 2024-09-18T11:31:36.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.181e-06 2024-09-18T11:31:36.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043579+0000", 2024-09-18T11:31:36.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9559999999999997e-06 2024-09-18T11:31:36.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043583+0000", 2024-09-18T11:31:36.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0770000000000001e-06 2024-09-18T11:31:36.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043610+0000", 2024-09-18T11:31:36.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6778e-05 2024-09-18T11:31:36.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043633+0000", 2024-09-18T11:31:36.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2904e-05 2024-09-18T11:31:36.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.047991+0000", 2024-09-18T11:31:36.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043582910000000002 2024-09-18T11:31:36.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.048015+0000", 2024-09-18T11:31:36.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3813000000000001e-05 2024-09-18T11:31:36.832 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.832 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.832 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.833 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2754 4.4 e92/22 4:2cb3e6a5:::smithi11939506-3:head v 92'147, mlcod=92'143)", 2024-09-18T11:31:36.833 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:17.887302+0000", 2024-09-18T11:31:36.833 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.862499143000001, 2024-09-18T11:31:36.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044319160000000002, 2024-09-18T11:31:36.833 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.833 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.833 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887302+0000", 2024-09-18T11:31:36.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887302+0000", 2024-09-18T11:31:36.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887301+0000", 2024-09-18T11:31:36.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:36.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887313+0000", 2024-09-18T11:31:36.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2468e-05 2024-09-18T11:31:36.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887331+0000", 2024-09-18T11:31:36.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7362000000000001e-05 2024-09-18T11:31:36.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.835 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887338+0000", 2024-09-18T11:31:36.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7980000000000006e-06 2024-09-18T11:31:36.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.835 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887416+0000", 2024-09-18T11:31:36.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7577999999999995e-05 2024-09-18T11:31:36.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887441+0000", 2024-09-18T11:31:36.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.531e-05 2024-09-18T11:31:36.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.891671+0000", 2024-09-18T11:31:36.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00422989 2024-09-18T11:31:36.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.891734+0000", 2024-09-18T11:31:36.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2885999999999994e-05 2024-09-18T11:31:36.837 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.837 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.837 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.837 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2769 4.4 e92/22 4:2cb3e6a5:::smithi11939506-3:head v 92'150, mlcod=92'148)", 2024-09-18T11:31:36.837 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:18.744829+0000", 2024-09-18T11:31:36.837 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.004971813000001, 2024-09-18T11:31:36.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043433719999999999, 2024-09-18T11:31:36.838 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.838 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.838 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744829+0000", 2024-09-18T11:31:36.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744829+0000", 2024-09-18T11:31:36.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744828+0000", 2024-09-18T11:31:36.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:36.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.745480+0000", 2024-09-18T11:31:36.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00065170400000000002 2024-09-18T11:31:36.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.745499+0000", 2024-09-18T11:31:36.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9218000000000001e-05 2024-09-18T11:31:36.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.745507+0000", 2024-09-18T11:31:36.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1950000000000003e-06 2024-09-18T11:31:36.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.745567+0000", 2024-09-18T11:31:36.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0040000000000001e-05 2024-09-18T11:31:36.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.745594+0000", 2024-09-18T11:31:36.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6803000000000001e-05 2024-09-18T11:31:36.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.749126+0000", 2024-09-18T11:31:36.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003532622 2024-09-18T11:31:36.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.749173+0000", 2024-09-18T11:31:36.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6334000000000001e-05 2024-09-18T11:31:36.841 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.842 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.842 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.842 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3118 4.4 e105/22 4:2d9c3f7f:::smithi11939506-5:head v 105'176, mlcod=103'173)", 2024-09-18T11:31:36.842 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:33.211199+0000", 2024-09-18T11:31:36.842 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.538602515, 2024-09-18T11:31:36.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041330630000000002, 2024-09-18T11:31:36.842 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:36.842 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:36.842 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:36.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:36.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:33.211199+0000", 2024-09-18T11:31:36.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:36.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:33.211199+0000", 2024-09-18T11:31:36.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:36.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:36.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:33.211198+0000", 2024-09-18T11:31:36.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:36.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:36.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:33.211645+0000", 2024-09-18T11:31:36.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00044782900000000001 2024-09-18T11:31:36.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:36.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:33.211666+0000", 2024-09-18T11:31:36.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0665999999999998e-05 2024-09-18T11:31:36.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:36.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:33.211674+0000", 2024-09-18T11:31:36.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1850000000000004e-06 2024-09-18T11:31:36.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:36.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:33.211725+0000", 2024-09-18T11:31:36.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0498999999999999e-05 2024-09-18T11:31:36.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:36.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:33.211748+0000", 2024-09-18T11:31:36.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3411999999999999e-05 2024-09-18T11:31:36.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:36.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:33.215293+0000", 2024-09-18T11:31:36.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035451760000000001 2024-09-18T11:31:36.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:36.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:36.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:36.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:33.215332+0000", 2024-09-18T11:31:36.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8417999999999997e-05 2024-09-18T11:31:36.846 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.846 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.846 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.846 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:36.847 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:36.847 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:36.848 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:31:36.849 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:36.847+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:36.946 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:36.946 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:36.947 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:36.947 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:36.949 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:36.948+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:36.955 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:31:37.050 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:37.048+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:37.052 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:37.052 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:37.052 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:37.052 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:37.052 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:37.060 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:31:37.089 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:37.087+0000 7f2d449e8640 -1 osd.1 108 handle_osd_map injecting map crc failure 2024-09-18T11:31:37.091 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is set 2024-09-18T11:31:37.092 INFO:tasks.rados.rados.0.smithi119.stdout:1469: done (0 left) 2024-09-18T11:31:37.092 INFO:tasks.rados.rados.0.smithi119.stdout:1470: write oid 40 current snap is 37 2024-09-18T11:31:37.092 INFO:tasks.rados.rados.0.smithi119.stdout:1470: seq_num 276 ranges {638732=557432,1617736=710086} 2024-09-18T11:31:37.098 INFO:tasks.rados.rados.0.smithi119.stdout:1470: writing smithi11939506-40 from 638732 to 1196164 tid 1 2024-09-18T11:31:37.105 INFO:tasks.rados.rados.0.smithi119.stdout:1470: writing smithi11939506-40 from 1617736 to 2327822 tid 2 2024-09-18T11:31:37.105 INFO:tasks.rados.rados.0.smithi119.stdout:1471: read oid 2 snap -1 2024-09-18T11:31:37.105 INFO:tasks.rados.rados.0.smithi119.stdout:1471: expect deleted 2024-09-18T11:31:37.105 INFO:tasks.rados.rados.0.smithi119.stdout:1472: read oid 7 snap 35 2024-09-18T11:31:37.105 INFO:tasks.rados.rados.0.smithi119.stdout:1472: expect (ObjNum 145 snap 18 seq_num 145) 2024-09-18T11:31:37.105 INFO:tasks.rados.rados.0.smithi119.stdout:1473: set_chunk oid 9 target oid 2024-09-18T11:31:37.121 INFO:tasks.rados.rados.0.smithi119.stdout:1473: set_chunk oid 9 offset: 1465344 length: 1337344 target oid efae800be116a8f4318957447289bc2a2c2d33d65c7604341c6a4a86aad75d7e offset: 0 snap -1 2024-09-18T11:31:37.121 INFO:tasks.rados.rados.0.smithi119.stdout:1474: read oid 8 snap -1 2024-09-18T11:31:37.121 INFO:tasks.rados.rados.0.smithi119.stdout:1474: expect deleted 2024-09-18T11:31:37.121 INFO:tasks.rados.rados.0.smithi119.stdout:1475: read oid 11 snap -1 2024-09-18T11:31:37.121 INFO:tasks.rados.rados.0.smithi119.stdout:1475: expect (ObjNum 249 snap 30 seq_num 249) 2024-09-18T11:31:37.121 INFO:tasks.rados.rados.0.smithi119.stdout:1476: read oid 41 snap 36 2024-09-18T11:31:37.121 INFO:tasks.rados.rados.0.smithi119.stdout:1476: expect (ObjNum 237 snap 28 seq_num 237) 2024-09-18T11:31:37.121 INFO:tasks.rados.rados.0.smithi119.stdout:1477: read oid 20 snap 29 2024-09-18T11:31:37.121 INFO:tasks.rados.rados.0.smithi119.stdout:1477: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:31:37.121 INFO:tasks.rados.rados.0.smithi119.stdout:1478: write (excl) oid 1 current snap is 37 2024-09-18T11:31:37.121 INFO:tasks.rados.rados.0.smithi119.stdout:1478: seq_num 277 ranges {655539=632681,2076218=570097} 2024-09-18T11:31:37.128 INFO:tasks.rados.rados.0.smithi119.stdout:1478: writing smithi11939506-1 from 655539 to 1288220 tid 1 2024-09-18T11:31:37.136 INFO:tasks.rados.rados.0.smithi119.stdout:1478: writing smithi11939506-1 from 2076218 to 2646315 tid 2 2024-09-18T11:31:37.136 INFO:tasks.rados.rados.0.smithi119.stdout:1479: delete oid 15 current snap is 37 2024-09-18T11:31:37.136 INFO:tasks.rados.rados.0.smithi119.stdout:1470: finishing write tid 1 to smithi11939506-40 2024-09-18T11:31:37.136 INFO:tasks.rados.rados.0.smithi119.stdout:1470: finishing write tid 2 to smithi11939506-40 2024-09-18T11:31:37.136 INFO:tasks.rados.rados.0.smithi119.stdout:1470: finishing write tid 3 to smithi11939506-40 2024-09-18T11:31:37.136 INFO:tasks.rados.rados.0.smithi119.stdout:1470: finishing write tid 4 to smithi11939506-40 2024-09-18T11:31:37.136 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 151 (ObjNum 276 snap 37 seq_num 276) dirty exists 2024-09-18T11:31:37.136 INFO:tasks.rados.rados.0.smithi119.stdout:1470: left oid 40 (ObjNum 276 snap 37 seq_num 276) 2024-09-18T11:31:37.136 INFO:tasks.rados.rados.0.smithi119.stdout:1472: expect (ObjNum 145 snap 18 seq_num 145) 2024-09-18T11:31:37.152 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:37.150+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:37.198 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:37.199 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:37.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:37.199 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:37.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.199 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:37.199 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:37.199 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 86.027045307999998, 2024-09-18T11:31:37.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:31:37.200 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:37.200 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:37.200 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:37.200 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:37.200 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.200 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:31:37.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.200 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:37.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:37.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:37.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:37.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:37.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:37.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:31:37.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:37.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:37.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:31:37.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:31:37.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:37.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:31:37.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:31:37.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:31:37.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:31:37.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:31:37.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:31:37.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:37.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:31:37.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:31:37.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:37.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:31:37.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:31:37.204 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.204 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:37.204 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.204 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1001 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:37.204 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:37.204 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 82.031894030000004, 2024-09-18T11:31:37.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007461082, 2024-09-18T11:31:37.204 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:37.204 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:37.205 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:37.205 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:37.205 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.205 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1001 2024-09-18T11:31:37.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.205 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:37.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:37.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:37.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:37.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:37.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:37.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164570+0000", 2024-09-18T11:31:37.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:37.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:37.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164574+0000", 2024-09-18T11:31:37.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.822e-06 2024-09-18T11:31:37.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:37.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164582+0000", 2024-09-18T11:31:37.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3410000000000003e-06 2024-09-18T11:31:37.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164587+0000", 2024-09-18T11:31:37.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769999999999999e-06 2024-09-18T11:31:37.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164702+0000", 2024-09-18T11:31:37.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115705 2024-09-18T11:31:37.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:37.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164709+0000", 2024-09-18T11:31:37.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-09-18T11:31:37.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170296+0000", 2024-09-18T11:31:37.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055876390000000001 2024-09-18T11:31:37.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:37.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170418+0000", 2024-09-18T11:31:37.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121549 2024-09-18T11:31:37.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:37.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172032+0000", 2024-09-18T11:31:37.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00161407 2024-09-18T11:31:37.209 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.209 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:37.209 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.210 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:37.210 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:37.210 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 82.031867313999996, 2024-09-18T11:31:37.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:31:37.210 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:37.210 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:37.210 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:37.210 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:37.210 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.210 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:31:37.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.210 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:37.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:37.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:37.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:37.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:37.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:37.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:37.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:37.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:37.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:31:37.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:31:37.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:37.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:31:37.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:31:37.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:31:37.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:31:37.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:31:37.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:31:37.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:37.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:31:37.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:31:37.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:31:37.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:31:37.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:37.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:31:37.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:31:37.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:37.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:31:37.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:31:37.215 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.215 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:37.215 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.215 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:37.215 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:37.215 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 82.031849025, 2024-09-18T11:31:37.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:31:37.215 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:37.216 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:37.216 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:37.216 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:37.216 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.216 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:31:37.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.216 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:37.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:37.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:37.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:37.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:37.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:37.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:31:37.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:37.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:37.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:31:37.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:31:37.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:37.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:31:37.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:31:37.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:31:37.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:31:37.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:31:37.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:31:37.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:37.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:31:37.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:31:37.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:31:37.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:31:37.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:37.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:31:37.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:31:37.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:37.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:31:37.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:31:37.221 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.221 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:37.221 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.221 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:37.221 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:37.221 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 82.031788255999999, 2024-09-18T11:31:37.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:31:37.221 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:37.221 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:37.221 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:37.221 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:37.222 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.222 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:31:37.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.222 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:37.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:37.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:37.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:37.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:37.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:37.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:31:37.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:37.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:37.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:31:37.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:31:37.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:37.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:31:37.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:31:37.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:31:37.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:31:37.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:31:37.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:31:37.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:37.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:31:37.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:31:37.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:31:37.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:31:37.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:37.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:31:37.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:31:37.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:37.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:31:37.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:31:37.228 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.228 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:37.228 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.229 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:37.229 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:37.229 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 82.031765934999996, 2024-09-18T11:31:37.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:31:37.229 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:37.229 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:37.229 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:37.229 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:37.230 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.230 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:31:37.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.230 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:37.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:37.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:37.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:37.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:37.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:37.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:31:37.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:37.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:37.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:31:37.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:31:37.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:37.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:31:37.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:31:37.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:31:37.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:31:37.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:31:37.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:31:37.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:37.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:31:37.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:31:37.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:31:37.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:31:37.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:37.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:31:37.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:31:37.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:37.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:31:37.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:31:37.237 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.237 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:37.237 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.238 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:31:37.238 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:37.238 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.250641052000006, 2024-09-18T11:31:37.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:31:37.238 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:37.238 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.238 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:37.238 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:37.238 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.238 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:31:37.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.238 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:37.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:37.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:37.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:37.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:37.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:37.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:31:37.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:31:37.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:37.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:31:37.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:31:37.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:37.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:31:37.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:31:37.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:31:37.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:31:37.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:31:37.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:31:37.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:37.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:31:37.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:31:37.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:37.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:31:37.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:31:37.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:37.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:31:37.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:31:37.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:37.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:31:37.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:31:37.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:37.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:31:37.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:31:37.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:37.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:31:37.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:31:37.243 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.244 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:37.244 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.244 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1661 4.0 4:019b6e09:::smithi11939506-40:17 [read 0~3445402,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:37.244 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:37.244 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.846568349000002, 2024-09-18T11:31:37.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075190880000000002, 2024-09-18T11:31:37.245 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:37.245 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:37.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:37.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:37.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.245 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1661 2024-09-18T11:31:37.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.245 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:37.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:37.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:37.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:37.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:37.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:37.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:37.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:37.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:37.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349899+0000", 2024-09-18T11:31:37.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9749999999999999e-06 2024-09-18T11:31:37.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:37.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349906+0000", 2024-09-18T11:31:37.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9290000000000001e-06 2024-09-18T11:31:37.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349912+0000", 2024-09-18T11:31:37.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9900000000000002e-06 2024-09-18T11:31:37.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355645+0000", 2024-09-18T11:31:37.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057329549999999996 2024-09-18T11:31:37.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:37.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355776+0000", 2024-09-18T11:31:37.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130975 2024-09-18T11:31:37.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:37.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357416+0000", 2024-09-18T11:31:37.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001639649 2024-09-18T11:31:37.248 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.249 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:37.249 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.249 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1761 4.0 4:0aec5e1c:::smithi11939506-50:head [write 769787~704463,stat] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:37.249 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:37.249 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.894075935000004, 2024-09-18T11:31:37.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074082879999999999, 2024-09-18T11:31:37.249 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:37.249 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.249 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:37.249 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:37.249 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.249 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1761 2024-09-18T11:31:37.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.250 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:37.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:37.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:37.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:37.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:37.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:37.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302386+0000", 2024-09-18T11:31:37.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:37.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:37.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302889+0000", 2024-09-18T11:31:37.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050318499999999996 2024-09-18T11:31:37.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:37.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302930+0000", 2024-09-18T11:31:37.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0198999999999999e-05 2024-09-18T11:31:37.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302965+0000", 2024-09-18T11:31:37.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5292e-05 2024-09-18T11:31:37.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303046+0000", 2024-09-18T11:31:37.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1254999999999994e-05 2024-09-18T11:31:37.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:37.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303054+0000", 2024-09-18T11:31:37.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2800000000000003e-06 2024-09-18T11:31:37.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304658+0000", 2024-09-18T11:31:37.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016039520000000001 2024-09-18T11:31:37.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:37.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304724+0000", 2024-09-18T11:31:37.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5888000000000003e-05 2024-09-18T11:31:37.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:37.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304853+0000", 2024-09-18T11:31:37.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012912700000000001 2024-09-18T11:31:37.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:37.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309084+0000", 2024-09-18T11:31:37.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042311229999999998 2024-09-18T11:31:37.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:37.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309738+0000", 2024-09-18T11:31:37.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00065401399999999996 2024-09-18T11:31:37.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:37.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309770+0000", 2024-09-18T11:31:37.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1652000000000001e-05 2024-09-18T11:31:37.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:37.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309797+0000", 2024-09-18T11:31:37.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7052000000000001e-05 2024-09-18T11:31:37.255 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.255 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:37.255 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.256 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1951 4.0 4:019b6e09:::smithi11939506-40:13 [sparse-read 0~2626478] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:37.256 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.339659+0000", 2024-09-18T11:31:37.256 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.856806229, 2024-09-18T11:31:37.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067895999999999998, 2024-09-18T11:31:37.256 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:37.256 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:37.256 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:37.256 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:37.256 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.256 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1951 2024-09-18T11:31:37.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.257 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:37.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:37.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339659+0000", 2024-09-18T11:31:37.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:37.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339659+0000", 2024-09-18T11:31:37.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:37.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339658+0000", 2024-09-18T11:31:37.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:37.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:37.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339661+0000", 2024-09-18T11:31:37.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3749999999999999e-06 2024-09-18T11:31:37.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:37.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339668+0000", 2024-09-18T11:31:37.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1380000000000001e-06 2024-09-18T11:31:37.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339675+0000", 2024-09-18T11:31:37.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7499999999999997e-06 2024-09-18T11:31:37.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339748+0000", 2024-09-18T11:31:37.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2885000000000004e-05 2024-09-18T11:31:37.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:37.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339751+0000", 2024-09-18T11:31:37.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.446e-06 2024-09-18T11:31:37.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345249+0000", 2024-09-18T11:31:37.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054983360000000004 2024-09-18T11:31:37.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:37.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345332+0000", 2024-09-18T11:31:37.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3245999999999995e-05 2024-09-18T11:31:37.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:37.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346448+0000", 2024-09-18T11:31:37.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0011160530000000001 2024-09-18T11:31:37.261 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.261 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:37.261 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.261 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:306 4.f e69/22 4:f5d7df41:::smithi11939506-16:17 v 69'94, mlcod=65'88)", 2024-09-18T11:31:37.261 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:37.261 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.845349218000003, 2024-09-18T11:31:37.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068024720000000004, 2024-09-18T11:31:37.262 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:37.262 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.262 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:37.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:37.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:37.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:37.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:37.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:37.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351115+0000", 2024-09-18T11:31:37.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:37.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:37.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351121+0000", 2024-09-18T11:31:37.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-09-18T11:31:37.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:37.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351126+0000", 2024-09-18T11:31:37.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5969999999999999e-06 2024-09-18T11:31:37.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351147+0000", 2024-09-18T11:31:37.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1160000000000001e-05 2024-09-18T11:31:37.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351518+0000", 2024-09-18T11:31:37.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00037079300000000001 2024-09-18T11:31:37.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:37.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351536+0000", 2024-09-18T11:31:37.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8168999999999998e-05 2024-09-18T11:31:37.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:37.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357870+0000", 2024-09-18T11:31:37.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063332650000000002 2024-09-18T11:31:37.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:37.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357918+0000", 2024-09-18T11:31:37.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8347999999999999e-05 2024-09-18T11:31:37.266 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.266 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:37.266 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.266 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2022 4.0 4:0aec5e1c:::smithi11939506-50:head [write 457190~552096,stat] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-18T11:31:37.266 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:37.266 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 44.200311081000002, 2024-09-18T11:31:37.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006803974, 2024-09-18T11:31:37.266 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:37.266 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.266 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:37.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:37.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.267 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2022 2024-09-18T11:31:37.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.267 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:37.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:37.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:37.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:37.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:37.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:37.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996151+0000", 2024-09-18T11:31:37.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:37.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:37.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996461+0000", 2024-09-18T11:31:37.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000309587 2024-09-18T11:31:37.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:37.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996481+0000", 2024-09-18T11:31:37.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9949000000000002e-05 2024-09-18T11:31:37.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996489+0000", 2024-09-18T11:31:37.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9129999999999992e-06 2024-09-18T11:31:37.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996562+0000", 2024-09-18T11:31:37.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2807999999999998e-05 2024-09-18T11:31:37.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:37.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996687+0000", 2024-09-18T11:31:37.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012458200000000001 2024-09-18T11:31:37.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:37.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996909+0000", 2024-09-18T11:31:37.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222124 2024-09-18T11:31:37.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:37.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.000741+0000", 2024-09-18T11:31:37.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038320720000000002 2024-09-18T11:31:37.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:37.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002854+0000", 2024-09-18T11:31:37.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002113228 2024-09-18T11:31:37.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:37.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002911+0000", 2024-09-18T11:31:37.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7192999999999999e-05 2024-09-18T11:31:37.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:37.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002958+0000", 2024-09-18T11:31:37.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6242000000000002e-05 2024-09-18T11:31:37.272 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.272 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:37.272 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.272 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2503 3.1 e85/67 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head v 85'117, mlcod=84'115)", 2024-09-18T11:31:37.272 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:31:37.273 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.219242364999999, 2024-09-18T11:31:37.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086527619999999996, 2024-09-18T11:31:37.273 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:37.273 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.273 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:37.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:37.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:31:37.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:37.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:31:37.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:37.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977220+0000", 2024-09-18T11:31:37.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:37.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:37.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980771+0000", 2024-09-18T11:31:37.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035505200000000002 2024-09-18T11:31:37.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:37.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980797+0000", 2024-09-18T11:31:37.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.618e-05 2024-09-18T11:31:37.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980807+0000", 2024-09-18T11:31:37.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0433e-05 2024-09-18T11:31:37.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980861+0000", 2024-09-18T11:31:37.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3415999999999998e-05 2024-09-18T11:31:37.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:37.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980885+0000", 2024-09-18T11:31:37.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4212000000000001e-05 2024-09-18T11:31:37.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:37.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985826+0000", 2024-09-18T11:31:37.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049412450000000004 2024-09-18T11:31:37.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:37.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985875+0000", 2024-09-18T11:31:37.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9116000000000002e-05 2024-09-18T11:31:37.277 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.277 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:37.277 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.277 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2648 4.0 4:059d8d95:::smithi11939506-32:head [read 0~3647119,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:37.277 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:31:37.277 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.865153995, 2024-09-18T11:31:37.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0076286949999999996, 2024-09-18T11:31:37.277 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:37.277 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:37.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:37.278 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:37.278 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.278 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2648 2024-09-18T11:31:37.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.278 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:37.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:37.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:31:37.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:37.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:31:37.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:37.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331309+0000", 2024-09-18T11:31:37.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:37.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:37.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331321+0000", 2024-09-18T11:31:37.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:31:37.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:37.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331340+0000", 2024-09-18T11:31:37.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8227e-05 2024-09-18T11:31:37.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331348+0000", 2024-09-18T11:31:37.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4640000000000006e-06 2024-09-18T11:31:37.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331406+0000", 2024-09-18T11:31:37.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7445000000000003e-05 2024-09-18T11:31:37.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:37.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331521+0000", 2024-09-18T11:31:37.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115945 2024-09-18T11:31:37.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:37.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338939+0000", 2024-09-18T11:31:37.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074179859999999997 2024-09-18T11:31:37.281 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.281 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:37.281 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.282 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2649 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:37.282 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:31:37.282 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.865098996, 2024-09-18T11:31:37.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566240000000007, 2024-09-18T11:31:37.282 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:37.282 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:37.282 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:37.282 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:37.282 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.282 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2649 2024-09-18T11:31:37.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.282 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:37.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:37.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:31:37.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:37.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:31:37.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:37.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331365+0000", 2024-09-18T11:31:37.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:37.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:37.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331369+0000", 2024-09-18T11:31:37.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.551e-06 2024-09-18T11:31:37.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:37.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331377+0000", 2024-09-18T11:31:37.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8769999999999993e-06 2024-09-18T11:31:37.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331381+0000", 2024-09-18T11:31:37.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.16e-06 2024-09-18T11:31:37.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338973+0000", 2024-09-18T11:31:37.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075918460000000002 2024-09-18T11:31:37.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:37.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339048+0000", 2024-09-18T11:31:37.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5813000000000001e-05 2024-09-18T11:31:37.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:37.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339322+0000", 2024-09-18T11:31:37.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027399899999999998 2024-09-18T11:31:37.286 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.286 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:37.286 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.286 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2650 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:37.286 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:37.286 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.865078778000001, 2024-09-18T11:31:37.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084392180000000001, 2024-09-18T11:31:37.287 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:37.287 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:37.287 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:37.287 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:37.287 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.287 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2650 2024-09-18T11:31:37.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.287 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:37.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:37.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:37.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:37.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:37.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:37.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:37.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:37.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:37.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331388+0000", 2024-09-18T11:31:37.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3429999999999998e-06 2024-09-18T11:31:37.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:37.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331394+0000", 2024-09-18T11:31:37.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.19e-06 2024-09-18T11:31:37.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331398+0000", 2024-09-18T11:31:37.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.495e-06 2024-09-18T11:31:37.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339362+0000", 2024-09-18T11:31:37.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079641799999999995 2024-09-18T11:31:37.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:37.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339460+0000", 2024-09-18T11:31:37.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8329e-05 2024-09-18T11:31:37.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:37.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339825+0000", 2024-09-18T11:31:37.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036510500000000001 2024-09-18T11:31:37.291 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.291 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:37.291 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.291 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:119 3.2 3:5595a68d:::07854d2fef297a06ba81685e660c332de36d5d18d546927d30daad6d7fda1541:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:37.291 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:31:37.291 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.152927021, 2024-09-18T11:31:37.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074145979999999997, 2024-09-18T11:31:37.291 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:37.291 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.291 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:37.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:37.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:31:37.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:37.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:31:37.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:37.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043535+0000", 2024-09-18T11:31:37.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:37.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:37.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043970+0000", 2024-09-18T11:31:37.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000435148 2024-09-18T11:31:37.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:37.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043993+0000", 2024-09-18T11:31:37.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3113999999999999e-05 2024-09-18T11:31:37.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044003+0000", 2024-09-18T11:31:37.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5310000000000007e-06 2024-09-18T11:31:37.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044062+0000", 2024-09-18T11:31:37.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9487000000000002e-05 2024-09-18T11:31:37.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:37.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044071+0000", 2024-09-18T11:31:37.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1610000000000003e-06 2024-09-18T11:31:37.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046423+0000", 2024-09-18T11:31:37.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023523680000000001 2024-09-18T11:31:37.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:37.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046502+0000", 2024-09-18T11:31:37.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9339999999999996e-05 2024-09-18T11:31:37.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:37.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046664+0000", 2024-09-18T11:31:37.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016185700000000001 2024-09-18T11:31:37.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:37.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050870+0000", 2024-09-18T11:31:37.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042060070000000003 2024-09-18T11:31:37.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:37.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050878+0000", 2024-09-18T11:31:37.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8960000000000003e-06 2024-09-18T11:31:37.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:37.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050918+0000", 2024-09-18T11:31:37.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0367000000000002e-05 2024-09-18T11:31:37.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:37.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050952+0000", 2024-09-18T11:31:37.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3989999999999998e-05 2024-09-18T11:31:37.297 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.297 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:37.297 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.298 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:124 3.2 3:4326e841:::a6567eb8c0be0365cdfccf2ece66d60995301bc908261a0c1daff039d2080a5f:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:37.298 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044017+0000", 2024-09-18T11:31:37.298 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.152447411000001, 2024-09-18T11:31:37.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070611140000000003, 2024-09-18T11:31:37.298 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:37.298 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.298 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:37.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.298 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:37.298 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044017+0000", 2024-09-18T11:31:37.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:37.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044017+0000", 2024-09-18T11:31:37.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:37.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044016+0000", 2024-09-18T11:31:37.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:37.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:37.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044102+0000", 2024-09-18T11:31:37.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5756999999999999e-05 2024-09-18T11:31:37.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:37.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044114+0000", 2024-09-18T11:31:37.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1505000000000001e-05 2024-09-18T11:31:37.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044120+0000", 2024-09-18T11:31:37.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8549999999999998e-06 2024-09-18T11:31:37.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044154+0000", 2024-09-18T11:31:37.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4773000000000001e-05 2024-09-18T11:31:37.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:37.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044159+0000", 2024-09-18T11:31:37.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5310000000000003e-06 2024-09-18T11:31:37.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046975+0000", 2024-09-18T11:31:37.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028162700000000001 2024-09-18T11:31:37.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:37.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.047116+0000", 2024-09-18T11:31:37.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014086400000000001 2024-09-18T11:31:37.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:37.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.047336+0000", 2024-09-18T11:31:37.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021975299999999999 2024-09-18T11:31:37.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:37.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050964+0000", 2024-09-18T11:31:37.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036283000000000001 2024-09-18T11:31:37.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:37.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.051005+0000", 2024-09-18T11:31:37.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0763999999999998e-05 2024-09-18T11:31:37.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:37.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.051050+0000", 2024-09-18T11:31:37.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4993000000000001e-05 2024-09-18T11:31:37.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:37.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.051078+0000", 2024-09-18T11:31:37.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8694999999999999e-05 2024-09-18T11:31:37.304 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.304 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:37.304 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.304 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:127 3.2 3:4d3b277a:::3266c7aaa53a7d2c10c3c54d852d6cbbed49fb041bc0df41a3ac83b56e2b0615:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:37.304 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:31:37.304 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.152063839, 2024-09-18T11:31:37.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093148009999999993, 2024-09-18T11:31:37.305 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:37.305 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.305 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:37.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:37.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:31:37.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:37.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:31:37.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:37.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044400+0000", 2024-09-18T11:31:37.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:37.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:37.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044442+0000", 2024-09-18T11:31:37.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2247999999999999e-05 2024-09-18T11:31:37.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:37.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044453+0000", 2024-09-18T11:31:37.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0617e-05 2024-09-18T11:31:37.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044457+0000", 2024-09-18T11:31:37.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2520000000000001e-06 2024-09-18T11:31:37.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044495+0000", 2024-09-18T11:31:37.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7710000000000003e-05 2024-09-18T11:31:37.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:37.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044499+0000", 2024-09-18T11:31:37.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6269999999999999e-06 2024-09-18T11:31:37.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048239+0000", 2024-09-18T11:31:37.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037396719999999999 2024-09-18T11:31:37.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:37.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048351+0000", 2024-09-18T11:31:37.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112342 2024-09-18T11:31:37.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:37.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048545+0000", 2024-09-18T11:31:37.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000193881 2024-09-18T11:31:37.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:37.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.052689+0000", 2024-09-18T11:31:37.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041435780000000002 2024-09-18T11:31:37.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:37.309 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053654+0000", 2024-09-18T11:31:37.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096558199999999999 2024-09-18T11:31:37.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:37.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053683+0000", 2024-09-18T11:31:37.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8850999999999999e-05 2024-09-18T11:31:37.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:37.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053716+0000", 2024-09-18T11:31:37.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2468000000000002e-05 2024-09-18T11:31:37.310 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.310 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:37.310 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.311 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2963 4.0 4:019b6e09:::smithi11939506-40:head [sparse-read 0~1093084] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:37.311 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:31:37.311 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.169055041, 2024-09-18T11:31:37.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072292880000000004, 2024-09-18T11:31:37.311 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:37.311 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:37.311 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:37.311 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:37.311 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.311 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2963 2024-09-18T11:31:37.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.312 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:37.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:37.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:31:37.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:37.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:31:37.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:37.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:37.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027409+0000", 2024-09-18T11:31:37.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:37.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:37.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027412+0000", 2024-09-18T11:31:37.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.751e-06 2024-09-18T11:31:37.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:37.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027418+0000", 2024-09-18T11:31:37.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.404e-06 2024-09-18T11:31:37.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027423+0000", 2024-09-18T11:31:37.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7119999999999999e-06 2024-09-18T11:31:37.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:37.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034156+0000", 2024-09-18T11:31:37.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067331639999999998 2024-09-18T11:31:37.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:37.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034255+0000", 2024-09-18T11:31:37.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8701000000000002e-05 2024-09-18T11:31:37.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:37.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:37.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:37.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034639+0000", 2024-09-18T11:31:37.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000384006 2024-09-18T11:31:37.315 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.315 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:37.315 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.315 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:37.315 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:37.315 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:37.317 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:31:37.317 INFO:tasks.rados.rados.0.smithi119.stdout:1473: finishing set_chunk to oid 9 2024-09-18T11:31:37.317 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 9 set_chunk returned error code -95 offset: 1465344 length: 1337344 2024-09-18T11:31:37.318 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 0 (ObjNum 199 snap 23 seq_num 199) dirty exists 2024-09-18T11:31:37.318 INFO:tasks.rados.rados.0.smithi119.stdout:1476: expect (ObjNum 237 snap 28 seq_num 237) 2024-09-18T11:31:37.318 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:37.314+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:37.410 INFO:tasks.rados.rados.0.smithi119.stdout:1475: expect (ObjNum 249 snap 30 seq_num 249) 2024-09-18T11:31:37.417 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:37.415+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:37.419 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:37.419 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:37.419 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:37.419 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:37.428 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:31:37.487 INFO:tasks.rados.rados.0.smithi119.stdout:1478: finishing write tid 1 to smithi11939506-1 2024-09-18T11:31:37.487 INFO:tasks.rados.rados.0.smithi119.stdout:1477: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:31:37.518 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:37.517+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:37.526 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:37.526 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:37.526 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:37.526 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:37.526 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:37.534 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:31:37.619 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:37.618+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:37.674 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:31:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 63.255395069000002, 2024-09-18T11:31:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:31:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:37.674 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:31:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:31:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:31:37.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:31:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:31:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:31:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:31:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:31:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.677 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:31:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:31:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:31:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:31:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:31:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:31:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:31:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:31:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:31:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:31:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:31:37.679 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:31:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:31:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:31:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1628 3.3 3:f80aa31b:::77cdbf3abbada57d0636b2198046a030673818705cbf13bb77b30c3e68b24345:head [write 0~582144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:37.680 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 60.360505068999998, 2024-09-18T11:31:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081714639999999998, 2024-09-18T11:31:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1628 2024-09-18T11:31:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:37.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310975+0000", 2024-09-18T11:31:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.682 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312086+0000", 2024-09-18T11:31:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001111288 2024-09-18T11:31:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312104+0000", 2024-09-18T11:31:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7427000000000001e-05 2024-09-18T11:31:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312110+0000", 2024-09-18T11:31:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5320000000000003e-06 2024-09-18T11:31:37.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312179+0000", 2024-09-18T11:31:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9196000000000006e-05 2024-09-18T11:31:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312375+0000", 2024-09-18T11:31:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00019550899999999999 2024-09-18T11:31:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.684 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312558+0000", 2024-09-18T11:31:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000182566 2024-09-18T11:31:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.314286+0000", 2024-09-18T11:31:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00172831 2024-09-18T11:31:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319079+0000", 2024-09-18T11:31:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047929340000000004 2024-09-18T11:31:37.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319116+0000", 2024-09-18T11:31:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6993999999999999e-05 2024-09-18T11:31:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319148+0000", 2024-09-18T11:31:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2539999999999997e-05 2024-09-18T11:31:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1663 4.1 4:87078dfe:::smithi11939506-14:head [write 1916458~318873] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 60.307558505999999, 2024-09-18T11:31:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079505749999999997, 2024-09-18T11:31:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1663 2024-09-18T11:31:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:37.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.688 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:37.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:37.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.688 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:37.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:37.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.688 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:37.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363922+0000", 2024-09-18T11:31:37.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:37.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.689 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:37.689 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364211+0000", 2024-09-18T11:31:37.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000288385 2024-09-18T11:31:37.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.689 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:37.689 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364223+0000", 2024-09-18T11:31:37.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2582000000000001e-05 2024-09-18T11:31:37.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.690 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.690 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364228+0000", 2024-09-18T11:31:37.690 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0000000000000004e-06 2024-09-18T11:31:37.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.690 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.690 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.690 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364268+0000", 2024-09-18T11:31:37.690 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0235999999999999e-05 2024-09-18T11:31:37.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.690 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.690 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:37.691 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364341+0000", 2024-09-18T11:31:37.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2266000000000002e-05 2024-09-18T11:31:37.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.691 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:37.691 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364444+0000", 2024-09-18T11:31:37.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103632 2024-09-18T11:31:37.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.691 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:37.691 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369461+0000", 2024-09-18T11:31:37.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005016848 2024-09-18T11:31:37.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:37.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371820+0000", 2024-09-18T11:31:37.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023586169999999999 2024-09-18T11:31:37.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:37.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371851+0000", 2024-09-18T11:31:37.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1297999999999997e-05 2024-09-18T11:31:37.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:37.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371874+0000", 2024-09-18T11:31:37.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2841e-05 2024-09-18T11:31:37.693 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.693 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:37.693 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.693 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1664 4.1 4:87078dfe:::smithi11939506-14:head [setxattr _header (57) in=7b,truncate 2235331] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:37.693 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:37.693 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 60.307244849, 2024-09-18T11:31:37.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010368235999999999, 2024-09-18T11:31:37.693 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:37.693 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.694 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:37.694 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:37.694 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.694 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1664 2024-09-18T11:31:37.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.694 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:37.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.694 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:37.694 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:37.694 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:37.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:37.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:37.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364236+0000", 2024-09-18T11:31:37.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:37.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:37.696 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364240+0000", 2024-09-18T11:31:37.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9829999999999998e-06 2024-09-18T11:31:37.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.696 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.696 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:37.696 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364247+0000", 2024-09-18T11:31:37.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2140000000000001e-06 2024-09-18T11:31:37.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.696 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.696 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.696 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364256+0000", 2024-09-18T11:31:37.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1509999999999993e-06 2024-09-18T11:31:37.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.696 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.697 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.697 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366529+0000", 2024-09-18T11:31:37.697 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022731409999999998 2024-09-18T11:31:37.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.697 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:37.697 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366621+0000", 2024-09-18T11:31:37.697 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2408e-05 2024-09-18T11:31:37.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.697 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:37.697 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366801+0000", 2024-09-18T11:31:37.697 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018002200000000001 2024-09-18T11:31:37.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.698 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.698 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:37.698 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372155+0000", 2024-09-18T11:31:37.698 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053531519999999999 2024-09-18T11:31:37.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.698 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.698 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:37.698 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374539+0000", 2024-09-18T11:31:37.698 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023842580000000002 2024-09-18T11:31:37.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.698 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.698 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:37.698 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374573+0000", 2024-09-18T11:31:37.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4205999999999997e-05 2024-09-18T11:31:37.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.699 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.699 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:37.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374605+0000", 2024-09-18T11:31:37.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2283999999999998e-05 2024-09-18T11:31:37.699 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.699 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:37.699 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.699 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.699 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1665 4.1 4:87078dfe:::smithi11939506-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:37.700 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:37.700 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 60.307220675000003, 2024-09-18T11:31:37.700 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010471897000000001, 2024-09-18T11:31:37.700 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:37.700 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:37.700 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:37.700 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:37.700 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.700 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1665 2024-09-18T11:31:37.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.700 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:37.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.700 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:37.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:37.701 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.701 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:37.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:37.701 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.701 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:37.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:37.701 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:37.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.702 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:37.702 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364263+0000", 2024-09-18T11:31:37.702 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6400000000000001e-06 2024-09-18T11:31:37.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.702 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.702 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:37.702 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364270+0000", 2024-09-18T11:31:37.702 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:31:37.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.702 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.702 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.702 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364274+0000", 2024-09-18T11:31:37.702 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.968e-06 2024-09-18T11:31:37.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.703 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.703 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.703 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367354+0000", 2024-09-18T11:31:37.703 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030803430000000001 2024-09-18T11:31:37.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.703 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.703 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:37.703 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367406+0000", 2024-09-18T11:31:37.703 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2330999999999998e-05 2024-09-18T11:31:37.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.703 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.703 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.703 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374625+0000", 2024-09-18T11:31:37.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072190689999999998 2024-09-18T11:31:37.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.704 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.704 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:37.704 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374673+0000", 2024-09-18T11:31:37.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7525999999999999e-05 2024-09-18T11:31:37.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.704 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.704 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:37.704 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374733+0000", 2024-09-18T11:31:37.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0359000000000003e-05 2024-09-18T11:31:37.704 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.704 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:37.704 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.705 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1947 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865577,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:37.705 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:37.705 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 46.331940965000001, 2024-09-18T11:31:37.705 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010043066999999999, 2024-09-18T11:31:37.705 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:37.705 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:37.705 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:37.705 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:37.705 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.705 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1947 2024-09-18T11:31:37.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.706 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:37.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:37.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:37.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:37.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:37.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:37.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339540+0000", 2024-09-18T11:31:37.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:37.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:37.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339545+0000", 2024-09-18T11:31:37.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6290000000000001e-06 2024-09-18T11:31:37.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:37.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339553+0000", 2024-09-18T11:31:37.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6299999999999998e-06 2024-09-18T11:31:37.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.708 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.708 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339557+0000", 2024-09-18T11:31:37.708 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3590000000000001e-06 2024-09-18T11:31:37.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.708 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.708 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.708 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345600+0000", 2024-09-18T11:31:37.708 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060434640000000001 2024-09-18T11:31:37.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.708 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.708 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:37.708 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345710+0000", 2024-09-18T11:31:37.708 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010924700000000001 2024-09-18T11:31:37.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.709 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:37.709 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349584+0000", 2024-09-18T11:31:37.709 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038743459999999999 2024-09-18T11:31:37.709 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.709 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:37.709 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.709 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1948 4.d 4:b5a463bf:::smithi11939506-20:head [sparse-read 0~2642181] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:37.709 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:37.709 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 46.331920427999997, 2024-09-18T11:31:37.709 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010701952000000001, 2024-09-18T11:31:37.710 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:37.710 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:37.710 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:37.710 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:37.710 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.710 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1948 2024-09-18T11:31:37.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.710 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:37.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.710 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:37.710 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:37.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:37.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:37.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:37.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:37.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:37.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:37.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339563+0000", 2024-09-18T11:31:37.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1950000000000002e-06 2024-09-18T11:31:37.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.712 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:37.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339569+0000", 2024-09-18T11:31:37.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5459999999999999e-06 2024-09-18T11:31:37.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.712 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339572+0000", 2024-09-18T11:31:37.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.337e-06 2024-09-18T11:31:37.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349693+0000", 2024-09-18T11:31:37.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010120987999999999 2024-09-18T11:31:37.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:37.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349816+0000", 2024-09-18T11:31:37.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012241199999999999 2024-09-18T11:31:37.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:37.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350263+0000", 2024-09-18T11:31:37.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00044787199999999998 2024-09-18T11:31:37.714 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.714 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:37.714 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.714 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:172 4.5 e78/22 4:a087a566:::smithi11939506-46:19 v 78'135, mlcod=77'132)", 2024-09-18T11:31:37.714 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:37.714 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.285658415, 2024-09-18T11:31:37.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010938658, 2024-09-18T11:31:37.714 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:37.714 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.714 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:37.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.714 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:37.714 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:37.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.715 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:37.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:37.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.715 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:37.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385823+0000", 2024-09-18T11:31:37.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:37.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.715 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:37.716 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385830+0000", 2024-09-18T11:31:37.716 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2790000000000001e-06 2024-09-18T11:31:37.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.716 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:37.716 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385843+0000", 2024-09-18T11:31:37.716 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2785e-05 2024-09-18T11:31:37.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.716 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.716 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385848+0000", 2024-09-18T11:31:37.716 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5570000000000003e-06 2024-09-18T11:31:37.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.717 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385914+0000", 2024-09-18T11:31:37.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5168999999999999e-05 2024-09-18T11:31:37.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.717 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:37.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385930+0000", 2024-09-18T11:31:37.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6884999999999998e-05 2024-09-18T11:31:37.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.717 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:37.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396721+0000", 2024-09-18T11:31:37.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010790137 2024-09-18T11:31:37.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:37.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396762+0000", 2024-09-18T11:31:37.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1635999999999999e-05 2024-09-18T11:31:37.718 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.718 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:37.718 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.718 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:31:37.718 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:37.718 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.285165663000001, 2024-09-18T11:31:37.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:31:37.719 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:37.719 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.719 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:37.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:37.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:37.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:37.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:37.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.720 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:37.720 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:37.720 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:37.720 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.720 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.720 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:37.720 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:31:37.720 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:31:37.720 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.720 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.720 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:37.720 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:31:37.720 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:31:37.721 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.721 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.721 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.721 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:31:37.721 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:31:37.721 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.721 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.721 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.721 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:31:37.721 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:31:37.721 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.721 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.721 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:37.722 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:31:37.722 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:31:37.722 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.722 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.722 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:37.722 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:31:37.722 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:31:37.722 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.722 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.722 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:37.722 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:31:37.722 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:31:37.722 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.723 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:37.723 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.723 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:37.723 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:37.723 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 28.699053057, 2024-09-18T11:31:37.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:31:37.723 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:37.723 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.723 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:37.723 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:37.723 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.723 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:31:37.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.724 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:37.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:37.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:37.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:37.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:37.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:37.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:31:37.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:37.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:37.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:31:37.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:31:37.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:37.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:31:37.726 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:31:37.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.726 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.726 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:31:37.726 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:31:37.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.726 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.726 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:31:37.726 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:31:37.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.726 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:37.727 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:31:37.727 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:31:37.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.727 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:37.727 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:31:37.727 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:31:37.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.727 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:37.727 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:31:37.727 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:31:37.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.728 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:37.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:31:37.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:31:37.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.728 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:37.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:31:37.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:31:37.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.728 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:37.729 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:31:37.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:31:37.729 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.729 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:37.729 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.729 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2647 4.e 4:78eefc4b:::smithi11939506-7:head [append 0~0] snapc 30=[30,2f,2b,2a,25,24,1c] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:37.729 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:37.729 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.661638376999999, 2024-09-18T11:31:37.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096752739999999993, 2024-09-18T11:31:37.729 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:37.729 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.730 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:37.730 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:37.730 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.730 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2647 2024-09-18T11:31:37.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.730 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:37.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.730 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:37.730 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:37.730 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.730 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:37.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:37.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.731 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:37.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009843+0000", 2024-09-18T11:31:37.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:37.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.731 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:37.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009846+0000", 2024-09-18T11:31:37.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3870000000000002e-06 2024-09-18T11:31:37.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.732 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:37.732 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009853+0000", 2024-09-18T11:31:37.732 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.238e-06 2024-09-18T11:31:37.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.732 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.732 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009860+0000", 2024-09-18T11:31:37.732 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1999999999999997e-06 2024-09-18T11:31:37.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.732 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.732 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.013798+0000", 2024-09-18T11:31:37.732 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003938441 2024-09-18T11:31:37.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.733 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:37.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.013889+0000", 2024-09-18T11:31:37.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0332000000000002e-05 2024-09-18T11:31:37.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.733 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 6", 2024-09-18T11:31:37.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.014066+0000", 2024-09-18T11:31:37.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017728700000000001 2024-09-18T11:31:37.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.733 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:37.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015813+0000", 2024-09-18T11:31:37.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001746804 2024-09-18T11:31:37.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.734 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:37.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019419+0000", 2024-09-18T11:31:37.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036058399999999999 2024-09-18T11:31:37.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.734 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:37.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019478+0000", 2024-09-18T11:31:37.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8860999999999998e-05 2024-09-18T11:31:37.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.735 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:37.735 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019519+0000", 2024-09-18T11:31:37.735 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1208000000000003e-05 2024-09-18T11:31:37.735 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.735 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:37.735 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.735 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.735 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.735 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:209 4.5 e91/22 4:af4ba75a:::smithi11939506-37:19 v 91'165, mlcod=91'161)", 2024-09-18T11:31:37.735 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:37.735 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.653561649, 2024-09-18T11:31:37.735 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0091546930000000002, 2024-09-18T11:31:37.735 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:37.735 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.736 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:37.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.736 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:37.736 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:37.736 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.736 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:37.736 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:37.736 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:37.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:37.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:37.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.737 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:37.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017925+0000", 2024-09-18T11:31:37.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0819999999999998e-06 2024-09-18T11:31:37.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.737 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:37.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017935+0000", 2024-09-18T11:31:37.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0597999999999999e-05 2024-09-18T11:31:37.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017941+0000", 2024-09-18T11:31:37.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2920000000000003e-06 2024-09-18T11:31:37.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017978+0000", 2024-09-18T11:31:37.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7667999999999999e-05 2024-09-18T11:31:37.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:37.739 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018018+0000", 2024-09-18T11:31:37.739 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9700999999999998e-05 2024-09-18T11:31:37.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.739 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.739 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:37.739 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.027039+0000", 2024-09-18T11:31:37.739 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090214079999999999 2024-09-18T11:31:37.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.739 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.739 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:37.739 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.027075+0000", 2024-09-18T11:31:37.739 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5605000000000002e-05 2024-09-18T11:31:37.739 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.739 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:37.740 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.740 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:570 4.b e91/67 4:ddcb7ff4:::smithi11939506-9:18 v 91'251, mlcod=90'245)", 2024-09-18T11:31:37.740 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:37.740 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.653525877, 2024-09-18T11:31:37.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089212549999999995, 2024-09-18T11:31:37.740 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:37.740 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.740 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:37.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.740 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:37.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:37.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.741 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:37.741 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:37.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.741 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:37.741 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:37.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:37.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:37.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017960+0000", 2024-09-18T11:31:37.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.777e-06 2024-09-18T11:31:37.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:37.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017966+0000", 2024-09-18T11:31:37.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4389999999999997e-06 2024-09-18T11:31:37.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017970+0000", 2024-09-18T11:31:37.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5549999999999998e-06 2024-09-18T11:31:37.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018032+0000", 2024-09-18T11:31:37.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2976000000000001e-05 2024-09-18T11:31:37.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:37.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018053+0000", 2024-09-18T11:31:37.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0322e-05 2024-09-18T11:31:37.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:37.744 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026844+0000", 2024-09-18T11:31:37.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087915790000000008 2024-09-18T11:31:37.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.744 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:37.744 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026877+0000", 2024-09-18T11:31:37.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2916000000000001e-05 2024-09-18T11:31:37.744 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.744 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:37.744 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.744 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.5.13:15 3.1 3:bf804683:::e58794d0a579007c3bd048992c5646e06b9775f9667464eee6a2a0508e23a0a6:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:37.745 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:37.745 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.653371029999999, 2024-09-18T11:31:37.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008816908, 2024-09-18T11:31:37.745 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:37.745 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.745 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:37.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:37.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:37.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:37.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:37.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.746 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:37.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018110+0000", 2024-09-18T11:31:37.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:37.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.746 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:37.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018114+0000", 2024-09-18T11:31:37.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0339999999999998e-06 2024-09-18T11:31:37.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:37.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018121+0000", 2024-09-18T11:31:37.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0199999999999997e-06 2024-09-18T11:31:37.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018125+0000", 2024-09-18T11:31:37.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0409999999999999e-06 2024-09-18T11:31:37.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018793+0000", 2024-09-18T11:31:37.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00066779199999999995 2024-09-18T11:31:37.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:37.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018868+0000", 2024-09-18T11:31:37.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4814999999999994e-05 2024-09-18T11:31:37.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:37.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019008+0000", 2024-09-18T11:31:37.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014031800000000001 2024-09-18T11:31:37.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.749 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:37.749 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.023494+0000", 2024-09-18T11:31:37.749 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0044861720000000001 2024-09-18T11:31:37.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.749 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:37.749 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026796+0000", 2024-09-18T11:31:37.749 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0033010969999999998 2024-09-18T11:31:37.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.749 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:37.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026891+0000", 2024-09-18T11:31:37.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5828000000000005e-05 2024-09-18T11:31:37.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:37.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026928+0000", 2024-09-18T11:31:37.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6393999999999997e-05 2024-09-18T11:31:37.750 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.750 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:37.750 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.750 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2793 4.1 4:87078dfe:::smithi11939506-14:head [write 534522~634550] snapc 30=[30,2f,2b,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:31:37.751 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:37.751 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.447798402, 2024-09-18T11:31:37.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085134289999999994, 2024-09-18T11:31:37.751 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:37.751 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.751 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:37.751 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:37.751 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.751 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2793 2024-09-18T11:31:37.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.751 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:37.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.751 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:37.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:37.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:37.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:37.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:37.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223681+0000", 2024-09-18T11:31:37.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:37.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.753 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:37.753 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224251+0000", 2024-09-18T11:31:37.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056940699999999996 2024-09-18T11:31:37.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.753 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:37.753 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224267+0000", 2024-09-18T11:31:37.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6829000000000001e-05 2024-09-18T11:31:37.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.753 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224292+0000", 2024-09-18T11:31:37.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4198000000000001e-05 2024-09-18T11:31:37.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.754 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224352+0000", 2024-09-18T11:31:37.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0608e-05 2024-09-18T11:31:37.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.754 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:37.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224456+0000", 2024-09-18T11:31:37.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010412300000000001 2024-09-18T11:31:37.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224662+0000", 2024-09-18T11:31:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020561399999999999 2024-09-18T11:31:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231800+0000", 2024-09-18T11:31:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071384530000000003 2024-09-18T11:31:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232126+0000", 2024-09-18T11:31:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00032601400000000001 2024-09-18T11:31:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232158+0000", 2024-09-18T11:31:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1472e-05 2024-09-18T11:31:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232197+0000", 2024-09-18T11:31:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9011999999999999e-05 2024-09-18T11:31:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:123 3.1 3:bae7b855:::93307fb660774b1d286e3149e1a75006840a12081178bf31d320df6af68ad704:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:23.045517+0000", 2024-09-18T11:31:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.625965206, 2024-09-18T11:31:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081745810000000002, 2024-09-18T11:31:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.045517+0000", 2024-09-18T11:31:37.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.758 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:37.758 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.045517+0000", 2024-09-18T11:31:37.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.758 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:37.758 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.045516+0000", 2024-09-18T11:31:37.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:37.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:37.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046503+0000", 2024-09-18T11:31:37.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00098663900000000009 2024-09-18T11:31:37.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:37.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046516+0000", 2024-09-18T11:31:37.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2971e-05 2024-09-18T11:31:37.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046522+0000", 2024-09-18T11:31:37.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.618e-06 2024-09-18T11:31:37.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.760 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.760 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.760 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046562+0000", 2024-09-18T11:31:37.760 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9400999999999998e-05 2024-09-18T11:31:37.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.760 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.760 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:37.760 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046639+0000", 2024-09-18T11:31:37.760 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7037000000000001e-05 2024-09-18T11:31:37.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.760 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.761 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:37.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046809+0000", 2024-09-18T11:31:37.761 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017013799999999999 2024-09-18T11:31:37.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.761 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:37.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.049963+0000", 2024-09-18T11:31:37.761 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031541389999999998 2024-09-18T11:31:37.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.761 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:37.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.053637+0000", 2024-09-18T11:31:37.761 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036736550000000001 2024-09-18T11:31:37.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:37.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.053665+0000", 2024-09-18T11:31:37.762 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8759999999999999e-05 2024-09-18T11:31:37.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:37.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.053691+0000", 2024-09-18T11:31:37.762 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5978000000000001e-05 2024-09-18T11:31:37.762 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.762 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:37.762 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.763 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2955 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:37.763 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:31:37.763 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.644290628, 2024-09-18T11:31:37.763 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010869531, 2024-09-18T11:31:37.763 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:37.763 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:37.763 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:37.763 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:37.763 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.763 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2955 2024-09-18T11:31:37.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.764 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:37.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:37.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:31:37.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:37.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:31:37.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.765 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:37.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027190+0000", 2024-09-18T11:31:37.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:37.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.765 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:37.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027196+0000", 2024-09-18T11:31:37.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5590000000000002e-06 2024-09-18T11:31:37.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.765 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:37.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027208+0000", 2024-09-18T11:31:37.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1816e-05 2024-09-18T11:31:37.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.766 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.766 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027215+0000", 2024-09-18T11:31:37.766 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4509999999999997e-06 2024-09-18T11:31:37.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.766 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.766 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027725+0000", 2024-09-18T11:31:37.766 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000510053 2024-09-18T11:31:37.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.767 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:37.767 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027803+0000", 2024-09-18T11:31:37.767 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8750000000000003e-05 2024-09-18T11:31:37.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.767 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.767 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.767 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032412+0000", 2024-09-18T11:31:37.767 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046089030000000001 2024-09-18T11:31:37.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.767 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.767 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:37.767 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032499+0000", 2024-09-18T11:31:37.767 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6316000000000005e-05 2024-09-18T11:31:37.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.768 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:37.768 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038061+0000", 2024-09-18T11:31:37.768 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005562259 2024-09-18T11:31:37.768 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.768 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:37.768 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.768 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2956 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~2100908,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:37.768 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:37.769 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.644256807, 2024-09-18T11:31:37.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011863831, 2024-09-18T11:31:37.769 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:37.769 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:37.769 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:37.769 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:37.769 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.769 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2956 2024-09-18T11:31:37.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.769 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:37.769 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.769 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:37.769 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:37.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.770 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:37.770 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:37.770 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.770 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:37.770 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:37.770 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:37.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:37.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027228+0000", 2024-09-18T11:31:37.771 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:31:37.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:37.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027235+0000", 2024-09-18T11:31:37.771 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:31:37.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027239+0000", 2024-09-18T11:31:37.771 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.535e-06 2024-09-18T11:31:37.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.772 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.772 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027831+0000", 2024-09-18T11:31:37.772 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059244699999999996 2024-09-18T11:31:37.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.772 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:37.772 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027879+0000", 2024-09-18T11:31:37.772 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:31:37.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.772 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.773 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038098+0000", 2024-09-18T11:31:37.773 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010219137 2024-09-18T11:31:37.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.773 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.773 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:37.773 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038207+0000", 2024-09-18T11:31:37.773 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010951999999999999 2024-09-18T11:31:37.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.773 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.773 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:37.773 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039089+0000", 2024-09-18T11:31:37.773 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00088151699999999996 2024-09-18T11:31:37.773 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.774 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:37.774 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.774 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2957 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:37.774 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:37.774 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.644239023000001, 2024-09-18T11:31:37.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012041979, 2024-09-18T11:31:37.774 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:37.774 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:37.774 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:37.774 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:37.774 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.775 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2957 2024-09-18T11:31:37.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.775 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:37.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.775 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:37.775 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:37.775 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.775 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:37.775 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:37.775 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.776 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:37.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:37.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:37.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.776 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:37.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027245+0000", 2024-09-18T11:31:37.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:31:37.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.776 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:37.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027251+0000", 2024-09-18T11:31:37.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0090000000000004e-06 2024-09-18T11:31:37.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.777 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027255+0000", 2024-09-18T11:31:37.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:31:37.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.777 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027895+0000", 2024-09-18T11:31:37.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064076400000000005 2024-09-18T11:31:37.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:37.778 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027930+0000", 2024-09-18T11:31:37.778 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4334e-05 2024-09-18T11:31:37.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.778 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.778 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039111+0000", 2024-09-18T11:31:37.778 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181455999999999 2024-09-18T11:31:37.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:37.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039156+0000", 2024-09-18T11:31:37.780 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5256000000000002e-05 2024-09-18T11:31:37.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:37.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039285+0000", 2024-09-18T11:31:37.780 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012839200000000001 2024-09-18T11:31:37.780 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.780 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:37.780 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.781 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3220 3.3 3:d97f8959:::efae800be116a8f4318957447289bc2a2c2d33d65c7604341c6a4a86aad75d7e:head [write 0~1337344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-18T11:31:37.781 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:37.110836+0000", 2024-09-18T11:31:37.781 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.56064577299999996, 2024-09-18T11:31:37.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089207350000000008, 2024-09-18T11:31:37.781 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:37.781 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:37.781 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:37.781 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:37.781 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:37.781 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3220 2024-09-18T11:31:37.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.782 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:37.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:37.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.110836+0000", 2024-09-18T11:31:37.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:37.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.110836+0000", 2024-09-18T11:31:37.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:37.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:37.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.110834+0000", 2024-09-18T11:31:37.783 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:37.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.783 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:37.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114557+0000", 2024-09-18T11:31:37.783 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003723166 2024-09-18T11:31:37.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.783 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:37.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114576+0000", 2024-09-18T11:31:37.783 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.861e-05 2024-09-18T11:31:37.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.784 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:37.784 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114585+0000", 2024-09-18T11:31:37.784 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0780000000000002e-06 2024-09-18T11:31:37.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.784 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:37.784 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114630+0000", 2024-09-18T11:31:37.784 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5118999999999999e-05 2024-09-18T11:31:37.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.784 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:37.784 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114748+0000", 2024-09-18T11:31:37.784 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011817899999999999 2024-09-18T11:31:37.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.785 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 6", 2024-09-18T11:31:37.785 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114862+0000", 2024-09-18T11:31:37.785 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113978 2024-09-18T11:31:37.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.785 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:37.785 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.116796+0000", 2024-09-18T11:31:37.785 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001934343 2024-09-18T11:31:37.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.785 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:37.785 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.119693+0000", 2024-09-18T11:31:37.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028964820000000001 2024-09-18T11:31:37.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:37.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.119729+0000", 2024-09-18T11:31:37.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6022000000000002e-05 2024-09-18T11:31:37.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:37.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:37.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:37.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.119757+0000", 2024-09-18T11:31:37.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7880000000000001e-05 2024-09-18T11:31:37.786 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.786 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:37.787 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.787 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:37.787 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:37.787 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:37.787 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:31:37.789 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:31:37.791 INFO:tasks.rados.rados.0.smithi119.stdout:1478: finishing write tid 2 to smithi11939506-1 2024-09-18T11:31:37.791 INFO:tasks.rados.rados.0.smithi119.stdout:1478: finishing write tid 3 to smithi11939506-1 2024-09-18T11:31:37.791 INFO:tasks.rados.rados.0.smithi119.stdout:1478: finishing write tid 4 to smithi11939506-1 2024-09-18T11:31:37.791 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 208 (ObjNum 277 snap 37 seq_num 277) dirty exists 2024-09-18T11:31:37.791 INFO:tasks.rados.rados.0.smithi119.stdout:1478: left oid 1 (ObjNum 277 snap 37 seq_num 277) 2024-09-18T11:31:37.791 INFO:tasks.rados.rados.0.smithi119.stdout:1470: done (9 left) 2024-09-18T11:31:37.792 INFO:tasks.rados.rados.0.smithi119.stdout:1471: done (8 left) 2024-09-18T11:31:37.792 INFO:tasks.rados.rados.0.smithi119.stdout:1472: done (7 left) 2024-09-18T11:31:37.792 INFO:tasks.rados.rados.0.smithi119.stdout:1473: done (6 left) 2024-09-18T11:31:37.792 INFO:tasks.rados.rados.0.smithi119.stdout:1474: done (5 left) 2024-09-18T11:31:37.792 INFO:tasks.rados.rados.0.smithi119.stdout:1475: done (4 left) 2024-09-18T11:31:37.792 INFO:tasks.rados.rados.0.smithi119.stdout:1476: done (3 left) 2024-09-18T11:31:37.792 INFO:tasks.rados.rados.0.smithi119.stdout:1477: done (2 left) 2024-09-18T11:31:37.792 INFO:tasks.rados.rados.0.smithi119.stdout:1478: done (1 left) 2024-09-18T11:31:37.792 INFO:tasks.rados.rados.0.smithi119.stdout:1479: done (0 left) 2024-09-18T11:31:37.792 INFO:tasks.rados.rados.0.smithi119.stdout:1480: rollback oid 19 current snap is 37 2024-09-18T11:31:37.792 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 19 to 31 2024-09-18T11:31:37.792 INFO:tasks.rados.rados.0.smithi119.stdout:1481: write oid 30 current snap is 37 2024-09-18T11:31:37.792 INFO:tasks.rados.rados.0.smithi119.stdout:1481: seq_num 279 ranges {689153=783179,2076829=457413} 2024-09-18T11:31:37.793 INFO:tasks.rados.rados.0.smithi119.stdout:1481: writing smithi11939506-30 from 689153 to 1472332 tid 1 2024-09-18T11:31:37.793 INFO:tasks.rados.rados.0.smithi119.stdout:1481: writing smithi11939506-30 from 2076829 to 2534242 tid 2 2024-09-18T11:31:37.793 INFO:tasks.rados.rados.0.smithi119.stdout:1482: set_chunk oid 11 target oid 2024-09-18T11:31:37.793 INFO:tasks.rados.rados.0.smithi119.stdout:1482: set_chunk oid 11 offset: 974848 length: 1041408 target oid 1a7ecbf758006ed330dbf75062b5e72b4b54001af0a00bc410868e3dc3984fe2 offset: 0 snap -1 2024-09-18T11:31:37.793 INFO:tasks.rados.rados.0.smithi119.stdout:1483: write oid 15 current snap is 37 2024-09-18T11:31:37.793 INFO:tasks.rados.rados.0.smithi119.stdout:1480: finishing rollback tid 0 to smithi11939506-19 2024-09-18T11:31:37.793 INFO:tasks.rados.rados.0.smithi119.stdout:1483: seq_num 280 ranges {705960=458428,1735311=717424} 2024-09-18T11:31:37.793 INFO:tasks.rados.rados.0.smithi119.stdout:1483: writing smithi11939506-15 from 705960 to 1164388 tid 1 2024-09-18T11:31:37.793 INFO:tasks.rados.rados.0.smithi119.stdout:1483: writing smithi11939506-15 from 1735311 to 2452735 tid 2 2024-09-18T11:31:37.793 INFO:tasks.rados.rados.0.smithi119.stdout:1484: read oid 1 snap -1 2024-09-18T11:31:37.793 INFO:tasks.rados.rados.0.smithi119.stdout:1484: expect (ObjNum 277 snap 37 seq_num 277) 2024-09-18T11:31:37.793 INFO:tasks.rados.rados.0.smithi119.stdout:1485: snap_create 2024-09-18T11:31:37.793 INFO:tasks.rados.rados.0.smithi119.stdout:1480: finishing rollback tid 1 to smithi11939506-19 2024-09-18T11:31:37.793 INFO:tasks.rados.rados.0.smithi119.stdout:1480: finishing rollback tid 2 to smithi11939506-19 2024-09-18T11:31:37.794 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 212 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-18T11:31:37.794 INFO:tasks.rados.rados.0.smithi119.stdout:1481: finishing write tid 1 to smithi11939506-30 2024-09-18T11:31:37.794 INFO:tasks.rados.rados.0.smithi119.stdout:1481: finishing write tid 2 to smithi11939506-30 2024-09-18T11:31:37.794 INFO:tasks.rados.rados.0.smithi119.stdout:1481: finishing write tid 3 to smithi11939506-30 2024-09-18T11:31:37.794 INFO:tasks.rados.rados.0.smithi119.stdout:1481: finishing write tid 4 to smithi11939506-30 2024-09-18T11:31:37.794 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 30 v 106 (ObjNum 279 snap 37 seq_num 279) dirty exists 2024-09-18T11:31:37.794 INFO:tasks.rados.rados.0.smithi119.stdout:1481: left oid 30 (ObjNum 279 snap 37 seq_num 279) 2024-09-18T11:31:37.794 INFO:tasks.rados.rados.0.smithi119.stdout:1482: finishing set_chunk to oid 11 2024-09-18T11:31:37.794 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 11 set_chunk returned error code -95 offset: 974848 length: 1041408 2024-09-18T11:31:37.794 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 11 v 0 (ObjNum 249 snap 30 seq_num 249) dirty exists 2024-09-18T11:31:37.794 INFO:tasks.rados.rados.0.smithi119.stdout:1483: finishing write tid 1 to smithi11939506-15 2024-09-18T11:31:37.794 INFO:tasks.rados.rados.0.smithi119.stdout:1483: finishing write tid 2 to smithi11939506-15 2024-09-18T11:31:37.794 INFO:tasks.rados.rados.0.smithi119.stdout:1483: finishing write tid 3 to smithi11939506-15 2024-09-18T11:31:37.795 INFO:tasks.rados.rados.0.smithi119.stdout:1483: finishing write tid 4 to smithi11939506-15 2024-09-18T11:31:37.795 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 109 (ObjNum 280 snap 37 seq_num 280) dirty exists 2024-09-18T11:31:37.795 INFO:tasks.rados.rados.0.smithi119.stdout:1483: left oid 15 (ObjNum 280 snap 37 seq_num 280) 2024-09-18T11:31:37.795 INFO:tasks.rados.rados.0.smithi119.stdout:1484: expect (ObjNum 277 snap 37 seq_num 277) 2024-09-18T11:31:37.795 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:37.786+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:37.889 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:37.887+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:37.892 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:37.908 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:37.908 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:37.923 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:37.923 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:37.928 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:37.928 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:37.928 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:37.928 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:37.936 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:31:37.938 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:37.939 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:37.954 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:37.954 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:37.969 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:37.969 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:37.984 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:37.984 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:37.989 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:37.988+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:38.001 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:38.001 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:38.004 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:38.004 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:38.011 INFO:tasks.rados.rados.0.smithi119.stdout:1480: done (5 left) 2024-09-18T11:31:38.011 INFO:tasks.rados.rados.0.smithi119.stdout:1481: done (4 left) 2024-09-18T11:31:38.011 INFO:tasks.rados.rados.0.smithi119.stdout:1482: done (3 left) 2024-09-18T11:31:38.011 INFO:tasks.rados.rados.0.smithi119.stdout:1483: done (2 left) 2024-09-18T11:31:38.012 INFO:tasks.rados.rados.0.smithi119.stdout:1484: done (1 left) 2024-09-18T11:31:38.012 INFO:tasks.rados.rados.0.smithi119.stdout:1485: done (0 left) 2024-09-18T11:31:38.012 INFO:tasks.rados.rados.0.smithi119.stdout:1486: tier_promote oid 20 2024-09-18T11:31:38.012 INFO:tasks.rados.rados.0.smithi119.stdout:1487: snap_remove snap 29 2024-09-18T11:31:38.039 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:38.039 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:38.048 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:31:38.091 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:38.089+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:38.186 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:38.186 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:38.191 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:38.189+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:38.195 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:31:38.292 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:38.290+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:38.293 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:38.293 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:38.301 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:31:38.393 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:38.392+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:38.399 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:38.399 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:38.407 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:31:38.495 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:38.493+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:38.546 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:38.547 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:38.554 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:31:38.596 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:38.594+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:38.693 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:38.693 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:38.696 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:38.695+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:38.702 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:31:38.797 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:38.796+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:38.840 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:38.840 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:38.848 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:31:38.898 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:38.896+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:38.986 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:38.986 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:38.994 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:31:39.018 INFO:tasks.rados.rados.0.smithi119.stdout:1487: done (1 left) 2024-09-18T11:31:39.018 INFO:tasks.rados.rados.0.smithi119.stdout:1488: read oid 44 snap -1 2024-09-18T11:31:39.018 INFO:tasks.rados.rados.0.smithi119.stdout:1488: expect (ObjNum 252 snap 30 seq_num 252) 2024-09-18T11:31:39.018 INFO:tasks.rados.rados.0.smithi119.stdout:1489: read oid 43 snap -1 2024-09-18T11:31:39.018 INFO:tasks.rados.rados.0.smithi119.stdout:1489: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:31:39.018 INFO:tasks.rados.rados.0.smithi119.stdout:1490: snap_create 2024-09-18T11:31:39.019 INFO:tasks.rados.rados.0.smithi119.stdout:1486: got (0) Success 2024-09-18T11:31:39.019 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 20 v 66 (ObjNum 111 snap 9 seq_num 111) dirty exists 2024-09-18T11:31:39.019 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:39.016+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:39.030 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:39.028+0000 7fd27659f640 -1 osd.6 110 handle_osd_map injecting map crc failure 2024-09-18T11:31:39.032 INFO:tasks.rados.rados.0.smithi119.stdout:1489: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:31:39.100 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:39.098+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:39.106 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T11:31:39.128 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:39.128 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:39.138 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:31:39.145 INFO:tasks.rados.rados.0.smithi119.stdout:1488: expect (ObjNum 252 snap 30 seq_num 252) 2024-09-18T11:31:39.200 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:39.198+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:39.246 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:39.246 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:39.254 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:31:39.302 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:39.301+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:39.395 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:39.395 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:39.403 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:31:39.404 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:39.402+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:39.501 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:39.501 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:39.504 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:39.502+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:39.510 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:31:39.604 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:39.602+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:39.609 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:39.609 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:39.617 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:31:39.705 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:39.703+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:39.754 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:39.755 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:39.763 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:31:39.847 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:39.845+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:39.901 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:39.901 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:39.907 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:39.906+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:39.909 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:31:40.005 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:31:40.008 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:40.006+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:40.018 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is unset 2024-09-18T11:31:40.019 INFO:tasks.rados.rados.0.smithi119.stdout:1486: done (3 left) 2024-09-18T11:31:40.019 INFO:tasks.rados.rados.0.smithi119.stdout:1488: done (2 left) 2024-09-18T11:31:40.019 INFO:tasks.rados.rados.0.smithi119.stdout:1489: done (1 left) 2024-09-18T11:31:40.019 INFO:tasks.rados.rados.0.smithi119.stdout:1490: done (0 left) 2024-09-18T11:31:40.019 INFO:tasks.rados.rados.0.smithi119.stdout:1491: set_chunk oid 40 target oid 2024-09-18T11:31:40.025 INFO:tasks.rados.rados.0.smithi119.stdout:1491: set_chunk oid 40 offset: 53248 length: 590848 target oid cf90902f562beb44ceaf96ec662616d6505d7cb47fdea97dacad2878857bc69b offset: 0 snap -1 2024-09-18T11:31:40.025 INFO:tasks.rados.rados.0.smithi119.stdout:1492: write oid 13 current snap is 39 2024-09-18T11:31:40.025 INFO:tasks.rados.rados.0.smithi119.stdout:1492: seq_num 281 ranges {722767=533677,1793793=461082} 2024-09-18T11:31:40.032 INFO:tasks.rados.rados.0.smithi119.stdout:1492: writing smithi11939506-13 from 722767 to 1256444 tid 1 2024-09-18T11:31:40.038 INFO:tasks.rados.rados.0.smithi119.stdout:1492: writing smithi11939506-13 from 1793793 to 2254875 tid 2 2024-09-18T11:31:40.038 INFO:tasks.rados.rados.0.smithi119.stdout:1493: read oid 42 snap 34 2024-09-18T11:31:40.038 INFO:tasks.rados.rados.0.smithi119.stdout:1493: expect (ObjNum 251 snap 30 seq_num 251) 2024-09-18T11:31:40.038 INFO:tasks.rados.rados.0.smithi119.stdout:1494: tier_evict oid 32 2024-09-18T11:31:40.038 INFO:tasks.rados.rados.0.smithi119.stdout:1494: tier_evict oid 32 snap -1 2024-09-18T11:31:40.038 INFO:tasks.rados.rados.0.smithi119.stdout:1491: finishing set_chunk to oid 40 2024-09-18T11:31:40.038 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 151 (ObjNum 276 snap 37 seq_num 276) dirty exists 2024-09-18T11:31:40.038 INFO:tasks.rados.rados.0.smithi119.stdout:1491: done (3 left) 2024-09-18T11:31:40.039 INFO:tasks.rados.rados.0.smithi119.stdout:1495: write oid 25 current snap is 39 2024-09-18T11:31:40.039 INFO:tasks.rados.rados.0.smithi119.stdout:1495: seq_num 282 ranges {739574=725279,1968628=721093} 2024-09-18T11:31:40.046 INFO:tasks.rados.rados.0.smithi119.stdout:1495: writing smithi11939506-25 from 739574 to 1464853 tid 1 2024-09-18T11:31:40.053 INFO:tasks.rados.rados.0.smithi119.stdout:1495: writing smithi11939506-25 from 1968628 to 2689721 tid 2 2024-09-18T11:31:40.053 INFO:tasks.rados.rados.0.smithi119.stdout:1496: read oid 49 snap -1 2024-09-18T11:31:40.054 INFO:tasks.rados.rados.0.smithi119.stdout:1496: expect (ObjNum 216 snap 26 seq_num 216) 2024-09-18T11:31:40.054 INFO:tasks.rados.rados.0.smithi119.stdout:1497: read oid 8 snap -1 2024-09-18T11:31:40.054 INFO:tasks.rados.rados.0.smithi119.stdout:1497: expect deleted 2024-09-18T11:31:40.054 INFO:tasks.rados.rados.0.smithi119.stdout:1498: write oid 12 current snap is 39 2024-09-18T11:31:40.054 INFO:tasks.rados.rados.0.smithi119.stdout:1498: seq_num 283 ranges {756381=400528,1910757=464751} 2024-09-18T11:31:40.055 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:40.055 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:40.060 INFO:tasks.rados.rados.0.smithi119.stdout:1498: writing smithi11939506-12 from 756381 to 1156909 tid 1 2024-09-18T11:31:40.068 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:31:40.068 INFO:tasks.rados.rados.0.smithi119.stdout:1498: writing smithi11939506-12 from 1910757 to 2375508 tid 2 2024-09-18T11:31:40.068 INFO:tasks.rados.rados.0.smithi119.stdout:1499: read oid 5 snap 38 2024-09-18T11:31:40.068 INFO:tasks.rados.rados.0.smithi119.stdout:1499: expect (ObjNum 272 snap 34 seq_num 272) 2024-09-18T11:31:40.069 INFO:tasks.rados.rados.0.smithi119.stdout:1500: set_chunk oid 40 target oid 2024-09-18T11:31:40.073 INFO:tasks.rados.rados.0.smithi119.stdout:1500: set_chunk oid 40 offset: 424448 length: 492032 target oid 375ff7167fbb5370317c878e5fbd545bd80e4875f5f1b54622da521f9043cc20 offset: 0 snap -1 2024-09-18T11:31:40.073 INFO:tasks.rados.rados.0.smithi119.stdout:1501: read oid 2 snap -1 2024-09-18T11:31:40.073 INFO:tasks.rados.rados.0.smithi119.stdout:1501: expect deleted 2024-09-18T11:31:40.073 INFO:tasks.rados.rados.0.smithi119.stdout:1502: set_chunk oid 22 target oid 2024-09-18T11:31:40.087 INFO:tasks.rados.rados.0.smithi119.stdout:1502: set_chunk oid 22 offset: 900096 length: 1426944 target oid a6e087f195b578083fcff7dae53d705e70b54d59c211bc44f60547f1c8ffbcf5 offset: 0 snap -1 2024-09-18T11:31:40.087 INFO:tasks.rados.rados.0.smithi119.stdout:1494: got (22) Invalid argument 2024-09-18T11:31:40.087 INFO:tasks.rados.rados.0.smithi119.stdout:1503: read oid 14 snap -1 2024-09-18T11:31:40.088 INFO:tasks.rados.rados.0.smithi119.stdout:1503: expect (ObjNum 246 snap 28 seq_num 246) 2024-09-18T11:31:40.088 INFO:tasks.rados.rados.0.smithi119.stdout:1494: done (11 left) 2024-09-18T11:31:40.088 INFO:tasks.rados.rados.0.smithi119.stdout:1504: write oid 37 current snap is 39 2024-09-18T11:31:40.088 INFO:tasks.rados.rados.0.smithi119.stdout:1492: finishing write tid 1 to smithi11939506-13 2024-09-18T11:31:40.088 INFO:tasks.rados.rados.0.smithi119.stdout:1492: finishing write tid 2 to smithi11939506-13 2024-09-18T11:31:40.088 INFO:tasks.rados.rados.0.smithi119.stdout:1492: finishing write tid 3 to smithi11939506-13 2024-09-18T11:31:40.088 INFO:tasks.rados.rados.0.smithi119.stdout:1504: seq_num 284 ranges {773188=475777,1969239=724762} 2024-09-18T11:31:40.089 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:40.088+0000 7fd27659f640 -1 osd.6 111 handle_osd_map injecting map crc failure 2024-09-18T11:31:40.095 INFO:tasks.rados.rados.0.smithi119.stdout:1504: writing smithi11939506-37 from 773188 to 1248965 tid 1 2024-09-18T11:31:40.103 INFO:tasks.rados.rados.0.smithi119.stdout:1504: writing smithi11939506-37 from 1969239 to 2694001 tid 2 2024-09-18T11:31:40.103 INFO:tasks.rados.rados.0.smithi119.stdout:1505: write oid 36 current snap is 39 2024-09-18T11:31:40.104 INFO:tasks.rados.rados.0.smithi119.stdout:1505: seq_num 285 ranges {789995=551026,2027721=468420} 2024-09-18T11:31:40.109 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:40.107+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:40.110 INFO:tasks.rados.rados.0.smithi119.stdout:1505: writing smithi11939506-36 from 789995 to 1341021 tid 1 2024-09-18T11:31:40.117 INFO:tasks.rados.rados.0.smithi119.stdout:1505: writing smithi11939506-36 from 2027721 to 2496141 tid 2 2024-09-18T11:31:40.117 INFO:tasks.rados.rados.0.smithi119.stdout:1506: read oid 35 snap -1 2024-09-18T11:31:40.118 INFO:tasks.rados.rados.0.smithi119.stdout:1506: expect (ObjNum 155 snap 20 seq_num 155) 2024-09-18T11:31:40.118 INFO:tasks.rados.rados.0.smithi119.stdout:1507: rollback oid 32 current snap is 39 2024-09-18T11:31:40.118 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 32 to 33 2024-09-18T11:31:40.118 INFO:tasks.rados.rados.0.smithi119.stdout:1492: finishing write tid 4 to smithi11939506-13 2024-09-18T11:31:40.118 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 13 v 337 (ObjNum 281 snap 39 seq_num 281) dirty exists 2024-09-18T11:31:40.118 INFO:tasks.rados.rados.0.smithi119.stdout:1492: left oid 13 (ObjNum 281 snap 39 seq_num 281) 2024-09-18T11:31:40.118 INFO:tasks.rados.rados.0.smithi119.stdout:1492: done (14 left) 2024-09-18T11:31:40.118 INFO:tasks.rados.rados.0.smithi119.stdout:1508: tier_evict oid 34 2024-09-18T11:31:40.118 INFO:tasks.rados.rados.0.smithi119.stdout:1508: tier_evict oid 34 snap -1 2024-09-18T11:31:40.118 INFO:tasks.rados.rados.0.smithi119.stdout:1493: expect (ObjNum 251 snap 30 seq_num 251) 2024-09-18T11:31:40.125 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:40.141 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:40.142 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:40.159 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:40.159 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:40.173 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:40.174 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:40.182 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:40.182 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:40.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:40.182 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:40.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.182 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 89.010562938000007, 2024-09-18T11:31:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:31:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:31:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:40.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:31:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:40.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:31:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:31:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:31:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:31:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:31:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:31:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:31:40.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:31:40.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:40.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:31:40.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:31:40.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:40.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:31:40.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:31:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1001 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.015411659999998, 2024-09-18T11:31:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007461082, 2024-09-18T11:31:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:40.187 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:40.188 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.188 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1001 2024-09-18T11:31:40.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.188 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:40.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:40.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:40.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:40.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164570+0000", 2024-09-18T11:31:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164574+0000", 2024-09-18T11:31:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.822e-06 2024-09-18T11:31:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164582+0000", 2024-09-18T11:31:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3410000000000003e-06 2024-09-18T11:31:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164587+0000", 2024-09-18T11:31:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769999999999999e-06 2024-09-18T11:31:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164702+0000", 2024-09-18T11:31:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115705 2024-09-18T11:31:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164709+0000", 2024-09-18T11:31:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-09-18T11:31:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170296+0000", 2024-09-18T11:31:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055876390000000001 2024-09-18T11:31:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170418+0000", 2024-09-18T11:31:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121549 2024-09-18T11:31:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172032+0000", 2024-09-18T11:31:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00161407 2024-09-18T11:31:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.015384944000004, 2024-09-18T11:31:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:31:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:40.192 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:31:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:40.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:40.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:40.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:40.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:40.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:31:40.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:31:40.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:31:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:31:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:31:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:31:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:31:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:31:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:31:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:31:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:31:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:31:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:31:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:31:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:40.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:31:40.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:31:40.197 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.197 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:40.197 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.197 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:40.197 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:40.197 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.015366654999994, 2024-09-18T11:31:40.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:31:40.197 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:31:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:31:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:31:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:31:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:31:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:31:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:31:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:31:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:31:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:31:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:31:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:31:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:31:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:31:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:31:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:31:40.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:31:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:31:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.015305885999993, 2024-09-18T11:31:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:31:40.202 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:31:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:40.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:31:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:31:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:31:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:40.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:31:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:31:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:31:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:31:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:31:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:31:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:31:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:31:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:31:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:31:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:31:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:31:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:31:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:31:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.015283565000004, 2024-09-18T11:31:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:31:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:40.207 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:31:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:40.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:31:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:31:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:31:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:31:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:31:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:31:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:31:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:31:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:31:40.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:31:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:31:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:31:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:31:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:31:40.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:31:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:31:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:31:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:31:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:40.212 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 80.234158682, 2024-09-18T11:31:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:31:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:31:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:40.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:31:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:31:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:31:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:31:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:31:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:31:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:31:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:31:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:31:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:31:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:31:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:31:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:31:40.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:31:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:31:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:31:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:31:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:31:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:40.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:31:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:31:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:31:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:31:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1661 4.0 4:019b6e09:::smithi11939506-40:17 [read 0~3445402,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 62.830085979000003, 2024-09-18T11:31:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075190880000000002, 2024-09-18T11:31:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1661 2024-09-18T11:31:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349899+0000", 2024-09-18T11:31:40.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9749999999999999e-06 2024-09-18T11:31:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349906+0000", 2024-09-18T11:31:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9290000000000001e-06 2024-09-18T11:31:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349912+0000", 2024-09-18T11:31:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9900000000000002e-06 2024-09-18T11:31:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355645+0000", 2024-09-18T11:31:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057329549999999996 2024-09-18T11:31:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355776+0000", 2024-09-18T11:31:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130975 2024-09-18T11:31:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357416+0000", 2024-09-18T11:31:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001639649 2024-09-18T11:31:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:40.222 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1761 4.0 4:0aec5e1c:::smithi11939506-50:head [write 769787~704463,stat] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 57.877593564999998, 2024-09-18T11:31:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074082879999999999, 2024-09-18T11:31:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1761 2024-09-18T11:31:40.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302386+0000", 2024-09-18T11:31:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302889+0000", 2024-09-18T11:31:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050318499999999996 2024-09-18T11:31:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302930+0000", 2024-09-18T11:31:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0198999999999999e-05 2024-09-18T11:31:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302965+0000", 2024-09-18T11:31:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5292e-05 2024-09-18T11:31:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303046+0000", 2024-09-18T11:31:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1254999999999994e-05 2024-09-18T11:31:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303054+0000", 2024-09-18T11:31:40.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2800000000000003e-06 2024-09-18T11:31:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304658+0000", 2024-09-18T11:31:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016039520000000001 2024-09-18T11:31:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304724+0000", 2024-09-18T11:31:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5888000000000003e-05 2024-09-18T11:31:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304853+0000", 2024-09-18T11:31:40.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012912700000000001 2024-09-18T11:31:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309084+0000", 2024-09-18T11:31:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042311229999999998 2024-09-18T11:31:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309738+0000", 2024-09-18T11:31:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00065401399999999996 2024-09-18T11:31:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309770+0000", 2024-09-18T11:31:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1652000000000001e-05 2024-09-18T11:31:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309797+0000", 2024-09-18T11:31:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7052000000000001e-05 2024-09-18T11:31:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.229 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:306 4.f e69/22 4:f5d7df41:::smithi11939506-16:17 v 69'94, mlcod=65'88)", 2024-09-18T11:31:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.828866847999997, 2024-09-18T11:31:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068024720000000004, 2024-09-18T11:31:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:40.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351115+0000", 2024-09-18T11:31:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351121+0000", 2024-09-18T11:31:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-09-18T11:31:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351126+0000", 2024-09-18T11:31:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5969999999999999e-06 2024-09-18T11:31:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351147+0000", 2024-09-18T11:31:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1160000000000001e-05 2024-09-18T11:31:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351518+0000", 2024-09-18T11:31:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00037079300000000001 2024-09-18T11:31:40.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351536+0000", 2024-09-18T11:31:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8168999999999998e-05 2024-09-18T11:31:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357870+0000", 2024-09-18T11:31:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063332650000000002 2024-09-18T11:31:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:40.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357918+0000", 2024-09-18T11:31:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8347999999999999e-05 2024-09-18T11:31:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2022 4.0 4:0aec5e1c:::smithi11939506-50:head [write 457190~552096,stat] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-18T11:31:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 47.183828710999997, 2024-09-18T11:31:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006803974, 2024-09-18T11:31:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.234 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2022 2024-09-18T11:31:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:40.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996151+0000", 2024-09-18T11:31:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996461+0000", 2024-09-18T11:31:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000309587 2024-09-18T11:31:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996481+0000", 2024-09-18T11:31:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9949000000000002e-05 2024-09-18T11:31:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996489+0000", 2024-09-18T11:31:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9129999999999992e-06 2024-09-18T11:31:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996562+0000", 2024-09-18T11:31:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2807999999999998e-05 2024-09-18T11:31:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996687+0000", 2024-09-18T11:31:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012458200000000001 2024-09-18T11:31:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996909+0000", 2024-09-18T11:31:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222124 2024-09-18T11:31:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.000741+0000", 2024-09-18T11:31:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038320720000000002 2024-09-18T11:31:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002854+0000", 2024-09-18T11:31:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002113228 2024-09-18T11:31:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002911+0000", 2024-09-18T11:31:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7192999999999999e-05 2024-09-18T11:31:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:40.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002958+0000", 2024-09-18T11:31:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6242000000000002e-05 2024-09-18T11:31:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2503 3.1 e85/67 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head v 85'117, mlcod=84'115)", 2024-09-18T11:31:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:31:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.202759995000001, 2024-09-18T11:31:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086527619999999996, 2024-09-18T11:31:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:40.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:31:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:31:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977220+0000", 2024-09-18T11:31:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:40.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980771+0000", 2024-09-18T11:31:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035505200000000002 2024-09-18T11:31:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980797+0000", 2024-09-18T11:31:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.618e-05 2024-09-18T11:31:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980807+0000", 2024-09-18T11:31:40.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0433e-05 2024-09-18T11:31:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980861+0000", 2024-09-18T11:31:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3415999999999998e-05 2024-09-18T11:31:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980885+0000", 2024-09-18T11:31:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4212000000000001e-05 2024-09-18T11:31:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:40.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985826+0000", 2024-09-18T11:31:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049412450000000004 2024-09-18T11:31:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985875+0000", 2024-09-18T11:31:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9116000000000002e-05 2024-09-18T11:31:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2648 4.0 4:059d8d95:::smithi11939506-32:head [read 0~3647119,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:31:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 25.848671625000001, 2024-09-18T11:31:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0076286949999999996, 2024-09-18T11:31:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2648 2024-09-18T11:31:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:31:40.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:31:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331309+0000", 2024-09-18T11:31:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:40.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331321+0000", 2024-09-18T11:31:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:31:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331340+0000", 2024-09-18T11:31:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8227e-05 2024-09-18T11:31:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331348+0000", 2024-09-18T11:31:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4640000000000006e-06 2024-09-18T11:31:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331406+0000", 2024-09-18T11:31:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7445000000000003e-05 2024-09-18T11:31:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331521+0000", 2024-09-18T11:31:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115945 2024-09-18T11:31:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338939+0000", 2024-09-18T11:31:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074179859999999997 2024-09-18T11:31:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:40.248 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2649 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:31:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 25.848616625999998, 2024-09-18T11:31:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566240000000007, 2024-09-18T11:31:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2649 2024-09-18T11:31:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.249 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:31:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:31:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331365+0000", 2024-09-18T11:31:40.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331369+0000", 2024-09-18T11:31:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.551e-06 2024-09-18T11:31:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331377+0000", 2024-09-18T11:31:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8769999999999993e-06 2024-09-18T11:31:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331381+0000", 2024-09-18T11:31:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.16e-06 2024-09-18T11:31:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338973+0000", 2024-09-18T11:31:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075918460000000002 2024-09-18T11:31:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339048+0000", 2024-09-18T11:31:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5813000000000001e-05 2024-09-18T11:31:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339322+0000", 2024-09-18T11:31:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027399899999999998 2024-09-18T11:31:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2650 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 25.848596407999999, 2024-09-18T11:31:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084392180000000001, 2024-09-18T11:31:40.253 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2650 2024-09-18T11:31:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:40.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331388+0000", 2024-09-18T11:31:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3429999999999998e-06 2024-09-18T11:31:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331394+0000", 2024-09-18T11:31:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.19e-06 2024-09-18T11:31:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331398+0000", 2024-09-18T11:31:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.495e-06 2024-09-18T11:31:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339362+0000", 2024-09-18T11:31:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079641799999999995 2024-09-18T11:31:40.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339460+0000", 2024-09-18T11:31:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8329e-05 2024-09-18T11:31:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339825+0000", 2024-09-18T11:31:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036510500000000001 2024-09-18T11:31:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:119 3.2 3:5595a68d:::07854d2fef297a06ba81685e660c332de36d5d18d546927d30daad6d7fda1541:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:31:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.136444651000001, 2024-09-18T11:31:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074145979999999997, 2024-09-18T11:31:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:31:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:31:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043535+0000", 2024-09-18T11:31:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043970+0000", 2024-09-18T11:31:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000435148 2024-09-18T11:31:40.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043993+0000", 2024-09-18T11:31:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3113999999999999e-05 2024-09-18T11:31:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044003+0000", 2024-09-18T11:31:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5310000000000007e-06 2024-09-18T11:31:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044062+0000", 2024-09-18T11:31:40.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9487000000000002e-05 2024-09-18T11:31:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044071+0000", 2024-09-18T11:31:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1610000000000003e-06 2024-09-18T11:31:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046423+0000", 2024-09-18T11:31:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023523680000000001 2024-09-18T11:31:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:40.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046502+0000", 2024-09-18T11:31:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9339999999999996e-05 2024-09-18T11:31:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046664+0000", 2024-09-18T11:31:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016185700000000001 2024-09-18T11:31:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050870+0000", 2024-09-18T11:31:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042060070000000003 2024-09-18T11:31:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050878+0000", 2024-09-18T11:31:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8960000000000003e-06 2024-09-18T11:31:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050918+0000", 2024-09-18T11:31:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0367000000000002e-05 2024-09-18T11:31:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050952+0000", 2024-09-18T11:31:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3989999999999998e-05 2024-09-18T11:31:40.263 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:124 3.2 3:4326e841:::a6567eb8c0be0365cdfccf2ece66d60995301bc908261a0c1daff039d2080a5f:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044017+0000", 2024-09-18T11:31:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.135965040999999, 2024-09-18T11:31:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070611140000000003, 2024-09-18T11:31:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044017+0000", 2024-09-18T11:31:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044017+0000", 2024-09-18T11:31:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044016+0000", 2024-09-18T11:31:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044102+0000", 2024-09-18T11:31:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5756999999999999e-05 2024-09-18T11:31:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044114+0000", 2024-09-18T11:31:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1505000000000001e-05 2024-09-18T11:31:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044120+0000", 2024-09-18T11:31:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8549999999999998e-06 2024-09-18T11:31:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044154+0000", 2024-09-18T11:31:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4773000000000001e-05 2024-09-18T11:31:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044159+0000", 2024-09-18T11:31:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5310000000000003e-06 2024-09-18T11:31:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046975+0000", 2024-09-18T11:31:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028162700000000001 2024-09-18T11:31:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.047116+0000", 2024-09-18T11:31:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014086400000000001 2024-09-18T11:31:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.047336+0000", 2024-09-18T11:31:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021975299999999999 2024-09-18T11:31:40.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050964+0000", 2024-09-18T11:31:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036283000000000001 2024-09-18T11:31:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.051005+0000", 2024-09-18T11:31:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0763999999999998e-05 2024-09-18T11:31:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.051050+0000", 2024-09-18T11:31:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4993000000000001e-05 2024-09-18T11:31:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.051078+0000", 2024-09-18T11:31:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8694999999999999e-05 2024-09-18T11:31:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:127 3.2 3:4d3b277a:::3266c7aaa53a7d2c10c3c54d852d6cbbed49fb041bc0df41a3ac83b56e2b0615:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:40.270 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:31:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.135581469000002, 2024-09-18T11:31:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093148009999999993, 2024-09-18T11:31:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:31:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:40.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:31:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044400+0000", 2024-09-18T11:31:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044442+0000", 2024-09-18T11:31:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2247999999999999e-05 2024-09-18T11:31:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044453+0000", 2024-09-18T11:31:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0617e-05 2024-09-18T11:31:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044457+0000", 2024-09-18T11:31:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2520000000000001e-06 2024-09-18T11:31:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044495+0000", 2024-09-18T11:31:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7710000000000003e-05 2024-09-18T11:31:40.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044499+0000", 2024-09-18T11:31:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6269999999999999e-06 2024-09-18T11:31:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048239+0000", 2024-09-18T11:31:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037396719999999999 2024-09-18T11:31:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048351+0000", 2024-09-18T11:31:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112342 2024-09-18T11:31:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048545+0000", 2024-09-18T11:31:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000193881 2024-09-18T11:31:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.052689+0000", 2024-09-18T11:31:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041435780000000002 2024-09-18T11:31:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053654+0000", 2024-09-18T11:31:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096558199999999999 2024-09-18T11:31:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053683+0000", 2024-09-18T11:31:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8850999999999999e-05 2024-09-18T11:31:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053716+0000", 2024-09-18T11:31:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2468000000000002e-05 2024-09-18T11:31:40.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2963 4.0 4:019b6e09:::smithi11939506-40:head [sparse-read 0~1093084] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:31:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 16.152572671000001, 2024-09-18T11:31:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072292880000000004, 2024-09-18T11:31:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:40.277 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2963 2024-09-18T11:31:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:31:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:31:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027409+0000", 2024-09-18T11:31:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027412+0000", 2024-09-18T11:31:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.751e-06 2024-09-18T11:31:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027418+0000", 2024-09-18T11:31:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.404e-06 2024-09-18T11:31:40.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027423+0000", 2024-09-18T11:31:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7119999999999999e-06 2024-09-18T11:31:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034156+0000", 2024-09-18T11:31:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067331639999999998 2024-09-18T11:31:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034255+0000", 2024-09-18T11:31:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8701000000000002e-05 2024-09-18T11:31:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034639+0000", 2024-09-18T11:31:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000384006 2024-09-18T11:31:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.281 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3209 4.0 4:019b6e09:::smithi11939506-40:head [write 638732~557432] snapc 42=[42,41,3f,3e,3d,3b,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-18T11:31:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:31:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.08189943, 2024-09-18T11:31:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091774999999999999, 2024-09-18T11:31:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3209 2024-09-18T11:31:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:40.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:31:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:31:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098080+0000", 2024-09-18T11:31:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098676+0000", 2024-09-18T11:31:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059598099999999998 2024-09-18T11:31:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098706+0000", 2024-09-18T11:31:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0020000000000001e-05 2024-09-18T11:31:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098716+0000", 2024-09-18T11:31:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0225e-05 2024-09-18T11:31:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098782+0000", 2024-09-18T11:31:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5530000000000004e-05 2024-09-18T11:31:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098883+0000", 2024-09-18T11:31:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101609 2024-09-18T11:31:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.099098+0000", 2024-09-18T11:31:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000215052 2024-09-18T11:31:40.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.102511+0000", 2024-09-18T11:31:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034129429999999999 2024-09-18T11:31:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107165+0000", 2024-09-18T11:31:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046531480000000002 2024-09-18T11:31:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:40.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107217+0000", 2024-09-18T11:31:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2058000000000002e-05 2024-09-18T11:31:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107260+0000", 2024-09-18T11:31:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3865000000000001e-05 2024-09-18T11:31:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:40.287 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:40.287 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:40.288 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:40.289 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:40.290 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:31:40.290 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:40.290 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:40.291 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:40.291 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:40.291 INFO:tasks.rados.rados.0.smithi119.stdout:1495: finishing write tid 1 to smithi11939506-25 2024-09-18T11:31:40.291 INFO:tasks.rados.rados.0.smithi119.stdout:1493: done (14 left) 2024-09-18T11:31:40.292 INFO:tasks.rados.rados.0.smithi119.stdout:1509: snap_create 2024-09-18T11:31:40.292 INFO:tasks.rados.rados.0.smithi119.stdout:1495: finishing write tid 2 to smithi11939506-25 2024-09-18T11:31:40.292 INFO:tasks.rados.rados.0.smithi119.stdout:1495: finishing write tid 3 to smithi11939506-25 2024-09-18T11:31:40.292 INFO:tasks.rados.rados.0.smithi119.stdout:1495: finishing write tid 4 to smithi11939506-25 2024-09-18T11:31:40.292 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 50 (ObjNum 282 snap 39 seq_num 282) dirty exists 2024-09-18T11:31:40.292 INFO:tasks.rados.rados.0.smithi119.stdout:1495: left oid 25 (ObjNum 282 snap 39 seq_num 282) 2024-09-18T11:31:40.292 INFO:tasks.rados.rados.0.smithi119.stdout:1496: expect (ObjNum 216 snap 26 seq_num 216) 2024-09-18T11:31:40.292 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:40.293 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:40.293 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:40.293 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:40.293 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:40.286+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:40.390 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:40.388+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:40.392 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:40.392 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:40.392 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:40.393 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:40.401 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:31:40.465 INFO:tasks.rados.rados.0.smithi119.stdout:1498: finishing write tid 1 to smithi11939506-12 2024-09-18T11:31:40.466 INFO:tasks.rados.rados.0.smithi119.stdout:1498: finishing write tid 2 to smithi11939506-12 2024-09-18T11:31:40.466 INFO:tasks.rados.rados.0.smithi119.stdout:1499: expect (ObjNum 272 snap 34 seq_num 272) 2024-09-18T11:31:40.491 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:40.489+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:40.498 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:40.498 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:40.498 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:40.498 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:40.498 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:40.506 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:31:40.582 INFO:tasks.rados.rados.0.smithi119.stdout:1498: finishing write tid 3 to smithi11939506-12 2024-09-18T11:31:40.582 INFO:tasks.rados.rados.0.smithi119.stdout:1498: finishing write tid 4 to smithi11939506-12 2024-09-18T11:31:40.582 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 12 v 86 (ObjNum 283 snap 39 seq_num 283) dirty exists 2024-09-18T11:31:40.583 INFO:tasks.rados.rados.0.smithi119.stdout:1498: left oid 12 (ObjNum 283 snap 39 seq_num 283) 2024-09-18T11:31:40.583 INFO:tasks.rados.rados.0.smithi119.stdout:1500: finishing set_chunk to oid 40 2024-09-18T11:31:40.583 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 40 set_chunk returned error code -95 offset: 424448 length: 492032 2024-09-18T11:31:40.583 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 0 (ObjNum 276 snap 37 seq_num 276) dirty exists 2024-09-18T11:31:40.583 INFO:tasks.rados.rados.0.smithi119.stdout:1502: finishing set_chunk to oid 22 2024-09-18T11:31:40.583 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 155 (ObjNum 263 snap 31 seq_num 263) dirty exists 2024-09-18T11:31:40.583 INFO:tasks.rados.rados.0.smithi119.stdout:1503: expect (ObjNum 246 snap 28 seq_num 246) 2024-09-18T11:31:40.591 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:40.590+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:40.606 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:40.606 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:40.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:40.606 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:40.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.606 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:31:40.606 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:40.606 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.187796253000002, 2024-09-18T11:31:40.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:31:40.607 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:40.607 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.607 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:40.607 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:40.607 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.607 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:31:40.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.607 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:40.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.607 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:40.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:40.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.608 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:40.608 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:40.608 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.608 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:40.608 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:31:40.608 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:40.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.608 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:40.609 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:31:40.609 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:31:40.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.609 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:40.609 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:31:40.609 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:31:40.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.609 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.609 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:31:40.609 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:31:40.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.610 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.610 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:31:40.610 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:31:40.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.610 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.610 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:40.610 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:31:40.610 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:31:40.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.610 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.610 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:40.610 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:31:40.610 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:31:40.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:40.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:31:40.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:31:40.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:40.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:31:40.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:31:40.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:40.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:31:40.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:31:40.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.612 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:40.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:31:40.612 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:31:40.612 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.612 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:40.612 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.612 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1628 3.3 3:f80aa31b:::77cdbf3abbada57d0636b2198046a030673818705cbf13bb77b30c3e68b24345:head [write 0~582144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:40.612 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:40.612 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 63.292906252999998, 2024-09-18T11:31:40.612 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081714639999999998, 2024-09-18T11:31:40.613 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:40.613 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.613 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:40.613 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:40.613 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.613 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1628 2024-09-18T11:31:40.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.613 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:40.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.613 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:40.613 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:40.613 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.614 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:40.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:40.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.614 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:40.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310975+0000", 2024-09-18T11:31:40.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:40.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.614 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:40.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312086+0000", 2024-09-18T11:31:40.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001111288 2024-09-18T11:31:40.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.615 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:40.615 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312104+0000", 2024-09-18T11:31:40.615 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7427000000000001e-05 2024-09-18T11:31:40.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.615 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.615 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312110+0000", 2024-09-18T11:31:40.615 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5320000000000003e-06 2024-09-18T11:31:40.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.615 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.615 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312179+0000", 2024-09-18T11:31:40.615 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9196000000000006e-05 2024-09-18T11:31:40.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.616 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:40.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312375+0000", 2024-09-18T11:31:40.616 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00019550899999999999 2024-09-18T11:31:40.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.616 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:40.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312558+0000", 2024-09-18T11:31:40.616 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000182566 2024-09-18T11:31:40.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.616 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:40.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.314286+0000", 2024-09-18T11:31:40.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00172831 2024-09-18T11:31:40.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.617 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:40.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319079+0000", 2024-09-18T11:31:40.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047929340000000004 2024-09-18T11:31:40.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.617 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:40.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319116+0000", 2024-09-18T11:31:40.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6993999999999999e-05 2024-09-18T11:31:40.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:40.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319148+0000", 2024-09-18T11:31:40.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2539999999999997e-05 2024-09-18T11:31:40.618 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.618 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:40.618 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.618 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1663 4.1 4:87078dfe:::smithi11939506-14:head [write 1916458~318873] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:40.618 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:40.618 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 63.239959689999999, 2024-09-18T11:31:40.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079505749999999997, 2024-09-18T11:31:40.619 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:40.619 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.619 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:40.619 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:40.619 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.619 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1663 2024-09-18T11:31:40.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.619 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:40.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.619 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:40.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:40.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.620 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:40.620 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363923+0000", 2024-09-18T11:31:40.620 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.620 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:40.620 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.363922+0000", 2024-09-18T11:31:40.620 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:40.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.620 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:40.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364211+0000", 2024-09-18T11:31:40.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000288385 2024-09-18T11:31:40.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:40.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364223+0000", 2024-09-18T11:31:40.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2582000000000001e-05 2024-09-18T11:31:40.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364228+0000", 2024-09-18T11:31:40.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0000000000000004e-06 2024-09-18T11:31:40.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364268+0000", 2024-09-18T11:31:40.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0235999999999999e-05 2024-09-18T11:31:40.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:40.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364341+0000", 2024-09-18T11:31:40.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2266000000000002e-05 2024-09-18T11:31:40.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364444+0000", 2024-09-18T11:31:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103632 2024-09-18T11:31:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.369461+0000", 2024-09-18T11:31:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005016848 2024-09-18T11:31:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371820+0000", 2024-09-18T11:31:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023586169999999999 2024-09-18T11:31:40.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.624 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:40.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371851+0000", 2024-09-18T11:31:40.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1297999999999997e-05 2024-09-18T11:31:40.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.624 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:40.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.371874+0000", 2024-09-18T11:31:40.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2841e-05 2024-09-18T11:31:40.624 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.625 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:40.625 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.625 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1664 4.1 4:87078dfe:::smithi11939506-14:head [setxattr _header (57) in=7b,truncate 2235331] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:40.625 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:40.625 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 63.239646033, 2024-09-18T11:31:40.625 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010368235999999999, 2024-09-18T11:31:40.625 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:40.625 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.625 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:40.625 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:40.625 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.626 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1664 2024-09-18T11:31:40.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.626 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:40.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:40.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:40.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:40.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:40.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:40.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364236+0000", 2024-09-18T11:31:40.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:40.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:40.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364240+0000", 2024-09-18T11:31:40.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9829999999999998e-06 2024-09-18T11:31:40.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:40.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364247+0000", 2024-09-18T11:31:40.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2140000000000001e-06 2024-09-18T11:31:40.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.628 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364256+0000", 2024-09-18T11:31:40.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1509999999999993e-06 2024-09-18T11:31:40.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.628 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366529+0000", 2024-09-18T11:31:40.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022731409999999998 2024-09-18T11:31:40.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:40.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366621+0000", 2024-09-18T11:31:40.629 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2408e-05 2024-09-18T11:31:40.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:40.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366801+0000", 2024-09-18T11:31:40.629 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018002200000000001 2024-09-18T11:31:40.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:40.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372155+0000", 2024-09-18T11:31:40.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053531519999999999 2024-09-18T11:31:40.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:40.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374539+0000", 2024-09-18T11:31:40.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023842580000000002 2024-09-18T11:31:40.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:40.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374573+0000", 2024-09-18T11:31:40.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4205999999999997e-05 2024-09-18T11:31:40.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:40.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374605+0000", 2024-09-18T11:31:40.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2283999999999998e-05 2024-09-18T11:31:40.631 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.631 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:40.631 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.631 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1665 4.1 4:87078dfe:::smithi11939506-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:40.631 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:40.631 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 63.239621859000003, 2024-09-18T11:31:40.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010471897000000001, 2024-09-18T11:31:40.631 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:40.631 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:40.631 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:40.632 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:40.632 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.632 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1665 2024-09-18T11:31:40.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.632 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:40.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.632 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:40.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:40.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.632 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:40.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:40.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:40.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:40.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:40.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:40.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364263+0000", 2024-09-18T11:31:40.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6400000000000001e-06 2024-09-18T11:31:40.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:40.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364270+0000", 2024-09-18T11:31:40.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:31:40.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364274+0000", 2024-09-18T11:31:40.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.968e-06 2024-09-18T11:31:40.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367354+0000", 2024-09-18T11:31:40.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030803430000000001 2024-09-18T11:31:40.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:40.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367406+0000", 2024-09-18T11:31:40.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2330999999999998e-05 2024-09-18T11:31:40.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.635 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374625+0000", 2024-09-18T11:31:40.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072190689999999998 2024-09-18T11:31:40.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.635 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:40.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374673+0000", 2024-09-18T11:31:40.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7525999999999999e-05 2024-09-18T11:31:40.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.636 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:40.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374733+0000", 2024-09-18T11:31:40.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0359000000000003e-05 2024-09-18T11:31:40.636 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.636 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:40.636 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.636 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1947 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865577,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:40.636 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:40.636 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 49.264342149000001, 2024-09-18T11:31:40.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010043066999999999, 2024-09-18T11:31:40.637 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:40.637 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:40.637 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:40.637 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:40.637 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.637 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1947 2024-09-18T11:31:40.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.637 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:40.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:40.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:40.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:40.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:40.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:40.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339540+0000", 2024-09-18T11:31:40.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:40.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.639 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:40.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339545+0000", 2024-09-18T11:31:40.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6290000000000001e-06 2024-09-18T11:31:40.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.639 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:40.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339553+0000", 2024-09-18T11:31:40.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6299999999999998e-06 2024-09-18T11:31:40.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.639 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339557+0000", 2024-09-18T11:31:40.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3590000000000001e-06 2024-09-18T11:31:40.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.640 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.640 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345600+0000", 2024-09-18T11:31:40.640 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060434640000000001 2024-09-18T11:31:40.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.640 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:40.640 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345710+0000", 2024-09-18T11:31:40.640 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010924700000000001 2024-09-18T11:31:40.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.641 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.641 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:40.641 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349584+0000", 2024-09-18T11:31:40.641 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038743459999999999 2024-09-18T11:31:40.641 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.641 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:40.641 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.641 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.641 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1948 4.d 4:b5a463bf:::smithi11939506-20:head [sparse-read 0~2642181] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:40.641 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:40.641 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 49.264321612000003, 2024-09-18T11:31:40.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010701952000000001, 2024-09-18T11:31:40.642 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:40.642 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:40.642 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:40.642 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:40.642 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.642 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1948 2024-09-18T11:31:40.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.642 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:40.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:40.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:40.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:40.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:40.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:40.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:40.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:40.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.644 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:40.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339563+0000", 2024-09-18T11:31:40.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1950000000000002e-06 2024-09-18T11:31:40.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.644 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:40.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339569+0000", 2024-09-18T11:31:40.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5459999999999999e-06 2024-09-18T11:31:40.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.644 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339572+0000", 2024-09-18T11:31:40.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.337e-06 2024-09-18T11:31:40.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.645 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349693+0000", 2024-09-18T11:31:40.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010120987999999999 2024-09-18T11:31:40.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.645 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:40.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349816+0000", 2024-09-18T11:31:40.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012241199999999999 2024-09-18T11:31:40.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.646 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:40.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350263+0000", 2024-09-18T11:31:40.646 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00044787199999999998 2024-09-18T11:31:40.646 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.646 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:40.646 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.646 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:172 4.5 e78/22 4:a087a566:::smithi11939506-46:19 v 78'135, mlcod=77'132)", 2024-09-18T11:31:40.646 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:40.646 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 39.218059599, 2024-09-18T11:31:40.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010938658, 2024-09-18T11:31:40.647 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:40.647 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.647 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:40.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:40.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:40.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:40.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:40.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:40.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385823+0000", 2024-09-18T11:31:40.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:40.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:40.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385830+0000", 2024-09-18T11:31:40.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2790000000000001e-06 2024-09-18T11:31:40.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:40.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385843+0000", 2024-09-18T11:31:40.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2785e-05 2024-09-18T11:31:40.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.649 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385848+0000", 2024-09-18T11:31:40.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5570000000000003e-06 2024-09-18T11:31:40.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.649 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385914+0000", 2024-09-18T11:31:40.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5168999999999999e-05 2024-09-18T11:31:40.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.650 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:40.650 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385930+0000", 2024-09-18T11:31:40.650 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6884999999999998e-05 2024-09-18T11:31:40.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.650 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:40.650 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396721+0000", 2024-09-18T11:31:40.650 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010790137 2024-09-18T11:31:40.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.650 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:40.650 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396762+0000", 2024-09-18T11:31:40.651 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1635999999999999e-05 2024-09-18T11:31:40.651 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.651 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:40.651 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.651 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:31:40.651 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:40.651 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 39.217566847000001, 2024-09-18T11:31:40.651 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:31:40.651 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:40.651 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.651 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:40.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:40.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:40.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:40.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:40.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:40.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:40.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:40.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:40.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:31:40.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:31:40.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:40.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:31:40.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:31:40.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:31:40.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:31:40.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:31:40.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:31:40.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:40.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:31:40.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:31:40.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:40.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:31:40.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:31:40.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:40.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:31:40.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:31:40.656 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.656 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:40.656 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.656 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:40.656 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:40.656 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.631454241, 2024-09-18T11:31:40.656 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:31:40.656 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:40.656 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.657 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:40.657 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:40.657 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.657 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:31:40.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.657 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:40.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.657 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:40.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:40.657 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:40.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:40.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:40.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:31:40.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:40.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:40.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:31:40.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:31:40.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:40.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:31:40.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:31:40.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:31:40.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:31:40.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:31:40.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:31:40.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:40.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:31:40.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:31:40.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:40.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:31:40.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:31:40.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:40.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:31:40.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:31:40.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:40.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:31:40.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:31:40.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:40.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:31:40.662 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:31:40.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.662 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:40.662 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:31:40.662 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:31:40.662 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.662 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:40.662 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.662 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2647 4.e 4:78eefc4b:::smithi11939506-7:head [append 0~0] snapc 30=[30,2f,2b,2a,25,24,1c] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:40.663 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:40.663 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.594039560999999, 2024-09-18T11:31:40.663 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096752739999999993, 2024-09-18T11:31:40.663 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:40.663 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.663 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:40.663 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:40.663 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.663 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2647 2024-09-18T11:31:40.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.663 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:40.663 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:40.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:40.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:40.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:40.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:40.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009843+0000", 2024-09-18T11:31:40.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:40.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.665 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:40.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009846+0000", 2024-09-18T11:31:40.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3870000000000002e-06 2024-09-18T11:31:40.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.665 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:40.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009853+0000", 2024-09-18T11:31:40.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.238e-06 2024-09-18T11:31:40.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009860+0000", 2024-09-18T11:31:40.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1999999999999997e-06 2024-09-18T11:31:40.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.013798+0000", 2024-09-18T11:31:40.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003938441 2024-09-18T11:31:40.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:40.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.013889+0000", 2024-09-18T11:31:40.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0332000000000002e-05 2024-09-18T11:31:40.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.667 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.667 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 6", 2024-09-18T11:31:40.667 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.014066+0000", 2024-09-18T11:31:40.667 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017728700000000001 2024-09-18T11:31:40.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.667 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.667 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:40.667 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015813+0000", 2024-09-18T11:31:40.667 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001746804 2024-09-18T11:31:40.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.667 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.667 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:40.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019419+0000", 2024-09-18T11:31:40.668 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036058399999999999 2024-09-18T11:31:40.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.668 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:40.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019478+0000", 2024-09-18T11:31:40.668 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8860999999999998e-05 2024-09-18T11:31:40.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.668 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:40.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019519+0000", 2024-09-18T11:31:40.668 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1208000000000003e-05 2024-09-18T11:31:40.668 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.669 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:40.669 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.669 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.669 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:209 4.5 e91/22 4:af4ba75a:::smithi11939506-37:19 v 91'165, mlcod=91'161)", 2024-09-18T11:31:40.669 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:40.669 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.585962833, 2024-09-18T11:31:40.669 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0091546930000000002, 2024-09-18T11:31:40.669 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:40.669 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.669 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:40.669 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.669 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:40.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:40.670 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.670 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:40.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:40.670 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.670 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:40.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:40.670 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:40.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.671 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:40.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017925+0000", 2024-09-18T11:31:40.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0819999999999998e-06 2024-09-18T11:31:40.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.671 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:40.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017935+0000", 2024-09-18T11:31:40.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0597999999999999e-05 2024-09-18T11:31:40.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.671 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017941+0000", 2024-09-18T11:31:40.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2920000000000003e-06 2024-09-18T11:31:40.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017978+0000", 2024-09-18T11:31:40.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7667999999999999e-05 2024-09-18T11:31:40.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:40.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018018+0000", 2024-09-18T11:31:40.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9700999999999998e-05 2024-09-18T11:31:40.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.673 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:40.673 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.027039+0000", 2024-09-18T11:31:40.673 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090214079999999999 2024-09-18T11:31:40.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.673 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.673 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:40.673 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.027075+0000", 2024-09-18T11:31:40.673 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5605000000000002e-05 2024-09-18T11:31:40.673 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.673 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:40.673 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.674 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:570 4.b e91/67 4:ddcb7ff4:::smithi11939506-9:18 v 91'251, mlcod=90'245)", 2024-09-18T11:31:40.674 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:40.674 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.585927061, 2024-09-18T11:31:40.674 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089212549999999995, 2024-09-18T11:31:40.674 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:40.674 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.674 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:40.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.674 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:40.674 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:40.674 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:40.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:40.675 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:40.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:40.675 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:40.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:40.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017960+0000", 2024-09-18T11:31:40.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.777e-06 2024-09-18T11:31:40.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.676 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:40.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017966+0000", 2024-09-18T11:31:40.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4389999999999997e-06 2024-09-18T11:31:40.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.676 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017970+0000", 2024-09-18T11:31:40.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5549999999999998e-06 2024-09-18T11:31:40.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.677 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.677 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018032+0000", 2024-09-18T11:31:40.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2976000000000001e-05 2024-09-18T11:31:40.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.677 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:40.677 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018053+0000", 2024-09-18T11:31:40.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0322e-05 2024-09-18T11:31:40.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026844+0000", 2024-09-18T11:31:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087915790000000008 2024-09-18T11:31:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026877+0000", 2024-09-18T11:31:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2916000000000001e-05 2024-09-18T11:31:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.5.13:15 3.1 3:bf804683:::e58794d0a579007c3bd048992c5646e06b9775f9667464eee6a2a0508e23a0a6:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.585772213999999, 2024-09-18T11:31:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008816908, 2024-09-18T11:31:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018110+0000", 2024-09-18T11:31:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018114+0000", 2024-09-18T11:31:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0339999999999998e-06 2024-09-18T11:31:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018121+0000", 2024-09-18T11:31:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0199999999999997e-06 2024-09-18T11:31:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018125+0000", 2024-09-18T11:31:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0409999999999999e-06 2024-09-18T11:31:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018793+0000", 2024-09-18T11:31:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00066779199999999995 2024-09-18T11:31:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018868+0000", 2024-09-18T11:31:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4814999999999994e-05 2024-09-18T11:31:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019008+0000", 2024-09-18T11:31:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014031800000000001 2024-09-18T11:31:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.023494+0000", 2024-09-18T11:31:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0044861720000000001 2024-09-18T11:31:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026796+0000", 2024-09-18T11:31:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0033010969999999998 2024-09-18T11:31:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026891+0000", 2024-09-18T11:31:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5828000000000005e-05 2024-09-18T11:31:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026928+0000", 2024-09-18T11:31:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6393999999999997e-05 2024-09-18T11:31:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2793 4.1 4:87078dfe:::smithi11939506-14:head [write 534522~634550] snapc 30=[30,2f,2b,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:31:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.380199586, 2024-09-18T11:31:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085134289999999994, 2024-09-18T11:31:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2793 2024-09-18T11:31:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223681+0000", 2024-09-18T11:31:40.687 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:40.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.687 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:40.687 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224251+0000", 2024-09-18T11:31:40.687 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056940699999999996 2024-09-18T11:31:40.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.687 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:40.687 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224267+0000", 2024-09-18T11:31:40.687 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6829000000000001e-05 2024-09-18T11:31:40.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.688 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224292+0000", 2024-09-18T11:31:40.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4198000000000001e-05 2024-09-18T11:31:40.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.688 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224352+0000", 2024-09-18T11:31:40.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0608e-05 2024-09-18T11:31:40.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.688 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:40.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224456+0000", 2024-09-18T11:31:40.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010412300000000001 2024-09-18T11:31:40.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.689 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:40.689 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224662+0000", 2024-09-18T11:31:40.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020561399999999999 2024-09-18T11:31:40.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.689 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:40.689 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231800+0000", 2024-09-18T11:31:40.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071384530000000003 2024-09-18T11:31:40.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.689 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:40.690 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232126+0000", 2024-09-18T11:31:40.690 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00032601400000000001 2024-09-18T11:31:40.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.690 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.690 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:40.690 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232158+0000", 2024-09-18T11:31:40.690 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1472e-05 2024-09-18T11:31:40.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.690 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.690 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:40.690 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232197+0000", 2024-09-18T11:31:40.690 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9011999999999999e-05 2024-09-18T11:31:40.690 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.690 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:40.690 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.691 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:123 3.1 3:bae7b855:::93307fb660774b1d286e3149e1a75006840a12081178bf31d320df6af68ad704:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:40.691 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:23.045517+0000", 2024-09-18T11:31:40.691 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.55836639, 2024-09-18T11:31:40.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081745810000000002, 2024-09-18T11:31:40.691 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:40.691 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.691 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:40.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.691 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:40.691 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.045517+0000", 2024-09-18T11:31:40.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:40.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.045517+0000", 2024-09-18T11:31:40.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:40.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.045516+0000", 2024-09-18T11:31:40.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:40.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:40.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046503+0000", 2024-09-18T11:31:40.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00098663900000000009 2024-09-18T11:31:40.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.693 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:40.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046516+0000", 2024-09-18T11:31:40.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2971e-05 2024-09-18T11:31:40.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.693 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046522+0000", 2024-09-18T11:31:40.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.618e-06 2024-09-18T11:31:40.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.694 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.694 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046562+0000", 2024-09-18T11:31:40.694 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9400999999999998e-05 2024-09-18T11:31:40.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.694 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:40.694 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046639+0000", 2024-09-18T11:31:40.694 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7037000000000001e-05 2024-09-18T11:31:40.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.694 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:40.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046809+0000", 2024-09-18T11:31:40.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017013799999999999 2024-09-18T11:31:40.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:40.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.049963+0000", 2024-09-18T11:31:40.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031541389999999998 2024-09-18T11:31:40.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:40.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.053637+0000", 2024-09-18T11:31:40.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036736550000000001 2024-09-18T11:31:40.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.696 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:40.696 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.053665+0000", 2024-09-18T11:31:40.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8759999999999999e-05 2024-09-18T11:31:40.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.696 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.696 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:40.696 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.053691+0000", 2024-09-18T11:31:40.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5978000000000001e-05 2024-09-18T11:31:40.696 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.696 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:40.696 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.696 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.697 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2955 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:40.697 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:31:40.697 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 16.576691812, 2024-09-18T11:31:40.697 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010869531, 2024-09-18T11:31:40.697 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:40.697 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:40.697 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:40.697 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:40.697 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.697 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2955 2024-09-18T11:31:40.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.697 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:40.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.698 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:40.698 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:31:40.698 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.698 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.698 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:40.698 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:31:40.698 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.698 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.698 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:40.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027190+0000", 2024-09-18T11:31:40.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:40.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.699 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.699 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:40.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027196+0000", 2024-09-18T11:31:40.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5590000000000002e-06 2024-09-18T11:31:40.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.699 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.699 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:40.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027208+0000", 2024-09-18T11:31:40.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1816e-05 2024-09-18T11:31:40.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.700 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.700 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027215+0000", 2024-09-18T11:31:40.700 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4509999999999997e-06 2024-09-18T11:31:40.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.700 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.700 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027725+0000", 2024-09-18T11:31:40.700 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000510053 2024-09-18T11:31:40.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.700 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:40.700 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027803+0000", 2024-09-18T11:31:40.701 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8750000000000003e-05 2024-09-18T11:31:40.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.701 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032412+0000", 2024-09-18T11:31:40.701 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046089030000000001 2024-09-18T11:31:40.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.701 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:40.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032499+0000", 2024-09-18T11:31:40.701 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6316000000000005e-05 2024-09-18T11:31:40.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.702 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:40.702 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038061+0000", 2024-09-18T11:31:40.702 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005562259 2024-09-18T11:31:40.702 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.702 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:40.702 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.702 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.702 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2956 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~2100908,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:40.702 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:40.702 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 16.576657991000001, 2024-09-18T11:31:40.702 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011863831, 2024-09-18T11:31:40.702 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:40.703 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:40.703 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:40.703 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:40.703 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.703 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2956 2024-09-18T11:31:40.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.703 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:40.703 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.703 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:40.703 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:40.703 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.703 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.704 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:40.704 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:40.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.704 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.704 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:40.704 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:40.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:40.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.704 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.704 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:40.704 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027228+0000", 2024-09-18T11:31:40.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:31:40.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.705 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:40.705 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027235+0000", 2024-09-18T11:31:40.705 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:31:40.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.705 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.705 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027239+0000", 2024-09-18T11:31:40.705 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.535e-06 2024-09-18T11:31:40.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027831+0000", 2024-09-18T11:31:40.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059244699999999996 2024-09-18T11:31:40.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:40.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027879+0000", 2024-09-18T11:31:40.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:31:40.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038098+0000", 2024-09-18T11:31:40.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010219137 2024-09-18T11:31:40.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:40.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038207+0000", 2024-09-18T11:31:40.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010951999999999999 2024-09-18T11:31:40.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:40.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039089+0000", 2024-09-18T11:31:40.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00088151699999999996 2024-09-18T11:31:40.707 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.707 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:40.707 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.708 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.708 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2957 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:40.708 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:40.708 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 16.576640207000001, 2024-09-18T11:31:40.708 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012041979, 2024-09-18T11:31:40.708 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:40.708 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:40.708 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:40.708 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:40.708 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.709 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2957 2024-09-18T11:31:40.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.709 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:40.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.709 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:40.709 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:40.709 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.709 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:40.709 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:40.709 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.710 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:40.710 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:40.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:40.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.710 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:40.710 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027245+0000", 2024-09-18T11:31:40.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:31:40.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.710 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:40.710 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027251+0000", 2024-09-18T11:31:40.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0090000000000004e-06 2024-09-18T11:31:40.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027255+0000", 2024-09-18T11:31:40.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:31:40.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027895+0000", 2024-09-18T11:31:40.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064076400000000005 2024-09-18T11:31:40.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:40.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027930+0000", 2024-09-18T11:31:40.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4334e-05 2024-09-18T11:31:40.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.712 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039111+0000", 2024-09-18T11:31:40.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181455999999999 2024-09-18T11:31:40.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.712 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:40.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039156+0000", 2024-09-18T11:31:40.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5256000000000002e-05 2024-09-18T11:31:40.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:40.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039285+0000", 2024-09-18T11:31:40.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012839200000000001 2024-09-18T11:31:40.713 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.713 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:40.713 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.713 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3220 3.3 3:d97f8959:::efae800be116a8f4318957447289bc2a2c2d33d65c7604341c6a4a86aad75d7e:head [write 0~1337344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-18T11:31:40.713 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:37.110836+0000", 2024-09-18T11:31:40.713 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.4930469569999998, 2024-09-18T11:31:40.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089207350000000008, 2024-09-18T11:31:40.714 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:40.714 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:40.714 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:40.714 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:40.714 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:40.714 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3220 2024-09-18T11:31:40.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.714 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:40.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.714 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:40.714 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.110836+0000", 2024-09-18T11:31:40.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.715 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:40.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.110836+0000", 2024-09-18T11:31:40.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:40.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.715 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:40.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.110834+0000", 2024-09-18T11:31:40.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:40.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.716 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:40.716 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114557+0000", 2024-09-18T11:31:40.716 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003723166 2024-09-18T11:31:40.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.716 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:40.716 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114576+0000", 2024-09-18T11:31:40.716 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.861e-05 2024-09-18T11:31:40.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.716 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:40.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114585+0000", 2024-09-18T11:31:40.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0780000000000002e-06 2024-09-18T11:31:40.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.717 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:40.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114630+0000", 2024-09-18T11:31:40.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5118999999999999e-05 2024-09-18T11:31:40.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.717 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:40.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114748+0000", 2024-09-18T11:31:40.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011817899999999999 2024-09-18T11:31:40.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 6", 2024-09-18T11:31:40.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114862+0000", 2024-09-18T11:31:40.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113978 2024-09-18T11:31:40.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:40.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.116796+0000", 2024-09-18T11:31:40.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001934343 2024-09-18T11:31:40.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:40.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.119693+0000", 2024-09-18T11:31:40.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028964820000000001 2024-09-18T11:31:40.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:40.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.119729+0000", 2024-09-18T11:31:40.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6022000000000002e-05 2024-09-18T11:31:40.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:40.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:40.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:40.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.119757+0000", 2024-09-18T11:31:40.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7880000000000001e-05 2024-09-18T11:31:40.719 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.720 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:40.720 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.720 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:40.720 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:40.720 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:40.721 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:31:40.722 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:40.720+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:40.750 INFO:tasks.rados.rados.0.smithi119.stdout:1504: finishing write tid 1 to smithi11939506-37 2024-09-18T11:31:40.750 INFO:tasks.rados.rados.0.smithi119.stdout:1504: finishing write tid 2 to smithi11939506-37 2024-09-18T11:31:40.750 INFO:tasks.rados.rados.0.smithi119.stdout:1504: finishing write tid 3 to smithi11939506-37 2024-09-18T11:31:40.750 INFO:tasks.rados.rados.0.smithi119.stdout:1504: finishing write tid 4 to smithi11939506-37 2024-09-18T11:31:40.750 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 227 (ObjNum 284 snap 39 seq_num 284) dirty exists 2024-09-18T11:31:40.750 INFO:tasks.rados.rados.0.smithi119.stdout:1504: left oid 37 (ObjNum 284 snap 39 seq_num 284) 2024-09-18T11:31:40.750 INFO:tasks.rados.rados.0.smithi119.stdout:1505: finishing write tid 1 to smithi11939506-36 2024-09-18T11:31:40.751 INFO:tasks.rados.rados.0.smithi119.stdout:1508: got (22) Invalid argument 2024-09-18T11:31:40.751 INFO:tasks.rados.rados.0.smithi119.stdout:1505: finishing write tid 2 to smithi11939506-36 2024-09-18T11:31:40.751 INFO:tasks.rados.rados.0.smithi119.stdout:1505: finishing write tid 3 to smithi11939506-36 2024-09-18T11:31:40.751 INFO:tasks.rados.rados.0.smithi119.stdout:1505: finishing write tid 4 to smithi11939506-36 2024-09-18T11:31:40.751 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 36 v 167 (ObjNum 285 snap 39 seq_num 285) dirty exists 2024-09-18T11:31:40.751 INFO:tasks.rados.rados.0.smithi119.stdout:1505: left oid 36 (ObjNum 285 snap 39 seq_num 285) 2024-09-18T11:31:40.751 INFO:tasks.rados.rados.0.smithi119.stdout:1507: finishing rollback tid 0 to smithi11939506-32 2024-09-18T11:31:40.751 INFO:tasks.rados.rados.0.smithi119.stdout:1507: finishing rollback tid 1 to smithi11939506-32 2024-09-18T11:31:40.751 INFO:tasks.rados.rados.0.smithi119.stdout:1506: expect (ObjNum 155 snap 20 seq_num 155) 2024-09-18T11:31:40.822 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:40.819+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:40.859 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:40.859 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:40.859 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:40.859 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:40.867 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:31:40.889 INFO:tasks.rados.rados.0.smithi119.stdout:1507: finishing rollback tid 2 to smithi11939506-32 2024-09-18T11:31:40.889 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 158 (ObjNum 217 snap 26 seq_num 217) dirty exists 2024-09-18T11:31:40.922 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:40.921+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:40.965 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:40.965 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:40.965 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:40.966 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:40.966 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:40.974 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:31:41.058 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:41.057+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:41.107 INFO:tasks.rados.rados.0.smithi119.stdout:1495: done (14 left) 2024-09-18T11:31:41.107 INFO:tasks.rados.rados.0.smithi119.stdout:1496: done (13 left) 2024-09-18T11:31:41.107 INFO:tasks.rados.rados.0.smithi119.stdout:1497: done (12 left) 2024-09-18T11:31:41.107 INFO:tasks.rados.rados.0.smithi119.stdout:1498: done (11 left) 2024-09-18T11:31:41.107 INFO:tasks.rados.rados.0.smithi119.stdout:1499: done (10 left) 2024-09-18T11:31:41.107 INFO:tasks.rados.rados.0.smithi119.stdout:1500: done (9 left) 2024-09-18T11:31:41.107 INFO:tasks.rados.rados.0.smithi119.stdout:1501: done (8 left) 2024-09-18T11:31:41.107 INFO:tasks.rados.rados.0.smithi119.stdout:1502: done (7 left) 2024-09-18T11:31:41.107 INFO:tasks.rados.rados.0.smithi119.stdout:1503: done (6 left) 2024-09-18T11:31:41.107 INFO:tasks.rados.rados.0.smithi119.stdout:1504: done (5 left) 2024-09-18T11:31:41.107 INFO:tasks.rados.rados.0.smithi119.stdout:1505: done (4 left) 2024-09-18T11:31:41.108 INFO:tasks.rados.rados.0.smithi119.stdout:1506: done (3 left) 2024-09-18T11:31:41.108 INFO:tasks.rados.rados.0.smithi119.stdout:1507: done (2 left) 2024-09-18T11:31:41.108 INFO:tasks.rados.rados.0.smithi119.stdout:1508: done (1 left) 2024-09-18T11:31:41.108 INFO:tasks.rados.rados.0.smithi119.stdout:1509: done (0 left) 2024-09-18T11:31:41.108 INFO:tasks.rados.rados.0.smithi119.stdout:1510: snap_create 2024-09-18T11:31:41.114 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:41.114 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:41.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:41.114 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:41.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.115 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:31:41.115 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:41.115 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 111.085848279, 2024-09-18T11:31:41.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073601550000000002, 2024-09-18T11:31:41.115 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.115 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:41.115 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.115 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:31:41.115 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:31:41.115 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1 2024-09-18T11:31:41.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.115 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:41.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:41.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026524+0000", 2024-09-18T11:31:41.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999905 2024-09-18T11:31:41.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026541+0000", 2024-09-18T11:31:41.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7070000000000001e-05 2024-09-18T11:31:41.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026554+0000", 2024-09-18T11:31:41.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.288e-05 2024-09-18T11:31:41.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026564+0000", 2024-09-18T11:31:41.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0197000000000001e-05 2024-09-18T11:31:41.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.118 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026826+0000", 2024-09-18T11:31:41.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026212400000000003 2024-09-18T11:31:41.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.118 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:31:41.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026839+0000", 2024-09-18T11:31:41.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2831e-05 2024-09-18T11:31:41.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.119 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.119 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028605+0000", 2024-09-18T11:31:41.119 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001765794 2024-09-18T11:31:41.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.119 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.119 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028709+0000", 2024-09-18T11:31:41.119 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104348 2024-09-18T11:31:41.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.119 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:41.119 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028824+0000", 2024-09-18T11:31:41.119 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011527399999999999 2024-09-18T11:31:41.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:41.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030319+0000", 2024-09-18T11:31:41.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001494439 2024-09-18T11:31:41.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:41.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033805+0000", 2024-09-18T11:31:41.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034857820000000002 2024-09-18T11:31:41.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:41.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033856+0000", 2024-09-18T11:31:41.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1687999999999997e-05 2024-09-18T11:31:41.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033894+0000", 2024-09-18T11:31:41.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7481000000000002e-05 2024-09-18T11:31:41.121 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.121 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.121 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.122 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1118 3.2 e38/21 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head v 38'76, mlcod=38'74)", 2024-09-18T11:31:41.122 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:41.122 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 81.164438126999997, 2024-09-18T11:31:41.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006048688, 2024-09-18T11:31:41.122 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.122 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:41.122 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.122 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.122 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:41.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.122 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:41.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947941+0000", 2024-09-18T11:31:41.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:41.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949868+0000", 2024-09-18T11:31:41.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019267659999999999 2024-09-18T11:31:41.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.124 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.124 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949893+0000", 2024-09-18T11:31:41.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5239999999999999e-05 2024-09-18T11:31:41.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.124 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.124 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949901+0000", 2024-09-18T11:31:41.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8420000000000005e-06 2024-09-18T11:31:41.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.124 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.124 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949954+0000", 2024-09-18T11:31:41.125 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3464999999999997e-05 2024-09-18T11:31:41.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.125 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.125 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949974+0000", 2024-09-18T11:31:41.125 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9731000000000001e-05 2024-09-18T11:31:41.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.125 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:41.125 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953948+0000", 2024-09-18T11:31:41.125 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039735489999999998 2024-09-18T11:31:41.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.125 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.126 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953992+0000", 2024-09-18T11:31:41.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4700999999999998e-05 2024-09-18T11:31:41.126 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.126 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.126 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.126 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1662 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'73, mlcod=51'69)", 2024-09-18T11:31:41.126 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:41.126 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 63.750833092000001, 2024-09-18T11:31:41.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059408539999999998, 2024-09-18T11:31:41.126 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.126 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:41.126 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.127 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:41.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.127 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:41.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.127 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:41.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361914+0000", 2024-09-18T11:31:41.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036569 2024-09-18T11:31:41.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361926+0000", 2024-09-18T11:31:41.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1960999999999999e-05 2024-09-18T11:31:41.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361937+0000", 2024-09-18T11:31:41.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0876999999999999e-05 2024-09-18T11:31:41.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361986+0000", 2024-09-18T11:31:41.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8652000000000001e-05 2024-09-18T11:31:41.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.362012+0000", 2024-09-18T11:31:41.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5854999999999999e-05 2024-09-18T11:31:41.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:41.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367460+0000", 2024-09-18T11:31:41.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054482100000000002 2024-09-18T11:31:41.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.130 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367490+0000", 2024-09-18T11:31:41.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9909000000000001e-05 2024-09-18T11:31:41.130 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.130 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.130 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.130 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:269 4.7 e58/22 4:e73ba68f:::smithi11939506-1:13 v 58'72, mlcod=57'69)", 2024-09-18T11:31:41.131 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:41.131 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 61.847001194999997, 2024-09-18T11:31:41.131 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065512569999999996, 2024-09-18T11:31:41.131 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.131 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:41.131 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.131 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.131 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:41.131 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.131 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.131 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:41.132 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.132 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.132 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265380+0000", 2024-09-18T11:31:41.132 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.132 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.132 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265388+0000", 2024-09-18T11:31:41.132 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.174e-06 2024-09-18T11:31:41.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265398+0000", 2024-09-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2599999999999994e-06 2024-09-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265402+0000", 2024-09-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0289999999999999e-06 2024-09-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265434+0000", 2024-09-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2511000000000001e-05 2024-09-18T11:31:41.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265447+0000", 2024-09-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2947e-05 2024-09-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271892+0000", 2024-09-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064445930000000002 2024-09-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271932+0000", 2024-09-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0303999999999999e-05 2024-09-18T11:31:41.134 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1788 4.8 4:1898b15d:::smithi11939506-25:head [write 453822~797061] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 58.196568612, 2024-09-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094305159999999999, 2024-09-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.135 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1788 2024-09-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.137 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.137 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915811+0000", 2024-09-18T11:31:41.137 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:41.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.137 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.137 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.137 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916608+0000", 2024-09-18T11:31:41.137 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079635000000000003 2024-09-18T11:31:41.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.137 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.137 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.137 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916624+0000", 2024-09-18T11:31:41.137 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6735000000000002e-05 2024-09-18T11:31:41.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.138 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916630+0000", 2024-09-18T11:31:41.138 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5709999999999997e-06 2024-09-18T11:31:41.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.138 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916674+0000", 2024-09-18T11:31:41.138 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4540999999999999e-05 2024-09-18T11:31:41.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.138 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916751+0000", 2024-09-18T11:31:41.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6695999999999999e-05 2024-09-18T11:31:41.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.139 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:41.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916872+0000", 2024-09-18T11:31:41.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120648 2024-09-18T11:31:41.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.139 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:41.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.919076+0000", 2024-09-18T11:31:41.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022046829999999998 2024-09-18T11:31:41.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.139 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:41.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925151+0000", 2024-09-18T11:31:41.140 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060740239999999999 2024-09-18T11:31:41.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.140 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:41.140 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925198+0000", 2024-09-18T11:31:41.140 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7896999999999999e-05 2024-09-18T11:31:41.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.140 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.140 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925244+0000", 2024-09-18T11:31:41.140 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5398999999999997e-05 2024-09-18T11:31:41.140 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.140 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.141 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.141 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1934 4.2 e65/22 4:425afe9c:::smithi11939506-29:head v 65'44, mlcod=60'40)", 2024-09-18T11:31:41.141 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:41.141 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 52.812389842000002, 2024-09-18T11:31:41.141 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064585119999999996, 2024-09-18T11:31:41.141 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.141 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:41.141 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.141 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.141 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:41.141 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.142 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:41.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.142 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299990+0000", 2024-09-18T11:31:41.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:41.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.142 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300789+0000", 2024-09-18T11:31:41.143 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079851499999999997 2024-09-18T11:31:41.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.143 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.143 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300806+0000", 2024-09-18T11:31:41.143 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7096999999999999e-05 2024-09-18T11:31:41.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.143 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.143 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300813+0000", 2024-09-18T11:31:41.143 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5079999999999999e-06 2024-09-18T11:31:41.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.144 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.144 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300870+0000", 2024-09-18T11:31:41.144 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7178e-05 2024-09-18T11:31:41.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.144 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.144 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300897+0000", 2024-09-18T11:31:41.144 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6284000000000001e-05 2024-09-18T11:31:41.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.144 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:41.144 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306410+0000", 2024-09-18T11:31:41.144 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055135210000000004 2024-09-18T11:31:41.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.145 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.145 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306451+0000", 2024-09-18T11:31:41.145 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0379000000000001e-05 2024-09-18T11:31:41.145 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.145 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.145 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.145 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2119 4.7 e73/22 4:e0d95e10:::smithi11939506-22:head v 73'89, mlcod=73'86)", 2024-09-18T11:31:41.145 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:55.701097+0000", 2024-09-18T11:31:41.145 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.411285196000001, 2024-09-18T11:31:41.145 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056939520000000004, 2024-09-18T11:31:41.146 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.146 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:41.146 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.146 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.146 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.146 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701097+0000", 2024-09-18T11:31:41.146 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.146 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.146 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.146 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701097+0000", 2024-09-18T11:31:41.146 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.147 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.147 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701095+0000", 2024-09-18T11:31:41.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:41.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.147 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.147 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701919+0000", 2024-09-18T11:31:41.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000824181 2024-09-18T11:31:41.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.147 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.147 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701938+0000", 2024-09-18T11:31:41.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8614000000000002e-05 2024-09-18T11:31:41.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701947+0000", 2024-09-18T11:31:41.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0699999999999996e-06 2024-09-18T11:31:41.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.701992+0000", 2024-09-18T11:31:41.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5614000000000001e-05 2024-09-18T11:31:41.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.702012+0000", 2024-09-18T11:31:41.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9639999999999998e-05 2024-09-18T11:31:41.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.149 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:41.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.706755+0000", 2024-09-18T11:31:41.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004742708 2024-09-18T11:31:41.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.149 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:55.706791+0000", 2024-09-18T11:31:41.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6106999999999998e-05 2024-09-18T11:31:41.149 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.149 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.150 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.150 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.150 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2452 4.3 4:ca2b2309:::smithi11939506-44:head [read 0~3462242,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:31:41.150 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:31:41.150 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.075614485000003, 2024-09-18T11:31:41.150 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060906450000000004, 2024-09-18T11:31:41.150 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.150 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:41.150 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.150 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.150 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.150 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2452 2024-09-18T11:31:41.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.151 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.151 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.151 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.151 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:31:41.151 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.151 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.151 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.151 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:31:41.151 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.151 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.151 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.151 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036765+0000", 2024-09-18T11:31:41.152 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:41.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.152 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.152 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.152 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036777+0000", 2024-09-18T11:31:41.152 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1422e-05 2024-09-18T11:31:41.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.152 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.152 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.152 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036793+0000", 2024-09-18T11:31:41.152 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6575999999999999e-05 2024-09-18T11:31:41.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.152 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.153 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036800+0000", 2024-09-18T11:31:41.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9940000000000003e-06 2024-09-18T11:31:41.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.153 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:41.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5847999999999999e-05 2024-09-18T11:31:41.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.153 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:41.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036853+0000", 2024-09-18T11:31:41.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0119999999999999e-06 2024-09-18T11:31:41.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038527+0000", 2024-09-18T11:31:41.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016732730000000001 2024-09-18T11:31:41.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038580+0000", 2024-09-18T11:31:41.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3443999999999999e-05 2024-09-18T11:31:41.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042858+0000", 2024-09-18T11:31:41.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042781049999999999 2024-09-18T11:31:41.155 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.155 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.155 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.155 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2453 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:31:41.155 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:31:41.155 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.075572252999997, 2024-09-18T11:31:41.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063798429999999996, 2024-09-18T11:31:41.155 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.155 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:41.155 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.155 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.155 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.156 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2453 2024-09-18T11:31:41.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.156 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.156 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:31:41.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.156 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:31:41.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.157 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.157 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036809+0000", 2024-09-18T11:31:41.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.157 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.157 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036812+0000", 2024-09-18T11:31:41.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8339999999999999e-06 2024-09-18T11:31:41.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.157 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.157 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036819+0000", 2024-09-18T11:31:41.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9890000000000003e-06 2024-09-18T11:31:41.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.158 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.158 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036823+0000", 2024-09-18T11:31:41.158 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8099999999999999e-06 2024-09-18T11:31:41.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.158 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.158 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036912+0000", 2024-09-18T11:31:41.158 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9083000000000001e-05 2024-09-18T11:31:41.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.158 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:41.158 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036932+0000", 2024-09-18T11:31:41.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0222000000000001e-05 2024-09-18T11:31:41.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.159 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042896+0000", 2024-09-18T11:31:41.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059637240000000001 2024-09-18T11:31:41.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.159 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042952+0000", 2024-09-18T11:31:41.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5877000000000001e-05 2024-09-18T11:31:41.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.159 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.160 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043190+0000", 2024-09-18T11:31:41.160 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00023774500000000001 2024-09-18T11:31:41.160 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.160 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.160 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.160 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2454 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:31:41.160 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:41.160 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.075536296999999, 2024-09-18T11:31:41.160 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066404469999999998, 2024-09-18T11:31:41.160 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.160 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:41.160 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.161 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.161 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.161 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2454 2024-09-18T11:31:41.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.161 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:41.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:41.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036845+0000", 2024-09-18T11:31:41.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036856+0000", 2024-09-18T11:31:41.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1338000000000001e-05 2024-09-18T11:31:41.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.163 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036863+0000", 2024-09-18T11:31:41.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4749999999999998e-06 2024-09-18T11:31:41.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.163 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.163 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036867+0000", 2024-09-18T11:31:41.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3839999999999999e-06 2024-09-18T11:31:41.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.163 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.163 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036946+0000", 2024-09-18T11:31:41.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9054000000000006e-05 2024-09-18T11:31:41.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.164 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:41.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036949+0000", 2024-09-18T11:31:41.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9390000000000002e-06 2024-09-18T11:31:41.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.164 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043213+0000", 2024-09-18T11:31:41.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062640090000000001 2024-09-18T11:31:41.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.164 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043261+0000", 2024-09-18T11:31:41.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7750999999999997e-05 2024-09-18T11:31:41.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043486+0000", 2024-09-18T11:31:41.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000224985 2024-09-18T11:31:41.165 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.165 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.165 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.165 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2553 4.3 4:ca2b2309:::smithi11939506-44:head [write 564768~630013,stat] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-18T11:31:41.165 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:31:41.165 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 30.915258664, 2024-09-18T11:31:41.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065686879999999996, 2024-09-18T11:31:41.166 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.166 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:41.166 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.166 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.166 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.166 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2553 2024-09-18T11:31:41.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.166 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:31:41.166 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:31:41.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197120+0000", 2024-09-18T11:31:41.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:41.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199000+0000", 2024-09-18T11:31:41.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018792080000000001 2024-09-18T11:31:41.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199044+0000", 2024-09-18T11:31:41.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4048999999999998e-05 2024-09-18T11:31:41.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199071+0000", 2024-09-18T11:31:41.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7288e-05 2024-09-18T11:31:41.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199119+0000", 2024-09-18T11:31:41.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8000999999999997e-05 2024-09-18T11:31:41.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199208+0000", 2024-09-18T11:31:41.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9209000000000006e-05 2024-09-18T11:31:41.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:41.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199385+0000", 2024-09-18T11:31:41.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000177095 2024-09-18T11:31:41.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:41.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.202282+0000", 2024-09-18T11:31:41.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028970469999999998 2024-09-18T11:31:41.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:41.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203578+0000", 2024-09-18T11:31:41.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012960490000000001 2024-09-18T11:31:41.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:41.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203638+0000", 2024-09-18T11:31:41.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9599000000000003e-05 2024-09-18T11:31:41.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203692+0000", 2024-09-18T11:31:41.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3908e-05 2024-09-18T11:31:41.171 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.171 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.171 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.171 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2751 4.3 4:ca2b2309:::smithi11939506-44:25 [read 0~3210104,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:31:41.171 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:17.886856+0000", 2024-09-18T11:31:41.171 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.225525523000002, 2024-09-18T11:31:41.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057395420000000003, 2024-09-18T11:31:41.171 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.172 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:41.172 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.172 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.172 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.172 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2751 2024-09-18T11:31:41.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.172 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.172 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.172 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886856+0000", 2024-09-18T11:31:41.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.173 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886856+0000", 2024-09-18T11:31:41.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.173 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886856+0000", 2024-09-18T11:31:41.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.173 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886859+0000", 2024-09-18T11:31:41.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.886e-06 2024-09-18T11:31:41.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.174 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.174 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886871+0000", 2024-09-18T11:31:41.174 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2249000000000001e-05 2024-09-18T11:31:41.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.174 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.174 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886874+0000", 2024-09-18T11:31:41.174 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2160000000000002e-06 2024-09-18T11:31:41.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.174 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.174 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.890901+0000", 2024-09-18T11:31:41.174 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040264860000000001 2024-09-18T11:31:41.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.175 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.175 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.891032+0000", 2024-09-18T11:31:41.175 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131438 2024-09-18T11:31:41.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.175 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.175 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.892596+0000", 2024-09-18T11:31:41.175 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015636350000000001 2024-09-18T11:31:41.175 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.175 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.175 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.176 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2752 4.3 4:ca2b2309:::smithi11939506-44:25 [read 0~3210104,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:31:41.176 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:31:41.176 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.225489503999999, 2024-09-18T11:31:41.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068715460000000001, 2024-09-18T11:31:41.176 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.176 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:41.176 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.176 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.176 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.176 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2752 2024-09-18T11:31:41.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.176 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:31:41.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:31:41.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:31:41.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886895+0000", 2024-09-18T11:31:41.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.993e-06 2024-09-18T11:31:41.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886901+0000", 2024-09-18T11:31:41.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1859999999999997e-06 2024-09-18T11:31:41.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886904+0000", 2024-09-18T11:31:41.179 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1920000000000001e-06 2024-09-18T11:31:41.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.179 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.179 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.179 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.892622+0000", 2024-09-18T11:31:41.179 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057175660000000003 2024-09-18T11:31:41.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.179 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.179 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.179 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.892693+0000", 2024-09-18T11:31:41.179 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1265000000000003e-05 2024-09-18T11:31:41.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.179 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.179 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.180 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.893764+0000", 2024-09-18T11:31:41.180 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001070753 2024-09-18T11:31:41.180 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.180 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.180 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.180 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.180 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.180 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2793 4.1 e92/22 4:87078dfe:::smithi11939506-14:head v 92'195, mlcod=92'191)", 2024-09-18T11:31:41.180 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:31:41.180 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.887589470000002, 2024-09-18T11:31:41.180 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062761989999999997, 2024-09-18T11:31:41.180 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.181 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:41.181 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.181 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:31:41.181 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.181 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:31:41.181 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.181 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.182 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:31:41.182 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:41.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.182 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.182 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.182 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224994+0000", 2024-09-18T11:31:41.182 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020197600000000001 2024-09-18T11:31:41.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.182 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.182 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.182 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225007+0000", 2024-09-18T11:31:41.182 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3791999999999999e-05 2024-09-18T11:31:41.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.183 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.183 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.183 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225013+0000", 2024-09-18T11:31:41.183 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3630000000000002e-06 2024-09-18T11:31:41.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.183 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.183 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.183 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225062+0000", 2024-09-18T11:31:41.183 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8925000000000003e-05 2024-09-18T11:31:41.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.183 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.183 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.183 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225086+0000", 2024-09-18T11:31:41.183 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4737999999999999e-05 2024-09-18T11:31:41.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.184 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.184 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:41.184 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231022+0000", 2024-09-18T11:31:41.184 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059360690000000004 2024-09-18T11:31:41.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.184 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.184 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.184 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231069+0000", 2024-09-18T11:31:41.184 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6252999999999999e-05 2024-09-18T11:31:41.184 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.184 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.184 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.185 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.185 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2876 4.3 4:cc46532a:::smithi11939506-33:2f [read 0~3680733,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:41.185 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:31:41.185 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.605520569999999, 2024-09-18T11:31:41.185 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068391210000000001, 2024-09-18T11:31:41.185 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.185 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:41.185 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.185 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.185 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.185 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2876 2024-09-18T11:31:41.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.186 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.186 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:31:41.186 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.186 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:31:41.186 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.186 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506860+0000", 2024-09-18T11:31:41.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:41.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506869+0000", 2024-09-18T11:31:41.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5539999999999998e-06 2024-09-18T11:31:41.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506880+0000", 2024-09-18T11:31:41.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1552999999999999e-05 2024-09-18T11:31:41.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506887+0000", 2024-09-18T11:31:41.188 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7329999999999997e-06 2024-09-18T11:31:41.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.188 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.188 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506937+0000", 2024-09-18T11:31:41.188 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0317e-05 2024-09-18T11:31:41.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.188 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.188 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:41.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506945+0000", 2024-09-18T11:31:41.188 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1550000000000007e-06 2024-09-18T11:31:41.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.188 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.189 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.189 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.509021+0000", 2024-09-18T11:31:41.189 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020759870000000001 2024-09-18T11:31:41.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.189 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.189 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.189 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.509123+0000", 2024-09-18T11:31:41.189 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010132900000000001 2024-09-18T11:31:41.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.189 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.189 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.189 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513700+0000", 2024-09-18T11:31:41.189 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0045777320000000002 2024-09-18T11:31:41.190 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.190 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.190 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.190 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.190 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2877 4.3 4:cc46532a:::smithi11939506-33:2f [read 0~3680733,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:41.190 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:31:41.190 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.605486156000001, 2024-09-18T11:31:41.190 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085715170000000007, 2024-09-18T11:31:41.190 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.190 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:41.190 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.190 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.190 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.191 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2877 2024-09-18T11:31:41.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.191 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.191 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.191 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.191 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:31:41.191 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.191 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.191 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.191 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:31:41.191 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.191 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.192 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.192 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:31:41.192 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.192 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.192 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.192 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506898+0000", 2024-09-18T11:31:41.192 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4760000000000001e-06 2024-09-18T11:31:41.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.192 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.192 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.192 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506903+0000", 2024-09-18T11:31:41.192 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:41.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.193 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.193 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.193 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506906+0000", 2024-09-18T11:31:41.193 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1769999999999998e-06 2024-09-18T11:31:41.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.193 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.193 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.193 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507007+0000", 2024-09-18T11:31:41.193 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101479 2024-09-18T11:31:41.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.193 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.193 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:41.193 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507015+0000", 2024-09-18T11:31:41.194 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8490000000000006e-06 2024-09-18T11:31:41.194 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.194 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.194 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.194 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513757+0000", 2024-09-18T11:31:41.194 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0067417800000000002 2024-09-18T11:31:41.194 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.194 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.194 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.194 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513867+0000", 2024-09-18T11:31:41.194 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000109987 2024-09-18T11:31:41.194 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.194 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.194 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.195 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515467+0000", 2024-09-18T11:31:41.195 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00160037 2024-09-18T11:31:41.195 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.195 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.195 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.195 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.195 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2878 4.3 4:cc46532a:::smithi11939506-33:2f [sparse-read 0~1770976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:41.195 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:31:41.195 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.605455701, 2024-09-18T11:31:41.195 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090602640000000002, 2024-09-18T11:31:41.195 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.195 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:41.195 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.196 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.196 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.196 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2878 2024-09-18T11:31:41.196 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.196 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.196 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.196 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.196 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:31:41.196 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.196 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.196 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.196 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.196 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:31:41.197 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.197 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.197 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.197 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.197 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:31:41.197 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.197 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.197 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.197 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.197 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506929+0000", 2024-09-18T11:31:41.197 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.163e-06 2024-09-18T11:31:41.197 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.197 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.198 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.198 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506936+0000", 2024-09-18T11:31:41.198 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5519999999999999e-06 2024-09-18T11:31:41.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.198 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.198 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.198 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506941+0000", 2024-09-18T11:31:41.198 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:41.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.198 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.198 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.198 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507036+0000", 2024-09-18T11:31:41.198 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5070999999999998e-05 2024-09-18T11:31:41.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.199 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.199 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:41.199 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507042+0000", 2024-09-18T11:31:41.199 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3959999999999997e-06 2024-09-18T11:31:41.199 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.199 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.199 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.199 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515504+0000", 2024-09-18T11:31:41.199 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084620300000000006 2024-09-18T11:31:41.199 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.199 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.199 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.199 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515625+0000", 2024-09-18T11:31:41.199 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012150999999999999 2024-09-18T11:31:41.200 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.200 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.200 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.200 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515986+0000", 2024-09-18T11:31:41.200 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036140100000000002 2024-09-18T11:31:41.200 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.200 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.200 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.200 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.200 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.200 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:686 4.7 e98/22 4:e5707c18:::smithi11939506-10:30 v 98'160, mlcod=97'155)", 2024-09-18T11:31:41.200 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:31:41.200 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.093614011, 2024-09-18T11:31:41.201 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074641239999999999, 2024-09-18T11:31:41.201 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.201 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:41.201 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.201 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.201 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.201 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:31:41.201 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.201 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.201 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.201 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:31:41.201 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.202 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.202 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.202 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018767+0000", 2024-09-18T11:31:41.202 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.202 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.202 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.202 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018773+0000", 2024-09-18T11:31:41.202 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1460000000000004e-06 2024-09-18T11:31:41.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.202 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.202 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.202 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018781+0000", 2024-09-18T11:31:41.203 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9780000000000005e-06 2024-09-18T11:31:41.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.203 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.203 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.203 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018785+0000", 2024-09-18T11:31:41.203 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0029999999999996e-06 2024-09-18T11:31:41.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.203 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.203 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.203 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018826+0000", 2024-09-18T11:31:41.203 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1616999999999998e-05 2024-09-18T11:31:41.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.203 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.203 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.204 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018843+0000", 2024-09-18T11:31:41.204 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6753999999999999e-05 2024-09-18T11:31:41.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.204 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.204 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:41.204 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026202+0000", 2024-09-18T11:31:41.204 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073592620000000001 2024-09-18T11:31:41.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.204 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.204 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.204 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026232+0000", 2024-09-18T11:31:41.204 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9779000000000001e-05 2024-09-18T11:31:41.204 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.204 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.205 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.205 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.205 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.205 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.4.0:621 4.1 e98/22 4:854aa293:::smithi11939506-26:35 v 98'215, mlcod=96'210)", 2024-09-18T11:31:41.205 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:31:41.205 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.092730572000001, 2024-09-18T11:31:41.205 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007014752, 2024-09-18T11:31:41.205 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.205 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:41.205 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.205 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.205 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.205 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:31:41.206 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.206 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.206 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.206 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.206 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:31:41.206 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.206 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.206 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.206 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.206 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:31:41.206 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.206 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.206 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.206 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.207 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019659+0000", 2024-09-18T11:31:41.207 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8229999999999995e-06 2024-09-18T11:31:41.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.207 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.207 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.207 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019671+0000", 2024-09-18T11:31:41.207 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2636000000000001e-05 2024-09-18T11:31:41.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.207 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.207 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.207 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019677+0000", 2024-09-18T11:31:41.207 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5470000000000004e-06 2024-09-18T11:31:41.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.207 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.208 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.208 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019710+0000", 2024-09-18T11:31:41.208 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3191999999999997e-05 2024-09-18T11:31:41.208 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.208 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.208 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.208 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019722+0000", 2024-09-18T11:31:41.208 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2357e-05 2024-09-18T11:31:41.208 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.208 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.208 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:41.208 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026639+0000", 2024-09-18T11:31:41.208 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0069167630000000003 2024-09-18T11:31:41.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.209 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.209 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.209 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026666+0000", 2024-09-18T11:31:41.209 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7087999999999999e-05 2024-09-18T11:31:41.209 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.209 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.209 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.209 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.209 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3234 4.7 e109/22 4:e73ba68f:::smithi11939506-1:head v 109'206, mlcod=107'202)", 2024-09-18T11:31:41.209 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:37.128363+0000", 2024-09-18T11:31:41.209 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.984019188, 2024-09-18T11:31:41.209 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059367100000000004, 2024-09-18T11:31:41.210 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.210 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:41.210 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.210 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.210 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.210 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.128363+0000", 2024-09-18T11:31:41.210 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.210 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.210 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.210 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.128363+0000", 2024-09-18T11:31:41.210 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.210 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.211 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.211 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.128362+0000", 2024-09-18T11:31:41.211 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.211 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.211 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.211 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131291+0000", 2024-09-18T11:31:41.211 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029282840000000002 2024-09-18T11:31:41.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.211 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.211 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.211 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131305+0000", 2024-09-18T11:31:41.211 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4222999999999999e-05 2024-09-18T11:31:41.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.212 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.212 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.212 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131313+0000", 2024-09-18T11:31:41.212 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.208e-06 2024-09-18T11:31:41.212 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.212 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.212 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.212 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131354+0000", 2024-09-18T11:31:41.212 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1158000000000001e-05 2024-09-18T11:31:41.212 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.212 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.212 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.212 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131378+0000", 2024-09-18T11:31:41.212 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3828999999999998e-05 2024-09-18T11:31:41.213 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.213 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.213 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:41.213 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.134258+0000", 2024-09-18T11:31:41.213 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028798920000000002 2024-09-18T11:31:41.213 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.213 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.213 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.213 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.134299+0000", 2024-09-18T11:31:41.213 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1355e-05 2024-09-18T11:31:41.213 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.213 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.213 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.213 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.213 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.214 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:41.215 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:41.212+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:41.215 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:31:41.315 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:41.314+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:41.353 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:41.353 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:41.353 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:41.353 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:41.361 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:31:41.417 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:41.415+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:41.498 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:41.498 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:41.498 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:41.498 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:41.499 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:41.506 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:31:41.518 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:41.516+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:41.606 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:41.606 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:41.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:41.606 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:41.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.606 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:41.607 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:41.607 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 90.487573146000003, 2024-09-18T11:31:41.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:31:41.607 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.607 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:41.607 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.607 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.607 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.607 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:31:41.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.607 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.608 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.608 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:41.608 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.608 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.608 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:41.608 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.608 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.608 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:31:41.608 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:41.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.609 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.609 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:31:41.609 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:31:41.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.609 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.609 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:31:41.609 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:31:41.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.609 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.610 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:31:41.610 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:31:41.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.610 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.610 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.610 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:31:41.610 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:31:41.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.610 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.610 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:41.610 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:31:41.610 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:31:41.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:31:41.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:31:41.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:31:41.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:31:41.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:31:41.612 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:31:41.612 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.612 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.612 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.612 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:31:41.612 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:41.612 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 80.934547745000003, 2024-09-18T11:31:41.612 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:31:41.612 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.612 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:41.613 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.613 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.613 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.613 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:31:41.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.613 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.613 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.613 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:41.613 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.614 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:41.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.614 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:31:41.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:41.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.614 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:31:41.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:31:41.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.615 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.615 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:31:41.615 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:31:41.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.615 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.615 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:31:41.615 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:31:41.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.616 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:31:41.616 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:31:41.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.616 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:31:41.616 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:31:41.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.616 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:31:41.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:31:41.617 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.617 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.617 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.617 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1688 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 19=[19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-18T11:31:41.617 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:41.617 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 62.071393151999999, 2024-09-18T11:31:41.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071575789999999999, 2024-09-18T11:31:41.617 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.617 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:41.617 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.618 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.618 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.618 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1688 2024-09-18T11:31:41.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.618 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:41.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:41.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.619 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532499+0000", 2024-09-18T11:31:41.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:41.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.619 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532500+0000", 2024-09-18T11:31:41.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5260000000000001e-06 2024-09-18T11:31:41.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.619 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532503+0000", 2024-09-18T11:31:41.620 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:31:41.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.620 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.620 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.532505+0000", 2024-09-18T11:31:41.620 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.889e-06 2024-09-18T11:31:41.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.620 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.620 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.534722+0000", 2024-09-18T11:31:41.620 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002217119 2024-09-18T11:31:41.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.620 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.534827+0000", 2024-09-18T11:31:41.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0001049 2024-09-18T11:31:41.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:41.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.535007+0000", 2024-09-18T11:31:41.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017965599999999999 2024-09-18T11:31:41.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:41.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539463+0000", 2024-09-18T11:31:41.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004455713 2024-09-18T11:31:41.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:41.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539607+0000", 2024-09-18T11:31:41.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014440300000000001 2024-09-18T11:31:41.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:41.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539634+0000", 2024-09-18T11:31:41.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7398000000000001e-05 2024-09-18T11:31:41.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.539657+0000", 2024-09-18T11:31:41.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2228000000000001e-05 2024-09-18T11:31:41.623 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.623 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.623 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.623 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1760 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2319366,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:41.623 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:31:41.623 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 59.309898601999997, 2024-09-18T11:31:41.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070132529999999997, 2024-09-18T11:31:41.623 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.623 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:41.623 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.623 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.624 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.624 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1760 2024-09-18T11:31:41.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.624 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.624 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:31:41.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.624 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293994+0000", 2024-09-18T11:31:41.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.625 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.625 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293993+0000", 2024-09-18T11:31:41.625 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.625 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.625 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.293997+0000", 2024-09-18T11:31:41.625 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4999999999999999e-06 2024-09-18T11:31:41.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294004+0000", 2024-09-18T11:31:41.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3950000000000003e-06 2024-09-18T11:31:41.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294009+0000", 2024-09-18T11:31:41.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4989999999999996e-06 2024-09-18T11:31:41.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294103+0000", 2024-09-18T11:31:41.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4817999999999999e-05 2024-09-18T11:31:41.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:41.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.294110+0000", 2024-09-18T11:31:41.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7240000000000003e-06 2024-09-18T11:31:41.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300406+0000", 2024-09-18T11:31:41.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062964470000000002 2024-09-18T11:31:41.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.628 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.300457+0000", 2024-09-18T11:31:41.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0915000000000003e-05 2024-09-18T11:31:41.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.628 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.301007+0000", 2024-09-18T11:31:41.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00054942299999999995 2024-09-18T11:31:41.628 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.628 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.628 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1775 4.a 4:588debc2:::smithi11939506-18:head [write 1527054~766050] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:31:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 58.707545861, 2024-09-18T11:31:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070939050000000002, 2024-09-18T11:31:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1775 2024-09-18T11:31:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:31:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896346+0000", 2024-09-18T11:31:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896345+0000", 2024-09-18T11:31:41.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:41.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.896993+0000", 2024-09-18T11:31:41.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064769999999999997 2024-09-18T11:31:41.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897011+0000", 2024-09-18T11:31:41.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8355999999999998e-05 2024-09-18T11:31:41.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897016+0000", 2024-09-18T11:31:41.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3890000000000004e-06 2024-09-18T11:31:41.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.632 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897054+0000", 2024-09-18T11:31:41.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7963000000000002e-05 2024-09-18T11:31:41.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.632 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897123+0000", 2024-09-18T11:31:41.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8128999999999998e-05 2024-09-18T11:31:41.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:41.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897224+0000", 2024-09-18T11:31:41.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010168800000000001 2024-09-18T11:31:41.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:41.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899515+0000", 2024-09-18T11:31:41.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022910280000000001 2024-09-18T11:31:41.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:41.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903333+0000", 2024-09-18T11:31:41.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003817459 2024-09-18T11:31:41.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:41.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903388+0000", 2024-09-18T11:31:41.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5651e-05 2024-09-18T11:31:41.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903440+0000", 2024-09-18T11:31:41.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1836999999999999e-05 2024-09-18T11:31:41.634 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.634 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.634 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.635 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1777 4.a 4:588debc2:::smithi11939506-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:41.635 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:31:41.635 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 58.706841408999999, 2024-09-18T11:31:41.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0069010809999999999, 2024-09-18T11:31:41.635 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.635 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:41.635 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.635 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.635 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.635 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1777 2024-09-18T11:31:41.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.636 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.636 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:31:41.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.636 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897051+0000", 2024-09-18T11:31:41.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897050+0000", 2024-09-18T11:31:41.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897052+0000", 2024-09-18T11:31:41.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:31:41.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897057+0000", 2024-09-18T11:31:41.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9470000000000002e-06 2024-09-18T11:31:41.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.897060+0000", 2024-09-18T11:31:41.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3410000000000001e-06 2024-09-18T11:31:41.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899417+0000", 2024-09-18T11:31:41.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023571170000000002 2024-09-18T11:31:41.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:41.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.899501+0000", 2024-09-18T11:31:41.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4160999999999996e-05 2024-09-18T11:31:41.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.639 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903661+0000", 2024-09-18T11:31:41.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041599510000000003 2024-09-18T11:31:41.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.639 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903790+0000", 2024-09-18T11:31:41.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012941499999999999 2024-09-18T11:31:41.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.640 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.640 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.903952+0000", 2024-09-18T11:31:41.640 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016146800000000001 2024-09-18T11:31:41.640 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.640 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.640 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.640 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1895 4.5 4:a087a566:::smithi11939506-46:head [read 0~2756315,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:41.640 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:41.640 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.295585903999999, 2024-09-18T11:31:41.640 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007540377, 2024-09-18T11:31:41.640 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.641 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:41.641 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.641 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.641 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.641 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1895 2024-09-18T11:31:41.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.641 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.641 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:41.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:41.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308306+0000", 2024-09-18T11:31:41.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308310+0000", 2024-09-18T11:31:41.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0559999999999998e-06 2024-09-18T11:31:41.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308317+0000", 2024-09-18T11:31:41.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7209999999999997e-06 2024-09-18T11:31:41.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308320+0000", 2024-09-18T11:31:41.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.721e-06 2024-09-18T11:31:41.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.644 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308447+0000", 2024-09-18T11:31:41.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012645199999999999 2024-09-18T11:31:41.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.644 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:41.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.308451+0000", 2024-09-18T11:31:41.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6800000000000001e-06 2024-09-18T11:31:41.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.645 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.312462+0000", 2024-09-18T11:31:41.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040102139999999998 2024-09-18T11:31:41.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.645 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.312585+0000", 2024-09-18T11:31:41.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012388899999999999 2024-09-18T11:31:41.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.645 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.315847+0000", 2024-09-18T11:31:41.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003261197 2024-09-18T11:31:41.646 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.646 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.646 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.646 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1926 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-18T11:31:41.646 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:41.646 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.241094797000002, 2024-09-18T11:31:41.646 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066190219999999996, 2024-09-18T11:31:41.646 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.646 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:41.646 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.647 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.647 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.647 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1926 2024-09-18T11:31:41.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.647 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:41.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:41.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362797+0000", 2024-09-18T11:31:41.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362799+0000", 2024-09-18T11:31:41.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.029e-06 2024-09-18T11:31:41.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362804+0000", 2024-09-18T11:31:41.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3989999999999997e-06 2024-09-18T11:31:41.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.649 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.362806+0000", 2024-09-18T11:31:41.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3020000000000002e-06 2024-09-18T11:31:41.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.649 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364105+0000", 2024-09-18T11:31:41.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001299478 2024-09-18T11:31:41.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.650 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:41.650 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.364153+0000", 2024-09-18T11:31:41.650 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8059000000000002e-05 2024-09-18T11:31:41.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.650 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.650 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369273+0000", 2024-09-18T11:31:41.650 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051199540000000003 2024-09-18T11:31:41.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.650 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.650 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369343+0000", 2024-09-18T11:31:41.650 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0191999999999995e-05 2024-09-18T11:31:41.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.651 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.651 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:47.369416+0000", 2024-09-18T11:31:41.651 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2925999999999999e-05 2024-09-18T11:31:41.651 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.651 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.651 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.651 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1969 4.a 4:588debc2:::smithi11939506-18:18 [read 0~3949612,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:41.651 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:41.651 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 50.229905907999999, 2024-09-18T11:31:41.651 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075409190000000001, 2024-09-18T11:31:41.651 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.652 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:41.652 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.652 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.652 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.652 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1969 2024-09-18T11:31:41.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.652 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:41.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:41.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:41.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373990+0000", 2024-09-18T11:31:41.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1119999999999998e-06 2024-09-18T11:31:41.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373997+0000", 2024-09-18T11:31:41.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4200000000000001e-06 2024-09-18T11:31:41.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.374002+0000", 2024-09-18T11:31:41.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:31:41.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380200+0000", 2024-09-18T11:31:41.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006197302 2024-09-18T11:31:41.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380261+0000", 2024-09-18T11:31:41.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1389999999999993e-05 2024-09-18T11:31:41.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.381527+0000", 2024-09-18T11:31:41.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012660080000000001 2024-09-18T11:31:41.655 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.655 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.655 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.656 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2207 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1410296~761389] snapc 24=[24,1e,1c,1b,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T11:31:41.656 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:41.656 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 44.181395082000002, 2024-09-18T11:31:41.656 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008410743, 2024-09-18T11:31:41.656 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.656 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:41.656 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.656 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.656 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.656 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2207 2024-09-18T11:31:41.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.656 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.657 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:41.657 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.657 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:41.657 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.657 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422496+0000", 2024-09-18T11:31:41.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:41.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423190+0000", 2024-09-18T11:31:41.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00069405499999999995 2024-09-18T11:31:41.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423204+0000", 2024-09-18T11:31:41.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:31:41.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423208+0000", 2024-09-18T11:31:41.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:31:41.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423293+0000", 2024-09-18T11:31:41.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4524999999999995e-05 2024-09-18T11:31:41.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423394+0000", 2024-09-18T11:31:41.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101395 2024-09-18T11:31:41.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:41.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423579+0000", 2024-09-18T11:31:41.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018461000000000001 2024-09-18T11:31:41.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:41.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.428574+0000", 2024-09-18T11:31:41.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049944960000000002 2024-09-18T11:31:41.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:41.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430809+0000", 2024-09-18T11:31:41.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022351820000000001 2024-09-18T11:31:41.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:41.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430862+0000", 2024-09-18T11:31:41.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3189999999999997e-05 2024-09-18T11:31:41.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.662 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.662 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430908+0000", 2024-09-18T11:31:41.662 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5973999999999997e-05 2024-09-18T11:31:41.662 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.662 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.662 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.662 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2339 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:41.662 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:41.662 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.217090452999997, 2024-09-18T11:31:41.663 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011809992, 2024-09-18T11:31:41.663 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.663 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:41.663 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.663 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.663 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.663 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2339 2024-09-18T11:31:41.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.663 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.663 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.663 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.663 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:41.663 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:41.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:31:41.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:41.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:31:41.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4710000000000001e-06 2024-09-18T11:31:41.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.665 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386824+0000", 2024-09-18T11:31:41.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6104e-05 2024-09-18T11:31:41.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.665 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386836+0000", 2024-09-18T11:31:41.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1827999999999999e-05 2024-09-18T11:31:41.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388390+0000", 2024-09-18T11:31:41.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015539499999999999 2024-09-18T11:31:41.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:41.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388443+0000", 2024-09-18T11:31:41.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3785000000000001e-05 2024-09-18T11:31:41.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398238+0000", 2024-09-18T11:31:41.667 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097947989999999999 2024-09-18T11:31:41.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.667 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.667 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.667 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398315+0000", 2024-09-18T11:31:41.667 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6909999999999994e-05 2024-09-18T11:31:41.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.667 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.667 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.667 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398612+0000", 2024-09-18T11:31:41.667 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029671999999999999 2024-09-18T11:31:41.667 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.667 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.667 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.668 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2340 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:41.668 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:41.668 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.217047141000002, 2024-09-18T11:31:41.668 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012086448, 2024-09-18T11:31:41.668 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.668 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:41.668 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.668 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.668 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.668 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2340 2024-09-18T11:31:41.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.669 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.669 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.669 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.669 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:41.669 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.669 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.669 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.669 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:41.669 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.670 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:41.670 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.670 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386848+0000", 2024-09-18T11:31:41.670 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3069999999999998e-06 2024-09-18T11:31:41.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.670 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386855+0000", 2024-09-18T11:31:41.670 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0979999999999996e-06 2024-09-18T11:31:41.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.671 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386859+0000", 2024-09-18T11:31:41.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9360000000000001e-06 2024-09-18T11:31:41.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.671 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388466+0000", 2024-09-18T11:31:41.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001607154 2024-09-18T11:31:41.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.671 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:41.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388507+0000", 2024-09-18T11:31:41.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1093000000000001e-05 2024-09-18T11:31:41.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398647+0000", 2024-09-18T11:31:41.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010139686 2024-09-18T11:31:41.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398718+0000", 2024-09-18T11:31:41.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1138999999999998e-05 2024-09-18T11:31:41.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.673 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.673 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398931+0000", 2024-09-18T11:31:41.673 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021356899999999999 2024-09-18T11:31:41.673 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.673 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.673 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.673 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.673 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2341 4.5 4:aa575018:::smithi11939506-11:head [read 0~2487436,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:41.673 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:41.673 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.217028919000001, 2024-09-18T11:31:41.673 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013751648999999999, 2024-09-18T11:31:41.673 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.674 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:41.674 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.674 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.674 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.674 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2341 2024-09-18T11:31:41.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.674 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.674 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.674 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:41.674 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:41.675 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:41.675 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386866+0000", 2024-09-18T11:31:41.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4249999999999998e-06 2024-09-18T11:31:41.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.676 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386872+0000", 2024-09-18T11:31:41.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.003e-06 2024-09-18T11:31:41.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.676 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:31:41.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.507e-06 2024-09-18T11:31:41.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.677 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.677 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388520+0000", 2024-09-18T11:31:41.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001644146 2024-09-18T11:31:41.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.677 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:41.677 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388561+0000", 2024-09-18T11:31:41.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1486999999999998e-05 2024-09-18T11:31:41.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.677 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.677 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398963+0000", 2024-09-18T11:31:41.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010401353 2024-09-18T11:31:41.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.678 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.399046+0000", 2024-09-18T11:31:41.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3141999999999998e-05 2024-09-18T11:31:41.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.678 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.400615+0000", 2024-09-18T11:31:41.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015690960000000001 2024-09-18T11:31:41.678 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.678 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.678 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.679 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2529 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:41.679 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:31:41.679 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.430555654999999, 2024-09-18T11:31:41.679 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079268120000000001, 2024-09-18T11:31:41.679 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.679 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:41.679 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.679 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.679 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.679 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2529 2024-09-18T11:31:41.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.680 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.680 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.680 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:31:41.680 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.680 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.680 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:31:41.680 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.681 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173336+0000", 2024-09-18T11:31:41.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.681 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173339+0000", 2024-09-18T11:31:41.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9019999999999999e-06 2024-09-18T11:31:41.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.681 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173381+0000", 2024-09-18T11:31:41.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1836999999999999e-05 2024-09-18T11:31:41.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.682 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173390+0000", 2024-09-18T11:31:41.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.302e-06 2024-09-18T11:31:41.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.682 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174507+0000", 2024-09-18T11:31:41.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001116868 2024-09-18T11:31:41.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.682 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.683 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174603+0000", 2024-09-18T11:31:41.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5598000000000002e-05 2024-09-18T11:31:41.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.683 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:41.683 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174759+0000", 2024-09-18T11:31:41.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015616299999999999 2024-09-18T11:31:41.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.683 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:41.683 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178943+0000", 2024-09-18T11:31:41.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041846180000000002 2024-09-18T11:31:41.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.684 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:41.684 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181175+0000", 2024-09-18T11:31:41.684 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00223215 2024-09-18T11:31:41.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.684 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.684 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:41.684 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181229+0000", 2024-09-18T11:31:41.684 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3585999999999998e-05 2024-09-18T11:31:41.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.684 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.684 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.684 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181263+0000", 2024-09-18T11:31:41.684 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4292999999999998e-05 2024-09-18T11:31:41.685 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.685 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.685 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.685 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.685 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2530 4.5 4:af4ba75a:::smithi11939506-37:head [rollback 25] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:41.685 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:31:41.685 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.430493349000002, 2024-09-18T11:31:41.685 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080198800000000001, 2024-09-18T11:31:41.685 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.685 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:41.685 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.685 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.686 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.686 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2530 2024-09-18T11:31:41.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.686 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:31:41.686 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:31:41.686 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.687 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.687 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173398+0000", 2024-09-18T11:31:41.687 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:41.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.687 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.687 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173403+0000", 2024-09-18T11:31:41.687 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4389999999999999e-06 2024-09-18T11:31:41.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.687 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.687 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173414+0000", 2024-09-18T11:31:41.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0411000000000001e-05 2024-09-18T11:31:41.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.688 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173420+0000", 2024-09-18T11:31:41.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9610000000000002e-06 2024-09-18T11:31:41.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.688 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176390+0000", 2024-09-18T11:31:41.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029706289999999998 2024-09-18T11:31:41.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.689 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.689 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176501+0000", 2024-09-18T11:31:41.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011090300000000001 2024-09-18T11:31:41.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.689 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:41.689 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176827+0000", 2024-09-18T11:31:41.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00032526899999999999 2024-09-18T11:31:41.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.689 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:41.689 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178958+0000", 2024-09-18T11:31:41.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021314860000000001 2024-09-18T11:31:41.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.690 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.690 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:41.690 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181318+0000", 2024-09-18T11:31:41.690 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002359571 2024-09-18T11:31:41.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.690 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.690 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:41.690 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181372+0000", 2024-09-18T11:31:41.690 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3884000000000001e-05 2024-09-18T11:31:41.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.690 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.690 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.691 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181419+0000", 2024-09-18T11:31:41.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7148000000000003e-05 2024-09-18T11:31:41.691 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.691 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.691 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.691 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2531 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:41.691 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:41.691 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.430467225999998, 2024-09-18T11:31:41.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080687239999999993, 2024-09-18T11:31:41.691 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.692 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:41.692 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.692 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.692 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.692 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2531 2024-09-18T11:31:41.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.692 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:41.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.693 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:41.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.693 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:41.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.693 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173429+0000", 2024-09-18T11:31:41.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2989999999999998e-06 2024-09-18T11:31:41.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.694 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.694 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173436+0000", 2024-09-18T11:31:41.694 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1799999999999999e-06 2024-09-18T11:31:41.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.694 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.694 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173441+0000", 2024-09-18T11:31:41.694 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8810000000000004e-06 2024-09-18T11:31:41.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.694 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.694 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178267+0000", 2024-09-18T11:31:41.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048257969999999997 2024-09-18T11:31:41.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178388+0000", 2024-09-18T11:31:41.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121779 2024-09-18T11:31:41.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:41.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178599+0000", 2024-09-18T11:31:41.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000210094 2024-09-18T11:31:41.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:41.696 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.179339+0000", 2024-09-18T11:31:41.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00074063799999999995 2024-09-18T11:31:41.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.696 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.696 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:41.696 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181444+0000", 2024-09-18T11:31:41.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021044100000000001 2024-09-18T11:31:41.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.696 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.696 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:41.696 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181470+0000", 2024-09-18T11:31:41.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6415999999999999e-05 2024-09-18T11:31:41.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.697 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.697 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181494+0000", 2024-09-18T11:31:41.697 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3672e-05 2024-09-18T11:31:41.697 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.697 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.697 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.697 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2910 4.5 4:af4ba75a:::smithi11939506-37:head [write 786627~795991,stat] snapc 37=[37,35,34,30,2f,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:41.697 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:31:41.697 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.600409397, 2024-09-18T11:31:41.697 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084093710000000006, 2024-09-18T11:31:41.698 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.698 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:41.698 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.698 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.698 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.698 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2910 2024-09-18T11:31:41.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.698 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.698 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.698 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.698 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:31:41.698 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.698 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.699 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:31:41.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.699 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.699 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003481+0000", 2024-09-18T11:31:41.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:41.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.699 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.699 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004300+0000", 2024-09-18T11:31:41.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081934200000000003 2024-09-18T11:31:41.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.700 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.700 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004324+0000", 2024-09-18T11:31:41.700 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4618999999999999e-05 2024-09-18T11:31:41.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.700 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.700 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004335+0000", 2024-09-18T11:31:41.700 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.092e-05 2024-09-18T11:31:41.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.700 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004385+0000", 2024-09-18T11:31:41.701 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9709999999999997e-05 2024-09-18T11:31:41.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.701 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004470+0000", 2024-09-18T11:31:41.701 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4864000000000006e-05 2024-09-18T11:31:41.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.701 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:41.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004612+0000", 2024-09-18T11:31:41.701 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014241500000000001 2024-09-18T11:31:41.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.702 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.702 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:41.702 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011193+0000", 2024-09-18T11:31:41.702 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006580654 2024-09-18T11:31:41.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.702 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.702 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:41.702 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011788+0000", 2024-09-18T11:31:41.702 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059533600000000002 2024-09-18T11:31:41.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.702 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.702 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:41.702 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011842+0000", 2024-09-18T11:31:41.703 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3180000000000002e-05 2024-09-18T11:31:41.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.703 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.703 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.703 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011892+0000", 2024-09-18T11:31:41.703 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0618000000000002e-05 2024-09-18T11:31:41.703 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.703 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.703 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.703 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.703 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2953 4.a 4:588debc2:::smithi11939506-18:head [assert-version v113,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:41.703 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:31:41.703 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.577093879, 2024-09-18T11:31:41.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083931279999999997, 2024-09-18T11:31:41.704 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.704 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:41.704 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.704 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.704 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.704 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2953 2024-09-18T11:31:41.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.704 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.704 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.704 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.704 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:31:41.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.705 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.705 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:31:41.705 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.705 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.705 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026796+0000", 2024-09-18T11:31:41.705 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:41.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.705 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026806+0000", 2024-09-18T11:31:41.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9529999999999994e-06 2024-09-18T11:31:41.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026824+0000", 2024-09-18T11:31:41.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7901999999999999e-05 2024-09-18T11:31:41.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026832+0000", 2024-09-18T11:31:41.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8450000000000003e-06 2024-09-18T11:31:41.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026883+0000", 2024-09-18T11:31:41.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0883000000000001e-05 2024-09-18T11:31:41.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026980+0000", 2024-09-18T11:31:41.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7526000000000001e-05 2024-09-18T11:31:41.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:41.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032722+0000", 2024-09-18T11:31:41.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057414689999999999 2024-09-18T11:31:41.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.708 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.708 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:41.708 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.034230+0000", 2024-09-18T11:31:41.708 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001508035 2024-09-18T11:31:41.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.708 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.708 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:41.708 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035090+0000", 2024-09-18T11:31:41.708 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00086047300000000005 2024-09-18T11:31:41.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.708 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.708 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:41.708 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035147+0000", 2024-09-18T11:31:41.709 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6598000000000003e-05 2024-09-18T11:31:41.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.709 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.709 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035191+0000", 2024-09-18T11:31:41.709 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4747999999999999e-05 2024-09-18T11:31:41.709 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.709 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.709 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.709 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3104 4.5 4:af4ba75a:::smithi11939506-37:3d [sparse-read 0~1484290] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-18T11:31:41.709 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:33.018191+0000", 2024-09-18T11:31:41.710 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.5857013270000007, 2024-09-18T11:31:41.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068683290000000003, 2024-09-18T11:31:41.710 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.710 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:41.710 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.710 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.710 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.710 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3104 2024-09-18T11:31:41.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.710 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.710 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018191+0000", 2024-09-18T11:31:41.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018191+0000", 2024-09-18T11:31:41.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018190+0000", 2024-09-18T11:31:41.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.712 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018198+0000", 2024-09-18T11:31:41.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3250000000000002e-06 2024-09-18T11:31:41.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.712 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018204+0000", 2024-09-18T11:31:41.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4649999999999999e-06 2024-09-18T11:31:41.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.712 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018208+0000", 2024-09-18T11:31:41.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5659999999999998e-06 2024-09-18T11:31:41.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018271+0000", 2024-09-18T11:31:41.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2908000000000001e-05 2024-09-18T11:31:41.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:41.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018275+0000", 2024-09-18T11:31:41.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8920000000000002e-06 2024-09-18T11:31:41.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.024672+0000", 2024-09-18T11:31:41.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063969939999999996 2024-09-18T11:31:41.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.714 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.714 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.024772+0000", 2024-09-18T11:31:41.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000100097 2024-09-18T11:31:41.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.714 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.714 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.025059+0000", 2024-09-18T11:31:41.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00028746899999999999 2024-09-18T11:31:41.714 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.714 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.714 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.715 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3105 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 3e=[3e,3d,3b,37,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-18T11:31:41.715 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:31:41.715 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.5856811749999995, 2024-09-18T11:31:41.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011735906000000001, 2024-09-18T11:31:41.715 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:41.715 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:41.715 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:41.715 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:41.715 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:41.715 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3105 2024-09-18T11:31:41.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.716 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:41.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.716 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:41.716 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:31:41.716 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.716 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:41.716 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:31:41.716 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:41.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.716 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:41.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:31:41.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:41.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.717 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:41.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018212+0000", 2024-09-18T11:31:41.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6759999999999999e-06 2024-09-18T11:31:41.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.717 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:41.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018216+0000", 2024-09-18T11:31:41.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-06 2024-09-18T11:31:41.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:41.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018218+0000", 2024-09-18T11:31:41.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6110000000000001e-06 2024-09-18T11:31:41.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018246+0000", 2024-09-18T11:31:41.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7557000000000001e-05 2024-09-18T11:31:41.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:41.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018250+0000", 2024-09-18T11:31:41.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4299999999999999e-06 2024-09-18T11:31:41.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:41.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021364+0000", 2024-09-18T11:31:41.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031135389999999998 2024-09-18T11:31:41.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:41.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021494+0000", 2024-09-18T11:31:41.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000130333 2024-09-18T11:31:41.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:41.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:41.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:41.720 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.029947+0000", 2024-09-18T11:31:41.720 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084526150000000001 2024-09-18T11:31:41.720 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.720 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.720 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.720 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:41.720 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:41.720 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:41.721 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:31:41.722 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:41.719+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:41.820 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:41.820 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:41.820 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:41.820 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:41.822 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:41.821+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:41.828 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:31:41.923 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:41.922+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:41.925 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:41.925 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:41.925 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:41.925 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:41.925 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:41.934 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:31:42.024 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:42.023+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:42.035 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T11:31:42.075 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:42.075 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:42.075 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:42.075 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:42.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.076 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:42.076 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:42.076 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.91514727000001, 2024-09-18T11:31:42.076 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:31:42.076 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:42.076 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:42.076 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:42.076 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:42.076 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:42.076 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:31:42.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.076 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:42.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.077 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:42.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:42.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.077 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:42.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:42.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.077 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:42.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:31:42.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:42.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:42.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:31:42.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:31:42.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:42.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:31:42.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:31:42.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:42.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:31:42.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:42.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.079 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:42.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:31:42.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:31:42.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.079 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:42.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:31:42.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:31:42.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:42.080 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:31:42.080 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:31:42.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:42.080 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:31:42.080 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:31:42.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:42.081 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:31:42.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:31:42.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.081 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:42.081 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:31:42.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:31:42.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.081 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:42.081 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:31:42.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:31:42.081 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.082 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:42.082 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.082 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:42.082 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:42.082 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.915021008, 2024-09-18T11:31:42.082 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055717040000000002, 2024-09-18T11:31:42.082 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:42.082 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:42.082 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:42.082 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:42.082 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:42.083 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 45 2024-09-18T11:31:42.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.083 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:42.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.083 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:42.083 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:42.083 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.083 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:42.083 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:42.083 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.084 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:42.084 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157676+0000", 2024-09-18T11:31:42.084 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:42.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.084 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:42.084 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157694+0000", 2024-09-18T11:31:42.084 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7422999999999999e-05 2024-09-18T11:31:42.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:42.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157704+0000", 2024-09-18T11:31:42.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9000000000000001e-06 2024-09-18T11:31:42.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:42.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157718+0000", 2024-09-18T11:31:42.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4788e-05 2024-09-18T11:31:42.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:42.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160937+0000", 2024-09-18T11:31:42.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00321886 2024-09-18T11:31:42.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.086 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:42.086 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161023+0000", 2024-09-18T11:31:42.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6063999999999995e-05 2024-09-18T11:31:42.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.086 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:42.086 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161202+0000", 2024-09-18T11:31:42.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017912599999999999 2024-09-18T11:31:42.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.086 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:42.086 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163052+0000", 2024-09-18T11:31:42.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001849554 2024-09-18T11:31:42.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.087 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:42.087 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163204+0000", 2024-09-18T11:31:42.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015186900000000001 2024-09-18T11:31:42.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.087 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:42.087 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163229+0000", 2024-09-18T11:31:42.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:31:42.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.088 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:42.088 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163249+0000", 2024-09-18T11:31:42.088 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9992000000000001e-05 2024-09-18T11:31:42.088 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.088 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:42.088 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.088 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:42.088 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:42.088 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.91497189499999, 2024-09-18T11:31:42.088 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:31:42.088 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:42.089 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:42.089 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:42.089 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:42.089 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:42.089 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:31:42.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.089 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:42.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.089 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:42.089 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:42.089 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.089 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:42.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:42.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:42.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:42.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:42.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:42.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:31:42.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:31:42.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:42.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:31:42.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:31:42.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:42.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:31:42.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:31:42.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:42.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:31:42.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:31:42.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.092 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:42.092 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:31:42.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:31:42.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.092 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:42.092 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:31:42.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:31:42.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.092 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:42.093 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:31:42.093 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:31:42.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.093 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:42.093 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:31:42.093 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:31:42.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.093 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:42.093 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:31:42.093 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:31:42.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.094 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:42.094 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:31:42.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:31:42.094 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.094 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:42.094 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.094 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:42.094 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:42.094 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.914929364, 2024-09-18T11:31:42.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:31:42.094 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:31:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:42.096 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:42.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:42.096 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:31:42.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:42.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:42.096 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:31:42.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:31:42.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:42.097 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:31:42.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:31:42.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:42.097 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:31:42.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:31:42.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:42.097 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:31:42.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:31:42.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.098 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:42.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:31:42.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:31:42.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.098 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:42.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:31:42.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:31:42.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.098 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:42.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:31:42.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:31:42.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:42.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:31:42.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:31:42.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:42.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:31:42.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:31:42.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:42.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:31:42.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:31:42.100 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.100 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:42.100 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.100 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:42.100 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:42.100 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.914874434, 2024-09-18T11:31:42.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:31:42.100 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:42.100 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:42.101 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:42.101 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:42.101 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:42.101 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:31:42.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.101 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:42.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.101 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:42.101 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:42.101 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.101 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:42.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:42.102 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.102 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:42.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:42.102 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:42.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.102 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:42.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:31:42.102 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:31:42.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.103 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:42.103 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:31:42.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:31:42.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.103 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:42.103 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:31:42.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:31:42.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.103 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:42.103 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:31:42.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:31:42.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:42.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:31:42.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:31:42.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:42.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:31:42.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:31:42.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:42.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:31:42.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:31:42.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.105 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:42.105 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:31:42.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:31:42.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.105 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:42.105 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:31:42.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:31:42.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:42.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:31:42.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:31:42.106 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.106 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:42.106 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.106 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:42.106 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:42.106 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.914864454, 2024-09-18T11:31:42.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:31:42.106 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:42.106 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:42.107 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:42.107 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:42.107 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:42.107 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:31:42.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.107 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:42.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:42.107 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:42.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:42.107 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:42.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.108 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:42.108 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:42.108 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:42.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.108 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:42.108 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:31:42.108 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:31:42.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.108 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:42.109 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:31:42.109 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:31:42.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.109 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:42.109 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:31:42.109 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:31:42.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.109 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:42.109 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:31:42.109 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:31:42.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.109 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:42.110 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:31:42.110 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:31:42.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.110 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:42.110 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:31:42.110 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:31:42.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.110 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:42.110 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:31:42.110 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:31:42.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.111 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:42.111 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:31:42.111 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:31:42.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.111 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:42.111 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:31:42.111 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:31:42.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.111 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:42.111 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:31:42.111 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:31:42.111 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.111 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:42.111 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.112 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:42.112 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:42.112 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.91485285500001, 2024-09-18T11:31:42.112 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:31:42.112 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:42.112 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:42.112 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:42.112 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:42.113 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:42.113 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:31:42.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.113 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:42.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.113 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:42.113 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:42.113 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.113 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:42.113 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:42.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.114 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:42.114 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:42.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:42.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.114 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:42.114 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:31:42.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:31:42.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.114 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:42.114 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:31:42.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:31:42.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:42.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:31:42.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:31:42.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:42.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:31:42.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:31:42.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:42.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:31:42.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:31:42.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:42.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:31:42.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:31:42.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:42.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:31:42.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:31:42.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:42.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:31:42.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:31:42.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:42.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:31:42.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:31:42.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:42.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:31:42.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:31:42.118 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.118 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:42.118 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.118 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:42.118 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:42.118 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.91479637400001, 2024-09-18T11:31:42.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:31:42.118 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:42.119 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:42.119 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:42.119 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:42.119 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:42.119 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:31:42.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.119 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:42.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.119 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:42.119 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:42.119 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:42.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:42.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:42.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:31:42.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:42.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:42.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:31:42.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:31:42.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:42.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:31:42.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:31:42.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:42.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:31:42.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:31:42.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:42.122 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:31:42.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:31:42.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.122 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:42.122 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:31:42.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:31:42.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.122 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:42.122 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:31:42.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:31:42.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:42.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:31:42.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:31:42.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:42.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:31:42.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:31:42.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:42.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:31:42.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:31:42.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.124 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:42.124 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:31:42.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:31:42.124 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.124 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:42.124 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.124 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:42.124 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:42.125 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.91478255, 2024-09-18T11:31:42.125 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:31:42.125 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:42.125 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:42.125 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:42.125 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:42.125 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:42.125 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:31:42.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.125 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:42.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.125 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:42.125 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:42.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.126 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:42.126 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:42.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.126 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:42.126 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:42.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:42.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.127 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:42.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:31:42.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:31:42.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.127 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:42.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:31:42.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:31:42.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.127 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:42.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:31:42.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:31:42.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:42.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:31:42.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:31:42.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:42.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:31:42.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:31:42.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:42.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:31:42.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:31:42.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:42.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:31:42.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:31:42.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:42.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:31:42.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:31:42.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.130 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:42.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:31:42.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:31:42.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.130 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:42.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:31:42.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:31:42.130 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.130 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:42.130 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.130 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:42.131 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:42.131 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.498603643, 2024-09-18T11:31:42.131 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:31:42.131 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:42.131 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:42.131 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:42.131 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:42.131 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:42.131 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:31:42.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.131 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:42.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.131 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:42.131 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:42.132 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.132 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:42.132 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:42.132 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.132 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:42.132 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:42.132 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:42.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.133 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:42.133 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:31:42.133 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:31:42.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.133 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.133 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:42.133 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:31:42.133 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:31:42.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.133 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.133 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:42.133 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:31:42.133 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:31:42.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:42.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:31:42.134 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:31:42.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:42.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:31:42.134 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:31:42.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:42.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:31:42.134 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:31:42.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.135 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:42.135 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:31:42.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:31:42.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.135 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:42.135 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:31:42.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:31:42.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.135 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:42.136 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:31:42.136 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:31:42.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.136 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:42.136 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:31:42.136 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:31:42.136 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.136 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:42.136 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.137 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:42.137 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:42.137 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.498580343, 2024-09-18T11:31:42.137 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:31:42.137 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:42.137 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:42.137 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:42.137 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:42.137 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:42.137 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:31:42.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.137 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:42.137 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.137 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:42.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:42.138 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.138 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:42.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:42.138 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.138 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:42.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:31:42.138 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:42.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.139 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:42.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:31:42.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:31:42.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.139 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:42.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:31:42.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:31:42.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.139 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:42.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:31:42.140 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:31:42.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.140 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:42.140 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:31:42.140 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:31:42.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.140 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:42.140 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:31:42.140 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:31:42.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.141 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:42.141 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:31:42.141 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:31:42.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.141 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:42.141 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:31:42.141 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:31:42.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.141 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:42.141 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:31:42.141 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:31:42.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.142 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:42.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:31:42.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:31:42.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.142 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:42.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:31:42.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:31:42.142 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.142 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:42.142 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.143 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:42.143 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:42.143 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.498494734, 2024-09-18T11:31:42.143 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:31:42.143 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:42.143 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:42.143 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:42.143 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:42.143 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:42.143 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:31:42.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.143 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:42.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.144 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:42.144 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:42.144 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.144 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:42.144 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:42.144 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.145 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:42.145 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:42.145 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:42.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.145 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:42.145 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:31:42.145 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:31:42.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.145 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:42.145 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:31:42.146 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:31:42.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.146 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.146 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:42.146 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:31:42.146 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:31:42.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.146 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.146 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:42.146 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:31:42.146 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:31:42.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.146 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.146 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:42.147 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:31:42.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:31:42.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.147 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:42.147 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:31:42.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:31:42.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.147 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:42.147 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:31:42.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:31:42.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:42.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:31:42.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:31:42.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:42.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:31:42.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:31:42.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:42.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:31:42.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:31:42.149 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.149 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:42.149 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.149 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:42.149 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:42.149 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.498483769, 2024-09-18T11:31:42.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:31:42.149 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:42.149 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:42.149 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:42.149 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:42.150 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:42.150 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:31:42.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.150 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:42.150 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.150 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:42.150 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:42.150 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.150 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.150 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:42.150 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:42.150 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.151 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.151 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:42.151 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:42.151 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:42.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.151 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.151 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:42.151 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:31:42.151 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:31:42.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.151 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.151 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:42.151 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:31:42.152 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:31:42.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.152 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.152 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:42.152 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:31:42.152 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:31:42.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.152 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.152 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:42.152 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:31:42.152 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:31:42.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.152 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.152 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:42.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:31:42.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:31:42.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.153 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:42.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:31:42.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:31:42.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.153 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:42.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:31:42.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:31:42.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:42.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:31:42.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:31:42.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:42.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:31:42.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:31:42.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:42.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:31:42.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:31:42.155 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.155 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:42.155 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.155 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:42.155 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:42.155 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.498472601, 2024-09-18T11:31:42.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:31:42.155 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:42.155 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:42.155 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:42.155 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:42.156 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:42.156 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:31:42.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.156 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:42.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.156 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:42.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:42.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.156 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:42.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:42.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.157 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:42.157 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:42.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:42.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.157 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:42.157 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:31:42.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:31:42.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.157 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:42.157 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:31:42.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:31:42.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.158 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:42.158 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:31:42.158 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:31:42.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.158 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:42.158 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:31:42.158 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:31:42.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.158 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:42.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:31:42.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:31:42.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.159 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:42.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:31:42.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:31:42.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.159 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:42.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:31:42.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:31:42.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.160 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:42.160 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:31:42.160 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:31:42.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.160 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:42.160 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:31:42.160 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:31:42.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.160 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:42.160 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:31:42.160 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:31:42.161 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.161 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:42.161 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.161 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:42.161 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:42.161 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.49840727199999, 2024-09-18T11:31:42.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:31:42.161 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:42.161 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:42.161 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:42.161 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:42.162 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:42.162 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:31:42.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.162 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:42.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:42.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:42.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:42.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:42.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.163 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:42.163 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:31:42.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:42.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.163 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:42.163 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:31:42.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:31:42.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.163 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:42.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:31:42.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:31:42.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.164 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:42.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:31:42.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:31:42.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.164 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:42.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:31:42.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:31:42.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:42.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:31:42.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:31:42.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:42.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:31:42.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:31:42.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:42.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:31:42.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:31:42.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:42.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:31:42.166 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:31:42.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:42.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:31:42.166 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:31:42.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:42.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:31:42.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:31:42.167 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.167 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:42.167 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.167 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:42.167 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:42.167 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.488245294, 2024-09-18T11:31:42.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:31:42.167 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:42.167 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:42.168 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:42.168 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:42.168 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:42.168 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:31:42.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.168 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:42.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:42.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:42.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:42.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:42.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:42.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:31:42.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:42.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:42.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:31:42.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:31:42.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:42.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:31:42.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:31:42.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:42.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:31:42.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:31:42.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:42.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:31:42.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:31:42.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:42.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:31:42.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:31:42.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:42.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:31:42.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:31:42.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:42.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:31:42.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:31:42.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.172 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:42.172 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:31:42.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:31:42.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.172 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:42.172 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:31:42.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:31:42.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.173 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:42.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:31:42.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:31:42.173 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.173 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:42.173 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.173 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:42.173 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:42.173 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.48815281500001, 2024-09-18T11:31:42.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:31:42.173 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:42.173 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:42.174 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:42.174 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:42.174 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:42.174 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:31:42.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.174 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:42.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.174 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:42.174 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:42.174 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.174 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:42.175 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:42.175 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.175 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:42.175 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:42.175 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:42.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.175 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:42.175 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:31:42.175 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:31:42.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:42.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:31:42.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:31:42.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:42.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:31:42.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:31:42.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:42.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:31:42.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:31:42.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:42.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:31:42.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:31:42.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:42.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:31:42.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:31:42.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:42.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:31:42.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:31:42.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:42.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:31:42.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:31:42.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:42.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:31:42.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:31:42.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.179 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.179 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:42.179 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:31:42.179 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:31:42.179 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.179 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:42.179 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.179 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.179 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1788 4.8 e60/22 4:1898b15d:::smithi11939506-25:head v 60'12, mlcod=58'9)", 2024-09-18T11:31:42.179 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:42.179 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 59.155747325999997, 2024-09-18T11:31:42.179 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058348599999999999, 2024-09-18T11:31:42.179 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:42.180 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:42.180 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:42.180 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.180 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:42.180 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:42.180 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.180 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.180 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.180 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:42.180 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:42.180 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.180 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.180 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:42.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916949+0000", 2024-09-18T11:31:42.181 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:42.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.181 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:42.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917602+0000", 2024-09-18T11:31:42.181 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00065337100000000003 2024-09-18T11:31:42.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.181 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:42.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917621+0000", 2024-09-18T11:31:42.181 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9287e-05 2024-09-18T11:31:42.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.182 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.182 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:42.182 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917629+0000", 2024-09-18T11:31:42.182 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3780000000000003e-06 2024-09-18T11:31:42.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.182 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.182 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:42.182 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917677+0000", 2024-09-18T11:31:42.182 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8068000000000001e-05 2024-09-18T11:31:42.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.182 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.182 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:42.182 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917699+0000", 2024-09-18T11:31:42.183 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2149000000000001e-05 2024-09-18T11:31:42.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.183 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.183 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:42.183 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922747+0000", 2024-09-18T11:31:42.183 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050477609999999996 2024-09-18T11:31:42.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.183 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.183 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:42.183 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922785+0000", 2024-09-18T11:31:42.183 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8726000000000002e-05 2024-09-18T11:31:42.183 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.183 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:42.184 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.184 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.184 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:58 4.3 e91/67 4:c70c3162:::smithi11939506-35:18 v 91'101, mlcod=86'95)", 2024-09-18T11:31:42.184 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:42.184 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 28.057089102999999, 2024-09-18T11:31:42.184 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055647500000000003, 2024-09-18T11:31:42.184 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:42.184 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:42.184 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:42.184 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.184 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:42.184 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:42.185 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.185 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.185 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:42.185 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:42.185 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.185 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.185 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:42.185 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015608+0000", 2024-09-18T11:31:42.185 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:42.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.185 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.185 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:42.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015614+0000", 2024-09-18T11:31:42.186 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5310000000000001e-06 2024-09-18T11:31:42.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.186 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:42.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015622+0000", 2024-09-18T11:31:42.186 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0809999999999994e-06 2024-09-18T11:31:42.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.186 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:42.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015627+0000", 2024-09-18T11:31:42.186 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8439999999999997e-06 2024-09-18T11:31:42.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:42.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016003+0000", 2024-09-18T11:31:42.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037652999999999997 2024-09-18T11:31:42.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:42.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016024+0000", 2024-09-18T11:31:42.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0780000000000001e-05 2024-09-18T11:31:42.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:42.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021141+0000", 2024-09-18T11:31:42.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051166639999999999 2024-09-18T11:31:42.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.188 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.188 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:42.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021174+0000", 2024-09-18T11:31:42.188 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2839000000000002e-05 2024-09-18T11:31:42.188 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.188 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:42.188 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.188 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.188 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3209 4.0 e109/22 4:019b6e09:::smithi11939506-40:head v 109'149, mlcod=106'146)", 2024-09-18T11:31:42.188 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:31:42.189 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.9733119639999996, 2024-09-18T11:31:42.189 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063064039999999998, 2024-09-18T11:31:42.189 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:42.189 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:42.189 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:42.189 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.189 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:42.189 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:31:42.189 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.189 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.189 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:42.189 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:31:42.189 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:42.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.190 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.190 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:42.190 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099385+0000", 2024-09-18T11:31:42.190 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:42.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.190 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.190 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:42.190 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099951+0000", 2024-09-18T11:31:42.190 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056606600000000005 2024-09-18T11:31:42.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.190 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.190 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:42.190 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099964+0000", 2024-09-18T11:31:42.191 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2918999999999999e-05 2024-09-18T11:31:42.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.191 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.191 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:42.191 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099972+0000", 2024-09-18T11:31:42.191 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2300000000000002e-06 2024-09-18T11:31:42.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.191 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.191 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:42.191 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.100026+0000", 2024-09-18T11:31:42.191 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4456999999999997e-05 2024-09-18T11:31:42.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.191 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.191 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:42.192 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.100046+0000", 2024-09-18T11:31:42.192 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0417000000000001e-05 2024-09-18T11:31:42.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.192 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.192 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:42.192 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.105653+0000", 2024-09-18T11:31:42.192 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005606214 2024-09-18T11:31:42.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:42.192 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:42.192 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:42.192 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.105692+0000", 2024-09-18T11:31:42.192 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9671e-05 2024-09-18T11:31:42.192 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.193 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:42.193 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.193 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:42.193 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:42.193 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:42.194 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:42.112+0000 7f2d449e8640 -1 osd.1 113 handle_osd_map injecting map crc failure 2024-09-18T11:31:42.195 INFO:tasks.rados.rados.0.smithi119.stdout:1510: done (0 left) 2024-09-18T11:31:42.195 INFO:tasks.rados.rados.0.smithi119.stdout:1511: rollback oid 46 current snap is 41 2024-09-18T11:31:42.195 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 46 to 39 2024-09-18T11:31:42.195 INFO:tasks.rados.rados.0.smithi119.stdout:1512: write oid 16 current snap is 41 2024-09-18T11:31:42.195 INFO:tasks.rados.rados.0.smithi119.stdout:1512: seq_num 286 ranges {406802=626275,1686203=728431} 2024-09-18T11:31:42.195 INFO:tasks.rados.rados.0.smithi119.stdout:1512: writing smithi11939506-16 from 406802 to 1033077 tid 1 2024-09-18T11:31:42.196 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:42.121+0000 7f697aead640 -1 osd.3 112 handle_osd_map injecting map crc failure 2024-09-18T11:31:42.196 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:42.193+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:42.196 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:31:42.199 INFO:tasks.rados.rados.0.smithi119.stdout:1512: writing smithi11939506-16 from 1686203 to 2414634 tid 2 2024-09-18T11:31:42.199 INFO:tasks.rados.rados.0.smithi119.stdout:1513: set_chunk oid 17 target oid 2024-09-18T11:31:42.199 INFO:tasks.rados.rados.0.smithi119.stdout:1513: set_chunk oid 17 offset: 989184 length: 646656 target oid 3ad573069ee1a90519c1149f0e68c7295f8e6ea86b8ca098f07065c69df191d9 offset: 0 snap -1 2024-09-18T11:31:42.199 INFO:tasks.rados.rados.0.smithi119.stdout:1514: set_chunk oid 15 target oid 2024-09-18T11:31:42.199 INFO:tasks.rados.rados.0.smithi119.stdout:1514: set_chunk oid 15 offset: 1294848 length: 318976 target oid 2567d672fbc2e6d01adc33cc3a3dc4e2bbdf0e4582c06760d8e7ed919e2a224e offset: 0 snap -1 2024-09-18T11:31:42.199 INFO:tasks.rados.rados.0.smithi119.stdout:1515: tier_flush oid 34 2024-09-18T11:31:42.199 INFO:tasks.rados.rados.0.smithi119.stdout:1515: tier_flush oid 34 snap -1 2024-09-18T11:31:42.199 INFO:tasks.rados.rados.0.smithi119.stdout:1511: finishing rollback tid 0 to smithi11939506-46 2024-09-18T11:31:42.199 INFO:tasks.rados.rados.0.smithi119.stdout:1512: finishing write tid 1 to smithi11939506-16 2024-09-18T11:31:42.199 INFO:tasks.rados.rados.0.smithi119.stdout:1516: read oid 14 snap -1 2024-09-18T11:31:42.199 INFO:tasks.rados.rados.0.smithi119.stdout:1516: expect (ObjNum 246 snap 28 seq_num 246) 2024-09-18T11:31:42.199 INFO:tasks.rados.rados.0.smithi119.stdout:1517: write (excl) oid 40 current snap is 41 2024-09-18T11:31:42.200 INFO:tasks.rados.rados.0.smithi119.stdout:1517: seq_num 287 ranges {423609=701524,1628332=472089,2648987=174622} 2024-09-18T11:31:42.200 INFO:tasks.rados.rados.0.smithi119.stdout:1517: writing smithi11939506-40 from 423609 to 1125133 tid 1 2024-09-18T11:31:42.200 INFO:tasks.rados.rados.0.smithi119.stdout:1517: writing smithi11939506-40 from 1628332 to 2100421 tid 2 2024-09-18T11:31:42.200 INFO:tasks.rados.rados.0.smithi119.stdout:1517: writing smithi11939506-40 from 2648987 to 2823609 tid 3 2024-09-18T11:31:42.200 INFO:tasks.rados.rados.0.smithi119.stdout:1518: set_chunk oid 5 target oid 2024-09-18T11:31:42.200 INFO:tasks.rados.rados.0.smithi119.stdout:1518: set_chunk oid 5 offset: 1384448 length: 343552 target oid 58bd5e51cbbfbe5b90d071ceb3caefb26e683a7df363e2e2686a1656e999ddf2 offset: 0 snap -1 2024-09-18T11:31:42.200 INFO:tasks.rados.rados.0.smithi119.stdout:1519: snap_create 2024-09-18T11:31:42.200 INFO:tasks.rados.rados.0.smithi119.stdout:1511: finishing rollback tid 1 to smithi11939506-46 2024-09-18T11:31:42.200 INFO:tasks.rados.rados.0.smithi119.stdout:1511: finishing rollback tid 2 to smithi11939506-46 2024-09-18T11:31:42.200 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 231 (ObjNum 171 snap 20 seq_num 171) dirty exists 2024-09-18T11:31:42.200 INFO:tasks.rados.rados.0.smithi119.stdout:1512: finishing write tid 2 to smithi11939506-16 2024-09-18T11:31:42.200 INFO:tasks.rados.rados.0.smithi119.stdout:1512: finishing write tid 3 to smithi11939506-16 2024-09-18T11:31:42.200 INFO:tasks.rados.rados.0.smithi119.stdout:1512: finishing write tid 4 to smithi11939506-16 2024-09-18T11:31:42.201 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 256 (ObjNum 286 snap 41 seq_num 286) dirty exists 2024-09-18T11:31:42.201 INFO:tasks.rados.rados.0.smithi119.stdout:1512: left oid 16 (ObjNum 286 snap 41 seq_num 286) 2024-09-18T11:31:42.201 INFO:tasks.rados.rados.0.smithi119.stdout:1513: finishing set_chunk to oid 17 2024-09-18T11:31:42.201 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 17 set_chunk returned error code -95 offset: 989184 length: 646656 2024-09-18T11:31:42.201 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 0 (ObjNum 247 snap 28 seq_num 247) dirty exists 2024-09-18T11:31:42.201 INFO:tasks.rados.rados.0.smithi119.stdout:1515: got (0) Success 2024-09-18T11:31:42.201 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 182 (ObjNum 264 snap 31 seq_num 264) dirty exists 2024-09-18T11:31:42.201 INFO:tasks.rados.rados.0.smithi119.stdout:1514: finishing set_chunk to oid 15 2024-09-18T11:31:42.201 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 109 (ObjNum 280 snap 37 seq_num 280) dirty exists 2024-09-18T11:31:42.201 INFO:tasks.rados.rados.0.smithi119.stdout:1516: expect (ObjNum 246 snap 28 seq_num 246) 2024-09-18T11:31:42.295 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:31:42.296 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:42.294+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:42.300 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:42.300 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:42.300 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:42.300 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:42.309 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:31:42.338 INFO:tasks.rados.rados.0.smithi119.stdout:1517: finishing write tid 1 to smithi11939506-40 2024-09-18T11:31:42.338 INFO:tasks.rados.rados.0.smithi119.stdout:1517: finishing write tid 2 to smithi11939506-40 2024-09-18T11:31:42.338 INFO:tasks.rados.rados.0.smithi119.stdout:1517: finishing write tid 3 to smithi11939506-40 2024-09-18T11:31:42.338 INFO:tasks.rados.rados.0.smithi119.stdout:1517: finishing write tid 4 to smithi11939506-40 2024-09-18T11:31:42.338 INFO:tasks.rados.rados.0.smithi119.stdout:1517: finishing write tid 5 to smithi11939506-40 2024-09-18T11:31:42.339 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 163 (ObjNum 287 snap 41 seq_num 287) dirty exists 2024-09-18T11:31:42.339 INFO:tasks.rados.rados.0.smithi119.stdout:1517: left oid 40 (ObjNum 287 snap 41 seq_num 287) 2024-09-18T11:31:42.339 INFO:tasks.rados.rados.0.smithi119.stdout:1518: finishing set_chunk to oid 5 2024-09-18T11:31:42.339 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 5 set_chunk returned error code -95 offset: 1384448 length: 343552 2024-09-18T11:31:42.339 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 0 (ObjNum 272 snap 34 seq_num 272) dirty exists 2024-09-18T11:31:42.396 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:42.396 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:42.394+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:42.425 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:42.426 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:42.441 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:42.441 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:42.450 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:42.450 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:42.456 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:42.456 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:42.459 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:31:42.471 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:42.472 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:42.487 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:42.487 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:42.497 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:42.495+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:42.503 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:42.503 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:42.518 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:42.518 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:42.522 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:42.522 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:42.571 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:42.572 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:42.580 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:31:42.598 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:42.597+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:42.678 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:42.678 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:42.686 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:31:42.730 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:42.728+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:42.784 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:42.784 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:42.793 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:31:42.834 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:42.832+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:42.900 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:42.898+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:42.931 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:42.931 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:42.940 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:31:43.016 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is unset 2024-09-18T11:31:43.017 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:43.015+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:43.018 INFO:tasks.rados.rados.0.smithi119.stdout:1511: done (8 left) 2024-09-18T11:31:43.018 INFO:tasks.rados.rados.0.smithi119.stdout:1512: done (7 left) 2024-09-18T11:31:43.018 INFO:tasks.rados.rados.0.smithi119.stdout:1513: done (6 left) 2024-09-18T11:31:43.018 INFO:tasks.rados.rados.0.smithi119.stdout:1514: done (5 left) 2024-09-18T11:31:43.018 INFO:tasks.rados.rados.0.smithi119.stdout:1515: done (4 left) 2024-09-18T11:31:43.018 INFO:tasks.rados.rados.0.smithi119.stdout:1516: done (3 left) 2024-09-18T11:31:43.018 INFO:tasks.rados.rados.0.smithi119.stdout:1517: done (2 left) 2024-09-18T11:31:43.018 INFO:tasks.rados.rados.0.smithi119.stdout:1518: done (1 left) 2024-09-18T11:31:43.019 INFO:tasks.rados.rados.0.smithi119.stdout:1519: done (0 left) 2024-09-18T11:31:43.019 INFO:tasks.rados.rados.0.smithi119.stdout:1520: tier_evict oid 33 2024-09-18T11:31:43.019 INFO:tasks.rados.rados.0.smithi119.stdout:1520: tier_evict oid 33 snap -1 2024-09-18T11:31:43.019 INFO:tasks.rados.rados.0.smithi119.stdout:1521: read oid 43 snap -1 2024-09-18T11:31:43.019 INFO:tasks.rados.rados.0.smithi119.stdout:1521: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:31:43.019 INFO:tasks.rados.rados.0.smithi119.stdout:1522: write oid 19 current snap is 42 2024-09-18T11:31:43.019 INFO:tasks.rados.rados.0.smithi119.stdout:1522: seq_num 288 ranges {440416=776773,1686814=732100} 2024-09-18T11:31:43.025 INFO:tasks.rados.rados.0.smithi119.stdout:1522: writing smithi11939506-19 from 440416 to 1217189 tid 1 2024-09-18T11:31:43.034 INFO:tasks.rados.rados.0.smithi119.stdout:1522: writing smithi11939506-19 from 1686814 to 2418914 tid 2 2024-09-18T11:31:43.034 INFO:tasks.rados.rados.0.smithi119.stdout:1523: read oid 34 snap -1 2024-09-18T11:31:43.034 INFO:tasks.rados.rados.0.smithi119.stdout:1523: expect (ObjNum 264 snap 31 seq_num 264) 2024-09-18T11:31:43.034 INFO:tasks.rados.rados.0.smithi119.stdout:1524: set_chunk oid 40 target oid 2024-09-18T11:31:43.044 INFO:tasks.rados.rados.0.smithi119.stdout:1524: set_chunk oid 40 offset: 1815552 length: 148480 target oid 35f15f03f04b7749d0e0cb8c80956847af6700da18dec6a7ea14e8276f3a15f7 offset: 0 snap -1 2024-09-18T11:31:43.044 INFO:tasks.rados.rados.0.smithi119.stdout:1525: read oid 11 snap 23 2024-09-18T11:31:43.044 INFO:tasks.rados.rados.0.smithi119.stdout:1525: expect (ObjNum 148 snap 18 seq_num 148) 2024-09-18T11:31:43.045 INFO:tasks.rados.rados.0.smithi119.stdout:1526: read oid 38 snap -1 2024-09-18T11:31:43.045 INFO:tasks.rados.rados.0.smithi119.stdout:1526: expect (ObjNum 231 snap 27 seq_num 231) 2024-09-18T11:31:43.045 INFO:tasks.rados.rados.0.smithi119.stdout:1520: got (22) Invalid argument 2024-09-18T11:31:43.045 INFO:tasks.rados.rados.0.smithi119.stdout:1527: read oid 28 snap -1 2024-09-18T11:31:43.045 INFO:tasks.rados.rados.0.smithi119.stdout:1527: expect (ObjNum 200 snap 23 seq_num 200) 2024-09-18T11:31:43.045 INFO:tasks.rados.rados.0.smithi119.stdout:1520: done (7 left) 2024-09-18T11:31:43.045 INFO:tasks.rados.rados.0.smithi119.stdout:1528: set_chunk oid 32 target oid 2024-09-18T11:31:43.055 INFO:tasks.rados.rados.0.smithi119.stdout:1528: set_chunk oid 32 offset: 1270272 length: 897024 target oid 492662a7b69eabb3685cc3f1b3f65890cfe051a201a043002855a319b2787ef0 offset: 0 snap -1 2024-09-18T11:31:43.056 INFO:tasks.rados.rados.0.smithi119.stdout:1529: read oid 41 snap -1 2024-09-18T11:31:43.056 INFO:tasks.rados.rados.0.smithi119.stdout:1529: expect (ObjNum 237 snap 28 seq_num 237) 2024-09-18T11:31:43.056 INFO:tasks.rados.rados.0.smithi119.stdout:1530: delete oid 50 current snap is 42 2024-09-18T11:31:43.056 INFO:tasks.rados.rados.0.smithi119.stdout:1521: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:31:43.097 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:43.097 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:43.101 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:43.099+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:43.106 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:31:43.148 INFO:tasks.rados.rados.0.smithi119.stdout:1522: finishing write tid 1 to smithi11939506-19 2024-09-18T11:31:43.148 INFO:tasks.rados.rados.0.smithi119.stdout:1521: done (9 left) 2024-09-18T11:31:43.148 INFO:tasks.rados.rados.0.smithi119.stdout:1530: done (8 left) 2024-09-18T11:31:43.148 INFO:tasks.rados.rados.0.smithi119.stdout:1531: tier_promote oid 24 2024-09-18T11:31:43.148 INFO:tasks.rados.rados.0.smithi119.stdout:1522: finishing write tid 2 to smithi11939506-19 2024-09-18T11:31:43.148 INFO:tasks.rados.rados.0.smithi119.stdout:1522: finishing write tid 3 to smithi11939506-19 2024-09-18T11:31:43.148 INFO:tasks.rados.rados.0.smithi119.stdout:1522: finishing write tid 4 to smithi11939506-19 2024-09-18T11:31:43.148 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 227 (ObjNum 288 snap 42 seq_num 288) dirty exists 2024-09-18T11:31:43.148 INFO:tasks.rados.rados.0.smithi119.stdout:1522: left oid 19 (ObjNum 288 snap 42 seq_num 288) 2024-09-18T11:31:43.148 INFO:tasks.rados.rados.0.smithi119.stdout:1523: expect (ObjNum 264 snap 31 seq_num 264) 2024-09-18T11:31:43.202 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:43.200+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:43.230 INFO:tasks.rados.rados.0.smithi119.stdout:1524: finishing set_chunk to oid 40 2024-09-18T11:31:43.230 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 163 (ObjNum 287 snap 41 seq_num 287) dirty exists 2024-09-18T11:31:43.230 INFO:tasks.rados.rados.0.smithi119.stdout:1527: expect (ObjNum 200 snap 23 seq_num 200) 2024-09-18T11:31:43.245 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:43.245 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:43.254 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:31:43.303 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:43.301+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:43.393 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:43.393 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:43.402 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:31:43.403 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:43.401+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:43.410 INFO:tasks.rados.rados.0.smithi119.stdout:1528: finishing set_chunk to oid 32 2024-09-18T11:31:43.410 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 32 set_chunk returned error code -95 offset: 1270272 length: 897024 2024-09-18T11:31:43.410 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 0 (ObjNum 217 snap 26 seq_num 217) dirty exists 2024-09-18T11:31:43.410 INFO:tasks.rados.rados.0.smithi119.stdout:1525: expect (ObjNum 148 snap 18 seq_num 148) 2024-09-18T11:31:43.489 INFO:tasks.rados.rados.0.smithi119.stdout:1526: expect (ObjNum 231 snap 27 seq_num 231) 2024-09-18T11:31:43.504 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:43.504 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:43.505 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:43.504+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:43.510 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:31:43.606 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:43.605+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:43.648 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:43.648 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:43.656 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:31:43.660 INFO:tasks.rados.rados.0.smithi119.stdout:1529: expect (ObjNum 237 snap 28 seq_num 237) 2024-09-18T11:31:43.740 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:43.739+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:43.787 INFO:tasks.rados.rados.0.smithi119.stdout:1531: got (0) Success 2024-09-18T11:31:43.787 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 207 (ObjNum 258 snap 31 seq_num 258) dirty exists 2024-09-18T11:31:43.787 INFO:tasks.rados.rados.0.smithi119.stdout:1522: done (8 left) 2024-09-18T11:31:43.787 INFO:tasks.rados.rados.0.smithi119.stdout:1523: done (7 left) 2024-09-18T11:31:43.787 INFO:tasks.rados.rados.0.smithi119.stdout:1524: done (6 left) 2024-09-18T11:31:43.787 INFO:tasks.rados.rados.0.smithi119.stdout:1525: done (5 left) 2024-09-18T11:31:43.787 INFO:tasks.rados.rados.0.smithi119.stdout:1526: done (4 left) 2024-09-18T11:31:43.787 INFO:tasks.rados.rados.0.smithi119.stdout:1527: done (3 left) 2024-09-18T11:31:43.787 INFO:tasks.rados.rados.0.smithi119.stdout:1528: done (2 left) 2024-09-18T11:31:43.787 INFO:tasks.rados.rados.0.smithi119.stdout:1529: done (1 left) 2024-09-18T11:31:43.787 INFO:tasks.rados.rados.0.smithi119.stdout:1531: done (0 left) 2024-09-18T11:31:43.787 INFO:tasks.rados.rados.0.smithi119.stdout:1532: read oid 4 snap -1 2024-09-18T11:31:43.787 INFO:tasks.rados.rados.0.smithi119.stdout:1532: expect (ObjNum 110 snap 8 seq_num 110) 2024-09-18T11:31:43.788 INFO:tasks.rados.rados.0.smithi119.stdout:1533: set_chunk oid 17 target oid 2024-09-18T11:31:43.797 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:43.797 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:43.799 INFO:tasks.rados.rados.0.smithi119.stdout:1533: set_chunk oid 17 offset: 184320 length: 1029120 target oid d0344b61b5098f91a6846a67b8df5cc66c7b37c79895ca90885b0c7f9dc84c7d offset: 0 snap -1 2024-09-18T11:31:43.799 INFO:tasks.rados.rados.0.smithi119.stdout:1534: read oid 28 snap -1 2024-09-18T11:31:43.799 INFO:tasks.rados.rados.0.smithi119.stdout:1534: expect (ObjNum 200 snap 23 seq_num 200) 2024-09-18T11:31:43.799 INFO:tasks.rados.rados.0.smithi119.stdout:1535: rollback oid 18 current snap is 42 2024-09-18T11:31:43.800 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 18 to 38 2024-09-18T11:31:43.800 INFO:tasks.rados.rados.0.smithi119.stdout:1532: expect (ObjNum 110 snap 8 seq_num 110) 2024-09-18T11:31:43.806 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:31:43.847 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:43.846+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:43.908 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:43.907+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:43.924 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2024-09-18T11:31:43.924 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-18T11:31:43.924 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-18T11:31:43.926 INFO:tasks.ceph.osd.0:Started 2024-09-18T11:31:43.926 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:31:43.945 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:43.945 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:43.954 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:31:43.977 INFO:tasks.rados.rados.0.smithi119.stdout:1533: finishing set_chunk to oid 17 2024-09-18T11:31:43.978 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 281 (ObjNum 247 snap 28 seq_num 247) dirty exists 2024-09-18T11:31:43.978 INFO:tasks.rados.rados.0.smithi119.stdout:1534: expect (ObjNum 200 snap 23 seq_num 200) 2024-09-18T11:31:44.009 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:44.007+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:44.019 INFO:teuthology.orchestra.run.smithi119.stderr:no valid command found; 10 closest matches: 2024-09-18T11:31:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:0 2024-09-18T11:31:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:1 2024-09-18T11:31:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:2 2024-09-18T11:31:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:abort 2024-09-18T11:31:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:assert 2024-09-18T11:31:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:config diff 2024-09-18T11:31:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:config diff get 2024-09-18T11:31:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:config get 2024-09-18T11:31:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:config help [] 2024-09-18T11:31:44.021 INFO:teuthology.orchestra.run.smithi119.stderr:config set ... 2024-09-18T11:31:44.021 INFO:teuthology.orchestra.run.smithi119.stderr:admin_socket: invalid command 2024-09-18T11:31:44.022 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T11:31:44.022 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-18T11:31:44.091 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:44.092 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:44.099 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:31:44.109 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:44.107+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:44.155 INFO:tasks.rados.rados.0.smithi119.stdout:1535: finishing rollback tid 0 to smithi11939506-18 2024-09-18T11:31:44.155 INFO:tasks.rados.rados.0.smithi119.stdout:1535: finishing rollback tid 1 to smithi11939506-18 2024-09-18T11:31:44.155 INFO:tasks.rados.rados.0.smithi119.stdout:1535: finishing rollback tid 2 to smithi11939506-18 2024-09-18T11:31:44.155 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 171 (ObjNum 205 snap 23 seq_num 205) dirty exists 2024-09-18T11:31:44.155 INFO:tasks.rados.rados.0.smithi119.stdout:1532: done (3 left) 2024-09-18T11:31:44.155 INFO:tasks.rados.rados.0.smithi119.stdout:1533: done (2 left) 2024-09-18T11:31:44.155 INFO:tasks.rados.rados.0.smithi119.stdout:1534: done (1 left) 2024-09-18T11:31:44.155 INFO:tasks.rados.rados.0.smithi119.stdout:1535: done (0 left) 2024-09-18T11:31:44.156 INFO:tasks.rados.rados.0.smithi119.stdout:1536: read oid 28 snap -1 2024-09-18T11:31:44.156 INFO:tasks.rados.rados.0.smithi119.stdout:1536: expect (ObjNum 200 snap 23 seq_num 200) 2024-09-18T11:31:44.156 INFO:tasks.rados.rados.0.smithi119.stdout:1537: write oid 30 current snap is 42 2024-09-18T11:31:44.156 INFO:tasks.rados.rados.0.smithi119.stdout:1537: seq_num 290 ranges {474030=643624,1520131=735769} 2024-09-18T11:31:44.160 INFO:tasks.rados.rados.0.smithi119.stdout:1537: writing smithi11939506-30 from 474030 to 1117654 tid 1 2024-09-18T11:31:44.167 INFO:tasks.rados.rados.0.smithi119.stdout:1537: writing smithi11939506-30 from 1520131 to 2255900 tid 2 2024-09-18T11:31:44.168 INFO:tasks.rados.rados.0.smithi119.stdout:1538: read oid 34 snap -1 2024-09-18T11:31:44.168 INFO:tasks.rados.rados.0.smithi119.stdout:1538: expect (ObjNum 264 snap 31 seq_num 264) 2024-09-18T11:31:44.168 INFO:tasks.rados.rados.0.smithi119.stdout:1539: read oid 8 snap -1 2024-09-18T11:31:44.168 INFO:tasks.rados.rados.0.smithi119.stdout:1539: expect deleted 2024-09-18T11:31:44.168 INFO:tasks.rados.rados.0.smithi119.stdout:1540: tier_flush oid 27 2024-09-18T11:31:44.168 INFO:tasks.rados.rados.0.smithi119.stdout:1540: tier_flush oid 27 snap -1 2024-09-18T11:31:44.168 INFO:tasks.rados.rados.0.smithi119.stdout:1541: snap_create 2024-09-18T11:31:44.168 INFO:tasks.rados.rados.0.smithi119.stdout:1536: expect (ObjNum 200 snap 23 seq_num 200) 2024-09-18T11:31:44.199 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:44.199 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:44.207 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:31:44.210 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:44.208+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:44.305 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:44.306 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:44.311 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:44.310+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:44.315 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:31:44.344 INFO:tasks.rados.rados.0.smithi119.stdout:1537: finishing write tid 1 to smithi11939506-30 2024-09-18T11:31:44.344 INFO:tasks.rados.rados.0.smithi119.stdout:1537: finishing write tid 2 to smithi11939506-30 2024-09-18T11:31:44.344 INFO:tasks.rados.rados.0.smithi119.stdout:1537: finishing write tid 3 to smithi11939506-30 2024-09-18T11:31:44.344 INFO:tasks.rados.rados.0.smithi119.stdout:1537: finishing write tid 4 to smithi11939506-30 2024-09-18T11:31:44.344 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 30 v 116 (ObjNum 290 snap 42 seq_num 290) dirty exists 2024-09-18T11:31:44.344 INFO:tasks.rados.rados.0.smithi119.stdout:1537: left oid 30 (ObjNum 290 snap 42 seq_num 290) 2024-09-18T11:31:44.344 INFO:tasks.rados.rados.0.smithi119.stdout:1538: expect (ObjNum 264 snap 31 seq_num 264) 2024-09-18T11:31:44.412 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:44.410+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:44.424 INFO:tasks.rados.rados.0.smithi119.stdout:1540: got (0) Success 2024-09-18T11:31:44.424 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 194 (ObjNum 49 snap 0 seq_num 384708240) clean dne 2024-09-18T11:31:44.453 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:44.453 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:44.462 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:31:44.509 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:31:44.507+0000 7fe41cf66440 -1 Falling back to public interface 2024-09-18T11:31:44.513 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:44.511+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:44.523 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:31:44.558 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:44.559 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:44.567 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:31:44.614 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:44.612+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:44.622 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:44.637 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:44.638 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:44.653 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:44.653 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:44.665 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:44.665 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:44.667 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:44.667 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:44.674 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:31:44.682 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:44.683 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:44.698 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:44.698 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:44.713 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:44.714 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:44.714 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:44.712+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:44.730 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:44.730 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:44.733 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:44.733 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:44.773 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:44.773 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:44.773 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:44.773 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:44.774 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:44.781 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:31:44.826 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:44.825+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:44.880 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:44.881 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:44.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:44.881 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:44.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.881 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:44.881 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:44.881 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 115.720339732, 2024-09-18T11:31:44.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:31:44.881 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:44.881 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:44.881 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:44.881 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:44.882 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:44.882 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:31:44.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.882 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:44.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:44.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:44.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:44.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:44.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:44.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:31:44.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:44.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:44.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:31:44.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:31:44.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:44.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:31:44.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:31:44.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:44.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:31:44.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:44.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:44.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:31:44.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:31:44.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:44.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:31:44.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:31:44.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:44.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:31:44.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:31:44.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:44.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:31:44.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:31:44.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:44.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:31:44.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:31:44.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:44.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:31:44.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:31:44.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:44.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:31:44.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:31:44.887 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.887 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:44.887 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.887 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:44.887 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:44.887 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 115.72021347, 2024-09-18T11:31:44.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055717040000000002, 2024-09-18T11:31:44.887 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:44.887 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:44.887 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:44.888 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:44.888 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:44.888 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 45 2024-09-18T11:31:44.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.888 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:44.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:44.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:44.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:44.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:44.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:44.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157676+0000", 2024-09-18T11:31:44.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:44.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:44.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157694+0000", 2024-09-18T11:31:44.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7422999999999999e-05 2024-09-18T11:31:44.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:44.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157704+0000", 2024-09-18T11:31:44.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9000000000000001e-06 2024-09-18T11:31:44.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:44.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157718+0000", 2024-09-18T11:31:44.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4788e-05 2024-09-18T11:31:44.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:44.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160937+0000", 2024-09-18T11:31:44.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00321886 2024-09-18T11:31:44.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:44.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161023+0000", 2024-09-18T11:31:44.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6063999999999995e-05 2024-09-18T11:31:44.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:44.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161202+0000", 2024-09-18T11:31:44.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017912599999999999 2024-09-18T11:31:44.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:44.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163052+0000", 2024-09-18T11:31:44.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001849554 2024-09-18T11:31:44.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:44.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163204+0000", 2024-09-18T11:31:44.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015186900000000001 2024-09-18T11:31:44.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:44.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163229+0000", 2024-09-18T11:31:44.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:31:44.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:44.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163249+0000", 2024-09-18T11:31:44.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9992000000000001e-05 2024-09-18T11:31:44.893 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.893 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:44.893 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.893 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:44.893 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:44.893 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 115.720164357, 2024-09-18T11:31:44.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:31:44.893 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:44.893 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:44.893 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:44.894 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:44.894 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:44.894 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:31:44.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.894 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:44.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:44.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:44.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:44.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:44.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:44.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:44.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:44.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:44.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:31:44.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:31:44.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:44.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:31:44.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:31:44.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:44.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:31:44.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:31:44.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:44.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:31:44.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:31:44.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:44.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:31:44.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:31:44.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:44.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:31:44.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:31:44.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:44.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:31:44.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:31:44.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:44.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:31:44.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:31:44.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:44.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:31:44.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:31:44.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:44.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:31:44.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:31:44.898 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.899 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:44.899 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.899 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:44.899 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:44.899 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 115.720121826, 2024-09-18T11:31:44.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:31:44.899 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:44.899 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:44.899 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:44.899 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:44.899 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:44.900 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:31:44.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.900 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:44.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:44.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:44.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:44.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:44.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:44.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:31:44.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:44.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:44.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:31:44.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:31:44.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:44.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:31:44.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:31:44.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:44.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:31:44.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:31:44.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:44.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:31:44.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:31:44.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:44.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:31:44.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:31:44.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:44.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:31:44.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:31:44.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:44.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:31:44.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:31:44.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:44.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:31:44.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:31:44.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:44.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:31:44.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:31:44.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:44.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:31:44.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:31:44.904 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.904 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:44.904 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.905 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:44.905 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:44.905 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 115.72006689600001, 2024-09-18T11:31:44.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:31:44.905 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:44.905 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:44.905 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:44.905 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:44.905 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:44.905 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:31:44.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.905 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:44.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:44.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:44.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:44.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:44.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:44.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:44.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:44.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:44.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:31:44.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:31:44.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:44.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:31:44.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:31:44.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:44.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:31:44.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:31:44.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:44.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:31:44.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:31:44.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:44.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:31:44.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:31:44.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:44.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:31:44.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:31:44.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:44.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:31:44.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:31:44.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:44.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:31:44.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:31:44.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:44.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:31:44.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:31:44.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:44.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:31:44.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:31:44.910 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.910 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:44.910 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.910 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:44.910 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:44.910 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 115.720056916, 2024-09-18T11:31:44.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:31:44.910 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:44.910 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:44.910 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:44.911 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:44.911 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:44.911 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:31:44.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.911 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:44.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:44.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:44.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:44.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:44.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:44.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:44.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:44.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:44.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:31:44.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:31:44.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:44.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:31:44.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:31:44.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:44.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:31:44.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:31:44.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:44.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:31:44.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:31:44.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:44.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:31:44.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:31:44.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:44.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:31:44.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:31:44.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:44.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:31:44.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:31:44.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:44.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:31:44.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:31:44.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:44.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:31:44.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:31:44.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:44.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:31:44.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:31:44.916 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.916 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:44.916 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.916 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:44.916 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:44.916 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 115.720045317, 2024-09-18T11:31:44.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:31:44.916 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:44.916 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:44.916 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:44.916 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:44.917 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:44.917 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:31:44.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.917 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:44.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:44.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:44.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:44.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:44.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:44.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:44.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:44.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:44.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:31:44.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:31:44.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:44.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:31:44.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:31:44.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:44.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:31:44.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:31:44.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:44.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:31:44.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:31:44.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:44.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:31:44.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:31:44.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:44.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:31:44.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:31:44.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:44.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:31:44.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:31:44.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:44.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:31:44.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:31:44.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:44.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:31:44.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:31:44.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:44.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:31:44.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:31:44.921 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.922 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:44.922 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.922 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:44.922 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:44.922 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 115.719988836, 2024-09-18T11:31:44.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:31:44.922 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:44.922 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:44.922 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:44.922 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:44.922 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:44.922 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:31:44.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.923 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:44.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:44.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:44.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:44.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:44.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:44.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:31:44.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:44.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:44.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:31:44.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:31:44.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:44.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:31:44.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:31:44.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:44.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:31:44.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:31:44.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:44.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:31:44.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:31:44.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:44.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:31:44.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:31:44.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:44.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:31:44.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:31:44.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:44.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:31:44.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:31:44.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:44.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:31:44.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:31:44.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:44.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:31:44.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:31:44.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:44.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:31:44.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:31:44.928 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.928 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:44.928 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.928 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:44.928 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:44.928 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 115.71997501200001, 2024-09-18T11:31:44.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:31:44.929 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:44.929 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:44.929 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:44.929 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:44.929 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:44.929 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:31:44.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.929 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:44.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:44.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:44.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:44.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:44.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:44.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:44.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:44.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:44.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:31:44.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:31:44.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:44.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:31:44.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:31:44.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:44.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:31:44.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:31:44.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:44.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:31:44.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:31:44.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:44.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:31:44.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:31:44.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:44.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:31:44.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:31:44.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:44.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:31:44.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:31:44.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:44.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:31:44.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:31:44.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:44.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:31:44.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:31:44.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:44.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:31:44.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:31:44.934 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.934 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:44.934 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.934 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:44.934 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:44.934 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 115.303796105, 2024-09-18T11:31:44.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:31:44.935 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:44.935 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:44.935 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:44.935 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:44.935 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:44.935 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:31:44.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.935 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:44.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:44.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:44.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:44.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:44.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:44.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:44.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:44.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:44.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:31:44.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:31:44.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:44.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:31:44.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:31:44.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:44.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:31:44.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:31:44.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:44.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:31:44.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:31:44.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:44.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:31:44.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:31:44.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:44.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:31:44.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:31:44.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:44.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:31:44.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:31:44.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:44.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:31:44.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:31:44.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:44.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:31:44.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:31:44.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:44.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:31:44.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:31:44.943 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.943 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:44.943 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.943 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:44.943 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:44.943 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 115.30377280499999, 2024-09-18T11:31:44.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:31:44.943 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:44.943 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:44.944 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:44.944 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:44.944 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:44.944 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:31:44.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.944 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:44.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:44.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:44.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:44.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:44.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:44.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:31:44.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:44.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:44.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:31:44.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:31:44.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:44.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:31:44.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:31:44.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:44.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:31:44.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:31:44.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:44.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:31:44.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:31:44.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:44.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:31:44.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:31:44.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:44.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:31:44.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:31:44.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:44.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:31:44.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:31:44.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:44.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:31:44.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:31:44.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:44.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:31:44.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:31:44.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:44.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:31:44.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:31:44.949 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.949 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:44.949 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.949 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:44.949 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:44.949 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 115.303687196, 2024-09-18T11:31:44.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:31:44.949 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:44.949 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:44.949 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:44.949 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:44.949 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:44.950 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:31:44.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.950 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:44.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:44.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:44.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:44.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:44.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:44.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:44.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:44.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:44.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:31:44.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:31:44.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:44.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:31:44.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:31:44.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:44.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:31:44.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:31:44.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:44.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:31:44.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:31:44.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:44.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:31:44.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:31:44.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:44.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:31:44.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:31:44.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:44.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:31:44.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:31:44.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:44.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:31:44.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:31:44.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:44.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:31:44.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:31:44.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:44.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:31:44.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:31:44.954 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.954 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:44.954 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.955 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:44.955 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:44.955 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 115.303676231, 2024-09-18T11:31:44.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:31:44.955 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:44.955 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:44.955 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:44.955 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:44.955 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:44.955 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:31:44.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.955 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:44.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:44.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:44.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:44.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:44.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:44.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:44.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:44.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:44.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:31:44.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:31:44.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:44.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:31:44.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:31:44.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:44.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:31:44.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:31:44.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:44.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:31:44.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:31:44.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:44.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:31:44.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:31:44.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:44.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:31:44.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:31:44.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:44.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:31:44.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:31:44.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:44.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:31:44.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:31:44.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:44.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:31:44.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:31:44.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:44.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:31:44.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:31:44.960 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.960 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:44.960 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.960 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:44.960 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:44.961 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 115.303665063, 2024-09-18T11:31:44.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:31:44.961 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:44.961 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:44.961 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:44.961 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:44.961 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:44.961 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:31:44.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.961 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:44.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:44.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:44.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:44.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:44.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:44.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:44.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:44.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:44.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:31:44.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:31:44.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:44.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:31:44.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:31:44.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:44.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:31:44.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:31:44.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:44.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:31:44.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:31:44.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:44.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:31:44.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:31:44.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:44.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:31:44.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:31:44.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:44.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:31:44.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:31:44.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:44.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:31:44.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:31:44.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:44.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:31:44.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:31:44.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:44.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:31:44.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:31:44.966 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.966 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:44.966 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.966 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:44.966 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:44.966 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 115.303599734, 2024-09-18T11:31:44.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:31:44.967 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:44.967 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:44.967 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:44.967 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:44.967 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:44.967 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:31:44.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.967 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:44.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:44.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:44.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:44.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:44.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:44.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:31:44.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:44.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:44.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:31:44.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:31:44.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:44.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:31:44.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:31:44.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:44.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:31:44.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:31:44.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:44.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:31:44.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:31:44.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:44.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:31:44.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:31:44.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:44.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:31:44.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:31:44.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:44.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:31:44.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:31:44.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:44.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:31:44.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:31:44.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:44.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:31:44.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:31:44.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:44.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:31:44.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:31:44.972 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.972 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:44.972 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.973 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:44.973 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:44.973 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 115.293437756, 2024-09-18T11:31:44.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:31:44.973 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:44.973 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:44.973 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:44.973 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:44.973 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:44.973 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:31:44.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.973 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:44.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:44.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:44.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:44.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:44.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:44.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:31:44.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:44.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:44.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:31:44.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:31:44.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:44.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:31:44.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:31:44.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:44.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:31:44.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:31:44.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:44.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:31:44.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:31:44.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:44.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:31:44.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:31:44.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:44.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:31:44.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:31:44.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:44.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:31:44.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:31:44.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:44.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:31:44.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:31:44.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:44.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:31:44.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:31:44.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:44.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:31:44.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:31:44.978 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.978 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:44.978 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.978 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:44.978 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:44.978 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 115.293345277, 2024-09-18T11:31:44.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:31:44.979 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:44.979 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:44.979 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:44.979 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:44.979 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:44.979 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:31:44.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.979 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:44.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:44.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:44.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:44.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:44.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:44.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:44.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:44.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:44.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:31:44.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:31:44.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:44.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:31:44.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:31:44.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:44.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:31:44.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:31:44.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:44.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:31:44.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:31:44.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:44.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:31:44.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:31:44.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:44.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:31:44.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:31:44.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:44.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:31:44.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:31:44.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:44.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:31:44.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:31:44.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:44.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:31:44.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:31:44.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:44.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:31:44.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:31:44.984 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.984 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:44.984 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.984 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1788 4.8 e60/22 4:1898b15d:::smithi11939506-25:head v 60'12, mlcod=58'9)", 2024-09-18T11:31:44.984 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:44.984 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 61.960939787999997, 2024-09-18T11:31:44.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058348599999999999, 2024-09-18T11:31:44.984 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:44.984 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:44.984 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:44.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:44.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:44.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:44.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:44.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:44.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916949+0000", 2024-09-18T11:31:44.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:44.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:44.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917602+0000", 2024-09-18T11:31:44.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00065337100000000003 2024-09-18T11:31:44.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:44.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917621+0000", 2024-09-18T11:31:44.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9287e-05 2024-09-18T11:31:44.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:44.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917629+0000", 2024-09-18T11:31:44.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3780000000000003e-06 2024-09-18T11:31:44.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:44.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917677+0000", 2024-09-18T11:31:44.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8068000000000001e-05 2024-09-18T11:31:44.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:44.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917699+0000", 2024-09-18T11:31:44.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2149000000000001e-05 2024-09-18T11:31:44.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:44.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922747+0000", 2024-09-18T11:31:44.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050477609999999996 2024-09-18T11:31:44.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:44.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922785+0000", 2024-09-18T11:31:44.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8726000000000002e-05 2024-09-18T11:31:44.988 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.988 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:44.988 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.989 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:58 4.3 e91/67 4:c70c3162:::smithi11939506-35:18 v 91'101, mlcod=86'95)", 2024-09-18T11:31:44.989 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:44.989 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 30.862281565, 2024-09-18T11:31:44.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055647500000000003, 2024-09-18T11:31:44.989 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:44.989 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:44.989 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:44.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:44.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:44.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:44.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:44.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:44.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015608+0000", 2024-09-18T11:31:44.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:44.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:44.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015614+0000", 2024-09-18T11:31:44.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5310000000000001e-06 2024-09-18T11:31:44.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:44.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015622+0000", 2024-09-18T11:31:44.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0809999999999994e-06 2024-09-18T11:31:44.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:44.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015627+0000", 2024-09-18T11:31:44.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8439999999999997e-06 2024-09-18T11:31:44.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:44.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016003+0000", 2024-09-18T11:31:44.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037652999999999997 2024-09-18T11:31:44.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:44.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016024+0000", 2024-09-18T11:31:44.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0780000000000001e-05 2024-09-18T11:31:44.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:44.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021141+0000", 2024-09-18T11:31:44.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051166639999999999 2024-09-18T11:31:44.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:44.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021174+0000", 2024-09-18T11:31:44.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2839000000000002e-05 2024-09-18T11:31:44.993 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.993 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:44.993 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.993 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3209 4.0 e109/22 4:019b6e09:::smithi11939506-40:head v 109'149, mlcod=106'146)", 2024-09-18T11:31:44.993 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:31:44.993 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.7785044259999996, 2024-09-18T11:31:44.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063064039999999998, 2024-09-18T11:31:44.993 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:44.994 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:44.994 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:44.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:44.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:31:44.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:44.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:31:44.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:44.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:44.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099385+0000", 2024-09-18T11:31:44.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:44.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:44.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099951+0000", 2024-09-18T11:31:44.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056606600000000005 2024-09-18T11:31:44.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:44.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099964+0000", 2024-09-18T11:31:44.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2918999999999999e-05 2024-09-18T11:31:44.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:44.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099972+0000", 2024-09-18T11:31:44.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2300000000000002e-06 2024-09-18T11:31:44.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:44.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.100026+0000", 2024-09-18T11:31:44.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4456999999999997e-05 2024-09-18T11:31:44.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:44.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.100046+0000", 2024-09-18T11:31:44.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0417000000000001e-05 2024-09-18T11:31:44.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:44.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.105653+0000", 2024-09-18T11:31:44.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005606214 2024-09-18T11:31:44.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:44.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:44.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:44.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.105692+0000", 2024-09-18T11:31:44.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9671e-05 2024-09-18T11:31:44.997 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.997 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:44.998 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.998 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:44.998 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:44.998 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:44.999 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:31:45.000 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:44.998+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:45.032 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T11:31:45.100 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:45.098+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:45.102 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:45.102 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:45.103 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:45.103 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:45.112 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:31:45.123 INFO:tasks.rados.rados.0.smithi119.stdout:1536: done (5 left) 2024-09-18T11:31:45.123 INFO:tasks.rados.rados.0.smithi119.stdout:1537: done (4 left) 2024-09-18T11:31:45.123 INFO:tasks.rados.rados.0.smithi119.stdout:1538: done (3 left) 2024-09-18T11:31:45.123 INFO:tasks.rados.rados.0.smithi119.stdout:1539: done (2 left) 2024-09-18T11:31:45.123 INFO:tasks.rados.rados.0.smithi119.stdout:1540: done (1 left) 2024-09-18T11:31:45.123 INFO:tasks.rados.rados.0.smithi119.stdout:1541: done (0 left) 2024-09-18T11:31:45.123 INFO:tasks.rados.rados.0.smithi119.stdout:1542: read oid 16 snap 39 2024-09-18T11:31:45.123 INFO:tasks.rados.rados.0.smithi119.stdout:1542: expect (ObjNum 268 snap 31 seq_num 268) 2024-09-18T11:31:45.123 INFO:tasks.rados.rados.0.smithi119.stdout:1543: read oid 33 snap -1 2024-09-18T11:31:45.123 INFO:tasks.rados.rados.0.smithi119.stdout:1543: expect deleted 2024-09-18T11:31:45.123 INFO:tasks.rados.rados.0.smithi119.stdout:1544: write oid 36 current snap is 43 2024-09-18T11:31:45.123 INFO:tasks.rados.rados.0.smithi119.stdout:1544: seq_num 291 ranges {490837=718873,1862260=479427,2875032=15805} 2024-09-18T11:31:45.124 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:45.122+0000 7f6bf5056640 -1 osd.5 115 handle_osd_map injecting map crc failure 2024-09-18T11:31:45.130 INFO:tasks.rados.rados.0.smithi119.stdout:1544: writing smithi11939506-36 from 490837 to 1209710 tid 1 2024-09-18T11:31:45.136 INFO:tasks.rados.rados.0.smithi119.stdout:1544: writing smithi11939506-36 from 1862260 to 2341687 tid 2 2024-09-18T11:31:45.139 INFO:tasks.rados.rados.0.smithi119.stdout:1544: writing smithi11939506-36 from 2875032 to 2890837 tid 3 2024-09-18T11:31:45.139 INFO:tasks.rados.rados.0.smithi119.stdout:1545: read oid 38 snap 31 2024-09-18T11:31:45.140 INFO:tasks.rados.rados.0.smithi119.stdout:1545: expect (ObjNum 231 snap 27 seq_num 231) 2024-09-18T11:31:45.140 INFO:tasks.rados.rados.0.smithi119.stdout:1546: read oid 42 snap -1 2024-09-18T11:31:45.140 INFO:tasks.rados.rados.0.smithi119.stdout:1546: expect (ObjNum 251 snap 30 seq_num 251) 2024-09-18T11:31:45.140 INFO:tasks.rados.rados.0.smithi119.stdout:1547: write (excl) oid 49 current snap is 43 2024-09-18T11:31:45.140 INFO:tasks.rados.rados.0.smithi119.stdout:1547: seq_num 292 ranges {507644=794122,1920742=623085} 2024-09-18T11:31:45.147 INFO:tasks.rados.rados.0.smithi119.stdout:1547: writing smithi11939506-49 from 507644 to 1301766 tid 1 2024-09-18T11:31:45.154 INFO:tasks.rados.rados.0.smithi119.stdout:1547: writing smithi11939506-49 from 1920742 to 2543827 tid 2 2024-09-18T11:31:45.154 INFO:tasks.rados.rados.0.smithi119.stdout:1548: read oid 3 snap -1 2024-09-18T11:31:45.154 INFO:tasks.rados.rados.0.smithi119.stdout:1548: expect (ObjNum 243 snap 28 seq_num 243) 2024-09-18T11:31:45.154 INFO:tasks.rados.rados.0.smithi119.stdout:1549: delete oid 4 current snap is 43 2024-09-18T11:31:45.154 INFO:tasks.rados.rados.0.smithi119.stdout:1542: expect (ObjNum 268 snap 31 seq_num 268) 2024-09-18T11:31:45.201 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:45.199+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:45.223 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:45.223 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:45.224 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:45.224 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:45.224 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:45.232 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:31:45.316 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:45.314+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:45.349 INFO:tasks.rados.rados.0.smithi119.stdout:1544: finishing write tid 1 to smithi11939506-36 2024-09-18T11:31:45.349 INFO:tasks.rados.rados.0.smithi119.stdout:1544: finishing write tid 2 to smithi11939506-36 2024-09-18T11:31:45.349 INFO:tasks.rados.rados.0.smithi119.stdout:1546: expect (ObjNum 251 snap 30 seq_num 251) 2024-09-18T11:31:45.373 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:45.374 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:45.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:45.374 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:45.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.374 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:888 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:45.374 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:45.374 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 94.254312966000001, 2024-09-18T11:31:45.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082282120000000004, 2024-09-18T11:31:45.374 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.374 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:45.374 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:45.374 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:45.375 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:45.375 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 888 2024-09-18T11:31:45.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.375 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:45.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:45.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116420+0000", 2024-09-18T11:31:45.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:45.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116423+0000", 2024-09-18T11:31:45.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1e-06 2024-09-18T11:31:45.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:31:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5510000000000001e-06 2024-09-18T11:31:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116431+0000", 2024-09-18T11:31:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6109999999999998e-06 2024-09-18T11:31:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116527+0000", 2024-09-18T11:31:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6212999999999995e-05 2024-09-18T11:31:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116532+0000", 2024-09-18T11:31:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6140000000000002e-06 2024-09-18T11:31:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124314+0000", 2024-09-18T11:31:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077818929999999998 2024-09-18T11:31:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124411+0000", 2024-09-18T11:31:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6992999999999998e-05 2024-09-18T11:31:45.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124649+0000", 2024-09-18T11:31:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000237729 2024-09-18T11:31:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:889 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~1003172] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 94.254288316, 2024-09-18T11:31:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085383349999999993, 2024-09-18T11:31:45.379 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 889 2024-09-18T11:31:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:45.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116445+0000", 2024-09-18T11:31:45.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:45.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116447+0000", 2024-09-18T11:31:45.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.854e-06 2024-09-18T11:31:45.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116452+0000", 2024-09-18T11:31:45.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0250000000000002e-06 2024-09-18T11:31:45.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116455+0000", 2024-09-18T11:31:45.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8739999999999999e-06 2024-09-18T11:31:45.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116565+0000", 2024-09-18T11:31:45.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000110741 2024-09-18T11:31:45.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:45.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.116571+0000", 2024-09-18T11:31:45.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4990000000000002e-06 2024-09-18T11:31:45.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124691+0000", 2024-09-18T11:31:45.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0081200339999999999 2024-09-18T11:31:45.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124784+0000", 2024-09-18T11:31:45.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2886999999999993e-05 2024-09-18T11:31:45.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.124983+0000", 2024-09-18T11:31:45.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019969000000000001 2024-09-18T11:31:45.384 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.384 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.384 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.384 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1126 3.0 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head [write 0~2520064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:31:45.384 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:31:45.384 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.388846337999993, 2024-09-18T11:31:45.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086197050000000001, 2024-09-18T11:31:45.385 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.385 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.385 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:45.385 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:45.385 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:45.385 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1126 2024-09-18T11:31:45.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.385 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:31:45.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981887+0000", 2024-09-18T11:31:45.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.981886+0000", 2024-09-18T11:31:45.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:45.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983017+0000", 2024-09-18T11:31:45.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00113131 2024-09-18T11:31:45.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983037+0000", 2024-09-18T11:31:45.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0231e-05 2024-09-18T11:31:45.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983044+0000", 2024-09-18T11:31:45.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9539999999999998e-06 2024-09-18T11:31:45.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983087+0000", 2024-09-18T11:31:45.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3040000000000001e-05 2024-09-18T11:31:45.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983193+0000", 2024-09-18T11:31:45.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000105682 2024-09-18T11:31:45.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:45.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983287+0000", 2024-09-18T11:31:45.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4066000000000004e-05 2024-09-18T11:31:45.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:45.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.986918+0000", 2024-09-18T11:31:45.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036311970000000001 2024-09-18T11:31:45.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:45.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990442+0000", 2024-09-18T11:31:45.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035243119999999999 2024-09-18T11:31:45.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990476+0000", 2024-09-18T11:31:45.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4028e-05 2024-09-18T11:31:45.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990507+0000", 2024-09-18T11:31:45.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0284999999999999e-05 2024-09-18T11:31:45.390 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.390 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.390 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.390 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:262 4.9 e53/22 4:99feffe9:::smithi11939506-45:13 v 53'17, mlcod=51'14)", 2024-09-18T11:31:45.390 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:45.391 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 71.371742522999995, 2024-09-18T11:31:45.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013878679, 2024-09-18T11:31:45.391 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.391 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.391 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:45.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:45.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:45.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:45.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998997+0000", 2024-09-18T11:31:45.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1099999999999999e-06 2024-09-18T11:31:45.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999008+0000", 2024-09-18T11:31:45.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1321000000000001e-05 2024-09-18T11:31:45.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999015+0000", 2024-09-18T11:31:45.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3000000000000004e-06 2024-09-18T11:31:45.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999055+0000", 2024-09-18T11:31:45.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0058000000000002e-05 2024-09-18T11:31:45.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999070+0000", 2024-09-18T11:31:45.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4328000000000001e-05 2024-09-18T11:31:45.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012465+0000", 2024-09-18T11:31:45.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013395638 2024-09-18T11:31:45.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012870+0000", 2024-09-18T11:31:45.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040423400000000001 2024-09-18T11:31:45.395 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.395 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.395 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.395 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:205 4.b e53/22 4:dba5bd39:::smithi11939506-17:13 v 53'77, mlcod=51'74)", 2024-09-18T11:31:45.395 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:45.395 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 71.371693261999994, 2024-09-18T11:31:45.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013359586, 2024-09-18T11:31:45.395 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.395 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.395 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:45.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:45.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:45.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:45.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999048+0000", 2024-09-18T11:31:45.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3550000000000005e-06 2024-09-18T11:31:45.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999056+0000", 2024-09-18T11:31:45.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3259999999999996e-06 2024-09-18T11:31:45.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999063+0000", 2024-09-18T11:31:45.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6409999999999996e-06 2024-09-18T11:31:45.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999098+0000", 2024-09-18T11:31:45.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5203999999999997e-05 2024-09-18T11:31:45.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999115+0000", 2024-09-18T11:31:45.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6478999999999999e-05 2024-09-18T11:31:45.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012362+0000", 2024-09-18T11:31:45.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01324768 2024-09-18T11:31:45.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012400+0000", 2024-09-18T11:31:45.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.731e-05 2024-09-18T11:31:45.399 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.399 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.399 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.399 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:88 4.a e53/22 4:588debc2:::smithi11939506-18:13 v 53'45, mlcod=51'42)", 2024-09-18T11:31:45.400 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:45.400 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 71.368910838999994, 2024-09-18T11:31:45.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011307667, 2024-09-18T11:31:45.400 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.400 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.400 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:45.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:45.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001822+0000", 2024-09-18T11:31:45.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:45.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001827+0000", 2024-09-18T11:31:45.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.378e-06 2024-09-18T11:31:45.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001839+0000", 2024-09-18T11:31:45.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1225e-05 2024-09-18T11:31:45.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001844+0000", 2024-09-18T11:31:45.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2299999999999999e-06 2024-09-18T11:31:45.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001891+0000", 2024-09-18T11:31:45.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6961e-05 2024-09-18T11:31:45.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001906+0000", 2024-09-18T11:31:45.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5158e-05 2024-09-18T11:31:45.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013103+0000", 2024-09-18T11:31:45.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011197314 2024-09-18T11:31:45.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013130+0000", 2024-09-18T11:31:45.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6900999999999999e-05 2024-09-18T11:31:45.404 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.404 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.404 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.404 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:89 4.a e53/22 4:552523b6:::smithi11939506-38:13 v 53'47, mlcod=51'42)", 2024-09-18T11:31:45.404 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:45.404 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 71.368351618999995, 2024-09-18T11:31:45.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011327614, 2024-09-18T11:31:45.404 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.404 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.404 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:45.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:45.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002381+0000", 2024-09-18T11:31:45.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:45.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002397+0000", 2024-09-18T11:31:45.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5688e-05 2024-09-18T11:31:45.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002406+0000", 2024-09-18T11:31:45.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0229999999999998e-06 2024-09-18T11:31:45.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002415+0000", 2024-09-18T11:31:45.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8959999999999992e-06 2024-09-18T11:31:45.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012386+0000", 2024-09-18T11:31:45.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0099711450000000007 2024-09-18T11:31:45.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012405+0000", 2024-09-18T11:31:45.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9389e-05 2024-09-18T11:31:45.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013676+0000", 2024-09-18T11:31:45.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012708649999999999 2024-09-18T11:31:45.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013709+0000", 2024-09-18T11:31:45.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3207000000000002e-05 2024-09-18T11:31:45.408 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.408 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.408 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.409 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1644 4.7 4:e73ba68f:::smithi11939506-1:head [sparse-read 0~2695582] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:45.409 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:45.409 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.036829320999999, 2024-09-18T11:31:45.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084764039999999999, 2024-09-18T11:31:45.409 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.409 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:45.409 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:45.409 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:45.409 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:45.409 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1644 2024-09-18T11:31:45.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.409 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:45.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:45.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333904+0000", 2024-09-18T11:31:45.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:45.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333906+0000", 2024-09-18T11:31:45.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.243e-06 2024-09-18T11:31:45.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333911+0000", 2024-09-18T11:31:45.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2959999999999998e-06 2024-09-18T11:31:45.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333915+0000", 2024-09-18T11:31:45.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6720000000000002e-06 2024-09-18T11:31:45.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341879+0000", 2024-09-18T11:31:45.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079638699999999996 2024-09-18T11:31:45.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.341978+0000", 2024-09-18T11:31:45.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9562000000000007e-05 2024-09-18T11:31:45.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342380+0000", 2024-09-18T11:31:45.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000402113 2024-09-18T11:31:45.413 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.413 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.413 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.413 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1645 4.7 4:e73ba68f:::smithi11939506-1:head [read 0~3983226,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:45.413 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:45.413 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.036814426999996, 2024-09-18T11:31:45.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010539468999999999, 2024-09-18T11:31:45.413 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.413 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:45.413 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:45.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:45.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:45.414 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1645 2024-09-18T11:31:45.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.414 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:45.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:45.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:45.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:45.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333921+0000", 2024-09-18T11:31:45.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8909999999999999e-06 2024-09-18T11:31:45.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333927+0000", 2024-09-18T11:31:45.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4229999999999996e-06 2024-09-18T11:31:45.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333932+0000", 2024-09-18T11:31:45.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8280000000000002e-06 2024-09-18T11:31:45.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342418+0000", 2024-09-18T11:31:45.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084862219999999999 2024-09-18T11:31:45.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342515+0000", 2024-09-18T11:31:45.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7287999999999994e-05 2024-09-18T11:31:45.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.344458+0000", 2024-09-18T11:31:45.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019431660000000001 2024-09-18T11:31:45.417 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.417 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.417 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.417 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1669 4.4 4:2d9c3f7f:::smithi11939506-5:head [read 0~3764735,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-18T11:31:45.418 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:31:45.418 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 66.547865529000006, 2024-09-18T11:31:45.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0083694990000000007, 2024-09-18T11:31:45.418 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.418 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:45.418 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:45.418 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:45.418 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:45.418 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1669 2024-09-18T11:31:45.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.418 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:31:45.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822868+0000", 2024-09-18T11:31:45.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822867+0000", 2024-09-18T11:31:45.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:45.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822871+0000", 2024-09-18T11:31:45.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:31:45.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822877+0000", 2024-09-18T11:31:45.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8000000000000004e-06 2024-09-18T11:31:45.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822880+0000", 2024-09-18T11:31:45.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5669999999999999e-06 2024-09-18T11:31:45.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822974+0000", 2024-09-18T11:31:45.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4048999999999993e-05 2024-09-18T11:31:45.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:45.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.822980+0000", 2024-09-18T11:31:45.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2789999999999998e-06 2024-09-18T11:31:45.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.829160+0000", 2024-09-18T11:31:45.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0061800179999999998 2024-09-18T11:31:45.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.829248+0000", 2024-09-18T11:31:45.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7707000000000006e-05 2024-09-18T11:31:45.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:38.831237+0000", 2024-09-18T11:31:45.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001989827 2024-09-18T11:31:45.422 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.422 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.422 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.422 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:100 4.a e58/22 4:5975a5ec:::smithi11939506-36:18 v 58'55, mlcod=56'50)", 2024-09-18T11:31:45.423 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:45.423 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 66.097584291000004, 2024-09-18T11:31:45.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0088255, 2024-09-18T11:31:45.423 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.423 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.423 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:45.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:45.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:45.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:45.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273155+0000", 2024-09-18T11:31:45.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.212e-06 2024-09-18T11:31:45.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273164+0000", 2024-09-18T11:31:45.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7609999999999996e-06 2024-09-18T11:31:45.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273168+0000", 2024-09-18T11:31:45.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2819999999999998e-06 2024-09-18T11:31:45.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274162+0000", 2024-09-18T11:31:45.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099398600000000009 2024-09-18T11:31:45.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274180+0000", 2024-09-18T11:31:45.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8236e-05 2024-09-18T11:31:45.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281943+0000", 2024-09-18T11:31:45.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007763308 2024-09-18T11:31:45.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281975+0000", 2024-09-18T11:31:45.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1202999999999999e-05 2024-09-18T11:31:45.427 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.427 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.427 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.427 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1954 4.7 4:ef60946b:::smithi11939506-8:head [read 0~2890771,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:45.427 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:45.427 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.030833250999997, 2024-09-18T11:31:45.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091142180000000003, 2024-09-18T11:31:45.427 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.427 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:45.427 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:45.427 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:45.428 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:45.428 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1954 2024-09-18T11:31:45.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.428 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:45.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:45.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339903+0000", 2024-09-18T11:31:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7349999999999998e-06 2024-09-18T11:31:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339909+0000", 2024-09-18T11:31:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-09-18T11:31:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339913+0000", 2024-09-18T11:31:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7670000000000001e-06 2024-09-18T11:31:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339970+0000", 2024-09-18T11:31:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6988999999999999e-05 2024-09-18T11:31:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339975+0000", 2024-09-18T11:31:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7929999999999997e-06 2024-09-18T11:31:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346917+0000", 2024-09-18T11:31:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069426640000000003 2024-09-18T11:31:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.347026+0000", 2024-09-18T11:31:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108241 2024-09-18T11:31:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.349014+0000", 2024-09-18T11:31:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001988723 2024-09-18T11:31:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1956 4.6 4:67c1765a:::smithi11939506-30:head [write 705927~558502] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.018724311, 2024-09-18T11:31:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011813239, 2024-09-18T11:31:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1956 2024-09-18T11:31:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352007+0000", 2024-09-18T11:31:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352229+0000", 2024-09-18T11:31:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222024 2024-09-18T11:31:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352251+0000", 2024-09-18T11:31:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2512000000000001e-05 2024-09-18T11:31:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352260+0000", 2024-09-18T11:31:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7339999999999997e-06 2024-09-18T11:31:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352307+0000", 2024-09-18T11:31:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7061999999999998e-05 2024-09-18T11:31:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352408+0000", 2024-09-18T11:31:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101216 2024-09-18T11:31:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352550+0000", 2024-09-18T11:31:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014136199999999999 2024-09-18T11:31:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357684+0000", 2024-09-18T11:31:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051342499999999999 2024-09-18T11:31:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363736+0000", 2024-09-18T11:31:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006052577 2024-09-18T11:31:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363778+0000", 2024-09-18T11:31:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1989000000000001e-05 2024-09-18T11:31:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363822+0000", 2024-09-18T11:31:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3894999999999999e-05 2024-09-18T11:31:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2278 4.f 4:fc192b81:::smithi11939506-31:head [stat,delete] snapc 26=[26,25,24,1e,1c,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T11:31:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 44.987687098999999, 2024-09-18T11:31:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01018694, 2024-09-18T11:31:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2278 2024-09-18T11:31:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383044+0000", 2024-09-18T11:31:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383054+0000", 2024-09-18T11:31:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5969999999999998e-06 2024-09-18T11:31:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383071+0000", 2024-09-18T11:31:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7353000000000001e-05 2024-09-18T11:31:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383081+0000", 2024-09-18T11:31:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9170000000000001e-06 2024-09-18T11:31:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383150+0000", 2024-09-18T11:31:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8503999999999993e-05 2024-09-18T11:31:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383158+0000", 2024-09-18T11:31:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5089999999999993e-06 2024-09-18T11:31:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385396+0000", 2024-09-18T11:31:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022381850000000002 2024-09-18T11:31:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385494+0000", 2024-09-18T11:31:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7751000000000007e-05 2024-09-18T11:31:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385697+0000", 2024-09-18T11:31:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020247400000000001 2024-09-18T11:31:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.390505+0000", 2024-09-18T11:31:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048084039999999996 2024-09-18T11:31:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393148+0000", 2024-09-18T11:31:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026432399999999998 2024-09-18T11:31:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393194+0000", 2024-09-18T11:31:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5259999999999997e-05 2024-09-18T11:31:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393233+0000", 2024-09-18T11:31:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9659000000000001e-05 2024-09-18T11:31:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2342 4.f 4:f3c51b3f:::smithi11939506-21:head [stat,delete] snapc 26=[26,25,24,1e,1c,19,18] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 43.983931087000002, 2024-09-18T11:31:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011167565000000001, 2024-09-18T11:31:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2342 2024-09-18T11:31:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:31:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:31:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1439999999999997e-06 2024-09-18T11:31:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386820+0000", 2024-09-18T11:31:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2651e-05 2024-09-18T11:31:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386825+0000", 2024-09-18T11:31:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3150000000000002e-06 2024-09-18T11:31:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386868+0000", 2024-09-18T11:31:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3832999999999999e-05 2024-09-18T11:31:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:31:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0409999999999998e-06 2024-09-18T11:31:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390445+0000", 2024-09-18T11:31:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035689459999999999 2024-09-18T11:31:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390539+0000", 2024-09-18T11:31:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3791999999999999e-05 2024-09-18T11:31:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390748+0000", 2024-09-18T11:31:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000208863 2024-09-18T11:31:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397659+0000", 2024-09-18T11:31:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069110710000000004 2024-09-18T11:31:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397873+0000", 2024-09-18T11:31:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021343999999999999 2024-09-18T11:31:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397923+0000", 2024-09-18T11:31:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0300000000000003e-05 2024-09-18T11:31:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397970+0000", 2024-09-18T11:31:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6968999999999997e-05 2024-09-18T11:31:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2429 4.b 4:d8d14c58:::smithi11939506-49:head [write 796698~561170,stat] snapc 2b=[2b,2a,26,25,24,1e,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T11:31:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.355561913000003, 2024-09-18T11:31:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097188989999999996, 2024-09-18T11:31:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2429 2024-09-18T11:31:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015170+0000", 2024-09-18T11:31:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015435+0000", 2024-09-18T11:31:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026533299999999998 2024-09-18T11:31:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015455+0000", 2024-09-18T11:31:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9623000000000002e-05 2024-09-18T11:31:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015461+0000", 2024-09-18T11:31:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1859999999999997e-06 2024-09-18T11:31:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015535+0000", 2024-09-18T11:31:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4555999999999996e-05 2024-09-18T11:31:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015543+0000", 2024-09-18T11:31:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9549999999999992e-06 2024-09-18T11:31:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017106+0000", 2024-09-18T11:31:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015630290000000001 2024-09-18T11:31:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017183+0000", 2024-09-18T11:31:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6953999999999995e-05 2024-09-18T11:31:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017301+0000", 2024-09-18T11:31:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011734 2024-09-18T11:31:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.020225+0000", 2024-09-18T11:31:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029244090000000002 2024-09-18T11:31:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024801+0000", 2024-09-18T11:31:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045764129999999997 2024-09-18T11:31:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024840+0000", 2024-09-18T11:31:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8325999999999999e-05 2024-09-18T11:31:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024890+0000", 2024-09-18T11:31:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0596999999999997e-05 2024-09-18T11:31:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2769 4.4 4:2cb3e6a5:::smithi11939506-3:head [write 484101~692450] snapc 30=[30,2f,2b,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:31:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:18.743766+0000", 2024-09-18T11:31:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.62696772, 2024-09-18T11:31:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086202359999999999, 2024-09-18T11:31:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2769 2024-09-18T11:31:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.743766+0000", 2024-09-18T11:31:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.743766+0000", 2024-09-18T11:31:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.743764+0000", 2024-09-18T11:31:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744376+0000", 2024-09-18T11:31:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00061202299999999995 2024-09-18T11:31:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744393+0000", 2024-09-18T11:31:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6679000000000001e-05 2024-09-18T11:31:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744399+0000", 2024-09-18T11:31:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4840000000000001e-06 2024-09-18T11:31:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744452+0000", 2024-09-18T11:31:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2321999999999998e-05 2024-09-18T11:31:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744552+0000", 2024-09-18T11:31:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100536 2024-09-18T11:31:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744738+0000", 2024-09-18T11:31:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000185434 2024-09-18T11:31:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.749297+0000", 2024-09-18T11:31:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004559802 2024-09-18T11:31:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.752260+0000", 2024-09-18T11:31:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029623000000000002 2024-09-18T11:31:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.752338+0000", 2024-09-18T11:31:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8330000000000004e-05 2024-09-18T11:31:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.752386+0000", 2024-09-18T11:31:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:31:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2917 4.4 4:2c93196f:::smithi11939506-28:head [sparse-read 0~2075838] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.008117+0000", 2024-09-18T11:31:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.362616185, 2024-09-18T11:31:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084318729999999995, 2024-09-18T11:31:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2917 2024-09-18T11:31:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.008117+0000", 2024-09-18T11:31:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.008117+0000", 2024-09-18T11:31:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.008117+0000", 2024-09-18T11:31:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.008120+0000", 2024-09-18T11:31:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9100000000000001e-06 2024-09-18T11:31:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.008126+0000", 2024-09-18T11:31:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9699999999999996e-06 2024-09-18T11:31:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.008129+0000", 2024-09-18T11:31:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5190000000000001e-06 2024-09-18T11:31:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.015988+0000", 2024-09-18T11:31:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078590699999999993 2024-09-18T11:31:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.016095+0000", 2024-09-18T11:31:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107357 2024-09-18T11:31:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.016549+0000", 2024-09-18T11:31:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00045354999999999999 2024-09-18T11:31:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2937 4.7 4:e301aefd:::smithi11939506-19:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:31:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.335892949000002, 2024-09-18T11:31:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019287667000000001, 2024-09-18T11:31:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2937 2024-09-18T11:31:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:31:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:31:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:31:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:31:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034842+0000", 2024-09-18T11:31:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.372e-06 2024-09-18T11:31:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034846+0000", 2024-09-18T11:31:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3919999999999996e-06 2024-09-18T11:31:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034848+0000", 2024-09-18T11:31:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3539999999999998e-06 2024-09-18T11:31:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034893+0000", 2024-09-18T11:31:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4861000000000003e-05 2024-09-18T11:31:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034965+0000", 2024-09-18T11:31:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1860999999999995e-05 2024-09-18T11:31:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.054128+0000", 2024-09-18T11:31:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019163039 2024-09-18T11:31:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3020 4.2 4:425afe9c:::smithi11939506-29:35 [set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e99)", 2024-09-18T11:31:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:27.073196+0000", 2024-09-18T11:31:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.297537383000002, 2024-09-18T11:31:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085408519999999998, 2024-09-18T11:31:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3020 2024-09-18T11:31:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073196+0000", 2024-09-18T11:31:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073196+0000", 2024-09-18T11:31:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073194+0000", 2024-09-18T11:31:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073207+0000", 2024-09-18T11:31:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3244000000000001e-05 2024-09-18T11:31:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073230+0000", 2024-09-18T11:31:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2878999999999999e-05 2024-09-18T11:31:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073239+0000", 2024-09-18T11:31:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5589999999999994e-06 2024-09-18T11:31:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073305+0000", 2024-09-18T11:31:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5438000000000006e-05 2024-09-18T11:31:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.073414+0000", 2024-09-18T11:31:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000109473 2024-09-18T11:31:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.079319+0000", 2024-09-18T11:31:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059050320000000002 2024-09-18T11:31:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.080078+0000", 2024-09-18T11:31:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00075877799999999999 2024-09-18T11:31:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.081605+0000", 2024-09-18T11:31:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015268549999999999 2024-09-18T11:31:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.081681+0000", 2024-09-18T11:31:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6396999999999994e-05 2024-09-18T11:31:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:27.081737+0000", 2024-09-18T11:31:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5658999999999997e-05 2024-09-18T11:31:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.478 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:45.480 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:31:45.481 INFO:tasks.rados.rados.0.smithi119.stdout:1544: finishing write tid 3 to smithi11939506-36 2024-09-18T11:31:45.481 INFO:tasks.rados.rados.0.smithi119.stdout:1544: finishing write tid 4 to smithi11939506-36 2024-09-18T11:31:45.481 INFO:tasks.rados.rados.0.smithi119.stdout:1544: finishing write tid 5 to smithi11939506-36 2024-09-18T11:31:45.481 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 36 v 176 (ObjNum 291 snap 43 seq_num 291) dirty exists 2024-09-18T11:31:45.481 INFO:tasks.rados.rados.0.smithi119.stdout:1544: left oid 36 (ObjNum 291 snap 43 seq_num 291) 2024-09-18T11:31:45.481 INFO:tasks.rados.rados.0.smithi119.stdout:1547: finishing write tid 1 to smithi11939506-49 2024-09-18T11:31:45.481 INFO:tasks.rados.rados.0.smithi119.stdout:1542: done (7 left) 2024-09-18T11:31:45.481 INFO:tasks.rados.rados.0.smithi119.stdout:1543: done (6 left) 2024-09-18T11:31:45.481 INFO:tasks.rados.rados.0.smithi119.stdout:1544: done (5 left) 2024-09-18T11:31:45.481 INFO:tasks.rados.rados.0.smithi119.stdout:1546: done (4 left) 2024-09-18T11:31:45.481 INFO:tasks.rados.rados.0.smithi119.stdout:1549: done (3 left) 2024-09-18T11:31:45.481 INFO:tasks.rados.rados.0.smithi119.stdout:1550: write (excl) oid 17 current snap is 43 2024-09-18T11:31:45.482 INFO:tasks.rados.rados.0.smithi119.stdout:1548: expect (ObjNum 243 snap 28 seq_num 243) 2024-09-18T11:31:45.482 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:45.479+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:45.567 INFO:tasks.rados.rados.0.smithi119.stdout:1547: finishing write tid 2 to smithi11939506-49 2024-09-18T11:31:45.567 INFO:tasks.rados.rados.0.smithi119.stdout:1547: finishing write tid 3 to smithi11939506-49 2024-09-18T11:31:45.567 INFO:tasks.rados.rados.0.smithi119.stdout:1547: finishing write tid 4 to smithi11939506-49 2024-09-18T11:31:45.567 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 342 (ObjNum 292 snap 43 seq_num 292) dirty exists 2024-09-18T11:31:45.567 INFO:tasks.rados.rados.0.smithi119.stdout:1547: left oid 49 (ObjNum 292 snap 43 seq_num 292) 2024-09-18T11:31:45.567 INFO:tasks.rados.rados.0.smithi119.stdout:1545: expect (ObjNum 231 snap 27 seq_num 231) 2024-09-18T11:31:45.580 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:45.579+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:45.619 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:45.620 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:45.620 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:45.620 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:45.628 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:31:45.681 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:45.680+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:45.726 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:45.726 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:45.726 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:45.727 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:45.727 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:45.735 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:31:45.747 INFO:tasks.rados.rados.0.smithi119.stdout:1550: seq_num 294 ranges {541258=544620,1637706=626754,2840948=100310} 2024-09-18T11:31:45.753 INFO:tasks.rados.rados.0.smithi119.stdout:1550: writing smithi11939506-17 from 541258 to 1085878 tid 1 2024-09-18T11:31:45.760 INFO:tasks.rados.rados.0.smithi119.stdout:1550: writing smithi11939506-17 from 1637706 to 2264460 tid 2 2024-09-18T11:31:45.764 INFO:tasks.rados.rados.0.smithi119.stdout:1550: writing smithi11939506-17 from 2840948 to 2941258 tid 3 2024-09-18T11:31:45.764 INFO:tasks.rados.rados.0.smithi119.stdout:1545: done (3 left) 2024-09-18T11:31:45.764 INFO:tasks.rados.rados.0.smithi119.stdout:1547: done (2 left) 2024-09-18T11:31:45.764 INFO:tasks.rados.rados.0.smithi119.stdout:1548: done (1 left) 2024-09-18T11:31:45.764 INFO:tasks.rados.rados.0.smithi119.stdout:1551: delete oid 23 current snap is 43 2024-09-18T11:31:45.764 INFO:tasks.rados.rados.0.smithi119.stdout:1550: finishing write tid 1 to smithi11939506-17 2024-09-18T11:31:45.764 INFO:tasks.rados.rados.0.smithi119.stdout:1550: finishing write tid 2 to smithi11939506-17 2024-09-18T11:31:45.765 INFO:tasks.rados.rados.0.smithi119.stdout:1551: done (1 left) 2024-09-18T11:31:45.765 INFO:tasks.rados.rados.0.smithi119.stdout:1552: read oid 20 snap -1 2024-09-18T11:31:45.765 INFO:tasks.rados.rados.0.smithi119.stdout:1552: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:31:45.765 INFO:tasks.rados.rados.0.smithi119.stdout:1553: set_chunk oid 22 target oid 2024-09-18T11:31:45.773 INFO:tasks.rados.rados.0.smithi119.stdout:1553: set_chunk oid 22 offset: 1590272 length: 329728 target oid aa82eb792a8c2556e808357772a775ffcd4a83746ee76793c4f133f018db500a offset: 0 snap -1 2024-09-18T11:31:45.773 INFO:tasks.rados.rados.0.smithi119.stdout:1554: tier_evict oid 11 2024-09-18T11:31:45.773 INFO:tasks.rados.rados.0.smithi119.stdout:1550: finishing write tid 3 to smithi11939506-17 2024-09-18T11:31:45.773 INFO:tasks.rados.rados.0.smithi119.stdout:1554: tier_evict oid 11 snap -1 2024-09-18T11:31:45.773 INFO:tasks.rados.rados.0.smithi119.stdout:1550: finishing write tid 4 to smithi11939506-17 2024-09-18T11:31:45.774 INFO:tasks.rados.rados.0.smithi119.stdout:1550: finishing write tid 5 to smithi11939506-17 2024-09-18T11:31:45.774 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 347 (ObjNum 294 snap 43 seq_num 294) dirty exists 2024-09-18T11:31:45.774 INFO:tasks.rados.rados.0.smithi119.stdout:1550: left oid 17 (ObjNum 294 snap 43 seq_num 294) 2024-09-18T11:31:45.774 INFO:tasks.rados.rados.0.smithi119.stdout:1550: done (3 left) 2024-09-18T11:31:45.774 INFO:tasks.rados.rados.0.smithi119.stdout:1555: write oid 33 current snap is 43 2024-09-18T11:31:45.774 INFO:tasks.rados.rados.0.smithi119.stdout:1555: seq_num 296 ranges {574872=695118,2038317=630423} 2024-09-18T11:31:45.779 INFO:tasks.rados.rados.0.smithi119.stdout:1555: writing smithi11939506-33 from 574872 to 1269990 tid 1 2024-09-18T11:31:45.788 INFO:tasks.rados.rados.0.smithi119.stdout:1555: writing smithi11939506-33 from 2038317 to 2668740 tid 2 2024-09-18T11:31:45.788 INFO:tasks.rados.rados.0.smithi119.stdout:1556: read oid 49 snap -1 2024-09-18T11:31:45.788 INFO:tasks.rados.rados.0.smithi119.stdout:1556: expect (ObjNum 292 snap 43 seq_num 292) 2024-09-18T11:31:45.788 INFO:tasks.rados.rados.0.smithi119.stdout:1557: write oid 16 current snap is 43 2024-09-18T11:31:45.788 INFO:tasks.rados.rados.0.smithi119.stdout:1553: finishing set_chunk to oid 22 2024-09-18T11:31:45.788 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 22 set_chunk returned error code -95 offset: 1590272 length: 329728 2024-09-18T11:31:45.788 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 0 (ObjNum 263 snap 31 seq_num 263) dirty exists 2024-09-18T11:31:45.788 INFO:tasks.rados.rados.0.smithi119.stdout:1554: got (22) Invalid argument 2024-09-18T11:31:45.788 INFO:tasks.rados.rados.0.smithi119.stdout:1552: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:31:45.838 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:45.781+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:45.851 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:45.851 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:45.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:45.851 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:45.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.851 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1046 4.4 e35/22 4:2cb3e6a5:::smithi11939506-3:head v 35'24, mlcod=30'21)", 2024-09-18T11:31:45.851 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:31:45.852 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 89.675446360999999, 2024-09-18T11:31:45.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042519100000000002, 2024-09-18T11:31:45.852 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.852 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.852 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:31:45.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173783+0000", 2024-09-18T11:31:45.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.173782+0000", 2024-09-18T11:31:45.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:45.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174207+0000", 2024-09-18T11:31:45.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00042479799999999999 2024-09-18T11:31:45.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174224+0000", 2024-09-18T11:31:45.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7513999999999999e-05 2024-09-18T11:31:45.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174232+0000", 2024-09-18T11:31:45.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0840000000000003e-06 2024-09-18T11:31:45.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174297+0000", 2024-09-18T11:31:45.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5215000000000005e-05 2024-09-18T11:31:45.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.174336+0000", 2024-09-18T11:31:45.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9403000000000002e-05 2024-09-18T11:31:45.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.177990+0000", 2024-09-18T11:31:45.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036533709999999999 2024-09-18T11:31:45.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:16.178035+0000", 2024-09-18T11:31:45.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5495999999999999e-05 2024-09-18T11:31:45.856 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.856 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.856 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.856 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:31:45.856 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:31:45.856 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.865855494000002, 2024-09-18T11:31:45.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:31:45.857 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.857 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.857 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:31:45.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:31:45.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:31:45.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:45.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:31:45.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:31:45.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:31:45.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:31:45.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:31:45.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:31:45.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:31:45.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:31:45.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:31:45.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:31:45.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:31:45.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:31:45.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:31:45.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:31:45.860 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.861 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.861 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.861 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:31:45.861 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:45.861 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 76.631024541000002, 2024-09-18T11:31:45.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:31:45.861 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.861 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.861 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:45.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:45.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:45.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:31:45.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:31:45.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:31:45.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:31:45.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:31:45.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:31:45.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:31:45.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:31:45.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:31:45.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:31:45.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:31:45.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:31:45.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:31:45.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:31:45.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:31:45.865 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.865 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.865 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.865 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:31:45.865 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:31:45.865 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 71.457118256000001, 2024-09-18T11:31:45.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:31:45.865 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.866 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.866 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:31:45.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:31:45.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:31:45.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:45.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:31:45.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:31:45.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:31:45.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:31:45.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:31:45.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:31:45.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:31:45.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:31:45.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:31:45.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:31:45.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:31:45.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:31:45.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:31:45.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:31:45.869 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.869 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.870 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.870 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:304 4.6 e69/67 4:67c1765a:::smithi11939506-30:17 v 69'35, mlcod=58'31)", 2024-09-18T11:31:45.870 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:45.870 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.503277162000003, 2024-09-18T11:31:45.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006377611, 2024-09-18T11:31:45.870 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.870 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.870 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:45.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:45.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:45.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:45.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345961+0000", 2024-09-18T11:31:45.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2050000000000008e-06 2024-09-18T11:31:45.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345969+0000", 2024-09-18T11:31:45.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0339999999999997e-06 2024-09-18T11:31:45.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345976+0000", 2024-09-18T11:31:45.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-18T11:31:45.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346026+0000", 2024-09-18T11:31:45.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9286999999999998e-05 2024-09-18T11:31:45.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346050+0000", 2024-09-18T11:31:45.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4397e-05 2024-09-18T11:31:45.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352293+0000", 2024-09-18T11:31:45.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062427170000000001 2024-09-18T11:31:45.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352330+0000", 2024-09-18T11:31:45.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7348999999999998e-05 2024-09-18T11:31:45.874 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.874 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.874 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.874 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:347 4.d e69/22 4:b44229a1:::smithi11939506-42:19 v 69'99, mlcod=65'93)", 2024-09-18T11:31:45.874 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:31:45.874 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.497543966999999, 2024-09-18T11:31:45.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056998760000000004, 2024-09-18T11:31:45.875 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.875 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.875 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:31:45.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:31:45.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351684+0000", 2024-09-18T11:31:45.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:45.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351693+0000", 2024-09-18T11:31:45.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-18T11:31:45.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351704+0000", 2024-09-18T11:31:45.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1077000000000001e-05 2024-09-18T11:31:45.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351709+0000", 2024-09-18T11:31:45.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5419999999999999e-06 2024-09-18T11:31:45.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.878 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351956+0000", 2024-09-18T11:31:45.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000247252 2024-09-18T11:31:45.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.878 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351975+0000", 2024-09-18T11:31:45.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9201000000000001e-05 2024-09-18T11:31:45.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.878 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357334+0000", 2024-09-18T11:31:45.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053588749999999999 2024-09-18T11:31:45.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357385+0000", 2024-09-18T11:31:45.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1406999999999998e-05 2024-09-18T11:31:45.879 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.879 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.879 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.879 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1956 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'37, mlcod=58'31)", 2024-09-18T11:31:45.879 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:45.879 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.496599054000001, 2024-09-18T11:31:45.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010958842, 2024-09-18T11:31:45.880 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.880 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.880 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:45.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:45.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:45.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:45.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352996+0000", 2024-09-18T11:31:45.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036622 2024-09-18T11:31:45.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.882 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.882 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353011+0000", 2024-09-18T11:31:45.882 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4849e-05 2024-09-18T11:31:45.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.882 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.882 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353017+0000", 2024-09-18T11:31:45.882 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:31:45.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.882 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.882 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353068+0000", 2024-09-18T11:31:45.882 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0695000000000002e-05 2024-09-18T11:31:45.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.883 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.883 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353092+0000", 2024-09-18T11:31:45.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4260000000000002e-05 2024-09-18T11:31:45.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.883 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.883 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363551+0000", 2024-09-18T11:31:45.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010459154999999999 2024-09-18T11:31:45.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.883 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363589+0000", 2024-09-18T11:31:45.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8161000000000003e-05 2024-09-18T11:31:45.884 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.884 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.884 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.884 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1957 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'38, mlcod=65'33)", 2024-09-18T11:31:45.884 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:45.884 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.490183187, 2024-09-18T11:31:45.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050499400000000002, 2024-09-18T11:31:45.884 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.884 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.885 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:45.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:45.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:45.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:45.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359214+0000", 2024-09-18T11:31:45.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000168047 2024-09-18T11:31:45.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359227+0000", 2024-09-18T11:31:45.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3124e-05 2024-09-18T11:31:45.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.887 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.887 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359235+0000", 2024-09-18T11:31:45.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0760000000000003e-06 2024-09-18T11:31:45.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.887 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.887 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359287+0000", 2024-09-18T11:31:45.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.151e-05 2024-09-18T11:31:45.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.887 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.887 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359310+0000", 2024-09-18T11:31:45.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-09-18T11:31:45.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364065+0000", 2024-09-18T11:31:45.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047549439999999997 2024-09-18T11:31:45.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364096+0000", 2024-09-18T11:31:45.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1423000000000001e-05 2024-09-18T11:31:45.889 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.889 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.889 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.889 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2300 4.4 e77/22 4:2c93196f:::smithi11939506-28:head v 77'94, mlcod=77'90)", 2024-09-18T11:31:45.889 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:31:45.889 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 44.861120968999998, 2024-09-18T11:31:45.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050425380000000001, 2024-09-18T11:31:45.889 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.889 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.889 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:31:45.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:31:45.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988108+0000", 2024-09-18T11:31:45.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:45.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988388+0000", 2024-09-18T11:31:45.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028087700000000001 2024-09-18T11:31:45.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988405+0000", 2024-09-18T11:31:45.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6565000000000001e-05 2024-09-18T11:31:45.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988412+0000", 2024-09-18T11:31:45.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9709999999999998e-06 2024-09-18T11:31:45.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988464+0000", 2024-09-18T11:31:45.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1949e-05 2024-09-18T11:31:45.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988490+0000", 2024-09-18T11:31:45.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5780999999999999e-05 2024-09-18T11:31:45.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993112+0000", 2024-09-18T11:31:45.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046228399999999996 2024-09-18T11:31:45.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993151+0000", 2024-09-18T11:31:45.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8587999999999997e-05 2024-09-18T11:31:45.893 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.893 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.893 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.894 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.13:63 3.0 e78/21 3:24ebb740:::smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 78'80, mlcod=73'78)", 2024-09-18T11:31:45.894 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:31:45.894 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 44.444131552999998, 2024-09-18T11:31:45.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052262230000000003, 2024-09-18T11:31:45.894 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.894 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.894 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:31:45.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:31:45.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405097+0000", 2024-09-18T11:31:45.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:45.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405103+0000", 2024-09-18T11:31:45.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8780000000000003e-06 2024-09-18T11:31:45.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405117+0000", 2024-09-18T11:31:45.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3441000000000001e-05 2024-09-18T11:31:45.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405121+0000", 2024-09-18T11:31:45.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7099999999999998e-06 2024-09-18T11:31:45.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405158+0000", 2024-09-18T11:31:45.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6477999999999999e-05 2024-09-18T11:31:45.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405173+0000", 2024-09-18T11:31:45.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4796e-05 2024-09-18T11:31:45.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410285+0000", 2024-09-18T11:31:45.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051118919999999998 2024-09-18T11:31:45.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410324+0000", 2024-09-18T11:31:45.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9616999999999997e-05 2024-09-18T11:31:45.898 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.898 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.898 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.898 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:451 4.6 e78/67 4:6f86ffcc:::smithi11939506-15:26 v 78'58, mlcod=77'53)", 2024-09-18T11:31:45.898 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.414259+0000", 2024-09-18T11:31:45.899 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 44.434970970000002, 2024-09-18T11:31:45.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041821610000000002, 2024-09-18T11:31:45.899 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.899 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.899 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414259+0000", 2024-09-18T11:31:45.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414259+0000", 2024-09-18T11:31:45.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414258+0000", 2024-09-18T11:31:45.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:45.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414264+0000", 2024-09-18T11:31:45.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1850000000000001e-06 2024-09-18T11:31:45.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414273+0000", 2024-09-18T11:31:45.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7290000000000006e-06 2024-09-18T11:31:45.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414278+0000", 2024-09-18T11:31:45.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8470000000000003e-06 2024-09-18T11:31:45.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414315+0000", 2024-09-18T11:31:45.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7462000000000001e-05 2024-09-18T11:31:45.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.414334+0000", 2024-09-18T11:31:45.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9080999999999999e-05 2024-09-18T11:31:45.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.418419+0000", 2024-09-18T11:31:45.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040846540000000001 2024-09-18T11:31:45.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.418441+0000", 2024-09-18T11:31:45.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1844999999999998e-05 2024-09-18T11:31:45.903 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.903 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.903 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.903 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.13:18 3.0 e91/21 3:35e3c617:::2e64dd273da7619bd64b294f52905c1b9405a1ced2a8fd3c0b5bc481bc9732a6:head v 91'89, mlcod=85'87)", 2024-09-18T11:31:45.903 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:31:45.903 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.827356141999999, 2024-09-18T11:31:45.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048558539999999997, 2024-09-18T11:31:45.903 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.903 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.903 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:31:45.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:31:45.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:31:45.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:45.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021880+0000", 2024-09-18T11:31:45.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5860000000000001e-06 2024-09-18T11:31:45.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021890+0000", 2024-09-18T11:31:45.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8290000000000003e-06 2024-09-18T11:31:45.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021895+0000", 2024-09-18T11:31:45.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2480000000000004e-06 2024-09-18T11:31:45.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022166+0000", 2024-09-18T11:31:45.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027073499999999998 2024-09-18T11:31:45.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022184+0000", 2024-09-18T11:31:45.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8082000000000001e-05 2024-09-18T11:31:45.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026693+0000", 2024-09-18T11:31:45.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045093479999999998 2024-09-18T11:31:45.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026729+0000", 2024-09-18T11:31:45.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6282999999999998e-05 2024-09-18T11:31:45.907 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.907 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.907 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.908 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:573 4.4 e91/22 4:2c93196f:::smithi11939506-28:26 v 91'135, mlcod=90'131)", 2024-09-18T11:31:45.908 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:31:45.908 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.824889552999998, 2024-09-18T11:31:45.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050722739999999999, 2024-09-18T11:31:45.908 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.908 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.908 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:31:45.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:31:45.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024339+0000", 2024-09-18T11:31:45.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:45.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024349+0000", 2024-09-18T11:31:45.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6339999999999996e-06 2024-09-18T11:31:45.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024362+0000", 2024-09-18T11:31:45.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-18T11:31:45.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024368+0000", 2024-09-18T11:31:45.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8649999999999998e-06 2024-09-18T11:31:45.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024431+0000", 2024-09-18T11:31:45.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2780999999999994e-05 2024-09-18T11:31:45.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024448+0000", 2024-09-18T11:31:45.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7388999999999999e-05 2024-09-18T11:31:45.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029385+0000", 2024-09-18T11:31:45.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004936523 2024-09-18T11:31:45.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029412+0000", 2024-09-18T11:31:45.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7702999999999999e-05 2024-09-18T11:31:45.912 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.912 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.912 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.912 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:574 4.4 e91/22 4:2cb3e6a5:::smithi11939506-3:1e v 91'137, mlcod=90'131)", 2024-09-18T11:31:45.912 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:31:45.912 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.824451509999999, 2024-09-18T11:31:45.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070802850000000004, 2024-09-18T11:31:45.913 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.913 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.913 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:31:45.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:31:45.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024777+0000", 2024-09-18T11:31:45.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:45.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024785+0000", 2024-09-18T11:31:45.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8099999999999998e-06 2024-09-18T11:31:45.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024794+0000", 2024-09-18T11:31:45.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5839999999999995e-06 2024-09-18T11:31:45.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024798+0000", 2024-09-18T11:31:45.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.031e-06 2024-09-18T11:31:45.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024830+0000", 2024-09-18T11:31:45.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2268999999999999e-05 2024-09-18T11:31:45.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024847+0000", 2024-09-18T11:31:45.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7306999999999999e-05 2024-09-18T11:31:45.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031834+0000", 2024-09-18T11:31:45.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069868380000000004 2024-09-18T11:31:45.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031858+0000", 2024-09-18T11:31:45.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4082000000000001e-05 2024-09-18T11:31:45.917 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.917 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.917 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.917 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:541 4.d e91/22 4:b9bb642b:::smithi11939506-27:19 v 91'158, mlcod=91'154)", 2024-09-18T11:31:45.917 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:31:45.917 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.817070818000001, 2024-09-18T11:31:45.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052185, 2024-09-18T11:31:45.917 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.917 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.917 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:31:45.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:31:45.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032158+0000", 2024-09-18T11:31:45.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:45.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032165+0000", 2024-09-18T11:31:45.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-09-18T11:31:45.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032176+0000", 2024-09-18T11:31:45.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1460999999999999e-05 2024-09-18T11:31:45.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032182+0000", 2024-09-18T11:31:45.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0870000000000003e-06 2024-09-18T11:31:45.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032224+0000", 2024-09-18T11:31:45.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1372999999999998e-05 2024-09-18T11:31:45.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032246+0000", 2024-09-18T11:31:45.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2158e-05 2024-09-18T11:31:45.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037348+0000", 2024-09-18T11:31:45.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051025319999999999 2024-09-18T11:31:45.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037377+0000", 2024-09-18T11:31:45.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8937000000000001e-05 2024-09-18T11:31:45.921 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.922 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.922 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.922 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:582 4.6 e91/67 4:6f86ffcc:::smithi11939506-15:26 v 91'78, mlcod=90'72)", 2024-09-18T11:31:45.922 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.043567+0000", 2024-09-18T11:31:45.922 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.805662867999999, 2024-09-18T11:31:45.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004448446, 2024-09-18T11:31:45.922 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.922 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.922 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043567+0000", 2024-09-18T11:31:45.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043567+0000", 2024-09-18T11:31:45.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043566+0000", 2024-09-18T11:31:45.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:45.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043571+0000", 2024-09-18T11:31:45.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.181e-06 2024-09-18T11:31:45.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043579+0000", 2024-09-18T11:31:45.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9559999999999997e-06 2024-09-18T11:31:45.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043583+0000", 2024-09-18T11:31:45.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0770000000000001e-06 2024-09-18T11:31:45.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043610+0000", 2024-09-18T11:31:45.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6778e-05 2024-09-18T11:31:45.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043633+0000", 2024-09-18T11:31:45.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2904e-05 2024-09-18T11:31:45.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.047991+0000", 2024-09-18T11:31:45.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043582910000000002 2024-09-18T11:31:45.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.048015+0000", 2024-09-18T11:31:45.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3813000000000001e-05 2024-09-18T11:31:45.926 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.926 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.926 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.926 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2754 4.4 e92/22 4:2cb3e6a5:::smithi11939506-3:head v 92'147, mlcod=92'143)", 2024-09-18T11:31:45.926 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:17.887302+0000", 2024-09-18T11:31:45.927 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.96192744, 2024-09-18T11:31:45.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044319160000000002, 2024-09-18T11:31:45.927 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.927 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.927 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887302+0000", 2024-09-18T11:31:45.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887302+0000", 2024-09-18T11:31:45.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887301+0000", 2024-09-18T11:31:45.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:45.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887313+0000", 2024-09-18T11:31:45.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2468e-05 2024-09-18T11:31:45.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887331+0000", 2024-09-18T11:31:45.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7362000000000001e-05 2024-09-18T11:31:45.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887338+0000", 2024-09-18T11:31:45.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7980000000000006e-06 2024-09-18T11:31:45.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887416+0000", 2024-09-18T11:31:45.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7577999999999995e-05 2024-09-18T11:31:45.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887441+0000", 2024-09-18T11:31:45.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.531e-05 2024-09-18T11:31:45.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.891671+0000", 2024-09-18T11:31:45.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00422989 2024-09-18T11:31:45.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.891734+0000", 2024-09-18T11:31:45.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2885999999999994e-05 2024-09-18T11:31:45.931 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.931 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.931 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.931 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2769 4.4 e92/22 4:2cb3e6a5:::smithi11939506-3:head v 92'150, mlcod=92'148)", 2024-09-18T11:31:45.931 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:18.744829+0000", 2024-09-18T11:31:45.932 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.10440011, 2024-09-18T11:31:45.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043433719999999999, 2024-09-18T11:31:45.932 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.932 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.932 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744829+0000", 2024-09-18T11:31:45.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744829+0000", 2024-09-18T11:31:45.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744828+0000", 2024-09-18T11:31:45.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:45.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.745480+0000", 2024-09-18T11:31:45.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00065170400000000002 2024-09-18T11:31:45.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.745499+0000", 2024-09-18T11:31:45.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9218000000000001e-05 2024-09-18T11:31:45.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.745507+0000", 2024-09-18T11:31:45.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1950000000000003e-06 2024-09-18T11:31:45.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.745567+0000", 2024-09-18T11:31:45.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0040000000000001e-05 2024-09-18T11:31:45.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.745594+0000", 2024-09-18T11:31:45.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6803000000000001e-05 2024-09-18T11:31:45.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.749126+0000", 2024-09-18T11:31:45.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003532622 2024-09-18T11:31:45.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.749173+0000", 2024-09-18T11:31:45.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6334000000000001e-05 2024-09-18T11:31:45.936 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.936 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.936 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.936 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3250 4.6 e109/67 4:6f86ffcc:::smithi11939506-15:head v 109'108, mlcod=109'106)", 2024-09-18T11:31:45.936 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:37.733741+0000", 2024-09-18T11:31:45.936 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1154880980000002, 2024-09-18T11:31:45.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041911810000000004, 2024-09-18T11:31:45.936 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.936 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:45.937 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.733741+0000", 2024-09-18T11:31:45.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.733741+0000", 2024-09-18T11:31:45.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.733741+0000", 2024-09-18T11:31:45.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:45.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.733911+0000", 2024-09-18T11:31:45.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017019999999999999 2024-09-18T11:31:45.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.733925+0000", 2024-09-18T11:31:45.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4052e-05 2024-09-18T11:31:45.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.733931+0000", 2024-09-18T11:31:45.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2099999999999998e-06 2024-09-18T11:31:45.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.733995+0000", 2024-09-18T11:31:45.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4502e-05 2024-09-18T11:31:45.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:45.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.734012+0000", 2024-09-18T11:31:45.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6940999999999999e-05 2024-09-18T11:31:45.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:45.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.737892+0000", 2024-09-18T11:31:45.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038799849999999999 2024-09-18T11:31:45.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.737933+0000", 2024-09-18T11:31:45.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0197000000000002e-05 2024-09-18T11:31:45.940 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.940 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.940 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.941 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4178.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-18T11:31:45.941 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:31:45.941 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.9084087529999998, 2024-09-18T11:31:45.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048960139999999997, 2024-09-18T11:31:45.941 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:45.941 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:31:45.941 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:45.941 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4178", 2024-09-18T11:31:45.941 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/490761693", 2024-09-18T11:31:45.941 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4 2024-09-18T11:31:45.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.942 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:45.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:45.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:31:45.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:45.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:31:45.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:45.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:45.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940819+0000", 2024-09-18T11:31:45.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:45.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:45.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940832+0000", 2024-09-18T11:31:45.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3155e-05 2024-09-18T11:31:45.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:45.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940848+0000", 2024-09-18T11:31:45.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6288999999999999e-05 2024-09-18T11:31:45.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:45.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940856+0000", 2024-09-18T11:31:45.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4919999999999996e-06 2024-09-18T11:31:45.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:45.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945613+0000", 2024-09-18T11:31:45.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047574999999999996 2024-09-18T11:31:45.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:45.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:45.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:45.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945717+0000", 2024-09-18T11:31:45.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103553 2024-09-18T11:31:45.945 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.945 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.945 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.945 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:45.945 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:45.945 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:45.946 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:31:45.947 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:45.945+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:46.010 INFO:tasks.rados.rados.0.smithi119.stdout:1555: finishing write tid 1 to smithi11939506-33 2024-09-18T11:31:46.010 INFO:tasks.rados.rados.0.smithi119.stdout:1555: finishing write tid 2 to smithi11939506-33 2024-09-18T11:31:46.010 INFO:tasks.rados.rados.0.smithi119.stdout:1555: finishing write tid 3 to smithi11939506-33 2024-09-18T11:31:46.010 INFO:tasks.rados.rados.0.smithi119.stdout:1555: finishing write tid 4 to smithi11939506-33 2024-09-18T11:31:46.010 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 33 v 145 (ObjNum 296 snap 43 seq_num 296) dirty exists 2024-09-18T11:31:46.011 INFO:tasks.rados.rados.0.smithi119.stdout:1555: left oid 33 (ObjNum 296 snap 43 seq_num 296) 2024-09-18T11:31:46.011 INFO:tasks.rados.rados.0.smithi119.stdout:1557: seq_num 297 ranges {591679=486720,1813152=490434} 2024-09-18T11:31:46.016 INFO:tasks.rados.rados.0.smithi119.stdout:1557: writing smithi11939506-16 from 591679 to 1078399 tid 1 2024-09-18T11:31:46.023 INFO:tasks.rados.rados.0.smithi119.stdout:1557: writing smithi11939506-16 from 1813152 to 2303586 tid 2 2024-09-18T11:31:46.023 INFO:tasks.rados.rados.0.smithi119.stdout:1552: done (5 left) 2024-09-18T11:31:46.023 INFO:tasks.rados.rados.0.smithi119.stdout:1553: done (4 left) 2024-09-18T11:31:46.023 INFO:tasks.rados.rados.0.smithi119.stdout:1554: done (3 left) 2024-09-18T11:31:46.023 INFO:tasks.rados.rados.0.smithi119.stdout:1555: done (2 left) 2024-09-18T11:31:46.023 INFO:tasks.rados.rados.0.smithi119.stdout:1558: read oid 8 snap -1 2024-09-18T11:31:46.023 INFO:tasks.rados.rados.0.smithi119.stdout:1558: expect deleted 2024-09-18T11:31:46.023 INFO:tasks.rados.rados.0.smithi119.stdout:1559: snap_remove snap 34 2024-09-18T11:31:46.047 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:46.046+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:46.087 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:46.087 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:46.087 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:46.087 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:46.096 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:31:46.130 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:46.129+0000 7fb6950ff640 -1 osd.2 116 handle_osd_map injecting map crc failure 2024-09-18T11:31:46.133 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is set 2024-09-18T11:31:46.133 INFO:tasks.rados.rados.0.smithi119.stdout:1559: done (3 left) 2024-09-18T11:31:46.133 INFO:tasks.rados.rados.0.smithi119.stdout:1560: write (excl) oid 22 current snap is 43 2024-09-18T11:31:46.134 INFO:tasks.rados.rados.0.smithi119.stdout:1560: seq_num 298 ranges {608486=561969,1871634=634092,2950640=57846} 2024-09-18T11:31:46.134 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:46.133+0000 7f697aead640 -1 osd.3 116 handle_osd_map injecting map crc failure 2024-09-18T11:31:46.141 INFO:tasks.rados.rados.0.smithi119.stdout:1560: writing smithi11939506-22 from 608486 to 1170455 tid 1 2024-09-18T11:31:46.147 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:46.146+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:46.149 INFO:tasks.rados.rados.0.smithi119.stdout:1560: writing smithi11939506-22 from 1871634 to 2505726 tid 2 2024-09-18T11:31:46.152 INFO:tasks.rados.rados.0.smithi119.stdout:1560: writing smithi11939506-22 from 2950640 to 3008486 tid 3 2024-09-18T11:31:46.152 INFO:tasks.rados.rados.0.smithi119.stdout:1561: write (excl) oid 33 current snap is 43 2024-09-18T11:31:46.152 INFO:tasks.rados.rados.0.smithi119.stdout:1556: expect (ObjNum 292 snap 43 seq_num 292) 2024-09-18T11:31:46.235 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:46.235 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:46.235 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:46.235 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:46.235 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:46.243 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:31:46.287 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:46.286+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:46.344 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:46.344 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:46.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:46.345 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:46.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.345 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:46.345 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:46.346 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.172702548000004, 2024-09-18T11:31:46.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:31:46.346 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:46.346 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:46.346 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:46.346 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:46.346 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.347 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:31:46.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.347 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:46.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:46.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:46.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:46.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:46.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:46.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:31:46.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:46.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:46.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:31:46.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:31:46.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:46.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:31:46.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:31:46.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:31:46.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:31:46.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:31:46.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:31:46.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:46.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:31:46.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:31:46.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:46.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:31:46.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:31:46.351 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.351 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:46.351 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.352 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1001 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:46.352 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:46.352 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.177551269999995, 2024-09-18T11:31:46.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007461082, 2024-09-18T11:31:46.352 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:46.352 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:46.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:46.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:46.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.352 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1001 2024-09-18T11:31:46.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.352 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:46.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:46.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:46.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:46.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:46.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:46.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164570+0000", 2024-09-18T11:31:46.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:46.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:46.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164574+0000", 2024-09-18T11:31:46.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.822e-06 2024-09-18T11:31:46.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:46.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164582+0000", 2024-09-18T11:31:46.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3410000000000003e-06 2024-09-18T11:31:46.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164587+0000", 2024-09-18T11:31:46.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769999999999999e-06 2024-09-18T11:31:46.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164702+0000", 2024-09-18T11:31:46.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115705 2024-09-18T11:31:46.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:46.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164709+0000", 2024-09-18T11:31:46.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-09-18T11:31:46.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170296+0000", 2024-09-18T11:31:46.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055876390000000001 2024-09-18T11:31:46.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:46.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170418+0000", 2024-09-18T11:31:46.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121549 2024-09-18T11:31:46.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:46.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172032+0000", 2024-09-18T11:31:46.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00161407 2024-09-18T11:31:46.356 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.356 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:46.356 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.357 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:46.357 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:46.357 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.177524554000001, 2024-09-18T11:31:46.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:31:46.357 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:46.357 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:46.357 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:46.357 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:46.357 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.357 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:31:46.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.358 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:46.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:46.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:46.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:46.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:46.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:46.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:46.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:46.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:46.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:31:46.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:31:46.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:46.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:31:46.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:31:46.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:31:46.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:31:46.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:31:46.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:31:46.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:46.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:31:46.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:31:46.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:31:46.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:31:46.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:46.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:31:46.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:31:46.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:46.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:31:46.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:31:46.362 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.362 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:46.362 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.362 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:46.362 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:46.362 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.177506265000005, 2024-09-18T11:31:46.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:31:46.362 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:46.362 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:46.362 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:46.362 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:46.363 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.363 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:31:46.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.363 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:46.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:46.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:46.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:46.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:46.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:46.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:31:46.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:46.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:46.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:31:46.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:31:46.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:46.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:31:46.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:31:46.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:31:46.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:31:46.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:31:46.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:31:46.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:46.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:31:46.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:31:46.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:31:46.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:31:46.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:46.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:31:46.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:31:46.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:46.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:31:46.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:31:46.367 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.367 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:46.367 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.368 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:46.368 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:46.368 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.177445496000004, 2024-09-18T11:31:46.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:31:46.368 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:46.368 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:46.368 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:46.368 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:46.368 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.368 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:31:46.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.369 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:46.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:46.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:46.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:46.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:46.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:46.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:31:46.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:46.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:46.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:31:46.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:31:46.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:46.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:31:46.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:31:46.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:31:46.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:31:46.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:31:46.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:31:46.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:46.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:31:46.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:31:46.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:31:46.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:31:46.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:46.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:31:46.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:31:46.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:46.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:31:46.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:31:46.376 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.376 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:46.376 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.377 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:46.377 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:46.377 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.177423175000001, 2024-09-18T11:31:46.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:31:46.377 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:46.378 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:46.378 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:46.378 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:46.378 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.378 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:31:46.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.378 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:46.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.378 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:46.378 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:46.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:46.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:46.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:46.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:31:46.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:46.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:46.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:31:46.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:31:46.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:46.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:31:46.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:31:46.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:31:46.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:31:46.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:31:46.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:31:46.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:46.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:31:46.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:31:46.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:31:46.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:31:46.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:46.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:31:46.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:31:46.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:46.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:31:46.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:31:46.383 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.383 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:46.383 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.383 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:31:46.383 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:46.383 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 86.396298291999997, 2024-09-18T11:31:46.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:31:46.383 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:46.383 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.383 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:46.383 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:46.384 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.384 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:31:46.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.384 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:46.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:46.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:46.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:46.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:46.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:46.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:31:46.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:31:46.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:46.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:31:46.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:31:46.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:46.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:31:46.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:31:46.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:31:46.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:31:46.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:31:46.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:31:46.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:46.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:31:46.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:31:46.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.387 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:46.387 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:31:46.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:31:46.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:46.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:31:46.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:31:46.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:46.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:31:46.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:31:46.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:46.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:31:46.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:31:46.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:46.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:31:46.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:31:46.390 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.390 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:46.390 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.390 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1661 4.0 4:019b6e09:::smithi11939506-40:17 [read 0~3445402,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:46.390 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:46.390 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.992225589, 2024-09-18T11:31:46.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075190880000000002, 2024-09-18T11:31:46.390 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:46.390 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:46.390 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:46.391 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:46.391 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.391 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1661 2024-09-18T11:31:46.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.391 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:46.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:46.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:46.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:46.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:46.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:46.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:46.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:46.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:46.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349899+0000", 2024-09-18T11:31:46.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9749999999999999e-06 2024-09-18T11:31:46.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:46.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349906+0000", 2024-09-18T11:31:46.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9290000000000001e-06 2024-09-18T11:31:46.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349912+0000", 2024-09-18T11:31:46.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9900000000000002e-06 2024-09-18T11:31:46.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355645+0000", 2024-09-18T11:31:46.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057329549999999996 2024-09-18T11:31:46.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:46.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355776+0000", 2024-09-18T11:31:46.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130975 2024-09-18T11:31:46.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:46.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357416+0000", 2024-09-18T11:31:46.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001639649 2024-09-18T11:31:46.394 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.394 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:46.394 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.395 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1761 4.0 4:0aec5e1c:::smithi11939506-50:head [write 769787~704463,stat] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:46.395 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:46.395 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.039733174999995, 2024-09-18T11:31:46.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074082879999999999, 2024-09-18T11:31:46.395 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:46.395 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.395 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:46.395 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:46.395 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.395 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1761 2024-09-18T11:31:46.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.395 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:46.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:46.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:46.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:46.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:46.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:46.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302386+0000", 2024-09-18T11:31:46.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:46.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:46.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302889+0000", 2024-09-18T11:31:46.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050318499999999996 2024-09-18T11:31:46.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.397 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:46.397 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302930+0000", 2024-09-18T11:31:46.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0198999999999999e-05 2024-09-18T11:31:46.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302965+0000", 2024-09-18T11:31:46.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5292e-05 2024-09-18T11:31:46.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303046+0000", 2024-09-18T11:31:46.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1254999999999994e-05 2024-09-18T11:31:46.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:46.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303054+0000", 2024-09-18T11:31:46.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2800000000000003e-06 2024-09-18T11:31:46.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304658+0000", 2024-09-18T11:31:46.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016039520000000001 2024-09-18T11:31:46.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:46.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304724+0000", 2024-09-18T11:31:46.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5888000000000003e-05 2024-09-18T11:31:46.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:46.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304853+0000", 2024-09-18T11:31:46.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012912700000000001 2024-09-18T11:31:46.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:46.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309084+0000", 2024-09-18T11:31:46.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042311229999999998 2024-09-18T11:31:46.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:46.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309738+0000", 2024-09-18T11:31:46.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00065401399999999996 2024-09-18T11:31:46.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:46.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309770+0000", 2024-09-18T11:31:46.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1652000000000001e-05 2024-09-18T11:31:46.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:46.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309797+0000", 2024-09-18T11:31:46.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7052000000000001e-05 2024-09-18T11:31:46.401 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.402 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:46.402 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.402 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:306 4.f e69/22 4:f5d7df41:::smithi11939506-16:17 v 69'94, mlcod=65'88)", 2024-09-18T11:31:46.402 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:46.402 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.991006458000001, 2024-09-18T11:31:46.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068024720000000004, 2024-09-18T11:31:46.402 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:46.402 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.402 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:46.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:46.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:46.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:46.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:46.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:46.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351115+0000", 2024-09-18T11:31:46.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:46.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:46.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351121+0000", 2024-09-18T11:31:46.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-09-18T11:31:46.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:46.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351126+0000", 2024-09-18T11:31:46.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5969999999999999e-06 2024-09-18T11:31:46.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351147+0000", 2024-09-18T11:31:46.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1160000000000001e-05 2024-09-18T11:31:46.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351518+0000", 2024-09-18T11:31:46.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00037079300000000001 2024-09-18T11:31:46.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:46.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351536+0000", 2024-09-18T11:31:46.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8168999999999998e-05 2024-09-18T11:31:46.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:46.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357870+0000", 2024-09-18T11:31:46.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063332650000000002 2024-09-18T11:31:46.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:46.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357918+0000", 2024-09-18T11:31:46.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8347999999999999e-05 2024-09-18T11:31:46.407 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.407 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:46.407 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.407 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2022 4.0 4:0aec5e1c:::smithi11939506-50:head [write 457190~552096,stat] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-18T11:31:46.408 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:46.408 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.345968321000001, 2024-09-18T11:31:46.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006803974, 2024-09-18T11:31:46.408 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:46.408 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.408 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:46.408 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:46.408 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.408 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2022 2024-09-18T11:31:46.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.408 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:46.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:46.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:46.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:46.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:46.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:46.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996151+0000", 2024-09-18T11:31:46.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:46.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:46.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996461+0000", 2024-09-18T11:31:46.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000309587 2024-09-18T11:31:46.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:46.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996481+0000", 2024-09-18T11:31:46.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9949000000000002e-05 2024-09-18T11:31:46.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996489+0000", 2024-09-18T11:31:46.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9129999999999992e-06 2024-09-18T11:31:46.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996562+0000", 2024-09-18T11:31:46.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2807999999999998e-05 2024-09-18T11:31:46.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:46.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996687+0000", 2024-09-18T11:31:46.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012458200000000001 2024-09-18T11:31:46.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:46.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996909+0000", 2024-09-18T11:31:46.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222124 2024-09-18T11:31:46.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:46.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.000741+0000", 2024-09-18T11:31:46.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038320720000000002 2024-09-18T11:31:46.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:46.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002854+0000", 2024-09-18T11:31:46.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002113228 2024-09-18T11:31:46.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:46.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002911+0000", 2024-09-18T11:31:46.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7192999999999999e-05 2024-09-18T11:31:46.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:46.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002958+0000", 2024-09-18T11:31:46.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6242000000000002e-05 2024-09-18T11:31:46.413 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.413 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:46.413 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.414 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2503 3.1 e85/67 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head v 85'117, mlcod=84'115)", 2024-09-18T11:31:46.414 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:31:46.414 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.364899604999998, 2024-09-18T11:31:46.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086527619999999996, 2024-09-18T11:31:46.414 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:46.414 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.414 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:46.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:46.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:31:46.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:46.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:31:46.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:46.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977220+0000", 2024-09-18T11:31:46.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:46.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:46.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980771+0000", 2024-09-18T11:31:46.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035505200000000002 2024-09-18T11:31:46.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:46.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980797+0000", 2024-09-18T11:31:46.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.618e-05 2024-09-18T11:31:46.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980807+0000", 2024-09-18T11:31:46.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0433e-05 2024-09-18T11:31:46.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980861+0000", 2024-09-18T11:31:46.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3415999999999998e-05 2024-09-18T11:31:46.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:46.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980885+0000", 2024-09-18T11:31:46.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4212000000000001e-05 2024-09-18T11:31:46.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:46.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985826+0000", 2024-09-18T11:31:46.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049412450000000004 2024-09-18T11:31:46.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:46.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985875+0000", 2024-09-18T11:31:46.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9116000000000002e-05 2024-09-18T11:31:46.419 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.419 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:46.419 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.420 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2648 4.0 4:059d8d95:::smithi11939506-32:head [read 0~3647119,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:46.420 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:31:46.420 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 32.010811234999998, 2024-09-18T11:31:46.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0076286949999999996, 2024-09-18T11:31:46.420 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:46.420 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:46.420 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:46.420 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:46.420 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.420 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2648 2024-09-18T11:31:46.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.420 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:46.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:46.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:31:46.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:46.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:31:46.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:46.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331309+0000", 2024-09-18T11:31:46.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:46.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:46.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331321+0000", 2024-09-18T11:31:46.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:31:46.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:46.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331340+0000", 2024-09-18T11:31:46.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8227e-05 2024-09-18T11:31:46.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331348+0000", 2024-09-18T11:31:46.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4640000000000006e-06 2024-09-18T11:31:46.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331406+0000", 2024-09-18T11:31:46.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7445000000000003e-05 2024-09-18T11:31:46.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:46.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331521+0000", 2024-09-18T11:31:46.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115945 2024-09-18T11:31:46.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:46.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338939+0000", 2024-09-18T11:31:46.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074179859999999997 2024-09-18T11:31:46.425 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.425 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:46.425 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.425 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2649 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:46.425 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:31:46.425 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 32.010756235999999, 2024-09-18T11:31:46.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566240000000007, 2024-09-18T11:31:46.425 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:46.425 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:46.425 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:46.425 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:46.425 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.426 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2649 2024-09-18T11:31:46.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.426 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:46.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:46.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:31:46.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:46.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:31:46.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:46.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331365+0000", 2024-09-18T11:31:46.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:46.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:46.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331369+0000", 2024-09-18T11:31:46.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.551e-06 2024-09-18T11:31:46.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:46.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331377+0000", 2024-09-18T11:31:46.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8769999999999993e-06 2024-09-18T11:31:46.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331381+0000", 2024-09-18T11:31:46.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.16e-06 2024-09-18T11:31:46.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338973+0000", 2024-09-18T11:31:46.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075918460000000002 2024-09-18T11:31:46.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:46.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339048+0000", 2024-09-18T11:31:46.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5813000000000001e-05 2024-09-18T11:31:46.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:46.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339322+0000", 2024-09-18T11:31:46.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027399899999999998 2024-09-18T11:31:46.429 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.429 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:46.429 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.429 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2650 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:46.429 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:46.429 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 32.010736018000003, 2024-09-18T11:31:46.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084392180000000001, 2024-09-18T11:31:46.430 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:46.430 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:46.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:46.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:46.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.430 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2650 2024-09-18T11:31:46.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.430 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:46.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:46.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:46.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:46.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:46.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:46.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:46.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:46.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:46.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331388+0000", 2024-09-18T11:31:46.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3429999999999998e-06 2024-09-18T11:31:46.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:46.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331394+0000", 2024-09-18T11:31:46.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.19e-06 2024-09-18T11:31:46.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331398+0000", 2024-09-18T11:31:46.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.495e-06 2024-09-18T11:31:46.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339362+0000", 2024-09-18T11:31:46.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079641799999999995 2024-09-18T11:31:46.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:46.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339460+0000", 2024-09-18T11:31:46.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8329e-05 2024-09-18T11:31:46.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:46.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339825+0000", 2024-09-18T11:31:46.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036510500000000001 2024-09-18T11:31:46.434 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.434 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:46.434 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.434 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:119 3.2 3:5595a68d:::07854d2fef297a06ba81685e660c332de36d5d18d546927d30daad6d7fda1541:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:46.434 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:31:46.434 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.298584260999998, 2024-09-18T11:31:46.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074145979999999997, 2024-09-18T11:31:46.435 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:46.435 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.435 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:46.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:46.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:31:46.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:46.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:31:46.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:46.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043535+0000", 2024-09-18T11:31:46.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:46.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:46.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043970+0000", 2024-09-18T11:31:46.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000435148 2024-09-18T11:31:46.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:46.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043993+0000", 2024-09-18T11:31:46.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3113999999999999e-05 2024-09-18T11:31:46.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044003+0000", 2024-09-18T11:31:46.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5310000000000007e-06 2024-09-18T11:31:46.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044062+0000", 2024-09-18T11:31:46.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9487000000000002e-05 2024-09-18T11:31:46.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:46.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044071+0000", 2024-09-18T11:31:46.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1610000000000003e-06 2024-09-18T11:31:46.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046423+0000", 2024-09-18T11:31:46.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023523680000000001 2024-09-18T11:31:46.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:46.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046502+0000", 2024-09-18T11:31:46.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9339999999999996e-05 2024-09-18T11:31:46.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:46.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046664+0000", 2024-09-18T11:31:46.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016185700000000001 2024-09-18T11:31:46.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:46.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050870+0000", 2024-09-18T11:31:46.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042060070000000003 2024-09-18T11:31:46.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:46.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050878+0000", 2024-09-18T11:31:46.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8960000000000003e-06 2024-09-18T11:31:46.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:46.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050918+0000", 2024-09-18T11:31:46.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0367000000000002e-05 2024-09-18T11:31:46.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:46.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050952+0000", 2024-09-18T11:31:46.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3989999999999998e-05 2024-09-18T11:31:46.441 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.441 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:46.441 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.442 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:124 3.2 3:4326e841:::a6567eb8c0be0365cdfccf2ece66d60995301bc908261a0c1daff039d2080a5f:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:46.442 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044017+0000", 2024-09-18T11:31:46.442 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.298104650999999, 2024-09-18T11:31:46.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070611140000000003, 2024-09-18T11:31:46.442 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:46.442 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.442 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:46.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:46.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044017+0000", 2024-09-18T11:31:46.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:46.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044017+0000", 2024-09-18T11:31:46.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:46.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044016+0000", 2024-09-18T11:31:46.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:46.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:46.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044102+0000", 2024-09-18T11:31:46.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5756999999999999e-05 2024-09-18T11:31:46.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:46.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044114+0000", 2024-09-18T11:31:46.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1505000000000001e-05 2024-09-18T11:31:46.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044120+0000", 2024-09-18T11:31:46.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8549999999999998e-06 2024-09-18T11:31:46.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044154+0000", 2024-09-18T11:31:46.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4773000000000001e-05 2024-09-18T11:31:46.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:46.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044159+0000", 2024-09-18T11:31:46.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5310000000000003e-06 2024-09-18T11:31:46.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046975+0000", 2024-09-18T11:31:46.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028162700000000001 2024-09-18T11:31:46.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:46.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.047116+0000", 2024-09-18T11:31:46.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014086400000000001 2024-09-18T11:31:46.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:46.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.047336+0000", 2024-09-18T11:31:46.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021975299999999999 2024-09-18T11:31:46.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:46.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050964+0000", 2024-09-18T11:31:46.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036283000000000001 2024-09-18T11:31:46.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:46.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.051005+0000", 2024-09-18T11:31:46.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0763999999999998e-05 2024-09-18T11:31:46.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:46.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.051050+0000", 2024-09-18T11:31:46.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4993000000000001e-05 2024-09-18T11:31:46.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:46.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.051078+0000", 2024-09-18T11:31:46.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8694999999999999e-05 2024-09-18T11:31:46.449 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.450 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:46.450 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.450 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:127 3.2 3:4d3b277a:::3266c7aaa53a7d2c10c3c54d852d6cbbed49fb041bc0df41a3ac83b56e2b0615:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:46.450 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:31:46.450 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.297721078999999, 2024-09-18T11:31:46.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093148009999999993, 2024-09-18T11:31:46.450 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:46.450 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.450 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:46.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:46.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:31:46.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:46.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:31:46.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:46.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044400+0000", 2024-09-18T11:31:46.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:46.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:46.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044442+0000", 2024-09-18T11:31:46.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2247999999999999e-05 2024-09-18T11:31:46.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:46.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044453+0000", 2024-09-18T11:31:46.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0617e-05 2024-09-18T11:31:46.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044457+0000", 2024-09-18T11:31:46.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2520000000000001e-06 2024-09-18T11:31:46.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044495+0000", 2024-09-18T11:31:46.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7710000000000003e-05 2024-09-18T11:31:46.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:46.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044499+0000", 2024-09-18T11:31:46.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6269999999999999e-06 2024-09-18T11:31:46.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048239+0000", 2024-09-18T11:31:46.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037396719999999999 2024-09-18T11:31:46.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:46.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048351+0000", 2024-09-18T11:31:46.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112342 2024-09-18T11:31:46.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:46.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048545+0000", 2024-09-18T11:31:46.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000193881 2024-09-18T11:31:46.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:46.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.052689+0000", 2024-09-18T11:31:46.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041435780000000002 2024-09-18T11:31:46.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:46.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053654+0000", 2024-09-18T11:31:46.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096558199999999999 2024-09-18T11:31:46.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:46.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053683+0000", 2024-09-18T11:31:46.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8850999999999999e-05 2024-09-18T11:31:46.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:46.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053716+0000", 2024-09-18T11:31:46.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2468000000000002e-05 2024-09-18T11:31:46.457 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.457 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:46.457 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.458 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2963 4.0 4:019b6e09:::smithi11939506-40:head [sparse-read 0~1093084] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:46.458 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:31:46.458 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.314712280999998, 2024-09-18T11:31:46.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072292880000000004, 2024-09-18T11:31:46.459 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:46.459 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:46.459 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:46.459 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:46.459 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.459 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2963 2024-09-18T11:31:46.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.459 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:46.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:46.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:31:46.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:46.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:31:46.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:46.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027409+0000", 2024-09-18T11:31:46.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:46.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:46.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027412+0000", 2024-09-18T11:31:46.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.751e-06 2024-09-18T11:31:46.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:46.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027418+0000", 2024-09-18T11:31:46.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.404e-06 2024-09-18T11:31:46.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027423+0000", 2024-09-18T11:31:46.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7119999999999999e-06 2024-09-18T11:31:46.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034156+0000", 2024-09-18T11:31:46.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067331639999999998 2024-09-18T11:31:46.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:46.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034255+0000", 2024-09-18T11:31:46.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8701000000000002e-05 2024-09-18T11:31:46.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:46.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034639+0000", 2024-09-18T11:31:46.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000384006 2024-09-18T11:31:46.464 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.464 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:46.464 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.464 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3209 4.0 4:019b6e09:::smithi11939506-40:head [write 638732~557432] snapc 42=[42,41,3f,3e,3d,3b,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-18T11:31:46.464 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:31:46.465 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.2440390400000005, 2024-09-18T11:31:46.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091774999999999999, 2024-09-18T11:31:46.465 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:46.465 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.465 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:46.465 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:46.465 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.465 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3209 2024-09-18T11:31:46.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.465 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:46.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:46.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:31:46.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:46.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:31:46.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:46.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:46.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098080+0000", 2024-09-18T11:31:46.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:46.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:46.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098676+0000", 2024-09-18T11:31:46.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059598099999999998 2024-09-18T11:31:46.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:46.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098706+0000", 2024-09-18T11:31:46.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0020000000000001e-05 2024-09-18T11:31:46.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098716+0000", 2024-09-18T11:31:46.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0225e-05 2024-09-18T11:31:46.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:46.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098782+0000", 2024-09-18T11:31:46.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5530000000000004e-05 2024-09-18T11:31:46.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:46.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098883+0000", 2024-09-18T11:31:46.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101609 2024-09-18T11:31:46.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:46.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.099098+0000", 2024-09-18T11:31:46.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000215052 2024-09-18T11:31:46.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:46.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.102511+0000", 2024-09-18T11:31:46.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034129429999999999 2024-09-18T11:31:46.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:46.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107165+0000", 2024-09-18T11:31:46.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046531480000000002 2024-09-18T11:31:46.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:46.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107217+0000", 2024-09-18T11:31:46.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2058000000000002e-05 2024-09-18T11:31:46.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:46.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:46.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:46.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107260+0000", 2024-09-18T11:31:46.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3865000000000001e-05 2024-09-18T11:31:46.470 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.470 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:46.470 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.471 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:46.471 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:46.471 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:46.472 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:31:46.474 INFO:tasks.rados.rados.0.smithi119.stdout:1557: finishing write tid 1 to smithi11939506-16 2024-09-18T11:31:46.474 INFO:tasks.rados.rados.0.smithi119.stdout:1557: finishing write tid 2 to smithi11939506-16 2024-09-18T11:31:46.474 INFO:tasks.rados.rados.0.smithi119.stdout:1557: finishing write tid 3 to smithi11939506-16 2024-09-18T11:31:46.474 INFO:tasks.rados.rados.0.smithi119.stdout:1557: finishing write tid 4 to smithi11939506-16 2024-09-18T11:31:46.474 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 262 (ObjNum 297 snap 43 seq_num 297) dirty exists 2024-09-18T11:31:46.474 INFO:tasks.rados.rados.0.smithi119.stdout:1557: left oid 16 (ObjNum 297 snap 43 seq_num 297) 2024-09-18T11:31:46.474 INFO:tasks.rados.rados.0.smithi119.stdout:1560: finishing write tid 1 to smithi11939506-22 2024-09-18T11:31:46.475 INFO:tasks.rados.rados.0.smithi119.stdout:1561: seq_num 299 ranges {625293=637218,1813763=494103,2978063=47230} 2024-09-18T11:31:46.475 INFO:tasks.rados.rados.0.smithi119.stdout:1561: writing smithi11939506-33 from 625293 to 1262511 tid 1 2024-09-18T11:31:46.475 INFO:tasks.rados.rados.0.smithi119.stdout:1561: writing smithi11939506-33 from 1813763 to 2307866 tid 2 2024-09-18T11:31:46.475 INFO:tasks.rados.rados.0.smithi119.stdout:1561: writing smithi11939506-33 from 2978063 to 3025293 tid 3 2024-09-18T11:31:46.475 INFO:tasks.rados.rados.0.smithi119.stdout:1556: done (4 left) 2024-09-18T11:31:46.475 INFO:tasks.rados.rados.0.smithi119.stdout:1557: done (3 left) 2024-09-18T11:31:46.475 INFO:tasks.rados.rados.0.smithi119.stdout:1558: done (2 left) 2024-09-18T11:31:46.475 INFO:tasks.rados.rados.0.smithi119.stdout:1562: read oid 2 snap 33 2024-09-18T11:31:46.475 INFO:tasks.rados.rados.0.smithi119.stdout:1562: expect (ObjNum 195 snap 23 seq_num 195) 2024-09-18T11:31:46.475 INFO:tasks.rados.rados.0.smithi119.stdout:1563: write oid 48 current snap is 43 2024-09-18T11:31:46.476 INFO:tasks.rados.rados.0.smithi119.stdout:1560: finishing write tid 2 to smithi11939506-22 2024-09-18T11:31:46.476 INFO:tasks.rados.rados.0.smithi119.stdout:1560: finishing write tid 3 to smithi11939506-22 2024-09-18T11:31:46.476 INFO:tasks.rados.rados.0.smithi119.stdout:1563: seq_num 300 ranges {642100=712467,1872245=637761} 2024-09-18T11:31:46.476 INFO:tasks.rados.rados.0.smithi119.stdout:1563: writing smithi11939506-48 from 642100 to 1354567 tid 1 2024-09-18T11:31:46.476 INFO:tasks.rados.rados.0.smithi119.stdout:1563: writing smithi11939506-48 from 1872245 to 2510006 tid 2 2024-09-18T11:31:46.476 INFO:tasks.rados.rados.0.smithi119.stdout:1564: delete oid 26 current snap is 43 2024-09-18T11:31:46.476 INFO:tasks.rados.rados.0.smithi119.stdout:1560: finishing write tid 4 to smithi11939506-22 2024-09-18T11:31:46.476 INFO:tasks.rados.rados.0.smithi119.stdout:1560: finishing write tid 5 to smithi11939506-22 2024-09-18T11:31:46.476 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 238 (ObjNum 298 snap 43 seq_num 298) dirty exists 2024-09-18T11:31:46.476 INFO:tasks.rados.rados.0.smithi119.stdout:1560: left oid 22 (ObjNum 298 snap 43 seq_num 298) 2024-09-18T11:31:46.476 INFO:tasks.rados.rados.0.smithi119.stdout:1561: finishing write tid 1 to smithi11939506-33 2024-09-18T11:31:46.476 INFO:tasks.rados.rados.0.smithi119.stdout:1561: finishing write tid 2 to smithi11939506-33 2024-09-18T11:31:46.476 INFO:tasks.rados.rados.0.smithi119.stdout:1561: finishing write tid 3 to smithi11939506-33 2024-09-18T11:31:46.476 INFO:tasks.rados.rados.0.smithi119.stdout:1561: finishing write tid 4 to smithi11939506-33 2024-09-18T11:31:46.477 INFO:tasks.rados.rados.0.smithi119.stdout:1561: finishing write tid 5 to smithi11939506-33 2024-09-18T11:31:46.477 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 33 v 149 (ObjNum 299 snap 43 seq_num 299) dirty exists 2024-09-18T11:31:46.477 INFO:tasks.rados.rados.0.smithi119.stdout:1561: left oid 33 (ObjNum 299 snap 43 seq_num 299) 2024-09-18T11:31:46.477 INFO:tasks.rados.rados.0.smithi119.stdout:1562: expect (ObjNum 195 snap 23 seq_num 195) 2024-09-18T11:31:46.477 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:46.471+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:46.572 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:46.571+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:46.576 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:46.577 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:46.577 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:46.577 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:46.585 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:31:46.673 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:46.672+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:46.679 INFO:tasks.rados.rados.0.smithi119.stdout:1563: finishing write tid 1 to smithi11939506-48 2024-09-18T11:31:46.679 INFO:tasks.rados.rados.0.smithi119.stdout:1560: done (4 left) 2024-09-18T11:31:46.679 INFO:tasks.rados.rados.0.smithi119.stdout:1561: done (3 left) 2024-09-18T11:31:46.679 INFO:tasks.rados.rados.0.smithi119.stdout:1562: done (2 left) 2024-09-18T11:31:46.679 INFO:tasks.rados.rados.0.smithi119.stdout:1564: done (1 left) 2024-09-18T11:31:46.679 INFO:tasks.rados.rados.0.smithi119.stdout:1565: read oid 17 snap 23 2024-09-18T11:31:46.679 INFO:tasks.rados.rados.0.smithi119.stdout:1565: expect (ObjNum 179 snap 20 seq_num 179) 2024-09-18T11:31:46.679 INFO:tasks.rados.rados.0.smithi119.stdout:1566: tier_flush oid 5 2024-09-18T11:31:46.679 INFO:tasks.rados.rados.0.smithi119.stdout:1566: tier_flush oid 5 snap -1 2024-09-18T11:31:46.679 INFO:tasks.rados.rados.0.smithi119.stdout:1563: finishing write tid 2 to smithi11939506-48 2024-09-18T11:31:46.679 INFO:tasks.rados.rados.0.smithi119.stdout:1563: finishing write tid 3 to smithi11939506-48 2024-09-18T11:31:46.679 INFO:tasks.rados.rados.0.smithi119.stdout:1563: finishing write tid 4 to smithi11939506-48 2024-09-18T11:31:46.680 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 48 v 241 (ObjNum 300 snap 43 seq_num 300) dirty exists 2024-09-18T11:31:46.680 INFO:tasks.rados.rados.0.smithi119.stdout:1563: left oid 48 (ObjNum 300 snap 43 seq_num 300) 2024-09-18T11:31:46.680 INFO:tasks.rados.rados.0.smithi119.stdout:1563: done (2 left) 2024-09-18T11:31:46.680 INFO:tasks.rados.rados.0.smithi119.stdout:1567: write oid 30 current snap is 43 2024-09-18T11:31:46.680 INFO:tasks.rados.rados.0.smithi119.stdout:1567: seq_num 302 ranges {675714=462965,1589209=641430,2943979=131735} 2024-09-18T11:31:46.682 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:46.682 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:46.682 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:46.682 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:46.682 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:46.684 INFO:tasks.rados.rados.0.smithi119.stdout:1567: writing smithi11939506-30 from 675714 to 1138679 tid 1 2024-09-18T11:31:46.690 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:31:46.692 INFO:tasks.rados.rados.0.smithi119.stdout:1567: writing smithi11939506-30 from 1589209 to 2230639 tid 2 2024-09-18T11:31:46.695 INFO:tasks.rados.rados.0.smithi119.stdout:1567: writing smithi11939506-30 from 2943979 to 3075714 tid 3 2024-09-18T11:31:46.695 INFO:tasks.rados.rados.0.smithi119.stdout:1568: write oid 33 current snap is 43 2024-09-18T11:31:46.695 INFO:tasks.rados.rados.0.smithi119.stdout:1566: got (16) Device or resource busy 2024-09-18T11:31:46.695 INFO:tasks.rados.rados.0.smithi119.stdout:check_oldest_snap_flushed oid 5 v 177 (ObjNum 272 snap 34 seq_num 272) unflushed 35 2024-09-18T11:31:46.695 INFO:tasks.rados.rados.0.smithi119.stdout:1568: seq_num 303 ranges {692521=538214,1931338=785088} 2024-09-18T11:31:46.701 INFO:tasks.rados.rados.0.smithi119.stdout:1568: writing smithi11939506-33 from 692521 to 1230735 tid 1 2024-09-18T11:31:46.709 INFO:tasks.rados.rados.0.smithi119.stdout:1568: writing smithi11939506-33 from 1931338 to 2716426 tid 2 2024-09-18T11:31:46.709 INFO:tasks.rados.rados.0.smithi119.stdout:1566: done (3 left) 2024-09-18T11:31:46.709 INFO:tasks.rados.rados.0.smithi119.stdout:1569: write oid 42 current snap is 43 2024-09-18T11:31:46.709 INFO:tasks.rados.rados.0.smithi119.stdout:1569: seq_num 304 ranges {709328=613463,1989820=645099} 2024-09-18T11:31:46.716 INFO:tasks.rados.rados.0.smithi119.stdout:1569: writing smithi11939506-42 from 709328 to 1322791 tid 1 2024-09-18T11:31:46.724 INFO:tasks.rados.rados.0.smithi119.stdout:1569: writing smithi11939506-42 from 1989820 to 2634919 tid 2 2024-09-18T11:31:46.724 INFO:tasks.rados.rados.0.smithi119.stdout:1570: read oid 32 snap -1 2024-09-18T11:31:46.724 INFO:tasks.rados.rados.0.smithi119.stdout:1570: expect (ObjNum 217 snap 26 seq_num 217) 2024-09-18T11:31:46.724 INFO:tasks.rados.rados.0.smithi119.stdout:1571: read oid 10 snap -1 2024-09-18T11:31:46.724 INFO:tasks.rados.rados.0.smithi119.stdout:1571: expect (ObjNum 260 snap 31 seq_num 260) 2024-09-18T11:31:46.724 INFO:tasks.rados.rados.0.smithi119.stdout:1572: write (excl) oid 50 current snap is 43 2024-09-18T11:31:46.724 INFO:tasks.rados.rados.0.smithi119.stdout:1565: expect (ObjNum 179 snap 20 seq_num 179) 2024-09-18T11:31:46.735 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:31:46.774 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:46.773+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:46.792 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:46.792 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:46.792 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:31:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 72.374003259000006, 2024-09-18T11:31:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:31:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.793 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:31:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:31:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:31:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:31:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:31:46.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:31:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:31:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:31:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:31:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:31:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:31:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:31:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:31:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:31:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:31:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:31:46.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:31:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:31:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:31:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:31:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:46.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:31:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:31:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1628 3.3 3:f80aa31b:::77cdbf3abbada57d0636b2198046a030673818705cbf13bb77b30c3e68b24345:head [write 0~582144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 69.479113259000002, 2024-09-18T11:31:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081714639999999998, 2024-09-18T11:31:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:46.799 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1628 2024-09-18T11:31:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310975+0000", 2024-09-18T11:31:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312086+0000", 2024-09-18T11:31:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001111288 2024-09-18T11:31:46.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312104+0000", 2024-09-18T11:31:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7427000000000001e-05 2024-09-18T11:31:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312110+0000", 2024-09-18T11:31:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5320000000000003e-06 2024-09-18T11:31:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312179+0000", 2024-09-18T11:31:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9196000000000006e-05 2024-09-18T11:31:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312375+0000", 2024-09-18T11:31:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00019550899999999999 2024-09-18T11:31:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312558+0000", 2024-09-18T11:31:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000182566 2024-09-18T11:31:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.314286+0000", 2024-09-18T11:31:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00172831 2024-09-18T11:31:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319079+0000", 2024-09-18T11:31:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047929340000000004 2024-09-18T11:31:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319116+0000", 2024-09-18T11:31:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6993999999999999e-05 2024-09-18T11:31:46.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319148+0000", 2024-09-18T11:31:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2539999999999997e-05 2024-09-18T11:31:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1664 4.1 4:87078dfe:::smithi11939506-14:head [setxattr _header (57) in=7b,truncate 2235331] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 69.425853039000003, 2024-09-18T11:31:46.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010368235999999999, 2024-09-18T11:31:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1664 2024-09-18T11:31:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:46.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364236+0000", 2024-09-18T11:31:46.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364240+0000", 2024-09-18T11:31:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9829999999999998e-06 2024-09-18T11:31:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364247+0000", 2024-09-18T11:31:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2140000000000001e-06 2024-09-18T11:31:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364256+0000", 2024-09-18T11:31:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1509999999999993e-06 2024-09-18T11:31:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366529+0000", 2024-09-18T11:31:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022731409999999998 2024-09-18T11:31:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366621+0000", 2024-09-18T11:31:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2408e-05 2024-09-18T11:31:46.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366801+0000", 2024-09-18T11:31:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018002200000000001 2024-09-18T11:31:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372155+0000", 2024-09-18T11:31:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053531519999999999 2024-09-18T11:31:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374539+0000", 2024-09-18T11:31:46.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023842580000000002 2024-09-18T11:31:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374573+0000", 2024-09-18T11:31:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4205999999999997e-05 2024-09-18T11:31:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374605+0000", 2024-09-18T11:31:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2283999999999998e-05 2024-09-18T11:31:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1665 4.1 4:87078dfe:::smithi11939506-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 69.425828865, 2024-09-18T11:31:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010471897000000001, 2024-09-18T11:31:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1665 2024-09-18T11:31:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.812 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:46.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364263+0000", 2024-09-18T11:31:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6400000000000001e-06 2024-09-18T11:31:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364270+0000", 2024-09-18T11:31:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:31:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364274+0000", 2024-09-18T11:31:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.968e-06 2024-09-18T11:31:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367354+0000", 2024-09-18T11:31:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030803430000000001 2024-09-18T11:31:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367406+0000", 2024-09-18T11:31:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2330999999999998e-05 2024-09-18T11:31:46.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374625+0000", 2024-09-18T11:31:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072190689999999998 2024-09-18T11:31:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374673+0000", 2024-09-18T11:31:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7525999999999999e-05 2024-09-18T11:31:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374733+0000", 2024-09-18T11:31:46.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0359000000000003e-05 2024-09-18T11:31:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1947 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865577,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 55.450549154999997, 2024-09-18T11:31:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010043066999999999, 2024-09-18T11:31:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:46.817 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1947 2024-09-18T11:31:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:46.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339540+0000", 2024-09-18T11:31:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339545+0000", 2024-09-18T11:31:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6290000000000001e-06 2024-09-18T11:31:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339553+0000", 2024-09-18T11:31:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6299999999999998e-06 2024-09-18T11:31:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339557+0000", 2024-09-18T11:31:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3590000000000001e-06 2024-09-18T11:31:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345600+0000", 2024-09-18T11:31:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060434640000000001 2024-09-18T11:31:46.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345710+0000", 2024-09-18T11:31:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010924700000000001 2024-09-18T11:31:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349584+0000", 2024-09-18T11:31:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038743459999999999 2024-09-18T11:31:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1948 4.d 4:b5a463bf:::smithi11939506-20:head [sparse-read 0~2642181] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 55.450528618, 2024-09-18T11:31:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010701952000000001, 2024-09-18T11:31:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1948 2024-09-18T11:31:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:46.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:46.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339563+0000", 2024-09-18T11:31:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1950000000000002e-06 2024-09-18T11:31:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339569+0000", 2024-09-18T11:31:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5459999999999999e-06 2024-09-18T11:31:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339572+0000", 2024-09-18T11:31:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.337e-06 2024-09-18T11:31:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349693+0000", 2024-09-18T11:31:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010120987999999999 2024-09-18T11:31:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349816+0000", 2024-09-18T11:31:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012241199999999999 2024-09-18T11:31:46.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350263+0000", 2024-09-18T11:31:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00044787199999999998 2024-09-18T11:31:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:172 4.5 e78/22 4:a087a566:::smithi11939506-46:19 v 78'135, mlcod=77'132)", 2024-09-18T11:31:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.404266604999997, 2024-09-18T11:31:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010938658, 2024-09-18T11:31:46.826 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385823+0000", 2024-09-18T11:31:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385830+0000", 2024-09-18T11:31:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2790000000000001e-06 2024-09-18T11:31:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385843+0000", 2024-09-18T11:31:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2785e-05 2024-09-18T11:31:46.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385848+0000", 2024-09-18T11:31:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5570000000000003e-06 2024-09-18T11:31:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385914+0000", 2024-09-18T11:31:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5168999999999999e-05 2024-09-18T11:31:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:46.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385930+0000", 2024-09-18T11:31:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6884999999999998e-05 2024-09-18T11:31:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396721+0000", 2024-09-18T11:31:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010790137 2024-09-18T11:31:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396762+0000", 2024-09-18T11:31:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1635999999999999e-05 2024-09-18T11:31:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:46.830 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:31:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.403773852999997, 2024-09-18T11:31:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:31:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:46.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:31:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:31:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:31:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:31:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:31:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:31:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:31:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:31:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:31:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:31:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:31:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:31:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:31:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:31:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 37.817661246999997, 2024-09-18T11:31:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:31:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:46.835 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:31:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:31:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:31:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:31:46.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:31:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:31:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:31:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:31:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:31:46.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:31:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:31:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:31:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:31:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:31:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:31:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:31:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:31:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:31:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:31:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:31:46.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:31:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:31:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2647 4.e 4:78eefc4b:::smithi11939506-7:head [append 0~0] snapc 30=[30,2f,2b,2a,25,24,1c] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:46.841 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.780246566999999, 2024-09-18T11:31:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096752739999999993, 2024-09-18T11:31:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2647 2024-09-18T11:31:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:46.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009843+0000", 2024-09-18T11:31:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:46.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009846+0000", 2024-09-18T11:31:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3870000000000002e-06 2024-09-18T11:31:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009853+0000", 2024-09-18T11:31:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.238e-06 2024-09-18T11:31:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009860+0000", 2024-09-18T11:31:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1999999999999997e-06 2024-09-18T11:31:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.013798+0000", 2024-09-18T11:31:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003938441 2024-09-18T11:31:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.013889+0000", 2024-09-18T11:31:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0332000000000002e-05 2024-09-18T11:31:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 6", 2024-09-18T11:31:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.014066+0000", 2024-09-18T11:31:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017728700000000001 2024-09-18T11:31:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015813+0000", 2024-09-18T11:31:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001746804 2024-09-18T11:31:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019419+0000", 2024-09-18T11:31:46.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036058399999999999 2024-09-18T11:31:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019478+0000", 2024-09-18T11:31:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8860999999999998e-05 2024-09-18T11:31:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019519+0000", 2024-09-18T11:31:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1208000000000003e-05 2024-09-18T11:31:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:209 4.5 e91/22 4:af4ba75a:::smithi11939506-37:19 v 91'165, mlcod=91'161)", 2024-09-18T11:31:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.772169839, 2024-09-18T11:31:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0091546930000000002, 2024-09-18T11:31:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:46.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017925+0000", 2024-09-18T11:31:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0819999999999998e-06 2024-09-18T11:31:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017935+0000", 2024-09-18T11:31:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0597999999999999e-05 2024-09-18T11:31:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017941+0000", 2024-09-18T11:31:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2920000000000003e-06 2024-09-18T11:31:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017978+0000", 2024-09-18T11:31:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7667999999999999e-05 2024-09-18T11:31:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018018+0000", 2024-09-18T11:31:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9700999999999998e-05 2024-09-18T11:31:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.027039+0000", 2024-09-18T11:31:46.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090214079999999999 2024-09-18T11:31:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.027075+0000", 2024-09-18T11:31:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5605000000000002e-05 2024-09-18T11:31:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:570 4.b e91/67 4:ddcb7ff4:::smithi11939506-9:18 v 91'251, mlcod=90'245)", 2024-09-18T11:31:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:46.852 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.772134067000003, 2024-09-18T11:31:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089212549999999995, 2024-09-18T11:31:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017960+0000", 2024-09-18T11:31:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.777e-06 2024-09-18T11:31:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017966+0000", 2024-09-18T11:31:46.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4389999999999997e-06 2024-09-18T11:31:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017970+0000", 2024-09-18T11:31:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5549999999999998e-06 2024-09-18T11:31:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018032+0000", 2024-09-18T11:31:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2976000000000001e-05 2024-09-18T11:31:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018053+0000", 2024-09-18T11:31:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0322e-05 2024-09-18T11:31:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026844+0000", 2024-09-18T11:31:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087915790000000008 2024-09-18T11:31:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026877+0000", 2024-09-18T11:31:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2916000000000001e-05 2024-09-18T11:31:46.856 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.5.13:15 3.1 3:bf804683:::e58794d0a579007c3bd048992c5646e06b9775f9667464eee6a2a0508e23a0a6:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.771979219999999, 2024-09-18T11:31:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008816908, 2024-09-18T11:31:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018110+0000", 2024-09-18T11:31:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:46.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018114+0000", 2024-09-18T11:31:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0339999999999998e-06 2024-09-18T11:31:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018121+0000", 2024-09-18T11:31:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0199999999999997e-06 2024-09-18T11:31:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018125+0000", 2024-09-18T11:31:46.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0409999999999999e-06 2024-09-18T11:31:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018793+0000", 2024-09-18T11:31:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00066779199999999995 2024-09-18T11:31:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018868+0000", 2024-09-18T11:31:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4814999999999994e-05 2024-09-18T11:31:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019008+0000", 2024-09-18T11:31:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014031800000000001 2024-09-18T11:31:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.023494+0000", 2024-09-18T11:31:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0044861720000000001 2024-09-18T11:31:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026796+0000", 2024-09-18T11:31:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0033010969999999998 2024-09-18T11:31:46.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026891+0000", 2024-09-18T11:31:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5828000000000005e-05 2024-09-18T11:31:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026928+0000", 2024-09-18T11:31:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6393999999999997e-05 2024-09-18T11:31:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2793 4.1 4:87078dfe:::smithi11939506-14:head [write 534522~634550] snapc 30=[30,2f,2b,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:31:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.566406592, 2024-09-18T11:31:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085134289999999994, 2024-09-18T11:31:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2793 2024-09-18T11:31:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:46.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223681+0000", 2024-09-18T11:31:46.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224251+0000", 2024-09-18T11:31:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056940699999999996 2024-09-18T11:31:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224267+0000", 2024-09-18T11:31:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6829000000000001e-05 2024-09-18T11:31:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224292+0000", 2024-09-18T11:31:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4198000000000001e-05 2024-09-18T11:31:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224352+0000", 2024-09-18T11:31:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0608e-05 2024-09-18T11:31:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224456+0000", 2024-09-18T11:31:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010412300000000001 2024-09-18T11:31:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224662+0000", 2024-09-18T11:31:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020561399999999999 2024-09-18T11:31:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231800+0000", 2024-09-18T11:31:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071384530000000003 2024-09-18T11:31:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232126+0000", 2024-09-18T11:31:46.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00032601400000000001 2024-09-18T11:31:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232158+0000", 2024-09-18T11:31:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1472e-05 2024-09-18T11:31:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232197+0000", 2024-09-18T11:31:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9011999999999999e-05 2024-09-18T11:31:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:46.868 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:123 3.1 3:bae7b855:::93307fb660774b1d286e3149e1a75006840a12081178bf31d320df6af68ad704:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:23.045517+0000", 2024-09-18T11:31:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.744573396, 2024-09-18T11:31:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081745810000000002, 2024-09-18T11:31:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.045517+0000", 2024-09-18T11:31:46.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.045517+0000", 2024-09-18T11:31:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.045516+0000", 2024-09-18T11:31:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:46.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046503+0000", 2024-09-18T11:31:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00098663900000000009 2024-09-18T11:31:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046516+0000", 2024-09-18T11:31:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2971e-05 2024-09-18T11:31:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046522+0000", 2024-09-18T11:31:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.618e-06 2024-09-18T11:31:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046562+0000", 2024-09-18T11:31:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9400999999999998e-05 2024-09-18T11:31:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046639+0000", 2024-09-18T11:31:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7037000000000001e-05 2024-09-18T11:31:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046809+0000", 2024-09-18T11:31:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017013799999999999 2024-09-18T11:31:46.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.049963+0000", 2024-09-18T11:31:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031541389999999998 2024-09-18T11:31:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.053637+0000", 2024-09-18T11:31:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036736550000000001 2024-09-18T11:31:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:46.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.053665+0000", 2024-09-18T11:31:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8759999999999999e-05 2024-09-18T11:31:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.053691+0000", 2024-09-18T11:31:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5978000000000001e-05 2024-09-18T11:31:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2955 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:46.874 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:31:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.762898818, 2024-09-18T11:31:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010869531, 2024-09-18T11:31:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2955 2024-09-18T11:31:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:46.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:31:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:31:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027190+0000", 2024-09-18T11:31:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027196+0000", 2024-09-18T11:31:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5590000000000002e-06 2024-09-18T11:31:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027208+0000", 2024-09-18T11:31:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1816e-05 2024-09-18T11:31:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027215+0000", 2024-09-18T11:31:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4509999999999997e-06 2024-09-18T11:31:46.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027725+0000", 2024-09-18T11:31:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000510053 2024-09-18T11:31:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027803+0000", 2024-09-18T11:31:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8750000000000003e-05 2024-09-18T11:31:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032412+0000", 2024-09-18T11:31:46.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046089030000000001 2024-09-18T11:31:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032499+0000", 2024-09-18T11:31:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6316000000000005e-05 2024-09-18T11:31:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038061+0000", 2024-09-18T11:31:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005562259 2024-09-18T11:31:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:46.879 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2956 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~2100908,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.762864997000001, 2024-09-18T11:31:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011863831, 2024-09-18T11:31:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2956 2024-09-18T11:31:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.880 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:46.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027228+0000", 2024-09-18T11:31:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:31:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027235+0000", 2024-09-18T11:31:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:31:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027239+0000", 2024-09-18T11:31:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.535e-06 2024-09-18T11:31:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027831+0000", 2024-09-18T11:31:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059244699999999996 2024-09-18T11:31:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027879+0000", 2024-09-18T11:31:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:31:46.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038098+0000", 2024-09-18T11:31:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010219137 2024-09-18T11:31:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038207+0000", 2024-09-18T11:31:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010951999999999999 2024-09-18T11:31:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:46.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039089+0000", 2024-09-18T11:31:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00088151699999999996 2024-09-18T11:31:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2957 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.762847213000001, 2024-09-18T11:31:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012041979, 2024-09-18T11:31:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:46.885 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2957 2024-09-18T11:31:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:46.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027245+0000", 2024-09-18T11:31:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:31:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:46.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027251+0000", 2024-09-18T11:31:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0090000000000004e-06 2024-09-18T11:31:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027255+0000", 2024-09-18T11:31:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:31:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027895+0000", 2024-09-18T11:31:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064076400000000005 2024-09-18T11:31:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027930+0000", 2024-09-18T11:31:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4334e-05 2024-09-18T11:31:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039111+0000", 2024-09-18T11:31:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181455999999999 2024-09-18T11:31:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039156+0000", 2024-09-18T11:31:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5256000000000002e-05 2024-09-18T11:31:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039285+0000", 2024-09-18T11:31:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012839200000000001 2024-09-18T11:31:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3220 3.3 3:d97f8959:::efae800be116a8f4318957447289bc2a2c2d33d65c7604341c6a4a86aad75d7e:head [write 0~1337344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-18T11:31:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:37.110836+0000", 2024-09-18T11:31:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.6792539630000007, 2024-09-18T11:31:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089207350000000008, 2024-09-18T11:31:46.890 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3220 2024-09-18T11:31:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.110836+0000", 2024-09-18T11:31:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.110836+0000", 2024-09-18T11:31:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.110834+0000", 2024-09-18T11:31:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114557+0000", 2024-09-18T11:31:46.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003723166 2024-09-18T11:31:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114576+0000", 2024-09-18T11:31:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.861e-05 2024-09-18T11:31:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114585+0000", 2024-09-18T11:31:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0780000000000002e-06 2024-09-18T11:31:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114630+0000", 2024-09-18T11:31:46.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5118999999999999e-05 2024-09-18T11:31:46.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:46.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114748+0000", 2024-09-18T11:31:46.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011817899999999999 2024-09-18T11:31:46.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 6", 2024-09-18T11:31:46.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114862+0000", 2024-09-18T11:31:46.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113978 2024-09-18T11:31:46.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:46.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.116796+0000", 2024-09-18T11:31:46.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001934343 2024-09-18T11:31:46.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:46.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.119693+0000", 2024-09-18T11:31:46.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028964820000000001 2024-09-18T11:31:46.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:46.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.119729+0000", 2024-09-18T11:31:46.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6022000000000002e-05 2024-09-18T11:31:46.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:46.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.119757+0000", 2024-09-18T11:31:46.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7880000000000001e-05 2024-09-18T11:31:46.896 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.896 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:46.896 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.896 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3474 4.5 e117/22 4:a56ea1fb:::smithi11939506-48:head v 117'240, mlcod=117'237)", 2024-09-18T11:31:46.896 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.444713+0000", 2024-09-18T11:31:46.896 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.34537685800000001, 2024-09-18T11:31:46.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081017789999999999, 2024-09-18T11:31:46.897 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:46.897 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:46.897 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:46.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:46.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444713+0000", 2024-09-18T11:31:46.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:46.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444713+0000", 2024-09-18T11:31:46.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:46.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:46.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444712+0000", 2024-09-18T11:31:46.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:46.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:46.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444858+0000", 2024-09-18T11:31:46.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014590299999999999 2024-09-18T11:31:46.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:46.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444867+0000", 2024-09-18T11:31:46.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2480000000000002e-06 2024-09-18T11:31:46.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:46.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444872+0000", 2024-09-18T11:31:46.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2950000000000004e-06 2024-09-18T11:31:46.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:46.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444906+0000", 2024-09-18T11:31:46.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4017999999999999e-05 2024-09-18T11:31:46.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:46.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444920+0000", 2024-09-18T11:31:46.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4321000000000001e-05 2024-09-18T11:31:46.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:46.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452785+0000", 2024-09-18T11:31:46.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0078646500000000008 2024-09-18T11:31:46.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:46.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:46.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:46.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452815+0000", 2024-09-18T11:31:46.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0321e-05 2024-09-18T11:31:46.900 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.900 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:46.901 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.901 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:46.901 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:46.901 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:46.902 INFO:tasks.rados.rados.0.smithi119.stdout:1567: finishing write tid 1 to smithi11939506-30 2024-09-18T11:31:46.902 INFO:tasks.rados.rados.0.smithi119.stdout:1567: finishing write tid 2 to smithi11939506-30 2024-09-18T11:31:46.902 INFO:tasks.rados.rados.0.smithi119.stdout:1572: seq_num 305 ranges {726135=405065,1764655=788757,3026248=99887} 2024-09-18T11:31:46.902 INFO:tasks.rados.rados.0.smithi119.stdout:1572: writing smithi11939506-50 from 726135 to 1131200 tid 1 2024-09-18T11:31:46.903 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:46.903 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:31:46.905 INFO:tasks.rados.rados.0.smithi119.stdout:1572: writing smithi11939506-50 from 1764655 to 2553412 tid 2 2024-09-18T11:31:46.905 INFO:tasks.rados.rados.0.smithi119.stdout:1572: writing smithi11939506-50 from 3026248 to 3126135 tid 3 2024-09-18T11:31:46.905 INFO:tasks.rados.rados.0.smithi119.stdout:1565: done (6 left) 2024-09-18T11:31:46.905 INFO:tasks.rados.rados.0.smithi119.stdout:1573: write oid 38 current snap is 43 2024-09-18T11:31:46.905 INFO:tasks.rados.rados.0.smithi119.stdout:1567: finishing write tid 3 to smithi11939506-30 2024-09-18T11:31:46.905 INFO:tasks.rados.rados.0.smithi119.stdout:1567: finishing write tid 4 to smithi11939506-30 2024-09-18T11:31:46.905 INFO:tasks.rados.rados.0.smithi119.stdout:1567: finishing write tid 5 to smithi11939506-30 2024-09-18T11:31:46.905 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 30 v 123 (ObjNum 302 snap 43 seq_num 302) dirty exists 2024-09-18T11:31:46.905 INFO:tasks.rados.rados.0.smithi119.stdout:1567: left oid 30 (ObjNum 302 snap 43 seq_num 302) 2024-09-18T11:31:46.905 INFO:tasks.rados.rados.0.smithi119.stdout:1568: finishing write tid 1 to smithi11939506-33 2024-09-18T11:31:46.905 INFO:tasks.rados.rados.0.smithi119.stdout:1568: finishing write tid 2 to smithi11939506-33 2024-09-18T11:31:46.905 INFO:tasks.rados.rados.0.smithi119.stdout:1568: finishing write tid 3 to smithi11939506-33 2024-09-18T11:31:46.906 INFO:tasks.rados.rados.0.smithi119.stdout:1568: finishing write tid 4 to smithi11939506-33 2024-09-18T11:31:46.906 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 33 v 152 (ObjNum 303 snap 43 seq_num 303) dirty exists 2024-09-18T11:31:46.906 INFO:tasks.rados.rados.0.smithi119.stdout:1568: left oid 33 (ObjNum 303 snap 43 seq_num 303) 2024-09-18T11:31:46.906 INFO:tasks.rados.rados.0.smithi119.stdout:1569: finishing write tid 1 to smithi11939506-42 2024-09-18T11:31:46.906 INFO:tasks.rados.rados.0.smithi119.stdout:1573: seq_num 306 ranges {742942=480314,1823137=648768} 2024-09-18T11:31:46.906 INFO:tasks.rados.rados.0.smithi119.stdout:1573: writing smithi11939506-38 from 742942 to 1223256 tid 1 2024-09-18T11:31:46.906 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:46.906 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:46.908 INFO:tasks.rados.rados.0.smithi119.stdout:1573: writing smithi11939506-38 from 1823137 to 2471905 tid 2 2024-09-18T11:31:46.908 INFO:tasks.rados.rados.0.smithi119.stdout:1567: done (6 left) 2024-09-18T11:31:46.908 INFO:tasks.rados.rados.0.smithi119.stdout:1568: done (5 left) 2024-09-18T11:31:46.908 INFO:tasks.rados.rados.0.smithi119.stdout:1574: read oid 18 snap 37 2024-09-18T11:31:46.908 INFO:tasks.rados.rados.0.smithi119.stdout:1574: expect (ObjNum 205 snap 23 seq_num 205) 2024-09-18T11:31:46.908 INFO:tasks.rados.rados.0.smithi119.stdout:1575: read oid 7 snap -1 2024-09-18T11:31:46.909 INFO:tasks.rados.rados.0.smithi119.stdout:1575: expect (ObjNum 145 snap 18 seq_num 145) 2024-09-18T11:31:46.909 INFO:tasks.rados.rados.0.smithi119.stdout:1576: read oid 33 snap -1 2024-09-18T11:31:46.909 INFO:tasks.rados.rados.0.smithi119.stdout:1576: expect (ObjNum 303 snap 43 seq_num 303) 2024-09-18T11:31:46.909 INFO:tasks.rados.rados.0.smithi119.stdout:1577: read oid 47 snap -1 2024-09-18T11:31:46.909 INFO:tasks.rados.rados.0.smithi119.stdout:1577: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T11:31:46.909 INFO:tasks.rados.rados.0.smithi119.stdout:1578: write (excl) oid 2 current snap is 43 2024-09-18T11:31:46.909 INFO:tasks.rados.rados.0.smithi119.stdout:1578: seq_num 307 ranges {759749=555563,1765266=792426,2964741=195008} 2024-09-18T11:31:46.909 INFO:tasks.rados.rados.0.smithi119.stdout:1578: writing smithi11939506-2 from 759749 to 1315312 tid 1 2024-09-18T11:31:46.909 INFO:tasks.rados.rados.0.smithi119.stdout:1578: writing smithi11939506-2 from 1765266 to 2557692 tid 2 2024-09-18T11:31:46.909 INFO:tasks.rados.rados.0.smithi119.stdout:1578: writing smithi11939506-2 from 2964741 to 3159749 tid 3 2024-09-18T11:31:46.909 INFO:tasks.rados.rados.0.smithi119.stdout:1579: set_chunk oid 20 target oid 2024-09-18T11:31:46.910 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:46.910 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:46.910 INFO:tasks.rados.rados.0.smithi119.stdout:1579: set_chunk oid 20 offset: 370176 length: 1182720 target oid 4d1090af523cbb0eaff747c2198c91c1e2c181b2770a04f54ea68cb3101a2412 offset: 0 snap -1 2024-09-18T11:31:46.910 INFO:tasks.rados.rados.0.smithi119.stdout:1580: write (excl) oid 36 current snap is 43 2024-09-18T11:31:46.910 INFO:tasks.rados.rados.0.smithi119.stdout:1570: expect (ObjNum 217 snap 26 seq_num 217) 2024-09-18T11:31:46.911 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:46.911 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:46.911 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:46.901+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:46.911 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:46.912 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:46.923 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:46.923 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:46.949 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:46.949 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:46.975 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:46.975 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:46.978 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:46.978 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:47.020 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:47.019+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:47.041 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:47.041 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:47.049 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:31:47.093 INFO:tasks.rados.rados.0.smithi119.stdout:1569: finishing write tid 2 to smithi11939506-42 2024-09-18T11:31:47.093 INFO:tasks.rados.rados.0.smithi119.stdout:1569: finishing write tid 3 to smithi11939506-42 2024-09-18T11:31:47.094 INFO:tasks.rados.rados.0.smithi119.stdout:1569: finishing write tid 4 to smithi11939506-42 2024-09-18T11:31:47.094 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 217 (ObjNum 304 snap 43 seq_num 304) dirty exists 2024-09-18T11:31:47.094 INFO:tasks.rados.rados.0.smithi119.stdout:1569: left oid 42 (ObjNum 304 snap 43 seq_num 304) 2024-09-18T11:31:47.094 INFO:tasks.rados.rados.0.smithi119.stdout:1571: expect (ObjNum 260 snap 31 seq_num 260) 2024-09-18T11:31:47.104 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:47.103+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:47.146 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:47.147 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:47.155 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:31:47.194 INFO:tasks.rados.rados.0.smithi119.stdout:1572: finishing write tid 1 to smithi11939506-50 2024-09-18T11:31:47.194 INFO:tasks.rados.rados.0.smithi119.stdout:1572: finishing write tid 2 to smithi11939506-50 2024-09-18T11:31:47.194 INFO:tasks.rados.rados.0.smithi119.stdout:1572: finishing write tid 3 to smithi11939506-50 2024-09-18T11:31:47.194 INFO:tasks.rados.rados.0.smithi119.stdout:1572: finishing write tid 4 to smithi11939506-50 2024-09-18T11:31:47.194 INFO:tasks.rados.rados.0.smithi119.stdout:1572: finishing write tid 5 to smithi11939506-50 2024-09-18T11:31:47.194 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 176 (ObjNum 305 snap 43 seq_num 305) dirty exists 2024-09-18T11:31:47.194 INFO:tasks.rados.rados.0.smithi119.stdout:1572: left oid 50 (ObjNum 305 snap 43 seq_num 305) 2024-09-18T11:31:47.195 INFO:tasks.rados.rados.0.smithi119.stdout:1573: finishing write tid 1 to smithi11939506-38 2024-09-18T11:31:47.195 INFO:tasks.rados.rados.0.smithi119.stdout:1580: seq_num 308 ranges {776556=630812,1823748=652437,3108517=68039} 2024-09-18T11:31:47.201 INFO:tasks.rados.rados.0.smithi119.stdout:1580: writing smithi11939506-36 from 776556 to 1407368 tid 1 2024-09-18T11:31:47.205 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:47.203+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:47.207 INFO:tasks.rados.rados.0.smithi119.stdout:1580: writing smithi11939506-36 from 1823748 to 2476185 tid 2 2024-09-18T11:31:47.211 INFO:tasks.rados.rados.0.smithi119.stdout:1580: writing smithi11939506-36 from 3108517 to 3176556 tid 3 2024-09-18T11:31:47.212 INFO:tasks.rados.rados.0.smithi119.stdout:1569: done (11 left) 2024-09-18T11:31:47.212 INFO:tasks.rados.rados.0.smithi119.stdout:1570: done (10 left) 2024-09-18T11:31:47.212 INFO:tasks.rados.rados.0.smithi119.stdout:1571: done (9 left) 2024-09-18T11:31:47.212 INFO:tasks.rados.rados.0.smithi119.stdout:1572: done (8 left) 2024-09-18T11:31:47.212 INFO:tasks.rados.rados.0.smithi119.stdout:1581: read oid 9 snap -1 2024-09-18T11:31:47.212 INFO:tasks.rados.rados.0.smithi119.stdout:1581: expect (ObjNum 199 snap 23 seq_num 199) 2024-09-18T11:31:47.212 INFO:tasks.rados.rados.0.smithi119.stdout:1582: write (excl) oid 13 current snap is 43 2024-09-18T11:31:47.212 INFO:tasks.rados.rados.0.smithi119.stdout:1582: seq_num 309 ranges {793363=706061,2282230=796095} 2024-09-18T11:31:47.220 INFO:tasks.rados.rados.0.smithi119.stdout:1582: writing smithi11939506-13 from 793363 to 1499424 tid 1 2024-09-18T11:31:47.229 INFO:tasks.rados.rados.0.smithi119.stdout:1582: writing smithi11939506-13 from 2282230 to 3078325 tid 2 2024-09-18T11:31:47.229 INFO:tasks.rados.rados.0.smithi119.stdout:1583: rollback oid 49 current snap is 43 2024-09-18T11:31:47.229 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 49 to 36 2024-09-18T11:31:47.229 INFO:tasks.rados.rados.0.smithi119.stdout:1575: expect (ObjNum 145 snap 18 seq_num 145) 2024-09-18T11:31:47.258 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:47.258 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:47.267 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:31:47.308 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:47.307+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:47.380 INFO:tasks.rados.rados.0.smithi119.stdout:1578: finishing write tid 1 to smithi11939506-2 2024-09-18T11:31:47.381 INFO:tasks.rados.rados.0.smithi119.stdout:1578: finishing write tid 2 to smithi11939506-2 2024-09-18T11:31:47.381 INFO:tasks.rados.rados.0.smithi119.stdout:1578: finishing write tid 3 to smithi11939506-2 2024-09-18T11:31:47.381 INFO:tasks.rados.rados.0.smithi119.stdout:1578: finishing write tid 4 to smithi11939506-2 2024-09-18T11:31:47.381 INFO:tasks.rados.rados.0.smithi119.stdout:1578: finishing write tid 5 to smithi11939506-2 2024-09-18T11:31:47.381 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 357 (ObjNum 307 snap 43 seq_num 307) dirty exists 2024-09-18T11:31:47.381 INFO:tasks.rados.rados.0.smithi119.stdout:1578: left oid 2 (ObjNum 307 snap 43 seq_num 307) 2024-09-18T11:31:47.381 INFO:tasks.rados.rados.0.smithi119.stdout:1577: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T11:31:47.405 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:47.405 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:47.406 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:47.405+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:47.413 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:31:47.470 INFO:tasks.rados.rados.0.smithi119.stdout:1576: expect (ObjNum 303 snap 43 seq_num 303) 2024-09-18T11:31:47.507 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:47.506+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:47.551 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:47.552 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:47.560 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:31:47.608 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:47.607+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:47.657 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:47.657 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:47.659 INFO:tasks.rados.rados.0.smithi119.stdout:1573: finishing write tid 2 to smithi11939506-38 2024-09-18T11:31:47.659 INFO:tasks.rados.rados.0.smithi119.stdout:1573: finishing write tid 3 to smithi11939506-38 2024-09-18T11:31:47.659 INFO:tasks.rados.rados.0.smithi119.stdout:1573: finishing write tid 4 to smithi11939506-38 2024-09-18T11:31:47.659 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 182 (ObjNum 306 snap 43 seq_num 306) dirty exists 2024-09-18T11:31:47.659 INFO:tasks.rados.rados.0.smithi119.stdout:1573: left oid 38 (ObjNum 306 snap 43 seq_num 306) 2024-09-18T11:31:47.659 INFO:tasks.rados.rados.0.smithi119.stdout:1574: expect (ObjNum 205 snap 23 seq_num 205) 2024-09-18T11:31:47.666 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:31:47.709 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:47.708+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:47.763 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:47.764 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:47.772 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:31:47.813 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:47.812+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:47.844 INFO:tasks.rados.rados.0.smithi119.stdout:1579: finishing set_chunk to oid 20 2024-09-18T11:31:47.844 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 20 set_chunk returned error code -95 offset: 370176 length: 1182720 2024-09-18T11:31:47.845 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 20 v 0 (ObjNum 111 snap 9 seq_num 111) dirty exists 2024-09-18T11:31:47.845 INFO:tasks.rados.rados.0.smithi119.stdout:1580: finishing write tid 1 to smithi11939506-36 2024-09-18T11:31:47.845 INFO:tasks.rados.rados.0.smithi119.stdout:1580: finishing write tid 2 to smithi11939506-36 2024-09-18T11:31:47.845 INFO:tasks.rados.rados.0.smithi119.stdout:1580: finishing write tid 3 to smithi11939506-36 2024-09-18T11:31:47.845 INFO:tasks.rados.rados.0.smithi119.stdout:1580: finishing write tid 4 to smithi11939506-36 2024-09-18T11:31:47.845 INFO:tasks.rados.rados.0.smithi119.stdout:1573: done (10 left) 2024-09-18T11:31:47.845 INFO:tasks.rados.rados.0.smithi119.stdout:1574: done (9 left) 2024-09-18T11:31:47.845 INFO:tasks.rados.rados.0.smithi119.stdout:1575: done (8 left) 2024-09-18T11:31:47.845 INFO:tasks.rados.rados.0.smithi119.stdout:1576: done (7 left) 2024-09-18T11:31:47.845 INFO:tasks.rados.rados.0.smithi119.stdout:1577: done (6 left) 2024-09-18T11:31:47.845 INFO:tasks.rados.rados.0.smithi119.stdout:1578: done (5 left) 2024-09-18T11:31:47.845 INFO:tasks.rados.rados.0.smithi119.stdout:1579: done (4 left) 2024-09-18T11:31:47.845 INFO:tasks.rados.rados.0.smithi119.stdout:1584: set_chunk oid 37 target oid 2024-09-18T11:31:47.848 INFO:tasks.rados.rados.0.smithi119.stdout:1584: set_chunk oid 37 offset: 309760 length: 215040 target oid 2ec3a6d3c0274e885a97dda06c53eb2e4d02aeaed87ff502d7435a8efc3fe7a9 offset: 0 snap -1 2024-09-18T11:31:47.848 INFO:tasks.rados.rados.0.smithi119.stdout:1585: read oid 7 snap 40 2024-09-18T11:31:47.848 INFO:tasks.rados.rados.0.smithi119.stdout:1585: expect (ObjNum 145 snap 18 seq_num 145) 2024-09-18T11:31:47.848 INFO:tasks.rados.rados.0.smithi119.stdout:1586: write oid 4 current snap is 43 2024-09-18T11:31:47.848 INFO:tasks.rados.rados.0.smithi119.stdout:1580: finishing write tid 5 to smithi11939506-36 2024-09-18T11:31:47.848 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 36 v 186 (ObjNum 308 snap 43 seq_num 308) dirty exists 2024-09-18T11:31:47.848 INFO:tasks.rados.rados.0.smithi119.stdout:1580: left oid 36 (ObjNum 308 snap 43 seq_num 308) 2024-09-18T11:31:47.848 INFO:tasks.rados.rados.0.smithi119.stdout:1581: expect (ObjNum 199 snap 23 seq_num 199) 2024-09-18T11:31:47.910 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:47.909+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:47.911 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:47.911 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:47.920 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:31:47.949 INFO:tasks.rados.rados.0.smithi119.stdout:1582: finishing write tid 1 to smithi11939506-13 2024-09-18T11:31:47.949 INFO:tasks.rados.rados.0.smithi119.stdout:1582: finishing write tid 2 to smithi11939506-13 2024-09-18T11:31:47.949 INFO:tasks.rados.rados.0.smithi119.stdout:1582: finishing write tid 3 to smithi11939506-13 2024-09-18T11:31:47.949 INFO:tasks.rados.rados.0.smithi119.stdout:1582: finishing write tid 4 to smithi11939506-13 2024-09-18T11:31:47.949 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 13 v 361 (ObjNum 309 snap 43 seq_num 309) dirty exists 2024-09-18T11:31:47.949 INFO:tasks.rados.rados.0.smithi119.stdout:1582: left oid 13 (ObjNum 309 snap 43 seq_num 309) 2024-09-18T11:31:47.950 INFO:tasks.rados.rados.0.smithi119.stdout:1583: finishing rollback tid 0 to smithi11939506-49 2024-09-18T11:31:47.950 INFO:tasks.rados.rados.0.smithi119.stdout:1583: finishing rollback tid 1 to smithi11939506-49 2024-09-18T11:31:47.950 INFO:tasks.rados.rados.0.smithi119.stdout:1586: seq_num 310 ranges {410170=781310,1940712=656106,3163363=46807} 2024-09-18T11:31:47.955 INFO:tasks.rados.rados.0.smithi119.stdout:1586: writing smithi11939506-4 from 410170 to 1191480 tid 1 2024-09-18T11:31:47.963 INFO:tasks.rados.rados.0.smithi119.stdout:1586: writing smithi11939506-4 from 1940712 to 2596818 tid 2 2024-09-18T11:31:47.967 INFO:tasks.rados.rados.0.smithi119.stdout:1586: writing smithi11939506-4 from 3163363 to 3210170 tid 3 2024-09-18T11:31:47.967 INFO:tasks.rados.rados.0.smithi119.stdout:1580: done (6 left) 2024-09-18T11:31:47.967 INFO:tasks.rados.rados.0.smithi119.stdout:1581: done (5 left) 2024-09-18T11:31:47.967 INFO:tasks.rados.rados.0.smithi119.stdout:1582: done (4 left) 2024-09-18T11:31:47.967 INFO:tasks.rados.rados.0.smithi119.stdout:1587: read oid 34 snap 36 2024-09-18T11:31:47.967 INFO:tasks.rados.rados.0.smithi119.stdout:1587: expect (ObjNum 264 snap 31 seq_num 264) 2024-09-18T11:31:47.967 INFO:tasks.rados.rados.0.smithi119.stdout:1588: set_chunk oid 10 target oid 2024-09-18T11:31:47.976 INFO:tasks.rados.rados.0.smithi119.stdout:1588: set_chunk oid 10 offset: 1384448 length: 893952 target oid ea50179e2a52b4df6c69ddf07ef89a24b62f1455f87e130946c3a67180e4f811 offset: 0 snap -1 2024-09-18T11:31:47.976 INFO:tasks.rados.rados.0.smithi119.stdout:1589: set_chunk oid 13 target oid 2024-09-18T11:31:47.985 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:31:47.984+0000 7fe41cf66440 -1 osd.0 65 log_to_monitors true 2024-09-18T11:31:47.987 INFO:tasks.rados.rados.0.smithi119.stdout:1589: set_chunk oid 13 offset: 47616 length: 1477120 target oid e3b9350008f8e312916aaf41e2db77ddcaf6e803922b1e7adb2e9b83a0ad5bf1 offset: 0 snap -1 2024-09-18T11:31:47.987 INFO:tasks.rados.rados.0.smithi119.stdout:1590: read oid 3 snap 27 2024-09-18T11:31:47.987 INFO:tasks.rados.rados.0.smithi119.stdout:1590: expect (ObjNum 230 snap 26 seq_num 230) 2024-09-18T11:31:47.988 INFO:tasks.rados.rados.0.smithi119.stdout:1591: set_chunk oid 46 target oid 2024-09-18T11:31:47.994 INFO:tasks.rados.rados.0.smithi119.stdout:1591: set_chunk oid 46 offset: 658944 length: 339456 target oid ff1e7f9a889b848985b6793919d738d410fdf107ac67378c927eda8971ab7ed9 offset: 0 snap -1 2024-09-18T11:31:47.994 INFO:tasks.rados.rados.0.smithi119.stdout:1592: read oid 24 snap 35 2024-09-18T11:31:47.994 INFO:tasks.rados.rados.0.smithi119.stdout:1592: expect (ObjNum 258 snap 31 seq_num 258) 2024-09-18T11:31:47.995 INFO:tasks.rados.rados.0.smithi119.stdout:1593: read oid 44 snap -1 2024-09-18T11:31:47.995 INFO:tasks.rados.rados.0.smithi119.stdout:1593: expect (ObjNum 252 snap 30 seq_num 252) 2024-09-18T11:31:47.995 INFO:tasks.rados.rados.0.smithi119.stdout:1594: write oid 21 current snap is 43 2024-09-18T11:31:47.995 INFO:tasks.rados.rados.0.smithi119.stdout:1583: finishing rollback tid 2 to smithi11939506-49 2024-09-18T11:31:47.995 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 364 (ObjNum 216 snap 26 seq_num 216) dirty exists 2024-09-18T11:31:47.995 INFO:tasks.rados.rados.0.smithi119.stdout:1594: seq_num 311 ranges {426977=456559,1482841=799764,2958080=268897} 2024-09-18T11:31:47.999 INFO:tasks.rados.rados.0.smithi119.stdout:1594: writing smithi11939506-21 from 426977 to 883536 tid 1 2024-09-18T11:31:48.006 INFO:tasks.rados.rados.0.smithi119.stdout:1594: writing smithi11939506-21 from 1482841 to 2282605 tid 2 2024-09-18T11:31:48.011 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:48.010+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:48.012 INFO:tasks.rados.rados.0.smithi119.stdout:1594: writing smithi11939506-21 from 2958080 to 3226977 tid 3 2024-09-18T11:31:48.012 INFO:tasks.rados.rados.0.smithi119.stdout:1583: done (11 left) 2024-09-18T11:31:48.012 INFO:tasks.rados.rados.0.smithi119.stdout:1595: read oid 25 snap -1 2024-09-18T11:31:48.013 INFO:tasks.rados.rados.0.smithi119.stdout:1595: expect (ObjNum 282 snap 39 seq_num 282) 2024-09-18T11:31:48.013 INFO:tasks.rados.rados.0.smithi119.stdout:1596: tier_evict oid 47 2024-09-18T11:31:48.013 INFO:tasks.rados.rados.0.smithi119.stdout:1596: tier_evict oid 47 snap -1 2024-09-18T11:31:48.013 INFO:tasks.rados.rados.0.smithi119.stdout:1597: read oid 27 snap 36 2024-09-18T11:31:48.013 INFO:tasks.rados.rados.0.smithi119.stdout:1597: expect deleted 2024-09-18T11:31:48.013 INFO:tasks.rados.rados.0.smithi119.stdout:1598: read oid 29 snap -1 2024-09-18T11:31:48.013 INFO:tasks.rados.rados.0.smithi119.stdout:1598: expect (ObjNum 210 snap 23 seq_num 210) 2024-09-18T11:31:48.013 INFO:tasks.rados.rados.0.smithi119.stdout:1599: read oid 14 snap -1 2024-09-18T11:31:48.013 INFO:tasks.rados.rados.0.smithi119.stdout:1599: expect (ObjNum 246 snap 28 seq_num 246) 2024-09-18T11:31:48.013 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:31:48.013 INFO:tasks.rados.rados.0.smithi119.stdout:1584: finishing set_chunk to oid 37 2024-09-18T11:31:48.013 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 37 set_chunk returned error code -95 offset: 309760 length: 215040 2024-09-18T11:31:48.013 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 0 (ObjNum 284 snap 39 seq_num 284) dirty exists 2024-09-18T11:31:48.013 INFO:tasks.rados.rados.0.smithi119.stdout:1585: expect (ObjNum 145 snap 18 seq_num 145) 2024-09-18T11:31:48.066 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:48.066 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:48.074 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:31:48.115 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:48.114+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:48.148 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:31:48.147+0000 7fe40be50640 -1 osd.0 65 handle_osd_map injecting map crc failure 2024-09-18T11:31:48.155 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T11:31:48.164 INFO:tasks.rados.rados.0.smithi119.stdout:1586: finishing write tid 1 to smithi11939506-4 2024-09-18T11:31:48.164 INFO:tasks.rados.rados.0.smithi119.stdout:1587: expect (ObjNum 264 snap 31 seq_num 264) 2024-09-18T11:31:48.191 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:48.191 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:48.196 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:31:48.195+0000 7fe4109e8640 -1 osd.0 118 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T11:31:48.201 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:31:48.242 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:48.241+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:48.254 INFO:tasks.rados.rados.0.smithi119.stdout:1586: finishing write tid 2 to smithi11939506-4 2024-09-18T11:31:48.254 INFO:tasks.rados.rados.0.smithi119.stdout:1586: finishing write tid 3 to smithi11939506-4 2024-09-18T11:31:48.254 INFO:tasks.rados.rados.0.smithi119.stdout:1584: done (15 left) 2024-09-18T11:31:48.254 INFO:tasks.rados.rados.0.smithi119.stdout:1585: done (14 left) 2024-09-18T11:31:48.254 INFO:tasks.rados.rados.0.smithi119.stdout:1587: done (13 left) 2024-09-18T11:31:48.254 INFO:tasks.rados.rados.0.smithi119.stdout:1600: delete oid 34 current snap is 43 2024-09-18T11:31:48.254 INFO:tasks.rados.rados.0.smithi119.stdout:1586: finishing write tid 4 to smithi11939506-4 2024-09-18T11:31:48.254 INFO:tasks.rados.rados.0.smithi119.stdout:1586: finishing write tid 5 to smithi11939506-4 2024-09-18T11:31:48.254 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 4 v 96 (ObjNum 310 snap 43 seq_num 310) dirty exists 2024-09-18T11:31:48.254 INFO:tasks.rados.rados.0.smithi119.stdout:1586: left oid 4 (ObjNum 310 snap 43 seq_num 310) 2024-09-18T11:31:48.255 INFO:tasks.rados.rados.0.smithi119.stdout:1588: finishing set_chunk to oid 10 2024-09-18T11:31:48.255 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 10 v 151 (ObjNum 260 snap 31 seq_num 260) dirty exists 2024-09-18T11:31:48.255 INFO:tasks.rados.rados.0.smithi119.stdout:1589: finishing set_chunk to oid 13 2024-09-18T11:31:48.255 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 13 v 361 (ObjNum 309 snap 43 seq_num 309) dirty exists 2024-09-18T11:31:48.255 INFO:tasks.rados.rados.0.smithi119.stdout:1590: expect (ObjNum 230 snap 26 seq_num 230) 2024-09-18T11:31:48.314 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:48.313+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:48.356 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:48.356 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:48.365 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:31:48.449 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:48.448+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:48.503 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:48.503 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:48.511 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:31:48.540 INFO:tasks.rados.rados.0.smithi119.stdout:1592: expect (ObjNum 258 snap 31 seq_num 258) 2024-09-18T11:31:48.540 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:48.539+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:48.617 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:48.616+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:48.637 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:48.638 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:48.646 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:31:48.709 INFO:tasks.rados.rados.0.smithi119.stdout:1593: expect (ObjNum 252 snap 30 seq_num 252) 2024-09-18T11:31:48.718 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:48.717+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:48.785 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:48.785 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:48.793 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:31:48.819 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:48.818+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:48.892 INFO:tasks.rados.rados.0.smithi119.stdout:1591: finishing set_chunk to oid 46 2024-09-18T11:31:48.892 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 231 (ObjNum 171 snap 20 seq_num 171) dirty exists 2024-09-18T11:31:48.892 INFO:tasks.rados.rados.0.smithi119.stdout:1594: finishing write tid 1 to smithi11939506-21 2024-09-18T11:31:48.892 INFO:tasks.rados.rados.0.smithi119.stdout:1594: finishing write tid 2 to smithi11939506-21 2024-09-18T11:31:48.892 INFO:tasks.rados.rados.0.smithi119.stdout:1586: done (13 left) 2024-09-18T11:31:48.892 INFO:tasks.rados.rados.0.smithi119.stdout:1588: done (12 left) 2024-09-18T11:31:48.892 INFO:tasks.rados.rados.0.smithi119.stdout:1589: done (11 left) 2024-09-18T11:31:48.892 INFO:tasks.rados.rados.0.smithi119.stdout:1590: done (10 left) 2024-09-18T11:31:48.893 INFO:tasks.rados.rados.0.smithi119.stdout:1591: done (9 left) 2024-09-18T11:31:48.893 INFO:tasks.rados.rados.0.smithi119.stdout:1592: done (8 left) 2024-09-18T11:31:48.893 INFO:tasks.rados.rados.0.smithi119.stdout:1593: done (7 left) 2024-09-18T11:31:48.893 INFO:tasks.rados.rados.0.smithi119.stdout:1600: done (6 left) 2024-09-18T11:31:48.893 INFO:tasks.rados.rados.0.smithi119.stdout:1601: read oid 22 snap -1 2024-09-18T11:31:48.893 INFO:tasks.rados.rados.0.smithi119.stdout:1601: expect (ObjNum 298 snap 43 seq_num 298) 2024-09-18T11:31:48.893 INFO:tasks.rados.rados.0.smithi119.stdout:1602: read oid 28 snap -1 2024-09-18T11:31:48.893 INFO:tasks.rados.rados.0.smithi119.stdout:1602: expect (ObjNum 200 snap 23 seq_num 200) 2024-09-18T11:31:48.893 INFO:tasks.rados.rados.0.smithi119.stdout:1603: write oid 23 current snap is 43 2024-09-18T11:31:48.893 INFO:tasks.rados.rados.0.smithi119.stdout:1596: got (22) Invalid argument 2024-09-18T11:31:48.893 INFO:tasks.rados.rados.0.smithi119.stdout:1603: seq_num 313 ranges {460591=723410,1716158=403433,2729279=440140} 2024-09-18T11:31:48.898 INFO:tasks.rados.rados.0.smithi119.stdout:1603: writing smithi11939506-23 from 460591 to 1184001 tid 1 2024-09-18T11:31:48.903 INFO:tasks.rados.rados.0.smithi119.stdout:1603: writing smithi11939506-23 from 1716158 to 2119591 tid 2 2024-09-18T11:31:48.909 INFO:tasks.rados.rados.0.smithi119.stdout:1603: writing smithi11939506-23 from 2729279 to 3169419 tid 3 2024-09-18T11:31:48.909 INFO:tasks.rados.rados.0.smithi119.stdout:1596: done (8 left) 2024-09-18T11:31:48.909 INFO:tasks.rados.rados.0.smithi119.stdout:1597: done (7 left) 2024-09-18T11:31:48.909 INFO:tasks.rados.rados.0.smithi119.stdout:1604: write oid 42 current snap is 43 2024-09-18T11:31:48.909 INFO:tasks.rados.rados.0.smithi119.stdout:1598: expect (ObjNum 210 snap 23 seq_num 210) 2024-09-18T11:31:48.920 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:48.919+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:48.933 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:48.933 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:48.941 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:31:48.980 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:31:49.020 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:49.020+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:49.023 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:31:49.042 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:49.042 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:49.052 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:31:49.085 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-18T11:31:49.101 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:49.101 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:49.118 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:49.119 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:49.121 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:49.120+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:49.130 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:49.130 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:49.130 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:49.130 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:49.135 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:49.135 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:49.139 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T11:31:49.139 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-18T11:31:49.139 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.000000 2024-09-18T11:31:49.139 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0 2024-09-18T11:31:49.149 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is set 2024-09-18T11:31:49.157 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:49.158 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:49.186 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:49.186 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:49.196 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:49.196 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:49.196 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:49.197 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:49.197 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:49.205 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:31:49.210 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:49.210 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:49.228 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:49.229 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:49.229 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:49.228+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:49.231 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T11:31:49.232 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T11:31:49.266 INFO:tasks.rados.rados.0.smithi119.stdout:1594: finishing write tid 3 to smithi11939506-21 2024-09-18T11:31:49.266 INFO:tasks.rados.rados.0.smithi119.stdout:1594: finishing write tid 4 to smithi11939506-21 2024-09-18T11:31:49.266 INFO:tasks.rados.rados.0.smithi119.stdout:1594: finishing write tid 5 to smithi11939506-21 2024-09-18T11:31:49.267 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 269 (ObjNum 311 snap 43 seq_num 311) dirty exists 2024-09-18T11:31:49.267 INFO:tasks.rados.rados.0.smithi119.stdout:1594: left oid 21 (ObjNum 311 snap 43 seq_num 311) 2024-09-18T11:31:49.267 INFO:tasks.rados.rados.0.smithi119.stdout:1599: expect (ObjNum 246 snap 28 seq_num 246) 2024-09-18T11:31:49.314 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 98.142266638999999, 2024-09-18T11:31:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:31:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:49.315 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:31:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:31:49.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:31:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:31:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:31:49.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:31:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:31:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:31:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:31:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:31:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:31:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:31:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:31:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:31:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:31:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:49.319 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1001 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 94.147115361000004, 2024-09-18T11:31:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007461082, 2024-09-18T11:31:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1001 2024-09-18T11:31:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:49.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:31:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164570+0000", 2024-09-18T11:31:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:49.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164574+0000", 2024-09-18T11:31:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.822e-06 2024-09-18T11:31:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164582+0000", 2024-09-18T11:31:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3410000000000003e-06 2024-09-18T11:31:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164587+0000", 2024-09-18T11:31:49.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769999999999999e-06 2024-09-18T11:31:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164702+0000", 2024-09-18T11:31:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115705 2024-09-18T11:31:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164709+0000", 2024-09-18T11:31:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-09-18T11:31:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170296+0000", 2024-09-18T11:31:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055876390000000001 2024-09-18T11:31:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170418+0000", 2024-09-18T11:31:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121549 2024-09-18T11:31:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172032+0000", 2024-09-18T11:31:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00161407 2024-09-18T11:31:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.324 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 94.147088644999997, 2024-09-18T11:31:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:31:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:31:49.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:49.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:31:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:31:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:31:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:31:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:31:49.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:31:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:31:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:31:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:31:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:31:49.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:31:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:31:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:31:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:31:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:31:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.329 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:31:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:31:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 94.147070356, 2024-09-18T11:31:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:31:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:49.330 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:31:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:31:49.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:31:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:31:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:31:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:31:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:31:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:31:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:31:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:31:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:31:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:31:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:31:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:31:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:31:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:31:49.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:31:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:31:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:31:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 94.147009586999999, 2024-09-18T11:31:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:31:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:31:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:31:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:31:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:49.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:31:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:31:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:31:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:31:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:31:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:31:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:31:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:31:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:31:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:31:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:31:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:31:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:31:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:31:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:31:49.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:31:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:31:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 94.146987265999996, 2024-09-18T11:31:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:31:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.341 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:31:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:31:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:49.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:31:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:31:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:31:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:31:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:31:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:31:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:31:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:31:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:31:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:31:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:31:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:31:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:31:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:31:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:31:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:31:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:31:49.345 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:31:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 89.365862383000007, 2024-09-18T11:31:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:31:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:31:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:31:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:31:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:31:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:31:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:31:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:49.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:31:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:31:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:31:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:31:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:31:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:31:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:31:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:31:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:31:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:31:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:31:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:31:49.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:31:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:31:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:31:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:31:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:31:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:31:49.351 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1661 4.0 4:019b6e09:::smithi11939506-40:17 [read 0~3445402,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 71.961789679999995, 2024-09-18T11:31:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075190880000000002, 2024-09-18T11:31:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.352 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1661 2024-09-18T11:31:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:31:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349899+0000", 2024-09-18T11:31:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9749999999999999e-06 2024-09-18T11:31:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349906+0000", 2024-09-18T11:31:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9290000000000001e-06 2024-09-18T11:31:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349912+0000", 2024-09-18T11:31:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9900000000000002e-06 2024-09-18T11:31:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355645+0000", 2024-09-18T11:31:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057329549999999996 2024-09-18T11:31:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355776+0000", 2024-09-18T11:31:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130975 2024-09-18T11:31:49.355 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357416+0000", 2024-09-18T11:31:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001639649 2024-09-18T11:31:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1761 4.0 4:0aec5e1c:::smithi11939506-50:head [write 769787~704463,stat] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:49.356 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 67.009297266000004, 2024-09-18T11:31:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074082879999999999, 2024-09-18T11:31:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1761 2024-09-18T11:31:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:31:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302386+0000", 2024-09-18T11:31:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302889+0000", 2024-09-18T11:31:49.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050318499999999996 2024-09-18T11:31:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302930+0000", 2024-09-18T11:31:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0198999999999999e-05 2024-09-18T11:31:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302965+0000", 2024-09-18T11:31:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5292e-05 2024-09-18T11:31:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303046+0000", 2024-09-18T11:31:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1254999999999994e-05 2024-09-18T11:31:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303054+0000", 2024-09-18T11:31:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2800000000000003e-06 2024-09-18T11:31:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304658+0000", 2024-09-18T11:31:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016039520000000001 2024-09-18T11:31:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304724+0000", 2024-09-18T11:31:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5888000000000003e-05 2024-09-18T11:31:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304853+0000", 2024-09-18T11:31:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012912700000000001 2024-09-18T11:31:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309084+0000", 2024-09-18T11:31:49.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042311229999999998 2024-09-18T11:31:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309738+0000", 2024-09-18T11:31:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00065401399999999996 2024-09-18T11:31:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309770+0000", 2024-09-18T11:31:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1652000000000001e-05 2024-09-18T11:31:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:49.362 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309797+0000", 2024-09-18T11:31:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7052000000000001e-05 2024-09-18T11:31:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:306 4.f e69/22 4:f5d7df41:::smithi11939506-16:17 v 69'94, mlcod=65'88)", 2024-09-18T11:31:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 57.960570549000003, 2024-09-18T11:31:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068024720000000004, 2024-09-18T11:31:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:49.363 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:31:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:49.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351115+0000", 2024-09-18T11:31:49.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:49.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:49.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351121+0000", 2024-09-18T11:31:49.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-09-18T11:31:49.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:49.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351126+0000", 2024-09-18T11:31:49.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5969999999999999e-06 2024-09-18T11:31:49.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351147+0000", 2024-09-18T11:31:49.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1160000000000001e-05 2024-09-18T11:31:49.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351518+0000", 2024-09-18T11:31:49.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00037079300000000001 2024-09-18T11:31:49.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:49.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351536+0000", 2024-09-18T11:31:49.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8168999999999998e-05 2024-09-18T11:31:49.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:49.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357870+0000", 2024-09-18T11:31:49.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063332650000000002 2024-09-18T11:31:49.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:49.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357918+0000", 2024-09-18T11:31:49.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8347999999999999e-05 2024-09-18T11:31:49.367 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.367 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:49.367 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.368 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2022 4.0 4:0aec5e1c:::smithi11939506-50:head [write 457190~552096,stat] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-18T11:31:49.368 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:49.368 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.315532412000003, 2024-09-18T11:31:49.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006803974, 2024-09-18T11:31:49.368 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:49.368 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.368 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:49.368 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:49.368 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.368 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2022 2024-09-18T11:31:49.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.368 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:49.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:49.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:49.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:49.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:31:49.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:49.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996151+0000", 2024-09-18T11:31:49.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:49.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:49.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996461+0000", 2024-09-18T11:31:49.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000309587 2024-09-18T11:31:49.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:49.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996481+0000", 2024-09-18T11:31:49.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9949000000000002e-05 2024-09-18T11:31:49.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996489+0000", 2024-09-18T11:31:49.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9129999999999992e-06 2024-09-18T11:31:49.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996562+0000", 2024-09-18T11:31:49.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2807999999999998e-05 2024-09-18T11:31:49.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:49.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996687+0000", 2024-09-18T11:31:49.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012458200000000001 2024-09-18T11:31:49.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:49.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996909+0000", 2024-09-18T11:31:49.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222124 2024-09-18T11:31:49.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:49.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.000741+0000", 2024-09-18T11:31:49.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038320720000000002 2024-09-18T11:31:49.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:49.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002854+0000", 2024-09-18T11:31:49.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002113228 2024-09-18T11:31:49.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:49.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002911+0000", 2024-09-18T11:31:49.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7192999999999999e-05 2024-09-18T11:31:49.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:49.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002958+0000", 2024-09-18T11:31:49.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6242000000000002e-05 2024-09-18T11:31:49.373 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.373 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:49.373 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.373 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2503 3.1 e85/67 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head v 85'117, mlcod=84'115)", 2024-09-18T11:31:49.374 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:31:49.374 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.334463696, 2024-09-18T11:31:49.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086527619999999996, 2024-09-18T11:31:49.374 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:49.374 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.374 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:49.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:49.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:31:49.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:49.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:31:49.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:49.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977220+0000", 2024-09-18T11:31:49.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:49.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:49.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980771+0000", 2024-09-18T11:31:49.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035505200000000002 2024-09-18T11:31:49.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:49.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980797+0000", 2024-09-18T11:31:49.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.618e-05 2024-09-18T11:31:49.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980807+0000", 2024-09-18T11:31:49.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0433e-05 2024-09-18T11:31:49.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980861+0000", 2024-09-18T11:31:49.376 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3415999999999998e-05 2024-09-18T11:31:49.376 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.376 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.376 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:49.376 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980885+0000", 2024-09-18T11:31:49.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4212000000000001e-05 2024-09-18T11:31:49.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:49.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985826+0000", 2024-09-18T11:31:49.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049412450000000004 2024-09-18T11:31:49.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.377 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.377 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:49.377 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985875+0000", 2024-09-18T11:31:49.377 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9116000000000002e-05 2024-09-18T11:31:49.377 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.377 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:49.378 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.378 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2648 4.0 4:059d8d95:::smithi11939506-32:head [read 0~3647119,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:49.378 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:31:49.378 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 34.980375326000001, 2024-09-18T11:31:49.378 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0076286949999999996, 2024-09-18T11:31:49.378 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:49.378 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:49.378 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:49.378 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:49.378 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.378 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2648 2024-09-18T11:31:49.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.378 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:49.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:49.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:31:49.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:49.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:31:49.379 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.379 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.379 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.379 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:49.379 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331309+0000", 2024-09-18T11:31:49.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:49.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:49.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331321+0000", 2024-09-18T11:31:49.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:31:49.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:49.380 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331340+0000", 2024-09-18T11:31:49.380 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8227e-05 2024-09-18T11:31:49.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.380 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331348+0000", 2024-09-18T11:31:49.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4640000000000006e-06 2024-09-18T11:31:49.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331406+0000", 2024-09-18T11:31:49.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7445000000000003e-05 2024-09-18T11:31:49.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.381 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:49.381 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331521+0000", 2024-09-18T11:31:49.381 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115945 2024-09-18T11:31:49.381 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.381 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.382 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:49.382 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338939+0000", 2024-09-18T11:31:49.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074179859999999997 2024-09-18T11:31:49.382 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.382 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:49.382 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.382 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.382 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2649 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:49.382 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:31:49.382 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 34.980320327000001, 2024-09-18T11:31:49.382 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566240000000007, 2024-09-18T11:31:49.382 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:49.383 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:49.383 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:49.383 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:49.383 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.383 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2649 2024-09-18T11:31:49.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.383 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:49.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:49.383 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:31:49.383 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.383 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.383 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:49.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:31:49.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:49.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331365+0000", 2024-09-18T11:31:49.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:49.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:49.384 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331369+0000", 2024-09-18T11:31:49.384 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.551e-06 2024-09-18T11:31:49.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.384 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:49.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331377+0000", 2024-09-18T11:31:49.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8769999999999993e-06 2024-09-18T11:31:49.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331381+0000", 2024-09-18T11:31:49.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.16e-06 2024-09-18T11:31:49.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.385 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.385 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.385 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338973+0000", 2024-09-18T11:31:49.385 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075918460000000002 2024-09-18T11:31:49.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:49.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339048+0000", 2024-09-18T11:31:49.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5813000000000001e-05 2024-09-18T11:31:49.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.386 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.386 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:49.386 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339322+0000", 2024-09-18T11:31:49.386 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027399899999999998 2024-09-18T11:31:49.386 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.386 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:49.386 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.387 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2650 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:49.387 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:49.387 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 34.980300108999998, 2024-09-18T11:31:49.387 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084392180000000001, 2024-09-18T11:31:49.387 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:49.387 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:49.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:49.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:49.387 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.387 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2650 2024-09-18T11:31:49.387 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.387 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:49.387 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:49.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:49.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:49.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:49.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.388 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.388 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:49.388 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:31:49.388 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:49.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:49.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331388+0000", 2024-09-18T11:31:49.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3429999999999998e-06 2024-09-18T11:31:49.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:49.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331394+0000", 2024-09-18T11:31:49.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.19e-06 2024-09-18T11:31:49.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.389 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.389 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331398+0000", 2024-09-18T11:31:49.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.495e-06 2024-09-18T11:31:49.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339362+0000", 2024-09-18T11:31:49.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079641799999999995 2024-09-18T11:31:49.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.390 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:49.390 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339460+0000", 2024-09-18T11:31:49.390 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8329e-05 2024-09-18T11:31:49.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.390 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.391 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:49.391 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339825+0000", 2024-09-18T11:31:49.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036510500000000001 2024-09-18T11:31:49.391 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.391 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:49.391 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.391 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.391 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:119 3.2 3:5595a68d:::07854d2fef297a06ba81685e660c332de36d5d18d546927d30daad6d7fda1541:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:49.391 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:31:49.391 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.268148352000001, 2024-09-18T11:31:49.391 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074145979999999997, 2024-09-18T11:31:49.391 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:49.391 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.391 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:49.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:49.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:31:49.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:49.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:31:49.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.392 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.392 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:49.392 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043535+0000", 2024-09-18T11:31:49.392 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:49.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:49.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043970+0000", 2024-09-18T11:31:49.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000435148 2024-09-18T11:31:49.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:49.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043993+0000", 2024-09-18T11:31:49.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3113999999999999e-05 2024-09-18T11:31:49.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.393 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.393 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.393 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044003+0000", 2024-09-18T11:31:49.393 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5310000000000007e-06 2024-09-18T11:31:49.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044062+0000", 2024-09-18T11:31:49.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9487000000000002e-05 2024-09-18T11:31:49.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:49.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044071+0000", 2024-09-18T11:31:49.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1610000000000003e-06 2024-09-18T11:31:49.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.394 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.394 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.394 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046423+0000", 2024-09-18T11:31:49.394 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023523680000000001 2024-09-18T11:31:49.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:49.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046502+0000", 2024-09-18T11:31:49.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9339999999999996e-05 2024-09-18T11:31:49.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:49.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046664+0000", 2024-09-18T11:31:49.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016185700000000001 2024-09-18T11:31:49.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.395 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:49.395 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050870+0000", 2024-09-18T11:31:49.395 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042060070000000003 2024-09-18T11:31:49.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:49.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050878+0000", 2024-09-18T11:31:49.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8960000000000003e-06 2024-09-18T11:31:49.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:49.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050918+0000", 2024-09-18T11:31:49.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0367000000000002e-05 2024-09-18T11:31:49.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.396 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:49.396 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050952+0000", 2024-09-18T11:31:49.396 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3989999999999998e-05 2024-09-18T11:31:49.397 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.397 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:49.397 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.397 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:124 3.2 3:4326e841:::a6567eb8c0be0365cdfccf2ece66d60995301bc908261a0c1daff039d2080a5f:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:49.397 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044017+0000", 2024-09-18T11:31:49.397 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.267668742000001, 2024-09-18T11:31:49.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070611140000000003, 2024-09-18T11:31:49.397 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:49.397 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.397 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:49.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:49.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044017+0000", 2024-09-18T11:31:49.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:49.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044017+0000", 2024-09-18T11:31:49.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:49.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044016+0000", 2024-09-18T11:31:49.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:49.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:49.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044102+0000", 2024-09-18T11:31:49.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5756999999999999e-05 2024-09-18T11:31:49.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:49.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044114+0000", 2024-09-18T11:31:49.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1505000000000001e-05 2024-09-18T11:31:49.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044120+0000", 2024-09-18T11:31:49.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8549999999999998e-06 2024-09-18T11:31:49.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044154+0000", 2024-09-18T11:31:49.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4773000000000001e-05 2024-09-18T11:31:49.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:49.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044159+0000", 2024-09-18T11:31:49.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5310000000000003e-06 2024-09-18T11:31:49.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046975+0000", 2024-09-18T11:31:49.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028162700000000001 2024-09-18T11:31:49.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:49.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.047116+0000", 2024-09-18T11:31:49.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014086400000000001 2024-09-18T11:31:49.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:49.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.047336+0000", 2024-09-18T11:31:49.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021975299999999999 2024-09-18T11:31:49.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:49.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050964+0000", 2024-09-18T11:31:49.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036283000000000001 2024-09-18T11:31:49.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:49.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.051005+0000", 2024-09-18T11:31:49.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0763999999999998e-05 2024-09-18T11:31:49.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:49.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.051050+0000", 2024-09-18T11:31:49.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4993000000000001e-05 2024-09-18T11:31:49.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:49.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.051078+0000", 2024-09-18T11:31:49.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8694999999999999e-05 2024-09-18T11:31:49.403 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.403 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:49.403 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.403 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:127 3.2 3:4d3b277a:::3266c7aaa53a7d2c10c3c54d852d6cbbed49fb041bc0df41a3ac83b56e2b0615:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:49.403 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:31:49.403 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.267285170000001, 2024-09-18T11:31:49.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093148009999999993, 2024-09-18T11:31:49.404 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:49.404 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.404 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:49.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:49.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:31:49.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:49.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:31:49.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:49.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044400+0000", 2024-09-18T11:31:49.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:49.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:49.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044442+0000", 2024-09-18T11:31:49.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2247999999999999e-05 2024-09-18T11:31:49.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:49.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044453+0000", 2024-09-18T11:31:49.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0617e-05 2024-09-18T11:31:49.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044457+0000", 2024-09-18T11:31:49.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2520000000000001e-06 2024-09-18T11:31:49.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044495+0000", 2024-09-18T11:31:49.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7710000000000003e-05 2024-09-18T11:31:49.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:49.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044499+0000", 2024-09-18T11:31:49.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6269999999999999e-06 2024-09-18T11:31:49.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048239+0000", 2024-09-18T11:31:49.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037396719999999999 2024-09-18T11:31:49.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:49.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048351+0000", 2024-09-18T11:31:49.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112342 2024-09-18T11:31:49.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:49.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048545+0000", 2024-09-18T11:31:49.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000193881 2024-09-18T11:31:49.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:49.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.052689+0000", 2024-09-18T11:31:49.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041435780000000002 2024-09-18T11:31:49.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:49.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053654+0000", 2024-09-18T11:31:49.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096558199999999999 2024-09-18T11:31:49.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:49.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053683+0000", 2024-09-18T11:31:49.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8850999999999999e-05 2024-09-18T11:31:49.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:49.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053716+0000", 2024-09-18T11:31:49.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2468000000000002e-05 2024-09-18T11:31:49.409 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.409 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:49.409 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.409 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2963 4.0 4:019b6e09:::smithi11939506-40:head [sparse-read 0~1093084] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:49.409 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:31:49.410 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 25.284276372000001, 2024-09-18T11:31:49.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072292880000000004, 2024-09-18T11:31:49.410 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:49.410 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:49.410 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:49.410 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:49.410 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.410 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2963 2024-09-18T11:31:49.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.410 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:49.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:49.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:31:49.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:49.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:31:49.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:49.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027409+0000", 2024-09-18T11:31:49.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:49.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:49.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027412+0000", 2024-09-18T11:31:49.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.751e-06 2024-09-18T11:31:49.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:49.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027418+0000", 2024-09-18T11:31:49.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.404e-06 2024-09-18T11:31:49.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027423+0000", 2024-09-18T11:31:49.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7119999999999999e-06 2024-09-18T11:31:49.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034156+0000", 2024-09-18T11:31:49.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067331639999999998 2024-09-18T11:31:49.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:49.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034255+0000", 2024-09-18T11:31:49.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8701000000000002e-05 2024-09-18T11:31:49.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:49.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034639+0000", 2024-09-18T11:31:49.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000384006 2024-09-18T11:31:49.414 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.414 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:49.414 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.414 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3209 4.0 4:019b6e09:::smithi11939506-40:head [write 638732~557432] snapc 42=[42,41,3f,3e,3d,3b,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-18T11:31:49.414 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:31:49.414 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.213603130999999, 2024-09-18T11:31:49.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091774999999999999, 2024-09-18T11:31:49.414 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:49.414 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:49.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:49.415 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.415 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3209 2024-09-18T11:31:49.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.415 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:49.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:49.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:31:49.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:49.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:31:49.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:49.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:49.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098080+0000", 2024-09-18T11:31:49.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:49.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:49.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098676+0000", 2024-09-18T11:31:49.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059598099999999998 2024-09-18T11:31:49.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:49.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098706+0000", 2024-09-18T11:31:49.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0020000000000001e-05 2024-09-18T11:31:49.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098716+0000", 2024-09-18T11:31:49.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0225e-05 2024-09-18T11:31:49.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:49.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098782+0000", 2024-09-18T11:31:49.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5530000000000004e-05 2024-09-18T11:31:49.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:49.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098883+0000", 2024-09-18T11:31:49.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101609 2024-09-18T11:31:49.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:49.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.099098+0000", 2024-09-18T11:31:49.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000215052 2024-09-18T11:31:49.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:49.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.102511+0000", 2024-09-18T11:31:49.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034129429999999999 2024-09-18T11:31:49.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:49.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107165+0000", 2024-09-18T11:31:49.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046531480000000002 2024-09-18T11:31:49.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:49.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107217+0000", 2024-09-18T11:31:49.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2058000000000002e-05 2024-09-18T11:31:49.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:49.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:49.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:49.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107260+0000", 2024-09-18T11:31:49.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3865000000000001e-05 2024-09-18T11:31:49.420 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.420 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:49.420 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.420 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:49.420 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:49.420 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:49.422 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:31:49.422 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:49.420+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:49.442 INFO:tasks.rados.rados.0.smithi119.stdout:1595: expect (ObjNum 282 snap 39 seq_num 282) 2024-09-18T11:31:49.522 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:31:49.521+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:31:49.532 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:49.532 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:49.532 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:49.532 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:49.541 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:31:49.556 INFO:tasks.rados.rados.0.smithi119.stdout:1601: expect (ObjNum 298 snap 43 seq_num 298) 2024-09-18T11:31:49.623 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:49.622+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:49.644 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:49.644 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:49.644 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:49.644 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:49.644 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:49.650 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:31:49.729 INFO:tasks.rados.rados.0.smithi119.stdout:1602: expect (ObjNum 200 snap 23 seq_num 200) 2024-09-18T11:31:49.735 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:49.734+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:49.790 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:49.790 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:49.790 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:49.790 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:49.790 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.790 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:31:49.790 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:49.790 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 75.371177966999994, 2024-09-18T11:31:49.790 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:31:49.790 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:49.791 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.791 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:49.791 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:49.791 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.791 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:31:49.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.791 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:49.791 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.791 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:49.791 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:49.791 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.791 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.791 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:49.792 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:31:49.792 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.792 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:49.792 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:31:49.792 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:49.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.792 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:49.792 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:31:49.792 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:31:49.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.793 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:49.793 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:31:49.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:31:49.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.793 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.793 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:31:49.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:31:49.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.793 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.793 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:31:49.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:31:49.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:49.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:31:49.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:31:49.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:49.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:31:49.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:31:49.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:49.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:31:49.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:31:49.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:49.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:31:49.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:31:49.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:49.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:31:49.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:31:49.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:49.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:31:49.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:31:49.796 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.796 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:49.796 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.796 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1628 3.3 3:f80aa31b:::77cdbf3abbada57d0636b2198046a030673818705cbf13bb77b30c3e68b24345:head [write 0~582144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:49.796 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:49.796 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 72.476287967000005, 2024-09-18T11:31:49.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081714639999999998, 2024-09-18T11:31:49.796 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:49.797 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.797 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:49.797 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:49.797 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.797 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1628 2024-09-18T11:31:49.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.797 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:49.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.797 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:49.797 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:49.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:49.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310977+0000", 2024-09-18T11:31:49.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:49.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.310975+0000", 2024-09-18T11:31:49.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:49.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:49.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312086+0000", 2024-09-18T11:31:49.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001111288 2024-09-18T11:31:49.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:49.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312104+0000", 2024-09-18T11:31:49.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7427000000000001e-05 2024-09-18T11:31:49.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312110+0000", 2024-09-18T11:31:49.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5320000000000003e-06 2024-09-18T11:31:49.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312179+0000", 2024-09-18T11:31:49.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9196000000000006e-05 2024-09-18T11:31:49.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:49.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312375+0000", 2024-09-18T11:31:49.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00019550899999999999 2024-09-18T11:31:49.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:49.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.312558+0000", 2024-09-18T11:31:49.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000182566 2024-09-18T11:31:49.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:49.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.314286+0000", 2024-09-18T11:31:49.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00172831 2024-09-18T11:31:49.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:49.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319079+0000", 2024-09-18T11:31:49.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047929340000000004 2024-09-18T11:31:49.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:49.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319116+0000", 2024-09-18T11:31:49.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6993999999999999e-05 2024-09-18T11:31:49.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:49.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.319148+0000", 2024-09-18T11:31:49.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2539999999999997e-05 2024-09-18T11:31:49.802 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.802 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:49.802 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.802 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1664 4.1 4:87078dfe:::smithi11939506-14:head [setxattr _header (57) in=7b,truncate 2235331] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:49.802 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:49.802 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 72.423027747000006, 2024-09-18T11:31:49.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010368235999999999, 2024-09-18T11:31:49.802 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:49.803 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.803 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:49.803 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:49.803 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.803 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1664 2024-09-18T11:31:49.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.803 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:49.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:49.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:49.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:49.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:31:49.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.804 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:49.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364236+0000", 2024-09-18T11:31:49.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:49.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.804 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:49.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364240+0000", 2024-09-18T11:31:49.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9829999999999998e-06 2024-09-18T11:31:49.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:49.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364247+0000", 2024-09-18T11:31:49.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2140000000000001e-06 2024-09-18T11:31:49.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364256+0000", 2024-09-18T11:31:49.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1509999999999993e-06 2024-09-18T11:31:49.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366529+0000", 2024-09-18T11:31:49.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022731409999999998 2024-09-18T11:31:49.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:49.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366621+0000", 2024-09-18T11:31:49.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2408e-05 2024-09-18T11:31:49.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:49.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366801+0000", 2024-09-18T11:31:49.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018002200000000001 2024-09-18T11:31:49.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:49.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372155+0000", 2024-09-18T11:31:49.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053531519999999999 2024-09-18T11:31:49.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:49.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374539+0000", 2024-09-18T11:31:49.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023842580000000002 2024-09-18T11:31:49.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:49.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374573+0000", 2024-09-18T11:31:49.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4205999999999997e-05 2024-09-18T11:31:49.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:49.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374605+0000", 2024-09-18T11:31:49.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2283999999999998e-05 2024-09-18T11:31:49.808 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.808 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:49.808 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.808 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1665 4.1 4:87078dfe:::smithi11939506-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:49.808 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:49.808 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 72.423003573000003, 2024-09-18T11:31:49.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010471897000000001, 2024-09-18T11:31:49.808 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:49.809 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:49.809 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:49.809 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:49.809 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.809 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1665 2024-09-18T11:31:49.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.809 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:49.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:49.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:49.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:49.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:49.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.810 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:49.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:31:49.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:49.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.810 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:49.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364263+0000", 2024-09-18T11:31:49.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6400000000000001e-06 2024-09-18T11:31:49.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.810 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:49.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364270+0000", 2024-09-18T11:31:49.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:31:49.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364274+0000", 2024-09-18T11:31:49.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.968e-06 2024-09-18T11:31:49.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367354+0000", 2024-09-18T11:31:49.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030803430000000001 2024-09-18T11:31:49.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.812 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:49.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367406+0000", 2024-09-18T11:31:49.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2330999999999998e-05 2024-09-18T11:31:49.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.812 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374625+0000", 2024-09-18T11:31:49.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072190689999999998 2024-09-18T11:31:49.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.812 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:49.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374673+0000", 2024-09-18T11:31:49.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7525999999999999e-05 2024-09-18T11:31:49.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:49.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374733+0000", 2024-09-18T11:31:49.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0359000000000003e-05 2024-09-18T11:31:49.813 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.813 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:49.813 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.813 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1947 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865577,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:49.813 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:49.813 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 58.447723863, 2024-09-18T11:31:49.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010043066999999999, 2024-09-18T11:31:49.814 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:49.814 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:49.814 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:49.814 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:49.814 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.814 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1947 2024-09-18T11:31:49.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.814 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:49.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:49.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:49.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.815 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:49.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:31:49.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.815 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:49.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339540+0000", 2024-09-18T11:31:49.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:49.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.815 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:49.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339545+0000", 2024-09-18T11:31:49.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6290000000000001e-06 2024-09-18T11:31:49.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:49.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339553+0000", 2024-09-18T11:31:49.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6299999999999998e-06 2024-09-18T11:31:49.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339557+0000", 2024-09-18T11:31:49.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3590000000000001e-06 2024-09-18T11:31:49.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345600+0000", 2024-09-18T11:31:49.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060434640000000001 2024-09-18T11:31:49.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:49.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345710+0000", 2024-09-18T11:31:49.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010924700000000001 2024-09-18T11:31:49.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:49.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349584+0000", 2024-09-18T11:31:49.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038743459999999999 2024-09-18T11:31:49.817 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.817 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:49.817 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.818 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1948 4.d 4:b5a463bf:::smithi11939506-20:head [sparse-read 0~2642181] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:49.818 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:49.818 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 58.447703326000003, 2024-09-18T11:31:49.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010701952000000001, 2024-09-18T11:31:49.818 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:49.818 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:49.818 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:49.818 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:49.818 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.818 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1948 2024-09-18T11:31:49.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.819 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:49.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:49.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:49.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:49.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:49.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:49.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:31:49.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:49.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:49.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339563+0000", 2024-09-18T11:31:49.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1950000000000002e-06 2024-09-18T11:31:49.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:49.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339569+0000", 2024-09-18T11:31:49.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5459999999999999e-06 2024-09-18T11:31:49.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339572+0000", 2024-09-18T11:31:49.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.337e-06 2024-09-18T11:31:49.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349693+0000", 2024-09-18T11:31:49.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010120987999999999 2024-09-18T11:31:49.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:49.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349816+0000", 2024-09-18T11:31:49.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012241199999999999 2024-09-18T11:31:49.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.822 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:49.822 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350263+0000", 2024-09-18T11:31:49.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00044787199999999998 2024-09-18T11:31:49.822 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.822 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:49.822 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.822 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:172 4.5 e78/22 4:a087a566:::smithi11939506-46:19 v 78'135, mlcod=77'132)", 2024-09-18T11:31:49.822 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:49.822 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 48.401441312999999, 2024-09-18T11:31:49.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010938658, 2024-09-18T11:31:49.823 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:49.823 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.823 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:49.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:49.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:49.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:49.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:31:49.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:49.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385823+0000", 2024-09-18T11:31:49.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:49.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:49.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385830+0000", 2024-09-18T11:31:49.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2790000000000001e-06 2024-09-18T11:31:49.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:49.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385843+0000", 2024-09-18T11:31:49.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2785e-05 2024-09-18T11:31:49.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385848+0000", 2024-09-18T11:31:49.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5570000000000003e-06 2024-09-18T11:31:49.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385914+0000", 2024-09-18T11:31:49.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5168999999999999e-05 2024-09-18T11:31:49.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:49.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385930+0000", 2024-09-18T11:31:49.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6884999999999998e-05 2024-09-18T11:31:49.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:49.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396721+0000", 2024-09-18T11:31:49.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010790137 2024-09-18T11:31:49.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:49.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396762+0000", 2024-09-18T11:31:49.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1635999999999999e-05 2024-09-18T11:31:49.826 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.826 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:49.826 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.827 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:31:49.827 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:49.827 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 48.400948561, 2024-09-18T11:31:49.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:31:49.827 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:49.827 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.827 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:49.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:49.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:49.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:49.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:49.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:49.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:31:49.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:49.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:49.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:31:49.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:31:49.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:49.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:31:49.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:31:49.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:31:49.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:31:49.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:31:49.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:31:49.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:49.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:31:49.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:31:49.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:49.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:31:49.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:31:49.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:49.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:31:49.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:31:49.831 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.831 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:49.831 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.831 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:49.831 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:49.831 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.814835955, 2024-09-18T11:31:49.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:31:49.831 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:49.831 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.831 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:49.832 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:49.832 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.832 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:31:49.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.832 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:49.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:49.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:49.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:49.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:31:49.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:49.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:31:49.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:49.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:49.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:31:49.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:31:49.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:49.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:31:49.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:31:49.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:31:49.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:31:49.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:31:49.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:31:49.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:49.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:31:49.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:31:49.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:49.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:31:49.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:31:49.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:49.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:31:49.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:31:49.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:49.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:31:49.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:31:49.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:49.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:31:49.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:31:49.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:49.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:31:49.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:31:49.837 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.837 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:49.837 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.837 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2647 4.e 4:78eefc4b:::smithi11939506-7:head [append 0~0] snapc 30=[30,2f,2b,2a,25,24,1c] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:49.837 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:49.837 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.777421275000002, 2024-09-18T11:31:49.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096752739999999993, 2024-09-18T11:31:49.837 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:49.837 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.837 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:49.838 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:49.838 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.838 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2647 2024-09-18T11:31:49.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.838 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:49.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:49.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:49.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:49.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:31:49.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:49.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009843+0000", 2024-09-18T11:31:49.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:49.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:49.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009846+0000", 2024-09-18T11:31:49.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3870000000000002e-06 2024-09-18T11:31:49.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:49.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009853+0000", 2024-09-18T11:31:49.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.238e-06 2024-09-18T11:31:49.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009860+0000", 2024-09-18T11:31:49.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1999999999999997e-06 2024-09-18T11:31:49.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.013798+0000", 2024-09-18T11:31:49.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003938441 2024-09-18T11:31:49.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:49.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.013889+0000", 2024-09-18T11:31:49.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0332000000000002e-05 2024-09-18T11:31:49.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 6", 2024-09-18T11:31:49.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.014066+0000", 2024-09-18T11:31:49.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017728700000000001 2024-09-18T11:31:49.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:49.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015813+0000", 2024-09-18T11:31:49.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001746804 2024-09-18T11:31:49.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:49.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019419+0000", 2024-09-18T11:31:49.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036058399999999999 2024-09-18T11:31:49.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:49.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019478+0000", 2024-09-18T11:31:49.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8860999999999998e-05 2024-09-18T11:31:49.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:49.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019519+0000", 2024-09-18T11:31:49.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1208000000000003e-05 2024-09-18T11:31:49.843 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.843 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:49.843 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.843 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:209 4.5 e91/22 4:af4ba75a:::smithi11939506-37:19 v 91'165, mlcod=91'161)", 2024-09-18T11:31:49.843 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:49.843 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.769344547000003, 2024-09-18T11:31:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0091546930000000002, 2024-09-18T11:31:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:31:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017925+0000", 2024-09-18T11:31:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0819999999999998e-06 2024-09-18T11:31:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017935+0000", 2024-09-18T11:31:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0597999999999999e-05 2024-09-18T11:31:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017941+0000", 2024-09-18T11:31:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2920000000000003e-06 2024-09-18T11:31:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017978+0000", 2024-09-18T11:31:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7667999999999999e-05 2024-09-18T11:31:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018018+0000", 2024-09-18T11:31:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9700999999999998e-05 2024-09-18T11:31:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.027039+0000", 2024-09-18T11:31:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090214079999999999 2024-09-18T11:31:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.027075+0000", 2024-09-18T11:31:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5605000000000002e-05 2024-09-18T11:31:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:570 4.b e91/67 4:ddcb7ff4:::smithi11939506-9:18 v 91'251, mlcod=90'245)", 2024-09-18T11:31:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.769308774999999, 2024-09-18T11:31:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089212549999999995, 2024-09-18T11:31:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017956+0000", 2024-09-18T11:31:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017960+0000", 2024-09-18T11:31:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.777e-06 2024-09-18T11:31:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017966+0000", 2024-09-18T11:31:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4389999999999997e-06 2024-09-18T11:31:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017970+0000", 2024-09-18T11:31:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5549999999999998e-06 2024-09-18T11:31:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018032+0000", 2024-09-18T11:31:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2976000000000001e-05 2024-09-18T11:31:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018053+0000", 2024-09-18T11:31:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0322e-05 2024-09-18T11:31:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026844+0000", 2024-09-18T11:31:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087915790000000008 2024-09-18T11:31:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026877+0000", 2024-09-18T11:31:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2916000000000001e-05 2024-09-18T11:31:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.5.13:15 3.1 3:bf804683:::e58794d0a579007c3bd048992c5646e06b9775f9667464eee6a2a0508e23a0a6:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:31:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.769153928000001, 2024-09-18T11:31:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008816908, 2024-09-18T11:31:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018111+0000", 2024-09-18T11:31:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018110+0000", 2024-09-18T11:31:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018114+0000", 2024-09-18T11:31:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0339999999999998e-06 2024-09-18T11:31:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018121+0000", 2024-09-18T11:31:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0199999999999997e-06 2024-09-18T11:31:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018125+0000", 2024-09-18T11:31:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0409999999999999e-06 2024-09-18T11:31:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018793+0000", 2024-09-18T11:31:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00066779199999999995 2024-09-18T11:31:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018868+0000", 2024-09-18T11:31:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4814999999999994e-05 2024-09-18T11:31:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019008+0000", 2024-09-18T11:31:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014031800000000001 2024-09-18T11:31:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.023494+0000", 2024-09-18T11:31:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0044861720000000001 2024-09-18T11:31:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026796+0000", 2024-09-18T11:31:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0033010969999999998 2024-09-18T11:31:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026891+0000", 2024-09-18T11:31:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5828000000000005e-05 2024-09-18T11:31:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.026928+0000", 2024-09-18T11:31:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6393999999999997e-05 2024-09-18T11:31:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.858 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2793 4.1 4:87078dfe:::smithi11939506-14:head [write 534522~634550] snapc 30=[30,2f,2b,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:31:49.858 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:49.858 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 30.563581299999999, 2024-09-18T11:31:49.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085134289999999994, 2024-09-18T11:31:49.858 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:49.858 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.858 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:49.858 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:49.858 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.859 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2793 2024-09-18T11:31:49.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.859 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:49.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:49.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:49.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:49.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223684+0000", 2024-09-18T11:31:49.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:49.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.223681+0000", 2024-09-18T11:31:49.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:49.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:49.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224251+0000", 2024-09-18T11:31:49.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056940699999999996 2024-09-18T11:31:49.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:49.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224267+0000", 2024-09-18T11:31:49.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6829000000000001e-05 2024-09-18T11:31:49.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224292+0000", 2024-09-18T11:31:49.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4198000000000001e-05 2024-09-18T11:31:49.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224352+0000", 2024-09-18T11:31:49.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0608e-05 2024-09-18T11:31:49.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:49.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224456+0000", 2024-09-18T11:31:49.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010412300000000001 2024-09-18T11:31:49.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:31:49.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224662+0000", 2024-09-18T11:31:49.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020561399999999999 2024-09-18T11:31:49.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:49.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231800+0000", 2024-09-18T11:31:49.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0071384530000000003 2024-09-18T11:31:49.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:49.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232126+0000", 2024-09-18T11:31:49.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00032601400000000001 2024-09-18T11:31:49.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:49.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232158+0000", 2024-09-18T11:31:49.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1472e-05 2024-09-18T11:31:49.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:49.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.232197+0000", 2024-09-18T11:31:49.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9011999999999999e-05 2024-09-18T11:31:49.863 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.863 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:49.864 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.864 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:123 3.1 3:bae7b855:::93307fb660774b1d286e3149e1a75006840a12081178bf31d320df6af68ad704:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:49.864 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:23.045517+0000", 2024-09-18T11:31:49.864 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.741748103999999, 2024-09-18T11:31:49.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081745810000000002, 2024-09-18T11:31:49.864 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:49.864 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.864 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:49.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:49.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.045517+0000", 2024-09-18T11:31:49.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:49.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.045517+0000", 2024-09-18T11:31:49.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:49.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.045516+0000", 2024-09-18T11:31:49.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:49.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:49.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046503+0000", 2024-09-18T11:31:49.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00098663900000000009 2024-09-18T11:31:49.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:49.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046516+0000", 2024-09-18T11:31:49.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2971e-05 2024-09-18T11:31:49.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046522+0000", 2024-09-18T11:31:49.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.618e-06 2024-09-18T11:31:49.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046562+0000", 2024-09-18T11:31:49.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9400999999999998e-05 2024-09-18T11:31:49.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:49.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046639+0000", 2024-09-18T11:31:49.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7037000000000001e-05 2024-09-18T11:31:49.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:49.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.046809+0000", 2024-09-18T11:31:49.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017013799999999999 2024-09-18T11:31:49.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:49.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.049963+0000", 2024-09-18T11:31:49.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031541389999999998 2024-09-18T11:31:49.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:49.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.053637+0000", 2024-09-18T11:31:49.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036736550000000001 2024-09-18T11:31:49.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:49.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.053665+0000", 2024-09-18T11:31:49.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8759999999999999e-05 2024-09-18T11:31:49.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:49.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.053691+0000", 2024-09-18T11:31:49.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5978000000000001e-05 2024-09-18T11:31:49.869 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.869 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:49.869 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.869 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2955 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:49.869 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:31:49.870 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.760073525999999, 2024-09-18T11:31:49.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010869531, 2024-09-18T11:31:49.870 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:49.870 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:49.870 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:49.870 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:49.870 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.870 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2955 2024-09-18T11:31:49.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.870 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:49.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:49.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:31:49.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:49.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:31:49.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:49.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027190+0000", 2024-09-18T11:31:49.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:49.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:49.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027196+0000", 2024-09-18T11:31:49.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5590000000000002e-06 2024-09-18T11:31:49.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:49.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027208+0000", 2024-09-18T11:31:49.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1816e-05 2024-09-18T11:31:49.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027215+0000", 2024-09-18T11:31:49.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4509999999999997e-06 2024-09-18T11:31:49.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027725+0000", 2024-09-18T11:31:49.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000510053 2024-09-18T11:31:49.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:49.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027803+0000", 2024-09-18T11:31:49.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8750000000000003e-05 2024-09-18T11:31:49.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032412+0000", 2024-09-18T11:31:49.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046089030000000001 2024-09-18T11:31:49.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:49.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032499+0000", 2024-09-18T11:31:49.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6316000000000005e-05 2024-09-18T11:31:49.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:49.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038061+0000", 2024-09-18T11:31:49.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005562259 2024-09-18T11:31:49.874 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.874 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:49.874 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.874 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2956 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~2100908,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:49.875 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:49.875 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.760039705000001, 2024-09-18T11:31:49.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011863831, 2024-09-18T11:31:49.875 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:49.875 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:49.875 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:49.875 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:49.875 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.875 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2956 2024-09-18T11:31:49.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.875 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:49.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:49.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:49.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:49.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:49.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:49.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:31:49.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:49.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:49.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027228+0000", 2024-09-18T11:31:49.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:31:49.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:49.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027235+0000", 2024-09-18T11:31:49.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:31:49.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027239+0000", 2024-09-18T11:31:49.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.535e-06 2024-09-18T11:31:49.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027831+0000", 2024-09-18T11:31:49.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059244699999999996 2024-09-18T11:31:49.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:49.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027879+0000", 2024-09-18T11:31:49.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:31:49.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038098+0000", 2024-09-18T11:31:49.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010219137 2024-09-18T11:31:49.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:49.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038207+0000", 2024-09-18T11:31:49.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010951999999999999 2024-09-18T11:31:49.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:49.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039089+0000", 2024-09-18T11:31:49.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00088151699999999996 2024-09-18T11:31:49.879 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.879 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:49.879 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.879 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2957 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:49.879 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:49.879 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.760021921, 2024-09-18T11:31:49.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012041979, 2024-09-18T11:31:49.880 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:49.880 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:49.880 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:49.880 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:49.880 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.880 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2957 2024-09-18T11:31:49.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.880 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:49.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:49.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:49.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:49.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:49.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:49.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:31:49.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:49.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:49.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027245+0000", 2024-09-18T11:31:49.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:31:49.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:49.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027251+0000", 2024-09-18T11:31:49.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0090000000000004e-06 2024-09-18T11:31:49.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027255+0000", 2024-09-18T11:31:49.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:31:49.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027895+0000", 2024-09-18T11:31:49.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064076400000000005 2024-09-18T11:31:49.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:49.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027930+0000", 2024-09-18T11:31:49.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4334e-05 2024-09-18T11:31:49.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039111+0000", 2024-09-18T11:31:49.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181455999999999 2024-09-18T11:31:49.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:49.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039156+0000", 2024-09-18T11:31:49.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5256000000000002e-05 2024-09-18T11:31:49.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:49.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039285+0000", 2024-09-18T11:31:49.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012839200000000001 2024-09-18T11:31:49.884 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.884 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:49.884 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.884 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3220 3.3 3:d97f8959:::efae800be116a8f4318957447289bc2a2c2d33d65c7604341c6a4a86aad75d7e:head [write 0~1337344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-18T11:31:49.885 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:37.110836+0000", 2024-09-18T11:31:49.885 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.676428671, 2024-09-18T11:31:49.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089207350000000008, 2024-09-18T11:31:49.885 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:49.885 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.885 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:49.885 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:49.885 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:49.885 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3220 2024-09-18T11:31:49.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.885 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:49.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:49.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.110836+0000", 2024-09-18T11:31:49.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:49.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.110836+0000", 2024-09-18T11:31:49.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:49.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.110834+0000", 2024-09-18T11:31:49.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:49.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:49.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114557+0000", 2024-09-18T11:31:49.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003723166 2024-09-18T11:31:49.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:49.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114576+0000", 2024-09-18T11:31:49.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.861e-05 2024-09-18T11:31:49.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114585+0000", 2024-09-18T11:31:49.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0780000000000002e-06 2024-09-18T11:31:49.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114630+0000", 2024-09-18T11:31:49.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5118999999999999e-05 2024-09-18T11:31:49.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:49.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114748+0000", 2024-09-18T11:31:49.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011817899999999999 2024-09-18T11:31:49.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 6", 2024-09-18T11:31:49.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.114862+0000", 2024-09-18T11:31:49.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113978 2024-09-18T11:31:49.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:49.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.116796+0000", 2024-09-18T11:31:49.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001934343 2024-09-18T11:31:49.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:49.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.119693+0000", 2024-09-18T11:31:49.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028964820000000001 2024-09-18T11:31:49.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:49.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.119729+0000", 2024-09-18T11:31:49.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6022000000000002e-05 2024-09-18T11:31:49.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:49.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.119757+0000", 2024-09-18T11:31:49.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7880000000000001e-05 2024-09-18T11:31:49.890 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.890 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:49.890 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.890 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3474 4.5 e117/22 4:a56ea1fb:::smithi11939506-48:head v 117'240, mlcod=117'237)", 2024-09-18T11:31:49.890 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.444713+0000", 2024-09-18T11:31:49.890 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.342551566, 2024-09-18T11:31:49.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0081017789999999999, 2024-09-18T11:31:49.891 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:49.891 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:49.891 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:49.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:49.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444713+0000", 2024-09-18T11:31:49.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:49.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444713+0000", 2024-09-18T11:31:49.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:49.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:49.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444712+0000", 2024-09-18T11:31:49.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:49.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:49.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444858+0000", 2024-09-18T11:31:49.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014590299999999999 2024-09-18T11:31:49.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:49.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444867+0000", 2024-09-18T11:31:49.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2480000000000002e-06 2024-09-18T11:31:49.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:49.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444872+0000", 2024-09-18T11:31:49.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2950000000000004e-06 2024-09-18T11:31:49.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:49.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444906+0000", 2024-09-18T11:31:49.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4017999999999999e-05 2024-09-18T11:31:49.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:49.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444920+0000", 2024-09-18T11:31:49.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4321000000000001e-05 2024-09-18T11:31:49.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:49.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452785+0000", 2024-09-18T11:31:49.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0078646500000000008 2024-09-18T11:31:49.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:49.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:49.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:49.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452815+0000", 2024-09-18T11:31:49.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0321e-05 2024-09-18T11:31:49.894 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.894 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:49.894 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.894 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:49.895 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:49.895 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:49.896 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:31:49.897 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:49.895+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:49.900 INFO:tasks.rados.rados.0.smithi119.stdout:1603: finishing write tid 1 to smithi11939506-23 2024-09-18T11:31:49.900 INFO:tasks.rados.rados.0.smithi119.stdout:1603: finishing write tid 2 to smithi11939506-23 2024-09-18T11:31:49.900 INFO:tasks.rados.rados.0.smithi119.stdout:1603: finishing write tid 3 to smithi11939506-23 2024-09-18T11:31:49.900 INFO:tasks.rados.rados.0.smithi119.stdout:1603: finishing write tid 4 to smithi11939506-23 2024-09-18T11:31:49.900 INFO:tasks.rados.rados.0.smithi119.stdout:1603: finishing write tid 5 to smithi11939506-23 2024-09-18T11:31:49.900 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 243 (ObjNum 313 snap 43 seq_num 313) dirty exists 2024-09-18T11:31:49.900 INFO:tasks.rados.rados.0.smithi119.stdout:1603: left oid 23 (ObjNum 313 snap 43 seq_num 313) 2024-09-18T11:31:49.900 INFO:tasks.rados.rados.0.smithi119.stdout:1604: seq_num 314 ranges {477398=798659,1774640=663444,2873055=404343} 2024-09-18T11:31:49.906 INFO:tasks.rados.rados.0.smithi119.stdout:1604: writing smithi11939506-42 from 477398 to 1276057 tid 1 2024-09-18T11:31:49.913 INFO:tasks.rados.rados.0.smithi119.stdout:1604: writing smithi11939506-42 from 1774640 to 2438084 tid 2 2024-09-18T11:31:49.919 INFO:tasks.rados.rados.0.smithi119.stdout:1604: writing smithi11939506-42 from 2873055 to 3277398 tid 3 2024-09-18T11:31:49.919 INFO:tasks.rados.rados.0.smithi119.stdout:1594: done (7 left) 2024-09-18T11:31:49.919 INFO:tasks.rados.rados.0.smithi119.stdout:1595: done (6 left) 2024-09-18T11:31:49.919 INFO:tasks.rados.rados.0.smithi119.stdout:1598: done (5 left) 2024-09-18T11:31:49.919 INFO:tasks.rados.rados.0.smithi119.stdout:1599: done (4 left) 2024-09-18T11:31:49.919 INFO:tasks.rados.rados.0.smithi119.stdout:1601: done (3 left) 2024-09-18T11:31:49.919 INFO:tasks.rados.rados.0.smithi119.stdout:1602: done (2 left) 2024-09-18T11:31:49.919 INFO:tasks.rados.rados.0.smithi119.stdout:1603: done (1 left) 2024-09-18T11:31:49.919 INFO:tasks.rados.rados.0.smithi119.stdout:1605: set_chunk oid 6 target oid 2024-09-18T11:31:49.919 INFO:tasks.rados.rados.0.smithi119.stdout:1605: set_chunk oid 6 offset: 0 length: 4000000 target oid smithi11939506- offset: 0 snap -1 2024-09-18T11:31:49.919 INFO:tasks.rados.rados.0.smithi119.stdout:1606: delete oid 24 current snap is 43 2024-09-18T11:31:49.919 INFO:tasks.rados.rados.0.smithi119.stdout:1604: finishing write tid 1 to smithi11939506-42 2024-09-18T11:31:49.919 INFO:tasks.rados.rados.0.smithi119.stdout:1604: finishing write tid 2 to smithi11939506-42 2024-09-18T11:31:49.920 INFO:tasks.rados.rados.0.smithi119.stdout:1605: finishing set_chunk to oid 6 2024-09-18T11:31:49.920 INFO:tasks.rados.rados.0.smithi119.stdout:1605: got expected ERANGE (src dne) 2024-09-18T11:31:49.921 INFO:tasks.rados.rados.0.smithi119.stdout:1604: finishing write tid 3 to smithi11939506-42 2024-09-18T11:31:49.921 INFO:tasks.rados.rados.0.smithi119.stdout:1605: done (2 left) 2024-09-18T11:31:49.921 INFO:tasks.rados.rados.0.smithi119.stdout:1606: done (1 left) 2024-09-18T11:31:49.922 INFO:tasks.rados.rados.0.smithi119.stdout:1607: read oid 43 snap -1 2024-09-18T11:31:49.922 INFO:tasks.rados.rados.0.smithi119.stdout:1607: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:31:49.922 INFO:tasks.rados.rados.0.smithi119.stdout:1608: write oid 33 current snap is 43 2024-09-18T11:31:49.922 INFO:tasks.rados.rados.0.smithi119.stdout:1608: seq_num 316 ranges {511012=549157,1775251=550760,3095195=215817} 2024-09-18T11:31:49.929 INFO:tasks.rados.rados.0.smithi119.stdout:1608: writing smithi11939506-33 from 511012 to 1060169 tid 1 2024-09-18T11:31:49.935 INFO:tasks.rados.rados.0.smithi119.stdout:1608: writing smithi11939506-33 from 1775251 to 2326011 tid 2 2024-09-18T11:31:49.941 INFO:tasks.rados.rados.0.smithi119.stdout:1608: writing smithi11939506-33 from 3095195 to 3311012 tid 3 2024-09-18T11:31:49.941 INFO:tasks.rados.rados.0.smithi119.stdout:1609: read oid 22 snap 27 2024-09-18T11:31:49.941 INFO:tasks.rados.rados.0.smithi119.stdout:1609: expect (ObjNum 181 snap 20 seq_num 181) 2024-09-18T11:31:49.941 INFO:tasks.rados.rados.0.smithi119.stdout:1610: write (excl) oid 16 current snap is 43 2024-09-18T11:31:49.942 INFO:tasks.rados.rados.0.smithi119.stdout:1610: seq_num 317 ranges {527819=624406,1833733=410771,2722618=601581} 2024-09-18T11:31:49.947 INFO:tasks.rados.rados.0.smithi119.stdout:1610: writing smithi11939506-16 from 527819 to 1152225 tid 1 2024-09-18T11:31:49.953 INFO:tasks.rados.rados.0.smithi119.stdout:1610: writing smithi11939506-16 from 1833733 to 2244504 tid 2 2024-09-18T11:31:49.960 INFO:tasks.rados.rados.0.smithi119.stdout:1610: writing smithi11939506-16 from 2722618 to 3324199 tid 3 2024-09-18T11:31:49.960 INFO:tasks.rados.rados.0.smithi119.stdout:1611: read oid 48 snap -1 2024-09-18T11:31:49.960 INFO:tasks.rados.rados.0.smithi119.stdout:1611: expect (ObjNum 300 snap 43 seq_num 300) 2024-09-18T11:31:49.960 INFO:tasks.rados.rados.0.smithi119.stdout:1612: rollback oid 17 current snap is 43 2024-09-18T11:31:49.960 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 17 to 23 2024-09-18T11:31:49.960 INFO:tasks.rados.rados.0.smithi119.stdout:1607: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:31:49.996 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:49.995+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:50.037 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:50.037 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:50.037 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:50.037 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:50.045 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:31:50.049 INFO:tasks.rados.rados.0.smithi119.stdout:1604: finishing write tid 4 to smithi11939506-42 2024-09-18T11:31:50.049 INFO:tasks.rados.rados.0.smithi119.stdout:1604: finishing write tid 5 to smithi11939506-42 2024-09-18T11:31:50.049 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 223 (ObjNum 314 snap 43 seq_num 314) dirty exists 2024-09-18T11:31:50.049 INFO:tasks.rados.rados.0.smithi119.stdout:1604: left oid 42 (ObjNum 314 snap 43 seq_num 314) 2024-09-18T11:31:50.050 INFO:tasks.rados.rados.0.smithi119.stdout:1608: finishing write tid 1 to smithi11939506-33 2024-09-18T11:31:50.050 INFO:tasks.rados.rados.0.smithi119.stdout:1608: finishing write tid 2 to smithi11939506-33 2024-09-18T11:31:50.050 INFO:tasks.rados.rados.0.smithi119.stdout:1604: done (6 left) 2024-09-18T11:31:50.050 INFO:tasks.rados.rados.0.smithi119.stdout:1607: done (5 left) 2024-09-18T11:31:50.050 INFO:tasks.rados.rados.0.smithi119.stdout:1613: read oid 39 snap 42 2024-09-18T11:31:50.050 INFO:tasks.rados.rados.0.smithi119.stdout:1613: expect (ObjNum 259 snap 31 seq_num 259) 2024-09-18T11:31:50.050 INFO:tasks.rados.rados.0.smithi119.stdout:1614: delete oid 2 current snap is 43 2024-09-18T11:31:50.050 INFO:tasks.rados.rados.0.smithi119.stdout:1608: finishing write tid 3 to smithi11939506-33 2024-09-18T11:31:50.050 INFO:tasks.rados.rados.0.smithi119.stdout:1608: finishing write tid 4 to smithi11939506-33 2024-09-18T11:31:50.050 INFO:tasks.rados.rados.0.smithi119.stdout:1608: finishing write tid 5 to smithi11939506-33 2024-09-18T11:31:50.050 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 33 v 156 (ObjNum 316 snap 43 seq_num 316) dirty exists 2024-09-18T11:31:50.050 INFO:tasks.rados.rados.0.smithi119.stdout:1608: left oid 33 (ObjNum 316 snap 43 seq_num 316) 2024-09-18T11:31:50.050 INFO:tasks.rados.rados.0.smithi119.stdout:1609: expect (ObjNum 181 snap 20 seq_num 181) 2024-09-18T11:31:50.097 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:50.096+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:50.155 INFO:teuthology.orchestra.run.smithi119.stderr:set osd.2 primary-affinity to 0 (802) 2024-09-18T11:31:50.186 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:50.186 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:50.186 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:50.186 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:50.186 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:50.194 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:31:50.238 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:50.237+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:50.263 INFO:tasks.rados.rados.0.smithi119.stdout:1610: finishing write tid 1 to smithi11939506-16 2024-09-18T11:31:50.263 INFO:tasks.rados.rados.0.smithi119.stdout:1610: finishing write tid 2 to smithi11939506-16 2024-09-18T11:31:50.263 INFO:tasks.rados.rados.0.smithi119.stdout:1612: finishing rollback tid 0 to smithi11939506-17 2024-09-18T11:31:50.263 INFO:tasks.rados.rados.0.smithi119.stdout:1610: finishing write tid 3 to smithi11939506-16 2024-09-18T11:31:50.263 INFO:tasks.rados.rados.0.smithi119.stdout:1610: finishing write tid 4 to smithi11939506-16 2024-09-18T11:31:50.263 INFO:tasks.rados.rados.0.smithi119.stdout:1610: finishing write tid 5 to smithi11939506-16 2024-09-18T11:31:50.263 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 275 (ObjNum 317 snap 43 seq_num 317) dirty exists 2024-09-18T11:31:50.263 INFO:tasks.rados.rados.0.smithi119.stdout:1610: left oid 16 (ObjNum 317 snap 43 seq_num 317) 2024-09-18T11:31:50.263 INFO:tasks.rados.rados.0.smithi119.stdout:1612: finishing rollback tid 1 to smithi11939506-17 2024-09-18T11:31:50.264 INFO:tasks.rados.rados.0.smithi119.stdout:1612: finishing rollback tid 2 to smithi11939506-17 2024-09-18T11:31:50.264 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 368 (ObjNum 179 snap 20 seq_num 179) dirty exists 2024-09-18T11:31:50.264 INFO:tasks.rados.rados.0.smithi119.stdout:1611: expect (ObjNum 300 snap 43 seq_num 300) 2024-09-18T11:31:50.293 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:31:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 120.264050458, 2024-09-18T11:31:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073601550000000002, 2024-09-18T11:31:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:31:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:31:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1 2024-09-18T11:31:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026524+0000", 2024-09-18T11:31:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999905 2024-09-18T11:31:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026541+0000", 2024-09-18T11:31:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7070000000000001e-05 2024-09-18T11:31:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026554+0000", 2024-09-18T11:31:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.288e-05 2024-09-18T11:31:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026564+0000", 2024-09-18T11:31:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0197000000000001e-05 2024-09-18T11:31:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026826+0000", 2024-09-18T11:31:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026212400000000003 2024-09-18T11:31:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:31:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026839+0000", 2024-09-18T11:31:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2831e-05 2024-09-18T11:31:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028605+0000", 2024-09-18T11:31:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001765794 2024-09-18T11:31:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028709+0000", 2024-09-18T11:31:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104348 2024-09-18T11:31:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028824+0000", 2024-09-18T11:31:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011527399999999999 2024-09-18T11:31:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030319+0000", 2024-09-18T11:31:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001494439 2024-09-18T11:31:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033805+0000", 2024-09-18T11:31:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034857820000000002 2024-09-18T11:31:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033856+0000", 2024-09-18T11:31:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1687999999999997e-05 2024-09-18T11:31:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033894+0000", 2024-09-18T11:31:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7481000000000002e-05 2024-09-18T11:31:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1118 3.2 e38/21 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head v 38'76, mlcod=38'74)", 2024-09-18T11:31:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 90.342640306000007, 2024-09-18T11:31:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006048688, 2024-09-18T11:31:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947941+0000", 2024-09-18T11:31:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949868+0000", 2024-09-18T11:31:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019267659999999999 2024-09-18T11:31:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949893+0000", 2024-09-18T11:31:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5239999999999999e-05 2024-09-18T11:31:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949901+0000", 2024-09-18T11:31:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8420000000000005e-06 2024-09-18T11:31:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949954+0000", 2024-09-18T11:31:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3464999999999997e-05 2024-09-18T11:31:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949974+0000", 2024-09-18T11:31:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9731000000000001e-05 2024-09-18T11:31:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953948+0000", 2024-09-18T11:31:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039735489999999998 2024-09-18T11:31:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953992+0000", 2024-09-18T11:31:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4700999999999998e-05 2024-09-18T11:31:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1662 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'73, mlcod=51'69)", 2024-09-18T11:31:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 72.929035271000004, 2024-09-18T11:31:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059408539999999998, 2024-09-18T11:31:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361914+0000", 2024-09-18T11:31:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036569 2024-09-18T11:31:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361926+0000", 2024-09-18T11:31:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1960999999999999e-05 2024-09-18T11:31:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361937+0000", 2024-09-18T11:31:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0876999999999999e-05 2024-09-18T11:31:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361986+0000", 2024-09-18T11:31:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8652000000000001e-05 2024-09-18T11:31:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.362012+0000", 2024-09-18T11:31:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5854999999999999e-05 2024-09-18T11:31:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367460+0000", 2024-09-18T11:31:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054482100000000002 2024-09-18T11:31:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367490+0000", 2024-09-18T11:31:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9909000000000001e-05 2024-09-18T11:31:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:269 4.7 e58/22 4:e73ba68f:::smithi11939506-1:13 v 58'72, mlcod=57'69)", 2024-09-18T11:31:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 71.025203374, 2024-09-18T11:31:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065512569999999996, 2024-09-18T11:31:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:50.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265380+0000", 2024-09-18T11:31:50.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:50.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265388+0000", 2024-09-18T11:31:50.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.174e-06 2024-09-18T11:31:50.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.312 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265398+0000", 2024-09-18T11:31:50.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2599999999999994e-06 2024-09-18T11:31:50.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.312 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265402+0000", 2024-09-18T11:31:50.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0289999999999999e-06 2024-09-18T11:31:50.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.313 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265434+0000", 2024-09-18T11:31:50.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2511000000000001e-05 2024-09-18T11:31:50.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.313 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265447+0000", 2024-09-18T11:31:50.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2947e-05 2024-09-18T11:31:50.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.313 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271892+0000", 2024-09-18T11:31:50.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064445930000000002 2024-09-18T11:31:50.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.314 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.314 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271932+0000", 2024-09-18T11:31:50.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0303999999999999e-05 2024-09-18T11:31:50.314 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.314 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.314 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.314 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1788 4.8 4:1898b15d:::smithi11939506-25:head [write 453822~797061] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:50.314 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:50.314 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 67.374770791000003, 2024-09-18T11:31:50.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094305159999999999, 2024-09-18T11:31:50.315 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.315 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.315 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.315 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.315 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.315 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1788 2024-09-18T11:31:50.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.315 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:50.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:50.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915811+0000", 2024-09-18T11:31:50.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:50.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916608+0000", 2024-09-18T11:31:50.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079635000000000003 2024-09-18T11:31:50.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916624+0000", 2024-09-18T11:31:50.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6735000000000002e-05 2024-09-18T11:31:50.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916630+0000", 2024-09-18T11:31:50.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5709999999999997e-06 2024-09-18T11:31:50.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916674+0000", 2024-09-18T11:31:50.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4540999999999999e-05 2024-09-18T11:31:50.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916751+0000", 2024-09-18T11:31:50.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6695999999999999e-05 2024-09-18T11:31:50.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:50.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916872+0000", 2024-09-18T11:31:50.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120648 2024-09-18T11:31:50.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:50.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.919076+0000", 2024-09-18T11:31:50.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022046829999999998 2024-09-18T11:31:50.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:50.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925151+0000", 2024-09-18T11:31:50.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060740239999999999 2024-09-18T11:31:50.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925198+0000", 2024-09-18T11:31:50.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7896999999999999e-05 2024-09-18T11:31:50.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925244+0000", 2024-09-18T11:31:50.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5398999999999997e-05 2024-09-18T11:31:50.320 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.320 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.320 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.320 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1934 4.2 e65/22 4:425afe9c:::smithi11939506-29:head v 65'44, mlcod=60'40)", 2024-09-18T11:31:50.321 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:50.321 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 61.990592020999998, 2024-09-18T11:31:50.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064585119999999996, 2024-09-18T11:31:50.321 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.321 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.321 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:50.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:50.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299990+0000", 2024-09-18T11:31:50.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:50.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300789+0000", 2024-09-18T11:31:50.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079851499999999997 2024-09-18T11:31:50.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300806+0000", 2024-09-18T11:31:50.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7096999999999999e-05 2024-09-18T11:31:50.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300813+0000", 2024-09-18T11:31:50.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5079999999999999e-06 2024-09-18T11:31:50.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300870+0000", 2024-09-18T11:31:50.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7178e-05 2024-09-18T11:31:50.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300897+0000", 2024-09-18T11:31:50.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6284000000000001e-05 2024-09-18T11:31:50.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306410+0000", 2024-09-18T11:31:50.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055135210000000004 2024-09-18T11:31:50.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306451+0000", 2024-09-18T11:31:50.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0379000000000001e-05 2024-09-18T11:31:50.325 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.325 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.325 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.325 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2452 4.3 4:ca2b2309:::smithi11939506-44:head [read 0~3462242,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:31:50.325 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:31:50.325 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 44.253816663999999, 2024-09-18T11:31:50.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060906450000000004, 2024-09-18T11:31:50.326 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.326 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:50.326 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.326 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.326 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.326 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2452 2024-09-18T11:31:50.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.326 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:31:50.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:31:50.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036765+0000", 2024-09-18T11:31:50.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:50.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036777+0000", 2024-09-18T11:31:50.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1422e-05 2024-09-18T11:31:50.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.328 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036793+0000", 2024-09-18T11:31:50.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6575999999999999e-05 2024-09-18T11:31:50.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.328 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036800+0000", 2024-09-18T11:31:50.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9940000000000003e-06 2024-09-18T11:31:50.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:50.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5847999999999999e-05 2024-09-18T11:31:50.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:50.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036853+0000", 2024-09-18T11:31:50.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0119999999999999e-06 2024-09-18T11:31:50.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038527+0000", 2024-09-18T11:31:50.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016732730000000001 2024-09-18T11:31:50.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038580+0000", 2024-09-18T11:31:50.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3443999999999999e-05 2024-09-18T11:31:50.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042858+0000", 2024-09-18T11:31:50.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042781049999999999 2024-09-18T11:31:50.331 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.331 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.331 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.331 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2453 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:31:50.331 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:31:50.331 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 44.253774432, 2024-09-18T11:31:50.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063798429999999996, 2024-09-18T11:31:50.331 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.331 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:50.331 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.332 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.332 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.332 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2453 2024-09-18T11:31:50.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.332 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:31:50.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:31:50.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036809+0000", 2024-09-18T11:31:50.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:50.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036812+0000", 2024-09-18T11:31:50.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8339999999999999e-06 2024-09-18T11:31:50.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036819+0000", 2024-09-18T11:31:50.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9890000000000003e-06 2024-09-18T11:31:50.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036823+0000", 2024-09-18T11:31:50.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8099999999999999e-06 2024-09-18T11:31:50.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036912+0000", 2024-09-18T11:31:50.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9083000000000001e-05 2024-09-18T11:31:50.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:50.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036932+0000", 2024-09-18T11:31:50.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0222000000000001e-05 2024-09-18T11:31:50.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042896+0000", 2024-09-18T11:31:50.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059637240000000001 2024-09-18T11:31:50.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042952+0000", 2024-09-18T11:31:50.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5877000000000001e-05 2024-09-18T11:31:50.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043190+0000", 2024-09-18T11:31:50.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00023774500000000001 2024-09-18T11:31:50.336 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.336 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.336 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.337 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2454 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:31:50.337 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:50.337 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 44.253738476000002, 2024-09-18T11:31:50.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066404469999999998, 2024-09-18T11:31:50.337 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.337 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:50.337 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.337 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.337 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.337 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2454 2024-09-18T11:31:50.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.337 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:50.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:50.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036845+0000", 2024-09-18T11:31:50.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:50.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036856+0000", 2024-09-18T11:31:50.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1338000000000001e-05 2024-09-18T11:31:50.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036863+0000", 2024-09-18T11:31:50.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4749999999999998e-06 2024-09-18T11:31:50.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036867+0000", 2024-09-18T11:31:50.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3839999999999999e-06 2024-09-18T11:31:50.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036946+0000", 2024-09-18T11:31:50.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9054000000000006e-05 2024-09-18T11:31:50.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:50.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036949+0000", 2024-09-18T11:31:50.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9390000000000002e-06 2024-09-18T11:31:50.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043213+0000", 2024-09-18T11:31:50.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062640090000000001 2024-09-18T11:31:50.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043261+0000", 2024-09-18T11:31:50.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7750999999999997e-05 2024-09-18T11:31:50.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043486+0000", 2024-09-18T11:31:50.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000224985 2024-09-18T11:31:50.342 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.342 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.342 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.342 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2553 4.3 4:ca2b2309:::smithi11939506-44:head [write 564768~630013,stat] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-18T11:31:50.342 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:31:50.342 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.093460843000003, 2024-09-18T11:31:50.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065686879999999996, 2024-09-18T11:31:50.342 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.342 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.342 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.342 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.343 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.343 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2553 2024-09-18T11:31:50.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.343 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:31:50.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:31:50.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197120+0000", 2024-09-18T11:31:50.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:50.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199000+0000", 2024-09-18T11:31:50.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018792080000000001 2024-09-18T11:31:50.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199044+0000", 2024-09-18T11:31:50.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4048999999999998e-05 2024-09-18T11:31:50.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199071+0000", 2024-09-18T11:31:50.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7288e-05 2024-09-18T11:31:50.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199119+0000", 2024-09-18T11:31:50.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8000999999999997e-05 2024-09-18T11:31:50.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199208+0000", 2024-09-18T11:31:50.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9209000000000006e-05 2024-09-18T11:31:50.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:50.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199385+0000", 2024-09-18T11:31:50.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000177095 2024-09-18T11:31:50.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:50.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.202282+0000", 2024-09-18T11:31:50.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028970469999999998 2024-09-18T11:31:50.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:50.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203578+0000", 2024-09-18T11:31:50.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012960490000000001 2024-09-18T11:31:50.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203638+0000", 2024-09-18T11:31:50.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9599000000000003e-05 2024-09-18T11:31:50.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203692+0000", 2024-09-18T11:31:50.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3908e-05 2024-09-18T11:31:50.348 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.348 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.348 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.348 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2752 4.3 4:ca2b2309:::smithi11939506-44:25 [read 0~3210104,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:31:50.348 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:31:50.349 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.403691682999998, 2024-09-18T11:31:50.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068715460000000001, 2024-09-18T11:31:50.349 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.349 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:50.349 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.349 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.349 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.349 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2752 2024-09-18T11:31:50.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.349 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:31:50.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:31:50.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:31:50.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:50.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886895+0000", 2024-09-18T11:31:50.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.993e-06 2024-09-18T11:31:50.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886901+0000", 2024-09-18T11:31:50.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1859999999999997e-06 2024-09-18T11:31:50.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886904+0000", 2024-09-18T11:31:50.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1920000000000001e-06 2024-09-18T11:31:50.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.892622+0000", 2024-09-18T11:31:50.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057175660000000003 2024-09-18T11:31:50.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.892693+0000", 2024-09-18T11:31:50.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1265000000000003e-05 2024-09-18T11:31:50.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.893764+0000", 2024-09-18T11:31:50.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001070753 2024-09-18T11:31:50.353 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.353 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.353 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.353 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2793 4.1 e92/22 4:87078dfe:::smithi11939506-14:head v 92'195, mlcod=92'191)", 2024-09-18T11:31:50.353 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:31:50.353 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.065791649000001, 2024-09-18T11:31:50.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062761989999999997, 2024-09-18T11:31:50.353 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.353 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.353 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:31:50.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:31:50.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:31:50.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:50.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224994+0000", 2024-09-18T11:31:50.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020197600000000001 2024-09-18T11:31:50.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225007+0000", 2024-09-18T11:31:50.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3791999999999999e-05 2024-09-18T11:31:50.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225013+0000", 2024-09-18T11:31:50.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3630000000000002e-06 2024-09-18T11:31:50.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225062+0000", 2024-09-18T11:31:50.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8925000000000003e-05 2024-09-18T11:31:50.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225086+0000", 2024-09-18T11:31:50.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4737999999999999e-05 2024-09-18T11:31:50.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231022+0000", 2024-09-18T11:31:50.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059360690000000004 2024-09-18T11:31:50.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231069+0000", 2024-09-18T11:31:50.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6252999999999999e-05 2024-09-18T11:31:50.357 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.357 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.357 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.358 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2876 4.3 4:cc46532a:::smithi11939506-33:2f [read 0~3680733,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:50.358 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:31:50.358 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.783722748999999, 2024-09-18T11:31:50.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068391210000000001, 2024-09-18T11:31:50.358 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.358 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:50.358 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.358 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.358 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.358 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2876 2024-09-18T11:31:50.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.358 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:31:50.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:31:50.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506860+0000", 2024-09-18T11:31:50.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:50.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506869+0000", 2024-09-18T11:31:50.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5539999999999998e-06 2024-09-18T11:31:50.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506880+0000", 2024-09-18T11:31:50.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1552999999999999e-05 2024-09-18T11:31:50.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506887+0000", 2024-09-18T11:31:50.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7329999999999997e-06 2024-09-18T11:31:50.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506937+0000", 2024-09-18T11:31:50.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0317e-05 2024-09-18T11:31:50.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:50.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506945+0000", 2024-09-18T11:31:50.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1550000000000007e-06 2024-09-18T11:31:50.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.509021+0000", 2024-09-18T11:31:50.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020759870000000001 2024-09-18T11:31:50.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.509123+0000", 2024-09-18T11:31:50.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010132900000000001 2024-09-18T11:31:50.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513700+0000", 2024-09-18T11:31:50.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0045777320000000002 2024-09-18T11:31:50.363 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.363 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.363 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.363 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2877 4.3 4:cc46532a:::smithi11939506-33:2f [read 0~3680733,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:50.363 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:31:50.363 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.783688335000001, 2024-09-18T11:31:50.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085715170000000007, 2024-09-18T11:31:50.363 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.363 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:50.364 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.364 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.364 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.364 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2877 2024-09-18T11:31:50.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.364 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:31:50.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:31:50.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:31:50.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:50.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506898+0000", 2024-09-18T11:31:50.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4760000000000001e-06 2024-09-18T11:31:50.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506903+0000", 2024-09-18T11:31:50.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:50.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506906+0000", 2024-09-18T11:31:50.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1769999999999998e-06 2024-09-18T11:31:50.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507007+0000", 2024-09-18T11:31:50.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101479 2024-09-18T11:31:50.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:50.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507015+0000", 2024-09-18T11:31:50.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8490000000000006e-06 2024-09-18T11:31:50.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513757+0000", 2024-09-18T11:31:50.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0067417800000000002 2024-09-18T11:31:50.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513867+0000", 2024-09-18T11:31:50.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000109987 2024-09-18T11:31:50.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515467+0000", 2024-09-18T11:31:50.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00160037 2024-09-18T11:31:50.368 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.368 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.368 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.369 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2878 4.3 4:cc46532a:::smithi11939506-33:2f [sparse-read 0~1770976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:50.369 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:31:50.369 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.78365788, 2024-09-18T11:31:50.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090602640000000002, 2024-09-18T11:31:50.369 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.369 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:50.369 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.369 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.369 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.369 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2878 2024-09-18T11:31:50.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.369 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:31:50.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:31:50.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:31:50.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:50.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506929+0000", 2024-09-18T11:31:50.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.163e-06 2024-09-18T11:31:50.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506936+0000", 2024-09-18T11:31:50.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5519999999999999e-06 2024-09-18T11:31:50.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506941+0000", 2024-09-18T11:31:50.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:50.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507036+0000", 2024-09-18T11:31:50.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5070999999999998e-05 2024-09-18T11:31:50.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:50.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507042+0000", 2024-09-18T11:31:50.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3959999999999997e-06 2024-09-18T11:31:50.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515504+0000", 2024-09-18T11:31:50.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084620300000000006 2024-09-18T11:31:50.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515625+0000", 2024-09-18T11:31:50.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012150999999999999 2024-09-18T11:31:50.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515986+0000", 2024-09-18T11:31:50.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036140100000000002 2024-09-18T11:31:50.374 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.374 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.374 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.374 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:686 4.7 e98/22 4:e5707c18:::smithi11939506-10:30 v 98'160, mlcod=97'155)", 2024-09-18T11:31:50.374 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:31:50.374 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.271816189999999, 2024-09-18T11:31:50.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074641239999999999, 2024-09-18T11:31:50.374 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.374 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.374 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:31:50.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:31:50.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018767+0000", 2024-09-18T11:31:50.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:50.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018773+0000", 2024-09-18T11:31:50.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1460000000000004e-06 2024-09-18T11:31:50.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018781+0000", 2024-09-18T11:31:50.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9780000000000005e-06 2024-09-18T11:31:50.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018785+0000", 2024-09-18T11:31:50.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0029999999999996e-06 2024-09-18T11:31:50.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018826+0000", 2024-09-18T11:31:50.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1616999999999998e-05 2024-09-18T11:31:50.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018843+0000", 2024-09-18T11:31:50.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6753999999999999e-05 2024-09-18T11:31:50.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026202+0000", 2024-09-18T11:31:50.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073592620000000001 2024-09-18T11:31:50.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026232+0000", 2024-09-18T11:31:50.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9779000000000001e-05 2024-09-18T11:31:50.378 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.378 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.378 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.379 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.4.0:621 4.1 e98/22 4:854aa293:::smithi11939506-26:35 v 98'215, mlcod=96'210)", 2024-09-18T11:31:50.379 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:31:50.379 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.270932751, 2024-09-18T11:31:50.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007014752, 2024-09-18T11:31:50.379 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.379 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.379 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:31:50.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:31:50.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:31:50.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:50.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019659+0000", 2024-09-18T11:31:50.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8229999999999995e-06 2024-09-18T11:31:50.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019671+0000", 2024-09-18T11:31:50.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2636000000000001e-05 2024-09-18T11:31:50.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019677+0000", 2024-09-18T11:31:50.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5470000000000004e-06 2024-09-18T11:31:50.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019710+0000", 2024-09-18T11:31:50.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3191999999999997e-05 2024-09-18T11:31:50.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019722+0000", 2024-09-18T11:31:50.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2357e-05 2024-09-18T11:31:50.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026639+0000", 2024-09-18T11:31:50.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0069167630000000003 2024-09-18T11:31:50.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026666+0000", 2024-09-18T11:31:50.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7087999999999999e-05 2024-09-18T11:31:50.383 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.383 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.383 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.383 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3234 4.7 e109/22 4:e73ba68f:::smithi11939506-1:head v 109'206, mlcod=107'202)", 2024-09-18T11:31:50.383 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:37.128363+0000", 2024-09-18T11:31:50.383 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.162221367000001, 2024-09-18T11:31:50.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059367100000000004, 2024-09-18T11:31:50.384 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.384 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.384 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.128363+0000", 2024-09-18T11:31:50.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.128363+0000", 2024-09-18T11:31:50.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.128362+0000", 2024-09-18T11:31:50.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:50.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131291+0000", 2024-09-18T11:31:50.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029282840000000002 2024-09-18T11:31:50.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131305+0000", 2024-09-18T11:31:50.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4222999999999999e-05 2024-09-18T11:31:50.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131313+0000", 2024-09-18T11:31:50.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.208e-06 2024-09-18T11:31:50.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131354+0000", 2024-09-18T11:31:50.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1158000000000001e-05 2024-09-18T11:31:50.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131378+0000", 2024-09-18T11:31:50.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3828999999999998e-05 2024-09-18T11:31:50.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.134258+0000", 2024-09-18T11:31:50.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028798920000000002 2024-09-18T11:31:50.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.134299+0000", 2024-09-18T11:31:50.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1355e-05 2024-09-18T11:31:50.388 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.388 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.388 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.388 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3489 4.3 4:cc46532a:::smithi11939506-33:head [setxattr _header (57) in=7b,truncate 3092521] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:31:50.388 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.709941+0000", 2024-09-18T11:31:50.388 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.5806434029999998, 2024-09-18T11:31:50.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057848090000000001, 2024-09-18T11:31:50.388 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.389 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.389 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.389 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.389 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.389 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3489 2024-09-18T11:31:50.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.389 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709941+0000", 2024-09-18T11:31:50.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709941+0000", 2024-09-18T11:31:50.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709940+0000", 2024-09-18T11:31:50.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:50.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709944+0000", 2024-09-18T11:31:50.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0249999999999996e-06 2024-09-18T11:31:50.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709950+0000", 2024-09-18T11:31:50.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4690000000000002e-06 2024-09-18T11:31:50.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709954+0000", 2024-09-18T11:31:50.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:31:50.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.711862+0000", 2024-09-18T11:31:50.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001907607 2024-09-18T11:31:50.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.711944+0000", 2024-09-18T11:31:50.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2182999999999996e-05 2024-09-18T11:31:50.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:50.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.712056+0000", 2024-09-18T11:31:50.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011131100000000001 2024-09-18T11:31:50.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:50.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.715101+0000", 2024-09-18T11:31:50.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030452460000000002 2024-09-18T11:31:50.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:50.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.715672+0000", 2024-09-18T11:31:50.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000570917 2024-09-18T11:31:50.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.715699+0000", 2024-09-18T11:31:50.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7069000000000001e-05 2024-09-18T11:31:50.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.715726+0000", 2024-09-18T11:31:50.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6718000000000001e-05 2024-09-18T11:31:50.394 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.394 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.394 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.394 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3490 4.3 4:cc46532a:::smithi11939506-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:31:50.395 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.709960+0000", 2024-09-18T11:31:50.395 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.5806243530000001, 2024-09-18T11:31:50.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058965190000000002, 2024-09-18T11:31:50.395 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.395 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:50.395 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.395 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.395 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.395 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3490 2024-09-18T11:31:50.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.395 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709960+0000", 2024-09-18T11:31:50.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709960+0000", 2024-09-18T11:31:50.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709959+0000", 2024-09-18T11:31:50.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:50.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709964+0000", 2024-09-18T11:31:50.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0900000000000004e-06 2024-09-18T11:31:50.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709971+0000", 2024-09-18T11:31:50.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5950000000000004e-06 2024-09-18T11:31:50.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709975+0000", 2024-09-18T11:31:50.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0690000000000003e-06 2024-09-18T11:31:50.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.712315+0000", 2024-09-18T11:31:50.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002339556 2024-09-18T11:31:50.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:50.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.712392+0000", 2024-09-18T11:31:50.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7868999999999996e-05 2024-09-18T11:31:50.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.715759+0000", 2024-09-18T11:31:50.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0033669989999999999 2024-09-18T11:31:50.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.715802+0000", 2024-09-18T11:31:50.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2636000000000003e-05 2024-09-18T11:31:50.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.715856+0000", 2024-09-18T11:31:50.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4153000000000001e-05 2024-09-18T11:31:50.399 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.400 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.400 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.400 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.400 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.400 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:50.401 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:31:50.402 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:50.400+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:50.458 INFO:tasks.rados.rados.0.smithi119.stdout:1613: expect (ObjNum 259 snap 31 seq_num 259) 2024-09-18T11:31:50.502 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:50.501+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:50.539 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:50.539 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:50.539 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:50.539 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:50.547 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:31:50.586 INFO:tasks.rados.rados.0.smithi119.stdout:1608: done (6 left) 2024-09-18T11:31:50.586 INFO:tasks.rados.rados.0.smithi119.stdout:1609: done (5 left) 2024-09-18T11:31:50.586 INFO:tasks.rados.rados.0.smithi119.stdout:1610: done (4 left) 2024-09-18T11:31:50.586 INFO:tasks.rados.rados.0.smithi119.stdout:1611: done (3 left) 2024-09-18T11:31:50.586 INFO:tasks.rados.rados.0.smithi119.stdout:1612: done (2 left) 2024-09-18T11:31:50.586 INFO:tasks.rados.rados.0.smithi119.stdout:1613: done (1 left) 2024-09-18T11:31:50.586 INFO:tasks.rados.rados.0.smithi119.stdout:1614: done (0 left) 2024-09-18T11:31:50.586 INFO:tasks.rados.rados.0.smithi119.stdout:1615: read oid 40 snap -1 2024-09-18T11:31:50.586 INFO:tasks.rados.rados.0.smithi119.stdout:1615: expect (ObjNum 287 snap 41 seq_num 287) 2024-09-18T11:31:50.586 INFO:tasks.rados.rados.0.smithi119.stdout:1616: set_chunk oid 42 target oid 2024-09-18T11:31:50.602 INFO:tasks.rados.rados.0.smithi119.stdout:1616: set_chunk oid 42 offset: 431104 length: 2180608 target oid 3fee8fd0e55960eee7af3d438f8c3184fb777d5a47530bbd9d979d65260961aa offset: 0 snap -1 2024-09-18T11:31:50.602 INFO:tasks.rados.rados.0.smithi119.stdout:1617: delete oid 5 current snap is 43 2024-09-18T11:31:50.602 INFO:tasks.rados.rados.0.smithi119.stdout:1615: expect (ObjNum 287 snap 41 seq_num 287) 2024-09-18T11:31:50.602 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:50.601+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:50.646 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:50.646 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:50.646 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:50.646 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:50.646 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:50.654 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:31:50.703 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:50.702+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:50.753 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:50.753 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:50.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:50.754 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:50.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.754 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:50.754 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:50.754 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 99.634667106999999, 2024-09-18T11:31:50.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:31:50.754 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.755 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:50.755 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.755 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.755 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.755 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:31:50.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.755 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:50.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:50.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:31:50.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:50.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:31:50.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:31:50.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:31:50.757 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:31:50.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:31:50.757 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:31:50.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:31:50.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:31:50.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.758 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:50.758 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:31:50.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:31:50.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:31:50.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:31:50.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:31:50.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:31:50.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.760 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:31:50.760 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:31:50.760 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.760 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.760 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.760 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.760 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:31:50.760 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:50.760 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 90.081641705999999, 2024-09-18T11:31:50.760 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:31:50.760 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.760 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:50.761 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.761 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.761 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.761 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:31:50.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.761 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.761 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:50.761 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:50.762 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.763 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.763 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:31:50.763 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:50.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:31:50.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:31:50.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.765 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:31:50.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:31:50.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.765 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.766 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:31:50.766 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:31:50.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.766 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.766 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:31:50.767 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:31:50.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.767 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.767 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.767 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:31:50.767 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:31:50.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.768 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.768 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:31:50.768 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:31:50.768 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.768 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.768 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.768 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1969 4.a 4:588debc2:::smithi11939506-18:18 [read 0~3949612,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:50.768 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:50.769 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 59.376999869000002, 2024-09-18T11:31:50.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075409190000000001, 2024-09-18T11:31:50.769 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.769 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:50.769 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.770 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.770 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.770 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1969 2024-09-18T11:31:50.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.770 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.770 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.770 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:50.770 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:50.771 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:50.771 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:50.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.772 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.772 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373990+0000", 2024-09-18T11:31:50.772 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1119999999999998e-06 2024-09-18T11:31:50.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.772 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.772 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373997+0000", 2024-09-18T11:31:50.772 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4200000000000001e-06 2024-09-18T11:31:50.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.773 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.773 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.374002+0000", 2024-09-18T11:31:50.773 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:31:50.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.773 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.773 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380200+0000", 2024-09-18T11:31:50.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006197302 2024-09-18T11:31:50.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.774 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380261+0000", 2024-09-18T11:31:50.775 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1389999999999993e-05 2024-09-18T11:31:50.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.775 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.775 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.381527+0000", 2024-09-18T11:31:50.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012660080000000001 2024-09-18T11:31:50.776 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.776 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.776 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.776 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2207 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1410296~761389] snapc 24=[24,1e,1c,1b,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T11:31:50.776 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:50.776 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.328489042999998, 2024-09-18T11:31:50.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008410743, 2024-09-18T11:31:50.776 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.777 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.777 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.777 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.777 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.777 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2207 2024-09-18T11:31:50.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.777 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.777 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:50.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.778 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.778 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:50.778 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.778 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.778 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422496+0000", 2024-09-18T11:31:50.778 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:50.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.778 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.778 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423190+0000", 2024-09-18T11:31:50.778 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00069405499999999995 2024-09-18T11:31:50.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.779 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.779 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.779 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423204+0000", 2024-09-18T11:31:50.779 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:31:50.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.779 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423208+0000", 2024-09-18T11:31:50.780 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:31:50.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423293+0000", 2024-09-18T11:31:50.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4524999999999995e-05 2024-09-18T11:31:50.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.781 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423394+0000", 2024-09-18T11:31:50.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101395 2024-09-18T11:31:50.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.781 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:50.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423579+0000", 2024-09-18T11:31:50.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018461000000000001 2024-09-18T11:31:50.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:50.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.428574+0000", 2024-09-18T11:31:50.783 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049944960000000002 2024-09-18T11:31:50.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.783 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:50.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430809+0000", 2024-09-18T11:31:50.783 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022351820000000001 2024-09-18T11:31:50.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.783 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430862+0000", 2024-09-18T11:31:50.783 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3189999999999997e-05 2024-09-18T11:31:50.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.784 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.784 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430908+0000", 2024-09-18T11:31:50.784 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5973999999999997e-05 2024-09-18T11:31:50.784 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.784 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.784 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.784 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2339 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:50.784 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:50.784 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 49.364184414, 2024-09-18T11:31:50.784 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011809992, 2024-09-18T11:31:50.785 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.785 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:50.785 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.785 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.785 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.785 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2339 2024-09-18T11:31:50.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.785 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.785 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.785 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:50.785 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:50.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:31:50.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:50.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:31:50.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4710000000000001e-06 2024-09-18T11:31:50.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386824+0000", 2024-09-18T11:31:50.788 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6104e-05 2024-09-18T11:31:50.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.788 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.788 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386836+0000", 2024-09-18T11:31:50.788 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1827999999999999e-05 2024-09-18T11:31:50.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.788 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.788 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388390+0000", 2024-09-18T11:31:50.789 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015539499999999999 2024-09-18T11:31:50.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.789 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:50.789 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388443+0000", 2024-09-18T11:31:50.789 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3785000000000001e-05 2024-09-18T11:31:50.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.790 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.790 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398238+0000", 2024-09-18T11:31:50.790 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097947989999999999 2024-09-18T11:31:50.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.790 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.791 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.791 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398315+0000", 2024-09-18T11:31:50.791 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6909999999999994e-05 2024-09-18T11:31:50.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.791 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.791 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.791 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398612+0000", 2024-09-18T11:31:50.792 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029671999999999999 2024-09-18T11:31:50.792 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.792 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.792 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.792 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2340 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:50.792 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:50.792 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 49.364141101999998, 2024-09-18T11:31:50.792 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012086448, 2024-09-18T11:31:50.792 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.792 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:50.793 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.793 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.793 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.793 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2340 2024-09-18T11:31:50.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.793 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.793 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.793 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:50.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:50.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:50.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:50.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386848+0000", 2024-09-18T11:31:50.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3069999999999998e-06 2024-09-18T11:31:50.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386855+0000", 2024-09-18T11:31:50.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0979999999999996e-06 2024-09-18T11:31:50.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386859+0000", 2024-09-18T11:31:50.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9360000000000001e-06 2024-09-18T11:31:50.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388466+0000", 2024-09-18T11:31:50.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001607154 2024-09-18T11:31:50.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:50.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388507+0000", 2024-09-18T11:31:50.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1093000000000001e-05 2024-09-18T11:31:50.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.797 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.797 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398647+0000", 2024-09-18T11:31:50.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010139686 2024-09-18T11:31:50.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.797 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.797 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398718+0000", 2024-09-18T11:31:50.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1138999999999998e-05 2024-09-18T11:31:50.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.797 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398931+0000", 2024-09-18T11:31:50.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021356899999999999 2024-09-18T11:31:50.798 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.798 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.798 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.798 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2341 4.5 4:aa575018:::smithi11939506-11:head [read 0~2487436,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:50.798 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:50.798 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 49.364122879999996, 2024-09-18T11:31:50.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013751648999999999, 2024-09-18T11:31:50.798 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.798 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:50.798 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.799 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.799 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.799 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2341 2024-09-18T11:31:50.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.799 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:50.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:50.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:50.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:50.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386866+0000", 2024-09-18T11:31:50.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4249999999999998e-06 2024-09-18T11:31:50.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386872+0000", 2024-09-18T11:31:50.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.003e-06 2024-09-18T11:31:50.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:31:50.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.507e-06 2024-09-18T11:31:50.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388520+0000", 2024-09-18T11:31:50.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001644146 2024-09-18T11:31:50.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:50.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388561+0000", 2024-09-18T11:31:50.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1486999999999998e-05 2024-09-18T11:31:50.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398963+0000", 2024-09-18T11:31:50.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010401353 2024-09-18T11:31:50.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.399046+0000", 2024-09-18T11:31:50.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3141999999999998e-05 2024-09-18T11:31:50.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.804 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.400615+0000", 2024-09-18T11:31:50.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015690960000000001 2024-09-18T11:31:50.804 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.804 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.804 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.804 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2529 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:50.804 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:31:50.804 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 41.577649616000002, 2024-09-18T11:31:50.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079268120000000001, 2024-09-18T11:31:50.804 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.805 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.805 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.805 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.805 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.805 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2529 2024-09-18T11:31:50.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.805 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:31:50.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:31:50.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173336+0000", 2024-09-18T11:31:50.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:50.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173339+0000", 2024-09-18T11:31:50.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9019999999999999e-06 2024-09-18T11:31:50.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173381+0000", 2024-09-18T11:31:50.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1836999999999999e-05 2024-09-18T11:31:50.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173390+0000", 2024-09-18T11:31:50.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.302e-06 2024-09-18T11:31:50.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174507+0000", 2024-09-18T11:31:50.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001116868 2024-09-18T11:31:50.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174603+0000", 2024-09-18T11:31:50.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5598000000000002e-05 2024-09-18T11:31:50.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:50.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174759+0000", 2024-09-18T11:31:50.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015616299999999999 2024-09-18T11:31:50.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:50.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178943+0000", 2024-09-18T11:31:50.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041846180000000002 2024-09-18T11:31:50.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:50.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181175+0000", 2024-09-18T11:31:50.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00223215 2024-09-18T11:31:50.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181229+0000", 2024-09-18T11:31:50.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3585999999999998e-05 2024-09-18T11:31:50.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.810 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181263+0000", 2024-09-18T11:31:50.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4292999999999998e-05 2024-09-18T11:31:50.810 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.810 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.810 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.810 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2530 4.5 4:af4ba75a:::smithi11939506-37:head [rollback 25] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:50.811 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:31:50.811 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 41.577587309999998, 2024-09-18T11:31:50.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080198800000000001, 2024-09-18T11:31:50.811 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.811 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.811 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.811 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.812 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.812 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2530 2024-09-18T11:31:50.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.812 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.812 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:31:50.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.812 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:31:50.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173398+0000", 2024-09-18T11:31:50.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:50.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173403+0000", 2024-09-18T11:31:50.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4389999999999999e-06 2024-09-18T11:31:50.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.815 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173414+0000", 2024-09-18T11:31:50.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0411000000000001e-05 2024-09-18T11:31:50.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.815 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173420+0000", 2024-09-18T11:31:50.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9610000000000002e-06 2024-09-18T11:31:50.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176390+0000", 2024-09-18T11:31:50.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029706289999999998 2024-09-18T11:31:50.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176501+0000", 2024-09-18T11:31:50.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011090300000000001 2024-09-18T11:31:50.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:50.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176827+0000", 2024-09-18T11:31:50.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00032526899999999999 2024-09-18T11:31:50.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:50.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178958+0000", 2024-09-18T11:31:50.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021314860000000001 2024-09-18T11:31:50.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:50.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181318+0000", 2024-09-18T11:31:50.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002359571 2024-09-18T11:31:50.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181372+0000", 2024-09-18T11:31:50.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3884000000000001e-05 2024-09-18T11:31:50.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181419+0000", 2024-09-18T11:31:50.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7148000000000003e-05 2024-09-18T11:31:50.819 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.819 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.819 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.820 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2531 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:50.820 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:50.820 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 41.577561187000001, 2024-09-18T11:31:50.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080687239999999993, 2024-09-18T11:31:50.821 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.821 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.821 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.821 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.821 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.821 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2531 2024-09-18T11:31:50.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.821 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:50.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.822 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.822 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:50.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.822 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:50.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:50.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173429+0000", 2024-09-18T11:31:50.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2989999999999998e-06 2024-09-18T11:31:50.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173436+0000", 2024-09-18T11:31:50.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1799999999999999e-06 2024-09-18T11:31:50.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173441+0000", 2024-09-18T11:31:50.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8810000000000004e-06 2024-09-18T11:31:50.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178267+0000", 2024-09-18T11:31:50.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048257969999999997 2024-09-18T11:31:50.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178388+0000", 2024-09-18T11:31:50.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121779 2024-09-18T11:31:50.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:50.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178599+0000", 2024-09-18T11:31:50.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000210094 2024-09-18T11:31:50.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:50.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.179339+0000", 2024-09-18T11:31:50.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00074063799999999995 2024-09-18T11:31:50.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:50.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181444+0000", 2024-09-18T11:31:50.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021044100000000001 2024-09-18T11:31:50.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181470+0000", 2024-09-18T11:31:50.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6415999999999999e-05 2024-09-18T11:31:50.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181494+0000", 2024-09-18T11:31:50.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3672e-05 2024-09-18T11:31:50.827 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.827 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.827 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.827 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2910 4.5 4:af4ba75a:::smithi11939506-37:head [write 786627~795991,stat] snapc 37=[37,35,34,30,2f,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:50.827 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:31:50.827 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.747503357999999, 2024-09-18T11:31:50.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084093710000000006, 2024-09-18T11:31:50.827 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.827 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.827 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.827 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.827 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.828 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2910 2024-09-18T11:31:50.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.828 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:31:50.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:31:50.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003481+0000", 2024-09-18T11:31:50.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:50.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004300+0000", 2024-09-18T11:31:50.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081934200000000003 2024-09-18T11:31:50.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004324+0000", 2024-09-18T11:31:50.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4618999999999999e-05 2024-09-18T11:31:50.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004335+0000", 2024-09-18T11:31:50.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.092e-05 2024-09-18T11:31:50.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004385+0000", 2024-09-18T11:31:50.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9709999999999997e-05 2024-09-18T11:31:50.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004470+0000", 2024-09-18T11:31:50.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4864000000000006e-05 2024-09-18T11:31:50.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:50.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004612+0000", 2024-09-18T11:31:50.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014241500000000001 2024-09-18T11:31:50.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:50.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011193+0000", 2024-09-18T11:31:50.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006580654 2024-09-18T11:31:50.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:50.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011788+0000", 2024-09-18T11:31:50.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059533600000000002 2024-09-18T11:31:50.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011842+0000", 2024-09-18T11:31:50.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3180000000000002e-05 2024-09-18T11:31:50.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011892+0000", 2024-09-18T11:31:50.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0618000000000002e-05 2024-09-18T11:31:50.833 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.833 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.834 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.834 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2953 4.a 4:588debc2:::smithi11939506-18:head [assert-version v113,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:50.834 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:31:50.834 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.724187839999999, 2024-09-18T11:31:50.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083931279999999997, 2024-09-18T11:31:50.834 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.834 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.834 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.834 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.834 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.834 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2953 2024-09-18T11:31:50.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.835 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:31:50.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:31:50.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026796+0000", 2024-09-18T11:31:50.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:50.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026806+0000", 2024-09-18T11:31:50.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9529999999999994e-06 2024-09-18T11:31:50.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026824+0000", 2024-09-18T11:31:50.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7901999999999999e-05 2024-09-18T11:31:50.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026832+0000", 2024-09-18T11:31:50.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8450000000000003e-06 2024-09-18T11:31:50.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026883+0000", 2024-09-18T11:31:50.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0883000000000001e-05 2024-09-18T11:31:50.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026980+0000", 2024-09-18T11:31:50.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7526000000000001e-05 2024-09-18T11:31:50.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:50.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032722+0000", 2024-09-18T11:31:50.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057414689999999999 2024-09-18T11:31:50.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:50.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.034230+0000", 2024-09-18T11:31:50.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001508035 2024-09-18T11:31:50.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:50.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035090+0000", 2024-09-18T11:31:50.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00086047300000000005 2024-09-18T11:31:50.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035147+0000", 2024-09-18T11:31:50.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6598000000000003e-05 2024-09-18T11:31:50.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035191+0000", 2024-09-18T11:31:50.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4747999999999999e-05 2024-09-18T11:31:50.840 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.840 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.840 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.840 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3105 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 3e=[3e,3d,3b,37,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-18T11:31:50.840 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:31:50.841 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.732775136000001, 2024-09-18T11:31:50.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011735906000000001, 2024-09-18T11:31:50.841 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.841 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:50.841 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.841 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.841 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.841 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3105 2024-09-18T11:31:50.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.841 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:31:50.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:31:50.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:31:50.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:50.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018212+0000", 2024-09-18T11:31:50.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6759999999999999e-06 2024-09-18T11:31:50.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018216+0000", 2024-09-18T11:31:50.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-06 2024-09-18T11:31:50.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018218+0000", 2024-09-18T11:31:50.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6110000000000001e-06 2024-09-18T11:31:50.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018246+0000", 2024-09-18T11:31:50.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7557000000000001e-05 2024-09-18T11:31:50.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:50.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018250+0000", 2024-09-18T11:31:50.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4299999999999999e-06 2024-09-18T11:31:50.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021364+0000", 2024-09-18T11:31:50.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031135389999999998 2024-09-18T11:31:50.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021494+0000", 2024-09-18T11:31:50.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000130333 2024-09-18T11:31:50.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.029947+0000", 2024-09-18T11:31:50.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084526150000000001 2024-09-18T11:31:50.845 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.845 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.846 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.846 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3323 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:31:50.846 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:31:50.846 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.6340870919999997, 2024-09-18T11:31:50.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0088051689999999998, 2024-09-18T11:31:50.846 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.846 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.846 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.846 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.846 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.846 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3323 2024-09-18T11:31:50.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.847 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:31:50.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:31:50.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116897+0000", 2024-09-18T11:31:50.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:50.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116908+0000", 2024-09-18T11:31:50.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1511e-05 2024-09-18T11:31:50.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116925+0000", 2024-09-18T11:31:50.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7028e-05 2024-09-18T11:31:50.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116932+0000", 2024-09-18T11:31:50.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5919999999999997e-06 2024-09-18T11:31:50.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116988+0000", 2024-09-18T11:31:50.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5928999999999999e-05 2024-09-18T11:31:50.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:50.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116995+0000", 2024-09-18T11:31:50.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7409999999999992e-06 2024-09-18T11:31:50.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122390+0000", 2024-09-18T11:31:50.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053941889999999998 2024-09-18T11:31:50.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122453+0000", 2024-09-18T11:31:50.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3833999999999999e-05 2024-09-18T11:31:50.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:50.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122581+0000", 2024-09-18T11:31:50.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012751200000000001 2024-09-18T11:31:50.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:50.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125034+0000", 2024-09-18T11:31:50.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024525609999999998 2024-09-18T11:31:50.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:50.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125607+0000", 2024-09-18T11:31:50.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00057356499999999995 2024-09-18T11:31:50.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125655+0000", 2024-09-18T11:31:50.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8179000000000001e-05 2024-09-18T11:31:50.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125704+0000", 2024-09-18T11:31:50.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8950999999999999e-05 2024-09-18T11:31:50.853 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.853 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.853 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.853 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3324 4.5 4:a087a566:::smithi11939506-46:head [rollback 46] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:31:50.853 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:31:50.853 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.6340445330000009, 2024-09-18T11:31:50.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010927767999999999, 2024-09-18T11:31:50.853 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.853 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.854 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.854 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.854 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.854 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3324 2024-09-18T11:31:50.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.854 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:31:50.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:31:50.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116941+0000", 2024-09-18T11:31:50.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:50.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116949+0000", 2024-09-18T11:31:50.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4660000000000002e-06 2024-09-18T11:31:50.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116960+0000", 2024-09-18T11:31:50.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1484999999999999e-05 2024-09-18T11:31:50.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116964+0000", 2024-09-18T11:31:50.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2810000000000002e-06 2024-09-18T11:31:50.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117053+0000", 2024-09-18T11:31:50.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8747999999999998e-05 2024-09-18T11:31:50.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:50.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117059+0000", 2024-09-18T11:31:50.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.246e-06 2024-09-18T11:31:50.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123604+0000", 2024-09-18T11:31:50.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065445319999999996 2024-09-18T11:31:50.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123663+0000", 2024-09-18T11:31:50.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8740999999999999e-05 2024-09-18T11:31:50.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:50.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123803+0000", 2024-09-18T11:31:50.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014049200000000001 2024-09-18T11:31:50.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:50.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127600+0000", 2024-09-18T11:31:50.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037971179999999999 2024-09-18T11:31:50.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:50.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127814+0000", 2024-09-18T11:31:50.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021333600000000001 2024-09-18T11:31:50.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127847+0000", 2024-09-18T11:31:50.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3188999999999997e-05 2024-09-18T11:31:50.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127869+0000", 2024-09-18T11:31:50.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2515e-05 2024-09-18T11:31:50.859 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.860 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.860 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.860 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3325 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:31:50.860 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:31:50.860 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.6339973469999993, 2024-09-18T11:31:50.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181030999999999, 2024-09-18T11:31:50.860 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.860 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.860 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.860 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.860 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.861 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3325 2024-09-18T11:31:50.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.861 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:31:50.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:31:50.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116988+0000", 2024-09-18T11:31:50.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:50.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116991+0000", 2024-09-18T11:31:50.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9890000000000001e-06 2024-09-18T11:31:50.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116998+0000", 2024-09-18T11:31:50.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5309999999999998e-06 2024-09-18T11:31:50.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117005+0000", 2024-09-18T11:31:50.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6420000000000001e-06 2024-09-18T11:31:50.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117091+0000", 2024-09-18T11:31:50.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6835999999999993e-05 2024-09-18T11:31:50.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:50.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117097+0000", 2024-09-18T11:31:50.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:31:50.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124717+0000", 2024-09-18T11:31:50.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076204899999999997 2024-09-18T11:31:50.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124773+0000", 2024-09-18T11:31:50.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5470999999999998e-05 2024-09-18T11:31:50.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:50.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124877+0000", 2024-09-18T11:31:50.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010426500000000001 2024-09-18T11:31:50.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:50.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127901+0000", 2024-09-18T11:31:50.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030243449999999999 2024-09-18T11:31:50.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:50.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128115+0000", 2024-09-18T11:31:50.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021355300000000001 2024-09-18T11:31:50.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128145+0000", 2024-09-18T11:31:50.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0290999999999998e-05 2024-09-18T11:31:50.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128170+0000", 2024-09-18T11:31:50.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4811e-05 2024-09-18T11:31:50.866 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.866 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.867 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.867 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3474 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1872245~637761] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:31:50.867 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:31:50.867 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.3072206699999995, 2024-09-18T11:31:50.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0092038580000000005, 2024-09-18T11:31:50.867 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.867 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.867 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.867 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.867 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.868 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3474 2024-09-18T11:31:50.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.868 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:31:50.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:31:50.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443764+0000", 2024-09-18T11:31:50.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:50.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444359+0000", 2024-09-18T11:31:50.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059519 2024-09-18T11:31:50.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444374+0000", 2024-09-18T11:31:50.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4949e-05 2024-09-18T11:31:50.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444382+0000", 2024-09-18T11:31:50.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1369999999999996e-06 2024-09-18T11:31:50.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444428+0000", 2024-09-18T11:31:50.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6783000000000003e-05 2024-09-18T11:31:50.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444507+0000", 2024-09-18T11:31:50.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8415000000000001e-05 2024-09-18T11:31:50.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:50.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444638+0000", 2024-09-18T11:31:50.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013129 2024-09-18T11:31:50.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:50.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.449059+0000", 2024-09-18T11:31:50.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004420719 2024-09-18T11:31:50.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:50.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452909+0000", 2024-09-18T11:31:50.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003850773 2024-09-18T11:31:50.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452944+0000", 2024-09-18T11:31:50.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-05 2024-09-18T11:31:50.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452969+0000", 2024-09-18T11:31:50.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5795999999999998e-05 2024-09-18T11:31:50.873 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.873 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.873 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.873 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3475 4.5 4:a56ea1fb:::smithi11939506-48:head [setxattr _header (57) in=7b,truncate 3042100] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:31:50.873 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:31:50.873 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.306591955, 2024-09-18T11:31:50.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086700660000000006, 2024-09-18T11:31:50.873 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.874 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:50.874 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.874 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.874 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.874 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3475 2024-09-18T11:31:50.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.874 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:31:50.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:31:50.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444393+0000", 2024-09-18T11:31:50.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:50.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444398+0000", 2024-09-18T11:31:50.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4909999999999999e-06 2024-09-18T11:31:50.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444405+0000", 2024-09-18T11:31:50.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5329999999999997e-06 2024-09-18T11:31:50.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444410+0000", 2024-09-18T11:31:50.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2470000000000002e-06 2024-09-18T11:31:50.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446046+0000", 2024-09-18T11:31:50.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001636722 2024-09-18T11:31:50.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446121+0000", 2024-09-18T11:31:50.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4604000000000006e-05 2024-09-18T11:31:50.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:50.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446274+0000", 2024-09-18T11:31:50.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015350300000000001 2024-09-18T11:31:50.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:50.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452884+0000", 2024-09-18T11:31:50.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066096439999999996 2024-09-18T11:31:50.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:50.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453000+0000", 2024-09-18T11:31:50.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000116204 2024-09-18T11:31:50.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:50.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453032+0000", 2024-09-18T11:31:50.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1327e-05 2024-09-18T11:31:50.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453064+0000", 2024-09-18T11:31:50.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2645000000000003e-05 2024-09-18T11:31:50.881 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.881 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.881 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.881 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3476 4.5 4:a56ea1fb:::smithi11939506-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:31:50.881 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:31:50.881 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.306552688, 2024-09-18T11:31:50.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087596800000000006, 2024-09-18T11:31:50.881 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.881 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:50.881 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.882 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.882 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.882 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3476 2024-09-18T11:31:50.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.882 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:31:50.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:31:50.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:31:50.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:50.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444436+0000", 2024-09-18T11:31:50.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9330000000000001e-06 2024-09-18T11:31:50.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444443+0000", 2024-09-18T11:31:50.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9369999999999998e-06 2024-09-18T11:31:50.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444447+0000", 2024-09-18T11:31:50.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1309999999999999e-06 2024-09-18T11:31:50.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446599+0000", 2024-09-18T11:31:50.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021521190000000001 2024-09-18T11:31:50.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:50.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446675+0000", 2024-09-18T11:31:50.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6099000000000004e-05 2024-09-18T11:31:50.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453088+0000", 2024-09-18T11:31:50.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006413307 2024-09-18T11:31:50.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453132+0000", 2024-09-18T11:31:50.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3548999999999999e-05 2024-09-18T11:31:50.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453193+0000", 2024-09-18T11:31:50.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1116000000000002e-05 2024-09-18T11:31:50.886 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.886 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.886 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.886 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3512 4.a 4:588debc2:::smithi11939506-18:43 [read 0~3445435,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:31:50.886 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.855533+0000", 2024-09-18T11:31:50.887 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.8954530809999999, 2024-09-18T11:31:50.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076145450000000003, 2024-09-18T11:31:50.887 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:50.887 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:50.887 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:50.887 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:50.887 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:50.887 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3512 2024-09-18T11:31:50.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.887 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:50.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:50.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.855533+0000", 2024-09-18T11:31:50.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:50.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.855533+0000", 2024-09-18T11:31:50.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:50.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:50.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.855533+0000", 2024-09-18T11:31:50.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:50.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:50.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.855535+0000", 2024-09-18T11:31:50.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.729e-06 2024-09-18T11:31:50.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:50.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.855541+0000", 2024-09-18T11:31:50.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0159999999999997e-06 2024-09-18T11:31:50.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:50.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.855545+0000", 2024-09-18T11:31:50.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7139999999999999e-06 2024-09-18T11:31:50.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:50.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.862093+0000", 2024-09-18T11:31:50.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065479639999999999 2024-09-18T11:31:50.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:50.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.862196+0000", 2024-09-18T11:31:50.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010261 2024-09-18T11:31:50.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:50.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:50.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:50.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.863148+0000", 2024-09-18T11:31:50.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00095185500000000004 2024-09-18T11:31:50.891 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.891 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.891 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.891 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:50.891 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:50.891 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:50.893 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:31:50.894 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:50.892+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:50.894 INFO:tasks.rados.rados.0.smithi119.stdout:1616: finishing set_chunk to oid 42 2024-09-18T11:31:50.894 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 223 (ObjNum 314 snap 43 seq_num 314) dirty exists 2024-09-18T11:31:50.894 INFO:tasks.rados.rados.0.smithi119.stdout:1615: done (2 left) 2024-09-18T11:31:50.895 INFO:tasks.rados.rados.0.smithi119.stdout:1616: done (1 left) 2024-09-18T11:31:50.895 INFO:tasks.rados.rados.0.smithi119.stdout:1617: done (0 left) 2024-09-18T11:31:50.895 INFO:tasks.rados.rados.0.smithi119.stdout:1618: read oid 33 snap -1 2024-09-18T11:31:50.895 INFO:tasks.rados.rados.0.smithi119.stdout:1618: expect (ObjNum 316 snap 43 seq_num 316) 2024-09-18T11:31:50.895 INFO:tasks.rados.rados.0.smithi119.stdout:1619: read oid 30 snap -1 2024-09-18T11:31:50.895 INFO:tasks.rados.rados.0.smithi119.stdout:1619: expect (ObjNum 302 snap 43 seq_num 302) 2024-09-18T11:31:50.895 INFO:tasks.rados.rados.0.smithi119.stdout:1620: read oid 27 snap -1 2024-09-18T11:31:50.895 INFO:tasks.rados.rados.0.smithi119.stdout:1620: expect deleted 2024-09-18T11:31:50.895 INFO:tasks.rados.rados.0.smithi119.stdout:1621: read oid 43 snap -1 2024-09-18T11:31:50.895 INFO:tasks.rados.rados.0.smithi119.stdout:1621: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:31:50.895 INFO:tasks.rados.rados.0.smithi119.stdout:1622: snap_create 2024-09-18T11:31:50.895 INFO:tasks.rados.rados.0.smithi119.stdout:1619: expect (ObjNum 302 snap 43 seq_num 302) 2024-09-18T11:31:50.946 INFO:tasks.rados.rados.0.smithi119.stdout:1618: expect (ObjNum 316 snap 43 seq_num 316) 2024-09-18T11:31:50.993 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:50.992+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:51.032 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:51.032 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:51.033 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:51.033 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:51.041 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:31:51.095 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:31:51.093+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:31:51.162 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:31:51.161+0000 7fe40be50640 -1 osd.0 120 handle_osd_map injecting map crc failure 2024-09-18T11:31:51.170 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T11:31:51.180 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:51.181 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:51.181 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:51.181 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:51.181 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:51.189 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:31:51.190 INFO:tasks.rados.rados.0.smithi119.stdout:1621: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:31:51.190 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:51.189+0000 7fcb662ad640 -1 osd.7 120 handle_osd_map injecting map crc failure 2024-09-18T11:31:51.191 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:51.189+0000 7fb6930fb640 -1 osd.2 120 handle_osd_map injecting map crc failure 2024-09-18T11:31:51.191 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:51.189+0000 7fd27659f640 -1 osd.6 120 handle_osd_map injecting map crc failure 2024-09-18T11:31:51.196 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:51.195+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:51.234 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:31:51.285 INFO:tasks.rados.rados.0.smithi119.stdout:1618: done (4 left) 2024-09-18T11:31:51.285 INFO:tasks.rados.rados.0.smithi119.stdout:1619: done (3 left) 2024-09-18T11:31:51.286 INFO:tasks.rados.rados.0.smithi119.stdout:1620: done (2 left) 2024-09-18T11:31:51.286 INFO:tasks.rados.rados.0.smithi119.stdout:1621: done (1 left) 2024-09-18T11:31:51.286 INFO:tasks.rados.rados.0.smithi119.stdout:1622: done (0 left) 2024-09-18T11:31:51.286 INFO:tasks.rados.rados.0.smithi119.stdout:1623: read oid 34 snap -1 2024-09-18T11:31:51.286 INFO:tasks.rados.rados.0.smithi119.stdout:1623: expect deleted 2024-09-18T11:31:51.286 INFO:tasks.rados.rados.0.smithi119.stdout:1624: write (excl) oid 46 current snap is 44 2024-09-18T11:31:51.287 INFO:tasks.rados.rados.0.smithi119.stdout:1624: seq_num 320 ranges {578240=566506,1609179=558098,2804887=551750} 2024-09-18T11:31:51.289 INFO:tasks.rados.rados.0.smithi119.stdout:1624: writing smithi11939506-46 from 578240 to 1144746 tid 1 2024-09-18T11:31:51.291 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:51.291 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:51.292 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:51.292 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:51.292 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.292 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:51.292 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:51.293 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 122.13078096700001, 2024-09-18T11:31:51.293 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:31:51.293 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:51.293 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:51.293 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:51.293 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:51.294 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:51.294 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:31:51.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.294 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:51.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.294 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:51.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:51.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.296 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:51.296 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:51.296 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.297 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:51.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:31:51.297 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:51.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.298 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:51.298 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:31:51.298 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:31:51.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.299 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:51.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:31:51.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:31:51.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:51.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:31:51.300 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:51.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:51.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:31:51.301 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:31:51.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.302 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:51.302 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:31:51.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:31:51.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.303 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:51.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:31:51.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:31:51.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.303 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:51.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:31:51.304 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:31:51.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.304 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:51.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:31:51.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:31:51.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:51.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:31:51.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:31:51.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:51.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:31:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:31:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.308 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:51.308 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:51.308 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 122.130654705, 2024-09-18T11:31:51.308 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055717040000000002, 2024-09-18T11:31:51.308 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:51.309 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:51.309 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:51.309 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:51.309 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:51.309 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 45 2024-09-18T11:31:51.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.310 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:51.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:51.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:51.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:51.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:51.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.312 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:51.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157676+0000", 2024-09-18T11:31:51.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:51.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.313 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:51.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157694+0000", 2024-09-18T11:31:51.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7422999999999999e-05 2024-09-18T11:31:51.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.313 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:51.314 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157704+0000", 2024-09-18T11:31:51.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9000000000000001e-06 2024-09-18T11:31:51.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.314 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157718+0000", 2024-09-18T11:31:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4788e-05 2024-09-18T11:31:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:51.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160937+0000", 2024-09-18T11:31:51.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00321886 2024-09-18T11:31:51.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:51.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161023+0000", 2024-09-18T11:31:51.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6063999999999995e-05 2024-09-18T11:31:51.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:51.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161202+0000", 2024-09-18T11:31:51.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017912599999999999 2024-09-18T11:31:51.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:51.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163052+0000", 2024-09-18T11:31:51.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001849554 2024-09-18T11:31:51.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163204+0000", 2024-09-18T11:31:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015186900000000001 2024-09-18T11:31:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163229+0000", 2024-09-18T11:31:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:31:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163249+0000", 2024-09-18T11:31:51.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9992000000000001e-05 2024-09-18T11:31:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 122.13060559199999, 2024-09-18T11:31:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:31:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:51.321 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:31:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:51.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:31:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:31:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:31:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:31:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:31:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:31:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:31:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:31:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:31:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:31:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:31:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:31:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:31:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:31:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:31:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:31:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:31:51.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:31:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:31:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:31:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:51.327 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 122.130563061, 2024-09-18T11:31:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:31:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:31:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:51.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:31:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:31:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:31:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:31:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:31:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:31:51.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:31:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:31:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:31:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:31:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:31:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:31:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:31:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:31:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:31:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:31:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:31:51.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:31:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:31:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:31:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:31:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 122.130508131, 2024-09-18T11:31:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:31:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:31:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:51.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:51.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:31:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:31:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:31:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:31:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:51.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:31:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:31:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:31:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:31:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:31:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:31:51.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:31:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:31:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:31:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:31:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:31:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:31:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:31:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:31:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:31:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:31:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.339 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 122.130498151, 2024-09-18T11:31:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:31:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:51.340 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:31:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:31:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:31:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:51.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:31:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:31:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:31:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:31:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:31:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:31:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:31:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:31:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:31:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:31:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:51.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:31:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:31:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:31:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:31:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:31:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:31:51.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:31:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:31:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:51.346 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 122.13048655199999, 2024-09-18T11:31:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:31:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:31:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:51.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:31:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:31:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:31:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:31:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:51.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:31:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:31:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:31:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:31:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:31:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:31:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:31:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:31:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:31:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:31:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:31:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:31:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:31:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:31:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:31:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:31:51.352 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 122.13043007100001, 2024-09-18T11:31:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:31:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:51.353 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:31:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:31:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:31:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:31:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:31:51.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:31:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:31:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:31:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:31:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:31:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:31:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:31:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:31:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:31:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:31:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:31:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:31:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:31:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:31:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:31:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:31:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:31:51.358 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 122.130416247, 2024-09-18T11:31:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:31:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:51.359 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:31:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:51.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:31:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:31:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:51.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:31:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:31:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:31:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:31:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:31:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:31:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:31:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:31:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:31:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:31:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:31:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:31:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:31:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:31:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:31:51.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:31:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:31:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:31:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:51.365 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 121.71423734, 2024-09-18T11:31:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:31:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:31:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:51.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:31:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:31:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:31:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:31:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:31:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:31:51.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:31:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:31:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:31:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:31:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:51.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:31:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:31:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:31:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:31:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:31:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:31:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:31:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:31:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:31:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:31:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.371 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 121.71421404, 2024-09-18T11:31:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:31:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:31:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:31:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:51.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:31:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:31:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:31:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:31:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:51.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:31:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:31:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:31:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:31:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:31:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:31:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:31:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:31:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:31:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:31:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:31:51.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:31:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:31:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:31:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:31:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:31:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:51.377 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 121.71412843100001, 2024-09-18T11:31:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:31:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:31:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.378 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:51.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:31:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:31:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:31:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:31:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:31:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:31:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:31:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:31:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:31:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:31:51.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:31:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:31:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:31:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:31:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:51.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:31:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:31:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:31:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:31:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:31:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:31:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:51.383 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 121.714117466, 2024-09-18T11:31:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:31:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:31:51.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:51.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:31:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:31:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:31:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:31:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:31:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:31:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:31:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:31:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:31:51.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:31:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:31:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:31:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:31:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:31:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:31:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:31:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:31:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:31:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:31:51.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:31:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 121.714106298, 2024-09-18T11:31:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:31:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:51.390 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:31:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:31:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:31:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:31:51.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:31:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:31:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:31:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:31:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:31:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:51.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:31:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:31:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:31:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:31:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:31:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:31:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:31:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:31:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:31:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:31:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:31:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:31:51.395 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 121.714040969, 2024-09-18T11:31:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:31:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:51.396 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:31:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:51.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:31:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:31:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:31:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:51.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:31:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:31:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:31:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:31:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:31:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:31:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:31:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:31:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:31:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:31:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:31:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:31:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:31:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:31:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:31:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:31:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:31:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:31:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:51.402 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 121.703878991, 2024-09-18T11:31:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:31:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:31:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:51.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:31:51.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:31:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:31:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:31:51.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:31:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:31:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:31:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:31:51.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:31:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:31:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:31:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:31:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:31:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:31:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:31:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:31:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:31:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:31:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:31:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:31:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:31:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 121.70378651199999, 2024-09-18T11:31:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:31:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:51.410 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:31:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:51.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:31:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:31:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:31:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:31:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:31:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:31:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:31:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:31:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:31:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:31:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:31:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:31:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:31:51.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:31:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:31:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:31:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:31:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:31:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:31:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:31:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1788 4.8 e60/22 4:1898b15d:::smithi11939506-25:head v 60'12, mlcod=58'9)", 2024-09-18T11:31:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 68.371381022999998, 2024-09-18T11:31:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058348599999999999, 2024-09-18T11:31:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:51.417 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:51.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916949+0000", 2024-09-18T11:31:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917602+0000", 2024-09-18T11:31:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00065337100000000003 2024-09-18T11:31:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917621+0000", 2024-09-18T11:31:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9287e-05 2024-09-18T11:31:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917629+0000", 2024-09-18T11:31:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3780000000000003e-06 2024-09-18T11:31:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917677+0000", 2024-09-18T11:31:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8068000000000001e-05 2024-09-18T11:31:51.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917699+0000", 2024-09-18T11:31:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2149000000000001e-05 2024-09-18T11:31:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922747+0000", 2024-09-18T11:31:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050477609999999996 2024-09-18T11:31:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922785+0000", 2024-09-18T11:31:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8726000000000002e-05 2024-09-18T11:31:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:58 4.3 e91/67 4:c70c3162:::smithi11939506-35:18 v 91'101, mlcod=86'95)", 2024-09-18T11:31:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 37.272722799999997, 2024-09-18T11:31:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055647500000000003, 2024-09-18T11:31:51.423 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015608+0000", 2024-09-18T11:31:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015614+0000", 2024-09-18T11:31:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5310000000000001e-06 2024-09-18T11:31:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015622+0000", 2024-09-18T11:31:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0809999999999994e-06 2024-09-18T11:31:51.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015627+0000", 2024-09-18T11:31:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8439999999999997e-06 2024-09-18T11:31:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016003+0000", 2024-09-18T11:31:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037652999999999997 2024-09-18T11:31:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:51.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016024+0000", 2024-09-18T11:31:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0780000000000001e-05 2024-09-18T11:31:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021141+0000", 2024-09-18T11:31:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051166639999999999 2024-09-18T11:31:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021174+0000", 2024-09-18T11:31:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2839000000000002e-05 2024-09-18T11:31:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:51.427 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3209 4.0 e109/22 4:019b6e09:::smithi11939506-40:head v 109'149, mlcod=106'146)", 2024-09-18T11:31:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:31:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.188945661, 2024-09-18T11:31:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063064039999999998, 2024-09-18T11:31:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:31:51.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:31:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099385+0000", 2024-09-18T11:31:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099951+0000", 2024-09-18T11:31:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056606600000000005 2024-09-18T11:31:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099964+0000", 2024-09-18T11:31:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2918999999999999e-05 2024-09-18T11:31:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099972+0000", 2024-09-18T11:31:51.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2300000000000002e-06 2024-09-18T11:31:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.100026+0000", 2024-09-18T11:31:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4456999999999997e-05 2024-09-18T11:31:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.100046+0000", 2024-09-18T11:31:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0417000000000001e-05 2024-09-18T11:31:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.105653+0000", 2024-09-18T11:31:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005606214 2024-09-18T11:31:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.105692+0000", 2024-09-18T11:31:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9671e-05 2024-09-18T11:31:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:51.432 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:51.432 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:51.434 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:31:51.435 INFO:tasks.rados.rados.0.smithi119.stdout:1624: writing smithi11939506-46 from 1609179 to 2167277 tid 2 2024-09-18T11:31:51.436 INFO:tasks.rados.rados.0.smithi119.stdout:1624: writing smithi11939506-46 from 2804887 to 3356637 tid 3 2024-09-18T11:31:51.436 INFO:tasks.rados.rados.0.smithi119.stdout:1625: tier_evict oid 42 2024-09-18T11:31:51.436 INFO:tasks.rados.rados.0.smithi119.stdout:1625: tier_evict oid 42 snap -1 2024-09-18T11:31:51.436 INFO:tasks.rados.rados.0.smithi119.stdout:1623: done (2 left) 2024-09-18T11:31:51.436 INFO:tasks.rados.rados.0.smithi119.stdout:1626: read oid 18 snap -1 2024-09-18T11:31:51.436 INFO:tasks.rados.rados.0.smithi119.stdout:1626: expect (ObjNum 205 snap 23 seq_num 205) 2024-09-18T11:31:51.436 INFO:tasks.rados.rados.0.smithi119.stdout:1627: tier_evict oid 11 2024-09-18T11:31:51.436 INFO:tasks.rados.rados.0.smithi119.stdout:1627: tier_evict oid 11 snap -1 2024-09-18T11:31:51.436 INFO:tasks.rados.rados.0.smithi119.stdout:1624: finishing write tid 1 to smithi11939506-46 2024-09-18T11:31:51.436 INFO:tasks.rados.rados.0.smithi119.stdout:1624: finishing write tid 2 to smithi11939506-46 2024-09-18T11:31:51.436 INFO:tasks.rados.rados.0.smithi119.stdout:1628: delete oid 25 current snap is 44 2024-09-18T11:31:51.436 INFO:tasks.rados.rados.0.smithi119.stdout:1625: got (22) Invalid argument 2024-09-18T11:31:51.437 INFO:tasks.rados.rados.0.smithi119.stdout:1625: done (4 left) 2024-09-18T11:31:51.437 INFO:tasks.rados.rados.0.smithi119.stdout:1628: done (3 left) 2024-09-18T11:31:51.437 INFO:tasks.rados.rados.0.smithi119.stdout:1629: snap_create 2024-09-18T11:31:51.437 INFO:tasks.rados.rados.0.smithi119.stdout:1627: got (22) Invalid argument 2024-09-18T11:31:51.437 INFO:tasks.rados.rados.0.smithi119.stdout:1624: finishing write tid 3 to smithi11939506-46 2024-09-18T11:31:51.437 INFO:tasks.rados.rados.0.smithi119.stdout:1626: expect (ObjNum 205 snap 23 seq_num 205) 2024-09-18T11:31:51.437 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:31:51.437 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:31:51.438 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:51.438 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:51.438 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:51.439 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:51.439 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:51.439 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:51.439 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:51.440 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:51.440 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:51.432+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:51.445 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:51.446 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:51.461 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:51.462 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:51.478 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:51.478 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:51.505 INFO:tasks.rados.rados.0.smithi119.stdout:1624: finishing write tid 4 to smithi11939506-46 2024-09-18T11:31:51.505 INFO:tasks.rados.rados.0.smithi119.stdout:1624: finishing write tid 5 to smithi11939506-46 2024-09-18T11:31:51.505 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 247 (ObjNum 320 snap 44 seq_num 320) dirty exists 2024-09-18T11:31:51.505 INFO:tasks.rados.rados.0.smithi119.stdout:1624: left oid 46 (ObjNum 320 snap 44 seq_num 320) 2024-09-18T11:31:51.535 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:51.534+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:51.540 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:51.540 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:51.548 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:31:51.635 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:51.634+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:51.688 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:51.688 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:51.697 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:31:51.736 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:51.735+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:51.835 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:51.872 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:51.874 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:31:51.874 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:51.873+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:51.974 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:51.974 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:51.975 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:51.973+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:51.983 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:31:52.076 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:52.074+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:52.122 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:52.122 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:52.130 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:31:52.179 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:52.178+0000 7fb6950ff640 -1 osd.2 121 handle_osd_map injecting map crc failure 2024-09-18T11:31:52.180 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:52.178+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:52.181 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is unset 2024-09-18T11:31:52.182 INFO:tasks.rados.rados.0.smithi119.stdout:1624: done (3 left) 2024-09-18T11:31:52.183 INFO:tasks.rados.rados.0.smithi119.stdout:1626: done (2 left) 2024-09-18T11:31:52.183 INFO:tasks.rados.rados.0.smithi119.stdout:1627: done (1 left) 2024-09-18T11:31:52.183 INFO:tasks.rados.rados.0.smithi119.stdout:1629: done (0 left) 2024-09-18T11:31:52.183 INFO:tasks.rados.rados.0.smithi119.stdout:1630: write (excl) oid 36 current snap is 45 2024-09-18T11:31:52.183 INFO:tasks.rados.rados.0.smithi119.stdout:1630: seq_num 322 ranges {611854=717004,2126143=561767,3259733=152121} 2024-09-18T11:31:52.190 INFO:tasks.rados.rados.0.smithi119.stdout:1630: writing smithi11939506-36 from 611854 to 1328858 tid 1 2024-09-18T11:31:52.197 INFO:tasks.rados.rados.0.smithi119.stdout:1630: writing smithi11939506-36 from 2126143 to 2687910 tid 2 2024-09-18T11:31:52.202 INFO:tasks.rados.rados.0.smithi119.stdout:1630: writing smithi11939506-36 from 3259733 to 3411854 tid 3 2024-09-18T11:31:52.202 INFO:tasks.rados.rados.0.smithi119.stdout:1631: tier_evict oid 43 2024-09-18T11:31:52.202 INFO:tasks.rados.rados.0.smithi119.stdout:1631: tier_evict oid 43 snap -1 2024-09-18T11:31:52.202 INFO:tasks.rados.rados.0.smithi119.stdout:1630: finishing write tid 1 to smithi11939506-36 2024-09-18T11:31:52.202 INFO:tasks.rados.rados.0.smithi119.stdout:1630: finishing write tid 2 to smithi11939506-36 2024-09-18T11:31:52.202 INFO:tasks.rados.rados.0.smithi119.stdout:1632: tier_evict oid 26 2024-09-18T11:31:52.202 INFO:tasks.rados.rados.0.smithi119.stdout:1632: tier_evict oid 26 snap -1 2024-09-18T11:31:52.203 INFO:tasks.rados.rados.0.smithi119.stdout:1633: read oid 20 snap -1 2024-09-18T11:31:52.203 INFO:tasks.rados.rados.0.smithi119.stdout:1633: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:31:52.203 INFO:tasks.rados.rados.0.smithi119.stdout:1634: write (excl) oid 49 current snap is 45 2024-09-18T11:31:52.203 INFO:tasks.rados.rados.0.smithi119.stdout:1634: seq_num 323 ranges {628661=792253,2184625=421778,3403509=25152} 2024-09-18T11:31:52.209 INFO:tasks.rados.rados.0.smithi119.stdout:1634: writing smithi11939506-49 from 628661 to 1420914 tid 1 2024-09-18T11:31:52.216 INFO:tasks.rados.rados.0.smithi119.stdout:1634: writing smithi11939506-49 from 2184625 to 2606403 tid 2 2024-09-18T11:31:52.221 INFO:tasks.rados.rados.0.smithi119.stdout:1634: writing smithi11939506-49 from 3403509 to 3428661 tid 3 2024-09-18T11:31:52.221 INFO:tasks.rados.rados.0.smithi119.stdout:1635: write (excl) oid 9 current snap is 45 2024-09-18T11:31:52.221 INFO:tasks.rados.rados.0.smithi119.stdout:1632: got (22) Invalid argument 2024-09-18T11:31:52.221 INFO:tasks.rados.rados.0.smithi119.stdout:1631: got (22) Invalid argument 2024-09-18T11:31:52.222 INFO:tasks.rados.rados.0.smithi119.stdout:1630: finishing write tid 3 to smithi11939506-36 2024-09-18T11:31:52.222 INFO:tasks.rados.rados.0.smithi119.stdout:1635: seq_num 324 ranges {645468=467502,1726754=565436,2798226=596838} 2024-09-18T11:31:52.227 INFO:tasks.rados.rados.0.smithi119.stdout:1635: writing smithi11939506-9 from 645468 to 1112970 tid 1 2024-09-18T11:31:52.233 INFO:tasks.rados.rados.0.smithi119.stdout:1635: writing smithi11939506-9 from 1726754 to 2292190 tid 2 2024-09-18T11:31:52.239 INFO:tasks.rados.rados.0.smithi119.stdout:1635: writing smithi11939506-9 from 2798226 to 3395064 tid 3 2024-09-18T11:31:52.240 INFO:tasks.rados.rados.0.smithi119.stdout:1631: done (5 left) 2024-09-18T11:31:52.240 INFO:tasks.rados.rados.0.smithi119.stdout:1632: done (4 left) 2024-09-18T11:31:52.240 INFO:tasks.rados.rados.0.smithi119.stdout:1636: snap_remove snap 35 2024-09-18T11:31:52.274 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:52.274 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:52.278 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:52.277+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:52.283 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:31:52.290 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:52.289+0000 7fcb662ad640 -1 osd.7 121 handle_osd_map injecting map crc failure 2024-09-18T11:31:52.379 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:52.378+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:52.422 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:52.422 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:52.430 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:31:52.480 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:52.479+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:52.528 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:52.528 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:52.537 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:31:52.581 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:52.580+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:52.674 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:52.674 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:52.682 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:31:52.683 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:52.681+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:52.782 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:52.781+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:52.820 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:52.821 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:52.830 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:31:52.883 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:52.882+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:52.928 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:52.928 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:52.936 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:31:52.984 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:52.983+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:53.016 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:53.015+0000 7f6bf3052640 -1 osd.5 122 handle_osd_map injecting map crc failure 2024-09-18T11:31:53.017 INFO:tasks.rados.rados.0.smithi119.stdout:1636: done (4 left) 2024-09-18T11:31:53.018 INFO:tasks.rados.rados.0.smithi119.stdout:1637: rollback oid 16 current snap is 45 2024-09-18T11:31:53.018 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 16 to 41 2024-09-18T11:31:53.018 INFO:tasks.rados.rados.0.smithi119.stdout:1630: finishing write tid 4 to smithi11939506-36 2024-09-18T11:31:53.018 INFO:tasks.rados.rados.0.smithi119.stdout:1630: finishing write tid 5 to smithi11939506-36 2024-09-18T11:31:53.018 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 36 v 191 (ObjNum 322 snap 45 seq_num 322) dirty exists 2024-09-18T11:31:53.018 INFO:tasks.rados.rados.0.smithi119.stdout:1630: left oid 36 (ObjNum 322 snap 45 seq_num 322) 2024-09-18T11:31:53.018 INFO:tasks.rados.rados.0.smithi119.stdout:1633: expect (ObjNum 111 snap 9 seq_num 111) 2024-09-18T11:31:53.082 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:53.083 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:53.085 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:53.084+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:53.091 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:31:53.186 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:53.185+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:53.200 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:53.201 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:53.209 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:31:53.241 INFO:tasks.rados.rados.0.smithi119.stdout:1634: finishing write tid 1 to smithi11939506-49 2024-09-18T11:31:53.242 INFO:tasks.rados.rados.0.smithi119.stdout:1634: finishing write tid 2 to smithi11939506-49 2024-09-18T11:31:53.242 INFO:tasks.rados.rados.0.smithi119.stdout:1634: finishing write tid 3 to smithi11939506-49 2024-09-18T11:31:53.242 INFO:tasks.rados.rados.0.smithi119.stdout:1634: finishing write tid 4 to smithi11939506-49 2024-09-18T11:31:53.242 INFO:tasks.rados.rados.0.smithi119.stdout:1634: finishing write tid 5 to smithi11939506-49 2024-09-18T11:31:53.242 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 374 (ObjNum 323 snap 45 seq_num 323) dirty exists 2024-09-18T11:31:53.242 INFO:tasks.rados.rados.0.smithi119.stdout:1634: left oid 49 (ObjNum 323 snap 45 seq_num 323) 2024-09-18T11:31:53.242 INFO:tasks.rados.rados.0.smithi119.stdout:1635: finishing write tid 1 to smithi11939506-9 2024-09-18T11:31:53.242 INFO:tasks.rados.rados.0.smithi119.stdout:1635: finishing write tid 2 to smithi11939506-9 2024-09-18T11:31:53.242 INFO:tasks.rados.rados.0.smithi119.stdout:1630: done (4 left) 2024-09-18T11:31:53.242 INFO:tasks.rados.rados.0.smithi119.stdout:1633: done (3 left) 2024-09-18T11:31:53.243 INFO:tasks.rados.rados.0.smithi119.stdout:1634: done (2 left) 2024-09-18T11:31:53.243 INFO:tasks.rados.rados.0.smithi119.stdout:1638: read oid 38 snap -1 2024-09-18T11:31:53.243 INFO:tasks.rados.rados.0.smithi119.stdout:1638: expect (ObjNum 306 snap 43 seq_num 306) 2024-09-18T11:31:53.243 INFO:tasks.rados.rados.0.smithi119.stdout:1639: read oid 23 snap -1 2024-09-18T11:31:53.243 INFO:tasks.rados.rados.0.smithi119.stdout:1639: expect (ObjNum 313 snap 43 seq_num 313) 2024-09-18T11:31:53.243 INFO:tasks.rados.rados.0.smithi119.stdout:1640: write (excl) oid 37 current snap is 45 2024-09-18T11:31:53.243 INFO:tasks.rados.rados.0.smithi119.stdout:1635: finishing write tid 3 to smithi11939506-9 2024-09-18T11:31:53.243 INFO:tasks.rados.rados.0.smithi119.stdout:1635: finishing write tid 4 to smithi11939506-9 2024-09-18T11:31:53.243 INFO:tasks.rados.rados.0.smithi119.stdout:1640: seq_num 325 ranges {662275=542751,1785236=425447,2942002=520273} 2024-09-18T11:31:53.247 INFO:tasks.rados.rados.0.smithi119.stdout:1640: writing smithi11939506-37 from 662275 to 1205026 tid 1 2024-09-18T11:31:53.253 INFO:tasks.rados.rados.0.smithi119.stdout:1640: writing smithi11939506-37 from 1785236 to 2210683 tid 2 2024-09-18T11:31:53.260 INFO:tasks.rados.rados.0.smithi119.stdout:1640: writing smithi11939506-37 from 2942002 to 3462275 tid 3 2024-09-18T11:31:53.260 INFO:tasks.rados.rados.0.smithi119.stdout:1641: write oid 39 current snap is 45 2024-09-18T11:31:53.261 INFO:tasks.rados.rados.0.smithi119.stdout:1635: finishing write tid 5 to smithi11939506-9 2024-09-18T11:31:53.261 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 379 (ObjNum 324 snap 45 seq_num 324) dirty exists 2024-09-18T11:31:53.261 INFO:tasks.rados.rados.0.smithi119.stdout:1635: left oid 9 (ObjNum 324 snap 45 seq_num 324) 2024-09-18T11:31:53.261 INFO:tasks.rados.rados.0.smithi119.stdout:1637: finishing rollback tid 0 to smithi11939506-16 2024-09-18T11:31:53.261 INFO:tasks.rados.rados.0.smithi119.stdout:1637: finishing rollback tid 1 to smithi11939506-16 2024-09-18T11:31:53.261 INFO:tasks.rados.rados.0.smithi119.stdout:1637: finishing rollback tid 2 to smithi11939506-16 2024-09-18T11:31:53.261 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 279 (ObjNum 286 snap 41 seq_num 286) dirty exists 2024-09-18T11:31:53.261 INFO:tasks.rados.rados.0.smithi119.stdout:1641: seq_num 326 ranges {679082=618000,1843718=569105,2853072=626010} 2024-09-18T11:31:53.267 INFO:tasks.rados.rados.0.smithi119.stdout:1641: writing smithi11939506-39 from 679082 to 1297082 tid 1 2024-09-18T11:31:53.273 INFO:tasks.rados.rados.0.smithi119.stdout:1641: writing smithi11939506-39 from 1843718 to 2412823 tid 2 2024-09-18T11:31:53.278 INFO:tasks.rados.rados.0.smithi119.stdout:1641: writing smithi11939506-39 from 2853072 to 3479082 tid 3 2024-09-18T11:31:53.279 INFO:tasks.rados.rados.0.smithi119.stdout:1635: done (5 left) 2024-09-18T11:31:53.279 INFO:tasks.rados.rados.0.smithi119.stdout:1637: done (4 left) 2024-09-18T11:31:53.279 INFO:tasks.rados.rados.0.smithi119.stdout:1642: write (excl) oid 6 current snap is 45 2024-09-18T11:31:53.279 INFO:tasks.rados.rados.0.smithi119.stdout:1642: seq_num 327 ranges {695889=409602,1618553=712763,2996848=499041} 2024-09-18T11:31:53.284 INFO:tasks.rados.rados.0.smithi119.stdout:1642: writing smithi11939506-6 from 695889 to 1105491 tid 1 2024-09-18T11:31:53.286 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:31:53.285+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:31:53.291 INFO:tasks.rados.rados.0.smithi119.stdout:1642: writing smithi11939506-6 from 1618553 to 2331316 tid 2 2024-09-18T11:31:53.297 INFO:tasks.rados.rados.0.smithi119.stdout:1642: writing smithi11939506-6 from 2996848 to 3495889 tid 3 2024-09-18T11:31:53.297 INFO:tasks.rados.rados.0.smithi119.stdout:1643: write (excl) oid 27 current snap is 45 2024-09-18T11:31:53.297 INFO:tasks.rados.rados.0.smithi119.stdout:1643: seq_num 328 ranges {712696=484851,1960682=572774,3307918=204778} 2024-09-18T11:31:53.303 INFO:tasks.rados.rados.0.smithi119.stdout:1643: writing smithi11939506-27 from 712696 to 1197547 tid 1 2024-09-18T11:31:53.310 INFO:tasks.rados.rados.0.smithi119.stdout:1643: writing smithi11939506-27 from 1960682 to 2533456 tid 2 2024-09-18T11:31:53.316 INFO:tasks.rados.rados.0.smithi119.stdout:1643: writing smithi11939506-27 from 3307918 to 3512696 tid 3 2024-09-18T11:31:53.316 INFO:tasks.rados.rados.0.smithi119.stdout:1644: snap_create 2024-09-18T11:31:53.316 INFO:tasks.rados.rados.0.smithi119.stdout:1638: expect (ObjNum 306 snap 43 seq_num 306) 2024-09-18T11:31:53.320 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:53.320 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:53.329 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:31:53.387 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:31:53.386+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:31:53.426 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:53.427 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:53.435 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:31:53.488 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:53.487+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:53.490 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:31:53.502 INFO:tasks.rados.rados.0.smithi119.stdout:1639: expect (ObjNum 313 snap 43 seq_num 313) 2024-09-18T11:31:53.533 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:53.534 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:53.542 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:31:53.605 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:31:53.605 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:31:53.610 INFO:tasks.rados.rados.0.smithi119.stdout:1640: finishing write tid 1 to smithi11939506-37 2024-09-18T11:31:53.610 INFO:tasks.rados.rados.0.smithi119.stdout:1640: finishing write tid 2 to smithi11939506-37 2024-09-18T11:31:53.610 INFO:tasks.rados.rados.0.smithi119.stdout:1640: finishing write tid 3 to smithi11939506-37 2024-09-18T11:31:53.611 INFO:tasks.rados.rados.0.smithi119.stdout:1640: finishing write tid 4 to smithi11939506-37 2024-09-18T11:31:53.611 INFO:tasks.rados.rados.0.smithi119.stdout:1640: finishing write tid 5 to smithi11939506-37 2024-09-18T11:31:53.611 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 258 (ObjNum 325 snap 45 seq_num 325) dirty exists 2024-09-18T11:31:53.611 INFO:tasks.rados.rados.0.smithi119.stdout:1640: left oid 37 (ObjNum 325 snap 45 seq_num 325) 2024-09-18T11:31:53.611 INFO:tasks.rados.rados.0.smithi119.stdout:1641: finishing write tid 1 to smithi11939506-39 2024-09-18T11:31:53.611 INFO:tasks.rados.rados.0.smithi119.stdout:1641: finishing write tid 2 to smithi11939506-39 2024-09-18T11:31:53.611 INFO:tasks.rados.rados.0.smithi119.stdout:1641: finishing write tid 3 to smithi11939506-39 2024-09-18T11:31:53.611 INFO:tasks.rados.rados.0.smithi119.stdout:1641: finishing write tid 4 to smithi11939506-39 2024-09-18T11:31:53.611 INFO:tasks.rados.rados.0.smithi119.stdout:1641: finishing write tid 5 to smithi11939506-39 2024-09-18T11:31:53.611 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 39 v 250 (ObjNum 326 snap 45 seq_num 326) dirty exists 2024-09-18T11:31:53.611 INFO:tasks.rados.rados.0.smithi119.stdout:1641: left oid 39 (ObjNum 326 snap 45 seq_num 326) 2024-09-18T11:31:53.611 INFO:tasks.rados.rados.0.smithi119.stdout:1642: finishing write tid 1 to smithi11939506-6 2024-09-18T11:31:53.611 INFO:tasks.rados.rados.0.smithi119.stdout:1642: finishing write tid 2 to smithi11939506-6 2024-09-18T11:31:53.612 INFO:tasks.rados.rados.0.smithi119.stdout:1642: finishing write tid 3 to smithi11939506-6 2024-09-18T11:31:53.612 INFO:tasks.rados.rados.0.smithi119.stdout:1642: finishing write tid 4 to smithi11939506-6 2024-09-18T11:31:53.612 INFO:tasks.rados.rados.0.smithi119.stdout:1642: finishing write tid 5 to smithi11939506-6 2024-09-18T11:31:53.612 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 6 v 140 (ObjNum 327 snap 45 seq_num 327) dirty exists 2024-09-18T11:31:53.612 INFO:tasks.rados.rados.0.smithi119.stdout:1642: left oid 6 (ObjNum 327 snap 45 seq_num 327) 2024-09-18T11:31:53.612 INFO:tasks.rados.rados.0.smithi119.stdout:1643: finishing write tid 1 to smithi11939506-27 2024-09-18T11:31:53.612 INFO:tasks.rados.rados.0.smithi119.stdout:1643: finishing write tid 2 to smithi11939506-27 2024-09-18T11:31:53.612 INFO:tasks.rados.rados.0.smithi119.stdout:1643: finishing write tid 3 to smithi11939506-27 2024-09-18T11:31:53.612 INFO:tasks.rados.rados.0.smithi119.stdout:1643: finishing write tid 4 to smithi11939506-27 2024-09-18T11:31:53.612 INFO:tasks.rados.rados.0.smithi119.stdout:1643: finishing write tid 5 to smithi11939506-27 2024-09-18T11:31:53.612 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 232 (ObjNum 328 snap 45 seq_num 328) dirty exists 2024-09-18T11:31:53.612 INFO:tasks.rados.rados.0.smithi119.stdout:1643: left oid 27 (ObjNum 328 snap 45 seq_num 328) 2024-09-18T11:31:53.620 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:53.621 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:53.626 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:53.625+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:53.635 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:53.635 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:53.649 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:53.650 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:53.665 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:53.665 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:53.679 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:53.680 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:53.681 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:53.681 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:53.681 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:53.681 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:53.681 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:53.690 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:31:53.690 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:53.688+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:53.695 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:53.695 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:53.710 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:53.710 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:53.790 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:53.789+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:53.830 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:53.830 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:53.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:53.830 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:53.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.830 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:31:53.830 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:53.831 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 123.80148627600001, 2024-09-18T11:31:53.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073601550000000002, 2024-09-18T11:31:53.831 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:53.831 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:53.831 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:53.831 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:31:53.831 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:31:53.831 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1 2024-09-18T11:31:53.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.831 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:53.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:53.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:53.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:53.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:31:53.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:53.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026524+0000", 2024-09-18T11:31:53.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999905 2024-09-18T11:31:53.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:53.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026541+0000", 2024-09-18T11:31:53.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7070000000000001e-05 2024-09-18T11:31:53.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:53.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026554+0000", 2024-09-18T11:31:53.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.288e-05 2024-09-18T11:31:53.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:53.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026564+0000", 2024-09-18T11:31:53.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0197000000000001e-05 2024-09-18T11:31:53.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026826+0000", 2024-09-18T11:31:53.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026212400000000003 2024-09-18T11:31:53.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:31:53.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026839+0000", 2024-09-18T11:31:53.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2831e-05 2024-09-18T11:31:53.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028605+0000", 2024-09-18T11:31:53.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001765794 2024-09-18T11:31:53.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:53.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028709+0000", 2024-09-18T11:31:53.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104348 2024-09-18T11:31:53.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:53.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028824+0000", 2024-09-18T11:31:53.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011527399999999999 2024-09-18T11:31:53.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:53.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030319+0000", 2024-09-18T11:31:53.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001494439 2024-09-18T11:31:53.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:53.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033805+0000", 2024-09-18T11:31:53.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034857820000000002 2024-09-18T11:31:53.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:53.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033856+0000", 2024-09-18T11:31:53.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1687999999999997e-05 2024-09-18T11:31:53.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:53.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033894+0000", 2024-09-18T11:31:53.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7481000000000002e-05 2024-09-18T11:31:53.837 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.837 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:53.837 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.837 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1118 3.2 e38/21 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head v 38'76, mlcod=38'74)", 2024-09-18T11:31:53.837 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:53.838 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 93.880076123999999, 2024-09-18T11:31:53.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006048688, 2024-09-18T11:31:53.838 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:53.838 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:53.838 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:53.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:53.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:53.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:53.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:31:53.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:53.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947941+0000", 2024-09-18T11:31:53.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:53.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:53.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949868+0000", 2024-09-18T11:31:53.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019267659999999999 2024-09-18T11:31:53.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:53.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949893+0000", 2024-09-18T11:31:53.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5239999999999999e-05 2024-09-18T11:31:53.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:53.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949901+0000", 2024-09-18T11:31:53.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8420000000000005e-06 2024-09-18T11:31:53.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949954+0000", 2024-09-18T11:31:53.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3464999999999997e-05 2024-09-18T11:31:53.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:53.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949974+0000", 2024-09-18T11:31:53.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9731000000000001e-05 2024-09-18T11:31:53.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:53.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953948+0000", 2024-09-18T11:31:53.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039735489999999998 2024-09-18T11:31:53.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:53.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953992+0000", 2024-09-18T11:31:53.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4700999999999998e-05 2024-09-18T11:31:53.841 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.842 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:53.842 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.842 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1662 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'73, mlcod=51'69)", 2024-09-18T11:31:53.842 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:53.842 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 76.466471088999995, 2024-09-18T11:31:53.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059408539999999998, 2024-09-18T11:31:53.842 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:53.842 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:53.842 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:53.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:53.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:53.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:53.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:53.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:53.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:31:53.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:53.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:53.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361914+0000", 2024-09-18T11:31:53.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036569 2024-09-18T11:31:53.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:53.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361926+0000", 2024-09-18T11:31:53.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1960999999999999e-05 2024-09-18T11:31:53.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:53.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361937+0000", 2024-09-18T11:31:53.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0876999999999999e-05 2024-09-18T11:31:53.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361986+0000", 2024-09-18T11:31:53.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8652000000000001e-05 2024-09-18T11:31:53.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:53.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.362012+0000", 2024-09-18T11:31:53.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5854999999999999e-05 2024-09-18T11:31:53.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:53.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367460+0000", 2024-09-18T11:31:53.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054482100000000002 2024-09-18T11:31:53.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:53.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367490+0000", 2024-09-18T11:31:53.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9909000000000001e-05 2024-09-18T11:31:53.846 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.846 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:53.846 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.846 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:269 4.7 e58/22 4:e73ba68f:::smithi11939506-1:13 v 58'72, mlcod=57'69)", 2024-09-18T11:31:53.846 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:53.846 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 74.562639192000006, 2024-09-18T11:31:53.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065512569999999996, 2024-09-18T11:31:53.847 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:53.847 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:53.847 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:53.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:53.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:53.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:53.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:31:53.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:53.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265380+0000", 2024-09-18T11:31:53.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:53.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:53.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265388+0000", 2024-09-18T11:31:53.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.174e-06 2024-09-18T11:31:53.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:53.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265398+0000", 2024-09-18T11:31:53.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2599999999999994e-06 2024-09-18T11:31:53.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:53.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265402+0000", 2024-09-18T11:31:53.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0289999999999999e-06 2024-09-18T11:31:53.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265434+0000", 2024-09-18T11:31:53.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2511000000000001e-05 2024-09-18T11:31:53.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:53.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265447+0000", 2024-09-18T11:31:53.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2947e-05 2024-09-18T11:31:53.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:53.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271892+0000", 2024-09-18T11:31:53.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064445930000000002 2024-09-18T11:31:53.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:53.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271932+0000", 2024-09-18T11:31:53.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0303999999999999e-05 2024-09-18T11:31:53.850 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.850 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:53.850 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.851 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1788 4.8 4:1898b15d:::smithi11939506-25:head [write 453822~797061] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:31:53.851 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:53.851 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 70.912206608999995, 2024-09-18T11:31:53.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094305159999999999, 2024-09-18T11:31:53.851 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:53.851 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:53.851 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:53.851 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:53.851 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:53.851 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1788 2024-09-18T11:31:53.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.851 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:53.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:53.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:53.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:53.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:31:53.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:53.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915811+0000", 2024-09-18T11:31:53.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:53.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:53.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916608+0000", 2024-09-18T11:31:53.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079635000000000003 2024-09-18T11:31:53.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:53.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916624+0000", 2024-09-18T11:31:53.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6735000000000002e-05 2024-09-18T11:31:53.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:53.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916630+0000", 2024-09-18T11:31:53.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5709999999999997e-06 2024-09-18T11:31:53.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916674+0000", 2024-09-18T11:31:53.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4540999999999999e-05 2024-09-18T11:31:53.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:53.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916751+0000", 2024-09-18T11:31:53.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6695999999999999e-05 2024-09-18T11:31:53.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:53.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916872+0000", 2024-09-18T11:31:53.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120648 2024-09-18T11:31:53.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:53.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.919076+0000", 2024-09-18T11:31:53.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022046829999999998 2024-09-18T11:31:53.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:53.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925151+0000", 2024-09-18T11:31:53.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060740239999999999 2024-09-18T11:31:53.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:53.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925198+0000", 2024-09-18T11:31:53.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7896999999999999e-05 2024-09-18T11:31:53.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:53.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925244+0000", 2024-09-18T11:31:53.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5398999999999997e-05 2024-09-18T11:31:53.856 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.856 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:53.856 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.857 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1934 4.2 e65/22 4:425afe9c:::smithi11939506-29:head v 65'44, mlcod=60'40)", 2024-09-18T11:31:53.857 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:53.857 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 65.528027839000003, 2024-09-18T11:31:53.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064585119999999996, 2024-09-18T11:31:53.857 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:53.857 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:53.857 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:53.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:53.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:53.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:53.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:31:53.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:53.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299990+0000", 2024-09-18T11:31:53.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:53.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:53.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300789+0000", 2024-09-18T11:31:53.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079851499999999997 2024-09-18T11:31:53.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:53.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300806+0000", 2024-09-18T11:31:53.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7096999999999999e-05 2024-09-18T11:31:53.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:53.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300813+0000", 2024-09-18T11:31:53.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5079999999999999e-06 2024-09-18T11:31:53.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300870+0000", 2024-09-18T11:31:53.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7178e-05 2024-09-18T11:31:53.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:53.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300897+0000", 2024-09-18T11:31:53.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6284000000000001e-05 2024-09-18T11:31:53.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:53.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306410+0000", 2024-09-18T11:31:53.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055135210000000004 2024-09-18T11:31:53.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:53.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306451+0000", 2024-09-18T11:31:53.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0379000000000001e-05 2024-09-18T11:31:53.861 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.861 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:53.861 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.861 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2452 4.3 4:ca2b2309:::smithi11939506-44:head [read 0~3462242,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:31:53.861 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:31:53.861 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 47.791252481999997, 2024-09-18T11:31:53.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060906450000000004, 2024-09-18T11:31:53.861 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:53.861 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:53.861 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:53.862 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:53.862 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:53.862 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2452 2024-09-18T11:31:53.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.862 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:53.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:53.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:31:53.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:53.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:31:53.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:53.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036765+0000", 2024-09-18T11:31:53.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:53.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:53.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036777+0000", 2024-09-18T11:31:53.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1422e-05 2024-09-18T11:31:53.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:53.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036793+0000", 2024-09-18T11:31:53.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6575999999999999e-05 2024-09-18T11:31:53.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:53.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036800+0000", 2024-09-18T11:31:53.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9940000000000003e-06 2024-09-18T11:31:53.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:53.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5847999999999999e-05 2024-09-18T11:31:53.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:53.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036853+0000", 2024-09-18T11:31:53.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0119999999999999e-06 2024-09-18T11:31:53.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038527+0000", 2024-09-18T11:31:53.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016732730000000001 2024-09-18T11:31:53.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:53.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038580+0000", 2024-09-18T11:31:53.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3443999999999999e-05 2024-09-18T11:31:53.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:53.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042858+0000", 2024-09-18T11:31:53.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042781049999999999 2024-09-18T11:31:53.866 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.866 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:53.866 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.866 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2453 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:31:53.866 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:31:53.866 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 47.791210249999999, 2024-09-18T11:31:53.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063798429999999996, 2024-09-18T11:31:53.867 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:53.867 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:53.867 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:53.867 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:53.867 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:53.867 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2453 2024-09-18T11:31:53.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.867 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:53.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:53.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:31:53.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:53.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:31:53.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:53.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036809+0000", 2024-09-18T11:31:53.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:53.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:53.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036812+0000", 2024-09-18T11:31:53.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8339999999999999e-06 2024-09-18T11:31:53.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:53.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036819+0000", 2024-09-18T11:31:53.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9890000000000003e-06 2024-09-18T11:31:53.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:53.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036823+0000", 2024-09-18T11:31:53.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8099999999999999e-06 2024-09-18T11:31:53.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036912+0000", 2024-09-18T11:31:53.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9083000000000001e-05 2024-09-18T11:31:53.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:53.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036932+0000", 2024-09-18T11:31:53.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0222000000000001e-05 2024-09-18T11:31:53.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042896+0000", 2024-09-18T11:31:53.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059637240000000001 2024-09-18T11:31:53.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:53.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042952+0000", 2024-09-18T11:31:53.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5877000000000001e-05 2024-09-18T11:31:53.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:53.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043190+0000", 2024-09-18T11:31:53.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00023774500000000001 2024-09-18T11:31:53.871 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.871 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:53.871 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.871 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2454 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:31:53.871 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:53.871 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 47.791174294000001, 2024-09-18T11:31:53.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066404469999999998, 2024-09-18T11:31:53.872 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:53.872 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:53.872 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:53.872 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:53.872 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:53.872 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2454 2024-09-18T11:31:53.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.872 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:53.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:53.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:53.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:53.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:31:53.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:53.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036845+0000", 2024-09-18T11:31:53.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:53.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:53.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036856+0000", 2024-09-18T11:31:53.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1338000000000001e-05 2024-09-18T11:31:53.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:53.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036863+0000", 2024-09-18T11:31:53.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4749999999999998e-06 2024-09-18T11:31:53.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:53.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036867+0000", 2024-09-18T11:31:53.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3839999999999999e-06 2024-09-18T11:31:53.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036946+0000", 2024-09-18T11:31:53.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9054000000000006e-05 2024-09-18T11:31:53.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:53.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036949+0000", 2024-09-18T11:31:53.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9390000000000002e-06 2024-09-18T11:31:53.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043213+0000", 2024-09-18T11:31:53.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062640090000000001 2024-09-18T11:31:53.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:53.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043261+0000", 2024-09-18T11:31:53.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7750999999999997e-05 2024-09-18T11:31:53.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:53.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043486+0000", 2024-09-18T11:31:53.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000224985 2024-09-18T11:31:53.876 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.876 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:53.876 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.876 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2553 4.3 4:ca2b2309:::smithi11939506-44:head [write 564768~630013,stat] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-18T11:31:53.877 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:31:53.877 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 43.630896661000001, 2024-09-18T11:31:53.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065686879999999996, 2024-09-18T11:31:53.877 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:53.877 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:53.877 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:53.877 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:53.877 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:53.877 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2553 2024-09-18T11:31:53.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.877 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:53.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:53.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:31:53.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:53.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:31:53.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:53.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197120+0000", 2024-09-18T11:31:53.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:31:53.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:53.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199000+0000", 2024-09-18T11:31:53.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018792080000000001 2024-09-18T11:31:53.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:53.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199044+0000", 2024-09-18T11:31:53.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4048999999999998e-05 2024-09-18T11:31:53.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:53.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199071+0000", 2024-09-18T11:31:53.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7288e-05 2024-09-18T11:31:53.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199119+0000", 2024-09-18T11:31:53.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8000999999999997e-05 2024-09-18T11:31:53.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:53.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199208+0000", 2024-09-18T11:31:53.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9209000000000006e-05 2024-09-18T11:31:53.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:53.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199385+0000", 2024-09-18T11:31:53.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000177095 2024-09-18T11:31:53.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:53.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.202282+0000", 2024-09-18T11:31:53.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028970469999999998 2024-09-18T11:31:53.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:53.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203578+0000", 2024-09-18T11:31:53.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012960490000000001 2024-09-18T11:31:53.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:53.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203638+0000", 2024-09-18T11:31:53.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9599000000000003e-05 2024-09-18T11:31:53.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:53.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203692+0000", 2024-09-18T11:31:53.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3908e-05 2024-09-18T11:31:53.882 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.882 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:53.882 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.882 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2752 4.3 4:ca2b2309:::smithi11939506-44:25 [read 0~3210104,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:31:53.882 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:31:53.883 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.941127500999997, 2024-09-18T11:31:53.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068715460000000001, 2024-09-18T11:31:53.883 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:53.883 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:53.883 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:53.883 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:53.883 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:53.883 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2752 2024-09-18T11:31:53.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.883 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:53.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:53.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:31:53.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:53.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:31:53.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:53.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:31:53.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:53.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:53.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886895+0000", 2024-09-18T11:31:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.993e-06 2024-09-18T11:31:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886901+0000", 2024-09-18T11:31:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1859999999999997e-06 2024-09-18T11:31:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886904+0000", 2024-09-18T11:31:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1920000000000001e-06 2024-09-18T11:31:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.892622+0000", 2024-09-18T11:31:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057175660000000003 2024-09-18T11:31:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.892693+0000", 2024-09-18T11:31:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1265000000000003e-05 2024-09-18T11:31:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.893764+0000", 2024-09-18T11:31:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001070753 2024-09-18T11:31:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2793 4.1 e92/22 4:87078dfe:::smithi11939506-14:head v 92'195, mlcod=92'191)", 2024-09-18T11:31:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:31:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 34.603227466999996, 2024-09-18T11:31:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062761989999999997, 2024-09-18T11:31:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:31:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:31:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:31:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224994+0000", 2024-09-18T11:31:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020197600000000001 2024-09-18T11:31:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225007+0000", 2024-09-18T11:31:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3791999999999999e-05 2024-09-18T11:31:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225013+0000", 2024-09-18T11:31:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3630000000000002e-06 2024-09-18T11:31:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225062+0000", 2024-09-18T11:31:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8925000000000003e-05 2024-09-18T11:31:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225086+0000", 2024-09-18T11:31:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4737999999999999e-05 2024-09-18T11:31:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231022+0000", 2024-09-18T11:31:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059360690000000004 2024-09-18T11:31:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231069+0000", 2024-09-18T11:31:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6252999999999999e-05 2024-09-18T11:31:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2876 4.3 4:cc46532a:::smithi11939506-33:2f [read 0~3680733,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:31:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.321158567000001, 2024-09-18T11:31:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068391210000000001, 2024-09-18T11:31:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2876 2024-09-18T11:31:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:31:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:31:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506860+0000", 2024-09-18T11:31:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506869+0000", 2024-09-18T11:31:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5539999999999998e-06 2024-09-18T11:31:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506880+0000", 2024-09-18T11:31:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1552999999999999e-05 2024-09-18T11:31:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506887+0000", 2024-09-18T11:31:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7329999999999997e-06 2024-09-18T11:31:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506937+0000", 2024-09-18T11:31:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0317e-05 2024-09-18T11:31:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506945+0000", 2024-09-18T11:31:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1550000000000007e-06 2024-09-18T11:31:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.509021+0000", 2024-09-18T11:31:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020759870000000001 2024-09-18T11:31:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.509123+0000", 2024-09-18T11:31:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010132900000000001 2024-09-18T11:31:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513700+0000", 2024-09-18T11:31:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0045777320000000002 2024-09-18T11:31:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2877 4.3 4:cc46532a:::smithi11939506-33:2f [read 0~3680733,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:31:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.321124153, 2024-09-18T11:31:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085715170000000007, 2024-09-18T11:31:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2877 2024-09-18T11:31:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:31:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:31:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:31:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:53.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506898+0000", 2024-09-18T11:31:53.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4760000000000001e-06 2024-09-18T11:31:53.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:53.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506903+0000", 2024-09-18T11:31:53.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:53.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:53.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506906+0000", 2024-09-18T11:31:53.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1769999999999998e-06 2024-09-18T11:31:53.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507007+0000", 2024-09-18T11:31:53.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101479 2024-09-18T11:31:53.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:53.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507015+0000", 2024-09-18T11:31:53.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8490000000000006e-06 2024-09-18T11:31:53.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513757+0000", 2024-09-18T11:31:53.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0067417800000000002 2024-09-18T11:31:53.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513867+0000", 2024-09-18T11:31:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000109987 2024-09-18T11:31:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515467+0000", 2024-09-18T11:31:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00160037 2024-09-18T11:31:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2878 4.3 4:cc46532a:::smithi11939506-33:2f [sparse-read 0~1770976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:31:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.321093697999999, 2024-09-18T11:31:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090602640000000002, 2024-09-18T11:31:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2878 2024-09-18T11:31:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:31:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:31:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:31:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506929+0000", 2024-09-18T11:31:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.163e-06 2024-09-18T11:31:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506936+0000", 2024-09-18T11:31:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5519999999999999e-06 2024-09-18T11:31:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506941+0000", 2024-09-18T11:31:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507036+0000", 2024-09-18T11:31:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5070999999999998e-05 2024-09-18T11:31:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507042+0000", 2024-09-18T11:31:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3959999999999997e-06 2024-09-18T11:31:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515504+0000", 2024-09-18T11:31:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084620300000000006 2024-09-18T11:31:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515625+0000", 2024-09-18T11:31:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012150999999999999 2024-09-18T11:31:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515986+0000", 2024-09-18T11:31:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036140100000000002 2024-09-18T11:31:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:686 4.7 e98/22 4:e5707c18:::smithi11939506-10:30 v 98'160, mlcod=97'155)", 2024-09-18T11:31:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:31:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 29.809252008000001, 2024-09-18T11:31:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074641239999999999, 2024-09-18T11:31:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:31:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:31:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018767+0000", 2024-09-18T11:31:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018773+0000", 2024-09-18T11:31:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1460000000000004e-06 2024-09-18T11:31:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018781+0000", 2024-09-18T11:31:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9780000000000005e-06 2024-09-18T11:31:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018785+0000", 2024-09-18T11:31:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0029999999999996e-06 2024-09-18T11:31:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018826+0000", 2024-09-18T11:31:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1616999999999998e-05 2024-09-18T11:31:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018843+0000", 2024-09-18T11:31:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6753999999999999e-05 2024-09-18T11:31:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026202+0000", 2024-09-18T11:31:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073592620000000001 2024-09-18T11:31:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026232+0000", 2024-09-18T11:31:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9779000000000001e-05 2024-09-18T11:31:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.4.0:621 4.1 e98/22 4:854aa293:::smithi11939506-26:35 v 98'215, mlcod=96'210)", 2024-09-18T11:31:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:31:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 29.808368568999999, 2024-09-18T11:31:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007014752, 2024-09-18T11:31:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:31:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:31:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:31:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019659+0000", 2024-09-18T11:31:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8229999999999995e-06 2024-09-18T11:31:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019671+0000", 2024-09-18T11:31:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2636000000000001e-05 2024-09-18T11:31:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019677+0000", 2024-09-18T11:31:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5470000000000004e-06 2024-09-18T11:31:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019710+0000", 2024-09-18T11:31:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3191999999999997e-05 2024-09-18T11:31:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019722+0000", 2024-09-18T11:31:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2357e-05 2024-09-18T11:31:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026639+0000", 2024-09-18T11:31:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0069167630000000003 2024-09-18T11:31:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026666+0000", 2024-09-18T11:31:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7087999999999999e-05 2024-09-18T11:31:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.916 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3234 4.7 e109/22 4:e73ba68f:::smithi11939506-1:head v 109'206, mlcod=107'202)", 2024-09-18T11:31:53.916 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:37.128363+0000", 2024-09-18T11:31:53.916 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 16.699657185, 2024-09-18T11:31:53.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059367100000000004, 2024-09-18T11:31:53.916 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:53.916 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:53.916 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:53.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:53.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.128363+0000", 2024-09-18T11:31:53.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.128363+0000", 2024-09-18T11:31:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.128362+0000", 2024-09-18T11:31:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131291+0000", 2024-09-18T11:31:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029282840000000002 2024-09-18T11:31:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131305+0000", 2024-09-18T11:31:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4222999999999999e-05 2024-09-18T11:31:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131313+0000", 2024-09-18T11:31:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.208e-06 2024-09-18T11:31:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131354+0000", 2024-09-18T11:31:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1158000000000001e-05 2024-09-18T11:31:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131378+0000", 2024-09-18T11:31:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3828999999999998e-05 2024-09-18T11:31:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.134258+0000", 2024-09-18T11:31:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028798920000000002 2024-09-18T11:31:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.134299+0000", 2024-09-18T11:31:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1355e-05 2024-09-18T11:31:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3489 4.3 4:cc46532a:::smithi11939506-33:head [setxattr _header (57) in=7b,truncate 3092521] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:31:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.709941+0000", 2024-09-18T11:31:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.1180792210000003, 2024-09-18T11:31:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057848090000000001, 2024-09-18T11:31:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3489 2024-09-18T11:31:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709941+0000", 2024-09-18T11:31:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709941+0000", 2024-09-18T11:31:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709940+0000", 2024-09-18T11:31:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709944+0000", 2024-09-18T11:31:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0249999999999996e-06 2024-09-18T11:31:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709950+0000", 2024-09-18T11:31:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4690000000000002e-06 2024-09-18T11:31:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709954+0000", 2024-09-18T11:31:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:31:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.711862+0000", 2024-09-18T11:31:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001907607 2024-09-18T11:31:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:53.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.711944+0000", 2024-09-18T11:31:53.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2182999999999996e-05 2024-09-18T11:31:53.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:31:53.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.712056+0000", 2024-09-18T11:31:53.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011131100000000001 2024-09-18T11:31:53.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:53.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.715101+0000", 2024-09-18T11:31:53.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030452460000000002 2024-09-18T11:31:53.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:53.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.715672+0000", 2024-09-18T11:31:53.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000570917 2024-09-18T11:31:53.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:53.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.715699+0000", 2024-09-18T11:31:53.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7069000000000001e-05 2024-09-18T11:31:53.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:53.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.715726+0000", 2024-09-18T11:31:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6718000000000001e-05 2024-09-18T11:31:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3490 4.3 4:cc46532a:::smithi11939506-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:31:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.709960+0000", 2024-09-18T11:31:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.1180601709999998, 2024-09-18T11:31:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058965190000000002, 2024-09-18T11:31:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3490 2024-09-18T11:31:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709960+0000", 2024-09-18T11:31:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709960+0000", 2024-09-18T11:31:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709959+0000", 2024-09-18T11:31:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709964+0000", 2024-09-18T11:31:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0900000000000004e-06 2024-09-18T11:31:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709971+0000", 2024-09-18T11:31:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5950000000000004e-06 2024-09-18T11:31:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709975+0000", 2024-09-18T11:31:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0690000000000003e-06 2024-09-18T11:31:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.712315+0000", 2024-09-18T11:31:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002339556 2024-09-18T11:31:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.712392+0000", 2024-09-18T11:31:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7868999999999996e-05 2024-09-18T11:31:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.715759+0000", 2024-09-18T11:31:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0033669989999999999 2024-09-18T11:31:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.715802+0000", 2024-09-18T11:31:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2636000000000003e-05 2024-09-18T11:31:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.715856+0000", 2024-09-18T11:31:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4153000000000001e-05 2024-09-18T11:31:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:53.931 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:53.933 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:31:53.933 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:53.932+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:54.033 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:54.032+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:54.071 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:54.071 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:54.071 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:54.071 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:54.081 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:31:54.165 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:54.164+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:54.174 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:54.173+0000 7f6bf5056640 -1 osd.5 123 handle_osd_map injecting map crc failure 2024-09-18T11:31:54.175 INFO:tasks.rados.rados.0.smithi119.stdout:1638: done (6 left) 2024-09-18T11:31:54.175 INFO:tasks.rados.rados.0.smithi119.stdout:1639: done (5 left) 2024-09-18T11:31:54.175 INFO:tasks.rados.rados.0.smithi119.stdout:1640: done (4 left) 2024-09-18T11:31:54.175 INFO:tasks.rados.rados.0.smithi119.stdout:1641: done (3 left) 2024-09-18T11:31:54.176 INFO:tasks.rados.rados.0.smithi119.stdout:1642: done (2 left) 2024-09-18T11:31:54.176 INFO:tasks.rados.rados.0.smithi119.stdout:1643: done (1 left) 2024-09-18T11:31:54.176 INFO:tasks.rados.rados.0.smithi119.stdout:1644: done (0 left) 2024-09-18T11:31:54.176 INFO:tasks.rados.rados.0.smithi119.stdout:1645: read oid 3 snap -1 2024-09-18T11:31:54.176 INFO:tasks.rados.rados.0.smithi119.stdout:1645: expect (ObjNum 243 snap 28 seq_num 243) 2024-09-18T11:31:54.176 INFO:tasks.rados.rados.0.smithi119.stdout:1646: snap_create 2024-09-18T11:31:54.178 INFO:tasks.rados.rados.0.smithi119.stdout:1645: expect (ObjNum 243 snap 28 seq_num 243) 2024-09-18T11:31:54.196 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T11:31:54.222 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:54.222 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:54.222 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:54.222 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:54.222 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:54.230 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:31:54.240 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:54.239+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:54.335 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:54.334+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:54.371 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:54.371 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:54.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:54.371 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:54.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.371 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:31:54.371 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:54.372 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.252362417, 2024-09-18T11:31:54.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:31:54.372 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.372 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:54.372 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.372 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:54.372 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:54.372 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:31:54.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.372 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:54.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:31:54.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:31:54.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:54.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:31:54.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:31:54.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:31:54.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:31:54.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:31:54.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:31:54.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:31:54.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:31:54.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:54.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:31:54.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:31:54.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:31:54.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:31:54.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:31:54.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:31:54.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:31:54.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:31:54.377 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.377 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.377 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.377 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:31:54.377 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:54.377 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 93.699337016000001, 2024-09-18T11:31:54.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:31:54.377 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.377 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:54.377 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.377 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:54.378 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:54.378 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:31:54.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.378 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:54.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:31:54.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:31:54.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:54.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:31:54.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:31:54.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:31:54.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:31:54.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:31:54.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:31:54.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:31:54.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:31:54.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:31:54.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:31:54.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:31:54.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:31:54.381 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.381 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.382 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.382 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1969 4.a 4:588debc2:::smithi11939506-18:18 [read 0~3949612,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:54.382 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:54.382 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 62.994695178999997, 2024-09-18T11:31:54.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075409190000000001, 2024-09-18T11:31:54.382 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.382 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:54.382 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.382 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:54.382 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:54.382 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1969 2024-09-18T11:31:54.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.383 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:54.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:54.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373986+0000", 2024-09-18T11:31:54.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:54.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373990+0000", 2024-09-18T11:31:54.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1119999999999998e-06 2024-09-18T11:31:54.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.373997+0000", 2024-09-18T11:31:54.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4200000000000001e-06 2024-09-18T11:31:54.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.374002+0000", 2024-09-18T11:31:54.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:31:54.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380200+0000", 2024-09-18T11:31:54.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006197302 2024-09-18T11:31:54.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.380261+0000", 2024-09-18T11:31:54.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1389999999999993e-05 2024-09-18T11:31:54.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.381527+0000", 2024-09-18T11:31:54.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012660080000000001 2024-09-18T11:31:54.386 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.386 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.386 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.386 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2207 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1410296~761389] snapc 24=[24,1e,1c,1b,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T11:31:54.386 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:54.386 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 56.946184353, 2024-09-18T11:31:54.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008410743, 2024-09-18T11:31:54.387 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.387 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.387 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.387 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:54.387 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:54.387 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2207 2024-09-18T11:31:54.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.387 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:54.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:31:54.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422496+0000", 2024-09-18T11:31:54.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:54.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423190+0000", 2024-09-18T11:31:54.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00069405499999999995 2024-09-18T11:31:54.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423204+0000", 2024-09-18T11:31:54.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:31:54.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423208+0000", 2024-09-18T11:31:54.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:31:54.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423293+0000", 2024-09-18T11:31:54.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4524999999999995e-05 2024-09-18T11:31:54.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423394+0000", 2024-09-18T11:31:54.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101395 2024-09-18T11:31:54.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:54.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423579+0000", 2024-09-18T11:31:54.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018461000000000001 2024-09-18T11:31:54.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.428574+0000", 2024-09-18T11:31:54.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049944960000000002 2024-09-18T11:31:54.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430809+0000", 2024-09-18T11:31:54.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022351820000000001 2024-09-18T11:31:54.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430862+0000", 2024-09-18T11:31:54.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3189999999999997e-05 2024-09-18T11:31:54.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430908+0000", 2024-09-18T11:31:54.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5973999999999997e-05 2024-09-18T11:31:54.392 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.392 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.392 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.392 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2339 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:54.393 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:54.393 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 52.981879724000002, 2024-09-18T11:31:54.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011809992, 2024-09-18T11:31:54.393 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.393 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:54.393 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.393 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:54.393 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:54.393 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2339 2024-09-18T11:31:54.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.393 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:54.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:54.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:31:54.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:54.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:31:54.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4710000000000001e-06 2024-09-18T11:31:54.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386824+0000", 2024-09-18T11:31:54.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6104e-05 2024-09-18T11:31:54.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386836+0000", 2024-09-18T11:31:54.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1827999999999999e-05 2024-09-18T11:31:54.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388390+0000", 2024-09-18T11:31:54.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015539499999999999 2024-09-18T11:31:54.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:54.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388443+0000", 2024-09-18T11:31:54.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3785000000000001e-05 2024-09-18T11:31:54.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398238+0000", 2024-09-18T11:31:54.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097947989999999999 2024-09-18T11:31:54.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398315+0000", 2024-09-18T11:31:54.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6909999999999994e-05 2024-09-18T11:31:54.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398612+0000", 2024-09-18T11:31:54.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029671999999999999 2024-09-18T11:31:54.397 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.398 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.398 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.398 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2340 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:54.398 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:54.398 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 52.981836412, 2024-09-18T11:31:54.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012086448, 2024-09-18T11:31:54.398 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.398 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:54.398 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.398 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:54.398 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:54.399 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2340 2024-09-18T11:31:54.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.399 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:54.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:54.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:31:54.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:54.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386848+0000", 2024-09-18T11:31:54.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3069999999999998e-06 2024-09-18T11:31:54.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386855+0000", 2024-09-18T11:31:54.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0979999999999996e-06 2024-09-18T11:31:54.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386859+0000", 2024-09-18T11:31:54.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9360000000000001e-06 2024-09-18T11:31:54.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388466+0000", 2024-09-18T11:31:54.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001607154 2024-09-18T11:31:54.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:54.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388507+0000", 2024-09-18T11:31:54.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1093000000000001e-05 2024-09-18T11:31:54.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398647+0000", 2024-09-18T11:31:54.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010139686 2024-09-18T11:31:54.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398718+0000", 2024-09-18T11:31:54.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1138999999999998e-05 2024-09-18T11:31:54.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398931+0000", 2024-09-18T11:31:54.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021356899999999999 2024-09-18T11:31:54.403 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.403 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.403 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.403 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2341 4.5 4:aa575018:::smithi11939506-11:head [read 0~2487436,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:54.403 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:54.403 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 52.981818189999998, 2024-09-18T11:31:54.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013751648999999999, 2024-09-18T11:31:54.404 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.404 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:54.404 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.404 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:54.404 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:54.404 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2341 2024-09-18T11:31:54.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.404 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:54.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:54.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:31:54.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:54.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386866+0000", 2024-09-18T11:31:54.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4249999999999998e-06 2024-09-18T11:31:54.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386872+0000", 2024-09-18T11:31:54.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.003e-06 2024-09-18T11:31:54.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:31:54.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.507e-06 2024-09-18T11:31:54.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388520+0000", 2024-09-18T11:31:54.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001644146 2024-09-18T11:31:54.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:54.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388561+0000", 2024-09-18T11:31:54.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1486999999999998e-05 2024-09-18T11:31:54.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398963+0000", 2024-09-18T11:31:54.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010401353 2024-09-18T11:31:54.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.399046+0000", 2024-09-18T11:31:54.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3141999999999998e-05 2024-09-18T11:31:54.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.400615+0000", 2024-09-18T11:31:54.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015690960000000001 2024-09-18T11:31:54.408 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.409 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.409 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.409 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2529 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:54.409 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:31:54.409 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.195344925999997, 2024-09-18T11:31:54.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079268120000000001, 2024-09-18T11:31:54.409 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.409 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.409 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.409 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:54.410 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:54.410 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2529 2024-09-18T11:31:54.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.410 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:31:54.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:31:54.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173336+0000", 2024-09-18T11:31:54.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:54.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173339+0000", 2024-09-18T11:31:54.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9019999999999999e-06 2024-09-18T11:31:54.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173381+0000", 2024-09-18T11:31:54.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1836999999999999e-05 2024-09-18T11:31:54.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173390+0000", 2024-09-18T11:31:54.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.302e-06 2024-09-18T11:31:54.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174507+0000", 2024-09-18T11:31:54.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001116868 2024-09-18T11:31:54.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174603+0000", 2024-09-18T11:31:54.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5598000000000002e-05 2024-09-18T11:31:54.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:54.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174759+0000", 2024-09-18T11:31:54.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015616299999999999 2024-09-18T11:31:54.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178943+0000", 2024-09-18T11:31:54.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041846180000000002 2024-09-18T11:31:54.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181175+0000", 2024-09-18T11:31:54.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00223215 2024-09-18T11:31:54.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181229+0000", 2024-09-18T11:31:54.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3585999999999998e-05 2024-09-18T11:31:54.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181263+0000", 2024-09-18T11:31:54.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4292999999999998e-05 2024-09-18T11:31:54.415 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.415 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.415 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.415 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2530 4.5 4:af4ba75a:::smithi11939506-37:head [rollback 25] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:54.415 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:31:54.415 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.19528262, 2024-09-18T11:31:54.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080198800000000001, 2024-09-18T11:31:54.415 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.415 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2530 2024-09-18T11:31:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:31:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:31:54.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173398+0000", 2024-09-18T11:31:54.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:54.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173403+0000", 2024-09-18T11:31:54.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4389999999999999e-06 2024-09-18T11:31:54.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173414+0000", 2024-09-18T11:31:54.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0411000000000001e-05 2024-09-18T11:31:54.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173420+0000", 2024-09-18T11:31:54.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9610000000000002e-06 2024-09-18T11:31:54.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176390+0000", 2024-09-18T11:31:54.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029706289999999998 2024-09-18T11:31:54.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176501+0000", 2024-09-18T11:31:54.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011090300000000001 2024-09-18T11:31:54.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:54.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176827+0000", 2024-09-18T11:31:54.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00032526899999999999 2024-09-18T11:31:54.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178958+0000", 2024-09-18T11:31:54.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021314860000000001 2024-09-18T11:31:54.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181318+0000", 2024-09-18T11:31:54.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002359571 2024-09-18T11:31:54.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181372+0000", 2024-09-18T11:31:54.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3884000000000001e-05 2024-09-18T11:31:54.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181419+0000", 2024-09-18T11:31:54.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7148000000000003e-05 2024-09-18T11:31:54.421 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.421 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.421 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.421 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2531 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:31:54.421 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:54.421 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.195256497000003, 2024-09-18T11:31:54.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080687239999999993, 2024-09-18T11:31:54.421 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.421 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.422 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.422 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:54.422 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:54.422 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2531 2024-09-18T11:31:54.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.422 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:54.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:54.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:31:54.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:54.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173429+0000", 2024-09-18T11:31:54.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2989999999999998e-06 2024-09-18T11:31:54.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173436+0000", 2024-09-18T11:31:54.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1799999999999999e-06 2024-09-18T11:31:54.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173441+0000", 2024-09-18T11:31:54.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8810000000000004e-06 2024-09-18T11:31:54.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178267+0000", 2024-09-18T11:31:54.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048257969999999997 2024-09-18T11:31:54.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178388+0000", 2024-09-18T11:31:54.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121779 2024-09-18T11:31:54.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:54.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178599+0000", 2024-09-18T11:31:54.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000210094 2024-09-18T11:31:54.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.179339+0000", 2024-09-18T11:31:54.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00074063799999999995 2024-09-18T11:31:54.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181444+0000", 2024-09-18T11:31:54.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021044100000000001 2024-09-18T11:31:54.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181470+0000", 2024-09-18T11:31:54.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6415999999999999e-05 2024-09-18T11:31:54.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181494+0000", 2024-09-18T11:31:54.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3672e-05 2024-09-18T11:31:54.427 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.427 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.427 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.427 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2910 4.5 4:af4ba75a:::smithi11939506-37:head [write 786627~795991,stat] snapc 37=[37,35,34,30,2f,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:31:54.427 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:31:54.427 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.365198668000001, 2024-09-18T11:31:54.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084093710000000006, 2024-09-18T11:31:54.427 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.427 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.428 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.428 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:54.428 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:54.428 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2910 2024-09-18T11:31:54.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.428 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:31:54.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:31:54.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003481+0000", 2024-09-18T11:31:54.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:54.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004300+0000", 2024-09-18T11:31:54.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081934200000000003 2024-09-18T11:31:54.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004324+0000", 2024-09-18T11:31:54.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4618999999999999e-05 2024-09-18T11:31:54.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004335+0000", 2024-09-18T11:31:54.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.092e-05 2024-09-18T11:31:54.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004385+0000", 2024-09-18T11:31:54.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9709999999999997e-05 2024-09-18T11:31:54.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004470+0000", 2024-09-18T11:31:54.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4864000000000006e-05 2024-09-18T11:31:54.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:54.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004612+0000", 2024-09-18T11:31:54.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014241500000000001 2024-09-18T11:31:54.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011193+0000", 2024-09-18T11:31:54.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006580654 2024-09-18T11:31:54.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011788+0000", 2024-09-18T11:31:54.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059533600000000002 2024-09-18T11:31:54.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011842+0000", 2024-09-18T11:31:54.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3180000000000002e-05 2024-09-18T11:31:54.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011892+0000", 2024-09-18T11:31:54.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0618000000000002e-05 2024-09-18T11:31:54.433 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.433 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.433 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.433 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2953 4.a 4:588debc2:::smithi11939506-18:head [assert-version v113,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:31:54.433 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:31:54.433 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 30.341883150000001, 2024-09-18T11:31:54.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083931279999999997, 2024-09-18T11:31:54.433 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.434 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.434 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.434 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:54.434 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:54.434 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2953 2024-09-18T11:31:54.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.434 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:31:54.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:31:54.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026796+0000", 2024-09-18T11:31:54.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:54.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026806+0000", 2024-09-18T11:31:54.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9529999999999994e-06 2024-09-18T11:31:54.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026824+0000", 2024-09-18T11:31:54.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7901999999999999e-05 2024-09-18T11:31:54.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026832+0000", 2024-09-18T11:31:54.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8450000000000003e-06 2024-09-18T11:31:54.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026883+0000", 2024-09-18T11:31:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0883000000000001e-05 2024-09-18T11:31:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026980+0000", 2024-09-18T11:31:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7526000000000001e-05 2024-09-18T11:31:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:31:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032722+0000", 2024-09-18T11:31:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057414689999999999 2024-09-18T11:31:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.034230+0000", 2024-09-18T11:31:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001508035 2024-09-18T11:31:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035090+0000", 2024-09-18T11:31:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00086047300000000005 2024-09-18T11:31:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035147+0000", 2024-09-18T11:31:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6598000000000003e-05 2024-09-18T11:31:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035191+0000", 2024-09-18T11:31:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4747999999999999e-05 2024-09-18T11:31:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3105 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 3e=[3e,3d,3b,37,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-18T11:31:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:31:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.350470445999999, 2024-09-18T11:31:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011735906000000001, 2024-09-18T11:31:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3105 2024-09-18T11:31:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:31:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:31:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:31:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018212+0000", 2024-09-18T11:31:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6759999999999999e-06 2024-09-18T11:31:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018216+0000", 2024-09-18T11:31:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-06 2024-09-18T11:31:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018218+0000", 2024-09-18T11:31:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6110000000000001e-06 2024-09-18T11:31:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018246+0000", 2024-09-18T11:31:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7557000000000001e-05 2024-09-18T11:31:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:54.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018250+0000", 2024-09-18T11:31:54.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4299999999999999e-06 2024-09-18T11:31:54.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021364+0000", 2024-09-18T11:31:54.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031135389999999998 2024-09-18T11:31:54.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021494+0000", 2024-09-18T11:31:54.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000130333 2024-09-18T11:31:54.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.029947+0000", 2024-09-18T11:31:54.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084526150000000001 2024-09-18T11:31:54.445 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.445 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.445 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.446 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3323 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:31:54.446 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:31:54.446 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.251782402, 2024-09-18T11:31:54.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0088051689999999998, 2024-09-18T11:31:54.446 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.446 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.446 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.446 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3323 2024-09-18T11:31:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:31:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:31:54.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116897+0000", 2024-09-18T11:31:54.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:54.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116908+0000", 2024-09-18T11:31:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1511e-05 2024-09-18T11:31:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116925+0000", 2024-09-18T11:31:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7028e-05 2024-09-18T11:31:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116932+0000", 2024-09-18T11:31:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5919999999999997e-06 2024-09-18T11:31:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116988+0000", 2024-09-18T11:31:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5928999999999999e-05 2024-09-18T11:31:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:31:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116995+0000", 2024-09-18T11:31:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7409999999999992e-06 2024-09-18T11:31:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122390+0000", 2024-09-18T11:31:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053941889999999998 2024-09-18T11:31:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122453+0000", 2024-09-18T11:31:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3833999999999999e-05 2024-09-18T11:31:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122581+0000", 2024-09-18T11:31:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012751200000000001 2024-09-18T11:31:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125034+0000", 2024-09-18T11:31:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024525609999999998 2024-09-18T11:31:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125607+0000", 2024-09-18T11:31:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00057356499999999995 2024-09-18T11:31:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125655+0000", 2024-09-18T11:31:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8179000000000001e-05 2024-09-18T11:31:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125704+0000", 2024-09-18T11:31:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8950999999999999e-05 2024-09-18T11:31:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3324 4.5 4:a087a566:::smithi11939506-46:head [rollback 46] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:31:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:31:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.251739842999999, 2024-09-18T11:31:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010927767999999999, 2024-09-18T11:31:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.455 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.455 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.455 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:54.455 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:54.455 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3324 2024-09-18T11:31:54.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.455 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:31:54.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:31:54.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116941+0000", 2024-09-18T11:31:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116949+0000", 2024-09-18T11:31:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4660000000000002e-06 2024-09-18T11:31:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116960+0000", 2024-09-18T11:31:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1484999999999999e-05 2024-09-18T11:31:54.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116964+0000", 2024-09-18T11:31:54.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2810000000000002e-06 2024-09-18T11:31:54.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117053+0000", 2024-09-18T11:31:54.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8747999999999998e-05 2024-09-18T11:31:54.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117059+0000", 2024-09-18T11:31:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.246e-06 2024-09-18T11:31:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123604+0000", 2024-09-18T11:31:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065445319999999996 2024-09-18T11:31:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123663+0000", 2024-09-18T11:31:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8740999999999999e-05 2024-09-18T11:31:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123803+0000", 2024-09-18T11:31:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014049200000000001 2024-09-18T11:31:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127600+0000", 2024-09-18T11:31:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037971179999999999 2024-09-18T11:31:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127814+0000", 2024-09-18T11:31:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021333600000000001 2024-09-18T11:31:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127847+0000", 2024-09-18T11:31:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3188999999999997e-05 2024-09-18T11:31:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127869+0000", 2024-09-18T11:31:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2515e-05 2024-09-18T11:31:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3325 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:31:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:31:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.251692657, 2024-09-18T11:31:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181030999999999, 2024-09-18T11:31:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3325 2024-09-18T11:31:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:31:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:31:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116988+0000", 2024-09-18T11:31:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116991+0000", 2024-09-18T11:31:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9890000000000001e-06 2024-09-18T11:31:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116998+0000", 2024-09-18T11:31:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5309999999999998e-06 2024-09-18T11:31:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117005+0000", 2024-09-18T11:31:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6420000000000001e-06 2024-09-18T11:31:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117091+0000", 2024-09-18T11:31:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6835999999999993e-05 2024-09-18T11:31:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117097+0000", 2024-09-18T11:31:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:31:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124717+0000", 2024-09-18T11:31:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076204899999999997 2024-09-18T11:31:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124773+0000", 2024-09-18T11:31:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5470999999999998e-05 2024-09-18T11:31:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124877+0000", 2024-09-18T11:31:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010426500000000001 2024-09-18T11:31:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127901+0000", 2024-09-18T11:31:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030243449999999999 2024-09-18T11:31:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128115+0000", 2024-09-18T11:31:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021355300000000001 2024-09-18T11:31:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128145+0000", 2024-09-18T11:31:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0290999999999998e-05 2024-09-18T11:31:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128170+0000", 2024-09-18T11:31:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4811e-05 2024-09-18T11:31:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3474 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1872245~637761] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:31:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:31:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.9249159799999997, 2024-09-18T11:31:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0092038580000000005, 2024-09-18T11:31:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3474 2024-09-18T11:31:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:31:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:31:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443764+0000", 2024-09-18T11:31:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444359+0000", 2024-09-18T11:31:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059519 2024-09-18T11:31:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444374+0000", 2024-09-18T11:31:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4949e-05 2024-09-18T11:31:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444382+0000", 2024-09-18T11:31:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1369999999999996e-06 2024-09-18T11:31:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444428+0000", 2024-09-18T11:31:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6783000000000003e-05 2024-09-18T11:31:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444507+0000", 2024-09-18T11:31:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8415000000000001e-05 2024-09-18T11:31:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444638+0000", 2024-09-18T11:31:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013129 2024-09-18T11:31:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.449059+0000", 2024-09-18T11:31:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004420719 2024-09-18T11:31:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452909+0000", 2024-09-18T11:31:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003850773 2024-09-18T11:31:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452944+0000", 2024-09-18T11:31:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-05 2024-09-18T11:31:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452969+0000", 2024-09-18T11:31:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5795999999999998e-05 2024-09-18T11:31:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3475 4.5 4:a56ea1fb:::smithi11939506-48:head [setxattr _header (57) in=7b,truncate 3042100] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:31:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:31:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.9242872650000002, 2024-09-18T11:31:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086700660000000006, 2024-09-18T11:31:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3475 2024-09-18T11:31:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:31:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:31:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444393+0000", 2024-09-18T11:31:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:31:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444398+0000", 2024-09-18T11:31:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4909999999999999e-06 2024-09-18T11:31:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444405+0000", 2024-09-18T11:31:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5329999999999997e-06 2024-09-18T11:31:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444410+0000", 2024-09-18T11:31:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2470000000000002e-06 2024-09-18T11:31:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446046+0000", 2024-09-18T11:31:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001636722 2024-09-18T11:31:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446121+0000", 2024-09-18T11:31:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4604000000000006e-05 2024-09-18T11:31:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446274+0000", 2024-09-18T11:31:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015350300000000001 2024-09-18T11:31:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452884+0000", 2024-09-18T11:31:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066096439999999996 2024-09-18T11:31:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453000+0000", 2024-09-18T11:31:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000116204 2024-09-18T11:31:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453032+0000", 2024-09-18T11:31:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1327e-05 2024-09-18T11:31:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453064+0000", 2024-09-18T11:31:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2645000000000003e-05 2024-09-18T11:31:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3476 4.5 4:a56ea1fb:::smithi11939506-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:31:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:31:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.9242479980000002, 2024-09-18T11:31:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087596800000000006, 2024-09-18T11:31:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3476 2024-09-18T11:31:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:31:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:31:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:31:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444436+0000", 2024-09-18T11:31:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9330000000000001e-06 2024-09-18T11:31:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444443+0000", 2024-09-18T11:31:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9369999999999998e-06 2024-09-18T11:31:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444447+0000", 2024-09-18T11:31:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1309999999999999e-06 2024-09-18T11:31:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446599+0000", 2024-09-18T11:31:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021521190000000001 2024-09-18T11:31:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446675+0000", 2024-09-18T11:31:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6099000000000004e-05 2024-09-18T11:31:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453088+0000", 2024-09-18T11:31:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006413307 2024-09-18T11:31:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453132+0000", 2024-09-18T11:31:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3548999999999999e-05 2024-09-18T11:31:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453193+0000", 2024-09-18T11:31:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1116000000000002e-05 2024-09-18T11:31:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3512 4.a 4:588debc2:::smithi11939506-18:43 [read 0~3445435,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:31:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.855533+0000", 2024-09-18T11:31:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.5131483909999996, 2024-09-18T11:31:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076145450000000003, 2024-09-18T11:31:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:31:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:31:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3512 2024-09-18T11:31:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.855533+0000", 2024-09-18T11:31:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.855533+0000", 2024-09-18T11:31:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.855533+0000", 2024-09-18T11:31:54.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:54.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.855535+0000", 2024-09-18T11:31:54.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.729e-06 2024-09-18T11:31:54.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.855541+0000", 2024-09-18T11:31:54.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0159999999999997e-06 2024-09-18T11:31:54.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.855545+0000", 2024-09-18T11:31:54.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7139999999999999e-06 2024-09-18T11:31:54.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.862093+0000", 2024-09-18T11:31:54.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065479639999999999 2024-09-18T11:31:54.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.862196+0000", 2024-09-18T11:31:54.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010261 2024-09-18T11:31:54.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.863148+0000", 2024-09-18T11:31:54.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00095185500000000004 2024-09-18T11:31:54.492 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.492 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.492 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.492 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.492 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.492 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:54.494 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:31:54.494 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:31:54.492+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:31:54.594 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:54.593+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:54.633 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:54.633 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:54.633 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:31:54.634 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:54.642 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:31:54.695 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:54.694+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:54.780 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:54.781 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:31:54.781 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:31:54.781 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:31:54.781 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:54.789 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:31:54.796 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:54.795+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:54.890 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:54.890 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:54.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:54.890 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:54.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.890 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:54.890 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:54.890 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 125.72979569899999, 2024-09-18T11:31:54.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:31:54.890 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.890 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.891 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.891 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:54.891 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:54.891 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:31:54.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.891 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:54.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:54.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:31:54.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:54.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:31:54.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:31:54.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:31:54.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:31:54.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:31:54.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:54.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:31:54.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:31:54.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:31:54.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:31:54.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:54.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:31:54.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:31:54.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:31:54.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:31:54.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:31:54.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:31:54.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:31:54.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:31:54.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:31:54.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:31:54.896 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.896 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.896 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.897 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:54.897 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:54.897 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 125.729669437, 2024-09-18T11:31:54.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055717040000000002, 2024-09-18T11:31:54.897 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.897 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.897 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.897 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:54.898 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:54.898 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 45 2024-09-18T11:31:54.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.898 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:54.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:54.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157676+0000", 2024-09-18T11:31:54.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:54.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157694+0000", 2024-09-18T11:31:54.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7422999999999999e-05 2024-09-18T11:31:54.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157704+0000", 2024-09-18T11:31:54.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9000000000000001e-06 2024-09-18T11:31:54.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157718+0000", 2024-09-18T11:31:54.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4788e-05 2024-09-18T11:31:54.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160937+0000", 2024-09-18T11:31:54.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00321886 2024-09-18T11:31:54.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161023+0000", 2024-09-18T11:31:54.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6063999999999995e-05 2024-09-18T11:31:54.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:54.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161202+0000", 2024-09-18T11:31:54.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017912599999999999 2024-09-18T11:31:54.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163052+0000", 2024-09-18T11:31:54.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001849554 2024-09-18T11:31:54.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163204+0000", 2024-09-18T11:31:54.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015186900000000001 2024-09-18T11:31:54.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163229+0000", 2024-09-18T11:31:54.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:31:54.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163249+0000", 2024-09-18T11:31:54.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9992000000000001e-05 2024-09-18T11:31:54.905 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.905 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.905 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.905 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:54.905 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:54.905 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 125.729620324, 2024-09-18T11:31:54.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:31:54.906 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.906 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.906 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.906 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:54.906 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:54.906 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:31:54.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.906 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:54.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:54.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:54.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:54.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:31:54.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:31:54.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:31:54.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:31:54.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:31:54.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:31:54.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:31:54.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:31:54.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:31:54.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:31:54.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:54.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:31:54.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:31:54.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:31:54.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:31:54.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:31:54.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:31:54.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:31:54.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:31:54.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:31:54.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:31:54.913 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.914 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.914 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.914 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:54.914 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:54.914 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 125.729577793, 2024-09-18T11:31:54.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:31:54.914 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.914 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.915 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.915 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:54.915 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:54.915 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:31:54.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.915 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:54.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:54.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:31:54.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:54.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:31:54.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:31:54.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:31:54.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:31:54.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:31:54.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:31:54.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:31:54.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:31:54.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:31:54.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:31:54.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:54.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:31:54.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:31:54.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:31:54.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:31:54.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:31:54.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:31:54.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:31:54.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:31:54.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:31:54.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:31:54.922 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.922 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.922 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.922 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:54.923 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:54.923 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 125.729522863, 2024-09-18T11:31:54.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:31:54.923 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.923 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.923 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.924 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:54.924 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:54.924 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:31:54.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.924 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:54.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:54.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:54.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:54.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:31:54.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:31:54.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:31:54.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:31:54.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:31:54.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:31:54.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:31:54.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:31:54.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:31:54.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:31:54.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:54.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:31:54.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:31:54.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:31:54.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:31:54.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:31:54.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:31:54.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:31:54.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:31:54.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:31:54.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:31:54.931 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.931 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.931 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.931 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:54.931 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:54.931 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 125.729512883, 2024-09-18T11:31:54.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:31:54.931 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.932 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.932 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.932 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:54.932 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:54.932 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:31:54.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.932 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:54.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:54.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:54.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:54.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:31:54.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:31:54.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:31:54.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:31:54.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:31:54.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:31:54.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:31:54.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:31:54.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:31:54.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:31:54.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:54.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:31:54.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:31:54.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:31:54.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:31:54.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:31:54.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:31:54.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:31:54.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:31:54.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:31:54.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:31:54.938 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.938 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.938 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.939 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:54.939 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:54.939 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 125.72950128399999, 2024-09-18T11:31:54.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:31:54.939 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.939 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.939 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.939 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:54.939 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:54.939 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:31:54.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.940 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:54.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:54.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:54.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:54.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:31:54.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:31:54.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:31:54.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:31:54.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:31:54.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:31:54.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:31:54.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:31:54.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:31:54.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:31:54.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:54.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:31:54.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:31:54.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:31:54.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:31:54.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:31:54.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:31:54.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:31:54.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:31:54.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:31:54.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:31:54.944 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.944 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.944 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.945 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:54.945 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:54.945 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 125.72944480300001, 2024-09-18T11:31:54.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:31:54.945 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.945 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.945 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.945 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:54.945 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:54.945 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:31:54.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.946 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:54.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:54.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:31:54.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:54.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:31:54.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:31:54.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:31:54.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:31:54.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:31:54.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:31:54.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:31:54.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:31:54.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:31:54.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:31:54.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:54.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:31:54.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:31:54.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:31:54.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:31:54.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:31:54.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:31:54.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:31:54.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:31:54.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:31:54.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:31:54.951 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.951 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.951 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.951 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:54.951 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:54.951 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 125.729430979, 2024-09-18T11:31:54.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:31:54.952 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.952 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:54.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:54.952 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:31:54.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.952 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:54.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:54.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:54.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:54.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:31:54.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:31:54.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:31:54.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:31:54.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:31:54.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:31:54.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:31:54.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:31:54.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:31:54.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:31:54.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:54.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:31:54.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:31:54.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:31:54.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:31:54.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:31:54.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:31:54.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:31:54.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:31:54.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:31:54.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:31:54.957 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.957 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.957 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.957 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:54.958 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:54.958 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 125.313252072, 2024-09-18T11:31:54.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:31:54.958 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.958 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.958 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.958 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:54.958 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:54.958 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:31:54.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.958 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:54.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:54.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:54.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:54.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:31:54.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:31:54.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:31:54.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:31:54.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:31:54.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:31:54.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:31:54.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:31:54.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:31:54.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:31:54.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:54.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:31:54.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:31:54.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:31:54.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:31:54.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:31:54.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:31:54.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:31:54.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:31:54.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:31:54.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:31:54.963 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.963 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.963 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.963 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:54.963 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:54.964 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 125.313228772, 2024-09-18T11:31:54.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:31:54.964 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.964 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.964 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.964 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:54.964 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:54.964 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:31:54.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.964 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:54.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:54.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:31:54.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:54.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:31:54.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:31:54.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:31:54.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:31:54.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:31:54.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:31:54.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:31:54.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:31:54.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:31:54.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:31:54.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:54.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:31:54.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:31:54.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:31:54.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:31:54.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:31:54.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:31:54.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:31:54.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:31:54.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:31:54.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:31:54.969 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.969 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.969 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.969 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:54.969 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:54.969 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 125.31314316300001, 2024-09-18T11:31:54.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:31:54.969 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.970 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.970 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.970 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:54.970 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:54.970 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:31:54.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.970 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:54.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:54.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:54.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:54.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:31:54.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:31:54.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:31:54.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:31:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:31:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:31:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:31:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:31:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:31:54.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:31:54.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:54.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:31:54.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:31:54.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:31:54.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:31:54.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:31:54.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:31:54.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:31:54.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:31:54.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:31:54.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:31:54.974 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.975 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.975 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.975 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:54.975 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:54.975 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 125.31313219800001, 2024-09-18T11:31:54.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:31:54.975 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.975 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.975 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.975 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:54.975 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:54.976 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:31:54.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.976 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:54.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:54.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:54.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:54.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:31:54.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:31:54.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:31:54.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:31:54.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:31:54.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:31:54.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:31:54.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:31:54.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:31:54.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:31:54.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:54.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:31:54.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:31:54.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:31:54.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:31:54.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:31:54.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:31:54.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:31:54.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:31:54.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:31:54.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:31:54.981 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.981 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.981 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.981 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:54.981 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:54.981 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 125.31312103, 2024-09-18T11:31:54.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:31:54.982 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.982 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.982 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.982 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:54.982 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:54.982 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:31:54.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.982 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:54.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:54.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:54.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:54.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:31:54.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:31:54.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:31:54.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:31:54.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:31:54.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:31:54.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:31:54.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:31:54.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:31:54.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:31:54.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:54.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:31:54.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:31:54.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:31:54.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:31:54.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:31:54.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:31:54.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:31:54.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:31:54.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:31:54.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:31:54.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.987 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.987 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:54.987 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:54.987 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 125.313055701, 2024-09-18T11:31:54.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:31:54.988 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.988 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.988 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.988 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:54.988 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:54.988 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:31:54.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.988 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:54.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:54.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:31:54.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:54.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:31:54.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:31:54.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:31:54.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:31:54.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:31:54.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:31:54.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:31:54.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:31:54.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:31:54.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:31:54.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:54.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:31:54.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:31:54.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:31:54.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:31:54.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:31:54.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:31:54.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:31:54.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:31:54.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:31:54.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:31:54.993 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.993 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.993 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.993 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:54.994 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:54.994 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 125.302893723, 2024-09-18T11:31:54.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:31:54.994 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:54.994 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:54.994 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:54.994 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:54.994 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:54.994 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:31:54.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.994 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:54.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:54.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:54.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:54.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:54.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:54.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:54.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:31:54.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:54.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:54.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:31:54.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:31:54.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:54.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:31:54.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:31:54.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:54.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:31:54.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:31:54.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:54.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:31:54.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:31:54.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:54.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:31:54.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:31:54.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:54.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:31:54.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:31:54.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:54.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:31:54.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:31:54.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:54.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:31:54.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:31:54.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:54.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:31:54.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:31:54.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:54.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:54.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:54.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:31:54.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:31:54.999 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.999 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:54.999 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:54.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.000 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:55.000 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:55.000 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 125.30280124399999, 2024-09-18T11:31:55.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:31:55.000 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:55.000 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:55.000 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:55.000 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:55.000 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:55.000 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:31:55.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.000 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:55.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:55.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:55.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:55.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:55.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:55.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:55.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:55.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:55.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:55.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:55.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:31:55.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:31:55.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:55.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:31:55.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:31:55.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:31:55.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:31:55.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:55.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:31:55.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:31:55.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:55.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:31:55.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:31:55.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:55.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:31:55.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:31:55.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:55.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:31:55.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:31:55.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:55.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:31:55.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:31:55.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:55.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:31:55.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:31:55.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:55.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:31:55.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:31:55.006 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:55.006 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:55.006 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:55.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.006 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1788 4.8 e60/22 4:1898b15d:::smithi11939506-25:head v 60'12, mlcod=58'9)", 2024-09-18T11:31:55.006 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:55.006 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 71.970395754999998, 2024-09-18T11:31:55.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058348599999999999, 2024-09-18T11:31:55.006 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:55.006 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:55.006 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:55.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:55.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:55.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:55.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:55.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:55.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:55.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:55.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916949+0000", 2024-09-18T11:31:55.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:55.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:55.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917602+0000", 2024-09-18T11:31:55.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00065337100000000003 2024-09-18T11:31:55.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:55.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917621+0000", 2024-09-18T11:31:55.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9287e-05 2024-09-18T11:31:55.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917629+0000", 2024-09-18T11:31:55.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3780000000000003e-06 2024-09-18T11:31:55.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:55.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917677+0000", 2024-09-18T11:31:55.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8068000000000001e-05 2024-09-18T11:31:55.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:55.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917699+0000", 2024-09-18T11:31:55.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2149000000000001e-05 2024-09-18T11:31:55.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:55.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922747+0000", 2024-09-18T11:31:55.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050477609999999996 2024-09-18T11:31:55.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:55.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922785+0000", 2024-09-18T11:31:55.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8726000000000002e-05 2024-09-18T11:31:55.011 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:55.011 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:55.011 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:55.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.011 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:58 4.3 e91/67 4:c70c3162:::smithi11939506-35:18 v 91'101, mlcod=86'95)", 2024-09-18T11:31:55.011 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:55.012 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.871737531999997, 2024-09-18T11:31:55.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055647500000000003, 2024-09-18T11:31:55.012 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:55.012 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:55.012 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:55.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:55.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:55.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:55.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:55.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:55.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:55.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:55.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015608+0000", 2024-09-18T11:31:55.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:55.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:55.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015614+0000", 2024-09-18T11:31:55.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5310000000000001e-06 2024-09-18T11:31:55.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:55.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015622+0000", 2024-09-18T11:31:55.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0809999999999994e-06 2024-09-18T11:31:55.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015627+0000", 2024-09-18T11:31:55.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8439999999999997e-06 2024-09-18T11:31:55.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:55.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016003+0000", 2024-09-18T11:31:55.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037652999999999997 2024-09-18T11:31:55.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:55.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016024+0000", 2024-09-18T11:31:55.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0780000000000001e-05 2024-09-18T11:31:55.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:55.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021141+0000", 2024-09-18T11:31:55.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051166639999999999 2024-09-18T11:31:55.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:55.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021174+0000", 2024-09-18T11:31:55.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2839000000000002e-05 2024-09-18T11:31:55.016 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:55.016 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:55.016 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:55.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.016 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3209 4.0 e109/22 4:019b6e09:::smithi11939506-40:head v 109'149, mlcod=106'146)", 2024-09-18T11:31:55.016 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:31:55.016 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.787960392999999, 2024-09-18T11:31:55.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063064039999999998, 2024-09-18T11:31:55.016 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:55.017 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:55.017 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:55.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:55.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:31:55.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:55.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:55.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:31:55.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:55.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:55.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099385+0000", 2024-09-18T11:31:55.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:55.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:55.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099951+0000", 2024-09-18T11:31:55.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056606600000000005 2024-09-18T11:31:55.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:55.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099964+0000", 2024-09-18T11:31:55.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2918999999999999e-05 2024-09-18T11:31:55.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099972+0000", 2024-09-18T11:31:55.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2300000000000002e-06 2024-09-18T11:31:55.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:55.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.100026+0000", 2024-09-18T11:31:55.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4456999999999997e-05 2024-09-18T11:31:55.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:55.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.100046+0000", 2024-09-18T11:31:55.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0417000000000001e-05 2024-09-18T11:31:55.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:55.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.105653+0000", 2024-09-18T11:31:55.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005606214 2024-09-18T11:31:55.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:55.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:55.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:55.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.105692+0000", 2024-09-18T11:31:55.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9671e-05 2024-09-18T11:31:55.021 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:55.021 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:55.021 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:55.022 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:55.022 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:55.022 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:55.023 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:31:55.024 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:55.022+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:55.123 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:55.124 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:55.124 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:55.124 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:55.124 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:55.123+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:55.132 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:31:55.170 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-18T11:31:55.170 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T11:31:55.171 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_0 2024-09-18T11:31:55.171 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:31:55.179 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:55.178+0000 7f3157e90640 -1 osd.4 124 handle_osd_map injecting map crc failure 2024-09-18T11:31:55.181 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is unset 2024-09-18T11:31:55.183 INFO:tasks.rados.rados.0.smithi119.stdout:1645: done (1 left) 2024-09-18T11:31:55.183 INFO:tasks.rados.rados.0.smithi119.stdout:1646: done (0 left) 2024-09-18T11:31:55.183 INFO:tasks.rados.rados.0.smithi119.stdout:1647: read oid 5 snap -1 2024-09-18T11:31:55.183 INFO:tasks.rados.rados.0.smithi119.stdout:1647: expect deleted 2024-09-18T11:31:55.183 INFO:tasks.rados.rados.0.smithi119.stdout:1648: read oid 47 snap -1 2024-09-18T11:31:55.183 INFO:tasks.rados.rados.0.smithi119.stdout:1648: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T11:31:55.183 INFO:tasks.rados.rados.0.smithi119.stdout:1649: snap_remove snap 23 2024-09-18T11:31:55.225 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:55.224+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:55.277 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:55.277 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:55.278 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:55.278 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:55.278 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:55.288 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:31:55.326 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:55.325+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:55.389 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:55.389 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:55.389 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:55.389 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [] 2024-09-18T11:31:55.389 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:55.397 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:31:55.427 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:55.426+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:55.452 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:31:55.453 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:31:55.464 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-09-18T11:31:55.179604+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":388749020,"num_objects":254,"num_object_clones":83,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":9,"num_read":4157,"num_read_kb":4123960,"num_write":2132,"num_write_kb":599535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":63687773,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":71,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3542,"ondisk_log_size":3542,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":2581928,"kb_used_data":483772,"kb_used_omap":11,"kb_used_meta":2098036,"kb_avail":658020952,"statfs":{"total":676457349120,"available":673813454848,"internally_reserved":0,"allocated":495382528,"data_stored":608194227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2148389224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19086194,"num_objects":14,"num_object_clones":9,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":187,"num_read_kb":214338,"num_write":67,"num_write_kb":19801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":5,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.017374"},"pg_stats":[{"pgid":"4.f","version":"119'275","reported_seq":596,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.163852+0000","last_change":"2024-09-18T11:31:46.136778+0000","last_active":"2024-09-18T11:31:51.163852+0000","last_peered":"2024-09-18T11:31:51.163852+0000","last_clean":"2024-09-18T11:31:51.163852+0000","last_became_active":"2024-09-18T11:30:02.074391+0000","last_became_peered":"2024-09-18T11:30:02.074391+0000","last_unstale":"2024-09-18T11:31:51.163852+0000","last_undegraded":"2024-09-18T11:31:51.163852+0000","last_fullsized":"2024-09-18T11:31:51.163852+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"92'190","last_scrub_stamp":"2024-09-18T11:31:17.744924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:17.744924+0000","objects_scrubbed":10,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:40.468788+0000","scrub_duration":0.002417757,"objects_trimmed":0,"snaptrim_duration":0.0045641789999999998,"stat_sum":{"num_bytes":22445538,"num_objects":12,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":1,"num_read":277,"num_read_kb":285913,"num_write":146,"num_write_kb":39636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":6,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"35","length":"1"},{"start":"3f","length":"1"}]},{"pgid":"4.e","version":"103'136","reported_seq":459,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.191490+0000","last_change":"2024-09-18T11:31:46.134511+0000","last_active":"2024-09-18T11:31:51.191490+0000","last_peered":"2024-09-18T11:31:51.191490+0000","last_clean":"2024-09-18T11:31:51.191490+0000","last_became_active":"2024-09-18T11:30:02.073185+0000","last_became_peered":"2024-09-18T11:30:02.073185+0000","last_unstale":"2024-09-18T11:31:51.191490+0000","last_undegraded":"2024-09-18T11:31:51.191490+0000","last_fullsized":"2024-09-18T11:31:51.191490+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"92'121","last_scrub_stamp":"2024-09-18T11:31:20.234551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:20.234551+0000","objects_scrubbed":5,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:32.824082+0000","scrub_duration":0.0040340640000000004,"objects_trimmed":0,"snaptrim_duration":0.00031780900000000001,"stat_sum":{"num_bytes":8353046,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":237,"num_read_kb":263752,"num_write":48,"num_write_kb":12076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"35","length":"1"},{"start":"3f","length":"1"}]},{"pgid":"4.d","version":"120'224","reported_seq":633,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-18T11:31:52.204215+0000","last_change":"2024-09-18T11:31:46.135621+0000","last_active":"2024-09-18T11:31:52.204215+0000","last_peered":"2024-09-18T11:31:52.204215+0000","last_clean":"2024-09-18T11:31:52.204215+0000","last_became_active":"2024-09-18T11:30:02.334498+0000","last_became_peered":"2024-09-18T11:30:02.334498+0000","last_unstale":"2024-09-18T11:31:52.204215+0000","last_undegraded":"2024-09-18T11:31:52.204215+0000","last_fullsized":"2024-09-18T11:31:52.204215+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'144","last_scrub_stamp":"2024-09-18T11:31:11.112252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:11.112252+0000","objects_scrubbed":10,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:20.999892+0000","scrub_duration":0.0033897469999999998,"objects_trimmed":0,"snaptrim_duration":0.00127363,"stat_sum":{"num_bytes":20735487,"num_objects":10,"num_object_clones":5,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":501,"num_read_kb":633254,"num_write":104,"num_write_kb":29112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":7,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"35","length":"1"},{"start":"3f","length":"1"}]},{"pgid":"4.c","version":"117'96","reported_seq":371,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.191461+0000","last_change":"2024-09-18T11:31:46.137885+0000","last_active":"2024-09-18T11:31:51.191461+0000","last_peered":"2024-09-18T11:31:51.191461+0000","last_clean":"2024-09-18T11:31:51.191461+0000","last_became_active":"2024-09-18T11:30:02.073883+0000","last_became_peered":"2024-09-18T11:30:02.073883+0000","last_unstale":"2024-09-18T11:31:51.191461+0000","last_undegraded":"2024-09-18T11:31:51.191461+0000","last_fullsized":"2024-09-18T11:31:51.191461+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"101'76","last_scrub_stamp":"2024-09-18T11:31:29.293475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:29.293475+0000","objects_scrubbed":4,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:53.466013+0000","scrub_duration":0.002355139,"objects_trimmed":0,"snaptrim_duration":0.0037347359999999998,"stat_sum":{"num_bytes":11749681,"num_objects":5,"num_object_clones":3,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":89,"num_read_kb":104276,"num_write":39,"num_write_kb":9469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"35","length":"1"},{"start":"3f","length":"1"}]},{"pgid":"4.b","version":"122'379","reported_seq":777,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-18T11:31:52.242465+0000","last_change":"2024-09-18T11:31:46.142266+0000","last_active":"2024-09-18T11:31:52.242465+0000","last_peered":"2024-09-18T11:31:52.242465+0000","last_clean":"2024-09-18T11:31:52.242465+0000","last_became_active":"2024-09-18T11:30:50.333230+0000","last_became_peered":"2024-09-18T11:30:50.333230+0000","last_unstale":"2024-09-18T11:31:52.242465+0000","last_undegraded":"2024-09-18T11:31:52.242465+0000","last_fullsized":"2024-09-18T11:31:52.242465+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'286","last_scrub_stamp":"2024-09-18T11:31:21.818375+0000","last_deep_scrub":"95'286","last_deep_scrub_stamp":"2024-09-18T11:31:21.818375+0000","last_clean_scrub_stamp":"2024-09-18T11:31:21.818375+0000","objects_scrubbed":11,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:51.543117+0000","scrub_duration":0.044204375999999997,"objects_trimmed":0,"snaptrim_duration":0.010122506,"stat_sum":{"num_bytes":39734331,"num_objects":17,"num_object_clones":11,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":403,"num_read_kb":431749,"num_write":183,"num_write_kb":48821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":17763749,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":7,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"35","length":"1"},{"start":"3f","length":"1"}]},{"pgid":"4.a","version":"122'191","reported_seq":503,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-18T11:31:52.203831+0000","last_change":"2024-09-18T11:31:46.250689+0000","last_active":"2024-09-18T11:31:52.203831+0000","last_peered":"2024-09-18T11:31:52.203831+0000","last_clean":"2024-09-18T11:31:52.203831+0000","last_became_active":"2024-09-18T11:30:02.075538+0000","last_became_peered":"2024-09-18T11:30:02.075538+0000","last_unstale":"2024-09-18T11:31:52.203831+0000","last_undegraded":"2024-09-18T11:31:52.203831+0000","last_fullsized":"2024-09-18T11:31:52.203831+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'152","last_scrub_stamp":"2024-09-18T11:31:30.146842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:30.146842+0000","objects_scrubbed":6,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:43.709866+0000","scrub_duration":0.0032260840000000002,"objects_trimmed":0,"snaptrim_duration":0.003571051,"stat_sum":{"num_bytes":19472399,"num_objects":9,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":249,"num_read_kb":363649,"num_write":83,"num_write_kb":23249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":6,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"35","length":"1"},{"start":"3f","length":"1"}]},{"pgid":"4.9","version":"111'74","reported_seq":348,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.191324+0000","last_change":"2024-09-18T11:31:46.134817+0000","last_active":"2024-09-18T11:31:51.191324+0000","last_peered":"2024-09-18T11:31:51.191324+0000","last_clean":"2024-09-18T11:31:51.191324+0000","last_became_active":"2024-09-18T11:30:02.074414+0000","last_became_peered":"2024-09-18T11:30:02.074414+0000","last_unstale":"2024-09-18T11:31:51.191324+0000","last_undegraded":"2024-09-18T11:31:51.191324+0000","last_fullsized":"2024-09-18T11:31:51.191324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'45","last_scrub_stamp":"2024-09-18T11:31:08.104353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:08.104353+0000","objects_scrubbed":3,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:23.202655+0000","scrub_duration":0.0019130519999999999,"objects_trimmed":0,"snaptrim_duration":0.00067337400000000002,"stat_sum":{"num_bytes":6007124,"num_objects":3,"num_object_clones":2,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":91211,"num_write":36,"num_write_kb":10894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"35","length":"1"},{"start":"3f","length":"1"}]},{"pgid":"4.8","version":"121'54","reported_seq":308,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.306215+0000","last_change":"2024-09-18T11:31:46.150613+0000","last_active":"2024-09-18T11:31:51.306215+0000","last_peered":"2024-09-18T11:31:51.306215+0000","last_clean":"2024-09-18T11:31:51.306215+0000","last_became_active":"2024-09-18T11:30:02.076172+0000","last_became_peered":"2024-09-18T11:30:02.076172+0000","last_unstale":"2024-09-18T11:31:51.306215+0000","last_undegraded":"2024-09-18T11:31:51.306215+0000","last_fullsized":"2024-09-18T11:31:51.306215+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'44","last_scrub_stamp":"2024-09-18T11:31:29.773113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:29.773113+0000","objects_scrubbed":2,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:34.758090+0000","scrub_duration":0.0016956790000000001,"objects_trimmed":0,"snaptrim_duration":0.0056871730000000002,"stat_sum":{"num_bytes":8572540,"num_objects":4,"num_object_clones":3,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":36,"num_read_kb":31271,"num_write":25,"num_write_kb":6824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"35","length":"1"},{"start":"3f","length":"1"}]},{"pgid":"4.5","version":"121'247","reported_seq":577,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.306141+0000","last_change":"2024-09-18T11:31:46.256715+0000","last_active":"2024-09-18T11:31:51.306141+0000","last_peered":"2024-09-18T11:31:51.306141+0000","last_clean":"2024-09-18T11:31:51.306141+0000","last_became_active":"2024-09-18T11:30:02.074697+0000","last_became_peered":"2024-09-18T11:30:02.074697+0000","last_unstale":"2024-09-18T11:31:51.306141+0000","last_undegraded":"2024-09-18T11:31:51.306141+0000","last_fullsized":"2024-09-18T11:31:51.306141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'148","last_scrub_stamp":"2024-09-18T11:31:08.098466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:08.098466+0000","objects_scrubbed":9,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:36.120154+0000","scrub_duration":0.0023687140000000001,"objects_trimmed":0,"snaptrim_duration":0.009560018,"stat_sum":{"num_bytes":21089784,"num_objects":13,"num_object_clones":9,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":286,"num_read_kb":299223,"num_write":115,"num_write_kb":28971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":9,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"35","length":"1"},{"start":"3f","length":"1"}]},{"pgid":"1.0","version":"16'72","reported_seq":228,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.192216+0000","last_change":"2024-09-18T11:30:57.650616+0000","last_active":"2024-09-18T11:31:51.192216+0000","last_peered":"2024-09-18T11:31:51.192216+0000","last_clean":"2024-09-18T11:31:51.192216+0000","last_became_active":"2024-09-18T11:30:50.741897+0000","last_became_peered":"2024-09-18T11:30:50.741897+0000","last_unstale":"2024-09-18T11:31:51.192216+0000","last_undegraded":"2024-09-18T11:31:51.192216+0000","last_fullsized":"2024-09-18T11:31:51.192216+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'72","last_scrub_stamp":"2024-09-18T11:30:57.650532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_clean_scrub_stamp":"2024-09-18T11:30:57.650532+0000","objects_scrubbed":2,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:22.503050+0000","scrub_duration":0.0016372979999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":127,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.159152+0000","last_change":"2024-09-18T11:31:06.619925+0000","last_active":"2024-09-18T11:31:51.159152+0000","last_peered":"2024-09-18T11:31:51.159152+0000","last_clean":"2024-09-18T11:31:51.159152+0000","last_became_active":"2024-09-18T11:29:50.028407+0000","last_became_peered":"2024-09-18T11:29:50.028407+0000","last_unstale":"2024-09-18T11:31:51.159152+0000","last_undegraded":"2024-09-18T11:31:51.159152+0000","last_fullsized":"2024-09-18T11:31:51.159152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'1","last_scrub_stamp":"2024-09-18T11:31:06.619850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:31:06.619850+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:30.529219+0000","scrub_duration":0.0016216189999999999,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"120'174","reported_seq":301,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.165535+0000","last_change":"2024-09-18T11:31:28.795160+0000","last_active":"2024-09-18T11:31:51.165535+0000","last_peered":"2024-09-18T11:31:51.165535+0000","last_clean":"2024-09-18T11:31:51.165535+0000","last_became_active":"2024-09-18T11:30:01.077612+0000","last_became_peered":"2024-09-18T11:30:01.077612+0000","last_unstale":"2024-09-18T11:31:51.165535+0000","last_undegraded":"2024-09-18T11:31:51.165535+0000","last_fullsized":"2024-09-18T11:31:51.165535+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'157","last_scrub_stamp":"2024-09-18T11:31:28.795082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:31:28.795082+0000","objects_scrubbed":30,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:33.013075+0000","scrub_duration":0.004667582,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38465056,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":134,"num_read_kb":3253,"num_write":193,"num_write_kb":69569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"117'123","reported_seq":423,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.163741+0000","last_change":"2024-09-18T11:31:46.139151+0000","last_active":"2024-09-18T11:31:51.163741+0000","last_peered":"2024-09-18T11:31:51.163741+0000","last_clean":"2024-09-18T11:31:51.163741+0000","last_became_active":"2024-09-18T11:30:50.743510+0000","last_became_peered":"2024-09-18T11:30:50.743510+0000","last_unstale":"2024-09-18T11:31:51.163741+0000","last_undegraded":"2024-09-18T11:31:51.163741+0000","last_fullsized":"2024-09-18T11:31:51.163741+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'66","last_scrub_stamp":"2024-09-18T11:31:07.766611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:07.766611+0000","objects_scrubbed":5,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:11.299507+0000","scrub_duration":0.0022223149999999999,"objects_trimmed":0,"snaptrim_duration":0.0068829010000000003,"stat_sum":{"num_bytes":15748417,"num_objects":7,"num_object_clones":5,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":123,"num_read_kb":118568,"num_write":56,"num_write_kb":13991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":2949092,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"35","length":"1"},{"start":"3f","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":125,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.192192+0000","last_change":"2024-09-18T11:31:09.752752+0000","last_active":"2024-09-18T11:31:51.192192+0000","last_peered":"2024-09-18T11:31:51.192192+0000","last_clean":"2024-09-18T11:31:51.192192+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:31:51.192192+0000","last_undegraded":"2024-09-18T11:31:51.192192+0000","last_fullsized":"2024-09-18T11:31:51.192192+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:31:09.752649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:31:09.752649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:10.252850+0000","scrub_duration":0.0020126839999999998,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"119'161","reported_seq":381,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.191385+0000","last_change":"2024-09-18T11:31:33.251215+0000","last_active":"2024-09-18T11:31:51.191385+0000","last_peered":"2024-09-18T11:31:51.191385+0000","last_clean":"2024-09-18T11:31:51.191385+0000","last_became_active":"2024-09-18T11:30:50.329599+0000","last_became_peered":"2024-09-18T11:30:50.329599+0000","last_unstale":"2024-09-18T11:31:51.191385+0000","last_undegraded":"2024-09-18T11:31:51.191385+0000","last_fullsized":"2024-09-18T11:31:51.191385+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'145","last_scrub_stamp":"2024-09-18T11:31:33.251143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:31:33.251143+0000","objects_scrubbed":28,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:33.814152+0000","scrub_duration":0.0067874609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39129522,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":138,"num_read_kb":9923,"num_write":172,"num_write_kb":63575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":20418189,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"117'239","reported_seq":608,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.163862+0000","last_change":"2024-09-18T11:31:46.149602+0000","last_active":"2024-09-18T11:31:51.163862+0000","last_peered":"2024-09-18T11:31:51.163862+0000","last_clean":"2024-09-18T11:31:51.163862+0000","last_became_active":"2024-09-18T11:30:02.076465+0000","last_became_peered":"2024-09-18T11:30:02.076465+0000","last_unstale":"2024-09-18T11:31:51.163862+0000","last_undegraded":"2024-09-18T11:31:51.163862+0000","last_fullsized":"2024-09-18T11:31:51.163862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"92'140","last_scrub_stamp":"2024-09-18T11:31:18.755540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:18.755540+0000","objects_scrubbed":8,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:28.845667+0000","scrub_duration":0.0038425999999999998,"objects_trimmed":0,"snaptrim_duration":0.017317078999999999,"stat_sum":{"num_bytes":24647186,"num_objects":15,"num_object_clones":10,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":386,"num_read_kb":394674,"num_write":118,"num_write_kb":30689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":9,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"35","length":"1"},{"start":"3f","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.194579+0000","last_change":"2024-09-18T11:31:51.194045+0000","last_active":"2024-09-18T11:31:51.194579+0000","last_peered":"2024-09-18T11:31:51.194579+0000","last_clean":"2024-09-18T11:31:51.194579+0000","last_became_active":"2024-09-18T11:31:51.193770+0000","last_became_peered":"2024-09-18T11:31:51.193770+0000","last_unstale":"2024-09-18T11:31:51.194579+0000","last_undegraded":"2024-09-18T11:31:51.194579+0000","last_fullsized":"2024-09-18T11:31:51.194579+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:31:19.127335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:31:19.127335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:19.511895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"117'123","reported_seq":246,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.163565+0000","last_change":"2024-09-18T11:31:06.801577+0000","last_active":"2024-09-18T11:31:51.163565+0000","last_peered":"2024-09-18T11:31:51.163565+0000","last_clean":"2024-09-18T11:31:51.163565+0000","last_became_active":"2024-09-18T11:30:01.071435+0000","last_became_peered":"2024-09-18T11:30:01.071435+0000","last_unstale":"2024-09-18T11:31:51.163565+0000","last_undegraded":"2024-09-18T11:31:51.163565+0000","last_fullsized":"2024-09-18T11:31:51.163565+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'83","last_scrub_stamp":"2024-09-18T11:31:06.801507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:31:06.801507+0000","objects_scrubbed":15,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:22.446373+0000","scrub_duration":0.0021766860000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19989790,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":93,"num_read_kb":939,"num_write":131,"num_write_kb":40496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"120'180","reported_seq":484,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.163732+0000","last_change":"2024-09-18T11:31:46.132503+0000","last_active":"2024-09-18T11:31:51.163732+0000","last_peered":"2024-09-18T11:31:51.163732+0000","last_clean":"2024-09-18T11:31:51.163732+0000","last_became_active":"2024-09-18T11:30:02.334751+0000","last_became_peered":"2024-09-18T11:30:02.334751+0000","last_unstale":"2024-09-18T11:31:51.163732+0000","last_undegraded":"2024-09-18T11:31:51.163732+0000","last_fullsized":"2024-09-18T11:31:51.163732+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'169","last_scrub_stamp":"2024-09-18T11:31:29.756257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:29.756257+0000","objects_scrubbed":7,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:31.853717+0000","scrub_duration":0.002423396,"objects_trimmed":0,"snaptrim_duration":0.00037696700000000002,"stat_sum":{"num_bytes":17361379,"num_objects":8,"num_object_clones":5,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":209,"num_read_kb":247285,"num_write":89,"num_write_kb":24684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":6,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"35","length":"1"},{"start":"3f","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":128,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.159081+0000","last_change":"2024-09-18T11:31:16.657820+0000","last_active":"2024-09-18T11:31:51.159081+0000","last_peered":"2024-09-18T11:31:51.159081+0000","last_clean":"2024-09-18T11:31:51.159081+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:31:51.159081+0000","last_undegraded":"2024-09-18T11:31:51.159081+0000","last_fullsized":"2024-09-18T11:31:51.159081+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"18'2","last_scrub_stamp":"2024-09-18T11:31:16.657749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:31:16.657749+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:30.076694+0000","scrub_duration":0.00143407,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"115'127","reported_seq":335,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.191412+0000","last_change":"2024-09-18T11:31:09.149266+0000","last_active":"2024-09-18T11:31:51.191412+0000","last_peered":"2024-09-18T11:31:51.191412+0000","last_clean":"2024-09-18T11:31:51.191412+0000","last_became_active":"2024-09-18T11:30:50.741594+0000","last_became_peered":"2024-09-18T11:30:50.741594+0000","last_unstale":"2024-09-18T11:31:51.191412+0000","last_undegraded":"2024-09-18T11:31:51.191412+0000","last_fullsized":"2024-09-18T11:31:51.191412+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'103","last_scrub_stamp":"2024-09-18T11:31:09.149191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:31:09.149191+0000","objects_scrubbed":26,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:33.612849+0000","scrub_duration":0.0034665379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19141892,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":105,"num_read_kb":4773,"num_write":136,"num_write_kb":48190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":16290641,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"111'89","reported_seq":365,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.163598+0000","last_change":"2024-09-18T11:31:46.143892+0000","last_active":"2024-09-18T11:31:51.163598+0000","last_peered":"2024-09-18T11:31:51.163598+0000","last_clean":"2024-09-18T11:31:51.163598+0000","last_became_active":"2024-09-18T11:30:02.076874+0000","last_became_peered":"2024-09-18T11:30:02.076874+0000","last_unstale":"2024-09-18T11:31:51.163598+0000","last_undegraded":"2024-09-18T11:31:51.163598+0000","last_fullsized":"2024-09-18T11:31:51.163598+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'70","last_scrub_stamp":"2024-09-18T11:31:09.807739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:09.807739+0000","objects_scrubbed":3,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:22.006274+0000","scrub_duration":0.002685074,"objects_trimmed":0,"snaptrim_duration":0.011571985999999999,"stat_sum":{"num_bytes":3529470,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":102005,"num_write":47,"num_write_kb":13042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"35","length":"1"},{"start":"3f","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":139,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.163750+0000","last_change":"2024-09-18T11:31:16.777136+0000","last_active":"2024-09-18T11:31:51.163750+0000","last_peered":"2024-09-18T11:31:51.163750+0000","last_clean":"2024-09-18T11:31:51.163750+0000","last_became_active":"2024-09-18T11:30:50.331026+0000","last_became_peered":"2024-09-18T11:30:50.331026+0000","last_unstale":"2024-09-18T11:31:51.163750+0000","last_undegraded":"2024-09-18T11:31:51.163750+0000","last_fullsized":"2024-09-18T11:31:51.163750+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:31:16.777069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:31:16.777069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:23.123141+0000","scrub_duration":0.001722945,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"119'156","reported_seq":485,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.159133+0000","last_change":"2024-09-18T11:31:46.145245+0000","last_active":"2024-09-18T11:31:51.159133+0000","last_peered":"2024-09-18T11:31:51.159133+0000","last_clean":"2024-09-18T11:31:51.159133+0000","last_became_active":"2024-09-18T11:30:50.333722+0000","last_became_peered":"2024-09-18T11:30:50.333722+0000","last_unstale":"2024-09-18T11:31:51.159133+0000","last_undegraded":"2024-09-18T11:31:51.159133+0000","last_fullsized":"2024-09-18T11:31:51.159133+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'129","last_scrub_stamp":"2024-09-18T11:31:28.771630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:28.771630+0000","objects_scrubbed":9,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:47.590830+0000","scrub_duration":0.00269141,"objects_trimmed":0,"snaptrim_duration":0.00027041899999999999,"stat_sum":{"num_bytes":16096672,"num_objects":6,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":199,"num_read_kb":219524,"num_write":85,"num_write_kb":22803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":5737174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"35","length":"1"},{"start":"3f","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":136,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.192174+0000","last_change":"2024-09-18T11:31:16.761649+0000","last_active":"2024-09-18T11:31:51.192174+0000","last_peered":"2024-09-18T11:31:51.192174+0000","last_clean":"2024-09-18T11:31:51.192174+0000","last_became_active":"2024-09-18T11:30:50.327483+0000","last_became_peered":"2024-09-18T11:30:50.327483+0000","last_unstale":"2024-09-18T11:31:51.192174+0000","last_undegraded":"2024-09-18T11:31:51.192174+0000","last_fullsized":"2024-09-18T11:31:51.192174+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:31:16.761562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:31:16.761562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:26.978531+0000","scrub_duration":0.0012814949999999999,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"117'176","reported_seq":479,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.165632+0000","last_change":"2024-09-18T11:31:46.146536+0000","last_active":"2024-09-18T11:31:51.165632+0000","last_peered":"2024-09-18T11:31:51.165632+0000","last_clean":"2024-09-18T11:31:51.165632+0000","last_became_active":"2024-09-18T11:30:02.072165+0000","last_became_peered":"2024-09-18T11:30:02.072165+0000","last_unstale":"2024-09-18T11:31:51.165632+0000","last_undegraded":"2024-09-18T11:31:51.165632+0000","last_fullsized":"2024-09-18T11:31:51.165632+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'120","last_scrub_stamp":"2024-09-18T11:31:06.491290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:06.491290+0000","objects_scrubbed":8,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:16.762073+0000","scrub_duration":0.002837347,"objects_trimmed":0,"snaptrim_duration":0.0093151109999999992,"stat_sum":{"num_bytes":17146208,"num_objects":8,"num_object_clones":5,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":187,"num_read_kb":203718,"num_write":101,"num_write_kb":29095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":5,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"35","length":"1"},{"start":"3f","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":128,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.163749+0000","last_change":"2024-09-18T11:30:57.765542+0000","last_active":"2024-09-18T11:31:51.163749+0000","last_peered":"2024-09-18T11:31:51.163749+0000","last_clean":"2024-09-18T11:31:51.163749+0000","last_became_active":"2024-09-18T11:29:50.028674+0000","last_became_peered":"2024-09-18T11:29:50.028674+0000","last_unstale":"2024-09-18T11:31:51.163749+0000","last_undegraded":"2024-09-18T11:31:51.163749+0000","last_fullsized":"2024-09-18T11:31:51.163749+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:57.765475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:30:57.765475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:20.322582+0000","scrub_duration":0.001835167,"objects_trimmed":0,"snaptrim_duration":0.00031621200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"118'243","reported_seq":568,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.191431+0000","last_change":"2024-09-18T11:31:46.135427+0000","last_active":"2024-09-18T11:31:51.191431+0000","last_peered":"2024-09-18T11:31:51.191431+0000","last_clean":"2024-09-18T11:31:51.191431+0000","last_became_active":"2024-09-18T11:30:02.075529+0000","last_became_peered":"2024-09-18T11:30:02.075529+0000","last_unstale":"2024-09-18T11:31:51.191431+0000","last_undegraded":"2024-09-18T11:31:51.191431+0000","last_fullsized":"2024-09-18T11:31:51.191431+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'230","last_scrub_stamp":"2024-09-18T11:31:32.253118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:32.253118+0000","objects_scrubbed":9,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:49.207202+0000","scrub_duration":0.0038969130000000001,"objects_trimmed":0,"snaptrim_duration":0.001190057,"stat_sum":{"num_bytes":18743111,"num_objects":9,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":278,"num_read_kb":314945,"num_write":122,"num_write_kb":32719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"35","length":"1"},{"start":"3f","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":125,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-18T11:31:51.192297+0000","last_change":"2024-09-18T11:30:57.053072+0000","last_active":"2024-09-18T11:31:51.192297+0000","last_peered":"2024-09-18T11:31:51.192297+0000","last_clean":"2024-09-18T11:31:51.192297+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:31:51.192297+0000","last_undegraded":"2024-09-18T11:31:51.192297+0000","last_fullsized":"2024-09-18T11:31:51.192297+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:57.052994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:30:57.052994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:58.065434+0000","scrub_duration":0.0012785050000000001,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":271432373,"num_objects":133,"num_object_clones":83,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":9,"num_read":3621,"num_read_kb":4105017,"num_write":1397,"num_write_kb":376075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":26450015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":71,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":373469184,"data_stored":491152510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2882,"ondisk_log_size":2882,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":116726260,"num_objects":117,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":0,"num_read":470,"num_read_kb":18888,"num_write":632,"num_write_kb":221830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":36708830,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":177790976,"data_stored":177451074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4182,"internal_metadata":0},"log_size":585,"ondisk_log_size":585,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":323532,"kb_used_data":26172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94048308,"statfs":{"total":96636764160,"available":96305467392,"internally_reserved":0,"allocated":26800128,"data_stored":32554679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":1,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.46000000000000002,"15min":0.46500000000000002},"min":{"1min":0.33100000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.52000000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.51100000000000001,"15min":0.52000000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.60599999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.41599999999999998}]},{"osd":2,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41999999999999998,"15min":0.42099999999999999},"min":{"1min":0.32400000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.53100000000000003,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.42999999999999999,"15min":0.436},"min":{"1min":0.34000000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.47699999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.438}]},{"osd":3,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44,"15min":0.442},"min":{"1min":0.34300000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.52400000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.47699999999999998,"15min":0.48799999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.48299999999999998,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.34799999999999998}]},{"osd":4,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.44800000000000001,"15min":0.46000000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.48899999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.49199999999999999,"15min":0.502},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.57299999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.42899999999999999}]},{"osd":5,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.42899999999999999,"15min":0.443},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.58799999999999997,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.45900000000000002,"15min":0.47099999999999997},"min":{"1min":0.28699999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.68400000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.32200000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.51300000000000001,"15min":0.52700000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54700000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.49399999999999999,"15min":0.504},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.626,"5min":0.75,"15min":0.75},"last":0.375}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":343732,"kb_used_data":46372,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94028108,"statfs":{"total":96636764160,"available":96284782592,"internally_reserved":0,"allocated":47484928,"data_stored":61430230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":1,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46600000000000003,"15min":0.46899999999999997},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.77200000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.438},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.45100000000000001,"15min":0.45500000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.65500000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.57499999999999996}]},{"osd":2,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46100000000000002,"15min":0.46300000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.75600000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.443,"15min":0.443},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.55400000000000005}]},{"osd":3,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.503,"15min":0.51500000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59599999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66100000000000003,"5min":1.032,"15min":1.032},"last":0.59699999999999998}]},{"osd":4,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.38500000000000001,"15min":0.38800000000000001},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.48699999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.41399999999999998,"15min":0.41799999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.54700000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.626}]},{"osd":5,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.27800000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.42499999999999999,"15min":0.42699999999999999},"min":{"1min":0.29099999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60499999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.68999999999999995}]},{"osd":7,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.45400000000000001,"15min":0.46000000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.622,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47099999999999997,"15min":0.47399999999999998},"min":{"1min":0.32300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.69999999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":369368,"kb_used_data":67848,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94002472,"statfs":{"total":96636764160,"available":96258531328,"internally_reserved":0,"allocated":69476352,"data_stored":83109377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":1,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.34100000000000003}]},{"osd":2,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.29199999999999998}]},{"osd":3,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.316},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.41299999999999998}]},{"osd":4,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.32700000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.245},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.26000000000000001}]},{"osd":7,"last update":"Wed Sep 18 11:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.28199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":414788,"kb_used_data":113268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93957052,"statfs":{"total":96636764160,"available":96212021248,"internally_reserved":0,"allocated":115986432,"data_stored":144201294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.42799999999999999}]},{"osd":1,"last update":"Wed Sep 18 11:31:47 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.48899999999999999,"15min":0.504},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.54400000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.48799999999999999,"15min":0.5},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.57299999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.33800000000000002}]},{"osd":2,"last update":"Wed Sep 18 11:31:47 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.48599999999999999,"15min":0.49399999999999999},"min":{"1min":0.32200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68899999999999995,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.50700000000000001,"15min":0.52100000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59199999999999997,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.38200000000000001}]},{"osd":3,"last update":"Wed Sep 18 11:31:47 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.505,"15min":0.51800000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.63500000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.51500000000000001,"15min":0.53000000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.66700000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.40699999999999997}]},{"osd":5,"last update":"Wed Sep 18 11:31:47 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.45800000000000002,"15min":0.46800000000000003},"min":{"1min":0.27700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.65600000000000003,"5min":1.002,"15min":1.002},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.496,"15min":0.51300000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.63600000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.29599999999999999}]},{"osd":6,"last update":"Wed Sep 18 11:31:47 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.46200000000000002,"15min":0.47299999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.70699999999999996,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.49399999999999999,"15min":0.50700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.60999999999999999,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.311}]},{"osd":7,"last update":"Wed Sep 18 11:31:47 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.5,"15min":0.51200000000000001},"min":{"1min":0.27400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67700000000000005,"5min":0.75,"15min":0.75},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.53400000000000003,"15min":0.54800000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72999999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.42399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":382728,"kb_used_data":81208,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93989112,"statfs":{"total":96636764160,"available":96244850688,"internally_reserved":0,"allocated":83156992,"data_stored":91651924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.32400000000000001}]},{"osd":1,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.33700000000000002,"15min":0.34300000000000003},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51300000000000001,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.29099999999999998,"5min":0.35799999999999998,"15min":0.36899999999999999},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.40899999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.34599999999999997}]},{"osd":2,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.23499999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.45600000000000002,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.315,"5min":0.39100000000000001,"15min":0.40300000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.52000000000000002,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.29799999999999999}]},{"osd":4,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51100000000000001,"15min":0.51400000000000001},"min":{"1min":0.376,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.66700000000000004,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65700000000000003,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47199999999999998}]},{"osd":5,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.497,"15min":0.505},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.58999999999999997,"5min":0.752,"15min":0.752},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.54800000000000004,"15min":0.56100000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63600000000000001,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.44}]},{"osd":6,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.61799999999999999,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.55500000000000005,"15min":0.56499999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.65800000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.42999999999999999}]},{"osd":7,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.58699999999999997,"15min":0.60699999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.622,"5min":1.26,"15min":1.26},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.56699999999999995,"15min":0.57899999999999996},"min":{"1min":0.40999999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.65400000000000003,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.52900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":340620,"kb_used_data":43264,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94031220,"statfs":{"total":96636764160,"available":96287969280,"internally_reserved":0,"allocated":44302336,"data_stored":51719469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.252}]},{"osd":1,"last update":"Wed Sep 18 11:31:52 2024","interfaces":[{"interface":"back","average":{"1min":0.28999999999999998,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.46000000000000002,"5min":1.621,"15min":1.621},"last":0.312},{"interface":"front","average":{"1min":0.29199999999999998,"5min":0.38600000000000001,"15min":0.40200000000000002},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.40100000000000002,"5min":1.589,"15min":1.589},"last":0.36599999999999999}]},{"osd":3,"last update":"Wed Sep 18 11:31:52 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.36199999999999999,"15min":0.37},"min":{"1min":0.254,"5min":0.249,"15min":0.249},"max":{"1min":0.48299999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.30499999999999999,"5min":0.40100000000000002,"15min":0.41699999999999998},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.434,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.35199999999999998}]},{"osd":4,"last update":"Wed Sep 18 11:31:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.54200000000000004,"15min":0.55400000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76100000000000001,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.442,"5min":0.50700000000000001,"15min":0.51800000000000002},"min":{"1min":0.35799999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.67300000000000004,"5min":1.012,"15min":1.012},"last":0.53500000000000003}]},{"osd":5,"last update":"Wed Sep 18 11:31:52 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.53500000000000003,"15min":0.54300000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66200000000000003,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69899999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.57999999999999996}]},{"osd":6,"last update":"Wed Sep 18 11:31:52 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.53100000000000003,"15min":0.54300000000000004},"min":{"1min":0.34799999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.64800000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.55000000000000004,"15min":0.56599999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64200000000000002,"5min":1.165,"15min":1.165},"last":0.55900000000000005}]},{"osd":7,"last update":"Wed Sep 18 11:31:52 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.371,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68899999999999995,"5min":1.073,"15min":1.073},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.54000000000000004,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.65600000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.52100000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":407160,"kb_used_data":105640,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93964680,"statfs":{"total":96636764160,"available":96219832320,"internally_reserved":0,"allocated":108175360,"data_stored":143527254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Wed Sep 18 11:31:51 2024","interfaces":[{"interface":"back","average":{"1min":0.255,"5min":0.36199999999999999,"15min":0.38},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.434,"5min":1.333,"15min":1.333},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.313,"5min":0.40799999999999997,"15min":0.42399999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.44500000000000001,"5min":1.298,"15min":1.298},"last":0.30299999999999999}]},{"osd":3,"last update":"Wed Sep 18 11:31:51 2024","interfaces":[{"interface":"back","average":{"1min":0.26300000000000001,"5min":0.36299999999999999,"15min":0.379},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.41999999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.30099999999999999,"5min":0.35299999999999998,"15min":0.36099999999999999},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.495,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.28799999999999998}]},{"osd":4,"last update":"Wed Sep 18 11:31:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.51500000000000001,"15min":0.52500000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.73999999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.49299999999999999,"15min":0.505},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69199999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.50900000000000001}]},{"osd":5,"last update":"Wed Sep 18 11:31:51 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.504,"15min":0.51700000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.59399999999999997,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.436,"5min":0.51200000000000001,"15min":0.52400000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.58499999999999996,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.63700000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:31:51 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.52800000000000002,"15min":0.53600000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70999999999999996,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.52900000000000003,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.57799999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56499999999999995}]},{"osd":7,"last update":"Wed Sep 18 11:31:51 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.53100000000000003,"15min":0.54500000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.55900000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73999999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.48399999999999999}]}]},{"osd":0,"up_from":0,"seq":511101108227,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36782080,"data_stored":36708830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4182,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12357632,"data_stored":12334503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12357632,"data_stored":12334503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":46129152,"data_stored":46044442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":35090432,"data_stored":35026346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":23056384,"data_stored":23010273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12017664,"data_stored":11992177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25010176,"data_stored":27495860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":95109120,"data_stored":130799193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":31277056,"data_stored":39001172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36323328,"data_stored":45214021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":80183296,"data_stored":108780197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45740032,"data_stored":59709214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":34238464,"data_stored":48516657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":25587712,"data_stored":31636196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:31:55.467 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pgp_num 16 2024-09-18T11:31:55.497 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:55.498 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:55.498 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:55.498 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:55.507 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:31:55.548 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:55.547+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:55.629 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:55.628+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:55.647 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:55.648 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:55.648 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:55.648 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:55.648 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:55.657 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:31:55.720 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:31:55.730 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:55.729+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:55.758 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:55.758 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:55.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:55.758 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:55.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.758 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:262 4.9 e53/22 4:99feffe9:::smithi11939506-45:13 v 53'17, mlcod=51'14)", 2024-09-18T11:31:55.758 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:55.758 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.755837876000001, 2024-09-18T11:31:55.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013878679, 2024-09-18T11:31:55.759 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:55.759 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:55.759 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:55.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:55.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:55.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:55.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:55.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:55.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:55.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:55.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:55.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998997+0000", 2024-09-18T11:31:55.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1099999999999999e-06 2024-09-18T11:31:55.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:55.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999008+0000", 2024-09-18T11:31:55.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1321000000000001e-05 2024-09-18T11:31:55.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999015+0000", 2024-09-18T11:31:55.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3000000000000004e-06 2024-09-18T11:31:55.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999055+0000", 2024-09-18T11:31:55.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0058000000000002e-05 2024-09-18T11:31:55.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:55.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999070+0000", 2024-09-18T11:31:55.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4328000000000001e-05 2024-09-18T11:31:55.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:55.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012465+0000", 2024-09-18T11:31:55.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013395638 2024-09-18T11:31:55.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:55.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012870+0000", 2024-09-18T11:31:55.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040423400000000001 2024-09-18T11:31:55.763 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.763 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:55.763 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.763 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:205 4.b e53/22 4:dba5bd39:::smithi11939506-17:13 v 53'77, mlcod=51'74)", 2024-09-18T11:31:55.763 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:55.763 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.755788615, 2024-09-18T11:31:55.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013359586, 2024-09-18T11:31:55.763 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:55.763 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:55.764 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:55.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:55.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:55.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:55.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:55.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:55.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:55.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:55.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:55.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999048+0000", 2024-09-18T11:31:55.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3550000000000005e-06 2024-09-18T11:31:55.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:55.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999056+0000", 2024-09-18T11:31:55.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3259999999999996e-06 2024-09-18T11:31:55.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999063+0000", 2024-09-18T11:31:55.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6409999999999996e-06 2024-09-18T11:31:55.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999098+0000", 2024-09-18T11:31:55.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5203999999999997e-05 2024-09-18T11:31:55.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:55.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999115+0000", 2024-09-18T11:31:55.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6478999999999999e-05 2024-09-18T11:31:55.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:55.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012362+0000", 2024-09-18T11:31:55.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01324768 2024-09-18T11:31:55.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:55.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012400+0000", 2024-09-18T11:31:55.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.731e-05 2024-09-18T11:31:55.767 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.767 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:55.767 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.768 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:88 4.a e53/22 4:588debc2:::smithi11939506-18:13 v 53'45, mlcod=51'42)", 2024-09-18T11:31:55.768 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:55.768 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.753006192000001, 2024-09-18T11:31:55.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011307667, 2024-09-18T11:31:55.768 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:55.768 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:55.768 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:55.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:55.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:55.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:55.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:55.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:55.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001822+0000", 2024-09-18T11:31:55.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:55.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:55.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001827+0000", 2024-09-18T11:31:55.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.378e-06 2024-09-18T11:31:55.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:55.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001839+0000", 2024-09-18T11:31:55.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1225e-05 2024-09-18T11:31:55.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001844+0000", 2024-09-18T11:31:55.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2299999999999999e-06 2024-09-18T11:31:55.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001891+0000", 2024-09-18T11:31:55.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6961e-05 2024-09-18T11:31:55.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:55.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001906+0000", 2024-09-18T11:31:55.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5158e-05 2024-09-18T11:31:55.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:55.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013103+0000", 2024-09-18T11:31:55.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011197314 2024-09-18T11:31:55.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.772 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:55.772 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013130+0000", 2024-09-18T11:31:55.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6900999999999999e-05 2024-09-18T11:31:55.772 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.772 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:55.772 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.772 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:89 4.a e53/22 4:552523b6:::smithi11939506-38:13 v 53'47, mlcod=51'42)", 2024-09-18T11:31:55.772 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:55.772 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.752446972000001, 2024-09-18T11:31:55.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011327614, 2024-09-18T11:31:55.772 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:55.772 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:55.773 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:55.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:55.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:55.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:55.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:55.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:55.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002381+0000", 2024-09-18T11:31:55.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:55.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:55.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002397+0000", 2024-09-18T11:31:55.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5688e-05 2024-09-18T11:31:55.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:55.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002406+0000", 2024-09-18T11:31:55.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0229999999999998e-06 2024-09-18T11:31:55.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002415+0000", 2024-09-18T11:31:55.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8959999999999992e-06 2024-09-18T11:31:55.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012386+0000", 2024-09-18T11:31:55.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0099711450000000007 2024-09-18T11:31:55.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:55.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012405+0000", 2024-09-18T11:31:55.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9389e-05 2024-09-18T11:31:55.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:55.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013676+0000", 2024-09-18T11:31:55.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012708649999999999 2024-09-18T11:31:55.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:55.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013709+0000", 2024-09-18T11:31:55.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3207000000000002e-05 2024-09-18T11:31:55.776 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.777 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:55.777 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.777 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1645 4.7 4:e73ba68f:::smithi11939506-1:head [read 0~3983226,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:55.777 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:55.777 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 78.420909780000002, 2024-09-18T11:31:55.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010539468999999999, 2024-09-18T11:31:55.777 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:55.777 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:55.777 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:55.777 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:55.777 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:55.777 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1645 2024-09-18T11:31:55.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.778 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:55.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:55.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:55.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:55.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:55.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:55.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:55.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:55.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:55.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333921+0000", 2024-09-18T11:31:55.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8909999999999999e-06 2024-09-18T11:31:55.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.779 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:55.779 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333927+0000", 2024-09-18T11:31:55.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4229999999999996e-06 2024-09-18T11:31:55.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333932+0000", 2024-09-18T11:31:55.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8280000000000002e-06 2024-09-18T11:31:55.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342418+0000", 2024-09-18T11:31:55.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084862219999999999 2024-09-18T11:31:55.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:55.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342515+0000", 2024-09-18T11:31:55.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7287999999999994e-05 2024-09-18T11:31:55.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:55.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.344458+0000", 2024-09-18T11:31:55.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019431660000000001 2024-09-18T11:31:55.781 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.782 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:55.782 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.782 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:100 4.a e58/22 4:5975a5ec:::smithi11939506-36:18 v 58'55, mlcod=56'50)", 2024-09-18T11:31:55.782 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:55.782 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 76.481679643999996, 2024-09-18T11:31:55.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0088255, 2024-09-18T11:31:55.782 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:55.782 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:55.782 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:55.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:55.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:55.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:55.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:55.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:55.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:55.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:55.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:55.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273155+0000", 2024-09-18T11:31:55.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.212e-06 2024-09-18T11:31:55.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:55.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273164+0000", 2024-09-18T11:31:55.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7609999999999996e-06 2024-09-18T11:31:55.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273168+0000", 2024-09-18T11:31:55.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2819999999999998e-06 2024-09-18T11:31:55.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274162+0000", 2024-09-18T11:31:55.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099398600000000009 2024-09-18T11:31:55.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:55.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274180+0000", 2024-09-18T11:31:55.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8236e-05 2024-09-18T11:31:55.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:55.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281943+0000", 2024-09-18T11:31:55.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007763308 2024-09-18T11:31:55.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.786 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:55.786 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281975+0000", 2024-09-18T11:31:55.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1202999999999999e-05 2024-09-18T11:31:55.786 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.786 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:55.786 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.786 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1954 4.7 4:ef60946b:::smithi11939506-8:head [read 0~2890771,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:55.787 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:55.787 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.414928603999996, 2024-09-18T11:31:55.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091142180000000003, 2024-09-18T11:31:55.787 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:55.787 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:55.787 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:55.787 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:55.787 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:55.787 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1954 2024-09-18T11:31:55.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.787 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:55.787 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.787 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:55.787 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:55.787 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:55.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:55.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:55.788 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:55.788 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:55.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.788 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.788 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:55.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339903+0000", 2024-09-18T11:31:55.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7349999999999998e-06 2024-09-18T11:31:55.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:55.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339909+0000", 2024-09-18T11:31:55.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-09-18T11:31:55.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.789 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.789 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339913+0000", 2024-09-18T11:31:55.789 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7670000000000001e-06 2024-09-18T11:31:55.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339970+0000", 2024-09-18T11:31:55.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6988999999999999e-05 2024-09-18T11:31:55.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:55.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339975+0000", 2024-09-18T11:31:55.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7929999999999997e-06 2024-09-18T11:31:55.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.790 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.790 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.790 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346917+0000", 2024-09-18T11:31:55.790 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069426640000000003 2024-09-18T11:31:55.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:55.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.347026+0000", 2024-09-18T11:31:55.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108241 2024-09-18T11:31:55.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.791 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.791 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:55.791 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.349014+0000", 2024-09-18T11:31:55.791 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001988723 2024-09-18T11:31:55.791 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.791 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:55.791 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.792 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1956 4.6 4:67c1765a:::smithi11939506-30:head [write 705927~558502] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:55.792 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:55.792 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.402819664000006, 2024-09-18T11:31:55.792 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011813239, 2024-09-18T11:31:55.792 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:55.792 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:55.792 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:55.792 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:55.792 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:55.792 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1956 2024-09-18T11:31:55.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.792 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:55.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:55.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:55.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:55.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:55.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.793 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.793 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.793 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:55.793 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352007+0000", 2024-09-18T11:31:55.793 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:55.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:55.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352229+0000", 2024-09-18T11:31:55.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222024 2024-09-18T11:31:55.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.794 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:55.794 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352251+0000", 2024-09-18T11:31:55.794 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2512000000000001e-05 2024-09-18T11:31:55.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.794 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352260+0000", 2024-09-18T11:31:55.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7339999999999997e-06 2024-09-18T11:31:55.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352307+0000", 2024-09-18T11:31:55.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7061999999999998e-05 2024-09-18T11:31:55.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.795 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.795 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:55.795 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352408+0000", 2024-09-18T11:31:55.795 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101216 2024-09-18T11:31:55.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:55.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352550+0000", 2024-09-18T11:31:55.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014136199999999999 2024-09-18T11:31:55.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:55.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357684+0000", 2024-09-18T11:31:55.796 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051342499999999999 2024-09-18T11:31:55.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.796 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:55.796 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363736+0000", 2024-09-18T11:31:55.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006052577 2024-09-18T11:31:55.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:55.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363778+0000", 2024-09-18T11:31:55.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1989000000000001e-05 2024-09-18T11:31:55.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.797 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:55.797 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363822+0000", 2024-09-18T11:31:55.797 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3894999999999999e-05 2024-09-18T11:31:55.797 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.797 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:55.797 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.798 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2278 4.f 4:fc192b81:::smithi11939506-31:head [stat,delete] snapc 26=[26,25,24,1e,1c,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T11:31:55.798 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:55.798 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.371782451999998, 2024-09-18T11:31:55.798 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01018694, 2024-09-18T11:31:55.798 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:55.798 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:55.798 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:55.798 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:55.798 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:55.798 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2278 2024-09-18T11:31:55.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.798 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:55.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:55.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:55.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:55.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:55.799 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.799 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.799 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:55.799 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383044+0000", 2024-09-18T11:31:55.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:55.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:55.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383054+0000", 2024-09-18T11:31:55.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5969999999999998e-06 2024-09-18T11:31:55.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.800 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:55.800 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383071+0000", 2024-09-18T11:31:55.800 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7353000000000001e-05 2024-09-18T11:31:55.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383081+0000", 2024-09-18T11:31:55.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9170000000000001e-06 2024-09-18T11:31:55.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383150+0000", 2024-09-18T11:31:55.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8503999999999993e-05 2024-09-18T11:31:55.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.801 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:55.801 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383158+0000", 2024-09-18T11:31:55.801 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5089999999999993e-06 2024-09-18T11:31:55.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385396+0000", 2024-09-18T11:31:55.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022381850000000002 2024-09-18T11:31:55.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:55.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385494+0000", 2024-09-18T11:31:55.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7751000000000007e-05 2024-09-18T11:31:55.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.802 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:55.802 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385697+0000", 2024-09-18T11:31:55.802 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020247400000000001 2024-09-18T11:31:55.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:55.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.390505+0000", 2024-09-18T11:31:55.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048084039999999996 2024-09-18T11:31:55.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:55.803 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393148+0000", 2024-09-18T11:31:55.803 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026432399999999998 2024-09-18T11:31:55.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.803 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:55.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393194+0000", 2024-09-18T11:31:55.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5259999999999997e-05 2024-09-18T11:31:55.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.804 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:55.804 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393233+0000", 2024-09-18T11:31:55.804 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9659000000000001e-05 2024-09-18T11:31:55.804 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.804 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:55.804 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.804 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2342 4.f 4:f3c51b3f:::smithi11939506-21:head [stat,delete] snapc 26=[26,25,24,1e,1c,19,18] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:55.805 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:55.805 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.368026440000001, 2024-09-18T11:31:55.805 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011167565000000001, 2024-09-18T11:31:55.805 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:55.805 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:55.805 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:55.805 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:55.805 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:55.805 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2342 2024-09-18T11:31:55.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.805 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:55.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.805 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:55.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:55.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:55.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:55.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.806 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:55.806 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:31:55.806 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:55.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:55.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:31:55.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1439999999999997e-06 2024-09-18T11:31:55.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:55.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386820+0000", 2024-09-18T11:31:55.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2651e-05 2024-09-18T11:31:55.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.807 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.807 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386825+0000", 2024-09-18T11:31:55.807 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3150000000000002e-06 2024-09-18T11:31:55.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386868+0000", 2024-09-18T11:31:55.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3832999999999999e-05 2024-09-18T11:31:55.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:55.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:31:55.808 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0409999999999998e-06 2024-09-18T11:31:55.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.808 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.808 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390445+0000", 2024-09-18T11:31:55.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035689459999999999 2024-09-18T11:31:55.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:55.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390539+0000", 2024-09-18T11:31:55.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3791999999999999e-05 2024-09-18T11:31:55.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.809 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:55.809 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390748+0000", 2024-09-18T11:31:55.809 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000208863 2024-09-18T11:31:55.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:55.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397659+0000", 2024-09-18T11:31:55.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069110710000000004 2024-09-18T11:31:55.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:55.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397873+0000", 2024-09-18T11:31:55.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021343999999999999 2024-09-18T11:31:55.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.810 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:55.810 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397923+0000", 2024-09-18T11:31:55.810 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0300000000000003e-05 2024-09-18T11:31:55.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.811 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:55.811 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397970+0000", 2024-09-18T11:31:55.811 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6968999999999997e-05 2024-09-18T11:31:55.811 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.811 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:55.811 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.811 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2429 4.b 4:d8d14c58:::smithi11939506-49:head [write 796698~561170,stat] snapc 2b=[2b,2a,26,25,24,1e,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T11:31:55.811 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:55.811 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 50.739657266000002, 2024-09-18T11:31:55.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097188989999999996, 2024-09-18T11:31:55.812 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:55.812 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:55.812 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:55.812 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:55.812 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:55.812 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2429 2024-09-18T11:31:55.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.812 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:55.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.812 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:55.812 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:55.812 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:55.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:55.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:55.813 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015170+0000", 2024-09-18T11:31:55.813 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:55.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.813 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:55.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015435+0000", 2024-09-18T11:31:55.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026533299999999998 2024-09-18T11:31:55.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:55.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015455+0000", 2024-09-18T11:31:55.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9623000000000002e-05 2024-09-18T11:31:55.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.814 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.814 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015461+0000", 2024-09-18T11:31:55.814 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1859999999999997e-06 2024-09-18T11:31:55.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015535+0000", 2024-09-18T11:31:55.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4555999999999996e-05 2024-09-18T11:31:55.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:55.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015543+0000", 2024-09-18T11:31:55.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9549999999999992e-06 2024-09-18T11:31:55.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.815 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.815 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017106+0000", 2024-09-18T11:31:55.815 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015630290000000001 2024-09-18T11:31:55.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:55.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017183+0000", 2024-09-18T11:31:55.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6953999999999995e-05 2024-09-18T11:31:55.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:55.816 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017301+0000", 2024-09-18T11:31:55.816 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011734 2024-09-18T11:31:55.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.816 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:55.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.020225+0000", 2024-09-18T11:31:55.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029244090000000002 2024-09-18T11:31:55.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:55.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024801+0000", 2024-09-18T11:31:55.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045764129999999997 2024-09-18T11:31:55.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.817 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.817 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:55.817 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024840+0000", 2024-09-18T11:31:55.817 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8325999999999999e-05 2024-09-18T11:31:55.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.818 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:55.818 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024890+0000", 2024-09-18T11:31:55.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0596999999999997e-05 2024-09-18T11:31:55.818 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.818 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:55.818 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.818 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.818 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2769 4.4 4:2cb3e6a5:::smithi11939506-3:head [write 484101~692450] snapc 30=[30,2f,2b,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:31:55.818 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:18.743766+0000", 2024-09-18T11:31:55.818 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.011063073000003, 2024-09-18T11:31:55.818 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086202359999999999, 2024-09-18T11:31:55.819 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:55.819 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:55.819 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:55.819 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:55.819 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:55.819 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2769 2024-09-18T11:31:55.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.819 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:55.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.819 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:55.819 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.743766+0000", 2024-09-18T11:31:55.819 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.819 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:55.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.743766+0000", 2024-09-18T11:31:55.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:55.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.743764+0000", 2024-09-18T11:31:55.820 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:55.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.820 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.820 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:55.820 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744376+0000", 2024-09-18T11:31:55.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00061202299999999995 2024-09-18T11:31:55.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:55.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744393+0000", 2024-09-18T11:31:55.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6679000000000001e-05 2024-09-18T11:31:55.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.821 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.821 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744399+0000", 2024-09-18T11:31:55.821 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4840000000000001e-06 2024-09-18T11:31:55.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.821 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744452+0000", 2024-09-18T11:31:55.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2321999999999998e-05 2024-09-18T11:31:55.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:55.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744552+0000", 2024-09-18T11:31:55.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100536 2024-09-18T11:31:55.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.822 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.822 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:55.822 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744738+0000", 2024-09-18T11:31:55.822 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000185434 2024-09-18T11:31:55.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:55.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.749297+0000", 2024-09-18T11:31:55.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004559802 2024-09-18T11:31:55.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:55.823 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.752260+0000", 2024-09-18T11:31:55.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029623000000000002 2024-09-18T11:31:55.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.823 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.823 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:55.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.752338+0000", 2024-09-18T11:31:55.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8330000000000004e-05 2024-09-18T11:31:55.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.824 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:55.824 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.752386+0000", 2024-09-18T11:31:55.824 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:31:55.824 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.824 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:55.824 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.824 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.824 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2937 4.7 4:e301aefd:::smithi11939506-19:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:55.825 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:31:55.825 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 32.719988301999997, 2024-09-18T11:31:55.825 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019287667000000001, 2024-09-18T11:31:55.825 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:55.825 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:55.825 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:55.825 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:55.825 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:55.825 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2937 2024-09-18T11:31:55.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.825 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:55.825 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.825 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:55.825 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:31:55.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:55.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:31:55.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.826 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:55.826 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:31:55.826 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:31:55.826 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.826 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:55.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034842+0000", 2024-09-18T11:31:55.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.372e-06 2024-09-18T11:31:55.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:55.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034846+0000", 2024-09-18T11:31:55.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3919999999999996e-06 2024-09-18T11:31:55.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.827 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.827 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034848+0000", 2024-09-18T11:31:55.827 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3539999999999998e-06 2024-09-18T11:31:55.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034893+0000", 2024-09-18T11:31:55.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4861000000000003e-05 2024-09-18T11:31:55.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:55.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034965+0000", 2024-09-18T11:31:55.828 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1860999999999995e-05 2024-09-18T11:31:55.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.828 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.828 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:55.828 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.054128+0000", 2024-09-18T11:31:55.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019163039 2024-09-18T11:31:55.829 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.829 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:55.829 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.829 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.829 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3453 4.f 4:f5d7df41:::smithi11939506-16:head [write 1813152~490434] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:31:55.829 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:31:55.829 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.732076374, 2024-09-18T11:31:55.829 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050685840000000003, 2024-09-18T11:31:55.829 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:55.829 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:55.829 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3453 2024-09-18T11:31:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:31:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:31:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022751+0000", 2024-09-18T11:31:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022871+0000", 2024-09-18T11:31:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000119781 2024-09-18T11:31:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022882+0000", 2024-09-18T11:31:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1627000000000001e-05 2024-09-18T11:31:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022887+0000", 2024-09-18T11:31:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2390000000000004e-06 2024-09-18T11:31:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023123+0000", 2024-09-18T11:31:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000236598 2024-09-18T11:31:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023207+0000", 2024-09-18T11:31:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4303999999999998e-05 2024-09-18T11:31:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023367+0000", 2024-09-18T11:31:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015983600000000001 2024-09-18T11:31:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073237+0000", 2024-09-18T11:31:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049869400000000001 2024-09-18T11:31:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073352+0000", 2024-09-18T11:31:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011579899999999999 2024-09-18T11:31:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073394+0000", 2024-09-18T11:31:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1445e-05 2024-09-18T11:31:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073438+0000", 2024-09-18T11:31:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4261999999999997e-05 2024-09-18T11:31:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3454 4.f 4:f5d7df41:::smithi11939506-16:head [setxattr _header (57) in=7b,truncate 2991679] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:31:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:31:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.7313779320000009, 2024-09-18T11:31:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050365656000000002, 2024-09-18T11:31:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3454 2024-09-18T11:31:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:31:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:31:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023450+0000", 2024-09-18T11:31:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023456+0000", 2024-09-18T11:31:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8329999999999999e-06 2024-09-18T11:31:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023465+0000", 2024-09-18T11:31:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1130000000000007e-06 2024-09-18T11:31:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023470+0000", 2024-09-18T11:31:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9139999999999999e-06 2024-09-18T11:31:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072611+0000", 2024-09-18T11:31:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141643999999998 2024-09-18T11:31:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072729+0000", 2024-09-18T11:31:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117667 2024-09-18T11:31:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072887+0000", 2024-09-18T11:31:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015760000000000001 2024-09-18T11:31:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073558+0000", 2024-09-18T11:31:55.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00067171699999999995 2024-09-18T11:31:55.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:55.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073764+0000", 2024-09-18T11:31:55.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000205749 2024-09-18T11:31:55.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:55.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073790+0000", 2024-09-18T11:31:55.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5643000000000002e-05 2024-09-18T11:31:55.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:55.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073816+0000", 2024-09-18T11:31:55.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6588e-05 2024-09-18T11:31:55.841 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.841 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:55.841 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.841 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3455 4.f 4:f5d7df41:::smithi11939506-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:31:55.841 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:31:55.841 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.7313483139999999, 2024-09-18T11:31:55.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050460466000000002, 2024-09-18T11:31:55.841 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:55.842 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:55.842 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:55.842 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:55.842 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:55.842 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3455 2024-09-18T11:31:55.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.842 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:55.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:55.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:31:55.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:55.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:31:55.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:55.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:31:55.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:55.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:55.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023483+0000", 2024-09-18T11:31:55.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.695e-06 2024-09-18T11:31:55.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:55.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023489+0000", 2024-09-18T11:31:55.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1079999999999998e-06 2024-09-18T11:31:55.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023492+0000", 2024-09-18T11:31:55.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.478e-06 2024-09-18T11:31:55.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073273+0000", 2024-09-18T11:31:55.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049780638000000002 2024-09-18T11:31:55.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:55.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073343+0000", 2024-09-18T11:31:55.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9770000000000005e-05 2024-09-18T11:31:55.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073846+0000", 2024-09-18T11:31:55.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050289499999999995 2024-09-18T11:31:55.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:55.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073886+0000", 2024-09-18T11:31:55.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0658999999999998e-05 2024-09-18T11:31:55.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:55.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073941+0000", 2024-09-18T11:31:55.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4642000000000003e-05 2024-09-18T11:31:55.846 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.846 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:55.846 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.846 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3456 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:31:55.846 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:31:55.847 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.7313232490000008, 2024-09-18T11:31:55.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049859120999999999, 2024-09-18T11:31:55.847 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:55.847 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:31:55.847 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:55.847 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:55.847 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:55.847 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3456 2024-09-18T11:31:55.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.847 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:55.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:55.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:31:55.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:55.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:31:55.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:55.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:31:55.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:55.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:55.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023508+0000", 2024-09-18T11:31:55.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9490000000000001e-06 2024-09-18T11:31:55.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:55.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023513+0000", 2024-09-18T11:31:55.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3800000000000002e-06 2024-09-18T11:31:55.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023517+0000", 2024-09-18T11:31:55.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5300000000000001e-06 2024-09-18T11:31:55.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073262+0000", 2024-09-18T11:31:55.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049745194999999999 2024-09-18T11:31:55.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:55.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073365+0000", 2024-09-18T11:31:55.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102462 2024-09-18T11:31:55.850 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.850 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:55.850 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.850 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3457 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:31:55.851 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:31:55.851 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.7313061530000002, 2024-09-18T11:31:55.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049928567, 2024-09-18T11:31:55.851 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:55.851 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:31:55.851 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:55.851 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:55.851 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:55.851 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3457 2024-09-18T11:31:55.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.851 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:55.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.851 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:55.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:31:55.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:55.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:31:55.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:55.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023522+0000", 2024-09-18T11:31:55.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:55.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:55.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023524+0000", 2024-09-18T11:31:55.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8500000000000001e-06 2024-09-18T11:31:55.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:55.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023529+0000", 2024-09-18T11:31:55.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7509999999999999e-06 2024-09-18T11:31:55.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023532+0000", 2024-09-18T11:31:55.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.163e-06 2024-09-18T11:31:55.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073386+0000", 2024-09-18T11:31:55.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049853899 2024-09-18T11:31:55.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:55.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073451+0000", 2024-09-18T11:31:55.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5197000000000006e-05 2024-09-18T11:31:55.854 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.854 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:55.854 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.855 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3458 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:31:55.855 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:31:55.855 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.7312935960000004, 2024-09-18T11:31:55.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.05001659, 2024-09-18T11:31:55.855 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:55.855 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:31:55.855 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:55.855 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:55.855 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:55.855 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3458 2024-09-18T11:31:55.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.856 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:55.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:55.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:31:55.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:55.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:31:55.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:55.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:31:55.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:55.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:55.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023537+0000", 2024-09-18T11:31:55.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.64e-06 2024-09-18T11:31:55.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:55.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023543+0000", 2024-09-18T11:31:55.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3369999999999996e-06 2024-09-18T11:31:55.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023546+0000", 2024-09-18T11:31:55.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:31:55.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073466+0000", 2024-09-18T11:31:55.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049919719000000001 2024-09-18T11:31:55.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:55.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073552+0000", 2024-09-18T11:31:55.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6168999999999994e-05 2024-09-18T11:31:55.859 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.859 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:55.859 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.859 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(client.4590.0:3453 4.f e116/22 ondisk, result = 0)", 2024-09-18T11:31:55.859 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:31:55.859 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.7296885119999992, 2024-09-18T11:31:55.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048099441, 2024-09-18T11:31:55.859 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:55.859 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:55.859 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:55.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:55.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:31:55.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:55.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:31:55.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:55.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:55.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025139+0000", 2024-09-18T11:31:55.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:55.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:55.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025144+0000", 2024-09-18T11:31:55.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4059999999999998e-06 2024-09-18T11:31:55.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:55.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072212+0000", 2024-09-18T11:31:55.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047068482000000002 2024-09-18T11:31:55.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:55.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072230+0000", 2024-09-18T11:31:55.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7632999999999999e-05 2024-09-18T11:31:55.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:55.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073224+0000", 2024-09-18T11:31:55.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099436899999999994 2024-09-18T11:31:55.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:55.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073231+0000", 2024-09-18T11:31:55.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-09-18T11:31:55.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:55.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:55.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:55.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073240+0000", 2024-09-18T11:31:55.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5699999999999993e-06 2024-09-18T11:31:55.863 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.863 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:55.863 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.863 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:55.863 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:55.863 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:55.865 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:31:55.865 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:31:55.865 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:31:55.866 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:55.866 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:55.866 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:31:55.863+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:31:55.866 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:55.867 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:55.882 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:55.882 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:55.898 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:55.898 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:55.914 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:55.914 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:55.930 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:55.930 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:55.946 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:55.947 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:55.967 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:55.967 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:55.967 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:55.966+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:55.975 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:31:56.066 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:56.065+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:56.113 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:56.113 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:56.122 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:31:56.166 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:31:56.165+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:31:56.192 INFO:teuthology.orchestra.run.smithi119.stderr:set pool 4 pgp_num to 16 2024-09-18T11:31:56.193 INFO:tasks.rados.rados.0.smithi119.stdout:1649: done (2 left) 2024-09-18T11:31:56.193 INFO:tasks.rados.rados.0.smithi119.stdout:1650: read oid 40 snap -1 2024-09-18T11:31:56.193 INFO:tasks.rados.rados.0.smithi119.stdout:1650: expect (ObjNum 287 snap 41 seq_num 287) 2024-09-18T11:31:56.194 INFO:tasks.rados.rados.0.smithi119.stdout:1651: tier_flush oid 7 2024-09-18T11:31:56.194 INFO:tasks.rados.rados.0.smithi119.stdout:1651: tier_flush oid 7 snap -1 2024-09-18T11:31:56.194 INFO:tasks.rados.rados.0.smithi119.stdout:1652: rollback oid 35 current snap is 47 2024-09-18T11:31:56.194 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 35 to 33 2024-09-18T11:31:56.194 INFO:tasks.rados.rados.0.smithi119.stdout:1653: read oid 1 snap 33 2024-09-18T11:31:56.194 INFO:tasks.rados.rados.0.smithi119.stdout:1653: expect (ObjNum 265 snap 31 seq_num 265) 2024-09-18T11:31:56.194 INFO:tasks.rados.rados.0.smithi119.stdout:1654: read oid 14 snap 42 2024-09-18T11:31:56.195 INFO:tasks.rados.rados.0.smithi119.stdout:1654: expect (ObjNum 246 snap 28 seq_num 246) 2024-09-18T11:31:56.195 INFO:tasks.rados.rados.0.smithi119.stdout:1655: snap_create 2024-09-18T11:31:56.195 INFO:tasks.rados.rados.0.smithi119.stdout:1648: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T11:31:56.195 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:56.193+0000 7f3155e8c640 -1 osd.4 125 handle_osd_map injecting map crc failure 2024-09-18T11:31:56.237 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:56.237 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:56.245 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:31:56.287 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:56.286+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:56.295 INFO:tasks.rados.rados.0.smithi119.stdout:1651: got (16) Device or resource busy 2024-09-18T11:31:56.295 INFO:tasks.rados.rados.0.smithi119.stdout:check_oldest_snap_flushed oid 7 v 113 (ObjNum 145 snap 18 seq_num 145) unflushed 31 2024-09-18T11:31:56.295 INFO:tasks.rados.rados.0.smithi119.stdout:1652: finishing rollback tid 0 to smithi11939506-35 2024-09-18T11:31:56.295 INFO:tasks.rados.rados.0.smithi119.stdout:1650: expect (ObjNum 287 snap 41 seq_num 287) 2024-09-18T11:31:56.368 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:56.367+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:56.386 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:56.386 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:56.394 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:31:56.467 INFO:tasks.rados.rados.0.smithi119.stdout:1652: finishing rollback tid 1 to smithi11939506-35 2024-09-18T11:31:56.467 INFO:tasks.rados.rados.0.smithi119.stdout:1652: finishing rollback tid 2 to smithi11939506-35 2024-09-18T11:31:56.467 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 162 (ObjNum 155 snap 20 seq_num 155) dirty exists 2024-09-18T11:31:56.467 INFO:tasks.rados.rados.0.smithi119.stdout:1653: expect (ObjNum 265 snap 31 seq_num 265) 2024-09-18T11:31:56.468 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:31:56.467+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:31:56.534 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:56.534 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:56.542 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:31:56.569 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:56.568+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:56.601 INFO:tasks.rados.rados.0.smithi119.stdout:1654: expect (ObjNum 246 snap 28 seq_num 246) 2024-09-18T11:31:56.670 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:56.669+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:56.681 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:56.681 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:56.689 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:31:56.771 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:56.770+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:56.805 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:56.805 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:56.811 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:31:56.897 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:56.896+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:56.951 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:56.951 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:56.958 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:31:56.973 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:31:56.972+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:31:57.074 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:57.073+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:57.097 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:57.098 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:57.106 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:31:57.175 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:31:57.174+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:31:57.200 INFO:tasks.rados.rados.0.smithi119.stdout:1647: done (7 left) 2024-09-18T11:31:57.200 INFO:tasks.rados.rados.0.smithi119.stdout:1648: done (6 left) 2024-09-18T11:31:57.200 INFO:tasks.rados.rados.0.smithi119.stdout:1650: done (5 left) 2024-09-18T11:31:57.200 INFO:tasks.rados.rados.0.smithi119.stdout:1651: done (4 left) 2024-09-18T11:31:57.201 INFO:tasks.rados.rados.0.smithi119.stdout:1652: done (3 left) 2024-09-18T11:31:57.201 INFO:tasks.rados.rados.0.smithi119.stdout:1653: done (2 left) 2024-09-18T11:31:57.201 INFO:tasks.rados.rados.0.smithi119.stdout:1654: done (1 left) 2024-09-18T11:31:57.201 INFO:tasks.rados.rados.0.smithi119.stdout:1655: done (0 left) 2024-09-18T11:31:57.201 INFO:tasks.rados.rados.0.smithi119.stdout:1656: write (excl) oid 34 current snap is 48 2024-09-18T11:31:57.201 INFO:tasks.rados.rados.0.smithi119.stdout:1656: seq_num 329 ranges {729503=560100,2019164=716432,3335341=194162} 2024-09-18T11:31:57.201 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T11:31:57.207 INFO:tasks.rados.rados.0.smithi119.stdout:1656: writing smithi11939506-34 from 729503 to 1289603 tid 1 2024-09-18T11:31:57.215 INFO:tasks.rados.rados.0.smithi119.stdout:1656: writing smithi11939506-34 from 2019164 to 2735596 tid 2 2024-09-18T11:31:57.220 INFO:tasks.rados.rados.0.smithi119.stdout:1656: writing smithi11939506-34 from 3335341 to 3529503 tid 3 2024-09-18T11:31:57.220 INFO:tasks.rados.rados.0.smithi119.stdout:1657: read oid 45 snap -1 2024-09-18T11:31:57.220 INFO:tasks.rados.rados.0.smithi119.stdout:1657: expect (ObjNum 254 snap 30 seq_num 254) 2024-09-18T11:31:57.220 INFO:tasks.rados.rados.0.smithi119.stdout:1658: delete oid 37 current snap is 48 2024-09-18T11:31:57.220 INFO:tasks.rados.rados.0.smithi119.stdout:1656: finishing write tid 1 to smithi11939506-34 2024-09-18T11:31:57.225 INFO:tasks.rados.rados.0.smithi119.stdout:1658: done (2 left) 2024-09-18T11:31:57.225 INFO:tasks.rados.rados.0.smithi119.stdout:1659: write oid 25 current snap is 48 2024-09-18T11:31:57.225 INFO:tasks.rados.rados.0.smithi119.stdout:1659: seq_num 331 ranges {763117=710598,2136128=720101,3390187=172930} 2024-09-18T11:31:57.233 INFO:tasks.rados.rados.0.smithi119.stdout:1659: writing smithi11939506-25 from 763117 to 1473715 tid 1 2024-09-18T11:31:57.240 INFO:tasks.rados.rados.0.smithi119.stdout:1659: writing smithi11939506-25 from 2136128 to 2856229 tid 2 2024-09-18T11:31:57.244 INFO:tasks.rados.rados.0.smithi119.stdout:1659: writing smithi11939506-25 from 3390187 to 3563117 tid 3 2024-09-18T11:31:57.244 INFO:tasks.rados.rados.0.smithi119.stdout:1660: tier_promote oid 26 2024-09-18T11:31:57.244 INFO:tasks.rados.rados.0.smithi119.stdout:1657: expect (ObjNum 254 snap 30 seq_num 254) 2024-09-18T11:31:57.253 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:57.254 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:57.263 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:31:57.307 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:57.305+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:57.360 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:57.360 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:57.368 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:31:57.377 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:57.376+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:57.412 INFO:tasks.rados.rados.0.smithi119.stdout:1656: finishing write tid 2 to smithi11939506-34 2024-09-18T11:31:57.412 INFO:tasks.rados.rados.0.smithi119.stdout:1656: finishing write tid 3 to smithi11939506-34 2024-09-18T11:31:57.412 INFO:tasks.rados.rados.0.smithi119.stdout:1656: finishing write tid 4 to smithi11939506-34 2024-09-18T11:31:57.413 INFO:tasks.rados.rados.0.smithi119.stdout:1657: done (3 left) 2024-09-18T11:31:57.413 INFO:tasks.rados.rados.0.smithi119.stdout:1661: write oid 23 current snap is 48 2024-09-18T11:31:57.413 INFO:tasks.rados.rados.0.smithi119.stdout:1661: seq_num 332 ranges {779924=785847,2078257=580112,3301257=278667} 2024-09-18T11:31:57.420 INFO:tasks.rados.rados.0.smithi119.stdout:1661: writing smithi11939506-23 from 779924 to 1565771 tid 1 2024-09-18T11:31:57.426 INFO:tasks.rados.rados.0.smithi119.stdout:1661: writing smithi11939506-23 from 2078257 to 2658369 tid 2 2024-09-18T11:31:57.431 INFO:tasks.rados.rados.0.smithi119.stdout:1661: writing smithi11939506-23 from 3301257 to 3579924 tid 3 2024-09-18T11:31:57.431 INFO:tasks.rados.rados.0.smithi119.stdout:1662: read oid 46 snap -1 2024-09-18T11:31:57.431 INFO:tasks.rados.rados.0.smithi119.stdout:1662: expect (ObjNum 320 snap 44 seq_num 320) 2024-09-18T11:31:57.432 INFO:tasks.rados.rados.0.smithi119.stdout:1663: snap_remove snap 39 2024-09-18T11:31:57.478 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:57.476+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:57.509 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:57.509 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:57.518 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:31:57.579 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:31:57.578+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:31:57.656 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:57.656 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:57.664 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:31:57.680 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:31:57.679+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:31:57.761 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:57.761 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:57.769 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:31:57.781 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:57.780+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:57.866 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:57.866 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:57.875 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:31:57.882 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:31:57.881+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:31:57.958 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:31:57.976 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:57.976 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:57.983 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:57.981+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:57.984 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:31:58.080 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:31:58.080 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:31:58.083 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:58.082+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:58.095 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:31:58.095 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:31:58.110 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:31:58.110 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:31:58.122 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:31:58.122 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:31:58.124 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:31:58.125 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:31:58.131 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:31:58.140 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:31:58.140 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:31:58.155 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:31:58.156 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:31:58.171 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:31:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:31:58.184 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:58.182+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:58.186 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:31:58.186 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:31:58.207 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:58.206+0000 7f2d449e8640 -1 osd.1 127 handle_osd_map injecting map crc failure 2024-09-18T11:31:58.210 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is set 2024-09-18T11:31:58.211 INFO:tasks.rados.rados.0.smithi119.stdout:1663: done (5 left) 2024-09-18T11:31:58.211 INFO:tasks.rados.rados.0.smithi119.stdout:1664: write (excl) oid 35 current snap is 48 2024-09-18T11:31:58.211 INFO:tasks.rados.rados.0.smithi119.stdout:1656: finishing write tid 5 to smithi11939506-34 2024-09-18T11:31:58.211 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 240 (ObjNum 329 snap 48 seq_num 329) dirty exists 2024-09-18T11:31:58.211 INFO:tasks.rados.rados.0.smithi119.stdout:1656: left oid 34 (ObjNum 329 snap 48 seq_num 329) 2024-09-18T11:31:58.211 INFO:tasks.rados.rados.0.smithi119.stdout:1664: seq_num 333 ranges {796731=461096,1736739=723770,2928680=668051} 2024-09-18T11:31:58.217 INFO:tasks.rados.rados.0.smithi119.stdout:1664: writing smithi11939506-35 from 796731 to 1257827 tid 1 2024-09-18T11:31:58.225 INFO:tasks.rados.rados.0.smithi119.stdout:1664: writing smithi11939506-35 from 1736739 to 2460509 tid 2 2024-09-18T11:31:58.231 INFO:tasks.rados.rados.0.smithi119.stdout:1664: writing smithi11939506-35 from 2928680 to 3596731 tid 3 2024-09-18T11:31:58.231 INFO:tasks.rados.rados.0.smithi119.stdout:1656: done (5 left) 2024-09-18T11:31:58.232 INFO:tasks.rados.rados.0.smithi119.stdout:1665: write (excl) oid 15 current snap is 48 2024-09-18T11:31:58.232 INFO:tasks.rados.rados.0.smithi119.stdout:1659: finishing write tid 1 to smithi11939506-25 2024-09-18T11:31:58.232 INFO:tasks.rados.rados.0.smithi119.stdout:1659: finishing write tid 2 to smithi11939506-25 2024-09-18T11:31:58.232 INFO:tasks.rados.rados.0.smithi119.stdout:1660: got (0) Success 2024-09-18T11:31:58.232 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 239 (ObjNum 273 snap 0 seq_num 0) clean dne 2024-09-18T11:31:58.232 INFO:tasks.rados.rados.0.smithi119.stdout:1659: finishing write tid 3 to smithi11939506-25 2024-09-18T11:31:58.232 INFO:tasks.rados.rados.0.smithi119.stdout:1659: finishing write tid 4 to smithi11939506-25 2024-09-18T11:31:58.232 INFO:tasks.rados.rados.0.smithi119.stdout:1659: finishing write tid 5 to smithi11939506-25 2024-09-18T11:31:58.232 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 62 (ObjNum 331 snap 48 seq_num 331) dirty exists 2024-09-18T11:31:58.232 INFO:tasks.rados.rados.0.smithi119.stdout:1659: left oid 25 (ObjNum 331 snap 48 seq_num 331) 2024-09-18T11:31:58.232 INFO:tasks.rados.rados.0.smithi119.stdout:1665: seq_num 334 ranges {413538=536345,1395221=583781,2556103=658617} 2024-09-18T11:31:58.236 INFO:tasks.rados.rados.0.smithi119.stdout:1665: writing smithi11939506-15 from 413538 to 949883 tid 1 2024-09-18T11:31:58.241 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:31:58.241 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:31:58.241 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:31:58.241 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:31:58.241 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.241 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:58.241 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:58.241 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 129.08077179599999, 2024-09-18T11:31:58.241 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:31:58.241 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:58.241 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:58.241 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:58.242 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:58.242 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:58.242 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:31:58.242 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.242 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:58.242 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.242 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:58.242 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:58.242 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.242 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.242 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.242 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:58.242 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:31:58.243 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.243 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.243 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:58.243 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:31:58.243 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:58.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.243 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.243 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:58.243 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:31:58.243 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:31:58.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.243 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.244 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:58.244 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:31:58.244 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:31:58.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.244 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.244 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:58.244 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:31:58.244 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:31:58.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.244 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.244 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:58.244 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:31:58.244 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:31:58.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.245 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.245 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:58.245 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:31:58.245 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:31:58.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.245 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.245 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:58.245 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:31:58.245 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:31:58.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.245 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.245 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:58.245 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:31:58.246 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:31:58.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.246 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.246 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:58.246 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:31:58.246 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:31:58.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.246 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.246 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:58.246 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:31:58.246 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:31:58.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.246 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.247 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:58.247 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:31:58.247 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:31:58.247 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.247 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:58.247 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.247 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.247 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:58.247 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:58.247 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 129.08064553400001, 2024-09-18T11:31:58.247 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055717040000000002, 2024-09-18T11:31:58.247 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:58.247 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:58.248 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:58.248 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:58.248 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:58.248 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 45 2024-09-18T11:31:58.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.248 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:58.248 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.248 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:58.248 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:58.248 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.248 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.249 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:58.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157677+0000", 2024-09-18T11:31:58.249 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.249 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.249 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:58.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157676+0000", 2024-09-18T11:31:58.249 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:58.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.249 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.249 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:58.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157694+0000", 2024-09-18T11:31:58.249 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7422999999999999e-05 2024-09-18T11:31:58.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.250 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.250 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:58.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157704+0000", 2024-09-18T11:31:58.250 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9000000000000001e-06 2024-09-18T11:31:58.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.250 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.250 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:58.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157718+0000", 2024-09-18T11:31:58.250 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4788e-05 2024-09-18T11:31:58.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.250 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.250 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:58.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160937+0000", 2024-09-18T11:31:58.250 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00321886 2024-09-18T11:31:58.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.251 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.251 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:58.251 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161023+0000", 2024-09-18T11:31:58.251 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6063999999999995e-05 2024-09-18T11:31:58.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.251 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.251 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:58.251 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161202+0000", 2024-09-18T11:31:58.251 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017912599999999999 2024-09-18T11:31:58.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.251 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.251 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:58.252 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163052+0000", 2024-09-18T11:31:58.252 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001849554 2024-09-18T11:31:58.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.252 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.252 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:58.252 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163204+0000", 2024-09-18T11:31:58.252 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015186900000000001 2024-09-18T11:31:58.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.252 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.252 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:58.252 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163229+0000", 2024-09-18T11:31:58.252 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4803999999999998e-05 2024-09-18T11:31:58.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.253 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.253 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:58.253 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163249+0000", 2024-09-18T11:31:58.253 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9992000000000001e-05 2024-09-18T11:31:58.253 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.253 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:58.253 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.253 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.253 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:58.253 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:58.253 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 129.080596421, 2024-09-18T11:31:58.253 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:31:58.254 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:58.254 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:58.254 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:58.254 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:58.254 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:58.254 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:31:58.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.254 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:58.254 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.254 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:58.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:58.254 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.254 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.255 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:58.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:58.255 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.255 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.255 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:58.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:31:58.255 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:58.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.255 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.255 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:58.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:31:58.255 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:31:58.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.256 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.256 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:58.256 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:31:58.256 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:31:58.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.256 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.256 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:58.256 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:31:58.256 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:31:58.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.256 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.256 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:58.257 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:31:58.257 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:31:58.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.257 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.257 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:58.257 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:31:58.257 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:31:58.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.257 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.257 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:58.257 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:31:58.257 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:31:58.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.258 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.258 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:58.258 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:31:58.258 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:31:58.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.258 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.258 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:58.258 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:31:58.258 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:31:58.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.258 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.258 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:58.258 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:31:58.259 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:31:58.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.259 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.259 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:58.259 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:31:58.259 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:31:58.259 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.259 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:58.259 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.259 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.259 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:58.259 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:58.259 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 129.08055389, 2024-09-18T11:31:58.260 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:31:58.260 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:58.260 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:58.260 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:58.260 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:58.260 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:58.260 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:31:58.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.260 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:58.260 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.260 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:58.260 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:58.260 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.261 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.261 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:58.261 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:31:58.261 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.261 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.261 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:58.261 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:31:58.261 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:58.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.261 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.261 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:58.261 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:31:58.262 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:31:58.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.262 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.262 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:58.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:31:58.262 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:31:58.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.262 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.262 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:58.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:31:58.262 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:31:58.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.262 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.262 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:58.263 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:31:58.263 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:31:58.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.263 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.263 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:58.263 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:31:58.263 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:31:58.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.263 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.263 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:58.263 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:31:58.263 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:31:58.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.263 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.264 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:58.264 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:31:58.264 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:31:58.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.264 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.264 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:58.264 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:31:58.264 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:31:58.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.264 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.264 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:58.264 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:31:58.264 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:31:58.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.265 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.265 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:58.265 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:31:58.265 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:31:58.265 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.265 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:58.265 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.265 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.265 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:58.265 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:58.265 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 129.08049896, 2024-09-18T11:31:58.265 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:31:58.266 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:58.266 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:58.266 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:58.266 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:58.266 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:58.266 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:31:58.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.266 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:58.266 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.266 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:58.266 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:58.266 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.267 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.267 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:58.267 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:58.267 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.267 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.267 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:58.267 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:31:58.267 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:58.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.267 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.267 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:58.267 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:31:58.268 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:31:58.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.268 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.268 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:58.268 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:31:58.268 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:31:58.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.268 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.268 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:58.268 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:31:58.268 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:31:58.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.268 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.268 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:58.269 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:31:58.269 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:31:58.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.269 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.269 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:58.269 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:31:58.269 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:31:58.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.269 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.269 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:58.269 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:31:58.270 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:31:58.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.270 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.270 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:58.270 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:31:58.270 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:31:58.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.270 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.270 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:58.270 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:31:58.270 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:31:58.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.270 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.271 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:58.271 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:31:58.271 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:31:58.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.271 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.271 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:58.271 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:31:58.271 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:31:58.271 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.271 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:58.271 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.271 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.272 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:58.272 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:58.272 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 129.08048898000001, 2024-09-18T11:31:58.272 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:31:58.272 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:58.272 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:58.272 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:58.273 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:58.273 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:58.273 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:31:58.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.273 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:58.273 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.273 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:58.273 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:58.273 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.273 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.273 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:58.273 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:58.273 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.274 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.274 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:58.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:31:58.274 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:58.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.274 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.274 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:58.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:31:58.274 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:31:58.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.274 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.275 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:58.275 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:31:58.275 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:31:58.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.275 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.275 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:58.275 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:31:58.275 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:31:58.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.275 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.275 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:58.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:31:58.276 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:31:58.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.276 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.276 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:58.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:31:58.276 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:31:58.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.276 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.276 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:58.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:31:58.276 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:31:58.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.277 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:58.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:31:58.277 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:31:58.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.277 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:58.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:31:58.277 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:31:58.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.277 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:58.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:31:58.278 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:31:58.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.278 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.278 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:58.278 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:31:58.278 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:31:58.278 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.278 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:58.278 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.278 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.278 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:58.278 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:58.279 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 129.08047738100001, 2024-09-18T11:31:58.279 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:31:58.279 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:58.279 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:58.279 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:58.279 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:58.279 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:58.279 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:31:58.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.279 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:58.279 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.279 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:58.279 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:58.279 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.280 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:58.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:58.280 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.280 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:58.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:31:58.280 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:58.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.280 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:58.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:31:58.281 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:31:58.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.281 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:58.281 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:31:58.281 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:31:58.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.281 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:58.281 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:31:58.281 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:31:58.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.281 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:58.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:31:58.282 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:31:58.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.282 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.282 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:58.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:31:58.282 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:31:58.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.282 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.282 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:58.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:31:58.282 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:31:58.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.282 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.283 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:58.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:31:58.283 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:31:58.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.283 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:58.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:31:58.283 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:31:58.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.283 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:58.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:31:58.283 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:31:58.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.284 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.284 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:58.284 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:31:58.284 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:31:58.284 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.284 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:58.284 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.284 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.284 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:58.284 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:58.284 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 129.08042090000001, 2024-09-18T11:31:58.285 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:31:58.285 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:58.285 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:58.285 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:58.285 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:58.285 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:58.285 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:31:58.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.285 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:58.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.285 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:58.285 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:58.285 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.286 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.286 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:58.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:31:58.286 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.286 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.286 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:58.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:31:58.286 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:58.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.286 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.286 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:58.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:31:58.287 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:31:58.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.287 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:58.287 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:31:58.287 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:31:58.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.287 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:58.287 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:31:58.287 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:31:58.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.287 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:58.287 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:31:58.288 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:31:58.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.288 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.288 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:58.288 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:31:58.288 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:31:58.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.288 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.288 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:58.288 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:31:58.288 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:31:58.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.288 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.289 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:58.289 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:31:58.289 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:31:58.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.289 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.289 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:58.289 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:31:58.289 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:31:58.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.289 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.289 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:58.289 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:31:58.289 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:31:58.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.290 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.290 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:58.290 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:31:58.290 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:31:58.290 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.290 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:58.290 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.290 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.290 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:58.290 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:58.290 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 129.080407076, 2024-09-18T11:31:58.291 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:31:58.291 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:58.291 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:58.291 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:58.291 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:58.291 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:58.291 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:31:58.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.291 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:58.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.291 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:58.291 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:58.291 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.292 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.292 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:58.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:58.292 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.292 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.292 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:58.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:31:58.292 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:58.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.292 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.292 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:58.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:31:58.293 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:31:58.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.293 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:58.293 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:31:58.293 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:31:58.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.293 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:58.293 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:31:58.293 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:31:58.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.293 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:58.293 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:31:58.294 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:31:58.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.294 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:58.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:31:58.294 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:31:58.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.294 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:58.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:31:58.294 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:31:58.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.294 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:58.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:31:58.294 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:31:58.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.295 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:58.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:31:58.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:31:58.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.295 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:58.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:31:58.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:31:58.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.295 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:58.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:31:58.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:31:58.296 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.296 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:58.296 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.296 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:58.296 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:58.296 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 128.66422816900001, 2024-09-18T11:31:58.296 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:31:58.296 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:58.296 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:58.296 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:58.297 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:58.297 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:58.297 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:31:58.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.297 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:58.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.297 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:58.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:58.297 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.297 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:58.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:58.297 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.298 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:58.298 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:31:58.298 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:58.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.298 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:58.298 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:31:58.298 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:31:58.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.298 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:58.298 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:31:58.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:31:58.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.299 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:58.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:31:58.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:31:58.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.299 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:58.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:31:58.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:31:58.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.299 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:58.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:31:58.300 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:31:58.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:58.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:31:58.300 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:31:58.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:58.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:31:58.300 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:31:58.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.301 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:58.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:31:58.301 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:31:58.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.301 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:58.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:31:58.301 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:31:58.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.301 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:58.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:31:58.301 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:31:58.302 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.302 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:58.302 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.302 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:58.302 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:58.302 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 128.664204869, 2024-09-18T11:31:58.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:31:58.302 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:58.302 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:58.302 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:58.302 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:58.303 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:58.303 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:31:58.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.303 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:58.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.303 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:58.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:58.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.303 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:58.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:31:58.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.304 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:58.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:31:58.304 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:58.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.304 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:58.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:31:58.304 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:31:58.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.304 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:58.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:31:58.304 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:31:58.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:58.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:31:58.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:31:58.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:58.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:31:58.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:31:58.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:58.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:31:58.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:31:58.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:58.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:31:58.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:31:58.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:58.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:31:58.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:31:58.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:58.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:31:58.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:31:58.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:58.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:31:58.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:31:58.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:58.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:31:58.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:31:58.307 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.308 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:58.308 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.308 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.308 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:58.308 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:58.308 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 128.66411926000001, 2024-09-18T11:31:58.308 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:31:58.308 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:58.308 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:58.308 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:58.308 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:58.308 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:58.309 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:31:58.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.309 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:58.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.309 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:58.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:58.309 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.309 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:58.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:58.309 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:58.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:31:58.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:58.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:58.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:31:58.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:31:58.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:58.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:31:58.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:31:58.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:58.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:31:58.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:31:58.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:58.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:31:58.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:31:58.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:58.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:31:58.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:31:58.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.312 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:58.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:31:58.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:31:58.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.312 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:58.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:31:58.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:31:58.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.313 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:58.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:31:58.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:31:58.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.313 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:58.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:31:58.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:31:58.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.313 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:58.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:31:58.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:31:58.314 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.314 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:58.314 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.314 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:58.314 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:58.314 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 128.66410829500001, 2024-09-18T11:31:58.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:31:58.314 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:58.314 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:58.314 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:58.315 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:58.315 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:58.315 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:31:58.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.315 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:58.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:58.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:58.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:58.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:58.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:58.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:31:58.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:58.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:58.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:31:58.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:31:58.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:58.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:31:58.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:31:58.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:58.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:31:58.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:31:58.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:58.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:31:58.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:31:58.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:58.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:31:58.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:31:58.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:58.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:31:58.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:31:58.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:58.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:31:58.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:31:58.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:58.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:31:58.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:31:58.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:58.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:31:58.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:31:58.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:58.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:31:58.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:31:58.319 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.319 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:58.320 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.320 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:58.320 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:58.320 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 128.66409712699999, 2024-09-18T11:31:58.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:31:58.320 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:58.320 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:58.320 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:58.320 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:58.320 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:58.320 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:31:58.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.321 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:58.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:58.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:58.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:58.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:58.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:58.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:31:58.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:31:58.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:58.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:31:58.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:31:58.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:58.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:31:58.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:31:58.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:58.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:31:58.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:31:58.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:58.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:31:58.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:31:58.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:58.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:31:58.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:31:58.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:58.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:31:58.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:31:58.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:58.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:31:58.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:31:58.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:58.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:31:58.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:31:58.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:58.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:31:58.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:31:58.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:58.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:31:58.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:31:58.325 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.325 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:58.325 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.325 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:58.326 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:58.326 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 128.664031798, 2024-09-18T11:31:58.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:31:58.326 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:58.326 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:58.326 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:58.326 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:58.326 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:58.326 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:31:58.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.326 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:58.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:58.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:58.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:58.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:31:58.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:58.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:31:58.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:58.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:58.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:31:58.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:31:58.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.328 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:58.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:31:58.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:31:58.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.328 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:58.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:31:58.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:31:58.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:58.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:31:58.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:31:58.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:58.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:31:58.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:31:58.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:58.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:31:58.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:31:58.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:58.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:31:58.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:31:58.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:58.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:31:58.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:31:58.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:58.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:31:58.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:31:58.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:58.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:31:58.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:31:58.331 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.331 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:58.331 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.331 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:58.332 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:58.332 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 128.65386982000001, 2024-09-18T11:31:58.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:31:58.332 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:58.332 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:58.332 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:58.332 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:58.332 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:58.332 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:31:58.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.332 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:58.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:58.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:58.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:58.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:31:58.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:58.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:31:58.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:58.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:58.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:31:58.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:31:58.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:58.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:31:58.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:31:58.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:58.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:31:58.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:31:58.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:58.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:31:58.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:31:58.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:58.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:31:58.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:31:58.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:58.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:31:58.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:31:58.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:58.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:31:58.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:31:58.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:58.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:31:58.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:31:58.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:58.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:31:58.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:31:58.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:58.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:31:58.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:31:58.337 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.337 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:58.337 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.337 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:31:58.337 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:58.337 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 128.65377734099999, 2024-09-18T11:31:58.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:31:58.338 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:58.338 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:58.338 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:31:58.338 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:31:58.338 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:31:58.338 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:31:58.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.338 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:58.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:58.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:58.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:58.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:58.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:58.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:31:58.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:58.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:58.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:31:58.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:31:58.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:58.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:31:58.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:31:58.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:58.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:31:58.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:31:58.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:58.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:31:58.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:31:58.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:58.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:31:58.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:31:58.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:31:58.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:31:58.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:31:58.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:31:58.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:31:58.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:31:58.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:58.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:31:58.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:31:58.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:58.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:31:58.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:31:58.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:58.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:31:58.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:31:58.343 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.343 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:58.343 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.343 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1788 4.8 e60/22 4:1898b15d:::smithi11939506-25:head v 60'12, mlcod=58'9)", 2024-09-18T11:31:58.344 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:58.344 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 75.321371851999999, 2024-09-18T11:31:58.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058348599999999999, 2024-09-18T11:31:58.344 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:58.344 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:58.344 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:58.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:58.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:58.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:58.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:31:58.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:58.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916949+0000", 2024-09-18T11:31:58.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:58.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:58.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917602+0000", 2024-09-18T11:31:58.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00065337100000000003 2024-09-18T11:31:58.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:58.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917621+0000", 2024-09-18T11:31:58.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9287e-05 2024-09-18T11:31:58.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:58.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917629+0000", 2024-09-18T11:31:58.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3780000000000003e-06 2024-09-18T11:31:58.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:58.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917677+0000", 2024-09-18T11:31:58.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8068000000000001e-05 2024-09-18T11:31:58.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:58.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917699+0000", 2024-09-18T11:31:58.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2149000000000001e-05 2024-09-18T11:31:58.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:58.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922747+0000", 2024-09-18T11:31:58.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050477609999999996 2024-09-18T11:31:58.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:58.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922785+0000", 2024-09-18T11:31:58.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8726000000000002e-05 2024-09-18T11:31:58.348 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.348 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:58.348 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.348 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:58 4.3 e91/67 4:c70c3162:::smithi11939506-35:18 v 91'101, mlcod=86'95)", 2024-09-18T11:31:58.348 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:58.348 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 44.222713628999998, 2024-09-18T11:31:58.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055647500000000003, 2024-09-18T11:31:58.348 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:58.349 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:58.349 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:58.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:58.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:58.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:58.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015609+0000", 2024-09-18T11:31:58.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:58.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015608+0000", 2024-09-18T11:31:58.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:58.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:58.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015614+0000", 2024-09-18T11:31:58.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5310000000000001e-06 2024-09-18T11:31:58.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:58.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015622+0000", 2024-09-18T11:31:58.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0809999999999994e-06 2024-09-18T11:31:58.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:58.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015627+0000", 2024-09-18T11:31:58.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8439999999999997e-06 2024-09-18T11:31:58.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:58.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016003+0000", 2024-09-18T11:31:58.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037652999999999997 2024-09-18T11:31:58.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:58.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.016024+0000", 2024-09-18T11:31:58.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0780000000000001e-05 2024-09-18T11:31:58.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:58.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021141+0000", 2024-09-18T11:31:58.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0051166639999999999 2024-09-18T11:31:58.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:58.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.021174+0000", 2024-09-18T11:31:58.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2839000000000002e-05 2024-09-18T11:31:58.352 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.352 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:58.352 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.353 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3209 4.0 e109/22 4:019b6e09:::smithi11939506-40:head v 109'149, mlcod=106'146)", 2024-09-18T11:31:58.353 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:31:58.353 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.138936489999999, 2024-09-18T11:31:58.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063064039999999998, 2024-09-18T11:31:58.353 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:31:58.353 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:58.353 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:31:58.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:31:58.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:31:58.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:31:58.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:31:58.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:31:58.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:31:58.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099385+0000", 2024-09-18T11:31:58.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:58.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:31:58.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099951+0000", 2024-09-18T11:31:58.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056606600000000005 2024-09-18T11:31:58.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:31:58.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099964+0000", 2024-09-18T11:31:58.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2918999999999999e-05 2024-09-18T11:31:58.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:31:58.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099972+0000", 2024-09-18T11:31:58.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2300000000000002e-06 2024-09-18T11:31:58.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:31:58.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.100026+0000", 2024-09-18T11:31:58.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4456999999999997e-05 2024-09-18T11:31:58.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:31:58.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.100046+0000", 2024-09-18T11:31:58.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0417000000000001e-05 2024-09-18T11:31:58.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:31:58.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.105653+0000", 2024-09-18T11:31:58.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005606214 2024-09-18T11:31:58.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:31:58.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:31:58.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:31:58.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.105692+0000", 2024-09-18T11:31:58.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9671e-05 2024-09-18T11:31:58.357 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.357 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:58.357 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.357 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:31:58.357 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:31:58.357 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:31:58.359 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:31:58.360 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:58.357+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:58.361 INFO:tasks.rados.rados.0.smithi119.stdout:1665: writing smithi11939506-15 from 1395221 to 1979002 tid 2 2024-09-18T11:31:58.361 INFO:tasks.rados.rados.0.smithi119.stdout:1665: writing smithi11939506-15 from 2556103 to 3214720 tid 3 2024-09-18T11:31:58.361 INFO:tasks.rados.rados.0.smithi119.stdout:1659: done (5 left) 2024-09-18T11:31:58.361 INFO:tasks.rados.rados.0.smithi119.stdout:1660: done (4 left) 2024-09-18T11:31:58.361 INFO:tasks.rados.rados.0.smithi119.stdout:1666: write (excl) oid 9 current snap is 48 2024-09-18T11:31:58.361 INFO:tasks.rados.rados.0.smithi119.stdout:1661: finishing write tid 1 to smithi11939506-23 2024-09-18T11:31:58.361 INFO:tasks.rados.rados.0.smithi119.stdout:1661: finishing write tid 2 to smithi11939506-23 2024-09-18T11:31:58.361 INFO:tasks.rados.rados.0.smithi119.stdout:1661: finishing write tid 3 to smithi11939506-23 2024-09-18T11:31:58.361 INFO:tasks.rados.rados.0.smithi119.stdout:1661: finishing write tid 4 to smithi11939506-23 2024-09-18T11:31:58.361 INFO:tasks.rados.rados.0.smithi119.stdout:1666: seq_num 335 ranges {430345=727947,1570056=727439,2699879=469889} 2024-09-18T11:31:58.362 INFO:tasks.rados.rados.0.smithi119.stdout:1666: writing smithi11939506-9 from 430345 to 1158292 tid 1 2024-09-18T11:31:58.362 INFO:tasks.rados.rados.0.smithi119.stdout:1666: writing smithi11939506-9 from 1570056 to 2297495 tid 2 2024-09-18T11:31:58.362 INFO:tasks.rados.rados.0.smithi119.stdout:1666: writing smithi11939506-9 from 2699879 to 3169768 tid 3 2024-09-18T11:31:58.362 INFO:tasks.rados.rados.0.smithi119.stdout:1667: read oid 47 snap -1 2024-09-18T11:31:58.362 INFO:tasks.rados.rados.0.smithi119.stdout:1667: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T11:31:58.362 INFO:tasks.rados.rados.0.smithi119.stdout:1668: write oid 28 current snap is 48 2024-09-18T11:31:58.362 INFO:tasks.rados.rados.0.smithi119.stdout:1661: finishing write tid 5 to smithi11939506-23 2024-09-18T11:31:58.362 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 260 (ObjNum 332 snap 48 seq_num 332) dirty exists 2024-09-18T11:31:58.362 INFO:tasks.rados.rados.0.smithi119.stdout:1661: left oid 23 (ObjNum 332 snap 48 seq_num 332) 2024-09-18T11:31:58.362 INFO:tasks.rados.rados.0.smithi119.stdout:1662: expect (ObjNum 320 snap 44 seq_num 320) 2024-09-18T11:31:58.455 INFO:tasks.rados.rados.0.smithi119.stdout:1664: finishing write tid 1 to smithi11939506-35 2024-09-18T11:31:58.456 INFO:tasks.rados.rados.0.smithi119.stdout:1664: finishing write tid 2 to smithi11939506-35 2024-09-18T11:31:58.456 INFO:tasks.rados.rados.0.smithi119.stdout:1664: finishing write tid 3 to smithi11939506-35 2024-09-18T11:31:58.456 INFO:tasks.rados.rados.0.smithi119.stdout:1664: finishing write tid 4 to smithi11939506-35 2024-09-18T11:31:58.456 INFO:tasks.rados.rados.0.smithi119.stdout:1668: seq_num 336 ranges {447152=403196,1512185=587450,2727302=681161} 2024-09-18T11:31:58.459 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:58.457+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:58.460 INFO:tasks.rados.rados.0.smithi119.stdout:1668: writing smithi11939506-28 from 447152 to 850348 tid 1 2024-09-18T11:31:58.463 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:58.463 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:58.463 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:58.463 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:58.467 INFO:tasks.rados.rados.0.smithi119.stdout:1668: writing smithi11939506-28 from 1512185 to 2099635 tid 2 2024-09-18T11:31:58.473 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:31:58.474 INFO:tasks.rados.rados.0.smithi119.stdout:1668: writing smithi11939506-28 from 2727302 to 3408463 tid 3 2024-09-18T11:31:58.474 INFO:tasks.rados.rados.0.smithi119.stdout:1661: done (6 left) 2024-09-18T11:31:58.474 INFO:tasks.rados.rados.0.smithi119.stdout:1662: done (5 left) 2024-09-18T11:31:58.474 INFO:tasks.rados.rados.0.smithi119.stdout:1669: read oid 12 snap -1 2024-09-18T11:31:58.474 INFO:tasks.rados.rados.0.smithi119.stdout:1669: expect (ObjNum 283 snap 39 seq_num 283) 2024-09-18T11:31:58.476 INFO:tasks.rados.rados.0.smithi119.stdout:1670: write oid 8 current snap is 48 2024-09-18T11:31:58.476 INFO:tasks.rados.rados.0.smithi119.stdout:1664: finishing write tid 5 to smithi11939506-35 2024-09-18T11:31:58.476 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 169 (ObjNum 333 snap 48 seq_num 333) dirty exists 2024-09-18T11:31:58.476 INFO:tasks.rados.rados.0.smithi119.stdout:1664: left oid 35 (ObjNum 333 snap 48 seq_num 333) 2024-09-18T11:31:58.476 INFO:tasks.rados.rados.0.smithi119.stdout:1665: finishing write tid 1 to smithi11939506-15 2024-09-18T11:31:58.476 INFO:tasks.rados.rados.0.smithi119.stdout:1670: seq_num 337 ranges {463959=478445,1570667=731108,3038372=492433} 2024-09-18T11:31:58.479 INFO:tasks.rados.rados.0.smithi119.stdout:1670: writing smithi11939506-8 from 463959 to 942404 tid 1 2024-09-18T11:31:58.487 INFO:tasks.rados.rados.0.smithi119.stdout:1670: writing smithi11939506-8 from 1570667 to 2301775 tid 2 2024-09-18T11:31:58.494 INFO:tasks.rados.rados.0.smithi119.stdout:1670: writing smithi11939506-8 from 3038372 to 3530805 tid 3 2024-09-18T11:31:58.494 INFO:tasks.rados.rados.0.smithi119.stdout:1664: done (6 left) 2024-09-18T11:31:58.494 INFO:tasks.rados.rados.0.smithi119.stdout:1671: read oid 36 snap -1 2024-09-18T11:31:58.495 INFO:tasks.rados.rados.0.smithi119.stdout:1671: expect (ObjNum 322 snap 45 seq_num 322) 2024-09-18T11:31:58.495 INFO:tasks.rados.rados.0.smithi119.stdout:1672: write (excl) oid 42 current snap is 48 2024-09-18T11:31:58.495 INFO:tasks.rados.rados.0.smithi119.stdout:1665: finishing write tid 2 to smithi11939506-15 2024-09-18T11:31:58.495 INFO:tasks.rados.rados.0.smithi119.stdout:1665: finishing write tid 3 to smithi11939506-15 2024-09-18T11:31:58.495 INFO:tasks.rados.rados.0.smithi119.stdout:1665: finishing write tid 4 to smithi11939506-15 2024-09-18T11:31:58.495 INFO:tasks.rados.rados.0.smithi119.stdout:1672: seq_num 338 ranges {480766=553694,1629149=591119,2782148=420058} 2024-09-18T11:31:58.500 INFO:tasks.rados.rados.0.smithi119.stdout:1672: writing smithi11939506-42 from 480766 to 1034460 tid 1 2024-09-18T11:31:58.507 INFO:tasks.rados.rados.0.smithi119.stdout:1672: writing smithi11939506-42 from 1629149 to 2220268 tid 2 2024-09-18T11:31:58.512 INFO:tasks.rados.rados.0.smithi119.stdout:1672: writing smithi11939506-42 from 2782148 to 3202206 tid 3 2024-09-18T11:31:58.512 INFO:tasks.rados.rados.0.smithi119.stdout:1673: set_chunk oid 18 target oid 2024-09-18T11:31:58.519 INFO:tasks.rados.rados.0.smithi119.stdout:1673: set_chunk oid 18 offset: 1964544 length: 409088 target oid 4e6b745cb5fde8912306f5565be1f3431b12724130448c673a2d7140c97aedc1 offset: 0 snap -1 2024-09-18T11:31:58.519 INFO:tasks.rados.rados.0.smithi119.stdout:1674: read oid 7 snap -1 2024-09-18T11:31:58.519 INFO:tasks.rados.rados.0.smithi119.stdout:1674: expect (ObjNum 145 snap 18 seq_num 145) 2024-09-18T11:31:58.519 INFO:tasks.rados.rados.0.smithi119.stdout:1675: read oid 26 snap -1 2024-09-18T11:31:58.519 INFO:tasks.rados.rados.0.smithi119.stdout:1675: expect deleted 2024-09-18T11:31:58.519 INFO:tasks.rados.rados.0.smithi119.stdout:1676: write oid 17 current snap is 48 2024-09-18T11:31:58.519 INFO:tasks.rados.rados.0.smithi119.stdout:1665: finishing write tid 5 to smithi11939506-15 2024-09-18T11:31:58.519 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 136 (ObjNum 334 snap 48 seq_num 334) dirty exists 2024-09-18T11:31:58.520 INFO:tasks.rados.rados.0.smithi119.stdout:1665: left oid 15 (ObjNum 334 snap 48 seq_num 334) 2024-09-18T11:31:58.520 INFO:tasks.rados.rados.0.smithi119.stdout:1666: finishing write tid 1 to smithi11939506-9 2024-09-18T11:31:58.520 INFO:tasks.rados.rados.0.smithi119.stdout:1666: finishing write tid 2 to smithi11939506-9 2024-09-18T11:31:58.520 INFO:tasks.rados.rados.0.smithi119.stdout:1666: finishing write tid 3 to smithi11939506-9 2024-09-18T11:31:58.520 INFO:tasks.rados.rados.0.smithi119.stdout:1666: finishing write tid 4 to smithi11939506-9 2024-09-18T11:31:58.520 INFO:tasks.rados.rados.0.smithi119.stdout:1666: finishing write tid 5 to smithi11939506-9 2024-09-18T11:31:58.520 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 410 (ObjNum 335 snap 48 seq_num 335) dirty exists 2024-09-18T11:31:58.520 INFO:tasks.rados.rados.0.smithi119.stdout:1666: left oid 9 (ObjNum 335 snap 48 seq_num 335) 2024-09-18T11:31:58.520 INFO:tasks.rados.rados.0.smithi119.stdout:1667: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T11:31:58.560 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:31:58.559+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:31:58.578 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:58.579 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:58.579 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:58.579 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:58.579 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:58.587 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:31:58.611 INFO:tasks.rados.rados.0.smithi119.stdout:1668: finishing write tid 1 to smithi11939506-28 2024-09-18T11:31:58.611 INFO:tasks.rados.rados.0.smithi119.stdout:1668: finishing write tid 2 to smithi11939506-28 2024-09-18T11:31:58.611 INFO:tasks.rados.rados.0.smithi119.stdout:1676: seq_num 339 ranges {497573=628943,1687631=734777,3093218=604355} 2024-09-18T11:31:58.617 INFO:tasks.rados.rados.0.smithi119.stdout:1676: writing smithi11939506-17 from 497573 to 1126516 tid 1 2024-09-18T11:31:58.624 INFO:tasks.rados.rados.0.smithi119.stdout:1676: writing smithi11939506-17 from 1687631 to 2422408 tid 2 2024-09-18T11:31:58.631 INFO:tasks.rados.rados.0.smithi119.stdout:1676: writing smithi11939506-17 from 3093218 to 3697573 tid 3 2024-09-18T11:31:58.632 INFO:tasks.rados.rados.0.smithi119.stdout:1665: done (11 left) 2024-09-18T11:31:58.632 INFO:tasks.rados.rados.0.smithi119.stdout:1666: done (10 left) 2024-09-18T11:31:58.632 INFO:tasks.rados.rados.0.smithi119.stdout:1667: done (9 left) 2024-09-18T11:31:58.632 INFO:tasks.rados.rados.0.smithi119.stdout:1677: read oid 23 snap -1 2024-09-18T11:31:58.632 INFO:tasks.rados.rados.0.smithi119.stdout:1677: expect (ObjNum 332 snap 48 seq_num 332) 2024-09-18T11:31:58.632 INFO:tasks.rados.rados.0.smithi119.stdout:1678: snap_create 2024-09-18T11:31:58.632 INFO:tasks.rados.rados.0.smithi119.stdout:1668: finishing write tid 3 to smithi11939506-28 2024-09-18T11:31:58.632 INFO:tasks.rados.rados.0.smithi119.stdout:1668: finishing write tid 4 to smithi11939506-28 2024-09-18T11:31:58.632 INFO:tasks.rados.rados.0.smithi119.stdout:1668: finishing write tid 5 to smithi11939506-28 2024-09-18T11:31:58.632 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 28 v 193 (ObjNum 336 snap 48 seq_num 336) dirty exists 2024-09-18T11:31:58.632 INFO:tasks.rados.rados.0.smithi119.stdout:1668: left oid 28 (ObjNum 336 snap 48 seq_num 336) 2024-09-18T11:31:58.632 INFO:tasks.rados.rados.0.smithi119.stdout:1669: expect (ObjNum 283 snap 39 seq_num 283) 2024-09-18T11:31:58.662 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:58.660+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:58.711 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [] 2024-09-18T11:31:58.711 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:58.719 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:31:58.763 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:58.761+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:58.839 INFO:tasks.rados.rados.0.smithi119.stdout:1670: finishing write tid 1 to smithi11939506-8 2024-09-18T11:31:58.839 INFO:tasks.rados.rados.0.smithi119.stdout:1670: finishing write tid 2 to smithi11939506-8 2024-09-18T11:31:58.840 INFO:tasks.rados.rados.0.smithi119.stdout:1670: finishing write tid 3 to smithi11939506-8 2024-09-18T11:31:58.840 INFO:tasks.rados.rados.0.smithi119.stdout:1670: finishing write tid 4 to smithi11939506-8 2024-09-18T11:31:58.840 INFO:tasks.rados.rados.0.smithi119.stdout:1670: finishing write tid 5 to smithi11939506-8 2024-09-18T11:31:58.840 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 8 v 265 (ObjNum 337 snap 48 seq_num 337) dirty exists 2024-09-18T11:31:58.840 INFO:tasks.rados.rados.0.smithi119.stdout:1670: left oid 8 (ObjNum 337 snap 48 seq_num 337) 2024-09-18T11:31:58.840 INFO:tasks.rados.rados.0.smithi119.stdout:1672: finishing write tid 1 to smithi11939506-42 2024-09-18T11:31:58.840 INFO:tasks.rados.rados.0.smithi119.stdout:1671: expect (ObjNum 322 snap 45 seq_num 322) 2024-09-18T11:31:58.858 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:58.858 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:58.858 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:58.858 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:58.862 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:58.860+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:58.867 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:31:58.964 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:58.964 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:58.965 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:58.965 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:58.965 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:58.966 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:58.963+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:31:58.974 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:31:59.065 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:59.063+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:59.075 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:59.075 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:59.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:59.075 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:59.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.075 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:262 4.9 e53/22 4:99feffe9:::smithi11939506-45:13 v 53'17, mlcod=51'14)", 2024-09-18T11:31:59.075 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:59.075 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.072876493999999, 2024-09-18T11:31:59.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013878679, 2024-09-18T11:31:59.075 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.076 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.076 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:59.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:59.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:31:59.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:59.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998997+0000", 2024-09-18T11:31:59.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1099999999999999e-06 2024-09-18T11:31:59.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999008+0000", 2024-09-18T11:31:59.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1321000000000001e-05 2024-09-18T11:31:59.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999015+0000", 2024-09-18T11:31:59.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3000000000000004e-06 2024-09-18T11:31:59.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999055+0000", 2024-09-18T11:31:59.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0058000000000002e-05 2024-09-18T11:31:59.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999070+0000", 2024-09-18T11:31:59.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4328000000000001e-05 2024-09-18T11:31:59.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012465+0000", 2024-09-18T11:31:59.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013395638 2024-09-18T11:31:59.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012870+0000", 2024-09-18T11:31:59.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040423400000000001 2024-09-18T11:31:59.080 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.080 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.080 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.080 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:205 4.b e53/22 4:dba5bd39:::smithi11939506-17:13 v 53'77, mlcod=51'74)", 2024-09-18T11:31:59.080 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:59.080 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.072827232999998, 2024-09-18T11:31:59.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013359586, 2024-09-18T11:31:59.080 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.080 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.080 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:59.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:59.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:31:59.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:59.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999048+0000", 2024-09-18T11:31:59.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3550000000000005e-06 2024-09-18T11:31:59.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999056+0000", 2024-09-18T11:31:59.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3259999999999996e-06 2024-09-18T11:31:59.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999063+0000", 2024-09-18T11:31:59.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6409999999999996e-06 2024-09-18T11:31:59.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999098+0000", 2024-09-18T11:31:59.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5203999999999997e-05 2024-09-18T11:31:59.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999115+0000", 2024-09-18T11:31:59.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6478999999999999e-05 2024-09-18T11:31:59.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012362+0000", 2024-09-18T11:31:59.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01324768 2024-09-18T11:31:59.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012400+0000", 2024-09-18T11:31:59.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.731e-05 2024-09-18T11:31:59.084 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.084 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.084 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.084 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:88 4.a e53/22 4:588debc2:::smithi11939506-18:13 v 53'45, mlcod=51'42)", 2024-09-18T11:31:59.085 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:59.085 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.070044809999999, 2024-09-18T11:31:59.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011307667, 2024-09-18T11:31:59.085 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.085 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.085 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:59.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001823+0000", 2024-09-18T11:31:59.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001822+0000", 2024-09-18T11:31:59.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:59.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001827+0000", 2024-09-18T11:31:59.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.378e-06 2024-09-18T11:31:59.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001839+0000", 2024-09-18T11:31:59.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1225e-05 2024-09-18T11:31:59.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001844+0000", 2024-09-18T11:31:59.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2299999999999999e-06 2024-09-18T11:31:59.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001891+0000", 2024-09-18T11:31:59.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6961e-05 2024-09-18T11:31:59.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.001906+0000", 2024-09-18T11:31:59.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5158e-05 2024-09-18T11:31:59.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013103+0000", 2024-09-18T11:31:59.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011197314 2024-09-18T11:31:59.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013130+0000", 2024-09-18T11:31:59.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6900999999999999e-05 2024-09-18T11:31:59.088 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.089 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.089 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.089 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:89 4.a e53/22 4:552523b6:::smithi11939506-38:13 v 53'47, mlcod=51'42)", 2024-09-18T11:31:59.089 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:59.089 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.069485589999999, 2024-09-18T11:31:59.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011327614, 2024-09-18T11:31:59.089 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.089 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.089 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:59.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002382+0000", 2024-09-18T11:31:59.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002381+0000", 2024-09-18T11:31:59.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:59.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002397+0000", 2024-09-18T11:31:59.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5688e-05 2024-09-18T11:31:59.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002406+0000", 2024-09-18T11:31:59.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0229999999999998e-06 2024-09-18T11:31:59.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.002415+0000", 2024-09-18T11:31:59.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.8959999999999992e-06 2024-09-18T11:31:59.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012386+0000", 2024-09-18T11:31:59.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0099711450000000007 2024-09-18T11:31:59.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012405+0000", 2024-09-18T11:31:59.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9389e-05 2024-09-18T11:31:59.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013676+0000", 2024-09-18T11:31:59.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012708649999999999 2024-09-18T11:31:59.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.013709+0000", 2024-09-18T11:31:59.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3207000000000002e-05 2024-09-18T11:31:59.093 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.093 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.093 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.093 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1645 4.7 4:e73ba68f:::smithi11939506-1:head [read 0~3983226,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:31:59.093 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:59.093 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.737948398, 2024-09-18T11:31:59.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010539468999999999, 2024-09-18T11:31:59.094 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.094 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:59.094 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:59.094 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:59.094 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:59.094 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1645 2024-09-18T11:31:59.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.094 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:59.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:59.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333919+0000", 2024-09-18T11:31:59.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:59.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333921+0000", 2024-09-18T11:31:59.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8909999999999999e-06 2024-09-18T11:31:59.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333927+0000", 2024-09-18T11:31:59.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4229999999999996e-06 2024-09-18T11:31:59.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.333932+0000", 2024-09-18T11:31:59.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8280000000000002e-06 2024-09-18T11:31:59.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342418+0000", 2024-09-18T11:31:59.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084862219999999999 2024-09-18T11:31:59.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.342515+0000", 2024-09-18T11:31:59.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7287999999999994e-05 2024-09-18T11:31:59.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.344458+0000", 2024-09-18T11:31:59.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019431660000000001 2024-09-18T11:31:59.097 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.097 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.098 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.098 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:100 4.a e58/22 4:5975a5ec:::smithi11939506-36:18 v 58'55, mlcod=56'50)", 2024-09-18T11:31:59.098 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:59.098 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 79.798718261999994, 2024-09-18T11:31:59.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0088255, 2024-09-18T11:31:59.098 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.098 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.098 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:59.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:59.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273149+0000", 2024-09-18T11:31:59.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:59.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273155+0000", 2024-09-18T11:31:59.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.212e-06 2024-09-18T11:31:59.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273164+0000", 2024-09-18T11:31:59.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7609999999999996e-06 2024-09-18T11:31:59.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.273168+0000", 2024-09-18T11:31:59.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2819999999999998e-06 2024-09-18T11:31:59.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274162+0000", 2024-09-18T11:31:59.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099398600000000009 2024-09-18T11:31:59.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.274180+0000", 2024-09-18T11:31:59.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8236e-05 2024-09-18T11:31:59.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281943+0000", 2024-09-18T11:31:59.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007763308 2024-09-18T11:31:59.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:39.281975+0000", 2024-09-18T11:31:59.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1202999999999999e-05 2024-09-18T11:31:59.102 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.102 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.102 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.102 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1954 4.7 4:ef60946b:::smithi11939506-8:head [read 0~2890771,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:59.102 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:59.102 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 67.731967221999994, 2024-09-18T11:31:59.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091142180000000003, 2024-09-18T11:31:59.102 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.103 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:59.103 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:59.103 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:59.103 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:59.103 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1954 2024-09-18T11:31:59.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.103 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:59.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:59.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339900+0000", 2024-09-18T11:31:59.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:59.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339903+0000", 2024-09-18T11:31:59.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7349999999999998e-06 2024-09-18T11:31:59.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339909+0000", 2024-09-18T11:31:59.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7950000000000004e-06 2024-09-18T11:31:59.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339913+0000", 2024-09-18T11:31:59.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7670000000000001e-06 2024-09-18T11:31:59.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339970+0000", 2024-09-18T11:31:59.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6988999999999999e-05 2024-09-18T11:31:59.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:31:59.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.339975+0000", 2024-09-18T11:31:59.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7929999999999997e-06 2024-09-18T11:31:59.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346917+0000", 2024-09-18T11:31:59.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069426640000000003 2024-09-18T11:31:59.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.347026+0000", 2024-09-18T11:31:59.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108241 2024-09-18T11:31:59.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.349014+0000", 2024-09-18T11:31:59.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001988723 2024-09-18T11:31:59.107 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.107 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.107 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.107 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1956 4.6 4:67c1765a:::smithi11939506-30:head [write 705927~558502] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:31:59.107 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:59.107 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 67.719858282000004, 2024-09-18T11:31:59.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011813239, 2024-09-18T11:31:59.108 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.108 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.108 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:59.108 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:59.108 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:59.108 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1956 2024-09-18T11:31:59.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.108 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:59.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:31:59.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352007+0000", 2024-09-18T11:31:59.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:31:59.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352229+0000", 2024-09-18T11:31:59.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222024 2024-09-18T11:31:59.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352251+0000", 2024-09-18T11:31:59.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2512000000000001e-05 2024-09-18T11:31:59.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352260+0000", 2024-09-18T11:31:59.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7339999999999997e-06 2024-09-18T11:31:59.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352307+0000", 2024-09-18T11:31:59.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7061999999999998e-05 2024-09-18T11:31:59.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352408+0000", 2024-09-18T11:31:59.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101216 2024-09-18T11:31:59.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:59.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352550+0000", 2024-09-18T11:31:59.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014136199999999999 2024-09-18T11:31:59.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:59.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357684+0000", 2024-09-18T11:31:59.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051342499999999999 2024-09-18T11:31:59.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:59.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363736+0000", 2024-09-18T11:31:59.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006052577 2024-09-18T11:31:59.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363778+0000", 2024-09-18T11:31:59.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1989000000000001e-05 2024-09-18T11:31:59.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363822+0000", 2024-09-18T11:31:59.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3894999999999999e-05 2024-09-18T11:31:59.113 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.113 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.113 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.114 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2278 4.f 4:fc192b81:::smithi11939506-31:head [stat,delete] snapc 26=[26,25,24,1e,1c,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T11:31:59.114 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:59.114 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.688821070000003, 2024-09-18T11:31:59.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01018694, 2024-09-18T11:31:59.114 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.114 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.114 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:59.114 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:59.114 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:59.114 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2278 2024-09-18T11:31:59.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.114 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:59.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383046+0000", 2024-09-18T11:31:59.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383044+0000", 2024-09-18T11:31:59.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:59.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383054+0000", 2024-09-18T11:31:59.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5969999999999998e-06 2024-09-18T11:31:59.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383071+0000", 2024-09-18T11:31:59.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7353000000000001e-05 2024-09-18T11:31:59.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383081+0000", 2024-09-18T11:31:59.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9170000000000001e-06 2024-09-18T11:31:59.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383150+0000", 2024-09-18T11:31:59.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8503999999999993e-05 2024-09-18T11:31:59.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:59.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.383158+0000", 2024-09-18T11:31:59.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5089999999999993e-06 2024-09-18T11:31:59.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385396+0000", 2024-09-18T11:31:59.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022381850000000002 2024-09-18T11:31:59.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385494+0000", 2024-09-18T11:31:59.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7751000000000007e-05 2024-09-18T11:31:59.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:59.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.385697+0000", 2024-09-18T11:31:59.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00020247400000000001 2024-09-18T11:31:59.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:59.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.390505+0000", 2024-09-18T11:31:59.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048084039999999996 2024-09-18T11:31:59.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:59.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393148+0000", 2024-09-18T11:31:59.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026432399999999998 2024-09-18T11:31:59.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393194+0000", 2024-09-18T11:31:59.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5259999999999997e-05 2024-09-18T11:31:59.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.393233+0000", 2024-09-18T11:31:59.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9659000000000001e-05 2024-09-18T11:31:59.120 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.120 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.120 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.120 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2342 4.f 4:f3c51b3f:::smithi11939506-21:head [stat,delete] snapc 26=[26,25,24,1e,1c,19,18] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:31:59.120 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:59.120 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 57.685065057999999, 2024-09-18T11:31:59.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011167565000000001, 2024-09-18T11:31:59.121 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.121 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.121 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:59.121 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:59.121 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:59.121 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2342 2024-09-18T11:31:59.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.121 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:59.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:31:59.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:31:59.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:59.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:31:59.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1439999999999997e-06 2024-09-18T11:31:59.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386820+0000", 2024-09-18T11:31:59.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2651e-05 2024-09-18T11:31:59.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386825+0000", 2024-09-18T11:31:59.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3150000000000002e-06 2024-09-18T11:31:59.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386868+0000", 2024-09-18T11:31:59.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3832999999999999e-05 2024-09-18T11:31:59.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:59.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:31:59.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0409999999999998e-06 2024-09-18T11:31:59.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390445+0000", 2024-09-18T11:31:59.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035689459999999999 2024-09-18T11:31:59.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390539+0000", 2024-09-18T11:31:59.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3791999999999999e-05 2024-09-18T11:31:59.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:59.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.390748+0000", 2024-09-18T11:31:59.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000208863 2024-09-18T11:31:59.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:59.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397659+0000", 2024-09-18T11:31:59.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069110710000000004 2024-09-18T11:31:59.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:59.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397873+0000", 2024-09-18T11:31:59.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021343999999999999 2024-09-18T11:31:59.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397923+0000", 2024-09-18T11:31:59.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0300000000000003e-05 2024-09-18T11:31:59.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.397970+0000", 2024-09-18T11:31:59.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6968999999999997e-05 2024-09-18T11:31:59.127 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.127 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.127 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.127 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2429 4.b 4:d8d14c58:::smithi11939506-49:head [write 796698~561170,stat] snapc 2b=[2b,2a,26,25,24,1e,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-18T11:31:59.127 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:59.127 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.056695884, 2024-09-18T11:31:59.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0097188989999999996, 2024-09-18T11:31:59.128 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.128 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:59.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:59.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:59.128 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2429 2024-09-18T11:31:59.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.128 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:59.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015171+0000", 2024-09-18T11:31:59.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015170+0000", 2024-09-18T11:31:59.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:59.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015435+0000", 2024-09-18T11:31:59.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00026533299999999998 2024-09-18T11:31:59.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015455+0000", 2024-09-18T11:31:59.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9623000000000002e-05 2024-09-18T11:31:59.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015461+0000", 2024-09-18T11:31:59.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1859999999999997e-06 2024-09-18T11:31:59.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015535+0000", 2024-09-18T11:31:59.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4555999999999996e-05 2024-09-18T11:31:59.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:31:59.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.015543+0000", 2024-09-18T11:31:59.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9549999999999992e-06 2024-09-18T11:31:59.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017106+0000", 2024-09-18T11:31:59.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015630290000000001 2024-09-18T11:31:59.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017183+0000", 2024-09-18T11:31:59.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6953999999999995e-05 2024-09-18T11:31:59.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:31:59.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.017301+0000", 2024-09-18T11:31:59.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011734 2024-09-18T11:31:59.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:59.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.020225+0000", 2024-09-18T11:31:59.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029244090000000002 2024-09-18T11:31:59.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:59.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024801+0000", 2024-09-18T11:31:59.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045764129999999997 2024-09-18T11:31:59.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024840+0000", 2024-09-18T11:31:59.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8325999999999999e-05 2024-09-18T11:31:59.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:05.024890+0000", 2024-09-18T11:31:59.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0596999999999997e-05 2024-09-18T11:31:59.134 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.134 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.134 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.134 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2769 4.4 4:2cb3e6a5:::smithi11939506-3:head [write 484101~692450] snapc 30=[30,2f,2b,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:31:59.134 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:18.743766+0000", 2024-09-18T11:31:59.134 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.328101691000001, 2024-09-18T11:31:59.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086202359999999999, 2024-09-18T11:31:59.134 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.134 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.135 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:59.135 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:59.135 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:59.135 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2769 2024-09-18T11:31:59.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.135 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.743766+0000", 2024-09-18T11:31:59.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.743766+0000", 2024-09-18T11:31:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.743764+0000", 2024-09-18T11:31:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744376+0000", 2024-09-18T11:31:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00061202299999999995 2024-09-18T11:31:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744393+0000", 2024-09-18T11:31:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6679000000000001e-05 2024-09-18T11:31:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744399+0000", 2024-09-18T11:31:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4840000000000001e-06 2024-09-18T11:31:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744452+0000", 2024-09-18T11:31:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2321999999999998e-05 2024-09-18T11:31:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744552+0000", 2024-09-18T11:31:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000100536 2024-09-18T11:31:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:31:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.744738+0000", 2024-09-18T11:31:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000185434 2024-09-18T11:31:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.749297+0000", 2024-09-18T11:31:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004559802 2024-09-18T11:31:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.752260+0000", 2024-09-18T11:31:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029623000000000002 2024-09-18T11:31:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.752338+0000", 2024-09-18T11:31:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8330000000000004e-05 2024-09-18T11:31:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:18.752386+0000", 2024-09-18T11:31:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:31:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2937 4.7 4:e301aefd:::smithi11939506-19:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:31:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:31:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.037026920000002, 2024-09-18T11:31:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019287667000000001, 2024-09-18T11:31:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2937 2024-09-18T11:31:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:31:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:31:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:31:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:31:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034842+0000", 2024-09-18T11:31:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.372e-06 2024-09-18T11:31:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034846+0000", 2024-09-18T11:31:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3919999999999996e-06 2024-09-18T11:31:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034848+0000", 2024-09-18T11:31:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3539999999999998e-06 2024-09-18T11:31:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034893+0000", 2024-09-18T11:31:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4861000000000003e-05 2024-09-18T11:31:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034965+0000", 2024-09-18T11:31:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1860999999999995e-05 2024-09-18T11:31:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.054128+0000", 2024-09-18T11:31:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019163039 2024-09-18T11:31:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3453 4.f 4:f5d7df41:::smithi11939506-16:head [write 1813152~490434] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:31:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:31:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.049114992, 2024-09-18T11:31:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050685840000000003, 2024-09-18T11:31:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3453 2024-09-18T11:31:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:31:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:31:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022751+0000", 2024-09-18T11:31:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022871+0000", 2024-09-18T11:31:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000119781 2024-09-18T11:31:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022882+0000", 2024-09-18T11:31:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1627000000000001e-05 2024-09-18T11:31:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022887+0000", 2024-09-18T11:31:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2390000000000004e-06 2024-09-18T11:31:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023123+0000", 2024-09-18T11:31:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000236598 2024-09-18T11:31:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023207+0000", 2024-09-18T11:31:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4303999999999998e-05 2024-09-18T11:31:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023367+0000", 2024-09-18T11:31:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015983600000000001 2024-09-18T11:31:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073237+0000", 2024-09-18T11:31:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049869400000000001 2024-09-18T11:31:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073352+0000", 2024-09-18T11:31:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011579899999999999 2024-09-18T11:31:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073394+0000", 2024-09-18T11:31:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1445e-05 2024-09-18T11:31:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073438+0000", 2024-09-18T11:31:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4261999999999997e-05 2024-09-18T11:31:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3454 4.f 4:f5d7df41:::smithi11939506-16:head [setxattr _header (57) in=7b,truncate 2991679] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:31:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:31:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.048416550000001, 2024-09-18T11:31:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050365656000000002, 2024-09-18T11:31:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3454 2024-09-18T11:31:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:31:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:31:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023450+0000", 2024-09-18T11:31:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023456+0000", 2024-09-18T11:31:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8329999999999999e-06 2024-09-18T11:31:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023465+0000", 2024-09-18T11:31:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1130000000000007e-06 2024-09-18T11:31:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023470+0000", 2024-09-18T11:31:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9139999999999999e-06 2024-09-18T11:31:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072611+0000", 2024-09-18T11:31:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141643999999998 2024-09-18T11:31:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072729+0000", 2024-09-18T11:31:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117667 2024-09-18T11:31:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:31:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072887+0000", 2024-09-18T11:31:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015760000000000001 2024-09-18T11:31:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:31:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073558+0000", 2024-09-18T11:31:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00067171699999999995 2024-09-18T11:31:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:31:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073764+0000", 2024-09-18T11:31:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000205749 2024-09-18T11:31:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073790+0000", 2024-09-18T11:31:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5643000000000002e-05 2024-09-18T11:31:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073816+0000", 2024-09-18T11:31:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6588e-05 2024-09-18T11:31:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3455 4.f 4:f5d7df41:::smithi11939506-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:31:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:31:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.048386932, 2024-09-18T11:31:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050460466000000002, 2024-09-18T11:31:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3455 2024-09-18T11:31:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:31:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:31:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:31:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023483+0000", 2024-09-18T11:31:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.695e-06 2024-09-18T11:31:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023489+0000", 2024-09-18T11:31:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1079999999999998e-06 2024-09-18T11:31:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023492+0000", 2024-09-18T11:31:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.478e-06 2024-09-18T11:31:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073273+0000", 2024-09-18T11:31:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049780638000000002 2024-09-18T11:31:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:31:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073343+0000", 2024-09-18T11:31:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9770000000000005e-05 2024-09-18T11:31:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073846+0000", 2024-09-18T11:31:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050289499999999995 2024-09-18T11:31:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073886+0000", 2024-09-18T11:31:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0658999999999998e-05 2024-09-18T11:31:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073941+0000", 2024-09-18T11:31:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4642000000000003e-05 2024-09-18T11:31:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3456 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:31:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:31:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.048361867000001, 2024-09-18T11:31:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049859120999999999, 2024-09-18T11:31:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:31:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3456 2024-09-18T11:31:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:31:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:31:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:31:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023508+0000", 2024-09-18T11:31:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9490000000000001e-06 2024-09-18T11:31:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023513+0000", 2024-09-18T11:31:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3800000000000002e-06 2024-09-18T11:31:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023517+0000", 2024-09-18T11:31:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5300000000000001e-06 2024-09-18T11:31:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073262+0000", 2024-09-18T11:31:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049745194999999999 2024-09-18T11:31:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073365+0000", 2024-09-18T11:31:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102462 2024-09-18T11:31:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3457 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:31:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:31:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.048344771, 2024-09-18T11:31:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049928567, 2024-09-18T11:31:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:31:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3457 2024-09-18T11:31:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:31:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:31:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023522+0000", 2024-09-18T11:31:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023524+0000", 2024-09-18T11:31:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8500000000000001e-06 2024-09-18T11:31:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023529+0000", 2024-09-18T11:31:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7509999999999999e-06 2024-09-18T11:31:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023532+0000", 2024-09-18T11:31:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.163e-06 2024-09-18T11:31:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073386+0000", 2024-09-18T11:31:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049853899 2024-09-18T11:31:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073451+0000", 2024-09-18T11:31:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5197000000000006e-05 2024-09-18T11:31:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3458 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:31:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:31:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.048332214, 2024-09-18T11:31:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.05001659, 2024-09-18T11:31:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:31:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:31:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:31:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3458 2024-09-18T11:31:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:31:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:31:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:31:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023537+0000", 2024-09-18T11:31:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.64e-06 2024-09-18T11:31:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023543+0000", 2024-09-18T11:31:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3369999999999996e-06 2024-09-18T11:31:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023546+0000", 2024-09-18T11:31:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:31:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073466+0000", 2024-09-18T11:31:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049919719000000001 2024-09-18T11:31:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073552+0000", 2024-09-18T11:31:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6168999999999994e-05 2024-09-18T11:31:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(client.4590.0:3453 4.f e116/22 ondisk, result = 0)", 2024-09-18T11:31:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:31:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.046727130000001, 2024-09-18T11:31:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048099441, 2024-09-18T11:31:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:31:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:31:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:31:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025139+0000", 2024-09-18T11:31:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025144+0000", 2024-09-18T11:31:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4059999999999998e-06 2024-09-18T11:31:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072212+0000", 2024-09-18T11:31:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047068482000000002 2024-09-18T11:31:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072230+0000", 2024-09-18T11:31:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7632999999999999e-05 2024-09-18T11:31:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073224+0000", 2024-09-18T11:31:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099436899999999994 2024-09-18T11:31:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073231+0000", 2024-09-18T11:31:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-09-18T11:31:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073240+0000", 2024-09-18T11:31:59.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5699999999999993e-06 2024-09-18T11:31:59.178 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.178 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.178 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.178 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.178 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.178 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:59.180 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:31:59.180 INFO:tasks.rados.rados.0.smithi119.stdout:1672: finishing write tid 2 to smithi11939506-42 2024-09-18T11:31:59.181 INFO:tasks.rados.rados.0.smithi119.stdout:1672: finishing write tid 3 to smithi11939506-42 2024-09-18T11:31:59.181 INFO:tasks.rados.rados.0.smithi119.stdout:1672: finishing write tid 4 to smithi11939506-42 2024-09-18T11:31:59.181 INFO:tasks.rados.rados.0.smithi119.stdout:1672: finishing write tid 5 to smithi11939506-42 2024-09-18T11:31:59.181 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 249 (ObjNum 338 snap 48 seq_num 338) dirty exists 2024-09-18T11:31:59.181 INFO:tasks.rados.rados.0.smithi119.stdout:1672: left oid 42 (ObjNum 338 snap 48 seq_num 338) 2024-09-18T11:31:59.181 INFO:tasks.rados.rados.0.smithi119.stdout:1673: finishing set_chunk to oid 18 2024-09-18T11:31:59.181 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 18 set_chunk returned error code -95 offset: 1964544 length: 409088 2024-09-18T11:31:59.181 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 0 (ObjNum 205 snap 23 seq_num 205) dirty exists 2024-09-18T11:31:59.181 INFO:tasks.rados.rados.0.smithi119.stdout:1674: expect (ObjNum 145 snap 18 seq_num 145) 2024-09-18T11:31:59.181 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:59.178+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:59.236 INFO:tasks.rados.rados.0.smithi119.stdout:1676: finishing write tid 1 to smithi11939506-17 2024-09-18T11:31:59.236 INFO:tasks.rados.rados.0.smithi119.stdout:1676: finishing write tid 2 to smithi11939506-17 2024-09-18T11:31:59.236 INFO:tasks.rados.rados.0.smithi119.stdout:1676: finishing write tid 3 to smithi11939506-17 2024-09-18T11:31:59.237 INFO:tasks.rados.rados.0.smithi119.stdout:1668: done (10 left) 2024-09-18T11:31:59.237 INFO:tasks.rados.rados.0.smithi119.stdout:1669: done (9 left) 2024-09-18T11:31:59.237 INFO:tasks.rados.rados.0.smithi119.stdout:1670: done (8 left) 2024-09-18T11:31:59.237 INFO:tasks.rados.rados.0.smithi119.stdout:1671: done (7 left) 2024-09-18T11:31:59.237 INFO:tasks.rados.rados.0.smithi119.stdout:1672: done (6 left) 2024-09-18T11:31:59.237 INFO:tasks.rados.rados.0.smithi119.stdout:1673: done (5 left) 2024-09-18T11:31:59.237 INFO:tasks.rados.rados.0.smithi119.stdout:1674: done (4 left) 2024-09-18T11:31:59.237 INFO:tasks.rados.rados.0.smithi119.stdout:1675: done (3 left) 2024-09-18T11:31:59.237 INFO:tasks.rados.rados.0.smithi119.stdout:1678: done (2 left) 2024-09-18T11:31:59.237 INFO:tasks.rados.rados.0.smithi119.stdout:1679: read oid 21 snap -1 2024-09-18T11:31:59.237 INFO:tasks.rados.rados.0.smithi119.stdout:1679: expect (ObjNum 311 snap 43 seq_num 311) 2024-09-18T11:31:59.237 INFO:tasks.rados.rados.0.smithi119.stdout:1680: read oid 10 snap -1 2024-09-18T11:31:59.237 INFO:tasks.rados.rados.0.smithi119.stdout:1680: expect (ObjNum 260 snap 31 seq_num 260) 2024-09-18T11:31:59.238 INFO:tasks.rados.rados.0.smithi119.stdout:1681: read oid 12 snap -1 2024-09-18T11:31:59.238 INFO:tasks.rados.rados.0.smithi119.stdout:1681: expect (ObjNum 283 snap 39 seq_num 283) 2024-09-18T11:31:59.238 INFO:tasks.rados.rados.0.smithi119.stdout:1682: set_chunk oid 36 target oid 2024-09-18T11:31:59.238 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:31:59.236+0000 7f6bf3052640 -1 osd.5 128 handle_osd_map injecting map crc failure 2024-09-18T11:31:59.239 INFO:tasks.rados.rados.0.smithi119.stdout:1682: set_chunk oid 36 offset: 150016 length: 245760 target oid 9eb8fa54d87a9b9775dd55c07b3d164453884695bd1ed83fe2f63259ee9e6afc offset: 0 snap -1 2024-09-18T11:31:59.240 INFO:tasks.rados.rados.0.smithi119.stdout:1683: set_chunk oid 40 target oid 2024-09-18T11:31:59.248 INFO:tasks.rados.rados.0.smithi119.stdout:1683: set_chunk oid 40 offset: 1875968 length: 497152 target oid bd5c52e5687865fe60152785248ba268d9e60a99a953fff24704bce0f39353ec offset: 0 snap 37 2024-09-18T11:31:59.248 INFO:tasks.rados.rados.0.smithi119.stdout:1684: read oid 5 snap 32 2024-09-18T11:31:59.248 INFO:tasks.rados.rados.0.smithi119.stdout:1684: expect (ObjNum 269 snap 31 seq_num 269) 2024-09-18T11:31:59.248 INFO:tasks.rados.rados.0.smithi119.stdout:1685: read oid 2 snap -1 2024-09-18T11:31:59.248 INFO:tasks.rados.rados.0.smithi119.stdout:1685: expect deleted 2024-09-18T11:31:59.249 INFO:tasks.rados.rados.0.smithi119.stdout:1686: read oid 15 snap 47 2024-09-18T11:31:59.249 INFO:tasks.rados.rados.0.smithi119.stdout:1686: expect (ObjNum 280 snap 37 seq_num 280) 2024-09-18T11:31:59.249 INFO:tasks.rados.rados.0.smithi119.stdout:1687: read oid 35 snap 45 2024-09-18T11:31:59.249 INFO:tasks.rados.rados.0.smithi119.stdout:1687: expect (ObjNum 155 snap 20 seq_num 155) 2024-09-18T11:31:59.249 INFO:tasks.rados.rados.0.smithi119.stdout:1688: write oid 45 current snap is 49 2024-09-18T11:31:59.249 INFO:tasks.rados.rados.0.smithi119.stdout:1676: finishing write tid 4 to smithi11939506-17 2024-09-18T11:31:59.249 INFO:tasks.rados.rados.0.smithi119.stdout:1676: finishing write tid 5 to smithi11939506-17 2024-09-18T11:31:59.249 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 415 (ObjNum 339 snap 48 seq_num 339) dirty exists 2024-09-18T11:31:59.249 INFO:tasks.rados.rados.0.smithi119.stdout:1676: left oid 17 (ObjNum 339 snap 48 seq_num 339) 2024-09-18T11:31:59.249 INFO:tasks.rados.rados.0.smithi119.stdout:1677: expect (ObjNum 332 snap 48 seq_num 332) 2024-09-18T11:31:59.281 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:31:59.278+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:31:59.327 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:59.327 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:59.327 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:59.327 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:59.335 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:31:59.405 INFO:tasks.rados.rados.0.smithi119.stdout:1679: expect (ObjNum 311 snap 43 seq_num 311) 2024-09-18T11:31:59.406 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:31:59.405+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:31:59.475 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:59.475 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:59.475 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:31:59.475 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:31:59.475 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:59.483 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:59.481+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:59.484 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:31:59.565 INFO:tasks.rados.rados.0.smithi119.stdout:1680: expect (ObjNum 260 snap 31 seq_num 260) 2024-09-18T11:31:59.584 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:31:59.582+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:31:59.623 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:59.623 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:31:59.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:31:59.623 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:31:59.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.623 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:31:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:31:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.637736571000005, 2024-09-18T11:31:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:31:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:31:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:31:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:31:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:31:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:31:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:31:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:31:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:31:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:31:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:31:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:31:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:31:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:31:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:31:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:31:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:31:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:31:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:31:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 90.402905618000005, 2024-09-18T11:31:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:31:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:31:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:31:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:31:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:31:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:31:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:31:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:31:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:31:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:31:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:31:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:31:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:31:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:31:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:31:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:31:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:31:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:31:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:31:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.228999333000004, 2024-09-18T11:31:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:31:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:31:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:31:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:31:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:31:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:31:59.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:31:59.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:31:59.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:31:59.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:31:59.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:31:59.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:31:59.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:31:59.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:31:59.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:31:59.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:31:59.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:31:59.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:31:59.641 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.641 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.641 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.642 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:304 4.6 e69/67 4:67c1765a:::smithi11939506-30:17 v 69'35, mlcod=58'31)", 2024-09-18T11:31:59.642 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:59.642 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.275158239000007, 2024-09-18T11:31:59.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006377611, 2024-09-18T11:31:59.642 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.642 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.642 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:59.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:59.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:31:59.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:59.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345961+0000", 2024-09-18T11:31:59.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2050000000000008e-06 2024-09-18T11:31:59.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345969+0000", 2024-09-18T11:31:59.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0339999999999997e-06 2024-09-18T11:31:59.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345976+0000", 2024-09-18T11:31:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-18T11:31:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346026+0000", 2024-09-18T11:31:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9286999999999998e-05 2024-09-18T11:31:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346050+0000", 2024-09-18T11:31:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4397e-05 2024-09-18T11:31:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352293+0000", 2024-09-18T11:31:59.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062427170000000001 2024-09-18T11:31:59.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352330+0000", 2024-09-18T11:31:59.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7348999999999998e-05 2024-09-18T11:31:59.648 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:347 4.d e69/22 4:b44229a1:::smithi11939506-42:19 v 69'99, mlcod=65'93)", 2024-09-18T11:31:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:31:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.269425044000002, 2024-09-18T11:31:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056998760000000004, 2024-09-18T11:31:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:31:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:31:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351684+0000", 2024-09-18T11:31:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351693+0000", 2024-09-18T11:31:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-18T11:31:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351704+0000", 2024-09-18T11:31:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1077000000000001e-05 2024-09-18T11:31:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351709+0000", 2024-09-18T11:31:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5419999999999999e-06 2024-09-18T11:31:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351956+0000", 2024-09-18T11:31:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000247252 2024-09-18T11:31:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351975+0000", 2024-09-18T11:31:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9201000000000001e-05 2024-09-18T11:31:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357334+0000", 2024-09-18T11:31:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053588749999999999 2024-09-18T11:31:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357385+0000", 2024-09-18T11:31:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1406999999999998e-05 2024-09-18T11:31:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.654 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1956 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'37, mlcod=58'31)", 2024-09-18T11:31:59.654 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:59.654 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.268480131000004, 2024-09-18T11:31:59.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010958842, 2024-09-18T11:31:59.654 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.654 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.654 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:59.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:31:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:59.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352996+0000", 2024-09-18T11:31:59.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036622 2024-09-18T11:31:59.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353011+0000", 2024-09-18T11:31:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4849e-05 2024-09-18T11:31:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353017+0000", 2024-09-18T11:31:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:31:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353068+0000", 2024-09-18T11:31:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0695000000000002e-05 2024-09-18T11:31:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353092+0000", 2024-09-18T11:31:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4260000000000002e-05 2024-09-18T11:31:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363551+0000", 2024-09-18T11:31:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010459154999999999 2024-09-18T11:31:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363589+0000", 2024-09-18T11:31:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8161000000000003e-05 2024-09-18T11:31:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1957 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'38, mlcod=65'33)", 2024-09-18T11:31:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.262064264000003, 2024-09-18T11:31:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050499400000000002, 2024-09-18T11:31:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:31:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359214+0000", 2024-09-18T11:31:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000168047 2024-09-18T11:31:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359227+0000", 2024-09-18T11:31:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3124e-05 2024-09-18T11:31:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359235+0000", 2024-09-18T11:31:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0760000000000003e-06 2024-09-18T11:31:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359287+0000", 2024-09-18T11:31:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.151e-05 2024-09-18T11:31:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359310+0000", 2024-09-18T11:31:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-09-18T11:31:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364065+0000", 2024-09-18T11:31:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047549439999999997 2024-09-18T11:31:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364096+0000", 2024-09-18T11:31:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1423000000000001e-05 2024-09-18T11:31:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2300 4.4 e77/22 4:2c93196f:::smithi11939506-28:head v 77'94, mlcod=77'90)", 2024-09-18T11:31:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:31:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.633002046000001, 2024-09-18T11:31:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050425380000000001, 2024-09-18T11:31:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:31:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:31:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988108+0000", 2024-09-18T11:31:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988388+0000", 2024-09-18T11:31:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028087700000000001 2024-09-18T11:31:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988405+0000", 2024-09-18T11:31:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6565000000000001e-05 2024-09-18T11:31:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988412+0000", 2024-09-18T11:31:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9709999999999998e-06 2024-09-18T11:31:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988464+0000", 2024-09-18T11:31:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1949e-05 2024-09-18T11:31:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988490+0000", 2024-09-18T11:31:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5780999999999999e-05 2024-09-18T11:31:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993112+0000", 2024-09-18T11:31:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046228399999999996 2024-09-18T11:31:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993151+0000", 2024-09-18T11:31:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8587999999999997e-05 2024-09-18T11:31:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.13:63 3.0 e78/21 3:24ebb740:::smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 78'80, mlcod=73'78)", 2024-09-18T11:31:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:31:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.216012630000002, 2024-09-18T11:31:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052262230000000003, 2024-09-18T11:31:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:31:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:31:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405097+0000", 2024-09-18T11:31:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405103+0000", 2024-09-18T11:31:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8780000000000003e-06 2024-09-18T11:31:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405117+0000", 2024-09-18T11:31:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3441000000000001e-05 2024-09-18T11:31:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405121+0000", 2024-09-18T11:31:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7099999999999998e-06 2024-09-18T11:31:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405158+0000", 2024-09-18T11:31:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6477999999999999e-05 2024-09-18T11:31:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405173+0000", 2024-09-18T11:31:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4796e-05 2024-09-18T11:31:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410285+0000", 2024-09-18T11:31:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051118919999999998 2024-09-18T11:31:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410324+0000", 2024-09-18T11:31:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9616999999999997e-05 2024-09-18T11:31:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.13:18 3.0 e91/21 3:35e3c617:::2e64dd273da7619bd64b294f52905c1b9405a1ced2a8fd3c0b5bc481bc9732a6:head v 91'89, mlcod=85'87)", 2024-09-18T11:31:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:31:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.599237219000003, 2024-09-18T11:31:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048558539999999997, 2024-09-18T11:31:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:31:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:31:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:31:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021880+0000", 2024-09-18T11:31:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5860000000000001e-06 2024-09-18T11:31:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021890+0000", 2024-09-18T11:31:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8290000000000003e-06 2024-09-18T11:31:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021895+0000", 2024-09-18T11:31:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2480000000000004e-06 2024-09-18T11:31:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022166+0000", 2024-09-18T11:31:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027073499999999998 2024-09-18T11:31:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022184+0000", 2024-09-18T11:31:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8082000000000001e-05 2024-09-18T11:31:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026693+0000", 2024-09-18T11:31:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045093479999999998 2024-09-18T11:31:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026729+0000", 2024-09-18T11:31:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6282999999999998e-05 2024-09-18T11:31:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:573 4.4 e91/22 4:2c93196f:::smithi11939506-28:26 v 91'135, mlcod=90'131)", 2024-09-18T11:31:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:31:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.596770630000002, 2024-09-18T11:31:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050722739999999999, 2024-09-18T11:31:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:31:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:31:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024339+0000", 2024-09-18T11:31:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024349+0000", 2024-09-18T11:31:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6339999999999996e-06 2024-09-18T11:31:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024362+0000", 2024-09-18T11:31:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-18T11:31:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024368+0000", 2024-09-18T11:31:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8649999999999998e-06 2024-09-18T11:31:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024431+0000", 2024-09-18T11:31:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2780999999999994e-05 2024-09-18T11:31:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024448+0000", 2024-09-18T11:31:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7388999999999999e-05 2024-09-18T11:31:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029385+0000", 2024-09-18T11:31:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004936523 2024-09-18T11:31:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029412+0000", 2024-09-18T11:31:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7702999999999999e-05 2024-09-18T11:31:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:574 4.4 e91/22 4:2cb3e6a5:::smithi11939506-3:1e v 91'137, mlcod=90'131)", 2024-09-18T11:31:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:31:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.596332586999999, 2024-09-18T11:31:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070802850000000004, 2024-09-18T11:31:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:31:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:31:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024777+0000", 2024-09-18T11:31:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024785+0000", 2024-09-18T11:31:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8099999999999998e-06 2024-09-18T11:31:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024794+0000", 2024-09-18T11:31:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5839999999999995e-06 2024-09-18T11:31:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024798+0000", 2024-09-18T11:31:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.031e-06 2024-09-18T11:31:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024830+0000", 2024-09-18T11:31:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2268999999999999e-05 2024-09-18T11:31:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024847+0000", 2024-09-18T11:31:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7306999999999999e-05 2024-09-18T11:31:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031834+0000", 2024-09-18T11:31:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069868380000000004 2024-09-18T11:31:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031858+0000", 2024-09-18T11:31:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4082000000000001e-05 2024-09-18T11:31:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:541 4.d e91/22 4:b9bb642b:::smithi11939506-27:19 v 91'158, mlcod=91'154)", 2024-09-18T11:31:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:31:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.588951895000001, 2024-09-18T11:31:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052185, 2024-09-18T11:31:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:31:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:31:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032158+0000", 2024-09-18T11:31:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:31:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032165+0000", 2024-09-18T11:31:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-09-18T11:31:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032176+0000", 2024-09-18T11:31:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1460999999999999e-05 2024-09-18T11:31:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032182+0000", 2024-09-18T11:31:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0870000000000003e-06 2024-09-18T11:31:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032224+0000", 2024-09-18T11:31:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1372999999999998e-05 2024-09-18T11:31:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032246+0000", 2024-09-18T11:31:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2158e-05 2024-09-18T11:31:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037348+0000", 2024-09-18T11:31:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051025319999999999 2024-09-18T11:31:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037377+0000", 2024-09-18T11:31:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8937000000000001e-05 2024-09-18T11:31:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:582 4.6 e91/67 4:6f86ffcc:::smithi11939506-15:26 v 91'78, mlcod=90'72)", 2024-09-18T11:31:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.043567+0000", 2024-09-18T11:31:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.577543945000002, 2024-09-18T11:31:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004448446, 2024-09-18T11:31:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043567+0000", 2024-09-18T11:31:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043567+0000", 2024-09-18T11:31:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043566+0000", 2024-09-18T11:31:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043571+0000", 2024-09-18T11:31:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.181e-06 2024-09-18T11:31:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043579+0000", 2024-09-18T11:31:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9559999999999997e-06 2024-09-18T11:31:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043583+0000", 2024-09-18T11:31:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0770000000000001e-06 2024-09-18T11:31:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043610+0000", 2024-09-18T11:31:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6778e-05 2024-09-18T11:31:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043633+0000", 2024-09-18T11:31:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2904e-05 2024-09-18T11:31:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.047991+0000", 2024-09-18T11:31:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043582910000000002 2024-09-18T11:31:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.048015+0000", 2024-09-18T11:31:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3813000000000001e-05 2024-09-18T11:31:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2754 4.4 e92/22 4:2cb3e6a5:::smithi11939506-3:head v 92'147, mlcod=92'143)", 2024-09-18T11:31:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:17.887302+0000", 2024-09-18T11:31:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.733808517, 2024-09-18T11:31:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044319160000000002, 2024-09-18T11:31:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.700 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.700 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887302+0000", 2024-09-18T11:31:59.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887302+0000", 2024-09-18T11:31:59.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887301+0000", 2024-09-18T11:31:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887313+0000", 2024-09-18T11:31:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2468e-05 2024-09-18T11:31:59.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887331+0000", 2024-09-18T11:31:59.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7362000000000001e-05 2024-09-18T11:31:59.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887338+0000", 2024-09-18T11:31:59.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7980000000000006e-06 2024-09-18T11:31:59.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887416+0000", 2024-09-18T11:31:59.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7577999999999995e-05 2024-09-18T11:31:59.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887441+0000", 2024-09-18T11:31:59.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.531e-05 2024-09-18T11:31:59.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.891671+0000", 2024-09-18T11:31:59.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00422989 2024-09-18T11:31:59.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.891734+0000", 2024-09-18T11:31:59.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2885999999999994e-05 2024-09-18T11:31:59.707 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.707 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.707 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.708 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4178.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-18T11:31:59.708 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:31:59.708 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.68028983, 2024-09-18T11:31:59.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048960139999999997, 2024-09-18T11:31:59.709 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.709 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:31:59.709 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:31:59.709 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4178", 2024-09-18T11:31:59.709 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/490761693", 2024-09-18T11:31:59.709 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4 2024-09-18T11:31:59.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.710 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:31:59.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:31:59.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940819+0000", 2024-09-18T11:31:59.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:31:59.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940832+0000", 2024-09-18T11:31:59.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3155e-05 2024-09-18T11:31:59.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940848+0000", 2024-09-18T11:31:59.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6288999999999999e-05 2024-09-18T11:31:59.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940856+0000", 2024-09-18T11:31:59.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4919999999999996e-06 2024-09-18T11:31:59.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945613+0000", 2024-09-18T11:31:59.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047574999999999996 2024-09-18T11:31:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945717+0000", 2024-09-18T11:31:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103553 2024-09-18T11:31:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.716 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3482 4.6 e117/67 4:67c1765a:::smithi11939506-30:head v 117'120, mlcod=115'116)", 2024-09-18T11:31:59.716 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:31:59.716 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.936435607, 2024-09-18T11:31:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050037479999999997, 2024-09-18T11:31:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:31:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:31:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684674+0000", 2024-09-18T11:31:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684950+0000", 2024-09-18T11:31:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027621200000000002 2024-09-18T11:31:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684968+0000", 2024-09-18T11:31:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8e-05 2024-09-18T11:31:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684984+0000", 2024-09-18T11:31:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5775000000000001e-05 2024-09-18T11:31:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685033+0000", 2024-09-18T11:31:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8878000000000002e-05 2024-09-18T11:31:59.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685059+0000", 2024-09-18T11:31:59.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6801999999999999e-05 2024-09-18T11:31:59.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689640+0000", 2024-09-18T11:31:59.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045809029999999999 2024-09-18T11:31:59.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689679+0000", 2024-09-18T11:31:59.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8507000000000002e-05 2024-09-18T11:31:59.721 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.721 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.721 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.721 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:935 4.4 e128/22 4:2d9c3f7f:::smithi11939506-5:49 v 128'188, mlcod=126'184)", 2024-09-18T11:31:59.721 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:58.221275+0000", 2024-09-18T11:31:59.721 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.399835701, 2024-09-18T11:31:59.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044057319999999999, 2024-09-18T11:31:59.721 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.721 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.722 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.221275+0000", 2024-09-18T11:31:59.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.221275+0000", 2024-09-18T11:31:59.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.221275+0000", 2024-09-18T11:31:59.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:31:59.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.221283+0000", 2024-09-18T11:31:59.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7479999999999999e-06 2024-09-18T11:31:59.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.221294+0000", 2024-09-18T11:31:59.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0986e-05 2024-09-18T11:31:59.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.221302+0000", 2024-09-18T11:31:59.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3069999999999997e-06 2024-09-18T11:31:59.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.221354+0000", 2024-09-18T11:31:59.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.198e-05 2024-09-18T11:31:59.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.221378+0000", 2024-09-18T11:31:59.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4354000000000001e-05 2024-09-18T11:31:59.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.225641+0000", 2024-09-18T11:31:59.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042632939999999999 2024-09-18T11:31:59.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.225681+0000", 2024-09-18T11:31:59.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9289999999999998e-05 2024-09-18T11:31:59.726 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.726 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.726 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.726 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:938 4.6 e128/67 4:67c1765a:::smithi11939506-30:48 v 128'131, mlcod=126'127)", 2024-09-18T11:31:59.727 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:58.226489+0000", 2024-09-18T11:31:59.727 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.3946217359999999, 2024-09-18T11:31:59.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045876129999999999, 2024-09-18T11:31:59.727 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.727 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.728 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.226489+0000", 2024-09-18T11:31:59.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.226489+0000", 2024-09-18T11:31:59.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.226488+0000", 2024-09-18T11:31:59.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:31:59.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.226494+0000", 2024-09-18T11:31:59.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4920000000000001e-06 2024-09-18T11:31:59.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.226503+0000", 2024-09-18T11:31:59.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.721e-06 2024-09-18T11:31:59.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.226508+0000", 2024-09-18T11:31:59.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0629999999999996e-06 2024-09-18T11:31:59.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.226549+0000", 2024-09-18T11:31:59.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1794e-05 2024-09-18T11:31:59.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.226571+0000", 2024-09-18T11:31:59.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1831000000000002e-05 2024-09-18T11:31:59.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.231042+0000", 2024-09-18T11:31:59.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044710729999999999 2024-09-18T11:31:59.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.231076+0000", 2024-09-18T11:31:59.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4122000000000003e-05 2024-09-18T11:31:59.735 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.735 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.735 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.736 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3809 4.d e128/22 4:b44229a1:::smithi11939506-42:head v 128'246, mlcod=128'242)", 2024-09-18T11:31:59.736 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:58.501935+0000", 2024-09-18T11:31:59.736 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.119175928, 2024-09-18T11:31:59.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048034009999999997, 2024-09-18T11:31:59.736 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:31:59.736 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:31:59.737 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:31:59.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:31:59.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.501935+0000", 2024-09-18T11:31:59.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:31:59.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.501935+0000", 2024-09-18T11:31:59.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:31:59.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:31:59.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.501933+0000", 2024-09-18T11:31:59.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:31:59.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:31:59.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.504308+0000", 2024-09-18T11:31:59.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023750609999999999 2024-09-18T11:31:59.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:31:59.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.504329+0000", 2024-09-18T11:31:59.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0933000000000001e-05 2024-09-18T11:31:59.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:31:59.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.504337+0000", 2024-09-18T11:31:59.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0430000000000008e-06 2024-09-18T11:31:59.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:31:59.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.504384+0000", 2024-09-18T11:31:59.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7216000000000003e-05 2024-09-18T11:31:59.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:31:59.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.504405+0000", 2024-09-18T11:31:59.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0341000000000001e-05 2024-09-18T11:31:59.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:31:59.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.506694+0000", 2024-09-18T11:31:59.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022896290000000001 2024-09-18T11:31:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:31:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:31:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:31:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.506738+0000", 2024-09-18T11:31:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3738e-05 2024-09-18T11:31:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.744 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.745 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:31:59.745 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:31:59.745 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:59.747 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:31:59.748 INFO:tasks.rados.rados.0.smithi119.stdout:1682: finishing set_chunk to oid 36 2024-09-18T11:31:59.748 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 36 v 191 (ObjNum 322 snap 45 seq_num 322) dirty exists 2024-09-18T11:31:59.748 INFO:tasks.rados.rados.0.smithi119.stdout:1681: expect (ObjNum 283 snap 39 seq_num 283) 2024-09-18T11:31:59.748 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:31:59.745+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:31:59.848 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:31:59.847+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:31:59.868 INFO:tasks.rados.rados.0.smithi119.stdout:1683: finishing set_chunk to oid 40 2024-09-18T11:31:59.869 INFO:tasks.rados.rados.0.smithi119.stdout:1686: expect (ObjNum 280 snap 37 seq_num 280) 2024-09-18T11:31:59.886 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:31:59.886 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:31:59.886 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:31:59.886 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:31:59.895 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:31:59.948 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:31:59.947+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:31:59.959 INFO:tasks.rados.rados.0.smithi119.stdout:1684: expect (ObjNum 269 snap 31 seq_num 269) 2024-09-18T11:32:00.033 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:00.033 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:00.034 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:00.034 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:00.034 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:00.041 INFO:tasks.rados.rados.0.smithi119.stdout:1687: expect (ObjNum 155 snap 20 seq_num 155) 2024-09-18T11:32:00.042 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:32:00.083 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:00.082+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:00.150 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:00.149+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:00.183 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:00.183 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:00.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:00.183 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:00.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.183 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:32:00.183 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:00.183 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.011137582, 2024-09-18T11:32:00.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:32:00.183 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:00.183 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:00.183 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:32:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:00.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:32:00.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:00.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:00.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:32:00.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:32:00.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:00.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:32:00.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:32:00.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:00.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:32:00.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:32:00.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:32:00.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:32:00.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:00.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:32:00.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:32:00.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:00.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:32:00.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:32:00.187 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.187 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:00.187 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.188 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1001 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:00.188 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:32:00.188 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.01598630399999, 2024-09-18T11:32:00.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007461082, 2024-09-18T11:32:00.188 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:00.188 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:00.188 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:00.188 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:00.188 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:00.188 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1001 2024-09-18T11:32:00.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.188 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:00.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:00.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:32:00.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:00.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:32:00.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:00.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164570+0000", 2024-09-18T11:32:00.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:00.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:00.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164574+0000", 2024-09-18T11:32:00.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.822e-06 2024-09-18T11:32:00.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:00.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164582+0000", 2024-09-18T11:32:00.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3410000000000003e-06 2024-09-18T11:32:00.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:00.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164587+0000", 2024-09-18T11:32:00.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769999999999999e-06 2024-09-18T11:32:00.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164702+0000", 2024-09-18T11:32:00.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115705 2024-09-18T11:32:00.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:00.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164709+0000", 2024-09-18T11:32:00.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-09-18T11:32:00.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170296+0000", 2024-09-18T11:32:00.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055876390000000001 2024-09-18T11:32:00.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:00.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170418+0000", 2024-09-18T11:32:00.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121549 2024-09-18T11:32:00.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:00.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172032+0000", 2024-09-18T11:32:00.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00161407 2024-09-18T11:32:00.192 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.193 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:00.193 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.193 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:00.193 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:00.193 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.015959588, 2024-09-18T11:32:00.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:32:00.193 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:00.193 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:00.193 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:00.193 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:00.193 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:00.193 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:32:00.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.194 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:00.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:00.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:00.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:00.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:00.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:00.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:00.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:00.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:00.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:32:00.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:32:00.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:00.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:32:00.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:32:00.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:00.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:32:00.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:32:00.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:32:00.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:32:00.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:00.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:32:00.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:32:00.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:32:00.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:32:00.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:00.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:32:00.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:32:00.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:00.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:32:00.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:32:00.198 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.198 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:00.198 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.198 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:00.198 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:00.198 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.015941299, 2024-09-18T11:32:00.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:32:00.198 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:00.199 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:00.199 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:00.199 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:00.199 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:00.199 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:32:00.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.199 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:00.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:00.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:00.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:00.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:00.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:00.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:32:00.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:00.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:00.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:32:00.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:32:00.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:00.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:32:00.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:32:00.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:00.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:32:00.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:32:00.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:32:00.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:32:00.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:00.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:32:00.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:32:00.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:32:00.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:32:00.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:00.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:32:00.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:32:00.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:00.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:32:00.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:32:00.203 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.203 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:00.203 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.203 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:00.203 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:00.204 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.01588053, 2024-09-18T11:32:00.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:32:00.204 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:00.204 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:00.204 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:00.204 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:00.204 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:00.204 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:32:00.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.204 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:00.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:00.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:00.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:00.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:00.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:00.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:32:00.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:00.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:00.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:32:00.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:32:00.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:00.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:32:00.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:32:00.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:00.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:32:00.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:32:00.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:32:00.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:32:00.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:00.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:32:00.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:32:00.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:32:00.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:32:00.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:00.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:32:00.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:32:00.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:00.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:32:00.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:32:00.208 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.208 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:00.208 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.209 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:00.209 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:00.209 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.015858209, 2024-09-18T11:32:00.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:32:00.209 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:00.209 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:00.209 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:00.209 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:00.209 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:00.209 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:32:00.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.209 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:00.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:00.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:00.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:00.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:00.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:00.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:32:00.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:00.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:00.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:32:00.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:32:00.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:00.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:32:00.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:32:00.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:00.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:32:00.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:32:00.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:32:00.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:32:00.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:00.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:32:00.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:32:00.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:32:00.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:32:00.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:00.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:32:00.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:32:00.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:00.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:32:00.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:32:00.213 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.214 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:00.214 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.214 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:32:00.214 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:00.214 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 100.234733326, 2024-09-18T11:32:00.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:32:00.214 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:00.214 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:00.214 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:00.214 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:00.214 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:00.215 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:32:00.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.215 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:00.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:00.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:00.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:00.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:00.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:00.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:32:00.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:32:00.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:00.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:32:00.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:32:00.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:00.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:32:00.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:32:00.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:00.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:32:00.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:32:00.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:32:00.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:32:00.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:00.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:32:00.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:32:00.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:00.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:32:00.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:32:00.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:00.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:32:00.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:32:00.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:00.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:32:00.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:32:00.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:00.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:32:00.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:32:00.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:00.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:32:00.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:32:00.219 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.219 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:00.219 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.220 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1661 4.0 4:019b6e09:::smithi11939506-40:17 [read 0~3445402,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:32:00.220 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:00.220 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 82.830660623, 2024-09-18T11:32:00.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075190880000000002, 2024-09-18T11:32:00.220 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:00.220 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:00.220 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:00.220 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:00.220 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:00.220 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1661 2024-09-18T11:32:00.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.220 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:00.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:00.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:00.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:00.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:00.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:00.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:00.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:00.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:00.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349899+0000", 2024-09-18T11:32:00.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9749999999999999e-06 2024-09-18T11:32:00.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:00.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349906+0000", 2024-09-18T11:32:00.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9290000000000001e-06 2024-09-18T11:32:00.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:00.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349912+0000", 2024-09-18T11:32:00.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9900000000000002e-06 2024-09-18T11:32:00.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355645+0000", 2024-09-18T11:32:00.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057329549999999996 2024-09-18T11:32:00.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:00.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355776+0000", 2024-09-18T11:32:00.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130975 2024-09-18T11:32:00.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:00.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357416+0000", 2024-09-18T11:32:00.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001639649 2024-09-18T11:32:00.224 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.224 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:00.224 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.224 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1761 4.0 4:0aec5e1c:::smithi11939506-50:head [write 769787~704463,stat] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:32:00.224 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:32:00.224 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.878168208999995, 2024-09-18T11:32:00.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074082879999999999, 2024-09-18T11:32:00.224 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:00.224 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:00.224 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:00.225 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:00.225 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:00.225 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1761 2024-09-18T11:32:00.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.225 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:00.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:00.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:32:00.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:00.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:32:00.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:00.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302386+0000", 2024-09-18T11:32:00.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:00.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:00.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302889+0000", 2024-09-18T11:32:00.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050318499999999996 2024-09-18T11:32:00.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:00.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302930+0000", 2024-09-18T11:32:00.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0198999999999999e-05 2024-09-18T11:32:00.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:00.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302965+0000", 2024-09-18T11:32:00.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5292e-05 2024-09-18T11:32:00.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303046+0000", 2024-09-18T11:32:00.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1254999999999994e-05 2024-09-18T11:32:00.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:32:00.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303054+0000", 2024-09-18T11:32:00.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2800000000000003e-06 2024-09-18T11:32:00.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304658+0000", 2024-09-18T11:32:00.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016039520000000001 2024-09-18T11:32:00.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:00.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304724+0000", 2024-09-18T11:32:00.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5888000000000003e-05 2024-09-18T11:32:00.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:00.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304853+0000", 2024-09-18T11:32:00.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012912700000000001 2024-09-18T11:32:00.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:00.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309084+0000", 2024-09-18T11:32:00.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042311229999999998 2024-09-18T11:32:00.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:00.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309738+0000", 2024-09-18T11:32:00.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00065401399999999996 2024-09-18T11:32:00.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:00.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309770+0000", 2024-09-18T11:32:00.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1652000000000001e-05 2024-09-18T11:32:00.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:00.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309797+0000", 2024-09-18T11:32:00.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7052000000000001e-05 2024-09-18T11:32:00.231 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.231 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:00.231 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.231 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:306 4.f e69/22 4:f5d7df41:::smithi11939506-16:17 v 69'94, mlcod=65'88)", 2024-09-18T11:32:00.231 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:32:00.231 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.829441492000001, 2024-09-18T11:32:00.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068024720000000004, 2024-09-18T11:32:00.231 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:00.231 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:00.231 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:00.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:00.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:32:00.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:00.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:32:00.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:00.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351115+0000", 2024-09-18T11:32:00.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:00.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:00.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351121+0000", 2024-09-18T11:32:00.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-09-18T11:32:00.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:00.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351126+0000", 2024-09-18T11:32:00.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5969999999999999e-06 2024-09-18T11:32:00.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:00.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351147+0000", 2024-09-18T11:32:00.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1160000000000001e-05 2024-09-18T11:32:00.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351518+0000", 2024-09-18T11:32:00.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00037079300000000001 2024-09-18T11:32:00.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:00.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351536+0000", 2024-09-18T11:32:00.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8168999999999998e-05 2024-09-18T11:32:00.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:00.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357870+0000", 2024-09-18T11:32:00.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063332650000000002 2024-09-18T11:32:00.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:00.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357918+0000", 2024-09-18T11:32:00.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8347999999999999e-05 2024-09-18T11:32:00.235 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.235 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:00.235 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.235 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2022 4.0 4:0aec5e1c:::smithi11939506-50:head [write 457190~552096,stat] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-18T11:32:00.235 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:32:00.236 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 67.184403355000001, 2024-09-18T11:32:00.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006803974, 2024-09-18T11:32:00.236 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:00.236 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:00.236 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:00.236 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:00.236 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:00.236 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2022 2024-09-18T11:32:00.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.236 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:00.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:00.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:32:00.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:00.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:32:00.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:00.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996151+0000", 2024-09-18T11:32:00.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:00.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:00.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996461+0000", 2024-09-18T11:32:00.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000309587 2024-09-18T11:32:00.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:00.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996481+0000", 2024-09-18T11:32:00.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9949000000000002e-05 2024-09-18T11:32:00.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:00.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996489+0000", 2024-09-18T11:32:00.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9129999999999992e-06 2024-09-18T11:32:00.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996562+0000", 2024-09-18T11:32:00.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2807999999999998e-05 2024-09-18T11:32:00.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:00.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996687+0000", 2024-09-18T11:32:00.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012458200000000001 2024-09-18T11:32:00.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.239 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:00.239 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996909+0000", 2024-09-18T11:32:00.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222124 2024-09-18T11:32:00.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:00.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.000741+0000", 2024-09-18T11:32:00.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038320720000000002 2024-09-18T11:32:00.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:00.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002854+0000", 2024-09-18T11:32:00.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002113228 2024-09-18T11:32:00.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:00.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002911+0000", 2024-09-18T11:32:00.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7192999999999999e-05 2024-09-18T11:32:00.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:00.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002958+0000", 2024-09-18T11:32:00.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6242000000000002e-05 2024-09-18T11:32:00.241 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.241 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:00.241 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.241 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2503 3.1 e85/67 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head v 85'117, mlcod=84'115)", 2024-09-18T11:32:00.241 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:00.241 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 51.203334638999998, 2024-09-18T11:32:00.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086527619999999996, 2024-09-18T11:32:00.242 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:00.242 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:00.242 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:00.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:00.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:00.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:00.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:00.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:00.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977220+0000", 2024-09-18T11:32:00.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:00.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:00.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980771+0000", 2024-09-18T11:32:00.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035505200000000002 2024-09-18T11:32:00.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:00.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980797+0000", 2024-09-18T11:32:00.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.618e-05 2024-09-18T11:32:00.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:00.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980807+0000", 2024-09-18T11:32:00.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0433e-05 2024-09-18T11:32:00.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980861+0000", 2024-09-18T11:32:00.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3415999999999998e-05 2024-09-18T11:32:00.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:00.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980885+0000", 2024-09-18T11:32:00.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4212000000000001e-05 2024-09-18T11:32:00.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:00.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985826+0000", 2024-09-18T11:32:00.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049412450000000004 2024-09-18T11:32:00.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:00.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985875+0000", 2024-09-18T11:32:00.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9116000000000002e-05 2024-09-18T11:32:00.245 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.245 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:00.246 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.246 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2648 4.0 4:059d8d95:::smithi11939506-32:head [read 0~3647119,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:00.246 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:32:00.246 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.849246268999998, 2024-09-18T11:32:00.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0076286949999999996, 2024-09-18T11:32:00.246 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:00.246 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:00.246 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:00.246 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:00.246 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:00.246 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2648 2024-09-18T11:32:00.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.247 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:00.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:00.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:32:00.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:00.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:32:00.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:00.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331309+0000", 2024-09-18T11:32:00.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:00.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:00.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331321+0000", 2024-09-18T11:32:00.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:32:00.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:00.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331340+0000", 2024-09-18T11:32:00.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8227e-05 2024-09-18T11:32:00.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:00.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331348+0000", 2024-09-18T11:32:00.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4640000000000006e-06 2024-09-18T11:32:00.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331406+0000", 2024-09-18T11:32:00.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7445000000000003e-05 2024-09-18T11:32:00.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:00.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331521+0000", 2024-09-18T11:32:00.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115945 2024-09-18T11:32:00.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:00.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338939+0000", 2024-09-18T11:32:00.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074179859999999997 2024-09-18T11:32:00.250 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.250 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:00.250 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.250 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2649 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:00.251 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:00.251 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.849191269999999, 2024-09-18T11:32:00.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566240000000007, 2024-09-18T11:32:00.251 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:00.251 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:00.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:00.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:00.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:00.251 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2649 2024-09-18T11:32:00.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.251 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:00.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:00.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:00.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:00.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:00.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:00.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331365+0000", 2024-09-18T11:32:00.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:00.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:00.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331369+0000", 2024-09-18T11:32:00.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.551e-06 2024-09-18T11:32:00.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:00.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331377+0000", 2024-09-18T11:32:00.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8769999999999993e-06 2024-09-18T11:32:00.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:00.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331381+0000", 2024-09-18T11:32:00.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.16e-06 2024-09-18T11:32:00.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338973+0000", 2024-09-18T11:32:00.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075918460000000002 2024-09-18T11:32:00.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:00.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339048+0000", 2024-09-18T11:32:00.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5813000000000001e-05 2024-09-18T11:32:00.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:00.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339322+0000", 2024-09-18T11:32:00.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027399899999999998 2024-09-18T11:32:00.255 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.255 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:00.255 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.255 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2650 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:00.255 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:00.255 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.849171052000003, 2024-09-18T11:32:00.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084392180000000001, 2024-09-18T11:32:00.255 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:00.255 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:00.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:00.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:00.256 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:00.256 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2650 2024-09-18T11:32:00.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.256 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:00.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:00.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:00.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:00.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:00.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:00.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:00.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:00.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:00.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331388+0000", 2024-09-18T11:32:00.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3429999999999998e-06 2024-09-18T11:32:00.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:00.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331394+0000", 2024-09-18T11:32:00.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.19e-06 2024-09-18T11:32:00.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:00.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331398+0000", 2024-09-18T11:32:00.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.495e-06 2024-09-18T11:32:00.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339362+0000", 2024-09-18T11:32:00.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079641799999999995 2024-09-18T11:32:00.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:00.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339460+0000", 2024-09-18T11:32:00.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8329e-05 2024-09-18T11:32:00.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.259 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:00.259 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339825+0000", 2024-09-18T11:32:00.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036510500000000001 2024-09-18T11:32:00.259 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.259 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:00.259 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.260 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:119 3.2 3:5595a68d:::07854d2fef297a06ba81685e660c332de36d5d18d546927d30daad6d7fda1541:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:00.260 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:32:00.260 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.137019295000002, 2024-09-18T11:32:00.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074145979999999997, 2024-09-18T11:32:00.260 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:00.260 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:00.260 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:00.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:00.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:32:00.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:00.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:32:00.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:00.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043535+0000", 2024-09-18T11:32:00.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:00.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:00.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043970+0000", 2024-09-18T11:32:00.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000435148 2024-09-18T11:32:00.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:00.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043993+0000", 2024-09-18T11:32:00.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3113999999999999e-05 2024-09-18T11:32:00.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:00.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044003+0000", 2024-09-18T11:32:00.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5310000000000007e-06 2024-09-18T11:32:00.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044062+0000", 2024-09-18T11:32:00.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9487000000000002e-05 2024-09-18T11:32:00.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:32:00.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044071+0000", 2024-09-18T11:32:00.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1610000000000003e-06 2024-09-18T11:32:00.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.263 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046423+0000", 2024-09-18T11:32:00.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023523680000000001 2024-09-18T11:32:00.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:00.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046502+0000", 2024-09-18T11:32:00.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9339999999999996e-05 2024-09-18T11:32:00.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:00.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046664+0000", 2024-09-18T11:32:00.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016185700000000001 2024-09-18T11:32:00.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:00.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050870+0000", 2024-09-18T11:32:00.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042060070000000003 2024-09-18T11:32:00.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:00.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050878+0000", 2024-09-18T11:32:00.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8960000000000003e-06 2024-09-18T11:32:00.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:00.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050918+0000", 2024-09-18T11:32:00.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0367000000000002e-05 2024-09-18T11:32:00.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:00.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050952+0000", 2024-09-18T11:32:00.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3989999999999998e-05 2024-09-18T11:32:00.266 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.266 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:00.266 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.266 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:124 3.2 3:4326e841:::a6567eb8c0be0365cdfccf2ece66d60995301bc908261a0c1daff039d2080a5f:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:00.266 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044017+0000", 2024-09-18T11:32:00.266 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.136539685000002, 2024-09-18T11:32:00.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070611140000000003, 2024-09-18T11:32:00.266 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:00.266 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:00.266 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:00.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:00.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044017+0000", 2024-09-18T11:32:00.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:00.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044017+0000", 2024-09-18T11:32:00.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:00.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044016+0000", 2024-09-18T11:32:00.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:00.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.267 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:00.267 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044102+0000", 2024-09-18T11:32:00.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5756999999999999e-05 2024-09-18T11:32:00.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:00.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044114+0000", 2024-09-18T11:32:00.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1505000000000001e-05 2024-09-18T11:32:00.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:00.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044120+0000", 2024-09-18T11:32:00.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8549999999999998e-06 2024-09-18T11:32:00.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044154+0000", 2024-09-18T11:32:00.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4773000000000001e-05 2024-09-18T11:32:00.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:00.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044159+0000", 2024-09-18T11:32:00.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5310000000000003e-06 2024-09-18T11:32:00.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046975+0000", 2024-09-18T11:32:00.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028162700000000001 2024-09-18T11:32:00.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:00.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.047116+0000", 2024-09-18T11:32:00.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014086400000000001 2024-09-18T11:32:00.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:00.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.047336+0000", 2024-09-18T11:32:00.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021975299999999999 2024-09-18T11:32:00.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:00.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050964+0000", 2024-09-18T11:32:00.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036283000000000001 2024-09-18T11:32:00.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:00.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.051005+0000", 2024-09-18T11:32:00.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0763999999999998e-05 2024-09-18T11:32:00.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:00.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.051050+0000", 2024-09-18T11:32:00.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4993000000000001e-05 2024-09-18T11:32:00.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:00.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.051078+0000", 2024-09-18T11:32:00.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8694999999999999e-05 2024-09-18T11:32:00.272 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.272 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:00.272 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.272 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:127 3.2 3:4d3b277a:::3266c7aaa53a7d2c10c3c54d852d6cbbed49fb041bc0df41a3ac83b56e2b0615:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:00.272 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:00.272 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 37.136156112999998, 2024-09-18T11:32:00.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093148009999999993, 2024-09-18T11:32:00.272 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:00.272 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:00.272 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:00.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:00.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:00.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:00.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:00.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:00.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044400+0000", 2024-09-18T11:32:00.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:00.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:00.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044442+0000", 2024-09-18T11:32:00.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2247999999999999e-05 2024-09-18T11:32:00.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:00.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044453+0000", 2024-09-18T11:32:00.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0617e-05 2024-09-18T11:32:00.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:00.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044457+0000", 2024-09-18T11:32:00.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2520000000000001e-06 2024-09-18T11:32:00.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044495+0000", 2024-09-18T11:32:00.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7710000000000003e-05 2024-09-18T11:32:00.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:00.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044499+0000", 2024-09-18T11:32:00.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6269999999999999e-06 2024-09-18T11:32:00.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048239+0000", 2024-09-18T11:32:00.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037396719999999999 2024-09-18T11:32:00.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:00.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048351+0000", 2024-09-18T11:32:00.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112342 2024-09-18T11:32:00.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:00.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048545+0000", 2024-09-18T11:32:00.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000193881 2024-09-18T11:32:00.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:00.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.052689+0000", 2024-09-18T11:32:00.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041435780000000002 2024-09-18T11:32:00.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:00.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053654+0000", 2024-09-18T11:32:00.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096558199999999999 2024-09-18T11:32:00.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:00.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053683+0000", 2024-09-18T11:32:00.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8850999999999999e-05 2024-09-18T11:32:00.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:00.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053716+0000", 2024-09-18T11:32:00.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2468000000000002e-05 2024-09-18T11:32:00.278 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.278 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:00.278 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.278 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2963 4.0 4:019b6e09:::smithi11939506-40:head [sparse-read 0~1093084] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:00.278 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:32:00.278 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.153147314999998, 2024-09-18T11:32:00.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072292880000000004, 2024-09-18T11:32:00.278 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:00.278 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:00.279 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:00.279 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:00.279 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:00.279 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2963 2024-09-18T11:32:00.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.279 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:00.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:00.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:32:00.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:00.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:32:00.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:00.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027409+0000", 2024-09-18T11:32:00.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:00.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:00.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027412+0000", 2024-09-18T11:32:00.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.751e-06 2024-09-18T11:32:00.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:00.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027418+0000", 2024-09-18T11:32:00.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.404e-06 2024-09-18T11:32:00.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:00.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027423+0000", 2024-09-18T11:32:00.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7119999999999999e-06 2024-09-18T11:32:00.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034156+0000", 2024-09-18T11:32:00.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067331639999999998 2024-09-18T11:32:00.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:00.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034255+0000", 2024-09-18T11:32:00.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8701000000000002e-05 2024-09-18T11:32:00.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:00.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034639+0000", 2024-09-18T11:32:00.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000384006 2024-09-18T11:32:00.282 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.282 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:00.282 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.283 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3209 4.0 4:019b6e09:::smithi11939506-40:head [write 638732~557432] snapc 42=[42,41,3f,3e,3d,3b,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-18T11:32:00.283 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:00.283 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.082474074, 2024-09-18T11:32:00.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091774999999999999, 2024-09-18T11:32:00.283 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:00.283 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:00.283 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:00.283 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:00.283 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:00.283 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3209 2024-09-18T11:32:00.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.283 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:00.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098080+0000", 2024-09-18T11:32:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:00.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098676+0000", 2024-09-18T11:32:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059598099999999998 2024-09-18T11:32:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098706+0000", 2024-09-18T11:32:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0020000000000001e-05 2024-09-18T11:32:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098716+0000", 2024-09-18T11:32:00.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0225e-05 2024-09-18T11:32:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098782+0000", 2024-09-18T11:32:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5530000000000004e-05 2024-09-18T11:32:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098883+0000", 2024-09-18T11:32:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101609 2024-09-18T11:32:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.099098+0000", 2024-09-18T11:32:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000215052 2024-09-18T11:32:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.102511+0000", 2024-09-18T11:32:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034129429999999999 2024-09-18T11:32:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107165+0000", 2024-09-18T11:32:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046531480000000002 2024-09-18T11:32:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107217+0000", 2024-09-18T11:32:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2058000000000002e-05 2024-09-18T11:32:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107260+0000", 2024-09-18T11:32:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3865000000000001e-05 2024-09-18T11:32:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:00.288 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:00.288 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:00.292 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:32:00.292 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T11:32:00.293 INFO:tasks.rados.rados.0.smithi119.stdout:1688: seq_num 340 ranges {514380=704192,1629760=478435,2604288=442602} 2024-09-18T11:32:00.294 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:32:00.298 INFO:tasks.rados.rados.0.smithi119.stdout:1688: writing smithi11939506-45 from 514380 to 1218572 tid 1 2024-09-18T11:32:00.298 INFO:tasks.rados.rados.0.smithi119.stdout:1688: writing smithi11939506-45 from 1629760 to 2108195 tid 2 2024-09-18T11:32:00.298 INFO:tasks.rados.rados.0.smithi119.stdout:1688: writing smithi11939506-45 from 2604288 to 3046890 tid 3 2024-09-18T11:32:00.298 INFO:tasks.rados.rados.0.smithi119.stdout:1676: done (11 left) 2024-09-18T11:32:00.298 INFO:tasks.rados.rados.0.smithi119.stdout:1677: done (10 left) 2024-09-18T11:32:00.298 INFO:tasks.rados.rados.0.smithi119.stdout:1679: done (9 left) 2024-09-18T11:32:00.298 INFO:tasks.rados.rados.0.smithi119.stdout:1680: done (8 left) 2024-09-18T11:32:00.298 INFO:tasks.rados.rados.0.smithi119.stdout:1681: done (7 left) 2024-09-18T11:32:00.298 INFO:tasks.rados.rados.0.smithi119.stdout:1682: done (6 left) 2024-09-18T11:32:00.299 INFO:tasks.rados.rados.0.smithi119.stdout:1683: done (5 left) 2024-09-18T11:32:00.299 INFO:tasks.rados.rados.0.smithi119.stdout:1684: done (4 left) 2024-09-18T11:32:00.299 INFO:tasks.rados.rados.0.smithi119.stdout:1685: done (3 left) 2024-09-18T11:32:00.299 INFO:tasks.rados.rados.0.smithi119.stdout:1686: done (2 left) 2024-09-18T11:32:00.299 INFO:tasks.rados.rados.0.smithi119.stdout:1687: done (1 left) 2024-09-18T11:32:00.299 INFO:tasks.rados.rados.0.smithi119.stdout:1689: rollback oid 29 current snap is 49 2024-09-18T11:32:00.299 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 29 to 46 2024-09-18T11:32:00.299 INFO:tasks.rados.rados.0.smithi119.stdout:1688: finishing write tid 1 to smithi11939506-45 2024-09-18T11:32:00.299 INFO:tasks.rados.rados.0.smithi119.stdout:1688: finishing write tid 2 to smithi11939506-45 2024-09-18T11:32:00.299 INFO:tasks.rados.rados.0.smithi119.stdout:1690: read oid 49 snap -1 2024-09-18T11:32:00.299 INFO:tasks.rados.rados.0.smithi119.stdout:1690: expect (ObjNum 323 snap 45 seq_num 323) 2024-09-18T11:32:00.299 INFO:tasks.rados.rados.0.smithi119.stdout:1691: write (excl) oid 50 current snap is 49 2024-09-18T11:32:00.299 INFO:tasks.rados.rados.0.smithi119.stdout:1691: seq_num 341 ranges {531187=779441,2088242=738446,3431711=299476} 2024-09-18T11:32:00.299 INFO:tasks.rados.rados.0.smithi119.stdout:1691: writing smithi11939506-50 from 531187 to 1310628 tid 1 2024-09-18T11:32:00.300 INFO:tasks.rados.rados.0.smithi119.stdout:1691: writing smithi11939506-50 from 2088242 to 2826688 tid 2 2024-09-18T11:32:00.300 INFO:tasks.rados.rados.0.smithi119.stdout:1691: writing smithi11939506-50 from 3431711 to 3731187 tid 3 2024-09-18T11:32:00.300 INFO:tasks.rados.rados.0.smithi119.stdout:1692: read oid 13 snap 47 2024-09-18T11:32:00.300 INFO:tasks.rados.rados.0.smithi119.stdout:1692: expect (ObjNum 309 snap 43 seq_num 309) 2024-09-18T11:32:00.300 INFO:tasks.rados.rados.0.smithi119.stdout:1693: read oid 6 snap -1 2024-09-18T11:32:00.300 INFO:tasks.rados.rados.0.smithi119.stdout:1693: expect (ObjNum 327 snap 45 seq_num 327) 2024-09-18T11:32:00.300 INFO:tasks.rados.rados.0.smithi119.stdout:1694: set_chunk oid 48 target oid 2024-09-18T11:32:00.300 INFO:tasks.rados.rados.0.smithi119.stdout:1694: set_chunk oid 48 offset: 281088 length: 1125888 target oid 5f3769fbb26ca4116f6139cc8c9e0881e30aa6aaed021f12c71627df8132d073 offset: 0 snap -1 2024-09-18T11:32:00.300 INFO:tasks.rados.rados.0.smithi119.stdout:1695: read oid 3 snap -1 2024-09-18T11:32:00.300 INFO:tasks.rados.rados.0.smithi119.stdout:1695: expect (ObjNum 243 snap 28 seq_num 243) 2024-09-18T11:32:00.300 INFO:tasks.rados.rados.0.smithi119.stdout:1696: write (excl) oid 37 current snap is 49 2024-09-18T11:32:00.300 INFO:tasks.rados.rados.0.smithi119.stdout:1689: finishing rollback tid 0 to smithi11939506-29 2024-09-18T11:32:00.300 INFO:tasks.rados.rados.0.smithi119.stdout:1696: seq_num 342 ranges {547994=454690,1746724=482104,2659134=465146,3605976=142018} 2024-09-18T11:32:00.301 INFO:tasks.rados.rados.0.smithi119.stdout:1696: writing smithi11939506-37 from 547994 to 1002684 tid 1 2024-09-18T11:32:00.301 INFO:tasks.rados.rados.0.smithi119.stdout:1696: writing smithi11939506-37 from 1746724 to 2228828 tid 2 2024-09-18T11:32:00.301 INFO:tasks.rados.rados.0.smithi119.stdout:1696: writing smithi11939506-37 from 2659134 to 3124280 tid 3 2024-09-18T11:32:00.301 INFO:tasks.rados.rados.0.smithi119.stdout:1696: writing smithi11939506-37 from 3605976 to 3747994 tid 4 2024-09-18T11:32:00.301 INFO:tasks.rados.rados.0.smithi119.stdout:1697: read oid 14 snap -1 2024-09-18T11:32:00.301 INFO:tasks.rados.rados.0.smithi119.stdout:1697: expect (ObjNum 246 snap 28 seq_num 246) 2024-09-18T11:32:00.301 INFO:tasks.rados.rados.0.smithi119.stdout:1698: delete oid 30 current snap is 49 2024-09-18T11:32:00.301 INFO:tasks.rados.rados.0.smithi119.stdout:1688: finishing write tid 3 to smithi11939506-45 2024-09-18T11:32:00.301 INFO:tasks.rados.rados.0.smithi119.stdout:1688: finishing write tid 4 to smithi11939506-45 2024-09-18T11:32:00.301 INFO:tasks.rados.rados.0.smithi119.stdout:1689: finishing rollback tid 1 to smithi11939506-29 2024-09-18T11:32:00.301 INFO:tasks.rados.rados.0.smithi119.stdout:1689: finishing rollback tid 2 to smithi11939506-29 2024-09-18T11:32:00.301 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 95 (ObjNum 210 snap 23 seq_num 210) dirty exists 2024-09-18T11:32:00.301 INFO:tasks.rados.rados.0.smithi119.stdout:1688: finishing write tid 5 to smithi11939506-45 2024-09-18T11:32:00.301 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 81 (ObjNum 340 snap 49 seq_num 340) dirty exists 2024-09-18T11:32:00.302 INFO:tasks.rados.rados.0.smithi119.stdout:1688: left oid 45 (ObjNum 340 snap 49 seq_num 340) 2024-09-18T11:32:00.302 INFO:tasks.rados.rados.0.smithi119.stdout:1690: expect (ObjNum 323 snap 45 seq_num 323) 2024-09-18T11:32:00.302 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:00.292+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:00.394 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:00.392+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:00.403 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:00.403 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:00.404 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:00.404 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:00.412 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:32:00.452 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:00.453 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:00.482 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:00.482 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:00.497 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:00.496+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:00.499 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:00.499 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:00.506 INFO:tasks.rados.rados.0.smithi119.stdout:1691: finishing write tid 1 to smithi11939506-50 2024-09-18T11:32:00.506 INFO:tasks.rados.rados.0.smithi119.stdout:1691: finishing write tid 2 to smithi11939506-50 2024-09-18T11:32:00.506 INFO:tasks.rados.rados.0.smithi119.stdout:1691: finishing write tid 3 to smithi11939506-50 2024-09-18T11:32:00.507 INFO:tasks.rados.rados.0.smithi119.stdout:1691: finishing write tid 4 to smithi11939506-50 2024-09-18T11:32:00.507 INFO:tasks.rados.rados.0.smithi119.stdout:1691: finishing write tid 5 to smithi11939506-50 2024-09-18T11:32:00.507 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 198 (ObjNum 341 snap 49 seq_num 341) dirty exists 2024-09-18T11:32:00.507 INFO:tasks.rados.rados.0.smithi119.stdout:1691: left oid 50 (ObjNum 341 snap 49 seq_num 341) 2024-09-18T11:32:00.507 INFO:tasks.rados.rados.0.smithi119.stdout:1693: expect (ObjNum 327 snap 45 seq_num 327) 2024-09-18T11:32:00.514 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:00.514 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:00.543 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:00.543 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:00.551 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:00.551 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:00.558 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:00.558 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:00.559 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:32:00.589 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:00.590 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:00.598 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:00.597+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:00.606 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:00.607 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:00.622 INFO:tasks.rados.rados.0.smithi119.stdout:1692: expect (ObjNum 309 snap 43 seq_num 309) 2024-09-18T11:32:00.660 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:00.660 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:00.669 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:32:00.712 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:00.710+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:00.766 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:00.767 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:00.775 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:32:00.799 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:00.798+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:00.822 INFO:tasks.rados.rados.0.smithi119.stdout:1695: expect (ObjNum 243 snap 28 seq_num 243) 2024-09-18T11:32:00.901 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:00.900+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:00.913 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:00.914 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:00.922 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:32:00.935 INFO:tasks.rados.rados.0.smithi119.stdout:1694: finishing set_chunk to oid 48 2024-09-18T11:32:00.936 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 48 v 241 (ObjNum 300 snap 43 seq_num 300) dirty exists 2024-09-18T11:32:00.936 INFO:tasks.rados.rados.0.smithi119.stdout:1696: finishing write tid 1 to smithi11939506-37 2024-09-18T11:32:00.936 INFO:tasks.rados.rados.0.smithi119.stdout:1696: finishing write tid 2 to smithi11939506-37 2024-09-18T11:32:00.936 INFO:tasks.rados.rados.0.smithi119.stdout:1696: finishing write tid 3 to smithi11939506-37 2024-09-18T11:32:00.936 INFO:tasks.rados.rados.0.smithi119.stdout:1696: finishing write tid 4 to smithi11939506-37 2024-09-18T11:32:00.936 INFO:tasks.rados.rados.0.smithi119.stdout:1696: finishing write tid 5 to smithi11939506-37 2024-09-18T11:32:00.936 INFO:tasks.rados.rados.0.smithi119.stdout:1696: finishing write tid 6 to smithi11939506-37 2024-09-18T11:32:00.936 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 280 (ObjNum 342 snap 49 seq_num 342) dirty exists 2024-09-18T11:32:00.936 INFO:tasks.rados.rados.0.smithi119.stdout:1696: left oid 37 (ObjNum 342 snap 49 seq_num 342) 2024-09-18T11:32:00.936 INFO:tasks.rados.rados.0.smithi119.stdout:1697: expect (ObjNum 246 snap 28 seq_num 246) 2024-09-18T11:32:01.002 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:00.999+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:01.019 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:01.020 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:01.028 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:32:01.102 INFO:tasks.rados.rados.0.smithi119.stdout:1688: done (10 left) 2024-09-18T11:32:01.102 INFO:tasks.rados.rados.0.smithi119.stdout:1689: done (9 left) 2024-09-18T11:32:01.102 INFO:tasks.rados.rados.0.smithi119.stdout:1690: done (8 left) 2024-09-18T11:32:01.102 INFO:tasks.rados.rados.0.smithi119.stdout:1691: done (7 left) 2024-09-18T11:32:01.103 INFO:tasks.rados.rados.0.smithi119.stdout:1692: done (6 left) 2024-09-18T11:32:01.103 INFO:tasks.rados.rados.0.smithi119.stdout:1693: done (5 left) 2024-09-18T11:32:01.103 INFO:tasks.rados.rados.0.smithi119.stdout:1694: done (4 left) 2024-09-18T11:32:01.103 INFO:tasks.rados.rados.0.smithi119.stdout:1695: done (3 left) 2024-09-18T11:32:01.103 INFO:tasks.rados.rados.0.smithi119.stdout:1696: done (2 left) 2024-09-18T11:32:01.103 INFO:tasks.rados.rados.0.smithi119.stdout:1697: done (1 left) 2024-09-18T11:32:01.103 INFO:tasks.rados.rados.0.smithi119.stdout:1698: done (0 left) 2024-09-18T11:32:01.103 INFO:tasks.rados.rados.0.smithi119.stdout:1699: read oid 9 snap -1 2024-09-18T11:32:01.103 INFO:tasks.rados.rados.0.smithi119.stdout:1699: expect (ObjNum 335 snap 48 seq_num 335) 2024-09-18T11:32:01.103 INFO:tasks.rados.rados.0.smithi119.stdout:1700: snap_create 2024-09-18T11:32:01.108 INFO:tasks.rados.rados.0.smithi119.stdout:1699: expect (ObjNum 335 snap 48 seq_num 335) 2024-09-18T11:32:01.112 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:01.110+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:01.166 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:01.167 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:01.175 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:32:01.204 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:01.202+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:01.218 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-18T11:32:01.219 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T11:32:01.219 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-18T11:32:01.219 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 0.05 2024-09-18T11:32:01.220 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:01.218+0000 7fb6950ff640 -1 osd.2 129 handle_osd_map injecting map crc failure 2024-09-18T11:32:01.222 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is set 2024-09-18T11:32:01.305 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:01.302+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:01.313 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:01.313 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:01.321 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:32:01.327 INFO:tasks.rados.rados.0.smithi119.stdout:1699: done (1 left) 2024-09-18T11:32:01.327 INFO:tasks.rados.rados.0.smithi119.stdout:1700: done (0 left) 2024-09-18T11:32:01.327 INFO:tasks.rados.rados.0.smithi119.stdout:1701: set_chunk oid 50 target oid 2024-09-18T11:32:01.336 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:01.333+0000 7fd27659f640 -1 osd.6 129 handle_osd_map injecting map crc failure 2024-09-18T11:32:01.339 INFO:tasks.rados.rados.0.smithi119.stdout:1701: set_chunk oid 50 offset: 2331136 length: 1193472 target oid e4b2e4b69a5b1963beb46fe9c96fc726683a817b38123dd3551bacc24d120d56 offset: 0 snap -1 2024-09-18T11:32:01.339 INFO:tasks.rados.rados.0.smithi119.stdout:1702: write oid 47 current snap is 50 2024-09-18T11:32:01.339 INFO:tasks.rados.rados.0.smithi119.stdout:1702: seq_num 344 ranges {581608=721541,1863688=485773,3113980=604043} 2024-09-18T11:32:01.346 INFO:tasks.rados.rados.0.smithi119.stdout:1702: writing smithi11939506-47 from 581608 to 1303149 tid 1 2024-09-18T11:32:01.351 INFO:tasks.rados.rados.0.smithi119.stdout:1702: writing smithi11939506-47 from 1863688 to 2349461 tid 2 2024-09-18T11:32:01.359 INFO:tasks.rados.rados.0.smithi119.stdout:1702: writing smithi11939506-47 from 3113980 to 3718023 tid 3 2024-09-18T11:32:01.359 INFO:tasks.rados.rados.0.smithi119.stdout:1703: snap_create 2024-09-18T11:32:01.359 INFO:tasks.rados.rados.0.smithi119.stdout:1701: finishing set_chunk to oid 50 2024-09-18T11:32:01.359 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 198 (ObjNum 341 snap 49 seq_num 341) dirty exists 2024-09-18T11:32:01.359 INFO:tasks.rados.rados.0.smithi119.stdout:1702: finishing write tid 1 to smithi11939506-47 2024-09-18T11:32:01.360 INFO:tasks.rados.rados.0.smithi119.stdout:1702: finishing write tid 2 to smithi11939506-47 2024-09-18T11:32:01.362 INFO:tasks.rados.rados.0.smithi119.stdout:1702: finishing write tid 3 to smithi11939506-47 2024-09-18T11:32:01.362 INFO:tasks.rados.rados.0.smithi119.stdout:1702: finishing write tid 4 to smithi11939506-47 2024-09-18T11:32:01.362 INFO:tasks.rados.rados.0.smithi119.stdout:1702: finishing write tid 5 to smithi11939506-47 2024-09-18T11:32:01.362 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 149 (ObjNum 344 snap 50 seq_num 344) dirty exists 2024-09-18T11:32:01.363 INFO:tasks.rados.rados.0.smithi119.stdout:1702: left oid 47 (ObjNum 344 snap 50 seq_num 344) 2024-09-18T11:32:01.406 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:01.405+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:01.436 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:01.436 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:01.444 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:32:01.486 INFO:teuthology.orchestra.run.smithi119.stdout:moved 0 / 58 (0%) 2024-09-18T11:32:01.487 INFO:teuthology.orchestra.run.smithi119.stdout:avg 8.28571 2024-09-18T11:32:01.487 INFO:teuthology.orchestra.run.smithi119.stdout:stddev 2.31234 -> 2.31234 (expected baseline 2.66497) 2024-09-18T11:32:01.487 INFO:teuthology.orchestra.run.smithi119.stdout:min osd.2 with 6 -> 6 pgs (0.724138 -> 0.724138 * mean) 2024-09-18T11:32:01.487 INFO:teuthology.orchestra.run.smithi119.stdout:max osd.1 with 13 -> 13 pgs (1.56897 -> 1.56897 * mean) 2024-09-18T11:32:01.487 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:32:01.487 INFO:teuthology.orchestra.run.smithi119.stdout:oload 1000 2024-09-18T11:32:01.487 INFO:teuthology.orchestra.run.smithi119.stdout:max_change 0.05 2024-09-18T11:32:01.487 INFO:teuthology.orchestra.run.smithi119.stdout:max_change_osds 4 2024-09-18T11:32:01.487 INFO:teuthology.orchestra.run.smithi119.stdout:average_utilization 0.0039 2024-09-18T11:32:01.487 INFO:teuthology.orchestra.run.smithi119.stdout:overload_utilization 0.0389 2024-09-18T11:32:01.488 INFO:teuthology.orchestra.run.smithi119.stderr:no change 2024-09-18T11:32:01.499 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-18T11:32:01.500 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2024-09-18T11:32:01.507 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:01.505+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:01.542 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:01.542 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:01.551 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:32:01.636 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:01.634+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:01.689 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:01.690 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:01.698 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:32:01.739 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:01.737+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:01.799 INFO:teuthology.orchestra.run.smithi119.stdout:moved 0 / 58 (0%) 2024-09-18T11:32:01.799 INFO:teuthology.orchestra.run.smithi119.stdout:avg 8.28571 2024-09-18T11:32:01.799 INFO:teuthology.orchestra.run.smithi119.stdout:stddev 2.31234 -> 2.31234 (expected baseline 2.66497) 2024-09-18T11:32:01.799 INFO:teuthology.orchestra.run.smithi119.stdout:min osd.2 with 6 -> 6 pgs (0.724138 -> 0.724138 * mean) 2024-09-18T11:32:01.799 INFO:teuthology.orchestra.run.smithi119.stdout:max osd.1 with 13 -> 13 pgs (1.56897 -> 1.56897 * mean) 2024-09-18T11:32:01.799 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:32:01.799 INFO:teuthology.orchestra.run.smithi119.stdout:oload 1000 2024-09-18T11:32:01.799 INFO:teuthology.orchestra.run.smithi119.stdout:max_change 0.05 2024-09-18T11:32:01.800 INFO:teuthology.orchestra.run.smithi119.stdout:max_change_osds 4 2024-09-18T11:32:01.800 INFO:teuthology.orchestra.run.smithi119.stdout:average_utilization 0.0039 2024-09-18T11:32:01.800 INFO:teuthology.orchestra.run.smithi119.stdout:overload_utilization 0.0389 2024-09-18T11:32:01.800 INFO:teuthology.orchestra.run.smithi119.stderr:no change 2024-09-18T11:32:01.809 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:01.807+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:01.811 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-18T11:32:01.811 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 3.0 2024-09-18T11:32:01.836 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:01.836 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:01.844 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:32:01.910 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:01.909+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:01.956 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:01.956 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:01.966 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:32:02.011 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:02.009+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:02.083 INFO:teuthology.orchestra.run.smithi119.stdout:moved 0 / 58 (0%) 2024-09-18T11:32:02.083 INFO:teuthology.orchestra.run.smithi119.stdout:avg 8.28571 2024-09-18T11:32:02.083 INFO:teuthology.orchestra.run.smithi119.stdout:stddev 2.31234 -> 2.31234 (expected baseline 2.66497) 2024-09-18T11:32:02.083 INFO:teuthology.orchestra.run.smithi119.stdout:min osd.2 with 6 -> 6 pgs (0.724138 -> 0.724138 * mean) 2024-09-18T11:32:02.083 INFO:teuthology.orchestra.run.smithi119.stdout:max osd.1 with 13 -> 13 pgs (1.56897 -> 1.56897 * mean) 2024-09-18T11:32:02.083 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:32:02.084 INFO:teuthology.orchestra.run.smithi119.stdout:oload 1000 2024-09-18T11:32:02.084 INFO:teuthology.orchestra.run.smithi119.stdout:max_change 3 2024-09-18T11:32:02.084 INFO:teuthology.orchestra.run.smithi119.stdout:max_change_osds 4 2024-09-18T11:32:02.084 INFO:teuthology.orchestra.run.smithi119.stdout:average_utilization 0.0039 2024-09-18T11:32:02.084 INFO:teuthology.orchestra.run.smithi119.stdout:overload_utilization 0.0389 2024-09-18T11:32:02.084 INFO:teuthology.orchestra.run.smithi119.stderr:no change 2024-09-18T11:32:02.096 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-18T11:32:02.096 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-18T11:32:02.106 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:02.106 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:02.111 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:02.110+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:02.115 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:32:02.212 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:02.210+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:02.230 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:02.228+0000 7f3157e90640 -1 osd.4 130 handle_osd_map injecting map crc failure 2024-09-18T11:32:02.239 INFO:tasks.rados.rados.0.smithi119.stdout:1701: done (2 left) 2024-09-18T11:32:02.239 INFO:tasks.rados.rados.0.smithi119.stdout:1702: done (1 left) 2024-09-18T11:32:02.239 INFO:tasks.rados.rados.0.smithi119.stdout:1703: done (0 left) 2024-09-18T11:32:02.239 INFO:tasks.rados.rados.0.smithi119.stdout:1704: write oid 31 current snap is 51 2024-09-18T11:32:02.239 INFO:tasks.rados.rados.0.smithi119.stdout:1704: seq_num 345 ranges {598415=796790,1922170=745784,3141403=415315} 2024-09-18T11:32:02.246 INFO:tasks.rados.rados.0.smithi119.stdout:1704: writing smithi11939506-31 from 598415 to 1395205 tid 1 2024-09-18T11:32:02.253 INFO:tasks.rados.rados.0.smithi119.stdout:1704: writing smithi11939506-31 from 1922170 to 2667954 tid 2 2024-09-18T11:32:02.259 INFO:tasks.rados.rados.0.smithi119.stdout:1704: writing smithi11939506-31 from 3141403 to 3556718 tid 3 2024-09-18T11:32:02.259 INFO:tasks.rados.rados.0.smithi119.stdout:1705: write oid 43 current snap is 51 2024-09-18T11:32:02.259 INFO:tasks.rados.rados.0.smithi119.stdout:1704: finishing write tid 1 to smithi11939506-31 2024-09-18T11:32:02.259 INFO:tasks.rados.rados.0.smithi119.stdout:1705: seq_num 346 ranges {615222=472039,1580652=489442,2768826=626587} 2024-09-18T11:32:02.262 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:02.263 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:02.264 INFO:tasks.rados.rados.0.smithi119.stdout:1705: writing smithi11939506-43 from 615222 to 1087261 tid 1 2024-09-18T11:32:02.270 INFO:tasks.rados.rados.0.smithi119.stdout:1705: writing smithi11939506-43 from 1580652 to 2070094 tid 2 2024-09-18T11:32:02.272 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:32:02.277 INFO:tasks.rados.rados.0.smithi119.stdout:1705: writing smithi11939506-43 from 2768826 to 3395413 tid 3 2024-09-18T11:32:02.277 INFO:tasks.rados.rados.0.smithi119.stdout:1706: set_chunk oid 1 target oid 2024-09-18T11:32:02.295 INFO:tasks.rados.rados.0.smithi119.stdout:1706: set_chunk oid 1 offset: 44544 length: 2356736 target oid 648484a68db83d57b38b973486edf8e5d6790961a5196f8a6b4bcf33da1348ab offset: 0 snap -1 2024-09-18T11:32:02.295 INFO:tasks.rados.rados.0.smithi119.stdout:1707: read oid 29 snap -1 2024-09-18T11:32:02.295 INFO:tasks.rados.rados.0.smithi119.stdout:1707: expect (ObjNum 210 snap 23 seq_num 210) 2024-09-18T11:32:02.295 INFO:tasks.rados.rados.0.smithi119.stdout:1708: snap_remove snap 27 2024-09-18T11:32:02.297 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:02.295+0000 7f6bf3052640 -1 osd.5 130 handle_osd_map injecting map crc failure 2024-09-18T11:32:02.313 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:02.311+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:02.374 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:02.374 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:02.379 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:32:02.377+0000 7fd482ca8640 -1 mgr.server reply reply (1) Operation not permitted no change 2024-09-18T11:32:02.379 INFO:teuthology.orchestra.run.smithi119.stdout:moved 2 / 58 (3.44828%) 2024-09-18T11:32:02.379 INFO:teuthology.orchestra.run.smithi119.stdout:avg 8.28571 2024-09-18T11:32:02.379 INFO:teuthology.orchestra.run.smithi119.stdout:stddev 2.31234 -> 2.54751 (expected baseline 2.66497) 2024-09-18T11:32:02.379 INFO:teuthology.orchestra.run.smithi119.stdout:min osd.2 with 6 -> 6 pgs (0.724138 -> 0.724138 * mean) 2024-09-18T11:32:02.379 INFO:teuthology.orchestra.run.smithi119.stdout:max osd.1 with 13 -> 14 pgs (1.56897 -> 1.68966 * mean) 2024-09-18T11:32:02.379 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:32:02.379 INFO:teuthology.orchestra.run.smithi119.stdout:oload 110 2024-09-18T11:32:02.379 INFO:teuthology.orchestra.run.smithi119.stdout:max_change 1 2024-09-18T11:32:02.380 INFO:teuthology.orchestra.run.smithi119.stdout:max_change_osds 4 2024-09-18T11:32:02.380 INFO:teuthology.orchestra.run.smithi119.stdout:average_utilization 0.0039 2024-09-18T11:32:02.380 INFO:teuthology.orchestra.run.smithi119.stdout:overload_utilization 0.0043 2024-09-18T11:32:02.380 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 weight 1.0000 -> 0.8790 2024-09-18T11:32:02.380 INFO:teuthology.orchestra.run.smithi119.stderr:no change 2024-09-18T11:32:02.383 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:32:02.392 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-18T11:32:02.392 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-09-18T11:32:02.434 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:02.433+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:02.515 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:02.514+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:02.531 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:02.532 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:02.541 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:32:02.616 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:02.614+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:02.620 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:32:02.681 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:02.682 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:02.691 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:32:02.706 INFO:teuthology.orchestra.run.smithi119.stdout:moved 0 / 58 (0%) 2024-09-18T11:32:02.706 INFO:teuthology.orchestra.run.smithi119.stdout:avg 8.28571 2024-09-18T11:32:02.707 INFO:teuthology.orchestra.run.smithi119.stdout:stddev 2.31234 -> 2.31234 (expected baseline 2.66497) 2024-09-18T11:32:02.707 INFO:teuthology.orchestra.run.smithi119.stdout:min osd.2 with 6 -> 6 pgs (0.724138 -> 0.724138 * mean) 2024-09-18T11:32:02.707 INFO:teuthology.orchestra.run.smithi119.stdout:max osd.1 with 13 -> 13 pgs (1.56897 -> 1.56897 * mean) 2024-09-18T11:32:02.707 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:32:02.707 INFO:teuthology.orchestra.run.smithi119.stdout:oload 1000 2024-09-18T11:32:02.707 INFO:teuthology.orchestra.run.smithi119.stdout:max_change 3 2024-09-18T11:32:02.707 INFO:teuthology.orchestra.run.smithi119.stdout:max_change_osds 4 2024-09-18T11:32:02.708 INFO:teuthology.orchestra.run.smithi119.stdout:average_utilization 0.0039 2024-09-18T11:32:02.708 INFO:teuthology.orchestra.run.smithi119.stdout:overload_utilization 0.0389 2024-09-18T11:32:02.708 INFO:teuthology.orchestra.run.smithi119.stderr:no change 2024-09-18T11:32:02.717 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:02.715+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:02.743 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:02.743 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:02.759 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:02.759 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:02.774 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:02.774 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:02.789 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:02.789 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:02.795 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:02.795 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:02.795 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:02.795 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:02.804 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:32:02.806 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:02.806 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:02.817 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:02.816+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:02.821 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:02.822 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:02.837 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:02.837 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:02.853 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:02.853 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:02.905 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:02.906 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:02.906 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:02.906 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:02.906 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:02.914 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:32:02.958 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:02.957+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:03.015 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:03.015 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:03.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:03.015 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:03.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.015 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:32:03.015 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:03.015 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 103.029568364, 2024-09-18T11:32:03.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:32:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:32:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:32:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:32:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:32:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:32:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:32:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:32:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:32:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:32:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:32:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:32:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:32:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:32:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:32:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:32:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:32:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 93.794737411, 2024-09-18T11:32:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:32:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:32:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:32:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:32:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:32:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:32:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:32:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:32:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:32:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:32:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:32:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:32:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:32:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:32:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:32:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:32:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 88.620831125999999, 2024-09-18T11:32:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:32:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:32:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:32:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:32:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:32:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:32:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:32:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:32:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:32:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:32:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:32:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:32:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:32:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:32:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:32:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:32:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:304 4.6 e69/67 4:67c1765a:::smithi11939506-30:17 v 69'35, mlcod=58'31)", 2024-09-18T11:32:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 71.666990032000001, 2024-09-18T11:32:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006377611, 2024-09-18T11:32:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345961+0000", 2024-09-18T11:32:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2050000000000008e-06 2024-09-18T11:32:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345969+0000", 2024-09-18T11:32:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0339999999999997e-06 2024-09-18T11:32:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345976+0000", 2024-09-18T11:32:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-18T11:32:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346026+0000", 2024-09-18T11:32:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9286999999999998e-05 2024-09-18T11:32:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346050+0000", 2024-09-18T11:32:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4397e-05 2024-09-18T11:32:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352293+0000", 2024-09-18T11:32:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062427170000000001 2024-09-18T11:32:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352330+0000", 2024-09-18T11:32:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7348999999999998e-05 2024-09-18T11:32:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:347 4.d e69/22 4:b44229a1:::smithi11939506-42:19 v 69'99, mlcod=65'93)", 2024-09-18T11:32:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 71.661256836999996, 2024-09-18T11:32:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056998760000000004, 2024-09-18T11:32:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351684+0000", 2024-09-18T11:32:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351693+0000", 2024-09-18T11:32:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-18T11:32:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351704+0000", 2024-09-18T11:32:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1077000000000001e-05 2024-09-18T11:32:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351709+0000", 2024-09-18T11:32:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5419999999999999e-06 2024-09-18T11:32:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351956+0000", 2024-09-18T11:32:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000247252 2024-09-18T11:32:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351975+0000", 2024-09-18T11:32:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9201000000000001e-05 2024-09-18T11:32:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357334+0000", 2024-09-18T11:32:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053588749999999999 2024-09-18T11:32:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357385+0000", 2024-09-18T11:32:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1406999999999998e-05 2024-09-18T11:32:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1956 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'37, mlcod=58'31)", 2024-09-18T11:32:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 71.660311923999998, 2024-09-18T11:32:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010958842, 2024-09-18T11:32:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352996+0000", 2024-09-18T11:32:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036622 2024-09-18T11:32:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353011+0000", 2024-09-18T11:32:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4849e-05 2024-09-18T11:32:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353017+0000", 2024-09-18T11:32:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:32:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353068+0000", 2024-09-18T11:32:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0695000000000002e-05 2024-09-18T11:32:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353092+0000", 2024-09-18T11:32:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4260000000000002e-05 2024-09-18T11:32:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363551+0000", 2024-09-18T11:32:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010459154999999999 2024-09-18T11:32:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363589+0000", 2024-09-18T11:32:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8161000000000003e-05 2024-09-18T11:32:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1957 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'38, mlcod=65'33)", 2024-09-18T11:32:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 71.653896056999997, 2024-09-18T11:32:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050499400000000002, 2024-09-18T11:32:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359214+0000", 2024-09-18T11:32:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000168047 2024-09-18T11:32:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359227+0000", 2024-09-18T11:32:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3124e-05 2024-09-18T11:32:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359235+0000", 2024-09-18T11:32:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0760000000000003e-06 2024-09-18T11:32:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359287+0000", 2024-09-18T11:32:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.151e-05 2024-09-18T11:32:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359310+0000", 2024-09-18T11:32:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-09-18T11:32:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364065+0000", 2024-09-18T11:32:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047549439999999997 2024-09-18T11:32:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364096+0000", 2024-09-18T11:32:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1423000000000001e-05 2024-09-18T11:32:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2300 4.4 e77/22 4:2c93196f:::smithi11939506-28:head v 77'94, mlcod=77'90)", 2024-09-18T11:32:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 62.024833839000003, 2024-09-18T11:32:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050425380000000001, 2024-09-18T11:32:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988108+0000", 2024-09-18T11:32:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988388+0000", 2024-09-18T11:32:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028087700000000001 2024-09-18T11:32:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988405+0000", 2024-09-18T11:32:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6565000000000001e-05 2024-09-18T11:32:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988412+0000", 2024-09-18T11:32:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9709999999999998e-06 2024-09-18T11:32:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988464+0000", 2024-09-18T11:32:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1949e-05 2024-09-18T11:32:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988490+0000", 2024-09-18T11:32:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5780999999999999e-05 2024-09-18T11:32:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993112+0000", 2024-09-18T11:32:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046228399999999996 2024-09-18T11:32:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993151+0000", 2024-09-18T11:32:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8587999999999997e-05 2024-09-18T11:32:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.13:63 3.0 e78/21 3:24ebb740:::smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 78'80, mlcod=73'78)", 2024-09-18T11:32:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:03.052 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 61.607844423000003, 2024-09-18T11:32:03.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052262230000000003, 2024-09-18T11:32:03.052 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.052 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.052 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405097+0000", 2024-09-18T11:32:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405103+0000", 2024-09-18T11:32:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8780000000000003e-06 2024-09-18T11:32:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405117+0000", 2024-09-18T11:32:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3441000000000001e-05 2024-09-18T11:32:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405121+0000", 2024-09-18T11:32:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7099999999999998e-06 2024-09-18T11:32:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405158+0000", 2024-09-18T11:32:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6477999999999999e-05 2024-09-18T11:32:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405173+0000", 2024-09-18T11:32:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4796e-05 2024-09-18T11:32:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410285+0000", 2024-09-18T11:32:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051118919999999998 2024-09-18T11:32:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410324+0000", 2024-09-18T11:32:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9616999999999997e-05 2024-09-18T11:32:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.13:18 3.0 e91/21 3:35e3c617:::2e64dd273da7619bd64b294f52905c1b9405a1ced2a8fd3c0b5bc481bc9732a6:head v 91'89, mlcod=85'87)", 2024-09-18T11:32:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.991069011999997, 2024-09-18T11:32:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048558539999999997, 2024-09-18T11:32:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021880+0000", 2024-09-18T11:32:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5860000000000001e-06 2024-09-18T11:32:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021890+0000", 2024-09-18T11:32:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8290000000000003e-06 2024-09-18T11:32:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021895+0000", 2024-09-18T11:32:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2480000000000004e-06 2024-09-18T11:32:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022166+0000", 2024-09-18T11:32:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027073499999999998 2024-09-18T11:32:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022184+0000", 2024-09-18T11:32:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8082000000000001e-05 2024-09-18T11:32:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026693+0000", 2024-09-18T11:32:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045093479999999998 2024-09-18T11:32:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026729+0000", 2024-09-18T11:32:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6282999999999998e-05 2024-09-18T11:32:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:573 4.4 e91/22 4:2c93196f:::smithi11939506-28:26 v 91'135, mlcod=90'131)", 2024-09-18T11:32:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.988602423000003, 2024-09-18T11:32:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050722739999999999, 2024-09-18T11:32:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024339+0000", 2024-09-18T11:32:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024349+0000", 2024-09-18T11:32:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6339999999999996e-06 2024-09-18T11:32:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024362+0000", 2024-09-18T11:32:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-18T11:32:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024368+0000", 2024-09-18T11:32:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8649999999999998e-06 2024-09-18T11:32:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024431+0000", 2024-09-18T11:32:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2780999999999994e-05 2024-09-18T11:32:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024448+0000", 2024-09-18T11:32:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7388999999999999e-05 2024-09-18T11:32:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029385+0000", 2024-09-18T11:32:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004936523 2024-09-18T11:32:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029412+0000", 2024-09-18T11:32:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7702999999999999e-05 2024-09-18T11:32:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:574 4.4 e91/22 4:2cb3e6a5:::smithi11939506-3:1e v 91'137, mlcod=90'131)", 2024-09-18T11:32:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.988164380000001, 2024-09-18T11:32:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070802850000000004, 2024-09-18T11:32:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024777+0000", 2024-09-18T11:32:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024785+0000", 2024-09-18T11:32:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8099999999999998e-06 2024-09-18T11:32:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024794+0000", 2024-09-18T11:32:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5839999999999995e-06 2024-09-18T11:32:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024798+0000", 2024-09-18T11:32:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.031e-06 2024-09-18T11:32:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024830+0000", 2024-09-18T11:32:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2268999999999999e-05 2024-09-18T11:32:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024847+0000", 2024-09-18T11:32:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7306999999999999e-05 2024-09-18T11:32:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031834+0000", 2024-09-18T11:32:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069868380000000004 2024-09-18T11:32:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031858+0000", 2024-09-18T11:32:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4082000000000001e-05 2024-09-18T11:32:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:541 4.d e91/22 4:b9bb642b:::smithi11939506-27:19 v 91'158, mlcod=91'154)", 2024-09-18T11:32:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.980783688000002, 2024-09-18T11:32:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052185, 2024-09-18T11:32:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032158+0000", 2024-09-18T11:32:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032165+0000", 2024-09-18T11:32:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-09-18T11:32:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032176+0000", 2024-09-18T11:32:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1460999999999999e-05 2024-09-18T11:32:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032182+0000", 2024-09-18T11:32:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0870000000000003e-06 2024-09-18T11:32:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032224+0000", 2024-09-18T11:32:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1372999999999998e-05 2024-09-18T11:32:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032246+0000", 2024-09-18T11:32:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2158e-05 2024-09-18T11:32:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037348+0000", 2024-09-18T11:32:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051025319999999999 2024-09-18T11:32:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037377+0000", 2024-09-18T11:32:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8937000000000001e-05 2024-09-18T11:32:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:582 4.6 e91/67 4:6f86ffcc:::smithi11939506-15:26 v 91'78, mlcod=90'72)", 2024-09-18T11:32:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.043567+0000", 2024-09-18T11:32:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.969375737999997, 2024-09-18T11:32:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004448446, 2024-09-18T11:32:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043567+0000", 2024-09-18T11:32:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043567+0000", 2024-09-18T11:32:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043566+0000", 2024-09-18T11:32:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043571+0000", 2024-09-18T11:32:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.181e-06 2024-09-18T11:32:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043579+0000", 2024-09-18T11:32:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9559999999999997e-06 2024-09-18T11:32:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043583+0000", 2024-09-18T11:32:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0770000000000001e-06 2024-09-18T11:32:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043610+0000", 2024-09-18T11:32:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6778e-05 2024-09-18T11:32:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.043633+0000", 2024-09-18T11:32:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2904e-05 2024-09-18T11:32:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.047991+0000", 2024-09-18T11:32:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043582910000000002 2024-09-18T11:32:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.048015+0000", 2024-09-18T11:32:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3813000000000001e-05 2024-09-18T11:32:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2754 4.4 e92/22 4:2cb3e6a5:::smithi11939506-3:head v 92'147, mlcod=92'143)", 2024-09-18T11:32:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:17.887302+0000", 2024-09-18T11:32:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.125640310000001, 2024-09-18T11:32:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044319160000000002, 2024-09-18T11:32:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887302+0000", 2024-09-18T11:32:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887302+0000", 2024-09-18T11:32:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887301+0000", 2024-09-18T11:32:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887313+0000", 2024-09-18T11:32:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2468e-05 2024-09-18T11:32:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887331+0000", 2024-09-18T11:32:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7362000000000001e-05 2024-09-18T11:32:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887338+0000", 2024-09-18T11:32:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7980000000000006e-06 2024-09-18T11:32:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887416+0000", 2024-09-18T11:32:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7577999999999995e-05 2024-09-18T11:32:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.887441+0000", 2024-09-18T11:32:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.531e-05 2024-09-18T11:32:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.891671+0000", 2024-09-18T11:32:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00422989 2024-09-18T11:32:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:17.891734+0000", 2024-09-18T11:32:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2885999999999994e-05 2024-09-18T11:32:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4178.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-18T11:32:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.072121623000001, 2024-09-18T11:32:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048960139999999997, 2024-09-18T11:32:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4178", 2024-09-18T11:32:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/490761693", 2024-09-18T11:32:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4 2024-09-18T11:32:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940819+0000", 2024-09-18T11:32:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940832+0000", 2024-09-18T11:32:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3155e-05 2024-09-18T11:32:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940848+0000", 2024-09-18T11:32:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6288999999999999e-05 2024-09-18T11:32:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940856+0000", 2024-09-18T11:32:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4919999999999996e-06 2024-09-18T11:32:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945613+0000", 2024-09-18T11:32:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047574999999999996 2024-09-18T11:32:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945717+0000", 2024-09-18T11:32:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103553 2024-09-18T11:32:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3482 4.6 e117/67 4:67c1765a:::smithi11939506-30:head v 117'120, mlcod=115'116)", 2024-09-18T11:32:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 16.328267400000001, 2024-09-18T11:32:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050037479999999997, 2024-09-18T11:32:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684674+0000", 2024-09-18T11:32:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684950+0000", 2024-09-18T11:32:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027621200000000002 2024-09-18T11:32:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684968+0000", 2024-09-18T11:32:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8e-05 2024-09-18T11:32:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684984+0000", 2024-09-18T11:32:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5775000000000001e-05 2024-09-18T11:32:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685033+0000", 2024-09-18T11:32:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8878000000000002e-05 2024-09-18T11:32:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685059+0000", 2024-09-18T11:32:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6801999999999999e-05 2024-09-18T11:32:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689640+0000", 2024-09-18T11:32:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045809029999999999 2024-09-18T11:32:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689679+0000", 2024-09-18T11:32:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8507000000000002e-05 2024-09-18T11:32:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:935 4.4 e128/22 4:2d9c3f7f:::smithi11939506-5:49 v 128'188, mlcod=126'184)", 2024-09-18T11:32:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:58.221275+0000", 2024-09-18T11:32:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.7916674940000004, 2024-09-18T11:32:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044057319999999999, 2024-09-18T11:32:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.221275+0000", 2024-09-18T11:32:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.221275+0000", 2024-09-18T11:32:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.221275+0000", 2024-09-18T11:32:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.221283+0000", 2024-09-18T11:32:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7479999999999999e-06 2024-09-18T11:32:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.221294+0000", 2024-09-18T11:32:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0986e-05 2024-09-18T11:32:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.221302+0000", 2024-09-18T11:32:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3069999999999997e-06 2024-09-18T11:32:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.221354+0000", 2024-09-18T11:32:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.198e-05 2024-09-18T11:32:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.221378+0000", 2024-09-18T11:32:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4354000000000001e-05 2024-09-18T11:32:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.225641+0000", 2024-09-18T11:32:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042632939999999999 2024-09-18T11:32:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.225681+0000", 2024-09-18T11:32:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9289999999999998e-05 2024-09-18T11:32:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:938 4.6 e128/67 4:67c1765a:::smithi11939506-30:48 v 128'131, mlcod=126'127)", 2024-09-18T11:32:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:58.226489+0000", 2024-09-18T11:32:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.7864535290000001, 2024-09-18T11:32:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045876129999999999, 2024-09-18T11:32:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.226489+0000", 2024-09-18T11:32:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.226489+0000", 2024-09-18T11:32:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.226488+0000", 2024-09-18T11:32:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.226494+0000", 2024-09-18T11:32:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4920000000000001e-06 2024-09-18T11:32:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.226503+0000", 2024-09-18T11:32:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.721e-06 2024-09-18T11:32:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.226508+0000", 2024-09-18T11:32:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0629999999999996e-06 2024-09-18T11:32:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.226549+0000", 2024-09-18T11:32:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1794e-05 2024-09-18T11:32:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.226571+0000", 2024-09-18T11:32:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1831000000000002e-05 2024-09-18T11:32:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.231042+0000", 2024-09-18T11:32:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044710729999999999 2024-09-18T11:32:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.231076+0000", 2024-09-18T11:32:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4122000000000003e-05 2024-09-18T11:32:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3809 4.d e128/22 4:b44229a1:::smithi11939506-42:head v 128'246, mlcod=128'242)", 2024-09-18T11:32:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:58.501935+0000", 2024-09-18T11:32:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5110077210000004, 2024-09-18T11:32:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048034009999999997, 2024-09-18T11:32:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.501935+0000", 2024-09-18T11:32:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.501935+0000", 2024-09-18T11:32:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.501933+0000", 2024-09-18T11:32:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.504308+0000", 2024-09-18T11:32:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023750609999999999 2024-09-18T11:32:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.504329+0000", 2024-09-18T11:32:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0933000000000001e-05 2024-09-18T11:32:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.504337+0000", 2024-09-18T11:32:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0430000000000008e-06 2024-09-18T11:32:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.504384+0000", 2024-09-18T11:32:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7216000000000003e-05 2024-09-18T11:32:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.504405+0000", 2024-09-18T11:32:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0341000000000001e-05 2024-09-18T11:32:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.506694+0000", 2024-09-18T11:32:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022896290000000001 2024-09-18T11:32:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:58.506738+0000", 2024-09-18T11:32:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3738e-05 2024-09-18T11:32:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.105 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:03.106 INFO:tasks.rados.rados.0.smithi119.stdout:1708: done (4 left) 2024-09-18T11:32:03.106 INFO:tasks.rados.rados.0.smithi119.stdout:1709: read oid 37 snap -1 2024-09-18T11:32:03.106 INFO:tasks.rados.rados.0.smithi119.stdout:1709: expect (ObjNum 342 snap 49 seq_num 342) 2024-09-18T11:32:03.106 INFO:tasks.rados.rados.0.smithi119.stdout:1710: write oid 41 current snap is 51 2024-09-18T11:32:03.107 INFO:tasks.rados.rados.0.smithi119.stdout:1704: finishing write tid 2 to smithi11939506-31 2024-09-18T11:32:03.107 INFO:tasks.rados.rados.0.smithi119.stdout:1704: finishing write tid 3 to smithi11939506-31 2024-09-18T11:32:03.107 INFO:tasks.rados.rados.0.smithi119.stdout:1704: finishing write tid 4 to smithi11939506-31 2024-09-18T11:32:03.107 INFO:tasks.rados.rados.0.smithi119.stdout:1704: finishing write tid 5 to smithi11939506-31 2024-09-18T11:32:03.107 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 300 (ObjNum 345 snap 51 seq_num 345) dirty exists 2024-09-18T11:32:03.107 INFO:tasks.rados.rados.0.smithi119.stdout:1704: left oid 31 (ObjNum 345 snap 51 seq_num 345) 2024-09-18T11:32:03.107 INFO:tasks.rados.rados.0.smithi119.stdout:1705: finishing write tid 1 to smithi11939506-43 2024-09-18T11:32:03.107 INFO:tasks.rados.rados.0.smithi119.stdout:1705: finishing write tid 2 to smithi11939506-43 2024-09-18T11:32:03.107 INFO:tasks.rados.rados.0.smithi119.stdout:1705: finishing write tid 3 to smithi11939506-43 2024-09-18T11:32:03.107 INFO:tasks.rados.rados.0.smithi119.stdout:1705: finishing write tid 4 to smithi11939506-43 2024-09-18T11:32:03.107 INFO:tasks.rados.rados.0.smithi119.stdout:1710: seq_num 347 ranges {632029=547288,1639134=749453,2796249=437859,3653524=178505} 2024-09-18T11:32:03.107 INFO:tasks.rados.rados.0.smithi119.stdout:1710: writing smithi11939506-41 from 632029 to 1179317 tid 1 2024-09-18T11:32:03.108 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:32:03.110 INFO:tasks.rados.rados.0.smithi119.stdout:1710: writing smithi11939506-41 from 1639134 to 2388587 tid 2 2024-09-18T11:32:03.110 INFO:tasks.rados.rados.0.smithi119.stdout:1710: writing smithi11939506-41 from 2796249 to 3234108 tid 3 2024-09-18T11:32:03.110 INFO:tasks.rados.rados.0.smithi119.stdout:1710: writing smithi11939506-41 from 3653524 to 3832029 tid 4 2024-09-18T11:32:03.110 INFO:tasks.rados.rados.0.smithi119.stdout:1704: done (5 left) 2024-09-18T11:32:03.110 INFO:tasks.rados.rados.0.smithi119.stdout:1711: set_chunk oid 39 target oid 2024-09-18T11:32:03.110 INFO:tasks.rados.rados.0.smithi119.stdout:1711: set_chunk oid 39 offset: 858624 length: 858624 target oid 7dc50ee24489a52812346b920f93a02a4d58b00ca87db4986330d3284dc336e5 offset: 0 snap -1 2024-09-18T11:32:03.110 INFO:tasks.rados.rados.0.smithi119.stdout:1712: read oid 10 snap 49 2024-09-18T11:32:03.110 INFO:tasks.rados.rados.0.smithi119.stdout:1712: expect (ObjNum 260 snap 31 seq_num 260) 2024-09-18T11:32:03.110 INFO:tasks.rados.rados.0.smithi119.stdout:1713: tier_evict oid 32 2024-09-18T11:32:03.110 INFO:tasks.rados.rados.0.smithi119.stdout:1713: tier_evict oid 32 snap -1 2024-09-18T11:32:03.110 INFO:tasks.rados.rados.0.smithi119.stdout:1705: finishing write tid 5 to smithi11939506-43 2024-09-18T11:32:03.110 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 43 v 254 (ObjNum 346 snap 51 seq_num 346) dirty exists 2024-09-18T11:32:03.111 INFO:tasks.rados.rados.0.smithi119.stdout:1705: left oid 43 (ObjNum 346 snap 51 seq_num 346) 2024-09-18T11:32:03.111 INFO:tasks.rados.rados.0.smithi119.stdout:1706: finishing set_chunk to oid 1 2024-09-18T11:32:03.111 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 208 (ObjNum 277 snap 37 seq_num 277) dirty exists 2024-09-18T11:32:03.111 INFO:tasks.rados.rados.0.smithi119.stdout:1707: expect (ObjNum 210 snap 23 seq_num 210) 2024-09-18T11:32:03.111 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:03.104+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:03.207 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:03.205+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:03.236 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T11:32:03.249 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:03.249 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:03.249 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:03.249 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:03.258 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:32:03.342 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:03.341+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:03.398 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:03.399 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:03.399 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:03.399 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:03.399 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:03.399 INFO:tasks.rados.rados.0.smithi119.stdout:1709: expect (ObjNum 342 snap 49 seq_num 342) 2024-09-18T11:32:03.407 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:32:03.408 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:03.407+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:03.510 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:03.509+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:03.527 INFO:tasks.rados.rados.0.smithi119.stdout:1710: finishing write tid 1 to smithi11939506-41 2024-09-18T11:32:03.528 INFO:tasks.rados.rados.0.smithi119.stdout:1710: finishing write tid 2 to smithi11939506-41 2024-09-18T11:32:03.528 INFO:tasks.rados.rados.0.smithi119.stdout:1710: finishing write tid 3 to smithi11939506-41 2024-09-18T11:32:03.528 INFO:tasks.rados.rados.0.smithi119.stdout:1713: got (22) Invalid argument 2024-09-18T11:32:03.528 INFO:tasks.rados.rados.0.smithi119.stdout:1710: finishing write tid 4 to smithi11939506-41 2024-09-18T11:32:03.528 INFO:tasks.rados.rados.0.smithi119.stdout:1705: done (7 left) 2024-09-18T11:32:03.528 INFO:tasks.rados.rados.0.smithi119.stdout:1706: done (6 left) 2024-09-18T11:32:03.528 INFO:tasks.rados.rados.0.smithi119.stdout:1707: done (5 left) 2024-09-18T11:32:03.528 INFO:tasks.rados.rados.0.smithi119.stdout:1709: done (4 left) 2024-09-18T11:32:03.528 INFO:tasks.rados.rados.0.smithi119.stdout:1713: done (3 left) 2024-09-18T11:32:03.528 INFO:tasks.rados.rados.0.smithi119.stdout:1714: snap_create 2024-09-18T11:32:03.529 INFO:tasks.rados.rados.0.smithi119.stdout:1710: finishing write tid 5 to smithi11939506-41 2024-09-18T11:32:03.529 INFO:tasks.rados.rados.0.smithi119.stdout:1710: finishing write tid 6 to smithi11939506-41 2024-09-18T11:32:03.529 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 41 v 429 (ObjNum 347 snap 51 seq_num 347) dirty exists 2024-09-18T11:32:03.529 INFO:tasks.rados.rados.0.smithi119.stdout:1710: left oid 41 (ObjNum 347 snap 51 seq_num 347) 2024-09-18T11:32:03.529 INFO:tasks.rados.rados.0.smithi119.stdout:1712: expect (ObjNum 260 snap 31 seq_num 260) 2024-09-18T11:32:03.548 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:32:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 112.37723577200001, 2024-09-18T11:32:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:32:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:03.549 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:32:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:03.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:32:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:32:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:32:03.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:32:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:32:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:32:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:32:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:32:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:32:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:32:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:32:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:32:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:32:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1001 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:32:03.554 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.382084494, 2024-09-18T11:32:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007461082, 2024-09-18T11:32:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1001 2024-09-18T11:32:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:32:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:32:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164570+0000", 2024-09-18T11:32:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:03.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164574+0000", 2024-09-18T11:32:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.822e-06 2024-09-18T11:32:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164582+0000", 2024-09-18T11:32:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3410000000000003e-06 2024-09-18T11:32:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164587+0000", 2024-09-18T11:32:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769999999999999e-06 2024-09-18T11:32:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164702+0000", 2024-09-18T11:32:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115705 2024-09-18T11:32:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164709+0000", 2024-09-18T11:32:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-09-18T11:32:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170296+0000", 2024-09-18T11:32:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055876390000000001 2024-09-18T11:32:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170418+0000", 2024-09-18T11:32:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121549 2024-09-18T11:32:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172032+0000", 2024-09-18T11:32:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00161407 2024-09-18T11:32:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.382057778, 2024-09-18T11:32:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:32:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:03.560 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:32:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:32:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:32:03.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:32:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:32:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:32:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:32:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:32:03.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:32:03.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:03.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:32:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:32:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:32:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:32:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:32:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:32:03.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:32:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:32:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:03.567 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.38203948899999, 2024-09-18T11:32:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:32:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:32:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:32:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:32:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:32:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:32:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:32:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:32:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:32:03.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:32:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:32:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:32:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:32:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:32:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:32:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:32:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:32:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:32:03.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:32:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.38197872000001, 2024-09-18T11:32:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:32:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:03.573 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:32:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:32:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:32:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:32:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:32:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:32:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:32:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:32:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:32:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:32:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:32:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:32:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:32:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:32:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:32:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:32:03.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:32:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:32:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:03.578 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.381956399, 2024-09-18T11:32:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:32:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:32:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:32:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:32:03.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:32:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:32:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:32:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:32:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:32:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:32:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:32:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:32:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:32:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:32:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:32:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:32:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:32:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:32:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:32:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:32:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 103.600831516, 2024-09-18T11:32:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:32:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:32:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:03.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:03.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:32:03.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:32:03.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:32:03.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:32:03.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:32:03.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:32:03.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:32:03.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:32:03.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:32:03.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:32:03.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:32:03.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:32:03.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:03.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:32:03.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:32:03.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:03.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:32:03.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:32:03.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:03.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:32:03.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:32:03.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:32:03.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:32:03.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:32:03.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:32:03.590 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.591 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.591 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.591 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1661 4.0 4:019b6e09:::smithi11939506-40:17 [read 0~3445402,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:32:03.591 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:03.591 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 86.196758813000002, 2024-09-18T11:32:03.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075190880000000002, 2024-09-18T11:32:03.591 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.591 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:03.591 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:03.591 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:03.591 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:03.591 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1661 2024-09-18T11:32:03.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.592 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:03.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:03.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:03.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:03.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349899+0000", 2024-09-18T11:32:03.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9749999999999999e-06 2024-09-18T11:32:03.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349906+0000", 2024-09-18T11:32:03.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9290000000000001e-06 2024-09-18T11:32:03.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349912+0000", 2024-09-18T11:32:03.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9900000000000002e-06 2024-09-18T11:32:03.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355645+0000", 2024-09-18T11:32:03.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057329549999999996 2024-09-18T11:32:03.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355776+0000", 2024-09-18T11:32:03.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130975 2024-09-18T11:32:03.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357416+0000", 2024-09-18T11:32:03.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001639649 2024-09-18T11:32:03.595 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.595 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.595 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.596 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1761 4.0 4:0aec5e1c:::smithi11939506-50:head [write 769787~704463,stat] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:32:03.596 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:32:03.596 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.244266398999997, 2024-09-18T11:32:03.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074082879999999999, 2024-09-18T11:32:03.596 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.596 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.596 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:03.596 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:03.596 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:03.596 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1761 2024-09-18T11:32:03.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.596 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:32:03.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:32:03.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302386+0000", 2024-09-18T11:32:03.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:03.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302889+0000", 2024-09-18T11:32:03.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050318499999999996 2024-09-18T11:32:03.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302930+0000", 2024-09-18T11:32:03.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0198999999999999e-05 2024-09-18T11:32:03.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302965+0000", 2024-09-18T11:32:03.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5292e-05 2024-09-18T11:32:03.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303046+0000", 2024-09-18T11:32:03.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1254999999999994e-05 2024-09-18T11:32:03.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:32:03.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303054+0000", 2024-09-18T11:32:03.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2800000000000003e-06 2024-09-18T11:32:03.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304658+0000", 2024-09-18T11:32:03.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016039520000000001 2024-09-18T11:32:03.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304724+0000", 2024-09-18T11:32:03.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5888000000000003e-05 2024-09-18T11:32:03.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:03.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304853+0000", 2024-09-18T11:32:03.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012912700000000001 2024-09-18T11:32:03.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:03.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309084+0000", 2024-09-18T11:32:03.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042311229999999998 2024-09-18T11:32:03.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:03.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309738+0000", 2024-09-18T11:32:03.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00065401399999999996 2024-09-18T11:32:03.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309770+0000", 2024-09-18T11:32:03.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1652000000000001e-05 2024-09-18T11:32:03.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309797+0000", 2024-09-18T11:32:03.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7052000000000001e-05 2024-09-18T11:32:03.602 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.602 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.602 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.603 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:306 4.f e69/22 4:f5d7df41:::smithi11939506-16:17 v 69'94, mlcod=65'88)", 2024-09-18T11:32:03.603 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:32:03.603 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.195539682000003, 2024-09-18T11:32:03.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0068024720000000004, 2024-09-18T11:32:03.603 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.603 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.603 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:32:03.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351116+0000", 2024-09-18T11:32:03.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351115+0000", 2024-09-18T11:32:03.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:03.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351121+0000", 2024-09-18T11:32:03.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5389999999999998e-06 2024-09-18T11:32:03.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351126+0000", 2024-09-18T11:32:03.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5969999999999999e-06 2024-09-18T11:32:03.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351147+0000", 2024-09-18T11:32:03.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1160000000000001e-05 2024-09-18T11:32:03.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351518+0000", 2024-09-18T11:32:03.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00037079300000000001 2024-09-18T11:32:03.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351536+0000", 2024-09-18T11:32:03.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8168999999999998e-05 2024-09-18T11:32:03.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357870+0000", 2024-09-18T11:32:03.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0063332650000000002 2024-09-18T11:32:03.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357918+0000", 2024-09-18T11:32:03.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8347999999999999e-05 2024-09-18T11:32:03.607 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.608 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.608 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.608 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2022 4.0 4:0aec5e1c:::smithi11939506-50:head [write 457190~552096,stat] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-18T11:32:03.608 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:32:03.608 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 70.550501545000003, 2024-09-18T11:32:03.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006803974, 2024-09-18T11:32:03.608 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.608 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.608 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:03.608 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:03.609 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:03.609 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2022 2024-09-18T11:32:03.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.609 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:32:03.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996154+0000", 2024-09-18T11:32:03.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996151+0000", 2024-09-18T11:32:03.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:03.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996461+0000", 2024-09-18T11:32:03.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000309587 2024-09-18T11:32:03.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996481+0000", 2024-09-18T11:32:03.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9949000000000002e-05 2024-09-18T11:32:03.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996489+0000", 2024-09-18T11:32:03.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9129999999999992e-06 2024-09-18T11:32:03.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996562+0000", 2024-09-18T11:32:03.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2807999999999998e-05 2024-09-18T11:32:03.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996687+0000", 2024-09-18T11:32:03.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012458200000000001 2024-09-18T11:32:03.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:03.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:52.996909+0000", 2024-09-18T11:32:03.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222124 2024-09-18T11:32:03.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:03.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.000741+0000", 2024-09-18T11:32:03.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0038320720000000002 2024-09-18T11:32:03.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:03.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002854+0000", 2024-09-18T11:32:03.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002113228 2024-09-18T11:32:03.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002911+0000", 2024-09-18T11:32:03.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7192999999999999e-05 2024-09-18T11:32:03.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:53.002958+0000", 2024-09-18T11:32:03.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6242000000000002e-05 2024-09-18T11:32:03.614 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.614 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.614 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.614 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2503 3.1 e85/67 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head v 85'117, mlcod=84'115)", 2024-09-18T11:32:03.614 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:03.614 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.569432829, 2024-09-18T11:32:03.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086527619999999996, 2024-09-18T11:32:03.614 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.614 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.614 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:03.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:03.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977220+0000", 2024-09-18T11:32:03.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:03.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980771+0000", 2024-09-18T11:32:03.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035505200000000002 2024-09-18T11:32:03.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980797+0000", 2024-09-18T11:32:03.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.618e-05 2024-09-18T11:32:03.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980807+0000", 2024-09-18T11:32:03.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0433e-05 2024-09-18T11:32:03.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980861+0000", 2024-09-18T11:32:03.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3415999999999998e-05 2024-09-18T11:32:03.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980885+0000", 2024-09-18T11:32:03.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4212000000000001e-05 2024-09-18T11:32:03.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985826+0000", 2024-09-18T11:32:03.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049412450000000004 2024-09-18T11:32:03.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985875+0000", 2024-09-18T11:32:03.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9116000000000002e-05 2024-09-18T11:32:03.618 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.618 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.618 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.618 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2648 4.0 4:059d8d95:::smithi11939506-32:head [read 0~3647119,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:03.619 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:32:03.619 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 49.215344459000001, 2024-09-18T11:32:03.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0076286949999999996, 2024-09-18T11:32:03.619 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.619 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:03.619 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:03.619 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:03.619 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:03.620 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2648 2024-09-18T11:32:03.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.620 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:32:03.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:32:03.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331309+0000", 2024-09-18T11:32:03.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:03.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331321+0000", 2024-09-18T11:32:03.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:32:03.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331340+0000", 2024-09-18T11:32:03.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8227e-05 2024-09-18T11:32:03.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331348+0000", 2024-09-18T11:32:03.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4640000000000006e-06 2024-09-18T11:32:03.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331406+0000", 2024-09-18T11:32:03.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7445000000000003e-05 2024-09-18T11:32:03.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331521+0000", 2024-09-18T11:32:03.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115945 2024-09-18T11:32:03.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338939+0000", 2024-09-18T11:32:03.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074179859999999997 2024-09-18T11:32:03.626 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.626 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.626 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.626 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2649 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:03.627 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:03.627 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 49.215289460000001, 2024-09-18T11:32:03.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566240000000007, 2024-09-18T11:32:03.627 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.627 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:03.627 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:03.627 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:03.627 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:03.627 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2649 2024-09-18T11:32:03.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.628 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:03.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:03.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331365+0000", 2024-09-18T11:32:03.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:03.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331369+0000", 2024-09-18T11:32:03.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.551e-06 2024-09-18T11:32:03.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331377+0000", 2024-09-18T11:32:03.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8769999999999993e-06 2024-09-18T11:32:03.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331381+0000", 2024-09-18T11:32:03.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.16e-06 2024-09-18T11:32:03.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338973+0000", 2024-09-18T11:32:03.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075918460000000002 2024-09-18T11:32:03.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339048+0000", 2024-09-18T11:32:03.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5813000000000001e-05 2024-09-18T11:32:03.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339322+0000", 2024-09-18T11:32:03.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027399899999999998 2024-09-18T11:32:03.632 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.632 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.632 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.632 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2650 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:03.632 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:03.632 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 49.215269241999998, 2024-09-18T11:32:03.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084392180000000001, 2024-09-18T11:32:03.632 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.632 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:03.632 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:03.632 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:03.633 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:03.633 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2650 2024-09-18T11:32:03.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.633 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:03.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:03.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:03.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:03.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331388+0000", 2024-09-18T11:32:03.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3429999999999998e-06 2024-09-18T11:32:03.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331394+0000", 2024-09-18T11:32:03.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.19e-06 2024-09-18T11:32:03.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331398+0000", 2024-09-18T11:32:03.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.495e-06 2024-09-18T11:32:03.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339362+0000", 2024-09-18T11:32:03.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079641799999999995 2024-09-18T11:32:03.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339460+0000", 2024-09-18T11:32:03.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8329e-05 2024-09-18T11:32:03.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339825+0000", 2024-09-18T11:32:03.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036510500000000001 2024-09-18T11:32:03.636 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.636 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.636 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.636 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:119 3.2 3:5595a68d:::07854d2fef297a06ba81685e660c332de36d5d18d546927d30daad6d7fda1541:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:03.636 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:32:03.637 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.503117484999997, 2024-09-18T11:32:03.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074145979999999997, 2024-09-18T11:32:03.637 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.637 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.637 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:32:03.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:32:03.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043535+0000", 2024-09-18T11:32:03.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:03.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043970+0000", 2024-09-18T11:32:03.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000435148 2024-09-18T11:32:03.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043993+0000", 2024-09-18T11:32:03.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3113999999999999e-05 2024-09-18T11:32:03.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044003+0000", 2024-09-18T11:32:03.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5310000000000007e-06 2024-09-18T11:32:03.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044062+0000", 2024-09-18T11:32:03.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9487000000000002e-05 2024-09-18T11:32:03.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:32:03.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044071+0000", 2024-09-18T11:32:03.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1610000000000003e-06 2024-09-18T11:32:03.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046423+0000", 2024-09-18T11:32:03.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023523680000000001 2024-09-18T11:32:03.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046502+0000", 2024-09-18T11:32:03.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9339999999999996e-05 2024-09-18T11:32:03.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:03.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046664+0000", 2024-09-18T11:32:03.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016185700000000001 2024-09-18T11:32:03.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:03.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050870+0000", 2024-09-18T11:32:03.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042060070000000003 2024-09-18T11:32:03.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:03.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050878+0000", 2024-09-18T11:32:03.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8960000000000003e-06 2024-09-18T11:32:03.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050918+0000", 2024-09-18T11:32:03.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0367000000000002e-05 2024-09-18T11:32:03.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050952+0000", 2024-09-18T11:32:03.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3989999999999998e-05 2024-09-18T11:32:03.642 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.642 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.643 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.643 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:124 3.2 3:4326e841:::a6567eb8c0be0365cdfccf2ece66d60995301bc908261a0c1daff039d2080a5f:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:03.643 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044017+0000", 2024-09-18T11:32:03.643 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.502637874999998, 2024-09-18T11:32:03.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070611140000000003, 2024-09-18T11:32:03.643 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.643 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.643 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044017+0000", 2024-09-18T11:32:03.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044017+0000", 2024-09-18T11:32:03.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044016+0000", 2024-09-18T11:32:03.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:03.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044102+0000", 2024-09-18T11:32:03.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5756999999999999e-05 2024-09-18T11:32:03.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044114+0000", 2024-09-18T11:32:03.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1505000000000001e-05 2024-09-18T11:32:03.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044120+0000", 2024-09-18T11:32:03.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8549999999999998e-06 2024-09-18T11:32:03.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044154+0000", 2024-09-18T11:32:03.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4773000000000001e-05 2024-09-18T11:32:03.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:03.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044159+0000", 2024-09-18T11:32:03.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5310000000000003e-06 2024-09-18T11:32:03.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046975+0000", 2024-09-18T11:32:03.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028162700000000001 2024-09-18T11:32:03.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.047116+0000", 2024-09-18T11:32:03.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014086400000000001 2024-09-18T11:32:03.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:03.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.047336+0000", 2024-09-18T11:32:03.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021975299999999999 2024-09-18T11:32:03.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:03.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050964+0000", 2024-09-18T11:32:03.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0036283000000000001 2024-09-18T11:32:03.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:03.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.051005+0000", 2024-09-18T11:32:03.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0763999999999998e-05 2024-09-18T11:32:03.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.051050+0000", 2024-09-18T11:32:03.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4993000000000001e-05 2024-09-18T11:32:03.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.051078+0000", 2024-09-18T11:32:03.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8694999999999999e-05 2024-09-18T11:32:03.649 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.649 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.649 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.649 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:127 3.2 3:4d3b277a:::3266c7aaa53a7d2c10c3c54d852d6cbbed49fb041bc0df41a3ac83b56e2b0615:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:03.649 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:03.649 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.502254303000001, 2024-09-18T11:32:03.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093148009999999993, 2024-09-18T11:32:03.650 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.650 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.650 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:03.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:03.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044400+0000", 2024-09-18T11:32:03.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:03.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044442+0000", 2024-09-18T11:32:03.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2247999999999999e-05 2024-09-18T11:32:03.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044453+0000", 2024-09-18T11:32:03.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0617e-05 2024-09-18T11:32:03.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044457+0000", 2024-09-18T11:32:03.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2520000000000001e-06 2024-09-18T11:32:03.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044495+0000", 2024-09-18T11:32:03.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7710000000000003e-05 2024-09-18T11:32:03.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:03.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044499+0000", 2024-09-18T11:32:03.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6269999999999999e-06 2024-09-18T11:32:03.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048239+0000", 2024-09-18T11:32:03.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037396719999999999 2024-09-18T11:32:03.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048351+0000", 2024-09-18T11:32:03.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112342 2024-09-18T11:32:03.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:03.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048545+0000", 2024-09-18T11:32:03.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000193881 2024-09-18T11:32:03.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:03.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.052689+0000", 2024-09-18T11:32:03.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041435780000000002 2024-09-18T11:32:03.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:03.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053654+0000", 2024-09-18T11:32:03.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096558199999999999 2024-09-18T11:32:03.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053683+0000", 2024-09-18T11:32:03.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8850999999999999e-05 2024-09-18T11:32:03.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053716+0000", 2024-09-18T11:32:03.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2468000000000002e-05 2024-09-18T11:32:03.656 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.656 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.656 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.656 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2963 4.0 4:019b6e09:::smithi11939506-40:head [sparse-read 0~1093084] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:03.656 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:32:03.656 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.519245505000001, 2024-09-18T11:32:03.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072292880000000004, 2024-09-18T11:32:03.657 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.657 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:03.657 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:03.657 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:03.657 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:03.657 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2963 2024-09-18T11:32:03.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.657 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:32:03.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:32:03.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027409+0000", 2024-09-18T11:32:03.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:03.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027412+0000", 2024-09-18T11:32:03.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.751e-06 2024-09-18T11:32:03.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027418+0000", 2024-09-18T11:32:03.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.404e-06 2024-09-18T11:32:03.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027423+0000", 2024-09-18T11:32:03.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7119999999999999e-06 2024-09-18T11:32:03.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034156+0000", 2024-09-18T11:32:03.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067331639999999998 2024-09-18T11:32:03.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034255+0000", 2024-09-18T11:32:03.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8701000000000002e-05 2024-09-18T11:32:03.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034639+0000", 2024-09-18T11:32:03.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000384006 2024-09-18T11:32:03.661 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.661 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.661 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.661 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3209 4.0 4:019b6e09:::smithi11939506-40:head [write 638732~557432] snapc 42=[42,41,3f,3e,3d,3b,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-18T11:32:03.661 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:03.661 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.448572263999999, 2024-09-18T11:32:03.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091774999999999999, 2024-09-18T11:32:03.661 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:03.661 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:03.661 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:03.661 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:03.662 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:03.662 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3209 2024-09-18T11:32:03.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.662 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:03.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:03.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:03.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:03.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:03.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:03.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:03.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098080+0000", 2024-09-18T11:32:03.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:03.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:03.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098676+0000", 2024-09-18T11:32:03.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059598099999999998 2024-09-18T11:32:03.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:03.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098706+0000", 2024-09-18T11:32:03.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0020000000000001e-05 2024-09-18T11:32:03.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:03.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098716+0000", 2024-09-18T11:32:03.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0225e-05 2024-09-18T11:32:03.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:03.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098782+0000", 2024-09-18T11:32:03.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5530000000000004e-05 2024-09-18T11:32:03.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:03.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098883+0000", 2024-09-18T11:32:03.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101609 2024-09-18T11:32:03.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:03.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.099098+0000", 2024-09-18T11:32:03.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000215052 2024-09-18T11:32:03.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:03.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.102511+0000", 2024-09-18T11:32:03.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034129429999999999 2024-09-18T11:32:03.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:03.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107165+0000", 2024-09-18T11:32:03.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046531480000000002 2024-09-18T11:32:03.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:03.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107217+0000", 2024-09-18T11:32:03.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2058000000000002e-05 2024-09-18T11:32:03.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:03.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:03.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:03.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107260+0000", 2024-09-18T11:32:03.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3865000000000001e-05 2024-09-18T11:32:03.667 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.667 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.667 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.667 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:03.667 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:03.667 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:03.669 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:32:03.669 INFO:tasks.rados.rados.0.smithi119.stdout:1711: finishing set_chunk to oid 39 2024-09-18T11:32:03.669 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 39 v 250 (ObjNum 326 snap 45 seq_num 326) dirty exists 2024-09-18T11:32:03.670 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:03.667+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:03.770 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:03.767+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:03.779 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:03.779 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:03.779 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:03.779 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:03.788 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:32:03.872 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:03.870+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:03.926 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:03.927 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:03.927 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:03.927 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:03.927 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:03.935 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:32:03.973 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:03.972+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:04.074 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:04.074 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:04.075 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:04.075 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:04.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.075 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:32:04.075 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:04.075 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 89.656004073000005, 2024-09-18T11:32:04.075 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:32:04.075 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.075 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.075 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.075 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.075 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.075 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:32:04.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.076 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.076 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.076 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:04.076 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.076 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.076 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:04.076 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.077 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:32:04.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:04.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.077 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:32:04.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:32:04.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.077 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:32:04.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:32:04.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:32:04.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:32:04.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:32:04.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:32:04.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:32:04.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:32:04.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.079 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:04.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:32:04.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:32:04.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.079 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:04.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:32:04.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:32:04.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:04.080 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:32:04.080 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:32:04.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.080 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:32:04.080 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:32:04.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.080 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:32:04.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:32:04.081 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.081 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.081 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.081 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1664 4.1 4:87078dfe:::smithi11939506-14:head [setxattr _header (57) in=7b,truncate 2235331] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:32:04.081 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:32:04.081 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 86.707853853000003, 2024-09-18T11:32:04.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010368235999999999, 2024-09-18T11:32:04.081 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.081 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.081 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.081 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.082 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.082 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1664 2024-09-18T11:32:04.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.082 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.082 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.082 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:32:04.082 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.082 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.082 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:32:04.082 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.083 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.083 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364236+0000", 2024-09-18T11:32:04.083 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:04.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.083 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.083 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364240+0000", 2024-09-18T11:32:04.083 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9829999999999998e-06 2024-09-18T11:32:04.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.083 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.083 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364247+0000", 2024-09-18T11:32:04.083 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2140000000000001e-06 2024-09-18T11:32:04.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.084 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.084 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364256+0000", 2024-09-18T11:32:04.084 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1509999999999993e-06 2024-09-18T11:32:04.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.084 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.084 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366529+0000", 2024-09-18T11:32:04.084 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022731409999999998 2024-09-18T11:32:04.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.084 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.084 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366621+0000", 2024-09-18T11:32:04.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2408e-05 2024-09-18T11:32:04.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:04.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366801+0000", 2024-09-18T11:32:04.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018002200000000001 2024-09-18T11:32:04.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:04.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372155+0000", 2024-09-18T11:32:04.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053531519999999999 2024-09-18T11:32:04.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:04.086 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374539+0000", 2024-09-18T11:32:04.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023842580000000002 2024-09-18T11:32:04.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.086 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.086 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374573+0000", 2024-09-18T11:32:04.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4205999999999997e-05 2024-09-18T11:32:04.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.086 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.086 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374605+0000", 2024-09-18T11:32:04.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2283999999999998e-05 2024-09-18T11:32:04.086 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.087 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.087 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.087 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1665 4.1 4:87078dfe:::smithi11939506-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:32:04.087 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:32:04.087 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 86.707829679, 2024-09-18T11:32:04.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010471897000000001, 2024-09-18T11:32:04.087 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.087 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:04.087 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.087 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.087 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.088 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1665 2024-09-18T11:32:04.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.088 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.088 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.088 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:32:04.088 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.088 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.088 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:32:04.088 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.089 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.089 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:32:04.089 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:04.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.089 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.089 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364263+0000", 2024-09-18T11:32:04.089 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6400000000000001e-06 2024-09-18T11:32:04.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.089 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.089 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364270+0000", 2024-09-18T11:32:04.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:32:04.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364274+0000", 2024-09-18T11:32:04.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.968e-06 2024-09-18T11:32:04.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367354+0000", 2024-09-18T11:32:04.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030803430000000001 2024-09-18T11:32:04.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:04.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367406+0000", 2024-09-18T11:32:04.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2330999999999998e-05 2024-09-18T11:32:04.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374625+0000", 2024-09-18T11:32:04.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072190689999999998 2024-09-18T11:32:04.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374673+0000", 2024-09-18T11:32:04.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7525999999999999e-05 2024-09-18T11:32:04.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.092 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.092 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374733+0000", 2024-09-18T11:32:04.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0359000000000003e-05 2024-09-18T11:32:04.092 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.092 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.092 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.092 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1947 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865577,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:32:04.092 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:32:04.092 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 72.732549969000004, 2024-09-18T11:32:04.093 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010043066999999999, 2024-09-18T11:32:04.093 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.093 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:04.093 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.093 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.093 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.093 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1947 2024-09-18T11:32:04.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.093 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.093 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.093 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:32:04.093 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.094 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.094 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:32:04.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.094 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.094 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339540+0000", 2024-09-18T11:32:04.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:04.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.094 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.094 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339545+0000", 2024-09-18T11:32:04.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6290000000000001e-06 2024-09-18T11:32:04.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.095 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.095 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339553+0000", 2024-09-18T11:32:04.095 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6299999999999998e-06 2024-09-18T11:32:04.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.095 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.095 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339557+0000", 2024-09-18T11:32:04.095 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3590000000000001e-06 2024-09-18T11:32:04.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.095 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.096 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345600+0000", 2024-09-18T11:32:04.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060434640000000001 2024-09-18T11:32:04.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.096 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345710+0000", 2024-09-18T11:32:04.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010924700000000001 2024-09-18T11:32:04.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.096 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349584+0000", 2024-09-18T11:32:04.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038743459999999999 2024-09-18T11:32:04.096 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.096 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.097 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.097 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1948 4.d 4:b5a463bf:::smithi11939506-20:head [sparse-read 0~2642181] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:32:04.097 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:32:04.097 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 72.732529432000007, 2024-09-18T11:32:04.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010701952000000001, 2024-09-18T11:32:04.097 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.097 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:04.097 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.097 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.097 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.097 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1948 2024-09-18T11:32:04.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.098 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.098 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:32:04.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.098 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:32:04.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.098 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:32:04.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:04.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339563+0000", 2024-09-18T11:32:04.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1950000000000002e-06 2024-09-18T11:32:04.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339569+0000", 2024-09-18T11:32:04.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5459999999999999e-06 2024-09-18T11:32:04.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339572+0000", 2024-09-18T11:32:04.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.337e-06 2024-09-18T11:32:04.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349693+0000", 2024-09-18T11:32:04.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010120987999999999 2024-09-18T11:32:04.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349816+0000", 2024-09-18T11:32:04.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012241199999999999 2024-09-18T11:32:04.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.101 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350263+0000", 2024-09-18T11:32:04.101 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00044787199999999998 2024-09-18T11:32:04.101 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.101 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.101 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.101 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:172 4.5 e78/22 4:a087a566:::smithi11939506-46:19 v 78'135, mlcod=77'132)", 2024-09-18T11:32:04.101 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:32:04.101 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 62.686267419000004, 2024-09-18T11:32:04.101 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010938658, 2024-09-18T11:32:04.101 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.101 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.101 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.102 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:32:04.102 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.102 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:32:04.102 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.102 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385823+0000", 2024-09-18T11:32:04.102 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:04.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.103 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.103 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385830+0000", 2024-09-18T11:32:04.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2790000000000001e-06 2024-09-18T11:32:04.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.103 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.103 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385843+0000", 2024-09-18T11:32:04.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2785e-05 2024-09-18T11:32:04.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.103 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385848+0000", 2024-09-18T11:32:04.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5570000000000003e-06 2024-09-18T11:32:04.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385914+0000", 2024-09-18T11:32:04.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5168999999999999e-05 2024-09-18T11:32:04.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385930+0000", 2024-09-18T11:32:04.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6884999999999998e-05 2024-09-18T11:32:04.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.105 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.105 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396721+0000", 2024-09-18T11:32:04.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010790137 2024-09-18T11:32:04.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.105 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.105 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396762+0000", 2024-09-18T11:32:04.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1635999999999999e-05 2024-09-18T11:32:04.105 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.105 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.105 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.106 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:32:04.106 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:04.106 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 62.685774666999997, 2024-09-18T11:32:04.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:32:04.106 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.106 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.106 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:04.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:04.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.107 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:04.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:04.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.107 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:32:04.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:32:04.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.108 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:32:04.108 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:32:04.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.108 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.108 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:32:04.108 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:32:04.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.108 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.108 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:32:04.108 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:32:04.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.109 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.109 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:32:04.109 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:32:04.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.109 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.109 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:32:04.109 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:32:04.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.109 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.109 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:32:04.109 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:32:04.109 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.110 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.110 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.110 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:04.110 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:04.110 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 55.099662060999997, 2024-09-18T11:32:04.110 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:32:04.110 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.110 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.110 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.110 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.110 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.111 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:32:04.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.111 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.111 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.111 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:04.111 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.111 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.111 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:04.111 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.112 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.112 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:32:04.112 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:04.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.112 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.112 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:32:04.112 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:32:04.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.112 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.112 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:32:04.113 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:32:04.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.113 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.113 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:32:04.113 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:32:04.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.113 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.113 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:32:04.113 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:32:04.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.113 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.114 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:32:04.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:32:04.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.114 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:04.114 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:32:04.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:32:04.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.114 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:04.114 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:32:04.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:32:04.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:04.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:32:04.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:32:04.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:32:04.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:32:04.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:32:04.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:32:04.116 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.116 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.116 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.116 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2647 4.e 4:78eefc4b:::smithi11939506-7:head [append 0~0] snapc 30=[30,2f,2b,2a,25,24,1c] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:04.116 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:32:04.116 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 50.062247380999999, 2024-09-18T11:32:04.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096752739999999993, 2024-09-18T11:32:04.116 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.116 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.116 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.116 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.116 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.116 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2647 2024-09-18T11:32:04.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.117 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:32:04.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:32:04.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009843+0000", 2024-09-18T11:32:04.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:04.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.118 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009846+0000", 2024-09-18T11:32:04.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3870000000000002e-06 2024-09-18T11:32:04.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.118 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009853+0000", 2024-09-18T11:32:04.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.238e-06 2024-09-18T11:32:04.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.119 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.119 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009860+0000", 2024-09-18T11:32:04.119 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1999999999999997e-06 2024-09-18T11:32:04.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.119 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.119 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.013798+0000", 2024-09-18T11:32:04.119 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003938441 2024-09-18T11:32:04.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.119 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.119 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.013889+0000", 2024-09-18T11:32:04.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0332000000000002e-05 2024-09-18T11:32:04.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 6", 2024-09-18T11:32:04.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.014066+0000", 2024-09-18T11:32:04.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017728700000000001 2024-09-18T11:32:04.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:04.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015813+0000", 2024-09-18T11:32:04.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001746804 2024-09-18T11:32:04.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:04.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019419+0000", 2024-09-18T11:32:04.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036058399999999999 2024-09-18T11:32:04.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019478+0000", 2024-09-18T11:32:04.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8860999999999998e-05 2024-09-18T11:32:04.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019519+0000", 2024-09-18T11:32:04.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1208000000000003e-05 2024-09-18T11:32:04.122 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.122 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.122 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.122 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:209 4.5 e91/22 4:af4ba75a:::smithi11939506-37:19 v 91'165, mlcod=91'161)", 2024-09-18T11:32:04.122 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:32:04.122 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 50.054170653, 2024-09-18T11:32:04.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0091546930000000002, 2024-09-18T11:32:04.122 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.122 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.122 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.122 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:32:04.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:32:04.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:32:04.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:04.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.124 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.124 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017925+0000", 2024-09-18T11:32:04.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0819999999999998e-06 2024-09-18T11:32:04.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.124 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.124 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017935+0000", 2024-09-18T11:32:04.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0597999999999999e-05 2024-09-18T11:32:04.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.124 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.124 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017941+0000", 2024-09-18T11:32:04.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2920000000000003e-06 2024-09-18T11:32:04.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.125 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.125 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017978+0000", 2024-09-18T11:32:04.125 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7667999999999999e-05 2024-09-18T11:32:04.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.125 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.125 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018018+0000", 2024-09-18T11:32:04.125 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9700999999999998e-05 2024-09-18T11:32:04.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.125 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.125 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.027039+0000", 2024-09-18T11:32:04.125 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090214079999999999 2024-09-18T11:32:04.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.126 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.126 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.027075+0000", 2024-09-18T11:32:04.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5605000000000002e-05 2024-09-18T11:32:04.126 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.126 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.126 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.126 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2955 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:04.126 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:32:04.126 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.044899632000003, 2024-09-18T11:32:04.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010869531, 2024-09-18T11:32:04.127 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.127 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:04.127 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.127 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.127 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.127 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2955 2024-09-18T11:32:04.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.127 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.127 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:32:04.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:32:04.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027190+0000", 2024-09-18T11:32:04.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:04.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027196+0000", 2024-09-18T11:32:04.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5590000000000002e-06 2024-09-18T11:32:04.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027208+0000", 2024-09-18T11:32:04.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1816e-05 2024-09-18T11:32:04.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027215+0000", 2024-09-18T11:32:04.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4509999999999997e-06 2024-09-18T11:32:04.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.130 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027725+0000", 2024-09-18T11:32:04.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000510053 2024-09-18T11:32:04.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.130 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:04.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027803+0000", 2024-09-18T11:32:04.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8750000000000003e-05 2024-09-18T11:32:04.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.130 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032412+0000", 2024-09-18T11:32:04.131 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046089030000000001 2024-09-18T11:32:04.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.131 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.131 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032499+0000", 2024-09-18T11:32:04.131 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6316000000000005e-05 2024-09-18T11:32:04.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.131 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.131 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038061+0000", 2024-09-18T11:32:04.131 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005562259 2024-09-18T11:32:04.131 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.131 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.131 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.132 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2956 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~2100908,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:04.132 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:04.132 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.044865811000001, 2024-09-18T11:32:04.132 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011863831, 2024-09-18T11:32:04.132 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.132 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:04.132 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.132 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.132 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.132 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2956 2024-09-18T11:32:04.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.132 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.133 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.133 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:04.133 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.133 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.133 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.133 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:04.133 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.133 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.133 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.133 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:04.133 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:04.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027228+0000", 2024-09-18T11:32:04.134 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:32:04.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027235+0000", 2024-09-18T11:32:04.134 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:32:04.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027239+0000", 2024-09-18T11:32:04.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.535e-06 2024-09-18T11:32:04.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.135 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.135 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027831+0000", 2024-09-18T11:32:04.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059244699999999996 2024-09-18T11:32:04.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.135 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:04.135 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027879+0000", 2024-09-18T11:32:04.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:32:04.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.136 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.136 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038098+0000", 2024-09-18T11:32:04.136 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010219137 2024-09-18T11:32:04.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.136 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.136 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038207+0000", 2024-09-18T11:32:04.136 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010951999999999999 2024-09-18T11:32:04.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.136 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.136 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039089+0000", 2024-09-18T11:32:04.136 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00088151699999999996 2024-09-18T11:32:04.137 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.137 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.137 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.137 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.137 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2957 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:04.137 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:04.137 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.044848027, 2024-09-18T11:32:04.137 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012041979, 2024-09-18T11:32:04.137 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.137 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:04.137 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.137 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.137 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.138 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2957 2024-09-18T11:32:04.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.138 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.138 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:04.138 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.138 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:04.138 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.139 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:04.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:04.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.139 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027245+0000", 2024-09-18T11:32:04.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:32:04.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.139 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027251+0000", 2024-09-18T11:32:04.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0090000000000004e-06 2024-09-18T11:32:04.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.140 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.140 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027255+0000", 2024-09-18T11:32:04.140 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:32:04.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.140 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.140 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027895+0000", 2024-09-18T11:32:04.140 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064076400000000005 2024-09-18T11:32:04.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.140 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:04.140 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027930+0000", 2024-09-18T11:32:04.141 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4334e-05 2024-09-18T11:32:04.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.141 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.141 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039111+0000", 2024-09-18T11:32:04.141 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181455999999999 2024-09-18T11:32:04.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.141 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.141 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039156+0000", 2024-09-18T11:32:04.141 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5256000000000002e-05 2024-09-18T11:32:04.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.141 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039285+0000", 2024-09-18T11:32:04.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012839200000000001 2024-09-18T11:32:04.142 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.142 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.142 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.142 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:978 4.b e132/67 4:d8d14c58:::smithi11939506-49:30 v 132'417, mlcod=128'414)", 2024-09-18T11:32:04.142 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:04.142 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.0468167450000001, 2024-09-18T11:32:04.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011942183, 2024-09-18T11:32:04.142 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.142 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.143 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.143 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.143 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:04.143 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.143 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.143 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:04.143 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.143 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.143 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:04.144 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:04.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.144 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.144 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025278+0000", 2024-09-18T11:32:04.144 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9060000000000004e-06 2024-09-18T11:32:04.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.144 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.144 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025288+0000", 2024-09-18T11:32:04.144 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.028e-05 2024-09-18T11:32:04.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.145 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.145 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025293+0000", 2024-09-18T11:32:04.145 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2700000000000004e-06 2024-09-18T11:32:04.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.145 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.145 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025346+0000", 2024-09-18T11:32:04.145 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2862999999999999e-05 2024-09-18T11:32:04.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.145 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.145 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025370+0000", 2024-09-18T11:32:04.146 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4005000000000001e-05 2024-09-18T11:32:04.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.146 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.146 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.146 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037159+0000", 2024-09-18T11:32:04.146 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011788343 2024-09-18T11:32:04.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.146 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.146 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.146 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037216+0000", 2024-09-18T11:32:04.146 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7695000000000002e-05 2024-09-18T11:32:04.146 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.147 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.147 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.147 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:980 4.b e132/67 4:d13decd8:::smithi11939506-2:3e v 132'419, mlcod=128'414)", 2024-09-18T11:32:04.147 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.026846+0000", 2024-09-18T11:32:04.147 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.0452447899999999, 2024-09-18T11:32:04.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010397910999999999, 2024-09-18T11:32:04.147 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.147 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.147 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.147 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.026846+0000", 2024-09-18T11:32:04.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.026846+0000", 2024-09-18T11:32:04.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.026845+0000", 2024-09-18T11:32:04.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:04.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.149 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.026857+0000", 2024-09-18T11:32:04.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1533e-05 2024-09-18T11:32:04.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.149 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.026869+0000", 2024-09-18T11:32:04.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1775e-05 2024-09-18T11:32:04.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.149 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.026875+0000", 2024-09-18T11:32:04.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5420000000000002e-06 2024-09-18T11:32:04.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.150 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.150 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.150 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.026931+0000", 2024-09-18T11:32:04.150 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5748000000000002e-05 2024-09-18T11:32:04.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.150 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.150 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.150 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.026951+0000", 2024-09-18T11:32:04.150 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0567000000000001e-05 2024-09-18T11:32:04.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.150 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.150 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.151 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037222+0000", 2024-09-18T11:32:04.151 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010270153000000001 2024-09-18T11:32:04.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.151 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.151 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.151 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037244+0000", 2024-09-18T11:32:04.151 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2524e-05 2024-09-18T11:32:04.151 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.151 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.151 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.151 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.151 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3930 4.b e132/67 4:d3ae8215:::smithi11939506-41:head v 132'421, mlcod=128'414)", 2024-09-18T11:32:04.151 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:04.152 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.0432684969999999, 2024-09-18T11:32:04.152 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015093554, 2024-09-18T11:32:04.152 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.152 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.152 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.152 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.152 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.152 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:04.152 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.152 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.152 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.152 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:04.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.153 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028821+0000", 2024-09-18T11:32:04.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:04.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.153 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029805+0000", 2024-09-18T11:32:04.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00098410899999999994 2024-09-18T11:32:04.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029817+0000", 2024-09-18T11:32:04.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2058e-05 2024-09-18T11:32:04.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029823+0000", 2024-09-18T11:32:04.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6450000000000002e-06 2024-09-18T11:32:04.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032330+0000", 2024-09-18T11:32:04.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025067710000000001 2024-09-18T11:32:04.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.155 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032349+0000", 2024-09-18T11:32:04.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9729e-05 2024-09-18T11:32:04.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.155 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043878+0000", 2024-09-18T11:32:04.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011528514 2024-09-18T11:32:04.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.155 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043916+0000", 2024-09-18T11:32:04.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8049000000000001e-05 2024-09-18T11:32:04.156 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.156 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.156 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.156 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:337 4.5 e132/22 4:aa575018:::smithi11939506-11:35 v 132'284, mlcod=129'279)", 2024-09-18T11:32:04.156 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.030014+0000", 2024-09-18T11:32:04.156 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.0420766939999999, 2024-09-18T11:32:04.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098918919999999994, 2024-09-18T11:32:04.156 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.156 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.156 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.157 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.157 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.030014+0000", 2024-09-18T11:32:04.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.157 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.157 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.030014+0000", 2024-09-18T11:32:04.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.157 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.157 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.030014+0000", 2024-09-18T11:32:04.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:04.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.158 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.158 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.030019+0000", 2024-09-18T11:32:04.158 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7250000000000002e-06 2024-09-18T11:32:04.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.030030+0000", 2024-09-18T11:32:04.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0720000000000001e-05 2024-09-18T11:32:04.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.030035+0000", 2024-09-18T11:32:04.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2190000000000003e-06 2024-09-18T11:32:04.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032640+0000", 2024-09-18T11:32:04.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026045869999999998 2024-09-18T11:32:04.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032658+0000", 2024-09-18T11:32:04.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7983e-05 2024-09-18T11:32:04.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.039871+0000", 2024-09-18T11:32:04.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072134520000000004 2024-09-18T11:32:04.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.163 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.039906+0000", 2024-09-18T11:32:04.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4811000000000003e-05 2024-09-18T11:32:04.163 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.163 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.163 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.163 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3931 4.b e132/67 4:d3ae8215:::smithi11939506-41:head v 132'422, mlcod=128'414)", 2024-09-18T11:32:04.163 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.034467+0000", 2024-09-18T11:32:04.163 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.037623771, 2024-09-18T11:32:04.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094796440000000006, 2024-09-18T11:32:04.163 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.163 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.163 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.164 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.034467+0000", 2024-09-18T11:32:04.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.164 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.034467+0000", 2024-09-18T11:32:04.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.164 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.034466+0000", 2024-09-18T11:32:04.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:04.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.035250+0000", 2024-09-18T11:32:04.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078353699999999995 2024-09-18T11:32:04.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.035404+0000", 2024-09-18T11:32:04.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015400200000000001 2024-09-18T11:32:04.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.035410+0000", 2024-09-18T11:32:04.166 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0039999999999996e-06 2024-09-18T11:32:04.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037281+0000", 2024-09-18T11:32:04.166 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018710199999999999 2024-09-18T11:32:04.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037300+0000", 2024-09-18T11:32:04.166 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9215000000000002e-05 2024-09-18T11:32:04.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043922+0000", 2024-09-18T11:32:04.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066223799999999998 2024-09-18T11:32:04.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043947+0000", 2024-09-18T11:32:04.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4440999999999999e-05 2024-09-18T11:32:04.167 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.167 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.167 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.167 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3933 4.b e132/67 4:d3ae8215:::smithi11939506-41:head v 132'428, mlcod=132'417)", 2024-09-18T11:32:04.168 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.044472+0000", 2024-09-18T11:32:04.168 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.0276186650000001, 2024-09-18T11:32:04.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096047660000000007, 2024-09-18T11:32:04.168 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.168 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.168 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.044472+0000", 2024-09-18T11:32:04.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.044472+0000", 2024-09-18T11:32:04.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.044472+0000", 2024-09-18T11:32:04.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:04.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.044650+0000", 2024-09-18T11:32:04.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017815999999999999 2024-09-18T11:32:04.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.044660+0000", 2024-09-18T11:32:04.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0265000000000001e-05 2024-09-18T11:32:04.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.044666+0000", 2024-09-18T11:32:04.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4289999999999998e-06 2024-09-18T11:32:04.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.045521+0000", 2024-09-18T11:32:04.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00085499999999999997 2024-09-18T11:32:04.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.045539+0000", 2024-09-18T11:32:04.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7815000000000001e-05 2024-09-18T11:32:04.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.054043+0000", 2024-09-18T11:32:04.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085036420000000005 2024-09-18T11:32:04.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.054077+0000", 2024-09-18T11:32:04.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4208999999999997e-05 2024-09-18T11:32:04.172 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.172 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.172 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.172 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3938 4.1 4:86d80ae6:::smithi11939506-39:head [assert-version v250,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:04.172 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.053159+0000", 2024-09-18T11:32:04.172 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.0189317330000001, 2024-09-18T11:32:04.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097869289999999998, 2024-09-18T11:32:04.172 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.172 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.173 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.173 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.173 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.173 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3938 2024-09-18T11:32:04.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.173 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.173 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.053159+0000", 2024-09-18T11:32:04.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.173 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.053159+0000", 2024-09-18T11:32:04.174 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.174 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.174 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.053158+0000", 2024-09-18T11:32:04.174 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:04.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.174 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.174 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.053164+0000", 2024-09-18T11:32:04.174 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4920000000000001e-06 2024-09-18T11:32:04.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.175 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.175 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.053174+0000", 2024-09-18T11:32:04.175 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0292000000000001e-05 2024-09-18T11:32:04.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.175 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.175 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.053178+0000", 2024-09-18T11:32:04.175 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2169999999999996e-06 2024-09-18T11:32:04.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.175 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.175 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.053219+0000", 2024-09-18T11:32:04.175 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.074e-05 2024-09-18T11:32:04.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:04.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.053307+0000", 2024-09-18T11:32:04.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7869999999999997e-05 2024-09-18T11:32:04.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.054152+0000", 2024-09-18T11:32:04.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00084497900000000004 2024-09-18T11:32:04.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.054219+0000", 2024-09-18T11:32:04.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6971000000000006e-05 2024-09-18T11:32:04.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:04.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.058088+0000", 2024-09-18T11:32:04.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038691680000000001 2024-09-18T11:32:04.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:04.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.062835+0000", 2024-09-18T11:32:04.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004746859 2024-09-18T11:32:04.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:04.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.062855+0000", 2024-09-18T11:32:04.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0109e-05 2024-09-18T11:32:04.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.062900+0000", 2024-09-18T11:32:04.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5522999999999998e-05 2024-09-18T11:32:04.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.062946+0000", 2024-09-18T11:32:04.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5643999999999999e-05 2024-09-18T11:32:04.178 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.178 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.178 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.178 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.179 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.179 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:04.179 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:04.071+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:04.180 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:32:04.221 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:04.219+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:04.239 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is unset 2024-09-18T11:32:04.239 INFO:tasks.rados.rados.0.smithi119.stdout:1710: done (3 left) 2024-09-18T11:32:04.239 INFO:tasks.rados.rados.0.smithi119.stdout:1711: done (2 left) 2024-09-18T11:32:04.239 INFO:tasks.rados.rados.0.smithi119.stdout:1712: done (1 left) 2024-09-18T11:32:04.239 INFO:tasks.rados.rados.0.smithi119.stdout:1714: done (0 left) 2024-09-18T11:32:04.240 INFO:tasks.rados.rados.0.smithi119.stdout:1715: snap_remove snap 48 2024-09-18T11:32:04.282 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:04.279+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:04.319 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:04.319 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:04.320 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:04.320 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:04.327 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:32:04.383 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:04.382+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:04.465 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:04.465 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:04.465 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:04.465 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:04.466 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:04.473 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:32:04.517 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:04.515+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:04.573 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:04.573 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:04.573 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:04.573 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:04.573 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.574 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:32:04.574 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:32:04.574 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 134.54429317, 2024-09-18T11:32:04.574 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073601550000000002, 2024-09-18T11:32:04.574 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.574 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.574 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.574 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:32:04.574 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:32:04.574 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1 2024-09-18T11:32:04.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.574 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.575 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.575 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.575 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:32:04.575 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.575 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.575 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.575 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:32:04.575 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.575 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.575 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.575 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026524+0000", 2024-09-18T11:32:04.576 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999905 2024-09-18T11:32:04.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.576 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.576 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026541+0000", 2024-09-18T11:32:04.576 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7070000000000001e-05 2024-09-18T11:32:04.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.576 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.576 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026554+0000", 2024-09-18T11:32:04.576 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.288e-05 2024-09-18T11:32:04.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.577 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.577 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026564+0000", 2024-09-18T11:32:04.577 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0197000000000001e-05 2024-09-18T11:32:04.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.577 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.577 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026826+0000", 2024-09-18T11:32:04.577 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026212400000000003 2024-09-18T11:32:04.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.578 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:32:04.578 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026839+0000", 2024-09-18T11:32:04.578 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2831e-05 2024-09-18T11:32:04.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.578 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.578 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.578 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028605+0000", 2024-09-18T11:32:04.578 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001765794 2024-09-18T11:32:04.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.578 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.578 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.578 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028709+0000", 2024-09-18T11:32:04.578 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104348 2024-09-18T11:32:04.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.579 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.579 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:32:04.579 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028824+0000", 2024-09-18T11:32:04.579 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011527399999999999 2024-09-18T11:32:04.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.579 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.579 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:04.579 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030319+0000", 2024-09-18T11:32:04.579 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001494439 2024-09-18T11:32:04.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.579 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.579 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:04.579 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033805+0000", 2024-09-18T11:32:04.580 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034857820000000002 2024-09-18T11:32:04.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.580 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.580 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.580 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033856+0000", 2024-09-18T11:32:04.580 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1687999999999997e-05 2024-09-18T11:32:04.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.580 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.580 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.580 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033894+0000", 2024-09-18T11:32:04.580 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7481000000000002e-05 2024-09-18T11:32:04.580 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.581 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.581 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.581 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.581 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1118 3.2 e38/21 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head v 38'76, mlcod=38'74)", 2024-09-18T11:32:04.581 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:32:04.581 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 104.622883018, 2024-09-18T11:32:04.581 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006048688, 2024-09-18T11:32:04.581 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.581 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.581 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.581 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.584 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.584 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:32:04.584 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.584 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.585 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.585 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:32:04.585 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.585 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.585 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.585 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947941+0000", 2024-09-18T11:32:04.585 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:04.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.585 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.586 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.586 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949868+0000", 2024-09-18T11:32:04.586 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019267659999999999 2024-09-18T11:32:04.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.586 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.586 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.586 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949893+0000", 2024-09-18T11:32:04.586 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5239999999999999e-05 2024-09-18T11:32:04.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.586 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.586 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.586 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949901+0000", 2024-09-18T11:32:04.587 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8420000000000005e-06 2024-09-18T11:32:04.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.587 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.587 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949954+0000", 2024-09-18T11:32:04.587 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3464999999999997e-05 2024-09-18T11:32:04.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.587 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.587 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949974+0000", 2024-09-18T11:32:04.588 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9731000000000001e-05 2024-09-18T11:32:04.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.588 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.588 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.588 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953948+0000", 2024-09-18T11:32:04.588 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039735489999999998 2024-09-18T11:32:04.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.588 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.588 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.588 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953992+0000", 2024-09-18T11:32:04.588 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4700999999999998e-05 2024-09-18T11:32:04.589 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.589 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.589 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.589 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1662 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'73, mlcod=51'69)", 2024-09-18T11:32:04.589 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:32:04.589 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 87.209277983000007, 2024-09-18T11:32:04.589 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059408539999999998, 2024-09-18T11:32:04.589 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.589 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.589 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.590 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.590 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:32:04.590 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.590 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.590 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:32:04.590 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.590 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.590 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:32:04.590 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:04.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.591 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.591 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.591 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361914+0000", 2024-09-18T11:32:04.591 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036569 2024-09-18T11:32:04.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.591 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.591 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.591 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361926+0000", 2024-09-18T11:32:04.591 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1960999999999999e-05 2024-09-18T11:32:04.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.591 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.591 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.591 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361937+0000", 2024-09-18T11:32:04.592 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0876999999999999e-05 2024-09-18T11:32:04.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.592 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.592 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.592 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361986+0000", 2024-09-18T11:32:04.592 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8652000000000001e-05 2024-09-18T11:32:04.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.592 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.592 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.592 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.362012+0000", 2024-09-18T11:32:04.592 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5854999999999999e-05 2024-09-18T11:32:04.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.593 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.593 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.593 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367460+0000", 2024-09-18T11:32:04.593 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054482100000000002 2024-09-18T11:32:04.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.593 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.593 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.593 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367490+0000", 2024-09-18T11:32:04.593 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9909000000000001e-05 2024-09-18T11:32:04.593 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.594 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.594 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.594 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.594 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:269 4.7 e58/22 4:e73ba68f:::smithi11939506-1:13 v 58'72, mlcod=57'69)", 2024-09-18T11:32:04.594 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:32:04.594 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.305446086000003, 2024-09-18T11:32:04.594 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065512569999999996, 2024-09-18T11:32:04.594 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.594 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.594 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.594 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.594 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.595 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:32:04.595 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.595 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.595 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.595 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:32:04.595 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.595 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.595 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.595 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265380+0000", 2024-09-18T11:32:04.595 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:04.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265388+0000", 2024-09-18T11:32:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.174e-06 2024-09-18T11:32:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265398+0000", 2024-09-18T11:32:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2599999999999994e-06 2024-09-18T11:32:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265402+0000", 2024-09-18T11:32:04.597 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0289999999999999e-06 2024-09-18T11:32:04.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.597 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.597 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.597 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265434+0000", 2024-09-18T11:32:04.597 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2511000000000001e-05 2024-09-18T11:32:04.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.597 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.597 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.597 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265447+0000", 2024-09-18T11:32:04.598 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2947e-05 2024-09-18T11:32:04.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.598 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.598 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.598 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271892+0000", 2024-09-18T11:32:04.598 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064445930000000002 2024-09-18T11:32:04.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.598 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.598 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.598 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271932+0000", 2024-09-18T11:32:04.598 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0303999999999999e-05 2024-09-18T11:32:04.598 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.599 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.599 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.599 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.599 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1788 4.8 4:1898b15d:::smithi11939506-25:head [write 453822~797061] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:32:04.599 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:04.599 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 81.655013503000006, 2024-09-18T11:32:04.599 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094305159999999999, 2024-09-18T11:32:04.599 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.599 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.599 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.599 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.599 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.599 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1788 2024-09-18T11:32:04.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.600 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.600 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.600 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.600 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:04.600 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.600 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.600 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.600 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:04.600 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.600 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.600 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.601 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915811+0000", 2024-09-18T11:32:04.601 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:04.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.601 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.601 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.601 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916608+0000", 2024-09-18T11:32:04.601 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079635000000000003 2024-09-18T11:32:04.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.601 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.601 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.601 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916624+0000", 2024-09-18T11:32:04.601 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6735000000000002e-05 2024-09-18T11:32:04.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.602 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.602 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.602 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916630+0000", 2024-09-18T11:32:04.602 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5709999999999997e-06 2024-09-18T11:32:04.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.602 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.602 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.602 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916674+0000", 2024-09-18T11:32:04.602 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4540999999999999e-05 2024-09-18T11:32:04.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.602 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.602 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.602 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916751+0000", 2024-09-18T11:32:04.603 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6695999999999999e-05 2024-09-18T11:32:04.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.603 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:04.603 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916872+0000", 2024-09-18T11:32:04.603 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120648 2024-09-18T11:32:04.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.603 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:04.603 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.919076+0000", 2024-09-18T11:32:04.603 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022046829999999998 2024-09-18T11:32:04.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.603 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:04.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925151+0000", 2024-09-18T11:32:04.604 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060740239999999999 2024-09-18T11:32:04.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.604 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.604 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925198+0000", 2024-09-18T11:32:04.604 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7896999999999999e-05 2024-09-18T11:32:04.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.604 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.604 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925244+0000", 2024-09-18T11:32:04.604 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5398999999999997e-05 2024-09-18T11:32:04.604 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.605 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.605 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.605 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1934 4.2 e65/22 4:425afe9c:::smithi11939506-29:head v 65'44, mlcod=60'40)", 2024-09-18T11:32:04.605 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:32:04.605 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 76.270834733000001, 2024-09-18T11:32:04.605 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064585119999999996, 2024-09-18T11:32:04.605 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.605 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.605 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.605 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:32:04.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.606 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:32:04.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.606 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299990+0000", 2024-09-18T11:32:04.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:04.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.607 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300789+0000", 2024-09-18T11:32:04.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079851499999999997 2024-09-18T11:32:04.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.607 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300806+0000", 2024-09-18T11:32:04.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7096999999999999e-05 2024-09-18T11:32:04.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.607 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300813+0000", 2024-09-18T11:32:04.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5079999999999999e-06 2024-09-18T11:32:04.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.608 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.608 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300870+0000", 2024-09-18T11:32:04.608 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7178e-05 2024-09-18T11:32:04.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.608 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.608 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300897+0000", 2024-09-18T11:32:04.608 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6284000000000001e-05 2024-09-18T11:32:04.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.608 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.609 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306410+0000", 2024-09-18T11:32:04.609 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055135210000000004 2024-09-18T11:32:04.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.609 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.609 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306451+0000", 2024-09-18T11:32:04.609 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0379000000000001e-05 2024-09-18T11:32:04.609 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.609 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.609 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.609 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2452 4.3 4:ca2b2309:::smithi11939506-44:head [read 0~3462242,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:32:04.609 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:32:04.610 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 58.534059376000002, 2024-09-18T11:32:04.610 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060906450000000004, 2024-09-18T11:32:04.610 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.610 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:04.610 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.610 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.610 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.610 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2452 2024-09-18T11:32:04.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.610 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.610 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.610 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:32:04.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:32:04.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036765+0000", 2024-09-18T11:32:04.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:04.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.612 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036777+0000", 2024-09-18T11:32:04.612 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1422e-05 2024-09-18T11:32:04.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.612 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036793+0000", 2024-09-18T11:32:04.612 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6575999999999999e-05 2024-09-18T11:32:04.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.612 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036800+0000", 2024-09-18T11:32:04.613 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9940000000000003e-06 2024-09-18T11:32:04.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.613 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.613 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:32:04.613 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5847999999999999e-05 2024-09-18T11:32:04.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.613 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:04.613 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036853+0000", 2024-09-18T11:32:04.613 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0119999999999999e-06 2024-09-18T11:32:04.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.614 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038527+0000", 2024-09-18T11:32:04.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016732730000000001 2024-09-18T11:32:04.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.614 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038580+0000", 2024-09-18T11:32:04.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3443999999999999e-05 2024-09-18T11:32:04.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.614 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042858+0000", 2024-09-18T11:32:04.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042781049999999999 2024-09-18T11:32:04.615 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.615 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.615 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.615 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2453 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:32:04.615 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:32:04.615 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 58.534017144000003, 2024-09-18T11:32:04.615 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063798429999999996, 2024-09-18T11:32:04.615 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.615 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:04.615 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.615 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.615 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.615 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2453 2024-09-18T11:32:04.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.616 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.616 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:32:04.616 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.616 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:32:04.616 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.616 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036809+0000", 2024-09-18T11:32:04.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:04.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.617 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036812+0000", 2024-09-18T11:32:04.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8339999999999999e-06 2024-09-18T11:32:04.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.617 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036819+0000", 2024-09-18T11:32:04.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9890000000000003e-06 2024-09-18T11:32:04.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036823+0000", 2024-09-18T11:32:04.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8099999999999999e-06 2024-09-18T11:32:04.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036912+0000", 2024-09-18T11:32:04.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9083000000000001e-05 2024-09-18T11:32:04.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:04.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036932+0000", 2024-09-18T11:32:04.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0222000000000001e-05 2024-09-18T11:32:04.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.619 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042896+0000", 2024-09-18T11:32:04.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059637240000000001 2024-09-18T11:32:04.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.619 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042952+0000", 2024-09-18T11:32:04.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5877000000000001e-05 2024-09-18T11:32:04.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.620 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.620 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043190+0000", 2024-09-18T11:32:04.620 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00023774500000000001 2024-09-18T11:32:04.620 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.620 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.620 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.620 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2454 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:32:04.620 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:32:04.620 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 58.533981187999998, 2024-09-18T11:32:04.620 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066404469999999998, 2024-09-18T11:32:04.620 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.620 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:04.621 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.621 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.621 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.621 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2454 2024-09-18T11:32:04.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.621 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:32:04.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:32:04.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036845+0000", 2024-09-18T11:32:04.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:04.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036856+0000", 2024-09-18T11:32:04.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1338000000000001e-05 2024-09-18T11:32:04.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.623 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036863+0000", 2024-09-18T11:32:04.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4749999999999998e-06 2024-09-18T11:32:04.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.623 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036867+0000", 2024-09-18T11:32:04.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3839999999999999e-06 2024-09-18T11:32:04.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.624 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036946+0000", 2024-09-18T11:32:04.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9054000000000006e-05 2024-09-18T11:32:04.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.624 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:04.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036949+0000", 2024-09-18T11:32:04.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9390000000000002e-06 2024-09-18T11:32:04.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.625 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.625 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043213+0000", 2024-09-18T11:32:04.625 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062640090000000001 2024-09-18T11:32:04.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.625 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.625 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043261+0000", 2024-09-18T11:32:04.625 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7750999999999997e-05 2024-09-18T11:32:04.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.625 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.625 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043486+0000", 2024-09-18T11:32:04.625 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000224985 2024-09-18T11:32:04.626 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.626 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.626 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.626 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2553 4.3 4:ca2b2309:::smithi11939506-44:head [write 564768~630013,stat] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-18T11:32:04.626 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:32:04.626 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.373703554999999, 2024-09-18T11:32:04.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065686879999999996, 2024-09-18T11:32:04.627 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.627 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.627 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.627 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.627 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.627 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2553 2024-09-18T11:32:04.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.627 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:32:04.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.628 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:32:04.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.628 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197120+0000", 2024-09-18T11:32:04.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:04.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199000+0000", 2024-09-18T11:32:04.629 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018792080000000001 2024-09-18T11:32:04.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199044+0000", 2024-09-18T11:32:04.629 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4048999999999998e-05 2024-09-18T11:32:04.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199071+0000", 2024-09-18T11:32:04.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7288e-05 2024-09-18T11:32:04.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199119+0000", 2024-09-18T11:32:04.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8000999999999997e-05 2024-09-18T11:32:04.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199208+0000", 2024-09-18T11:32:04.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9209000000000006e-05 2024-09-18T11:32:04.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:04.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199385+0000", 2024-09-18T11:32:04.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000177095 2024-09-18T11:32:04.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:04.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.202282+0000", 2024-09-18T11:32:04.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028970469999999998 2024-09-18T11:32:04.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:04.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203578+0000", 2024-09-18T11:32:04.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012960490000000001 2024-09-18T11:32:04.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.632 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203638+0000", 2024-09-18T11:32:04.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9599000000000003e-05 2024-09-18T11:32:04.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203692+0000", 2024-09-18T11:32:04.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3908e-05 2024-09-18T11:32:04.633 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.633 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.633 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.633 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2752 4.3 4:ca2b2309:::smithi11939506-44:25 [read 0~3210104,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:32:04.633 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:32:04.633 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 46.683934395000001, 2024-09-18T11:32:04.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068715460000000001, 2024-09-18T11:32:04.634 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.634 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:04.634 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.634 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.634 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.634 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2752 2024-09-18T11:32:04.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.634 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:32:04.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.635 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:32:04.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.635 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:32:04.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:04.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.635 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886895+0000", 2024-09-18T11:32:04.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.993e-06 2024-09-18T11:32:04.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.636 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886901+0000", 2024-09-18T11:32:04.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1859999999999997e-06 2024-09-18T11:32:04.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.636 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886904+0000", 2024-09-18T11:32:04.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1920000000000001e-06 2024-09-18T11:32:04.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.892622+0000", 2024-09-18T11:32:04.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057175660000000003 2024-09-18T11:32:04.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.892693+0000", 2024-09-18T11:32:04.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1265000000000003e-05 2024-09-18T11:32:04.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.893764+0000", 2024-09-18T11:32:04.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001070753 2024-09-18T11:32:04.638 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.638 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.638 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.638 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2793 4.1 e92/22 4:87078dfe:::smithi11939506-14:head v 92'195, mlcod=92'191)", 2024-09-18T11:32:04.638 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:32:04.638 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.346034361000001, 2024-09-18T11:32:04.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062761989999999997, 2024-09-18T11:32:04.639 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.639 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.639 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.639 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:32:04.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.640 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.640 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:32:04.640 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.640 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.640 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:32:04.640 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:04.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.641 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.641 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224994+0000", 2024-09-18T11:32:04.641 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020197600000000001 2024-09-18T11:32:04.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.641 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.641 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.641 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225007+0000", 2024-09-18T11:32:04.641 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3791999999999999e-05 2024-09-18T11:32:04.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.641 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.641 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.641 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225013+0000", 2024-09-18T11:32:04.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3630000000000002e-06 2024-09-18T11:32:04.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225062+0000", 2024-09-18T11:32:04.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8925000000000003e-05 2024-09-18T11:32:04.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225086+0000", 2024-09-18T11:32:04.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4737999999999999e-05 2024-09-18T11:32:04.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231022+0000", 2024-09-18T11:32:04.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059360690000000004 2024-09-18T11:32:04.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231069+0000", 2024-09-18T11:32:04.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6252999999999999e-05 2024-09-18T11:32:04.643 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.643 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.643 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.644 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2876 4.3 4:cc46532a:::smithi11939506-33:2f [read 0~3680733,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:04.644 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:32:04.644 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 42.063965461000002, 2024-09-18T11:32:04.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068391210000000001, 2024-09-18T11:32:04.644 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.644 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:04.644 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.644 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.644 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.644 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2876 2024-09-18T11:32:04.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.645 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.645 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:32:04.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.645 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:32:04.646 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.646 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506860+0000", 2024-09-18T11:32:04.646 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:04.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.646 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506869+0000", 2024-09-18T11:32:04.646 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5539999999999998e-06 2024-09-18T11:32:04.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506880+0000", 2024-09-18T11:32:04.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1552999999999999e-05 2024-09-18T11:32:04.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506887+0000", 2024-09-18T11:32:04.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7329999999999997e-06 2024-09-18T11:32:04.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506937+0000", 2024-09-18T11:32:04.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0317e-05 2024-09-18T11:32:04.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:04.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506945+0000", 2024-09-18T11:32:04.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1550000000000007e-06 2024-09-18T11:32:04.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.509021+0000", 2024-09-18T11:32:04.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020759870000000001 2024-09-18T11:32:04.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.509123+0000", 2024-09-18T11:32:04.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010132900000000001 2024-09-18T11:32:04.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.649 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513700+0000", 2024-09-18T11:32:04.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0045777320000000002 2024-09-18T11:32:04.649 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.649 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.649 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.649 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2877 4.3 4:cc46532a:::smithi11939506-33:2f [read 0~3680733,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:04.650 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:04.650 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 42.063931046999997, 2024-09-18T11:32:04.650 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085715170000000007, 2024-09-18T11:32:04.650 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.650 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:04.650 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.650 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.650 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.650 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2877 2024-09-18T11:32:04.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.650 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.651 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.651 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:04.651 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.651 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.651 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:04.651 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:04.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:04.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506898+0000", 2024-09-18T11:32:04.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4760000000000001e-06 2024-09-18T11:32:04.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506903+0000", 2024-09-18T11:32:04.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:32:04.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506906+0000", 2024-09-18T11:32:04.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1769999999999998e-06 2024-09-18T11:32:04.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507007+0000", 2024-09-18T11:32:04.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101479 2024-09-18T11:32:04.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:04.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507015+0000", 2024-09-18T11:32:04.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8490000000000006e-06 2024-09-18T11:32:04.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513757+0000", 2024-09-18T11:32:04.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0067417800000000002 2024-09-18T11:32:04.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513867+0000", 2024-09-18T11:32:04.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000109987 2024-09-18T11:32:04.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515467+0000", 2024-09-18T11:32:04.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00160037 2024-09-18T11:32:04.655 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.655 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.656 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.656 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2878 4.3 4:cc46532a:::smithi11939506-33:2f [sparse-read 0~1770976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:04.656 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:04.656 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 42.063900592000003, 2024-09-18T11:32:04.656 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090602640000000002, 2024-09-18T11:32:04.656 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.656 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:04.656 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.657 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.657 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.657 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2878 2024-09-18T11:32:04.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.657 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.657 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:04.657 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:04.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:04.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:04.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506929+0000", 2024-09-18T11:32:04.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.163e-06 2024-09-18T11:32:04.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506936+0000", 2024-09-18T11:32:04.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5519999999999999e-06 2024-09-18T11:32:04.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506941+0000", 2024-09-18T11:32:04.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:32:04.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507036+0000", 2024-09-18T11:32:04.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5070999999999998e-05 2024-09-18T11:32:04.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:04.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507042+0000", 2024-09-18T11:32:04.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3959999999999997e-06 2024-09-18T11:32:04.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515504+0000", 2024-09-18T11:32:04.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084620300000000006 2024-09-18T11:32:04.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515625+0000", 2024-09-18T11:32:04.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012150999999999999 2024-09-18T11:32:04.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515986+0000", 2024-09-18T11:32:04.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036140100000000002 2024-09-18T11:32:04.662 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.662 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.662 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.662 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:686 4.7 e98/22 4:e5707c18:::smithi11939506-10:30 v 98'160, mlcod=97'155)", 2024-09-18T11:32:04.662 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:32:04.662 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.552058901999999, 2024-09-18T11:32:04.662 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074641239999999999, 2024-09-18T11:32:04.662 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.662 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.662 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.663 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.663 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.663 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:32:04.663 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.663 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.663 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.663 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:32:04.663 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.663 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018767+0000", 2024-09-18T11:32:04.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:04.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018773+0000", 2024-09-18T11:32:04.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1460000000000004e-06 2024-09-18T11:32:04.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018781+0000", 2024-09-18T11:32:04.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9780000000000005e-06 2024-09-18T11:32:04.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.665 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018785+0000", 2024-09-18T11:32:04.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0029999999999996e-06 2024-09-18T11:32:04.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.665 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018826+0000", 2024-09-18T11:32:04.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1616999999999998e-05 2024-09-18T11:32:04.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018843+0000", 2024-09-18T11:32:04.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6753999999999999e-05 2024-09-18T11:32:04.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026202+0000", 2024-09-18T11:32:04.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073592620000000001 2024-09-18T11:32:04.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026232+0000", 2024-09-18T11:32:04.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9779000000000001e-05 2024-09-18T11:32:04.667 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.667 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.667 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.667 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.667 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.4.0:621 4.1 e98/22 4:854aa293:::smithi11939506-26:35 v 98'215, mlcod=96'210)", 2024-09-18T11:32:04.667 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:32:04.667 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.551175463, 2024-09-18T11:32:04.667 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007014752, 2024-09-18T11:32:04.667 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.667 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.667 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.667 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.668 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:32:04.668 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.668 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:32:04.669 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.669 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.669 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.669 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:32:04.669 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:04.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.670 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019659+0000", 2024-09-18T11:32:04.670 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8229999999999995e-06 2024-09-18T11:32:04.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.670 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019671+0000", 2024-09-18T11:32:04.670 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2636000000000001e-05 2024-09-18T11:32:04.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.671 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019677+0000", 2024-09-18T11:32:04.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5470000000000004e-06 2024-09-18T11:32:04.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.671 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019710+0000", 2024-09-18T11:32:04.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3191999999999997e-05 2024-09-18T11:32:04.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019722+0000", 2024-09-18T11:32:04.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2357e-05 2024-09-18T11:32:04.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026639+0000", 2024-09-18T11:32:04.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0069167630000000003 2024-09-18T11:32:04.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026666+0000", 2024-09-18T11:32:04.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7087999999999999e-05 2024-09-18T11:32:04.673 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.673 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.673 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.673 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.673 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3234 4.7 e109/22 4:e73ba68f:::smithi11939506-1:head v 109'206, mlcod=107'202)", 2024-09-18T11:32:04.673 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:37.128363+0000", 2024-09-18T11:32:04.673 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.442464079000001, 2024-09-18T11:32:04.673 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059367100000000004, 2024-09-18T11:32:04.674 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.674 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.674 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.674 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.674 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.128363+0000", 2024-09-18T11:32:04.674 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.674 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.674 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.128363+0000", 2024-09-18T11:32:04.674 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.128362+0000", 2024-09-18T11:32:04.675 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:04.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131291+0000", 2024-09-18T11:32:04.675 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029282840000000002 2024-09-18T11:32:04.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131305+0000", 2024-09-18T11:32:04.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4222999999999999e-05 2024-09-18T11:32:04.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.676 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131313+0000", 2024-09-18T11:32:04.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.208e-06 2024-09-18T11:32:04.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.676 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131354+0000", 2024-09-18T11:32:04.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1158000000000001e-05 2024-09-18T11:32:04.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.677 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.677 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131378+0000", 2024-09-18T11:32:04.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3828999999999998e-05 2024-09-18T11:32:04.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.677 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.677 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.134258+0000", 2024-09-18T11:32:04.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028798920000000002 2024-09-18T11:32:04.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.677 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.134299+0000", 2024-09-18T11:32:04.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1355e-05 2024-09-18T11:32:04.678 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.678 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.678 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.678 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3490 4.3 4:cc46532a:::smithi11939506-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:04.678 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.709960+0000", 2024-09-18T11:32:04.678 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.860867065000001, 2024-09-18T11:32:04.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058965190000000002, 2024-09-18T11:32:04.679 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.679 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:04.679 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.679 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.679 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.679 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3490 2024-09-18T11:32:04.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.679 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.679 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.679 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709960+0000", 2024-09-18T11:32:04.679 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.680 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.680 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709960+0000", 2024-09-18T11:32:04.680 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.681 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709959+0000", 2024-09-18T11:32:04.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:04.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.682 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709964+0000", 2024-09-18T11:32:04.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0900000000000004e-06 2024-09-18T11:32:04.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.683 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.683 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709971+0000", 2024-09-18T11:32:04.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5950000000000004e-06 2024-09-18T11:32:04.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.683 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.684 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709975+0000", 2024-09-18T11:32:04.684 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0690000000000003e-06 2024-09-18T11:32:04.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.684 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.684 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.685 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.712315+0000", 2024-09-18T11:32:04.685 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002339556 2024-09-18T11:32:04.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.685 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.685 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:04.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.712392+0000", 2024-09-18T11:32:04.686 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7868999999999996e-05 2024-09-18T11:32:04.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.715759+0000", 2024-09-18T11:32:04.687 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0033669989999999999 2024-09-18T11:32:04.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.687 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.687 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.715802+0000", 2024-09-18T11:32:04.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2636000000000003e-05 2024-09-18T11:32:04.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.688 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.715856+0000", 2024-09-18T11:32:04.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4153000000000001e-05 2024-09-18T11:32:04.689 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.689 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.689 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.689 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3775 4.3 4:c70c3162:::smithi11939506-35:head [write 796731~461096,stat] snapc 51=[51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b,30] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-18T11:32:04.689 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:58.217457+0000", 2024-09-18T11:32:04.689 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 6.3533699410000004, 2024-09-18T11:32:04.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064394090000000001, 2024-09-18T11:32:04.689 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:04.689 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:04.690 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:04.690 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:04.690 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:04.690 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3775 2024-09-18T11:32:04.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.690 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:04.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.691 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:04.691 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217457+0000", 2024-09-18T11:32:04.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:04.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217457+0000", 2024-09-18T11:32:04.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:04.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.693 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:04.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217455+0000", 2024-09-18T11:32:04.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:04.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.693 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:04.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217965+0000", 2024-09-18T11:32:04.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00050991999999999997 2024-09-18T11:32:04.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.694 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:04.694 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217977+0000", 2024-09-18T11:32:04.694 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1701e-05 2024-09-18T11:32:04.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:04.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217981+0000", 2024-09-18T11:32:04.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5600000000000004e-06 2024-09-18T11:32:04.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:04.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.218033+0000", 2024-09-18T11:32:04.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1728999999999999e-05 2024-09-18T11:32:04.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:04.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.218147+0000", 2024-09-18T11:32:04.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000114066 2024-09-18T11:32:04.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.696 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.696 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:04.696 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.218364+0000", 2024-09-18T11:32:04.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021718700000000001 2024-09-18T11:32:04.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.696 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.696 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:04.696 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.223702+0000", 2024-09-18T11:32:04.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005338036 2024-09-18T11:32:04.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.696 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.696 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:04.696 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.223812+0000", 2024-09-18T11:32:04.697 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000110011 2024-09-18T11:32:04.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.697 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:04.697 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.223868+0000", 2024-09-18T11:32:04.697 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5989999999999998e-05 2024-09-18T11:32:04.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:04.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:04.697 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:04.697 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.223896+0000", 2024-09-18T11:32:04.697 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7820000000000001e-05 2024-09-18T11:32:04.697 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.697 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.698 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.698 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:04.698 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:04.698 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:04.699 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:32:04.700 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:04.697+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:04.800 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:04.798+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:04.839 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:04.839 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:04.839 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:04.839 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:04.848 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:32:04.865 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:32:04.932 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:04.930+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:04.987 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:04.987 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:04.987 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:04.988 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:04.988 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:04.991 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:04.991 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:04.996 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:32:05.007 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:05.007 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:05.021 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:05.022 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:05.036 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:05.036 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:05.037 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:05.035+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:05.052 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:05.052 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:05.067 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:05.067 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:05.082 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:05.082 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:05.098 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:05.098 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:05.102 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:05.100+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:05.135 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:05.135 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:05.144 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:32:05.203 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:05.202+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:05.239 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:05.237+0000 7f6bf3052640 -1 osd.5 132 handle_osd_map injecting map crc failure 2024-09-18T11:32:05.239 INFO:tasks.rados.rados.0.smithi119.stdout:1715: done (0 left) 2024-09-18T11:32:05.240 INFO:tasks.rados.rados.0.smithi119.stdout:1716: set_chunk oid 44 target oid 2024-09-18T11:32:05.242 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:05.241+0000 7f697aead640 -1 osd.3 132 handle_osd_map injecting map crc failure 2024-09-18T11:32:05.253 INFO:tasks.rados.rados.0.smithi119.stdout:1716: set_chunk oid 44 offset: 1730560 length: 375808 target oid bcda10e691cfa11a400af8144b596b83091051358e306444dd4ee9e9eafb5116 offset: 0 snap 41 2024-09-18T11:32:05.253 INFO:tasks.rados.rados.0.smithi119.stdout:1717: read oid 40 snap -1 2024-09-18T11:32:05.253 INFO:tasks.rados.rados.0.smithi119.stdout:1717: expect (ObjNum 287 snap 41 seq_num 287) 2024-09-18T11:32:05.253 INFO:tasks.rados.rados.0.smithi119.stdout:1718: tier_flush oid 28 2024-09-18T11:32:05.254 INFO:tasks.rados.rados.0.smithi119.stdout:1718: tier_flush oid 28 snap -1 2024-09-18T11:32:05.254 INFO:tasks.rados.rados.0.smithi119.stdout:1719: snap_create 2024-09-18T11:32:05.254 INFO:tasks.rados.rados.0.smithi119.stdout:1718: got (0) Success 2024-09-18T11:32:05.254 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 28 v 193 (ObjNum 336 snap 48 seq_num 336) dirty exists 2024-09-18T11:32:05.255 INFO:tasks.rados.rados.0.smithi119.stdout:1716: finishing set_chunk to oid 44 2024-09-18T11:32:05.258 INFO:tasks.rados.rados.0.smithi119.stdout:1717: expect (ObjNum 287 snap 41 seq_num 287) 2024-09-18T11:32:05.286 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:05.286 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:05.294 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:32:05.303 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:05.302+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:05.391 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:05.391 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:05.400 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:32:05.446 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:05.444+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:05.498 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:05.498 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:05.507 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:32:05.507 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:05.504+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:05.609 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:05.609 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:05.609 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:05.608+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:05.614 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:32:05.710 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:05.708+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:05.711 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:05.711 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:05.719 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:32:05.813 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:05.811+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:05.817 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:05.817 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:05.825 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:32:05.914 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:05.913+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:05.922 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:05.923 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:05.931 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:32:06.017 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:06.015+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:06.029 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:06.029 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:06.038 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:32:06.118 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:06.117+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:06.136 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:06.136 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:06.144 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:32:06.229 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:06.228+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:06.253 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T11:32:06.257 INFO:tasks.rados.rados.0.smithi119.stdout:1716: done (3 left) 2024-09-18T11:32:06.257 INFO:tasks.rados.rados.0.smithi119.stdout:1717: done (2 left) 2024-09-18T11:32:06.257 INFO:tasks.rados.rados.0.smithi119.stdout:1718: done (1 left) 2024-09-18T11:32:06.257 INFO:tasks.rados.rados.0.smithi119.stdout:1719: done (0 left) 2024-09-18T11:32:06.257 INFO:tasks.rados.rados.0.smithi119.stdout:1720: snap_create 2024-09-18T11:32:06.288 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:06.288 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:06.298 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:32:06.339 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:06.338+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:06.420 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:06.418+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:06.438 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:06.438 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:06.447 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:32:06.493 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:06.492+0000 7f697aead640 -1 osd.3 134 handle_osd_map injecting map crc failure 2024-09-18T11:32:06.520 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:06.519+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:06.545 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:06.545 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:06.554 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:32:06.622 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:06.620+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:06.692 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:06.693 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:06.701 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:32:06.745 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:06.744+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:06.798 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:06.799 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:06.807 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:32:06.824 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:06.822+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:06.924 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:06.923+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:06.946 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:06.946 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:06.955 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:32:07.025 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:07.023+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:07.065 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:07.065 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:07.074 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:32:07.109 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:32:07.154 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:07.153+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:07.214 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:07.214 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:07.223 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:32:07.225 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:07.225 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:07.241 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:07.241 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:07.256 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:07.256 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:07.258 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is unset 2024-09-18T11:32:07.258 INFO:tasks.rados.rados.0.smithi119.stdout:1720: done (0 left) 2024-09-18T11:32:07.258 INFO:tasks.rados.rados.0.smithi119.stdout:1721: tier_flush oid 14 2024-09-18T11:32:07.258 INFO:tasks.rados.rados.0.smithi119.stdout:1721: tier_flush oid 14 snap -1 2024-09-18T11:32:07.259 INFO:tasks.rados.rados.0.smithi119.stdout:1722: read oid 10 snap -1 2024-09-18T11:32:07.259 INFO:tasks.rados.rados.0.smithi119.stdout:1722: expect (ObjNum 260 snap 31 seq_num 260) 2024-09-18T11:32:07.259 INFO:tasks.rados.rados.0.smithi119.stdout:1723: snap_remove snap 41 2024-09-18T11:32:07.267 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:07.265+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:07.277 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:07.278 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:07.293 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:07.293 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:07.309 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:07.309 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:07.325 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:07.325 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:07.327 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:07.326+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:07.330 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:32:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 92.911554476000006, 2024-09-18T11:32:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:32:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:32:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:07.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:32:07.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:07.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:32:07.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:32:07.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:32:07.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:32:07.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:32:07.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:32:07.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:32:07.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:32:07.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:32:07.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:32:07.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:07.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:32:07.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:32:07.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:07.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:32:07.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:32:07.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:07.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:32:07.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:32:07.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:32:07.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:32:07.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:32:07.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:32:07.336 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.336 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.336 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.337 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1664 4.1 4:87078dfe:::smithi11939506-14:head [setxattr _header (57) in=7b,truncate 2235331] snapc 18=[18,17,16,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:32:07.337 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:32:07.337 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 89.963404256000004, 2024-09-18T11:32:07.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010368235999999999, 2024-09-18T11:32:07.337 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.337 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.337 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.337 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.337 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.337 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1664 2024-09-18T11:32:07.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.337 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:32:07.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364237+0000", 2024-09-18T11:32:07.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364236+0000", 2024-09-18T11:32:07.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:07.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364240+0000", 2024-09-18T11:32:07.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9829999999999998e-06 2024-09-18T11:32:07.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364247+0000", 2024-09-18T11:32:07.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2140000000000001e-06 2024-09-18T11:32:07.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364256+0000", 2024-09-18T11:32:07.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1509999999999993e-06 2024-09-18T11:32:07.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366529+0000", 2024-09-18T11:32:07.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022731409999999998 2024-09-18T11:32:07.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366621+0000", 2024-09-18T11:32:07.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2408e-05 2024-09-18T11:32:07.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:07.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.366801+0000", 2024-09-18T11:32:07.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018002200000000001 2024-09-18T11:32:07.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:07.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.372155+0000", 2024-09-18T11:32:07.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053531519999999999 2024-09-18T11:32:07.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:07.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374539+0000", 2024-09-18T11:32:07.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023842580000000002 2024-09-18T11:32:07.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374573+0000", 2024-09-18T11:32:07.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4205999999999997e-05 2024-09-18T11:32:07.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374605+0000", 2024-09-18T11:32:07.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2283999999999998e-05 2024-09-18T11:32:07.342 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.342 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.342 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.342 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1665 4.1 4:87078dfe:::smithi11939506-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:32:07.342 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:32:07.342 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 89.963380082, 2024-09-18T11:32:07.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010471897000000001, 2024-09-18T11:32:07.343 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.343 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:07.343 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.343 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.343 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.343 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1665 2024-09-18T11:32:07.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.343 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:32:07.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:32:07.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364261+0000", 2024-09-18T11:32:07.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:07.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364263+0000", 2024-09-18T11:32:07.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6400000000000001e-06 2024-09-18T11:32:07.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364270+0000", 2024-09-18T11:32:07.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0900000000000001e-06 2024-09-18T11:32:07.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.364274+0000", 2024-09-18T11:32:07.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.968e-06 2024-09-18T11:32:07.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367354+0000", 2024-09-18T11:32:07.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030803430000000001 2024-09-18T11:32:07.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:07.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367406+0000", 2024-09-18T11:32:07.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2330999999999998e-05 2024-09-18T11:32:07.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374625+0000", 2024-09-18T11:32:07.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072190689999999998 2024-09-18T11:32:07.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374673+0000", 2024-09-18T11:32:07.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7525999999999999e-05 2024-09-18T11:32:07.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.374733+0000", 2024-09-18T11:32:07.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0359000000000003e-05 2024-09-18T11:32:07.347 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.347 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.348 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.348 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1947 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865577,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:32:07.348 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:32:07.348 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 75.988100372000005, 2024-09-18T11:32:07.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010043066999999999, 2024-09-18T11:32:07.348 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.348 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:07.348 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.348 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.348 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.349 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1947 2024-09-18T11:32:07.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.349 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:32:07.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339541+0000", 2024-09-18T11:32:07.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339540+0000", 2024-09-18T11:32:07.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:07.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339545+0000", 2024-09-18T11:32:07.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6290000000000001e-06 2024-09-18T11:32:07.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339553+0000", 2024-09-18T11:32:07.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6299999999999998e-06 2024-09-18T11:32:07.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339557+0000", 2024-09-18T11:32:07.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3590000000000001e-06 2024-09-18T11:32:07.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345600+0000", 2024-09-18T11:32:07.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060434640000000001 2024-09-18T11:32:07.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.345710+0000", 2024-09-18T11:32:07.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010924700000000001 2024-09-18T11:32:07.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349584+0000", 2024-09-18T11:32:07.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038743459999999999 2024-09-18T11:32:07.352 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.352 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.352 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.352 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1948 4.d 4:b5a463bf:::smithi11939506-20:head [sparse-read 0~2642181] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:32:07.352 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:32:07.352 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 75.988079834999994, 2024-09-18T11:32:07.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010701952000000001, 2024-09-18T11:32:07.353 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.353 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:07.353 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.353 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.353 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.353 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1948 2024-09-18T11:32:07.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.353 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:32:07.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:32:07.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339561+0000", 2024-09-18T11:32:07.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:07.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339563+0000", 2024-09-18T11:32:07.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1950000000000002e-06 2024-09-18T11:32:07.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339569+0000", 2024-09-18T11:32:07.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5459999999999999e-06 2024-09-18T11:32:07.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.339572+0000", 2024-09-18T11:32:07.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.337e-06 2024-09-18T11:32:07.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349693+0000", 2024-09-18T11:32:07.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010120987999999999 2024-09-18T11:32:07.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.349816+0000", 2024-09-18T11:32:07.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012241199999999999 2024-09-18T11:32:07.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:51.350263+0000", 2024-09-18T11:32:07.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00044787199999999998 2024-09-18T11:32:07.357 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.357 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.357 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.357 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:172 4.5 e78/22 4:a087a566:::smithi11939506-46:19 v 78'135, mlcod=77'132)", 2024-09-18T11:32:07.357 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:32:07.357 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 65.941817822000004, 2024-09-18T11:32:07.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010938658, 2024-09-18T11:32:07.358 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.358 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.358 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:32:07.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:32:07.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385823+0000", 2024-09-18T11:32:07.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:07.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385830+0000", 2024-09-18T11:32:07.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2790000000000001e-06 2024-09-18T11:32:07.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385843+0000", 2024-09-18T11:32:07.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2785e-05 2024-09-18T11:32:07.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385848+0000", 2024-09-18T11:32:07.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5570000000000003e-06 2024-09-18T11:32:07.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385914+0000", 2024-09-18T11:32:07.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5168999999999999e-05 2024-09-18T11:32:07.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385930+0000", 2024-09-18T11:32:07.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6884999999999998e-05 2024-09-18T11:32:07.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396721+0000", 2024-09-18T11:32:07.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010790137 2024-09-18T11:32:07.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396762+0000", 2024-09-18T11:32:07.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1635999999999999e-05 2024-09-18T11:32:07.361 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.361 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.362 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.362 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:32:07.362 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:07.362 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 65.941325070000005, 2024-09-18T11:32:07.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:32:07.362 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.362 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.362 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:07.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:07.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:07.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:07.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:32:07.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:32:07.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:32:07.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:32:07.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:32:07.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:32:07.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:32:07.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:32:07.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:32:07.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:32:07.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:32:07.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:32:07.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:32:07.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:32:07.366 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.367 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.367 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.367 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:07.367 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:07.367 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 58.355212463999997, 2024-09-18T11:32:07.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:32:07.367 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.367 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.367 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.367 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.368 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.368 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:32:07.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.368 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:07.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:07.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:32:07.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:07.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:32:07.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:32:07.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:32:07.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:32:07.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:32:07.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:32:07.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:32:07.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:32:07.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:32:07.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:32:07.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:07.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:32:07.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:32:07.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:07.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:32:07.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:32:07.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:07.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:32:07.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:32:07.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:32:07.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:32:07.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:32:07.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:32:07.372 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.372 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.373 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.373 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2647 4.e 4:78eefc4b:::smithi11939506-7:head [append 0~0] snapc 30=[30,2f,2b,2a,25,24,1c] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:07.373 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:32:07.373 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.317797784, 2024-09-18T11:32:07.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096752739999999993, 2024-09-18T11:32:07.373 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.373 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.373 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.373 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.373 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.373 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2647 2024-09-18T11:32:07.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.373 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:32:07.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009844+0000", 2024-09-18T11:32:07.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009843+0000", 2024-09-18T11:32:07.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:07.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009846+0000", 2024-09-18T11:32:07.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3870000000000002e-06 2024-09-18T11:32:07.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009853+0000", 2024-09-18T11:32:07.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.238e-06 2024-09-18T11:32:07.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.009860+0000", 2024-09-18T11:32:07.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1999999999999997e-06 2024-09-18T11:32:07.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.013798+0000", 2024-09-18T11:32:07.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003938441 2024-09-18T11:32:07.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.013889+0000", 2024-09-18T11:32:07.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0332000000000002e-05 2024-09-18T11:32:07.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 6", 2024-09-18T11:32:07.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.014066+0000", 2024-09-18T11:32:07.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017728700000000001 2024-09-18T11:32:07.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:07.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.015813+0000", 2024-09-18T11:32:07.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001746804 2024-09-18T11:32:07.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:07.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019419+0000", 2024-09-18T11:32:07.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0036058399999999999 2024-09-18T11:32:07.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019478+0000", 2024-09-18T11:32:07.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8860999999999998e-05 2024-09-18T11:32:07.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.019519+0000", 2024-09-18T11:32:07.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1208000000000003e-05 2024-09-18T11:32:07.378 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.379 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.379 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.379 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:209 4.5 e91/22 4:af4ba75a:::smithi11939506-37:19 v 91'165, mlcod=91'161)", 2024-09-18T11:32:07.379 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:32:07.379 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.309721056000001, 2024-09-18T11:32:07.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0091546930000000002, 2024-09-18T11:32:07.379 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.379 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.379 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:32:07.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:32:07.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017920+0000", 2024-09-18T11:32:07.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:07.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017925+0000", 2024-09-18T11:32:07.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0819999999999998e-06 2024-09-18T11:32:07.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017935+0000", 2024-09-18T11:32:07.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0597999999999999e-05 2024-09-18T11:32:07.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017941+0000", 2024-09-18T11:32:07.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2920000000000003e-06 2024-09-18T11:32:07.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.017978+0000", 2024-09-18T11:32:07.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7667999999999999e-05 2024-09-18T11:32:07.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.018018+0000", 2024-09-18T11:32:07.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9700999999999998e-05 2024-09-18T11:32:07.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.027039+0000", 2024-09-18T11:32:07.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090214079999999999 2024-09-18T11:32:07.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:14.027075+0000", 2024-09-18T11:32:07.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5605000000000002e-05 2024-09-18T11:32:07.383 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.383 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.383 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.384 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2955 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:07.384 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:32:07.384 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 43.300450034999997, 2024-09-18T11:32:07.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010869531, 2024-09-18T11:32:07.384 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.384 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:07.384 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.384 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.384 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.384 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2955 2024-09-18T11:32:07.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.385 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:32:07.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027191+0000", 2024-09-18T11:32:07.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027190+0000", 2024-09-18T11:32:07.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:07.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027196+0000", 2024-09-18T11:32:07.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5590000000000002e-06 2024-09-18T11:32:07.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027208+0000", 2024-09-18T11:32:07.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1816e-05 2024-09-18T11:32:07.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027215+0000", 2024-09-18T11:32:07.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4509999999999997e-06 2024-09-18T11:32:07.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027725+0000", 2024-09-18T11:32:07.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000510053 2024-09-18T11:32:07.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:07.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027803+0000", 2024-09-18T11:32:07.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8750000000000003e-05 2024-09-18T11:32:07.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032412+0000", 2024-09-18T11:32:07.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046089030000000001 2024-09-18T11:32:07.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032499+0000", 2024-09-18T11:32:07.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6316000000000005e-05 2024-09-18T11:32:07.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038061+0000", 2024-09-18T11:32:07.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005562259 2024-09-18T11:32:07.389 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.389 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.389 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.389 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2956 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~2100908,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:07.389 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:07.390 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 43.300416214000002, 2024-09-18T11:32:07.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011863831, 2024-09-18T11:32:07.390 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.390 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:07.390 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.390 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.390 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.390 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2956 2024-09-18T11:32:07.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.390 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:07.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:07.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:07.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:07.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027228+0000", 2024-09-18T11:32:07.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:32:07.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027235+0000", 2024-09-18T11:32:07.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:32:07.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027239+0000", 2024-09-18T11:32:07.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.535e-06 2024-09-18T11:32:07.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027831+0000", 2024-09-18T11:32:07.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059244699999999996 2024-09-18T11:32:07.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:07.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027879+0000", 2024-09-18T11:32:07.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:32:07.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038098+0000", 2024-09-18T11:32:07.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010219137 2024-09-18T11:32:07.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038207+0000", 2024-09-18T11:32:07.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010951999999999999 2024-09-18T11:32:07.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039089+0000", 2024-09-18T11:32:07.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00088151699999999996 2024-09-18T11:32:07.395 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.395 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.395 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.395 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2957 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:07.395 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:07.395 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 43.300398430000001, 2024-09-18T11:32:07.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012041979, 2024-09-18T11:32:07.395 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.396 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:07.396 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.396 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.396 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.396 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2957 2024-09-18T11:32:07.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.396 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:07.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:07.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:07.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:07.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027245+0000", 2024-09-18T11:32:07.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:32:07.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027251+0000", 2024-09-18T11:32:07.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0090000000000004e-06 2024-09-18T11:32:07.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027255+0000", 2024-09-18T11:32:07.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:32:07.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027895+0000", 2024-09-18T11:32:07.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064076400000000005 2024-09-18T11:32:07.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:07.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027930+0000", 2024-09-18T11:32:07.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4334e-05 2024-09-18T11:32:07.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039111+0000", 2024-09-18T11:32:07.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181455999999999 2024-09-18T11:32:07.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039156+0000", 2024-09-18T11:32:07.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5256000000000002e-05 2024-09-18T11:32:07.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039285+0000", 2024-09-18T11:32:07.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012839200000000001 2024-09-18T11:32:07.400 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.400 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.400 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.401 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:978 4.b e132/67 4:d8d14c58:::smithi11939506-49:30 v 132'417, mlcod=128'414)", 2024-09-18T11:32:07.401 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:07.401 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.3023671480000001, 2024-09-18T11:32:07.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011942183, 2024-09-18T11:32:07.401 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.401 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.401 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:07.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:07.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:07.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:07.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025278+0000", 2024-09-18T11:32:07.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9060000000000004e-06 2024-09-18T11:32:07.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025288+0000", 2024-09-18T11:32:07.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.028e-05 2024-09-18T11:32:07.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025293+0000", 2024-09-18T11:32:07.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2700000000000004e-06 2024-09-18T11:32:07.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025346+0000", 2024-09-18T11:32:07.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2862999999999999e-05 2024-09-18T11:32:07.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025370+0000", 2024-09-18T11:32:07.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4005000000000001e-05 2024-09-18T11:32:07.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037159+0000", 2024-09-18T11:32:07.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011788343 2024-09-18T11:32:07.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037216+0000", 2024-09-18T11:32:07.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7695000000000002e-05 2024-09-18T11:32:07.405 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.405 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.405 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.405 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:980 4.b e132/67 4:d13decd8:::smithi11939506-2:3e v 132'419, mlcod=128'414)", 2024-09-18T11:32:07.405 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.026846+0000", 2024-09-18T11:32:07.405 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.3007951929999999, 2024-09-18T11:32:07.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010397910999999999, 2024-09-18T11:32:07.405 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.405 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.405 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.026846+0000", 2024-09-18T11:32:07.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.026846+0000", 2024-09-18T11:32:07.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.026845+0000", 2024-09-18T11:32:07.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:07.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.026857+0000", 2024-09-18T11:32:07.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1533e-05 2024-09-18T11:32:07.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.026869+0000", 2024-09-18T11:32:07.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1775e-05 2024-09-18T11:32:07.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.026875+0000", 2024-09-18T11:32:07.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5420000000000002e-06 2024-09-18T11:32:07.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.026931+0000", 2024-09-18T11:32:07.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5748000000000002e-05 2024-09-18T11:32:07.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.026951+0000", 2024-09-18T11:32:07.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0567000000000001e-05 2024-09-18T11:32:07.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037222+0000", 2024-09-18T11:32:07.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010270153000000001 2024-09-18T11:32:07.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037244+0000", 2024-09-18T11:32:07.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2524e-05 2024-09-18T11:32:07.409 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.410 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.410 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.410 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3930 4.b e132/67 4:d3ae8215:::smithi11939506-41:head v 132'421, mlcod=128'414)", 2024-09-18T11:32:07.410 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:07.410 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.2988188999999997, 2024-09-18T11:32:07.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015093554, 2024-09-18T11:32:07.410 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.410 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.410 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:07.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:07.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028821+0000", 2024-09-18T11:32:07.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:07.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029805+0000", 2024-09-18T11:32:07.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00098410899999999994 2024-09-18T11:32:07.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029817+0000", 2024-09-18T11:32:07.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2058e-05 2024-09-18T11:32:07.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029823+0000", 2024-09-18T11:32:07.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6450000000000002e-06 2024-09-18T11:32:07.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032330+0000", 2024-09-18T11:32:07.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025067710000000001 2024-09-18T11:32:07.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032349+0000", 2024-09-18T11:32:07.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9729e-05 2024-09-18T11:32:07.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043878+0000", 2024-09-18T11:32:07.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011528514 2024-09-18T11:32:07.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043916+0000", 2024-09-18T11:32:07.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8049000000000001e-05 2024-09-18T11:32:07.414 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.414 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.414 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.414 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:337 4.5 e132/22 4:aa575018:::smithi11939506-11:35 v 132'284, mlcod=129'279)", 2024-09-18T11:32:07.414 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.030014+0000", 2024-09-18T11:32:07.414 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.2976270970000003, 2024-09-18T11:32:07.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098918919999999994, 2024-09-18T11:32:07.414 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.415 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.415 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.030014+0000", 2024-09-18T11:32:07.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.030014+0000", 2024-09-18T11:32:07.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.030014+0000", 2024-09-18T11:32:07.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:07.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.030019+0000", 2024-09-18T11:32:07.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7250000000000002e-06 2024-09-18T11:32:07.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.030030+0000", 2024-09-18T11:32:07.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0720000000000001e-05 2024-09-18T11:32:07.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.030035+0000", 2024-09-18T11:32:07.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2190000000000003e-06 2024-09-18T11:32:07.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032640+0000", 2024-09-18T11:32:07.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026045869999999998 2024-09-18T11:32:07.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032658+0000", 2024-09-18T11:32:07.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7983e-05 2024-09-18T11:32:07.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.039871+0000", 2024-09-18T11:32:07.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072134520000000004 2024-09-18T11:32:07.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.039906+0000", 2024-09-18T11:32:07.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4811000000000003e-05 2024-09-18T11:32:07.419 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.419 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.419 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.419 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3931 4.b e132/67 4:d3ae8215:::smithi11939506-41:head v 132'422, mlcod=128'414)", 2024-09-18T11:32:07.419 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.034467+0000", 2024-09-18T11:32:07.419 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.2931741739999998, 2024-09-18T11:32:07.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094796440000000006, 2024-09-18T11:32:07.419 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.419 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.420 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.034467+0000", 2024-09-18T11:32:07.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.034467+0000", 2024-09-18T11:32:07.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.034466+0000", 2024-09-18T11:32:07.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:07.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.035250+0000", 2024-09-18T11:32:07.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078353699999999995 2024-09-18T11:32:07.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.035404+0000", 2024-09-18T11:32:07.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015400200000000001 2024-09-18T11:32:07.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.035410+0000", 2024-09-18T11:32:07.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0039999999999996e-06 2024-09-18T11:32:07.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037281+0000", 2024-09-18T11:32:07.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018710199999999999 2024-09-18T11:32:07.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037300+0000", 2024-09-18T11:32:07.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9215000000000002e-05 2024-09-18T11:32:07.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043922+0000", 2024-09-18T11:32:07.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066223799999999998 2024-09-18T11:32:07.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043947+0000", 2024-09-18T11:32:07.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4440999999999999e-05 2024-09-18T11:32:07.423 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.424 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.424 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.424 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3933 4.b e132/67 4:d3ae8215:::smithi11939506-41:head v 132'428, mlcod=132'417)", 2024-09-18T11:32:07.424 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.044472+0000", 2024-09-18T11:32:07.424 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.2831690680000003, 2024-09-18T11:32:07.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096047660000000007, 2024-09-18T11:32:07.424 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.424 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.424 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.044472+0000", 2024-09-18T11:32:07.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.044472+0000", 2024-09-18T11:32:07.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.044472+0000", 2024-09-18T11:32:07.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:07.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.044650+0000", 2024-09-18T11:32:07.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00017815999999999999 2024-09-18T11:32:07.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.044660+0000", 2024-09-18T11:32:07.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0265000000000001e-05 2024-09-18T11:32:07.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.044666+0000", 2024-09-18T11:32:07.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4289999999999998e-06 2024-09-18T11:32:07.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.045521+0000", 2024-09-18T11:32:07.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00085499999999999997 2024-09-18T11:32:07.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.045539+0000", 2024-09-18T11:32:07.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7815000000000001e-05 2024-09-18T11:32:07.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.054043+0000", 2024-09-18T11:32:07.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085036420000000005 2024-09-18T11:32:07.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.054077+0000", 2024-09-18T11:32:07.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4208999999999997e-05 2024-09-18T11:32:07.428 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.428 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.428 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.428 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3938 4.1 4:86d80ae6:::smithi11939506-39:head [assert-version v250,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:07.429 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.053159+0000", 2024-09-18T11:32:07.429 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.2744821359999996, 2024-09-18T11:32:07.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097869289999999998, 2024-09-18T11:32:07.429 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.429 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.429 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.429 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.429 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.429 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3938 2024-09-18T11:32:07.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.429 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.053159+0000", 2024-09-18T11:32:07.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.053159+0000", 2024-09-18T11:32:07.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.053158+0000", 2024-09-18T11:32:07.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:07.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.053164+0000", 2024-09-18T11:32:07.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4920000000000001e-06 2024-09-18T11:32:07.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.053174+0000", 2024-09-18T11:32:07.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0292000000000001e-05 2024-09-18T11:32:07.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.053178+0000", 2024-09-18T11:32:07.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2169999999999996e-06 2024-09-18T11:32:07.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.053219+0000", 2024-09-18T11:32:07.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.074e-05 2024-09-18T11:32:07.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:07.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.053307+0000", 2024-09-18T11:32:07.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7869999999999997e-05 2024-09-18T11:32:07.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.054152+0000", 2024-09-18T11:32:07.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00084497900000000004 2024-09-18T11:32:07.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.054219+0000", 2024-09-18T11:32:07.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6971000000000006e-05 2024-09-18T11:32:07.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:07.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.058088+0000", 2024-09-18T11:32:07.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0038691680000000001 2024-09-18T11:32:07.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:07.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.062835+0000", 2024-09-18T11:32:07.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004746859 2024-09-18T11:32:07.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:07.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.062855+0000", 2024-09-18T11:32:07.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0109e-05 2024-09-18T11:32:07.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.062900+0000", 2024-09-18T11:32:07.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5522999999999998e-05 2024-09-18T11:32:07.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.062946+0000", 2024-09-18T11:32:07.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5643999999999999e-05 2024-09-18T11:32:07.435 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.435 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.435 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.435 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.435 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.436 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:07.436 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:07.438 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:32:07.438 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:07.439 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:07.436+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:07.538 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:07.537+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:07.577 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:07.578 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:07.578 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:07.578 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:07.585 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:32:07.639 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:07.637+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:07.683 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:07.683 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:07.683 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:07.683 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:07.683 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:07.691 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:32:07.722 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-18T11:32:07.722 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T11:32:07.722 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-18T11:32:07.722 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T11:32:07.722 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T11:32:07.723 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-18T11:32:07.768 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:07.767+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:07.824 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:07.824 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T11:32:07.824 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:07.832 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:07.832 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:07.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:07.832 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:07.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.832 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:32:07.832 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:32:07.832 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 137.80324144400001, 2024-09-18T11:32:07.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073601550000000002, 2024-09-18T11:32:07.832 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.833 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.833 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.833 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:32:07.833 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:32:07.833 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1 2024-09-18T11:32:07.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.833 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:32:07.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:32:07.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026524+0000", 2024-09-18T11:32:07.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999905 2024-09-18T11:32:07.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026541+0000", 2024-09-18T11:32:07.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7070000000000001e-05 2024-09-18T11:32:07.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026554+0000", 2024-09-18T11:32:07.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.288e-05 2024-09-18T11:32:07.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026564+0000", 2024-09-18T11:32:07.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0197000000000001e-05 2024-09-18T11:32:07.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026826+0000", 2024-09-18T11:32:07.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026212400000000003 2024-09-18T11:32:07.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:32:07.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026839+0000", 2024-09-18T11:32:07.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2831e-05 2024-09-18T11:32:07.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028605+0000", 2024-09-18T11:32:07.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001765794 2024-09-18T11:32:07.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028709+0000", 2024-09-18T11:32:07.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104348 2024-09-18T11:32:07.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:32:07.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028824+0000", 2024-09-18T11:32:07.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011527399999999999 2024-09-18T11:32:07.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:07.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030319+0000", 2024-09-18T11:32:07.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001494439 2024-09-18T11:32:07.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:07.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033805+0000", 2024-09-18T11:32:07.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034857820000000002 2024-09-18T11:32:07.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033856+0000", 2024-09-18T11:32:07.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1687999999999997e-05 2024-09-18T11:32:07.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033894+0000", 2024-09-18T11:32:07.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7481000000000002e-05 2024-09-18T11:32:07.839 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.839 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.839 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.840 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1118 3.2 e38/21 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head v 38'76, mlcod=38'74)", 2024-09-18T11:32:07.840 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:32:07.840 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 107.881831292, 2024-09-18T11:32:07.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006048688, 2024-09-18T11:32:07.840 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.840 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.840 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:32:07.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:32:07.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947941+0000", 2024-09-18T11:32:07.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:07.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949868+0000", 2024-09-18T11:32:07.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019267659999999999 2024-09-18T11:32:07.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949893+0000", 2024-09-18T11:32:07.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5239999999999999e-05 2024-09-18T11:32:07.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949901+0000", 2024-09-18T11:32:07.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8420000000000005e-06 2024-09-18T11:32:07.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949954+0000", 2024-09-18T11:32:07.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3464999999999997e-05 2024-09-18T11:32:07.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949974+0000", 2024-09-18T11:32:07.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9731000000000001e-05 2024-09-18T11:32:07.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953948+0000", 2024-09-18T11:32:07.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039735489999999998 2024-09-18T11:32:07.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953992+0000", 2024-09-18T11:32:07.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4700999999999998e-05 2024-09-18T11:32:07.844 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.844 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.844 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.844 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1662 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'73, mlcod=51'69)", 2024-09-18T11:32:07.844 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:32:07.844 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 90.468226256999998, 2024-09-18T11:32:07.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059408539999999998, 2024-09-18T11:32:07.844 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.845 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.845 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:32:07.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:32:07.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:32:07.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:07.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361914+0000", 2024-09-18T11:32:07.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036569 2024-09-18T11:32:07.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361926+0000", 2024-09-18T11:32:07.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1960999999999999e-05 2024-09-18T11:32:07.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361937+0000", 2024-09-18T11:32:07.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0876999999999999e-05 2024-09-18T11:32:07.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361986+0000", 2024-09-18T11:32:07.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8652000000000001e-05 2024-09-18T11:32:07.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.362012+0000", 2024-09-18T11:32:07.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5854999999999999e-05 2024-09-18T11:32:07.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367460+0000", 2024-09-18T11:32:07.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054482100000000002 2024-09-18T11:32:07.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367490+0000", 2024-09-18T11:32:07.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9909000000000001e-05 2024-09-18T11:32:07.848 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.848 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.849 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.849 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:269 4.7 e58/22 4:e73ba68f:::smithi11939506-1:13 v 58'72, mlcod=57'69)", 2024-09-18T11:32:07.849 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:32:07.849 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.564394359999994, 2024-09-18T11:32:07.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065512569999999996, 2024-09-18T11:32:07.849 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.849 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.849 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:32:07.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:32:07.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265380+0000", 2024-09-18T11:32:07.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:07.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265388+0000", 2024-09-18T11:32:07.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.174e-06 2024-09-18T11:32:07.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265398+0000", 2024-09-18T11:32:07.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2599999999999994e-06 2024-09-18T11:32:07.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265402+0000", 2024-09-18T11:32:07.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0289999999999999e-06 2024-09-18T11:32:07.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265434+0000", 2024-09-18T11:32:07.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2511000000000001e-05 2024-09-18T11:32:07.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265447+0000", 2024-09-18T11:32:07.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2947e-05 2024-09-18T11:32:07.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271892+0000", 2024-09-18T11:32:07.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064445930000000002 2024-09-18T11:32:07.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271932+0000", 2024-09-18T11:32:07.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0303999999999999e-05 2024-09-18T11:32:07.856 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.856 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.856 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.857 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1788 4.8 4:1898b15d:::smithi11939506-25:head [write 453822~797061] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:32:07.857 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:07.857 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 84.913961776999997, 2024-09-18T11:32:07.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094305159999999999, 2024-09-18T11:32:07.857 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.857 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.857 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.857 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.857 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.857 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1788 2024-09-18T11:32:07.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.858 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:07.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:07.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915811+0000", 2024-09-18T11:32:07.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:07.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916608+0000", 2024-09-18T11:32:07.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079635000000000003 2024-09-18T11:32:07.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916624+0000", 2024-09-18T11:32:07.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6735000000000002e-05 2024-09-18T11:32:07.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916630+0000", 2024-09-18T11:32:07.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5709999999999997e-06 2024-09-18T11:32:07.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916674+0000", 2024-09-18T11:32:07.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4540999999999999e-05 2024-09-18T11:32:07.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916751+0000", 2024-09-18T11:32:07.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6695999999999999e-05 2024-09-18T11:32:07.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:07.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916872+0000", 2024-09-18T11:32:07.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120648 2024-09-18T11:32:07.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:07.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.919076+0000", 2024-09-18T11:32:07.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022046829999999998 2024-09-18T11:32:07.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:07.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925151+0000", 2024-09-18T11:32:07.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060740239999999999 2024-09-18T11:32:07.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925198+0000", 2024-09-18T11:32:07.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7896999999999999e-05 2024-09-18T11:32:07.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925244+0000", 2024-09-18T11:32:07.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5398999999999997e-05 2024-09-18T11:32:07.862 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.862 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.863 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.863 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1934 4.2 e65/22 4:425afe9c:::smithi11939506-29:head v 65'44, mlcod=60'40)", 2024-09-18T11:32:07.863 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:32:07.863 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 79.529783007000006, 2024-09-18T11:32:07.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064585119999999996, 2024-09-18T11:32:07.863 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.863 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.863 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:32:07.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:32:07.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299990+0000", 2024-09-18T11:32:07.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:07.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300789+0000", 2024-09-18T11:32:07.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079851499999999997 2024-09-18T11:32:07.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300806+0000", 2024-09-18T11:32:07.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7096999999999999e-05 2024-09-18T11:32:07.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300813+0000", 2024-09-18T11:32:07.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5079999999999999e-06 2024-09-18T11:32:07.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300870+0000", 2024-09-18T11:32:07.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7178e-05 2024-09-18T11:32:07.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300897+0000", 2024-09-18T11:32:07.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6284000000000001e-05 2024-09-18T11:32:07.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306410+0000", 2024-09-18T11:32:07.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055135210000000004 2024-09-18T11:32:07.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306451+0000", 2024-09-18T11:32:07.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0379000000000001e-05 2024-09-18T11:32:07.867 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.867 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.867 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.868 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2452 4.3 4:ca2b2309:::smithi11939506-44:head [read 0~3462242,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:32:07.868 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:32:07.868 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 61.79300765, 2024-09-18T11:32:07.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060906450000000004, 2024-09-18T11:32:07.868 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.868 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:07.868 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.868 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.868 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.868 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2452 2024-09-18T11:32:07.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.869 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:32:07.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:32:07.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036765+0000", 2024-09-18T11:32:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036777+0000", 2024-09-18T11:32:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1422e-05 2024-09-18T11:32:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036793+0000", 2024-09-18T11:32:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6575999999999999e-05 2024-09-18T11:32:07.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036800+0000", 2024-09-18T11:32:07.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9940000000000003e-06 2024-09-18T11:32:07.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:32:07.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5847999999999999e-05 2024-09-18T11:32:07.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:07.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036853+0000", 2024-09-18T11:32:07.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0119999999999999e-06 2024-09-18T11:32:07.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038527+0000", 2024-09-18T11:32:07.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016732730000000001 2024-09-18T11:32:07.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038580+0000", 2024-09-18T11:32:07.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3443999999999999e-05 2024-09-18T11:32:07.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042858+0000", 2024-09-18T11:32:07.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042781049999999999 2024-09-18T11:32:07.873 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.873 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.873 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.873 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2453 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:32:07.873 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:32:07.874 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 61.792965418000001, 2024-09-18T11:32:07.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063798429999999996, 2024-09-18T11:32:07.874 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.874 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:07.874 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.874 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.874 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.874 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2453 2024-09-18T11:32:07.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.874 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:32:07.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:32:07.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036809+0000", 2024-09-18T11:32:07.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:07.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036812+0000", 2024-09-18T11:32:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8339999999999999e-06 2024-09-18T11:32:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036819+0000", 2024-09-18T11:32:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9890000000000003e-06 2024-09-18T11:32:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036823+0000", 2024-09-18T11:32:07.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8099999999999999e-06 2024-09-18T11:32:07.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036912+0000", 2024-09-18T11:32:07.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9083000000000001e-05 2024-09-18T11:32:07.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:07.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036932+0000", 2024-09-18T11:32:07.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0222000000000001e-05 2024-09-18T11:32:07.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042896+0000", 2024-09-18T11:32:07.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059637240000000001 2024-09-18T11:32:07.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042952+0000", 2024-09-18T11:32:07.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5877000000000001e-05 2024-09-18T11:32:07.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043190+0000", 2024-09-18T11:32:07.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00023774500000000001 2024-09-18T11:32:07.878 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.878 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.878 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.879 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2454 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:32:07.879 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:32:07.879 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 61.792929462000004, 2024-09-18T11:32:07.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066404469999999998, 2024-09-18T11:32:07.879 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.879 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:07.879 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.879 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.879 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.879 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2454 2024-09-18T11:32:07.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.879 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:32:07.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:32:07.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036845+0000", 2024-09-18T11:32:07.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:07.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036856+0000", 2024-09-18T11:32:07.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1338000000000001e-05 2024-09-18T11:32:07.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036863+0000", 2024-09-18T11:32:07.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4749999999999998e-06 2024-09-18T11:32:07.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036867+0000", 2024-09-18T11:32:07.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3839999999999999e-06 2024-09-18T11:32:07.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036946+0000", 2024-09-18T11:32:07.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9054000000000006e-05 2024-09-18T11:32:07.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:07.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036949+0000", 2024-09-18T11:32:07.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9390000000000002e-06 2024-09-18T11:32:07.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043213+0000", 2024-09-18T11:32:07.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062640090000000001 2024-09-18T11:32:07.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043261+0000", 2024-09-18T11:32:07.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7750999999999997e-05 2024-09-18T11:32:07.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043486+0000", 2024-09-18T11:32:07.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000224985 2024-09-18T11:32:07.884 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.884 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.884 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.884 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2553 4.3 4:ca2b2309:::smithi11939506-44:head [write 564768~630013,stat] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-18T11:32:07.884 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:32:07.884 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.632651828999997, 2024-09-18T11:32:07.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065686879999999996, 2024-09-18T11:32:07.884 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.884 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.885 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.885 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.885 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.885 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2553 2024-09-18T11:32:07.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.885 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:32:07.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:32:07.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197120+0000", 2024-09-18T11:32:07.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:07.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199000+0000", 2024-09-18T11:32:07.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018792080000000001 2024-09-18T11:32:07.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199044+0000", 2024-09-18T11:32:07.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4048999999999998e-05 2024-09-18T11:32:07.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199071+0000", 2024-09-18T11:32:07.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7288e-05 2024-09-18T11:32:07.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199119+0000", 2024-09-18T11:32:07.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8000999999999997e-05 2024-09-18T11:32:07.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199208+0000", 2024-09-18T11:32:07.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9209000000000006e-05 2024-09-18T11:32:07.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:07.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199385+0000", 2024-09-18T11:32:07.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000177095 2024-09-18T11:32:07.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:07.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.202282+0000", 2024-09-18T11:32:07.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028970469999999998 2024-09-18T11:32:07.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:07.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203578+0000", 2024-09-18T11:32:07.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012960490000000001 2024-09-18T11:32:07.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203638+0000", 2024-09-18T11:32:07.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9599000000000003e-05 2024-09-18T11:32:07.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203692+0000", 2024-09-18T11:32:07.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3908e-05 2024-09-18T11:32:07.890 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.890 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.890 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.890 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2752 4.3 4:ca2b2309:::smithi11939506-44:25 [read 0~3210104,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:32:07.890 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:32:07.890 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 49.942882668999999, 2024-09-18T11:32:07.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068715460000000001, 2024-09-18T11:32:07.891 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.891 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:07.891 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.891 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.891 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.891 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2752 2024-09-18T11:32:07.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.891 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:32:07.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:32:07.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:32:07.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:07.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886895+0000", 2024-09-18T11:32:07.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.993e-06 2024-09-18T11:32:07.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886901+0000", 2024-09-18T11:32:07.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1859999999999997e-06 2024-09-18T11:32:07.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886904+0000", 2024-09-18T11:32:07.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1920000000000001e-06 2024-09-18T11:32:07.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.892622+0000", 2024-09-18T11:32:07.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057175660000000003 2024-09-18T11:32:07.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.892693+0000", 2024-09-18T11:32:07.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1265000000000003e-05 2024-09-18T11:32:07.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.893764+0000", 2024-09-18T11:32:07.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001070753 2024-09-18T11:32:07.894 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.894 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.895 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.895 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2793 4.1 e92/22 4:87078dfe:::smithi11939506-14:head v 92'195, mlcod=92'191)", 2024-09-18T11:32:07.895 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:32:07.895 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 48.604982634999999, 2024-09-18T11:32:07.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062761989999999997, 2024-09-18T11:32:07.895 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.895 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.895 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:32:07.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:32:07.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:32:07.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:07.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224994+0000", 2024-09-18T11:32:07.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020197600000000001 2024-09-18T11:32:07.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225007+0000", 2024-09-18T11:32:07.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3791999999999999e-05 2024-09-18T11:32:07.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225013+0000", 2024-09-18T11:32:07.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3630000000000002e-06 2024-09-18T11:32:07.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225062+0000", 2024-09-18T11:32:07.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8925000000000003e-05 2024-09-18T11:32:07.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225086+0000", 2024-09-18T11:32:07.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4737999999999999e-05 2024-09-18T11:32:07.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231022+0000", 2024-09-18T11:32:07.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059360690000000004 2024-09-18T11:32:07.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231069+0000", 2024-09-18T11:32:07.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6252999999999999e-05 2024-09-18T11:32:07.899 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.899 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.899 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.899 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2876 4.3 4:cc46532a:::smithi11939506-33:2f [read 0~3680733,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:07.900 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:32:07.900 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.322913735, 2024-09-18T11:32:07.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068391210000000001, 2024-09-18T11:32:07.900 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.900 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:07.900 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.900 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.900 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.900 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2876 2024-09-18T11:32:07.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.900 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:32:07.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:32:07.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506860+0000", 2024-09-18T11:32:07.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:07.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506869+0000", 2024-09-18T11:32:07.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5539999999999998e-06 2024-09-18T11:32:07.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506880+0000", 2024-09-18T11:32:07.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1552999999999999e-05 2024-09-18T11:32:07.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506887+0000", 2024-09-18T11:32:07.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7329999999999997e-06 2024-09-18T11:32:07.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506937+0000", 2024-09-18T11:32:07.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0317e-05 2024-09-18T11:32:07.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:07.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506945+0000", 2024-09-18T11:32:07.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1550000000000007e-06 2024-09-18T11:32:07.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.509021+0000", 2024-09-18T11:32:07.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020759870000000001 2024-09-18T11:32:07.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.509123+0000", 2024-09-18T11:32:07.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010132900000000001 2024-09-18T11:32:07.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513700+0000", 2024-09-18T11:32:07.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0045777320000000002 2024-09-18T11:32:07.904 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.904 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.905 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.905 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2877 4.3 4:cc46532a:::smithi11939506-33:2f [read 0~3680733,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:07.905 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:07.905 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.322879321000002, 2024-09-18T11:32:07.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085715170000000007, 2024-09-18T11:32:07.905 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.905 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:07.905 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.905 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.905 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.905 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2877 2024-09-18T11:32:07.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.906 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:07.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:07.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:07.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:07.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506898+0000", 2024-09-18T11:32:07.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4760000000000001e-06 2024-09-18T11:32:07.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506903+0000", 2024-09-18T11:32:07.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:32:07.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506906+0000", 2024-09-18T11:32:07.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1769999999999998e-06 2024-09-18T11:32:07.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507007+0000", 2024-09-18T11:32:07.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101479 2024-09-18T11:32:07.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:07.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507015+0000", 2024-09-18T11:32:07.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8490000000000006e-06 2024-09-18T11:32:07.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513757+0000", 2024-09-18T11:32:07.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0067417800000000002 2024-09-18T11:32:07.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513867+0000", 2024-09-18T11:32:07.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000109987 2024-09-18T11:32:07.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515467+0000", 2024-09-18T11:32:07.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00160037 2024-09-18T11:32:07.910 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.910 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.910 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.910 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2878 4.3 4:cc46532a:::smithi11939506-33:2f [sparse-read 0~1770976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:07.910 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:07.910 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.322848866000001, 2024-09-18T11:32:07.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090602640000000002, 2024-09-18T11:32:07.910 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.911 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:07.911 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.911 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.911 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.911 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2878 2024-09-18T11:32:07.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.911 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:07.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:07.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:07.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:07.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506929+0000", 2024-09-18T11:32:07.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.163e-06 2024-09-18T11:32:07.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506936+0000", 2024-09-18T11:32:07.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5519999999999999e-06 2024-09-18T11:32:07.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506941+0000", 2024-09-18T11:32:07.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:32:07.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507036+0000", 2024-09-18T11:32:07.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5070999999999998e-05 2024-09-18T11:32:07.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:07.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507042+0000", 2024-09-18T11:32:07.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3959999999999997e-06 2024-09-18T11:32:07.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515504+0000", 2024-09-18T11:32:07.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084620300000000006 2024-09-18T11:32:07.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515625+0000", 2024-09-18T11:32:07.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012150999999999999 2024-09-18T11:32:07.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515986+0000", 2024-09-18T11:32:07.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036140100000000002 2024-09-18T11:32:07.915 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.915 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.915 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.916 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:686 4.7 e98/22 4:e5707c18:::smithi11939506-10:30 v 98'160, mlcod=97'155)", 2024-09-18T11:32:07.916 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:32:07.916 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 43.811007175999997, 2024-09-18T11:32:07.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074641239999999999, 2024-09-18T11:32:07.916 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.916 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.916 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:32:07.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:32:07.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018767+0000", 2024-09-18T11:32:07.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:07.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018773+0000", 2024-09-18T11:32:07.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1460000000000004e-06 2024-09-18T11:32:07.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018781+0000", 2024-09-18T11:32:07.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9780000000000005e-06 2024-09-18T11:32:07.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018785+0000", 2024-09-18T11:32:07.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0029999999999996e-06 2024-09-18T11:32:07.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018826+0000", 2024-09-18T11:32:07.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1616999999999998e-05 2024-09-18T11:32:07.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018843+0000", 2024-09-18T11:32:07.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6753999999999999e-05 2024-09-18T11:32:07.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026202+0000", 2024-09-18T11:32:07.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073592620000000001 2024-09-18T11:32:07.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026232+0000", 2024-09-18T11:32:07.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9779000000000001e-05 2024-09-18T11:32:07.920 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.920 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.920 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.920 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.4.0:621 4.1 e98/22 4:854aa293:::smithi11939506-26:35 v 98'215, mlcod=96'210)", 2024-09-18T11:32:07.920 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:32:07.920 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 43.810123736999998, 2024-09-18T11:32:07.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007014752, 2024-09-18T11:32:07.921 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.921 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.921 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:32:07.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:32:07.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:32:07.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:07.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019659+0000", 2024-09-18T11:32:07.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8229999999999995e-06 2024-09-18T11:32:07.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019671+0000", 2024-09-18T11:32:07.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2636000000000001e-05 2024-09-18T11:32:07.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019677+0000", 2024-09-18T11:32:07.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5470000000000004e-06 2024-09-18T11:32:07.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019710+0000", 2024-09-18T11:32:07.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3191999999999997e-05 2024-09-18T11:32:07.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019722+0000", 2024-09-18T11:32:07.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2357e-05 2024-09-18T11:32:07.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026639+0000", 2024-09-18T11:32:07.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0069167630000000003 2024-09-18T11:32:07.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026666+0000", 2024-09-18T11:32:07.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7087999999999999e-05 2024-09-18T11:32:07.924 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.925 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.925 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.925 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3234 4.7 e109/22 4:e73ba68f:::smithi11939506-1:head v 109'206, mlcod=107'202)", 2024-09-18T11:32:07.925 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:37.128363+0000", 2024-09-18T11:32:07.925 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 30.701412352999998, 2024-09-18T11:32:07.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059367100000000004, 2024-09-18T11:32:07.925 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.925 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.925 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.128363+0000", 2024-09-18T11:32:07.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.128363+0000", 2024-09-18T11:32:07.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.128362+0000", 2024-09-18T11:32:07.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:07.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131291+0000", 2024-09-18T11:32:07.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029282840000000002 2024-09-18T11:32:07.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131305+0000", 2024-09-18T11:32:07.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4222999999999999e-05 2024-09-18T11:32:07.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131313+0000", 2024-09-18T11:32:07.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.208e-06 2024-09-18T11:32:07.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131354+0000", 2024-09-18T11:32:07.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1158000000000001e-05 2024-09-18T11:32:07.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.131378+0000", 2024-09-18T11:32:07.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3828999999999998e-05 2024-09-18T11:32:07.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.134258+0000", 2024-09-18T11:32:07.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028798920000000002 2024-09-18T11:32:07.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.134299+0000", 2024-09-18T11:32:07.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1355e-05 2024-09-18T11:32:07.929 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.929 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.929 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.929 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3490 4.3 4:cc46532a:::smithi11939506-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:07.930 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.709960+0000", 2024-09-18T11:32:07.930 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.119815338999999, 2024-09-18T11:32:07.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058965190000000002, 2024-09-18T11:32:07.930 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.930 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:07.930 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.930 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.930 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.930 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3490 2024-09-18T11:32:07.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.930 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709960+0000", 2024-09-18T11:32:07.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709960+0000", 2024-09-18T11:32:07.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709959+0000", 2024-09-18T11:32:07.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:07.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709964+0000", 2024-09-18T11:32:07.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0900000000000004e-06 2024-09-18T11:32:07.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709971+0000", 2024-09-18T11:32:07.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5950000000000004e-06 2024-09-18T11:32:07.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.709975+0000", 2024-09-18T11:32:07.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0690000000000003e-06 2024-09-18T11:32:07.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.712315+0000", 2024-09-18T11:32:07.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002339556 2024-09-18T11:32:07.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:07.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.712392+0000", 2024-09-18T11:32:07.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7868999999999996e-05 2024-09-18T11:32:07.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.715759+0000", 2024-09-18T11:32:07.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0033669989999999999 2024-09-18T11:32:07.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.715802+0000", 2024-09-18T11:32:07.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2636000000000003e-05 2024-09-18T11:32:07.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.715856+0000", 2024-09-18T11:32:07.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4153000000000001e-05 2024-09-18T11:32:07.934 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.934 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.935 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.935 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3775 4.3 4:c70c3162:::smithi11939506-35:head [write 796731~461096,stat] snapc 51=[51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b,30] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-18T11:32:07.935 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:58.217457+0000", 2024-09-18T11:32:07.935 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.6123182150000002, 2024-09-18T11:32:07.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064394090000000001, 2024-09-18T11:32:07.935 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:07.935 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:07.935 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:07.935 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:07.935 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:07.935 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3775 2024-09-18T11:32:07.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.936 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:07.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:07.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217457+0000", 2024-09-18T11:32:07.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:07.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217457+0000", 2024-09-18T11:32:07.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:07.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:07.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217455+0000", 2024-09-18T11:32:07.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:07.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:07.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217965+0000", 2024-09-18T11:32:07.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00050991999999999997 2024-09-18T11:32:07.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:07.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217977+0000", 2024-09-18T11:32:07.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1701e-05 2024-09-18T11:32:07.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:07.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217981+0000", 2024-09-18T11:32:07.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5600000000000004e-06 2024-09-18T11:32:07.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:07.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.218033+0000", 2024-09-18T11:32:07.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1728999999999999e-05 2024-09-18T11:32:07.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:07.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.218147+0000", 2024-09-18T11:32:07.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000114066 2024-09-18T11:32:07.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:07.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.218364+0000", 2024-09-18T11:32:07.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021718700000000001 2024-09-18T11:32:07.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:07.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.223702+0000", 2024-09-18T11:32:07.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005338036 2024-09-18T11:32:07.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:07.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.223812+0000", 2024-09-18T11:32:07.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000110011 2024-09-18T11:32:07.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:07.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.223868+0000", 2024-09-18T11:32:07.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5989999999999998e-05 2024-09-18T11:32:07.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:07.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:07.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:07.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.223896+0000", 2024-09-18T11:32:07.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7820000000000001e-05 2024-09-18T11:32:07.940 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.940 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.940 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.941 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:07.941 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:07.941 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:07.943 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:32:07.943 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:07.941+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:08.043 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:08.041+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:08.082 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:08.083 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:08.083 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:08.083 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:08.091 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:32:08.144 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:08.143+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:08.229 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:08.229 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:08.229 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:08.229 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:08.229 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:08.237 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:32:08.244 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:08.243+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:08.270 INFO:tasks.rados.rados.0.smithi119.stdout:1723: done (2 left) 2024-09-18T11:32:08.270 INFO:tasks.rados.rados.0.smithi119.stdout:1724: read oid 41 snap -1 2024-09-18T11:32:08.270 INFO:tasks.rados.rados.0.smithi119.stdout:1724: expect (ObjNum 347 snap 51 seq_num 347) 2024-09-18T11:32:08.270 INFO:tasks.rados.rados.0.smithi119.stdout:1725: snap_remove snap 53 2024-09-18T11:32:08.344 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:08.344 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:08.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:08.344 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:08.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.344 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:32:08.344 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:08.344 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 117.225109215, 2024-09-18T11:32:08.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:32:08.344 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.345 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:08.345 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.345 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:08.345 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:08.345 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:32:08.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.345 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:08.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:08.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:32:08.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:08.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:32:08.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:32:08.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:32:08.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:32:08.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:32:08.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:32:08.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:32:08.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:32:08.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:08.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:32:08.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:32:08.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:32:08.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:32:08.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:32:08.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:32:08.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:32:08.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:32:08.349 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.349 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.349 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.349 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:32:08.350 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:08.350 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 107.672083814, 2024-09-18T11:32:08.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:32:08.350 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.350 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:08.350 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.350 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:08.350 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:08.350 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:32:08.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.350 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:08.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:08.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:32:08.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:08.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:32:08.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:32:08.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:32:08.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:32:08.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:32:08.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:32:08.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:32:08.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:32:08.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:32:08.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:32:08.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:32:08.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:32:08.354 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.354 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.354 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.354 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2207 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1410296~761389] snapc 24=[24,1e,1c,1b,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T11:32:08.354 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:32:08.354 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 70.918931150999995, 2024-09-18T11:32:08.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008410743, 2024-09-18T11:32:08.354 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.354 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.354 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.355 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:08.355 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:08.355 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2207 2024-09-18T11:32:08.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.355 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:32:08.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:32:08.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422496+0000", 2024-09-18T11:32:08.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:08.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423190+0000", 2024-09-18T11:32:08.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00069405499999999995 2024-09-18T11:32:08.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423204+0000", 2024-09-18T11:32:08.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:32:08.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423208+0000", 2024-09-18T11:32:08.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:32:08.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423293+0000", 2024-09-18T11:32:08.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4524999999999995e-05 2024-09-18T11:32:08.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423394+0000", 2024-09-18T11:32:08.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101395 2024-09-18T11:32:08.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:08.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423579+0000", 2024-09-18T11:32:08.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018461000000000001 2024-09-18T11:32:08.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.428574+0000", 2024-09-18T11:32:08.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049944960000000002 2024-09-18T11:32:08.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430809+0000", 2024-09-18T11:32:08.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022351820000000001 2024-09-18T11:32:08.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430862+0000", 2024-09-18T11:32:08.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3189999999999997e-05 2024-09-18T11:32:08.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430908+0000", 2024-09-18T11:32:08.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5973999999999997e-05 2024-09-18T11:32:08.360 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.360 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.360 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.360 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2339 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:08.360 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:08.360 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.954626521999998, 2024-09-18T11:32:08.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011809992, 2024-09-18T11:32:08.361 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.361 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:08.361 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.361 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:08.361 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:08.361 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2339 2024-09-18T11:32:08.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.361 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:08.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:08.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:32:08.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:08.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:32:08.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4710000000000001e-06 2024-09-18T11:32:08.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386824+0000", 2024-09-18T11:32:08.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6104e-05 2024-09-18T11:32:08.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386836+0000", 2024-09-18T11:32:08.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1827999999999999e-05 2024-09-18T11:32:08.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388390+0000", 2024-09-18T11:32:08.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015539499999999999 2024-09-18T11:32:08.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:08.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388443+0000", 2024-09-18T11:32:08.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3785000000000001e-05 2024-09-18T11:32:08.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398238+0000", 2024-09-18T11:32:08.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097947989999999999 2024-09-18T11:32:08.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398315+0000", 2024-09-18T11:32:08.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6909999999999994e-05 2024-09-18T11:32:08.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398612+0000", 2024-09-18T11:32:08.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029671999999999999 2024-09-18T11:32:08.365 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.365 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.366 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.366 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2340 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:08.366 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:08.366 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.954583209999996, 2024-09-18T11:32:08.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012086448, 2024-09-18T11:32:08.366 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.366 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:08.366 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.366 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:08.366 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:08.367 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2340 2024-09-18T11:32:08.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.367 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:08.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:08.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:08.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386848+0000", 2024-09-18T11:32:08.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3069999999999998e-06 2024-09-18T11:32:08.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386855+0000", 2024-09-18T11:32:08.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0979999999999996e-06 2024-09-18T11:32:08.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386859+0000", 2024-09-18T11:32:08.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9360000000000001e-06 2024-09-18T11:32:08.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388466+0000", 2024-09-18T11:32:08.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001607154 2024-09-18T11:32:08.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:08.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388507+0000", 2024-09-18T11:32:08.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1093000000000001e-05 2024-09-18T11:32:08.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398647+0000", 2024-09-18T11:32:08.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010139686 2024-09-18T11:32:08.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398718+0000", 2024-09-18T11:32:08.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1138999999999998e-05 2024-09-18T11:32:08.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398931+0000", 2024-09-18T11:32:08.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021356899999999999 2024-09-18T11:32:08.371 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.371 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.371 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.371 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2341 4.5 4:aa575018:::smithi11939506-11:head [read 0~2487436,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:08.372 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:08.372 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.954564988000001, 2024-09-18T11:32:08.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013751648999999999, 2024-09-18T11:32:08.372 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.372 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:08.372 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.372 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:08.372 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:08.372 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2341 2024-09-18T11:32:08.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.372 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:08.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:08.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:08.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386866+0000", 2024-09-18T11:32:08.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4249999999999998e-06 2024-09-18T11:32:08.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386872+0000", 2024-09-18T11:32:08.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.003e-06 2024-09-18T11:32:08.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:32:08.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.507e-06 2024-09-18T11:32:08.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388520+0000", 2024-09-18T11:32:08.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001644146 2024-09-18T11:32:08.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:08.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388561+0000", 2024-09-18T11:32:08.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1486999999999998e-05 2024-09-18T11:32:08.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398963+0000", 2024-09-18T11:32:08.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010401353 2024-09-18T11:32:08.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.399046+0000", 2024-09-18T11:32:08.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3141999999999998e-05 2024-09-18T11:32:08.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.400615+0000", 2024-09-18T11:32:08.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015690960000000001 2024-09-18T11:32:08.376 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.376 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.377 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.377 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2529 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:08.377 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:32:08.377 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 59.168091724, 2024-09-18T11:32:08.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079268120000000001, 2024-09-18T11:32:08.377 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.377 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.377 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.377 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:08.377 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:08.377 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2529 2024-09-18T11:32:08.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.378 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:32:08.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:32:08.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173336+0000", 2024-09-18T11:32:08.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173339+0000", 2024-09-18T11:32:08.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9019999999999999e-06 2024-09-18T11:32:08.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173381+0000", 2024-09-18T11:32:08.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1836999999999999e-05 2024-09-18T11:32:08.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173390+0000", 2024-09-18T11:32:08.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.302e-06 2024-09-18T11:32:08.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174507+0000", 2024-09-18T11:32:08.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001116868 2024-09-18T11:32:08.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174603+0000", 2024-09-18T11:32:08.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5598000000000002e-05 2024-09-18T11:32:08.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:08.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174759+0000", 2024-09-18T11:32:08.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015616299999999999 2024-09-18T11:32:08.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178943+0000", 2024-09-18T11:32:08.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041846180000000002 2024-09-18T11:32:08.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181175+0000", 2024-09-18T11:32:08.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00223215 2024-09-18T11:32:08.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181229+0000", 2024-09-18T11:32:08.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3585999999999998e-05 2024-09-18T11:32:08.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181263+0000", 2024-09-18T11:32:08.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4292999999999998e-05 2024-09-18T11:32:08.382 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.382 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.383 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.383 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2530 4.5 4:af4ba75a:::smithi11939506-37:head [rollback 25] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:08.383 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:32:08.383 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 59.168029418000003, 2024-09-18T11:32:08.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080198800000000001, 2024-09-18T11:32:08.383 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.383 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.383 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.383 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:08.383 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:08.383 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2530 2024-09-18T11:32:08.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.384 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:32:08.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:32:08.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173398+0000", 2024-09-18T11:32:08.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:08.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173403+0000", 2024-09-18T11:32:08.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4389999999999999e-06 2024-09-18T11:32:08.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173414+0000", 2024-09-18T11:32:08.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0411000000000001e-05 2024-09-18T11:32:08.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173420+0000", 2024-09-18T11:32:08.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9610000000000002e-06 2024-09-18T11:32:08.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176390+0000", 2024-09-18T11:32:08.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029706289999999998 2024-09-18T11:32:08.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176501+0000", 2024-09-18T11:32:08.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011090300000000001 2024-09-18T11:32:08.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:08.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176827+0000", 2024-09-18T11:32:08.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00032526899999999999 2024-09-18T11:32:08.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178958+0000", 2024-09-18T11:32:08.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021314860000000001 2024-09-18T11:32:08.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181318+0000", 2024-09-18T11:32:08.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002359571 2024-09-18T11:32:08.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181372+0000", 2024-09-18T11:32:08.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3884000000000001e-05 2024-09-18T11:32:08.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181419+0000", 2024-09-18T11:32:08.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7148000000000003e-05 2024-09-18T11:32:08.389 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.389 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.389 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.389 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2531 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:08.389 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:08.389 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 59.168003294999998, 2024-09-18T11:32:08.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080687239999999993, 2024-09-18T11:32:08.389 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.389 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.389 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.389 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:08.390 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:08.390 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2531 2024-09-18T11:32:08.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.390 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:08.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:08.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:08.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173429+0000", 2024-09-18T11:32:08.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2989999999999998e-06 2024-09-18T11:32:08.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173436+0000", 2024-09-18T11:32:08.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1799999999999999e-06 2024-09-18T11:32:08.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173441+0000", 2024-09-18T11:32:08.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8810000000000004e-06 2024-09-18T11:32:08.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178267+0000", 2024-09-18T11:32:08.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048257969999999997 2024-09-18T11:32:08.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178388+0000", 2024-09-18T11:32:08.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121779 2024-09-18T11:32:08.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:08.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178599+0000", 2024-09-18T11:32:08.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000210094 2024-09-18T11:32:08.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.179339+0000", 2024-09-18T11:32:08.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00074063799999999995 2024-09-18T11:32:08.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181444+0000", 2024-09-18T11:32:08.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021044100000000001 2024-09-18T11:32:08.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181470+0000", 2024-09-18T11:32:08.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6415999999999999e-05 2024-09-18T11:32:08.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181494+0000", 2024-09-18T11:32:08.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3672e-05 2024-09-18T11:32:08.395 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.395 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.395 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.395 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2910 4.5 4:af4ba75a:::smithi11939506-37:head [write 786627~795991,stat] snapc 37=[37,35,34,30,2f,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:08.395 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:32:08.395 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.337945466000001, 2024-09-18T11:32:08.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084093710000000006, 2024-09-18T11:32:08.395 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.395 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.395 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.396 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:08.396 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:08.396 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2910 2024-09-18T11:32:08.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.396 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:32:08.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:32:08.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003481+0000", 2024-09-18T11:32:08.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:08.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004300+0000", 2024-09-18T11:32:08.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081934200000000003 2024-09-18T11:32:08.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004324+0000", 2024-09-18T11:32:08.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4618999999999999e-05 2024-09-18T11:32:08.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004335+0000", 2024-09-18T11:32:08.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.092e-05 2024-09-18T11:32:08.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004385+0000", 2024-09-18T11:32:08.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9709999999999997e-05 2024-09-18T11:32:08.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004470+0000", 2024-09-18T11:32:08.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4864000000000006e-05 2024-09-18T11:32:08.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:08.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004612+0000", 2024-09-18T11:32:08.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014241500000000001 2024-09-18T11:32:08.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011193+0000", 2024-09-18T11:32:08.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006580654 2024-09-18T11:32:08.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011788+0000", 2024-09-18T11:32:08.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059533600000000002 2024-09-18T11:32:08.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011842+0000", 2024-09-18T11:32:08.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3180000000000002e-05 2024-09-18T11:32:08.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011892+0000", 2024-09-18T11:32:08.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0618000000000002e-05 2024-09-18T11:32:08.401 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.401 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.401 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.401 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2953 4.a 4:588debc2:::smithi11939506-18:head [assert-version v113,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:08.401 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:32:08.401 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 44.314629947999997, 2024-09-18T11:32:08.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083931279999999997, 2024-09-18T11:32:08.401 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.401 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.401 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.402 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:08.402 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:08.402 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2953 2024-09-18T11:32:08.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.402 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:32:08.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:32:08.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026796+0000", 2024-09-18T11:32:08.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:08.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026806+0000", 2024-09-18T11:32:08.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9529999999999994e-06 2024-09-18T11:32:08.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026824+0000", 2024-09-18T11:32:08.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7901999999999999e-05 2024-09-18T11:32:08.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026832+0000", 2024-09-18T11:32:08.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8450000000000003e-06 2024-09-18T11:32:08.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026883+0000", 2024-09-18T11:32:08.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0883000000000001e-05 2024-09-18T11:32:08.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026980+0000", 2024-09-18T11:32:08.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7526000000000001e-05 2024-09-18T11:32:08.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:32:08.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032722+0000", 2024-09-18T11:32:08.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057414689999999999 2024-09-18T11:32:08.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.034230+0000", 2024-09-18T11:32:08.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001508035 2024-09-18T11:32:08.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035090+0000", 2024-09-18T11:32:08.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00086047300000000005 2024-09-18T11:32:08.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035147+0000", 2024-09-18T11:32:08.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6598000000000003e-05 2024-09-18T11:32:08.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035191+0000", 2024-09-18T11:32:08.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4747999999999999e-05 2024-09-18T11:32:08.407 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.407 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.407 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.407 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3105 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 3e=[3e,3d,3b,37,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-18T11:32:08.407 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:08.407 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.323217243999999, 2024-09-18T11:32:08.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011735906000000001, 2024-09-18T11:32:08.408 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.408 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:08.408 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.408 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:08.408 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:08.408 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3105 2024-09-18T11:32:08.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.408 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:08.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:08.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:08.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018212+0000", 2024-09-18T11:32:08.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6759999999999999e-06 2024-09-18T11:32:08.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018216+0000", 2024-09-18T11:32:08.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-06 2024-09-18T11:32:08.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018218+0000", 2024-09-18T11:32:08.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6110000000000001e-06 2024-09-18T11:32:08.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018246+0000", 2024-09-18T11:32:08.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7557000000000001e-05 2024-09-18T11:32:08.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:08.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018250+0000", 2024-09-18T11:32:08.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4299999999999999e-06 2024-09-18T11:32:08.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021364+0000", 2024-09-18T11:32:08.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031135389999999998 2024-09-18T11:32:08.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021494+0000", 2024-09-18T11:32:08.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000130333 2024-09-18T11:32:08.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.029947+0000", 2024-09-18T11:32:08.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084526150000000001 2024-09-18T11:32:08.412 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.412 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.413 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.413 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3323 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:08.413 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:32:08.413 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.224529199999999, 2024-09-18T11:32:08.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0088051689999999998, 2024-09-18T11:32:08.413 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.413 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.413 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.413 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:08.413 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:08.413 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3323 2024-09-18T11:32:08.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.414 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:32:08.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:32:08.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116897+0000", 2024-09-18T11:32:08.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:08.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116908+0000", 2024-09-18T11:32:08.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1511e-05 2024-09-18T11:32:08.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116925+0000", 2024-09-18T11:32:08.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7028e-05 2024-09-18T11:32:08.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116932+0000", 2024-09-18T11:32:08.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5919999999999997e-06 2024-09-18T11:32:08.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116988+0000", 2024-09-18T11:32:08.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5928999999999999e-05 2024-09-18T11:32:08.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:08.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116995+0000", 2024-09-18T11:32:08.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7409999999999992e-06 2024-09-18T11:32:08.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122390+0000", 2024-09-18T11:32:08.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053941889999999998 2024-09-18T11:32:08.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122453+0000", 2024-09-18T11:32:08.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3833999999999999e-05 2024-09-18T11:32:08.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:08.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122581+0000", 2024-09-18T11:32:08.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012751200000000001 2024-09-18T11:32:08.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125034+0000", 2024-09-18T11:32:08.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024525609999999998 2024-09-18T11:32:08.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125607+0000", 2024-09-18T11:32:08.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00057356499999999995 2024-09-18T11:32:08.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125655+0000", 2024-09-18T11:32:08.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8179000000000001e-05 2024-09-18T11:32:08.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125704+0000", 2024-09-18T11:32:08.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8950999999999999e-05 2024-09-18T11:32:08.419 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.419 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.419 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.419 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3324 4.5 4:a087a566:::smithi11939506-46:head [rollback 46] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:08.420 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:08.420 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.224486640999999, 2024-09-18T11:32:08.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010927767999999999, 2024-09-18T11:32:08.420 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.420 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.420 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.420 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:08.420 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:08.420 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3324 2024-09-18T11:32:08.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.420 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:08.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:08.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116941+0000", 2024-09-18T11:32:08.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116949+0000", 2024-09-18T11:32:08.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4660000000000002e-06 2024-09-18T11:32:08.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116960+0000", 2024-09-18T11:32:08.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1484999999999999e-05 2024-09-18T11:32:08.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116964+0000", 2024-09-18T11:32:08.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2810000000000002e-06 2024-09-18T11:32:08.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117053+0000", 2024-09-18T11:32:08.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8747999999999998e-05 2024-09-18T11:32:08.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:08.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117059+0000", 2024-09-18T11:32:08.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.246e-06 2024-09-18T11:32:08.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123604+0000", 2024-09-18T11:32:08.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065445319999999996 2024-09-18T11:32:08.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123663+0000", 2024-09-18T11:32:08.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8740999999999999e-05 2024-09-18T11:32:08.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:08.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123803+0000", 2024-09-18T11:32:08.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014049200000000001 2024-09-18T11:32:08.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127600+0000", 2024-09-18T11:32:08.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037971179999999999 2024-09-18T11:32:08.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127814+0000", 2024-09-18T11:32:08.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021333600000000001 2024-09-18T11:32:08.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127847+0000", 2024-09-18T11:32:08.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3188999999999997e-05 2024-09-18T11:32:08.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127869+0000", 2024-09-18T11:32:08.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2515e-05 2024-09-18T11:32:08.426 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.426 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.426 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.426 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3325 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:08.426 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:08.426 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.224439454999999, 2024-09-18T11:32:08.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181030999999999, 2024-09-18T11:32:08.427 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.427 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.427 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.427 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:08.427 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:08.427 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3325 2024-09-18T11:32:08.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.427 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:08.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:08.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116988+0000", 2024-09-18T11:32:08.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116991+0000", 2024-09-18T11:32:08.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9890000000000001e-06 2024-09-18T11:32:08.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116998+0000", 2024-09-18T11:32:08.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5309999999999998e-06 2024-09-18T11:32:08.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117005+0000", 2024-09-18T11:32:08.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6420000000000001e-06 2024-09-18T11:32:08.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117091+0000", 2024-09-18T11:32:08.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6835999999999993e-05 2024-09-18T11:32:08.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:08.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117097+0000", 2024-09-18T11:32:08.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:32:08.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124717+0000", 2024-09-18T11:32:08.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076204899999999997 2024-09-18T11:32:08.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124773+0000", 2024-09-18T11:32:08.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5470999999999998e-05 2024-09-18T11:32:08.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:08.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124877+0000", 2024-09-18T11:32:08.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010426500000000001 2024-09-18T11:32:08.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127901+0000", 2024-09-18T11:32:08.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030243449999999999 2024-09-18T11:32:08.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128115+0000", 2024-09-18T11:32:08.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021355300000000001 2024-09-18T11:32:08.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128145+0000", 2024-09-18T11:32:08.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0290999999999998e-05 2024-09-18T11:32:08.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128170+0000", 2024-09-18T11:32:08.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4811e-05 2024-09-18T11:32:08.433 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.433 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.433 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.433 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3474 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1872245~637761] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:08.433 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:08.433 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.897662778000001, 2024-09-18T11:32:08.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0092038580000000005, 2024-09-18T11:32:08.434 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.434 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.434 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.434 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:08.434 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:08.434 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3474 2024-09-18T11:32:08.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.434 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:08.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:08.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443764+0000", 2024-09-18T11:32:08.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:08.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444359+0000", 2024-09-18T11:32:08.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059519 2024-09-18T11:32:08.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444374+0000", 2024-09-18T11:32:08.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4949e-05 2024-09-18T11:32:08.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444382+0000", 2024-09-18T11:32:08.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1369999999999996e-06 2024-09-18T11:32:08.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444428+0000", 2024-09-18T11:32:08.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6783000000000003e-05 2024-09-18T11:32:08.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444507+0000", 2024-09-18T11:32:08.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8415000000000001e-05 2024-09-18T11:32:08.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:08.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444638+0000", 2024-09-18T11:32:08.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013129 2024-09-18T11:32:08.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.449059+0000", 2024-09-18T11:32:08.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004420719 2024-09-18T11:32:08.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452909+0000", 2024-09-18T11:32:08.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003850773 2024-09-18T11:32:08.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452944+0000", 2024-09-18T11:32:08.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-05 2024-09-18T11:32:08.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452969+0000", 2024-09-18T11:32:08.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5795999999999998e-05 2024-09-18T11:32:08.439 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.439 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.439 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.439 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3475 4.5 4:a56ea1fb:::smithi11939506-48:head [setxattr _header (57) in=7b,truncate 3042100] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:08.439 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:32:08.439 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.897034063, 2024-09-18T11:32:08.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086700660000000006, 2024-09-18T11:32:08.439 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.440 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.440 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.440 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:08.440 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:08.440 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3475 2024-09-18T11:32:08.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.440 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:32:08.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:32:08.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444393+0000", 2024-09-18T11:32:08.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:08.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444398+0000", 2024-09-18T11:32:08.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4909999999999999e-06 2024-09-18T11:32:08.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444405+0000", 2024-09-18T11:32:08.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5329999999999997e-06 2024-09-18T11:32:08.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444410+0000", 2024-09-18T11:32:08.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2470000000000002e-06 2024-09-18T11:32:08.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446046+0000", 2024-09-18T11:32:08.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001636722 2024-09-18T11:32:08.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446121+0000", 2024-09-18T11:32:08.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4604000000000006e-05 2024-09-18T11:32:08.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:08.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446274+0000", 2024-09-18T11:32:08.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015350300000000001 2024-09-18T11:32:08.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452884+0000", 2024-09-18T11:32:08.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066096439999999996 2024-09-18T11:32:08.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453000+0000", 2024-09-18T11:32:08.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000116204 2024-09-18T11:32:08.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453032+0000", 2024-09-18T11:32:08.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1327e-05 2024-09-18T11:32:08.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453064+0000", 2024-09-18T11:32:08.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2645000000000003e-05 2024-09-18T11:32:08.445 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.445 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.445 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.445 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3476 4.5 4:a56ea1fb:::smithi11939506-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:08.445 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:08.445 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.896994796000001, 2024-09-18T11:32:08.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087596800000000006, 2024-09-18T11:32:08.445 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.445 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:08.446 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.446 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:08.446 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:08.446 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3476 2024-09-18T11:32:08.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.446 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:08.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:08.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:08.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444436+0000", 2024-09-18T11:32:08.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9330000000000001e-06 2024-09-18T11:32:08.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444443+0000", 2024-09-18T11:32:08.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9369999999999998e-06 2024-09-18T11:32:08.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444447+0000", 2024-09-18T11:32:08.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1309999999999999e-06 2024-09-18T11:32:08.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446599+0000", 2024-09-18T11:32:08.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021521190000000001 2024-09-18T11:32:08.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:08.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446675+0000", 2024-09-18T11:32:08.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6099000000000004e-05 2024-09-18T11:32:08.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453088+0000", 2024-09-18T11:32:08.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006413307 2024-09-18T11:32:08.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453132+0000", 2024-09-18T11:32:08.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3548999999999999e-05 2024-09-18T11:32:08.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453193+0000", 2024-09-18T11:32:08.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1116000000000002e-05 2024-09-18T11:32:08.450 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.450 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.450 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.450 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3928 4.5 4:af4ba75a:::smithi11939506-37:head [sparse-read 0~1543958] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:08.450 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:08.451 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.3221978859999997, 2024-09-18T11:32:08.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080807250000000004, 2024-09-18T11:32:08.451 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.451 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:08.451 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.451 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:08.451 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:08.451 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3928 2024-09-18T11:32:08.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.451 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:08.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:08.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:08.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019232+0000", 2024-09-18T11:32:08.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1050000000000002e-06 2024-09-18T11:32:08.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019236+0000", 2024-09-18T11:32:08.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7910000000000002e-06 2024-09-18T11:32:08.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019238+0000", 2024-09-18T11:32:08.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4339999999999999e-06 2024-09-18T11:32:08.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019326+0000", 2024-09-18T11:32:08.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8097999999999996e-05 2024-09-18T11:32:08.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:08.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019330+0000", 2024-09-18T11:32:08.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9349999999999996e-06 2024-09-18T11:32:08.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027035+0000", 2024-09-18T11:32:08.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0077047269999999998 2024-09-18T11:32:08.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027110+0000", 2024-09-18T11:32:08.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4443000000000005e-05 2024-09-18T11:32:08.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027311+0000", 2024-09-18T11:32:08.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000201528 2024-09-18T11:32:08.455 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.455 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.455 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.455 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3929 4.5 4:af4ba75a:::smithi11939506-37:head [sparse-read 0~1543958] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:08.456 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:08.456 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.3221673330000003, 2024-09-18T11:32:08.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084586699999999997, 2024-09-18T11:32:08.456 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.456 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:08.456 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.456 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:08.456 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:08.456 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3929 2024-09-18T11:32:08.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.456 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:08.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:08.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:08.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019262+0000", 2024-09-18T11:32:08.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8029999999999999e-06 2024-09-18T11:32:08.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019267+0000", 2024-09-18T11:32:08.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1119999999999998e-06 2024-09-18T11:32:08.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019269+0000", 2024-09-18T11:32:08.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4269999999999998e-06 2024-09-18T11:32:08.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019344+0000", 2024-09-18T11:32:08.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4979000000000002e-05 2024-09-18T11:32:08.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:08.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019348+0000", 2024-09-18T11:32:08.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3930000000000001e-06 2024-09-18T11:32:08.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027385+0000", 2024-09-18T11:32:08.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080368409999999994 2024-09-18T11:32:08.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027463+0000", 2024-09-18T11:32:08.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7316000000000003e-05 2024-09-18T11:32:08.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027720+0000", 2024-09-18T11:32:08.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000257068 2024-09-18T11:32:08.460 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.460 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.460 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.461 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.461 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.461 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:08.462 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:32:08.463 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:08.461+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:08.561 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:08.561 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:08.561 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:08.561 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:08.563 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:08.562+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:08.569 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:32:08.664 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:08.663+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:08.706 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:08.707 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:08.707 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:08.707 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:08.707 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:08.715 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:32:08.764 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:08.763+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:08.814 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:08.815 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:08.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:08.815 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:08.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.815 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:08.815 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:32:08.815 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 139.654698776, 2024-09-18T11:32:08.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:32:08.815 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.815 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.815 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.815 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:08.815 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:08.816 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:32:08.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.816 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:32:08.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:32:08.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:32:08.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:32:08.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:32:08.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:32:08.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:32:08.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:32:08.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:32:08.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:32:08.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:32:08.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:32:08.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:32:08.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:08.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:32:08.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:32:08.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:32:08.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:32:08.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:32:08.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:32:08.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:32:08.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:32:08.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:32:08.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:32:08.821 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.821 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.821 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.821 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:08.821 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:32:08.821 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 139.65452340100001, 2024-09-18T11:32:08.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:32:08.821 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.821 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.822 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.822 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:08.822 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:08.822 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:32:08.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.822 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.822 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.822 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:32:08.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.822 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.822 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:32:08.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:32:08.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:32:08.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:32:08.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:32:08.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:32:08.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:32:08.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:32:08.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:32:08.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:32:08.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:32:08.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:32:08.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:08.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:32:08.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:32:08.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:32:08.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:32:08.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:32:08.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:32:08.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:32:08.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:32:08.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:32:08.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:32:08.827 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.827 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.827 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.827 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:08.827 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:08.827 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 139.65448086999999, 2024-09-18T11:32:08.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:32:08.827 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.827 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.827 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.828 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:08.828 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:08.828 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:32:08.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.828 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:08.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:08.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:32:08.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:32:08.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:32:08.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:32:08.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:32:08.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:32:08.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:32:08.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:32:08.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:32:08.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:32:08.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:32:08.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:08.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:32:08.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:32:08.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:32:08.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:32:08.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:32:08.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:32:08.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:32:08.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:32:08.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:32:08.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:32:08.833 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.833 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.833 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.833 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:08.833 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:08.833 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 139.65442594000001, 2024-09-18T11:32:08.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:32:08.834 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.834 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.834 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.834 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:08.834 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:08.834 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:32:08.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.834 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:08.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:08.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:08.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:32:08.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:32:08.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:32:08.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:32:08.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:32:08.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:32:08.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:32:08.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:32:08.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:32:08.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:32:08.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:08.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:32:08.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:32:08.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:32:08.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:32:08.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:32:08.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:32:08.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:32:08.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:32:08.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:32:08.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:32:08.839 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.839 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.839 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.839 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:08.839 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:08.839 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 139.65441595999999, 2024-09-18T11:32:08.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:32:08.839 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.840 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.840 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.840 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:08.840 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:08.840 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:32:08.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.840 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:08.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:08.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:08.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:08.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:32:08.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:32:08.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:32:08.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:32:08.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:32:08.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:32:08.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:32:08.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:32:08.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:32:08.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:32:08.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:08.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:32:08.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:32:08.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:32:08.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:32:08.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:32:08.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:32:08.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:32:08.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:32:08.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:32:08.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:32:08.845 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.845 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.845 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.845 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:08.845 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:08.845 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 139.65440436099999, 2024-09-18T11:32:08.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:32:08.845 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.846 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.846 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.846 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:08.846 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:08.846 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:32:08.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.846 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:08.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:08.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:08.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:08.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:32:08.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:32:08.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:32:08.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:32:08.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:32:08.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:32:08.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:32:08.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:32:08.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:32:08.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:32:08.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:08.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:32:08.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:32:08.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:32:08.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:32:08.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:32:08.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:32:08.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:32:08.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:32:08.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:32:08.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:32:08.851 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.851 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.851 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.851 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:08.851 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:08.851 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 139.65434787999999, 2024-09-18T11:32:08.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:32:08.851 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.852 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.852 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.852 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:08.852 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:08.852 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:32:08.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.852 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:08.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:32:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:32:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:32:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:32:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:32:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:32:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:32:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:32:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:32:08.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:32:08.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:32:08.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:08.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:32:08.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:32:08.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:32:08.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:32:08.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:32:08.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:32:08.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:32:08.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:32:08.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:32:08.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:32:08.857 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.857 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.857 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.857 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:08.857 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:08.857 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 139.65433405600001, 2024-09-18T11:32:08.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:32:08.857 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.857 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.857 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.858 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:08.858 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:08.858 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:32:08.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.858 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:08.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:08.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:08.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:08.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:32:08.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:32:08.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:32:08.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:32:08.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:32:08.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:32:08.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:32:08.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:32:08.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:32:08.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:32:08.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:08.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:32:08.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:32:08.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:32:08.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:32:08.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:32:08.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:32:08.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:32:08.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:32:08.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:32:08.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:32:08.863 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.863 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.863 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.863 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:08.863 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:08.863 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 139.23815514899999, 2024-09-18T11:32:08.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:32:08.863 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.863 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.864 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.864 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:08.864 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:08.864 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:32:08.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.864 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:08.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:08.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:08.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:08.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:32:08.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:32:08.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:32:08.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:32:08.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:32:08.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:32:08.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:32:08.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:32:08.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:32:08.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:32:08.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:08.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:32:08.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:32:08.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:32:08.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:32:08.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:32:08.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:32:08.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:32:08.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:32:08.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:32:08.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:32:08.869 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.869 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.869 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.869 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:08.869 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:08.869 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 139.23813184900001, 2024-09-18T11:32:08.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:32:08.870 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.870 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.870 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.870 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:08.870 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:08.870 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:32:08.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.870 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:08.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:08.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:32:08.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:32:08.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:32:08.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:32:08.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:32:08.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:32:08.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:32:08.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:32:08.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:32:08.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:32:08.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:32:08.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:08.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:32:08.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:32:08.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:32:08.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:32:08.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:32:08.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:32:08.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:32:08.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:32:08.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:32:08.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:32:08.878 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.878 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.878 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.878 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:08.878 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:08.878 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 139.23804623999999, 2024-09-18T11:32:08.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:32:08.878 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.879 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.879 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.879 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:08.879 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:08.879 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:32:08.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.879 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:08.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:08.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:08.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:32:08.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:32:08.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:32:08.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:32:08.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:32:08.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:32:08.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:32:08.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:32:08.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:32:08.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:32:08.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:08.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:32:08.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:32:08.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:32:08.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:32:08.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:32:08.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:32:08.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:32:08.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:32:08.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:32:08.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:32:08.884 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.884 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.884 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.884 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:08.884 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:08.884 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 139.23803527499999, 2024-09-18T11:32:08.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:32:08.884 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.884 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.885 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.885 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:08.885 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:08.885 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:32:08.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.885 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:08.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:08.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:08.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:08.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:32:08.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:32:08.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:32:08.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:32:08.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:32:08.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:32:08.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:32:08.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:32:08.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:32:08.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:32:08.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:08.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:32:08.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:32:08.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:32:08.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:32:08.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:32:08.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:32:08.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:32:08.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:32:08.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:32:08.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:32:08.890 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.890 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.890 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.890 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:08.890 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:08.890 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 139.238024107, 2024-09-18T11:32:08.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:32:08.890 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.890 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.890 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.891 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:08.891 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:08.891 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:32:08.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.891 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:08.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:08.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:08.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:08.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:32:08.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:32:08.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:32:08.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:32:08.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:32:08.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:32:08.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:32:08.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:32:08.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:32:08.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:32:08.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:08.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:32:08.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:32:08.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:32:08.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:32:08.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:32:08.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:32:08.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:32:08.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:32:08.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:32:08.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:32:08.896 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.896 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.896 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.896 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:08.896 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:08.896 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 139.23795877800001, 2024-09-18T11:32:08.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:32:08.896 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.896 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.896 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.896 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:08.897 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:08.897 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:32:08.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.897 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:08.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:08.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:32:08.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:32:08.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:32:08.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:32:08.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:32:08.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:32:08.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:32:08.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:32:08.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:32:08.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:32:08.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:32:08.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:08.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:32:08.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:32:08.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:32:08.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:32:08.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:32:08.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:32:08.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:32:08.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:32:08.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:32:08.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:32:08.901 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.901 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.901 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.902 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:08.902 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:08.902 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 139.22779679999999, 2024-09-18T11:32:08.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:32:08.902 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.902 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.902 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.902 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:08.902 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:08.902 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:32:08.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.902 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:08.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:08.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:32:08.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:32:08.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:32:08.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:32:08.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:32:08.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:32:08.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:32:08.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:32:08.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:32:08.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:32:08.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:32:08.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:08.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:32:08.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:32:08.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:32:08.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:32:08.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:32:08.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:32:08.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:32:08.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:32:08.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:32:08.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:32:08.907 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.907 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.907 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.907 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:08.907 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:08.907 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 139.227704321, 2024-09-18T11:32:08.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:32:08.907 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.907 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.908 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:08.908 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:08.908 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:08.908 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:32:08.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.908 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:08.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:08.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:08.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:32:08.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:32:08.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:32:08.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:32:08.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:32:08.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:32:08.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:32:08.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:32:08.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:32:08.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:32:08.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:08.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:32:08.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:32:08.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:08.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:32:08.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:32:08.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:08.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:32:08.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:32:08.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:32:08.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:32:08.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:32:08.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:32:08.913 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.913 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.913 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.913 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1788 4.8 e60/22 4:1898b15d:::smithi11939506-25:head v 60'12, mlcod=58'9)", 2024-09-18T11:32:08.913 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:32:08.913 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.895298831999995, 2024-09-18T11:32:08.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058348599999999999, 2024-09-18T11:32:08.913 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.913 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.913 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:32:08.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:32:08.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916949+0000", 2024-09-18T11:32:08.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:08.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917602+0000", 2024-09-18T11:32:08.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00065337100000000003 2024-09-18T11:32:08.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917621+0000", 2024-09-18T11:32:08.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9287e-05 2024-09-18T11:32:08.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917629+0000", 2024-09-18T11:32:08.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3780000000000003e-06 2024-09-18T11:32:08.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917677+0000", 2024-09-18T11:32:08.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8068000000000001e-05 2024-09-18T11:32:08.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917699+0000", 2024-09-18T11:32:08.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2149000000000001e-05 2024-09-18T11:32:08.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922747+0000", 2024-09-18T11:32:08.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050477609999999996 2024-09-18T11:32:08.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922785+0000", 2024-09-18T11:32:08.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8726000000000002e-05 2024-09-18T11:32:08.917 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.917 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.917 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.917 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3209 4.0 e109/22 4:019b6e09:::smithi11939506-40:head v 109'149, mlcod=106'146)", 2024-09-18T11:32:08.917 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:32:08.917 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.712863469999999, 2024-09-18T11:32:08.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063064039999999998, 2024-09-18T11:32:08.918 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.918 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.918 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:32:08.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:32:08.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099385+0000", 2024-09-18T11:32:08.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099951+0000", 2024-09-18T11:32:08.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056606600000000005 2024-09-18T11:32:08.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099964+0000", 2024-09-18T11:32:08.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2918999999999999e-05 2024-09-18T11:32:08.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099972+0000", 2024-09-18T11:32:08.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2300000000000002e-06 2024-09-18T11:32:08.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.100026+0000", 2024-09-18T11:32:08.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4456999999999997e-05 2024-09-18T11:32:08.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.100046+0000", 2024-09-18T11:32:08.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0417000000000001e-05 2024-09-18T11:32:08.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.105653+0000", 2024-09-18T11:32:08.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005606214 2024-09-18T11:32:08.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.105692+0000", 2024-09-18T11:32:08.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9671e-05 2024-09-18T11:32:08.921 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.921 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.921 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.922 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:123 4.3 e132/67 4:cc46532a:::smithi11939506-33:30 v 132'171, mlcod=128'168)", 2024-09-18T11:32:08.922 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:08.922 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.7843976960000001, 2024-09-18T11:32:08.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087627039999999996, 2024-09-18T11:32:08.922 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.922 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.922 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:08.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:08.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027851+0000", 2024-09-18T11:32:08.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:08.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027871+0000", 2024-09-18T11:32:08.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9749e-05 2024-09-18T11:32:08.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027882+0000", 2024-09-18T11:32:08.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1242e-05 2024-09-18T11:32:08.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027889+0000", 2024-09-18T11:32:08.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7649999999999996e-06 2024-09-18T11:32:08.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027929+0000", 2024-09-18T11:32:08.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9984999999999997e-05 2024-09-18T11:32:08.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027951+0000", 2024-09-18T11:32:08.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2047000000000001e-05 2024-09-18T11:32:08.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.036565+0000", 2024-09-18T11:32:08.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086135110000000008 2024-09-18T11:32:08.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.036614+0000", 2024-09-18T11:32:08.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9747000000000003e-05 2024-09-18T11:32:08.926 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.926 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.926 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.926 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:124 4.3 e132/67 4:ca2b2309:::smithi11939506-44:30 v 132'173, mlcod=128'168)", 2024-09-18T11:32:08.926 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:08.926 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.7837057139999999, 2024-09-18T11:32:08.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0088947309999999995, 2024-09-18T11:32:08.926 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:08.926 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:08.926 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:08.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:08.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:08.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:08.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:08.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:08.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:08.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028543+0000", 2024-09-18T11:32:08.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:08.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:08.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028551+0000", 2024-09-18T11:32:08.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8720000000000002e-06 2024-09-18T11:32:08.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:08.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028562+0000", 2024-09-18T11:32:08.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1039000000000001e-05 2024-09-18T11:32:08.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:08.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028569+0000", 2024-09-18T11:32:08.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0350000000000004e-06 2024-09-18T11:32:08.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:08.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028848+0000", 2024-09-18T11:32:08.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00027935600000000003 2024-09-18T11:32:08.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:08.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028868+0000", 2024-09-18T11:32:08.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9947999999999999e-05 2024-09-18T11:32:08.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:08.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037405+0000", 2024-09-18T11:32:08.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085369920000000002 2024-09-18T11:32:08.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:08.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:08.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:08.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037438+0000", 2024-09-18T11:32:08.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3244000000000002e-05 2024-09-18T11:32:08.930 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.930 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.930 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.930 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:08.930 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:08.930 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:08.932 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:32:08.932 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:08.930+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:09.033 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:09.032+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:09.036 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:09.036 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:09.036 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:09.036 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:09.045 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:32:09.134 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:09.133+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:09.183 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:09.184 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:09.184 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:09.184 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:09.184 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:09.192 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:32:09.235 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:09.234+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:09.271 INFO:tasks.rados.rados.0.smithi119.stdout:1722: expect (ObjNum 260 snap 31 seq_num 260) 2024-09-18T11:32:09.276 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T11:32:09.294 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:09.294 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [] 2024-09-18T11:32:09.294 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:09.303 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:32:09.344 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:09.343+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:09.376 INFO:tasks.rados.rados.0.smithi119.stdout:1721: got (0) Success 2024-09-18T11:32:09.376 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 0 (ObjNum 246 snap 28 seq_num 246) dirty exists 2024-09-18T11:32:09.376 INFO:tasks.rados.rados.0.smithi119.stdout:1721: done (3 left) 2024-09-18T11:32:09.376 INFO:tasks.rados.rados.0.smithi119.stdout:1722: done (2 left) 2024-09-18T11:32:09.376 INFO:tasks.rados.rados.0.smithi119.stdout:1725: done (1 left) 2024-09-18T11:32:09.376 INFO:tasks.rados.rados.0.smithi119.stdout:1726: tier_promote oid 35 2024-09-18T11:32:09.377 INFO:tasks.rados.rados.0.smithi119.stdout:1724: expect (ObjNum 347 snap 51 seq_num 347) 2024-09-18T11:32:09.437 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:09.436+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:09.439 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:32:09.445 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:09.445 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:09.445 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:09.445 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:09.454 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:32:09.538 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:09.537+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:09.553 INFO:tasks.rados.rados.0.smithi119.stdout:1726: got (0) Success 2024-09-18T11:32:09.553 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 169 (ObjNum 333 snap 48 seq_num 333) dirty exists 2024-09-18T11:32:09.553 INFO:tasks.rados.rados.0.smithi119.stdout:1724: done (1 left) 2024-09-18T11:32:09.553 INFO:tasks.rados.rados.0.smithi119.stdout:1726: done (0 left) 2024-09-18T11:32:09.553 INFO:tasks.rados.rados.0.smithi119.stdout:1727: write oid 35 current snap is 54 2024-09-18T11:32:09.553 INFO:tasks.rados.rados.0.smithi119.stdout:1727: seq_num 348 ranges {648836=622537,1981263=493111,3107319=741517} 2024-09-18T11:32:09.559 INFO:tasks.rados.rados.0.smithi119.stdout:1727: writing smithi11939506-35 from 648836 to 1271373 tid 1 2024-09-18T11:32:09.566 INFO:tasks.rados.rados.0.smithi119.stdout:1727: writing smithi11939506-35 from 1981263 to 2474374 tid 2 2024-09-18T11:32:09.570 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:09.570 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:09.574 INFO:tasks.rados.rados.0.smithi119.stdout:1727: writing smithi11939506-35 from 3107319 to 3848836 tid 3 2024-09-18T11:32:09.574 INFO:tasks.rados.rados.0.smithi119.stdout:1728: tier_flush oid 3 2024-09-18T11:32:09.574 INFO:tasks.rados.rados.0.smithi119.stdout:1728: tier_flush oid 3 snap -1 2024-09-18T11:32:09.574 INFO:tasks.rados.rados.0.smithi119.stdout:1727: finishing write tid 1 to smithi11939506-35 2024-09-18T11:32:09.574 INFO:tasks.rados.rados.0.smithi119.stdout:1727: finishing write tid 2 to smithi11939506-35 2024-09-18T11:32:09.575 INFO:tasks.rados.rados.0.smithi119.stdout:1729: read oid 7 snap -1 2024-09-18T11:32:09.575 INFO:tasks.rados.rados.0.smithi119.stdout:1729: expect (ObjNum 145 snap 18 seq_num 145) 2024-09-18T11:32:09.575 INFO:tasks.rados.rados.0.smithi119.stdout:1730: tier_promote oid 1 2024-09-18T11:32:09.575 INFO:tasks.rados.rados.0.smithi119.stdout:1731: write (excl) oid 20 current snap is 54 2024-09-18T11:32:09.575 INFO:tasks.rados.rados.0.smithi119.stdout:1731: seq_num 349 ranges {665643=697786,2039745=753122,3534742=330901} 2024-09-18T11:32:09.581 INFO:tasks.rados.rados.0.smithi119.stdout:1731: writing smithi11939506-20 from 665643 to 1363429 tid 1 2024-09-18T11:32:09.591 INFO:tasks.rados.rados.0.smithi119.stdout:1731: writing smithi11939506-20 from 2039745 to 2792867 tid 2 2024-09-18T11:32:09.597 INFO:tasks.rados.rados.0.smithi119.stdout:1731: writing smithi11939506-20 from 3534742 to 3865643 tid 3 2024-09-18T11:32:09.597 INFO:tasks.rados.rados.0.smithi119.stdout:1732: write (excl) oid 9 current snap is 54 2024-09-18T11:32:09.597 INFO:tasks.rados.rados.0.smithi119.stdout:1727: finishing write tid 3 to smithi11939506-35 2024-09-18T11:32:09.597 INFO:tasks.rados.rados.0.smithi119.stdout:1732: seq_num 350 ranges {682450=489388,1814580=496780,2878518=788028} 2024-09-18T11:32:09.600 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:09.601 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:09.603 INFO:tasks.rados.rados.0.smithi119.stdout:1732: writing smithi11939506-9 from 682450 to 1171838 tid 1 2024-09-18T11:32:09.611 INFO:tasks.rados.rados.0.smithi119.stdout:1732: writing smithi11939506-9 from 1814580 to 2311360 tid 2 2024-09-18T11:32:09.616 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:09.616 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:09.620 INFO:tasks.rados.rados.0.smithi119.stdout:1732: writing smithi11939506-9 from 2878518 to 3666546 tid 3 2024-09-18T11:32:09.620 INFO:tasks.rados.rados.0.smithi119.stdout:1733: write oid 17 current snap is 54 2024-09-18T11:32:09.620 INFO:tasks.rados.rados.0.smithi119.stdout:1733: seq_num 351 ranges {699257=564637,1873062=640438,3305941=593316} 2024-09-18T11:32:09.628 INFO:tasks.rados.rados.0.smithi119.stdout:1733: writing smithi11939506-17 from 699257 to 1263894 tid 1 2024-09-18T11:32:09.628 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:09.629 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:09.633 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:09.633 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:09.636 INFO:tasks.rados.rados.0.smithi119.stdout:1733: writing smithi11939506-17 from 1873062 to 2513500 tid 2 2024-09-18T11:32:09.638 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:09.637+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:09.641 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:32:09.644 INFO:tasks.rados.rados.0.smithi119.stdout:1733: writing smithi11939506-17 from 3305941 to 3899257 tid 3 2024-09-18T11:32:09.644 INFO:tasks.rados.rados.0.smithi119.stdout:1734: set_chunk oid 38 target oid 2024-09-18T11:32:09.649 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:09.649 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:09.660 INFO:tasks.rados.rados.0.smithi119.stdout:1734: set_chunk oid 38 offset: 1048576 length: 1709568 target oid 22b064a083114a2756d85a07d2762a393d3e92dd898a5334bc045c9031244d76 offset: 0 snap -1 2024-09-18T11:32:09.660 INFO:tasks.rados.rados.0.smithi119.stdout:1735: read oid 11 snap 49 2024-09-18T11:32:09.660 INFO:tasks.rados.rados.0.smithi119.stdout:1735: expect (ObjNum 249 snap 30 seq_num 249) 2024-09-18T11:32:09.660 INFO:tasks.rados.rados.0.smithi119.stdout:1736: read oid 13 snap -1 2024-09-18T11:32:09.660 INFO:tasks.rados.rados.0.smithi119.stdout:1736: expect (ObjNum 309 snap 43 seq_num 309) 2024-09-18T11:32:09.660 INFO:tasks.rados.rados.0.smithi119.stdout:1727: finishing write tid 4 to smithi11939506-35 2024-09-18T11:32:09.661 INFO:tasks.rados.rados.0.smithi119.stdout:1727: finishing write tid 5 to smithi11939506-35 2024-09-18T11:32:09.661 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 185 (ObjNum 348 snap 54 seq_num 348) dirty exists 2024-09-18T11:32:09.661 INFO:tasks.rados.rados.0.smithi119.stdout:1727: left oid 35 (ObjNum 348 snap 54 seq_num 348) 2024-09-18T11:32:09.661 INFO:tasks.rados.rados.0.smithi119.stdout:1729: expect (ObjNum 145 snap 18 seq_num 145) 2024-09-18T11:32:09.669 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:09.669 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:09.695 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:09.695 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:09.711 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:09.711 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:09.748 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:09.748 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:09.748 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:09.747+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:09.757 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:32:09.820 INFO:tasks.rados.rados.0.smithi119.stdout:1730: got (0) Success 2024-09-18T11:32:09.821 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 208 (ObjNum 277 snap 37 seq_num 277) dirty exists 2024-09-18T11:32:09.821 INFO:tasks.rados.rados.0.smithi119.stdout:1728: got (0) Success 2024-09-18T11:32:09.821 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 0 (ObjNum 243 snap 28 seq_num 243) dirty exists 2024-09-18T11:32:09.821 INFO:tasks.rados.rados.0.smithi119.stdout:1731: finishing write tid 1 to smithi11939506-20 2024-09-18T11:32:09.821 INFO:tasks.rados.rados.0.smithi119.stdout:1731: finishing write tid 2 to smithi11939506-20 2024-09-18T11:32:09.821 INFO:tasks.rados.rados.0.smithi119.stdout:1737: read oid 12 snap -1 2024-09-18T11:32:09.821 INFO:tasks.rados.rados.0.smithi119.stdout:1737: expect (ObjNum 283 snap 39 seq_num 283) 2024-09-18T11:32:09.821 INFO:tasks.rados.rados.0.smithi119.stdout:1727: done (10 left) 2024-09-18T11:32:09.821 INFO:tasks.rados.rados.0.smithi119.stdout:1728: done (9 left) 2024-09-18T11:32:09.821 INFO:tasks.rados.rados.0.smithi119.stdout:1729: done (8 left) 2024-09-18T11:32:09.821 INFO:tasks.rados.rados.0.smithi119.stdout:1730: done (7 left) 2024-09-18T11:32:09.821 INFO:tasks.rados.rados.0.smithi119.stdout:1738: tier_promote oid 16 2024-09-18T11:32:09.822 INFO:tasks.rados.rados.0.smithi119.stdout:1731: finishing write tid 3 to smithi11939506-20 2024-09-18T11:32:09.822 INFO:tasks.rados.rados.0.smithi119.stdout:1731: finishing write tid 4 to smithi11939506-20 2024-09-18T11:32:09.822 INFO:tasks.rados.rados.0.smithi119.stdout:1731: finishing write tid 5 to smithi11939506-20 2024-09-18T11:32:09.822 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 20 v 273 (ObjNum 349 snap 54 seq_num 349) dirty exists 2024-09-18T11:32:09.822 INFO:tasks.rados.rados.0.smithi119.stdout:1731: left oid 20 (ObjNum 349 snap 54 seq_num 349) 2024-09-18T11:32:09.822 INFO:tasks.rados.rados.0.smithi119.stdout:1732: finishing write tid 1 to smithi11939506-9 2024-09-18T11:32:09.822 INFO:tasks.rados.rados.0.smithi119.stdout:1731: done (7 left) 2024-09-18T11:32:09.822 INFO:tasks.rados.rados.0.smithi119.stdout:1739: write oid 14 current snap is 54 2024-09-18T11:32:09.822 INFO:tasks.rados.rados.0.smithi119.stdout:1732: finishing write tid 2 to smithi11939506-9 2024-09-18T11:32:09.822 INFO:tasks.rados.rados.0.smithi119.stdout:1732: finishing write tid 3 to smithi11939506-9 2024-09-18T11:32:09.822 INFO:tasks.rados.rados.0.smithi119.stdout:1732: finishing write tid 4 to smithi11939506-9 2024-09-18T11:32:09.822 INFO:tasks.rados.rados.0.smithi119.stdout:1739: seq_num 352 ranges {716064=639886,1815191=500449,2817011=526925} 2024-09-18T11:32:09.828 INFO:tasks.rados.rados.0.smithi119.stdout:1739: writing smithi11939506-14 from 716064 to 1355950 tid 1 2024-09-18T11:32:09.834 INFO:tasks.rados.rados.0.smithi119.stdout:1739: writing smithi11939506-14 from 1815191 to 2315640 tid 2 2024-09-18T11:32:09.840 INFO:tasks.rados.rados.0.smithi119.stdout:1739: writing smithi11939506-14 from 2817011 to 3343936 tid 3 2024-09-18T11:32:09.840 INFO:tasks.rados.rados.0.smithi119.stdout:1740: read oid 50 snap 33 2024-09-18T11:32:09.840 INFO:tasks.rados.rados.0.smithi119.stdout:1740: expect (ObjNum 183 snap 20 seq_num 183) 2024-09-18T11:32:09.840 INFO:tasks.rados.rados.0.smithi119.stdout:1741: delete oid 36 current snap is 54 2024-09-18T11:32:09.841 INFO:tasks.rados.rados.0.smithi119.stdout:1732: finishing write tid 5 to smithi11939506-9 2024-09-18T11:32:09.841 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 446 (ObjNum 350 snap 54 seq_num 350) dirty exists 2024-09-18T11:32:09.841 INFO:tasks.rados.rados.0.smithi119.stdout:1732: left oid 9 (ObjNum 350 snap 54 seq_num 350) 2024-09-18T11:32:09.841 INFO:tasks.rados.rados.0.smithi119.stdout:1733: finishing write tid 1 to smithi11939506-17 2024-09-18T11:32:09.841 INFO:tasks.rados.rados.0.smithi119.stdout:1733: finishing write tid 2 to smithi11939506-17 2024-09-18T11:32:09.841 INFO:tasks.rados.rados.0.smithi119.stdout:1733: finishing write tid 3 to smithi11939506-17 2024-09-18T11:32:09.841 INFO:tasks.rados.rados.0.smithi119.stdout:1733: finishing write tid 4 to smithi11939506-17 2024-09-18T11:32:09.841 INFO:tasks.rados.rados.0.smithi119.stdout:1733: finishing write tid 5 to smithi11939506-17 2024-09-18T11:32:09.841 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 451 (ObjNum 351 snap 54 seq_num 351) dirty exists 2024-09-18T11:32:09.841 INFO:tasks.rados.rados.0.smithi119.stdout:1733: left oid 17 (ObjNum 351 snap 54 seq_num 351) 2024-09-18T11:32:09.841 INFO:tasks.rados.rados.0.smithi119.stdout:1736: expect (ObjNum 309 snap 43 seq_num 309) 2024-09-18T11:32:09.842 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:09.839+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:09.899 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:09.899 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:09.907 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:32:09.941 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:09.940+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:10.042 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:10.040+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:10.046 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:10.046 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:10.048 INFO:tasks.rados.rados.0.smithi119.stdout:1734: finishing set_chunk to oid 38 2024-09-18T11:32:10.048 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 182 (ObjNum 306 snap 43 seq_num 306) dirty exists 2024-09-18T11:32:10.049 INFO:tasks.rados.rados.0.smithi119.stdout:1735: expect (ObjNum 249 snap 30 seq_num 249) 2024-09-18T11:32:10.055 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:32:10.129 INFO:tasks.rados.rados.0.smithi119.stdout:1738: got (0) Success 2024-09-18T11:32:10.129 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 279 (ObjNum 286 snap 41 seq_num 286) dirty exists 2024-09-18T11:32:10.129 INFO:tasks.rados.rados.0.smithi119.stdout:1737: expect (ObjNum 283 snap 39 seq_num 283) 2024-09-18T11:32:10.143 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:10.141+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:10.154 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:10.154 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:10.163 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:32:10.243 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:10.242+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:10.262 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:10.262 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:10.271 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:32:10.278 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:10.277+0000 7f697ceb1640 -1 osd.3 138 handle_osd_map injecting map crc failure 2024-09-18T11:32:10.282 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is set 2024-09-18T11:32:10.339 INFO:tasks.rados.rados.0.smithi119.stdout:1739: finishing write tid 1 to smithi11939506-14 2024-09-18T11:32:10.339 INFO:tasks.rados.rados.0.smithi119.stdout:1739: finishing write tid 2 to smithi11939506-14 2024-09-18T11:32:10.339 INFO:tasks.rados.rados.0.smithi119.stdout:1732: done (9 left) 2024-09-18T11:32:10.339 INFO:tasks.rados.rados.0.smithi119.stdout:1733: done (8 left) 2024-09-18T11:32:10.339 INFO:tasks.rados.rados.0.smithi119.stdout:1734: done (7 left) 2024-09-18T11:32:10.339 INFO:tasks.rados.rados.0.smithi119.stdout:1735: done (6 left) 2024-09-18T11:32:10.339 INFO:tasks.rados.rados.0.smithi119.stdout:1736: done (5 left) 2024-09-18T11:32:10.339 INFO:tasks.rados.rados.0.smithi119.stdout:1737: done (4 left) 2024-09-18T11:32:10.340 INFO:tasks.rados.rados.0.smithi119.stdout:1738: done (3 left) 2024-09-18T11:32:10.340 INFO:tasks.rados.rados.0.smithi119.stdout:1741: done (2 left) 2024-09-18T11:32:10.340 INFO:tasks.rados.rados.0.smithi119.stdout:1742: write (excl) oid 38 current snap is 54 2024-09-18T11:32:10.340 INFO:tasks.rados.rados.0.smithi119.stdout:1740: expect (ObjNum 183 snap 20 seq_num 183) 2024-09-18T11:32:10.344 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:10.343+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:10.375 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:10.375 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:10.384 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:32:10.468 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:10.467+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:10.523 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:10.524 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:10.532 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:32:10.552 INFO:tasks.rados.rados.0.smithi119.stdout:1739: finishing write tid 3 to smithi11939506-14 2024-09-18T11:32:10.552 INFO:tasks.rados.rados.0.smithi119.stdout:1739: finishing write tid 4 to smithi11939506-14 2024-09-18T11:32:10.552 INFO:tasks.rados.rados.0.smithi119.stdout:1742: seq_num 354 ranges {749678=790384,2332155=504118,3271857=549469} 2024-09-18T11:32:10.552 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:10.551+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:10.560 INFO:tasks.rados.rados.0.smithi119.stdout:1742: writing smithi11939506-38 from 749678 to 1540062 tid 1 2024-09-18T11:32:10.567 INFO:tasks.rados.rados.0.smithi119.stdout:1742: writing smithi11939506-38 from 2332155 to 2836273 tid 2 2024-09-18T11:32:10.574 INFO:tasks.rados.rados.0.smithi119.stdout:1742: writing smithi11939506-38 from 3271857 to 3821326 tid 3 2024-09-18T11:32:10.574 INFO:tasks.rados.rados.0.smithi119.stdout:1740: done (2 left) 2024-09-18T11:32:10.574 INFO:tasks.rados.rados.0.smithi119.stdout:1743: tier_promote oid 19 2024-09-18T11:32:10.574 INFO:tasks.rados.rados.0.smithi119.stdout:1739: finishing write tid 5 to smithi11939506-14 2024-09-18T11:32:10.574 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 281 (ObjNum 352 snap 54 seq_num 352) dirty exists 2024-09-18T11:32:10.574 INFO:tasks.rados.rados.0.smithi119.stdout:1739: left oid 14 (ObjNum 352 snap 54 seq_num 352) 2024-09-18T11:32:10.574 INFO:tasks.rados.rados.0.smithi119.stdout:1739: done (2 left) 2024-09-18T11:32:10.574 INFO:tasks.rados.rados.0.smithi119.stdout:1744: read oid 30 snap -1 2024-09-18T11:32:10.574 INFO:tasks.rados.rados.0.smithi119.stdout:1744: expect deleted 2024-09-18T11:32:10.574 INFO:tasks.rados.rados.0.smithi119.stdout:1745: tier_flush oid 40 2024-09-18T11:32:10.574 INFO:tasks.rados.rados.0.smithi119.stdout:1745: tier_flush oid 40 snap -1 2024-09-18T11:32:10.575 INFO:tasks.rados.rados.0.smithi119.stdout:1742: finishing write tid 1 to smithi11939506-38 2024-09-18T11:32:10.575 INFO:tasks.rados.rados.0.smithi119.stdout:1742: finishing write tid 2 to smithi11939506-38 2024-09-18T11:32:10.575 INFO:tasks.rados.rados.0.smithi119.stdout:1746: read oid 48 snap -1 2024-09-18T11:32:10.575 INFO:tasks.rados.rados.0.smithi119.stdout:1746: expect (ObjNum 300 snap 43 seq_num 300) 2024-09-18T11:32:10.575 INFO:tasks.rados.rados.0.smithi119.stdout:1747: read oid 13 snap -1 2024-09-18T11:32:10.575 INFO:tasks.rados.rados.0.smithi119.stdout:1747: expect (ObjNum 309 snap 43 seq_num 309) 2024-09-18T11:32:10.575 INFO:tasks.rados.rados.0.smithi119.stdout:1748: snap_create 2024-09-18T11:32:10.575 INFO:tasks.rados.rados.0.smithi119.stdout:1745: got (16) Device or resource busy 2024-09-18T11:32:10.575 INFO:tasks.rados.rados.0.smithi119.stdout:check_oldest_snap_flushed oid 40 v 163 (ObjNum 287 snap 41 seq_num 287) unflushed 42 2024-09-18T11:32:10.575 INFO:tasks.rados.rados.0.smithi119.stdout:1743: got (0) Success 2024-09-18T11:32:10.576 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 227 (ObjNum 288 snap 42 seq_num 288) dirty exists 2024-09-18T11:32:10.577 INFO:tasks.rados.rados.0.smithi119.stdout:1747: expect (ObjNum 309 snap 43 seq_num 309) 2024-09-18T11:32:10.589 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:10.588+0000 7fe409e4c640 -1 osd.0 138 handle_osd_map injecting map crc failure 2024-09-18T11:32:10.636 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:10.636 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:10.644 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:32:10.647 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:10.645+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:10.742 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:10.742 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:10.746 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:10.745+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:10.751 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:32:10.777 INFO:tasks.rados.rados.0.smithi119.stdout:1742: finishing write tid 3 to smithi11939506-38 2024-09-18T11:32:10.777 INFO:tasks.rados.rados.0.smithi119.stdout:1742: finishing write tid 4 to smithi11939506-38 2024-09-18T11:32:10.777 INFO:tasks.rados.rados.0.smithi119.stdout:1742: finishing write tid 5 to smithi11939506-38 2024-09-18T11:32:10.777 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 228 (ObjNum 354 snap 54 seq_num 354) dirty exists 2024-09-18T11:32:10.777 INFO:tasks.rados.rados.0.smithi119.stdout:1742: left oid 38 (ObjNum 354 snap 54 seq_num 354) 2024-09-18T11:32:10.777 INFO:tasks.rados.rados.0.smithi119.stdout:1746: expect (ObjNum 300 snap 43 seq_num 300) 2024-09-18T11:32:10.847 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:10.846+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:10.889 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:10.890 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:10.898 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:32:10.982 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:10.981+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:11.036 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:11.037 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:11.045 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:32:11.049 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:11.047+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:11.150 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:11.148+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:11.183 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:11.183 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:11.191 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:32:11.251 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:11.249+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:11.287 INFO:tasks.rados.rados.0.smithi119.stdout:1742: done (6 left) 2024-09-18T11:32:11.287 INFO:tasks.rados.rados.0.smithi119.stdout:1743: done (5 left) 2024-09-18T11:32:11.287 INFO:tasks.rados.rados.0.smithi119.stdout:1744: done (4 left) 2024-09-18T11:32:11.287 INFO:tasks.rados.rados.0.smithi119.stdout:1745: done (3 left) 2024-09-18T11:32:11.287 INFO:tasks.rados.rados.0.smithi119.stdout:1746: done (2 left) 2024-09-18T11:32:11.287 INFO:tasks.rados.rados.0.smithi119.stdout:1747: done (1 left) 2024-09-18T11:32:11.287 INFO:tasks.rados.rados.0.smithi119.stdout:1748: done (0 left) 2024-09-18T11:32:11.287 INFO:tasks.rados.rados.0.smithi119.stdout:1749: rollback oid 33 current snap is 55 2024-09-18T11:32:11.287 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 33 to 47 2024-09-18T11:32:11.287 INFO:tasks.rados.rados.0.smithi119.stdout:1750: read oid 43 snap -1 2024-09-18T11:32:11.287 INFO:tasks.rados.rados.0.smithi119.stdout:1750: expect (ObjNum 346 snap 51 seq_num 346) 2024-09-18T11:32:11.287 INFO:tasks.rados.rados.0.smithi119.stdout:1751: read oid 12 snap -1 2024-09-18T11:32:11.288 INFO:tasks.rados.rados.0.smithi119.stdout:1751: expect (ObjNum 283 snap 39 seq_num 283) 2024-09-18T11:32:11.288 INFO:tasks.rados.rados.0.smithi119.stdout:1752: write (excl) oid 29 current snap is 55 2024-09-18T11:32:11.288 INFO:tasks.rados.rados.0.smithi119.stdout:1752: seq_num 355 ranges {766485=465633,1990637=647776,3299280=667205} 2024-09-18T11:32:11.291 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:11.292 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:11.293 INFO:tasks.rados.rados.0.smithi119.stdout:1752: writing smithi11939506-29 from 766485 to 1232118 tid 1 2024-09-18T11:32:11.301 INFO:tasks.rados.rados.0.smithi119.stdout:1752: writing smithi11939506-29 from 1990637 to 2638413 tid 2 2024-09-18T11:32:11.305 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:32:11.308 INFO:tasks.rados.rados.0.smithi119.stdout:1752: writing smithi11939506-29 from 3299280 to 3966485 tid 3 2024-09-18T11:32:11.308 INFO:tasks.rados.rados.0.smithi119.stdout:1753: write (excl) oid 37 current snap is 55 2024-09-18T11:32:11.308 INFO:tasks.rados.rados.0.smithi119.stdout:1753: seq_num 356 ranges {783292=540882,1932766=507787,3210350=572013} 2024-09-18T11:32:11.315 INFO:tasks.rados.rados.0.smithi119.stdout:1753: writing smithi11939506-37 from 783292 to 1324174 tid 1 2024-09-18T11:32:11.321 INFO:tasks.rados.rados.0.smithi119.stdout:1753: writing smithi11939506-37 from 1932766 to 2440553 tid 2 2024-09-18T11:32:11.328 INFO:tasks.rados.rados.0.smithi119.stdout:1753: writing smithi11939506-37 from 3210350 to 3782363 tid 3 2024-09-18T11:32:11.328 INFO:tasks.rados.rados.0.smithi119.stdout:1754: write oid 44 current snap is 55 2024-09-18T11:32:11.328 INFO:tasks.rados.rados.0.smithi119.stdout:1754: seq_num 357 ranges {400099=616131,1591248=408851} 2024-09-18T11:32:11.333 INFO:tasks.rados.rados.0.smithi119.stdout:1754: writing smithi11939506-44 from 400099 to 1016230 tid 1 2024-09-18T11:32:11.340 INFO:tasks.rados.rados.0.smithi119.stdout:1754: writing smithi11939506-44 from 1591248 to 2000099 tid 2 2024-09-18T11:32:11.340 INFO:tasks.rados.rados.0.smithi119.stdout:1755: read oid 48 snap -1 2024-09-18T11:32:11.340 INFO:tasks.rados.rados.0.smithi119.stdout:1755: expect (ObjNum 300 snap 43 seq_num 300) 2024-09-18T11:32:11.340 INFO:tasks.rados.rados.0.smithi119.stdout:1756: read oid 46 snap -1 2024-09-18T11:32:11.340 INFO:tasks.rados.rados.0.smithi119.stdout:1756: expect (ObjNum 320 snap 44 seq_num 320) 2024-09-18T11:32:11.340 INFO:tasks.rados.rados.0.smithi119.stdout:1757: read oid 19 snap -1 2024-09-18T11:32:11.340 INFO:tasks.rados.rados.0.smithi119.stdout:1757: expect (ObjNum 288 snap 42 seq_num 288) 2024-09-18T11:32:11.340 INFO:tasks.rados.rados.0.smithi119.stdout:1758: snap_remove snap 49 2024-09-18T11:32:11.352 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:11.351+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:11.407 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:11.407 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:11.415 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:32:11.429 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:11.427+0000 7fb6930fb640 -1 osd.2 139 handle_osd_map injecting map crc failure 2024-09-18T11:32:11.453 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:11.451+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:11.553 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:11.553 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:11.554 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:11.553+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:11.561 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:32:11.655 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:11.653+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:11.699 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:11.700 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:11.708 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:32:11.723 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:32:11.767 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:11.765+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:11.806 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:11.806 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:11.814 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:32:11.839 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:11.839 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:11.854 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:11.854 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:11.858 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:11.857+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:11.869 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:11.869 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:11.884 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:11.884 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:11.899 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:11.900 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:11.912 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:11.912 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:11.914 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:11.915 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:11.921 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:32:11.930 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:11.930 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:11.945 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:11.945 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:11.997 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:11.995+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:12.057 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:12.056+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:12.061 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:12.061 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:12.061 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:12.061 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:12.061 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.061 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:32:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 142.90129886599999, 2024-09-18T11:32:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056157139999999999, 2024-09-18T11:32:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 44 2024-09-18T11:32:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:32:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157551+0000", 2024-09-18T11:32:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.063 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157550+0000", 2024-09-18T11:32:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157655+0000", 2024-09-18T11:32:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010479099999999999 2024-09-18T11:32:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157664+0000", 2024-09-18T11:32:12.064 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4059999999999994e-06 2024-09-18T11:32:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157669+0000", 2024-09-18T11:32:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:32:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160305+0000", 2024-09-18T11:32:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026361499999999999 2024-09-18T11:32:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.065 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160400+0000", 2024-09-18T11:32:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5056000000000006e-05 2024-09-18T11:32:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.160552+0000", 2024-09-18T11:32:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015160299999999999 2024-09-18T11:32:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.162786+0000", 2024-09-18T11:32:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022340939999999998 2024-09-18T11:32:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163095+0000", 2024-09-18T11:32:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000308933 2024-09-18T11:32:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163137+0000", 2024-09-18T11:32:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1736999999999997e-05 2024-09-18T11:32:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163166+0000", 2024-09-18T11:32:12.067 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9679000000000002e-05 2024-09-18T11:32:12.068 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.068 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:12.068 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.068 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.068 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.068 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:12.068 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:32:12.068 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 142.90112349099999, 2024-09-18T11:32:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0056311659999999999, 2024-09-18T11:32:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 46 2024-09-18T11:32:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:32:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.069 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:32:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157726+0000", 2024-09-18T11:32:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:12.070 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157730+0000", 2024-09-18T11:32:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2339999999999996e-06 2024-09-18T11:32:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157756+0000", 2024-09-18T11:32:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.635e-05 2024-09-18T11:32:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.071 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.072 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157759+0000", 2024-09-18T11:32:12.072 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2310000000000001e-06 2024-09-18T11:32:12.072 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.072 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.072 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:12.072 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161557+0000", 2024-09-18T11:32:12.072 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037978560000000001 2024-09-18T11:32:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161633+0000", 2024-09-18T11:32:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5786999999999998e-05 2024-09-18T11:32:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.161759+0000", 2024-09-18T11:32:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000125649 2024-09-18T11:32:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.073 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163256+0000", 2024-09-18T11:32:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014971419999999999 2024-09-18T11:32:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163284+0000", 2024-09-18T11:32:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181e-05 2024-09-18T11:32:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163324+0000", 2024-09-18T11:32:12.074 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9511999999999997e-05 2024-09-18T11:32:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163357+0000", 2024-09-18T11:32:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3627999999999997e-05 2024-09-18T11:32:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 142.90108096, 2024-09-18T11:32:12.075 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:32:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:32:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:32:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:32:12.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:32:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:32:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:32:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:32:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:32:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:32:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:32:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:32:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:32:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:12.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:32:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:32:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:32:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:32:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:12.080 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:32:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:32:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:32:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:32:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:32:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:32:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:12.081 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 142.90102603, 2024-09-18T11:32:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:32:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:12.082 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:32:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:32:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:32:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:12.084 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:32:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:32:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:32:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:32:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:32:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:32:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:32:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:32:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:32:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:32:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:32:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:32:12.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:32:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:32:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:32:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:32:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:32:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:32:12.087 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 142.90101605000001, 2024-09-18T11:32:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:32:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:12.088 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:32:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:32:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:32:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:12.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:32:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:32:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:32:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:32:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:32:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:32:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:32:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:32:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:32:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:32:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:32:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:32:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:32:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:32:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:32:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:32:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:32:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:32:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 142.90100445100001, 2024-09-18T11:32:12.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:32:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:32:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:12.096 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:32:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:32:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:32:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:32:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:32:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:32:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:12.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:32:12.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:32:12.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.098 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:12.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:32:12.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:32:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:32:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:32:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:32:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:32:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:12.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:32:12.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:32:12.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:12.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:32:12.101 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:32:12.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.101 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:12.101 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:32:12.101 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:32:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 142.90094797, 2024-09-18T11:32:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:32:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:12.102 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:32:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:32:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.105 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:12.105 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:32:12.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:32:12.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.105 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:32:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:32:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:32:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:32:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:32:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:32:12.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:32:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:32:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:32:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:32:12.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:32:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:32:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:32:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:32:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.109 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:12.109 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:32:12.109 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:32:12.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.109 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:32:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:32:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 142.900934146, 2024-09-18T11:32:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:32:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:12.110 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:32:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.112 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:12.112 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:12.112 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.113 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:12.113 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:12.113 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:12.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.114 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:12.114 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:32:12.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:32:12.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.114 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:12.114 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:32:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:32:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:32:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:32:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:32:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:32:12.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:12.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:32:12.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:32:12.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:32:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:32:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:32:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:32:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:12.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:32:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:32:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:32:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:32:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:32:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:32:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.118 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 142.48475523900001, 2024-09-18T11:32:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:32:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:32:12.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:32:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:32:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:32:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:32:12.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:32:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:32:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:32:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:32:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:32:12.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:32:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:32:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:32:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:32:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:32:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:12.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:32:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:32:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:32:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:32:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:32:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:32:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.124 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 142.484731939, 2024-09-18T11:32:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:32:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:12.125 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:32:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:12.126 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:32:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:32:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:32:12.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:32:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:32:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:32:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:32:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:32:12.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:32:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:32:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:32:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:32:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:32:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:32:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:32:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:32:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:32:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:32:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:32:12.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.131 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:12.131 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:32:12.131 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:32:12.131 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.131 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:12.131 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.132 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:12.132 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:12.132 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 142.48464633, 2024-09-18T11:32:12.132 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:32:12.132 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:12.132 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.132 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:12.132 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:12.132 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:12.132 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:32:12.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.132 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:12.133 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.133 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:12.133 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:12.133 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.133 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.133 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:12.133 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:12.133 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.133 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.133 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:12.133 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:12.134 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:12.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:12.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:32:12.134 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:32:12.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:12.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:32:12.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:32:12.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.135 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.135 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:32:12.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:32:12.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.135 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:12.135 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:32:12.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:32:12.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.135 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:12.135 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:32:12.136 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:32:12.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.136 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:12.136 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:32:12.136 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:32:12.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.136 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:12.136 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:32:12.136 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:32:12.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.136 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:12.137 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:32:12.137 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:32:12.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.137 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.137 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:12.137 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:32:12.137 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:32:12.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.137 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.137 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:12.137 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:32:12.137 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:32:12.137 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.137 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:12.138 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.138 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:12.138 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:12.138 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 142.484635365, 2024-09-18T11:32:12.138 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:32:12.138 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:12.138 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.138 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:12.138 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:12.138 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:12.138 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:32:12.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.139 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:12.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.139 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:12.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:12.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.139 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:12.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:12.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.139 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:12.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:12.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:12.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.140 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:12.140 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:32:12.140 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:32:12.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.140 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:12.140 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:32:12.140 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:32:12.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.140 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.140 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:32:12.141 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:32:12.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.141 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:12.141 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:32:12.141 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:32:12.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.141 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:12.141 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:32:12.141 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:32:12.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.142 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:12.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:32:12.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:32:12.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.142 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:12.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:32:12.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:32:12.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.142 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:12.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:32:12.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:32:12.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.143 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:12.143 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:32:12.143 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:32:12.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.143 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:12.143 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:32:12.143 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:32:12.143 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.143 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:12.143 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.144 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:12.144 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:12.144 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 142.48462419699999, 2024-09-18T11:32:12.144 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:32:12.144 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:12.144 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.144 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:12.144 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:12.144 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:12.144 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:32:12.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.144 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:12.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.145 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:12.145 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:12.145 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.145 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:12.145 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:12.145 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.145 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:12.145 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:12.146 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:12.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.146 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.146 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:12.146 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:32:12.146 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:32:12.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.146 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.146 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:12.146 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:32:12.146 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:32:12.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.146 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.146 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.147 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:32:12.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:32:12.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.147 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:12.147 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:32:12.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:32:12.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.147 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:12.147 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:32:12.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:32:12.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:12.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:32:12.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:32:12.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:12.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:32:12.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:32:12.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:12.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:32:12.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:32:12.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.149 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:12.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:32:12.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:32:12.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.149 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:12.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:32:12.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:32:12.149 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.149 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:12.149 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.150 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.150 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:12.150 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:12.150 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 142.48455886799999, 2024-09-18T11:32:12.150 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:32:12.150 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:12.150 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.150 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:12.150 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:12.150 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:12.150 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:32:12.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.151 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:12.151 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.151 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:12.151 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:12.151 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.151 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.151 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:12.151 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:12.151 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.151 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.151 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:12.151 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:32:12.152 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:12.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.152 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.152 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:12.152 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:32:12.152 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:32:12.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.152 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.152 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:12.152 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:32:12.152 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:32:12.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.152 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.153 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:32:12.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:32:12.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.153 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:12.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:32:12.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:32:12.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.153 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:12.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:32:12.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:32:12.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:12.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:32:12.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:32:12.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:12.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:32:12.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:32:12.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:12.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:32:12.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:32:12.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.155 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:12.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:32:12.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:32:12.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.155 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:12.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:32:12.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:32:12.155 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.156 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:12.156 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.156 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:12.156 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:12.156 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 142.47439689000001, 2024-09-18T11:32:12.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:32:12.156 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:12.156 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.156 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:12.156 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:12.156 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:12.157 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:32:12.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.157 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:12.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.157 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:12.157 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:12.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.157 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:12.157 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:12.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.158 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:12.158 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:32:12.158 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:12.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.158 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:12.158 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:32:12.158 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:32:12.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.158 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:12.158 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:32:12.158 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:32:12.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.159 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:32:12.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:32:12.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.159 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:12.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:32:12.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:32:12.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.159 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:12.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:32:12.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:32:12.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.160 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:12.160 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:32:12.160 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:32:12.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.160 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:12.160 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:32:12.160 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:32:12.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:12.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:32:12.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:32:12.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:12.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:32:12.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:32:12.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:12.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:32:12.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:32:12.161 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.162 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:12.162 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.162 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:12.162 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:12.162 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 142.47430441099999, 2024-09-18T11:32:12.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:32:12.162 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:12.162 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.162 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:12.162 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:12.162 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:12.162 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:32:12.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.163 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:12.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.163 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:12.163 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:12.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.163 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:12.163 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:12.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.163 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:12.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:12.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:12.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.164 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:12.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:32:12.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:32:12.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.164 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:12.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:32:12.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:32:12.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:32:12.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:32:12.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:12.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:32:12.166 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:32:12.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:12.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:32:12.166 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:32:12.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:12.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:32:12.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:32:12.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:12.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:32:12.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:32:12.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:12.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:32:12.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:32:12.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:12.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:32:12.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:32:12.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:12.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:32:12.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:32:12.168 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.168 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:12.168 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.169 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1788 4.8 e60/22 4:1898b15d:::smithi11939506-25:head v 60'12, mlcod=58'9)", 2024-09-18T11:32:12.169 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:32:12.169 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 89.141898921999996, 2024-09-18T11:32:12.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058348599999999999, 2024-09-18T11:32:12.169 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:12.169 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.169 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:12.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:12.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:32:12.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:12.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:32:12.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:12.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916949+0000", 2024-09-18T11:32:12.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:12.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:12.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917602+0000", 2024-09-18T11:32:12.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00065337100000000003 2024-09-18T11:32:12.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:12.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917621+0000", 2024-09-18T11:32:12.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9287e-05 2024-09-18T11:32:12.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917629+0000", 2024-09-18T11:32:12.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3780000000000003e-06 2024-09-18T11:32:12.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:12.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917677+0000", 2024-09-18T11:32:12.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8068000000000001e-05 2024-09-18T11:32:12.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.172 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:12.172 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917699+0000", 2024-09-18T11:32:12.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2149000000000001e-05 2024-09-18T11:32:12.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.172 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:12.172 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922747+0000", 2024-09-18T11:32:12.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050477609999999996 2024-09-18T11:32:12.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.173 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:12.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922785+0000", 2024-09-18T11:32:12.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8726000000000002e-05 2024-09-18T11:32:12.173 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.173 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:12.173 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.173 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3209 4.0 e109/22 4:019b6e09:::smithi11939506-40:head v 109'149, mlcod=106'146)", 2024-09-18T11:32:12.173 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:32:12.173 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 34.959463560000003, 2024-09-18T11:32:12.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063064039999999998, 2024-09-18T11:32:12.174 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:12.174 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.174 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:12.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.174 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:12.174 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:32:12.174 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.174 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:12.174 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:32:12.174 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.175 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:12.175 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099385+0000", 2024-09-18T11:32:12.175 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:12.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.175 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:12.175 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099951+0000", 2024-09-18T11:32:12.175 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056606600000000005 2024-09-18T11:32:12.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:12.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099964+0000", 2024-09-18T11:32:12.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2918999999999999e-05 2024-09-18T11:32:12.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099972+0000", 2024-09-18T11:32:12.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2300000000000002e-06 2024-09-18T11:32:12.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:12.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.100026+0000", 2024-09-18T11:32:12.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4456999999999997e-05 2024-09-18T11:32:12.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:12.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.100046+0000", 2024-09-18T11:32:12.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0417000000000001e-05 2024-09-18T11:32:12.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:12.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.105653+0000", 2024-09-18T11:32:12.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005606214 2024-09-18T11:32:12.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:12.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.105692+0000", 2024-09-18T11:32:12.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9671e-05 2024-09-18T11:32:12.178 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.178 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:12.178 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.178 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:123 4.3 e132/67 4:cc46532a:::smithi11939506-33:30 v 132'171, mlcod=128'168)", 2024-09-18T11:32:12.178 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:12.178 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.0309977860000004, 2024-09-18T11:32:12.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087627039999999996, 2024-09-18T11:32:12.178 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:12.179 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.179 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:12.179 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.179 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:12.179 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:12.179 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.179 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.179 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:12.179 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:12.179 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.179 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.179 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:12.179 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027851+0000", 2024-09-18T11:32:12.179 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:12.180 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.180 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.180 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:12.180 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027871+0000", 2024-09-18T11:32:12.180 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9749e-05 2024-09-18T11:32:12.180 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.180 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.180 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:12.180 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027882+0000", 2024-09-18T11:32:12.180 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1242e-05 2024-09-18T11:32:12.180 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.180 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027889+0000", 2024-09-18T11:32:12.181 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7649999999999996e-06 2024-09-18T11:32:12.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.181 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:12.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027929+0000", 2024-09-18T11:32:12.181 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9984999999999997e-05 2024-09-18T11:32:12.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.181 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:12.182 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027951+0000", 2024-09-18T11:32:12.182 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2047000000000001e-05 2024-09-18T11:32:12.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.182 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.182 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:12.182 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.036565+0000", 2024-09-18T11:32:12.182 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086135110000000008 2024-09-18T11:32:12.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.183 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.183 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:12.183 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.036614+0000", 2024-09-18T11:32:12.183 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9747000000000003e-05 2024-09-18T11:32:12.183 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.183 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:12.183 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.183 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.183 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:124 4.3 e132/67 4:ca2b2309:::smithi11939506-44:30 v 132'173, mlcod=128'168)", 2024-09-18T11:32:12.183 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:12.183 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.0303058039999993, 2024-09-18T11:32:12.184 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0088947309999999995, 2024-09-18T11:32:12.184 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:12.184 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.184 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:12.184 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.184 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:12.184 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:12.184 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.184 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.184 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:12.184 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:12.184 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:12.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.185 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.185 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:12.185 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028543+0000", 2024-09-18T11:32:12.185 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:12.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.185 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.185 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:12.185 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028551+0000", 2024-09-18T11:32:12.185 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8720000000000002e-06 2024-09-18T11:32:12.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.185 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.185 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:12.185 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028562+0000", 2024-09-18T11:32:12.185 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1039000000000001e-05 2024-09-18T11:32:12.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.186 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028569+0000", 2024-09-18T11:32:12.186 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0350000000000004e-06 2024-09-18T11:32:12.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.186 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:12.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028848+0000", 2024-09-18T11:32:12.186 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00027935600000000003 2024-09-18T11:32:12.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:12.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028868+0000", 2024-09-18T11:32:12.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9947999999999999e-05 2024-09-18T11:32:12.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:12.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037405+0000", 2024-09-18T11:32:12.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085369920000000002 2024-09-18T11:32:12.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:12.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:12.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:12.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037438+0000", 2024-09-18T11:32:12.188 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3244000000000002e-05 2024-09-18T11:32:12.188 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.188 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:12.188 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.188 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:12.188 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:12.188 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:12.189 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:32:12.190 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:12.188+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:12.290 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:12.290 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:12.290 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:12.290 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:12.291 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:12.289+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:12.297 INFO:tasks.rados.rados.0.smithi119.stdout:1758: done (9 left) 2024-09-18T11:32:12.297 INFO:tasks.rados.rados.0.smithi119.stdout:1759: read oid 27 snap -1 2024-09-18T11:32:12.297 INFO:tasks.rados.rados.0.smithi119.stdout:1759: expect (ObjNum 328 snap 45 seq_num 328) 2024-09-18T11:32:12.297 INFO:tasks.rados.rados.0.smithi119.stdout:1760: read oid 14 snap -1 2024-09-18T11:32:12.297 INFO:tasks.rados.rados.0.smithi119.stdout:1760: expect (ObjNum 352 snap 54 seq_num 352) 2024-09-18T11:32:12.297 INFO:tasks.rados.rados.0.smithi119.stdout:1761: set_chunk oid 21 target oid 2024-09-18T11:32:12.301 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T11:32:12.302 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:32:12.309 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:12.308+0000 7fd27659f640 -1 osd.6 140 handle_osd_map injecting map crc failure 2024-09-18T11:32:12.328 INFO:tasks.rados.rados.0.smithi119.stdout:1761: set_chunk oid 21 offset: 829440 length: 1015808 target oid 3526641c07d3efa989be9684355570f39c331008b8a02c9fa22b38a94b6c6b9f offset: 0 snap -1 2024-09-18T11:32:12.328 INFO:tasks.rados.rados.0.smithi119.stdout:1762: write oid 10 current snap is 55 2024-09-18T11:32:12.328 INFO:tasks.rados.rados.0.smithi119.stdout:1762: seq_num 358 ranges {416906=407733,1366083=511456} 2024-09-18T11:32:12.332 INFO:tasks.rados.rados.0.smithi119.stdout:1762: writing smithi11939506-10 from 416906 to 824639 tid 1 2024-09-18T11:32:12.339 INFO:tasks.rados.rados.0.smithi119.stdout:1762: writing smithi11939506-10 from 1366083 to 1877539 tid 2 2024-09-18T11:32:12.339 INFO:tasks.rados.rados.0.smithi119.stdout:1763: read oid 18 snap -1 2024-09-18T11:32:12.339 INFO:tasks.rados.rados.0.smithi119.stdout:1763: expect (ObjNum 205 snap 23 seq_num 205) 2024-09-18T11:32:12.339 INFO:tasks.rados.rados.0.smithi119.stdout:1764: set_chunk oid 30 target oid 2024-09-18T11:32:12.339 INFO:tasks.rados.rados.0.smithi119.stdout:1764: set_chunk oid 30 offset: 0 length: 4000000 target oid smithi11939506- offset: 0 snap -1 2024-09-18T11:32:12.339 INFO:tasks.rados.rados.0.smithi119.stdout:1765: read oid 6 snap -1 2024-09-18T11:32:12.339 INFO:tasks.rados.rados.0.smithi119.stdout:1765: expect (ObjNum 327 snap 45 seq_num 327) 2024-09-18T11:32:12.339 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:32:12.339 INFO:tasks.rados.rados.0.smithi119.stdout:1749: finishing rollback tid 0 to smithi11939506-33 2024-09-18T11:32:12.340 INFO:tasks.rados.rados.0.smithi119.stdout:1749: finishing rollback tid 1 to smithi11939506-33 2024-09-18T11:32:12.340 INFO:tasks.rados.rados.0.smithi119.stdout:1749: finishing rollback tid 2 to smithi11939506-33 2024-09-18T11:32:12.340 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 33 v 189 (ObjNum 316 snap 43 seq_num 316) dirty exists 2024-09-18T11:32:12.340 INFO:tasks.rados.rados.0.smithi119.stdout:1751: expect (ObjNum 283 snap 39 seq_num 283) 2024-09-18T11:32:12.391 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:12.389+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:12.433 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:12.433 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:12.433 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:12.433 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:12.433 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:12.441 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:32:12.526 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:12.524+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:12.553 INFO:tasks.rados.rados.0.smithi119.stdout:1752: finishing write tid 1 to smithi11939506-29 2024-09-18T11:32:12.553 INFO:tasks.rados.rados.0.smithi119.stdout:1750: expect (ObjNum 346 snap 51 seq_num 346) 2024-09-18T11:32:12.581 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:12.581 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:12.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:12.581 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [] 2024-09-18T11:32:12.581 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:12.590 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:32:12.593 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:12.592+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:12.695 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:12.693+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:12.729 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:12.729 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:12.729 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:12.729 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:12.730 INFO:tasks.rados.rados.0.smithi119.stdout:1752: finishing write tid 2 to smithi11939506-29 2024-09-18T11:32:12.730 INFO:tasks.rados.rados.0.smithi119.stdout:1749: done (15 left) 2024-09-18T11:32:12.730 INFO:tasks.rados.rados.0.smithi119.stdout:1750: done (14 left) 2024-09-18T11:32:12.730 INFO:tasks.rados.rados.0.smithi119.stdout:1751: done (13 left) 2024-09-18T11:32:12.731 INFO:tasks.rados.rados.0.smithi119.stdout:1766: write oid 16 current snap is 55 2024-09-18T11:32:12.731 INFO:tasks.rados.rados.0.smithi119.stdout:1752: finishing write tid 3 to smithi11939506-29 2024-09-18T11:32:12.731 INFO:tasks.rados.rados.0.smithi119.stdout:1752: finishing write tid 4 to smithi11939506-29 2024-09-18T11:32:12.731 INFO:tasks.rados.rados.0.smithi119.stdout:1766: seq_num 359 ranges {433713=482982,1424565=609148} 2024-09-18T11:32:12.734 INFO:tasks.rados.rados.0.smithi119.stdout:1766: writing smithi11939506-16 from 433713 to 916695 tid 1 2024-09-18T11:32:12.738 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:32:12.741 INFO:tasks.rados.rados.0.smithi119.stdout:1766: writing smithi11939506-16 from 1424565 to 2033713 tid 2 2024-09-18T11:32:12.741 INFO:tasks.rados.rados.0.smithi119.stdout:1767: read oid 40 snap 46 2024-09-18T11:32:12.741 INFO:tasks.rados.rados.0.smithi119.stdout:1767: expect (ObjNum 287 snap 41 seq_num 287) 2024-09-18T11:32:12.741 INFO:tasks.rados.rados.0.smithi119.stdout:1768: set_chunk oid 17 target oid 2024-09-18T11:32:12.759 INFO:tasks.rados.rados.0.smithi119.stdout:1768: set_chunk oid 17 offset: 450560 length: 2091520 target oid 37b2d60b0981df382017e5a65f576cc464db41a8583063e706cb324cb2ad9a30 offset: 0 snap -1 2024-09-18T11:32:12.759 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 16 2024-09-18T11:32:12.759 INFO:tasks.rados.rados.0.smithi119.stdout:1752: finishing write tid 5 to smithi11939506-29 2024-09-18T11:32:12.759 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 106 (ObjNum 355 snap 55 seq_num 355) dirty exists 2024-09-18T11:32:12.759 INFO:tasks.rados.rados.0.smithi119.stdout:1752: left oid 29 (ObjNum 355 snap 55 seq_num 355) 2024-09-18T11:32:12.759 INFO:tasks.rados.rados.0.smithi119.stdout:1753: finishing write tid 1 to smithi11939506-37 2024-09-18T11:32:12.759 INFO:tasks.rados.rados.0.smithi119.stdout:1752: done (15 left) 2024-09-18T11:32:12.760 INFO:tasks.rados.rados.0.smithi119.stdout:1769: rollback oid 45 current snap is 55 2024-09-18T11:32:12.760 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 45 to 46 2024-09-18T11:32:12.760 INFO:tasks.rados.rados.0.smithi119.stdout:1753: finishing write tid 2 to smithi11939506-37 2024-09-18T11:32:12.760 INFO:tasks.rados.rados.0.smithi119.stdout:1753: finishing write tid 3 to smithi11939506-37 2024-09-18T11:32:12.760 INFO:tasks.rados.rados.0.smithi119.stdout:1753: finishing write tid 4 to smithi11939506-37 2024-09-18T11:32:12.760 INFO:tasks.rados.rados.0.smithi119.stdout:1753: finishing write tid 5 to smithi11939506-37 2024-09-18T11:32:12.760 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 299 (ObjNum 356 snap 55 seq_num 356) dirty exists 2024-09-18T11:32:12.760 INFO:tasks.rados.rados.0.smithi119.stdout:1753: left oid 37 (ObjNum 356 snap 55 seq_num 356) 2024-09-18T11:32:12.760 INFO:tasks.rados.rados.0.smithi119.stdout:1754: finishing write tid 1 to smithi11939506-44 2024-09-18T11:32:12.760 INFO:tasks.rados.rados.0.smithi119.stdout:1757: expect (ObjNum 288 snap 42 seq_num 288) 2024-09-18T11:32:12.795 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:12.793+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:12.843 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:12.843 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:12.843 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:12.843 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:12.843 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:12.851 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:32:12.888 INFO:tasks.rados.rados.0.smithi119.stdout:1754: finishing write tid 2 to smithi11939506-44 2024-09-18T11:32:12.888 INFO:tasks.rados.rados.0.smithi119.stdout:1754: finishing write tid 3 to smithi11939506-44 2024-09-18T11:32:12.888 INFO:tasks.rados.rados.0.smithi119.stdout:1754: finishing write tid 4 to smithi11939506-44 2024-09-18T11:32:12.888 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 44 v 193 (ObjNum 357 snap 55 seq_num 357) dirty exists 2024-09-18T11:32:12.888 INFO:tasks.rados.rados.0.smithi119.stdout:1754: left oid 44 (ObjNum 357 snap 55 seq_num 357) 2024-09-18T11:32:12.888 INFO:tasks.rados.rados.0.smithi119.stdout:1755: expect (ObjNum 300 snap 43 seq_num 300) 2024-09-18T11:32:12.932 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:12.930+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:12.943 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-18T11:32:12.943 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T11:32:12.944 INFO:tasks.thrashosds.thrasher:Reweighting osd 2 to 0.2804897951246304 2024-09-18T11:32:12.944 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 0.2804897951246304 2024-09-18T11:32:12.990 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:262 4.9 e53/22 4:99feffe9:::smithi11939506-45:13 v 53'17, mlcod=51'14)", 2024-09-18T11:32:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 98.988878970000002, 2024-09-18T11:32:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013878679, 2024-09-18T11:32:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998997+0000", 2024-09-18T11:32:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1099999999999999e-06 2024-09-18T11:32:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999008+0000", 2024-09-18T11:32:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1321000000000001e-05 2024-09-18T11:32:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999015+0000", 2024-09-18T11:32:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3000000000000004e-06 2024-09-18T11:32:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999055+0000", 2024-09-18T11:32:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0058000000000002e-05 2024-09-18T11:32:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999070+0000", 2024-09-18T11:32:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4328000000000001e-05 2024-09-18T11:32:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012465+0000", 2024-09-18T11:32:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013395638 2024-09-18T11:32:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012870+0000", 2024-09-18T11:32:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040423400000000001 2024-09-18T11:32:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:205 4.b e53/22 4:dba5bd39:::smithi11939506-17:13 v 53'77, mlcod=51'74)", 2024-09-18T11:32:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 98.988829709000001, 2024-09-18T11:32:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013359586, 2024-09-18T11:32:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:12.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:12.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999048+0000", 2024-09-18T11:32:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3550000000000005e-06 2024-09-18T11:32:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999056+0000", 2024-09-18T11:32:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3259999999999996e-06 2024-09-18T11:32:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:12.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999063+0000", 2024-09-18T11:32:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6409999999999996e-06 2024-09-18T11:32:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999098+0000", 2024-09-18T11:32:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5203999999999997e-05 2024-09-18T11:32:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999115+0000", 2024-09-18T11:32:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6478999999999999e-05 2024-09-18T11:32:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:12.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012362+0000", 2024-09-18T11:32:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01324768 2024-09-18T11:32:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012400+0000", 2024-09-18T11:32:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.731e-05 2024-09-18T11:32:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1956 4.6 4:67c1765a:::smithi11939506-30:head [write 705927~558502] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:32:13.001 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:32:13.001 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.635860758000007, 2024-09-18T11:32:13.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011813239, 2024-09-18T11:32:13.001 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.001 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.001 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:13.001 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:13.001 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:13.001 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1956 2024-09-18T11:32:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:32:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:32:13.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352007+0000", 2024-09-18T11:32:13.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:13.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352229+0000", 2024-09-18T11:32:13.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222024 2024-09-18T11:32:13.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352251+0000", 2024-09-18T11:32:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2512000000000001e-05 2024-09-18T11:32:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352260+0000", 2024-09-18T11:32:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7339999999999997e-06 2024-09-18T11:32:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352307+0000", 2024-09-18T11:32:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7061999999999998e-05 2024-09-18T11:32:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352408+0000", 2024-09-18T11:32:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101216 2024-09-18T11:32:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:32:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352550+0000", 2024-09-18T11:32:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014136199999999999 2024-09-18T11:32:13.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357684+0000", 2024-09-18T11:32:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051342499999999999 2024-09-18T11:32:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363736+0000", 2024-09-18T11:32:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006052577 2024-09-18T11:32:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363778+0000", 2024-09-18T11:32:13.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1989000000000001e-05 2024-09-18T11:32:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363822+0000", 2024-09-18T11:32:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3894999999999999e-05 2024-09-18T11:32:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2937 4.7 4:e301aefd:::smithi11939506-19:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:13.007 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 49.953029395999998, 2024-09-18T11:32:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019287667000000001, 2024-09-18T11:32:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2937 2024-09-18T11:32:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:13.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034842+0000", 2024-09-18T11:32:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.372e-06 2024-09-18T11:32:13.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034846+0000", 2024-09-18T11:32:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3919999999999996e-06 2024-09-18T11:32:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034848+0000", 2024-09-18T11:32:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3539999999999998e-06 2024-09-18T11:32:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034893+0000", 2024-09-18T11:32:13.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4861000000000003e-05 2024-09-18T11:32:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034965+0000", 2024-09-18T11:32:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1860999999999995e-05 2024-09-18T11:32:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.054128+0000", 2024-09-18T11:32:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019163039 2024-09-18T11:32:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.012 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3453 4.f 4:f5d7df41:::smithi11939506-16:head [write 1813152~490434] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:32:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.965117467999999, 2024-09-18T11:32:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050685840000000003, 2024-09-18T11:32:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:13.013 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3453 2024-09-18T11:32:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:32:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:32:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022751+0000", 2024-09-18T11:32:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022871+0000", 2024-09-18T11:32:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000119781 2024-09-18T11:32:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022882+0000", 2024-09-18T11:32:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1627000000000001e-05 2024-09-18T11:32:13.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022887+0000", 2024-09-18T11:32:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2390000000000004e-06 2024-09-18T11:32:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023123+0000", 2024-09-18T11:32:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000236598 2024-09-18T11:32:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023207+0000", 2024-09-18T11:32:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4303999999999998e-05 2024-09-18T11:32:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023367+0000", 2024-09-18T11:32:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015983600000000001 2024-09-18T11:32:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073237+0000", 2024-09-18T11:32:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049869400000000001 2024-09-18T11:32:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073352+0000", 2024-09-18T11:32:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011579899999999999 2024-09-18T11:32:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073394+0000", 2024-09-18T11:32:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1445e-05 2024-09-18T11:32:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073438+0000", 2024-09-18T11:32:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4261999999999997e-05 2024-09-18T11:32:13.018 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3454 4.f 4:f5d7df41:::smithi11939506-16:head [setxattr _header (57) in=7b,truncate 2991679] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:32:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.964419026000002, 2024-09-18T11:32:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050365656000000002, 2024-09-18T11:32:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:13.019 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3454 2024-09-18T11:32:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:32:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:32:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023450+0000", 2024-09-18T11:32:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023456+0000", 2024-09-18T11:32:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8329999999999999e-06 2024-09-18T11:32:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023465+0000", 2024-09-18T11:32:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1130000000000007e-06 2024-09-18T11:32:13.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023470+0000", 2024-09-18T11:32:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9139999999999999e-06 2024-09-18T11:32:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072611+0000", 2024-09-18T11:32:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141643999999998 2024-09-18T11:32:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072729+0000", 2024-09-18T11:32:13.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117667 2024-09-18T11:32:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072887+0000", 2024-09-18T11:32:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015760000000000001 2024-09-18T11:32:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073558+0000", 2024-09-18T11:32:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00067171699999999995 2024-09-18T11:32:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073764+0000", 2024-09-18T11:32:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000205749 2024-09-18T11:32:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073790+0000", 2024-09-18T11:32:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5643000000000002e-05 2024-09-18T11:32:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073816+0000", 2024-09-18T11:32:13.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6588e-05 2024-09-18T11:32:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3455 4.f 4:f5d7df41:::smithi11939506-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.964389407999999, 2024-09-18T11:32:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050460466000000002, 2024-09-18T11:32:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:13.025 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3455 2024-09-18T11:32:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023483+0000", 2024-09-18T11:32:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.695e-06 2024-09-18T11:32:13.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023489+0000", 2024-09-18T11:32:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1079999999999998e-06 2024-09-18T11:32:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023492+0000", 2024-09-18T11:32:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.478e-06 2024-09-18T11:32:13.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073273+0000", 2024-09-18T11:32:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049780638000000002 2024-09-18T11:32:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073343+0000", 2024-09-18T11:32:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9770000000000005e-05 2024-09-18T11:32:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073846+0000", 2024-09-18T11:32:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050289499999999995 2024-09-18T11:32:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073886+0000", 2024-09-18T11:32:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0658999999999998e-05 2024-09-18T11:32:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073941+0000", 2024-09-18T11:32:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4642000000000003e-05 2024-09-18T11:32:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.030 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3456 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.964364343, 2024-09-18T11:32:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049859120999999999, 2024-09-18T11:32:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:13.031 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3456 2024-09-18T11:32:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023508+0000", 2024-09-18T11:32:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9490000000000001e-06 2024-09-18T11:32:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023513+0000", 2024-09-18T11:32:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3800000000000002e-06 2024-09-18T11:32:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023517+0000", 2024-09-18T11:32:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5300000000000001e-06 2024-09-18T11:32:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073262+0000", 2024-09-18T11:32:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049745194999999999 2024-09-18T11:32:13.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073365+0000", 2024-09-18T11:32:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102462 2024-09-18T11:32:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3457 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:32:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.964347246999999, 2024-09-18T11:32:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049928567, 2024-09-18T11:32:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3457 2024-09-18T11:32:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:32:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:32:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023522+0000", 2024-09-18T11:32:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023524+0000", 2024-09-18T11:32:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8500000000000001e-06 2024-09-18T11:32:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023529+0000", 2024-09-18T11:32:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7509999999999999e-06 2024-09-18T11:32:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023532+0000", 2024-09-18T11:32:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.163e-06 2024-09-18T11:32:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073386+0000", 2024-09-18T11:32:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049853899 2024-09-18T11:32:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073451+0000", 2024-09-18T11:32:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5197000000000006e-05 2024-09-18T11:32:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3458 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.964334690000001, 2024-09-18T11:32:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.05001659, 2024-09-18T11:32:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3458 2024-09-18T11:32:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:13.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023537+0000", 2024-09-18T11:32:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.64e-06 2024-09-18T11:32:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023543+0000", 2024-09-18T11:32:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3369999999999996e-06 2024-09-18T11:32:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023546+0000", 2024-09-18T11:32:13.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:32:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073466+0000", 2024-09-18T11:32:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049919719000000001 2024-09-18T11:32:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073552+0000", 2024-09-18T11:32:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6168999999999994e-05 2024-09-18T11:32:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.043 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(client.4590.0:3453 4.f e116/22 ondisk, result = 0)", 2024-09-18T11:32:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:32:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.962729606, 2024-09-18T11:32:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048099441, 2024-09-18T11:32:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:32:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:32:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025139+0000", 2024-09-18T11:32:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025144+0000", 2024-09-18T11:32:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4059999999999998e-06 2024-09-18T11:32:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072212+0000", 2024-09-18T11:32:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047068482000000002 2024-09-18T11:32:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072230+0000", 2024-09-18T11:32:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7632999999999999e-05 2024-09-18T11:32:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073224+0000", 2024-09-18T11:32:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099436899999999994 2024-09-18T11:32:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073231+0000", 2024-09-18T11:32:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-09-18T11:32:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073240+0000", 2024-09-18T11:32:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5699999999999993e-06 2024-09-18T11:32:13.047 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3930 4.b 4:d3ae8215:::smithi11939506-41:head [write 632029~547288] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:32:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9618139259999996, 2024-09-18T11:32:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018962711, 2024-09-18T11:32:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:13.048 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3930 2024-09-18T11:32:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:32:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:32:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026055+0000", 2024-09-18T11:32:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026334+0000", 2024-09-18T11:32:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000278931 2024-09-18T11:32:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026344+0000", 2024-09-18T11:32:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0716999999999999e-05 2024-09-18T11:32:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026350+0000", 2024-09-18T11:32:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7620000000000001e-06 2024-09-18T11:32:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028429+0000", 2024-09-18T11:32:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020787319999999998 2024-09-18T11:32:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028525+0000", 2024-09-18T11:32:13.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6638999999999994e-05 2024-09-18T11:32:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028720+0000", 2024-09-18T11:32:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000194525 2024-09-18T11:32:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.036996+0000", 2024-09-18T11:32:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082756140000000006 2024-09-18T11:32:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044952+0000", 2024-09-18T11:32:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566740000000004 2024-09-18T11:32:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044985+0000", 2024-09-18T11:32:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3081999999999999e-05 2024-09-18T11:32:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045019+0000", 2024-09-18T11:32:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3257999999999999e-05 2024-09-18T11:32:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.053 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3931 4.b 4:d3ae8215:::smithi11939506-41:head [write 1639134~749453] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.033118+0000", 2024-09-18T11:32:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9547522859999997, 2024-09-18T11:32:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012000667, 2024-09-18T11:32:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3931 2024-09-18T11:32:13.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033118+0000", 2024-09-18T11:32:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033118+0000", 2024-09-18T11:32:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033116+0000", 2024-09-18T11:32:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033408+0000", 2024-09-18T11:32:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029145000000000001 2024-09-18T11:32:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033420+0000", 2024-09-18T11:32:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2748999999999999e-05 2024-09-18T11:32:13.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033426+0000", 2024-09-18T11:32:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8889999999999999e-06 2024-09-18T11:32:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.034205+0000", 2024-09-18T11:32:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00077899000000000002 2024-09-18T11:32:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.034280+0000", 2024-09-18T11:32:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4871999999999997e-05 2024-09-18T11:32:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.034392+0000", 2024-09-18T11:32:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000111666 2024-09-18T11:32:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.037001+0000", 2024-09-18T11:32:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026091310000000002 2024-09-18T11:32:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045055+0000", 2024-09-18T11:32:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080539700000000006 2024-09-18T11:32:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045090+0000", 2024-09-18T11:32:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4984000000000003e-05 2024-09-18T11:32:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045118+0000", 2024-09-18T11:32:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8343999999999998e-05 2024-09-18T11:32:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3932 4.b 4:d3ae8215:::smithi11939506-41:head [write 2796249~437859] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:32:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9498072860000004, 2024-09-18T11:32:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01458329, 2024-09-18T11:32:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.060 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3932 2024-09-18T11:32:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:32:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:32:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038061+0000", 2024-09-18T11:32:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038341+0000", 2024-09-18T11:32:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027989099999999999 2024-09-18T11:32:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038353+0000", 2024-09-18T11:32:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2247e-05 2024-09-18T11:32:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038358+0000", 2024-09-18T11:32:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-18T11:32:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043226+0000", 2024-09-18T11:32:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048673049999999997 2024-09-18T11:32:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043284+0000", 2024-09-18T11:32:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8338999999999998e-05 2024-09-18T11:32:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043367+0000", 2024-09-18T11:32:13.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3120000000000004e-05 2024-09-18T11:32:13.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:13.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.051544+0000", 2024-09-18T11:32:13.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00817734 2024-09-18T11:32:13.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:13.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052528+0000", 2024-09-18T11:32:13.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098318499999999992 2024-09-18T11:32:13.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052610+0000", 2024-09-18T11:32:13.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2300999999999997e-05 2024-09-18T11:32:13.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052646+0000", 2024-09-18T11:32:13.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5955999999999999e-05 2024-09-18T11:32:13.066 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.066 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.066 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.066 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3933 4.b 4:d3ae8215:::smithi11939506-41:head [write 3653524~178505] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:13.066 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041300+0000", 2024-09-18T11:32:13.067 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9465693900000005, 2024-09-18T11:32:13.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01299752, 2024-09-18T11:32:13.067 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.067 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.067 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:13.067 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:13.067 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:13.067 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3933 2024-09-18T11:32:13.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.067 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041300+0000", 2024-09-18T11:32:13.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041300+0000", 2024-09-18T11:32:13.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041299+0000", 2024-09-18T11:32:13.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:13.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041369+0000", 2024-09-18T11:32:13.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9838000000000005e-05 2024-09-18T11:32:13.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041380+0000", 2024-09-18T11:32:13.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0910000000000001e-05 2024-09-18T11:32:13.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041385+0000", 2024-09-18T11:32:13.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5800000000000002e-06 2024-09-18T11:32:13.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044260+0000", 2024-09-18T11:32:13.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028751229999999998 2024-09-18T11:32:13.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044312+0000", 2024-09-18T11:32:13.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1808000000000003e-05 2024-09-18T11:32:13.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:13.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044393+0000", 2024-09-18T11:32:13.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1806999999999998e-05 2024-09-18T11:32:13.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:13.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052654+0000", 2024-09-18T11:32:13.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082609320000000003 2024-09-18T11:32:13.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:13.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054207+0000", 2024-09-18T11:32:13.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001552091 2024-09-18T11:32:13.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054256+0000", 2024-09-18T11:32:13.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9437000000000001e-05 2024-09-18T11:32:13.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054298+0000", 2024-09-18T11:32:13.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1993000000000003e-05 2024-09-18T11:32:13.072 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.072 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.072 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.072 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3934 4.b 4:d3ae8215:::smithi11939506-41:head [setxattr _header (57) in=7b,truncate 3832029] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:13.073 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:13.073 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9464807690000008, 2024-09-18T11:32:13.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013018336, 2024-09-18T11:32:13.073 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.073 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.073 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:13.073 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:13.073 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:13.073 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3934 2024-09-18T11:32:13.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.073 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:13.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:13.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:13.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:13.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041391+0000", 2024-09-18T11:32:13.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.26e-06 2024-09-18T11:32:13.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041395+0000", 2024-09-18T11:32:13.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8999999999999999e-06 2024-09-18T11:32:13.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041397+0000", 2024-09-18T11:32:13.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3149999999999999e-06 2024-09-18T11:32:13.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045266+0000", 2024-09-18T11:32:13.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003869094 2024-09-18T11:32:13.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045316+0000", 2024-09-18T11:32:13.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0074000000000002e-05 2024-09-18T11:32:13.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045403+0000", 2024-09-18T11:32:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6737999999999995e-05 2024-09-18T11:32:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052659+0000", 2024-09-18T11:32:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072558259999999999 2024-09-18T11:32:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054330+0000", 2024-09-18T11:32:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016716000000000001 2024-09-18T11:32:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054362+0000", 2024-09-18T11:32:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1195000000000002e-05 2024-09-18T11:32:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054407+0000", 2024-09-18T11:32:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5707000000000001e-05 2024-09-18T11:32:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3935 4.b 4:d3ae8215:::smithi11939506-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:32:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9464701790000003, 2024-09-18T11:32:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013142333000000001, 2024-09-18T11:32:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3935 2024-09-18T11:32:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:32:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:32:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041399+0000", 2024-09-18T11:32:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041401+0000", 2024-09-18T11:32:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4890000000000001e-06 2024-09-18T11:32:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041404+0000", 2024-09-18T11:32:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5240000000000001e-06 2024-09-18T11:32:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041407+0000", 2024-09-18T11:32:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2230000000000001e-06 2024-09-18T11:32:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045699+0000", 2024-09-18T11:32:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042918970000000002 2024-09-18T11:32:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045750+0000", 2024-09-18T11:32:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1332999999999998e-05 2024-09-18T11:32:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054438+0000", 2024-09-18T11:32:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086882569999999996 2024-09-18T11:32:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054485+0000", 2024-09-18T11:32:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7151000000000003e-05 2024-09-18T11:32:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054542+0000", 2024-09-18T11:32:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6706999999999998e-05 2024-09-18T11:32:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:343 4.a e137/22 4:5975a5ec:::smithi11939506-36:49 v 137'218, mlcod=132'212)", 2024-09-18T11:32:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:08.278418+0000", 2024-09-18T11:32:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.70945167, 2024-09-18T11:32:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012396378, 2024-09-18T11:32:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278418+0000", 2024-09-18T11:32:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278418+0000", 2024-09-18T11:32:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278418+0000", 2024-09-18T11:32:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278422+0000", 2024-09-18T11:32:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.189e-06 2024-09-18T11:32:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278428+0000", 2024-09-18T11:32:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7400000000000001e-06 2024-09-18T11:32:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278431+0000", 2024-09-18T11:32:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4790000000000001e-06 2024-09-18T11:32:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.283059+0000", 2024-09-18T11:32:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046282750000000003 2024-09-18T11:32:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.283072+0000", 2024-09-18T11:32:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2819999999999999e-05 2024-09-18T11:32:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.290780+0000", 2024-09-18T11:32:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077080990000000004 2024-09-18T11:32:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.290815+0000", 2024-09-18T11:32:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4224000000000003e-05 2024-09-18T11:32:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3969 4.4 4:2cb3e6a5:::smithi11939506-3:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:32:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.4142299139999999, 2024-09-18T11:32:13.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.024248042000000001, 2024-09-18T11:32:13.089 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.089 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:13.089 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:13.089 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:13.089 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:13.089 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3969 2024-09-18T11:32:13.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.089 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:32:13.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:32:13.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573639+0000", 2024-09-18T11:32:13.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:13.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573646+0000", 2024-09-18T11:32:13.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6569999999999997e-06 2024-09-18T11:32:13.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573662+0000", 2024-09-18T11:32:13.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5187999999999999e-05 2024-09-18T11:32:13.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573670+0000", 2024-09-18T11:32:13.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5399999999999996e-06 2024-09-18T11:32:13.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8592999999999999e-05 2024-09-18T11:32:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573818+0000", 2024-09-18T11:32:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9388999999999995e-05 2024-09-18T11:32:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597888+0000", 2024-09-18T11:32:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.024069859999999998 2024-09-18T11:32:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3973 4.7 4:e73ba68f:::smithi11939506-1:head [tier-promote] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.4141308349999999, 2024-09-18T11:32:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.023911297000000001, 2024-09-18T11:32:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3973 2024-09-18T11:32:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573738+0000", 2024-09-18T11:32:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573743+0000", 2024-09-18T11:32:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0839999999999999e-06 2024-09-18T11:32:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573752+0000", 2024-09-18T11:32:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3080000000000008e-06 2024-09-18T11:32:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573758+0000", 2024-09-18T11:32:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2659999999999998e-06 2024-09-18T11:32:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573803+0000", 2024-09-18T11:32:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5243e-05 2024-09-18T11:32:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573897+0000", 2024-09-18T11:32:13.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3398000000000003e-05 2024-09-18T11:32:13.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597650+0000", 2024-09-18T11:32:13.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.023753672999999999 2024-09-18T11:32:13.097 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.097 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.097 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.097 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.097 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.097 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:13.099 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:32:13.100 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is set 2024-09-18T11:32:13.100 INFO:tasks.rados.rados.0.smithi119.stdout:1756: expect (ObjNum 320 snap 44 seq_num 320) 2024-09-18T11:32:13.100 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:13.096+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:13.199 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:13.198+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:13.238 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:13.239 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:13.239 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:13.239 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:13.247 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:32:13.287 INFO:tasks.rados.rados.0.smithi119.stdout:1760: expect (ObjNum 352 snap 54 seq_num 352) 2024-09-18T11:32:13.300 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:13.299+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:13.347 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:13.347 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:13.347 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:13.347 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:13.347 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:13.355 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:32:13.401 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:13.399+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:13.495 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:13.495 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:13.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:13.495 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:13.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.495 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:32:13.495 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:13.495 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.50967356, 2024-09-18T11:32:13.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:32:13.496 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.496 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.496 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:13.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:13.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:32:13.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:13.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:32:13.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:32:13.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:32:13.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:32:13.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:32:13.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:32:13.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:32:13.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:32:13.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:32:13.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:32:13.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:32:13.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:32:13.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:32:13.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:32:13.500 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.500 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.500 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.500 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:32:13.500 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:13.500 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 104.274842607, 2024-09-18T11:32:13.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:32:13.500 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.500 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.500 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:13.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:13.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:13.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:13.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:32:13.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:32:13.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:32:13.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:32:13.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:32:13.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:32:13.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:32:13.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:32:13.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:32:13.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:32:13.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:32:13.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:32:13.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:32:13.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:32:13.504 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.504 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.504 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.505 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:32:13.505 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:13.505 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.100936321999995, 2024-09-18T11:32:13.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:32:13.505 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.505 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.505 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:13.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:13.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:32:13.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:13.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:32:13.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:32:13.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:32:13.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:32:13.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:32:13.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:32:13.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:32:13.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:32:13.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:32:13.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:32:13.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:32:13.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:32:13.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:32:13.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:32:13.509 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.509 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.509 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.509 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:304 4.6 e69/67 4:67c1765a:::smithi11939506-30:17 v 69'35, mlcod=58'31)", 2024-09-18T11:32:13.509 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:13.509 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 82.147095227999998, 2024-09-18T11:32:13.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006377611, 2024-09-18T11:32:13.509 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.509 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.509 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:13.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:13.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:13.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:13.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345961+0000", 2024-09-18T11:32:13.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2050000000000008e-06 2024-09-18T11:32:13.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345969+0000", 2024-09-18T11:32:13.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0339999999999997e-06 2024-09-18T11:32:13.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345976+0000", 2024-09-18T11:32:13.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-18T11:32:13.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346026+0000", 2024-09-18T11:32:13.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9286999999999998e-05 2024-09-18T11:32:13.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346050+0000", 2024-09-18T11:32:13.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4397e-05 2024-09-18T11:32:13.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352293+0000", 2024-09-18T11:32:13.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062427170000000001 2024-09-18T11:32:13.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352330+0000", 2024-09-18T11:32:13.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7348999999999998e-05 2024-09-18T11:32:13.513 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.513 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.513 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.513 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:347 4.d e69/22 4:b44229a1:::smithi11939506-42:19 v 69'99, mlcod=65'93)", 2024-09-18T11:32:13.513 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:13.514 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 82.141362032999993, 2024-09-18T11:32:13.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056998760000000004, 2024-09-18T11:32:13.514 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.514 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.514 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:13.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:13.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351684+0000", 2024-09-18T11:32:13.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:13.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351693+0000", 2024-09-18T11:32:13.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-18T11:32:13.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351704+0000", 2024-09-18T11:32:13.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1077000000000001e-05 2024-09-18T11:32:13.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351709+0000", 2024-09-18T11:32:13.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5419999999999999e-06 2024-09-18T11:32:13.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351956+0000", 2024-09-18T11:32:13.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000247252 2024-09-18T11:32:13.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351975+0000", 2024-09-18T11:32:13.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9201000000000001e-05 2024-09-18T11:32:13.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357334+0000", 2024-09-18T11:32:13.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053588749999999999 2024-09-18T11:32:13.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357385+0000", 2024-09-18T11:32:13.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1406999999999998e-05 2024-09-18T11:32:13.518 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.518 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.518 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.518 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1956 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'37, mlcod=58'31)", 2024-09-18T11:32:13.518 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:13.518 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 82.140417119999995, 2024-09-18T11:32:13.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010958842, 2024-09-18T11:32:13.518 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.518 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.518 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:13.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:13.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:13.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:13.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352996+0000", 2024-09-18T11:32:13.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036622 2024-09-18T11:32:13.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353011+0000", 2024-09-18T11:32:13.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4849e-05 2024-09-18T11:32:13.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353017+0000", 2024-09-18T11:32:13.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:32:13.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353068+0000", 2024-09-18T11:32:13.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0695000000000002e-05 2024-09-18T11:32:13.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353092+0000", 2024-09-18T11:32:13.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4260000000000002e-05 2024-09-18T11:32:13.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363551+0000", 2024-09-18T11:32:13.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010459154999999999 2024-09-18T11:32:13.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363589+0000", 2024-09-18T11:32:13.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8161000000000003e-05 2024-09-18T11:32:13.522 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.522 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.522 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.522 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1957 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'38, mlcod=65'33)", 2024-09-18T11:32:13.522 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:13.522 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 82.134001252999994, 2024-09-18T11:32:13.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050499400000000002, 2024-09-18T11:32:13.523 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.523 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.523 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:13.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:13.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:13.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:13.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359214+0000", 2024-09-18T11:32:13.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000168047 2024-09-18T11:32:13.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359227+0000", 2024-09-18T11:32:13.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3124e-05 2024-09-18T11:32:13.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359235+0000", 2024-09-18T11:32:13.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0760000000000003e-06 2024-09-18T11:32:13.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359287+0000", 2024-09-18T11:32:13.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.151e-05 2024-09-18T11:32:13.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359310+0000", 2024-09-18T11:32:13.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-09-18T11:32:13.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364065+0000", 2024-09-18T11:32:13.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047549439999999997 2024-09-18T11:32:13.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364096+0000", 2024-09-18T11:32:13.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1423000000000001e-05 2024-09-18T11:32:13.526 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.526 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.526 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.527 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2300 4.4 e77/22 4:2c93196f:::smithi11939506-28:head v 77'94, mlcod=77'90)", 2024-09-18T11:32:13.527 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:13.527 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.504939035000007, 2024-09-18T11:32:13.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050425380000000001, 2024-09-18T11:32:13.527 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.527 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.527 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:13.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:13.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988108+0000", 2024-09-18T11:32:13.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:13.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988388+0000", 2024-09-18T11:32:13.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028087700000000001 2024-09-18T11:32:13.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988405+0000", 2024-09-18T11:32:13.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6565000000000001e-05 2024-09-18T11:32:13.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988412+0000", 2024-09-18T11:32:13.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9709999999999998e-06 2024-09-18T11:32:13.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988464+0000", 2024-09-18T11:32:13.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1949e-05 2024-09-18T11:32:13.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988490+0000", 2024-09-18T11:32:13.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5780999999999999e-05 2024-09-18T11:32:13.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993112+0000", 2024-09-18T11:32:13.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046228399999999996 2024-09-18T11:32:13.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993151+0000", 2024-09-18T11:32:13.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8587999999999997e-05 2024-09-18T11:32:13.531 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.531 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.531 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.531 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.13:63 3.0 e78/21 3:24ebb740:::smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 78'80, mlcod=73'78)", 2024-09-18T11:32:13.532 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:13.532 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.087949619, 2024-09-18T11:32:13.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052262230000000003, 2024-09-18T11:32:13.532 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.532 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.532 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:13.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:13.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405097+0000", 2024-09-18T11:32:13.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:13.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405103+0000", 2024-09-18T11:32:13.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8780000000000003e-06 2024-09-18T11:32:13.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405117+0000", 2024-09-18T11:32:13.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3441000000000001e-05 2024-09-18T11:32:13.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405121+0000", 2024-09-18T11:32:13.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7099999999999998e-06 2024-09-18T11:32:13.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405158+0000", 2024-09-18T11:32:13.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6477999999999999e-05 2024-09-18T11:32:13.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405173+0000", 2024-09-18T11:32:13.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4796e-05 2024-09-18T11:32:13.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410285+0000", 2024-09-18T11:32:13.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051118919999999998 2024-09-18T11:32:13.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410324+0000", 2024-09-18T11:32:13.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9616999999999997e-05 2024-09-18T11:32:13.536 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.536 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.536 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.536 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.13:18 3.0 e91/21 3:35e3c617:::2e64dd273da7619bd64b294f52905c1b9405a1ced2a8fd3c0b5bc481bc9732a6:head v 91'89, mlcod=85'87)", 2024-09-18T11:32:13.536 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:13.536 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.471174208000001, 2024-09-18T11:32:13.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048558539999999997, 2024-09-18T11:32:13.537 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.537 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.537 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:13.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:13.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:13.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:13.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021880+0000", 2024-09-18T11:32:13.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5860000000000001e-06 2024-09-18T11:32:13.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021890+0000", 2024-09-18T11:32:13.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8290000000000003e-06 2024-09-18T11:32:13.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021895+0000", 2024-09-18T11:32:13.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2480000000000004e-06 2024-09-18T11:32:13.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022166+0000", 2024-09-18T11:32:13.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027073499999999998 2024-09-18T11:32:13.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022184+0000", 2024-09-18T11:32:13.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8082000000000001e-05 2024-09-18T11:32:13.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026693+0000", 2024-09-18T11:32:13.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045093479999999998 2024-09-18T11:32:13.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026729+0000", 2024-09-18T11:32:13.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6282999999999998e-05 2024-09-18T11:32:13.541 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.541 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.541 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.541 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:573 4.4 e91/22 4:2c93196f:::smithi11939506-28:26 v 91'135, mlcod=90'131)", 2024-09-18T11:32:13.541 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:13.541 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.468707619, 2024-09-18T11:32:13.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050722739999999999, 2024-09-18T11:32:13.541 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.541 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.541 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:13.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:13.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024339+0000", 2024-09-18T11:32:13.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:13.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024349+0000", 2024-09-18T11:32:13.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6339999999999996e-06 2024-09-18T11:32:13.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024362+0000", 2024-09-18T11:32:13.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-18T11:32:13.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024368+0000", 2024-09-18T11:32:13.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8649999999999998e-06 2024-09-18T11:32:13.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024431+0000", 2024-09-18T11:32:13.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2780999999999994e-05 2024-09-18T11:32:13.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024448+0000", 2024-09-18T11:32:13.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7388999999999999e-05 2024-09-18T11:32:13.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029385+0000", 2024-09-18T11:32:13.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004936523 2024-09-18T11:32:13.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029412+0000", 2024-09-18T11:32:13.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7702999999999999e-05 2024-09-18T11:32:13.545 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.545 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.545 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.546 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:574 4.4 e91/22 4:2cb3e6a5:::smithi11939506-3:1e v 91'137, mlcod=90'131)", 2024-09-18T11:32:13.546 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:13.546 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.468269575999997, 2024-09-18T11:32:13.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070802850000000004, 2024-09-18T11:32:13.546 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.546 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.546 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:13.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:13.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024777+0000", 2024-09-18T11:32:13.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:13.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024785+0000", 2024-09-18T11:32:13.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8099999999999998e-06 2024-09-18T11:32:13.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024794+0000", 2024-09-18T11:32:13.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5839999999999995e-06 2024-09-18T11:32:13.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024798+0000", 2024-09-18T11:32:13.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.031e-06 2024-09-18T11:32:13.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024830+0000", 2024-09-18T11:32:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2268999999999999e-05 2024-09-18T11:32:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024847+0000", 2024-09-18T11:32:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7306999999999999e-05 2024-09-18T11:32:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031834+0000", 2024-09-18T11:32:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069868380000000004 2024-09-18T11:32:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031858+0000", 2024-09-18T11:32:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4082000000000001e-05 2024-09-18T11:32:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:541 4.d e91/22 4:b9bb642b:::smithi11939506-27:19 v 91'158, mlcod=91'154)", 2024-09-18T11:32:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.460888883999999, 2024-09-18T11:32:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052185, 2024-09-18T11:32:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.550 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.551 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:13.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:13.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032158+0000", 2024-09-18T11:32:13.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:13.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032165+0000", 2024-09-18T11:32:13.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-09-18T11:32:13.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032176+0000", 2024-09-18T11:32:13.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1460999999999999e-05 2024-09-18T11:32:13.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032182+0000", 2024-09-18T11:32:13.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0870000000000003e-06 2024-09-18T11:32:13.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032224+0000", 2024-09-18T11:32:13.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1372999999999998e-05 2024-09-18T11:32:13.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032246+0000", 2024-09-18T11:32:13.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2158e-05 2024-09-18T11:32:13.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037348+0000", 2024-09-18T11:32:13.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051025319999999999 2024-09-18T11:32:13.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037377+0000", 2024-09-18T11:32:13.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8937000000000001e-05 2024-09-18T11:32:13.554 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.554 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.554 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.555 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4178.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-18T11:32:13.555 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:13.555 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.552226819000001, 2024-09-18T11:32:13.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048960139999999997, 2024-09-18T11:32:13.555 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.555 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:13.555 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:13.555 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4178", 2024-09-18T11:32:13.555 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/490761693", 2024-09-18T11:32:13.555 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4 2024-09-18T11:32:13.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.555 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:13.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:13.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940819+0000", 2024-09-18T11:32:13.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:13.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940832+0000", 2024-09-18T11:32:13.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3155e-05 2024-09-18T11:32:13.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940848+0000", 2024-09-18T11:32:13.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6288999999999999e-05 2024-09-18T11:32:13.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940856+0000", 2024-09-18T11:32:13.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4919999999999996e-06 2024-09-18T11:32:13.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945613+0000", 2024-09-18T11:32:13.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047574999999999996 2024-09-18T11:32:13.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945717+0000", 2024-09-18T11:32:13.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103553 2024-09-18T11:32:13.559 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.559 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.559 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.559 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3482 4.6 e117/67 4:67c1765a:::smithi11939506-30:head v 117'120, mlcod=115'116)", 2024-09-18T11:32:13.559 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:13.559 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.808372596000002, 2024-09-18T11:32:13.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050037479999999997, 2024-09-18T11:32:13.559 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.559 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.559 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:13.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:13.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684674+0000", 2024-09-18T11:32:13.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:13.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684950+0000", 2024-09-18T11:32:13.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027621200000000002 2024-09-18T11:32:13.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684968+0000", 2024-09-18T11:32:13.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8e-05 2024-09-18T11:32:13.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684984+0000", 2024-09-18T11:32:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5775000000000001e-05 2024-09-18T11:32:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685033+0000", 2024-09-18T11:32:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8878000000000002e-05 2024-09-18T11:32:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685059+0000", 2024-09-18T11:32:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6801999999999999e-05 2024-09-18T11:32:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689640+0000", 2024-09-18T11:32:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045809029999999999 2024-09-18T11:32:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689679+0000", 2024-09-18T11:32:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8507000000000002e-05 2024-09-18T11:32:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:976 4.4 e132/22 4:2cb3e6a5:::smithi11939506-3:30 v 132'195, mlcod=128'192)", 2024-09-18T11:32:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 10.474401524999999, 2024-09-18T11:32:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0058526510000000004, 2024-09-18T11:32:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018661+0000", 2024-09-18T11:32:13.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5565000000000001e-05 2024-09-18T11:32:13.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018672+0000", 2024-09-18T11:32:13.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0210000000000001e-05 2024-09-18T11:32:13.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018678+0000", 2024-09-18T11:32:13.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1539999999999999e-06 2024-09-18T11:32:13.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018716+0000", 2024-09-18T11:32:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8389000000000001e-05 2024-09-18T11:32:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018737+0000", 2024-09-18T11:32:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0611e-05 2024-09-18T11:32:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.024458+0000", 2024-09-18T11:32:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057208629999999996 2024-09-18T11:32:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.024499+0000", 2024-09-18T11:32:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1100999999999998e-05 2024-09-18T11:32:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:823 4.d e137/22 4:b44229a1:::smithi11939506-42:49 v 137'266, mlcod=132'260)", 2024-09-18T11:32:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:32:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.2157529690000004, 2024-09-18T11:32:13.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054497770000000003, 2024-09-18T11:32:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:32:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:32:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277294+0000", 2024-09-18T11:32:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277299+0000", 2024-09-18T11:32:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6280000000000004e-06 2024-09-18T11:32:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277306+0000", 2024-09-18T11:32:13.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.533e-06 2024-09-18T11:32:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277316+0000", 2024-09-18T11:32:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0148e-05 2024-09-18T11:32:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277868+0000", 2024-09-18T11:32:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00055199999999999997 2024-09-18T11:32:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277898+0000", 2024-09-18T11:32:13.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0264999999999999e-05 2024-09-18T11:32:13.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.282715+0000", 2024-09-18T11:32:13.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048165619999999999 2024-09-18T11:32:13.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.282744+0000", 2024-09-18T11:32:13.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9855000000000001e-05 2024-09-18T11:32:13.572 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.572 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.573 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:163 3.0 e138/21 3:02e8b30e:::4456d047c09ae2a9a25ccebfd83e30cf78df6779d63bfba9365afd7301eaeb6a:head v 138'132, mlcod=134'130)", 2024-09-18T11:32:13.573 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:13.573 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.9108902260000002, 2024-09-18T11:32:13.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051351770000000003, 2024-09-18T11:32:13.573 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.573 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.573 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:13.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:13.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:13.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:13.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582232+0000", 2024-09-18T11:32:13.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5333999999999994e-05 2024-09-18T11:32:13.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582242+0000", 2024-09-18T11:32:13.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5200000000000003e-06 2024-09-18T11:32:13.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582248+0000", 2024-09-18T11:32:13.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8000000000000004e-06 2024-09-18T11:32:13.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582305+0000", 2024-09-18T11:32:13.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7813999999999999e-05 2024-09-18T11:32:13.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582329+0000", 2024-09-18T11:32:13.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3414999999999999e-05 2024-09-18T11:32:13.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587257+0000", 2024-09-18T11:32:13.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049279880000000003 2024-09-18T11:32:13.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587293+0000", 2024-09-18T11:32:13.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5802e-05 2024-09-18T11:32:13.577 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.577 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.577 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.577 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3974 4.d e138/22 4:b5a463bf:::smithi11939506-20:head v 138'270, mlcod=137'266)", 2024-09-18T11:32:13.577 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:32:13.578 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.9070089550000002, 2024-09-18T11:32:13.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011228458, 2024-09-18T11:32:13.578 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.578 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.578 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:32:13.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:32:13.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586038+0000", 2024-09-18T11:32:13.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:13.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587638+0000", 2024-09-18T11:32:13.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001599184 2024-09-18T11:32:13.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587647+0000", 2024-09-18T11:32:13.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3719999999999994e-06 2024-09-18T11:32:13.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587653+0000", 2024-09-18T11:32:13.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.105e-06 2024-09-18T11:32:13.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587712+0000", 2024-09-18T11:32:13.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9252000000000001e-05 2024-09-18T11:32:13.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587735+0000", 2024-09-18T11:32:13.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3132000000000001e-05 2024-09-18T11:32:13.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597228+0000", 2024-09-18T11:32:13.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0094930090000000002 2024-09-18T11:32:13.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597267+0000", 2024-09-18T11:32:13.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8693999999999999e-05 2024-09-18T11:32:13.581 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.581 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.581 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.582 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3975 4.d e138/22 4:b5a463bf:::smithi11939506-20:head v 138'271, mlcod=137'266)", 2024-09-18T11:32:13.582 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:13.582 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.9008082489999998, 2024-09-18T11:32:13.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053620450000000002, 2024-09-18T11:32:13.582 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:13.582 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:13.582 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:13.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:13.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:13.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:13.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:13.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:13.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:13.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:13.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:13.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:13.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592927+0000", 2024-09-18T11:32:13.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00068878699999999995 2024-09-18T11:32:13.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:13.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592940+0000", 2024-09-18T11:32:13.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:32:13.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:13.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592951+0000", 2024-09-18T11:32:13.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1046000000000001e-05 2024-09-18T11:32:13.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:13.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593008+0000", 2024-09-18T11:32:13.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6875e-05 2024-09-18T11:32:13.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:13.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593033+0000", 2024-09-18T11:32:13.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5341999999999999e-05 2024-09-18T11:32:13.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:13.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597565+0000", 2024-09-18T11:32:13.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045323150000000003 2024-09-18T11:32:13.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:13.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:13.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:13.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597601+0000", 2024-09-18T11:32:13.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6071000000000001e-05 2024-09-18T11:32:13.586 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.586 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.586 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.586 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:13.586 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:13.586 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:13.588 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:32:13.588 INFO:tasks.rados.rados.0.smithi119.stdout:1759: expect (ObjNum 328 snap 45 seq_num 328) 2024-09-18T11:32:13.588 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:13.585+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:13.688 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:13.686+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:13.695 INFO:tasks.rados.rados.0.smithi119.stdout:1761: finishing set_chunk to oid 21 2024-09-18T11:32:13.695 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 269 (ObjNum 311 snap 43 seq_num 311) dirty exists 2024-09-18T11:32:13.695 INFO:tasks.rados.rados.0.smithi119.stdout:1762: finishing write tid 1 to smithi11939506-10 2024-09-18T11:32:13.695 INFO:tasks.rados.rados.0.smithi119.stdout:1753: done (15 left) 2024-09-18T11:32:13.695 INFO:tasks.rados.rados.0.smithi119.stdout:1754: done (14 left) 2024-09-18T11:32:13.696 INFO:tasks.rados.rados.0.smithi119.stdout:1755: done (13 left) 2024-09-18T11:32:13.696 INFO:tasks.rados.rados.0.smithi119.stdout:1756: done (12 left) 2024-09-18T11:32:13.696 INFO:tasks.rados.rados.0.smithi119.stdout:1757: done (11 left) 2024-09-18T11:32:13.696 INFO:tasks.rados.rados.0.smithi119.stdout:1759: done (10 left) 2024-09-18T11:32:13.696 INFO:tasks.rados.rados.0.smithi119.stdout:1760: done (9 left) 2024-09-18T11:32:13.696 INFO:tasks.rados.rados.0.smithi119.stdout:1761: done (8 left) 2024-09-18T11:32:13.696 INFO:tasks.rados.rados.0.smithi119.stdout:1770: read oid 23 snap -1 2024-09-18T11:32:13.696 INFO:tasks.rados.rados.0.smithi119.stdout:1770: expect (ObjNum 332 snap 48 seq_num 332) 2024-09-18T11:32:13.696 INFO:tasks.rados.rados.0.smithi119.stdout:1771: tier_flush oid 24 2024-09-18T11:32:13.696 INFO:tasks.rados.rados.0.smithi119.stdout:1771: tier_flush oid 24 snap -1 2024-09-18T11:32:13.696 INFO:tasks.rados.rados.0.smithi119.stdout:1764: finishing set_chunk to oid 30 2024-09-18T11:32:13.696 INFO:tasks.rados.rados.0.smithi119.stdout:1764: got expected ERANGE (src dne) 2024-09-18T11:32:13.696 INFO:tasks.rados.rados.0.smithi119.stdout:1762: finishing write tid 2 to smithi11939506-10 2024-09-18T11:32:13.697 INFO:tasks.rados.rados.0.smithi119.stdout:1762: finishing write tid 3 to smithi11939506-10 2024-09-18T11:32:13.697 INFO:tasks.rados.rados.0.smithi119.stdout:1762: finishing write tid 4 to smithi11939506-10 2024-09-18T11:32:13.697 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 10 v 287 (ObjNum 358 snap 55 seq_num 358) dirty exists 2024-09-18T11:32:13.697 INFO:tasks.rados.rados.0.smithi119.stdout:1762: left oid 10 (ObjNum 358 snap 55 seq_num 358) 2024-09-18T11:32:13.697 INFO:tasks.rados.rados.0.smithi119.stdout:1765: expect (ObjNum 327 snap 45 seq_num 327) 2024-09-18T11:32:13.729 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:13.730 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:13.730 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:13.730 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:13.738 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:32:13.812 INFO:tasks.rados.rados.0.smithi119.stdout:1763: expect (ObjNum 205 snap 23 seq_num 205) 2024-09-18T11:32:13.813 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:13.811+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:13.876 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:13.876 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:13.876 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:13.876 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:13.877 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:13.885 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:32:13.890 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:13.889+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:13.957 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:32:14.004 INFO:tasks.rados.rados.0.smithi119.stdout:1766: finishing write tid 1 to smithi11939506-16 2024-09-18T11:32:14.004 INFO:tasks.rados.rados.0.smithi119.stdout:1767: expect (ObjNum 287 snap 41 seq_num 287) 2024-09-18T11:32:14.004 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:14.002+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:14.030 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.2 to 0.28049 (47ce) 2024-09-18T11:32:14.031 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:14.032 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:14.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:14.032 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:14.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.032 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:32:14.032 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:14.032 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 122.859222587, 2024-09-18T11:32:14.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:32:14.032 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:14.032 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:14.032 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:14.032 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:14.033 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:14.033 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:32:14.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.033 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:14.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:14.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:14.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:14.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:14.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:14.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:32:14.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:14.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:14.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:32:14.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:32:14.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:14.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:32:14.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:32:14.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:14.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:32:14.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:32:14.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:32:14.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:32:14.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:14.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:32:14.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:32:14.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:14.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:32:14.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:32:14.037 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.037 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:14.037 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.037 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1001 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:14.037 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:32:14.037 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.864071309, 2024-09-18T11:32:14.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007461082, 2024-09-18T11:32:14.037 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:14.037 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:14.038 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:14.038 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:14.038 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:14.038 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1001 2024-09-18T11:32:14.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.038 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:14.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:14.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:32:14.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:14.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:32:14.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:14.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164570+0000", 2024-09-18T11:32:14.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:14.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:14.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164574+0000", 2024-09-18T11:32:14.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.822e-06 2024-09-18T11:32:14.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:14.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164582+0000", 2024-09-18T11:32:14.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3410000000000003e-06 2024-09-18T11:32:14.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:14.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164587+0000", 2024-09-18T11:32:14.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769999999999999e-06 2024-09-18T11:32:14.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164702+0000", 2024-09-18T11:32:14.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115705 2024-09-18T11:32:14.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:14.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164709+0000", 2024-09-18T11:32:14.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-09-18T11:32:14.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170296+0000", 2024-09-18T11:32:14.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055876390000000001 2024-09-18T11:32:14.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:14.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170418+0000", 2024-09-18T11:32:14.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121549 2024-09-18T11:32:14.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:14.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172032+0000", 2024-09-18T11:32:14.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00161407 2024-09-18T11:32:14.042 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.042 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:14.042 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.043 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:14.043 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:14.043 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.864044593, 2024-09-18T11:32:14.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:32:14.043 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:14.043 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:14.043 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:14.043 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:14.043 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:14.043 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:32:14.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.044 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:14.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:14.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:14.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:14.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:14.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:14.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:14.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:14.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:14.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:32:14.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:32:14.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:14.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:32:14.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:32:14.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:14.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:32:14.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:32:14.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:32:14.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:32:14.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:14.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:32:14.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:32:14.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:32:14.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:32:14.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:14.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:32:14.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:32:14.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:14.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:32:14.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:32:14.047 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.048 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:14.048 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.048 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:14.048 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:14.048 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.86402630400001, 2024-09-18T11:32:14.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:32:14.048 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:14.048 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:14.048 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:14.048 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:14.048 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:14.049 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:32:14.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.049 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:14.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:14.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:14.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:14.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:14.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:14.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:32:14.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:14.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:14.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:32:14.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:32:14.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:14.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:32:14.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:32:14.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:14.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:32:14.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:32:14.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:32:14.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:32:14.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:14.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:32:14.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:32:14.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:32:14.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:32:14.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:14.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:32:14.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:32:14.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:14.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:32:14.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:32:14.053 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.053 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:14.053 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.053 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:14.053 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:14.053 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.86396553500001, 2024-09-18T11:32:14.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:32:14.053 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:14.054 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:14.054 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:14.054 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:14.054 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:14.054 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:32:14.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.054 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:14.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:14.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:14.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:14.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:14.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:14.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:32:14.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:14.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:14.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:32:14.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:32:14.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:14.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:32:14.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:32:14.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:14.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:32:14.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:32:14.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:32:14.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:32:14.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:14.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:32:14.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:32:14.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:32:14.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:32:14.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:14.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:32:14.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:32:14.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:14.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:32:14.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:32:14.058 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.058 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:14.058 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.059 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:14.059 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:14.059 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.863943214, 2024-09-18T11:32:14.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:32:14.059 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:14.059 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:14.059 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:14.059 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:14.059 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:14.059 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:32:14.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.059 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:14.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:14.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:14.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:14.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:14.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:14.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:32:14.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:14.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:14.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:32:14.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:32:14.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:14.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:32:14.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:32:14.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:14.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:32:14.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:32:14.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:32:14.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:32:14.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:14.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:32:14.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:32:14.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:32:14.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:32:14.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:14.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:32:14.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:32:14.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:14.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:32:14.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:32:14.063 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.064 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:14.064 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.064 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:32:14.064 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:14.064 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.082818331, 2024-09-18T11:32:14.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:32:14.064 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:14.064 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:14.064 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:14.064 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:14.065 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:14.065 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:32:14.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.065 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:14.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:14.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:14.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:14.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:14.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:14.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:32:14.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:32:14.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:14.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:32:14.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:32:14.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:14.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:32:14.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:32:14.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:14.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:32:14.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:32:14.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:32:14.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:32:14.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:14.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:32:14.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:32:14.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:14.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:32:14.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:32:14.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:14.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:32:14.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:32:14.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:14.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:32:14.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:32:14.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:14.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:32:14.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:32:14.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:14.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:32:14.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:32:14.069 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.070 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:14.070 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.070 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1661 4.0 4:019b6e09:::smithi11939506-40:17 [read 0~3445402,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:32:14.070 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:14.070 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.678745628000001, 2024-09-18T11:32:14.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075190880000000002, 2024-09-18T11:32:14.070 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:14.070 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:14.070 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:14.070 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:14.070 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:14.071 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1661 2024-09-18T11:32:14.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.071 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:14.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:14.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:14.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:14.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:14.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:14.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:14.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:14.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:14.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349899+0000", 2024-09-18T11:32:14.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9749999999999999e-06 2024-09-18T11:32:14.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:14.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349906+0000", 2024-09-18T11:32:14.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9290000000000001e-06 2024-09-18T11:32:14.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:14.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349912+0000", 2024-09-18T11:32:14.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9900000000000002e-06 2024-09-18T11:32:14.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355645+0000", 2024-09-18T11:32:14.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057329549999999996 2024-09-18T11:32:14.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:14.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355776+0000", 2024-09-18T11:32:14.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130975 2024-09-18T11:32:14.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:14.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357416+0000", 2024-09-18T11:32:14.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001639649 2024-09-18T11:32:14.074 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.074 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:14.074 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.074 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1761 4.0 4:0aec5e1c:::smithi11939506-50:head [write 769787~704463,stat] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:32:14.075 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:32:14.075 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.726253213999996, 2024-09-18T11:32:14.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074082879999999999, 2024-09-18T11:32:14.075 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:14.075 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:14.075 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:14.075 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:14.075 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:14.075 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1761 2024-09-18T11:32:14.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.075 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:14.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:14.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:32:14.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:14.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:32:14.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:14.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302386+0000", 2024-09-18T11:32:14.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:14.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:14.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302889+0000", 2024-09-18T11:32:14.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050318499999999996 2024-09-18T11:32:14.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:14.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302930+0000", 2024-09-18T11:32:14.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0198999999999999e-05 2024-09-18T11:32:14.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:14.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302965+0000", 2024-09-18T11:32:14.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5292e-05 2024-09-18T11:32:14.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303046+0000", 2024-09-18T11:32:14.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1254999999999994e-05 2024-09-18T11:32:14.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:32:14.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303054+0000", 2024-09-18T11:32:14.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2800000000000003e-06 2024-09-18T11:32:14.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304658+0000", 2024-09-18T11:32:14.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016039520000000001 2024-09-18T11:32:14.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:14.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304724+0000", 2024-09-18T11:32:14.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5888000000000003e-05 2024-09-18T11:32:14.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:14.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304853+0000", 2024-09-18T11:32:14.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012912700000000001 2024-09-18T11:32:14.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:14.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309084+0000", 2024-09-18T11:32:14.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042311229999999998 2024-09-18T11:32:14.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:14.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309738+0000", 2024-09-18T11:32:14.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00065401399999999996 2024-09-18T11:32:14.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:14.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309770+0000", 2024-09-18T11:32:14.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1652000000000001e-05 2024-09-18T11:32:14.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:14.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309797+0000", 2024-09-18T11:32:14.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7052000000000001e-05 2024-09-18T11:32:14.081 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.081 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:14.081 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.082 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2503 3.1 e85/67 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head v 85'117, mlcod=84'115)", 2024-09-18T11:32:14.082 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:14.082 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 65.051419644000006, 2024-09-18T11:32:14.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086527619999999996, 2024-09-18T11:32:14.082 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:14.082 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:14.082 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:14.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:14.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:14.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:14.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:14.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:14.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977220+0000", 2024-09-18T11:32:14.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:14.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:14.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980771+0000", 2024-09-18T11:32:14.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035505200000000002 2024-09-18T11:32:14.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:14.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980797+0000", 2024-09-18T11:32:14.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.618e-05 2024-09-18T11:32:14.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:14.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980807+0000", 2024-09-18T11:32:14.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0433e-05 2024-09-18T11:32:14.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980861+0000", 2024-09-18T11:32:14.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3415999999999998e-05 2024-09-18T11:32:14.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:14.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980885+0000", 2024-09-18T11:32:14.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4212000000000001e-05 2024-09-18T11:32:14.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:14.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985826+0000", 2024-09-18T11:32:14.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049412450000000004 2024-09-18T11:32:14.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:14.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985875+0000", 2024-09-18T11:32:14.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9116000000000002e-05 2024-09-18T11:32:14.086 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.086 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:14.086 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.086 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2648 4.0 4:059d8d95:::smithi11939506-32:head [read 0~3647119,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:14.086 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:32:14.086 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.697331274, 2024-09-18T11:32:14.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0076286949999999996, 2024-09-18T11:32:14.086 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:14.087 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:14.087 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:14.087 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:14.087 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:14.087 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2648 2024-09-18T11:32:14.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.087 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:14.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:14.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:32:14.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:14.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:32:14.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:14.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331309+0000", 2024-09-18T11:32:14.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:14.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:14.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331321+0000", 2024-09-18T11:32:14.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:32:14.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:14.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331340+0000", 2024-09-18T11:32:14.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8227e-05 2024-09-18T11:32:14.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:14.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331348+0000", 2024-09-18T11:32:14.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4640000000000006e-06 2024-09-18T11:32:14.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331406+0000", 2024-09-18T11:32:14.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7445000000000003e-05 2024-09-18T11:32:14.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:14.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331521+0000", 2024-09-18T11:32:14.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115945 2024-09-18T11:32:14.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:14.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338939+0000", 2024-09-18T11:32:14.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074179859999999997 2024-09-18T11:32:14.090 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.090 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:14.090 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.091 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2649 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:14.091 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:14.091 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.697276275, 2024-09-18T11:32:14.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566240000000007, 2024-09-18T11:32:14.091 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:14.091 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:14.091 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:14.091 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:14.091 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:14.091 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2649 2024-09-18T11:32:14.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.092 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:14.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:14.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:14.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:14.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:14.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:14.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331365+0000", 2024-09-18T11:32:14.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:14.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:14.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331369+0000", 2024-09-18T11:32:14.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.551e-06 2024-09-18T11:32:14.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:14.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331377+0000", 2024-09-18T11:32:14.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8769999999999993e-06 2024-09-18T11:32:14.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:14.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331381+0000", 2024-09-18T11:32:14.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.16e-06 2024-09-18T11:32:14.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338973+0000", 2024-09-18T11:32:14.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075918460000000002 2024-09-18T11:32:14.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:14.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339048+0000", 2024-09-18T11:32:14.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5813000000000001e-05 2024-09-18T11:32:14.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:14.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339322+0000", 2024-09-18T11:32:14.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027399899999999998 2024-09-18T11:32:14.095 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.095 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:14.095 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.095 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2650 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:14.095 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:14.096 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.697256056999997, 2024-09-18T11:32:14.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084392180000000001, 2024-09-18T11:32:14.096 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:14.096 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:14.096 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:14.096 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:14.096 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:14.096 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2650 2024-09-18T11:32:14.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.096 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:14.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:14.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:14.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:14.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:14.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:14.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:14.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:14.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:14.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331388+0000", 2024-09-18T11:32:14.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3429999999999998e-06 2024-09-18T11:32:14.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:14.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331394+0000", 2024-09-18T11:32:14.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.19e-06 2024-09-18T11:32:14.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:14.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331398+0000", 2024-09-18T11:32:14.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.495e-06 2024-09-18T11:32:14.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339362+0000", 2024-09-18T11:32:14.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079641799999999995 2024-09-18T11:32:14.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:14.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339460+0000", 2024-09-18T11:32:14.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8329e-05 2024-09-18T11:32:14.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:14.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339825+0000", 2024-09-18T11:32:14.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036510500000000001 2024-09-18T11:32:14.100 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.100 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:14.100 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.100 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:119 3.2 3:5595a68d:::07854d2fef297a06ba81685e660c332de36d5d18d546927d30daad6d7fda1541:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:14.100 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:32:14.100 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 50.985104300000003, 2024-09-18T11:32:14.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074145979999999997, 2024-09-18T11:32:14.101 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:14.101 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:14.101 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:14.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:14.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:32:14.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:14.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:32:14.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:14.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043535+0000", 2024-09-18T11:32:14.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:14.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:14.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043970+0000", 2024-09-18T11:32:14.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000435148 2024-09-18T11:32:14.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:14.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043993+0000", 2024-09-18T11:32:14.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3113999999999999e-05 2024-09-18T11:32:14.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:14.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044003+0000", 2024-09-18T11:32:14.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5310000000000007e-06 2024-09-18T11:32:14.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044062+0000", 2024-09-18T11:32:14.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9487000000000002e-05 2024-09-18T11:32:14.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:32:14.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044071+0000", 2024-09-18T11:32:14.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1610000000000003e-06 2024-09-18T11:32:14.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046423+0000", 2024-09-18T11:32:14.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023523680000000001 2024-09-18T11:32:14.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:14.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046502+0000", 2024-09-18T11:32:14.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9339999999999996e-05 2024-09-18T11:32:14.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:14.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046664+0000", 2024-09-18T11:32:14.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016185700000000001 2024-09-18T11:32:14.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:14.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050870+0000", 2024-09-18T11:32:14.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042060070000000003 2024-09-18T11:32:14.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:14.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050878+0000", 2024-09-18T11:32:14.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8960000000000003e-06 2024-09-18T11:32:14.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:14.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050918+0000", 2024-09-18T11:32:14.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0367000000000002e-05 2024-09-18T11:32:14.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:14.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050952+0000", 2024-09-18T11:32:14.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3989999999999998e-05 2024-09-18T11:32:14.107 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.107 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:14.107 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.107 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:127 3.2 3:4d3b277a:::3266c7aaa53a7d2c10c3c54d852d6cbbed49fb041bc0df41a3ac83b56e2b0615:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:14.107 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:14.108 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 50.984241118, 2024-09-18T11:32:14.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093148009999999993, 2024-09-18T11:32:14.108 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:14.108 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:14.108 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:14.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:14.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:14.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:14.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:14.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:14.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044400+0000", 2024-09-18T11:32:14.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:14.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:14.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044442+0000", 2024-09-18T11:32:14.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2247999999999999e-05 2024-09-18T11:32:14.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:14.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044453+0000", 2024-09-18T11:32:14.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0617e-05 2024-09-18T11:32:14.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:14.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044457+0000", 2024-09-18T11:32:14.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2520000000000001e-06 2024-09-18T11:32:14.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044495+0000", 2024-09-18T11:32:14.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7710000000000003e-05 2024-09-18T11:32:14.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:14.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044499+0000", 2024-09-18T11:32:14.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6269999999999999e-06 2024-09-18T11:32:14.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048239+0000", 2024-09-18T11:32:14.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037396719999999999 2024-09-18T11:32:14.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:14.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048351+0000", 2024-09-18T11:32:14.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112342 2024-09-18T11:32:14.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:14.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048545+0000", 2024-09-18T11:32:14.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000193881 2024-09-18T11:32:14.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:14.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.052689+0000", 2024-09-18T11:32:14.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041435780000000002 2024-09-18T11:32:14.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:14.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053654+0000", 2024-09-18T11:32:14.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096558199999999999 2024-09-18T11:32:14.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:14.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053683+0000", 2024-09-18T11:32:14.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8850999999999999e-05 2024-09-18T11:32:14.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:14.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053716+0000", 2024-09-18T11:32:14.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2468000000000002e-05 2024-09-18T11:32:14.114 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.114 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:14.114 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.114 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2963 4.0 4:019b6e09:::smithi11939506-40:head [sparse-read 0~1093084] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:14.114 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:32:14.114 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 50.00123232, 2024-09-18T11:32:14.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072292880000000004, 2024-09-18T11:32:14.114 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:14.114 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:14.114 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:14.114 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:14.115 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:14.115 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2963 2024-09-18T11:32:14.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.115 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:14.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:14.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:32:14.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:14.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:32:14.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:14.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027409+0000", 2024-09-18T11:32:14.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:14.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:14.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027412+0000", 2024-09-18T11:32:14.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.751e-06 2024-09-18T11:32:14.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:14.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027418+0000", 2024-09-18T11:32:14.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.404e-06 2024-09-18T11:32:14.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:14.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027423+0000", 2024-09-18T11:32:14.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7119999999999999e-06 2024-09-18T11:32:14.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034156+0000", 2024-09-18T11:32:14.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067331639999999998 2024-09-18T11:32:14.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:14.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034255+0000", 2024-09-18T11:32:14.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8701000000000002e-05 2024-09-18T11:32:14.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:14.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034639+0000", 2024-09-18T11:32:14.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000384006 2024-09-18T11:32:14.118 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.118 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:14.118 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.118 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3209 4.0 4:019b6e09:::smithi11939506-40:head [write 638732~557432] snapc 42=[42,41,3f,3e,3d,3b,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-18T11:32:14.118 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:14.119 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.930559078999998, 2024-09-18T11:32:14.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091774999999999999, 2024-09-18T11:32:14.119 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:14.119 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:14.119 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:14.119 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:14.119 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:14.119 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3209 2024-09-18T11:32:14.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.119 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:14.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:14.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:14.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:14.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:14.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:14.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098080+0000", 2024-09-18T11:32:14.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:14.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:14.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098676+0000", 2024-09-18T11:32:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059598099999999998 2024-09-18T11:32:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098706+0000", 2024-09-18T11:32:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0020000000000001e-05 2024-09-18T11:32:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098716+0000", 2024-09-18T11:32:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0225e-05 2024-09-18T11:32:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098782+0000", 2024-09-18T11:32:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5530000000000004e-05 2024-09-18T11:32:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098883+0000", 2024-09-18T11:32:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101609 2024-09-18T11:32:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.099098+0000", 2024-09-18T11:32:14.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000215052 2024-09-18T11:32:14.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:14.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.102511+0000", 2024-09-18T11:32:14.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034129429999999999 2024-09-18T11:32:14.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:14.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107165+0000", 2024-09-18T11:32:14.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046531480000000002 2024-09-18T11:32:14.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:14.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107217+0000", 2024-09-18T11:32:14.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2058000000000002e-05 2024-09-18T11:32:14.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:14.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107260+0000", 2024-09-18T11:32:14.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3865000000000001e-05 2024-09-18T11:32:14.124 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.124 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:14.124 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.125 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:162 3.1 e138/67 3:8c826a1f:::e682cfc1e82c88472b9acf1cd97978bc105aa035f10b685bb4cace95809b52bf:head v 138'176, mlcod=136'174)", 2024-09-18T11:32:14.125 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:32:14.125 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.4450853339999998, 2024-09-18T11:32:14.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010445024000000001, 2024-09-18T11:32:14.125 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:14.125 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:14.125 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:14.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:14.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:32:14.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:14.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:32:14.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:14.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583556+0000", 2024-09-18T11:32:14.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:14.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:14.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584899+0000", 2024-09-18T11:32:14.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001343056 2024-09-18T11:32:14.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:14.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584929+0000", 2024-09-18T11:32:14.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0133000000000001e-05 2024-09-18T11:32:14.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:14.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584937+0000", 2024-09-18T11:32:14.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6380000000000004e-06 2024-09-18T11:32:14.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584990+0000", 2024-09-18T11:32:14.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3186000000000002e-05 2024-09-18T11:32:14.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:14.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585012+0000", 2024-09-18T11:32:14.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1474000000000001e-05 2024-09-18T11:32:14.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:14.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593963+0000", 2024-09-18T11:32:14.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089512859999999993 2024-09-18T11:32:14.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:14.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594002+0000", 2024-09-18T11:32:14.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8859999999999997e-05 2024-09-18T11:32:14.129 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.129 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:14.129 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.129 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:166 3.1 e138/67 3:a73e635b:::b6ee478f0f661ec7bac3d17c2ca2fdaff71a47570f64844a7ba79bbb2a0e8d9d:head v 138'177, mlcod=136'174)", 2024-09-18T11:32:14.130 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:32:14.130 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.4435154499999996, 2024-09-18T11:32:14.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089018680000000003, 2024-09-18T11:32:14.130 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:14.130 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:14.130 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:14.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:14.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:32:14.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:14.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:32:14.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:14.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585126+0000", 2024-09-18T11:32:14.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:14.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:14.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585211+0000", 2024-09-18T11:32:14.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5432999999999993e-05 2024-09-18T11:32:14.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585220+0000", 2024-09-18T11:32:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5879999999999998e-06 2024-09-18T11:32:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585225+0000", 2024-09-18T11:32:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8999999999999997e-06 2024-09-18T11:32:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585705+0000", 2024-09-18T11:32:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00048042799999999997 2024-09-18T11:32:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585724+0000", 2024-09-18T11:32:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8352e-05 2024-09-18T11:32:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594008+0000", 2024-09-18T11:32:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082840919999999998 2024-09-18T11:32:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594028+0000", 2024-09-18T11:32:14.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0869e-05 2024-09-18T11:32:14.134 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.134 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:14.134 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.134 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:168 3.1 e138/67 3:b64d7bad:::7feb22256bdcec217720fd46184ab43ced8923ed66ec6319c57f078d9f123262:head v 138'178, mlcod=136'174)", 2024-09-18T11:32:14.134 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:14.134 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.4425237170000003, 2024-09-18T11:32:14.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079290650000000008, 2024-09-18T11:32:14.134 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:14.134 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:14.134 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:14.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:14.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:14.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:14.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:14.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:14.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:14.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:14.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:14.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:14.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586191+0000", 2024-09-18T11:32:14.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3627000000000005e-05 2024-09-18T11:32:14.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:14.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586202+0000", 2024-09-18T11:32:14.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0488e-05 2024-09-18T11:32:14.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:14.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586207+0000", 2024-09-18T11:32:14.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1719999999999998e-06 2024-09-18T11:32:14.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:14.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586239+0000", 2024-09-18T11:32:14.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2570000000000002e-05 2024-09-18T11:32:14.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:14.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586271+0000", 2024-09-18T11:32:14.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1341999999999999e-05 2024-09-18T11:32:14.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:14.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594031+0000", 2024-09-18T11:32:14.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077597459999999997 2024-09-18T11:32:14.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:14.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:14.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:14.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594047+0000", 2024-09-18T11:32:14.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6858e-05 2024-09-18T11:32:14.138 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.138 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:14.139 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.139 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:14.139 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:14.139 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:14.140 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:32:14.142 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:14.142 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:14.142 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:14.143 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:14.143 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:14.143 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:14.143 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:14.144 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:14.144 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:14.139+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:14.155 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:14.155 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:14.170 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:14.171 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:14.179 INFO:tasks.rados.rados.0.smithi119.stdout:1766: finishing write tid 2 to smithi11939506-16 2024-09-18T11:32:14.179 INFO:tasks.rados.rados.0.smithi119.stdout:1766: finishing write tid 3 to smithi11939506-16 2024-09-18T11:32:14.179 INFO:tasks.rados.rados.0.smithi119.stdout:1766: finishing write tid 4 to smithi11939506-16 2024-09-18T11:32:14.179 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 16 v 324 (ObjNum 359 snap 55 seq_num 359) dirty exists 2024-09-18T11:32:14.179 INFO:tasks.rados.rados.0.smithi119.stdout:1766: left oid 16 (ObjNum 359 snap 55 seq_num 359) 2024-09-18T11:32:14.179 INFO:tasks.rados.rados.0.smithi119.stdout:1768: finishing set_chunk to oid 17 2024-09-18T11:32:14.179 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 451 (ObjNum 351 snap 54 seq_num 351) dirty exists 2024-09-18T11:32:14.180 INFO:tasks.rados.rados.0.smithi119.stdout:1769: finishing rollback tid 0 to smithi11939506-45 2024-09-18T11:32:14.180 INFO:tasks.rados.rados.0.smithi119.stdout:1762: done (9 left) 2024-09-18T11:32:14.180 INFO:tasks.rados.rados.0.smithi119.stdout:1763: done (8 left) 2024-09-18T11:32:14.180 INFO:tasks.rados.rados.0.smithi119.stdout:1764: done (7 left) 2024-09-18T11:32:14.180 INFO:tasks.rados.rados.0.smithi119.stdout:1765: done (6 left) 2024-09-18T11:32:14.180 INFO:tasks.rados.rados.0.smithi119.stdout:1766: done (5 left) 2024-09-18T11:32:14.180 INFO:tasks.rados.rados.0.smithi119.stdout:1767: done (4 left) 2024-09-18T11:32:14.180 INFO:tasks.rados.rados.0.smithi119.stdout:1768: done (3 left) 2024-09-18T11:32:14.180 INFO:tasks.rados.rados.0.smithi119.stdout:1772: tier_flush oid 18 2024-09-18T11:32:14.180 INFO:tasks.rados.rados.0.smithi119.stdout:1772: tier_flush oid 18 snap -1 2024-09-18T11:32:14.180 INFO:tasks.rados.rados.0.smithi119.stdout:1769: finishing rollback tid 1 to smithi11939506-45 2024-09-18T11:32:14.180 INFO:tasks.rados.rados.0.smithi119.stdout:1769: finishing rollback tid 2 to smithi11939506-45 2024-09-18T11:32:14.180 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 91 (ObjNum 254 snap 30 seq_num 254) dirty exists 2024-09-18T11:32:14.181 INFO:tasks.rados.rados.0.smithi119.stdout:1771: got (0) Success 2024-09-18T11:32:14.181 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 271 (ObjNum 129 snap 0 seq_num 384687584) clean dne 2024-09-18T11:32:14.181 INFO:tasks.rados.rados.0.smithi119.stdout:1770: expect (ObjNum 332 snap 48 seq_num 332) 2024-09-18T11:32:14.186 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:14.186 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:14.201 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:14.202 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:14.241 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:14.240+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:14.255 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:14.255 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:14.264 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:32:14.333 INFO:tasks.rados.rados.0.smithi119.stdout:1772: got (16) Device or resource busy 2024-09-18T11:32:14.333 INFO:tasks.rados.rados.0.smithi119.stdout:check_oldest_snap_flushed oid 18 v 171 (ObjNum 205 snap 23 seq_num 205) unflushed 42 2024-09-18T11:32:14.334 INFO:tasks.rados.rados.0.smithi119.stdout:1769: done (3 left) 2024-09-18T11:32:14.334 INFO:tasks.rados.rados.0.smithi119.stdout:1770: done (2 left) 2024-09-18T11:32:14.334 INFO:tasks.rados.rados.0.smithi119.stdout:1771: done (1 left) 2024-09-18T11:32:14.334 INFO:tasks.rados.rados.0.smithi119.stdout:1772: done (0 left) 2024-09-18T11:32:14.334 INFO:tasks.rados.rados.0.smithi119.stdout:1773: snap_remove snap 31 2024-09-18T11:32:14.349 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:14.348+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:14.403 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:14.403 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:14.412 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:32:14.453 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:14.452+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:14.544 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:14.543+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:14.551 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:14.551 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:14.560 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:32:14.645 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:14.644+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:14.698 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:14.699 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:14.707 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:32:14.746 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:14.745+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:14.805 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:14.805 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:14.814 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:32:14.847 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:14.845+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:14.912 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:14.912 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:14.920 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:32:14.948 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:14.946+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:15.019 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:15.019 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:15.020 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:15.018+0000 7f697ceb1640 -1 osd.3 143 handle_osd_map injecting map crc failure 2024-09-18T11:32:15.021 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:15.019+0000 7f2d449e8640 -1 osd.1 143 handle_osd_map injecting map crc failure 2024-09-18T11:32:15.023 INFO:tasks.rados.rados.0.smithi119.stdout:1773: done (0 left) 2024-09-18T11:32:15.023 INFO:tasks.rados.rados.0.smithi119.stdout:1774: set_chunk oid 29 target oid 2024-09-18T11:32:15.031 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:32:15.043 INFO:tasks.rados.rados.0.smithi119.stdout:1774: set_chunk oid 29 offset: 2877952 length: 273920 target oid c267daa6438c469d1912ba01ca1fe2e798f4e90026535c99b45f85c41ec9edcf offset: 0 snap 45 2024-09-18T11:32:15.043 INFO:tasks.rados.rados.0.smithi119.stdout:1775: read oid 47 snap -1 2024-09-18T11:32:15.043 INFO:tasks.rados.rados.0.smithi119.stdout:1775: expect (ObjNum 344 snap 50 seq_num 344) 2024-09-18T11:32:15.043 INFO:tasks.rados.rados.0.smithi119.stdout:1776: rollback oid 35 current snap is 55 2024-09-18T11:32:15.043 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 35 to 32 2024-09-18T11:32:15.043 INFO:tasks.rados.rados.0.smithi119.stdout:1777: write oid 1 current snap is 55 2024-09-18T11:32:15.044 INFO:tasks.rados.rados.0.smithi119.stdout:1777: seq_num 360 ranges {450520=558231,1483047=515125} 2024-09-18T11:32:15.049 INFO:tasks.rados.rados.0.smithi119.stdout:1777: writing smithi11939506-1 from 450520 to 1008751 tid 1 2024-09-18T11:32:15.055 INFO:tasks.rados.rados.0.smithi119.stdout:1777: writing smithi11939506-1 from 1483047 to 1998172 tid 2 2024-09-18T11:32:15.055 INFO:tasks.rados.rados.0.smithi119.stdout:1778: write oid 26 current snap is 55 2024-09-18T11:32:15.055 INFO:tasks.rados.rados.0.smithi119.stdout:1778: seq_num 361 ranges {467327=633480,1825176=242151} 2024-09-18T11:32:15.061 INFO:tasks.rados.rados.0.smithi119.stdout:1778: writing smithi11939506-26 from 467327 to 1100807 tid 1 2024-09-18T11:32:15.067 INFO:tasks.rados.rados.0.smithi119.stdout:1778: writing smithi11939506-26 from 1825176 to 2067327 tid 2 2024-09-18T11:32:15.067 INFO:tasks.rados.rados.0.smithi119.stdout:1779: read oid 18 snap -1 2024-09-18T11:32:15.067 INFO:tasks.rados.rados.0.smithi119.stdout:1779: expect (ObjNum 205 snap 23 seq_num 205) 2024-09-18T11:32:15.067 INFO:tasks.rados.rados.0.smithi119.stdout:1774: finishing set_chunk to oid 29 2024-09-18T11:32:15.067 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 29 set_chunk returned error code -95 offset: 2877952 length: 273920 2024-09-18T11:32:15.067 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 29 v 0 (ObjNum 355 snap 55 seq_num 355) dirty exists 2024-09-18T11:32:15.067 INFO:tasks.rados.rados.0.smithi119.stdout:1776: finishing rollback tid 0 to smithi11939506-35 2024-09-18T11:32:15.067 INFO:tasks.rados.rados.0.smithi119.stdout:1775: expect (ObjNum 344 snap 50 seq_num 344) 2024-09-18T11:32:15.074 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:15.073+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:15.101 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T11:32:15.150 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:15.149+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:15.151 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:15.152 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:15.162 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:32:15.251 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:15.249+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:15.268 INFO:tasks.rados.rados.0.smithi119.stdout:1777: finishing write tid 1 to smithi11939506-1 2024-09-18T11:32:15.268 INFO:tasks.rados.rados.0.smithi119.stdout:1777: finishing write tid 2 to smithi11939506-1 2024-09-18T11:32:15.268 INFO:tasks.rados.rados.0.smithi119.stdout:1777: finishing write tid 3 to smithi11939506-1 2024-09-18T11:32:15.269 INFO:tasks.rados.rados.0.smithi119.stdout:1777: finishing write tid 4 to smithi11939506-1 2024-09-18T11:32:15.269 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 291 (ObjNum 360 snap 55 seq_num 360) dirty exists 2024-09-18T11:32:15.269 INFO:tasks.rados.rados.0.smithi119.stdout:1777: left oid 1 (ObjNum 360 snap 55 seq_num 360) 2024-09-18T11:32:15.269 INFO:tasks.rados.rados.0.smithi119.stdout:1780: read oid 17 snap -1 2024-09-18T11:32:15.269 INFO:tasks.rados.rados.0.smithi119.stdout:1780: expect (ObjNum 351 snap 54 seq_num 351) 2024-09-18T11:32:15.269 INFO:tasks.rados.rados.0.smithi119.stdout:1774: done (6 left) 2024-09-18T11:32:15.269 INFO:tasks.rados.rados.0.smithi119.stdout:1775: done (5 left) 2024-09-18T11:32:15.269 INFO:tasks.rados.rados.0.smithi119.stdout:1777: done (4 left) 2024-09-18T11:32:15.269 INFO:tasks.rados.rados.0.smithi119.stdout:1781: write (excl) oid 8 current snap is 55 2024-09-18T11:32:15.269 INFO:tasks.rados.rados.0.smithi119.stdout:1778: finishing write tid 1 to smithi11939506-26 2024-09-18T11:32:15.269 INFO:tasks.rados.rados.0.smithi119.stdout:1781: seq_num 362 ranges {484134=708729,1883658=200476} 2024-09-18T11:32:15.274 INFO:tasks.rados.rados.0.smithi119.stdout:1781: writing smithi11939506-8 from 484134 to 1192863 tid 1 2024-09-18T11:32:15.279 INFO:tasks.rados.rados.0.smithi119.stdout:1781: writing smithi11939506-8 from 1883658 to 2084134 tid 2 2024-09-18T11:32:15.279 INFO:tasks.rados.rados.0.smithi119.stdout:1782: set_chunk oid 19 target oid 2024-09-18T11:32:15.292 INFO:tasks.rados.rados.0.smithi119.stdout:1782: set_chunk oid 19 offset: 323584 length: 1522176 target oid 0492e4d28eaaf42cc03e6aa5ee3e7c037cd0e8b26b2c0b726597fd267a0dc5e0 offset: 0 snap -1 2024-09-18T11:32:15.292 INFO:tasks.rados.rados.0.smithi119.stdout:1783: write (excl) oid 23 current snap is 55 2024-09-18T11:32:15.292 INFO:tasks.rados.rados.0.smithi119.stdout:1776: finishing rollback tid 1 to smithi11939506-35 2024-09-18T11:32:15.292 INFO:tasks.rados.rados.0.smithi119.stdout:1783: seq_num 363 ranges {500941=783978,1942140=158801} 2024-09-18T11:32:15.298 INFO:tasks.rados.rados.0.smithi119.stdout:1783: writing smithi11939506-23 from 500941 to 1284919 tid 1 2024-09-18T11:32:15.303 INFO:tasks.rados.rados.0.smithi119.stdout:1783: writing smithi11939506-23 from 1942140 to 2100941 tid 2 2024-09-18T11:32:15.303 INFO:tasks.rados.rados.0.smithi119.stdout:1784: read oid 9 snap -1 2024-09-18T11:32:15.303 INFO:tasks.rados.rados.0.smithi119.stdout:1784: expect (ObjNum 350 snap 54 seq_num 350) 2024-09-18T11:32:15.304 INFO:tasks.rados.rados.0.smithi119.stdout:1785: set_chunk oid 44 target oid 2024-09-18T11:32:15.307 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:15.308 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:15.313 INFO:tasks.rados.rados.0.smithi119.stdout:1785: set_chunk oid 44 offset: 116224 length: 697344 target oid 0293314d859fc5d9467e10180a977e1250a9677a56d8145f1f1452aa07ec2998 offset: 0 snap -1 2024-09-18T11:32:15.313 INFO:tasks.rados.rados.0.smithi119.stdout:1786: read oid 47 snap 38 2024-09-18T11:32:15.314 INFO:tasks.rados.rados.0.smithi119.stdout:1786: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T11:32:15.314 INFO:tasks.rados.rados.0.smithi119.stdout:1787: snap_remove snap 40 2024-09-18T11:32:15.318 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:32:15.359 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:15.358+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:15.385 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:15.383+0000 7fe409e4c640 -1 osd.0 143 handle_osd_map injecting map crc failure 2024-09-18T11:32:15.453 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:15.451+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:15.456 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:15.456 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:15.464 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:32:15.554 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:15.552+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:15.562 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:15.562 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:15.571 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:32:15.655 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:15.654+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:15.669 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:15.669 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:15.677 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:32:15.757 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:15.756+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:15.775 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:15.776 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:15.784 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:32:15.858 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:15.857+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:15.880 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:15.881 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:15.888 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:32:15.959 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:15.958+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:15.987 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:15.987 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:15.995 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:32:16.059 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:16.058+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:16.092 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:16.092 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:16.100 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:32:16.184 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:16.182+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:16.212 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:32:16.239 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:16.239 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:16.249 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:32:16.290 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:16.288+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:16.313 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is unset 2024-09-18T11:32:16.314 INFO:tasks.rados.rados.0.smithi119.stdout:1787: done (10 left) 2024-09-18T11:32:16.314 INFO:tasks.rados.rados.0.smithi119.stdout:1788: read oid 12 snap -1 2024-09-18T11:32:16.314 INFO:tasks.rados.rados.0.smithi119.stdout:1788: expect (ObjNum 283 snap 39 seq_num 283) 2024-09-18T11:32:16.315 INFO:tasks.rados.rados.0.smithi119.stdout:1789: read oid 34 snap -1 2024-09-18T11:32:16.315 INFO:tasks.rados.rados.0.smithi119.stdout:1789: expect (ObjNum 329 snap 48 seq_num 329) 2024-09-18T11:32:16.315 INFO:tasks.rados.rados.0.smithi119.stdout:1790: tier_evict oid 21 2024-09-18T11:32:16.315 INFO:tasks.rados.rados.0.smithi119.stdout:1790: tier_evict oid 21 snap -1 2024-09-18T11:32:16.315 INFO:tasks.rados.rados.0.smithi119.stdout:1776: finishing rollback tid 2 to smithi11939506-35 2024-09-18T11:32:16.315 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 207 (ObjNum 155 snap 20 seq_num 155) dirty exists 2024-09-18T11:32:16.315 INFO:tasks.rados.rados.0.smithi119.stdout:1778: finishing write tid 2 to smithi11939506-26 2024-09-18T11:32:16.315 INFO:tasks.rados.rados.0.smithi119.stdout:1778: finishing write tid 3 to smithi11939506-26 2024-09-18T11:32:16.315 INFO:tasks.rados.rados.0.smithi119.stdout:1776: done (12 left) 2024-09-18T11:32:16.315 INFO:tasks.rados.rados.0.smithi119.stdout:1791: tier_promote oid 41 2024-09-18T11:32:16.316 INFO:tasks.rados.rados.0.smithi119.stdout:1778: finishing write tid 4 to smithi11939506-26 2024-09-18T11:32:16.316 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 292 (ObjNum 361 snap 55 seq_num 361) dirty exists 2024-09-18T11:32:16.316 INFO:tasks.rados.rados.0.smithi119.stdout:1778: left oid 26 (ObjNum 361 snap 55 seq_num 361) 2024-09-18T11:32:16.316 INFO:tasks.rados.rados.0.smithi119.stdout:1779: expect (ObjNum 205 snap 23 seq_num 205) 2024-09-18T11:32:16.317 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:16.313+0000 7fcb662ad640 -1 osd.7 144 handle_osd_map injecting map crc failure 2024-09-18T11:32:16.352 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:16.362 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:16.360+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:16.379 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:16.379 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:16.395 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:16.395 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:16.403 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:16.403 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:16.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:16.403 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:16.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.403 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:262 4.9 e53/22 4:99feffe9:::smithi11939506-45:13 v 53'17, mlcod=51'14)", 2024-09-18T11:32:16.404 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:16.404 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 102.40220003899999, 2024-09-18T11:32:16.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013878679, 2024-09-18T11:32:16.404 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.404 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.404 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:16.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:16.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:16.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:16.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998997+0000", 2024-09-18T11:32:16.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1099999999999999e-06 2024-09-18T11:32:16.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999008+0000", 2024-09-18T11:32:16.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1321000000000001e-05 2024-09-18T11:32:16.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999015+0000", 2024-09-18T11:32:16.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3000000000000004e-06 2024-09-18T11:32:16.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999055+0000", 2024-09-18T11:32:16.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0058000000000002e-05 2024-09-18T11:32:16.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999070+0000", 2024-09-18T11:32:16.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4328000000000001e-05 2024-09-18T11:32:16.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012465+0000", 2024-09-18T11:32:16.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013395638 2024-09-18T11:32:16.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012870+0000", 2024-09-18T11:32:16.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040423400000000001 2024-09-18T11:32:16.408 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.408 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.408 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.409 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:205 4.b e53/22 4:dba5bd39:::smithi11939506-17:13 v 53'77, mlcod=51'74)", 2024-09-18T11:32:16.409 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:16.409 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 102.40215077800001, 2024-09-18T11:32:16.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013359586, 2024-09-18T11:32:16.409 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.409 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.409 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:16.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:16.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:16.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:16.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999048+0000", 2024-09-18T11:32:16.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3550000000000005e-06 2024-09-18T11:32:16.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999056+0000", 2024-09-18T11:32:16.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3259999999999996e-06 2024-09-18T11:32:16.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999063+0000", 2024-09-18T11:32:16.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6409999999999996e-06 2024-09-18T11:32:16.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999098+0000", 2024-09-18T11:32:16.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5203999999999997e-05 2024-09-18T11:32:16.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999115+0000", 2024-09-18T11:32:16.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6478999999999999e-05 2024-09-18T11:32:16.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012362+0000", 2024-09-18T11:32:16.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01324768 2024-09-18T11:32:16.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012400+0000", 2024-09-18T11:32:16.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.731e-05 2024-09-18T11:32:16.413 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.413 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.413 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.413 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1956 4.6 4:67c1765a:::smithi11939506-30:head [write 705927~558502] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:32:16.413 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:32:16.413 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.049181826999998, 2024-09-18T11:32:16.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011813239, 2024-09-18T11:32:16.413 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.413 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:16.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:16.414 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:16.414 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1956 2024-09-18T11:32:16.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.414 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:32:16.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:32:16.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352007+0000", 2024-09-18T11:32:16.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:16.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352229+0000", 2024-09-18T11:32:16.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222024 2024-09-18T11:32:16.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352251+0000", 2024-09-18T11:32:16.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2512000000000001e-05 2024-09-18T11:32:16.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352260+0000", 2024-09-18T11:32:16.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7339999999999997e-06 2024-09-18T11:32:16.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352307+0000", 2024-09-18T11:32:16.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7061999999999998e-05 2024-09-18T11:32:16.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352408+0000", 2024-09-18T11:32:16.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101216 2024-09-18T11:32:16.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:32:16.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352550+0000", 2024-09-18T11:32:16.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014136199999999999 2024-09-18T11:32:16.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:16.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357684+0000", 2024-09-18T11:32:16.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051342499999999999 2024-09-18T11:32:16.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:16.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363736+0000", 2024-09-18T11:32:16.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006052577 2024-09-18T11:32:16.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363778+0000", 2024-09-18T11:32:16.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1989000000000001e-05 2024-09-18T11:32:16.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363822+0000", 2024-09-18T11:32:16.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3894999999999999e-05 2024-09-18T11:32:16.419 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.419 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.419 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.419 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2937 4.7 4:e301aefd:::smithi11939506-19:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:16.419 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:16.419 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.366350464999996, 2024-09-18T11:32:16.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019287667000000001, 2024-09-18T11:32:16.419 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.419 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:16.419 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:16.420 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:16.420 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:16.420 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2937 2024-09-18T11:32:16.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.420 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:16.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:16.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:16.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:16.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034842+0000", 2024-09-18T11:32:16.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.372e-06 2024-09-18T11:32:16.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034846+0000", 2024-09-18T11:32:16.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3919999999999996e-06 2024-09-18T11:32:16.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034848+0000", 2024-09-18T11:32:16.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3539999999999998e-06 2024-09-18T11:32:16.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034893+0000", 2024-09-18T11:32:16.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4861000000000003e-05 2024-09-18T11:32:16.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034965+0000", 2024-09-18T11:32:16.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1860999999999995e-05 2024-09-18T11:32:16.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.054128+0000", 2024-09-18T11:32:16.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019163039 2024-09-18T11:32:16.423 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.423 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.423 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.424 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3453 4.f 4:f5d7df41:::smithi11939506-16:head [write 1813152~490434] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:16.424 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:32:16.424 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.378438537000001, 2024-09-18T11:32:16.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050685840000000003, 2024-09-18T11:32:16.424 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.424 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.424 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:16.424 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:16.424 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:16.424 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3453 2024-09-18T11:32:16.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.424 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:32:16.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:32:16.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022751+0000", 2024-09-18T11:32:16.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:16.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022871+0000", 2024-09-18T11:32:16.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000119781 2024-09-18T11:32:16.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022882+0000", 2024-09-18T11:32:16.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1627000000000001e-05 2024-09-18T11:32:16.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022887+0000", 2024-09-18T11:32:16.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2390000000000004e-06 2024-09-18T11:32:16.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023123+0000", 2024-09-18T11:32:16.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000236598 2024-09-18T11:32:16.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023207+0000", 2024-09-18T11:32:16.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4303999999999998e-05 2024-09-18T11:32:16.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:16.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023367+0000", 2024-09-18T11:32:16.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015983600000000001 2024-09-18T11:32:16.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:16.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073237+0000", 2024-09-18T11:32:16.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049869400000000001 2024-09-18T11:32:16.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:16.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073352+0000", 2024-09-18T11:32:16.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011579899999999999 2024-09-18T11:32:16.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073394+0000", 2024-09-18T11:32:16.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1445e-05 2024-09-18T11:32:16.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073438+0000", 2024-09-18T11:32:16.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4261999999999997e-05 2024-09-18T11:32:16.429 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.429 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.429 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.430 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3454 4.f 4:f5d7df41:::smithi11939506-16:head [setxattr _header (57) in=7b,truncate 2991679] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:16.430 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:32:16.430 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.377740095, 2024-09-18T11:32:16.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050365656000000002, 2024-09-18T11:32:16.430 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.430 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:16.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:16.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:16.430 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3454 2024-09-18T11:32:16.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.430 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:32:16.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:32:16.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023450+0000", 2024-09-18T11:32:16.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:16.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023456+0000", 2024-09-18T11:32:16.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8329999999999999e-06 2024-09-18T11:32:16.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023465+0000", 2024-09-18T11:32:16.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1130000000000007e-06 2024-09-18T11:32:16.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023470+0000", 2024-09-18T11:32:16.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9139999999999999e-06 2024-09-18T11:32:16.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072611+0000", 2024-09-18T11:32:16.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141643999999998 2024-09-18T11:32:16.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072729+0000", 2024-09-18T11:32:16.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117667 2024-09-18T11:32:16.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:16.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072887+0000", 2024-09-18T11:32:16.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015760000000000001 2024-09-18T11:32:16.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:16.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073558+0000", 2024-09-18T11:32:16.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00067171699999999995 2024-09-18T11:32:16.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:16.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073764+0000", 2024-09-18T11:32:16.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000205749 2024-09-18T11:32:16.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073790+0000", 2024-09-18T11:32:16.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5643000000000002e-05 2024-09-18T11:32:16.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073816+0000", 2024-09-18T11:32:16.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6588e-05 2024-09-18T11:32:16.435 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.435 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.435 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.436 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3455 4.f 4:f5d7df41:::smithi11939506-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:16.436 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:16.436 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.377710477000001, 2024-09-18T11:32:16.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050460466000000002, 2024-09-18T11:32:16.436 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.436 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:16.436 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:16.436 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:16.436 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:16.436 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3455 2024-09-18T11:32:16.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.436 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:16.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:16.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:16.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:16.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023483+0000", 2024-09-18T11:32:16.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.695e-06 2024-09-18T11:32:16.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023489+0000", 2024-09-18T11:32:16.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1079999999999998e-06 2024-09-18T11:32:16.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023492+0000", 2024-09-18T11:32:16.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.478e-06 2024-09-18T11:32:16.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073273+0000", 2024-09-18T11:32:16.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049780638000000002 2024-09-18T11:32:16.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:16.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073343+0000", 2024-09-18T11:32:16.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9770000000000005e-05 2024-09-18T11:32:16.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073846+0000", 2024-09-18T11:32:16.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050289499999999995 2024-09-18T11:32:16.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073886+0000", 2024-09-18T11:32:16.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0658999999999998e-05 2024-09-18T11:32:16.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073941+0000", 2024-09-18T11:32:16.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4642000000000003e-05 2024-09-18T11:32:16.441 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.441 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.441 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.441 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3456 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:16.441 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:16.441 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.377685412000002, 2024-09-18T11:32:16.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049859120999999999, 2024-09-18T11:32:16.442 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.442 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:16.442 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:16.442 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:16.442 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:16.442 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3456 2024-09-18T11:32:16.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.442 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:16.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:16.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:16.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:16.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023508+0000", 2024-09-18T11:32:16.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9490000000000001e-06 2024-09-18T11:32:16.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023513+0000", 2024-09-18T11:32:16.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3800000000000002e-06 2024-09-18T11:32:16.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023517+0000", 2024-09-18T11:32:16.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5300000000000001e-06 2024-09-18T11:32:16.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073262+0000", 2024-09-18T11:32:16.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049745194999999999 2024-09-18T11:32:16.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073365+0000", 2024-09-18T11:32:16.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102462 2024-09-18T11:32:16.445 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.445 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.445 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.445 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3457 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:16.445 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:32:16.445 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.377668316000001, 2024-09-18T11:32:16.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049928567, 2024-09-18T11:32:16.446 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.446 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:16.446 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:16.446 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:16.446 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:16.446 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3457 2024-09-18T11:32:16.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.446 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:32:16.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:32:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023522+0000", 2024-09-18T11:32:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023524+0000", 2024-09-18T11:32:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8500000000000001e-06 2024-09-18T11:32:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023529+0000", 2024-09-18T11:32:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7509999999999999e-06 2024-09-18T11:32:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023532+0000", 2024-09-18T11:32:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.163e-06 2024-09-18T11:32:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073386+0000", 2024-09-18T11:32:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049853899 2024-09-18T11:32:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073451+0000", 2024-09-18T11:32:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5197000000000006e-05 2024-09-18T11:32:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3458 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.377655759, 2024-09-18T11:32:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.05001659, 2024-09-18T11:32:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3458 2024-09-18T11:32:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023537+0000", 2024-09-18T11:32:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.64e-06 2024-09-18T11:32:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023543+0000", 2024-09-18T11:32:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3369999999999996e-06 2024-09-18T11:32:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023546+0000", 2024-09-18T11:32:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:32:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073466+0000", 2024-09-18T11:32:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049919719000000001 2024-09-18T11:32:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073552+0000", 2024-09-18T11:32:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6168999999999994e-05 2024-09-18T11:32:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(client.4590.0:3453 4.f e116/22 ondisk, result = 0)", 2024-09-18T11:32:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:32:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.376050674999998, 2024-09-18T11:32:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048099441, 2024-09-18T11:32:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:32:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:32:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025139+0000", 2024-09-18T11:32:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025144+0000", 2024-09-18T11:32:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4059999999999998e-06 2024-09-18T11:32:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072212+0000", 2024-09-18T11:32:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047068482000000002 2024-09-18T11:32:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072230+0000", 2024-09-18T11:32:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7632999999999999e-05 2024-09-18T11:32:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073224+0000", 2024-09-18T11:32:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099436899999999994 2024-09-18T11:32:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073231+0000", 2024-09-18T11:32:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-09-18T11:32:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073240+0000", 2024-09-18T11:32:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5699999999999993e-06 2024-09-18T11:32:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3930 4.b 4:d3ae8215:::smithi11939506-41:head [write 632029~547288] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:32:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.375134995, 2024-09-18T11:32:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018962711, 2024-09-18T11:32:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3930 2024-09-18T11:32:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:32:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:32:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026055+0000", 2024-09-18T11:32:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026334+0000", 2024-09-18T11:32:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000278931 2024-09-18T11:32:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026344+0000", 2024-09-18T11:32:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0716999999999999e-05 2024-09-18T11:32:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026350+0000", 2024-09-18T11:32:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7620000000000001e-06 2024-09-18T11:32:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028429+0000", 2024-09-18T11:32:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020787319999999998 2024-09-18T11:32:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028525+0000", 2024-09-18T11:32:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6638999999999994e-05 2024-09-18T11:32:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028720+0000", 2024-09-18T11:32:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000194525 2024-09-18T11:32:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.036996+0000", 2024-09-18T11:32:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082756140000000006 2024-09-18T11:32:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044952+0000", 2024-09-18T11:32:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566740000000004 2024-09-18T11:32:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044985+0000", 2024-09-18T11:32:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3081999999999999e-05 2024-09-18T11:32:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045019+0000", 2024-09-18T11:32:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3257999999999999e-05 2024-09-18T11:32:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3931 4.b 4:d3ae8215:::smithi11939506-41:head [write 1639134~749453] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.033118+0000", 2024-09-18T11:32:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.368073355, 2024-09-18T11:32:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012000667, 2024-09-18T11:32:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3931 2024-09-18T11:32:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033118+0000", 2024-09-18T11:32:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033118+0000", 2024-09-18T11:32:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033116+0000", 2024-09-18T11:32:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033408+0000", 2024-09-18T11:32:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029145000000000001 2024-09-18T11:32:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033420+0000", 2024-09-18T11:32:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2748999999999999e-05 2024-09-18T11:32:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033426+0000", 2024-09-18T11:32:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8889999999999999e-06 2024-09-18T11:32:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.034205+0000", 2024-09-18T11:32:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00077899000000000002 2024-09-18T11:32:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.034280+0000", 2024-09-18T11:32:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4871999999999997e-05 2024-09-18T11:32:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.034392+0000", 2024-09-18T11:32:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000111666 2024-09-18T11:32:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.037001+0000", 2024-09-18T11:32:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026091310000000002 2024-09-18T11:32:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045055+0000", 2024-09-18T11:32:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080539700000000006 2024-09-18T11:32:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045090+0000", 2024-09-18T11:32:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4984000000000003e-05 2024-09-18T11:32:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045118+0000", 2024-09-18T11:32:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8343999999999998e-05 2024-09-18T11:32:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3932 4.b 4:d3ae8215:::smithi11939506-41:head [write 2796249~437859] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:32:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.363128355000001, 2024-09-18T11:32:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01458329, 2024-09-18T11:32:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3932 2024-09-18T11:32:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:32:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:32:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038061+0000", 2024-09-18T11:32:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038341+0000", 2024-09-18T11:32:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027989099999999999 2024-09-18T11:32:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038353+0000", 2024-09-18T11:32:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2247e-05 2024-09-18T11:32:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038358+0000", 2024-09-18T11:32:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-18T11:32:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043226+0000", 2024-09-18T11:32:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048673049999999997 2024-09-18T11:32:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043284+0000", 2024-09-18T11:32:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8338999999999998e-05 2024-09-18T11:32:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043367+0000", 2024-09-18T11:32:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3120000000000004e-05 2024-09-18T11:32:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.051544+0000", 2024-09-18T11:32:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00817734 2024-09-18T11:32:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052528+0000", 2024-09-18T11:32:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098318499999999992 2024-09-18T11:32:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052610+0000", 2024-09-18T11:32:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2300999999999997e-05 2024-09-18T11:32:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052646+0000", 2024-09-18T11:32:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5955999999999999e-05 2024-09-18T11:32:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3933 4.b 4:d3ae8215:::smithi11939506-41:head [write 3653524~178505] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041300+0000", 2024-09-18T11:32:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.359890459000001, 2024-09-18T11:32:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01299752, 2024-09-18T11:32:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3933 2024-09-18T11:32:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041300+0000", 2024-09-18T11:32:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041300+0000", 2024-09-18T11:32:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041299+0000", 2024-09-18T11:32:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041369+0000", 2024-09-18T11:32:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9838000000000005e-05 2024-09-18T11:32:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041380+0000", 2024-09-18T11:32:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0910000000000001e-05 2024-09-18T11:32:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041385+0000", 2024-09-18T11:32:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5800000000000002e-06 2024-09-18T11:32:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044260+0000", 2024-09-18T11:32:16.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028751229999999998 2024-09-18T11:32:16.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044312+0000", 2024-09-18T11:32:16.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1808000000000003e-05 2024-09-18T11:32:16.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:16.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044393+0000", 2024-09-18T11:32:16.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1806999999999998e-05 2024-09-18T11:32:16.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052654+0000", 2024-09-18T11:32:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082609320000000003 2024-09-18T11:32:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054207+0000", 2024-09-18T11:32:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001552091 2024-09-18T11:32:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054256+0000", 2024-09-18T11:32:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9437000000000001e-05 2024-09-18T11:32:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054298+0000", 2024-09-18T11:32:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1993000000000003e-05 2024-09-18T11:32:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3934 4.b 4:d3ae8215:::smithi11939506-41:head [setxattr _header (57) in=7b,truncate 3832029] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.359801837999999, 2024-09-18T11:32:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013018336, 2024-09-18T11:32:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3934 2024-09-18T11:32:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041391+0000", 2024-09-18T11:32:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.26e-06 2024-09-18T11:32:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041395+0000", 2024-09-18T11:32:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8999999999999999e-06 2024-09-18T11:32:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041397+0000", 2024-09-18T11:32:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3149999999999999e-06 2024-09-18T11:32:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045266+0000", 2024-09-18T11:32:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003869094 2024-09-18T11:32:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045316+0000", 2024-09-18T11:32:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0074000000000002e-05 2024-09-18T11:32:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045403+0000", 2024-09-18T11:32:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6737999999999995e-05 2024-09-18T11:32:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052659+0000", 2024-09-18T11:32:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072558259999999999 2024-09-18T11:32:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054330+0000", 2024-09-18T11:32:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016716000000000001 2024-09-18T11:32:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054362+0000", 2024-09-18T11:32:16.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1195000000000002e-05 2024-09-18T11:32:16.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054407+0000", 2024-09-18T11:32:16.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5707000000000001e-05 2024-09-18T11:32:16.488 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.488 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3935 4.b 4:d3ae8215:::smithi11939506-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:32:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.359791248000001, 2024-09-18T11:32:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013142333000000001, 2024-09-18T11:32:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3935 2024-09-18T11:32:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:32:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:32:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041399+0000", 2024-09-18T11:32:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041401+0000", 2024-09-18T11:32:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4890000000000001e-06 2024-09-18T11:32:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041404+0000", 2024-09-18T11:32:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5240000000000001e-06 2024-09-18T11:32:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041407+0000", 2024-09-18T11:32:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2230000000000001e-06 2024-09-18T11:32:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045699+0000", 2024-09-18T11:32:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042918970000000002 2024-09-18T11:32:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045750+0000", 2024-09-18T11:32:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1332999999999998e-05 2024-09-18T11:32:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054438+0000", 2024-09-18T11:32:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086882569999999996 2024-09-18T11:32:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054485+0000", 2024-09-18T11:32:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7151000000000003e-05 2024-09-18T11:32:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054542+0000", 2024-09-18T11:32:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6706999999999998e-05 2024-09-18T11:32:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:343 4.a e137/22 4:5975a5ec:::smithi11939506-36:49 v 137'218, mlcod=132'212)", 2024-09-18T11:32:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:08.278418+0000", 2024-09-18T11:32:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.1227727390000002, 2024-09-18T11:32:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012396378, 2024-09-18T11:32:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278418+0000", 2024-09-18T11:32:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278418+0000", 2024-09-18T11:32:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278418+0000", 2024-09-18T11:32:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278422+0000", 2024-09-18T11:32:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.189e-06 2024-09-18T11:32:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278428+0000", 2024-09-18T11:32:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7400000000000001e-06 2024-09-18T11:32:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278431+0000", 2024-09-18T11:32:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4790000000000001e-06 2024-09-18T11:32:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.283059+0000", 2024-09-18T11:32:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046282750000000003 2024-09-18T11:32:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.283072+0000", 2024-09-18T11:32:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2819999999999999e-05 2024-09-18T11:32:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.290780+0000", 2024-09-18T11:32:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077080990000000004 2024-09-18T11:32:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.290815+0000", 2024-09-18T11:32:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4224000000000003e-05 2024-09-18T11:32:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3969 4.4 4:2cb3e6a5:::smithi11939506-3:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:32:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.8275509830000001, 2024-09-18T11:32:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.024248042000000001, 2024-09-18T11:32:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3969 2024-09-18T11:32:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:32:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:32:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573639+0000", 2024-09-18T11:32:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573646+0000", 2024-09-18T11:32:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6569999999999997e-06 2024-09-18T11:32:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573662+0000", 2024-09-18T11:32:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5187999999999999e-05 2024-09-18T11:32:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573670+0000", 2024-09-18T11:32:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5399999999999996e-06 2024-09-18T11:32:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8592999999999999e-05 2024-09-18T11:32:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573818+0000", 2024-09-18T11:32:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9388999999999995e-05 2024-09-18T11:32:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597888+0000", 2024-09-18T11:32:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.024069859999999998 2024-09-18T11:32:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3973 4.7 4:e73ba68f:::smithi11939506-1:head [tier-promote] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.8274519040000001, 2024-09-18T11:32:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.023911297000000001, 2024-09-18T11:32:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3973 2024-09-18T11:32:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573738+0000", 2024-09-18T11:32:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573743+0000", 2024-09-18T11:32:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0839999999999999e-06 2024-09-18T11:32:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573752+0000", 2024-09-18T11:32:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3080000000000008e-06 2024-09-18T11:32:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573758+0000", 2024-09-18T11:32:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2659999999999998e-06 2024-09-18T11:32:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573803+0000", 2024-09-18T11:32:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5243e-05 2024-09-18T11:32:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573897+0000", 2024-09-18T11:32:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3398000000000003e-05 2024-09-18T11:32:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597650+0000", 2024-09-18T11:32:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.023753672999999999 2024-09-18T11:32:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.507 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:16.508 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:16.508 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:16.509 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:32:16.510 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:16.510 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:16.510 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:16.511 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:16.511 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:16.511 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:16.511 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:16.512 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:16.512 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:16.507+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:16.523 INFO:tasks.rados.rados.0.smithi119.stdout:1780: expect (ObjNum 351 snap 54 seq_num 351) 2024-09-18T11:32:16.609 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:16.608+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:16.622 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:16.622 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:16.622 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:16.622 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:16.630 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:32:16.710 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:16.709+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:16.729 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:16.729 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:16.729 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:16.729 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:16.729 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:16.737 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:32:16.742 INFO:tasks.rados.rados.0.smithi119.stdout:1781: finishing write tid 1 to smithi11939506-8 2024-09-18T11:32:16.742 INFO:tasks.rados.rados.0.smithi119.stdout:1781: finishing write tid 2 to smithi11939506-8 2024-09-18T11:32:16.743 INFO:tasks.rados.rados.0.smithi119.stdout:1781: finishing write tid 3 to smithi11939506-8 2024-09-18T11:32:16.743 INFO:tasks.rados.rados.0.smithi119.stdout:1781: finishing write tid 4 to smithi11939506-8 2024-09-18T11:32:16.743 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 8 v 305 (ObjNum 362 snap 55 seq_num 362) dirty exists 2024-09-18T11:32:16.743 INFO:tasks.rados.rados.0.smithi119.stdout:1781: left oid 8 (ObjNum 362 snap 55 seq_num 362) 2024-09-18T11:32:16.743 INFO:tasks.rados.rados.0.smithi119.stdout:1782: finishing set_chunk to oid 19 2024-09-18T11:32:16.743 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 227 (ObjNum 288 snap 42 seq_num 288) dirty exists 2024-09-18T11:32:16.743 INFO:tasks.rados.rados.0.smithi119.stdout:1783: finishing write tid 1 to smithi11939506-23 2024-09-18T11:32:16.743 INFO:tasks.rados.rados.0.smithi119.stdout:1778: done (12 left) 2024-09-18T11:32:16.743 INFO:tasks.rados.rados.0.smithi119.stdout:1779: done (11 left) 2024-09-18T11:32:16.743 INFO:tasks.rados.rados.0.smithi119.stdout:1780: done (10 left) 2024-09-18T11:32:16.743 INFO:tasks.rados.rados.0.smithi119.stdout:1781: done (9 left) 2024-09-18T11:32:16.743 INFO:tasks.rados.rados.0.smithi119.stdout:1782: done (8 left) 2024-09-18T11:32:16.743 INFO:tasks.rados.rados.0.smithi119.stdout:1792: snap_create 2024-09-18T11:32:16.744 INFO:tasks.rados.rados.0.smithi119.stdout:1783: finishing write tid 2 to smithi11939506-23 2024-09-18T11:32:16.744 INFO:tasks.rados.rados.0.smithi119.stdout:1783: finishing write tid 3 to smithi11939506-23 2024-09-18T11:32:16.744 INFO:tasks.rados.rados.0.smithi119.stdout:1783: finishing write tid 4 to smithi11939506-23 2024-09-18T11:32:16.744 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 296 (ObjNum 363 snap 55 seq_num 363) dirty exists 2024-09-18T11:32:16.744 INFO:tasks.rados.rados.0.smithi119.stdout:1783: left oid 23 (ObjNum 363 snap 55 seq_num 363) 2024-09-18T11:32:16.744 INFO:tasks.rados.rados.0.smithi119.stdout:1784: expect (ObjNum 350 snap 54 seq_num 350) 2024-09-18T11:32:16.811 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:16.810+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:16.837 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:16.837 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:16.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:16.837 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:16.837 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.837 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:32:16.837 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:16.837 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 116.85112184899999, 2024-09-18T11:32:16.837 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:32:16.838 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.838 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.838 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:16.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.838 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.838 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.838 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:16.838 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.838 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:32:16.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:16.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:32:16.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:32:16.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.839 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.839 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.839 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:32:16.839 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:32:16.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:32:16.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:32:16.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.840 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:32:16.840 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:32:16.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.840 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.840 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:32:16.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:32:16.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:32:16.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:32:16.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.841 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.841 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.841 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:32:16.841 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:32:16.841 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.841 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.841 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.842 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:32:16.842 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:16.842 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 107.616290896, 2024-09-18T11:32:16.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:32:16.842 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.842 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.842 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.842 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.842 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.842 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:16.842 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:16.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.843 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:16.843 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:16.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.843 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:32:16.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:32:16.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:32:16.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:32:16.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.844 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.844 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.844 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:32:16.844 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:32:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:32:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:32:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:32:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:32:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.845 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:32:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:32:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:32:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:32:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.846 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:32:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 102.44238461099999, 2024-09-18T11:32:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:32:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.847 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:32:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:32:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:32:16.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:32:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:32:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:32:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:32:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.849 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:32:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:32:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:32:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:32:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:32:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:32:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.850 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:32:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:32:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:304 4.6 e69/67 4:67c1765a:::smithi11939506-30:17 v 69'35, mlcod=58'31)", 2024-09-18T11:32:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.488543516999997, 2024-09-18T11:32:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006377611, 2024-09-18T11:32:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.852 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345961+0000", 2024-09-18T11:32:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2050000000000008e-06 2024-09-18T11:32:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345969+0000", 2024-09-18T11:32:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0339999999999997e-06 2024-09-18T11:32:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.853 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345976+0000", 2024-09-18T11:32:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-18T11:32:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346026+0000", 2024-09-18T11:32:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9286999999999998e-05 2024-09-18T11:32:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346050+0000", 2024-09-18T11:32:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4397e-05 2024-09-18T11:32:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352293+0000", 2024-09-18T11:32:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062427170000000001 2024-09-18T11:32:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352330+0000", 2024-09-18T11:32:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7348999999999998e-05 2024-09-18T11:32:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.855 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:347 4.d e69/22 4:b44229a1:::smithi11939506-42:19 v 69'99, mlcod=65'93)", 2024-09-18T11:32:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.482810322000006, 2024-09-18T11:32:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056998760000000004, 2024-09-18T11:32:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.856 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351684+0000", 2024-09-18T11:32:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351693+0000", 2024-09-18T11:32:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-18T11:32:16.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351704+0000", 2024-09-18T11:32:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1077000000000001e-05 2024-09-18T11:32:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351709+0000", 2024-09-18T11:32:16.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5419999999999999e-06 2024-09-18T11:32:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351956+0000", 2024-09-18T11:32:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000247252 2024-09-18T11:32:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351975+0000", 2024-09-18T11:32:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9201000000000001e-05 2024-09-18T11:32:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357334+0000", 2024-09-18T11:32:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053588749999999999 2024-09-18T11:32:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357385+0000", 2024-09-18T11:32:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1406999999999998e-05 2024-09-18T11:32:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1956 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'37, mlcod=58'31)", 2024-09-18T11:32:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:16.860 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.481865408999994, 2024-09-18T11:32:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010958842, 2024-09-18T11:32:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352996+0000", 2024-09-18T11:32:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036622 2024-09-18T11:32:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.862 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353011+0000", 2024-09-18T11:32:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4849e-05 2024-09-18T11:32:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353017+0000", 2024-09-18T11:32:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:32:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353068+0000", 2024-09-18T11:32:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0695000000000002e-05 2024-09-18T11:32:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353092+0000", 2024-09-18T11:32:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4260000000000002e-05 2024-09-18T11:32:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363551+0000", 2024-09-18T11:32:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010459154999999999 2024-09-18T11:32:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363589+0000", 2024-09-18T11:32:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8161000000000003e-05 2024-09-18T11:32:16.864 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1957 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'38, mlcod=65'33)", 2024-09-18T11:32:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.475449542000007, 2024-09-18T11:32:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050499400000000002, 2024-09-18T11:32:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359214+0000", 2024-09-18T11:32:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000168047 2024-09-18T11:32:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359227+0000", 2024-09-18T11:32:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3124e-05 2024-09-18T11:32:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359235+0000", 2024-09-18T11:32:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0760000000000003e-06 2024-09-18T11:32:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359287+0000", 2024-09-18T11:32:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.151e-05 2024-09-18T11:32:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359310+0000", 2024-09-18T11:32:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-09-18T11:32:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364065+0000", 2024-09-18T11:32:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047549439999999997 2024-09-18T11:32:16.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364096+0000", 2024-09-18T11:32:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1423000000000001e-05 2024-09-18T11:32:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2300 4.4 e77/22 4:2c93196f:::smithi11939506-28:head v 77'94, mlcod=77'90)", 2024-09-18T11:32:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 75.846387324000005, 2024-09-18T11:32:16.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050425380000000001, 2024-09-18T11:32:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988108+0000", 2024-09-18T11:32:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988388+0000", 2024-09-18T11:32:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028087700000000001 2024-09-18T11:32:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988405+0000", 2024-09-18T11:32:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6565000000000001e-05 2024-09-18T11:32:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988412+0000", 2024-09-18T11:32:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9709999999999998e-06 2024-09-18T11:32:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988464+0000", 2024-09-18T11:32:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1949e-05 2024-09-18T11:32:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988490+0000", 2024-09-18T11:32:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5780999999999999e-05 2024-09-18T11:32:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993112+0000", 2024-09-18T11:32:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046228399999999996 2024-09-18T11:32:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993151+0000", 2024-09-18T11:32:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8587999999999997e-05 2024-09-18T11:32:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.13:63 3.0 e78/21 3:24ebb740:::smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 78'80, mlcod=73'78)", 2024-09-18T11:32:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 75.429397907999999, 2024-09-18T11:32:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052262230000000003, 2024-09-18T11:32:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405097+0000", 2024-09-18T11:32:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405103+0000", 2024-09-18T11:32:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8780000000000003e-06 2024-09-18T11:32:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405117+0000", 2024-09-18T11:32:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3441000000000001e-05 2024-09-18T11:32:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405121+0000", 2024-09-18T11:32:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7099999999999998e-06 2024-09-18T11:32:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405158+0000", 2024-09-18T11:32:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6477999999999999e-05 2024-09-18T11:32:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405173+0000", 2024-09-18T11:32:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4796e-05 2024-09-18T11:32:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410285+0000", 2024-09-18T11:32:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051118919999999998 2024-09-18T11:32:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410324+0000", 2024-09-18T11:32:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9616999999999997e-05 2024-09-18T11:32:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.13:18 3.0 e91/21 3:35e3c617:::2e64dd273da7619bd64b294f52905c1b9405a1ced2a8fd3c0b5bc481bc9732a6:head v 91'89, mlcod=85'87)", 2024-09-18T11:32:16.878 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 62.812622497, 2024-09-18T11:32:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048558539999999997, 2024-09-18T11:32:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021880+0000", 2024-09-18T11:32:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5860000000000001e-06 2024-09-18T11:32:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021890+0000", 2024-09-18T11:32:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8290000000000003e-06 2024-09-18T11:32:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021895+0000", 2024-09-18T11:32:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2480000000000004e-06 2024-09-18T11:32:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022166+0000", 2024-09-18T11:32:16.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027073499999999998 2024-09-18T11:32:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022184+0000", 2024-09-18T11:32:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8082000000000001e-05 2024-09-18T11:32:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026693+0000", 2024-09-18T11:32:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045093479999999998 2024-09-18T11:32:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.882 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026729+0000", 2024-09-18T11:32:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6282999999999998e-05 2024-09-18T11:32:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:573 4.4 e91/22 4:2c93196f:::smithi11939506-28:26 v 91'135, mlcod=90'131)", 2024-09-18T11:32:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 62.810155907999999, 2024-09-18T11:32:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050722739999999999, 2024-09-18T11:32:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.883 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024339+0000", 2024-09-18T11:32:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024349+0000", 2024-09-18T11:32:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6339999999999996e-06 2024-09-18T11:32:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024362+0000", 2024-09-18T11:32:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-18T11:32:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024368+0000", 2024-09-18T11:32:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8649999999999998e-06 2024-09-18T11:32:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024431+0000", 2024-09-18T11:32:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2780999999999994e-05 2024-09-18T11:32:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024448+0000", 2024-09-18T11:32:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7388999999999999e-05 2024-09-18T11:32:16.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029385+0000", 2024-09-18T11:32:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004936523 2024-09-18T11:32:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029412+0000", 2024-09-18T11:32:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7702999999999999e-05 2024-09-18T11:32:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:574 4.4 e91/22 4:2cb3e6a5:::smithi11939506-3:1e v 91'137, mlcod=90'131)", 2024-09-18T11:32:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 62.809717865000003, 2024-09-18T11:32:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070802850000000004, 2024-09-18T11:32:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024777+0000", 2024-09-18T11:32:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024785+0000", 2024-09-18T11:32:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8099999999999998e-06 2024-09-18T11:32:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024794+0000", 2024-09-18T11:32:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5839999999999995e-06 2024-09-18T11:32:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024798+0000", 2024-09-18T11:32:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.031e-06 2024-09-18T11:32:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024830+0000", 2024-09-18T11:32:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2268999999999999e-05 2024-09-18T11:32:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024847+0000", 2024-09-18T11:32:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7306999999999999e-05 2024-09-18T11:32:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031834+0000", 2024-09-18T11:32:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069868380000000004 2024-09-18T11:32:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031858+0000", 2024-09-18T11:32:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4082000000000001e-05 2024-09-18T11:32:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:541 4.d e91/22 4:b9bb642b:::smithi11939506-27:19 v 91'158, mlcod=91'154)", 2024-09-18T11:32:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 62.802337172999998, 2024-09-18T11:32:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052185, 2024-09-18T11:32:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032158+0000", 2024-09-18T11:32:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032165+0000", 2024-09-18T11:32:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-09-18T11:32:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032176+0000", 2024-09-18T11:32:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1460999999999999e-05 2024-09-18T11:32:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032182+0000", 2024-09-18T11:32:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0870000000000003e-06 2024-09-18T11:32:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032224+0000", 2024-09-18T11:32:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1372999999999998e-05 2024-09-18T11:32:16.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032246+0000", 2024-09-18T11:32:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2158e-05 2024-09-18T11:32:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037348+0000", 2024-09-18T11:32:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051025319999999999 2024-09-18T11:32:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037377+0000", 2024-09-18T11:32:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8937000000000001e-05 2024-09-18T11:32:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4178.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-18T11:32:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 34.893675107999996, 2024-09-18T11:32:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048960139999999997, 2024-09-18T11:32:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:16.897 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4178", 2024-09-18T11:32:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/490761693", 2024-09-18T11:32:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4 2024-09-18T11:32:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940819+0000", 2024-09-18T11:32:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940832+0000", 2024-09-18T11:32:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3155e-05 2024-09-18T11:32:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940848+0000", 2024-09-18T11:32:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6288999999999999e-05 2024-09-18T11:32:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940856+0000", 2024-09-18T11:32:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4919999999999996e-06 2024-09-18T11:32:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945613+0000", 2024-09-18T11:32:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047574999999999996 2024-09-18T11:32:16.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945717+0000", 2024-09-18T11:32:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103553 2024-09-18T11:32:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3482 4.6 e117/67 4:67c1765a:::smithi11939506-30:head v 117'120, mlcod=115'116)", 2024-09-18T11:32:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 30.149820885, 2024-09-18T11:32:16.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050037479999999997, 2024-09-18T11:32:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684674+0000", 2024-09-18T11:32:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684950+0000", 2024-09-18T11:32:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027621200000000002 2024-09-18T11:32:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684968+0000", 2024-09-18T11:32:16.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8e-05 2024-09-18T11:32:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684984+0000", 2024-09-18T11:32:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5775000000000001e-05 2024-09-18T11:32:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685033+0000", 2024-09-18T11:32:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8878000000000002e-05 2024-09-18T11:32:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685059+0000", 2024-09-18T11:32:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6801999999999999e-05 2024-09-18T11:32:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689640+0000", 2024-09-18T11:32:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045809029999999999 2024-09-18T11:32:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689679+0000", 2024-09-18T11:32:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8507000000000002e-05 2024-09-18T11:32:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.905 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:976 4.4 e132/22 4:2cb3e6a5:::smithi11939506-3:30 v 132'195, mlcod=128'192)", 2024-09-18T11:32:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.815849814, 2024-09-18T11:32:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0058526510000000004, 2024-09-18T11:32:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:16.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018661+0000", 2024-09-18T11:32:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5565000000000001e-05 2024-09-18T11:32:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018672+0000", 2024-09-18T11:32:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0210000000000001e-05 2024-09-18T11:32:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018678+0000", 2024-09-18T11:32:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1539999999999999e-06 2024-09-18T11:32:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018716+0000", 2024-09-18T11:32:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8389000000000001e-05 2024-09-18T11:32:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018737+0000", 2024-09-18T11:32:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0611e-05 2024-09-18T11:32:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.024458+0000", 2024-09-18T11:32:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057208629999999996 2024-09-18T11:32:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.024499+0000", 2024-09-18T11:32:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1100999999999998e-05 2024-09-18T11:32:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:823 4.d e137/22 4:b44229a1:::smithi11939506-42:49 v 137'266, mlcod=132'260)", 2024-09-18T11:32:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:32:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.5572012579999992, 2024-09-18T11:32:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054497770000000003, 2024-09-18T11:32:16.910 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:32:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:32:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277294+0000", 2024-09-18T11:32:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277299+0000", 2024-09-18T11:32:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6280000000000004e-06 2024-09-18T11:32:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277306+0000", 2024-09-18T11:32:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.533e-06 2024-09-18T11:32:16.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277316+0000", 2024-09-18T11:32:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0148e-05 2024-09-18T11:32:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277868+0000", 2024-09-18T11:32:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00055199999999999997 2024-09-18T11:32:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277898+0000", 2024-09-18T11:32:16.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0264999999999999e-05 2024-09-18T11:32:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.282715+0000", 2024-09-18T11:32:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048165619999999999 2024-09-18T11:32:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.282744+0000", 2024-09-18T11:32:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9855000000000001e-05 2024-09-18T11:32:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:163 3.0 e138/21 3:02e8b30e:::4456d047c09ae2a9a25ccebfd83e30cf78df6779d63bfba9365afd7301eaeb6a:head v 138'132, mlcod=134'130)", 2024-09-18T11:32:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.2523385149999999, 2024-09-18T11:32:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051351770000000003, 2024-09-18T11:32:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582232+0000", 2024-09-18T11:32:16.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5333999999999994e-05 2024-09-18T11:32:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582242+0000", 2024-09-18T11:32:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5200000000000003e-06 2024-09-18T11:32:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582248+0000", 2024-09-18T11:32:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8000000000000004e-06 2024-09-18T11:32:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582305+0000", 2024-09-18T11:32:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7813999999999999e-05 2024-09-18T11:32:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582329+0000", 2024-09-18T11:32:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3414999999999999e-05 2024-09-18T11:32:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587257+0000", 2024-09-18T11:32:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049279880000000003 2024-09-18T11:32:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587293+0000", 2024-09-18T11:32:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5802e-05 2024-09-18T11:32:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3974 4.d e138/22 4:b5a463bf:::smithi11939506-20:head v 138'270, mlcod=137'266)", 2024-09-18T11:32:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:32:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.2484572439999999, 2024-09-18T11:32:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011228458, 2024-09-18T11:32:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:32:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:32:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586038+0000", 2024-09-18T11:32:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587638+0000", 2024-09-18T11:32:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001599184 2024-09-18T11:32:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587647+0000", 2024-09-18T11:32:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3719999999999994e-06 2024-09-18T11:32:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587653+0000", 2024-09-18T11:32:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.105e-06 2024-09-18T11:32:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587712+0000", 2024-09-18T11:32:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9252000000000001e-05 2024-09-18T11:32:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587735+0000", 2024-09-18T11:32:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3132000000000001e-05 2024-09-18T11:32:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597228+0000", 2024-09-18T11:32:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0094930090000000002 2024-09-18T11:32:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597267+0000", 2024-09-18T11:32:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8693999999999999e-05 2024-09-18T11:32:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3975 4.d e138/22 4:b5a463bf:::smithi11939506-20:head v 138'271, mlcod=137'266)", 2024-09-18T11:32:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.2422565380000004, 2024-09-18T11:32:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053620450000000002, 2024-09-18T11:32:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:16.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592927+0000", 2024-09-18T11:32:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00068878699999999995 2024-09-18T11:32:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592940+0000", 2024-09-18T11:32:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:32:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592951+0000", 2024-09-18T11:32:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1046000000000001e-05 2024-09-18T11:32:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593008+0000", 2024-09-18T11:32:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6875e-05 2024-09-18T11:32:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593033+0000", 2024-09-18T11:32:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5341999999999999e-05 2024-09-18T11:32:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597565+0000", 2024-09-18T11:32:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045323150000000003 2024-09-18T11:32:16.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597601+0000", 2024-09-18T11:32:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6071000000000001e-05 2024-09-18T11:32:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:16.928 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:16.928 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:16.930 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:32:16.930 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:16.927+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:16.955 INFO:tasks.rados.rados.0.smithi119.stdout:1785: finishing set_chunk to oid 44 2024-09-18T11:32:16.955 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 44 v 193 (ObjNum 357 snap 55 seq_num 357) dirty exists 2024-09-18T11:32:16.955 INFO:tasks.rados.rados.0.smithi119.stdout:1786: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T11:32:17.030 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:17.029+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:17.048 INFO:tasks.rados.rados.0.smithi119.stdout:1790: got (22) Invalid argument 2024-09-18T11:32:17.048 INFO:tasks.rados.rados.0.smithi119.stdout:1788: expect (ObjNum 283 snap 39 seq_num 283) 2024-09-18T11:32:17.055 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:17.055 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:17.055 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:17.055 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:17.064 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:32:17.131 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:17.130+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:17.163 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:17.163 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:17.163 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:17.163 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:17.163 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:17.171 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:32:17.232 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:17.231+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:17.257 INFO:tasks.rados.rados.0.smithi119.stdout:1791: got (0) Success 2024-09-18T11:32:17.257 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 41 v 429 (ObjNum 347 snap 51 seq_num 347) dirty exists 2024-09-18T11:32:17.257 INFO:tasks.rados.rados.0.smithi119.stdout:1789: expect (ObjNum 329 snap 48 seq_num 329) 2024-09-18T11:32:17.272 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:17.272 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:17.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:17.272 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:17.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:32:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 126.10086608100001, 2024-09-18T11:32:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:32:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:32:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.273 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:32:17.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:32:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:32:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:32:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:32:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:32:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:32:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:32:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:32:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:32:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:32:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:32:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:32:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1001 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:32:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 122.105714803, 2024-09-18T11:32:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007461082, 2024-09-18T11:32:17.277 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1001 2024-09-18T11:32:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:32:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:32:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164570+0000", 2024-09-18T11:32:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164574+0000", 2024-09-18T11:32:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.822e-06 2024-09-18T11:32:17.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164582+0000", 2024-09-18T11:32:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3410000000000003e-06 2024-09-18T11:32:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164587+0000", 2024-09-18T11:32:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769999999999999e-06 2024-09-18T11:32:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164702+0000", 2024-09-18T11:32:17.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115705 2024-09-18T11:32:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164709+0000", 2024-09-18T11:32:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-09-18T11:32:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170296+0000", 2024-09-18T11:32:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055876390000000001 2024-09-18T11:32:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170418+0000", 2024-09-18T11:32:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121549 2024-09-18T11:32:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172032+0000", 2024-09-18T11:32:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00161407 2024-09-18T11:32:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.282 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 122.105688087, 2024-09-18T11:32:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:32:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:32:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:32:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:32:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:32:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:32:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:32:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:32:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:32:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:32:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:32:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:32:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:32:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:32:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:32:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:32:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:32:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:32:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 122.10566979799999, 2024-09-18T11:32:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:32:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:32:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:32:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:32:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:32:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:32:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:32:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:32:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:32:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:32:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:32:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:32:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:32:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:32:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:32:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:32:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:32:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:32:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:32:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 122.10560902899999, 2024-09-18T11:32:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:32:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:32:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:32:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:32:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:32:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:32:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:32:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:32:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:32:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:32:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:32:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:32:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:32:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:32:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:32:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:32:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:32:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:32:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:32:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 122.105586708, 2024-09-18T11:32:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:32:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:32:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:32:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:32:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:32:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:32:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:32:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:32:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:32:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:32:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:32:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:32:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:32:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:32:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:32:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:32:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:32:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:32:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:32:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:32:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.324461825, 2024-09-18T11:32:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:32:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:32:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:32:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:32:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:32:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:32:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:32:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:32:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:32:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:32:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:32:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:32:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:32:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:32:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:32:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:32:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:32:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:32:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:32:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:32:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:32:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:32:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:32:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:32:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1661 4.0 4:019b6e09:::smithi11939506-40:17 [read 0~3445402,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:32:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.920389122000003, 2024-09-18T11:32:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075190880000000002, 2024-09-18T11:32:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1661 2024-09-18T11:32:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349899+0000", 2024-09-18T11:32:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9749999999999999e-06 2024-09-18T11:32:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349906+0000", 2024-09-18T11:32:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9290000000000001e-06 2024-09-18T11:32:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349912+0000", 2024-09-18T11:32:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9900000000000002e-06 2024-09-18T11:32:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355645+0000", 2024-09-18T11:32:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057329549999999996 2024-09-18T11:32:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355776+0000", 2024-09-18T11:32:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130975 2024-09-18T11:32:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357416+0000", 2024-09-18T11:32:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001639649 2024-09-18T11:32:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1761 4.0 4:0aec5e1c:::smithi11939506-50:head [write 769787~704463,stat] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:32:17.313 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:32:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 94.967896707999998, 2024-09-18T11:32:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074082879999999999, 2024-09-18T11:32:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1761 2024-09-18T11:32:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:32:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:32:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302386+0000", 2024-09-18T11:32:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302889+0000", 2024-09-18T11:32:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050318499999999996 2024-09-18T11:32:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302930+0000", 2024-09-18T11:32:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0198999999999999e-05 2024-09-18T11:32:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302965+0000", 2024-09-18T11:32:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5292e-05 2024-09-18T11:32:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303046+0000", 2024-09-18T11:32:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1254999999999994e-05 2024-09-18T11:32:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:32:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303054+0000", 2024-09-18T11:32:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2800000000000003e-06 2024-09-18T11:32:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304658+0000", 2024-09-18T11:32:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016039520000000001 2024-09-18T11:32:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304724+0000", 2024-09-18T11:32:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5888000000000003e-05 2024-09-18T11:32:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304853+0000", 2024-09-18T11:32:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012912700000000001 2024-09-18T11:32:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309084+0000", 2024-09-18T11:32:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042311229999999998 2024-09-18T11:32:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309738+0000", 2024-09-18T11:32:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00065401399999999996 2024-09-18T11:32:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309770+0000", 2024-09-18T11:32:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1652000000000001e-05 2024-09-18T11:32:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309797+0000", 2024-09-18T11:32:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7052000000000001e-05 2024-09-18T11:32:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2503 3.1 e85/67 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head v 85'117, mlcod=84'115)", 2024-09-18T11:32:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.293063137999994, 2024-09-18T11:32:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086527619999999996, 2024-09-18T11:32:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.320 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:17.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977220+0000", 2024-09-18T11:32:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980771+0000", 2024-09-18T11:32:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035505200000000002 2024-09-18T11:32:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980797+0000", 2024-09-18T11:32:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.618e-05 2024-09-18T11:32:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980807+0000", 2024-09-18T11:32:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0433e-05 2024-09-18T11:32:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980861+0000", 2024-09-18T11:32:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3415999999999998e-05 2024-09-18T11:32:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980885+0000", 2024-09-18T11:32:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4212000000000001e-05 2024-09-18T11:32:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.323 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985826+0000", 2024-09-18T11:32:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049412450000000004 2024-09-18T11:32:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985875+0000", 2024-09-18T11:32:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9116000000000002e-05 2024-09-18T11:32:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.324 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2648 4.0 4:059d8d95:::smithi11939506-32:head [read 0~3647119,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:32:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 62.938974768000001, 2024-09-18T11:32:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0076286949999999996, 2024-09-18T11:32:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2648 2024-09-18T11:32:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.325 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:32:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:32:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331309+0000", 2024-09-18T11:32:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.326 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331321+0000", 2024-09-18T11:32:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:32:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331340+0000", 2024-09-18T11:32:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8227e-05 2024-09-18T11:32:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331348+0000", 2024-09-18T11:32:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4640000000000006e-06 2024-09-18T11:32:17.327 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331406+0000", 2024-09-18T11:32:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7445000000000003e-05 2024-09-18T11:32:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331521+0000", 2024-09-18T11:32:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115945 2024-09-18T11:32:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:17.328 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338939+0000", 2024-09-18T11:32:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074179859999999997 2024-09-18T11:32:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2649 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 62.938919769000002, 2024-09-18T11:32:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566240000000007, 2024-09-18T11:32:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:17.329 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2649 2024-09-18T11:32:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.330 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331365+0000", 2024-09-18T11:32:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331369+0000", 2024-09-18T11:32:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.551e-06 2024-09-18T11:32:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331377+0000", 2024-09-18T11:32:17.331 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8769999999999993e-06 2024-09-18T11:32:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331381+0000", 2024-09-18T11:32:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.16e-06 2024-09-18T11:32:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338973+0000", 2024-09-18T11:32:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075918460000000002 2024-09-18T11:32:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:17.332 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339048+0000", 2024-09-18T11:32:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5813000000000001e-05 2024-09-18T11:32:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339322+0000", 2024-09-18T11:32:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027399899999999998 2024-09-18T11:32:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2650 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:17.333 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 62.938899550999999, 2024-09-18T11:32:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084392180000000001, 2024-09-18T11:32:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2650 2024-09-18T11:32:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:17.334 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:17.335 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331388+0000", 2024-09-18T11:32:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3429999999999998e-06 2024-09-18T11:32:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331394+0000", 2024-09-18T11:32:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.19e-06 2024-09-18T11:32:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331398+0000", 2024-09-18T11:32:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.495e-06 2024-09-18T11:32:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.336 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339362+0000", 2024-09-18T11:32:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079641799999999995 2024-09-18T11:32:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339460+0000", 2024-09-18T11:32:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8329e-05 2024-09-18T11:32:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339825+0000", 2024-09-18T11:32:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036510500000000001 2024-09-18T11:32:17.337 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:119 3.2 3:5595a68d:::07854d2fef297a06ba81685e660c332de36d5d18d546927d30daad6d7fda1541:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:32:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.226747793999998, 2024-09-18T11:32:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074145979999999997, 2024-09-18T11:32:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:32:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:32:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043535+0000", 2024-09-18T11:32:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043970+0000", 2024-09-18T11:32:17.339 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000435148 2024-09-18T11:32:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043993+0000", 2024-09-18T11:32:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3113999999999999e-05 2024-09-18T11:32:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044003+0000", 2024-09-18T11:32:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5310000000000007e-06 2024-09-18T11:32:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.340 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044062+0000", 2024-09-18T11:32:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9487000000000002e-05 2024-09-18T11:32:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:32:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044071+0000", 2024-09-18T11:32:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1610000000000003e-06 2024-09-18T11:32:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046423+0000", 2024-09-18T11:32:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023523680000000001 2024-09-18T11:32:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.341 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046502+0000", 2024-09-18T11:32:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9339999999999996e-05 2024-09-18T11:32:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046664+0000", 2024-09-18T11:32:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016185700000000001 2024-09-18T11:32:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050870+0000", 2024-09-18T11:32:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042060070000000003 2024-09-18T11:32:17.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050878+0000", 2024-09-18T11:32:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8960000000000003e-06 2024-09-18T11:32:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050918+0000", 2024-09-18T11:32:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0367000000000002e-05 2024-09-18T11:32:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:17.343 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050952+0000", 2024-09-18T11:32:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3989999999999998e-05 2024-09-18T11:32:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:127 3.2 3:4d3b277a:::3266c7aaa53a7d2c10c3c54d852d6cbbed49fb041bc0df41a3ac83b56e2b0615:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.225884612000002, 2024-09-18T11:32:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093148009999999993, 2024-09-18T11:32:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044400+0000", 2024-09-18T11:32:17.345 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044442+0000", 2024-09-18T11:32:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2247999999999999e-05 2024-09-18T11:32:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044453+0000", 2024-09-18T11:32:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0617e-05 2024-09-18T11:32:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.346 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044457+0000", 2024-09-18T11:32:17.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2520000000000001e-06 2024-09-18T11:32:17.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044495+0000", 2024-09-18T11:32:17.347 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7710000000000003e-05 2024-09-18T11:32:17.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.347 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.347 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:17.347 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044499+0000", 2024-09-18T11:32:17.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6269999999999999e-06 2024-09-18T11:32:17.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048239+0000", 2024-09-18T11:32:17.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037396719999999999 2024-09-18T11:32:17.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.348 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:17.348 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048351+0000", 2024-09-18T11:32:17.348 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112342 2024-09-18T11:32:17.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.348 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:17.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048545+0000", 2024-09-18T11:32:17.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000193881 2024-09-18T11:32:17.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:17.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.052689+0000", 2024-09-18T11:32:17.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041435780000000002 2024-09-18T11:32:17.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.349 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.349 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:17.349 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053654+0000", 2024-09-18T11:32:17.349 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096558199999999999 2024-09-18T11:32:17.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:17.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053683+0000", 2024-09-18T11:32:17.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8850999999999999e-05 2024-09-18T11:32:17.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.350 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:17.350 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053716+0000", 2024-09-18T11:32:17.350 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2468000000000002e-05 2024-09-18T11:32:17.350 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.350 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:17.350 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.350 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.351 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2963 4.0 4:019b6e09:::smithi11939506-40:head [sparse-read 0~1093084] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:17.351 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:32:17.351 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.242875814000001, 2024-09-18T11:32:17.351 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072292880000000004, 2024-09-18T11:32:17.351 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:17.351 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:17.351 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:17.351 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:17.351 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.351 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2963 2024-09-18T11:32:17.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.351 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:17.351 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.351 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:17.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:32:17.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:17.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027410+0000", 2024-09-18T11:32:17.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.352 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:17.352 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027409+0000", 2024-09-18T11:32:17.352 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:17.352 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.352 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:17.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027412+0000", 2024-09-18T11:32:17.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.751e-06 2024-09-18T11:32:17.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:17.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027418+0000", 2024-09-18T11:32:17.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.404e-06 2024-09-18T11:32:17.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.353 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.353 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.027423+0000", 2024-09-18T11:32:17.353 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7119999999999999e-06 2024-09-18T11:32:17.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034156+0000", 2024-09-18T11:32:17.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0067331639999999998 2024-09-18T11:32:17.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:17.354 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034255+0000", 2024-09-18T11:32:17.354 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8701000000000002e-05 2024-09-18T11:32:17.354 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.354 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.354 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:17.355 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:24.034639+0000", 2024-09-18T11:32:17.355 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000384006 2024-09-18T11:32:17.356 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.356 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:17.356 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.356 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.356 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3209 4.0 4:019b6e09:::smithi11939506-40:head [write 638732~557432] snapc 42=[42,41,3f,3e,3d,3b,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-18T11:32:17.356 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:17.356 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.172202573, 2024-09-18T11:32:17.356 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091774999999999999, 2024-09-18T11:32:17.356 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:17.356 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.356 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:17.357 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:17.357 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.357 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3209 2024-09-18T11:32:17.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.357 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:17.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:17.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:17.357 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.357 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.357 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:17.357 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:17.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:17.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098080+0000", 2024-09-18T11:32:17.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:17.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.358 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:17.358 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098676+0000", 2024-09-18T11:32:17.358 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059598099999999998 2024-09-18T11:32:17.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.358 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:17.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098706+0000", 2024-09-18T11:32:17.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0020000000000001e-05 2024-09-18T11:32:17.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098716+0000", 2024-09-18T11:32:17.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0225e-05 2024-09-18T11:32:17.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.359 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.359 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.359 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098782+0000", 2024-09-18T11:32:17.359 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5530000000000004e-05 2024-09-18T11:32:17.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:17.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098883+0000", 2024-09-18T11:32:17.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101609 2024-09-18T11:32:17.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:17.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.099098+0000", 2024-09-18T11:32:17.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000215052 2024-09-18T11:32:17.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.360 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.360 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:17.360 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.102511+0000", 2024-09-18T11:32:17.360 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034129429999999999 2024-09-18T11:32:17.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:17.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107165+0000", 2024-09-18T11:32:17.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046531480000000002 2024-09-18T11:32:17.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:17.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107217+0000", 2024-09-18T11:32:17.361 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2058000000000002e-05 2024-09-18T11:32:17.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.361 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:17.361 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107260+0000", 2024-09-18T11:32:17.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3865000000000001e-05 2024-09-18T11:32:17.362 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.362 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:17.362 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.362 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.362 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:162 3.1 e138/67 3:8c826a1f:::e682cfc1e82c88472b9acf1cd97978bc105aa035f10b685bb4cace95809b52bf:head v 138'176, mlcod=136'174)", 2024-09-18T11:32:17.362 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:32:17.362 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.6867288279999997, 2024-09-18T11:32:17.362 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010445024000000001, 2024-09-18T11:32:17.362 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:17.362 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.362 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:17.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:17.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:32:17.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:17.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:32:17.363 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.363 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.363 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:17.363 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583556+0000", 2024-09-18T11:32:17.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:17.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:17.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584899+0000", 2024-09-18T11:32:17.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001343056 2024-09-18T11:32:17.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.364 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:17.364 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584929+0000", 2024-09-18T11:32:17.364 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0133000000000001e-05 2024-09-18T11:32:17.364 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.364 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584937+0000", 2024-09-18T11:32:17.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6380000000000004e-06 2024-09-18T11:32:17.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584990+0000", 2024-09-18T11:32:17.365 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3186000000000002e-05 2024-09-18T11:32:17.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.365 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:17.365 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585012+0000", 2024-09-18T11:32:17.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1474000000000001e-05 2024-09-18T11:32:17.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:17.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593963+0000", 2024-09-18T11:32:17.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089512859999999993 2024-09-18T11:32:17.366 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.366 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.366 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:17.366 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594002+0000", 2024-09-18T11:32:17.366 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8859999999999997e-05 2024-09-18T11:32:17.366 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.366 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:17.366 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.367 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:166 3.1 e138/67 3:a73e635b:::b6ee478f0f661ec7bac3d17c2ca2fdaff71a47570f64844a7ba79bbb2a0e8d9d:head v 138'177, mlcod=136'174)", 2024-09-18T11:32:17.367 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:32:17.367 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.6851589440000003, 2024-09-18T11:32:17.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089018680000000003, 2024-09-18T11:32:17.367 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:17.367 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.367 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:17.367 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.367 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:17.367 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:32:17.367 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:17.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:32:17.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.368 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:17.368 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585126+0000", 2024-09-18T11:32:17.368 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:17.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.368 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:17.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585211+0000", 2024-09-18T11:32:17.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5432999999999993e-05 2024-09-18T11:32:17.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:17.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585220+0000", 2024-09-18T11:32:17.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5879999999999998e-06 2024-09-18T11:32:17.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.369 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.369 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585225+0000", 2024-09-18T11:32:17.369 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8999999999999997e-06 2024-09-18T11:32:17.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585705+0000", 2024-09-18T11:32:17.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00048042799999999997 2024-09-18T11:32:17.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:17.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585724+0000", 2024-09-18T11:32:17.370 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8352e-05 2024-09-18T11:32:17.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.370 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:17.370 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594008+0000", 2024-09-18T11:32:17.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082840919999999998 2024-09-18T11:32:17.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.371 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:17.371 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594028+0000", 2024-09-18T11:32:17.371 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0869e-05 2024-09-18T11:32:17.371 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.371 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:17.371 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.371 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.371 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:168 3.1 e138/67 3:b64d7bad:::7feb22256bdcec217720fd46184ab43ced8923ed66ec6319c57f078d9f123262:head v 138'178, mlcod=136'174)", 2024-09-18T11:32:17.371 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:17.371 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.6841672110000001, 2024-09-18T11:32:17.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079290650000000008, 2024-09-18T11:32:17.372 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:17.372 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.372 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:17.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:17.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:17.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.372 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.372 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:17.372 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:17.372 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:17.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:17.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:17.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:17.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:17.373 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586191+0000", 2024-09-18T11:32:17.373 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3627000000000005e-05 2024-09-18T11:32:17.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.373 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:17.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586202+0000", 2024-09-18T11:32:17.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0488e-05 2024-09-18T11:32:17.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586207+0000", 2024-09-18T11:32:17.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1719999999999998e-06 2024-09-18T11:32:17.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.374 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.374 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:17.374 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586239+0000", 2024-09-18T11:32:17.374 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2570000000000002e-05 2024-09-18T11:32:17.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:17.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586271+0000", 2024-09-18T11:32:17.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1341999999999999e-05 2024-09-18T11:32:17.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:17.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594031+0000", 2024-09-18T11:32:17.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077597459999999997 2024-09-18T11:32:17.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:17.375 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:17.375 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:17.375 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594047+0000", 2024-09-18T11:32:17.375 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6858e-05 2024-09-18T11:32:17.376 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.376 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:17.376 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.376 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:17.376 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:17.376 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:17.378 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:32:17.378 INFO:tasks.rados.rados.0.smithi119.stdout:1783: done (8 left) 2024-09-18T11:32:17.378 INFO:tasks.rados.rados.0.smithi119.stdout:1784: done (7 left) 2024-09-18T11:32:17.378 INFO:tasks.rados.rados.0.smithi119.stdout:1785: done (6 left) 2024-09-18T11:32:17.378 INFO:tasks.rados.rados.0.smithi119.stdout:1786: done (5 left) 2024-09-18T11:32:17.378 INFO:tasks.rados.rados.0.smithi119.stdout:1788: done (4 left) 2024-09-18T11:32:17.379 INFO:tasks.rados.rados.0.smithi119.stdout:1789: done (3 left) 2024-09-18T11:32:17.379 INFO:tasks.rados.rados.0.smithi119.stdout:1790: done (2 left) 2024-09-18T11:32:17.379 INFO:tasks.rados.rados.0.smithi119.stdout:1791: done (1 left) 2024-09-18T11:32:17.379 INFO:tasks.rados.rados.0.smithi119.stdout:1792: done (0 left) 2024-09-18T11:32:17.379 INFO:tasks.rados.rados.0.smithi119.stdout:1793: snap_remove snap 52 2024-09-18T11:32:17.379 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:17.375+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:17.479 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:17.477+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:17.488 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:17.489 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:17.489 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:17.489 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:17.497 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:32:17.579 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:17.578+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:17.596 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:17.596 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:17.597 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:17.597 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:17.597 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:17.605 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:32:17.680 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:17.679+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:17.704 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:17.705 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:17.705 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:17.705 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:17.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.705 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:32:17.705 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:17.705 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.286109895, 2024-09-18T11:32:17.705 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:32:17.705 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:17.705 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.706 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:17.706 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:17.706 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.706 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:32:17.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.706 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:17.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:17.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:17.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:17.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:17.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:17.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:32:17.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:17.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:17.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:32:17.708 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:32:17.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.708 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.708 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:17.708 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:32:17.708 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:32:17.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.708 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.709 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.709 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:32:17.709 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:32:17.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.709 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.709 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:32:17.709 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:32:17.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.710 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:17.710 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:32:17.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:32:17.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.710 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:17.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:32:17.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:32:17.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:17.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:32:17.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:32:17.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.712 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:17.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:32:17.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:32:17.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.712 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:17.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:32:17.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:32:17.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:17.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:32:17.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:32:17.713 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.713 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:17.713 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.714 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:172 4.5 e78/22 4:a087a566:::smithi11939506-46:19 v 78'135, mlcod=77'132)", 2024-09-18T11:32:17.714 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:32:17.714 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 76.316373240999994, 2024-09-18T11:32:17.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010938658, 2024-09-18T11:32:17.714 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:17.714 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.714 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:17.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.715 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:17.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:32:17.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.715 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:17.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:32:17.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.716 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:17.716 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385823+0000", 2024-09-18T11:32:17.716 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:17.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.716 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:17.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385830+0000", 2024-09-18T11:32:17.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2790000000000001e-06 2024-09-18T11:32:17.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.717 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:17.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385843+0000", 2024-09-18T11:32:17.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2785e-05 2024-09-18T11:32:17.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385848+0000", 2024-09-18T11:32:17.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5570000000000003e-06 2024-09-18T11:32:17.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385914+0000", 2024-09-18T11:32:17.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5168999999999999e-05 2024-09-18T11:32:17.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:17.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385930+0000", 2024-09-18T11:32:17.720 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6884999999999998e-05 2024-09-18T11:32:17.720 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.720 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.720 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:17.720 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396721+0000", 2024-09-18T11:32:17.720 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010790137 2024-09-18T11:32:17.720 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.720 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.721 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:17.721 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396762+0000", 2024-09-18T11:32:17.721 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1635999999999999e-05 2024-09-18T11:32:17.721 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.721 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:17.721 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.721 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.722 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.722 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:32:17.722 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:17.722 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 76.315880488999994, 2024-09-18T11:32:17.722 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:32:17.722 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:17.722 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.722 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:17.722 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.722 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:17.722 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:17.722 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.722 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.723 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:17.723 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:17.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.723 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:17.723 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:17.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:17.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.723 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:17.723 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:32:17.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:32:17.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:17.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:32:17.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:32:17.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:32:17.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:32:17.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:32:17.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:32:17.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:17.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:32:17.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:32:17.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:17.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:32:17.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:32:17.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.726 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:17.726 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:32:17.726 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:32:17.726 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.726 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:17.726 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.726 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:17.726 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:17.726 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 68.729767882999994, 2024-09-18T11:32:17.726 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:32:17.726 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:17.727 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.727 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:17.727 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:17.727 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.727 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:32:17.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.727 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:17.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.727 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:17.727 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:17.727 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.728 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:17.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:17.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.728 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:17.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:32:17.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:17.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.728 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:17.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:32:17.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:32:17.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.729 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:17.729 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:32:17.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:32:17.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.729 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.729 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:32:17.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:32:17.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.729 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.730 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:32:17.730 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:32:17.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.730 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:17.730 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:32:17.730 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:32:17.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.730 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:17.730 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:32:17.730 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:32:17.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.731 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:17.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:32:17.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:32:17.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.731 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:17.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:32:17.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:32:17.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.731 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:17.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:32:17.732 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:32:17.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.732 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:17.732 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:32:17.732 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:32:17.732 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.732 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:17.732 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.732 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2956 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~2100908,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:17.732 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:17.733 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.674971632999998, 2024-09-18T11:32:17.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011863831, 2024-09-18T11:32:17.733 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:17.733 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:17.733 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:17.733 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:17.733 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.733 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2956 2024-09-18T11:32:17.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.733 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:17.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.733 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:17.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:17.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.734 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:17.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:17.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.734 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:17.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:17.735 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:17.735 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.735 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.735 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:17.735 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027228+0000", 2024-09-18T11:32:17.735 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:32:17.735 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.735 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.735 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:17.735 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027235+0000", 2024-09-18T11:32:17.735 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:32:17.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.736 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.736 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027239+0000", 2024-09-18T11:32:17.736 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.535e-06 2024-09-18T11:32:17.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.736 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.736 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027831+0000", 2024-09-18T11:32:17.736 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059244699999999996 2024-09-18T11:32:17.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:17.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027879+0000", 2024-09-18T11:32:17.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:32:17.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.737 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038098+0000", 2024-09-18T11:32:17.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010219137 2024-09-18T11:32:17.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.737 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:17.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038207+0000", 2024-09-18T11:32:17.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010951999999999999 2024-09-18T11:32:17.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:17.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039089+0000", 2024-09-18T11:32:17.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00088151699999999996 2024-09-18T11:32:17.738 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.738 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:17.738 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.738 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2957 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:17.738 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:17.739 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.674953848999998, 2024-09-18T11:32:17.739 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012041979, 2024-09-18T11:32:17.739 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:17.739 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:17.739 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:17.739 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:17.739 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.739 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2957 2024-09-18T11:32:17.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.739 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:17.739 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.739 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:17.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:17.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.740 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:17.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:17.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.740 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:17.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:17.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:17.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.741 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:17.741 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027245+0000", 2024-09-18T11:32:17.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:32:17.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.741 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:17.741 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027251+0000", 2024-09-18T11:32:17.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0090000000000004e-06 2024-09-18T11:32:17.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.741 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.741 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027255+0000", 2024-09-18T11:32:17.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:32:17.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027895+0000", 2024-09-18T11:32:17.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064076400000000005 2024-09-18T11:32:17.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:17.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027930+0000", 2024-09-18T11:32:17.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4334e-05 2024-09-18T11:32:17.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039111+0000", 2024-09-18T11:32:17.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181455999999999 2024-09-18T11:32:17.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:17.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039156+0000", 2024-09-18T11:32:17.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5256000000000002e-05 2024-09-18T11:32:17.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:17.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039285+0000", 2024-09-18T11:32:17.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012839200000000001 2024-09-18T11:32:17.743 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.743 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:17.744 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.744 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:978 4.b e132/67 4:d8d14c58:::smithi11939506-49:30 v 132'417, mlcod=128'414)", 2024-09-18T11:32:17.744 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:17.744 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.676922567, 2024-09-18T11:32:17.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011942183, 2024-09-18T11:32:17.744 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:17.744 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.744 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:17.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:17.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:17.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:17.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:17.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:17.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:17.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:17.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.746 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:17.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025278+0000", 2024-09-18T11:32:17.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9060000000000004e-06 2024-09-18T11:32:17.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.746 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:17.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025288+0000", 2024-09-18T11:32:17.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.028e-05 2024-09-18T11:32:17.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025293+0000", 2024-09-18T11:32:17.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2700000000000004e-06 2024-09-18T11:32:17.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025346+0000", 2024-09-18T11:32:17.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2862999999999999e-05 2024-09-18T11:32:17.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:17.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025370+0000", 2024-09-18T11:32:17.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4005000000000001e-05 2024-09-18T11:32:17.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:17.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037159+0000", 2024-09-18T11:32:17.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011788343 2024-09-18T11:32:17.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:17.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037216+0000", 2024-09-18T11:32:17.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7695000000000002e-05 2024-09-18T11:32:17.749 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.749 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:17.749 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.749 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3930 4.b e132/67 4:d3ae8215:::smithi11939506-41:head v 132'421, mlcod=128'414)", 2024-09-18T11:32:17.749 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:17.749 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.673374319000001, 2024-09-18T11:32:17.749 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015093554, 2024-09-18T11:32:17.749 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:17.749 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.749 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:17.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:17.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:17.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:17.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:17.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:17.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028821+0000", 2024-09-18T11:32:17.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:17.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.751 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:17.751 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029805+0000", 2024-09-18T11:32:17.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00098410899999999994 2024-09-18T11:32:17.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.751 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:17.751 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029817+0000", 2024-09-18T11:32:17.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2058e-05 2024-09-18T11:32:17.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.751 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.751 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029823+0000", 2024-09-18T11:32:17.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6450000000000002e-06 2024-09-18T11:32:17.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032330+0000", 2024-09-18T11:32:17.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025067710000000001 2024-09-18T11:32:17.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:17.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032349+0000", 2024-09-18T11:32:17.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9729e-05 2024-09-18T11:32:17.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:17.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043878+0000", 2024-09-18T11:32:17.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011528514 2024-09-18T11:32:17.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.753 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:17.753 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043916+0000", 2024-09-18T11:32:17.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8049000000000001e-05 2024-09-18T11:32:17.753 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.753 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:17.753 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.753 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:162 3.1 3:8c826a1f:::e682cfc1e82c88472b9acf1cd97978bc105aa035f10b685bb4cace95809b52bf:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:17.753 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:32:17.753 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.1212521609999992, 2024-09-18T11:32:17.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013234571000000001, 2024-09-18T11:32:17.754 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:17.754 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.754 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:17.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.754 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:17.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:32:17.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.754 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:17.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:32:17.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:17.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580943+0000", 2024-09-18T11:32:17.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:17.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:17.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582556+0000", 2024-09-18T11:32:17.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016132970000000001 2024-09-18T11:32:17.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:17.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582572+0000", 2024-09-18T11:32:17.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5846999999999999e-05 2024-09-18T11:32:17.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582578+0000", 2024-09-18T11:32:17.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4810000000000003e-06 2024-09-18T11:32:17.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582632+0000", 2024-09-18T11:32:17.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4156000000000001e-05 2024-09-18T11:32:17.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:17.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582819+0000", 2024-09-18T11:32:17.757 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000186431 2024-09-18T11:32:17.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:17.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583118+0000", 2024-09-18T11:32:17.757 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029880100000000001 2024-09-18T11:32:17.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:17.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590676+0000", 2024-09-18T11:32:17.757 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075588670000000004 2024-09-18T11:32:17.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.758 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:17.758 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594111+0000", 2024-09-18T11:32:17.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034348550000000001 2024-09-18T11:32:17.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.758 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:17.758 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594142+0000", 2024-09-18T11:32:17.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.099e-05 2024-09-18T11:32:17.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:17.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594179+0000", 2024-09-18T11:32:17.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6831e-05 2024-09-18T11:32:17.759 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.759 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:17.759 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.759 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3974 4.d 4:b5a463bf:::smithi11939506-20:head [write 665643~697786,stat] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:17.759 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:32:17.759 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.1201654780000005, 2024-09-18T11:32:17.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016621174999999998, 2024-09-18T11:32:17.760 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:17.760 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.760 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:17.760 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:17.760 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.760 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3974 2024-09-18T11:32:17.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.760 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:17.760 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.760 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:17.760 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:32:17.760 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.761 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:17.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:32:17.761 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.761 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:17.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582030+0000", 2024-09-18T11:32:17.761 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:17.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:17.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585185+0000", 2024-09-18T11:32:17.762 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003155644 2024-09-18T11:32:17.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.763 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:17.763 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585208+0000", 2024-09-18T11:32:17.763 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3013000000000001e-05 2024-09-18T11:32:17.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.763 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.763 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585215+0000", 2024-09-18T11:32:17.763 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7569999999999998e-06 2024-09-18T11:32:17.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.763 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.763 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585278+0000", 2024-09-18T11:32:17.763 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2699000000000004e-05 2024-09-18T11:32:17.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:17.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585374+0000", 2024-09-18T11:32:17.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.6062000000000003e-05 2024-09-18T11:32:17.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:32:17.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585577+0000", 2024-09-18T11:32:17.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000202926 2024-09-18T11:32:17.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:17.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596544+0000", 2024-09-18T11:32:17.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010966848 2024-09-18T11:32:17.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.765 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:17.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598594+0000", 2024-09-18T11:32:17.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020502049999999998 2024-09-18T11:32:17.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.765 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:17.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598625+0000", 2024-09-18T11:32:17.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1412999999999999e-05 2024-09-18T11:32:17.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.766 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:17.766 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598652+0000", 2024-09-18T11:32:17.766 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7166999999999999e-05 2024-09-18T11:32:17.766 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.766 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:17.766 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.766 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:166 3.1 3:a73e635b:::b6ee478f0f661ec7bac3d17c2ca2fdaff71a47570f64844a7ba79bbb2a0e8d9d:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:17.766 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:32:17.766 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.1196061119999996, 2024-09-18T11:32:17.766 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011711037000000001, 2024-09-18T11:32:17.767 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:17.767 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.767 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:17.767 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.767 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:17.767 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:32:17.767 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.767 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.767 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:17.767 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:32:17.767 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.768 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:17.768 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582590+0000", 2024-09-18T11:32:17.768 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:17.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.768 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:17.768 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582673+0000", 2024-09-18T11:32:17.768 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2861000000000005e-05 2024-09-18T11:32:17.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.768 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:17.769 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582683+0000", 2024-09-18T11:32:17.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0081999999999999e-05 2024-09-18T11:32:17.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.769 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.769 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.769 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582688+0000", 2024-09-18T11:32:17.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6870000000000002e-06 2024-09-18T11:32:17.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.769 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.769 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.769 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584266+0000", 2024-09-18T11:32:17.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001578423 2024-09-18T11:32:17.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.770 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:17.770 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584407+0000", 2024-09-18T11:32:17.770 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014144599999999999 2024-09-18T11:32:17.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.770 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:17.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584599+0000", 2024-09-18T11:32:17.771 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000191881 2024-09-18T11:32:17.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:17.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590680+0000", 2024-09-18T11:32:17.772 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060801479999999996 2024-09-18T11:32:17.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.772 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:17.772 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594224+0000", 2024-09-18T11:32:17.773 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035441700000000001 2024-09-18T11:32:17.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.773 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.773 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:17.773 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594269+0000", 2024-09-18T11:32:17.773 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5297999999999999e-05 2024-09-18T11:32:17.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.774 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:17.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594302+0000", 2024-09-18T11:32:17.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2703000000000002e-05 2024-09-18T11:32:17.774 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.774 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:17.774 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.775 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:168 3.1 3:b64d7bad:::7feb22256bdcec217720fd46184ab43ced8923ed66ec6319c57f078d9f123262:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:17.775 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:32:17.775 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.1193786600000006, 2024-09-18T11:32:17.775 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011589809, 2024-09-18T11:32:17.775 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:17.776 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.776 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:17.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.776 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:17.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:32:17.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:17.777 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:32:17.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.778 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:17.778 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582817+0000", 2024-09-18T11:32:17.778 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:17.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.779 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:17.779 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582903+0000", 2024-09-18T11:32:17.779 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5915999999999995e-05 2024-09-18T11:32:17.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.779 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.779 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:17.779 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582913+0000", 2024-09-18T11:32:17.779 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1409999999999994e-06 2024-09-18T11:32:17.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.779 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582917+0000", 2024-09-18T11:32:17.780 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6419999999999998e-06 2024-09-18T11:32:17.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585540+0000", 2024-09-18T11:32:17.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026228940000000002 2024-09-18T11:32:17.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.781 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:17.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585721+0000", 2024-09-18T11:32:17.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018086299999999999 2024-09-18T11:32:17.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:17.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585918+0000", 2024-09-18T11:32:17.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000196758 2024-09-18T11:32:17.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:17.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590683+0000", 2024-09-18T11:32:17.783 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047657020000000001 2024-09-18T11:32:17.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.783 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:17.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594354+0000", 2024-09-18T11:32:17.783 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003671068 2024-09-18T11:32:17.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.783 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:17.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594386+0000", 2024-09-18T11:32:17.784 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1426e-05 2024-09-18T11:32:17.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.784 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:17.784 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594408+0000", 2024-09-18T11:32:17.784 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1987000000000001e-05 2024-09-18T11:32:17.784 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.784 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:17.784 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.784 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:172 3.1 3:8c3b343b:::d73444b0abf5ea0abb7f4bfb783ec9a4695bac9fb180180b88c2eef6a2e03b80:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:17.784 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.583414+0000", 2024-09-18T11:32:17.785 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.1187826180000009, 2024-09-18T11:32:17.785 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011061609999999999, 2024-09-18T11:32:17.785 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:17.785 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.785 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:17.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.785 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:17.785 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583414+0000", 2024-09-18T11:32:17.785 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.785 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:17.785 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583414+0000", 2024-09-18T11:32:17.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:17.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583413+0000", 2024-09-18T11:32:17.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:17.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:17.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583525+0000", 2024-09-18T11:32:17.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000111289 2024-09-18T11:32:17.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:17.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583537+0000", 2024-09-18T11:32:17.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1755e-05 2024-09-18T11:32:17.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583542+0000", 2024-09-18T11:32:17.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4399999999999996e-06 2024-09-18T11:32:17.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.788 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.588499+0000", 2024-09-18T11:32:17.788 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004957347 2024-09-18T11:32:17.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.788 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:17.788 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.588684+0000", 2024-09-18T11:32:17.788 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000185002 2024-09-18T11:32:17.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.788 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:17.788 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.588889+0000", 2024-09-18T11:32:17.788 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020478200000000001 2024-09-18T11:32:17.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.591496+0000", 2024-09-18T11:32:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026070619999999998 2024-09-18T11:32:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594430+0000", 2024-09-18T11:32:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029342219999999998 2024-09-18T11:32:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:17.789 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594458+0000", 2024-09-18T11:32:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7243e-05 2024-09-18T11:32:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594476+0000", 2024-09-18T11:32:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8139e-05 2024-09-18T11:32:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.790 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:175 3.1 3:b5e86b88:::e55cf3d5cf782ed327a67e746b287b76d05f74f58a4f3726e399a564440ceaf6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:17.791 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:32:17.791 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.1185297189999996, 2024-09-18T11:32:17.791 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013709627, 2024-09-18T11:32:17.791 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:17.791 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.791 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:17.791 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.792 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:17.792 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:32:17.792 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.792 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:17.792 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:32:17.792 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.792 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:17.792 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583666+0000", 2024-09-18T11:32:17.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:17.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.793 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:17.793 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584041+0000", 2024-09-18T11:32:17.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037511499999999998 2024-09-18T11:32:17.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.793 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:17.793 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584055+0000", 2024-09-18T11:32:17.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3295999999999999e-05 2024-09-18T11:32:17.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584061+0000", 2024-09-18T11:32:17.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1630000000000001e-06 2024-09-18T11:32:17.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.589980+0000", 2024-09-18T11:32:17.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005918998 2024-09-18T11:32:17.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:17.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590117+0000", 2024-09-18T11:32:17.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013715400000000001 2024-09-18T11:32:17.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:17.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590270+0000", 2024-09-18T11:32:17.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015245600000000001 2024-09-18T11:32:17.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:17.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596513+0000", 2024-09-18T11:32:17.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062438729999999996 2024-09-18T11:32:17.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:17.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597328+0000", 2024-09-18T11:32:17.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081489599999999996 2024-09-18T11:32:17.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:17.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597358+0000", 2024-09-18T11:32:17.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9291000000000001e-05 2024-09-18T11:32:17.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:17.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597377+0000", 2024-09-18T11:32:17.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:32:17.797 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.797 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:17.797 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.797 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4061 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:17.797 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:32:17.797 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.4053510300000003, 2024-09-18T11:32:17.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013340967, 2024-09-18T11:32:17.797 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:17.797 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:17.797 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:17.797 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:17.797 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.798 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4061 2024-09-18T11:32:17.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.798 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:17.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:17.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:32:17.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:17.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:32:17.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:17.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296844+0000", 2024-09-18T11:32:17.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:17.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:17.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296854+0000", 2024-09-18T11:32:17.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0037000000000001e-05 2024-09-18T11:32:17.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:17.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296873+0000", 2024-09-18T11:32:17.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:32:17.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296881+0000", 2024-09-18T11:32:17.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3790000000000006e-06 2024-09-18T11:32:17.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296975+0000", 2024-09-18T11:32:17.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3536999999999996e-05 2024-09-18T11:32:17.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:17.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296982+0000", 2024-09-18T11:32:17.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6399999999999997e-06 2024-09-18T11:32:17.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301933+0000", 2024-09-18T11:32:17.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049507190000000001 2024-09-18T11:32:17.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:17.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301984+0000", 2024-09-18T11:32:17.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1439e-05 2024-09-18T11:32:17.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:17.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:32:17.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082022050000000006 2024-09-18T11:32:17.803 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.803 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:17.803 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.803 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4062 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:17.803 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:17.803 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.4053024750000001, 2024-09-18T11:32:17.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016297147000000001, 2024-09-18T11:32:17.803 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:17.803 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:17.804 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:17.804 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:17.804 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.804 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4062 2024-09-18T11:32:17.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.804 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:17.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.804 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:17.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:17.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:17.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:17.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:17.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:17.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:17.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:17.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296898+0000", 2024-09-18T11:32:17.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9419999999999997e-06 2024-09-18T11:32:17.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:17.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296906+0000", 2024-09-18T11:32:17.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3520000000000007e-06 2024-09-18T11:32:17.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296910+0000", 2024-09-18T11:32:17.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4469999999999999e-06 2024-09-18T11:32:17.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297032+0000", 2024-09-18T11:32:17.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012168900000000001 2024-09-18T11:32:17.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:17.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297038+0000", 2024-09-18T11:32:17.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9320000000000001e-06 2024-09-18T11:32:17.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310221+0000", 2024-09-18T11:32:17.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013182887000000001 2024-09-18T11:32:17.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:17.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310323+0000", 2024-09-18T11:32:17.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010170000000000001 2024-09-18T11:32:17.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:17.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313191+0000", 2024-09-18T11:32:17.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002868808 2024-09-18T11:32:17.810 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.810 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:17.810 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.810 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4063 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:17.810 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:17.810 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.4052815829999998, 2024-09-18T11:32:17.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017648712, 2024-09-18T11:32:17.810 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:17.810 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:17.810 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:17.810 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:17.811 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.811 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4063 2024-09-18T11:32:17.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.811 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:17.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:17.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:17.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:17.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:17.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.812 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:17.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:17.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:17.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.812 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:17.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296918+0000", 2024-09-18T11:32:17.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8100000000000002e-06 2024-09-18T11:32:17.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:17.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296924+0000", 2024-09-18T11:32:17.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2519999999999996e-06 2024-09-18T11:32:17.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296928+0000", 2024-09-18T11:32:17.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0500000000000002e-06 2024-09-18T11:32:17.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297062+0000", 2024-09-18T11:32:17.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013425500000000001 2024-09-18T11:32:17.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:17.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297069+0000", 2024-09-18T11:32:17.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5239999999999997e-06 2024-09-18T11:32:17.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313424+0000", 2024-09-18T11:32:17.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016355277000000001 2024-09-18T11:32:17.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:17.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313514+0000", 2024-09-18T11:32:17.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0581999999999994e-05 2024-09-18T11:32:17.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.815 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:17.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.314564+0000", 2024-09-18T11:32:17.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001049384 2024-09-18T11:32:17.815 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.815 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:17.815 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.815 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4065 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:17.815 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:32:17.815 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.4052361419999997, 2024-09-18T11:32:17.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013225997, 2024-09-18T11:32:17.816 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:17.816 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:17.816 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:17.816 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:17.816 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.816 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4065 2024-09-18T11:32:17.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.816 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:17.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.816 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:17.816 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:32:17.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:17.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:32:17.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:17.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296960+0000", 2024-09-18T11:32:17.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:17.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:17.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296963+0000", 2024-09-18T11:32:17.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2270000000000002e-06 2024-09-18T11:32:17.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:17.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296970+0000", 2024-09-18T11:32:17.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6220000000000003e-06 2024-09-18T11:32:17.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296973+0000", 2024-09-18T11:32:17.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3730000000000001e-06 2024-09-18T11:32:17.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297097+0000", 2024-09-18T11:32:17.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012352399999999999 2024-09-18T11:32:17.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:17.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297103+0000", 2024-09-18T11:32:17.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7119999999999997e-06 2024-09-18T11:32:17.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307831+0000", 2024-09-18T11:32:17.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010728794 2024-09-18T11:32:17.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:17.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307924+0000", 2024-09-18T11:32:17.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2099e-05 2024-09-18T11:32:17.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:17.822 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:32:17.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022630469999999998 2024-09-18T11:32:17.822 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.822 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:17.822 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.823 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4066 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:17.823 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:32:17.823 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.405219067, 2024-09-18T11:32:17.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015656389999999999, 2024-09-18T11:32:17.824 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:17.824 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:17.824 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:17.824 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:17.824 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.824 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4066 2024-09-18T11:32:17.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.825 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:17.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:17.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:32:17.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:17.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:32:17.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:17.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296977+0000", 2024-09-18T11:32:17.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:17.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:17.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296980+0000", 2024-09-18T11:32:17.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8830000000000002e-06 2024-09-18T11:32:17.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:17.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296985+0000", 2024-09-18T11:32:17.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:32:17.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296988+0000", 2024-09-18T11:32:17.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0419999999999998e-06 2024-09-18T11:32:17.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297120+0000", 2024-09-18T11:32:17.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131693 2024-09-18T11:32:17.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:17.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297125+0000", 2024-09-18T11:32:17.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1100000000000002e-06 2024-09-18T11:32:17.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310212+0000", 2024-09-18T11:32:17.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013087395999999999 2024-09-18T11:32:17.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:17.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310292+0000", 2024-09-18T11:32:17.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9980000000000003e-05 2024-09-18T11:32:17.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:17.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312634+0000", 2024-09-18T11:32:17.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002341756 2024-09-18T11:32:17.830 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.830 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:17.830 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.831 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4068 3.1 3:87e79ed0:::3526641c07d3efa989be9684355570f39c331008b8a02c9fa22b38a94b6c6b9f:head [write 0~1015808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:17.831 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:32:17.831 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.4014428500000005, 2024-09-18T11:32:17.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016392648999999999, 2024-09-18T11:32:17.831 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:17.831 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:17.831 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:17.831 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:17.832 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:17.832 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4068 2024-09-18T11:32:17.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.832 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:17.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:17.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:32:17.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:17.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:32:17.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:17.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:17.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300753+0000", 2024-09-18T11:32:17.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:17.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:17.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301922+0000", 2024-09-18T11:32:17.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001169432 2024-09-18T11:32:17.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:17.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301936+0000", 2024-09-18T11:32:17.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3574000000000001e-05 2024-09-18T11:32:17.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:17.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301941+0000", 2024-09-18T11:32:17.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6069999999999998e-06 2024-09-18T11:32:17.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:17.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310234+0000", 2024-09-18T11:32:17.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082924820000000003 2024-09-18T11:32:17.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:17.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310337+0000", 2024-09-18T11:32:17.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102673 2024-09-18T11:32:17.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:17.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310469+0000", 2024-09-18T11:32:17.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131982 2024-09-18T11:32:17.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:17.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312844+0000", 2024-09-18T11:32:17.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023758680000000002 2024-09-18T11:32:17.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:17.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317092+0000", 2024-09-18T11:32:17.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042477549999999998 2024-09-18T11:32:17.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:17.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317124+0000", 2024-09-18T11:32:17.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1968999999999998e-05 2024-09-18T11:32:17.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:17.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:17.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:17.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317147+0000", 2024-09-18T11:32:17.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2339000000000001e-05 2024-09-18T11:32:17.836 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.837 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:17.837 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.837 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:17.837 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:17.837 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:17.838 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:32:17.839 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:17.837+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:17.897 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:17.895+0000 7f6bf3052640 -1 osd.5 145 handle_osd_map injecting map crc failure 2024-09-18T11:32:17.939 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:17.937+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:17.978 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:17.978 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:17.978 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:17.978 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:17.986 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:32:18.040 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:18.038+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:18.124 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:18.124 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:18.124 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:18.125 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:18.125 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:18.132 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:32:18.176 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:18.175+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:18.233 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:18.233 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:18.234 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:18.234 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:18.234 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.234 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:32:18.234 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:32:18.234 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 148.20426111899999, 2024-09-18T11:32:18.235 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073601550000000002, 2024-09-18T11:32:18.235 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.235 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.235 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.235 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:32:18.235 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:32:18.235 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1 2024-09-18T11:32:18.235 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.235 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.235 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.236 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.236 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:32:18.236 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.236 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.236 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.236 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.236 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:32:18.236 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.236 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.236 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.236 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.237 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026524+0000", 2024-09-18T11:32:18.237 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999905 2024-09-18T11:32:18.237 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.237 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.237 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.237 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026541+0000", 2024-09-18T11:32:18.237 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7070000000000001e-05 2024-09-18T11:32:18.237 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.237 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.237 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.237 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026554+0000", 2024-09-18T11:32:18.237 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.288e-05 2024-09-18T11:32:18.237 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.238 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.238 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.238 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026564+0000", 2024-09-18T11:32:18.238 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0197000000000001e-05 2024-09-18T11:32:18.238 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.238 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.238 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.238 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026826+0000", 2024-09-18T11:32:18.238 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026212400000000003 2024-09-18T11:32:18.238 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.238 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.238 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:32:18.238 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026839+0000", 2024-09-18T11:32:18.238 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2831e-05 2024-09-18T11:32:18.239 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.239 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.239 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.239 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028605+0000", 2024-09-18T11:32:18.239 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001765794 2024-09-18T11:32:18.239 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.239 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.240 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.240 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028709+0000", 2024-09-18T11:32:18.240 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104348 2024-09-18T11:32:18.240 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.240 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.240 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:32:18.240 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028824+0000", 2024-09-18T11:32:18.240 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011527399999999999 2024-09-18T11:32:18.240 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.240 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.240 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:18.241 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030319+0000", 2024-09-18T11:32:18.241 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001494439 2024-09-18T11:32:18.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.241 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.241 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:18.241 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033805+0000", 2024-09-18T11:32:18.241 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034857820000000002 2024-09-18T11:32:18.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.241 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.241 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.241 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033856+0000", 2024-09-18T11:32:18.241 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1687999999999997e-05 2024-09-18T11:32:18.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.241 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.242 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.242 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033894+0000", 2024-09-18T11:32:18.242 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7481000000000002e-05 2024-09-18T11:32:18.242 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.242 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.242 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.242 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.242 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.242 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1118 3.2 e38/21 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head v 38'76, mlcod=38'74)", 2024-09-18T11:32:18.242 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:32:18.242 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 118.282850967, 2024-09-18T11:32:18.242 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006048688, 2024-09-18T11:32:18.242 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.242 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.243 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.243 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.243 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.243 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:32:18.243 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.243 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.243 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.243 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:32:18.243 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.243 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.243 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.244 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947941+0000", 2024-09-18T11:32:18.244 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:18.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.244 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.244 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.244 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949868+0000", 2024-09-18T11:32:18.244 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019267659999999999 2024-09-18T11:32:18.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.244 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.244 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.245 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949893+0000", 2024-09-18T11:32:18.245 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5239999999999999e-05 2024-09-18T11:32:18.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.245 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.245 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.245 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949901+0000", 2024-09-18T11:32:18.245 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8420000000000005e-06 2024-09-18T11:32:18.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.245 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.245 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.245 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949954+0000", 2024-09-18T11:32:18.245 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3464999999999997e-05 2024-09-18T11:32:18.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.245 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.246 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.246 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949974+0000", 2024-09-18T11:32:18.246 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9731000000000001e-05 2024-09-18T11:32:18.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.246 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.246 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.246 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953948+0000", 2024-09-18T11:32:18.246 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039735489999999998 2024-09-18T11:32:18.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.246 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.246 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.246 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953992+0000", 2024-09-18T11:32:18.246 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4700999999999998e-05 2024-09-18T11:32:18.246 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.247 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.247 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.247 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.247 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1662 4.1 e56/22 4:87078dfe:::smithi11939506-14:head v 56'73, mlcod=51'69)", 2024-09-18T11:32:18.247 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:32:18.247 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 100.869245932, 2024-09-18T11:32:18.247 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059408539999999998, 2024-09-18T11:32:18.247 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.247 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.247 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.247 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.247 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.248 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:32:18.248 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.248 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.248 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.248 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:32:18.248 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.248 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.248 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.248 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361549+0000", 2024-09-18T11:32:18.249 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:18.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.249 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.249 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361914+0000", 2024-09-18T11:32:18.249 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036569 2024-09-18T11:32:18.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.249 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.249 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361926+0000", 2024-09-18T11:32:18.249 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1960999999999999e-05 2024-09-18T11:32:18.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.249 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.249 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361937+0000", 2024-09-18T11:32:18.250 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0876999999999999e-05 2024-09-18T11:32:18.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.250 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.250 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.361986+0000", 2024-09-18T11:32:18.250 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8652000000000001e-05 2024-09-18T11:32:18.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.250 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.250 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.362012+0000", 2024-09-18T11:32:18.250 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5854999999999999e-05 2024-09-18T11:32:18.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.250 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.251 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.251 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367460+0000", 2024-09-18T11:32:18.251 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0054482100000000002 2024-09-18T11:32:18.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.251 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.251 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.251 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:37.367490+0000", 2024-09-18T11:32:18.251 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9909000000000001e-05 2024-09-18T11:32:18.251 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.251 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.251 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.251 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.251 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:269 4.7 e58/22 4:e73ba68f:::smithi11939506-1:13 v 58'72, mlcod=57'69)", 2024-09-18T11:32:18.252 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:32:18.252 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 98.965414034999995, 2024-09-18T11:32:18.252 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065512569999999996, 2024-09-18T11:32:18.252 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.252 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.252 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.252 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.252 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.252 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:32:18.252 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.252 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.252 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.253 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:32:18.253 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.253 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.253 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.253 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265380+0000", 2024-09-18T11:32:18.253 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:18.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.253 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.253 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.253 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265388+0000", 2024-09-18T11:32:18.253 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.174e-06 2024-09-18T11:32:18.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.253 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.254 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265398+0000", 2024-09-18T11:32:18.254 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2599999999999994e-06 2024-09-18T11:32:18.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.254 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.254 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265402+0000", 2024-09-18T11:32:18.254 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0289999999999999e-06 2024-09-18T11:32:18.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.254 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.254 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265434+0000", 2024-09-18T11:32:18.254 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2511000000000001e-05 2024-09-18T11:32:18.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.255 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.255 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265447+0000", 2024-09-18T11:32:18.255 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2947e-05 2024-09-18T11:32:18.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.255 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.255 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271892+0000", 2024-09-18T11:32:18.255 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064445930000000002 2024-09-18T11:32:18.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.256 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.256 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.256 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271932+0000", 2024-09-18T11:32:18.256 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0303999999999999e-05 2024-09-18T11:32:18.256 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.256 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.256 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.256 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.256 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1788 4.8 4:1898b15d:::smithi11939506-25:head [write 453822~797061] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:32:18.256 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:18.257 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 95.314981451999998, 2024-09-18T11:32:18.257 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094305159999999999, 2024-09-18T11:32:18.257 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.257 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.257 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.257 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.257 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.257 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1788 2024-09-18T11:32:18.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.257 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.257 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.257 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.257 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:18.258 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.258 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.258 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.258 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:18.258 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.258 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.258 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.258 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915811+0000", 2024-09-18T11:32:18.258 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:18.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.258 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.258 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.259 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916608+0000", 2024-09-18T11:32:18.259 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079635000000000003 2024-09-18T11:32:18.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.259 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.259 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.259 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916624+0000", 2024-09-18T11:32:18.259 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6735000000000002e-05 2024-09-18T11:32:18.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.259 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.259 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.259 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916630+0000", 2024-09-18T11:32:18.259 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5709999999999997e-06 2024-09-18T11:32:18.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.260 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.260 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.260 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916674+0000", 2024-09-18T11:32:18.260 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4540999999999999e-05 2024-09-18T11:32:18.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.260 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.260 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.260 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916751+0000", 2024-09-18T11:32:18.260 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6695999999999999e-05 2024-09-18T11:32:18.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.260 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.261 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:18.261 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916872+0000", 2024-09-18T11:32:18.261 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120648 2024-09-18T11:32:18.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.261 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.261 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:18.261 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.919076+0000", 2024-09-18T11:32:18.261 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022046829999999998 2024-09-18T11:32:18.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.261 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.261 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:18.261 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925151+0000", 2024-09-18T11:32:18.261 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060740239999999999 2024-09-18T11:32:18.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.262 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.262 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925198+0000", 2024-09-18T11:32:18.262 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7896999999999999e-05 2024-09-18T11:32:18.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.262 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.262 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925244+0000", 2024-09-18T11:32:18.262 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5398999999999997e-05 2024-09-18T11:32:18.262 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.262 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.262 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.263 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.263 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1934 4.2 e65/22 4:425afe9c:::smithi11939506-29:head v 65'44, mlcod=60'40)", 2024-09-18T11:32:18.263 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:32:18.263 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 89.930802682000007, 2024-09-18T11:32:18.263 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064585119999999996, 2024-09-18T11:32:18.263 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.263 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.263 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.263 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.263 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.263 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:32:18.264 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.264 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.264 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.264 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:32:18.264 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.264 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.264 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.264 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299990+0000", 2024-09-18T11:32:18.264 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:18.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.265 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.265 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.265 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300789+0000", 2024-09-18T11:32:18.265 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079851499999999997 2024-09-18T11:32:18.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.265 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.265 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.265 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300806+0000", 2024-09-18T11:32:18.265 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7096999999999999e-05 2024-09-18T11:32:18.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.265 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.266 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.266 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300813+0000", 2024-09-18T11:32:18.266 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5079999999999999e-06 2024-09-18T11:32:18.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.266 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.266 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.266 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300870+0000", 2024-09-18T11:32:18.266 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7178e-05 2024-09-18T11:32:18.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.266 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.266 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.266 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300897+0000", 2024-09-18T11:32:18.266 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6284000000000001e-05 2024-09-18T11:32:18.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.267 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.267 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.267 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306410+0000", 2024-09-18T11:32:18.267 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055135210000000004 2024-09-18T11:32:18.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.267 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.268 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.268 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306451+0000", 2024-09-18T11:32:18.268 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0379000000000001e-05 2024-09-18T11:32:18.268 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.268 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.269 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.269 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.269 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2452 4.3 4:ca2b2309:::smithi11939506-44:head [read 0~3462242,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:32:18.269 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:32:18.269 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 72.194027324999993, 2024-09-18T11:32:18.270 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060906450000000004, 2024-09-18T11:32:18.270 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.270 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:18.270 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.270 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.270 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.270 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2452 2024-09-18T11:32:18.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.271 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.271 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.271 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.271 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:32:18.271 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.271 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.271 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.271 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:32:18.272 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.272 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.272 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.272 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036765+0000", 2024-09-18T11:32:18.272 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:18.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.273 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.273 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.273 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036777+0000", 2024-09-18T11:32:18.273 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1422e-05 2024-09-18T11:32:18.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.273 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.274 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036793+0000", 2024-09-18T11:32:18.274 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6575999999999999e-05 2024-09-18T11:32:18.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.274 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.274 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.275 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036800+0000", 2024-09-18T11:32:18.275 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9940000000000003e-06 2024-09-18T11:32:18.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.275 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.275 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.275 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:32:18.276 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5847999999999999e-05 2024-09-18T11:32:18.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.276 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.276 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:18.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036853+0000", 2024-09-18T11:32:18.276 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0119999999999999e-06 2024-09-18T11:32:18.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.277 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038527+0000", 2024-09-18T11:32:18.277 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016732730000000001 2024-09-18T11:32:18.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.277 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.278 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038580+0000", 2024-09-18T11:32:18.278 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3443999999999999e-05 2024-09-18T11:32:18.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.278 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.278 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.278 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042858+0000", 2024-09-18T11:32:18.278 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042781049999999999 2024-09-18T11:32:18.278 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.278 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.278 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.278 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.279 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2453 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:32:18.279 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:32:18.279 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 72.193985092999995, 2024-09-18T11:32:18.279 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063798429999999996, 2024-09-18T11:32:18.279 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.279 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:18.279 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.279 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.279 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.279 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2453 2024-09-18T11:32:18.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.280 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.280 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:32:18.280 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.280 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:32:18.280 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.281 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.281 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036809+0000", 2024-09-18T11:32:18.281 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:18.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.281 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.281 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036812+0000", 2024-09-18T11:32:18.281 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8339999999999999e-06 2024-09-18T11:32:18.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.281 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.281 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036819+0000", 2024-09-18T11:32:18.281 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9890000000000003e-06 2024-09-18T11:32:18.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.282 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.282 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036823+0000", 2024-09-18T11:32:18.282 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8099999999999999e-06 2024-09-18T11:32:18.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.282 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.282 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036912+0000", 2024-09-18T11:32:18.282 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9083000000000001e-05 2024-09-18T11:32:18.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.282 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.282 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:18.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036932+0000", 2024-09-18T11:32:18.282 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0222000000000001e-05 2024-09-18T11:32:18.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.283 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042896+0000", 2024-09-18T11:32:18.283 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059637240000000001 2024-09-18T11:32:18.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.283 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042952+0000", 2024-09-18T11:32:18.283 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5877000000000001e-05 2024-09-18T11:32:18.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.283 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.284 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043190+0000", 2024-09-18T11:32:18.284 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00023774500000000001 2024-09-18T11:32:18.284 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.284 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.284 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.284 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.284 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2454 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:32:18.284 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:32:18.284 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 72.193949137000004, 2024-09-18T11:32:18.284 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066404469999999998, 2024-09-18T11:32:18.284 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.284 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:18.285 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.285 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.285 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.285 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2454 2024-09-18T11:32:18.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.285 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.285 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.285 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:32:18.285 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.286 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:32:18.286 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.286 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.286 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036845+0000", 2024-09-18T11:32:18.286 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:18.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.286 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.286 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036856+0000", 2024-09-18T11:32:18.286 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1338000000000001e-05 2024-09-18T11:32:18.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.287 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.287 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036863+0000", 2024-09-18T11:32:18.287 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4749999999999998e-06 2024-09-18T11:32:18.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.287 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.287 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036867+0000", 2024-09-18T11:32:18.287 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3839999999999999e-06 2024-09-18T11:32:18.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.288 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.288 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036946+0000", 2024-09-18T11:32:18.288 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9054000000000006e-05 2024-09-18T11:32:18.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.288 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.288 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:18.288 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036949+0000", 2024-09-18T11:32:18.288 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9390000000000002e-06 2024-09-18T11:32:18.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.288 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.288 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.288 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043213+0000", 2024-09-18T11:32:18.288 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062640090000000001 2024-09-18T11:32:18.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.289 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.289 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.289 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043261+0000", 2024-09-18T11:32:18.289 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7750999999999997e-05 2024-09-18T11:32:18.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.289 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.289 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.289 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043486+0000", 2024-09-18T11:32:18.289 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000224985 2024-09-18T11:32:18.289 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.289 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.290 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.290 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.290 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2553 4.3 4:ca2b2309:::smithi11939506-44:head [write 564768~630013,stat] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-18T11:32:18.290 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:32:18.290 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 68.033671503999997, 2024-09-18T11:32:18.290 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065686879999999996, 2024-09-18T11:32:18.290 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.290 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.290 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.290 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.290 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.290 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2553 2024-09-18T11:32:18.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.291 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.291 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.291 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:32:18.291 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.291 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.291 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:32:18.291 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.291 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197120+0000", 2024-09-18T11:32:18.292 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:18.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.292 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.292 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199000+0000", 2024-09-18T11:32:18.292 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018792080000000001 2024-09-18T11:32:18.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.292 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.292 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199044+0000", 2024-09-18T11:32:18.292 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4048999999999998e-05 2024-09-18T11:32:18.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.293 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.293 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199071+0000", 2024-09-18T11:32:18.293 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7288e-05 2024-09-18T11:32:18.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.293 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.293 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199119+0000", 2024-09-18T11:32:18.293 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8000999999999997e-05 2024-09-18T11:32:18.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.294 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199208+0000", 2024-09-18T11:32:18.294 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9209000000000006e-05 2024-09-18T11:32:18.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.294 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:18.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199385+0000", 2024-09-18T11:32:18.294 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000177095 2024-09-18T11:32:18.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.294 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:18.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.202282+0000", 2024-09-18T11:32:18.294 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028970469999999998 2024-09-18T11:32:18.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.295 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:18.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203578+0000", 2024-09-18T11:32:18.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012960490000000001 2024-09-18T11:32:18.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.295 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203638+0000", 2024-09-18T11:32:18.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9599000000000003e-05 2024-09-18T11:32:18.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.295 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203692+0000", 2024-09-18T11:32:18.296 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3908e-05 2024-09-18T11:32:18.296 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.296 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.296 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.296 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2752 4.3 4:ca2b2309:::smithi11939506-44:25 [read 0~3210104,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:32:18.296 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:32:18.296 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 60.343902344, 2024-09-18T11:32:18.296 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068715460000000001, 2024-09-18T11:32:18.296 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.297 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:18.297 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.297 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.297 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.297 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2752 2024-09-18T11:32:18.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.298 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.299 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:32:18.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.299 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:32:18.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:32:18.300 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:18.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886895+0000", 2024-09-18T11:32:18.300 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.993e-06 2024-09-18T11:32:18.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.301 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886901+0000", 2024-09-18T11:32:18.301 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1859999999999997e-06 2024-09-18T11:32:18.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.301 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886904+0000", 2024-09-18T11:32:18.301 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1920000000000001e-06 2024-09-18T11:32:18.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.302 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.302 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.892622+0000", 2024-09-18T11:32:18.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057175660000000003 2024-09-18T11:32:18.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.302 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.892693+0000", 2024-09-18T11:32:18.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1265000000000003e-05 2024-09-18T11:32:18.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.303 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.893764+0000", 2024-09-18T11:32:18.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001070753 2024-09-18T11:32:18.303 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.303 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.303 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.303 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2793 4.1 e92/22 4:87078dfe:::smithi11939506-14:head v 92'195, mlcod=92'191)", 2024-09-18T11:32:18.304 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:32:18.304 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 59.00600231, 2024-09-18T11:32:18.304 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062761989999999997, 2024-09-18T11:32:18.304 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.304 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.304 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.304 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:32:18.304 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:32:18.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:32:18.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:18.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224994+0000", 2024-09-18T11:32:18.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020197600000000001 2024-09-18T11:32:18.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225007+0000", 2024-09-18T11:32:18.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3791999999999999e-05 2024-09-18T11:32:18.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225013+0000", 2024-09-18T11:32:18.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3630000000000002e-06 2024-09-18T11:32:18.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225062+0000", 2024-09-18T11:32:18.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8925000000000003e-05 2024-09-18T11:32:18.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225086+0000", 2024-09-18T11:32:18.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4737999999999999e-05 2024-09-18T11:32:18.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231022+0000", 2024-09-18T11:32:18.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059360690000000004 2024-09-18T11:32:18.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231069+0000", 2024-09-18T11:32:18.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6252999999999999e-05 2024-09-18T11:32:18.307 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.307 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.308 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.308 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.308 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2876 4.3 4:cc46532a:::smithi11939506-33:2f [read 0~3680733,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:18.308 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:32:18.308 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 55.723933410000001, 2024-09-18T11:32:18.308 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068391210000000001, 2024-09-18T11:32:18.308 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.308 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:18.308 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.308 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.308 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.308 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2876 2024-09-18T11:32:18.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.309 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.309 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:32:18.309 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.309 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:32:18.309 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.309 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506860+0000", 2024-09-18T11:32:18.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:18.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506869+0000", 2024-09-18T11:32:18.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5539999999999998e-06 2024-09-18T11:32:18.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506880+0000", 2024-09-18T11:32:18.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1552999999999999e-05 2024-09-18T11:32:18.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506887+0000", 2024-09-18T11:32:18.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7329999999999997e-06 2024-09-18T11:32:18.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506937+0000", 2024-09-18T11:32:18.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0317e-05 2024-09-18T11:32:18.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:18.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506945+0000", 2024-09-18T11:32:18.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1550000000000007e-06 2024-09-18T11:32:18.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.312 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.509021+0000", 2024-09-18T11:32:18.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020759870000000001 2024-09-18T11:32:18.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.312 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.509123+0000", 2024-09-18T11:32:18.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010132900000000001 2024-09-18T11:32:18.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.313 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513700+0000", 2024-09-18T11:32:18.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0045777320000000002 2024-09-18T11:32:18.313 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.313 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.313 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.313 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2877 4.3 4:cc46532a:::smithi11939506-33:2f [read 0~3680733,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:18.313 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:18.313 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 55.723898996000003, 2024-09-18T11:32:18.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085715170000000007, 2024-09-18T11:32:18.313 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.313 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:18.314 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.314 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.314 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.314 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2877 2024-09-18T11:32:18.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.314 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.314 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.314 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:18.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.314 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:18.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:18.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:18.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506898+0000", 2024-09-18T11:32:18.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4760000000000001e-06 2024-09-18T11:32:18.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506903+0000", 2024-09-18T11:32:18.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:32:18.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506906+0000", 2024-09-18T11:32:18.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1769999999999998e-06 2024-09-18T11:32:18.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507007+0000", 2024-09-18T11:32:18.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101479 2024-09-18T11:32:18.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507015+0000", 2024-09-18T11:32:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8490000000000006e-06 2024-09-18T11:32:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513757+0000", 2024-09-18T11:32:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0067417800000000002 2024-09-18T11:32:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513867+0000", 2024-09-18T11:32:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000109987 2024-09-18T11:32:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515467+0000", 2024-09-18T11:32:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00160037 2024-09-18T11:32:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2878 4.3 4:cc46532a:::smithi11939506-33:2f [sparse-read 0~1770976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 55.723868541000002, 2024-09-18T11:32:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090602640000000002, 2024-09-18T11:32:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2878 2024-09-18T11:32:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506929+0000", 2024-09-18T11:32:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.163e-06 2024-09-18T11:32:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506936+0000", 2024-09-18T11:32:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5519999999999999e-06 2024-09-18T11:32:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506941+0000", 2024-09-18T11:32:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:32:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507036+0000", 2024-09-18T11:32:18.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5070999999999998e-05 2024-09-18T11:32:18.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507042+0000", 2024-09-18T11:32:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3959999999999997e-06 2024-09-18T11:32:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515504+0000", 2024-09-18T11:32:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084620300000000006 2024-09-18T11:32:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515625+0000", 2024-09-18T11:32:18.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012150999999999999 2024-09-18T11:32:18.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515986+0000", 2024-09-18T11:32:18.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036140100000000002 2024-09-18T11:32:18.325 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.326 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.326 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.326 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:686 4.7 e98/22 4:e5707c18:::smithi11939506-10:30 v 98'160, mlcod=97'155)", 2024-09-18T11:32:18.327 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:32:18.327 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.212026850999997, 2024-09-18T11:32:18.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074641239999999999, 2024-09-18T11:32:18.327 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.327 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:32:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:32:18.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018767+0000", 2024-09-18T11:32:18.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:18.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018773+0000", 2024-09-18T11:32:18.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1460000000000004e-06 2024-09-18T11:32:18.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018781+0000", 2024-09-18T11:32:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9780000000000005e-06 2024-09-18T11:32:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018785+0000", 2024-09-18T11:32:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0029999999999996e-06 2024-09-18T11:32:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018826+0000", 2024-09-18T11:32:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1616999999999998e-05 2024-09-18T11:32:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018843+0000", 2024-09-18T11:32:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6753999999999999e-05 2024-09-18T11:32:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026202+0000", 2024-09-18T11:32:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073592620000000001 2024-09-18T11:32:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026232+0000", 2024-09-18T11:32:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9779000000000001e-05 2024-09-18T11:32:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.4.0:621 4.1 e98/22 4:854aa293:::smithi11939506-26:35 v 98'215, mlcod=96'210)", 2024-09-18T11:32:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:32:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.211143411999998, 2024-09-18T11:32:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007014752, 2024-09-18T11:32:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:32:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:32:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:32:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019659+0000", 2024-09-18T11:32:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8229999999999995e-06 2024-09-18T11:32:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019671+0000", 2024-09-18T11:32:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2636000000000001e-05 2024-09-18T11:32:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019677+0000", 2024-09-18T11:32:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5470000000000004e-06 2024-09-18T11:32:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019710+0000", 2024-09-18T11:32:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3191999999999997e-05 2024-09-18T11:32:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019722+0000", 2024-09-18T11:32:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2357e-05 2024-09-18T11:32:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026639+0000", 2024-09-18T11:32:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0069167630000000003 2024-09-18T11:32:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026666+0000", 2024-09-18T11:32:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7087999999999999e-05 2024-09-18T11:32:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3775 4.3 4:c70c3162:::smithi11939506-35:head [write 796731~461096,stat] snapc 51=[51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b,30] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-18T11:32:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:58.217457+0000", 2024-09-18T11:32:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 20.013337889999999, 2024-09-18T11:32:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064394090000000001, 2024-09-18T11:32:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3775 2024-09-18T11:32:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217457+0000", 2024-09-18T11:32:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217457+0000", 2024-09-18T11:32:18.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217455+0000", 2024-09-18T11:32:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217965+0000", 2024-09-18T11:32:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00050991999999999997 2024-09-18T11:32:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217977+0000", 2024-09-18T11:32:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1701e-05 2024-09-18T11:32:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217981+0000", 2024-09-18T11:32:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5600000000000004e-06 2024-09-18T11:32:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.218033+0000", 2024-09-18T11:32:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1728999999999999e-05 2024-09-18T11:32:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.218147+0000", 2024-09-18T11:32:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000114066 2024-09-18T11:32:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.218364+0000", 2024-09-18T11:32:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021718700000000001 2024-09-18T11:32:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.223702+0000", 2024-09-18T11:32:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005338036 2024-09-18T11:32:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.223812+0000", 2024-09-18T11:32:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000110011 2024-09-18T11:32:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.223868+0000", 2024-09-18T11:32:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5989999999999998e-05 2024-09-18T11:32:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.223896+0000", 2024-09-18T11:32:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7820000000000001e-05 2024-09-18T11:32:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3963 4.3 4:c70c3162:::smithi11939506-35:head [tier-promote] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:32:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.8549715310000003, 2024-09-18T11:32:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010675591999999999, 2024-09-18T11:32:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3963 2024-09-18T11:32:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:32:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:32:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375821+0000", 2024-09-18T11:32:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375829+0000", 2024-09-18T11:32:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7989999999999994e-06 2024-09-18T11:32:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375843+0000", 2024-09-18T11:32:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:32:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375849+0000", 2024-09-18T11:32:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1419999999999998e-06 2024-09-18T11:32:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375889+0000", 2024-09-18T11:32:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0197999999999997e-05 2024-09-18T11:32:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375894+0000", 2024-09-18T11:32:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0139999999999998e-06 2024-09-18T11:32:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.377699+0000", 2024-09-18T11:32:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001804688 2024-09-18T11:32:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.377776+0000", 2024-09-18T11:32:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7081000000000003e-05 2024-09-18T11:32:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.386499+0000", 2024-09-18T11:32:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008722746 2024-09-18T11:32:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1025 4.7 e138/22 4:e73ba68f:::smithi11939506-1:head v 138'281, mlcod=137'278)", 2024-09-18T11:32:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.587274+0000", 2024-09-18T11:32:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.643521217, 2024-09-18T11:32:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094231360000000004, 2024-09-18T11:32:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.587274+0000", 2024-09-18T11:32:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.587274+0000", 2024-09-18T11:32:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.587273+0000", 2024-09-18T11:32:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590258+0000", 2024-09-18T11:32:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002984998 2024-09-18T11:32:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590273+0000", 2024-09-18T11:32:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4999e-05 2024-09-18T11:32:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590283+0000", 2024-09-18T11:32:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3619999999999995e-06 2024-09-18T11:32:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590333+0000", 2024-09-18T11:32:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9978999999999997e-05 2024-09-18T11:32:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590352+0000", 2024-09-18T11:32:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9477999999999999e-05 2024-09-18T11:32:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596664+0000", 2024-09-18T11:32:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063120679999999997 2024-09-18T11:32:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596697+0000", 2024-09-18T11:32:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2456999999999998e-05 2024-09-18T11:32:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.355 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:18.355 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T11:32:18.357 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:32:18.358 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:18.321+0000 7f3157e90640 -1 osd.4 146 handle_osd_map injecting map crc failure 2024-09-18T11:32:18.358 INFO:tasks.rados.rados.0.smithi119.stdout:1793: done (0 left) 2024-09-18T11:32:18.358 INFO:tasks.rados.rados.0.smithi119.stdout:1794: set_chunk oid 27 target oid 2024-09-18T11:32:18.358 INFO:tasks.rados.rados.0.smithi119.stdout:1794: set_chunk oid 27 offset: 1552384 length: 263168 target oid 9b94b83334ada38b48b53f19ef2c45abc9f84fb723564671290a556f75d6fdc8 offset: 0 snap -1 2024-09-18T11:32:18.358 INFO:tasks.rados.rados.0.smithi119.stdout:1795: read oid 21 snap -1 2024-09-18T11:32:18.358 INFO:tasks.rados.rados.0.smithi119.stdout:1795: expect (ObjNum 311 snap 43 seq_num 311) 2024-09-18T11:32:18.358 INFO:tasks.rados.rados.0.smithi119.stdout:1796: set_chunk oid 20 target oid 2024-09-18T11:32:18.359 INFO:tasks.rados.rados.0.smithi119.stdout:1796: set_chunk oid 20 offset: 3053056 length: 735232 target oid ecc42ffd354f98bc072a943dfde710d701be4a7fdd4e2a7f16082bf7289e5c2c offset: 0 snap -1 2024-09-18T11:32:18.359 INFO:tasks.rados.rados.0.smithi119.stdout:1797: delete oid 48 current snap is 56 2024-09-18T11:32:18.359 INFO:tasks.rados.rados.0.smithi119.stdout:1795: expect (ObjNum 311 snap 43 seq_num 311) 2024-09-18T11:32:18.359 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:18.354+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:18.457 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:18.454+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:18.496 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:18.497 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:18.497 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:18.497 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:18.504 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:32:18.512 INFO:tasks.rados.rados.0.smithi119.stdout:1794: finishing set_chunk to oid 27 2024-09-18T11:32:18.512 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 232 (ObjNum 328 snap 45 seq_num 328) dirty exists 2024-09-18T11:32:18.513 INFO:tasks.rados.rados.0.smithi119.stdout:1796: finishing set_chunk to oid 20 2024-09-18T11:32:18.513 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 20 v 273 (ObjNum 349 snap 54 seq_num 349) dirty exists 2024-09-18T11:32:18.513 INFO:tasks.rados.rados.0.smithi119.stdout:1794: done (3 left) 2024-09-18T11:32:18.513 INFO:tasks.rados.rados.0.smithi119.stdout:1795: done (2 left) 2024-09-18T11:32:18.513 INFO:tasks.rados.rados.0.smithi119.stdout:1796: done (1 left) 2024-09-18T11:32:18.513 INFO:tasks.rados.rados.0.smithi119.stdout:1797: done (0 left) 2024-09-18T11:32:18.514 INFO:tasks.rados.rados.0.smithi119.stdout:1798: write (excl) oid 38 current snap is 56 2024-09-18T11:32:18.514 INFO:tasks.rados.rados.0.smithi119.stdout:1798: seq_num 365 ranges {534555=650829,1659104=475451} 2024-09-18T11:32:18.514 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:32:18.520 INFO:tasks.rados.rados.0.smithi119.stdout:1798: writing smithi11939506-38 from 534555 to 1185384 tid 1 2024-09-18T11:32:18.527 INFO:tasks.rados.rados.0.smithi119.stdout:1798: writing smithi11939506-38 from 1659104 to 2134555 tid 2 2024-09-18T11:32:18.527 INFO:tasks.rados.rados.0.smithi119.stdout:1799: read oid 3 snap -1 2024-09-18T11:32:18.527 INFO:tasks.rados.rados.0.smithi119.stdout:1799: expect (ObjNum 243 snap 28 seq_num 243) 2024-09-18T11:32:18.527 INFO:tasks.rados.rados.0.smithi119.stdout:1800: snap_remove snap 42 2024-09-18T11:32:18.557 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:18.555+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:18.604 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:18.604 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:18.604 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:18.604 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:18.604 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:18.612 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:32:18.630 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:18.631 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:18.646 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:18.646 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:18.658 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:18.657+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:18.661 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:18.661 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:18.676 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:18.676 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:18.691 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:18.691 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:18.706 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:18.706 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:18.713 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:18.714 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:18.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:18.714 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:18.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.714 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:32:18.714 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:18.714 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 127.59464634699999, 2024-09-18T11:32:18.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:32:18.715 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.715 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:18.715 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.715 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.715 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.715 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:32:18.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.715 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.715 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:18.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.716 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.716 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:18.716 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.716 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.716 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:32:18.716 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:18.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.717 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:32:18.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:32:18.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.717 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:32:18.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:32:18.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:32:18.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:32:18.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:32:18.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:32:18.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:18.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:32:18.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:32:18.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:32:18.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:32:18.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:32:18.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:32:18.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:32:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:32:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:32:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 118.04162094599999, 2024-09-18T11:32:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:32:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:32:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.722 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:18.722 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.722 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.722 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.722 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.722 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:18.722 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.722 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.722 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.722 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.723 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:32:18.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:18.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.723 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.723 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:32:18.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:32:18.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.723 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.723 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:32:18.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:32:18.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:32:18.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:32:18.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:32:18.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:32:18.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:32:18.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:32:18.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.726 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.726 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:32:18.726 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:32:18.726 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.727 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.727 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.727 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2207 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1410296~761389] snapc 24=[24,1e,1c,1b,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T11:32:18.728 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:32:18.728 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 81.288468283, 2024-09-18T11:32:18.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008410743, 2024-09-18T11:32:18.728 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.728 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.728 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.729 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.729 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.729 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2207 2024-09-18T11:32:18.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.729 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.729 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.729 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:32:18.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.730 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.730 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:32:18.730 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.730 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422496+0000", 2024-09-18T11:32:18.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:18.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.731 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423190+0000", 2024-09-18T11:32:18.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00069405499999999995 2024-09-18T11:32:18.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.732 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.732 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423204+0000", 2024-09-18T11:32:18.732 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:32:18.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.732 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.732 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423208+0000", 2024-09-18T11:32:18.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:32:18.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.733 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423293+0000", 2024-09-18T11:32:18.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4524999999999995e-05 2024-09-18T11:32:18.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.734 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423394+0000", 2024-09-18T11:32:18.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101395 2024-09-18T11:32:18.735 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.735 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.735 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:18.735 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423579+0000", 2024-09-18T11:32:18.735 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018461000000000001 2024-09-18T11:32:18.735 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.735 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.736 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:18.736 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.428574+0000", 2024-09-18T11:32:18.736 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049944960000000002 2024-09-18T11:32:18.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.736 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:18.736 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430809+0000", 2024-09-18T11:32:18.736 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022351820000000001 2024-09-18T11:32:18.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.737 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430862+0000", 2024-09-18T11:32:18.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3189999999999997e-05 2024-09-18T11:32:18.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.737 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430908+0000", 2024-09-18T11:32:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5973999999999997e-05 2024-09-18T11:32:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2339 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 77.324163654000003, 2024-09-18T11:32:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011809992, 2024-09-18T11:32:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.740 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.740 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2339 2024-09-18T11:32:18.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.740 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.740 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:18.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.741 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.741 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:18.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.741 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.741 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:32:18.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:18.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.741 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:32:18.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4710000000000001e-06 2024-09-18T11:32:18.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386824+0000", 2024-09-18T11:32:18.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6104e-05 2024-09-18T11:32:18.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386836+0000", 2024-09-18T11:32:18.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1827999999999999e-05 2024-09-18T11:32:18.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388390+0000", 2024-09-18T11:32:18.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015539499999999999 2024-09-18T11:32:18.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:18.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388443+0000", 2024-09-18T11:32:18.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3785000000000001e-05 2024-09-18T11:32:18.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398238+0000", 2024-09-18T11:32:18.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097947989999999999 2024-09-18T11:32:18.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.744 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.744 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398315+0000", 2024-09-18T11:32:18.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6909999999999994e-05 2024-09-18T11:32:18.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.744 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.744 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398612+0000", 2024-09-18T11:32:18.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029671999999999999 2024-09-18T11:32:18.744 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.744 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.744 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.745 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2340 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:18.745 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:18.745 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 77.324120342000001, 2024-09-18T11:32:18.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012086448, 2024-09-18T11:32:18.745 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.745 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:18.745 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.745 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.745 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.745 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2340 2024-09-18T11:32:18.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.746 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.746 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:18.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.746 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:18.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:18.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:18.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386848+0000", 2024-09-18T11:32:18.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3069999999999998e-06 2024-09-18T11:32:18.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386855+0000", 2024-09-18T11:32:18.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0979999999999996e-06 2024-09-18T11:32:18.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386859+0000", 2024-09-18T11:32:18.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9360000000000001e-06 2024-09-18T11:32:18.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388466+0000", 2024-09-18T11:32:18.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001607154 2024-09-18T11:32:18.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.749 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:18.749 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388507+0000", 2024-09-18T11:32:18.749 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1093000000000001e-05 2024-09-18T11:32:18.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.749 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.749 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398647+0000", 2024-09-18T11:32:18.749 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010139686 2024-09-18T11:32:18.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.749 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398718+0000", 2024-09-18T11:32:18.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1138999999999998e-05 2024-09-18T11:32:18.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398931+0000", 2024-09-18T11:32:18.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021356899999999999 2024-09-18T11:32:18.750 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.750 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.750 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.751 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2341 4.5 4:aa575018:::smithi11939506-11:head [read 0~2487436,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:18.751 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:18.751 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 77.324102120000006, 2024-09-18T11:32:18.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013751648999999999, 2024-09-18T11:32:18.751 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.751 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:18.751 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.751 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.751 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.751 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2341 2024-09-18T11:32:18.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.751 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:18.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:18.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:18.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:18.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.753 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.753 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386866+0000", 2024-09-18T11:32:18.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4249999999999998e-06 2024-09-18T11:32:18.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.753 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.753 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386872+0000", 2024-09-18T11:32:18.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.003e-06 2024-09-18T11:32:18.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.754 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:32:18.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.507e-06 2024-09-18T11:32:18.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.754 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388520+0000", 2024-09-18T11:32:18.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001644146 2024-09-18T11:32:18.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.754 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:18.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388561+0000", 2024-09-18T11:32:18.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1486999999999998e-05 2024-09-18T11:32:18.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398963+0000", 2024-09-18T11:32:18.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010401353 2024-09-18T11:32:18.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.399046+0000", 2024-09-18T11:32:18.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3141999999999998e-05 2024-09-18T11:32:18.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.400615+0000", 2024-09-18T11:32:18.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015690960000000001 2024-09-18T11:32:18.756 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.756 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.756 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.756 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2529 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:18.756 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:32:18.756 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 69.537628855999998, 2024-09-18T11:32:18.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079268120000000001, 2024-09-18T11:32:18.757 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.757 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.757 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.757 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.757 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.757 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2529 2024-09-18T11:32:18.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.757 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:32:18.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.758 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.758 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:32:18.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.758 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.758 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173336+0000", 2024-09-18T11:32:18.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:18.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173339+0000", 2024-09-18T11:32:18.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9019999999999999e-06 2024-09-18T11:32:18.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173381+0000", 2024-09-18T11:32:18.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1836999999999999e-05 2024-09-18T11:32:18.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.760 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.760 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173390+0000", 2024-09-18T11:32:18.760 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.302e-06 2024-09-18T11:32:18.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.760 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.760 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.760 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174507+0000", 2024-09-18T11:32:18.760 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001116868 2024-09-18T11:32:18.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.760 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.760 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.760 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174603+0000", 2024-09-18T11:32:18.760 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5598000000000002e-05 2024-09-18T11:32:18.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.761 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:18.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174759+0000", 2024-09-18T11:32:18.761 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015616299999999999 2024-09-18T11:32:18.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.761 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:18.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178943+0000", 2024-09-18T11:32:18.761 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041846180000000002 2024-09-18T11:32:18.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:18.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181175+0000", 2024-09-18T11:32:18.762 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00223215 2024-09-18T11:32:18.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181229+0000", 2024-09-18T11:32:18.762 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3585999999999998e-05 2024-09-18T11:32:18.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181263+0000", 2024-09-18T11:32:18.763 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4292999999999998e-05 2024-09-18T11:32:18.763 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.763 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.763 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.763 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2530 4.5 4:af4ba75a:::smithi11939506-37:head [rollback 25] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:18.763 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:32:18.763 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 69.537566549999994, 2024-09-18T11:32:18.763 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080198800000000001, 2024-09-18T11:32:18.763 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.763 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.763 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.764 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.764 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.764 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2530 2024-09-18T11:32:18.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.764 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:32:18.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:32:18.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.765 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173398+0000", 2024-09-18T11:32:18.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:18.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.765 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173403+0000", 2024-09-18T11:32:18.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4389999999999999e-06 2024-09-18T11:32:18.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.766 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.766 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173414+0000", 2024-09-18T11:32:18.766 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0411000000000001e-05 2024-09-18T11:32:18.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.766 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.766 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173420+0000", 2024-09-18T11:32:18.766 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9610000000000002e-06 2024-09-18T11:32:18.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.767 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.767 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176390+0000", 2024-09-18T11:32:18.767 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029706289999999998 2024-09-18T11:32:18.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.767 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.767 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.767 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176501+0000", 2024-09-18T11:32:18.767 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011090300000000001 2024-09-18T11:32:18.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.767 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.767 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:18.767 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176827+0000", 2024-09-18T11:32:18.768 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00032526899999999999 2024-09-18T11:32:18.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.768 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:18.768 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178958+0000", 2024-09-18T11:32:18.768 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021314860000000001 2024-09-18T11:32:18.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.768 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:18.768 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181318+0000", 2024-09-18T11:32:18.768 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002359571 2024-09-18T11:32:18.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.769 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.769 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181372+0000", 2024-09-18T11:32:18.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3884000000000001e-05 2024-09-18T11:32:18.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.769 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.769 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.769 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181419+0000", 2024-09-18T11:32:18.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7148000000000003e-05 2024-09-18T11:32:18.769 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.769 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.769 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.770 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2531 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:18.770 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:18.770 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 69.537540426999996, 2024-09-18T11:32:18.770 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080687239999999993, 2024-09-18T11:32:18.770 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.770 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.770 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.770 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.770 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.770 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2531 2024-09-18T11:32:18.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.771 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:18.771 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:18.771 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.772 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:18.772 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:18.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.772 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.772 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173429+0000", 2024-09-18T11:32:18.772 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2989999999999998e-06 2024-09-18T11:32:18.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.772 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.772 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173436+0000", 2024-09-18T11:32:18.772 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1799999999999999e-06 2024-09-18T11:32:18.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.773 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.773 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.773 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173441+0000", 2024-09-18T11:32:18.773 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8810000000000004e-06 2024-09-18T11:32:18.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.773 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.773 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.773 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178267+0000", 2024-09-18T11:32:18.773 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048257969999999997 2024-09-18T11:32:18.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.773 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.774 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178388+0000", 2024-09-18T11:32:18.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121779 2024-09-18T11:32:18.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.774 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:18.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178599+0000", 2024-09-18T11:32:18.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000210094 2024-09-18T11:32:18.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.774 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:18.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.179339+0000", 2024-09-18T11:32:18.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00074063799999999995 2024-09-18T11:32:18.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.775 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:18.775 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181444+0000", 2024-09-18T11:32:18.775 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021044100000000001 2024-09-18T11:32:18.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.775 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.775 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181470+0000", 2024-09-18T11:32:18.775 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6415999999999999e-05 2024-09-18T11:32:18.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.776 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181494+0000", 2024-09-18T11:32:18.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3672e-05 2024-09-18T11:32:18.776 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.776 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.776 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.776 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2910 4.5 4:af4ba75a:::smithi11939506-37:head [write 786627~795991,stat] snapc 37=[37,35,34,30,2f,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:18.776 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:32:18.776 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 55.707482597999999, 2024-09-18T11:32:18.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084093710000000006, 2024-09-18T11:32:18.777 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.777 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.777 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.777 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.777 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.777 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2910 2024-09-18T11:32:18.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.777 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.777 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:32:18.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.778 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.778 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:32:18.778 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.778 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.778 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003481+0000", 2024-09-18T11:32:18.778 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:18.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.778 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.779 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004300+0000", 2024-09-18T11:32:18.779 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081934200000000003 2024-09-18T11:32:18.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.779 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.779 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.779 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004324+0000", 2024-09-18T11:32:18.779 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4618999999999999e-05 2024-09-18T11:32:18.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.779 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.779 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.779 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004335+0000", 2024-09-18T11:32:18.779 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.092e-05 2024-09-18T11:32:18.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004385+0000", 2024-09-18T11:32:18.780 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9709999999999997e-05 2024-09-18T11:32:18.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004470+0000", 2024-09-18T11:32:18.780 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4864000000000006e-05 2024-09-18T11:32:18.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:18.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004612+0000", 2024-09-18T11:32:18.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014241500000000001 2024-09-18T11:32:18.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.781 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:18.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011193+0000", 2024-09-18T11:32:18.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006580654 2024-09-18T11:32:18.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.781 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:18.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011788+0000", 2024-09-18T11:32:18.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059533600000000002 2024-09-18T11:32:18.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011842+0000", 2024-09-18T11:32:18.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3180000000000002e-05 2024-09-18T11:32:18.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011892+0000", 2024-09-18T11:32:18.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0618000000000002e-05 2024-09-18T11:32:18.782 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.782 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.782 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.783 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2953 4.a 4:588debc2:::smithi11939506-18:head [assert-version v113,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:18.783 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:32:18.783 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.684167080000002, 2024-09-18T11:32:18.783 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083931279999999997, 2024-09-18T11:32:18.783 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.783 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.783 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.783 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.783 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.784 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2953 2024-09-18T11:32:18.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.784 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.784 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.784 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:32:18.784 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.784 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.784 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:32:18.784 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.785 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.785 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026796+0000", 2024-09-18T11:32:18.785 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:18.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.785 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.785 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026806+0000", 2024-09-18T11:32:18.785 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9529999999999994e-06 2024-09-18T11:32:18.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.785 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026824+0000", 2024-09-18T11:32:18.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7901999999999999e-05 2024-09-18T11:32:18.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026832+0000", 2024-09-18T11:32:18.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8450000000000003e-06 2024-09-18T11:32:18.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026883+0000", 2024-09-18T11:32:18.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0883000000000001e-05 2024-09-18T11:32:18.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026980+0000", 2024-09-18T11:32:18.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7526000000000001e-05 2024-09-18T11:32:18.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:32:18.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032722+0000", 2024-09-18T11:32:18.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057414689999999999 2024-09-18T11:32:18.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:18.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.034230+0000", 2024-09-18T11:32:18.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001508035 2024-09-18T11:32:18.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.788 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:18.788 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035090+0000", 2024-09-18T11:32:18.788 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00086047300000000005 2024-09-18T11:32:18.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.788 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.788 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035147+0000", 2024-09-18T11:32:18.788 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6598000000000003e-05 2024-09-18T11:32:18.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.789 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.789 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035191+0000", 2024-09-18T11:32:18.789 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4747999999999999e-05 2024-09-18T11:32:18.789 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.790 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.790 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.790 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.790 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3105 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 3e=[3e,3d,3b,37,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-18T11:32:18.790 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:18.791 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.692754376000003, 2024-09-18T11:32:18.791 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011735906000000001, 2024-09-18T11:32:18.791 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.791 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:18.791 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.791 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.791 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.792 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3105 2024-09-18T11:32:18.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.792 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.792 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.792 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:18.792 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.793 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.793 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:18.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:18.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:18.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018212+0000", 2024-09-18T11:32:18.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6759999999999999e-06 2024-09-18T11:32:18.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018216+0000", 2024-09-18T11:32:18.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-06 2024-09-18T11:32:18.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018218+0000", 2024-09-18T11:32:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6110000000000001e-06 2024-09-18T11:32:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018246+0000", 2024-09-18T11:32:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7557000000000001e-05 2024-09-18T11:32:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018250+0000", 2024-09-18T11:32:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4299999999999999e-06 2024-09-18T11:32:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021364+0000", 2024-09-18T11:32:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031135389999999998 2024-09-18T11:32:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021494+0000", 2024-09-18T11:32:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000130333 2024-09-18T11:32:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.029947+0000", 2024-09-18T11:32:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084526150000000001 2024-09-18T11:32:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3323 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:32:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.594066331999997, 2024-09-18T11:32:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0088051689999999998, 2024-09-18T11:32:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3323 2024-09-18T11:32:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:32:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:32:18.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116897+0000", 2024-09-18T11:32:18.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:18.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116908+0000", 2024-09-18T11:32:18.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1511e-05 2024-09-18T11:32:18.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116925+0000", 2024-09-18T11:32:18.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7028e-05 2024-09-18T11:32:18.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116932+0000", 2024-09-18T11:32:18.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5919999999999997e-06 2024-09-18T11:32:18.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116988+0000", 2024-09-18T11:32:18.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5928999999999999e-05 2024-09-18T11:32:18.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:18.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116995+0000", 2024-09-18T11:32:18.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7409999999999992e-06 2024-09-18T11:32:18.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.804 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122390+0000", 2024-09-18T11:32:18.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053941889999999998 2024-09-18T11:32:18.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.804 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122453+0000", 2024-09-18T11:32:18.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3833999999999999e-05 2024-09-18T11:32:18.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:18.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122581+0000", 2024-09-18T11:32:18.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012751200000000001 2024-09-18T11:32:18.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:18.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125034+0000", 2024-09-18T11:32:18.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024525609999999998 2024-09-18T11:32:18.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:18.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125607+0000", 2024-09-18T11:32:18.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00057356499999999995 2024-09-18T11:32:18.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125655+0000", 2024-09-18T11:32:18.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8179000000000001e-05 2024-09-18T11:32:18.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125704+0000", 2024-09-18T11:32:18.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8950999999999999e-05 2024-09-18T11:32:18.807 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.807 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.807 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.807 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3324 4.5 4:a087a566:::smithi11939506-46:head [rollback 46] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:18.807 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:18.807 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.594023773000004, 2024-09-18T11:32:18.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010927767999999999, 2024-09-18T11:32:18.807 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.807 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.807 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.808 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.808 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.808 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3324 2024-09-18T11:32:18.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.808 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:18.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:18.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116941+0000", 2024-09-18T11:32:18.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:18.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116949+0000", 2024-09-18T11:32:18.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4660000000000002e-06 2024-09-18T11:32:18.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.810 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116960+0000", 2024-09-18T11:32:18.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1484999999999999e-05 2024-09-18T11:32:18.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.810 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116964+0000", 2024-09-18T11:32:18.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2810000000000002e-06 2024-09-18T11:32:18.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.810 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.810 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117053+0000", 2024-09-18T11:32:18.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8747999999999998e-05 2024-09-18T11:32:18.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:18.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117059+0000", 2024-09-18T11:32:18.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.246e-06 2024-09-18T11:32:18.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123604+0000", 2024-09-18T11:32:18.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065445319999999996 2024-09-18T11:32:18.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.812 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123663+0000", 2024-09-18T11:32:18.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8740999999999999e-05 2024-09-18T11:32:18.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.812 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:18.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123803+0000", 2024-09-18T11:32:18.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014049200000000001 2024-09-18T11:32:18.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:18.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127600+0000", 2024-09-18T11:32:18.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037971179999999999 2024-09-18T11:32:18.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:18.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127814+0000", 2024-09-18T11:32:18.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021333600000000001 2024-09-18T11:32:18.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127847+0000", 2024-09-18T11:32:18.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3188999999999997e-05 2024-09-18T11:32:18.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127869+0000", 2024-09-18T11:32:18.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2515e-05 2024-09-18T11:32:18.815 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.815 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.815 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.815 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3325 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:18.815 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:18.815 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.593976587, 2024-09-18T11:32:18.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181030999999999, 2024-09-18T11:32:18.815 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.815 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.815 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.816 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.816 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.816 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3325 2024-09-18T11:32:18.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.817 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:18.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:18.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116988+0000", 2024-09-18T11:32:18.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:18.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116991+0000", 2024-09-18T11:32:18.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9890000000000001e-06 2024-09-18T11:32:18.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116998+0000", 2024-09-18T11:32:18.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5309999999999998e-06 2024-09-18T11:32:18.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117005+0000", 2024-09-18T11:32:18.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6420000000000001e-06 2024-09-18T11:32:18.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.822 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.822 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117091+0000", 2024-09-18T11:32:18.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6835999999999993e-05 2024-09-18T11:32:18.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:18.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117097+0000", 2024-09-18T11:32:18.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:32:18.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124717+0000", 2024-09-18T11:32:18.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076204899999999997 2024-09-18T11:32:18.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124773+0000", 2024-09-18T11:32:18.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5470999999999998e-05 2024-09-18T11:32:18.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:18.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124877+0000", 2024-09-18T11:32:18.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010426500000000001 2024-09-18T11:32:18.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:18.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127901+0000", 2024-09-18T11:32:18.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030243449999999999 2024-09-18T11:32:18.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:18.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128115+0000", 2024-09-18T11:32:18.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021355300000000001 2024-09-18T11:32:18.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128145+0000", 2024-09-18T11:32:18.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0290999999999998e-05 2024-09-18T11:32:18.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128170+0000", 2024-09-18T11:32:18.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4811e-05 2024-09-18T11:32:18.827 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.827 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.827 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.827 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3474 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1872245~637761] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:18.827 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:18.827 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.267199910000002, 2024-09-18T11:32:18.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0092038580000000005, 2024-09-18T11:32:18.827 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.827 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.827 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.828 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.828 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.828 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3474 2024-09-18T11:32:18.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.828 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:18.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:18.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443764+0000", 2024-09-18T11:32:18.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:18.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444359+0000", 2024-09-18T11:32:18.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059519 2024-09-18T11:32:18.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444374+0000", 2024-09-18T11:32:18.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4949e-05 2024-09-18T11:32:18.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444382+0000", 2024-09-18T11:32:18.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1369999999999996e-06 2024-09-18T11:32:18.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444428+0000", 2024-09-18T11:32:18.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6783000000000003e-05 2024-09-18T11:32:18.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444507+0000", 2024-09-18T11:32:18.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8415000000000001e-05 2024-09-18T11:32:18.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:18.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444638+0000", 2024-09-18T11:32:18.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013129 2024-09-18T11:32:18.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:18.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.449059+0000", 2024-09-18T11:32:18.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004420719 2024-09-18T11:32:18.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:18.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452909+0000", 2024-09-18T11:32:18.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003850773 2024-09-18T11:32:18.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452944+0000", 2024-09-18T11:32:18.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-05 2024-09-18T11:32:18.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452969+0000", 2024-09-18T11:32:18.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5795999999999998e-05 2024-09-18T11:32:18.833 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.833 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.833 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.834 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3475 4.5 4:a56ea1fb:::smithi11939506-48:head [setxattr _header (57) in=7b,truncate 3042100] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:18.834 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:32:18.834 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.266571194999997, 2024-09-18T11:32:18.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086700660000000006, 2024-09-18T11:32:18.834 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.834 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:18.834 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.834 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.834 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.834 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3475 2024-09-18T11:32:18.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.834 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:32:18.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:32:18.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444393+0000", 2024-09-18T11:32:18.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:18.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444398+0000", 2024-09-18T11:32:18.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4909999999999999e-06 2024-09-18T11:32:18.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444405+0000", 2024-09-18T11:32:18.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5329999999999997e-06 2024-09-18T11:32:18.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444410+0000", 2024-09-18T11:32:18.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2470000000000002e-06 2024-09-18T11:32:18.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446046+0000", 2024-09-18T11:32:18.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001636722 2024-09-18T11:32:18.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446121+0000", 2024-09-18T11:32:18.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4604000000000006e-05 2024-09-18T11:32:18.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:18.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446274+0000", 2024-09-18T11:32:18.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015350300000000001 2024-09-18T11:32:18.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:18.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452884+0000", 2024-09-18T11:32:18.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066096439999999996 2024-09-18T11:32:18.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:18.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453000+0000", 2024-09-18T11:32:18.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000116204 2024-09-18T11:32:18.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:18.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453032+0000", 2024-09-18T11:32:18.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1327e-05 2024-09-18T11:32:18.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453064+0000", 2024-09-18T11:32:18.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2645000000000003e-05 2024-09-18T11:32:18.839 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.840 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.840 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.840 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3476 4.5 4:a56ea1fb:::smithi11939506-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:18.840 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:18.840 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.266531927999999, 2024-09-18T11:32:18.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087596800000000006, 2024-09-18T11:32:18.840 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.840 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:18.840 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.841 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.841 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.841 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3476 2024-09-18T11:32:18.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.841 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:18.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:18.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:18.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:18.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444436+0000", 2024-09-18T11:32:18.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9330000000000001e-06 2024-09-18T11:32:18.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444443+0000", 2024-09-18T11:32:18.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9369999999999998e-06 2024-09-18T11:32:18.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444447+0000", 2024-09-18T11:32:18.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1309999999999999e-06 2024-09-18T11:32:18.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446599+0000", 2024-09-18T11:32:18.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021521190000000001 2024-09-18T11:32:18.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:18.844 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446675+0000", 2024-09-18T11:32:18.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6099000000000004e-05 2024-09-18T11:32:18.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.844 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453088+0000", 2024-09-18T11:32:18.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006413307 2024-09-18T11:32:18.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453132+0000", 2024-09-18T11:32:18.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3548999999999999e-05 2024-09-18T11:32:18.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453193+0000", 2024-09-18T11:32:18.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1116000000000002e-05 2024-09-18T11:32:18.845 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.846 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.846 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.846 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3928 4.5 4:af4ba75a:::smithi11939506-37:head [sparse-read 0~1543958] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:18.846 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:18.846 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 15.691735017999999, 2024-09-18T11:32:18.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080807250000000004, 2024-09-18T11:32:18.846 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.846 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:18.846 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.846 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.846 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.847 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3928 2024-09-18T11:32:18.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.847 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:18.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:18.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:18.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:18.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019232+0000", 2024-09-18T11:32:18.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1050000000000002e-06 2024-09-18T11:32:18.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019236+0000", 2024-09-18T11:32:18.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7910000000000002e-06 2024-09-18T11:32:18.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019238+0000", 2024-09-18T11:32:18.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4339999999999999e-06 2024-09-18T11:32:18.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019326+0000", 2024-09-18T11:32:18.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8097999999999996e-05 2024-09-18T11:32:18.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:18.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019330+0000", 2024-09-18T11:32:18.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9349999999999996e-06 2024-09-18T11:32:18.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027035+0000", 2024-09-18T11:32:18.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0077047269999999998 2024-09-18T11:32:18.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027110+0000", 2024-09-18T11:32:18.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4443000000000005e-05 2024-09-18T11:32:18.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027311+0000", 2024-09-18T11:32:18.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000201528 2024-09-18T11:32:18.853 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.853 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.853 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.854 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3929 4.5 4:af4ba75a:::smithi11939506-37:head [sparse-read 0~1543958] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:18.854 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:18.854 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 15.691704465000001, 2024-09-18T11:32:18.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084586699999999997, 2024-09-18T11:32:18.854 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:18.854 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:18.854 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:18.855 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:18.855 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:18.855 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3929 2024-09-18T11:32:18.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.855 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:18.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:18.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:18.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:18.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:18.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:18.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:18.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:18.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:18.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:18.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019262+0000", 2024-09-18T11:32:18.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8029999999999999e-06 2024-09-18T11:32:18.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:18.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019267+0000", 2024-09-18T11:32:18.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1119999999999998e-06 2024-09-18T11:32:18.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:18.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019269+0000", 2024-09-18T11:32:18.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4269999999999998e-06 2024-09-18T11:32:18.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019344+0000", 2024-09-18T11:32:18.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4979000000000002e-05 2024-09-18T11:32:18.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:18.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019348+0000", 2024-09-18T11:32:18.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3930000000000001e-06 2024-09-18T11:32:18.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:18.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027385+0000", 2024-09-18T11:32:18.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080368409999999994 2024-09-18T11:32:18.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:18.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027463+0000", 2024-09-18T11:32:18.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7316000000000003e-05 2024-09-18T11:32:18.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:18.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:18.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:18.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027720+0000", 2024-09-18T11:32:18.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000257068 2024-09-18T11:32:18.861 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.861 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.861 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.861 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:18.861 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:18.861 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:18.862 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:18.863 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:18.863 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:18.864 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:32:18.865 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:18.865 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:18.860+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:18.963 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:18.962+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:19.003 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:19.003 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:19.011 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:32:19.064 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:19.062+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:19.142 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-18T11:32:19.143 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T11:32:19.143 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-18T11:32:19.143 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T11:32:19.149 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:19.149 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:19.157 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:32:19.200 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:19.199+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:19.256 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:19.257 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:19.265 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:32:19.306 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:19.305+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:19.325 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:19.323+0000 7f3157e90640 -1 osd.4 147 handle_osd_map injecting map crc failure 2024-09-18T11:32:19.326 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is unset 2024-09-18T11:32:19.327 INFO:tasks.rados.rados.0.smithi119.stdout:1798: finishing write tid 1 to smithi11939506-38 2024-09-18T11:32:19.327 INFO:tasks.rados.rados.0.smithi119.stdout:1799: expect (ObjNum 243 snap 28 seq_num 243) 2024-09-18T11:32:19.368 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:19.366+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:19.375 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:19.375 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:19.384 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:32:19.424 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:32:19.424 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:19.424 INFO:teuthology.orchestra.run.smithi119.stdout: "epoch": 148, 2024-09-18T11:32:19.424 INFO:teuthology.orchestra.run.smithi119.stdout: "fsid": "f4cca259-b87a-45cc-8995-7e9fca891a9f", 2024-09-18T11:32:19.424 INFO:teuthology.orchestra.run.smithi119.stdout: "created": "2024-09-18T11:29:38.205487+0000", 2024-09-18T11:32:19.424 INFO:teuthology.orchestra.run.smithi119.stdout: "modified": "2024-09-18T11:32:19.317671+0000", 2024-09-18T11:32:19.424 INFO:teuthology.orchestra.run.smithi119.stdout: "last_up_change": "2024-09-18T11:31:49.142260+0000", 2024-09-18T11:32:19.424 INFO:teuthology.orchestra.run.smithi119.stdout: "last_in_change": "2024-09-18T11:30:49.312763+0000", 2024-09-18T11:32:19.424 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T11:32:19.425 INFO:teuthology.orchestra.run.smithi119.stdout: "flags_num": 5799936, 2024-09-18T11:32:19.425 INFO:teuthology.orchestra.run.smithi119.stdout: "flags_set": [ 2024-09-18T11:32:19.425 INFO:teuthology.orchestra.run.smithi119.stdout: "pglog_hardlimit", 2024-09-18T11:32:19.425 INFO:teuthology.orchestra.run.smithi119.stdout: "purged_snapdirs", 2024-09-18T11:32:19.425 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_deletes", 2024-09-18T11:32:19.425 INFO:teuthology.orchestra.run.smithi119.stdout: "sortbitwise" 2024-09-18T11:32:19.425 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-09-18T11:32:19.425 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_version": 6, 2024-09-18T11:32:19.425 INFO:teuthology.orchestra.run.smithi119.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T11:32:19.425 INFO:teuthology.orchestra.run.smithi119.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T11:32:19.425 INFO:teuthology.orchestra.run.smithi119.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T11:32:19.425 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_snapshot": "", 2024-09-18T11:32:19.426 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_max": 4, 2024-09-18T11:32:19.426 INFO:teuthology.orchestra.run.smithi119.stdout: "max_osd": 8, 2024-09-18T11:32:19.426 INFO:teuthology.orchestra.run.smithi119.stdout: "require_min_compat_client": "luminous", 2024-09-18T11:32:19.426 INFO:teuthology.orchestra.run.smithi119.stdout: "min_compat_client": "jewel", 2024-09-18T11:32:19.426 INFO:teuthology.orchestra.run.smithi119.stdout: "require_osd_release": "quincy", 2024-09-18T11:32:19.426 INFO:teuthology.orchestra.run.smithi119.stdout: "pools": [ 2024-09-18T11:32:19.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.426 INFO:teuthology.orchestra.run.smithi119.stdout: "pool": 1, 2024-09-18T11:32:19.426 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_name": ".mgr", 2024-09-18T11:32:19.426 INFO:teuthology.orchestra.run.smithi119.stdout: "create_time": "2024-09-18T11:29:47.951262+0000", 2024-09-18T11:32:19.426 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-09-18T11:32:19.426 INFO:teuthology.orchestra.run.smithi119.stdout: "flags_names": "hashpspool", 2024-09-18T11:32:19.427 INFO:teuthology.orchestra.run.smithi119.stdout: "type": 1, 2024-09-18T11:32:19.427 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 2, 2024-09-18T11:32:19.427 INFO:teuthology.orchestra.run.smithi119.stdout: "min_size": 2, 2024-09-18T11:32:19.427 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_rule": 0, 2024-09-18T11:32:19.427 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_count": 0, 2024-09-18T11:32:19.427 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_target": 0, 2024-09-18T11:32:19.427 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T11:32:19.427 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T11:32:19.427 INFO:teuthology.orchestra.run.smithi119.stdout: "object_hash": 2, 2024-09-18T11:32:19.427 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_autoscale_mode": "off", 2024-09-18T11:32:19.427 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num": 1, 2024-09-18T11:32:19.427 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_placement_num": 1, 2024-09-18T11:32:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_placement_num_target": 1, 2024-09-18T11:32:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num_target": 1, 2024-09-18T11:32:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num_pending": 1, 2024-09-18T11:32:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: "last_pg_merge_meta": { 2024-09-18T11:32:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: "source_pgid": "0.0", 2024-09-18T11:32:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: "ready_epoch": 0, 2024-09-18T11:32:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: "last_epoch_started": 0, 2024-09-18T11:32:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: "last_epoch_clean": 0, 2024-09-18T11:32:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: "source_version": "0'0", 2024-09-18T11:32:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: "target_version": "0'0" 2024-09-18T11:32:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: "last_change": "16", 2024-09-18T11:32:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend": "0", 2024-09-18T11:32:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T11:32:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T11:32:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: "auid": 0, 2024-09-18T11:32:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_mode": "selfmanaged", 2024-09-18T11:32:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_seq": 0, 2024-09-18T11:32:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_epoch": 0, 2024-09-18T11:32:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_snaps": [], 2024-09-18T11:32:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: "removed_snaps": "[]", 2024-09-18T11:32:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: "quota_max_bytes": 0, 2024-09-18T11:32:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: "quota_max_objects": 0, 2024-09-18T11:32:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: "tiers": [], 2024-09-18T11:32:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: "tier_of": -1, 2024-09-18T11:32:19.430 INFO:teuthology.orchestra.run.smithi119.stdout: "read_tier": -1, 2024-09-18T11:32:19.430 INFO:teuthology.orchestra.run.smithi119.stdout: "write_tier": -1, 2024-09-18T11:32:19.430 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_mode": "none", 2024-09-18T11:32:19.430 INFO:teuthology.orchestra.run.smithi119.stdout: "target_max_bytes": 0, 2024-09-18T11:32:19.430 INFO:teuthology.orchestra.run.smithi119.stdout: "target_max_objects": 0, 2024-09-18T11:32:19.430 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T11:32:19.430 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T11:32:19.430 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T11:32:19.430 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_min_flush_age": 0, 2024-09-18T11:32:19.430 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_min_evict_age": 0, 2024-09-18T11:32:19.430 INFO:teuthology.orchestra.run.smithi119.stdout: "erasure_code_profile": "", 2024-09-18T11:32:19.430 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_params": { 2024-09-18T11:32:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "none" 2024-09-18T11:32:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_period": 0, 2024-09-18T11:32:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_count": 0, 2024-09-18T11:32:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "use_gmt_hitset": true, 2024-09-18T11:32:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "min_read_recency_for_promote": 0, 2024-09-18T11:32:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "min_write_recency_for_promote": 0, 2024-09-18T11:32:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T11:32:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_search_last_n": 0, 2024-09-18T11:32:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "grade_table": [], 2024-09-18T11:32:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "stripe_width": 0, 2024-09-18T11:32:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "expected_num_objects": 0, 2024-09-18T11:32:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "fast_read": false, 2024-09-18T11:32:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: "options": { 2024-09-18T11:32:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num_max": 32, 2024-09-18T11:32:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num_min": 1 2024-09-18T11:32:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: "application_metadata": { 2024-09-18T11:32:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": {} 2024-09-18T11:32:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: "pool": 2, 2024-09-18T11:32:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_name": "rbd", 2024-09-18T11:32:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: "create_time": "2024-09-18T11:29:48.866238+0000", 2024-09-18T11:32:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 8193, 2024-09-18T11:32:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T11:32:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "type": 1, 2024-09-18T11:32:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 2, 2024-09-18T11:32:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "min_size": 2, 2024-09-18T11:32:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_rule": 0, 2024-09-18T11:32:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_count": 0, 2024-09-18T11:32:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_target": 0, 2024-09-18T11:32:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T11:32:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T11:32:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "object_hash": 2, 2024-09-18T11:32:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_autoscale_mode": "off", 2024-09-18T11:32:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num": 8, 2024-09-18T11:32:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_placement_num": 8, 2024-09-18T11:32:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_placement_num_target": 8, 2024-09-18T11:32:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num_target": 8, 2024-09-18T11:32:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num_pending": 8, 2024-09-18T11:32:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: "last_pg_merge_meta": { 2024-09-18T11:32:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: "source_pgid": "0.0", 2024-09-18T11:32:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: "ready_epoch": 0, 2024-09-18T11:32:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: "last_epoch_started": 0, 2024-09-18T11:32:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: "last_epoch_clean": 0, 2024-09-18T11:32:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: "source_version": "0'0", 2024-09-18T11:32:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: "target_version": "0'0" 2024-09-18T11:32:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: "last_change": "18", 2024-09-18T11:32:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend": "0", 2024-09-18T11:32:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T11:32:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T11:32:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: "auid": 0, 2024-09-18T11:32:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_mode": "selfmanaged", 2024-09-18T11:32:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_seq": 3, 2024-09-18T11:32:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_epoch": 18, 2024-09-18T11:32:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_snaps": [], 2024-09-18T11:32:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: "removed_snaps": "[]", 2024-09-18T11:32:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: "quota_max_bytes": 0, 2024-09-18T11:32:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: "quota_max_objects": 0, 2024-09-18T11:32:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: "tiers": [], 2024-09-18T11:32:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: "tier_of": -1, 2024-09-18T11:32:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: "read_tier": -1, 2024-09-18T11:32:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: "write_tier": -1, 2024-09-18T11:32:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_mode": "none", 2024-09-18T11:32:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "target_max_bytes": 0, 2024-09-18T11:32:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "target_max_objects": 0, 2024-09-18T11:32:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T11:32:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T11:32:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T11:32:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_min_flush_age": 0, 2024-09-18T11:32:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_min_evict_age": 0, 2024-09-18T11:32:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "erasure_code_profile": "", 2024-09-18T11:32:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_params": { 2024-09-18T11:32:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "none" 2024-09-18T11:32:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_period": 0, 2024-09-18T11:32:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_count": 0, 2024-09-18T11:32:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "use_gmt_hitset": true, 2024-09-18T11:32:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "min_read_recency_for_promote": 0, 2024-09-18T11:32:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "min_write_recency_for_promote": 0, 2024-09-18T11:32:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T11:32:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_search_last_n": 0, 2024-09-18T11:32:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "grade_table": [], 2024-09-18T11:32:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "stripe_width": 0, 2024-09-18T11:32:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "expected_num_objects": 0, 2024-09-18T11:32:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "fast_read": false, 2024-09-18T11:32:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "options": {}, 2024-09-18T11:32:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "application_metadata": { 2024-09-18T11:32:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd": {} 2024-09-18T11:32:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: "pool": 3, 2024-09-18T11:32:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_name": "low_tier", 2024-09-18T11:32:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: "create_time": "2024-09-18T11:29:59.343964+0000", 2024-09-18T11:32:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-09-18T11:32:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: "flags_names": "hashpspool", 2024-09-18T11:32:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: "type": 1, 2024-09-18T11:32:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 2, 2024-09-18T11:32:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: "min_size": 2, 2024-09-18T11:32:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_rule": 0, 2024-09-18T11:32:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_count": 0, 2024-09-18T11:32:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_target": 0, 2024-09-18T11:32:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T11:32:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T11:32:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "object_hash": 2, 2024-09-18T11:32:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_autoscale_mode": "off", 2024-09-18T11:32:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num": 4, 2024-09-18T11:32:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_placement_num": 4, 2024-09-18T11:32:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_placement_num_target": 4, 2024-09-18T11:32:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num_target": 4, 2024-09-18T11:32:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num_pending": 4, 2024-09-18T11:32:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "last_pg_merge_meta": { 2024-09-18T11:32:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "source_pgid": "0.0", 2024-09-18T11:32:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "ready_epoch": 0, 2024-09-18T11:32:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "last_epoch_started": 0, 2024-09-18T11:32:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "last_epoch_clean": 0, 2024-09-18T11:32:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "source_version": "0'0", 2024-09-18T11:32:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "target_version": "0'0" 2024-09-18T11:32:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "last_change": "23", 2024-09-18T11:32:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend": "0", 2024-09-18T11:32:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T11:32:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T11:32:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "auid": 0, 2024-09-18T11:32:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_mode": "selfmanaged", 2024-09-18T11:32:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_seq": 0, 2024-09-18T11:32:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_epoch": 0, 2024-09-18T11:32:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_snaps": [], 2024-09-18T11:32:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: "removed_snaps": "[]", 2024-09-18T11:32:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: "quota_max_bytes": 0, 2024-09-18T11:32:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: "quota_max_objects": 0, 2024-09-18T11:32:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: "tiers": [], 2024-09-18T11:32:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: "tier_of": -1, 2024-09-18T11:32:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: "read_tier": -1, 2024-09-18T11:32:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: "write_tier": -1, 2024-09-18T11:32:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_mode": "none", 2024-09-18T11:32:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: "target_max_bytes": 0, 2024-09-18T11:32:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: "target_max_objects": 0, 2024-09-18T11:32:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T11:32:19.442 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T11:32:19.442 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T11:32:19.442 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_min_flush_age": 0, 2024-09-18T11:32:19.442 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_min_evict_age": 0, 2024-09-18T11:32:19.442 INFO:teuthology.orchestra.run.smithi119.stdout: "erasure_code_profile": "", 2024-09-18T11:32:19.442 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_params": { 2024-09-18T11:32:19.442 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "none" 2024-09-18T11:32:19.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.442 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_period": 0, 2024-09-18T11:32:19.442 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_count": 0, 2024-09-18T11:32:19.442 INFO:teuthology.orchestra.run.smithi119.stdout: "use_gmt_hitset": true, 2024-09-18T11:32:19.442 INFO:teuthology.orchestra.run.smithi119.stdout: "min_read_recency_for_promote": 0, 2024-09-18T11:32:19.442 INFO:teuthology.orchestra.run.smithi119.stdout: "min_write_recency_for_promote": 0, 2024-09-18T11:32:19.442 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T11:32:19.443 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_search_last_n": 0, 2024-09-18T11:32:19.443 INFO:teuthology.orchestra.run.smithi119.stdout: "grade_table": [], 2024-09-18T11:32:19.443 INFO:teuthology.orchestra.run.smithi119.stdout: "stripe_width": 0, 2024-09-18T11:32:19.443 INFO:teuthology.orchestra.run.smithi119.stdout: "expected_num_objects": 0, 2024-09-18T11:32:19.443 INFO:teuthology.orchestra.run.smithi119.stdout: "fast_read": false, 2024-09-18T11:32:19.443 INFO:teuthology.orchestra.run.smithi119.stdout: "options": {}, 2024-09-18T11:32:19.443 INFO:teuthology.orchestra.run.smithi119.stdout: "application_metadata": {} 2024-09-18T11:32:19.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.443 INFO:teuthology.orchestra.run.smithi119.stdout: "pool": 4, 2024-09-18T11:32:19.443 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_name": "unique_pool_0", 2024-09-18T11:32:19.443 INFO:teuthology.orchestra.run.smithi119.stdout: "create_time": "2024-09-18T11:30:00.523818+0000", 2024-09-18T11:32:19.443 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 8193, 2024-09-18T11:32:19.444 INFO:teuthology.orchestra.run.smithi119.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T11:32:19.444 INFO:teuthology.orchestra.run.smithi119.stdout: "type": 1, 2024-09-18T11:32:19.444 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 2, 2024-09-18T11:32:19.444 INFO:teuthology.orchestra.run.smithi119.stdout: "min_size": 2, 2024-09-18T11:32:19.444 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_rule": 0, 2024-09-18T11:32:19.444 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_count": 0, 2024-09-18T11:32:19.444 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_target": 0, 2024-09-18T11:32:19.444 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T11:32:19.444 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T11:32:19.444 INFO:teuthology.orchestra.run.smithi119.stdout: "object_hash": 2, 2024-09-18T11:32:19.444 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_autoscale_mode": "off", 2024-09-18T11:32:19.444 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num": 16, 2024-09-18T11:32:19.445 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_placement_num": 16, 2024-09-18T11:32:19.445 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_placement_num_target": 16, 2024-09-18T11:32:19.445 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num_target": 16, 2024-09-18T11:32:19.445 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num_pending": 16, 2024-09-18T11:32:19.445 INFO:teuthology.orchestra.run.smithi119.stdout: "last_pg_merge_meta": { 2024-09-18T11:32:19.445 INFO:teuthology.orchestra.run.smithi119.stdout: "source_pgid": "0.0", 2024-09-18T11:32:19.445 INFO:teuthology.orchestra.run.smithi119.stdout: "ready_epoch": 0, 2024-09-18T11:32:19.445 INFO:teuthology.orchestra.run.smithi119.stdout: "last_epoch_started": 0, 2024-09-18T11:32:19.445 INFO:teuthology.orchestra.run.smithi119.stdout: "last_epoch_clean": 0, 2024-09-18T11:32:19.445 INFO:teuthology.orchestra.run.smithi119.stdout: "source_version": "0'0", 2024-09-18T11:32:19.445 INFO:teuthology.orchestra.run.smithi119.stdout: "target_version": "0'0" 2024-09-18T11:32:19.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.445 INFO:teuthology.orchestra.run.smithi119.stdout: "last_change": "148", 2024-09-18T11:32:19.445 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend": "0", 2024-09-18T11:32:19.446 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T11:32:19.446 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T11:32:19.446 INFO:teuthology.orchestra.run.smithi119.stdout: "auid": 0, 2024-09-18T11:32:19.446 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_mode": "selfmanaged", 2024-09-18T11:32:19.446 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_seq": 99, 2024-09-18T11:32:19.446 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_epoch": 148, 2024-09-18T11:32:19.446 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_snaps": [], 2024-09-18T11:32:19.446 INFO:teuthology.orchestra.run.smithi119.stdout: "removed_snaps": "[]", 2024-09-18T11:32:19.446 INFO:teuthology.orchestra.run.smithi119.stdout: "quota_max_bytes": 0, 2024-09-18T11:32:19.446 INFO:teuthology.orchestra.run.smithi119.stdout: "quota_max_objects": 0, 2024-09-18T11:32:19.446 INFO:teuthology.orchestra.run.smithi119.stdout: "tiers": [], 2024-09-18T11:32:19.446 INFO:teuthology.orchestra.run.smithi119.stdout: "tier_of": -1, 2024-09-18T11:32:19.447 INFO:teuthology.orchestra.run.smithi119.stdout: "read_tier": -1, 2024-09-18T11:32:19.447 INFO:teuthology.orchestra.run.smithi119.stdout: "write_tier": -1, 2024-09-18T11:32:19.447 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_mode": "none", 2024-09-18T11:32:19.447 INFO:teuthology.orchestra.run.smithi119.stdout: "target_max_bytes": 0, 2024-09-18T11:32:19.447 INFO:teuthology.orchestra.run.smithi119.stdout: "target_max_objects": 0, 2024-09-18T11:32:19.447 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T11:32:19.447 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T11:32:19.447 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T11:32:19.447 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_min_flush_age": 0, 2024-09-18T11:32:19.447 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_min_evict_age": 0, 2024-09-18T11:32:19.447 INFO:teuthology.orchestra.run.smithi119.stdout: "erasure_code_profile": "", 2024-09-18T11:32:19.447 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_params": { 2024-09-18T11:32:19.447 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "none" 2024-09-18T11:32:19.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.448 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_period": 0, 2024-09-18T11:32:19.448 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_count": 0, 2024-09-18T11:32:19.448 INFO:teuthology.orchestra.run.smithi119.stdout: "use_gmt_hitset": true, 2024-09-18T11:32:19.448 INFO:teuthology.orchestra.run.smithi119.stdout: "min_read_recency_for_promote": 0, 2024-09-18T11:32:19.448 INFO:teuthology.orchestra.run.smithi119.stdout: "min_write_recency_for_promote": 0, 2024-09-18T11:32:19.448 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T11:32:19.448 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_search_last_n": 0, 2024-09-18T11:32:19.448 INFO:teuthology.orchestra.run.smithi119.stdout: "grade_table": [], 2024-09-18T11:32:19.448 INFO:teuthology.orchestra.run.smithi119.stdout: "stripe_width": 0, 2024-09-18T11:32:19.448 INFO:teuthology.orchestra.run.smithi119.stdout: "expected_num_objects": 0, 2024-09-18T11:32:19.448 INFO:teuthology.orchestra.run.smithi119.stdout: "fast_read": false, 2024-09-18T11:32:19.448 INFO:teuthology.orchestra.run.smithi119.stdout: "options": { 2024-09-18T11:32:19.448 INFO:teuthology.orchestra.run.smithi119.stdout: "dedup_cdc_chunk_size": 131072, 2024-09-18T11:32:19.449 INFO:teuthology.orchestra.run.smithi119.stdout: "dedup_chunk_algorithm": "fastcdc", 2024-09-18T11:32:19.449 INFO:teuthology.orchestra.run.smithi119.stdout: "dedup_tier": 3, 2024-09-18T11:32:19.449 INFO:teuthology.orchestra.run.smithi119.stdout: "fingerprint_algorithm": "sha256" 2024-09-18T11:32:19.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.449 INFO:teuthology.orchestra.run.smithi119.stdout: "application_metadata": { 2024-09-18T11:32:19.449 INFO:teuthology.orchestra.run.smithi119.stdout: "rados": {} 2024-09-18T11:32:19.449 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.449 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.449 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-09-18T11:32:19.449 INFO:teuthology.orchestra.run.smithi119.stdout: "osds": [ 2024-09-18T11:32:19.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.449 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 0, 2024-09-18T11:32:19.449 INFO:teuthology.orchestra.run.smithi119.stdout: "uuid": "12faced2-7888-4265-ad12-815efe36466c", 2024-09-18T11:32:19.450 INFO:teuthology.orchestra.run.smithi119.stdout: "up": 1, 2024-09-18T11:32:19.450 INFO:teuthology.orchestra.run.smithi119.stdout: "in": 0, 2024-09-18T11:32:19.450 INFO:teuthology.orchestra.run.smithi119.stdout: "weight": 0, 2024-09-18T11:32:19.450 INFO:teuthology.orchestra.run.smithi119.stdout: "primary_affinity": 1, 2024-09-18T11:32:19.450 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_begin": 13, 2024-09-18T11:32:19.450 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_end": 65, 2024-09-18T11:32:19.450 INFO:teuthology.orchestra.run.smithi119.stdout: "up_from": 119, 2024-09-18T11:32:19.450 INFO:teuthology.orchestra.run.smithi119.stdout: "up_thru": 22, 2024-09-18T11:32:19.450 INFO:teuthology.orchestra.run.smithi119.stdout: "down_at": 66, 2024-09-18T11:32:19.450 INFO:teuthology.orchestra.run.smithi119.stdout: "lost_at": 0, 2024-09-18T11:32:19.450 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addrs": { 2024-09-18T11:32:19.450 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.451 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.451 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6800", 2024-09-18T11:32:19.451 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 56607 2024-09-18T11:32:19.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.451 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.451 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6801", 2024-09-18T11:32:19.451 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 56607 2024-09-18T11:32:19.451 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.452 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.452 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addrs": { 2024-09-18T11:32:19.452 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.452 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.452 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6802", 2024-09-18T11:32:19.452 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 56607 2024-09-18T11:32:19.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.453 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.453 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6803", 2024-09-18T11:32:19.453 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 56607 2024-09-18T11:32:19.453 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.453 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.453 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addrs": { 2024-09-18T11:32:19.453 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.453 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.453 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6806", 2024-09-18T11:32:19.453 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 56607 2024-09-18T11:32:19.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.454 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.454 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6807", 2024-09-18T11:32:19.454 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 56607 2024-09-18T11:32:19.454 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.454 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.454 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addrs": { 2024-09-18T11:32:19.454 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.454 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.454 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6804", 2024-09-18T11:32:19.454 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 56607 2024-09-18T11:32:19.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.455 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.455 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6805", 2024-09-18T11:32:19.455 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 56607 2024-09-18T11:32:19.455 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.455 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.455 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addr": "172.21.15.119:6801/56607", 2024-09-18T11:32:19.455 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addr": "172.21.15.119:6803/56607", 2024-09-18T11:32:19.455 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addr": "172.21.15.119:6807/56607", 2024-09-18T11:32:19.455 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addr": "172.21.15.119:6805/56607", 2024-09-18T11:32:19.455 INFO:teuthology.orchestra.run.smithi119.stdout: "state": [ 2024-09-18T11:32:19.455 INFO:teuthology.orchestra.run.smithi119.stdout: "exists", 2024-09-18T11:32:19.456 INFO:teuthology.orchestra.run.smithi119.stdout: "up" 2024-09-18T11:32:19.456 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.456 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 1, 2024-09-18T11:32:19.456 INFO:teuthology.orchestra.run.smithi119.stdout: "uuid": "20fee29d-b047-46ae-aa60-88e897fbb0fb", 2024-09-18T11:32:19.456 INFO:teuthology.orchestra.run.smithi119.stdout: "up": 1, 2024-09-18T11:32:19.456 INFO:teuthology.orchestra.run.smithi119.stdout: "in": 1, 2024-09-18T11:32:19.456 INFO:teuthology.orchestra.run.smithi119.stdout: "weight": 1, 2024-09-18T11:32:19.456 INFO:teuthology.orchestra.run.smithi119.stdout: "primary_affinity": 1, 2024-09-18T11:32:19.456 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_begin": 0, 2024-09-18T11:32:19.456 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_end": 0, 2024-09-18T11:32:19.456 INFO:teuthology.orchestra.run.smithi119.stdout: "up_from": 13, 2024-09-18T11:32:19.456 INFO:teuthology.orchestra.run.smithi119.stdout: "up_thru": 143, 2024-09-18T11:32:19.457 INFO:teuthology.orchestra.run.smithi119.stdout: "down_at": 0, 2024-09-18T11:32:19.457 INFO:teuthology.orchestra.run.smithi119.stdout: "lost_at": 0, 2024-09-18T11:32:19.457 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addrs": { 2024-09-18T11:32:19.457 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.457 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.457 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6824", 2024-09-18T11:32:19.457 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33350 2024-09-18T11:32:19.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.457 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.457 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6825", 2024-09-18T11:32:19.457 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33350 2024-09-18T11:32:19.458 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.458 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.458 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addrs": { 2024-09-18T11:32:19.458 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.458 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.458 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6826", 2024-09-18T11:32:19.458 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33350 2024-09-18T11:32:19.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.458 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.458 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6827", 2024-09-18T11:32:19.459 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33350 2024-09-18T11:32:19.459 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.459 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.459 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addrs": { 2024-09-18T11:32:19.459 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.459 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.459 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6830", 2024-09-18T11:32:19.459 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33350 2024-09-18T11:32:19.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.459 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.459 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6831", 2024-09-18T11:32:19.460 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33350 2024-09-18T11:32:19.460 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.460 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.460 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addrs": { 2024-09-18T11:32:19.460 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.460 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.460 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6828", 2024-09-18T11:32:19.460 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33350 2024-09-18T11:32:19.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.460 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.460 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6829", 2024-09-18T11:32:19.461 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33350 2024-09-18T11:32:19.461 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.461 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.461 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addr": "172.21.15.119:6825/33350", 2024-09-18T11:32:19.461 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addr": "172.21.15.119:6827/33350", 2024-09-18T11:32:19.461 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addr": "172.21.15.119:6831/33350", 2024-09-18T11:32:19.461 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addr": "172.21.15.119:6829/33350", 2024-09-18T11:32:19.461 INFO:teuthology.orchestra.run.smithi119.stdout: "state": [ 2024-09-18T11:32:19.461 INFO:teuthology.orchestra.run.smithi119.stdout: "exists", 2024-09-18T11:32:19.461 INFO:teuthology.orchestra.run.smithi119.stdout: "up" 2024-09-18T11:32:19.461 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.462 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 2, 2024-09-18T11:32:19.462 INFO:teuthology.orchestra.run.smithi119.stdout: "uuid": "21a296f3-b259-4841-9a17-4cd21b302b36", 2024-09-18T11:32:19.462 INFO:teuthology.orchestra.run.smithi119.stdout: "up": 1, 2024-09-18T11:32:19.462 INFO:teuthology.orchestra.run.smithi119.stdout: "in": 1, 2024-09-18T11:32:19.462 INFO:teuthology.orchestra.run.smithi119.stdout: "weight": 0.280487060546875, 2024-09-18T11:32:19.462 INFO:teuthology.orchestra.run.smithi119.stdout: "primary_affinity": 0, 2024-09-18T11:32:19.462 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_begin": 0, 2024-09-18T11:32:19.462 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_end": 0, 2024-09-18T11:32:19.462 INFO:teuthology.orchestra.run.smithi119.stdout: "up_from": 13, 2024-09-18T11:32:19.462 INFO:teuthology.orchestra.run.smithi119.stdout: "up_thru": 15, 2024-09-18T11:32:19.462 INFO:teuthology.orchestra.run.smithi119.stdout: "down_at": 0, 2024-09-18T11:32:19.462 INFO:teuthology.orchestra.run.smithi119.stdout: "lost_at": 0, 2024-09-18T11:32:19.462 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addrs": { 2024-09-18T11:32:19.463 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.463 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.463 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6808", 2024-09-18T11:32:19.463 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33351 2024-09-18T11:32:19.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.463 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.463 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6809", 2024-09-18T11:32:19.463 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33351 2024-09-18T11:32:19.463 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.463 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.463 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addrs": { 2024-09-18T11:32:19.464 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.464 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.464 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6811", 2024-09-18T11:32:19.464 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33351 2024-09-18T11:32:19.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.464 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.464 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6813", 2024-09-18T11:32:19.464 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33351 2024-09-18T11:32:19.464 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.464 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addrs": { 2024-09-18T11:32:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6819", 2024-09-18T11:32:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33351 2024-09-18T11:32:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6821", 2024-09-18T11:32:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33351 2024-09-18T11:32:19.465 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addrs": { 2024-09-18T11:32:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6815", 2024-09-18T11:32:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33351 2024-09-18T11:32:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6817", 2024-09-18T11:32:19.466 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33351 2024-09-18T11:32:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addr": "172.21.15.119:6809/33351", 2024-09-18T11:32:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addr": "172.21.15.119:6813/33351", 2024-09-18T11:32:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addr": "172.21.15.119:6821/33351", 2024-09-18T11:32:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addr": "172.21.15.119:6817/33351", 2024-09-18T11:32:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: "state": [ 2024-09-18T11:32:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: "exists", 2024-09-18T11:32:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: "up" 2024-09-18T11:32:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 3, 2024-09-18T11:32:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: "uuid": "59044f30-32e1-4942-9a8e-202c1def221a", 2024-09-18T11:32:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: "up": 1, 2024-09-18T11:32:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: "in": 1, 2024-09-18T11:32:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: "weight": 1, 2024-09-18T11:32:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: "primary_affinity": 1, 2024-09-18T11:32:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_begin": 0, 2024-09-18T11:32:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_end": 0, 2024-09-18T11:32:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: "up_from": 13, 2024-09-18T11:32:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: "up_thru": 143, 2024-09-18T11:32:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: "down_at": 0, 2024-09-18T11:32:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: "lost_at": 0, 2024-09-18T11:32:19.468 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addrs": { 2024-09-18T11:32:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6810", 2024-09-18T11:32:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33349 2024-09-18T11:32:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6812", 2024-09-18T11:32:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33349 2024-09-18T11:32:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addrs": { 2024-09-18T11:32:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6814", 2024-09-18T11:32:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33349 2024-09-18T11:32:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6816", 2024-09-18T11:32:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33349 2024-09-18T11:32:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addrs": { 2024-09-18T11:32:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6822", 2024-09-18T11:32:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33349 2024-09-18T11:32:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6823", 2024-09-18T11:32:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33349 2024-09-18T11:32:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addrs": { 2024-09-18T11:32:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6818", 2024-09-18T11:32:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33349 2024-09-18T11:32:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6820", 2024-09-18T11:32:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33349 2024-09-18T11:32:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.472 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addr": "172.21.15.119:6812/33349", 2024-09-18T11:32:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addr": "172.21.15.119:6816/33349", 2024-09-18T11:32:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addr": "172.21.15.119:6823/33349", 2024-09-18T11:32:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addr": "172.21.15.119:6820/33349", 2024-09-18T11:32:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: "state": [ 2024-09-18T11:32:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: "exists", 2024-09-18T11:32:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: "up" 2024-09-18T11:32:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 4, 2024-09-18T11:32:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: "uuid": "7d1ae3d0-8d69-4d3c-9e51-24fdd77a22a9", 2024-09-18T11:32:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: "up": 1, 2024-09-18T11:32:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: "in": 1, 2024-09-18T11:32:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: "weight": 1, 2024-09-18T11:32:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: "primary_affinity": 1, 2024-09-18T11:32:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_begin": 0, 2024-09-18T11:32:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_end": 0, 2024-09-18T11:32:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: "up_from": 13, 2024-09-18T11:32:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: "up_thru": 143, 2024-09-18T11:32:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: "down_at": 0, 2024-09-18T11:32:19.474 INFO:teuthology.orchestra.run.smithi119.stdout: "lost_at": 0, 2024-09-18T11:32:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addrs": { 2024-09-18T11:32:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6808", 2024-09-18T11:32:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33088 2024-09-18T11:32:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6809", 2024-09-18T11:32:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33088 2024-09-18T11:32:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addrs": { 2024-09-18T11:32:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6810", 2024-09-18T11:32:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33088 2024-09-18T11:32:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6811", 2024-09-18T11:32:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33088 2024-09-18T11:32:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addrs": { 2024-09-18T11:32:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6814", 2024-09-18T11:32:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33088 2024-09-18T11:32:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6815", 2024-09-18T11:32:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33088 2024-09-18T11:32:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addrs": { 2024-09-18T11:32:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6812", 2024-09-18T11:32:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33088 2024-09-18T11:32:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6813", 2024-09-18T11:32:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33088 2024-09-18T11:32:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.478 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addr": "172.21.15.148:6809/33088", 2024-09-18T11:32:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addr": "172.21.15.148:6811/33088", 2024-09-18T11:32:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addr": "172.21.15.148:6815/33088", 2024-09-18T11:32:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addr": "172.21.15.148:6813/33088", 2024-09-18T11:32:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: "state": [ 2024-09-18T11:32:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: "exists", 2024-09-18T11:32:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: "up" 2024-09-18T11:32:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 5, 2024-09-18T11:32:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: "uuid": "b0dd3d30-b273-428c-a4a9-4c238abf45fc", 2024-09-18T11:32:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: "up": 1, 2024-09-18T11:32:19.479 INFO:teuthology.orchestra.run.smithi119.stdout: "in": 1, 2024-09-18T11:32:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: "weight": 1, 2024-09-18T11:32:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: "primary_affinity": 1, 2024-09-18T11:32:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_begin": 0, 2024-09-18T11:32:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_end": 0, 2024-09-18T11:32:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: "up_from": 13, 2024-09-18T11:32:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: "up_thru": 143, 2024-09-18T11:32:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: "down_at": 0, 2024-09-18T11:32:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: "lost_at": 0, 2024-09-18T11:32:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addrs": { 2024-09-18T11:32:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6800", 2024-09-18T11:32:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33089 2024-09-18T11:32:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6801", 2024-09-18T11:32:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33089 2024-09-18T11:32:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addrs": { 2024-09-18T11:32:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.481 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6802", 2024-09-18T11:32:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33089 2024-09-18T11:32:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6803", 2024-09-18T11:32:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33089 2024-09-18T11:32:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addrs": { 2024-09-18T11:32:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.482 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6806", 2024-09-18T11:32:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33089 2024-09-18T11:32:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6807", 2024-09-18T11:32:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33089 2024-09-18T11:32:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addrs": { 2024-09-18T11:32:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.483 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6804", 2024-09-18T11:32:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33089 2024-09-18T11:32:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6805", 2024-09-18T11:32:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33089 2024-09-18T11:32:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addr": "172.21.15.148:6801/33089", 2024-09-18T11:32:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addr": "172.21.15.148:6803/33089", 2024-09-18T11:32:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addr": "172.21.15.148:6807/33089", 2024-09-18T11:32:19.484 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addr": "172.21.15.148:6805/33089", 2024-09-18T11:32:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: "state": [ 2024-09-18T11:32:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: "exists", 2024-09-18T11:32:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: "up" 2024-09-18T11:32:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 6, 2024-09-18T11:32:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: "uuid": "40290cb3-2334-4c68-8e6c-59d0d5455171", 2024-09-18T11:32:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: "up": 1, 2024-09-18T11:32:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: "in": 1, 2024-09-18T11:32:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: "weight": 1, 2024-09-18T11:32:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: "primary_affinity": 1, 2024-09-18T11:32:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_begin": 0, 2024-09-18T11:32:19.485 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_end": 0, 2024-09-18T11:32:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: "up_from": 13, 2024-09-18T11:32:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: "up_thru": 22, 2024-09-18T11:32:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: "down_at": 0, 2024-09-18T11:32:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: "lost_at": 0, 2024-09-18T11:32:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addrs": { 2024-09-18T11:32:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6816", 2024-09-18T11:32:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33107 2024-09-18T11:32:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.486 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6817", 2024-09-18T11:32:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33107 2024-09-18T11:32:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addrs": { 2024-09-18T11:32:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6818", 2024-09-18T11:32:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33107 2024-09-18T11:32:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.487 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6819", 2024-09-18T11:32:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33107 2024-09-18T11:32:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addrs": { 2024-09-18T11:32:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6822", 2024-09-18T11:32:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33107 2024-09-18T11:32:19.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6823", 2024-09-18T11:32:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33107 2024-09-18T11:32:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addrs": { 2024-09-18T11:32:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6820", 2024-09-18T11:32:19.489 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33107 2024-09-18T11:32:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6821", 2024-09-18T11:32:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33107 2024-09-18T11:32:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addr": "172.21.15.148:6817/33107", 2024-09-18T11:32:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addr": "172.21.15.148:6819/33107", 2024-09-18T11:32:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addr": "172.21.15.148:6823/33107", 2024-09-18T11:32:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addr": "172.21.15.148:6821/33107", 2024-09-18T11:32:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: "state": [ 2024-09-18T11:32:19.490 INFO:teuthology.orchestra.run.smithi119.stdout: "exists", 2024-09-18T11:32:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: "up" 2024-09-18T11:32:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 7, 2024-09-18T11:32:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: "uuid": "54802abe-ea1a-448e-85af-d1f14a45be27", 2024-09-18T11:32:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: "up": 1, 2024-09-18T11:32:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: "in": 1, 2024-09-18T11:32:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: "weight": 1, 2024-09-18T11:32:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: "primary_affinity": 1, 2024-09-18T11:32:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_begin": 0, 2024-09-18T11:32:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_end": 0, 2024-09-18T11:32:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: "up_from": 13, 2024-09-18T11:32:19.491 INFO:teuthology.orchestra.run.smithi119.stdout: "up_thru": 67, 2024-09-18T11:32:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: "down_at": 0, 2024-09-18T11:32:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: "lost_at": 0, 2024-09-18T11:32:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addrs": { 2024-09-18T11:32:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6824", 2024-09-18T11:32:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33119 2024-09-18T11:32:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6825", 2024-09-18T11:32:19.492 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33119 2024-09-18T11:32:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addrs": { 2024-09-18T11:32:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6826", 2024-09-18T11:32:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33119 2024-09-18T11:32:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.493 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.494 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6827", 2024-09-18T11:32:19.494 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33119 2024-09-18T11:32:19.494 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.494 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.494 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addrs": { 2024-09-18T11:32:19.494 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6830", 2024-09-18T11:32:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33119 2024-09-18T11:32:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6831", 2024-09-18T11:32:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33119 2024-09-18T11:32:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addrs": { 2024-09-18T11:32:19.495 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6828", 2024-09-18T11:32:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33119 2024-09-18T11:32:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6829", 2024-09-18T11:32:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33119 2024-09-18T11:32:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addr": "172.21.15.148:6825/33119", 2024-09-18T11:32:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addr": "172.21.15.148:6827/33119", 2024-09-18T11:32:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addr": "172.21.15.148:6831/33119", 2024-09-18T11:32:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addr": "172.21.15.148:6829/33119", 2024-09-18T11:32:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: "state": [ 2024-09-18T11:32:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: "exists", 2024-09-18T11:32:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: "up" 2024-09-18T11:32:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-09-18T11:32:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_xinfo": [ 2024-09-18T11:32:19.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 0, 2024-09-18T11:32:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: "down_stamp": "0.000000", 2024-09-18T11:32:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_probability": 0, 2024-09-18T11:32:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_interval": 0, 2024-09-18T11:32:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: "features": 4540138320759226367, 2024-09-18T11:32:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: "old_weight": 65536, 2024-09-18T11:32:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: "last_purged_snaps_scrub": "2024-09-18T11:31:49.015287+0000", 2024-09-18T11:32:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: "dead_epoch": 66 2024-09-18T11:32:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.498 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 1, 2024-09-18T11:32:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: "down_stamp": "0.000000", 2024-09-18T11:32:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_probability": 0, 2024-09-18T11:32:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_interval": 0, 2024-09-18T11:32:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: "features": 4540138320759226367, 2024-09-18T11:32:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: "old_weight": 0, 2024-09-18T11:32:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: "last_purged_snaps_scrub": "2024-09-18T11:32:14.430356+0000", 2024-09-18T11:32:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: "dead_epoch": 0 2024-09-18T11:32:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 2, 2024-09-18T11:32:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: "down_stamp": "0.000000", 2024-09-18T11:32:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_probability": 0, 2024-09-18T11:32:19.499 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_interval": 0, 2024-09-18T11:32:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: "features": 4540138320759226367, 2024-09-18T11:32:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: "old_weight": 0, 2024-09-18T11:32:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: "last_purged_snaps_scrub": "2024-09-18T11:31:09.297193+0000", 2024-09-18T11:32:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: "dead_epoch": 0 2024-09-18T11:32:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 3, 2024-09-18T11:32:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: "down_stamp": "0.000000", 2024-09-18T11:32:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_probability": 0, 2024-09-18T11:32:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_interval": 0, 2024-09-18T11:32:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: "features": 4540138320759226367, 2024-09-18T11:32:19.500 INFO:teuthology.orchestra.run.smithi119.stdout: "old_weight": 0, 2024-09-18T11:32:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: "last_purged_snaps_scrub": "2024-09-18T11:32:12.546041+0000", 2024-09-18T11:32:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: "dead_epoch": 0 2024-09-18T11:32:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 4, 2024-09-18T11:32:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: "down_stamp": "0.000000", 2024-09-18T11:32:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_probability": 0, 2024-09-18T11:32:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_interval": 0, 2024-09-18T11:32:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: "features": 4540138320759226367, 2024-09-18T11:32:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: "old_weight": 0, 2024-09-18T11:32:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: "last_purged_snaps_scrub": "2024-09-18T11:32:13.027907+0000", 2024-09-18T11:32:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: "dead_epoch": 0 2024-09-18T11:32:19.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.504 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 5, 2024-09-18T11:32:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: "down_stamp": "0.000000", 2024-09-18T11:32:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_probability": 0, 2024-09-18T11:32:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_interval": 0, 2024-09-18T11:32:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: "features": 4540138320759226367, 2024-09-18T11:32:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: "old_weight": 0, 2024-09-18T11:32:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: "last_purged_snaps_scrub": "2024-09-18T11:31:09.462670+0000", 2024-09-18T11:32:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: "dead_epoch": 0 2024-09-18T11:32:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 6, 2024-09-18T11:32:19.505 INFO:teuthology.orchestra.run.smithi119.stdout: "down_stamp": "0.000000", 2024-09-18T11:32:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_probability": 0, 2024-09-18T11:32:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_interval": 0, 2024-09-18T11:32:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: "features": 4540138320759226367, 2024-09-18T11:32:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: "old_weight": 0, 2024-09-18T11:32:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: "last_purged_snaps_scrub": "2024-09-18T11:32:14.951013+0000", 2024-09-18T11:32:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: "dead_epoch": 0 2024-09-18T11:32:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 7, 2024-09-18T11:32:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: "down_stamp": "0.000000", 2024-09-18T11:32:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_probability": 0, 2024-09-18T11:32:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_interval": 0, 2024-09-18T11:32:19.506 INFO:teuthology.orchestra.run.smithi119.stdout: "features": 4540138320759226367, 2024-09-18T11:32:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "old_weight": 0, 2024-09-18T11:32:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "last_purged_snaps_scrub": "2024-09-18T11:31:07.463783+0000", 2024-09-18T11:32:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "dead_epoch": 0 2024-09-18T11:32:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-09-18T11:32:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_upmap": [], 2024-09-18T11:32:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_upmap_items": [], 2024-09-18T11:32:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_temp": [], 2024-09-18T11:32:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "primary_temp": [], 2024-09-18T11:32:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "blocklist": {}, 2024-09-18T11:32:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "range_blocklist": {}, 2024-09-18T11:32:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "erasure_code_profiles": { 2024-09-18T11:32:19.507 INFO:teuthology.orchestra.run.smithi119.stdout: "default": { 2024-09-18T11:32:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: "crush-failure-domain": "osd", 2024-09-18T11:32:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: "k": "2", 2024-09-18T11:32:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: "m": "1", 2024-09-18T11:32:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: "plugin": "jerasure", 2024-09-18T11:32:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: "technique": "reed_sol_van" 2024-09-18T11:32:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: "removed_snaps_queue": [ 2024-09-18T11:32:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: "pool": 4, 2024-09-18T11:32:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: "snaps": [ 2024-09-18T11:32:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.508 INFO:teuthology.orchestra.run.smithi119.stdout: "begin": 59, 2024-09-18T11:32:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: "length": 1 2024-09-18T11:32:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: "begin": 71, 2024-09-18T11:32:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: "length": 3 2024-09-18T11:32:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: "begin": 84, 2024-09-18T11:32:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: "length": 1 2024-09-18T11:32:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: "begin": 89, 2024-09-18T11:32:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: "length": 2 2024-09-18T11:32:19.509 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-09-18T11:32:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: "new_removed_snaps": [ 2024-09-18T11:32:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: "pool": 4, 2024-09-18T11:32:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: "snaps": [ 2024-09-18T11:32:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: "begin": 73, 2024-09-18T11:32:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: "length": 1 2024-09-18T11:32:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.510 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-09-18T11:32:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: "new_purged_snaps": [], 2024-09-18T11:32:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_node_flags": {}, 2024-09-18T11:32:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: "device_class_flags": {}, 2024-09-18T11:32:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: "stretch_mode": { 2024-09-18T11:32:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: "stretch_mode_enabled": false, 2024-09-18T11:32:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: "stretch_bucket_count": 0, 2024-09-18T11:32:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: "degraded_stretch_mode": 0, 2024-09-18T11:32:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: "recovering_stretch_mode": 0, 2024-09-18T11:32:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: "stretch_mode_bucket": 0 2024-09-18T11:32:19.511 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:19.511 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:19.514 INFO:tasks.thrashosds.thrasher:j is {'epoch': 148, 'fsid': 'f4cca259-b87a-45cc-8995-7e9fca891a9f', 'created': '2024-09-18T11:29:38.205487+0000', 'modified': '2024-09-18T11:32:19.317671+0000', 'last_up_change': '2024-09-18T11:31:49.142260+0000', 'last_in_change': '2024-09-18T11:30:49.312763+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T11:29:47.951262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T11:29:48.866238+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'low_tier', 'create_time': '2024-09-18T11:29:59.343964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}}, {'pool': 4, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-18T11:30:00.523818+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 99, 'snap_epoch': 148, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'dedup_cdc_chunk_size': 131072, 'dedup_chunk_algorithm': 'fastcdc', 'dedup_tier': 3, 'fingerprint_algorithm': 'sha256'}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '12faced2-7888-4265-ad12-815efe36466c', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 13, 'last_clean_end': 65, 'up_from': 119, 'up_thru': 22, 'down_at': 66, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6800', 'nonce': 56607}, {'type': 'v1', 'addr': '172.21.15.119:6801', 'nonce': 56607}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6802', 'nonce': 56607}, {'type': 'v1', 'addr': '172.21.15.119:6803', 'nonce': 56607}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6806', 'nonce': 56607}, {'type': 'v1', 'addr': '172.21.15.119:6807', 'nonce': 56607}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6804', 'nonce': 56607}, {'type': 'v1', 'addr': '172.21.15.119:6805', 'nonce': 56607}]}, 'public_addr': '172.21.15.119:6801/56607', 'cluster_addr': '172.21.15.119:6803/56607', 'heartbeat_back_addr': '172.21.15.119:6807/56607', 'heartbeat_front_addr': '172.21.15.119:6805/56607', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '20fee29d-b047-46ae-aa60-88e897fbb0fb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 143, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6824', 'nonce': 33350}, {'type': 'v1', 'addr': '172.21.15.119:6825', 'nonce': 33350}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6826', 'nonce': 33350}, {'type': 'v1', 'addr': '172.21.15.119:6827', 'nonce': 33350}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6830', 'nonce': 33350}, {'type': 'v1', 'addr': '172.21.15.119:6831', 'nonce': 33350}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6828', 'nonce': 33350}, {'type': 'v1', 'addr': '172.21.15.119:6829', 'nonce': 33350}]}, 'public_addr': '172.21.15.119:6825/33350', 'cluster_addr': '172.21.15.119:6827/33350', 'heartbeat_back_addr': '172.21.15.119:6831/33350', 'heartbeat_front_addr': '172.21.15.119:6829/33350', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '21a296f3-b259-4841-9a17-4cd21b302b36', 'up': 1, 'in': 1, 'weight': 0.280487060546875, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 15, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6808', 'nonce': 33351}, {'type': 'v1', 'addr': '172.21.15.119:6809', 'nonce': 33351}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6811', 'nonce': 33351}, {'type': 'v1', 'addr': '172.21.15.119:6813', 'nonce': 33351}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6819', 'nonce': 33351}, {'type': 'v1', 'addr': '172.21.15.119:6821', 'nonce': 33351}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6815', 'nonce': 33351}, {'type': 'v1', 'addr': '172.21.15.119:6817', 'nonce': 33351}]}, 'public_addr': '172.21.15.119:6809/33351', 'cluster_addr': '172.21.15.119:6813/33351', 'heartbeat_back_addr': '172.21.15.119:6821/33351', 'heartbeat_front_addr': '172.21.15.119:6817/33351', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '59044f30-32e1-4942-9a8e-202c1def221a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 143, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6810', 'nonce': 33349}, {'type': 'v1', 'addr': '172.21.15.119:6812', 'nonce': 33349}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6814', 'nonce': 33349}, {'type': 'v1', 'addr': '172.21.15.119:6816', 'nonce': 33349}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6822', 'nonce': 33349}, {'type': 'v1', 'addr': '172.21.15.119:6823', 'nonce': 33349}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6818', 'nonce': 33349}, {'type': 'v1', 'addr': '172.21.15.119:6820', 'nonce': 33349}]}, 'public_addr': '172.21.15.119:6812/33349', 'cluster_addr': '172.21.15.119:6816/33349', 'heartbeat_back_addr': '172.21.15.119:6823/33349', 'heartbeat_front_addr': '172.21.15.119:6820/33349', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '7d1ae3d0-8d69-4d3c-9e51-24fdd77a22a9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 143, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6808', 'nonce': 33088}, {'type': 'v1', 'addr': '172.21.15.148:6809', 'nonce': 33088}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6810', 'nonce': 33088}, {'type': 'v1', 'addr': '172.21.15.148:6811', 'nonce': 33088}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6814', 'nonce': 33088}, {'type': 'v1', 'addr': '172.21.15.148:6815', 'nonce': 33088}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6812', 'nonce': 33088}, {'type': 'v1', 'addr': '172.21.15.148:6813', 'nonce': 33088}]}, 'public_addr': '172.21.15.148:6809/33088', 'cluster_addr': '172.21.15.148:6811/33088', 'heartbeat_back_addr': '172.21.15.148:6815/33088', 'heartbeat_front_addr': '172.21.15.148:6813/33088', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b0dd3d30-b273-428c-a4a9-4c238abf45fc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 143, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6800', 'nonce': 33089}, {'type': 'v1', 'addr': '172.21.15.148:6801', 'nonce': 33089}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6802', 'nonce': 33089}, {'type': 'v1', 'addr': '172.21.15.148:6803', 'nonce': 33089}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6806', 'nonce': 33089}, {'type': 'v1', 'addr': '172.21.15.148:6807', 'nonce': 33089}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6804', 'nonce': 33089}, {'type': 'v1', 'addr': '172.21.15.148:6805', 'nonce': 33089}]}, 'public_addr': '172.21.15.148:6801/33089', 'cluster_addr': '172.21.15.148:6803/33089', 'heartbeat_back_addr': '172.21.15.148:6807/33089', 'heartbeat_front_addr': '172.21.15.148:6805/33089', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '40290cb3-2334-4c68-8e6c-59d0d5455171', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6816', 'nonce': 33107}, {'type': 'v1', 'addr': '172.21.15.148:6817', 'nonce': 33107}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6818', 'nonce': 33107}, {'type': 'v1', 'addr': '172.21.15.148:6819', 'nonce': 33107}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6822', 'nonce': 33107}, {'type': 'v1', 'addr': '172.21.15.148:6823', 'nonce': 33107}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6820', 'nonce': 33107}, {'type': 'v1', 'addr': '172.21.15.148:6821', 'nonce': 33107}]}, 'public_addr': '172.21.15.148:6817/33107', 'cluster_addr': '172.21.15.148:6819/33107', 'heartbeat_back_addr': '172.21.15.148:6823/33107', 'heartbeat_front_addr': '172.21.15.148:6821/33107', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '54802abe-ea1a-448e-85af-d1f14a45be27', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 67, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6824', 'nonce': 33119}, {'type': 'v1', 'addr': '172.21.15.148:6825', 'nonce': 33119}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6826', 'nonce': 33119}, {'type': 'v1', 'addr': '172.21.15.148:6827', 'nonce': 33119}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6830', 'nonce': 33119}, {'type': 'v1', 'addr': '172.21.15.148:6831', 'nonce': 33119}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6828', 'nonce': 33119}, {'type': 'v1', 'addr': '172.21.15.148:6829', 'nonce': 33119}]}, 'public_addr': '172.21.15.148:6825/33119', 'cluster_addr': '172.21.15.148:6827/33119', 'heartbeat_back_addr': '172.21.15.148:6831/33119', 'heartbeat_front_addr': '172.21.15.148:6829/33119', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T11:31:49.015287+0000', 'dead_epoch': 66}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T11:32:14.430356+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T11:31:09.297193+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T11:32:12.546041+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T11:32:13.027907+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T11:31:09.462670+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T11:32:14.951013+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T11:31:07.463783+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 4, 'snaps': [{'begin': 59, 'length': 1}, {'begin': 71, 'length': 3}, {'begin': 84, 'length': 1}, {'begin': 89, 'length': 2}]}], 'new_removed_snaps': [{'pool': 4, 'snaps': [{'begin': 73, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T11:32:19.514 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:32:19.514 INFO:tasks.rados.rados.0.smithi119.stdout:1798: finishing write tid 2 to smithi11939506-38 2024-09-18T11:32:19.515 INFO:tasks.rados.rados.0.smithi119.stdout:1798: finishing write tid 3 to smithi11939506-38 2024-09-18T11:32:19.515 INFO:tasks.rados.rados.0.smithi119.stdout:1798: finishing write tid 4 to smithi11939506-38 2024-09-18T11:32:19.515 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 252 (ObjNum 365 snap 56 seq_num 365) dirty exists 2024-09-18T11:32:19.515 INFO:tasks.rados.rados.0.smithi119.stdout:1798: left oid 38 (ObjNum 365 snap 56 seq_num 365) 2024-09-18T11:32:19.515 INFO:tasks.rados.rados.0.smithi119.stdout:1798: done (2 left) 2024-09-18T11:32:19.515 INFO:tasks.rados.rados.0.smithi119.stdout:1799: done (1 left) 2024-09-18T11:32:19.515 INFO:tasks.rados.rados.0.smithi119.stdout:1800: done (0 left) 2024-09-18T11:32:19.515 INFO:tasks.rados.rados.0.smithi119.stdout:1801: snap_remove snap 54 2024-09-18T11:32:19.516 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:19.511+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:19.610 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:19.610 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:19.614 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:19.612+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:19.619 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:32:19.715 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:19.714+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:19.758 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:19.758 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:19.768 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:32:19.816 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:19.815+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:19.822 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:32:19.823 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:32:19.834 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-09-18T11:32:19.323310+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":391704240,"num_objects":280,"num_object_clones":81,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":278,"num_whiteouts":5,"num_read":4778,"num_read_kb":4846979,"num_write":2417,"num_write_kb":675834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":108843503,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":66,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4362,"ondisk_log_size":4362,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":6,"kb":660602880,"kb_used":2624568,"kb_used_data":518092,"kb_used_omap":11,"kb_used_meta":2106356,"kb_avail":657978312,"statfs":{"total":676457349120,"available":673769791488,"internally_reserved":0,"allocated":530526208,"data_stored":642650294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2156908904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4721008,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":-1,"num_read":81,"num_read_kb":106862,"num_write":25,"num_write_kb":6153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":45155730,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.302626"},"pg_stats":[{"pgid":"4.f","version":"145'340","reported_seq":722,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.336748+0000","last_change":"2024-09-18T11:32:16.336748+0000","last_active":"2024-09-18T11:32:16.336748+0000","last_peered":"2024-09-18T11:32:16.336748+0000","last_clean":"2024-09-18T11:32:16.336748+0000","last_became_active":"2024-09-18T11:30:02.074391+0000","last_became_peered":"2024-09-18T11:30:02.074391+0000","last_unstale":"2024-09-18T11:32:16.336748+0000","last_undegraded":"2024-09-18T11:32:16.336748+0000","last_fullsized":"2024-09-18T11:32:16.336748+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"92'190","last_scrub_stamp":"2024-09-18T11:31:17.744924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:17.744924+0000","objects_scrubbed":10,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:40.468788+0000","scrub_duration":0.002417757,"objects_trimmed":0,"snaptrim_duration":0.025749396000000001,"stat_sum":{"num_bytes":19954209,"num_objects":11,"num_object_clones":7,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":288,"num_read_kb":296941,"num_write":157,"num_write_kb":42617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":5,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3b","length":"1"},{"start":"47","length":"2"},{"start":"54","length":"1"},{"start":"5a","length":"1"}]},{"pgid":"4.e","version":"145'163","reported_seq":576,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.332687+0000","last_change":"2024-09-18T11:32:16.332687+0000","last_active":"2024-09-18T11:32:16.332687+0000","last_peered":"2024-09-18T11:32:16.332687+0000","last_clean":"2024-09-18T11:32:16.332687+0000","last_became_active":"2024-09-18T11:30:02.073185+0000","last_became_peered":"2024-09-18T11:30:02.073185+0000","last_unstale":"2024-09-18T11:32:16.332687+0000","last_undegraded":"2024-09-18T11:32:16.332687+0000","last_fullsized":"2024-09-18T11:32:16.332687+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"92'121","last_scrub_stamp":"2024-09-18T11:31:20.234551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:20.234551+0000","objects_scrubbed":5,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:32.824082+0000","scrub_duration":0.0040340640000000004,"objects_trimmed":0,"snaptrim_duration":0.017736694000000001,"stat_sum":{"num_bytes":10921826,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":309,"num_read_kb":354836,"num_write":58,"num_write_kb":15432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3b","length":"1"},{"start":"47","length":"2"},{"start":"54","length":"1"},{"start":"5a","length":"1"}]},{"pgid":"4.d","version":"145'293","reported_seq":827,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.348982+0000","last_change":"2024-09-18T11:32:16.348982+0000","last_active":"2024-09-18T11:32:16.348982+0000","last_peered":"2024-09-18T11:32:16.348982+0000","last_clean":"2024-09-18T11:32:16.348982+0000","last_became_active":"2024-09-18T11:32:15.034612+0000","last_became_peered":"2024-09-18T11:32:15.034612+0000","last_unstale":"2024-09-18T11:32:16.348982+0000","last_undegraded":"2024-09-18T11:32:16.348982+0000","last_fullsized":"2024-09-18T11:32:16.348982+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'144","last_scrub_stamp":"2024-09-18T11:31:11.112252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:11.112252+0000","objects_scrubbed":10,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:28.546599+0000","scrub_duration":0.0033897469999999998,"objects_trimmed":0,"snaptrim_duration":0.033964965,"stat_sum":{"num_bytes":26438283,"num_objects":10,"num_object_clones":5,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":538,"num_read_kb":685981,"num_write":129,"num_write_kb":36606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":16214313,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3b","length":"1"},{"start":"47","length":"2"},{"start":"54","length":"1"},{"start":"5a","length":"1"}]},{"pgid":"4.c","version":"145'118","reported_seq":471,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.322412+0000","last_change":"2024-09-18T11:32:16.322412+0000","last_active":"2024-09-18T11:32:16.322412+0000","last_peered":"2024-09-18T11:32:16.322412+0000","last_clean":"2024-09-18T11:32:16.322412+0000","last_became_active":"2024-09-18T11:30:02.073883+0000","last_became_peered":"2024-09-18T11:30:02.073883+0000","last_unstale":"2024-09-18T11:32:16.322412+0000","last_undegraded":"2024-09-18T11:32:16.322412+0000","last_fullsized":"2024-09-18T11:32:16.322412+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"101'76","last_scrub_stamp":"2024-09-18T11:31:29.293475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:29.293475+0000","objects_scrubbed":4,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:53.466013+0000","scrub_duration":0.002355139,"objects_trimmed":0,"snaptrim_duration":0.0075389109999999997,"stat_sum":{"num_bytes":10590877,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":134,"num_read_kb":166296,"num_write":39,"num_write_kb":9469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3b","length":"1"},{"start":"47","length":"2"},{"start":"54","length":"1"},{"start":"5a","length":"1"}]},{"pgid":"4.b","version":"145'481","reported_seq":955,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.343903+0000","last_change":"2024-09-18T11:32:16.343903+0000","last_active":"2024-09-18T11:32:16.343903+0000","last_peered":"2024-09-18T11:32:16.343903+0000","last_clean":"2024-09-18T11:32:16.343903+0000","last_became_active":"2024-09-18T11:30:50.333230+0000","last_became_peered":"2024-09-18T11:30:50.333230+0000","last_unstale":"2024-09-18T11:32:16.343903+0000","last_undegraded":"2024-09-18T11:32:16.343903+0000","last_fullsized":"2024-09-18T11:32:16.343903+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'286","last_scrub_stamp":"2024-09-18T11:31:21.818375+0000","last_deep_scrub":"95'286","last_deep_scrub_stamp":"2024-09-18T11:31:21.818375+0000","last_clean_scrub_stamp":"2024-09-18T11:31:21.818375+0000","objects_scrubbed":11,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:51.543117+0000","scrub_duration":0.044204375999999997,"objects_trimmed":0,"snaptrim_duration":0.033118914999999999,"stat_sum":{"num_bytes":39653857,"num_objects":18,"num_object_clones":12,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":1,"num_read":477,"num_read_kb":538278,"num_write":209,"num_write_kb":57988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":17763749,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":6,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3b","length":"1"},{"start":"47","length":"2"},{"start":"54","length":"1"},{"start":"5a","length":"1"}]},{"pgid":"4.a","version":"145'244","reported_seq":629,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.328774+0000","last_change":"2024-09-18T11:32:16.328774+0000","last_active":"2024-09-18T11:32:16.328774+0000","last_peered":"2024-09-18T11:32:16.328774+0000","last_clean":"2024-09-18T11:32:16.328774+0000","last_became_active":"2024-09-18T11:30:02.075538+0000","last_became_peered":"2024-09-18T11:30:02.075538+0000","last_unstale":"2024-09-18T11:32:16.328774+0000","last_undegraded":"2024-09-18T11:32:16.328774+0000","last_fullsized":"2024-09-18T11:32:16.328774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'152","last_scrub_stamp":"2024-09-18T11:31:30.146842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:30.146842+0000","objects_scrubbed":6,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:43.709866+0000","scrub_duration":0.0032260840000000002,"objects_trimmed":0,"snaptrim_duration":0.016512576000000001,"stat_sum":{"num_bytes":16817765,"num_objects":9,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":290,"num_read_kb":427037,"num_write":89,"num_write_kb":25051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":6,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3b","length":"1"},{"start":"47","length":"2"},{"start":"54","length":"1"},{"start":"5a","length":"1"}]},{"pgid":"4.9","version":"145'95","reported_seq":438,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.326095+0000","last_change":"2024-09-18T11:32:16.326095+0000","last_active":"2024-09-18T11:32:16.326095+0000","last_peered":"2024-09-18T11:32:16.326095+0000","last_clean":"2024-09-18T11:32:16.326095+0000","last_became_active":"2024-09-18T11:30:02.074414+0000","last_became_peered":"2024-09-18T11:30:02.074414+0000","last_unstale":"2024-09-18T11:32:16.326095+0000","last_undegraded":"2024-09-18T11:32:16.326095+0000","last_fullsized":"2024-09-18T11:32:16.326095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'45","last_scrub_stamp":"2024-09-18T11:31:08.104353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:08.104353+0000","objects_scrubbed":3,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:23.202655+0000","scrub_duration":0.0019130519999999999,"objects_trimmed":0,"snaptrim_duration":0.011228451,"stat_sum":{"num_bytes":6079634,"num_objects":3,"num_object_clones":2,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":96283,"num_write":43,"num_write_kb":12483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3b","length":"1"},{"start":"47","length":"2"},{"start":"54","length":"1"},{"start":"5a","length":"1"}]},{"pgid":"4.8","version":"145'68","reported_seq":389,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.317650+0000","last_change":"2024-09-18T11:32:16.317650+0000","last_active":"2024-09-18T11:32:16.317650+0000","last_peered":"2024-09-18T11:32:16.317650+0000","last_clean":"2024-09-18T11:32:16.317650+0000","last_became_active":"2024-09-18T11:30:02.076172+0000","last_became_peered":"2024-09-18T11:30:02.076172+0000","last_unstale":"2024-09-18T11:32:16.317650+0000","last_undegraded":"2024-09-18T11:32:16.317650+0000","last_fullsized":"2024-09-18T11:32:16.317650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'44","last_scrub_stamp":"2024-09-18T11:31:29.773113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:29.773113+0000","objects_scrubbed":2,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:34.758090+0000","scrub_duration":0.0016956790000000001,"objects_trimmed":0,"snaptrim_duration":0.0042401950000000004,"stat_sum":{"num_bytes":8572573,"num_objects":3,"num_object_clones":2,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":31272,"num_write":30,"num_write_kb":8391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3b","length":"1"},{"start":"47","length":"2"},{"start":"54","length":"1"},{"start":"5a","length":"1"}]},{"pgid":"4.5","version":"145'313","reported_seq":722,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.324817+0000","last_change":"2024-09-18T11:32:16.324817+0000","last_active":"2024-09-18T11:32:16.324817+0000","last_peered":"2024-09-18T11:32:16.324817+0000","last_clean":"2024-09-18T11:32:16.324817+0000","last_became_active":"2024-09-18T11:30:02.074697+0000","last_became_peered":"2024-09-18T11:30:02.074697+0000","last_unstale":"2024-09-18T11:32:16.324817+0000","last_undegraded":"2024-09-18T11:32:16.324817+0000","last_fullsized":"2024-09-18T11:32:16.324817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'148","last_scrub_stamp":"2024-09-18T11:31:08.098466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:08.098466+0000","objects_scrubbed":9,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:36.120154+0000","scrub_duration":0.0023687140000000001,"objects_trimmed":0,"snaptrim_duration":0.012357798999999999,"stat_sum":{"num_bytes":25174227,"num_objects":13,"num_object_clones":9,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":349,"num_read_kb":379930,"num_write":132,"num_write_kb":33521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":9,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3b","length":"1"},{"start":"47","length":"2"},{"start":"54","length":"1"},{"start":"5a","length":"1"}]},{"pgid":"1.0","version":"16'72","reported_seq":252,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.318871+0000","last_change":"2024-09-18T11:30:57.650616+0000","last_active":"2024-09-18T11:32:16.318871+0000","last_peered":"2024-09-18T11:32:16.318871+0000","last_clean":"2024-09-18T11:32:16.318871+0000","last_became_active":"2024-09-18T11:30:50.741897+0000","last_became_peered":"2024-09-18T11:30:50.741897+0000","last_unstale":"2024-09-18T11:32:16.318871+0000","last_undegraded":"2024-09-18T11:32:16.318871+0000","last_fullsized":"2024-09-18T11:32:16.318871+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'72","last_scrub_stamp":"2024-09-18T11:30:57.650532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_clean_scrub_stamp":"2024-09-18T11:30:57.650532+0000","objects_scrubbed":2,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:22.503050+0000","scrub_duration":0.0016372979999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":163,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.313671+0000","last_change":"2024-09-18T11:32:15.037039+0000","last_active":"2024-09-18T11:32:16.313671+0000","last_peered":"2024-09-18T11:32:16.313671+0000","last_clean":"2024-09-18T11:32:16.313671+0000","last_became_active":"2024-09-18T11:32:15.033796+0000","last_became_peered":"2024-09-18T11:32:15.033796+0000","last_unstale":"2024-09-18T11:32:16.313671+0000","last_undegraded":"2024-09-18T11:32:16.313671+0000","last_fullsized":"2024-09-18T11:32:16.313671+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'1","last_scrub_stamp":"2024-09-18T11:31:06.619850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:31:06.619850+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:32.788143+0000","scrub_duration":0.0016216189999999999,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"144'188","reported_seq":339,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.321453+0000","last_change":"2024-09-18T11:31:28.795160+0000","last_active":"2024-09-18T11:32:16.321453+0000","last_peered":"2024-09-18T11:32:16.321453+0000","last_clean":"2024-09-18T11:32:16.321453+0000","last_became_active":"2024-09-18T11:30:01.077612+0000","last_became_peered":"2024-09-18T11:30:01.077612+0000","last_unstale":"2024-09-18T11:32:16.321453+0000","last_undegraded":"2024-09-18T11:32:16.321453+0000","last_fullsized":"2024-09-18T11:32:16.321453+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'157","last_scrub_stamp":"2024-09-18T11:31:28.795082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:31:28.795082+0000","objects_scrubbed":30,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:33.013075+0000","scrub_duration":0.004667582,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37606502,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":147,"num_read_kb":3767,"num_write":215,"num_write_kb":71993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"145'152","reported_seq":557,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.339310+0000","last_change":"2024-09-18T11:32:16.339310+0000","last_active":"2024-09-18T11:32:16.339310+0000","last_peered":"2024-09-18T11:32:16.339310+0000","last_clean":"2024-09-18T11:32:16.339310+0000","last_became_active":"2024-09-18T11:32:15.034666+0000","last_became_peered":"2024-09-18T11:32:15.034666+0000","last_unstale":"2024-09-18T11:32:16.339310+0000","last_undegraded":"2024-09-18T11:32:16.339310+0000","last_fullsized":"2024-09-18T11:32:16.339310+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'66","last_scrub_stamp":"2024-09-18T11:31:07.766611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:07.766611+0000","objects_scrubbed":5,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:18.515409+0000","scrub_duration":0.0022223149999999999,"objects_trimmed":0,"snaptrim_duration":0.028306969000000001,"stat_sum":{"num_bytes":12913685,"num_objects":7,"num_object_clones":5,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":133,"num_read_kb":122020,"num_write":62,"num_write_kb":15730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":11402529,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3b","length":"1"},{"start":"47","length":"2"},{"start":"54","length":"1"},{"start":"5a","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":149,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.318776+0000","last_change":"2024-09-18T11:31:09.752752+0000","last_active":"2024-09-18T11:32:16.318776+0000","last_peered":"2024-09-18T11:32:16.318776+0000","last_clean":"2024-09-18T11:32:16.318776+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:32:16.318776+0000","last_undegraded":"2024-09-18T11:32:16.318776+0000","last_fullsized":"2024-09-18T11:32:16.318776+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:31:09.752649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:31:09.752649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020126839999999998,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"144'187","reported_seq":431,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.314722+0000","last_change":"2024-09-18T11:31:33.251215+0000","last_active":"2024-09-18T11:32:16.314722+0000","last_peered":"2024-09-18T11:32:16.314722+0000","last_clean":"2024-09-18T11:32:16.314722+0000","last_became_active":"2024-09-18T11:30:50.329599+0000","last_became_peered":"2024-09-18T11:30:50.329599+0000","last_unstale":"2024-09-18T11:32:16.314722+0000","last_undegraded":"2024-09-18T11:32:16.314722+0000","last_fullsized":"2024-09-18T11:32:16.314722+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'145","last_scrub_stamp":"2024-09-18T11:31:33.251143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:31:33.251143+0000","objects_scrubbed":28,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:33.814152+0000","scrub_duration":0.0067874609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40061727,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":160,"num_read_kb":9933,"num_write":206,"num_write_kb":70006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":20418189,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"145'314","reported_seq":756,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.352026+0000","last_change":"2024-09-18T11:32:16.352026+0000","last_active":"2024-09-18T11:32:16.352026+0000","last_peered":"2024-09-18T11:32:16.352026+0000","last_clean":"2024-09-18T11:32:16.352026+0000","last_became_active":"2024-09-18T11:30:02.076465+0000","last_became_peered":"2024-09-18T11:30:02.076465+0000","last_unstale":"2024-09-18T11:32:16.352026+0000","last_undegraded":"2024-09-18T11:32:16.352026+0000","last_fullsized":"2024-09-18T11:32:16.352026+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"92'140","last_scrub_stamp":"2024-09-18T11:31:18.755540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:18.755540+0000","objects_scrubbed":8,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:28.845667+0000","scrub_duration":0.0038425999999999998,"objects_trimmed":0,"snaptrim_duration":0.040818979999999998,"stat_sum":{"num_bytes":23844120,"num_objects":14,"num_object_clones":9,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":434,"num_read_kb":444109,"num_write":136,"num_write_kb":37492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":9,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3b","length":"1"},{"start":"47","length":"2"},{"start":"54","length":"1"},{"start":"5a","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.321505+0000","last_change":"2024-09-18T11:32:15.027026+0000","last_active":"2024-09-18T11:32:16.321505+0000","last_peered":"2024-09-18T11:32:16.321505+0000","last_clean":"2024-09-18T11:32:16.321505+0000","last_became_active":"2024-09-18T11:32:15.026717+0000","last_became_peered":"2024-09-18T11:32:15.026717+0000","last_unstale":"2024-09-18T11:32:16.321505+0000","last_undegraded":"2024-09-18T11:32:16.321505+0000","last_fullsized":"2024-09-18T11:32:16.321505+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:31:19.127335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:31:19.127335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:29.864700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"144'136","reported_seq":392,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.310531+0000","last_change":"2024-09-18T11:32:15.213232+0000","last_active":"2024-09-18T11:32:16.310531+0000","last_peered":"2024-09-18T11:32:16.310531+0000","last_clean":"2024-09-18T11:32:16.310531+0000","last_became_active":"2024-09-18T11:32:15.033700+0000","last_became_peered":"2024-09-18T11:32:15.033700+0000","last_unstale":"2024-09-18T11:32:16.310531+0000","last_undegraded":"2024-09-18T11:32:16.310531+0000","last_fullsized":"2024-09-18T11:32:16.310531+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'83","last_scrub_stamp":"2024-09-18T11:31:06.801507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:31:06.801507+0000","objects_scrubbed":15,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:19.898644+0000","scrub_duration":0.0021766860000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19887692,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":103,"num_read_kb":943,"num_write":148,"num_write_kb":42818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":12702421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"145'208","reported_seq":632,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.322616+0000","last_change":"2024-09-18T11:32:16.322616+0000","last_active":"2024-09-18T11:32:16.322616+0000","last_peered":"2024-09-18T11:32:16.322616+0000","last_clean":"2024-09-18T11:32:16.322616+0000","last_became_active":"2024-09-18T11:32:15.032533+0000","last_became_peered":"2024-09-18T11:32:15.032533+0000","last_unstale":"2024-09-18T11:32:16.322616+0000","last_undegraded":"2024-09-18T11:32:16.322616+0000","last_fullsized":"2024-09-18T11:32:16.322616+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'169","last_scrub_stamp":"2024-09-18T11:31:29.756257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:29.756257+0000","objects_scrubbed":7,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:42.087986+0000","scrub_duration":0.002423396,"objects_trimmed":0,"snaptrim_duration":0.011826122,"stat_sum":{"num_bytes":11170332,"num_objects":6,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":238,"num_read_kb":281338,"num_write":94,"num_write_kb":26318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":7785559,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3b","length":"1"},{"start":"47","length":"2"},{"start":"54","length":"1"},{"start":"5a","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":151,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.313598+0000","last_change":"2024-09-18T11:31:16.657820+0000","last_active":"2024-09-18T11:32:16.313598+0000","last_peered":"2024-09-18T11:32:16.313598+0000","last_clean":"2024-09-18T11:32:16.313598+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:32:16.313598+0000","last_undegraded":"2024-09-18T11:32:16.313598+0000","last_fullsized":"2024-09-18T11:32:16.313598+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"18'2","last_scrub_stamp":"2024-09-18T11:31:16.657749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:31:16.657749+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:30.076694+0000","scrub_duration":0.00143407,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"141'142","reported_seq":375,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.314739+0000","last_change":"2024-09-18T11:31:09.149266+0000","last_active":"2024-09-18T11:32:16.314739+0000","last_peered":"2024-09-18T11:32:16.314739+0000","last_clean":"2024-09-18T11:32:16.314739+0000","last_became_active":"2024-09-18T11:30:50.741594+0000","last_became_peered":"2024-09-18T11:30:50.741594+0000","last_unstale":"2024-09-18T11:32:16.314739+0000","last_undegraded":"2024-09-18T11:32:16.314739+0000","last_fullsized":"2024-09-18T11:32:16.314739+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'103","last_scrub_stamp":"2024-09-18T11:31:09.149191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:31:09.149191+0000","objects_scrubbed":26,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:33.612849+0000","scrub_duration":0.0034665379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24266822,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":116,"num_read_kb":7076,"num_write":156,"num_write_kb":55186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":16290641,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"145'112","reported_seq":458,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.313288+0000","last_change":"2024-09-18T11:32:16.313288+0000","last_active":"2024-09-18T11:32:16.313288+0000","last_peered":"2024-09-18T11:32:16.313288+0000","last_clean":"2024-09-18T11:32:16.313288+0000","last_became_active":"2024-09-18T11:30:02.076874+0000","last_became_peered":"2024-09-18T11:30:02.076874+0000","last_unstale":"2024-09-18T11:32:16.313288+0000","last_undegraded":"2024-09-18T11:32:16.313288+0000","last_fullsized":"2024-09-18T11:32:16.313288+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'70","last_scrub_stamp":"2024-09-18T11:31:09.807739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:09.807739+0000","objects_scrubbed":3,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:22.006274+0000","scrub_duration":0.002685074,"objects_trimmed":0,"snaptrim_duration":0.002664872,"stat_sum":{"num_bytes":5310084,"num_objects":3,"num_object_clones":2,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":118440,"num_write":54,"num_write_kb":14782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3b","length":"1"},{"start":"47","length":"2"},{"start":"54","length":"1"},{"start":"5a","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":163,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.310902+0000","last_change":"2024-09-18T11:31:16.777136+0000","last_active":"2024-09-18T11:32:16.310902+0000","last_peered":"2024-09-18T11:32:16.310902+0000","last_clean":"2024-09-18T11:32:16.310902+0000","last_became_active":"2024-09-18T11:30:50.331026+0000","last_became_peered":"2024-09-18T11:30:50.331026+0000","last_unstale":"2024-09-18T11:32:16.310902+0000","last_undegraded":"2024-09-18T11:32:16.310902+0000","last_fullsized":"2024-09-18T11:32:16.310902+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:31:16.777069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:31:16.777069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:23.123141+0000","scrub_duration":0.001722945,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"145'212","reported_seq":605,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.319519+0000","last_change":"2024-09-18T11:32:16.319519+0000","last_active":"2024-09-18T11:32:16.319519+0000","last_peered":"2024-09-18T11:32:16.319519+0000","last_clean":"2024-09-18T11:32:16.319519+0000","last_became_active":"2024-09-18T11:30:50.333722+0000","last_became_peered":"2024-09-18T11:30:50.333722+0000","last_unstale":"2024-09-18T11:32:16.319519+0000","last_undegraded":"2024-09-18T11:32:16.319519+0000","last_fullsized":"2024-09-18T11:32:16.319519+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"99'129","last_scrub_stamp":"2024-09-18T11:31:28.771630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:28.771630+0000","objects_scrubbed":9,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:47.590830+0000","scrub_duration":0.00269141,"objects_trimmed":0,"snaptrim_duration":0.0059858799999999998,"stat_sum":{"num_bytes":14570292,"num_objects":9,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":214,"num_read_kb":232344,"num_write":106,"num_write_kb":27943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":5737174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":7,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3b","length":"1"},{"start":"47","length":"2"},{"start":"54","length":"1"},{"start":"5a","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":160,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.318637+0000","last_change":"2024-09-18T11:31:16.761649+0000","last_active":"2024-09-18T11:32:16.318637+0000","last_peered":"2024-09-18T11:32:16.318637+0000","last_clean":"2024-09-18T11:32:16.318637+0000","last_became_active":"2024-09-18T11:30:50.327483+0000","last_became_peered":"2024-09-18T11:30:50.327483+0000","last_unstale":"2024-09-18T11:32:16.318637+0000","last_undegraded":"2024-09-18T11:32:16.318637+0000","last_fullsized":"2024-09-18T11:32:16.318637+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:31:16.761562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:31:16.761562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:26.978531+0000","scrub_duration":0.0012814949999999999,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"145'219","reported_seq":596,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.328282+0000","last_change":"2024-09-18T11:32:16.328282+0000","last_active":"2024-09-18T11:32:16.328282+0000","last_peered":"2024-09-18T11:32:16.328282+0000","last_clean":"2024-09-18T11:32:16.328282+0000","last_became_active":"2024-09-18T11:30:02.072165+0000","last_became_peered":"2024-09-18T11:30:02.072165+0000","last_unstale":"2024-09-18T11:32:16.328282+0000","last_undegraded":"2024-09-18T11:32:16.328282+0000","last_fullsized":"2024-09-18T11:32:16.328282+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'120","last_scrub_stamp":"2024-09-18T11:31:06.491290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:06.491290+0000","objects_scrubbed":8,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002837347,"objects_trimmed":0,"snaptrim_duration":0.0067295100000000002,"stat_sum":{"num_bytes":17539107,"num_objects":8,"num_object_clones":5,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":226,"num_read_kb":253817,"num_write":106,"num_write_kb":30872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":6,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3b","length":"1"},{"start":"47","length":"2"},{"start":"54","length":"1"},{"start":"5a","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":152,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.310980+0000","last_change":"2024-09-18T11:30:57.765542+0000","last_active":"2024-09-18T11:32:16.310980+0000","last_peered":"2024-09-18T11:32:16.310980+0000","last_clean":"2024-09-18T11:32:16.310980+0000","last_became_active":"2024-09-18T11:29:50.028674+0000","last_became_peered":"2024-09-18T11:29:50.028674+0000","last_unstale":"2024-09-18T11:32:16.310980+0000","last_undegraded":"2024-09-18T11:32:16.310980+0000","last_fullsized":"2024-09-18T11:32:16.310980+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:30:57.765475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:30:57.765475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:20.322582+0000","scrub_duration":0.001835167,"objects_trimmed":0,"snaptrim_duration":0.00031621200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"145'302","reported_seq":708,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.338200+0000","last_change":"2024-09-18T11:32:16.338200+0000","last_active":"2024-09-18T11:32:16.338200+0000","last_peered":"2024-09-18T11:32:16.338200+0000","last_clean":"2024-09-18T11:32:16.338200+0000","last_became_active":"2024-09-18T11:30:02.075529+0000","last_became_peered":"2024-09-18T11:30:02.075529+0000","last_unstale":"2024-09-18T11:32:16.338200+0000","last_undegraded":"2024-09-18T11:32:16.338200+0000","last_fullsized":"2024-09-18T11:32:16.338200+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'230","last_scrub_stamp":"2024-09-18T11:31:32.253118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:31:32.253118+0000","objects_scrubbed":9,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:49.207202+0000","scrub_duration":0.0038969130000000001,"objects_trimmed":0,"snaptrim_duration":0.023299302000000001,"stat_sum":{"num_bytes":19740239,"num_objects":9,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":338,"num_read_kb":396283,"num_write":145,"num_write_kb":39506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3b","length":"1"},{"start":"47","length":"2"},{"start":"54","length":"1"},{"start":"5a","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":159,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-18T11:32:16.312249+0000","last_change":"2024-09-18T11:32:06.213366+0000","last_active":"2024-09-18T11:32:16.312249+0000","last_peered":"2024-09-18T11:32:16.312249+0000","last_clean":"2024-09-18T11:32:16.312249+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:32:16.312249+0000","last_undegraded":"2024-09-18T11:32:16.312249+0000","last_fullsized":"2024-09-18T11:32:16.312249+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:32:06.213281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:32:06.213281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:23.965948+0000","scrub_duration":0.0013619280000000001,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":269291110,"num_objects":131,"num_object_clones":81,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129,"num_whiteouts":5,"num_read":4186,"num_read_kb":4825205,"num_write":1589,"num_write_kb":434201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":58903324,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":66,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":423403520,"data_stored":546848031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3634,"ondisk_log_size":3634,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":121822743,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":526,"num_read_kb":21719,"num_write":725,"num_write_kb":240003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":49411251,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":206528512,"data_stored":206050577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4182,"internal_metadata":0},"log_size":653,"ondisk_log_size":653,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332980,"kb_used_data":35620,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94038860,"statfs":{"total":96636764160,"available":96295792640,"internally_reserved":0,"allocated":36474880,"data_stored":48070489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":1,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.46000000000000002,"15min":0.46500000000000002},"min":{"1min":0.33100000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.52000000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.51100000000000001,"15min":0.52000000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.60599999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.48299999999999998}]},{"osd":2,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41999999999999998,"15min":0.42099999999999999},"min":{"1min":0.32400000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.53100000000000003,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.313},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.42999999999999999,"15min":0.436},"min":{"1min":0.34000000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.47699999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.46400000000000002}]},{"osd":3,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44,"15min":0.442},"min":{"1min":0.34300000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.52400000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.47699999999999998,"15min":0.48799999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.48299999999999998,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.34599999999999997}]},{"osd":4,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.44800000000000001,"15min":0.46000000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.48899999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.49199999999999999,"15min":0.502},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.57299999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.47499999999999998}]},{"osd":5,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.42899999999999999,"15min":0.443},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.58799999999999997,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.45900000000000002,"15min":0.47099999999999997},"min":{"1min":0.28699999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.68400000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.28399999999999997}]},{"osd":6,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.51300000000000001,"15min":0.52700000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54700000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.49399999999999999,"15min":0.504},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.626,"5min":0.75,"15min":0.75},"last":0.40000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":368568,"kb_used_data":67048,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94003272,"statfs":{"total":96636764160,"available":96259350528,"internally_reserved":0,"allocated":68657152,"data_stored":81734743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":1,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46600000000000003,"15min":0.46899999999999997},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.77200000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.45100000000000001,"15min":0.45500000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.65500000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.47299999999999998}]},{"osd":2,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46100000000000002,"15min":0.46300000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.75600000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.44,"5min":0.443,"15min":0.443},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.44900000000000001}]},{"osd":3,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.503,"15min":0.51500000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59599999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66100000000000003,"5min":1.032,"15min":1.032},"last":0.52700000000000002}]},{"osd":4,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.38500000000000001,"15min":0.38800000000000001},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.48699999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.41399999999999998,"15min":0.41799999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.54700000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.499}]},{"osd":5,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.27800000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.307},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.42499999999999999,"15min":0.42699999999999999},"min":{"1min":0.29099999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60499999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.34799999999999998}]},{"osd":7,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.45400000000000001,"15min":0.46000000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.622,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.37},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47099999999999997,"15min":0.47399999999999998},"min":{"1min":0.32300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":373700,"kb_used_data":72184,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93998140,"statfs":{"total":96636764160,"available":96254095360,"internally_reserved":0,"allocated":73916416,"data_stored":91504881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Wed Sep 18 11:31:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.443,"15min":0.44800000000000001},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.57199999999999995,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.45400000000000001}]},{"osd":2,"last update":"Wed Sep 18 11:31:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.48699999999999999,"15min":0.498},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.61199999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.48399999999999999,"15min":0.498},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.53700000000000003,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.42299999999999999}]},{"osd":3,"last update":"Wed Sep 18 11:31:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.48099999999999998,"15min":0.49299999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58399999999999996,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.434},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.49099999999999999,"15min":0.499},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.627,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.46300000000000002}]},{"osd":4,"last update":"Wed Sep 18 11:31:53 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.42999999999999999,"15min":0.44400000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54800000000000004,"5min":0.63,"15min":0.63},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.44400000000000001,"15min":0.46000000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.55900000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.25900000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:31:53 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.42099999999999999,"15min":0.42999999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.53600000000000003,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.56999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.27800000000000002}]},{"osd":7,"last update":"Wed Sep 18 11:31:53 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.48799999999999999,"15min":0.504},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.59799999999999998,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.50900000000000001,"15min":0.52200000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.58499999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.40100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434352,"kb_used_data":132832,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93937488,"statfs":{"total":96636764160,"available":96191987712,"internally_reserved":0,"allocated":136019968,"data_stored":162013507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.54200000000000004}]},{"osd":1,"last update":"Wed Sep 18 11:31:47 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.48899999999999999,"15min":0.504},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.54400000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.48799999999999999,"15min":0.5},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.57299999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.57099999999999995}]},{"osd":2,"last update":"Wed Sep 18 11:31:47 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.48599999999999999,"15min":0.49399999999999999},"min":{"1min":0.32200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68899999999999995,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.50700000000000001,"15min":0.52100000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59199999999999997,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.52800000000000002}]},{"osd":3,"last update":"Wed Sep 18 11:31:47 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.505,"15min":0.51800000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.63500000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.51500000000000001,"15min":0.53000000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.66700000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.58299999999999996}]},{"osd":5,"last update":"Wed Sep 18 11:31:47 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.45800000000000002,"15min":0.46800000000000003},"min":{"1min":0.27700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.65600000000000003,"5min":1.002,"15min":1.002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.496,"15min":0.51300000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.63600000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.42299999999999999}]},{"osd":6,"last update":"Wed Sep 18 11:31:47 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.46200000000000002,"15min":0.47299999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.70699999999999996,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.49399999999999999,"15min":0.50700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.60999999999999999,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.61299999999999999}]},{"osd":7,"last update":"Wed Sep 18 11:31:47 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.5,"15min":0.51200000000000001},"min":{"1min":0.27400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67700000000000005,"5min":0.75,"15min":0.75},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.53400000000000003,"15min":0.54800000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72999999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.629}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":388564,"kb_used_data":87040,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93983276,"statfs":{"total":96636764160,"available":96238874624,"internally_reserved":0,"allocated":89128960,"data_stored":97916280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.255}]},{"osd":1,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.33700000000000002,"15min":0.34300000000000003},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51300000000000001,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.313},{"interface":"front","average":{"1min":0.29099999999999998,"5min":0.35799999999999998,"15min":0.36899999999999999},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.40899999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.27100000000000002}]},{"osd":2,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.23499999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.45600000000000002,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.315,"5min":0.39100000000000001,"15min":0.40300000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.52000000000000002,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.38400000000000001}]},{"osd":4,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51100000000000001,"15min":0.51400000000000001},"min":{"1min":0.376,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.66700000000000004,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65700000000000003,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.32900000000000001}]},{"osd":5,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.497,"15min":0.505},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.58999999999999997,"5min":0.752,"15min":0.752},"last":0.441},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.54800000000000004,"15min":0.56100000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63600000000000001,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.40100000000000002}]},{"osd":6,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.61799999999999999,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.55500000000000005,"15min":0.56499999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.65800000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53400000000000003}]},{"osd":7,"last update":"Wed Sep 18 11:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.58699999999999997,"15min":0.60699999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.622,"5min":1.26,"15min":1.26},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.56699999999999995,"15min":0.57899999999999996},"min":{"1min":0.40999999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.65400000000000003,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.41199999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302448,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069392,"statfs":{"total":96636764160,"available":96327057408,"internally_reserved":0,"allocated":950272,"data_stored":546686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":1,"last update":"Wed Sep 18 11:31:52 2024","interfaces":[{"interface":"back","average":{"1min":0.28999999999999998,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.46000000000000002,"5min":1.621,"15min":1.621},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.29199999999999998,"5min":0.38600000000000001,"15min":0.40200000000000002},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.40100000000000002,"5min":1.589,"15min":1.589},"last":0.35699999999999998}]},{"osd":3,"last update":"Wed Sep 18 11:31:52 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.36199999999999999,"15min":0.37},"min":{"1min":0.254,"5min":0.249,"15min":0.249},"max":{"1min":0.48299999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.372},{"interface":"front","average":{"1min":0.30499999999999999,"5min":0.40100000000000002,"15min":0.41699999999999998},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.434,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.38500000000000001}]},{"osd":4,"last update":"Wed Sep 18 11:31:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.54200000000000004,"15min":0.55400000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76100000000000001,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.498},{"interface":"front","average":{"1min":0.442,"5min":0.50700000000000001,"15min":0.51800000000000002},"min":{"1min":0.35799999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.67300000000000004,"5min":1.012,"15min":1.012},"last":0.44600000000000001}]},{"osd":5,"last update":"Wed Sep 18 11:31:52 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.53500000000000003,"15min":0.54300000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66200000000000003,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69899999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.48099999999999998}]},{"osd":6,"last update":"Wed Sep 18 11:31:52 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.53100000000000003,"15min":0.54300000000000004},"min":{"1min":0.34799999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.64800000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.55000000000000004,"15min":0.56599999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64200000000000002,"5min":1.165,"15min":1.165},"last":0.54100000000000004}]},{"osd":7,"last update":"Wed Sep 18 11:31:52 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.371,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68899999999999995,"5min":1.073,"15min":1.073},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.54000000000000004,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.65600000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.46400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":423956,"kb_used_data":122440,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93947884,"statfs":{"total":96636764160,"available":96202633216,"internally_reserved":0,"allocated":125378560,"data_stored":160863708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24299999999999999}]},{"osd":2,"last update":"Wed Sep 18 11:31:51 2024","interfaces":[{"interface":"back","average":{"1min":0.255,"5min":0.36199999999999999,"15min":0.38},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.434,"5min":1.333,"15min":1.333},"last":0.31},{"interface":"front","average":{"1min":0.313,"5min":0.40799999999999997,"15min":0.42399999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.44500000000000001,"5min":1.298,"15min":1.298},"last":0.29799999999999999}]},{"osd":3,"last update":"Wed Sep 18 11:31:51 2024","interfaces":[{"interface":"back","average":{"1min":0.26300000000000001,"5min":0.36299999999999999,"15min":0.379},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.41999999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.30099999999999999,"5min":0.35299999999999998,"15min":0.36099999999999999},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.495,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.28100000000000003}]},{"osd":4,"last update":"Wed Sep 18 11:31:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.51500000000000001,"15min":0.52500000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.73999999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.49299999999999999,"15min":0.505},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69199999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.34999999999999998}]},{"osd":5,"last update":"Wed Sep 18 11:31:51 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.504,"15min":0.51700000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.59399999999999997,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.51200000000000001,"15min":0.52400000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.58499999999999996,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.32300000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:31:51 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.52800000000000002,"15min":0.53600000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70999999999999996,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.52900000000000003,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.57799999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.33400000000000002}]},{"osd":7,"last update":"Wed Sep 18 11:31:51 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.53100000000000003,"15min":0.54500000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.55900000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73999999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.34200000000000003}]}]},{"osd":0,"up_from":0,"seq":511101108232,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36782080,"data_stored":36708830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4182,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13004800,"data_stored":12976341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12730368,"data_stored":12702421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48484352,"data_stored":48361383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":42545152,"data_stored":42452212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":22958080,"data_stored":22891110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":30023680,"data_stored":29958280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25010176,"data_stored":27495860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":111374336,"data_stored":147329623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32714752,"data_stored":38252603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":39661568,"data_stored":48998748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":92491776,"data_stored":119010504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":50020352,"data_stored":68064927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":37138432,"data_stored":50699700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":34992128,"data_stored":46996066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:32:19.837 INFO:tasks.thrashosds.thrasher:Setting 4.b to [5, 3] 2024-09-18T11:32:19.837 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '4.b', '5', '3'] 2024-09-18T11:32:19.837 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 4.b 5 3 2024-09-18T11:32:19.869 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:19.869 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:19.878 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:32:19.919 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:19.917+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:20.017 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:20.018 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:20.018 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:20.015+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:20.026 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:32:20.119 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:20.118+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:20.164 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:20.164 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:20.172 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:32:20.219 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:20.218+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:20.270 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:20.270 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:20.278 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:32:20.322 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:20.320+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:20.330 INFO:teuthology.orchestra.run.smithi119.stderr:set 4.b pg_upmap mapping to [5,3] 2024-09-18T11:32:20.331 INFO:tasks.rados.rados.0.smithi119.stdout:1801: done (0 left) 2024-09-18T11:32:20.331 INFO:tasks.rados.rados.0.smithi119.stdout:1802: read oid 40 snap -1 2024-09-18T11:32:20.331 INFO:tasks.rados.rados.0.smithi119.stdout:1802: expect (ObjNum 287 snap 41 seq_num 287) 2024-09-18T11:32:20.331 INFO:tasks.rados.rados.0.smithi119.stdout:1803: read oid 45 snap -1 2024-09-18T11:32:20.331 INFO:tasks.rados.rados.0.smithi119.stdout:1803: expect (ObjNum 254 snap 30 seq_num 254) 2024-09-18T11:32:20.331 INFO:tasks.rados.rados.0.smithi119.stdout:1804: set_chunk oid 46 target oid 2024-09-18T11:32:20.347 INFO:tasks.rados.rados.0.smithi119.stdout:1804: set_chunk oid 46 offset: 379904 length: 948224 target oid d22605c6279c8fffe2b5ff0eb298c7f7c4347073f9c58d076a7466bbfe503dc9 offset: 0 snap 36 2024-09-18T11:32:20.347 INFO:tasks.rados.rados.0.smithi119.stdout:1805: snap_remove snap 51 2024-09-18T11:32:20.385 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:20.386 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:20.394 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:32:20.421 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:20.420+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:20.522 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:20.520+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:20.534 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:20.534 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:20.543 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:32:20.623 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:20.622+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:20.681 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:20.681 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:20.690 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:32:20.723 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:20.722+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:20.786 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:20.786 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:20.795 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:32:20.839 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:20.837+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:20.865 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:32:20.894 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:20.894 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:20.904 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:32:20.949 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:20.948+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:20.981 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:20.981 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:20.996 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:20.997 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:21.011 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:21.011 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:21.015 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:21.015 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:21.024 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:32:21.025 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:21.026 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:21.026 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:21.024+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:21.041 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:21.041 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:21.056 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:21.056 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:21.071 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:21.071 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:21.086 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:21.086 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:21.127 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:21.126+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:21.164 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:21.164 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:21.164 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:21.164 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:21.164 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:21.172 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:32:21.228 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:21.226+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:21.312 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:21.312 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:21.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:21.313 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:21.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.313 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:32:21.313 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:21.313 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 106.89389294199999, 2024-09-18T11:32:21.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:32:21.313 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.313 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.313 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.313 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.313 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.313 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:32:21.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.314 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.314 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.314 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:21.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.314 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.314 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:21.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.314 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.314 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:32:21.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:21.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:32:21.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:32:21.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:32:21.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:32:21.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:32:21.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:32:21.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:32:21.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:32:21.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:32:21.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:32:21.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:21.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:32:21.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:32:21.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:21.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:32:21.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:32:21.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:21.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:32:21.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:32:21.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:32:21.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:32:21.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:32:21.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:32:21.318 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.318 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.318 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.319 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:172 4.5 e78/22 4:a087a566:::smithi11939506-46:19 v 78'135, mlcod=77'132)", 2024-09-18T11:32:21.319 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:32:21.319 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 79.924156288000006, 2024-09-18T11:32:21.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010938658, 2024-09-18T11:32:21.319 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.319 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.320 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:32:21.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:32:21.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385823+0000", 2024-09-18T11:32:21.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:21.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385830+0000", 2024-09-18T11:32:21.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2790000000000001e-06 2024-09-18T11:32:21.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385843+0000", 2024-09-18T11:32:21.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2785e-05 2024-09-18T11:32:21.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385848+0000", 2024-09-18T11:32:21.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5570000000000003e-06 2024-09-18T11:32:21.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385914+0000", 2024-09-18T11:32:21.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5168999999999999e-05 2024-09-18T11:32:21.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385930+0000", 2024-09-18T11:32:21.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6884999999999998e-05 2024-09-18T11:32:21.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396721+0000", 2024-09-18T11:32:21.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010790137 2024-09-18T11:32:21.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396762+0000", 2024-09-18T11:32:21.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1635999999999999e-05 2024-09-18T11:32:21.323 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.323 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.323 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.324 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:32:21.324 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:21.324 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 79.923663536000007, 2024-09-18T11:32:21.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:32:21.324 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.324 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.324 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:21.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:21.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:21.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:21.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:32:21.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:32:21.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:32:21.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:32:21.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:32:21.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:32:21.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:32:21.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:32:21.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:32:21.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:32:21.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:32:21.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:32:21.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:32:21.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:32:21.328 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.328 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.328 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.328 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:21.328 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:21.328 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 72.337550930000006, 2024-09-18T11:32:21.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:32:21.328 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.328 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.329 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.329 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.329 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.329 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:32:21.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.329 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:21.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:21.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:32:21.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:21.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:32:21.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:32:21.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:32:21.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:32:21.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:32:21.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:32:21.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:32:21.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:32:21.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:32:21.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:32:21.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:21.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:32:21.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:32:21.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:21.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:32:21.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:32:21.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:21.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:32:21.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:32:21.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:32:21.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:32:21.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:32:21.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:32:21.334 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.334 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.334 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.334 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2956 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~2100908,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:21.334 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:21.334 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.282754679999996, 2024-09-18T11:32:21.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011863831, 2024-09-18T11:32:21.334 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.334 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:21.334 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.335 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.335 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.335 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2956 2024-09-18T11:32:21.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.335 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:21.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:21.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:21.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:21.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027228+0000", 2024-09-18T11:32:21.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:32:21.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027235+0000", 2024-09-18T11:32:21.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:32:21.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027239+0000", 2024-09-18T11:32:21.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.535e-06 2024-09-18T11:32:21.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027831+0000", 2024-09-18T11:32:21.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059244699999999996 2024-09-18T11:32:21.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:21.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027879+0000", 2024-09-18T11:32:21.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:32:21.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038098+0000", 2024-09-18T11:32:21.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010219137 2024-09-18T11:32:21.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038207+0000", 2024-09-18T11:32:21.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010951999999999999 2024-09-18T11:32:21.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039089+0000", 2024-09-18T11:32:21.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00088151699999999996 2024-09-18T11:32:21.339 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.339 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.339 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.339 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2957 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:21.339 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:21.339 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.282736896000003, 2024-09-18T11:32:21.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012041979, 2024-09-18T11:32:21.340 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.340 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:21.340 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.340 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.340 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.340 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2957 2024-09-18T11:32:21.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.340 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:21.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:21.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:21.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:21.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027245+0000", 2024-09-18T11:32:21.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:32:21.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027251+0000", 2024-09-18T11:32:21.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0090000000000004e-06 2024-09-18T11:32:21.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027255+0000", 2024-09-18T11:32:21.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:32:21.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027895+0000", 2024-09-18T11:32:21.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064076400000000005 2024-09-18T11:32:21.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:21.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027930+0000", 2024-09-18T11:32:21.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4334e-05 2024-09-18T11:32:21.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039111+0000", 2024-09-18T11:32:21.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181455999999999 2024-09-18T11:32:21.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039156+0000", 2024-09-18T11:32:21.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5256000000000002e-05 2024-09-18T11:32:21.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039285+0000", 2024-09-18T11:32:21.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012839200000000001 2024-09-18T11:32:21.344 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.344 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.344 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.345 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:978 4.b e132/67 4:d8d14c58:::smithi11939506-49:30 v 132'417, mlcod=128'414)", 2024-09-18T11:32:21.345 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:21.345 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.284705614, 2024-09-18T11:32:21.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011942183, 2024-09-18T11:32:21.345 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.345 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.345 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:21.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:21.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:21.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:21.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025278+0000", 2024-09-18T11:32:21.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9060000000000004e-06 2024-09-18T11:32:21.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025288+0000", 2024-09-18T11:32:21.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.028e-05 2024-09-18T11:32:21.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025293+0000", 2024-09-18T11:32:21.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2700000000000004e-06 2024-09-18T11:32:21.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025346+0000", 2024-09-18T11:32:21.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2862999999999999e-05 2024-09-18T11:32:21.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025370+0000", 2024-09-18T11:32:21.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4005000000000001e-05 2024-09-18T11:32:21.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037159+0000", 2024-09-18T11:32:21.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011788343 2024-09-18T11:32:21.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037216+0000", 2024-09-18T11:32:21.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7695000000000002e-05 2024-09-18T11:32:21.349 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.349 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.349 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.349 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3930 4.b e132/67 4:d3ae8215:::smithi11939506-41:head v 132'421, mlcod=128'414)", 2024-09-18T11:32:21.349 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:21.349 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.281157365999999, 2024-09-18T11:32:21.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015093554, 2024-09-18T11:32:21.349 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.349 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.349 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:21.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:21.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028821+0000", 2024-09-18T11:32:21.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:21.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029805+0000", 2024-09-18T11:32:21.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00098410899999999994 2024-09-18T11:32:21.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029817+0000", 2024-09-18T11:32:21.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2058e-05 2024-09-18T11:32:21.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029823+0000", 2024-09-18T11:32:21.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6450000000000002e-06 2024-09-18T11:32:21.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032330+0000", 2024-09-18T11:32:21.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025067710000000001 2024-09-18T11:32:21.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032349+0000", 2024-09-18T11:32:21.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9729e-05 2024-09-18T11:32:21.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043878+0000", 2024-09-18T11:32:21.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011528514 2024-09-18T11:32:21.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043916+0000", 2024-09-18T11:32:21.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8049000000000001e-05 2024-09-18T11:32:21.353 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.353 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.353 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.353 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:162 3.1 3:8c826a1f:::e682cfc1e82c88472b9acf1cd97978bc105aa035f10b685bb4cace95809b52bf:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:21.353 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:32:21.354 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 11.729035207999999, 2024-09-18T11:32:21.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013234571000000001, 2024-09-18T11:32:21.354 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.354 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.354 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:32:21.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:32:21.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580943+0000", 2024-09-18T11:32:21.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:21.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582556+0000", 2024-09-18T11:32:21.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016132970000000001 2024-09-18T11:32:21.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582572+0000", 2024-09-18T11:32:21.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5846999999999999e-05 2024-09-18T11:32:21.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582578+0000", 2024-09-18T11:32:21.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4810000000000003e-06 2024-09-18T11:32:21.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582632+0000", 2024-09-18T11:32:21.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4156000000000001e-05 2024-09-18T11:32:21.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582819+0000", 2024-09-18T11:32:21.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000186431 2024-09-18T11:32:21.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:21.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583118+0000", 2024-09-18T11:32:21.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029880100000000001 2024-09-18T11:32:21.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:21.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590676+0000", 2024-09-18T11:32:21.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075588670000000004 2024-09-18T11:32:21.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:21.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594111+0000", 2024-09-18T11:32:21.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034348550000000001 2024-09-18T11:32:21.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594142+0000", 2024-09-18T11:32:21.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.099e-05 2024-09-18T11:32:21.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594179+0000", 2024-09-18T11:32:21.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6831e-05 2024-09-18T11:32:21.359 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.359 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.359 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.359 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3974 4.d 4:b5a463bf:::smithi11939506-20:head [write 665643~697786,stat] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:21.359 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:32:21.359 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 11.727948525, 2024-09-18T11:32:21.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016621174999999998, 2024-09-18T11:32:21.359 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.359 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.359 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.359 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.359 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.360 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3974 2024-09-18T11:32:21.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.360 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:32:21.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:32:21.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582030+0000", 2024-09-18T11:32:21.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:21.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585185+0000", 2024-09-18T11:32:21.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003155644 2024-09-18T11:32:21.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585208+0000", 2024-09-18T11:32:21.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3013000000000001e-05 2024-09-18T11:32:21.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585215+0000", 2024-09-18T11:32:21.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7569999999999998e-06 2024-09-18T11:32:21.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585278+0000", 2024-09-18T11:32:21.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2699000000000004e-05 2024-09-18T11:32:21.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585374+0000", 2024-09-18T11:32:21.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.6062000000000003e-05 2024-09-18T11:32:21.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:32:21.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585577+0000", 2024-09-18T11:32:21.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000202926 2024-09-18T11:32:21.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:21.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596544+0000", 2024-09-18T11:32:21.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010966848 2024-09-18T11:32:21.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:21.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598594+0000", 2024-09-18T11:32:21.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020502049999999998 2024-09-18T11:32:21.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598625+0000", 2024-09-18T11:32:21.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1412999999999999e-05 2024-09-18T11:32:21.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598652+0000", 2024-09-18T11:32:21.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7166999999999999e-05 2024-09-18T11:32:21.365 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.365 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.365 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.365 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:166 3.1 3:a73e635b:::b6ee478f0f661ec7bac3d17c2ca2fdaff71a47570f64844a7ba79bbb2a0e8d9d:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:21.365 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:32:21.365 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 11.727389158999999, 2024-09-18T11:32:21.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011711037000000001, 2024-09-18T11:32:21.365 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.365 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.365 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:32:21.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:32:21.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582590+0000", 2024-09-18T11:32:21.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:21.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582673+0000", 2024-09-18T11:32:21.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2861000000000005e-05 2024-09-18T11:32:21.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582683+0000", 2024-09-18T11:32:21.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0081999999999999e-05 2024-09-18T11:32:21.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582688+0000", 2024-09-18T11:32:21.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6870000000000002e-06 2024-09-18T11:32:21.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584266+0000", 2024-09-18T11:32:21.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001578423 2024-09-18T11:32:21.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584407+0000", 2024-09-18T11:32:21.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014144599999999999 2024-09-18T11:32:21.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:21.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584599+0000", 2024-09-18T11:32:21.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000191881 2024-09-18T11:32:21.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:21.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590680+0000", 2024-09-18T11:32:21.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060801479999999996 2024-09-18T11:32:21.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:21.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594224+0000", 2024-09-18T11:32:21.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035441700000000001 2024-09-18T11:32:21.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594269+0000", 2024-09-18T11:32:21.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5297999999999999e-05 2024-09-18T11:32:21.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594302+0000", 2024-09-18T11:32:21.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2703000000000002e-05 2024-09-18T11:32:21.370 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.370 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.370 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.371 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:168 3.1 3:b64d7bad:::7feb22256bdcec217720fd46184ab43ced8923ed66ec6319c57f078d9f123262:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:21.371 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:32:21.371 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 11.727161707, 2024-09-18T11:32:21.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011589809, 2024-09-18T11:32:21.371 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.371 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.371 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:32:21.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:32:21.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582817+0000", 2024-09-18T11:32:21.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:21.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582903+0000", 2024-09-18T11:32:21.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5915999999999995e-05 2024-09-18T11:32:21.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582913+0000", 2024-09-18T11:32:21.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1409999999999994e-06 2024-09-18T11:32:21.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582917+0000", 2024-09-18T11:32:21.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6419999999999998e-06 2024-09-18T11:32:21.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585540+0000", 2024-09-18T11:32:21.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026228940000000002 2024-09-18T11:32:21.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585721+0000", 2024-09-18T11:32:21.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018086299999999999 2024-09-18T11:32:21.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:21.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585918+0000", 2024-09-18T11:32:21.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000196758 2024-09-18T11:32:21.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:21.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590683+0000", 2024-09-18T11:32:21.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047657020000000001 2024-09-18T11:32:21.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:21.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594354+0000", 2024-09-18T11:32:21.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003671068 2024-09-18T11:32:21.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594386+0000", 2024-09-18T11:32:21.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1426e-05 2024-09-18T11:32:21.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594408+0000", 2024-09-18T11:32:21.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1987000000000001e-05 2024-09-18T11:32:21.376 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.376 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.376 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.376 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:172 3.1 3:8c3b343b:::d73444b0abf5ea0abb7f4bfb783ec9a4695bac9fb180180b88c2eef6a2e03b80:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:21.376 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.583414+0000", 2024-09-18T11:32:21.376 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 11.726565665000001, 2024-09-18T11:32:21.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011061609999999999, 2024-09-18T11:32:21.376 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.376 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.377 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583414+0000", 2024-09-18T11:32:21.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583414+0000", 2024-09-18T11:32:21.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583413+0000", 2024-09-18T11:32:21.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:21.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583525+0000", 2024-09-18T11:32:21.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000111289 2024-09-18T11:32:21.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583537+0000", 2024-09-18T11:32:21.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1755e-05 2024-09-18T11:32:21.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583542+0000", 2024-09-18T11:32:21.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4399999999999996e-06 2024-09-18T11:32:21.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.588499+0000", 2024-09-18T11:32:21.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004957347 2024-09-18T11:32:21.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.588684+0000", 2024-09-18T11:32:21.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000185002 2024-09-18T11:32:21.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:21.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.588889+0000", 2024-09-18T11:32:21.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020478200000000001 2024-09-18T11:32:21.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:21.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.591496+0000", 2024-09-18T11:32:21.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026070619999999998 2024-09-18T11:32:21.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:21.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594430+0000", 2024-09-18T11:32:21.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029342219999999998 2024-09-18T11:32:21.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594458+0000", 2024-09-18T11:32:21.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7243e-05 2024-09-18T11:32:21.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594476+0000", 2024-09-18T11:32:21.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8139e-05 2024-09-18T11:32:21.381 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.381 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.382 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.382 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:175 3.1 3:b5e86b88:::e55cf3d5cf782ed327a67e746b287b76d05f74f58a4f3726e399a564440ceaf6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:21.382 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:32:21.382 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 11.726312765999999, 2024-09-18T11:32:21.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013709627, 2024-09-18T11:32:21.382 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.382 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.382 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:32:21.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:32:21.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583666+0000", 2024-09-18T11:32:21.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:21.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584041+0000", 2024-09-18T11:32:21.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037511499999999998 2024-09-18T11:32:21.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584055+0000", 2024-09-18T11:32:21.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3295999999999999e-05 2024-09-18T11:32:21.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584061+0000", 2024-09-18T11:32:21.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1630000000000001e-06 2024-09-18T11:32:21.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.589980+0000", 2024-09-18T11:32:21.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005918998 2024-09-18T11:32:21.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590117+0000", 2024-09-18T11:32:21.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013715400000000001 2024-09-18T11:32:21.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:21.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590270+0000", 2024-09-18T11:32:21.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015245600000000001 2024-09-18T11:32:21.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:21.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596513+0000", 2024-09-18T11:32:21.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062438729999999996 2024-09-18T11:32:21.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:21.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597328+0000", 2024-09-18T11:32:21.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081489599999999996 2024-09-18T11:32:21.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597358+0000", 2024-09-18T11:32:21.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9291000000000001e-05 2024-09-18T11:32:21.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597377+0000", 2024-09-18T11:32:21.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:32:21.387 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.387 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.387 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.387 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4061 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:21.387 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:32:21.387 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.0131340770000001, 2024-09-18T11:32:21.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013340967, 2024-09-18T11:32:21.387 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.388 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:21.388 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.388 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.388 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.388 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4061 2024-09-18T11:32:21.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.388 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:32:21.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:32:21.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296844+0000", 2024-09-18T11:32:21.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:21.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296854+0000", 2024-09-18T11:32:21.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0037000000000001e-05 2024-09-18T11:32:21.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296873+0000", 2024-09-18T11:32:21.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:32:21.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296881+0000", 2024-09-18T11:32:21.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3790000000000006e-06 2024-09-18T11:32:21.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296975+0000", 2024-09-18T11:32:21.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3536999999999996e-05 2024-09-18T11:32:21.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:21.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296982+0000", 2024-09-18T11:32:21.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6399999999999997e-06 2024-09-18T11:32:21.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301933+0000", 2024-09-18T11:32:21.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049507190000000001 2024-09-18T11:32:21.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301984+0000", 2024-09-18T11:32:21.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1439e-05 2024-09-18T11:32:21.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:32:21.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082022050000000006 2024-09-18T11:32:21.392 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.392 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.392 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.393 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4062 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:21.393 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:21.393 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.0130855220000008, 2024-09-18T11:32:21.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016297147000000001, 2024-09-18T11:32:21.393 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.393 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:21.393 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.393 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.393 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.393 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4062 2024-09-18T11:32:21.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.394 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:21.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:21.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:21.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:21.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296898+0000", 2024-09-18T11:32:21.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9419999999999997e-06 2024-09-18T11:32:21.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296906+0000", 2024-09-18T11:32:21.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3520000000000007e-06 2024-09-18T11:32:21.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296910+0000", 2024-09-18T11:32:21.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4469999999999999e-06 2024-09-18T11:32:21.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297032+0000", 2024-09-18T11:32:21.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012168900000000001 2024-09-18T11:32:21.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:21.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297038+0000", 2024-09-18T11:32:21.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9320000000000001e-06 2024-09-18T11:32:21.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310221+0000", 2024-09-18T11:32:21.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013182887000000001 2024-09-18T11:32:21.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310323+0000", 2024-09-18T11:32:21.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010170000000000001 2024-09-18T11:32:21.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313191+0000", 2024-09-18T11:32:21.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002868808 2024-09-18T11:32:21.398 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.398 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.398 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.399 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4063 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:21.399 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:21.399 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.0130646300000006, 2024-09-18T11:32:21.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017648712, 2024-09-18T11:32:21.399 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.399 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:21.399 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.399 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.399 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.399 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4063 2024-09-18T11:32:21.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.400 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:21.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:21.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:21.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:21.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296918+0000", 2024-09-18T11:32:21.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8100000000000002e-06 2024-09-18T11:32:21.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296924+0000", 2024-09-18T11:32:21.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2519999999999996e-06 2024-09-18T11:32:21.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296928+0000", 2024-09-18T11:32:21.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0500000000000002e-06 2024-09-18T11:32:21.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297062+0000", 2024-09-18T11:32:21.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013425500000000001 2024-09-18T11:32:21.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:21.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297069+0000", 2024-09-18T11:32:21.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5239999999999997e-06 2024-09-18T11:32:21.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313424+0000", 2024-09-18T11:32:21.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016355277000000001 2024-09-18T11:32:21.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313514+0000", 2024-09-18T11:32:21.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0581999999999994e-05 2024-09-18T11:32:21.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.314564+0000", 2024-09-18T11:32:21.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001049384 2024-09-18T11:32:21.404 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.404 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.404 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4065 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:32:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.0130191889999995, 2024-09-18T11:32:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013225997, 2024-09-18T11:32:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4065 2024-09-18T11:32:21.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:32:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:32:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296960+0000", 2024-09-18T11:32:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296963+0000", 2024-09-18T11:32:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2270000000000002e-06 2024-09-18T11:32:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296970+0000", 2024-09-18T11:32:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6220000000000003e-06 2024-09-18T11:32:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296973+0000", 2024-09-18T11:32:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3730000000000001e-06 2024-09-18T11:32:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297097+0000", 2024-09-18T11:32:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012352399999999999 2024-09-18T11:32:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297103+0000", 2024-09-18T11:32:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7119999999999997e-06 2024-09-18T11:32:21.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307831+0000", 2024-09-18T11:32:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010728794 2024-09-18T11:32:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307924+0000", 2024-09-18T11:32:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2099e-05 2024-09-18T11:32:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:32:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022630469999999998 2024-09-18T11:32:21.409 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4066 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:32:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.0130021140000007, 2024-09-18T11:32:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015656389999999999, 2024-09-18T11:32:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4066 2024-09-18T11:32:21.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:32:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:32:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296977+0000", 2024-09-18T11:32:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296980+0000", 2024-09-18T11:32:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8830000000000002e-06 2024-09-18T11:32:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296985+0000", 2024-09-18T11:32:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:32:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296988+0000", 2024-09-18T11:32:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0419999999999998e-06 2024-09-18T11:32:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297120+0000", 2024-09-18T11:32:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131693 2024-09-18T11:32:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297125+0000", 2024-09-18T11:32:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1100000000000002e-06 2024-09-18T11:32:21.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310212+0000", 2024-09-18T11:32:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013087395999999999 2024-09-18T11:32:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310292+0000", 2024-09-18T11:32:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9980000000000003e-05 2024-09-18T11:32:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312634+0000", 2024-09-18T11:32:21.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002341756 2024-09-18T11:32:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4068 3.1 3:87e79ed0:::3526641c07d3efa989be9684355570f39c331008b8a02c9fa22b38a94b6c6b9f:head [write 0~1015808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:32:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.0092258970000003, 2024-09-18T11:32:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016392648999999999, 2024-09-18T11:32:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4068 2024-09-18T11:32:21.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:32:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:32:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300753+0000", 2024-09-18T11:32:21.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:21.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301922+0000", 2024-09-18T11:32:21.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001169432 2024-09-18T11:32:21.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301936+0000", 2024-09-18T11:32:21.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3574000000000001e-05 2024-09-18T11:32:21.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301941+0000", 2024-09-18T11:32:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6069999999999998e-06 2024-09-18T11:32:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310234+0000", 2024-09-18T11:32:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082924820000000003 2024-09-18T11:32:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310337+0000", 2024-09-18T11:32:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102673 2024-09-18T11:32:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310469+0000", 2024-09-18T11:32:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131982 2024-09-18T11:32:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312844+0000", 2024-09-18T11:32:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023758680000000002 2024-09-18T11:32:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317092+0000", 2024-09-18T11:32:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042477549999999998 2024-09-18T11:32:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317124+0000", 2024-09-18T11:32:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1968999999999998e-05 2024-09-18T11:32:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317147+0000", 2024-09-18T11:32:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2339000000000001e-05 2024-09-18T11:32:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.420 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.421 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.421 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:21.421 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T11:32:21.422 INFO:tasks.rados.rados.0.smithi119.stdout:1805: done (3 left) 2024-09-18T11:32:21.423 INFO:tasks.rados.rados.0.smithi119.stdout:1806: read oid 28 snap -1 2024-09-18T11:32:21.423 INFO:tasks.rados.rados.0.smithi119.stdout:1806: expect (ObjNum 336 snap 48 seq_num 336) 2024-09-18T11:32:21.423 INFO:tasks.rados.rados.0.smithi119.stdout:1807: read oid 10 snap -1 2024-09-18T11:32:21.423 INFO:tasks.rados.rados.0.smithi119.stdout:1807: expect (ObjNum 358 snap 55 seq_num 358) 2024-09-18T11:32:21.423 INFO:tasks.rados.rados.0.smithi119.stdout:1808: snap_remove snap 37 2024-09-18T11:32:21.424 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:32:21.424 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:21.337+0000 7fcb662ad640 -1 osd.7 149 handle_osd_map injecting map crc failure 2024-09-18T11:32:21.424 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:21.342+0000 7f3155e8c640 -1 osd.4 149 handle_osd_map injecting map crc failure 2024-09-18T11:32:21.424 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:21.421+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:21.523 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:21.521+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:21.563 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:21.564 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:21.564 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:21.564 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:21.572 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:32:21.595 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:21.594+0000 7fd27659f640 -1 osd.6 149 handle_osd_map injecting map crc failure 2024-09-18T11:32:21.624 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:21.623+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:21.670 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:21.670 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:21.670 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:21.670 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:21.670 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:21.678 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:32:21.762 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:21.761+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:21.818 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:21.818 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:21.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:21.818 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:21.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.818 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4266.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T11:32:21.818 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:32:21.818 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 151.78921988100001, 2024-09-18T11:32:21.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073601550000000002, 2024-09-18T11:32:21.818 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.819 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.819 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.819 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4266", 2024-09-18T11:32:21.819 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/1431924034", 2024-09-18T11:32:21.819 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1 2024-09-18T11:32:21.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.819 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:32:21.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026534+0000", 2024-09-18T11:32:21.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026524+0000", 2024-09-18T11:32:21.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999905 2024-09-18T11:32:21.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026541+0000", 2024-09-18T11:32:21.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7070000000000001e-05 2024-09-18T11:32:21.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026554+0000", 2024-09-18T11:32:21.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.288e-05 2024-09-18T11:32:21.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026564+0000", 2024-09-18T11:32:21.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0197000000000001e-05 2024-09-18T11:32:21.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026826+0000", 2024-09-18T11:32:21.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00026212400000000003 2024-09-18T11:32:21.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.822 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:32:21.822 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.026839+0000", 2024-09-18T11:32:21.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2831e-05 2024-09-18T11:32:21.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.822 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.822 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028605+0000", 2024-09-18T11:32:21.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001765794 2024-09-18T11:32:21.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.822 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.822 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028709+0000", 2024-09-18T11:32:21.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000104348 2024-09-18T11:32:21.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:32:21.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.028824+0000", 2024-09-18T11:32:21.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011527399999999999 2024-09-18T11:32:21.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:21.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.030319+0000", 2024-09-18T11:32:21.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001494439 2024-09-18T11:32:21.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:21.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033805+0000", 2024-09-18T11:32:21.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034857820000000002 2024-09-18T11:32:21.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033856+0000", 2024-09-18T11:32:21.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1687999999999997e-05 2024-09-18T11:32:21.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:50.033894+0000", 2024-09-18T11:32:21.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7481000000000002e-05 2024-09-18T11:32:21.824 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.825 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.825 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.825 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1118 3.2 e38/21 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head v 38'76, mlcod=38'74)", 2024-09-18T11:32:21.825 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:32:21.825 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 121.867809729, 2024-09-18T11:32:21.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006048688, 2024-09-18T11:32:21.825 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.825 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.825 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:32:21.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947944+0000", 2024-09-18T11:32:21.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.947941+0000", 2024-09-18T11:32:21.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:21.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949868+0000", 2024-09-18T11:32:21.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019267659999999999 2024-09-18T11:32:21.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949893+0000", 2024-09-18T11:32:21.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5239999999999999e-05 2024-09-18T11:32:21.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.827 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.827 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949901+0000", 2024-09-18T11:32:21.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8420000000000005e-06 2024-09-18T11:32:21.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949954+0000", 2024-09-18T11:32:21.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3464999999999997e-05 2024-09-18T11:32:21.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.949974+0000", 2024-09-18T11:32:21.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9731000000000001e-05 2024-09-18T11:32:21.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953948+0000", 2024-09-18T11:32:21.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039735489999999998 2024-09-18T11:32:21.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:19.953992+0000", 2024-09-18T11:32:21.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4700999999999998e-05 2024-09-18T11:32:21.829 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.829 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.829 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.829 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:269 4.7 e58/22 4:e73ba68f:::smithi11939506-1:13 v 58'72, mlcod=57'69)", 2024-09-18T11:32:21.829 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:32:21.830 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 102.55037279699999, 2024-09-18T11:32:21.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065512569999999996, 2024-09-18T11:32:21.830 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.830 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.830 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:32:21.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265381+0000", 2024-09-18T11:32:21.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265380+0000", 2024-09-18T11:32:21.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:21.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265388+0000", 2024-09-18T11:32:21.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.174e-06 2024-09-18T11:32:21.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265398+0000", 2024-09-18T11:32:21.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2599999999999994e-06 2024-09-18T11:32:21.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265402+0000", 2024-09-18T11:32:21.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0289999999999999e-06 2024-09-18T11:32:21.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265434+0000", 2024-09-18T11:32:21.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2511000000000001e-05 2024-09-18T11:32:21.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.265447+0000", 2024-09-18T11:32:21.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2947e-05 2024-09-18T11:32:21.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271892+0000", 2024-09-18T11:32:21.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064445930000000002 2024-09-18T11:32:21.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:39.271932+0000", 2024-09-18T11:32:21.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0303999999999999e-05 2024-09-18T11:32:21.833 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.834 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.834 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.834 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1788 4.8 4:1898b15d:::smithi11939506-25:head [write 453822~797061] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:32:21.834 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:21.834 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 98.899940213999997, 2024-09-18T11:32:21.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094305159999999999, 2024-09-18T11:32:21.834 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.834 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.834 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.834 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.834 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.834 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1788 2024-09-18T11:32:21.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.835 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:21.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:21.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915811+0000", 2024-09-18T11:32:21.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:21.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916608+0000", 2024-09-18T11:32:21.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079635000000000003 2024-09-18T11:32:21.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916624+0000", 2024-09-18T11:32:21.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6735000000000002e-05 2024-09-18T11:32:21.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916630+0000", 2024-09-18T11:32:21.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5709999999999997e-06 2024-09-18T11:32:21.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916674+0000", 2024-09-18T11:32:21.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4540999999999999e-05 2024-09-18T11:32:21.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916751+0000", 2024-09-18T11:32:21.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6695999999999999e-05 2024-09-18T11:32:21.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:21.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916872+0000", 2024-09-18T11:32:21.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120648 2024-09-18T11:32:21.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:21.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.919076+0000", 2024-09-18T11:32:21.838 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022046829999999998 2024-09-18T11:32:21.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.838 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:21.838 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925151+0000", 2024-09-18T11:32:21.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060740239999999999 2024-09-18T11:32:21.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925198+0000", 2024-09-18T11:32:21.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7896999999999999e-05 2024-09-18T11:32:21.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.839 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.839 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925244+0000", 2024-09-18T11:32:21.839 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5398999999999997e-05 2024-09-18T11:32:21.839 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.839 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.839 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.840 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1934 4.2 e65/22 4:425afe9c:::smithi11939506-29:head v 65'44, mlcod=60'40)", 2024-09-18T11:32:21.840 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:32:21.840 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 93.515761444000006, 2024-09-18T11:32:21.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064585119999999996, 2024-09-18T11:32:21.840 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.840 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.840 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.840 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.840 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:32:21.840 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299992+0000", 2024-09-18T11:32:21.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.299990+0000", 2024-09-18T11:32:21.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:21.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.841 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.841 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300789+0000", 2024-09-18T11:32:21.841 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079851499999999997 2024-09-18T11:32:21.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300806+0000", 2024-09-18T11:32:21.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7096999999999999e-05 2024-09-18T11:32:21.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300813+0000", 2024-09-18T11:32:21.842 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5079999999999999e-06 2024-09-18T11:32:21.842 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.842 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.842 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300870+0000", 2024-09-18T11:32:21.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7178e-05 2024-09-18T11:32:21.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.300897+0000", 2024-09-18T11:32:21.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6284000000000001e-05 2024-09-18T11:32:21.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306410+0000", 2024-09-18T11:32:21.843 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055135210000000004 2024-09-18T11:32:21.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.843 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.843 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:48.306451+0000", 2024-09-18T11:32:21.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0379000000000001e-05 2024-09-18T11:32:21.844 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.844 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.844 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.844 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2452 4.3 4:ca2b2309:::smithi11939506-44:head [read 0~3462242,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:32:21.844 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:32:21.844 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 75.778986087000007, 2024-09-18T11:32:21.844 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060906450000000004, 2024-09-18T11:32:21.844 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.844 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:21.844 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.845 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.845 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.845 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2452 2024-09-18T11:32:21.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.845 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:32:21.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.845 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.845 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036767+0000", 2024-09-18T11:32:21.845 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036765+0000", 2024-09-18T11:32:21.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:21.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036777+0000", 2024-09-18T11:32:21.846 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1422e-05 2024-09-18T11:32:21.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.846 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.846 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036793+0000", 2024-09-18T11:32:21.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6575999999999999e-05 2024-09-18T11:32:21.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036800+0000", 2024-09-18T11:32:21.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9940000000000003e-06 2024-09-18T11:32:21.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.847 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:32:21.847 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5847999999999999e-05 2024-09-18T11:32:21.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.847 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:21.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036853+0000", 2024-09-18T11:32:21.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0119999999999999e-06 2024-09-18T11:32:21.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038527+0000", 2024-09-18T11:32:21.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016732730000000001 2024-09-18T11:32:21.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.848 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.848 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.038580+0000", 2024-09-18T11:32:21.848 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3443999999999999e-05 2024-09-18T11:32:21.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.849 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.849 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042858+0000", 2024-09-18T11:32:21.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042781049999999999 2024-09-18T11:32:21.849 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.849 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.849 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.849 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2453 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:32:21.849 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:32:21.849 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 75.778943854999994, 2024-09-18T11:32:21.849 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063798429999999996, 2024-09-18T11:32:21.849 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.850 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:21.850 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.850 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.850 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.850 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2453 2024-09-18T11:32:21.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.850 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:32:21.850 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.850 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.850 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036810+0000", 2024-09-18T11:32:21.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036809+0000", 2024-09-18T11:32:21.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:21.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.851 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036812+0000", 2024-09-18T11:32:21.851 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8339999999999999e-06 2024-09-18T11:32:21.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.851 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036819+0000", 2024-09-18T11:32:21.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9890000000000003e-06 2024-09-18T11:32:21.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036823+0000", 2024-09-18T11:32:21.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8099999999999999e-06 2024-09-18T11:32:21.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.852 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.852 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036912+0000", 2024-09-18T11:32:21.852 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9083000000000001e-05 2024-09-18T11:32:21.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:21.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036932+0000", 2024-09-18T11:32:21.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0222000000000001e-05 2024-09-18T11:32:21.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042896+0000", 2024-09-18T11:32:21.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059637240000000001 2024-09-18T11:32:21.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.853 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.853 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.042952+0000", 2024-09-18T11:32:21.853 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5877000000000001e-05 2024-09-18T11:32:21.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.854 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.854 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043190+0000", 2024-09-18T11:32:21.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00023774500000000001 2024-09-18T11:32:21.854 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.854 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.854 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.854 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2454 4.3 4:ca2b2309:::smithi11939506-44:head [sparse-read 0~2524964] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-18T11:32:21.854 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:32:21.854 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 75.778907899000004, 2024-09-18T11:32:21.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066404469999999998, 2024-09-18T11:32:21.855 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.855 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:21.855 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.855 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.855 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.855 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2454 2024-09-18T11:32:21.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.855 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.855 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.855 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:32:21.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036846+0000", 2024-09-18T11:32:21.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036845+0000", 2024-09-18T11:32:21.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:21.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036856+0000", 2024-09-18T11:32:21.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1338000000000001e-05 2024-09-18T11:32:21.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036863+0000", 2024-09-18T11:32:21.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4749999999999998e-06 2024-09-18T11:32:21.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036867+0000", 2024-09-18T11:32:21.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3839999999999999e-06 2024-09-18T11:32:21.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036946+0000", 2024-09-18T11:32:21.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9054000000000006e-05 2024-09-18T11:32:21.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:21.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.036949+0000", 2024-09-18T11:32:21.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9390000000000002e-06 2024-09-18T11:32:21.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043213+0000", 2024-09-18T11:32:21.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062640090000000001 2024-09-18T11:32:21.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043261+0000", 2024-09-18T11:32:21.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7750999999999997e-05 2024-09-18T11:32:21.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:06.043486+0000", 2024-09-18T11:32:21.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000224985 2024-09-18T11:32:21.859 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.859 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.859 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.859 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2553 4.3 4:ca2b2309:::smithi11939506-44:head [write 564768~630013,stat] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-18T11:32:21.859 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:32:21.859 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 71.618630265999997, 2024-09-18T11:32:21.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065686879999999996, 2024-09-18T11:32:21.860 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.860 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.860 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.860 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.860 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.860 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2553 2024-09-18T11:32:21.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.860 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:32:21.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197123+0000", 2024-09-18T11:32:21.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.197120+0000", 2024-09-18T11:32:21.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:21.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.861 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.861 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199000+0000", 2024-09-18T11:32:21.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018792080000000001 2024-09-18T11:32:21.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199044+0000", 2024-09-18T11:32:21.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4048999999999998e-05 2024-09-18T11:32:21.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199071+0000", 2024-09-18T11:32:21.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7288e-05 2024-09-18T11:32:21.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199119+0000", 2024-09-18T11:32:21.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8000999999999997e-05 2024-09-18T11:32:21.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199208+0000", 2024-09-18T11:32:21.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9209000000000006e-05 2024-09-18T11:32:21.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:21.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.199385+0000", 2024-09-18T11:32:21.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000177095 2024-09-18T11:32:21.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:21.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.202282+0000", 2024-09-18T11:32:21.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028970469999999998 2024-09-18T11:32:21.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:21.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203578+0000", 2024-09-18T11:32:21.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012960490000000001 2024-09-18T11:32:21.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203638+0000", 2024-09-18T11:32:21.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9599000000000003e-05 2024-09-18T11:32:21.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:10.203692+0000", 2024-09-18T11:32:21.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3908e-05 2024-09-18T11:32:21.865 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.865 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.865 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.865 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2752 4.3 4:ca2b2309:::smithi11939506-44:25 [read 0~3210104,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-18T11:32:21.866 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:32:21.866 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 63.928861105999999, 2024-09-18T11:32:21.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068715460000000001, 2024-09-18T11:32:21.866 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.866 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:21.866 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.866 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.866 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.866 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2752 2024-09-18T11:32:21.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.866 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.866 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.866 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:32:21.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:32:21.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886892+0000", 2024-09-18T11:32:21.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:21.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886895+0000", 2024-09-18T11:32:21.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.993e-06 2024-09-18T11:32:21.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886901+0000", 2024-09-18T11:32:21.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1859999999999997e-06 2024-09-18T11:32:21.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.886904+0000", 2024-09-18T11:32:21.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1920000000000001e-06 2024-09-18T11:32:21.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.892622+0000", 2024-09-18T11:32:21.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057175660000000003 2024-09-18T11:32:21.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.892693+0000", 2024-09-18T11:32:21.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1265000000000003e-05 2024-09-18T11:32:21.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:17.893764+0000", 2024-09-18T11:32:21.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001070753 2024-09-18T11:32:21.870 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.870 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.870 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.870 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:2793 4.1 e92/22 4:87078dfe:::smithi11939506-14:head v 92'195, mlcod=92'191)", 2024-09-18T11:32:21.870 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:32:21.870 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 62.590961071999999, 2024-09-18T11:32:21.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062761989999999997, 2024-09-18T11:32:21.870 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.870 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.870 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:32:21.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:32:21.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224792+0000", 2024-09-18T11:32:21.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:21.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.224994+0000", 2024-09-18T11:32:21.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020197600000000001 2024-09-18T11:32:21.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225007+0000", 2024-09-18T11:32:21.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3791999999999999e-05 2024-09-18T11:32:21.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225013+0000", 2024-09-18T11:32:21.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3630000000000002e-06 2024-09-18T11:32:21.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225062+0000", 2024-09-18T11:32:21.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8925000000000003e-05 2024-09-18T11:32:21.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.873 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.225086+0000", 2024-09-18T11:32:21.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4737999999999999e-05 2024-09-18T11:32:21.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231022+0000", 2024-09-18T11:32:21.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059360690000000004 2024-09-18T11:32:21.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:19.231069+0000", 2024-09-18T11:32:21.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6252999999999999e-05 2024-09-18T11:32:21.874 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.874 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.874 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.875 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2876 4.3 4:cc46532a:::smithi11939506-33:2f [read 0~3680733,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:21.875 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:32:21.875 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 59.308892172, 2024-09-18T11:32:21.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0068391210000000001, 2024-09-18T11:32:21.875 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.875 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:21.875 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.875 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.875 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.875 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2876 2024-09-18T11:32:21.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.875 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:32:21.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506861+0000", 2024-09-18T11:32:21.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506860+0000", 2024-09-18T11:32:21.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:21.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506869+0000", 2024-09-18T11:32:21.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5539999999999998e-06 2024-09-18T11:32:21.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506880+0000", 2024-09-18T11:32:21.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1552999999999999e-05 2024-09-18T11:32:21.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506887+0000", 2024-09-18T11:32:21.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7329999999999997e-06 2024-09-18T11:32:21.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506937+0000", 2024-09-18T11:32:21.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0317e-05 2024-09-18T11:32:21.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:21.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506945+0000", 2024-09-18T11:32:21.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1550000000000007e-06 2024-09-18T11:32:21.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.509021+0000", 2024-09-18T11:32:21.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020759870000000001 2024-09-18T11:32:21.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.509123+0000", 2024-09-18T11:32:21.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010132900000000001 2024-09-18T11:32:21.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513700+0000", 2024-09-18T11:32:21.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0045777320000000002 2024-09-18T11:32:21.879 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.880 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.880 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.880 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2877 4.3 4:cc46532a:::smithi11939506-33:2f [read 0~3680733,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:21.880 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:21.880 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 59.308857758000002, 2024-09-18T11:32:21.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085715170000000007, 2024-09-18T11:32:21.880 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.880 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:21.880 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.880 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.880 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.881 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2877 2024-09-18T11:32:21.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.881 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:21.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:21.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:21.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:21.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506898+0000", 2024-09-18T11:32:21.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4760000000000001e-06 2024-09-18T11:32:21.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506903+0000", 2024-09-18T11:32:21.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:32:21.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506906+0000", 2024-09-18T11:32:21.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1769999999999998e-06 2024-09-18T11:32:21.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507007+0000", 2024-09-18T11:32:21.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101479 2024-09-18T11:32:21.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:21.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507015+0000", 2024-09-18T11:32:21.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8490000000000006e-06 2024-09-18T11:32:21.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513757+0000", 2024-09-18T11:32:21.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0067417800000000002 2024-09-18T11:32:21.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513867+0000", 2024-09-18T11:32:21.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000109987 2024-09-18T11:32:21.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515467+0000", 2024-09-18T11:32:21.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00160037 2024-09-18T11:32:21.885 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.885 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.885 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.885 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2878 4.3 4:cc46532a:::smithi11939506-33:2f [sparse-read 0~1770976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:21.885 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:21.885 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 59.308827303000001, 2024-09-18T11:32:21.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090602640000000002, 2024-09-18T11:32:21.885 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.886 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:21.886 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.886 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.886 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.886 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2878 2024-09-18T11:32:21.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.886 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:21.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:21.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:21.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:21.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506929+0000", 2024-09-18T11:32:21.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.163e-06 2024-09-18T11:32:21.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506936+0000", 2024-09-18T11:32:21.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5519999999999999e-06 2024-09-18T11:32:21.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506941+0000", 2024-09-18T11:32:21.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:32:21.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507036+0000", 2024-09-18T11:32:21.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5070999999999998e-05 2024-09-18T11:32:21.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:21.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507042+0000", 2024-09-18T11:32:21.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3959999999999997e-06 2024-09-18T11:32:21.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515504+0000", 2024-09-18T11:32:21.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084620300000000006 2024-09-18T11:32:21.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515625+0000", 2024-09-18T11:32:21.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012150999999999999 2024-09-18T11:32:21.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515986+0000", 2024-09-18T11:32:21.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036140100000000002 2024-09-18T11:32:21.890 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.890 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.890 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.891 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:686 4.7 e98/22 4:e5707c18:::smithi11939506-10:30 v 98'160, mlcod=97'155)", 2024-09-18T11:32:21.891 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:32:21.891 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.796985612999997, 2024-09-18T11:32:21.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074641239999999999, 2024-09-18T11:32:21.891 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.891 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.891 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:32:21.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018768+0000", 2024-09-18T11:32:21.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018767+0000", 2024-09-18T11:32:21.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:21.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018773+0000", 2024-09-18T11:32:21.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1460000000000004e-06 2024-09-18T11:32:21.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018781+0000", 2024-09-18T11:32:21.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9780000000000005e-06 2024-09-18T11:32:21.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018785+0000", 2024-09-18T11:32:21.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0029999999999996e-06 2024-09-18T11:32:21.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018826+0000", 2024-09-18T11:32:21.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1616999999999998e-05 2024-09-18T11:32:21.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.018843+0000", 2024-09-18T11:32:21.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6753999999999999e-05 2024-09-18T11:32:21.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026202+0000", 2024-09-18T11:32:21.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073592620000000001 2024-09-18T11:32:21.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026232+0000", 2024-09-18T11:32:21.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9779000000000001e-05 2024-09-18T11:32:21.895 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.895 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.895 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.895 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.4.0:621 4.1 e98/22 4:854aa293:::smithi11939506-26:35 v 98'215, mlcod=96'210)", 2024-09-18T11:32:21.895 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:32:21.895 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.796102173999998, 2024-09-18T11:32:21.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007014752, 2024-09-18T11:32:21.895 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.895 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.896 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:32:21.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:32:21.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019651+0000", 2024-09-18T11:32:21.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:21.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019659+0000", 2024-09-18T11:32:21.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8229999999999995e-06 2024-09-18T11:32:21.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019671+0000", 2024-09-18T11:32:21.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2636000000000001e-05 2024-09-18T11:32:21.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019677+0000", 2024-09-18T11:32:21.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5470000000000004e-06 2024-09-18T11:32:21.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019710+0000", 2024-09-18T11:32:21.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3191999999999997e-05 2024-09-18T11:32:21.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.019722+0000", 2024-09-18T11:32:21.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2357e-05 2024-09-18T11:32:21.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026639+0000", 2024-09-18T11:32:21.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0069167630000000003 2024-09-18T11:32:21.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026666+0000", 2024-09-18T11:32:21.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7087999999999999e-05 2024-09-18T11:32:21.899 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.899 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.899 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.900 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3775 4.3 4:c70c3162:::smithi11939506-35:head [write 796731~461096,stat] snapc 51=[51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b,30] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-18T11:32:21.900 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:58.217457+0000", 2024-09-18T11:32:21.900 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.598296651999998, 2024-09-18T11:32:21.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064394090000000001, 2024-09-18T11:32:21.900 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.900 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.900 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.900 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.900 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.900 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3775 2024-09-18T11:32:21.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.900 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217457+0000", 2024-09-18T11:32:21.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217457+0000", 2024-09-18T11:32:21.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217455+0000", 2024-09-18T11:32:21.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:21.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217965+0000", 2024-09-18T11:32:21.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00050991999999999997 2024-09-18T11:32:21.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217977+0000", 2024-09-18T11:32:21.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1701e-05 2024-09-18T11:32:21.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.217981+0000", 2024-09-18T11:32:21.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5600000000000004e-06 2024-09-18T11:32:21.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.218033+0000", 2024-09-18T11:32:21.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1728999999999999e-05 2024-09-18T11:32:21.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.218147+0000", 2024-09-18T11:32:21.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000114066 2024-09-18T11:32:21.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:21.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.218364+0000", 2024-09-18T11:32:21.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021718700000000001 2024-09-18T11:32:21.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:21.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.223702+0000", 2024-09-18T11:32:21.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005338036 2024-09-18T11:32:21.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:21.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.223812+0000", 2024-09-18T11:32:21.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000110011 2024-09-18T11:32:21.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.223868+0000", 2024-09-18T11:32:21.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5989999999999998e-05 2024-09-18T11:32:21.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:58.223896+0000", 2024-09-18T11:32:21.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7820000000000001e-05 2024-09-18T11:32:21.905 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.905 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.905 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.906 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3963 4.3 4:c70c3162:::smithi11939506-35:head [tier-promote] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:21.906 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:32:21.906 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.439930293, 2024-09-18T11:32:21.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010675591999999999, 2024-09-18T11:32:21.906 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.906 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:21.906 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:21.906 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:21.906 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:21.906 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3963 2024-09-18T11:32:21.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.906 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:32:21.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:32:21.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375821+0000", 2024-09-18T11:32:21.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:21.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375829+0000", 2024-09-18T11:32:21.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7989999999999994e-06 2024-09-18T11:32:21.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375843+0000", 2024-09-18T11:32:21.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:32:21.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375849+0000", 2024-09-18T11:32:21.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1419999999999998e-06 2024-09-18T11:32:21.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375889+0000", 2024-09-18T11:32:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0197999999999997e-05 2024-09-18T11:32:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375894+0000", 2024-09-18T11:32:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0139999999999998e-06 2024-09-18T11:32:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.377699+0000", 2024-09-18T11:32:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001804688 2024-09-18T11:32:21.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.377776+0000", 2024-09-18T11:32:21.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7081000000000003e-05 2024-09-18T11:32:21.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.386499+0000", 2024-09-18T11:32:21.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008722746 2024-09-18T11:32:21.910 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.910 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.910 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.911 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1025 4.7 e138/22 4:e73ba68f:::smithi11939506-1:head v 138'281, mlcod=137'278)", 2024-09-18T11:32:21.911 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.587274+0000", 2024-09-18T11:32:21.911 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.228479978999999, 2024-09-18T11:32:21.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094231360000000004, 2024-09-18T11:32:21.911 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.911 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.911 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.587274+0000", 2024-09-18T11:32:21.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.587274+0000", 2024-09-18T11:32:21.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.587273+0000", 2024-09-18T11:32:21.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:21.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590258+0000", 2024-09-18T11:32:21.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002984998 2024-09-18T11:32:21.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590273+0000", 2024-09-18T11:32:21.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4999e-05 2024-09-18T11:32:21.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590283+0000", 2024-09-18T11:32:21.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3619999999999995e-06 2024-09-18T11:32:21.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590333+0000", 2024-09-18T11:32:21.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9978999999999997e-05 2024-09-18T11:32:21.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590352+0000", 2024-09-18T11:32:21.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9477999999999999e-05 2024-09-18T11:32:21.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596664+0000", 2024-09-18T11:32:21.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063120679999999997 2024-09-18T11:32:21.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596697+0000", 2024-09-18T11:32:21.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2456999999999998e-05 2024-09-18T11:32:21.915 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.915 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.915 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.915 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.4.0:892 4.1 e147/22 4:87078dfe:::smithi11939506-14:5a v 147'306, mlcod=145'300)", 2024-09-18T11:32:21.915 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:18.329208+0000", 2024-09-18T11:32:21.915 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.4865456890000002, 2024-09-18T11:32:21.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061449249999999999, 2024-09-18T11:32:21.915 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:21.916 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:21.916 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:21.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:21.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.329208+0000", 2024-09-18T11:32:21.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:21.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.329208+0000", 2024-09-18T11:32:21.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:21.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:21.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.329207+0000", 2024-09-18T11:32:21.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:21.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:21.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.329217+0000", 2024-09-18T11:32:21.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5670000000000001e-06 2024-09-18T11:32:21.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:21.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.329228+0000", 2024-09-18T11:32:21.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1202999999999999e-05 2024-09-18T11:32:21.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:21.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.329235+0000", 2024-09-18T11:32:21.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8229999999999997e-06 2024-09-18T11:32:21.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:21.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.329559+0000", 2024-09-18T11:32:21.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000324798 2024-09-18T11:32:21.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:21.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.329579+0000", 2024-09-18T11:32:21.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9468999999999999e-05 2024-09-18T11:32:21.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:21.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.335310+0000", 2024-09-18T11:32:21.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057313629999999997 2024-09-18T11:32:21.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:21.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:21.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:21.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.335353+0000", 2024-09-18T11:32:21.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2515999999999997e-05 2024-09-18T11:32:21.919 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.919 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.919 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.919 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:21.919 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:21.919 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:21.921 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:32:21.921 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:21.920+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:22.022 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:22.020+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:22.059 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:22.059 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:22.059 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:22.059 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:22.068 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:32:22.123 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:22.121+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:22.166 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:22.167 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:22.167 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:22.167 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:22.167 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:22.175 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:32:22.223 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:22.221+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:22.275 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:22.276 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:22.276 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:22.276 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:22.276 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.276 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:32:22.277 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:22.277 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 131.155925857, 2024-09-18T11:32:22.277 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:32:22.277 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.277 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:22.278 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.278 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:22.278 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:22.278 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:32:22.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.278 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.279 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.279 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.279 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:22.279 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.279 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.280 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:22.280 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.281 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.281 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:32:22.281 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:22.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.281 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:32:22.282 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:32:22.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.282 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.282 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:32:22.283 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:32:22.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.283 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:32:22.283 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:32:22.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.284 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.284 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.284 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:32:22.284 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:32:22.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.284 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.284 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:22.284 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:32:22.285 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:32:22.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.285 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.285 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:32:22.285 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:32:22.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.286 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:32:22.286 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:32:22.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.286 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.286 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:32:22.286 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:32:22.286 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.286 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.286 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.287 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:32:22.287 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:22.287 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 121.602900456, 2024-09-18T11:32:22.287 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:32:22.287 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.287 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:22.287 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.287 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:22.287 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:22.287 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:32:22.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.287 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.287 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.288 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:22.288 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.288 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.288 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.288 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:22.288 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.288 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.288 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.288 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:32:22.288 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:22.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.289 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.289 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.289 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:32:22.289 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:32:22.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.289 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.289 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.289 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:32:22.289 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:32:22.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.289 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.289 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.290 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:32:22.290 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:32:22.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.290 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.290 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.290 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:32:22.290 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:32:22.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.290 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.290 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.290 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:32:22.291 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:32:22.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.291 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.291 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:32:22.291 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:32:22.291 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.291 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.291 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.291 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2207 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1410296~761389] snapc 24=[24,1e,1c,1b,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T11:32:22.291 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:32:22.291 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 84.849747793000006, 2024-09-18T11:32:22.292 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008410743, 2024-09-18T11:32:22.292 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.292 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.292 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.292 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:22.292 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:22.292 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2207 2024-09-18T11:32:22.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.292 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.292 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.292 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:32:22.292 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.293 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.293 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:32:22.293 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.293 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.293 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422496+0000", 2024-09-18T11:32:22.293 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:22.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.293 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423190+0000", 2024-09-18T11:32:22.294 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00069405499999999995 2024-09-18T11:32:22.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.294 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423204+0000", 2024-09-18T11:32:22.294 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:32:22.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.294 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423208+0000", 2024-09-18T11:32:22.294 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:32:22.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.295 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423293+0000", 2024-09-18T11:32:22.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4524999999999995e-05 2024-09-18T11:32:22.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.295 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423394+0000", 2024-09-18T11:32:22.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101395 2024-09-18T11:32:22.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.296 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:22.296 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423579+0000", 2024-09-18T11:32:22.296 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018461000000000001 2024-09-18T11:32:22.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.296 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.296 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.428574+0000", 2024-09-18T11:32:22.296 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049944960000000002 2024-09-18T11:32:22.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.297 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430809+0000", 2024-09-18T11:32:22.297 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022351820000000001 2024-09-18T11:32:22.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.297 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430862+0000", 2024-09-18T11:32:22.298 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3189999999999997e-05 2024-09-18T11:32:22.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.298 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.298 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430908+0000", 2024-09-18T11:32:22.298 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5973999999999997e-05 2024-09-18T11:32:22.298 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.298 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.298 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.299 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2339 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:22.299 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:22.299 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 80.885443163999994, 2024-09-18T11:32:22.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011809992, 2024-09-18T11:32:22.299 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.299 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:22.299 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.299 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:22.299 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:22.299 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2339 2024-09-18T11:32:22.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.299 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:22.300 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:22.300 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:32:22.301 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:22.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.301 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:32:22.301 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4710000000000001e-06 2024-09-18T11:32:22.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.301 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386824+0000", 2024-09-18T11:32:22.301 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6104e-05 2024-09-18T11:32:22.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.301 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.302 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386836+0000", 2024-09-18T11:32:22.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1827999999999999e-05 2024-09-18T11:32:22.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.302 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.302 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388390+0000", 2024-09-18T11:32:22.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015539499999999999 2024-09-18T11:32:22.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.302 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:22.302 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388443+0000", 2024-09-18T11:32:22.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3785000000000001e-05 2024-09-18T11:32:22.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.303 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398238+0000", 2024-09-18T11:32:22.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097947989999999999 2024-09-18T11:32:22.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.303 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398315+0000", 2024-09-18T11:32:22.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6909999999999994e-05 2024-09-18T11:32:22.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.303 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398612+0000", 2024-09-18T11:32:22.304 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029671999999999999 2024-09-18T11:32:22.304 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.304 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.304 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.304 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2340 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:22.305 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:22.305 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 80.885399852000006, 2024-09-18T11:32:22.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012086448, 2024-09-18T11:32:22.305 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.305 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:22.305 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.305 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:22.305 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:22.305 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2340 2024-09-18T11:32:22.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.305 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:22.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:22.308 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.308 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.308 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.308 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:22.308 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:22.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.308 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.308 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386848+0000", 2024-09-18T11:32:22.309 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3069999999999998e-06 2024-09-18T11:32:22.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.309 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386855+0000", 2024-09-18T11:32:22.309 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0979999999999996e-06 2024-09-18T11:32:22.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386859+0000", 2024-09-18T11:32:22.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9360000000000001e-06 2024-09-18T11:32:22.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388466+0000", 2024-09-18T11:32:22.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001607154 2024-09-18T11:32:22.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:22.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388507+0000", 2024-09-18T11:32:22.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1093000000000001e-05 2024-09-18T11:32:22.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398647+0000", 2024-09-18T11:32:22.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010139686 2024-09-18T11:32:22.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398718+0000", 2024-09-18T11:32:22.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1138999999999998e-05 2024-09-18T11:32:22.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.312 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398931+0000", 2024-09-18T11:32:22.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021356899999999999 2024-09-18T11:32:22.312 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.312 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.312 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.312 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2341 4.5 4:aa575018:::smithi11939506-11:head [read 0~2487436,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:22.313 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:22.313 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 80.885381629999998, 2024-09-18T11:32:22.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013751648999999999, 2024-09-18T11:32:22.313 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.313 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:22.313 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.313 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:22.313 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:22.313 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2341 2024-09-18T11:32:22.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.313 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.314 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.314 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:22.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.314 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.314 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:22.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.314 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:22.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:22.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386866+0000", 2024-09-18T11:32:22.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4249999999999998e-06 2024-09-18T11:32:22.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386872+0000", 2024-09-18T11:32:22.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.003e-06 2024-09-18T11:32:22.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.316 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:32:22.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.507e-06 2024-09-18T11:32:22.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.316 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388520+0000", 2024-09-18T11:32:22.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001644146 2024-09-18T11:32:22.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:22.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388561+0000", 2024-09-18T11:32:22.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1486999999999998e-05 2024-09-18T11:32:22.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398963+0000", 2024-09-18T11:32:22.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010401353 2024-09-18T11:32:22.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.399046+0000", 2024-09-18T11:32:22.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3141999999999998e-05 2024-09-18T11:32:22.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.400615+0000", 2024-09-18T11:32:22.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015690960000000001 2024-09-18T11:32:22.318 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.318 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.319 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.319 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2529 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:22.319 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:32:22.319 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 73.098908366000003, 2024-09-18T11:32:22.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079268120000000001, 2024-09-18T11:32:22.319 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.319 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.319 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.320 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:22.320 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:22.320 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2529 2024-09-18T11:32:22.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.320 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:32:22.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:32:22.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173336+0000", 2024-09-18T11:32:22.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:22.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173339+0000", 2024-09-18T11:32:22.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9019999999999999e-06 2024-09-18T11:32:22.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173381+0000", 2024-09-18T11:32:22.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1836999999999999e-05 2024-09-18T11:32:22.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173390+0000", 2024-09-18T11:32:22.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.302e-06 2024-09-18T11:32:22.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174507+0000", 2024-09-18T11:32:22.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001116868 2024-09-18T11:32:22.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174603+0000", 2024-09-18T11:32:22.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5598000000000002e-05 2024-09-18T11:32:22.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:22.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174759+0000", 2024-09-18T11:32:22.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015616299999999999 2024-09-18T11:32:22.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178943+0000", 2024-09-18T11:32:22.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041846180000000002 2024-09-18T11:32:22.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181175+0000", 2024-09-18T11:32:22.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00223215 2024-09-18T11:32:22.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181229+0000", 2024-09-18T11:32:22.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3585999999999998e-05 2024-09-18T11:32:22.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181263+0000", 2024-09-18T11:32:22.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4292999999999998e-05 2024-09-18T11:32:22.325 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.325 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.325 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.325 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2530 4.5 4:af4ba75a:::smithi11939506-37:head [rollback 25] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:22.325 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:32:22.325 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 73.09884606, 2024-09-18T11:32:22.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080198800000000001, 2024-09-18T11:32:22.326 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.326 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.326 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.326 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:22.326 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:22.326 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2530 2024-09-18T11:32:22.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.326 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:32:22.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:32:22.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173398+0000", 2024-09-18T11:32:22.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:22.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.328 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173403+0000", 2024-09-18T11:32:22.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4389999999999999e-06 2024-09-18T11:32:22.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.328 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173414+0000", 2024-09-18T11:32:22.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0411000000000001e-05 2024-09-18T11:32:22.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173420+0000", 2024-09-18T11:32:22.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9610000000000002e-06 2024-09-18T11:32:22.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176390+0000", 2024-09-18T11:32:22.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029706289999999998 2024-09-18T11:32:22.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176501+0000", 2024-09-18T11:32:22.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011090300000000001 2024-09-18T11:32:22.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:22.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176827+0000", 2024-09-18T11:32:22.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00032526899999999999 2024-09-18T11:32:22.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178958+0000", 2024-09-18T11:32:22.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021314860000000001 2024-09-18T11:32:22.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181318+0000", 2024-09-18T11:32:22.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002359571 2024-09-18T11:32:22.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181372+0000", 2024-09-18T11:32:22.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3884000000000001e-05 2024-09-18T11:32:22.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181419+0000", 2024-09-18T11:32:22.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7148000000000003e-05 2024-09-18T11:32:22.332 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.332 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.332 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.333 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2531 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:22.333 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:22.333 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 73.098819937000002, 2024-09-18T11:32:22.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080687239999999993, 2024-09-18T11:32:22.333 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.333 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.333 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.333 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:22.333 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:22.333 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2531 2024-09-18T11:32:22.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.333 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:22.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:22.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:22.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:22.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173429+0000", 2024-09-18T11:32:22.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2989999999999998e-06 2024-09-18T11:32:22.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173436+0000", 2024-09-18T11:32:22.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1799999999999999e-06 2024-09-18T11:32:22.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173441+0000", 2024-09-18T11:32:22.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8810000000000004e-06 2024-09-18T11:32:22.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178267+0000", 2024-09-18T11:32:22.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048257969999999997 2024-09-18T11:32:22.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178388+0000", 2024-09-18T11:32:22.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121779 2024-09-18T11:32:22.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:22.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178599+0000", 2024-09-18T11:32:22.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000210094 2024-09-18T11:32:22.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.179339+0000", 2024-09-18T11:32:22.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00074063799999999995 2024-09-18T11:32:22.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181444+0000", 2024-09-18T11:32:22.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021044100000000001 2024-09-18T11:32:22.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181470+0000", 2024-09-18T11:32:22.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6415999999999999e-05 2024-09-18T11:32:22.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181494+0000", 2024-09-18T11:32:22.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3672e-05 2024-09-18T11:32:22.338 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.339 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.339 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.339 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2910 4.5 4:af4ba75a:::smithi11939506-37:head [write 786627~795991,stat] snapc 37=[37,35,34,30,2f,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:22.339 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:32:22.339 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 59.268762107999997, 2024-09-18T11:32:22.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084093710000000006, 2024-09-18T11:32:22.339 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.339 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.339 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.339 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:22.339 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:22.339 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2910 2024-09-18T11:32:22.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.340 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:32:22.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:32:22.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003481+0000", 2024-09-18T11:32:22.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:22.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004300+0000", 2024-09-18T11:32:22.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081934200000000003 2024-09-18T11:32:22.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004324+0000", 2024-09-18T11:32:22.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4618999999999999e-05 2024-09-18T11:32:22.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004335+0000", 2024-09-18T11:32:22.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.092e-05 2024-09-18T11:32:22.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004385+0000", 2024-09-18T11:32:22.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9709999999999997e-05 2024-09-18T11:32:22.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004470+0000", 2024-09-18T11:32:22.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4864000000000006e-05 2024-09-18T11:32:22.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:22.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004612+0000", 2024-09-18T11:32:22.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014241500000000001 2024-09-18T11:32:22.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011193+0000", 2024-09-18T11:32:22.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006580654 2024-09-18T11:32:22.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011788+0000", 2024-09-18T11:32:22.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059533600000000002 2024-09-18T11:32:22.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011842+0000", 2024-09-18T11:32:22.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3180000000000002e-05 2024-09-18T11:32:22.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011892+0000", 2024-09-18T11:32:22.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0618000000000002e-05 2024-09-18T11:32:22.345 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.345 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.346 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.346 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2953 4.a 4:588debc2:::smithi11939506-18:head [assert-version v113,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:22.346 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:32:22.346 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 58.24544659, 2024-09-18T11:32:22.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083931279999999997, 2024-09-18T11:32:22.347 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.347 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.347 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.347 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:22.347 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:22.347 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2953 2024-09-18T11:32:22.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.347 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:32:22.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:32:22.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026796+0000", 2024-09-18T11:32:22.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:22.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026806+0000", 2024-09-18T11:32:22.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9529999999999994e-06 2024-09-18T11:32:22.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026824+0000", 2024-09-18T11:32:22.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7901999999999999e-05 2024-09-18T11:32:22.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026832+0000", 2024-09-18T11:32:22.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8450000000000003e-06 2024-09-18T11:32:22.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026883+0000", 2024-09-18T11:32:22.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0883000000000001e-05 2024-09-18T11:32:22.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026980+0000", 2024-09-18T11:32:22.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7526000000000001e-05 2024-09-18T11:32:22.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:32:22.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032722+0000", 2024-09-18T11:32:22.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057414689999999999 2024-09-18T11:32:22.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.034230+0000", 2024-09-18T11:32:22.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001508035 2024-09-18T11:32:22.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035090+0000", 2024-09-18T11:32:22.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00086047300000000005 2024-09-18T11:32:22.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035147+0000", 2024-09-18T11:32:22.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6598000000000003e-05 2024-09-18T11:32:22.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035191+0000", 2024-09-18T11:32:22.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4747999999999999e-05 2024-09-18T11:32:22.353 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.353 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.353 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.353 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3105 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 3e=[3e,3d,3b,37,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-18T11:32:22.353 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:22.353 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 49.254033886000002, 2024-09-18T11:32:22.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011735906000000001, 2024-09-18T11:32:22.353 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.353 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:22.353 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.354 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:22.354 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:22.354 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3105 2024-09-18T11:32:22.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.354 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:22.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:22.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:22.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:22.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018212+0000", 2024-09-18T11:32:22.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6759999999999999e-06 2024-09-18T11:32:22.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018216+0000", 2024-09-18T11:32:22.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-06 2024-09-18T11:32:22.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018218+0000", 2024-09-18T11:32:22.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6110000000000001e-06 2024-09-18T11:32:22.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018246+0000", 2024-09-18T11:32:22.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7557000000000001e-05 2024-09-18T11:32:22.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:22.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018250+0000", 2024-09-18T11:32:22.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4299999999999999e-06 2024-09-18T11:32:22.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021364+0000", 2024-09-18T11:32:22.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031135389999999998 2024-09-18T11:32:22.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021494+0000", 2024-09-18T11:32:22.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000130333 2024-09-18T11:32:22.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.029947+0000", 2024-09-18T11:32:22.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084526150000000001 2024-09-18T11:32:22.358 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.358 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.358 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.359 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3323 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:22.359 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:32:22.359 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.155345842000003, 2024-09-18T11:32:22.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0088051689999999998, 2024-09-18T11:32:22.359 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.359 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.359 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.359 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:22.359 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:22.359 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3323 2024-09-18T11:32:22.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.359 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:32:22.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:32:22.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116897+0000", 2024-09-18T11:32:22.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:22.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116908+0000", 2024-09-18T11:32:22.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1511e-05 2024-09-18T11:32:22.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116925+0000", 2024-09-18T11:32:22.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7028e-05 2024-09-18T11:32:22.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116932+0000", 2024-09-18T11:32:22.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5919999999999997e-06 2024-09-18T11:32:22.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116988+0000", 2024-09-18T11:32:22.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5928999999999999e-05 2024-09-18T11:32:22.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:22.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116995+0000", 2024-09-18T11:32:22.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7409999999999992e-06 2024-09-18T11:32:22.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122390+0000", 2024-09-18T11:32:22.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053941889999999998 2024-09-18T11:32:22.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122453+0000", 2024-09-18T11:32:22.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3833999999999999e-05 2024-09-18T11:32:22.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:22.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122581+0000", 2024-09-18T11:32:22.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012751200000000001 2024-09-18T11:32:22.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125034+0000", 2024-09-18T11:32:22.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024525609999999998 2024-09-18T11:32:22.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125607+0000", 2024-09-18T11:32:22.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00057356499999999995 2024-09-18T11:32:22.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125655+0000", 2024-09-18T11:32:22.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8179000000000001e-05 2024-09-18T11:32:22.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125704+0000", 2024-09-18T11:32:22.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8950999999999999e-05 2024-09-18T11:32:22.365 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.365 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.365 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.365 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3324 4.5 4:a087a566:::smithi11939506-46:head [rollback 46] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:22.365 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:22.366 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.155303283000002, 2024-09-18T11:32:22.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010927767999999999, 2024-09-18T11:32:22.366 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.366 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.366 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.366 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:22.366 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:22.366 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3324 2024-09-18T11:32:22.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.366 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:22.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:22.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116941+0000", 2024-09-18T11:32:22.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:22.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116949+0000", 2024-09-18T11:32:22.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4660000000000002e-06 2024-09-18T11:32:22.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116960+0000", 2024-09-18T11:32:22.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1484999999999999e-05 2024-09-18T11:32:22.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116964+0000", 2024-09-18T11:32:22.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2810000000000002e-06 2024-09-18T11:32:22.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117053+0000", 2024-09-18T11:32:22.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8747999999999998e-05 2024-09-18T11:32:22.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:22.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117059+0000", 2024-09-18T11:32:22.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.246e-06 2024-09-18T11:32:22.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123604+0000", 2024-09-18T11:32:22.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065445319999999996 2024-09-18T11:32:22.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123663+0000", 2024-09-18T11:32:22.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8740999999999999e-05 2024-09-18T11:32:22.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:22.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123803+0000", 2024-09-18T11:32:22.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014049200000000001 2024-09-18T11:32:22.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127600+0000", 2024-09-18T11:32:22.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037971179999999999 2024-09-18T11:32:22.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127814+0000", 2024-09-18T11:32:22.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021333600000000001 2024-09-18T11:32:22.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127847+0000", 2024-09-18T11:32:22.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3188999999999997e-05 2024-09-18T11:32:22.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127869+0000", 2024-09-18T11:32:22.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2515e-05 2024-09-18T11:32:22.372 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.372 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.372 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.373 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3325 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:22.373 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:22.373 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.155256096999999, 2024-09-18T11:32:22.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181030999999999, 2024-09-18T11:32:22.373 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.373 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.373 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.373 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:22.373 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:22.373 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3325 2024-09-18T11:32:22.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.373 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:22.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:22.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116988+0000", 2024-09-18T11:32:22.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:22.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116991+0000", 2024-09-18T11:32:22.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9890000000000001e-06 2024-09-18T11:32:22.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116998+0000", 2024-09-18T11:32:22.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5309999999999998e-06 2024-09-18T11:32:22.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117005+0000", 2024-09-18T11:32:22.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6420000000000001e-06 2024-09-18T11:32:22.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117091+0000", 2024-09-18T11:32:22.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6835999999999993e-05 2024-09-18T11:32:22.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:22.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117097+0000", 2024-09-18T11:32:22.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:32:22.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124717+0000", 2024-09-18T11:32:22.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076204899999999997 2024-09-18T11:32:22.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124773+0000", 2024-09-18T11:32:22.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5470999999999998e-05 2024-09-18T11:32:22.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124877+0000", 2024-09-18T11:32:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010426500000000001 2024-09-18T11:32:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127901+0000", 2024-09-18T11:32:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030243449999999999 2024-09-18T11:32:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128115+0000", 2024-09-18T11:32:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021355300000000001 2024-09-18T11:32:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128145+0000", 2024-09-18T11:32:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0290999999999998e-05 2024-09-18T11:32:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128170+0000", 2024-09-18T11:32:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4811e-05 2024-09-18T11:32:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3474 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1872245~637761] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.828479420000001, 2024-09-18T11:32:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0092038580000000005, 2024-09-18T11:32:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3474 2024-09-18T11:32:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443764+0000", 2024-09-18T11:32:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444359+0000", 2024-09-18T11:32:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059519 2024-09-18T11:32:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444374+0000", 2024-09-18T11:32:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4949e-05 2024-09-18T11:32:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444382+0000", 2024-09-18T11:32:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1369999999999996e-06 2024-09-18T11:32:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444428+0000", 2024-09-18T11:32:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6783000000000003e-05 2024-09-18T11:32:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444507+0000", 2024-09-18T11:32:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8415000000000001e-05 2024-09-18T11:32:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444638+0000", 2024-09-18T11:32:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013129 2024-09-18T11:32:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.449059+0000", 2024-09-18T11:32:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004420719 2024-09-18T11:32:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452909+0000", 2024-09-18T11:32:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003850773 2024-09-18T11:32:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452944+0000", 2024-09-18T11:32:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-05 2024-09-18T11:32:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452969+0000", 2024-09-18T11:32:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5795999999999998e-05 2024-09-18T11:32:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3475 4.5 4:a56ea1fb:::smithi11939506-48:head [setxattr _header (57) in=7b,truncate 3042100] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:32:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.827850705000003, 2024-09-18T11:32:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086700660000000006, 2024-09-18T11:32:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3475 2024-09-18T11:32:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:32:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:32:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444393+0000", 2024-09-18T11:32:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444398+0000", 2024-09-18T11:32:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4909999999999999e-06 2024-09-18T11:32:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444405+0000", 2024-09-18T11:32:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5329999999999997e-06 2024-09-18T11:32:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444410+0000", 2024-09-18T11:32:22.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2470000000000002e-06 2024-09-18T11:32:22.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446046+0000", 2024-09-18T11:32:22.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001636722 2024-09-18T11:32:22.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446121+0000", 2024-09-18T11:32:22.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4604000000000006e-05 2024-09-18T11:32:22.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:22.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446274+0000", 2024-09-18T11:32:22.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015350300000000001 2024-09-18T11:32:22.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452884+0000", 2024-09-18T11:32:22.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066096439999999996 2024-09-18T11:32:22.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453000+0000", 2024-09-18T11:32:22.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000116204 2024-09-18T11:32:22.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453032+0000", 2024-09-18T11:32:22.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1327e-05 2024-09-18T11:32:22.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453064+0000", 2024-09-18T11:32:22.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2645000000000003e-05 2024-09-18T11:32:22.392 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.392 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.392 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.392 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3476 4.5 4:a56ea1fb:::smithi11939506-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:22.392 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:22.392 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.827811437999998, 2024-09-18T11:32:22.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087596800000000006, 2024-09-18T11:32:22.392 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.392 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:22.392 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.392 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:22.393 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:22.393 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3476 2024-09-18T11:32:22.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.393 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:22.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:22.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:22.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:22.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444436+0000", 2024-09-18T11:32:22.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9330000000000001e-06 2024-09-18T11:32:22.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444443+0000", 2024-09-18T11:32:22.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9369999999999998e-06 2024-09-18T11:32:22.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444447+0000", 2024-09-18T11:32:22.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1309999999999999e-06 2024-09-18T11:32:22.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446599+0000", 2024-09-18T11:32:22.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021521190000000001 2024-09-18T11:32:22.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:22.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446675+0000", 2024-09-18T11:32:22.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6099000000000004e-05 2024-09-18T11:32:22.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453088+0000", 2024-09-18T11:32:22.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006413307 2024-09-18T11:32:22.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453132+0000", 2024-09-18T11:32:22.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3548999999999999e-05 2024-09-18T11:32:22.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453193+0000", 2024-09-18T11:32:22.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1116000000000002e-05 2024-09-18T11:32:22.397 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.398 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.398 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.398 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3928 4.5 4:af4ba75a:::smithi11939506-37:head [sparse-read 0~1543958] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:22.398 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:22.398 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.253014528000001, 2024-09-18T11:32:22.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080807250000000004, 2024-09-18T11:32:22.398 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.398 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:22.398 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.398 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:22.398 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:22.398 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3928 2024-09-18T11:32:22.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.399 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:22.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:22.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:22.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:22.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019232+0000", 2024-09-18T11:32:22.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1050000000000002e-06 2024-09-18T11:32:22.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019236+0000", 2024-09-18T11:32:22.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7910000000000002e-06 2024-09-18T11:32:22.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019238+0000", 2024-09-18T11:32:22.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4339999999999999e-06 2024-09-18T11:32:22.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019326+0000", 2024-09-18T11:32:22.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8097999999999996e-05 2024-09-18T11:32:22.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:22.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019330+0000", 2024-09-18T11:32:22.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9349999999999996e-06 2024-09-18T11:32:22.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027035+0000", 2024-09-18T11:32:22.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0077047269999999998 2024-09-18T11:32:22.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027110+0000", 2024-09-18T11:32:22.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4443000000000005e-05 2024-09-18T11:32:22.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027311+0000", 2024-09-18T11:32:22.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000201528 2024-09-18T11:32:22.403 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.404 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.404 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.404 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3929 4.5 4:af4ba75a:::smithi11939506-37:head [sparse-read 0~1543958] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:22.404 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:22.404 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.252983974999999, 2024-09-18T11:32:22.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084586699999999997, 2024-09-18T11:32:22.404 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.404 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:22.404 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.404 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:22.404 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:22.405 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3929 2024-09-18T11:32:22.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.405 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:22.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:22.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:22.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:22.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019262+0000", 2024-09-18T11:32:22.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8029999999999999e-06 2024-09-18T11:32:22.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019267+0000", 2024-09-18T11:32:22.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1119999999999998e-06 2024-09-18T11:32:22.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019269+0000", 2024-09-18T11:32:22.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4269999999999998e-06 2024-09-18T11:32:22.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019344+0000", 2024-09-18T11:32:22.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4979000000000002e-05 2024-09-18T11:32:22.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:22.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019348+0000", 2024-09-18T11:32:22.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3930000000000001e-06 2024-09-18T11:32:22.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027385+0000", 2024-09-18T11:32:22.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080368409999999994 2024-09-18T11:32:22.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027463+0000", 2024-09-18T11:32:22.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7316000000000003e-05 2024-09-18T11:32:22.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027720+0000", 2024-09-18T11:32:22.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000257068 2024-09-18T11:32:22.410 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.410 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.410 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.410 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.410 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.410 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:22.411 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is set 2024-09-18T11:32:22.412 INFO:tasks.rados.rados.0.smithi119.stdout:1808: done (5 left) 2024-09-18T11:32:22.412 INFO:tasks.rados.rados.0.smithi119.stdout:1809: tier_evict oid 48 2024-09-18T11:32:22.412 INFO:tasks.rados.rados.0.smithi119.stdout:1809: tier_evict oid 48 snap -1 2024-09-18T11:32:22.412 INFO:tasks.rados.rados.0.smithi119.stdout:1802: expect (ObjNum 287 snap 41 seq_num 287) 2024-09-18T11:32:22.413 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:32:22.414 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:22.409+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:22.514 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:22.512+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:22.674 INFO:tasks.rados.rados.0.smithi119.stdout:1803: expect (ObjNum 254 snap 30 seq_num 254) 2024-09-18T11:32:22.674 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:22.674 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:22.674 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:22.674 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:22.675 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:32:22.675 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:22.673+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:22.704 INFO:tasks.rados.rados.0.smithi119.stdout:1804: finishing set_chunk to oid 46 2024-09-18T11:32:22.704 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 46 set_chunk returned error code -95 offset: 379904 length: 948224 2024-09-18T11:32:22.704 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 46 v 0 (ObjNum 320 snap 44 seq_num 320) dirty exists 2024-09-18T11:32:22.704 INFO:tasks.rados.rados.0.smithi119.stdout:1807: expect (ObjNum 358 snap 55 seq_num 358) 2024-09-18T11:32:22.776 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:22.772+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:22.812 INFO:tasks.rados.rados.0.smithi119.stdout:1806: expect (ObjNum 336 snap 48 seq_num 336) 2024-09-18T11:32:22.812 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:22.812 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:22.812 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:22.812 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:22.813 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:22.820 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:32:22.875 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:22.874+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:22.919 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:22.919 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:22.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:22.919 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:22.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.920 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:22.920 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:22.920 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 153.75905379700001, 2024-09-18T11:32:22.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:32:22.920 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.920 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.920 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.920 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:22.920 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:22.920 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:32:22.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.920 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:22.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:22.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:32:22.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:22.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:32:22.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:32:22.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:32:22.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:32:22.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:32:22.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:32:22.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:32:22.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:32:22.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:32:22.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:32:22.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:22.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:32:22.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:32:22.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:32:22.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:32:22.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:32:22.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:32:22.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:32:22.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:32:22.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:32:22.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:32:22.926 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.926 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.926 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.926 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:22.926 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:22.926 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 153.758998867, 2024-09-18T11:32:22.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:32:22.926 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.927 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.927 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.927 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:22.927 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:22.927 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:32:22.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.927 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:22.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:22.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:22.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:22.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:32:22.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:32:22.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:32:22.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:32:22.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:32:22.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:32:22.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:32:22.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:32:22.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:32:22.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:32:22.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:22.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:32:22.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:32:22.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:32:22.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:32:22.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:32:22.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:32:22.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:32:22.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:32:22.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:32:22.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:32:22.932 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.932 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.932 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.932 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:22.932 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:22.932 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 153.75898888699999, 2024-09-18T11:32:22.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:32:22.933 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.933 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.933 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.933 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:22.933 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:22.933 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:32:22.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.933 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:22.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:22.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:22.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:22.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:32:22.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:32:22.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:32:22.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:32:22.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:32:22.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:32:22.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:32:22.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:32:22.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:32:22.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:32:22.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:22.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:32:22.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:32:22.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:32:22.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:32:22.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:32:22.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:32:22.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:32:22.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:32:22.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:32:22.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:32:22.939 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.939 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.939 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.939 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:22.939 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:22.939 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 153.75897728800001, 2024-09-18T11:32:22.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:32:22.940 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.940 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.940 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.940 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:22.940 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:22.940 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:32:22.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.940 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:22.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:22.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:22.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:22.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:32:22.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:32:22.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:32:22.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:32:22.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:32:22.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:32:22.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:32:22.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:32:22.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:32:22.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:32:22.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:22.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:32:22.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:32:22.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:32:22.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:32:22.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:32:22.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:32:22.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:32:22.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:32:22.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:32:22.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:32:22.945 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.945 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.945 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.945 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:22.945 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:22.946 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 153.75892080700001, 2024-09-18T11:32:22.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:32:22.946 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.946 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.946 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.946 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:22.946 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:22.946 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:32:22.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.946 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:22.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:22.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:32:22.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:22.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:32:22.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:32:22.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:32:22.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:32:22.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:32:22.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:32:22.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:32:22.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:32:22.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:32:22.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:32:22.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:22.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:32:22.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:32:22.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:32:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:32:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:32:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:32:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:32:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:32:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:32:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:32:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 153.758906983, 2024-09-18T11:32:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:32:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.951 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:32:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:32:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:32:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:32:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:32:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:32:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:32:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:32:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:32:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:32:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:32:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:32:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:32:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:32:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:32:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:32:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:32:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:32:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:32:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:32:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:32:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.956 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 153.34272807599999, 2024-09-18T11:32:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:32:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:22.957 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:32:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:22.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:32:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:32:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:32:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:32:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:32:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:32:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:32:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:32:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:32:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:32:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:22.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:32:22.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:32:22.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:32:22.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:32:22.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:32:22.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:32:22.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:32:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:32:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:32:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:32:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 153.34270477600001, 2024-09-18T11:32:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:32:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:32:22.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:32:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:32:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:32:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:32:22.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:32:22.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:32:22.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:32:22.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:32:22.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:32:22.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:32:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:32:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:32:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:32:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:32:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:32:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:32:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:32:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:32:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:32:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:32:22.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:32:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 153.34261916700001, 2024-09-18T11:32:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:32:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.969 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:32:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:32:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:32:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:32:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:32:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:32:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:32:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:32:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:32:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:32:22.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:32:22.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:22.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:32:22.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:32:22.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:32:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:32:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:32:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:32:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:32:22.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:32:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:32:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:32:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 153.34260820200001, 2024-09-18T11:32:22.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:32:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:32:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:32:22.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:32:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:32:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:32:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:32:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:32:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:32:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:32:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:32:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:32:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:32:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:32:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:32:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:32:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:32:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:32:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:32:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:32:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:32:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:32:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:22.981 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 153.34259703399999, 2024-09-18T11:32:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:32:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:32:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:32:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:32:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:32:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:32:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:32:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:32:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:32:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:32:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:32:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:32:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:32:22.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:32:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:32:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:32:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:32:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:32:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:32:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:32:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:32:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:32:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:22.987 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 153.342531705, 2024-09-18T11:32:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:32:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:32:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:32:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:32:22.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:32:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:32:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:32:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:32:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:32:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:32:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:32:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:32:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:32:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:32:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:32:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:32:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:32:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:32:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:32:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:32:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:32:22.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:32:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:32:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 153.33236972700001, 2024-09-18T11:32:22.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:32:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:32:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:32:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:32:22.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:32:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:32:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:32:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:32:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:32:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:22.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:32:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:32:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:32:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:32:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:32:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:32:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:22.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:32:22.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:32:22.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:22.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:32:22.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:32:22.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:22.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:32:22.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:32:22.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:22.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:32:22.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:32:22.999 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.999 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:22.999 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:22.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:22.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:22.999 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:22.999 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:22.999 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 153.332277248, 2024-09-18T11:32:22.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:32:23.000 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:23.000 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:23.000 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:23.000 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:23.000 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:23.000 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:32:23.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.000 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:23.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:23.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:23.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:23.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:23.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:23.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:23.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:23.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:23.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:23.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:23.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:32:23.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:32:23.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:23.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:32:23.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:32:23.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:23.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:32:23.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:32:23.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:23.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:32:23.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:32:23.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:23.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:32:23.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:32:23.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:23.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:32:23.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:32:23.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:32:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:32:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:32:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:32:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:32:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:32:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:32:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:32:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1788 4.8 e60/22 4:1898b15d:::smithi11939506-25:head v 60'12, mlcod=58'9)", 2024-09-18T11:32:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:32:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 99.999871759000001, 2024-09-18T11:32:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058348599999999999, 2024-09-18T11:32:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:23.005 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:32:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:32:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916949+0000", 2024-09-18T11:32:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917602+0000", 2024-09-18T11:32:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00065337100000000003 2024-09-18T11:32:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917621+0000", 2024-09-18T11:32:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9287e-05 2024-09-18T11:32:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917629+0000", 2024-09-18T11:32:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3780000000000003e-06 2024-09-18T11:32:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917677+0000", 2024-09-18T11:32:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8068000000000001e-05 2024-09-18T11:32:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917699+0000", 2024-09-18T11:32:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2149000000000001e-05 2024-09-18T11:32:23.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922747+0000", 2024-09-18T11:32:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050477609999999996 2024-09-18T11:32:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922785+0000", 2024-09-18T11:32:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8726000000000002e-05 2024-09-18T11:32:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:23.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3209 4.0 e109/22 4:019b6e09:::smithi11939506-40:head v 109'149, mlcod=106'146)", 2024-09-18T11:32:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:32:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.817436397000002, 2024-09-18T11:32:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063064039999999998, 2024-09-18T11:32:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:32:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:32:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099385+0000", 2024-09-18T11:32:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099951+0000", 2024-09-18T11:32:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056606600000000005 2024-09-18T11:32:23.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099964+0000", 2024-09-18T11:32:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2918999999999999e-05 2024-09-18T11:32:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099972+0000", 2024-09-18T11:32:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2300000000000002e-06 2024-09-18T11:32:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:23.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.100026+0000", 2024-09-18T11:32:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4456999999999997e-05 2024-09-18T11:32:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.100046+0000", 2024-09-18T11:32:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0417000000000001e-05 2024-09-18T11:32:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.105653+0000", 2024-09-18T11:32:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005606214 2024-09-18T11:32:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:23.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.105692+0000", 2024-09-18T11:32:23.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9671e-05 2024-09-18T11:32:23.014 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:23.014 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:23.014 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:23.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.014 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:123 4.3 e132/67 4:cc46532a:::smithi11939506-33:30 v 132'171, mlcod=128'168)", 2024-09-18T11:32:23.014 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:23.014 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.888970622999999, 2024-09-18T11:32:23.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087627039999999996, 2024-09-18T11:32:23.014 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:23.014 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:23.014 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:23.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:23.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:23.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:23.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:23.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:23.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:23.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:23.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027851+0000", 2024-09-18T11:32:23.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027871+0000", 2024-09-18T11:32:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9749e-05 2024-09-18T11:32:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027882+0000", 2024-09-18T11:32:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1242e-05 2024-09-18T11:32:23.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027889+0000", 2024-09-18T11:32:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7649999999999996e-06 2024-09-18T11:32:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027929+0000", 2024-09-18T11:32:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9984999999999997e-05 2024-09-18T11:32:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:23.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027951+0000", 2024-09-18T11:32:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2047000000000001e-05 2024-09-18T11:32:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.036565+0000", 2024-09-18T11:32:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086135110000000008 2024-09-18T11:32:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.036614+0000", 2024-09-18T11:32:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9747000000000003e-05 2024-09-18T11:32:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:23.018 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:124 4.3 e132/67 4:ca2b2309:::smithi11939506-44:30 v 132'173, mlcod=128'168)", 2024-09-18T11:32:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.888278640999999, 2024-09-18T11:32:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0088947309999999995, 2024-09-18T11:32:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:23.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028543+0000", 2024-09-18T11:32:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:23.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028551+0000", 2024-09-18T11:32:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8720000000000002e-06 2024-09-18T11:32:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028562+0000", 2024-09-18T11:32:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1039000000000001e-05 2024-09-18T11:32:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028569+0000", 2024-09-18T11:32:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0350000000000004e-06 2024-09-18T11:32:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028848+0000", 2024-09-18T11:32:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00027935600000000003 2024-09-18T11:32:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028868+0000", 2024-09-18T11:32:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9947999999999999e-05 2024-09-18T11:32:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037405+0000", 2024-09-18T11:32:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085369920000000002 2024-09-18T11:32:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037438+0000", 2024-09-18T11:32:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3244000000000002e-05 2024-09-18T11:32:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:142 4.3 e141/67 4:ca2b2309:::smithi11939506-44:5d v 141'197, mlcod=140'192)", 2024-09-18T11:32:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.297859+0000", 2024-09-18T11:32:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 10.618963631, 2024-09-18T11:32:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063063160000000002, 2024-09-18T11:32:23.023 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297859+0000", 2024-09-18T11:32:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297859+0000", 2024-09-18T11:32:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:23.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297858+0000", 2024-09-18T11:32:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297863+0000", 2024-09-18T11:32:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9450000000000001e-06 2024-09-18T11:32:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297870+0000", 2024-09-18T11:32:23.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.43e-06 2024-09-18T11:32:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297875+0000", 2024-09-18T11:32:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8439999999999997e-06 2024-09-18T11:32:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297970+0000", 2024-09-18T11:32:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4968000000000003e-05 2024-09-18T11:32:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:23.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297991+0000", 2024-09-18T11:32:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0726000000000001e-05 2024-09-18T11:32:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.304127+0000", 2024-09-18T11:32:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061356700000000002 2024-09-18T11:32:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.304165+0000", 2024-09-18T11:32:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8305e-05 2024-09-18T11:32:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:23.027 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:154 4.3 e147/67 4:ca2b2309:::smithi11939506-44:5d v 147'216, mlcod=145'210)", 2024-09-18T11:32:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.579492192, 2024-09-18T11:32:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066667829999999999, 2024-09-18T11:32:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:23.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337335+0000", 2024-09-18T11:32:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1479999999999997e-06 2024-09-18T11:32:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337344+0000", 2024-09-18T11:32:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5640000000000002e-06 2024-09-18T11:32:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337349+0000", 2024-09-18T11:32:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9470000000000002e-06 2024-09-18T11:32:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337410+0000", 2024-09-18T11:32:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0448999999999997e-05 2024-09-18T11:32:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:23.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337425+0000", 2024-09-18T11:32:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.504e-05 2024-09-18T11:32:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.343972+0000", 2024-09-18T11:32:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065474169999999998 2024-09-18T11:32:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.343997+0000", 2024-09-18T11:32:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4672999999999999e-05 2024-09-18T11:32:23.032 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:23.033 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:23.033 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:23.033 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:23.033 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:23.033 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:23.035 INFO:tasks.rados.rados.0.smithi119.stdout:1809: got (22) Invalid argument 2024-09-18T11:32:23.035 INFO:tasks.rados.rados.0.smithi119.stdout:1802: done (5 left) 2024-09-18T11:32:23.035 INFO:tasks.rados.rados.0.smithi119.stdout:1803: done (4 left) 2024-09-18T11:32:23.035 INFO:tasks.rados.rados.0.smithi119.stdout:1804: done (3 left) 2024-09-18T11:32:23.035 INFO:tasks.rados.rados.0.smithi119.stdout:1806: done (2 left) 2024-09-18T11:32:23.035 INFO:tasks.rados.rados.0.smithi119.stdout:1807: done (1 left) 2024-09-18T11:32:23.035 INFO:tasks.rados.rados.0.smithi119.stdout:1809: done (0 left) 2024-09-18T11:32:23.035 INFO:tasks.rados.rados.0.smithi119.stdout:1810: tier_evict oid 10 2024-09-18T11:32:23.035 INFO:tasks.rados.rados.0.smithi119.stdout:1810: tier_evict oid 10 snap -1 2024-09-18T11:32:23.035 INFO:tasks.rados.rados.0.smithi119.stdout:1811: read oid 1 snap 50 2024-09-18T11:32:23.035 INFO:tasks.rados.rados.0.smithi119.stdout:1811: expect (ObjNum 277 snap 37 seq_num 277) 2024-09-18T11:32:23.036 INFO:tasks.rados.rados.0.smithi119.stdout:1812: write oid 2 current snap is 56 2024-09-18T11:32:23.036 INFO:tasks.rados.rados.0.smithi119.stdout:1812: seq_num 366 ranges {551362=726078,1717586=409779} 2024-09-18T11:32:23.036 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:32:23.036 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:23.033+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:23.037 INFO:tasks.rados.rados.0.smithi119.stdout:1812: writing smithi11939506-2 from 551362 to 1277440 tid 1 2024-09-18T11:32:23.042 INFO:tasks.rados.rados.0.smithi119.stdout:1812: writing smithi11939506-2 from 1717586 to 2127365 tid 2 2024-09-18T11:32:23.042 INFO:tasks.rados.rados.0.smithi119.stdout:1813: snap_create 2024-09-18T11:32:23.042 INFO:tasks.rados.rados.0.smithi119.stdout:1810: got (22) Invalid argument 2024-09-18T11:32:23.042 INFO:tasks.rados.rados.0.smithi119.stdout:1811: expect (ObjNum 277 snap 37 seq_num 277) 2024-09-18T11:32:23.098 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:32:23.135 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:23.136 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:23.136 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:23.136 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:23.136 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:23.134+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:23.144 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:32:23.224 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:23.225 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:23.236 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:23.235+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:23.240 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:23.240 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:23.255 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:23.255 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:23.270 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:23.270 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:23.284 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:23.284 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:23.285 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:23.286 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:23.293 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:32:23.301 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:23.302 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:23.317 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:23.317 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:23.333 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:23.333 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:23.336 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:23.335+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:23.402 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:23.402 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:23.410 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:32:23.455 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:23.453+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:23.508 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:23.508 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:23.516 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:32:23.558 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:23.556+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:23.638 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:23.636+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:23.654 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:23.654 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:23.663 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:32:23.739 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:23.738+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:23.800 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:23.800 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:23.808 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:32:23.853 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:23.850+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:23.905 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:23.906 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:23.914 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:32:23.940 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:23.939+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:24.023 INFO:tasks.rados.rados.0.smithi119.stdout:1810: done (3 left) 2024-09-18T11:32:24.023 INFO:tasks.rados.rados.0.smithi119.stdout:1811: done (2 left) 2024-09-18T11:32:24.023 INFO:tasks.rados.rados.0.smithi119.stdout:1813: done (1 left) 2024-09-18T11:32:24.023 INFO:tasks.rados.rados.0.smithi119.stdout:1814: write (excl) oid 27 current snap is 57 2024-09-18T11:32:24.023 INFO:tasks.rados.rados.0.smithi119.stdout:1814: seq_num 367 ranges {568169=401327,1376068=669790} 2024-09-18T11:32:24.028 INFO:tasks.rados.rados.0.smithi119.stdout:1814: writing smithi11939506-27 from 568169 to 969496 tid 1 2024-09-18T11:32:24.035 INFO:tasks.rados.rados.0.smithi119.stdout:1814: writing smithi11939506-27 from 1376068 to 2045858 tid 2 2024-09-18T11:32:24.035 INFO:tasks.rados.rados.0.smithi119.stdout:1815: tier_promote oid 6 2024-09-18T11:32:24.035 INFO:tasks.rados.rados.0.smithi119.stdout:1812: finishing write tid 1 to smithi11939506-2 2024-09-18T11:32:24.035 INFO:tasks.rados.rados.0.smithi119.stdout:1814: finishing write tid 1 to smithi11939506-27 2024-09-18T11:32:24.035 INFO:tasks.rados.rados.0.smithi119.stdout:1812: finishing write tid 2 to smithi11939506-2 2024-09-18T11:32:24.035 INFO:tasks.rados.rados.0.smithi119.stdout:1816: read oid 23 snap -1 2024-09-18T11:32:24.035 INFO:tasks.rados.rados.0.smithi119.stdout:1816: expect (ObjNum 363 snap 55 seq_num 363) 2024-09-18T11:32:24.035 INFO:tasks.rados.rados.0.smithi119.stdout:1817: write oid 25 current snap is 57 2024-09-18T11:32:24.036 INFO:tasks.rados.rados.0.smithi119.stdout:1817: seq_num 368 ranges {584976=476576,1834550=350426} 2024-09-18T11:32:24.041 INFO:tasks.rados.rados.0.smithi119.stdout:1817: writing smithi11939506-25 from 584976 to 1061552 tid 1 2024-09-18T11:32:24.041 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:24.039+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:24.048 INFO:tasks.rados.rados.0.smithi119.stdout:1817: writing smithi11939506-25 from 1834550 to 2184976 tid 2 2024-09-18T11:32:24.048 INFO:tasks.rados.rados.0.smithi119.stdout:1818: set_chunk oid 7 target oid 2024-09-18T11:32:24.053 INFO:tasks.rados.rados.0.smithi119.stdout:1818: set_chunk oid 7 offset: 1387520 length: 477696 target oid 9120c528f6604abe92f36811165977c81881f4ec2f8ce5bc701844e84b741212 offset: 0 snap 44 2024-09-18T11:32:24.053 INFO:tasks.rados.rados.0.smithi119.stdout:1819: read oid 49 snap 33 2024-09-18T11:32:24.053 INFO:tasks.rados.rados.0.smithi119.stdout:1819: expect (ObjNum 216 snap 26 seq_num 216) 2024-09-18T11:32:24.053 INFO:tasks.rados.rados.0.smithi119.stdout:1812: finishing write tid 3 to smithi11939506-2 2024-09-18T11:32:24.053 INFO:tasks.rados.rados.0.smithi119.stdout:1820: read oid 47 snap 32 2024-09-18T11:32:24.053 INFO:tasks.rados.rados.0.smithi119.stdout:1820: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T11:32:24.053 INFO:tasks.rados.rados.0.smithi119.stdout:1821: set_chunk oid 40 target oid 2024-09-18T11:32:24.059 INFO:tasks.rados.rados.0.smithi119.stdout:1821: set_chunk oid 40 offset: 653824 length: 184832 target oid f66bc6ef279694e71b7ad49aac79329d9efdd2edfb990fc66bcf42adee0b1a3d offset: 0 snap -1 2024-09-18T11:32:24.059 INFO:tasks.rados.rados.0.smithi119.stdout:1822: read oid 44 snap 47 2024-09-18T11:32:24.059 INFO:tasks.rados.rados.0.smithi119.stdout:1822: expect (ObjNum 252 snap 30 seq_num 252) 2024-09-18T11:32:24.059 INFO:tasks.rados.rados.0.smithi119.stdout:1823: write (excl) oid 32 current snap is 57 2024-09-18T11:32:24.060 INFO:tasks.rados.rados.0.smithi119.stdout:1812: finishing write tid 4 to smithi11939506-2 2024-09-18T11:32:24.060 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 498 (ObjNum 366 snap 56 seq_num 366) dirty exists 2024-09-18T11:32:24.060 INFO:tasks.rados.rados.0.smithi119.stdout:1812: left oid 2 (ObjNum 366 snap 56 seq_num 366) 2024-09-18T11:32:24.060 INFO:tasks.rados.rados.0.smithi119.stdout:1815: got (0) Success 2024-09-18T11:32:24.060 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 6 v 140 (ObjNum 327 snap 45 seq_num 327) dirty exists 2024-09-18T11:32:24.060 INFO:tasks.rados.rados.0.smithi119.stdout:1814: finishing write tid 2 to smithi11939506-27 2024-09-18T11:32:24.060 INFO:tasks.rados.rados.0.smithi119.stdout:1823: seq_num 369 ranges {601783=551825,1776679=425104} 2024-09-18T11:32:24.061 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:24.061 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:24.067 INFO:tasks.rados.rados.0.smithi119.stdout:1823: writing smithi11939506-32 from 601783 to 1153608 tid 1 2024-09-18T11:32:24.072 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:32:24.074 INFO:tasks.rados.rados.0.smithi119.stdout:1823: writing smithi11939506-32 from 1776679 to 2201783 tid 2 2024-09-18T11:32:24.074 INFO:tasks.rados.rados.0.smithi119.stdout:1812: done (10 left) 2024-09-18T11:32:24.074 INFO:tasks.rados.rados.0.smithi119.stdout:1815: done (9 left) 2024-09-18T11:32:24.074 INFO:tasks.rados.rados.0.smithi119.stdout:1824: write (excl) oid 11 current snap is 57 2024-09-18T11:32:24.074 INFO:tasks.rados.rados.0.smithi119.stdout:1824: seq_num 370 ranges {618590=627074,1835161=383429} 2024-09-18T11:32:24.080 INFO:tasks.rados.rados.0.smithi119.stdout:1824: writing smithi11939506-11 from 618590 to 1245664 tid 1 2024-09-18T11:32:24.086 INFO:tasks.rados.rados.0.smithi119.stdout:1824: writing smithi11939506-11 from 1835161 to 2218590 tid 2 2024-09-18T11:32:24.086 INFO:tasks.rados.rados.0.smithi119.stdout:1825: snap_remove snap 46 2024-09-18T11:32:24.142 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:24.140+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:24.172 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:24.172 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:24.181 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:32:24.266 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:24.264+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:24.320 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:24.320 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:24.328 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:32:24.344 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:24.343+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:24.415 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T11:32:24.445 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:24.444+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:24.468 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:24.469 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:24.477 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:32:24.546 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:24.545+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:24.581 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:24.581 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:24.589 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:32:24.646 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:24.644+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:24.694 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:24.694 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:24.703 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:32:24.748 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:24.746+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:24.800 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:24.800 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:24.809 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:32:24.852 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:24.851+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:24.908 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:24.908 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:24.917 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:32:24.949 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:24.948+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:25.025 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is set 2024-09-18T11:32:25.025 INFO:tasks.rados.rados.0.smithi119.stdout:1825: done (10 left) 2024-09-18T11:32:25.026 INFO:tasks.rados.rados.0.smithi119.stdout:1826: write (excl) oid 26 current snap is 57 2024-09-18T11:32:25.026 INFO:tasks.rados.rados.0.smithi119.stdout:1826: seq_num 371 ranges {635397=702323,1893643=341754} 2024-09-18T11:32:25.033 INFO:tasks.rados.rados.0.smithi119.stdout:1826: writing smithi11939506-26 from 635397 to 1337720 tid 1 2024-09-18T11:32:25.038 INFO:tasks.rados.rados.0.smithi119.stdout:1826: writing smithi11939506-26 from 1893643 to 2235397 tid 2 2024-09-18T11:32:25.038 INFO:tasks.rados.rados.0.smithi119.stdout:1827: snap_create 2024-09-18T11:32:25.038 INFO:tasks.rados.rados.0.smithi119.stdout:1816: expect (ObjNum 363 snap 55 seq_num 363) 2024-09-18T11:32:25.050 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:25.049+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:25.063 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:25.064 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:25.072 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:32:25.155 INFO:tasks.rados.rados.0.smithi119.stdout:1814: finishing write tid 3 to smithi11939506-27 2024-09-18T11:32:25.156 INFO:tasks.rados.rados.0.smithi119.stdout:1814: finishing write tid 4 to smithi11939506-27 2024-09-18T11:32:25.156 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 319 (ObjNum 367 snap 57 seq_num 367) dirty exists 2024-09-18T11:32:25.156 INFO:tasks.rados.rados.0.smithi119.stdout:1814: left oid 27 (ObjNum 367 snap 57 seq_num 367) 2024-09-18T11:32:25.156 INFO:tasks.rados.rados.0.smithi119.stdout:1817: finishing write tid 1 to smithi11939506-25 2024-09-18T11:32:25.156 INFO:tasks.rados.rados.0.smithi119.stdout:1817: finishing write tid 2 to smithi11939506-25 2024-09-18T11:32:25.156 INFO:tasks.rados.rados.0.smithi119.stdout:1817: finishing write tid 3 to smithi11939506-25 2024-09-18T11:32:25.156 INFO:tasks.rados.rados.0.smithi119.stdout:1817: finishing write tid 4 to smithi11939506-25 2024-09-18T11:32:25.156 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 76 (ObjNum 368 snap 57 seq_num 368) dirty exists 2024-09-18T11:32:25.156 INFO:tasks.rados.rados.0.smithi119.stdout:1817: left oid 25 (ObjNum 368 snap 57 seq_num 368) 2024-09-18T11:32:25.156 INFO:tasks.rados.rados.0.smithi119.stdout:1818: finishing set_chunk to oid 7 2024-09-18T11:32:25.156 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 7 set_chunk returned error code -95 offset: 1387520 length: 477696 2024-09-18T11:32:25.157 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 7 v 0 (ObjNum 145 snap 18 seq_num 145) dirty exists 2024-09-18T11:32:25.157 INFO:tasks.rados.rados.0.smithi119.stdout:1820: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-18T11:32:25.157 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:25.155+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:25.211 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:25.211 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:25.220 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:32:25.245 INFO:tasks.rados.rados.0.smithi119.stdout:1821: finishing set_chunk to oid 40 2024-09-18T11:32:25.245 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 163 (ObjNum 287 snap 41 seq_num 287) dirty exists 2024-09-18T11:32:25.245 INFO:tasks.rados.rados.0.smithi119.stdout:1819: expect (ObjNum 216 snap 26 seq_num 216) 2024-09-18T11:32:25.252 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:25.250+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:25.344 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:32:25.348 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-18T11:32:25.349 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T11:32:25.349 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-18T11:32:25.349 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-09-18T11:32:25.360 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:25.360 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:25.369 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:32:25.388 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:25.386+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:25.454 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:25.452+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:25.461 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:25.461 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:25.468 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:25.468 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:25.476 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:32:25.477 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:25.477 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:25.492 INFO:tasks.rados.rados.0.smithi119.stdout:1822: expect (ObjNum 252 snap 30 seq_num 252) 2024-09-18T11:32:25.493 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:25.494 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:25.508 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:25.508 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:25.524 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:25.524 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:25.540 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:25.540 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:25.554 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:25.552+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:25.556 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:25.556 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:25.571 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:25.571 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:25.615 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:25.615 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:25.615 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:25.615 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:25.615 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:25.623 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:32:25.658 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:32:25.656+0000 7fd482ca8640 -1 mgr.server reply reply (1) Operation not permitted no change 2024-09-18T11:32:25.658 INFO:teuthology.orchestra.run.smithi119.stdout:moved 1 / 58 (1.72414%) 2024-09-18T11:32:25.658 INFO:teuthology.orchestra.run.smithi119.stdout:avg 8.28571 2024-09-18T11:32:25.658 INFO:teuthology.orchestra.run.smithi119.stdout:stddev 3.88088 -> 3.53409 (expected baseline 2.66497) 2024-09-18T11:32:25.658 INFO:teuthology.orchestra.run.smithi119.stdout:min osd.2 with 0 -> 1 pgs (0 -> 0.12069 * mean) 2024-09-18T11:32:25.658 INFO:teuthology.orchestra.run.smithi119.stdout:max osd.1 with 14 -> 14 pgs (1.68966 -> 1.68966 * mean) 2024-09-18T11:32:25.658 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:32:25.658 INFO:teuthology.orchestra.run.smithi119.stdout:oload 1000 2024-09-18T11:32:25.659 INFO:teuthology.orchestra.run.smithi119.stdout:max_change 1 2024-09-18T11:32:25.659 INFO:teuthology.orchestra.run.smithi119.stdout:max_change_osds 4 2024-09-18T11:32:25.659 INFO:teuthology.orchestra.run.smithi119.stdout:average_utilization 0.0040 2024-09-18T11:32:25.659 INFO:teuthology.orchestra.run.smithi119.stdout:overload_utilization 0.0399 2024-09-18T11:32:25.659 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 weight 0.2805 -> 0.3497 2024-09-18T11:32:25.659 INFO:teuthology.orchestra.run.smithi119.stderr:no change 2024-09-18T11:32:25.667 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:25.666+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:25.670 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-18T11:32:25.670 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-18T11:32:25.676 INFO:tasks.rados.rados.0.smithi119.stdout:1823: finishing write tid 1 to smithi11939506-32 2024-09-18T11:32:25.676 INFO:tasks.rados.rados.0.smithi119.stdout:1823: finishing write tid 2 to smithi11939506-32 2024-09-18T11:32:25.676 INFO:tasks.rados.rados.0.smithi119.stdout:1823: finishing write tid 3 to smithi11939506-32 2024-09-18T11:32:25.676 INFO:tasks.rados.rados.0.smithi119.stdout:1823: finishing write tid 4 to smithi11939506-32 2024-09-18T11:32:25.676 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 230 (ObjNum 369 snap 57 seq_num 369) dirty exists 2024-09-18T11:32:25.676 INFO:tasks.rados.rados.0.smithi119.stdout:1823: left oid 32 (ObjNum 369 snap 57 seq_num 369) 2024-09-18T11:32:25.676 INFO:tasks.rados.rados.0.smithi119.stdout:1824: finishing write tid 1 to smithi11939506-11 2024-09-18T11:32:25.676 INFO:tasks.rados.rados.0.smithi119.stdout:1824: finishing write tid 2 to smithi11939506-11 2024-09-18T11:32:25.676 INFO:tasks.rados.rados.0.smithi119.stdout:1824: finishing write tid 3 to smithi11939506-11 2024-09-18T11:32:25.676 INFO:tasks.rados.rados.0.smithi119.stdout:1824: finishing write tid 4 to smithi11939506-11 2024-09-18T11:32:25.676 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 11 v 341 (ObjNum 370 snap 57 seq_num 370) dirty exists 2024-09-18T11:32:25.676 INFO:tasks.rados.rados.0.smithi119.stdout:1824: left oid 11 (ObjNum 370 snap 57 seq_num 370) 2024-09-18T11:32:25.677 INFO:tasks.rados.rados.0.smithi119.stdout:1826: finishing write tid 1 to smithi11939506-26 2024-09-18T11:32:25.677 INFO:tasks.rados.rados.0.smithi119.stdout:1826: finishing write tid 2 to smithi11939506-26 2024-09-18T11:32:25.677 INFO:tasks.rados.rados.0.smithi119.stdout:1826: finishing write tid 3 to smithi11939506-26 2024-09-18T11:32:25.677 INFO:tasks.rados.rados.0.smithi119.stdout:1826: finishing write tid 4 to smithi11939506-26 2024-09-18T11:32:25.677 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 332 (ObjNum 371 snap 57 seq_num 371) dirty exists 2024-09-18T11:32:25.677 INFO:tasks.rados.rados.0.smithi119.stdout:1826: left oid 26 (ObjNum 371 snap 57 seq_num 371) 2024-09-18T11:32:25.723 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:25.723 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:25.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:25.723 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:25.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.723 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:32:25.723 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:25.723 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 134.60425103099999, 2024-09-18T11:32:25.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:32:25.723 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:25.724 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:25.724 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:25.724 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:25.724 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:25.724 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:32:25.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.724 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:25.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:25.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:25.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:25.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:25.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:25.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:32:25.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:25.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:25.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:32:25.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:32:25.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.726 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:25.726 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:32:25.726 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:32:25.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.726 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:25.726 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:32:25.726 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:32:25.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.726 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.726 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:32:25.726 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:32:25.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.727 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:25.727 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:32:25.727 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:32:25.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.727 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.727 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:32:25.727 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:32:25.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.727 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:25.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:32:25.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:32:25.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.728 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:25.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:32:25.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:32:25.728 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.728 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:25.728 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.729 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:32:25.729 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:25.729 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 125.05122563, 2024-09-18T11:32:25.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:32:25.729 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:25.729 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:25.729 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:25.729 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:25.729 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:25.729 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:32:25.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.729 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:25.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.729 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:25.730 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:25.730 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.730 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:25.730 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:25.730 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.730 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:25.730 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:32:25.730 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:25.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.731 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:25.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:32:25.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:32:25.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.731 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:25.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:32:25.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:32:25.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.731 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:25.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:32:25.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:32:25.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.732 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.732 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:32:25.732 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:32:25.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.732 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:25.732 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:32:25.732 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:32:25.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.732 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:25.732 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:32:25.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:32:25.733 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.733 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:25.733 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.733 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2207 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1410296~761389] snapc 24=[24,1e,1c,1b,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T11:32:25.733 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:32:25.733 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.298072966999996, 2024-09-18T11:32:25.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008410743, 2024-09-18T11:32:25.733 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:25.733 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:25.733 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:25.733 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:25.734 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:25.734 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2207 2024-09-18T11:32:25.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.734 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:25.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.734 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:25.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:32:25.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.734 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:25.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:32:25.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.735 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.735 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.735 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:25.735 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422496+0000", 2024-09-18T11:32:25.735 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:25.735 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.735 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.735 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:25.735 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423190+0000", 2024-09-18T11:32:25.735 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00069405499999999995 2024-09-18T11:32:25.735 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.735 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.735 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:25.736 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423204+0000", 2024-09-18T11:32:25.736 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:32:25.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.736 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:25.736 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423208+0000", 2024-09-18T11:32:25.736 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:32:25.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.736 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.736 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423293+0000", 2024-09-18T11:32:25.736 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4524999999999995e-05 2024-09-18T11:32:25.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:25.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423394+0000", 2024-09-18T11:32:25.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101395 2024-09-18T11:32:25.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.737 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:25.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423579+0000", 2024-09-18T11:32:25.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018461000000000001 2024-09-18T11:32:25.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.737 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:25.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.428574+0000", 2024-09-18T11:32:25.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049944960000000002 2024-09-18T11:32:25.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:25.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430809+0000", 2024-09-18T11:32:25.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022351820000000001 2024-09-18T11:32:25.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:25.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430862+0000", 2024-09-18T11:32:25.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3189999999999997e-05 2024-09-18T11:32:25.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:25.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430908+0000", 2024-09-18T11:32:25.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5973999999999997e-05 2024-09-18T11:32:25.739 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.739 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:25.739 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.739 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.739 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2339 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:25.739 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:25.739 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 84.333768337999999, 2024-09-18T11:32:25.739 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011809992, 2024-09-18T11:32:25.739 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:25.739 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:25.739 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:25.739 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:25.739 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:25.740 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2339 2024-09-18T11:32:25.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.740 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:25.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.740 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:25.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:25.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.740 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:25.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:25.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.741 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:25.741 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:32:25.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:25.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.741 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:25.741 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:32:25.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4710000000000001e-06 2024-09-18T11:32:25.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.741 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:25.741 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386824+0000", 2024-09-18T11:32:25.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6104e-05 2024-09-18T11:32:25.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:25.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386836+0000", 2024-09-18T11:32:25.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1827999999999999e-05 2024-09-18T11:32:25.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388390+0000", 2024-09-18T11:32:25.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015539499999999999 2024-09-18T11:32:25.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:25.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388443+0000", 2024-09-18T11:32:25.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3785000000000001e-05 2024-09-18T11:32:25.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398238+0000", 2024-09-18T11:32:25.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097947989999999999 2024-09-18T11:32:25.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:25.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398315+0000", 2024-09-18T11:32:25.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6909999999999994e-05 2024-09-18T11:32:25.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:25.744 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398612+0000", 2024-09-18T11:32:25.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029671999999999999 2024-09-18T11:32:25.744 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.744 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:25.744 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.744 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2340 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:25.744 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:25.744 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 84.333725025999996, 2024-09-18T11:32:25.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012086448, 2024-09-18T11:32:25.744 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:25.744 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:25.744 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:25.745 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:25.745 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:25.745 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2340 2024-09-18T11:32:25.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.745 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:25.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:25.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:25.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:25.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:25.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.746 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:25.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:25.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:25.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.746 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:25.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386848+0000", 2024-09-18T11:32:25.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3069999999999998e-06 2024-09-18T11:32:25.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.746 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:25.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386855+0000", 2024-09-18T11:32:25.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0979999999999996e-06 2024-09-18T11:32:25.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:25.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386859+0000", 2024-09-18T11:32:25.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9360000000000001e-06 2024-09-18T11:32:25.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388466+0000", 2024-09-18T11:32:25.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001607154 2024-09-18T11:32:25.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:25.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388507+0000", 2024-09-18T11:32:25.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1093000000000001e-05 2024-09-18T11:32:25.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398647+0000", 2024-09-18T11:32:25.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010139686 2024-09-18T11:32:25.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:25.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398718+0000", 2024-09-18T11:32:25.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1138999999999998e-05 2024-09-18T11:32:25.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398931+0000", 2024-09-18T11:32:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021356899999999999 2024-09-18T11:32:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2341 4.5 4:aa575018:::smithi11939506-11:head [read 0~2487436,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 84.333706804000002, 2024-09-18T11:32:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013751648999999999, 2024-09-18T11:32:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:25.750 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:25.750 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:25.750 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:25.750 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:25.750 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2341 2024-09-18T11:32:25.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.750 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:25.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.750 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:25.750 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:25.750 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.751 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:25.751 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:25.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.751 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:25.751 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:25.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:25.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.751 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:25.751 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386866+0000", 2024-09-18T11:32:25.751 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4249999999999998e-06 2024-09-18T11:32:25.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:25.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386872+0000", 2024-09-18T11:32:25.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.003e-06 2024-09-18T11:32:25.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:25.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:32:25.752 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.507e-06 2024-09-18T11:32:25.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.752 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.752 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388520+0000", 2024-09-18T11:32:25.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001644146 2024-09-18T11:32:25.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.753 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:25.753 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388561+0000", 2024-09-18T11:32:25.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1486999999999998e-05 2024-09-18T11:32:25.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.753 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.753 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398963+0000", 2024-09-18T11:32:25.753 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010401353 2024-09-18T11:32:25.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.753 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.753 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:25.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.399046+0000", 2024-09-18T11:32:25.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3141999999999998e-05 2024-09-18T11:32:25.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.754 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:25.754 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.400615+0000", 2024-09-18T11:32:25.754 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015690960000000001 2024-09-18T11:32:25.754 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.754 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:25.754 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.754 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2529 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:25.755 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:32:25.755 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 76.547233539999993, 2024-09-18T11:32:25.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079268120000000001, 2024-09-18T11:32:25.755 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:25.755 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:25.755 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:25.755 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:25.755 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:25.755 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2529 2024-09-18T11:32:25.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.755 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:25.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.755 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:25.755 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:32:25.755 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:25.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173337+0000", 2024-09-18T11:32:25.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:25.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173336+0000", 2024-09-18T11:32:25.756 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:25.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.756 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:25.756 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173339+0000", 2024-09-18T11:32:25.757 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9019999999999999e-06 2024-09-18T11:32:25.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:25.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173381+0000", 2024-09-18T11:32:25.757 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1836999999999999e-05 2024-09-18T11:32:25.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:25.757 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173390+0000", 2024-09-18T11:32:25.757 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.302e-06 2024-09-18T11:32:25.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.757 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.758 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174507+0000", 2024-09-18T11:32:25.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001116868 2024-09-18T11:32:25.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.758 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:25.758 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174603+0000", 2024-09-18T11:32:25.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5598000000000002e-05 2024-09-18T11:32:25.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.758 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:25.758 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.174759+0000", 2024-09-18T11:32:25.758 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015616299999999999 2024-09-18T11:32:25.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.758 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:25.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178943+0000", 2024-09-18T11:32:25.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0041846180000000002 2024-09-18T11:32:25.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:25.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181175+0000", 2024-09-18T11:32:25.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00223215 2024-09-18T11:32:25.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.759 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:25.759 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181229+0000", 2024-09-18T11:32:25.759 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3585999999999998e-05 2024-09-18T11:32:25.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.760 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.760 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:25.760 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181263+0000", 2024-09-18T11:32:25.760 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4292999999999998e-05 2024-09-18T11:32:25.760 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.760 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:25.760 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.760 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.760 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2530 4.5 4:af4ba75a:::smithi11939506-37:head [rollback 25] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:25.760 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:32:25.760 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 76.547171234000004, 2024-09-18T11:32:25.760 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080198800000000001, 2024-09-18T11:32:25.761 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:25.761 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:25.761 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:25.761 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:25.761 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:25.761 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2530 2024-09-18T11:32:25.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.761 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:25.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.761 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:25.761 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:32:25.761 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:25.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:32:25.762 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:25.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173398+0000", 2024-09-18T11:32:25.762 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:25.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.762 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:25.762 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173403+0000", 2024-09-18T11:32:25.762 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4389999999999999e-06 2024-09-18T11:32:25.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.763 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:25.763 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173414+0000", 2024-09-18T11:32:25.763 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0411000000000001e-05 2024-09-18T11:32:25.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.763 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:25.763 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173420+0000", 2024-09-18T11:32:25.763 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9610000000000002e-06 2024-09-18T11:32:25.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.763 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.763 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176390+0000", 2024-09-18T11:32:25.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029706289999999998 2024-09-18T11:32:25.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:25.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176501+0000", 2024-09-18T11:32:25.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011090300000000001 2024-09-18T11:32:25.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:25.764 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176827+0000", 2024-09-18T11:32:25.764 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00032526899999999999 2024-09-18T11:32:25.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.764 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:25.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178958+0000", 2024-09-18T11:32:25.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021314860000000001 2024-09-18T11:32:25.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.765 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:25.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181318+0000", 2024-09-18T11:32:25.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002359571 2024-09-18T11:32:25.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.765 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:25.765 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181372+0000", 2024-09-18T11:32:25.765 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3884000000000001e-05 2024-09-18T11:32:25.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181419+0000", 2024-09-18T11:32:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7148000000000003e-05 2024-09-18T11:32:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2531 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 76.547145111000006, 2024-09-18T11:32:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080687239999999993, 2024-09-18T11:32:25.767 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:25.767 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:25.767 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:25.767 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:25.767 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:25.767 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2531 2024-09-18T11:32:25.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.767 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:25.767 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.767 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:25.767 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:25.767 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.767 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.768 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:25.768 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:25.768 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.768 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:25.768 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:25.768 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:25.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.768 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.768 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:25.768 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173429+0000", 2024-09-18T11:32:25.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2989999999999998e-06 2024-09-18T11:32:25.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.769 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.769 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:25.769 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173436+0000", 2024-09-18T11:32:25.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1799999999999999e-06 2024-09-18T11:32:25.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.769 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.769 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:25.769 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173441+0000", 2024-09-18T11:32:25.769 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8810000000000004e-06 2024-09-18T11:32:25.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.769 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.770 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.770 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178267+0000", 2024-09-18T11:32:25.770 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048257969999999997 2024-09-18T11:32:25.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.770 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:25.770 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178388+0000", 2024-09-18T11:32:25.770 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121779 2024-09-18T11:32:25.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.770 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:25.770 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178599+0000", 2024-09-18T11:32:25.770 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000210094 2024-09-18T11:32:25.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:25.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.179339+0000", 2024-09-18T11:32:25.771 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00074063799999999995 2024-09-18T11:32:25.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:25.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181444+0000", 2024-09-18T11:32:25.771 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021044100000000001 2024-09-18T11:32:25.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.771 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:25.771 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181470+0000", 2024-09-18T11:32:25.772 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6415999999999999e-05 2024-09-18T11:32:25.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.772 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:25.772 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181494+0000", 2024-09-18T11:32:25.772 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3672e-05 2024-09-18T11:32:25.772 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.772 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:25.772 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.772 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2910 4.5 4:af4ba75a:::smithi11939506-37:head [write 786627~795991,stat] snapc 37=[37,35,34,30,2f,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:25.772 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:32:25.773 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 62.717087282000001, 2024-09-18T11:32:25.773 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084093710000000006, 2024-09-18T11:32:25.773 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:25.773 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:25.773 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:25.773 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:25.773 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:25.773 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2910 2024-09-18T11:32:25.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.773 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:25.773 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.773 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:25.773 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:32:25.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.774 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:25.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:32:25.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.774 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:25.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003481+0000", 2024-09-18T11:32:25.774 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:25.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.774 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:25.774 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004300+0000", 2024-09-18T11:32:25.775 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081934200000000003 2024-09-18T11:32:25.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.775 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:25.775 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004324+0000", 2024-09-18T11:32:25.775 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4618999999999999e-05 2024-09-18T11:32:25.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.775 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:25.775 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004335+0000", 2024-09-18T11:32:25.775 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.092e-05 2024-09-18T11:32:25.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.776 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004385+0000", 2024-09-18T11:32:25.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9709999999999997e-05 2024-09-18T11:32:25.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.776 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:25.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004470+0000", 2024-09-18T11:32:25.776 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4864000000000006e-05 2024-09-18T11:32:25.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.776 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:25.776 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004612+0000", 2024-09-18T11:32:25.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014241500000000001 2024-09-18T11:32:25.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:25.777 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011193+0000", 2024-09-18T11:32:25.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006580654 2024-09-18T11:32:25.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:25.777 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011788+0000", 2024-09-18T11:32:25.777 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059533600000000002 2024-09-18T11:32:25.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.777 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:25.778 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011842+0000", 2024-09-18T11:32:25.778 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3180000000000002e-05 2024-09-18T11:32:25.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.778 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:25.778 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011892+0000", 2024-09-18T11:32:25.778 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0618000000000002e-05 2024-09-18T11:32:25.778 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.778 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:25.778 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.778 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2953 4.a 4:588debc2:::smithi11939506-18:head [assert-version v113,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:25.778 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:32:25.778 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 61.693771763999997, 2024-09-18T11:32:25.778 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083931279999999997, 2024-09-18T11:32:25.779 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:25.779 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:25.779 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:25.779 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:25.779 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:25.779 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2953 2024-09-18T11:32:25.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.779 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:25.779 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.779 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:25.779 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:32:25.779 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.779 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.779 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:25.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:32:25.780 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:25.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026796+0000", 2024-09-18T11:32:25.780 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:25.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.780 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:25.780 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026806+0000", 2024-09-18T11:32:25.780 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9529999999999994e-06 2024-09-18T11:32:25.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.781 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:25.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026824+0000", 2024-09-18T11:32:25.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7901999999999999e-05 2024-09-18T11:32:25.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.781 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:25.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026832+0000", 2024-09-18T11:32:25.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8450000000000003e-06 2024-09-18T11:32:25.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.781 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.781 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026883+0000", 2024-09-18T11:32:25.781 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0883000000000001e-05 2024-09-18T11:32:25.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:25.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026980+0000", 2024-09-18T11:32:25.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7526000000000001e-05 2024-09-18T11:32:25.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:32:25.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032722+0000", 2024-09-18T11:32:25.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057414689999999999 2024-09-18T11:32:25.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.782 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:25.782 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.034230+0000", 2024-09-18T11:32:25.782 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001508035 2024-09-18T11:32:25.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.783 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:25.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035090+0000", 2024-09-18T11:32:25.783 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00086047300000000005 2024-09-18T11:32:25.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.783 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:25.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035147+0000", 2024-09-18T11:32:25.783 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6598000000000003e-05 2024-09-18T11:32:25.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.783 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:25.783 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035191+0000", 2024-09-18T11:32:25.783 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4747999999999999e-05 2024-09-18T11:32:25.784 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.784 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:25.784 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.784 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3105 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 3e=[3e,3d,3b,37,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-18T11:32:25.784 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:25.784 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 52.702359059999999, 2024-09-18T11:32:25.784 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011735906000000001, 2024-09-18T11:32:25.784 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:25.784 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:25.785 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:25.785 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:25.785 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:25.785 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3105 2024-09-18T11:32:25.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.785 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:25.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.785 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:25.785 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:25.785 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.785 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:25.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:25.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:25.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:25.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:25.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.786 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:25.786 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018212+0000", 2024-09-18T11:32:25.786 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6759999999999999e-06 2024-09-18T11:32:25.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:25.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018216+0000", 2024-09-18T11:32:25.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-06 2024-09-18T11:32:25.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:25.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018218+0000", 2024-09-18T11:32:25.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6110000000000001e-06 2024-09-18T11:32:25.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.787 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.787 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018246+0000", 2024-09-18T11:32:25.787 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7557000000000001e-05 2024-09-18T11:32:25.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.788 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:25.788 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018250+0000", 2024-09-18T11:32:25.788 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4299999999999999e-06 2024-09-18T11:32:25.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.788 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.788 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021364+0000", 2024-09-18T11:32:25.788 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031135389999999998 2024-09-18T11:32:25.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.788 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:25.788 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021494+0000", 2024-09-18T11:32:25.788 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000130333 2024-09-18T11:32:25.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.789 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:25.789 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.029947+0000", 2024-09-18T11:32:25.789 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084526150000000001 2024-09-18T11:32:25.789 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.789 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:25.789 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.789 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3323 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:25.789 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:32:25.789 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 43.603671016, 2024-09-18T11:32:25.789 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0088051689999999998, 2024-09-18T11:32:25.790 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:25.790 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:25.790 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:25.790 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:25.790 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:25.790 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3323 2024-09-18T11:32:25.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.790 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:25.790 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.790 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:25.790 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:32:25.790 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.791 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.791 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:25.791 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:32:25.791 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.791 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.791 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:25.791 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116897+0000", 2024-09-18T11:32:25.791 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:25.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.792 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:25.792 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116908+0000", 2024-09-18T11:32:25.792 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1511e-05 2024-09-18T11:32:25.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.792 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:25.792 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116925+0000", 2024-09-18T11:32:25.792 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7028e-05 2024-09-18T11:32:25.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.792 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:25.793 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116932+0000", 2024-09-18T11:32:25.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5919999999999997e-06 2024-09-18T11:32:25.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.793 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.793 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116988+0000", 2024-09-18T11:32:25.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5928999999999999e-05 2024-09-18T11:32:25.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.793 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:25.793 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116995+0000", 2024-09-18T11:32:25.793 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7409999999999992e-06 2024-09-18T11:32:25.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122390+0000", 2024-09-18T11:32:25.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053941889999999998 2024-09-18T11:32:25.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:25.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122453+0000", 2024-09-18T11:32:25.794 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3833999999999999e-05 2024-09-18T11:32:25.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.794 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:25.794 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122581+0000", 2024-09-18T11:32:25.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012751200000000001 2024-09-18T11:32:25.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:25.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125034+0000", 2024-09-18T11:32:25.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024525609999999998 2024-09-18T11:32:25.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.795 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:25.795 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125607+0000", 2024-09-18T11:32:25.795 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00057356499999999995 2024-09-18T11:32:25.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:25.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125655+0000", 2024-09-18T11:32:25.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8179000000000001e-05 2024-09-18T11:32:25.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.796 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:25.796 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125704+0000", 2024-09-18T11:32:25.796 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8950999999999999e-05 2024-09-18T11:32:25.796 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.796 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:25.796 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.797 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3324 4.5 4:a087a566:::smithi11939506-46:head [rollback 46] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:25.797 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:25.797 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 43.603628456999999, 2024-09-18T11:32:25.797 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010927767999999999, 2024-09-18T11:32:25.797 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:25.797 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:25.797 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:25.797 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:25.797 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:25.797 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3324 2024-09-18T11:32:25.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.797 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:25.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:25.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:25.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:25.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:25.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.798 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:25.798 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116941+0000", 2024-09-18T11:32:25.798 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:25.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:25.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116949+0000", 2024-09-18T11:32:25.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4660000000000002e-06 2024-09-18T11:32:25.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:25.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116960+0000", 2024-09-18T11:32:25.799 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1484999999999999e-05 2024-09-18T11:32:25.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.799 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:25.799 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116964+0000", 2024-09-18T11:32:25.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2810000000000002e-06 2024-09-18T11:32:25.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117053+0000", 2024-09-18T11:32:25.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8747999999999998e-05 2024-09-18T11:32:25.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.800 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:25.800 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117059+0000", 2024-09-18T11:32:25.800 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.246e-06 2024-09-18T11:32:25.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123604+0000", 2024-09-18T11:32:25.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065445319999999996 2024-09-18T11:32:25.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:25.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123663+0000", 2024-09-18T11:32:25.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8740999999999999e-05 2024-09-18T11:32:25.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.801 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:25.801 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123803+0000", 2024-09-18T11:32:25.801 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014049200000000001 2024-09-18T11:32:25.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:25.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127600+0000", 2024-09-18T11:32:25.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037971179999999999 2024-09-18T11:32:25.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:25.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127814+0000", 2024-09-18T11:32:25.802 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021333600000000001 2024-09-18T11:32:25.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.802 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:25.802 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127847+0000", 2024-09-18T11:32:25.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3188999999999997e-05 2024-09-18T11:32:25.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.803 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:25.803 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127869+0000", 2024-09-18T11:32:25.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2515e-05 2024-09-18T11:32:25.803 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.803 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:25.803 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.803 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3325 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:25.803 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:25.803 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 43.603581271000003, 2024-09-18T11:32:25.803 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181030999999999, 2024-09-18T11:32:25.804 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:25.804 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:25.804 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:25.804 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:25.804 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:25.804 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3325 2024-09-18T11:32:25.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.804 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:25.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.804 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:25.804 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:25.804 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:25.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:25.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:25.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116988+0000", 2024-09-18T11:32:25.805 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:25.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.805 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:25.805 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116991+0000", 2024-09-18T11:32:25.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9890000000000001e-06 2024-09-18T11:32:25.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:25.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116998+0000", 2024-09-18T11:32:25.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5309999999999998e-06 2024-09-18T11:32:25.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:25.806 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117005+0000", 2024-09-18T11:32:25.806 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6420000000000001e-06 2024-09-18T11:32:25.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.806 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117091+0000", 2024-09-18T11:32:25.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6835999999999993e-05 2024-09-18T11:32:25.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:25.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117097+0000", 2024-09-18T11:32:25.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:32:25.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.807 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.807 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124717+0000", 2024-09-18T11:32:25.807 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076204899999999997 2024-09-18T11:32:25.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:25.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124773+0000", 2024-09-18T11:32:25.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5470999999999998e-05 2024-09-18T11:32:25.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:25.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124877+0000", 2024-09-18T11:32:25.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010426500000000001 2024-09-18T11:32:25.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.808 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:25.808 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127901+0000", 2024-09-18T11:32:25.808 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030243449999999999 2024-09-18T11:32:25.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:25.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128115+0000", 2024-09-18T11:32:25.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021355300000000001 2024-09-18T11:32:25.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:25.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128145+0000", 2024-09-18T11:32:25.809 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0290999999999998e-05 2024-09-18T11:32:25.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.809 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:25.809 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128170+0000", 2024-09-18T11:32:25.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4811e-05 2024-09-18T11:32:25.810 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.810 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:25.810 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.810 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3474 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1872245~637761] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:25.810 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:25.810 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 39.276804593999998, 2024-09-18T11:32:25.810 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0092038580000000005, 2024-09-18T11:32:25.810 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:25.810 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:25.810 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:25.810 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:25.811 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:25.811 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3474 2024-09-18T11:32:25.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.811 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:25.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:25.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:25.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.811 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:25.811 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:25.811 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.812 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:25.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443764+0000", 2024-09-18T11:32:25.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:25.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.812 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:25.812 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444359+0000", 2024-09-18T11:32:25.812 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059519 2024-09-18T11:32:25.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.812 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.812 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:25.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444374+0000", 2024-09-18T11:32:25.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4949e-05 2024-09-18T11:32:25.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:25.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444382+0000", 2024-09-18T11:32:25.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1369999999999996e-06 2024-09-18T11:32:25.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.813 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.813 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444428+0000", 2024-09-18T11:32:25.813 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6783000000000003e-05 2024-09-18T11:32:25.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:25.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444507+0000", 2024-09-18T11:32:25.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8415000000000001e-05 2024-09-18T11:32:25.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:25.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444638+0000", 2024-09-18T11:32:25.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013129 2024-09-18T11:32:25.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.814 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:25.814 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.449059+0000", 2024-09-18T11:32:25.814 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004420719 2024-09-18T11:32:25.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.815 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:25.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452909+0000", 2024-09-18T11:32:25.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003850773 2024-09-18T11:32:25.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.815 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:25.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452944+0000", 2024-09-18T11:32:25.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-05 2024-09-18T11:32:25.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.815 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:25.815 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452969+0000", 2024-09-18T11:32:25.815 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5795999999999998e-05 2024-09-18T11:32:25.816 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.816 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:25.816 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.816 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3475 4.5 4:a56ea1fb:::smithi11939506-48:head [setxattr _header (57) in=7b,truncate 3042100] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:25.816 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:32:25.816 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 39.276175879, 2024-09-18T11:32:25.816 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086700660000000006, 2024-09-18T11:32:25.816 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:25.816 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:25.816 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:25.816 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:25.817 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:25.817 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3475 2024-09-18T11:32:25.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.817 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:25.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:25.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:32:25.817 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.817 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.817 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:25.817 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:32:25.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:25.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444393+0000", 2024-09-18T11:32:25.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:25.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:25.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444398+0000", 2024-09-18T11:32:25.818 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4909999999999999e-06 2024-09-18T11:32:25.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.818 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:25.818 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444405+0000", 2024-09-18T11:32:25.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5329999999999997e-06 2024-09-18T11:32:25.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:25.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444410+0000", 2024-09-18T11:32:25.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2470000000000002e-06 2024-09-18T11:32:25.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.819 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446046+0000", 2024-09-18T11:32:25.819 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001636722 2024-09-18T11:32:25.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.819 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:25.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446121+0000", 2024-09-18T11:32:25.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4604000000000006e-05 2024-09-18T11:32:25.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:25.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446274+0000", 2024-09-18T11:32:25.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015350300000000001 2024-09-18T11:32:25.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.820 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:25.820 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452884+0000", 2024-09-18T11:32:25.820 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066096439999999996 2024-09-18T11:32:25.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:25.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453000+0000", 2024-09-18T11:32:25.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000116204 2024-09-18T11:32:25.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:25.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453032+0000", 2024-09-18T11:32:25.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1327e-05 2024-09-18T11:32:25.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.821 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:25.821 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453064+0000", 2024-09-18T11:32:25.821 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2645000000000003e-05 2024-09-18T11:32:25.822 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.822 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:25.822 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.822 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3476 4.5 4:a56ea1fb:::smithi11939506-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:25.822 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:25.822 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 39.276136612000002, 2024-09-18T11:32:25.822 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087596800000000006, 2024-09-18T11:32:25.822 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:25.822 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:25.822 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:25.822 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:25.822 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:25.823 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3476 2024-09-18T11:32:25.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.823 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:25.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:25.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:25.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.823 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:25.823 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:25.823 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:25.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:25.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:25.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:25.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444436+0000", 2024-09-18T11:32:25.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9330000000000001e-06 2024-09-18T11:32:25.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.824 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:25.824 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444443+0000", 2024-09-18T11:32:25.824 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9369999999999998e-06 2024-09-18T11:32:25.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:25.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444447+0000", 2024-09-18T11:32:25.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1309999999999999e-06 2024-09-18T11:32:25.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446599+0000", 2024-09-18T11:32:25.825 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021521190000000001 2024-09-18T11:32:25.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.825 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:25.825 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446675+0000", 2024-09-18T11:32:25.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6099000000000004e-05 2024-09-18T11:32:25.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453088+0000", 2024-09-18T11:32:25.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006413307 2024-09-18T11:32:25.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:25.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453132+0000", 2024-09-18T11:32:25.826 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3548999999999999e-05 2024-09-18T11:32:25.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.826 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:25.826 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453193+0000", 2024-09-18T11:32:25.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1116000000000002e-05 2024-09-18T11:32:25.827 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.827 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:25.827 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.827 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3928 4.5 4:af4ba75a:::smithi11939506-37:head [sparse-read 0~1543958] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:25.827 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:25.827 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.701339701999999, 2024-09-18T11:32:25.827 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080807250000000004, 2024-09-18T11:32:25.827 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:25.827 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:25.827 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:25.827 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:25.827 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:25.828 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3928 2024-09-18T11:32:25.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.828 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:25.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:25.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:25.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.828 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:25.828 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:25.828 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.828 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:25.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:25.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:25.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:25.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019232+0000", 2024-09-18T11:32:25.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1050000000000002e-06 2024-09-18T11:32:25.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.829 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:25.829 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019236+0000", 2024-09-18T11:32:25.829 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7910000000000002e-06 2024-09-18T11:32:25.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:25.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019238+0000", 2024-09-18T11:32:25.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4339999999999999e-06 2024-09-18T11:32:25.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.830 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019326+0000", 2024-09-18T11:32:25.830 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8097999999999996e-05 2024-09-18T11:32:25.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.830 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.830 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:25.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019330+0000", 2024-09-18T11:32:25.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9349999999999996e-06 2024-09-18T11:32:25.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027035+0000", 2024-09-18T11:32:25.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0077047269999999998 2024-09-18T11:32:25.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.831 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:25.831 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027110+0000", 2024-09-18T11:32:25.831 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4443000000000005e-05 2024-09-18T11:32:25.831 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.832 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:25.832 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027311+0000", 2024-09-18T11:32:25.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000201528 2024-09-18T11:32:25.832 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.832 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:25.832 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.832 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3929 4.5 4:af4ba75a:::smithi11939506-37:head [sparse-read 0~1543958] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:25.832 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:25.832 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.701309149, 2024-09-18T11:32:25.832 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084586699999999997, 2024-09-18T11:32:25.833 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:25.833 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:25.833 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:25.833 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:25.833 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:25.833 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3929 2024-09-18T11:32:25.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.833 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:25.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.833 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:25.833 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:25.833 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:25.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:25.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:25.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:25.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:25.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:25.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.834 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:25.834 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019262+0000", 2024-09-18T11:32:25.834 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8029999999999999e-06 2024-09-18T11:32:25.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:25.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019267+0000", 2024-09-18T11:32:25.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1119999999999998e-06 2024-09-18T11:32:25.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:25.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019269+0000", 2024-09-18T11:32:25.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4269999999999998e-06 2024-09-18T11:32:25.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.835 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.835 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019344+0000", 2024-09-18T11:32:25.835 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4979000000000002e-05 2024-09-18T11:32:25.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:25.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019348+0000", 2024-09-18T11:32:25.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3930000000000001e-06 2024-09-18T11:32:25.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:25.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027385+0000", 2024-09-18T11:32:25.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080368409999999994 2024-09-18T11:32:25.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.836 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:25.836 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027463+0000", 2024-09-18T11:32:25.836 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7316000000000003e-05 2024-09-18T11:32:25.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:25.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:25.837 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:25.837 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027720+0000", 2024-09-18T11:32:25.837 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000257068 2024-09-18T11:32:25.837 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.837 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:25.837 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.837 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:25.837 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:25.837 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:25.839 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:32:25.839 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:25.837+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:25.940 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:25.938+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:25.977 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:25.978 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:25.978 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:25.978 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:25.986 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:32:26.025 INFO:tasks.rados.rados.0.smithi119.stdout:1814: done (11 left) 2024-09-18T11:32:26.025 INFO:tasks.rados.rados.0.smithi119.stdout:1816: done (10 left) 2024-09-18T11:32:26.026 INFO:tasks.rados.rados.0.smithi119.stdout:1817: done (9 left) 2024-09-18T11:32:26.026 INFO:tasks.rados.rados.0.smithi119.stdout:1818: done (8 left) 2024-09-18T11:32:26.026 INFO:tasks.rados.rados.0.smithi119.stdout:1819: done (7 left) 2024-09-18T11:32:26.026 INFO:tasks.rados.rados.0.smithi119.stdout:1820: done (6 left) 2024-09-18T11:32:26.026 INFO:tasks.rados.rados.0.smithi119.stdout:1821: done (5 left) 2024-09-18T11:32:26.026 INFO:tasks.rados.rados.0.smithi119.stdout:1822: done (4 left) 2024-09-18T11:32:26.026 INFO:tasks.rados.rados.0.smithi119.stdout:1823: done (3 left) 2024-09-18T11:32:26.026 INFO:tasks.rados.rados.0.smithi119.stdout:1824: done (2 left) 2024-09-18T11:32:26.026 INFO:tasks.rados.rados.0.smithi119.stdout:1826: done (1 left) 2024-09-18T11:32:26.026 INFO:tasks.rados.rados.0.smithi119.stdout:1827: done (0 left) 2024-09-18T11:32:26.026 INFO:tasks.rados.rados.0.smithi119.stdout:1828: read oid 27 snap -1 2024-09-18T11:32:26.026 INFO:tasks.rados.rados.0.smithi119.stdout:1828: expect (ObjNum 367 snap 57 seq_num 367) 2024-09-18T11:32:26.027 INFO:teuthology.orchestra.run.smithi119.stdout:moved 3 / 58 (5.17241%) 2024-09-18T11:32:26.027 INFO:teuthology.orchestra.run.smithi119.stdout:avg 8.28571 2024-09-18T11:32:26.027 INFO:teuthology.orchestra.run.smithi119.stdout:stddev 3.88088 -> 3.41067 (expected baseline 2.66497) 2024-09-18T11:32:26.027 INFO:teuthology.orchestra.run.smithi119.stdout:min osd.2 with 0 -> 1 pgs (0 -> 0.12069 * mean) 2024-09-18T11:32:26.027 INFO:teuthology.orchestra.run.smithi119.stdout:max osd.1 with 14 -> 13 pgs (1.68966 -> 1.56897 * mean) 2024-09-18T11:32:26.027 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:32:26.027 INFO:teuthology.orchestra.run.smithi119.stdout:oload 110 2024-09-18T11:32:26.027 INFO:teuthology.orchestra.run.smithi119.stdout:max_change 0.05 2024-09-18T11:32:26.027 INFO:teuthology.orchestra.run.smithi119.stdout:max_change_osds 4 2024-09-18T11:32:26.028 INFO:teuthology.orchestra.run.smithi119.stdout:average_utilization 0.0040 2024-09-18T11:32:26.028 INFO:teuthology.orchestra.run.smithi119.stdout:overload_utilization 0.0044 2024-09-18T11:32:26.028 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 weight 0.2805 -> 0.3305 2024-09-18T11:32:26.028 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 weight 1.0000 -> 0.9500 2024-09-18T11:32:26.028 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 weight 1.0000 -> 0.9500 2024-09-18T11:32:26.028 INFO:tasks.rados.rados.0.smithi119.stdout:1829: write (excl) oid 2 current snap is 58 2024-09-18T11:32:26.028 INFO:tasks.rados.rados.0.smithi119.stdout:1829: seq_num 372 ranges {652204=777572,1952125=300079} 2024-09-18T11:32:26.029 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:26.025+0000 7f2d429e4640 -1 osd.1 154 handle_osd_map injecting map crc failure 2024-09-18T11:32:26.034 INFO:tasks.rados.rados.0.smithi119.stdout:1829: writing smithi11939506-2 from 652204 to 1429776 tid 1 2024-09-18T11:32:26.039 INFO:tasks.rados.rados.0.smithi119.stdout:1829: writing smithi11939506-2 from 1952125 to 2252204 tid 2 2024-09-18T11:32:26.039 INFO:tasks.rados.rados.0.smithi119.stdout:1830: read oid 42 snap -1 2024-09-18T11:32:26.039 INFO:tasks.rados.rados.0.smithi119.stdout:1830: expect (ObjNum 338 snap 48 seq_num 338) 2024-09-18T11:32:26.040 INFO:tasks.rados.rados.0.smithi119.stdout:1831: delete oid 37 current snap is 58 2024-09-18T11:32:26.040 INFO:tasks.rados.rados.0.smithi119.stdout:1828: expect (ObjNum 367 snap 57 seq_num 367) 2024-09-18T11:32:26.041 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-18T11:32:26.041 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-09-18T11:32:26.041 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:26.039+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:26.131 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:26.131 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:26.131 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:26.131 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:26.131 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:26.139 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:32:26.141 INFO:tasks.rados.rados.0.smithi119.stdout:1829: finishing write tid 1 to smithi11939506-2 2024-09-18T11:32:26.141 INFO:tasks.rados.rados.0.smithi119.stdout:1829: finishing write tid 2 to smithi11939506-2 2024-09-18T11:32:26.141 INFO:tasks.rados.rados.0.smithi119.stdout:1829: finishing write tid 3 to smithi11939506-2 2024-09-18T11:32:26.141 INFO:tasks.rados.rados.0.smithi119.stdout:1829: finishing write tid 4 to smithi11939506-2 2024-09-18T11:32:26.141 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 522 (ObjNum 372 snap 58 seq_num 372) dirty exists 2024-09-18T11:32:26.141 INFO:tasks.rados.rados.0.smithi119.stdout:1829: left oid 2 (ObjNum 372 snap 58 seq_num 372) 2024-09-18T11:32:26.141 INFO:tasks.rados.rados.0.smithi119.stdout:1830: expect (ObjNum 338 snap 48 seq_num 338) 2024-09-18T11:32:26.142 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:26.139+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:26.240 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:26.240 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:26.240 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:26.240 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:26.240 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.240 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:26.240 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:26.240 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 157.07970929999999, 2024-09-18T11:32:26.240 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:32:26.241 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:26.241 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:26.241 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:26.241 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:26.241 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:26.241 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:32:26.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.241 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:26.241 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.241 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:26.241 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:26.241 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.242 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.242 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.242 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:26.242 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:26.242 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.242 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.242 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.242 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:26.242 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:32:26.242 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:26.242 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.242 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.242 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:26.243 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:32:26.243 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:32:26.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.243 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.243 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:26.243 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:32:26.243 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:32:26.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.243 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.243 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:26.243 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:32:26.243 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:32:26.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.243 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.244 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:26.244 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:32:26.244 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:32:26.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.244 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.244 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:26.244 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:32:26.244 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:32:26.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.244 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.244 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:26.244 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:32:26.244 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:32:26.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.245 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.245 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:26.245 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:32:26.245 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:32:26.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.245 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.245 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:26.245 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:32:26.245 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:32:26.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.245 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.245 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:26.245 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:32:26.245 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:32:26.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.246 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.246 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:26.246 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:32:26.246 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:32:26.246 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.246 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:26.246 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.246 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.246 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:26.246 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:26.246 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 157.07965436999999, 2024-09-18T11:32:26.246 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:32:26.247 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:26.247 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:26.247 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:26.247 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:26.247 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:26.247 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:32:26.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.247 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:26.247 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.247 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:26.247 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:26.247 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.248 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.248 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:26.248 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:26.248 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.248 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.248 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:26.248 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:26.248 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:26.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.248 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.248 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:26.248 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:32:26.249 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:32:26.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.249 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.249 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:26.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:32:26.249 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:32:26.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.249 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.249 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:26.249 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:32:26.249 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:32:26.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.249 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.250 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:26.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:32:26.250 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:32:26.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.250 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.250 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:26.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:32:26.250 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:32:26.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.250 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.250 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:26.250 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:32:26.251 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:32:26.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.251 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.251 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:26.251 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:32:26.251 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:32:26.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.251 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.251 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:26.251 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:32:26.251 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:32:26.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.251 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.252 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:26.252 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:32:26.252 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:32:26.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.252 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.252 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:26.252 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:32:26.252 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:32:26.252 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.252 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:26.252 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.252 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.252 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.252 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:26.253 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:26.253 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 157.07964439, 2024-09-18T11:32:26.253 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:32:26.253 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:26.253 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:26.253 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:26.253 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:26.253 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:26.253 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:32:26.253 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.253 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:26.253 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.253 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:26.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:26.254 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.254 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.254 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:26.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:26.254 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.254 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.254 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:26.254 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:26.254 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:26.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.255 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.255 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:26.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:32:26.255 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:32:26.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.255 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.255 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:26.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:32:26.255 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:32:26.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.255 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.255 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:26.255 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:32:26.256 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:32:26.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.256 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.256 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:26.256 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:32:26.256 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:32:26.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.256 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.256 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:26.256 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:32:26.256 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:32:26.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.256 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.257 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:26.257 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:32:26.257 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:32:26.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.257 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.257 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:26.257 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:32:26.257 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:32:26.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.257 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.257 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:26.257 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:32:26.257 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:32:26.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.258 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.259 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:26.259 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:32:26.259 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:32:26.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.260 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.260 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:26.260 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:32:26.260 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:32:26.260 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.260 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:26.260 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.260 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.260 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:26.260 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:26.261 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 157.07963279099999, 2024-09-18T11:32:26.261 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:32:26.261 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:26.261 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:26.261 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:26.261 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:26.261 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:26.261 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:32:26.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.261 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:26.261 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.261 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:26.261 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:26.261 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.262 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.262 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:26.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:26.262 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.262 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.262 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:26.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:26.262 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:26.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.262 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.262 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:26.262 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:32:26.262 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:32:26.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.263 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.263 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:26.263 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:32:26.263 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:32:26.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.263 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.263 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:26.263 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:32:26.263 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:32:26.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.263 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.263 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:26.263 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:32:26.263 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:32:26.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.264 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.264 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:26.264 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:32:26.264 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:32:26.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.264 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.264 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:26.264 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:32:26.264 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:32:26.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.264 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.264 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:26.264 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:32:26.264 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:32:26.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.265 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.265 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:26.265 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:32:26.265 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:32:26.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.265 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.265 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:26.265 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:32:26.265 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:32:26.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.265 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.265 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:26.266 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:32:26.266 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:32:26.266 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.266 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:26.266 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.266 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.266 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:26.266 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:26.266 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 157.07957630999999, 2024-09-18T11:32:26.266 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:32:26.266 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:26.266 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:26.267 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:26.267 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:26.267 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:26.267 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:32:26.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.267 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:26.267 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.267 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:26.267 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:26.267 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.267 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.267 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:26.267 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:26.268 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.268 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.268 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:26.268 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:32:26.268 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:26.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.268 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.268 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:26.268 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:32:26.268 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:32:26.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.269 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.269 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:26.269 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:32:26.269 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:32:26.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.269 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.269 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:26.269 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:32:26.269 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:32:26.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.269 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.269 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:26.270 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:32:26.270 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:32:26.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.270 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.270 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:26.270 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:32:26.270 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:32:26.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.270 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.270 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:26.270 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:32:26.270 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:32:26.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.271 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.271 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:26.271 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:32:26.271 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:32:26.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.271 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.271 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:26.271 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:32:26.271 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:32:26.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.271 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.271 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:26.271 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:32:26.272 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:32:26.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.272 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.272 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:26.272 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:32:26.272 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:32:26.272 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.272 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:26.272 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.272 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.272 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:26.272 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:26.273 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 157.07956248599999, 2024-09-18T11:32:26.273 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:32:26.273 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:26.273 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:26.273 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:26.273 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:26.273 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:26.273 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:32:26.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.273 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:26.273 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.273 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:26.273 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:26.273 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.274 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.274 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:26.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:26.274 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.274 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.274 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:26.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:26.274 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:26.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.274 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.274 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:26.274 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:32:26.274 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:32:26.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:32:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:32:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:32:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:32:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:32:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:32:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:32:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:32:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:32:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:32:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:32:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:32:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:32:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:32:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:32:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:32:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:32:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:32:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 156.663383579, 2024-09-18T11:32:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:32:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:32:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:32:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:32:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:32:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:32:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:32:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:32:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:32:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:32:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:32:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:32:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:32:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:32:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:32:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:32:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:32:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:32:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:32:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:32:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:32:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:32:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 156.66336027899999, 2024-09-18T11:32:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:32:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:32:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:32:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.287 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:26.287 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:32:26.287 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:32:26.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.287 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:26.287 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:32:26.287 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:32:26.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.287 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:26.288 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:32:26.288 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:32:26.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.288 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.288 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:26.288 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:32:26.288 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:32:26.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.288 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.288 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:26.288 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:32:26.288 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:32:26.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.289 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.289 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:26.289 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:32:26.289 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:32:26.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.289 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.289 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:26.289 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:32:26.289 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:32:26.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.289 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.289 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:26.289 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:32:26.290 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:32:26.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.290 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.290 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:26.290 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:32:26.290 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:32:26.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.290 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.290 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:26.290 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:32:26.290 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:32:26.290 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.291 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:26.291 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.291 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:26.291 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:26.291 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 156.66327466999999, 2024-09-18T11:32:26.291 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:32:26.291 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:26.291 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:26.291 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:26.291 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:26.291 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:26.291 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:32:26.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.292 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:26.292 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.292 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:26.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:26.292 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.292 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.292 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:26.292 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:26.292 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.293 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:26.293 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:26.293 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:26.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.293 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:26.293 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:32:26.293 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:32:26.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.293 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:26.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:32:26.294 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:32:26.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.294 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:26.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:32:26.294 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:32:26.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.294 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:26.294 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:32:26.294 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:32:26.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.295 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:26.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:32:26.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:32:26.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.295 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:26.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:32:26.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:32:26.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.295 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:26.295 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:32:26.295 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:32:26.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.296 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:26.296 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:32:26.296 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:32:26.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.296 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:26.296 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:32:26.296 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:32:26.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.297 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:26.297 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:32:26.297 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:32:26.297 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.297 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:26.297 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.297 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:26.297 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:26.297 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 156.66326370499999, 2024-09-18T11:32:26.297 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:32:26.297 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:26.297 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:26.298 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:26.298 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:26.298 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:26.298 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:32:26.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.298 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:26.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.298 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:26.298 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:26.298 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.299 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:26.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:26.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.299 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:26.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:26.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:26.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.299 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:26.299 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:32:26.299 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:32:26.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:26.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:32:26.300 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:32:26.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:26.300 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:32:26.300 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:32:26.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.300 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:26.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:32:26.301 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:32:26.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.301 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:26.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:32:26.301 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:32:26.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.301 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:26.301 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:32:26.301 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:32:26.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.302 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:26.302 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:32:26.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:32:26.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.302 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:26.302 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:32:26.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:32:26.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.302 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:26.302 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:32:26.302 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:32:26.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.303 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:26.303 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:32:26.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:32:26.303 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.303 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:26.303 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.303 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:26.303 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:26.303 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 156.66325253700001, 2024-09-18T11:32:26.303 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:32:26.303 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:26.304 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:26.304 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:26.304 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:26.304 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:26.304 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:32:26.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.304 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:26.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.304 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:26.304 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:26.304 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:26.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:26.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:26.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:26.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:26.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.305 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:26.305 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:32:26.305 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:32:26.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:26.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:32:26.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:32:26.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:26.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:32:26.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:32:26.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.306 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:26.306 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:32:26.306 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:32:26.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:26.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:32:26.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:32:26.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:26.307 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:32:26.307 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:32:26.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.307 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:26.308 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:32:26.308 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:32:26.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.308 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.308 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:26.308 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:32:26.308 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:32:26.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.308 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.308 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:26.308 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:32:26.308 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:32:26.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.309 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:26.309 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:32:26.309 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:32:26.309 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.309 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:26.309 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.309 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:26.309 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:26.309 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 156.66318720800001, 2024-09-18T11:32:26.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:32:26.310 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:26.310 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:26.310 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:26.310 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:26.310 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:26.310 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:32:26.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.310 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:26.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.310 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:26.310 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:26.310 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:26.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:26.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:26.311 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:32:26.311 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:26.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.311 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:26.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:32:26.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:32:26.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.312 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:26.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:32:26.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:32:26.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.312 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:26.312 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:32:26.312 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:32:26.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.313 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:26.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:32:26.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:32:26.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.313 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:26.313 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:32:26.313 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:32:26.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.313 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:26.314 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:32:26.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:32:26.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.314 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:26.314 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:32:26.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:32:26.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.314 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:26.314 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:32:26.314 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:32:26.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:26.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:32:26.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:32:26.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.315 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:26.315 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:32:26.315 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:32:26.315 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.315 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:26.315 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.316 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:26.316 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:26.316 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 156.65302523, 2024-09-18T11:32:26.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:32:26.316 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:26.316 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:26.316 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:26.316 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:26.316 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:26.316 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:32:26.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.316 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:26.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:26.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:26.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:26.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:26.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.317 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:26.317 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:32:26.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:26.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:26.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:32:26.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:32:26.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:26.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:32:26.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:32:26.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:26.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:32:26.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:32:26.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:26.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:32:26.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:32:26.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:26.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:32:26.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:32:26.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:26.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:32:26.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:32:26.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:26.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:32:26.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:32:26.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:26.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:32:26.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:32:26.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:26.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:32:26.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:32:26.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:26.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:32:26.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:32:26.321 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.321 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:26.321 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.322 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:26.322 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:26.322 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 156.65293275100001, 2024-09-18T11:32:26.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:32:26.322 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:26.322 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:26.322 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:26.322 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:26.322 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:26.322 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:32:26.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.323 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:26.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:26.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:26.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:26.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:26.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.323 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:26.323 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:26.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:26.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:26.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:32:26.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:32:26.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:26.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:32:26.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:32:26.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:26.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:32:26.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:32:26.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:26.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:32:26.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:32:26.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:26.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:32:26.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:32:26.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:26.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:32:26.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:32:26.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:26.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:32:26.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:32:26.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:26.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:32:26.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:32:26.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:26.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:32:26.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:32:26.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:26.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:32:26.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:32:26.328 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.328 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:26.328 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.328 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:1788 4.8 e60/22 4:1898b15d:::smithi11939506-25:head v 60'12, mlcod=58'9)", 2024-09-18T11:32:26.328 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:32:26.328 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.320527262, 2024-09-18T11:32:26.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0058348599999999999, 2024-09-18T11:32:26.329 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:26.329 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:26.329 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:26.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:26.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:32:26.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:26.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916951+0000", 2024-09-18T11:32:26.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:26.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916949+0000", 2024-09-18T11:32:26.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:26.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:26.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917602+0000", 2024-09-18T11:32:26.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00065337100000000003 2024-09-18T11:32:26.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:26.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917621+0000", 2024-09-18T11:32:26.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9287e-05 2024-09-18T11:32:26.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:26.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917629+0000", 2024-09-18T11:32:26.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3780000000000003e-06 2024-09-18T11:32:26.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:26.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917677+0000", 2024-09-18T11:32:26.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8068000000000001e-05 2024-09-18T11:32:26.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:26.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.917699+0000", 2024-09-18T11:32:26.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2149000000000001e-05 2024-09-18T11:32:26.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:26.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922747+0000", 2024-09-18T11:32:26.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0050477609999999996 2024-09-18T11:32:26.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:26.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.922785+0000", 2024-09-18T11:32:26.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8726000000000002e-05 2024-09-18T11:32:26.333 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.334 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:26.334 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.334 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3209 4.0 e109/22 4:019b6e09:::smithi11939506-40:head v 109'149, mlcod=106'146)", 2024-09-18T11:32:26.334 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:32:26.334 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 49.138091899999999, 2024-09-18T11:32:26.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063064039999999998, 2024-09-18T11:32:26.334 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:26.334 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:26.334 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:26.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:26.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:32:26.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:26.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:32:26.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:26.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099385+0000", 2024-09-18T11:32:26.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:26.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:26.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099951+0000", 2024-09-18T11:32:26.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056606600000000005 2024-09-18T11:32:26.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:26.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099964+0000", 2024-09-18T11:32:26.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2918999999999999e-05 2024-09-18T11:32:26.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.336 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:26.336 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099972+0000", 2024-09-18T11:32:26.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2300000000000002e-06 2024-09-18T11:32:26.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:26.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.100026+0000", 2024-09-18T11:32:26.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4456999999999997e-05 2024-09-18T11:32:26.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:26.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.100046+0000", 2024-09-18T11:32:26.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0417000000000001e-05 2024-09-18T11:32:26.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:26.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.105653+0000", 2024-09-18T11:32:26.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005606214 2024-09-18T11:32:26.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:26.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.105692+0000", 2024-09-18T11:32:26.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9671e-05 2024-09-18T11:32:26.338 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.338 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:26.338 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.339 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:123 4.3 e132/67 4:cc46532a:::smithi11939506-33:30 v 132'171, mlcod=128'168)", 2024-09-18T11:32:26.339 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:26.339 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.209626126, 2024-09-18T11:32:26.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087627039999999996, 2024-09-18T11:32:26.339 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:26.339 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:26.339 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:26.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:26.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:26.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:26.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:26.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:26.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027851+0000", 2024-09-18T11:32:26.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:26.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:26.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027871+0000", 2024-09-18T11:32:26.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9749e-05 2024-09-18T11:32:26.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:26.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027882+0000", 2024-09-18T11:32:26.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1242e-05 2024-09-18T11:32:26.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:26.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027889+0000", 2024-09-18T11:32:26.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7649999999999996e-06 2024-09-18T11:32:26.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:26.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027929+0000", 2024-09-18T11:32:26.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9984999999999997e-05 2024-09-18T11:32:26.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:26.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027951+0000", 2024-09-18T11:32:26.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2047000000000001e-05 2024-09-18T11:32:26.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:26.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.036565+0000", 2024-09-18T11:32:26.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086135110000000008 2024-09-18T11:32:26.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.342 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:26.342 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.036614+0000", 2024-09-18T11:32:26.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9747000000000003e-05 2024-09-18T11:32:26.343 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.343 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:26.343 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.343 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:124 4.3 e132/67 4:ca2b2309:::smithi11939506-44:30 v 132'173, mlcod=128'168)", 2024-09-18T11:32:26.343 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:26.343 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.208934144000001, 2024-09-18T11:32:26.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0088947309999999995, 2024-09-18T11:32:26.343 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:26.343 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:26.343 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:26.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:26.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:26.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:26.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:26.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:26.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028543+0000", 2024-09-18T11:32:26.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:26.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:26.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028551+0000", 2024-09-18T11:32:26.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8720000000000002e-06 2024-09-18T11:32:26.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:26.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028562+0000", 2024-09-18T11:32:26.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1039000000000001e-05 2024-09-18T11:32:26.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:26.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028569+0000", 2024-09-18T11:32:26.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0350000000000004e-06 2024-09-18T11:32:26.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:26.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028848+0000", 2024-09-18T11:32:26.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00027935600000000003 2024-09-18T11:32:26.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:26.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028868+0000", 2024-09-18T11:32:26.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9947999999999999e-05 2024-09-18T11:32:26.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:26.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037405+0000", 2024-09-18T11:32:26.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085369920000000002 2024-09-18T11:32:26.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:26.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037438+0000", 2024-09-18T11:32:26.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3244000000000002e-05 2024-09-18T11:32:26.347 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.347 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:26.347 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.348 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:142 4.3 e141/67 4:ca2b2309:::smithi11939506-44:5d v 141'197, mlcod=140'192)", 2024-09-18T11:32:26.348 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.297859+0000", 2024-09-18T11:32:26.348 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.939619134000001, 2024-09-18T11:32:26.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063063160000000002, 2024-09-18T11:32:26.348 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:26.348 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:26.348 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:26.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:26.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297859+0000", 2024-09-18T11:32:26.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:26.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297859+0000", 2024-09-18T11:32:26.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:26.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297858+0000", 2024-09-18T11:32:26.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:26.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:26.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297863+0000", 2024-09-18T11:32:26.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9450000000000001e-06 2024-09-18T11:32:26.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:26.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297870+0000", 2024-09-18T11:32:26.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.43e-06 2024-09-18T11:32:26.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:26.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297875+0000", 2024-09-18T11:32:26.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8439999999999997e-06 2024-09-18T11:32:26.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:26.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297970+0000", 2024-09-18T11:32:26.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4968000000000003e-05 2024-09-18T11:32:26.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:26.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297991+0000", 2024-09-18T11:32:26.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0726000000000001e-05 2024-09-18T11:32:26.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:26.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.304127+0000", 2024-09-18T11:32:26.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061356700000000002 2024-09-18T11:32:26.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:26.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.304165+0000", 2024-09-18T11:32:26.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8305e-05 2024-09-18T11:32:26.352 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.352 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:26.352 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.352 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:154 4.3 e147/67 4:ca2b2309:::smithi11939506-44:5d v 147'216, mlcod=145'210)", 2024-09-18T11:32:26.352 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:26.352 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.9001476950000002, 2024-09-18T11:32:26.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066667829999999999, 2024-09-18T11:32:26.353 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:26.353 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:26.353 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:26.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:26.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:26.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:26.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:26.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:26.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:26.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:26.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:26.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:26.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337335+0000", 2024-09-18T11:32:26.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1479999999999997e-06 2024-09-18T11:32:26.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:26.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337344+0000", 2024-09-18T11:32:26.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5640000000000002e-06 2024-09-18T11:32:26.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:26.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337349+0000", 2024-09-18T11:32:26.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9470000000000002e-06 2024-09-18T11:32:26.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:26.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337410+0000", 2024-09-18T11:32:26.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0448999999999997e-05 2024-09-18T11:32:26.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.355 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:26.355 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337425+0000", 2024-09-18T11:32:26.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.504e-05 2024-09-18T11:32:26.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:26.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.343972+0000", 2024-09-18T11:32:26.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065474169999999998 2024-09-18T11:32:26.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:26.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:26.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:26.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.343997+0000", 2024-09-18T11:32:26.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4672999999999999e-05 2024-09-18T11:32:26.356 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.356 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:26.356 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.356 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:26.357 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:26.357 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:26.357 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-09-18T11:32:26.344+0000 7fd482ca8640 -1 mgr.server reply reply (1) Operation not permitted no change 2024-09-18T11:32:26.358 INFO:teuthology.orchestra.run.smithi119.stdout:moved 0 / 58 (0%) 2024-09-18T11:32:26.358 INFO:teuthology.orchestra.run.smithi119.stdout:avg 8.28571 2024-09-18T11:32:26.358 INFO:teuthology.orchestra.run.smithi119.stdout:stddev 3.41067 -> 3.41067 (expected baseline 2.66497) 2024-09-18T11:32:26.358 INFO:teuthology.orchestra.run.smithi119.stdout:min osd.2 with 1 -> 1 pgs (0.12069 -> 0.12069 * mean) 2024-09-18T11:32:26.358 INFO:teuthology.orchestra.run.smithi119.stdout:max osd.1 with 13 -> 13 pgs (1.56897 -> 1.56897 * mean) 2024-09-18T11:32:26.359 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:32:26.359 INFO:teuthology.orchestra.run.smithi119.stdout:oload 1000 2024-09-18T11:32:26.359 INFO:teuthology.orchestra.run.smithi119.stdout:max_change 1 2024-09-18T11:32:26.359 INFO:teuthology.orchestra.run.smithi119.stdout:max_change_osds 4 2024-09-18T11:32:26.359 INFO:teuthology.orchestra.run.smithi119.stdout:average_utilization 0.0040 2024-09-18T11:32:26.359 INFO:teuthology.orchestra.run.smithi119.stdout:overload_utilization 0.0399 2024-09-18T11:32:26.359 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 weight 0.3305 -> 0.4120 2024-09-18T11:32:26.360 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:32:26.360 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:26.356+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:26.361 INFO:teuthology.orchestra.run.smithi119.stderr:no change 2024-09-18T11:32:26.361 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-18T11:32:26.361 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-09-18T11:32:26.417 INFO:tasks.rados.rados.0.smithi119.stdout:1828: done (3 left) 2024-09-18T11:32:26.417 INFO:tasks.rados.rados.0.smithi119.stdout:1829: done (2 left) 2024-09-18T11:32:26.417 INFO:tasks.rados.rados.0.smithi119.stdout:1830: done (1 left) 2024-09-18T11:32:26.417 INFO:tasks.rados.rados.0.smithi119.stdout:1831: done (0 left) 2024-09-18T11:32:26.417 INFO:tasks.rados.rados.0.smithi119.stdout:1832: read oid 31 snap -1 2024-09-18T11:32:26.417 INFO:tasks.rados.rados.0.smithi119.stdout:1832: expect (ObjNum 345 snap 51 seq_num 345) 2024-09-18T11:32:26.417 INFO:tasks.rados.rados.0.smithi119.stdout:1833: snap_remove snap 44 2024-09-18T11:32:26.459 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:26.458+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:26.503 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:26.503 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:26.504 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:26.504 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:26.513 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:32:26.560 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:26.559+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:26.612 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:26.612 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:26.612 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:26.612 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:26.613 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:26.621 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:32:26.662 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:26.661+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:26.758 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:26.759 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:26.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:26.759 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [] 2024-09-18T11:32:26.759 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:26.761 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:26.760+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:26.767 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:32:26.863 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:26.862+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:26.864 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:26.864 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:26.864 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:26.864 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:26.873 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:32:26.963 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:26.962+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:26.970 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:26.970 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:26.970 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:26.970 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:26.970 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:26.979 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:32:27.030 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:27.029+0000 7fcb662ad640 -1 osd.7 155 handle_osd_map injecting map crc failure 2024-09-18T11:32:27.032 INFO:tasks.rados.rados.0.smithi119.stdout:1833: done (1 left) 2024-09-18T11:32:27.032 INFO:tasks.rados.rados.0.smithi119.stdout:1834: read oid 5 snap -1 2024-09-18T11:32:27.032 INFO:tasks.rados.rados.0.smithi119.stdout:1834: expect deleted 2024-09-18T11:32:27.032 INFO:tasks.rados.rados.0.smithi119.stdout:1835: write oid 17 current snap is 58 2024-09-18T11:32:27.032 INFO:tasks.rados.rados.0.smithi119.stdout:1835: seq_num 374 ranges {685818=644423,2069089=216729} 2024-09-18T11:32:27.032 INFO:teuthology.orchestra.run.smithi119.stdout:moved 0 / 58 (0%) 2024-09-18T11:32:27.033 INFO:teuthology.orchestra.run.smithi119.stdout:avg 8.28571 2024-09-18T11:32:27.033 INFO:teuthology.orchestra.run.smithi119.stdout:stddev 3.41067 -> 3.41067 (expected baseline 2.66497) 2024-09-18T11:32:27.033 INFO:teuthology.orchestra.run.smithi119.stdout:min osd.2 with 1 -> 1 pgs (0.12069 -> 0.12069 * mean) 2024-09-18T11:32:27.033 INFO:teuthology.orchestra.run.smithi119.stdout:max osd.1 with 13 -> 13 pgs (1.56897 -> 1.56897 * mean) 2024-09-18T11:32:27.033 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:32:27.033 INFO:teuthology.orchestra.run.smithi119.stdout:oload 1000 2024-09-18T11:32:27.033 INFO:teuthology.orchestra.run.smithi119.stdout:max_change 3 2024-09-18T11:32:27.033 INFO:teuthology.orchestra.run.smithi119.stdout:max_change_osds 4 2024-09-18T11:32:27.033 INFO:teuthology.orchestra.run.smithi119.stdout:average_utilization 0.0040 2024-09-18T11:32:27.033 INFO:teuthology.orchestra.run.smithi119.stdout:overload_utilization 0.0399 2024-09-18T11:32:27.033 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 weight 0.3305 -> 0.4120 2024-09-18T11:32:27.039 INFO:tasks.rados.rados.0.smithi119.stdout:1835: writing smithi11939506-17 from 685818 to 1330241 tid 1 2024-09-18T11:32:27.041 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T11:32:27.045 INFO:tasks.rados.rados.0.smithi119.stdout:1835: writing smithi11939506-17 from 2069089 to 2285818 tid 2 2024-09-18T11:32:27.046 INFO:tasks.rados.rados.0.smithi119.stdout:1836: read oid 22 snap -1 2024-09-18T11:32:27.046 INFO:tasks.rados.rados.0.smithi119.stdout:1836: expect (ObjNum 298 snap 43 seq_num 298) 2024-09-18T11:32:27.046 INFO:tasks.rados.rados.0.smithi119.stdout:1837: set_chunk oid 39 target oid 2024-09-18T11:32:27.049 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-18T11:32:27.049 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-18T11:32:27.065 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:27.063+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:27.067 INFO:tasks.rados.rados.0.smithi119.stdout:1837: set_chunk oid 39 offset: 956416 length: 2203648 target oid 92886a4340d03039afb2c877fad76e3afe7a80ee04388d55256c386022cd5d63 offset: 0 snap -1 2024-09-18T11:32:27.067 INFO:tasks.rados.rados.0.smithi119.stdout:1838: tier_promote oid 12 2024-09-18T11:32:27.067 INFO:tasks.rados.rados.0.smithi119.stdout:1832: expect (ObjNum 345 snap 51 seq_num 345) 2024-09-18T11:32:27.096 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:27.096 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:27.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:27.096 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:27.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.096 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:262 4.9 e53/22 4:99feffe9:::smithi11939506-45:13 v 53'17, mlcod=51'14)", 2024-09-18T11:32:27.096 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:27.096 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.09447702200001, 2024-09-18T11:32:27.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013878679, 2024-09-18T11:32:27.096 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.096 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.096 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:27.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:27.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:27.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:27.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998997+0000", 2024-09-18T11:32:27.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1099999999999999e-06 2024-09-18T11:32:27.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999008+0000", 2024-09-18T11:32:27.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1321000000000001e-05 2024-09-18T11:32:27.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999015+0000", 2024-09-18T11:32:27.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3000000000000004e-06 2024-09-18T11:32:27.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999055+0000", 2024-09-18T11:32:27.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0058000000000002e-05 2024-09-18T11:32:27.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999070+0000", 2024-09-18T11:32:27.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4328000000000001e-05 2024-09-18T11:32:27.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012465+0000", 2024-09-18T11:32:27.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013395638 2024-09-18T11:32:27.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012870+0000", 2024-09-18T11:32:27.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040423400000000001 2024-09-18T11:32:27.100 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.100 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.100 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.101 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:205 4.b e53/22 4:dba5bd39:::smithi11939506-17:13 v 53'77, mlcod=51'74)", 2024-09-18T11:32:27.101 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:27.101 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.09442776100001, 2024-09-18T11:32:27.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013359586, 2024-09-18T11:32:27.101 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.101 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.101 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:27.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:27.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:27.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:27.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999048+0000", 2024-09-18T11:32:27.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3550000000000005e-06 2024-09-18T11:32:27.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999056+0000", 2024-09-18T11:32:27.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3259999999999996e-06 2024-09-18T11:32:27.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999063+0000", 2024-09-18T11:32:27.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6409999999999996e-06 2024-09-18T11:32:27.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999098+0000", 2024-09-18T11:32:27.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5203999999999997e-05 2024-09-18T11:32:27.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999115+0000", 2024-09-18T11:32:27.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6478999999999999e-05 2024-09-18T11:32:27.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012362+0000", 2024-09-18T11:32:27.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01324768 2024-09-18T11:32:27.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012400+0000", 2024-09-18T11:32:27.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.731e-05 2024-09-18T11:32:27.105 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.105 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.105 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.105 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1956 4.6 4:67c1765a:::smithi11939506-30:head [write 705927~558502] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:32:27.105 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:32:27.105 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.741458809999997, 2024-09-18T11:32:27.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011813239, 2024-09-18T11:32:27.105 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.106 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.106 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:27.106 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:27.106 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:27.106 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1956 2024-09-18T11:32:27.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.106 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:32:27.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:32:27.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352007+0000", 2024-09-18T11:32:27.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:27.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352229+0000", 2024-09-18T11:32:27.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222024 2024-09-18T11:32:27.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352251+0000", 2024-09-18T11:32:27.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2512000000000001e-05 2024-09-18T11:32:27.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352260+0000", 2024-09-18T11:32:27.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7339999999999997e-06 2024-09-18T11:32:27.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352307+0000", 2024-09-18T11:32:27.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7061999999999998e-05 2024-09-18T11:32:27.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352408+0000", 2024-09-18T11:32:27.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101216 2024-09-18T11:32:27.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:32:27.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352550+0000", 2024-09-18T11:32:27.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014136199999999999 2024-09-18T11:32:27.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:27.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357684+0000", 2024-09-18T11:32:27.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051342499999999999 2024-09-18T11:32:27.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:27.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363736+0000", 2024-09-18T11:32:27.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006052577 2024-09-18T11:32:27.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363778+0000", 2024-09-18T11:32:27.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1989000000000001e-05 2024-09-18T11:32:27.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363822+0000", 2024-09-18T11:32:27.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3894999999999999e-05 2024-09-18T11:32:27.111 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.111 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.111 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.111 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2937 4.7 4:e301aefd:::smithi11939506-19:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:27.111 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:27.111 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.058627447999996, 2024-09-18T11:32:27.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019287667000000001, 2024-09-18T11:32:27.111 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.111 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:27.111 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:27.111 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:27.112 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:27.112 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2937 2024-09-18T11:32:27.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.112 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:27.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:27.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:27.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:27.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034842+0000", 2024-09-18T11:32:27.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.372e-06 2024-09-18T11:32:27.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034846+0000", 2024-09-18T11:32:27.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3919999999999996e-06 2024-09-18T11:32:27.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034848+0000", 2024-09-18T11:32:27.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3539999999999998e-06 2024-09-18T11:32:27.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034893+0000", 2024-09-18T11:32:27.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4861000000000003e-05 2024-09-18T11:32:27.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034965+0000", 2024-09-18T11:32:27.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1860999999999995e-05 2024-09-18T11:32:27.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.054128+0000", 2024-09-18T11:32:27.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019163039 2024-09-18T11:32:27.115 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.115 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.115 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.115 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3453 4.f 4:f5d7df41:::smithi11939506-16:head [write 1813152~490434] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:27.115 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:32:27.115 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.07071552, 2024-09-18T11:32:27.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050685840000000003, 2024-09-18T11:32:27.116 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.116 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.116 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:27.116 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:27.116 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:27.116 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3453 2024-09-18T11:32:27.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.116 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:32:27.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:32:27.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022751+0000", 2024-09-18T11:32:27.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:27.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022871+0000", 2024-09-18T11:32:27.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000119781 2024-09-18T11:32:27.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022882+0000", 2024-09-18T11:32:27.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1627000000000001e-05 2024-09-18T11:32:27.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022887+0000", 2024-09-18T11:32:27.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2390000000000004e-06 2024-09-18T11:32:27.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023123+0000", 2024-09-18T11:32:27.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000236598 2024-09-18T11:32:27.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023207+0000", 2024-09-18T11:32:27.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4303999999999998e-05 2024-09-18T11:32:27.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:27.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023367+0000", 2024-09-18T11:32:27.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015983600000000001 2024-09-18T11:32:27.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:27.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073237+0000", 2024-09-18T11:32:27.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049869400000000001 2024-09-18T11:32:27.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:27.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073352+0000", 2024-09-18T11:32:27.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011579899999999999 2024-09-18T11:32:27.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073394+0000", 2024-09-18T11:32:27.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1445e-05 2024-09-18T11:32:27.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073438+0000", 2024-09-18T11:32:27.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4261999999999997e-05 2024-09-18T11:32:27.121 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.121 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.121 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.121 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3454 4.f 4:f5d7df41:::smithi11939506-16:head [setxattr _header (57) in=7b,truncate 2991679] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:27.121 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:32:27.121 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.070017077999999, 2024-09-18T11:32:27.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050365656000000002, 2024-09-18T11:32:27.121 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.122 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.122 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:27.122 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:27.122 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:27.122 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3454 2024-09-18T11:32:27.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.122 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:32:27.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:32:27.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023450+0000", 2024-09-18T11:32:27.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:27.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023456+0000", 2024-09-18T11:32:27.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8329999999999999e-06 2024-09-18T11:32:27.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023465+0000", 2024-09-18T11:32:27.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1130000000000007e-06 2024-09-18T11:32:27.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023470+0000", 2024-09-18T11:32:27.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9139999999999999e-06 2024-09-18T11:32:27.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072611+0000", 2024-09-18T11:32:27.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141643999999998 2024-09-18T11:32:27.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072729+0000", 2024-09-18T11:32:27.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117667 2024-09-18T11:32:27.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:27.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072887+0000", 2024-09-18T11:32:27.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015760000000000001 2024-09-18T11:32:27.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:27.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073558+0000", 2024-09-18T11:32:27.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00067171699999999995 2024-09-18T11:32:27.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:27.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073764+0000", 2024-09-18T11:32:27.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000205749 2024-09-18T11:32:27.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073790+0000", 2024-09-18T11:32:27.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5643000000000002e-05 2024-09-18T11:32:27.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073816+0000", 2024-09-18T11:32:27.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6588e-05 2024-09-18T11:32:27.127 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.127 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.127 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.127 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3455 4.f 4:f5d7df41:::smithi11939506-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:27.127 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:27.128 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.06998746, 2024-09-18T11:32:27.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050460466000000002, 2024-09-18T11:32:27.128 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.128 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:27.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:27.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:27.128 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:27.128 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3455 2024-09-18T11:32:27.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.128 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:27.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:27.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:27.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:27.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023483+0000", 2024-09-18T11:32:27.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.695e-06 2024-09-18T11:32:27.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023489+0000", 2024-09-18T11:32:27.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1079999999999998e-06 2024-09-18T11:32:27.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023492+0000", 2024-09-18T11:32:27.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.478e-06 2024-09-18T11:32:27.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073273+0000", 2024-09-18T11:32:27.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049780638000000002 2024-09-18T11:32:27.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:27.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073343+0000", 2024-09-18T11:32:27.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9770000000000005e-05 2024-09-18T11:32:27.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073846+0000", 2024-09-18T11:32:27.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050289499999999995 2024-09-18T11:32:27.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073886+0000", 2024-09-18T11:32:27.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0658999999999998e-05 2024-09-18T11:32:27.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073941+0000", 2024-09-18T11:32:27.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4642000000000003e-05 2024-09-18T11:32:27.132 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.132 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.133 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.133 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3456 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:27.133 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:27.133 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.069962394999997, 2024-09-18T11:32:27.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049859120999999999, 2024-09-18T11:32:27.133 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.133 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:27.133 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:27.134 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:27.134 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:27.134 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3456 2024-09-18T11:32:27.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.134 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:27.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:27.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:27.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:27.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023508+0000", 2024-09-18T11:32:27.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9490000000000001e-06 2024-09-18T11:32:27.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023513+0000", 2024-09-18T11:32:27.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3800000000000002e-06 2024-09-18T11:32:27.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023517+0000", 2024-09-18T11:32:27.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5300000000000001e-06 2024-09-18T11:32:27.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073262+0000", 2024-09-18T11:32:27.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049745194999999999 2024-09-18T11:32:27.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073365+0000", 2024-09-18T11:32:27.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102462 2024-09-18T11:32:27.137 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.137 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.137 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.137 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3457 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:27.138 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:32:27.138 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.069945298999997, 2024-09-18T11:32:27.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049928567, 2024-09-18T11:32:27.138 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.138 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:27.138 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:27.138 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:27.138 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:27.138 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3457 2024-09-18T11:32:27.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.138 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:32:27.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:32:27.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023522+0000", 2024-09-18T11:32:27.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:27.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023524+0000", 2024-09-18T11:32:27.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8500000000000001e-06 2024-09-18T11:32:27.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023529+0000", 2024-09-18T11:32:27.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7509999999999999e-06 2024-09-18T11:32:27.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023532+0000", 2024-09-18T11:32:27.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.163e-06 2024-09-18T11:32:27.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073386+0000", 2024-09-18T11:32:27.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049853899 2024-09-18T11:32:27.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073451+0000", 2024-09-18T11:32:27.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5197000000000006e-05 2024-09-18T11:32:27.141 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.142 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.142 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.142 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3458 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:27.142 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:27.142 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.069932741999999, 2024-09-18T11:32:27.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.05001659, 2024-09-18T11:32:27.142 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.142 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:27.142 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:27.143 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:27.143 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:27.143 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3458 2024-09-18T11:32:27.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.143 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:27.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:27.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:27.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:27.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023537+0000", 2024-09-18T11:32:27.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.64e-06 2024-09-18T11:32:27.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023543+0000", 2024-09-18T11:32:27.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3369999999999996e-06 2024-09-18T11:32:27.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023546+0000", 2024-09-18T11:32:27.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:32:27.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073466+0000", 2024-09-18T11:32:27.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049919719000000001 2024-09-18T11:32:27.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073552+0000", 2024-09-18T11:32:27.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6168999999999994e-05 2024-09-18T11:32:27.146 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.146 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.147 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.147 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(client.4590.0:3453 4.f e116/22 ondisk, result = 0)", 2024-09-18T11:32:27.147 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:32:27.147 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.068327658000001, 2024-09-18T11:32:27.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048099441, 2024-09-18T11:32:27.147 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.147 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:27.147 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:32:27.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:32:27.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025139+0000", 2024-09-18T11:32:27.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:27.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025144+0000", 2024-09-18T11:32:27.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4059999999999998e-06 2024-09-18T11:32:27.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072212+0000", 2024-09-18T11:32:27.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047068482000000002 2024-09-18T11:32:27.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072230+0000", 2024-09-18T11:32:27.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7632999999999999e-05 2024-09-18T11:32:27.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073224+0000", 2024-09-18T11:32:27.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099436899999999994 2024-09-18T11:32:27.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073231+0000", 2024-09-18T11:32:27.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-09-18T11:32:27.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073240+0000", 2024-09-18T11:32:27.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5699999999999993e-06 2024-09-18T11:32:27.151 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.151 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.151 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.151 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3930 4.b 4:d3ae8215:::smithi11939506-41:head [write 632029~547288] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:27.151 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:32:27.151 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.067411977999999, 2024-09-18T11:32:27.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018962711, 2024-09-18T11:32:27.151 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.151 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:27.152 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:27.152 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:27.152 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3930 2024-09-18T11:32:27.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.152 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:32:27.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:32:27.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026055+0000", 2024-09-18T11:32:27.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:27.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026334+0000", 2024-09-18T11:32:27.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000278931 2024-09-18T11:32:27.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026344+0000", 2024-09-18T11:32:27.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0716999999999999e-05 2024-09-18T11:32:27.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026350+0000", 2024-09-18T11:32:27.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7620000000000001e-06 2024-09-18T11:32:27.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028429+0000", 2024-09-18T11:32:27.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020787319999999998 2024-09-18T11:32:27.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028525+0000", 2024-09-18T11:32:27.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6638999999999994e-05 2024-09-18T11:32:27.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:27.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028720+0000", 2024-09-18T11:32:27.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000194525 2024-09-18T11:32:27.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:27.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.036996+0000", 2024-09-18T11:32:27.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082756140000000006 2024-09-18T11:32:27.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:27.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044952+0000", 2024-09-18T11:32:27.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566740000000004 2024-09-18T11:32:27.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044985+0000", 2024-09-18T11:32:27.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3081999999999999e-05 2024-09-18T11:32:27.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045019+0000", 2024-09-18T11:32:27.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3257999999999999e-05 2024-09-18T11:32:27.157 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.157 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.157 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.157 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3931 4.b 4:d3ae8215:::smithi11939506-41:head [write 1639134~749453] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:27.157 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.033118+0000", 2024-09-18T11:32:27.157 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.060350337999999, 2024-09-18T11:32:27.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012000667, 2024-09-18T11:32:27.157 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.157 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.158 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:27.158 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:27.158 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:27.158 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3931 2024-09-18T11:32:27.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.158 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033118+0000", 2024-09-18T11:32:27.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033118+0000", 2024-09-18T11:32:27.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033116+0000", 2024-09-18T11:32:27.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:27.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033408+0000", 2024-09-18T11:32:27.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029145000000000001 2024-09-18T11:32:27.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033420+0000", 2024-09-18T11:32:27.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2748999999999999e-05 2024-09-18T11:32:27.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033426+0000", 2024-09-18T11:32:27.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8889999999999999e-06 2024-09-18T11:32:27.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.034205+0000", 2024-09-18T11:32:27.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00077899000000000002 2024-09-18T11:32:27.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.034280+0000", 2024-09-18T11:32:27.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4871999999999997e-05 2024-09-18T11:32:27.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:27.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.034392+0000", 2024-09-18T11:32:27.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000111666 2024-09-18T11:32:27.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:27.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.037001+0000", 2024-09-18T11:32:27.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026091310000000002 2024-09-18T11:32:27.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:27.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045055+0000", 2024-09-18T11:32:27.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080539700000000006 2024-09-18T11:32:27.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045090+0000", 2024-09-18T11:32:27.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4984000000000003e-05 2024-09-18T11:32:27.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045118+0000", 2024-09-18T11:32:27.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8343999999999998e-05 2024-09-18T11:32:27.163 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.163 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.163 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.163 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3932 4.b 4:d3ae8215:::smithi11939506-41:head [write 2796249~437859] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:27.163 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:32:27.163 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.055405338, 2024-09-18T11:32:27.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01458329, 2024-09-18T11:32:27.163 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.163 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.163 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:27.163 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:27.163 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:27.164 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3932 2024-09-18T11:32:27.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.164 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:32:27.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:32:27.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038061+0000", 2024-09-18T11:32:27.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:27.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038341+0000", 2024-09-18T11:32:27.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027989099999999999 2024-09-18T11:32:27.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038353+0000", 2024-09-18T11:32:27.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2247e-05 2024-09-18T11:32:27.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038358+0000", 2024-09-18T11:32:27.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-18T11:32:27.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043226+0000", 2024-09-18T11:32:27.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048673049999999997 2024-09-18T11:32:27.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043284+0000", 2024-09-18T11:32:27.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8338999999999998e-05 2024-09-18T11:32:27.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:27.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043367+0000", 2024-09-18T11:32:27.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3120000000000004e-05 2024-09-18T11:32:27.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:27.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.051544+0000", 2024-09-18T11:32:27.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00817734 2024-09-18T11:32:27.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:27.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052528+0000", 2024-09-18T11:32:27.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098318499999999992 2024-09-18T11:32:27.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052610+0000", 2024-09-18T11:32:27.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2300999999999997e-05 2024-09-18T11:32:27.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052646+0000", 2024-09-18T11:32:27.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5955999999999999e-05 2024-09-18T11:32:27.168 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.168 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.169 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.169 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3933 4.b 4:d3ae8215:::smithi11939506-41:head [write 3653524~178505] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:27.169 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041300+0000", 2024-09-18T11:32:27.169 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.052167441999998, 2024-09-18T11:32:27.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01299752, 2024-09-18T11:32:27.169 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.169 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.169 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:27.169 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:27.169 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:27.169 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3933 2024-09-18T11:32:27.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.169 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041300+0000", 2024-09-18T11:32:27.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041300+0000", 2024-09-18T11:32:27.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041299+0000", 2024-09-18T11:32:27.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:27.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041369+0000", 2024-09-18T11:32:27.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9838000000000005e-05 2024-09-18T11:32:27.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041380+0000", 2024-09-18T11:32:27.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0910000000000001e-05 2024-09-18T11:32:27.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041385+0000", 2024-09-18T11:32:27.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5800000000000002e-06 2024-09-18T11:32:27.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044260+0000", 2024-09-18T11:32:27.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028751229999999998 2024-09-18T11:32:27.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044312+0000", 2024-09-18T11:32:27.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1808000000000003e-05 2024-09-18T11:32:27.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:27.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044393+0000", 2024-09-18T11:32:27.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1806999999999998e-05 2024-09-18T11:32:27.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:27.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052654+0000", 2024-09-18T11:32:27.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082609320000000003 2024-09-18T11:32:27.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:27.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054207+0000", 2024-09-18T11:32:27.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001552091 2024-09-18T11:32:27.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054256+0000", 2024-09-18T11:32:27.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9437000000000001e-05 2024-09-18T11:32:27.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054298+0000", 2024-09-18T11:32:27.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1993000000000003e-05 2024-09-18T11:32:27.174 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.174 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.174 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.175 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3934 4.b 4:d3ae8215:::smithi11939506-41:head [setxattr _header (57) in=7b,truncate 3832029] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:27.175 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:27.175 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.052078820999998, 2024-09-18T11:32:27.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013018336, 2024-09-18T11:32:27.175 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.175 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.175 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:27.175 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:27.175 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:27.175 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3934 2024-09-18T11:32:27.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.175 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:27.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:27.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:27.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:27.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041391+0000", 2024-09-18T11:32:27.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.26e-06 2024-09-18T11:32:27.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041395+0000", 2024-09-18T11:32:27.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8999999999999999e-06 2024-09-18T11:32:27.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041397+0000", 2024-09-18T11:32:27.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3149999999999999e-06 2024-09-18T11:32:27.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045266+0000", 2024-09-18T11:32:27.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003869094 2024-09-18T11:32:27.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045316+0000", 2024-09-18T11:32:27.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0074000000000002e-05 2024-09-18T11:32:27.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:27.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045403+0000", 2024-09-18T11:32:27.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6737999999999995e-05 2024-09-18T11:32:27.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:27.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052659+0000", 2024-09-18T11:32:27.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072558259999999999 2024-09-18T11:32:27.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:27.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054330+0000", 2024-09-18T11:32:27.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016716000000000001 2024-09-18T11:32:27.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054362+0000", 2024-09-18T11:32:27.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1195000000000002e-05 2024-09-18T11:32:27.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054407+0000", 2024-09-18T11:32:27.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5707000000000001e-05 2024-09-18T11:32:27.180 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.180 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.180 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.181 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3935 4.b 4:d3ae8215:::smithi11939506-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:27.181 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:32:27.181 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.052068231, 2024-09-18T11:32:27.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013142333000000001, 2024-09-18T11:32:27.181 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.181 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:27.181 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:27.181 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:27.181 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:27.181 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3935 2024-09-18T11:32:27.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.182 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:32:27.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:32:27.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041399+0000", 2024-09-18T11:32:27.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:27.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041401+0000", 2024-09-18T11:32:27.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4890000000000001e-06 2024-09-18T11:32:27.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041404+0000", 2024-09-18T11:32:27.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5240000000000001e-06 2024-09-18T11:32:27.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041407+0000", 2024-09-18T11:32:27.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2230000000000001e-06 2024-09-18T11:32:27.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045699+0000", 2024-09-18T11:32:27.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042918970000000002 2024-09-18T11:32:27.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:27.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045750+0000", 2024-09-18T11:32:27.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1332999999999998e-05 2024-09-18T11:32:27.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054438+0000", 2024-09-18T11:32:27.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086882569999999996 2024-09-18T11:32:27.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054485+0000", 2024-09-18T11:32:27.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7151000000000003e-05 2024-09-18T11:32:27.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054542+0000", 2024-09-18T11:32:27.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6706999999999998e-05 2024-09-18T11:32:27.186 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.186 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.186 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.186 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:343 4.a e137/22 4:5975a5ec:::smithi11939506-36:49 v 137'218, mlcod=132'212)", 2024-09-18T11:32:27.186 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:08.278418+0000", 2024-09-18T11:32:27.186 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.815049722000001, 2024-09-18T11:32:27.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012396378, 2024-09-18T11:32:27.187 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.187 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.187 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278418+0000", 2024-09-18T11:32:27.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278418+0000", 2024-09-18T11:32:27.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278418+0000", 2024-09-18T11:32:27.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:27.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278422+0000", 2024-09-18T11:32:27.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.189e-06 2024-09-18T11:32:27.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278428+0000", 2024-09-18T11:32:27.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7400000000000001e-06 2024-09-18T11:32:27.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278431+0000", 2024-09-18T11:32:27.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4790000000000001e-06 2024-09-18T11:32:27.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.283059+0000", 2024-09-18T11:32:27.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046282750000000003 2024-09-18T11:32:27.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.283072+0000", 2024-09-18T11:32:27.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2819999999999999e-05 2024-09-18T11:32:27.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.290780+0000", 2024-09-18T11:32:27.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077080990000000004 2024-09-18T11:32:27.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.290815+0000", 2024-09-18T11:32:27.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4224000000000003e-05 2024-09-18T11:32:27.191 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.191 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.191 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.191 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3969 4.4 4:2cb3e6a5:::smithi11939506-3:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:27.191 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:32:27.191 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.519827966000001, 2024-09-18T11:32:27.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.024248042000000001, 2024-09-18T11:32:27.191 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.191 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:27.191 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:27.191 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:27.192 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:27.192 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3969 2024-09-18T11:32:27.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.192 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:32:27.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:32:27.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573639+0000", 2024-09-18T11:32:27.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:27.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573646+0000", 2024-09-18T11:32:27.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6569999999999997e-06 2024-09-18T11:32:27.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573662+0000", 2024-09-18T11:32:27.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5187999999999999e-05 2024-09-18T11:32:27.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573670+0000", 2024-09-18T11:32:27.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5399999999999996e-06 2024-09-18T11:32:27.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:27.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8592999999999999e-05 2024-09-18T11:32:27.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573818+0000", 2024-09-18T11:32:27.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9388999999999995e-05 2024-09-18T11:32:27.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597888+0000", 2024-09-18T11:32:27.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.024069859999999998 2024-09-18T11:32:27.195 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.195 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.195 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.195 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3973 4.7 4:e73ba68f:::smithi11939506-1:head [tier-promote] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:27.195 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:27.195 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.519728886999999, 2024-09-18T11:32:27.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.023911297000000001, 2024-09-18T11:32:27.196 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.196 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:27.196 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:27.196 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:27.196 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:27.196 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3973 2024-09-18T11:32:27.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.196 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:27.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:27.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573738+0000", 2024-09-18T11:32:27.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:27.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573743+0000", 2024-09-18T11:32:27.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0839999999999999e-06 2024-09-18T11:32:27.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573752+0000", 2024-09-18T11:32:27.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3080000000000008e-06 2024-09-18T11:32:27.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573758+0000", 2024-09-18T11:32:27.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2659999999999998e-06 2024-09-18T11:32:27.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573803+0000", 2024-09-18T11:32:27.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5243e-05 2024-09-18T11:32:27.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573897+0000", 2024-09-18T11:32:27.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3398000000000003e-05 2024-09-18T11:32:27.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597650+0000", 2024-09-18T11:32:27.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.023753672999999999 2024-09-18T11:32:27.199 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.199 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.199 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.200 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.200 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.200 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:27.201 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:32:27.202 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:27.200+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:27.287 INFO:tasks.rados.rados.0.smithi119.stdout:1835: finishing write tid 1 to smithi11939506-17 2024-09-18T11:32:27.287 INFO:tasks.rados.rados.0.smithi119.stdout:1836: expect (ObjNum 298 snap 43 seq_num 298) 2024-09-18T11:32:27.302 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:27.300+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:27.344 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:27.344 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:27.344 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:27.344 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:27.353 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:32:27.403 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:27.402+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:27.452 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:27.452 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:27.452 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:27.452 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:27.452 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:27.461 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:32:27.472 INFO:tasks.rados.rados.0.smithi119.stdout:1835: finishing write tid 2 to smithi11939506-17 2024-09-18T11:32:27.472 INFO:tasks.rados.rados.0.smithi119.stdout:1835: finishing write tid 3 to smithi11939506-17 2024-09-18T11:32:27.472 INFO:tasks.rados.rados.0.smithi119.stdout:1835: finishing write tid 4 to smithi11939506-17 2024-09-18T11:32:27.472 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 534 (ObjNum 374 snap 58 seq_num 374) dirty exists 2024-09-18T11:32:27.472 INFO:tasks.rados.rados.0.smithi119.stdout:1835: left oid 17 (ObjNum 374 snap 58 seq_num 374) 2024-09-18T11:32:27.472 INFO:tasks.rados.rados.0.smithi119.stdout:1832: done (5 left) 2024-09-18T11:32:27.472 INFO:tasks.rados.rados.0.smithi119.stdout:1834: done (4 left) 2024-09-18T11:32:27.472 INFO:tasks.rados.rados.0.smithi119.stdout:1835: done (3 left) 2024-09-18T11:32:27.472 INFO:tasks.rados.rados.0.smithi119.stdout:1836: done (2 left) 2024-09-18T11:32:27.472 INFO:tasks.rados.rados.0.smithi119.stdout:1839: tier_flush oid 38 2024-09-18T11:32:27.473 INFO:tasks.rados.rados.0.smithi119.stdout:1839: tier_flush oid 38 snap -1 2024-09-18T11:32:27.473 INFO:tasks.rados.rados.0.smithi119.stdout:1837: finishing set_chunk to oid 39 2024-09-18T11:32:27.473 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 39 set_chunk returned error code -95 offset: 956416 length: 2203648 2024-09-18T11:32:27.473 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 39 v 0 (ObjNum 326 snap 45 seq_num 326) dirty exists 2024-09-18T11:32:27.473 INFO:tasks.rados.rados.0.smithi119.stdout:1838: got (0) Success 2024-09-18T11:32:27.473 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 12 v 86 (ObjNum 283 snap 39 seq_num 283) dirty exists 2024-09-18T11:32:27.473 INFO:tasks.rados.rados.0.smithi119.stdout:1837: done (2 left) 2024-09-18T11:32:27.473 INFO:tasks.rados.rados.0.smithi119.stdout:1838: done (1 left) 2024-09-18T11:32:27.473 INFO:tasks.rados.rados.0.smithi119.stdout:1840: read oid 48 snap -1 2024-09-18T11:32:27.473 INFO:tasks.rados.rados.0.smithi119.stdout:1840: expect deleted 2024-09-18T11:32:27.473 INFO:tasks.rados.rados.0.smithi119.stdout:1841: write (excl) oid 2 current snap is 58 2024-09-18T11:32:27.473 INFO:tasks.rados.rados.0.smithi119.stdout:1841: seq_num 375 ranges {702625=719672,2127571=175054} 2024-09-18T11:32:27.479 INFO:tasks.rados.rados.0.smithi119.stdout:1841: writing smithi11939506-2 from 702625 to 1422297 tid 1 2024-09-18T11:32:27.484 INFO:tasks.rados.rados.0.smithi119.stdout:1841: writing smithi11939506-2 from 2127571 to 2302625 tid 2 2024-09-18T11:32:27.485 INFO:tasks.rados.rados.0.smithi119.stdout:1842: tier_evict oid 45 2024-09-18T11:32:27.485 INFO:tasks.rados.rados.0.smithi119.stdout:1842: tier_evict oid 45 snap -1 2024-09-18T11:32:27.485 INFO:tasks.rados.rados.0.smithi119.stdout:1839: got (16) Device or resource busy 2024-09-18T11:32:27.485 INFO:tasks.rados.rados.0.smithi119.stdout:check_oldest_snap_flushed oid 38 v 252 (ObjNum 365 snap 56 seq_num 365) unflushed 56 2024-09-18T11:32:27.485 INFO:tasks.rados.rados.0.smithi119.stdout:1839: done (3 left) 2024-09-18T11:32:27.485 INFO:tasks.rados.rados.0.smithi119.stdout:1840: done (2 left) 2024-09-18T11:32:27.485 INFO:tasks.rados.rados.0.smithi119.stdout:1843: tier_evict oid 43 2024-09-18T11:32:27.485 INFO:tasks.rados.rados.0.smithi119.stdout:1843: tier_evict oid 43 snap -1 2024-09-18T11:32:27.485 INFO:tasks.rados.rados.0.smithi119.stdout:1844: write (excl) oid 50 current snap is 58 2024-09-18T11:32:27.485 INFO:tasks.rados.rados.0.smithi119.stdout:1844: seq_num 376 ranges {719432=794921,2186053=133379} 2024-09-18T11:32:27.492 INFO:tasks.rados.rados.0.smithi119.stdout:1844: writing smithi11939506-50 from 719432 to 1514353 tid 1 2024-09-18T11:32:27.498 INFO:tasks.rados.rados.0.smithi119.stdout:1844: writing smithi11939506-50 from 2186053 to 2319432 tid 2 2024-09-18T11:32:27.498 INFO:tasks.rados.rados.0.smithi119.stdout:1845: tier_promote oid 25 2024-09-18T11:32:27.498 INFO:tasks.rados.rados.0.smithi119.stdout:1841: finishing write tid 1 to smithi11939506-2 2024-09-18T11:32:27.498 INFO:tasks.rados.rados.0.smithi119.stdout:1843: got (22) Invalid argument 2024-09-18T11:32:27.498 INFO:tasks.rados.rados.0.smithi119.stdout:1842: got (22) Invalid argument 2024-09-18T11:32:27.498 INFO:tasks.rados.rados.0.smithi119.stdout:1841: finishing write tid 2 to smithi11939506-2 2024-09-18T11:32:27.498 INFO:tasks.rados.rados.0.smithi119.stdout:1841: finishing write tid 3 to smithi11939506-2 2024-09-18T11:32:27.498 INFO:tasks.rados.rados.0.smithi119.stdout:1841: finishing write tid 4 to smithi11939506-2 2024-09-18T11:32:27.498 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 537 (ObjNum 375 snap 58 seq_num 375) dirty exists 2024-09-18T11:32:27.499 INFO:tasks.rados.rados.0.smithi119.stdout:1841: left oid 2 (ObjNum 375 snap 58 seq_num 375) 2024-09-18T11:32:27.499 INFO:tasks.rados.rados.0.smithi119.stdout:1841: done (4 left) 2024-09-18T11:32:27.499 INFO:tasks.rados.rados.0.smithi119.stdout:1842: done (3 left) 2024-09-18T11:32:27.499 INFO:tasks.rados.rados.0.smithi119.stdout:1843: done (2 left) 2024-09-18T11:32:27.499 INFO:tasks.rados.rados.0.smithi119.stdout:1846: tier_flush oid 8 2024-09-18T11:32:27.499 INFO:tasks.rados.rados.0.smithi119.stdout:1846: tier_flush oid 8 snap -1 2024-09-18T11:32:27.499 INFO:tasks.rados.rados.0.smithi119.stdout:1844: finishing write tid 1 to smithi11939506-50 2024-09-18T11:32:27.499 INFO:tasks.rados.rados.0.smithi119.stdout:1847: tier_promote oid 23 2024-09-18T11:32:27.499 INFO:tasks.rados.rados.0.smithi119.stdout:1848: read oid 27 snap -1 2024-09-18T11:32:27.499 INFO:tasks.rados.rados.0.smithi119.stdout:1848: expect (ObjNum 367 snap 57 seq_num 367) 2024-09-18T11:32:27.499 INFO:tasks.rados.rados.0.smithi119.stdout:1849: write oid 24 current snap is 58 2024-09-18T11:32:27.499 INFO:tasks.rados.rados.0.smithi119.stdout:1849: seq_num 377 ranges {736239=470170,1728182=571782} 2024-09-18T11:32:27.503 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:27.502+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:27.504 INFO:tasks.rados.rados.0.smithi119.stdout:1849: writing smithi11939506-24 from 736239 to 1206409 tid 1 2024-09-18T11:32:27.511 INFO:tasks.rados.rados.0.smithi119.stdout:1849: writing smithi11939506-24 from 1728182 to 2299964 tid 2 2024-09-18T11:32:27.511 INFO:tasks.rados.rados.0.smithi119.stdout:1850: read oid 42 snap -1 2024-09-18T11:32:27.511 INFO:tasks.rados.rados.0.smithi119.stdout:1850: expect (ObjNum 338 snap 48 seq_num 338) 2024-09-18T11:32:27.511 INFO:tasks.rados.rados.0.smithi119.stdout:1851: read oid 48 snap -1 2024-09-18T11:32:27.511 INFO:tasks.rados.rados.0.smithi119.stdout:1851: expect deleted 2024-09-18T11:32:27.511 INFO:tasks.rados.rados.0.smithi119.stdout:1852: read oid 17 snap 56 2024-09-18T11:32:27.511 INFO:tasks.rados.rados.0.smithi119.stdout:1852: expect (ObjNum 351 snap 54 seq_num 351) 2024-09-18T11:32:27.512 INFO:tasks.rados.rados.0.smithi119.stdout:1853: read oid 28 snap -1 2024-09-18T11:32:27.512 INFO:tasks.rados.rados.0.smithi119.stdout:1853: expect (ObjNum 336 snap 48 seq_num 336) 2024-09-18T11:32:27.512 INFO:tasks.rados.rados.0.smithi119.stdout:1854: rollback oid 10 current snap is 58 2024-09-18T11:32:27.512 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 10 to 56 2024-09-18T11:32:27.512 INFO:tasks.rados.rados.0.smithi119.stdout:1846: got (0) Success 2024-09-18T11:32:27.512 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 8 v 305 (ObjNum 362 snap 55 seq_num 362) dirty exists 2024-09-18T11:32:27.512 INFO:tasks.rados.rados.0.smithi119.stdout:1846: done (10 left) 2024-09-18T11:32:27.512 INFO:tasks.rados.rados.0.smithi119.stdout:1855: set_chunk oid 30 target oid 2024-09-18T11:32:27.512 INFO:tasks.rados.rados.0.smithi119.stdout:1855: set_chunk oid 30 offset: 0 length: 4000000 target oid smithi11939506- offset: 0 snap -1 2024-09-18T11:32:27.512 INFO:tasks.rados.rados.0.smithi119.stdout:1856: snap_remove snap 45 2024-09-18T11:32:27.579 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:27.579 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:27.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:27.579 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:27.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.579 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:32:27.579 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:27.579 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 127.59371285, 2024-09-18T11:32:27.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:32:27.579 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.580 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.580 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:27.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:27.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:32:27.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:27.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:32:27.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:32:27.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:32:27.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:32:27.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:32:27.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:32:27.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:32:27.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:32:27.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:32:27.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:32:27.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:32:27.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:32:27.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:32:27.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:32:27.584 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.584 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.584 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.585 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:32:27.585 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:27.585 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.358881897, 2024-09-18T11:32:27.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:32:27.585 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.585 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.585 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:27.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:27.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:27.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:27.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:32:27.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:32:27.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:32:27.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:32:27.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:32:27.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:32:27.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:32:27.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:32:27.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:32:27.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:32:27.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:32:27.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:32:27.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:32:27.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:32:27.589 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.589 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.589 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.590 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:32:27.590 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:27.590 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.184975612, 2024-09-18T11:32:27.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:32:27.590 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.590 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.590 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:27.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:27.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:32:27.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:27.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:32:27.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:32:27.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:32:27.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:32:27.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:32:27.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:32:27.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:32:27.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:32:27.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:32:27.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:32:27.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:32:27.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:32:27.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:32:27.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:32:27.595 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.595 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.595 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.595 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:304 4.6 e69/67 4:67c1765a:::smithi11939506-30:17 v 69'35, mlcod=58'31)", 2024-09-18T11:32:27.596 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:27.596 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.231134518000005, 2024-09-18T11:32:27.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006377611, 2024-09-18T11:32:27.596 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.596 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.596 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:27.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:27.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:27.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:27.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345961+0000", 2024-09-18T11:32:27.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2050000000000008e-06 2024-09-18T11:32:27.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345969+0000", 2024-09-18T11:32:27.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0339999999999997e-06 2024-09-18T11:32:27.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345976+0000", 2024-09-18T11:32:27.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-18T11:32:27.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346026+0000", 2024-09-18T11:32:27.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9286999999999998e-05 2024-09-18T11:32:27.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346050+0000", 2024-09-18T11:32:27.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4397e-05 2024-09-18T11:32:27.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352293+0000", 2024-09-18T11:32:27.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062427170000000001 2024-09-18T11:32:27.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352330+0000", 2024-09-18T11:32:27.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7348999999999998e-05 2024-09-18T11:32:27.600 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.600 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.600 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.600 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:347 4.d e69/22 4:b44229a1:::smithi11939506-42:19 v 69'99, mlcod=65'93)", 2024-09-18T11:32:27.601 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:27.601 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.225401323, 2024-09-18T11:32:27.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056998760000000004, 2024-09-18T11:32:27.601 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.601 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.601 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:27.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:27.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351684+0000", 2024-09-18T11:32:27.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:27.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351693+0000", 2024-09-18T11:32:27.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-18T11:32:27.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351704+0000", 2024-09-18T11:32:27.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1077000000000001e-05 2024-09-18T11:32:27.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351709+0000", 2024-09-18T11:32:27.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5419999999999999e-06 2024-09-18T11:32:27.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351956+0000", 2024-09-18T11:32:27.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000247252 2024-09-18T11:32:27.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351975+0000", 2024-09-18T11:32:27.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9201000000000001e-05 2024-09-18T11:32:27.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357334+0000", 2024-09-18T11:32:27.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053588749999999999 2024-09-18T11:32:27.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357385+0000", 2024-09-18T11:32:27.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1406999999999998e-05 2024-09-18T11:32:27.605 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.605 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.605 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.605 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1956 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'37, mlcod=58'31)", 2024-09-18T11:32:27.605 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:27.605 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.224456410000002, 2024-09-18T11:32:27.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010958842, 2024-09-18T11:32:27.605 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.606 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.606 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:27.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:27.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352996+0000", 2024-09-18T11:32:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036622 2024-09-18T11:32:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353011+0000", 2024-09-18T11:32:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4849e-05 2024-09-18T11:32:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353017+0000", 2024-09-18T11:32:27.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:32:27.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353068+0000", 2024-09-18T11:32:27.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0695000000000002e-05 2024-09-18T11:32:27.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353092+0000", 2024-09-18T11:32:27.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4260000000000002e-05 2024-09-18T11:32:27.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363551+0000", 2024-09-18T11:32:27.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010459154999999999 2024-09-18T11:32:27.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363589+0000", 2024-09-18T11:32:27.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8161000000000003e-05 2024-09-18T11:32:27.609 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.609 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.610 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.610 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1957 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'38, mlcod=65'33)", 2024-09-18T11:32:27.610 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:27.610 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.218040543000001, 2024-09-18T11:32:27.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050499400000000002, 2024-09-18T11:32:27.610 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.610 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.610 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:27.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:27.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:27.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:27.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359214+0000", 2024-09-18T11:32:27.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000168047 2024-09-18T11:32:27.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359227+0000", 2024-09-18T11:32:27.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3124e-05 2024-09-18T11:32:27.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359235+0000", 2024-09-18T11:32:27.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0760000000000003e-06 2024-09-18T11:32:27.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359287+0000", 2024-09-18T11:32:27.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.151e-05 2024-09-18T11:32:27.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359310+0000", 2024-09-18T11:32:27.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-09-18T11:32:27.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364065+0000", 2024-09-18T11:32:27.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047549439999999997 2024-09-18T11:32:27.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364096+0000", 2024-09-18T11:32:27.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1423000000000001e-05 2024-09-18T11:32:27.614 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.614 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.614 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.614 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2300 4.4 e77/22 4:2c93196f:::smithi11939506-28:head v 77'94, mlcod=77'90)", 2024-09-18T11:32:27.614 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:27.614 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 86.588978324999999, 2024-09-18T11:32:27.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050425380000000001, 2024-09-18T11:32:27.615 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.615 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.615 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:27.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:27.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988108+0000", 2024-09-18T11:32:27.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:27.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988388+0000", 2024-09-18T11:32:27.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028087700000000001 2024-09-18T11:32:27.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988405+0000", 2024-09-18T11:32:27.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6565000000000001e-05 2024-09-18T11:32:27.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988412+0000", 2024-09-18T11:32:27.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9709999999999998e-06 2024-09-18T11:32:27.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988464+0000", 2024-09-18T11:32:27.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1949e-05 2024-09-18T11:32:27.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988490+0000", 2024-09-18T11:32:27.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5780999999999999e-05 2024-09-18T11:32:27.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993112+0000", 2024-09-18T11:32:27.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046228399999999996 2024-09-18T11:32:27.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993151+0000", 2024-09-18T11:32:27.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8587999999999997e-05 2024-09-18T11:32:27.619 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.619 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.619 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.619 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.13:63 3.0 e78/21 3:24ebb740:::smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 78'80, mlcod=73'78)", 2024-09-18T11:32:27.619 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:27.619 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 86.171988909000007, 2024-09-18T11:32:27.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052262230000000003, 2024-09-18T11:32:27.619 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.619 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.619 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:27.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:27.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405097+0000", 2024-09-18T11:32:27.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:27.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405103+0000", 2024-09-18T11:32:27.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8780000000000003e-06 2024-09-18T11:32:27.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405117+0000", 2024-09-18T11:32:27.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3441000000000001e-05 2024-09-18T11:32:27.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405121+0000", 2024-09-18T11:32:27.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7099999999999998e-06 2024-09-18T11:32:27.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405158+0000", 2024-09-18T11:32:27.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6477999999999999e-05 2024-09-18T11:32:27.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405173+0000", 2024-09-18T11:32:27.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4796e-05 2024-09-18T11:32:27.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410285+0000", 2024-09-18T11:32:27.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051118919999999998 2024-09-18T11:32:27.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410324+0000", 2024-09-18T11:32:27.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9616999999999997e-05 2024-09-18T11:32:27.623 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.623 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.623 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.624 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.13:18 3.0 e91/21 3:35e3c617:::2e64dd273da7619bd64b294f52905c1b9405a1ced2a8fd3c0b5bc481bc9732a6:head v 91'89, mlcod=85'87)", 2024-09-18T11:32:27.624 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:27.624 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.555213498000001, 2024-09-18T11:32:27.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048558539999999997, 2024-09-18T11:32:27.624 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.624 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.624 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:27.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:27.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:27.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:27.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021880+0000", 2024-09-18T11:32:27.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5860000000000001e-06 2024-09-18T11:32:27.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021890+0000", 2024-09-18T11:32:27.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8290000000000003e-06 2024-09-18T11:32:27.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021895+0000", 2024-09-18T11:32:27.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2480000000000004e-06 2024-09-18T11:32:27.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022166+0000", 2024-09-18T11:32:27.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027073499999999998 2024-09-18T11:32:27.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022184+0000", 2024-09-18T11:32:27.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8082000000000001e-05 2024-09-18T11:32:27.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026693+0000", 2024-09-18T11:32:27.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045093479999999998 2024-09-18T11:32:27.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026729+0000", 2024-09-18T11:32:27.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6282999999999998e-05 2024-09-18T11:32:27.628 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.628 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.628 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.628 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:573 4.4 e91/22 4:2c93196f:::smithi11939506-28:26 v 91'135, mlcod=90'131)", 2024-09-18T11:32:27.628 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:27.628 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.552746909000007, 2024-09-18T11:32:27.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050722739999999999, 2024-09-18T11:32:27.628 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.628 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.628 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:27.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:27.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.629 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.629 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024339+0000", 2024-09-18T11:32:27.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:27.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024349+0000", 2024-09-18T11:32:27.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6339999999999996e-06 2024-09-18T11:32:27.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024362+0000", 2024-09-18T11:32:27.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-18T11:32:27.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024368+0000", 2024-09-18T11:32:27.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8649999999999998e-06 2024-09-18T11:32:27.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024431+0000", 2024-09-18T11:32:27.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2780999999999994e-05 2024-09-18T11:32:27.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024448+0000", 2024-09-18T11:32:27.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7388999999999999e-05 2024-09-18T11:32:27.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029385+0000", 2024-09-18T11:32:27.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004936523 2024-09-18T11:32:27.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029412+0000", 2024-09-18T11:32:27.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7702999999999999e-05 2024-09-18T11:32:27.632 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.632 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.632 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.633 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:574 4.4 e91/22 4:2cb3e6a5:::smithi11939506-3:1e v 91'137, mlcod=90'131)", 2024-09-18T11:32:27.633 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:27.633 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.552308866000004, 2024-09-18T11:32:27.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070802850000000004, 2024-09-18T11:32:27.633 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.633 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.633 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:27.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:27.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024777+0000", 2024-09-18T11:32:27.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:27.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024785+0000", 2024-09-18T11:32:27.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8099999999999998e-06 2024-09-18T11:32:27.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024794+0000", 2024-09-18T11:32:27.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5839999999999995e-06 2024-09-18T11:32:27.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024798+0000", 2024-09-18T11:32:27.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.031e-06 2024-09-18T11:32:27.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024830+0000", 2024-09-18T11:32:27.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2268999999999999e-05 2024-09-18T11:32:27.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024847+0000", 2024-09-18T11:32:27.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7306999999999999e-05 2024-09-18T11:32:27.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031834+0000", 2024-09-18T11:32:27.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069868380000000004 2024-09-18T11:32:27.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031858+0000", 2024-09-18T11:32:27.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4082000000000001e-05 2024-09-18T11:32:27.637 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.637 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.637 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.637 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:541 4.d e91/22 4:b9bb642b:::smithi11939506-27:19 v 91'158, mlcod=91'154)", 2024-09-18T11:32:27.638 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:27.638 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 73.544928174000006, 2024-09-18T11:32:27.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052185, 2024-09-18T11:32:27.638 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.638 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.638 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:27.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:27.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032158+0000", 2024-09-18T11:32:27.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:27.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032165+0000", 2024-09-18T11:32:27.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-09-18T11:32:27.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032176+0000", 2024-09-18T11:32:27.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1460999999999999e-05 2024-09-18T11:32:27.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032182+0000", 2024-09-18T11:32:27.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0870000000000003e-06 2024-09-18T11:32:27.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032224+0000", 2024-09-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1372999999999998e-05 2024-09-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032246+0000", 2024-09-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2158e-05 2024-09-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037348+0000", 2024-09-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051025319999999999 2024-09-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037377+0000", 2024-09-18T11:32:27.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8937000000000001e-05 2024-09-18T11:32:27.642 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.642 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.642 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.642 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4178.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-18T11:32:27.642 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:27.642 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.636266108999997, 2024-09-18T11:32:27.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048960139999999997, 2024-09-18T11:32:27.642 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.642 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:27.643 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:27.643 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4178", 2024-09-18T11:32:27.643 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/490761693", 2024-09-18T11:32:27.643 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4 2024-09-18T11:32:27.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.643 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:27.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:27.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940819+0000", 2024-09-18T11:32:27.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:27.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940832+0000", 2024-09-18T11:32:27.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3155e-05 2024-09-18T11:32:27.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940848+0000", 2024-09-18T11:32:27.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6288999999999999e-05 2024-09-18T11:32:27.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940856+0000", 2024-09-18T11:32:27.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4919999999999996e-06 2024-09-18T11:32:27.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945613+0000", 2024-09-18T11:32:27.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047574999999999996 2024-09-18T11:32:27.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945717+0000", 2024-09-18T11:32:27.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103553 2024-09-18T11:32:27.646 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.646 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.646 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.646 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3482 4.6 e117/67 4:67c1765a:::smithi11939506-30:head v 117'120, mlcod=115'116)", 2024-09-18T11:32:27.646 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:27.647 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.892411885999998, 2024-09-18T11:32:27.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050037479999999997, 2024-09-18T11:32:27.647 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.647 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.647 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:27.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:27.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684674+0000", 2024-09-18T11:32:27.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:27.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684950+0000", 2024-09-18T11:32:27.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027621200000000002 2024-09-18T11:32:27.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684968+0000", 2024-09-18T11:32:27.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8e-05 2024-09-18T11:32:27.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684984+0000", 2024-09-18T11:32:27.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5775000000000001e-05 2024-09-18T11:32:27.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685033+0000", 2024-09-18T11:32:27.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8878000000000002e-05 2024-09-18T11:32:27.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685059+0000", 2024-09-18T11:32:27.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6801999999999999e-05 2024-09-18T11:32:27.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689640+0000", 2024-09-18T11:32:27.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045809029999999999 2024-09-18T11:32:27.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689679+0000", 2024-09-18T11:32:27.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8507000000000002e-05 2024-09-18T11:32:27.651 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.651 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.651 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.652 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:976 4.4 e132/22 4:2cb3e6a5:::smithi11939506-3:30 v 132'195, mlcod=128'192)", 2024-09-18T11:32:27.652 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:27.652 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 24.558440815000001, 2024-09-18T11:32:27.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0058526510000000004, 2024-09-18T11:32:27.652 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.652 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.652 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:27.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:27.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:27.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:27.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018661+0000", 2024-09-18T11:32:27.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5565000000000001e-05 2024-09-18T11:32:27.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018672+0000", 2024-09-18T11:32:27.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0210000000000001e-05 2024-09-18T11:32:27.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018678+0000", 2024-09-18T11:32:27.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1539999999999999e-06 2024-09-18T11:32:27.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018716+0000", 2024-09-18T11:32:27.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8389000000000001e-05 2024-09-18T11:32:27.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018737+0000", 2024-09-18T11:32:27.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0611e-05 2024-09-18T11:32:27.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.024458+0000", 2024-09-18T11:32:27.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057208629999999996 2024-09-18T11:32:27.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.024499+0000", 2024-09-18T11:32:27.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1100999999999998e-05 2024-09-18T11:32:27.657 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.657 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.657 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.657 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:823 4.d e137/22 4:b44229a1:::smithi11939506-42:49 v 137'266, mlcod=132'260)", 2024-09-18T11:32:27.657 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:32:27.657 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.299792259, 2024-09-18T11:32:27.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054497770000000003, 2024-09-18T11:32:27.657 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.657 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.657 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:32:27.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:32:27.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277294+0000", 2024-09-18T11:32:27.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:27.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277299+0000", 2024-09-18T11:32:27.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6280000000000004e-06 2024-09-18T11:32:27.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277306+0000", 2024-09-18T11:32:27.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.533e-06 2024-09-18T11:32:27.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277316+0000", 2024-09-18T11:32:27.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0148e-05 2024-09-18T11:32:27.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277868+0000", 2024-09-18T11:32:27.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00055199999999999997 2024-09-18T11:32:27.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277898+0000", 2024-09-18T11:32:27.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0264999999999999e-05 2024-09-18T11:32:27.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.282715+0000", 2024-09-18T11:32:27.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048165619999999999 2024-09-18T11:32:27.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.282744+0000", 2024-09-18T11:32:27.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9855000000000001e-05 2024-09-18T11:32:27.661 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.661 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.661 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.661 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:163 3.0 e138/21 3:02e8b30e:::4456d047c09ae2a9a25ccebfd83e30cf78df6779d63bfba9365afd7301eaeb6a:head v 138'132, mlcod=134'130)", 2024-09-18T11:32:27.662 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:27.662 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.994929515999999, 2024-09-18T11:32:27.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051351770000000003, 2024-09-18T11:32:27.662 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.662 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.662 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:27.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:27.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:27.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:27.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582232+0000", 2024-09-18T11:32:27.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5333999999999994e-05 2024-09-18T11:32:27.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582242+0000", 2024-09-18T11:32:27.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5200000000000003e-06 2024-09-18T11:32:27.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582248+0000", 2024-09-18T11:32:27.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8000000000000004e-06 2024-09-18T11:32:27.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582305+0000", 2024-09-18T11:32:27.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7813999999999999e-05 2024-09-18T11:32:27.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582329+0000", 2024-09-18T11:32:27.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3414999999999999e-05 2024-09-18T11:32:27.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587257+0000", 2024-09-18T11:32:27.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049279880000000003 2024-09-18T11:32:27.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587293+0000", 2024-09-18T11:32:27.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5802e-05 2024-09-18T11:32:27.666 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.666 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.666 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.666 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3974 4.d e138/22 4:b5a463bf:::smithi11939506-20:head v 138'270, mlcod=137'266)", 2024-09-18T11:32:27.666 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:32:27.667 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.991048245000002, 2024-09-18T11:32:27.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011228458, 2024-09-18T11:32:27.667 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.667 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.667 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:32:27.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:32:27.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586038+0000", 2024-09-18T11:32:27.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:27.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587638+0000", 2024-09-18T11:32:27.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001599184 2024-09-18T11:32:27.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587647+0000", 2024-09-18T11:32:27.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3719999999999994e-06 2024-09-18T11:32:27.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587653+0000", 2024-09-18T11:32:27.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.105e-06 2024-09-18T11:32:27.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587712+0000", 2024-09-18T11:32:27.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9252000000000001e-05 2024-09-18T11:32:27.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587735+0000", 2024-09-18T11:32:27.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3132000000000001e-05 2024-09-18T11:32:27.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597228+0000", 2024-09-18T11:32:27.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0094930090000000002 2024-09-18T11:32:27.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597267+0000", 2024-09-18T11:32:27.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8693999999999999e-05 2024-09-18T11:32:27.670 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.670 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.671 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.671 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3975 4.d e138/22 4:b5a463bf:::smithi11939506-20:head v 138'271, mlcod=137'266)", 2024-09-18T11:32:27.671 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:27.671 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.984847539, 2024-09-18T11:32:27.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053620450000000002, 2024-09-18T11:32:27.671 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:27.671 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:27.671 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:27.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:27.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:27.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:27.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:27.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:27.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:27.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:27.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:27.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:27.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592927+0000", 2024-09-18T11:32:27.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00068878699999999995 2024-09-18T11:32:27.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:27.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592940+0000", 2024-09-18T11:32:27.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:32:27.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:27.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592951+0000", 2024-09-18T11:32:27.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1046000000000001e-05 2024-09-18T11:32:27.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:27.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593008+0000", 2024-09-18T11:32:27.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6875e-05 2024-09-18T11:32:27.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:27.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593033+0000", 2024-09-18T11:32:27.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5341999999999999e-05 2024-09-18T11:32:27.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:27.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597565+0000", 2024-09-18T11:32:27.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045323150000000003 2024-09-18T11:32:27.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:27.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:27.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:27.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597601+0000", 2024-09-18T11:32:27.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6071000000000001e-05 2024-09-18T11:32:27.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.675 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:27.675 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:27.675 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:27.675 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:32:27.677 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:32:27.678 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:27.676+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:27.778 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:27.777+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:27.817 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:27.817 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:27.817 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:27.818 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:27.826 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:32:27.832 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:27.833 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:27.849 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:27.849 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:27.864 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:27.864 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:27.879 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:27.878+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:27.879 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:27.880 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:27.895 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:27.895 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:27.910 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:27.910 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:27.925 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:27.925 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:27.926 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:27.926 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:27.933 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:32:27.942 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:27.942 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:27.979 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:27.978+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:28.030 INFO:tasks.rados.rados.0.smithi119.stdout:1856: done (11 left) 2024-09-18T11:32:28.031 INFO:tasks.rados.rados.0.smithi119.stdout:1857: tier_evict oid 31 2024-09-18T11:32:28.031 INFO:tasks.rados.rados.0.smithi119.stdout:1857: tier_evict oid 31 snap -1 2024-09-18T11:32:28.031 INFO:tasks.rados.rados.0.smithi119.stdout:1845: got (0) Success 2024-09-18T11:32:28.031 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 76 (ObjNum 368 snap 57 seq_num 368) dirty exists 2024-09-18T11:32:28.031 INFO:tasks.rados.rados.0.smithi119.stdout:1844: finishing write tid 2 to smithi11939506-50 2024-09-18T11:32:28.031 INFO:tasks.rados.rados.0.smithi119.stdout:1844: finishing write tid 3 to smithi11939506-50 2024-09-18T11:32:28.031 INFO:tasks.rados.rados.0.smithi119.stdout:1844: finishing write tid 4 to smithi11939506-50 2024-09-18T11:32:28.031 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 50 v 242 (ObjNum 376 snap 58 seq_num 376) dirty exists 2024-09-18T11:32:28.031 INFO:tasks.rados.rados.0.smithi119.stdout:1844: left oid 50 (ObjNum 376 snap 58 seq_num 376) 2024-09-18T11:32:28.031 INFO:tasks.rados.rados.0.smithi119.stdout:1848: expect (ObjNum 367 snap 57 seq_num 367) 2024-09-18T11:32:28.032 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is unset 2024-09-18T11:32:28.032 INFO:teuthology.orchestra.run.smithi119.stdout:moved 2 / 58 (3.44828%) 2024-09-18T11:32:28.032 INFO:teuthology.orchestra.run.smithi119.stdout:avg 8.28571 2024-09-18T11:32:28.032 INFO:teuthology.orchestra.run.smithi119.stdout:stddev 3.41067 -> 3.2388 (expected baseline 2.66497) 2024-09-18T11:32:28.032 INFO:teuthology.orchestra.run.smithi119.stdout:min osd.2 with 1 -> 1 pgs (0.12069 -> 0.12069 * mean) 2024-09-18T11:32:28.033 INFO:teuthology.orchestra.run.smithi119.stdout:max osd.1 with 13 -> 12 pgs (1.56897 -> 1.44828 * mean) 2024-09-18T11:32:28.033 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:32:28.033 INFO:teuthology.orchestra.run.smithi119.stdout:oload 110 2024-09-18T11:32:28.033 INFO:teuthology.orchestra.run.smithi119.stdout:max_change 0.05 2024-09-18T11:32:28.033 INFO:teuthology.orchestra.run.smithi119.stdout:max_change_osds 4 2024-09-18T11:32:28.033 INFO:teuthology.orchestra.run.smithi119.stdout:average_utilization 0.0040 2024-09-18T11:32:28.033 INFO:teuthology.orchestra.run.smithi119.stdout:overload_utilization 0.0044 2024-09-18T11:32:28.033 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 weight 0.4120 -> 0.4620 2024-09-18T11:32:28.033 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 weight 0.9500 -> 0.9000 2024-09-18T11:32:28.033 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 weight 0.9500 -> 0.9000 2024-09-18T11:32:28.046 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:28.047 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:28.056 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:32:28.098 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:28.097+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:28.136 INFO:tasks.rados.rados.0.smithi119.stdout:1847: got (0) Success 2024-09-18T11:32:28.136 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 296 (ObjNum 363 snap 55 seq_num 363) dirty exists 2024-09-18T11:32:28.136 INFO:tasks.rados.rados.0.smithi119.stdout:1849: finishing write tid 1 to smithi11939506-24 2024-09-18T11:32:28.136 INFO:tasks.rados.rados.0.smithi119.stdout:1844: done (11 left) 2024-09-18T11:32:28.136 INFO:tasks.rados.rados.0.smithi119.stdout:1845: done (10 left) 2024-09-18T11:32:28.137 INFO:tasks.rados.rados.0.smithi119.stdout:1847: done (9 left) 2024-09-18T11:32:28.137 INFO:tasks.rados.rados.0.smithi119.stdout:1848: done (8 left) 2024-09-18T11:32:28.137 INFO:tasks.rados.rados.0.smithi119.stdout:1858: snap_create 2024-09-18T11:32:28.137 INFO:tasks.rados.rados.0.smithi119.stdout:1855: finishing set_chunk to oid 30 2024-09-18T11:32:28.137 INFO:tasks.rados.rados.0.smithi119.stdout:1855: got expected ERANGE (src dne) 2024-09-18T11:32:28.137 INFO:tasks.rados.rados.0.smithi119.stdout:1849: finishing write tid 2 to smithi11939506-24 2024-09-18T11:32:28.137 INFO:tasks.rados.rados.0.smithi119.stdout:1853: expect (ObjNum 336 snap 48 seq_num 336) 2024-09-18T11:32:28.154 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:28.155 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:28.163 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:32:28.207 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:28.206+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:28.262 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:28.263 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:28.271 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:32:28.313 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:28.312+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:28.353 INFO:tasks.rados.rados.0.smithi119.stdout:1849: finishing write tid 3 to smithi11939506-24 2024-09-18T11:32:28.353 INFO:tasks.rados.rados.0.smithi119.stdout:1849: finishing write tid 4 to smithi11939506-24 2024-09-18T11:32:28.354 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 24 v 373 (ObjNum 377 snap 58 seq_num 377) dirty exists 2024-09-18T11:32:28.354 INFO:tasks.rados.rados.0.smithi119.stdout:1849: left oid 24 (ObjNum 377 snap 58 seq_num 377) 2024-09-18T11:32:28.354 INFO:tasks.rados.rados.0.smithi119.stdout:1850: expect (ObjNum 338 snap 48 seq_num 338) 2024-09-18T11:32:28.382 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:28.381+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:28.410 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:28.410 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:28.419 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:32:28.482 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:28.481+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:28.518 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:28.518 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:28.526 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:32:28.583 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:28.582+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:28.621 INFO:tasks.rados.rados.0.smithi119.stdout:1854: finishing rollback tid 0 to smithi11939506-10 2024-09-18T11:32:28.621 INFO:tasks.rados.rados.0.smithi119.stdout:1854: finishing rollback tid 1 to smithi11939506-10 2024-09-18T11:32:28.621 INFO:tasks.rados.rados.0.smithi119.stdout:1852: expect (ObjNum 351 snap 54 seq_num 351) 2024-09-18T11:32:28.668 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:28.668 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:28.676 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:32:28.720 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:28.718+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:28.773 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:28.773 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:28.781 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:32:28.785 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:28.784+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:28.830 INFO:tasks.rados.rados.0.smithi119.stdout:1854: finishing rollback tid 2 to smithi11939506-10 2024-09-18T11:32:28.830 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 10 v 358 (ObjNum 358 snap 55 seq_num 358) dirty exists 2024-09-18T11:32:28.830 INFO:tasks.rados.rados.0.smithi119.stdout:1857: got (22) Invalid argument 2024-09-18T11:32:28.886 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:28.885+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:28.921 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:28.921 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:28.929 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:32:29.013 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:29.012+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:29.031 INFO:tasks.rados.rados.0.smithi119.stdout:1849: done (8 left) 2024-09-18T11:32:29.031 INFO:tasks.rados.rados.0.smithi119.stdout:1850: done (7 left) 2024-09-18T11:32:29.031 INFO:tasks.rados.rados.0.smithi119.stdout:1851: done (6 left) 2024-09-18T11:32:29.031 INFO:tasks.rados.rados.0.smithi119.stdout:1852: done (5 left) 2024-09-18T11:32:29.031 INFO:tasks.rados.rados.0.smithi119.stdout:1853: done (4 left) 2024-09-18T11:32:29.031 INFO:tasks.rados.rados.0.smithi119.stdout:1854: done (3 left) 2024-09-18T11:32:29.031 INFO:tasks.rados.rados.0.smithi119.stdout:1855: done (2 left) 2024-09-18T11:32:29.031 INFO:tasks.rados.rados.0.smithi119.stdout:1857: done (1 left) 2024-09-18T11:32:29.031 INFO:tasks.rados.rados.0.smithi119.stdout:1858: done (0 left) 2024-09-18T11:32:29.032 INFO:tasks.rados.rados.0.smithi119.stdout:1859: tier_evict oid 23 2024-09-18T11:32:29.032 INFO:tasks.rados.rados.0.smithi119.stdout:1859: tier_evict oid 23 snap -1 2024-09-18T11:32:29.032 INFO:tasks.rados.rados.0.smithi119.stdout:1860: set_chunk oid 27 target oid 2024-09-18T11:32:29.072 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:29.072 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:29.081 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:32:29.088 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:29.087+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:29.189 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:29.188+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:29.220 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:29.220 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:29.228 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:32:29.290 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:29.288+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:29.367 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:29.367 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:29.375 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:32:29.420 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:29.419+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:29.473 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:29.473 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:29.481 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:32:29.492 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:29.491+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:29.511 INFO:tasks.rados.rados.0.smithi119.stdout:1860: set_chunk oid 27 offset: 1972224 length: 60416 target oid 7119225a1d868e5f6d083aa0281f3f3f95390adfec23405e2786ce6239c97210 offset: 0 snap -1 2024-09-18T11:32:29.512 INFO:tasks.rados.rados.0.smithi119.stdout:1861: rollback oid 34 current snap is 59 2024-09-18T11:32:29.512 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 34 to 58 2024-09-18T11:32:29.512 INFO:tasks.rados.rados.0.smithi119.stdout:1859: got (22) Invalid argument 2024-09-18T11:32:29.512 INFO:tasks.rados.rados.0.smithi119.stdout:1859: done (2 left) 2024-09-18T11:32:29.512 INFO:tasks.rados.rados.0.smithi119.stdout:1862: read oid 37 snap -1 2024-09-18T11:32:29.512 INFO:tasks.rados.rados.0.smithi119.stdout:1862: expect deleted 2024-09-18T11:32:29.512 INFO:tasks.rados.rados.0.smithi119.stdout:1863: write (excl) oid 17 current snap is 59 2024-09-18T11:32:29.512 INFO:tasks.rados.rados.0.smithi119.stdout:1863: seq_num 378 ranges {753046=545419,1786664=431793} 2024-09-18T11:32:29.518 INFO:tasks.rados.rados.0.smithi119.stdout:1863: writing smithi11939506-17 from 753046 to 1298465 tid 1 2024-09-18T11:32:29.523 INFO:tasks.rados.rados.0.smithi119.stdout:1863: writing smithi11939506-17 from 1786664 to 2218457 tid 2 2024-09-18T11:32:29.523 INFO:tasks.rados.rados.0.smithi119.stdout:1864: tier_promote oid 35 2024-09-18T11:32:29.524 INFO:tasks.rados.rados.0.smithi119.stdout:1862: done (4 left) 2024-09-18T11:32:29.524 INFO:tasks.rados.rados.0.smithi119.stdout:1865: write (excl) oid 21 current snap is 59 2024-09-18T11:32:29.524 INFO:tasks.rados.rados.0.smithi119.stdout:1865: seq_num 379 ranges {769853=620668,1845146=524707} 2024-09-18T11:32:29.532 INFO:tasks.rados.rados.0.smithi119.stdout:1865: writing smithi11939506-21 from 769853 to 1390521 tid 1 2024-09-18T11:32:29.538 INFO:tasks.rados.rados.0.smithi119.stdout:1865: writing smithi11939506-21 from 1845146 to 2369853 tid 2 2024-09-18T11:32:29.538 INFO:tasks.rados.rados.0.smithi119.stdout:1866: tier_evict oid 39 2024-09-18T11:32:29.538 INFO:tasks.rados.rados.0.smithi119.stdout:1866: tier_evict oid 39 snap -1 2024-09-18T11:32:29.538 INFO:tasks.rados.rados.0.smithi119.stdout:1863: finishing write tid 1 to smithi11939506-17 2024-09-18T11:32:29.538 INFO:tasks.rados.rados.0.smithi119.stdout:1867: read oid 43 snap 50 2024-09-18T11:32:29.538 INFO:tasks.rados.rados.0.smithi119.stdout:1867: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:32:29.538 INFO:tasks.rados.rados.0.smithi119.stdout:1868: set_chunk oid 23 target oid 2024-09-18T11:32:29.545 INFO:tasks.rados.rados.0.smithi119.stdout:1868: set_chunk oid 23 offset: 799232 length: 842752 target oid fa07574965f0bcbfc2c66a125ea410a3ec4242be802894d76113e2ceda1389d4 offset: 0 snap -1 2024-09-18T11:32:29.545 INFO:tasks.rados.rados.0.smithi119.stdout:1869: set_chunk oid 4 target oid 2024-09-18T11:32:29.563 INFO:tasks.rados.rados.0.smithi119.stdout:1869: set_chunk oid 4 offset: 239104 length: 1473536 target oid f0ad97abdbb4030303a86ed99e58813b09a84b6338a3d782aa32100aabc14586 offset: 0 snap -1 2024-09-18T11:32:29.563 INFO:tasks.rados.rados.0.smithi119.stdout:1870: snap_create 2024-09-18T11:32:29.563 INFO:tasks.rados.rados.0.smithi119.stdout:1863: finishing write tid 2 to smithi11939506-17 2024-09-18T11:32:29.563 INFO:tasks.rados.rados.0.smithi119.stdout:1863: finishing write tid 3 to smithi11939506-17 2024-09-18T11:32:29.563 INFO:tasks.rados.rados.0.smithi119.stdout:1863: finishing write tid 4 to smithi11939506-17 2024-09-18T11:32:29.563 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 547 (ObjNum 378 snap 59 seq_num 378) dirty exists 2024-09-18T11:32:29.563 INFO:tasks.rados.rados.0.smithi119.stdout:1863: left oid 17 (ObjNum 378 snap 59 seq_num 378) 2024-09-18T11:32:29.563 INFO:tasks.rados.rados.0.smithi119.stdout:1861: finishing rollback tid 0 to smithi11939506-34 2024-09-18T11:32:29.564 INFO:tasks.rados.rados.0.smithi119.stdout:1864: got (0) Success 2024-09-18T11:32:29.564 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 207 (ObjNum 155 snap 20 seq_num 155) dirty exists 2024-09-18T11:32:29.564 INFO:tasks.rados.rados.0.smithi119.stdout:1861: finishing rollback tid 1 to smithi11939506-34 2024-09-18T11:32:29.564 INFO:tasks.rados.rados.0.smithi119.stdout:1861: finishing rollback tid 2 to smithi11939506-34 2024-09-18T11:32:29.564 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 337 (ObjNum 329 snap 48 seq_num 329) dirty exists 2024-09-18T11:32:29.564 INFO:tasks.rados.rados.0.smithi119.stdout:1860: finishing set_chunk to oid 27 2024-09-18T11:32:29.564 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 319 (ObjNum 367 snap 57 seq_num 367) dirty exists 2024-09-18T11:32:29.564 INFO:tasks.rados.rados.0.smithi119.stdout:1865: finishing write tid 1 to smithi11939506-21 2024-09-18T11:32:29.564 INFO:tasks.rados.rados.0.smithi119.stdout:1866: got (22) Invalid argument 2024-09-18T11:32:29.564 INFO:tasks.rados.rados.0.smithi119.stdout:1865: finishing write tid 2 to smithi11939506-21 2024-09-18T11:32:29.564 INFO:tasks.rados.rados.0.smithi119.stdout:1865: finishing write tid 3 to smithi11939506-21 2024-09-18T11:32:29.564 INFO:tasks.rados.rados.0.smithi119.stdout:1865: finishing write tid 4 to smithi11939506-21 2024-09-18T11:32:29.564 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 381 (ObjNum 379 snap 59 seq_num 379) dirty exists 2024-09-18T11:32:29.565 INFO:tasks.rados.rados.0.smithi119.stdout:1865: left oid 21 (ObjNum 379 snap 59 seq_num 379) 2024-09-18T11:32:29.565 INFO:tasks.rados.rados.0.smithi119.stdout:1868: finishing set_chunk to oid 23 2024-09-18T11:32:29.565 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 296 (ObjNum 363 snap 55 seq_num 363) dirty exists 2024-09-18T11:32:29.565 INFO:tasks.rados.rados.0.smithi119.stdout:1867: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-18T11:32:29.592 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:29.591+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:29.630 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:29.630 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:29.638 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:32:29.653 INFO:tasks.rados.rados.0.smithi119.stdout:1869: finishing set_chunk to oid 4 2024-09-18T11:32:29.653 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 4 v 96 (ObjNum 310 snap 43 seq_num 310) dirty exists 2024-09-18T11:32:29.693 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:29.692+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:29.737 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:29.737 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:29.745 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:32:29.776 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:29.774+0000 7fb6930fb640 -1 osd.2 157 handle_osd_map injecting map crc failure 2024-09-18T11:32:29.819 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:29.818+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:29.873 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:29.873 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:29.881 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:32:29.896 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:29.895+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:29.954 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:32:29.997 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:29.995+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:30.022 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:30.022 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:30.031 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:32:30.047 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T11:32:30.082 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:30.082 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:30.097 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:30.096+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:30.100 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:30.100 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:30.115 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:30.131 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:30.131 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:30.134 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:30.134 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:30.134 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:30.134 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:30.143 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:32:30.147 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:30.147 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:30.162 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:30.163 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:30.179 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:30.179 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:30.195 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:30.195 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:30.197 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:30.196+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:30.248 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:30.249 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:30.249 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:30.249 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:30.249 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:30.257 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:32:30.298 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:30.297+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:30.377 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is unset 2024-09-18T11:32:30.378 INFO:tasks.rados.rados.0.smithi119.stdout:1860: done (9 left) 2024-09-18T11:32:30.378 INFO:tasks.rados.rados.0.smithi119.stdout:1861: done (8 left) 2024-09-18T11:32:30.378 INFO:tasks.rados.rados.0.smithi119.stdout:1863: done (7 left) 2024-09-18T11:32:30.378 INFO:tasks.rados.rados.0.smithi119.stdout:1864: done (6 left) 2024-09-18T11:32:30.379 INFO:tasks.rados.rados.0.smithi119.stdout:1865: done (5 left) 2024-09-18T11:32:30.379 INFO:tasks.rados.rados.0.smithi119.stdout:1866: done (4 left) 2024-09-18T11:32:30.379 INFO:tasks.rados.rados.0.smithi119.stdout:1867: done (3 left) 2024-09-18T11:32:30.379 INFO:tasks.rados.rados.0.smithi119.stdout:1868: done (2 left) 2024-09-18T11:32:30.379 INFO:tasks.rados.rados.0.smithi119.stdout:1869: done (1 left) 2024-09-18T11:32:30.379 INFO:tasks.rados.rados.0.smithi119.stdout:1870: done (0 left) 2024-09-18T11:32:30.379 INFO:tasks.rados.rados.0.smithi119.stdout:1871: write oid 47 current snap is 60 2024-09-18T11:32:30.379 INFO:tasks.rados.rados.0.smithi119.stdout:1871: seq_num 380 ranges {786660=695917,1903628=435462} 2024-09-18T11:32:30.386 INFO:tasks.rados.rados.0.smithi119.stdout:1871: writing smithi11939506-47 from 786660 to 1482577 tid 1 2024-09-18T11:32:30.391 INFO:tasks.rados.rados.0.smithi119.stdout:1871: writing smithi11939506-47 from 1903628 to 2339090 tid 2 2024-09-18T11:32:30.391 INFO:tasks.rados.rados.0.smithi119.stdout:1872: snap_remove snap 36 2024-09-18T11:32:30.399 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:30.398+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:30.401 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:262 4.9 e53/22 4:99feffe9:::smithi11939506-45:13 v 53'17, mlcod=51'14)", 2024-09-18T11:32:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 116.40025361399999, 2024-09-18T11:32:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013878679, 2024-09-18T11:32:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998997+0000", 2024-09-18T11:32:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1099999999999999e-06 2024-09-18T11:32:30.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999008+0000", 2024-09-18T11:32:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1321000000000001e-05 2024-09-18T11:32:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999015+0000", 2024-09-18T11:32:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3000000000000004e-06 2024-09-18T11:32:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999055+0000", 2024-09-18T11:32:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0058000000000002e-05 2024-09-18T11:32:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999070+0000", 2024-09-18T11:32:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4328000000000001e-05 2024-09-18T11:32:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012465+0000", 2024-09-18T11:32:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013395638 2024-09-18T11:32:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012870+0000", 2024-09-18T11:32:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040423400000000001 2024-09-18T11:32:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:205 4.b e53/22 4:dba5bd39:::smithi11939506-17:13 v 53'77, mlcod=51'74)", 2024-09-18T11:32:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 116.40020435300001, 2024-09-18T11:32:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013359586, 2024-09-18T11:32:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.406 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.407 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999048+0000", 2024-09-18T11:32:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3550000000000005e-06 2024-09-18T11:32:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999056+0000", 2024-09-18T11:32:30.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3259999999999996e-06 2024-09-18T11:32:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999063+0000", 2024-09-18T11:32:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6409999999999996e-06 2024-09-18T11:32:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999098+0000", 2024-09-18T11:32:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5203999999999997e-05 2024-09-18T11:32:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999115+0000", 2024-09-18T11:32:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6478999999999999e-05 2024-09-18T11:32:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012362+0000", 2024-09-18T11:32:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01324768 2024-09-18T11:32:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012400+0000", 2024-09-18T11:32:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.731e-05 2024-09-18T11:32:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.410 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1956 4.6 4:67c1765a:::smithi11939506-30:head [write 705927~558502] snapc 1e=[1e,1c,1b,19,18,16,13,10] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-18T11:32:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:32:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.047235401999998, 2024-09-18T11:32:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011813239, 2024-09-18T11:32:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1956 2024-09-18T11:32:30.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:32:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352009+0000", 2024-09-18T11:32:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.412 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352007+0000", 2024-09-18T11:32:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352229+0000", 2024-09-18T11:32:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000222024 2024-09-18T11:32:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352251+0000", 2024-09-18T11:32:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2512000000000001e-05 2024-09-18T11:32:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352260+0000", 2024-09-18T11:32:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7339999999999997e-06 2024-09-18T11:32:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352307+0000", 2024-09-18T11:32:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7061999999999998e-05 2024-09-18T11:32:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352408+0000", 2024-09-18T11:32:30.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101216 2024-09-18T11:32:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 2", 2024-09-18T11:32:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352550+0000", 2024-09-18T11:32:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014136199999999999 2024-09-18T11:32:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357684+0000", 2024-09-18T11:32:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051342499999999999 2024-09-18T11:32:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:30.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363736+0000", 2024-09-18T11:32:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006052577 2024-09-18T11:32:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363778+0000", 2024-09-18T11:32:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1989000000000001e-05 2024-09-18T11:32:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363822+0000", 2024-09-18T11:32:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3894999999999999e-05 2024-09-18T11:32:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.416 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2937 4.7 4:e301aefd:::smithi11939506-19:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 67.364404039999997, 2024-09-18T11:32:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019287667000000001, 2024-09-18T11:32:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2937 2024-09-18T11:32:30.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.418 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:30.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:30.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:30.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:30.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034842+0000", 2024-09-18T11:32:30.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.372e-06 2024-09-18T11:32:30.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.419 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.419 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034846+0000", 2024-09-18T11:32:30.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3919999999999996e-06 2024-09-18T11:32:30.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034848+0000", 2024-09-18T11:32:30.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3539999999999998e-06 2024-09-18T11:32:30.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034893+0000", 2024-09-18T11:32:30.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4861000000000003e-05 2024-09-18T11:32:30.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034965+0000", 2024-09-18T11:32:30.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1860999999999995e-05 2024-09-18T11:32:30.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.054128+0000", 2024-09-18T11:32:30.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019163039 2024-09-18T11:32:30.421 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.421 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.421 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.421 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3453 4.f 4:f5d7df41:::smithi11939506-16:head [write 1813152~490434] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:30.421 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:32:30.421 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 44.376492112000001, 2024-09-18T11:32:30.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050685840000000003, 2024-09-18T11:32:30.422 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.422 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.422 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:30.422 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:30.422 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:30.422 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3453 2024-09-18T11:32:30.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.422 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:32:30.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:32:30.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022751+0000", 2024-09-18T11:32:30.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:30.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022871+0000", 2024-09-18T11:32:30.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000119781 2024-09-18T11:32:30.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022882+0000", 2024-09-18T11:32:30.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1627000000000001e-05 2024-09-18T11:32:30.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022887+0000", 2024-09-18T11:32:30.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2390000000000004e-06 2024-09-18T11:32:30.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023123+0000", 2024-09-18T11:32:30.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000236598 2024-09-18T11:32:30.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023207+0000", 2024-09-18T11:32:30.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4303999999999998e-05 2024-09-18T11:32:30.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.425 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:30.425 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023367+0000", 2024-09-18T11:32:30.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015983600000000001 2024-09-18T11:32:30.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:30.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073237+0000", 2024-09-18T11:32:30.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049869400000000001 2024-09-18T11:32:30.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:30.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073352+0000", 2024-09-18T11:32:30.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011579899999999999 2024-09-18T11:32:30.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073394+0000", 2024-09-18T11:32:30.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1445e-05 2024-09-18T11:32:30.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073438+0000", 2024-09-18T11:32:30.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4261999999999997e-05 2024-09-18T11:32:30.427 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.427 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.427 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.427 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3454 4.f 4:f5d7df41:::smithi11939506-16:head [setxattr _header (57) in=7b,truncate 2991679] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:30.428 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:32:30.428 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 44.37579367, 2024-09-18T11:32:30.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050365656000000002, 2024-09-18T11:32:30.428 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.428 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.428 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:30.428 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:30.428 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:30.428 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3454 2024-09-18T11:32:30.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.428 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:32:30.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:32:30.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023450+0000", 2024-09-18T11:32:30.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:30.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023456+0000", 2024-09-18T11:32:30.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8329999999999999e-06 2024-09-18T11:32:30.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023465+0000", 2024-09-18T11:32:30.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1130000000000007e-06 2024-09-18T11:32:30.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023470+0000", 2024-09-18T11:32:30.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9139999999999999e-06 2024-09-18T11:32:30.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072611+0000", 2024-09-18T11:32:30.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141643999999998 2024-09-18T11:32:30.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072729+0000", 2024-09-18T11:32:30.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117667 2024-09-18T11:32:30.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:30.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072887+0000", 2024-09-18T11:32:30.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015760000000000001 2024-09-18T11:32:30.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:30.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073558+0000", 2024-09-18T11:32:30.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00067171699999999995 2024-09-18T11:32:30.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:30.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073764+0000", 2024-09-18T11:32:30.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000205749 2024-09-18T11:32:30.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073790+0000", 2024-09-18T11:32:30.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5643000000000002e-05 2024-09-18T11:32:30.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073816+0000", 2024-09-18T11:32:30.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6588e-05 2024-09-18T11:32:30.433 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.433 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.433 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.434 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3455 4.f 4:f5d7df41:::smithi11939506-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:30.434 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:30.434 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 44.375764052000001, 2024-09-18T11:32:30.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050460466000000002, 2024-09-18T11:32:30.434 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.434 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:30.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:30.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:30.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:30.434 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3455 2024-09-18T11:32:30.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.434 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:30.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:30.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:30.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:30.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023483+0000", 2024-09-18T11:32:30.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.695e-06 2024-09-18T11:32:30.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023489+0000", 2024-09-18T11:32:30.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1079999999999998e-06 2024-09-18T11:32:30.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023492+0000", 2024-09-18T11:32:30.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.478e-06 2024-09-18T11:32:30.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073273+0000", 2024-09-18T11:32:30.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049780638000000002 2024-09-18T11:32:30.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:30.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073343+0000", 2024-09-18T11:32:30.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9770000000000005e-05 2024-09-18T11:32:30.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073846+0000", 2024-09-18T11:32:30.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050289499999999995 2024-09-18T11:32:30.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073886+0000", 2024-09-18T11:32:30.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0658999999999998e-05 2024-09-18T11:32:30.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073941+0000", 2024-09-18T11:32:30.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4642000000000003e-05 2024-09-18T11:32:30.439 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.439 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.439 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.439 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3456 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:30.439 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:30.439 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 44.375738986999998, 2024-09-18T11:32:30.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049859120999999999, 2024-09-18T11:32:30.440 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.440 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:30.440 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:30.440 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:30.440 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:30.440 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3456 2024-09-18T11:32:30.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.440 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.440 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:30.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:30.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:30.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:30.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023508+0000", 2024-09-18T11:32:30.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9490000000000001e-06 2024-09-18T11:32:30.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023513+0000", 2024-09-18T11:32:30.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3800000000000002e-06 2024-09-18T11:32:30.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023517+0000", 2024-09-18T11:32:30.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5300000000000001e-06 2024-09-18T11:32:30.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073262+0000", 2024-09-18T11:32:30.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049745194999999999 2024-09-18T11:32:30.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073365+0000", 2024-09-18T11:32:30.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102462 2024-09-18T11:32:30.443 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.443 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.443 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.443 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3457 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:30.443 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:32:30.444 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 44.375721890999998, 2024-09-18T11:32:30.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049928567, 2024-09-18T11:32:30.444 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.444 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:30.444 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:30.444 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:30.444 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:30.444 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3457 2024-09-18T11:32:30.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.445 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:32:30.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:32:30.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023522+0000", 2024-09-18T11:32:30.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:30.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023524+0000", 2024-09-18T11:32:30.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8500000000000001e-06 2024-09-18T11:32:30.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023529+0000", 2024-09-18T11:32:30.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7509999999999999e-06 2024-09-18T11:32:30.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023532+0000", 2024-09-18T11:32:30.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.163e-06 2024-09-18T11:32:30.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073386+0000", 2024-09-18T11:32:30.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049853899 2024-09-18T11:32:30.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073451+0000", 2024-09-18T11:32:30.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5197000000000006e-05 2024-09-18T11:32:30.448 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.448 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.448 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.448 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3458 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:30.448 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:30.448 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 44.375709334, 2024-09-18T11:32:30.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.05001659, 2024-09-18T11:32:30.448 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.448 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:30.449 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:30.449 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:30.449 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:30.449 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3458 2024-09-18T11:32:30.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.449 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:30.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:30.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:30.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:30.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023537+0000", 2024-09-18T11:32:30.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.64e-06 2024-09-18T11:32:30.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023543+0000", 2024-09-18T11:32:30.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3369999999999996e-06 2024-09-18T11:32:30.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023546+0000", 2024-09-18T11:32:30.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:32:30.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073466+0000", 2024-09-18T11:32:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049919719000000001 2024-09-18T11:32:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073552+0000", 2024-09-18T11:32:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6168999999999994e-05 2024-09-18T11:32:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(client.4590.0:3453 4.f e116/22 ondisk, result = 0)", 2024-09-18T11:32:30.452 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:32:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 44.374104250000002, 2024-09-18T11:32:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048099441, 2024-09-18T11:32:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:32:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:32:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025139+0000", 2024-09-18T11:32:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025144+0000", 2024-09-18T11:32:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4059999999999998e-06 2024-09-18T11:32:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072212+0000", 2024-09-18T11:32:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047068482000000002 2024-09-18T11:32:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072230+0000", 2024-09-18T11:32:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7632999999999999e-05 2024-09-18T11:32:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073224+0000", 2024-09-18T11:32:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099436899999999994 2024-09-18T11:32:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073231+0000", 2024-09-18T11:32:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-09-18T11:32:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073240+0000", 2024-09-18T11:32:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5699999999999993e-06 2024-09-18T11:32:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3930 4.b 4:d3ae8215:::smithi11939506-41:head [write 632029~547288] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:32:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.37318857, 2024-09-18T11:32:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018962711, 2024-09-18T11:32:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3930 2024-09-18T11:32:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:32:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:32:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026055+0000", 2024-09-18T11:32:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:30.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026334+0000", 2024-09-18T11:32:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000278931 2024-09-18T11:32:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026344+0000", 2024-09-18T11:32:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0716999999999999e-05 2024-09-18T11:32:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026350+0000", 2024-09-18T11:32:30.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7620000000000001e-06 2024-09-18T11:32:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028429+0000", 2024-09-18T11:32:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020787319999999998 2024-09-18T11:32:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028525+0000", 2024-09-18T11:32:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6638999999999994e-05 2024-09-18T11:32:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028720+0000", 2024-09-18T11:32:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000194525 2024-09-18T11:32:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.036996+0000", 2024-09-18T11:32:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082756140000000006 2024-09-18T11:32:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044952+0000", 2024-09-18T11:32:30.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566740000000004 2024-09-18T11:32:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044985+0000", 2024-09-18T11:32:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3081999999999999e-05 2024-09-18T11:32:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045019+0000", 2024-09-18T11:32:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3257999999999999e-05 2024-09-18T11:32:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3931 4.b 4:d3ae8215:::smithi11939506-41:head [write 1639134~749453] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.033118+0000", 2024-09-18T11:32:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.36612693, 2024-09-18T11:32:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012000667, 2024-09-18T11:32:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3931 2024-09-18T11:32:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033118+0000", 2024-09-18T11:32:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033118+0000", 2024-09-18T11:32:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033116+0000", 2024-09-18T11:32:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033408+0000", 2024-09-18T11:32:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029145000000000001 2024-09-18T11:32:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033420+0000", 2024-09-18T11:32:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2748999999999999e-05 2024-09-18T11:32:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.033426+0000", 2024-09-18T11:32:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8889999999999999e-06 2024-09-18T11:32:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.034205+0000", 2024-09-18T11:32:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00077899000000000002 2024-09-18T11:32:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.034280+0000", 2024-09-18T11:32:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4871999999999997e-05 2024-09-18T11:32:30.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.034392+0000", 2024-09-18T11:32:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000111666 2024-09-18T11:32:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.037001+0000", 2024-09-18T11:32:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026091310000000002 2024-09-18T11:32:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:30.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045055+0000", 2024-09-18T11:32:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0080539700000000006 2024-09-18T11:32:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045090+0000", 2024-09-18T11:32:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4984000000000003e-05 2024-09-18T11:32:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045118+0000", 2024-09-18T11:32:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8343999999999998e-05 2024-09-18T11:32:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.468 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3932 4.b 4:d3ae8215:::smithi11939506-41:head [write 2796249~437859] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:32:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.361181930000001, 2024-09-18T11:32:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01458329, 2024-09-18T11:32:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3932 2024-09-18T11:32:30.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:32:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:32:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038061+0000", 2024-09-18T11:32:30.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038341+0000", 2024-09-18T11:32:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027989099999999999 2024-09-18T11:32:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038353+0000", 2024-09-18T11:32:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2247e-05 2024-09-18T11:32:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038358+0000", 2024-09-18T11:32:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-18T11:32:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043226+0000", 2024-09-18T11:32:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048673049999999997 2024-09-18T11:32:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043284+0000", 2024-09-18T11:32:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8338999999999998e-05 2024-09-18T11:32:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043367+0000", 2024-09-18T11:32:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3120000000000004e-05 2024-09-18T11:32:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.051544+0000", 2024-09-18T11:32:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00817734 2024-09-18T11:32:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052528+0000", 2024-09-18T11:32:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098318499999999992 2024-09-18T11:32:30.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052610+0000", 2024-09-18T11:32:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2300999999999997e-05 2024-09-18T11:32:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052646+0000", 2024-09-18T11:32:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5955999999999999e-05 2024-09-18T11:32:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3933 4.b 4:d3ae8215:::smithi11939506-41:head [write 3653524~178505] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041300+0000", 2024-09-18T11:32:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.357944033999999, 2024-09-18T11:32:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01299752, 2024-09-18T11:32:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3933 2024-09-18T11:32:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041300+0000", 2024-09-18T11:32:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041300+0000", 2024-09-18T11:32:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041299+0000", 2024-09-18T11:32:30.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041369+0000", 2024-09-18T11:32:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9838000000000005e-05 2024-09-18T11:32:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041380+0000", 2024-09-18T11:32:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0910000000000001e-05 2024-09-18T11:32:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041385+0000", 2024-09-18T11:32:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5800000000000002e-06 2024-09-18T11:32:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044260+0000", 2024-09-18T11:32:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028751229999999998 2024-09-18T11:32:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044312+0000", 2024-09-18T11:32:30.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1808000000000003e-05 2024-09-18T11:32:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044393+0000", 2024-09-18T11:32:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1806999999999998e-05 2024-09-18T11:32:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052654+0000", 2024-09-18T11:32:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082609320000000003 2024-09-18T11:32:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054207+0000", 2024-09-18T11:32:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001552091 2024-09-18T11:32:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054256+0000", 2024-09-18T11:32:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9437000000000001e-05 2024-09-18T11:32:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054298+0000", 2024-09-18T11:32:30.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1993000000000003e-05 2024-09-18T11:32:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3934 4.b 4:d3ae8215:::smithi11939506-41:head [setxattr _header (57) in=7b,truncate 3832029] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.357855412999999, 2024-09-18T11:32:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013018336, 2024-09-18T11:32:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:30.481 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3934 2024-09-18T11:32:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041391+0000", 2024-09-18T11:32:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.26e-06 2024-09-18T11:32:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041395+0000", 2024-09-18T11:32:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8999999999999999e-06 2024-09-18T11:32:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041397+0000", 2024-09-18T11:32:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3149999999999999e-06 2024-09-18T11:32:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045266+0000", 2024-09-18T11:32:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003869094 2024-09-18T11:32:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045316+0000", 2024-09-18T11:32:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0074000000000002e-05 2024-09-18T11:32:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045403+0000", 2024-09-18T11:32:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6737999999999995e-05 2024-09-18T11:32:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052659+0000", 2024-09-18T11:32:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072558259999999999 2024-09-18T11:32:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054330+0000", 2024-09-18T11:32:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016716000000000001 2024-09-18T11:32:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054362+0000", 2024-09-18T11:32:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1195000000000002e-05 2024-09-18T11:32:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054407+0000", 2024-09-18T11:32:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5707000000000001e-05 2024-09-18T11:32:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3935 4.b 4:d3ae8215:::smithi11939506-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:32:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.357844823000001, 2024-09-18T11:32:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013142333000000001, 2024-09-18T11:32:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3935 2024-09-18T11:32:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.487 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:32:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:32:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041399+0000", 2024-09-18T11:32:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041401+0000", 2024-09-18T11:32:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4890000000000001e-06 2024-09-18T11:32:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041404+0000", 2024-09-18T11:32:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5240000000000001e-06 2024-09-18T11:32:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041407+0000", 2024-09-18T11:32:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2230000000000001e-06 2024-09-18T11:32:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045699+0000", 2024-09-18T11:32:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042918970000000002 2024-09-18T11:32:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045750+0000", 2024-09-18T11:32:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1332999999999998e-05 2024-09-18T11:32:30.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054438+0000", 2024-09-18T11:32:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086882569999999996 2024-09-18T11:32:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054485+0000", 2024-09-18T11:32:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7151000000000003e-05 2024-09-18T11:32:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054542+0000", 2024-09-18T11:32:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6706999999999998e-05 2024-09-18T11:32:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.0:343 4.a e137/22 4:5975a5ec:::smithi11939506-36:49 v 137'218, mlcod=132'212)", 2024-09-18T11:32:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:08.278418+0000", 2024-09-18T11:32:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.120826313999999, 2024-09-18T11:32:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012396378, 2024-09-18T11:32:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.492 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278418+0000", 2024-09-18T11:32:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278418+0000", 2024-09-18T11:32:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278418+0000", 2024-09-18T11:32:30.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:30.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278422+0000", 2024-09-18T11:32:30.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.189e-06 2024-09-18T11:32:30.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278428+0000", 2024-09-18T11:32:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7400000000000001e-06 2024-09-18T11:32:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.278431+0000", 2024-09-18T11:32:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4790000000000001e-06 2024-09-18T11:32:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.283059+0000", 2024-09-18T11:32:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046282750000000003 2024-09-18T11:32:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.283072+0000", 2024-09-18T11:32:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2819999999999999e-05 2024-09-18T11:32:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.290780+0000", 2024-09-18T11:32:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077080990000000004 2024-09-18T11:32:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.290815+0000", 2024-09-18T11:32:30.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4224000000000003e-05 2024-09-18T11:32:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3969 4.4 4:2cb3e6a5:::smithi11939506-3:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:32:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 20.825604557999998, 2024-09-18T11:32:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.024248042000000001, 2024-09-18T11:32:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:30.497 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3969 2024-09-18T11:32:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:32:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:32:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573639+0000", 2024-09-18T11:32:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573646+0000", 2024-09-18T11:32:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6569999999999997e-06 2024-09-18T11:32:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573662+0000", 2024-09-18T11:32:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5187999999999999e-05 2024-09-18T11:32:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573670+0000", 2024-09-18T11:32:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5399999999999996e-06 2024-09-18T11:32:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8592999999999999e-05 2024-09-18T11:32:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573818+0000", 2024-09-18T11:32:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9388999999999995e-05 2024-09-18T11:32:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597888+0000", 2024-09-18T11:32:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.024069859999999998 2024-09-18T11:32:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.501 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3973 4.7 4:e73ba68f:::smithi11939506-1:head [tier-promote] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 20.825505479, 2024-09-18T11:32:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.023911297000000001, 2024-09-18T11:32:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3973 2024-09-18T11:32:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573738+0000", 2024-09-18T11:32:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573743+0000", 2024-09-18T11:32:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0839999999999999e-06 2024-09-18T11:32:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573752+0000", 2024-09-18T11:32:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3080000000000008e-06 2024-09-18T11:32:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573758+0000", 2024-09-18T11:32:30.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2659999999999998e-06 2024-09-18T11:32:30.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573803+0000", 2024-09-18T11:32:30.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5243e-05 2024-09-18T11:32:30.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573897+0000", 2024-09-18T11:32:30.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3398000000000003e-05 2024-09-18T11:32:30.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597650+0000", 2024-09-18T11:32:30.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.023753672999999999 2024-09-18T11:32:30.506 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.506 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.506 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.506 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.506 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.506 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:30.508 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:32:30.508 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:30.506+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:30.608 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:30.607+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:30.647 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:30.647 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:30.647 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:30.647 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:30.655 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:32:30.702 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:30.701+0000 7fcb662ad640 -1 osd.7 158 handle_osd_map injecting map crc failure 2024-09-18T11:32:30.739 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:30.738+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:30.792 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:30.793 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:30.793 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:30.793 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:30.793 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:30.801 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:32:30.809 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:30.808+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:30.910 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:30.909+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:30.940 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:30.941 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:30.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:30.941 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:30.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.941 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:32:30.941 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:30.941 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 130.955511882, 2024-09-18T11:32:30.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:32:30.941 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.941 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.941 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:30.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:30.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:32:30.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:30.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:32:30.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:32:30.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:32:30.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:32:30.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:32:30.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:32:30.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:32:30.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:32:30.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:32:30.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:32:30.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:32:30.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:32:30.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:32:30.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:32:30.945 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.946 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.946 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.946 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:32:30.946 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:30.946 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 121.720680929, 2024-09-18T11:32:30.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:32:30.946 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.946 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.946 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:30.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:30.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:30.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:30.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:32:30.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:32:30.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:32:30.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:32:30.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:32:30.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:32:30.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:32:30.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:32:30.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:32:30.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:32:30.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:32:30.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:32:30.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:32:30.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:32:30.950 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.950 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.950 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.950 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:32:30.950 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:30.950 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 116.546774644, 2024-09-18T11:32:30.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:32:30.951 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.951 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.951 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:30.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:30.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:32:30.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:30.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:32:30.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:32:30.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:32:30.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:32:30.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:32:30.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:32:30.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:32:30.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:32:30.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:32:30.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:32:30.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:32:30.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:32:30.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:32:30.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:32:30.955 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.955 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.955 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.955 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:304 4.6 e69/67 4:67c1765a:::smithi11939506-30:17 v 69'35, mlcod=58'31)", 2024-09-18T11:32:30.956 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:30.956 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.592933549999998, 2024-09-18T11:32:30.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006377611, 2024-09-18T11:32:30.956 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.956 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.956 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:30.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:30.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:30.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:30.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345961+0000", 2024-09-18T11:32:30.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2050000000000008e-06 2024-09-18T11:32:30.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345969+0000", 2024-09-18T11:32:30.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0339999999999997e-06 2024-09-18T11:32:30.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345976+0000", 2024-09-18T11:32:30.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-18T11:32:30.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346026+0000", 2024-09-18T11:32:30.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9286999999999998e-05 2024-09-18T11:32:30.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346050+0000", 2024-09-18T11:32:30.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4397e-05 2024-09-18T11:32:30.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352293+0000", 2024-09-18T11:32:30.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062427170000000001 2024-09-18T11:32:30.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352330+0000", 2024-09-18T11:32:30.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7348999999999998e-05 2024-09-18T11:32:30.960 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.960 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.960 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.960 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:347 4.d e69/22 4:b44229a1:::smithi11939506-42:19 v 69'99, mlcod=65'93)", 2024-09-18T11:32:30.960 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:30.960 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.587200354999993, 2024-09-18T11:32:30.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056998760000000004, 2024-09-18T11:32:30.960 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.960 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.960 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:30.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:30.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351684+0000", 2024-09-18T11:32:30.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:30.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351693+0000", 2024-09-18T11:32:30.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-18T11:32:30.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351704+0000", 2024-09-18T11:32:30.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1077000000000001e-05 2024-09-18T11:32:30.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351709+0000", 2024-09-18T11:32:30.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5419999999999999e-06 2024-09-18T11:32:30.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351956+0000", 2024-09-18T11:32:30.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000247252 2024-09-18T11:32:30.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351975+0000", 2024-09-18T11:32:30.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9201000000000001e-05 2024-09-18T11:32:30.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357334+0000", 2024-09-18T11:32:30.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053588749999999999 2024-09-18T11:32:30.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357385+0000", 2024-09-18T11:32:30.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1406999999999998e-05 2024-09-18T11:32:30.964 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.965 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.965 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.965 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1956 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'37, mlcod=58'31)", 2024-09-18T11:32:30.965 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:30.965 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.586255441999995, 2024-09-18T11:32:30.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010958842, 2024-09-18T11:32:30.965 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.965 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.965 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:30.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:30.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:30.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:30.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352996+0000", 2024-09-18T11:32:30.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036622 2024-09-18T11:32:30.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353011+0000", 2024-09-18T11:32:30.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4849e-05 2024-09-18T11:32:30.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353017+0000", 2024-09-18T11:32:30.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:32:30.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353068+0000", 2024-09-18T11:32:30.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0695000000000002e-05 2024-09-18T11:32:30.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353092+0000", 2024-09-18T11:32:30.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4260000000000002e-05 2024-09-18T11:32:30.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363551+0000", 2024-09-18T11:32:30.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010459154999999999 2024-09-18T11:32:30.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363589+0000", 2024-09-18T11:32:30.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8161000000000003e-05 2024-09-18T11:32:30.969 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.969 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.969 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.970 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1957 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'38, mlcod=65'33)", 2024-09-18T11:32:30.970 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:30.970 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.579839574999994, 2024-09-18T11:32:30.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050499400000000002, 2024-09-18T11:32:30.970 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.970 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.970 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:30.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:30.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:30.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:30.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359214+0000", 2024-09-18T11:32:30.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000168047 2024-09-18T11:32:30.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359227+0000", 2024-09-18T11:32:30.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3124e-05 2024-09-18T11:32:30.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359235+0000", 2024-09-18T11:32:30.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0760000000000003e-06 2024-09-18T11:32:30.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359287+0000", 2024-09-18T11:32:30.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.151e-05 2024-09-18T11:32:30.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359310+0000", 2024-09-18T11:32:30.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-09-18T11:32:30.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364065+0000", 2024-09-18T11:32:30.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047549439999999997 2024-09-18T11:32:30.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364096+0000", 2024-09-18T11:32:30.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1423000000000001e-05 2024-09-18T11:32:30.974 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.974 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.974 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.974 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2300 4.4 e77/22 4:2c93196f:::smithi11939506-28:head v 77'94, mlcod=77'90)", 2024-09-18T11:32:30.975 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:30.975 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 89.950777357000007, 2024-09-18T11:32:30.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050425380000000001, 2024-09-18T11:32:30.975 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.975 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.975 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:30.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:30.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988108+0000", 2024-09-18T11:32:30.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:30.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988388+0000", 2024-09-18T11:32:30.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028087700000000001 2024-09-18T11:32:30.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988405+0000", 2024-09-18T11:32:30.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6565000000000001e-05 2024-09-18T11:32:30.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988412+0000", 2024-09-18T11:32:30.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9709999999999998e-06 2024-09-18T11:32:30.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988464+0000", 2024-09-18T11:32:30.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1949e-05 2024-09-18T11:32:30.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988490+0000", 2024-09-18T11:32:30.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5780999999999999e-05 2024-09-18T11:32:30.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993112+0000", 2024-09-18T11:32:30.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046228399999999996 2024-09-18T11:32:30.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993151+0000", 2024-09-18T11:32:30.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8587999999999997e-05 2024-09-18T11:32:30.979 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.979 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.979 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.979 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.13:63 3.0 e78/21 3:24ebb740:::smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 78'80, mlcod=73'78)", 2024-09-18T11:32:30.979 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:30.979 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 89.533787941, 2024-09-18T11:32:30.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052262230000000003, 2024-09-18T11:32:30.979 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.979 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.979 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:30.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:30.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405097+0000", 2024-09-18T11:32:30.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:30.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405103+0000", 2024-09-18T11:32:30.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8780000000000003e-06 2024-09-18T11:32:30.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405117+0000", 2024-09-18T11:32:30.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3441000000000001e-05 2024-09-18T11:32:30.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405121+0000", 2024-09-18T11:32:30.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7099999999999998e-06 2024-09-18T11:32:30.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405158+0000", 2024-09-18T11:32:30.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6477999999999999e-05 2024-09-18T11:32:30.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405173+0000", 2024-09-18T11:32:30.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4796e-05 2024-09-18T11:32:30.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410285+0000", 2024-09-18T11:32:30.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051118919999999998 2024-09-18T11:32:30.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410324+0000", 2024-09-18T11:32:30.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9616999999999997e-05 2024-09-18T11:32:30.984 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.984 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.984 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.984 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.13:18 3.0 e91/21 3:35e3c617:::2e64dd273da7619bd64b294f52905c1b9405a1ced2a8fd3c0b5bc481bc9732a6:head v 91'89, mlcod=85'87)", 2024-09-18T11:32:30.984 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:30.984 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 76.917012529999994, 2024-09-18T11:32:30.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048558539999999997, 2024-09-18T11:32:30.984 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.984 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.984 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:30.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:30.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:30.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:30.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021880+0000", 2024-09-18T11:32:30.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5860000000000001e-06 2024-09-18T11:32:30.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021890+0000", 2024-09-18T11:32:30.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8290000000000003e-06 2024-09-18T11:32:30.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021895+0000", 2024-09-18T11:32:30.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2480000000000004e-06 2024-09-18T11:32:30.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022166+0000", 2024-09-18T11:32:30.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027073499999999998 2024-09-18T11:32:30.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022184+0000", 2024-09-18T11:32:30.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8082000000000001e-05 2024-09-18T11:32:30.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026693+0000", 2024-09-18T11:32:30.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045093479999999998 2024-09-18T11:32:30.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026729+0000", 2024-09-18T11:32:30.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6282999999999998e-05 2024-09-18T11:32:30.988 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.988 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.988 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.989 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:573 4.4 e91/22 4:2c93196f:::smithi11939506-28:26 v 91'135, mlcod=90'131)", 2024-09-18T11:32:30.989 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:30.989 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 76.914545941, 2024-09-18T11:32:30.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050722739999999999, 2024-09-18T11:32:30.989 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.989 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.989 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:30.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:30.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024339+0000", 2024-09-18T11:32:30.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:30.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024349+0000", 2024-09-18T11:32:30.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6339999999999996e-06 2024-09-18T11:32:30.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024362+0000", 2024-09-18T11:32:30.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-18T11:32:30.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024368+0000", 2024-09-18T11:32:30.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8649999999999998e-06 2024-09-18T11:32:30.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:30.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024431+0000", 2024-09-18T11:32:30.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2780999999999994e-05 2024-09-18T11:32:30.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:30.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024448+0000", 2024-09-18T11:32:30.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7388999999999999e-05 2024-09-18T11:32:30.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:30.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029385+0000", 2024-09-18T11:32:30.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004936523 2024-09-18T11:32:30.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:30.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029412+0000", 2024-09-18T11:32:30.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7702999999999999e-05 2024-09-18T11:32:30.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.994 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:30.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:30.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.995 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:574 4.4 e91/22 4:2cb3e6a5:::smithi11939506-3:1e v 91'137, mlcod=90'131)", 2024-09-18T11:32:30.995 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:30.995 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 76.914107897999997, 2024-09-18T11:32:30.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070802850000000004, 2024-09-18T11:32:30.995 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:30.995 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:30.996 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:30.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:30.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:30.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:30.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:30.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:30.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:30.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024777+0000", 2024-09-18T11:32:30.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:30.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:30.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024785+0000", 2024-09-18T11:32:30.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8099999999999998e-06 2024-09-18T11:32:30.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:30.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024794+0000", 2024-09-18T11:32:30.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5839999999999995e-06 2024-09-18T11:32:30.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:30.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:30.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:30.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024798+0000", 2024-09-18T11:32:31.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.031e-06 2024-09-18T11:32:31.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024830+0000", 2024-09-18T11:32:31.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2268999999999999e-05 2024-09-18T11:32:31.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024847+0000", 2024-09-18T11:32:31.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7306999999999999e-05 2024-09-18T11:32:31.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:31.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031834+0000", 2024-09-18T11:32:31.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069868380000000004 2024-09-18T11:32:31.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031858+0000", 2024-09-18T11:32:31.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4082000000000001e-05 2024-09-18T11:32:31.001 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.002 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.002 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.002 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:541 4.d e91/22 4:b9bb642b:::smithi11939506-27:19 v 91'158, mlcod=91'154)", 2024-09-18T11:32:31.002 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:31.002 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 76.906727205999999, 2024-09-18T11:32:31.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052185, 2024-09-18T11:32:31.002 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.002 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.002 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:31.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:31.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032158+0000", 2024-09-18T11:32:31.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:31.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032165+0000", 2024-09-18T11:32:31.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-09-18T11:32:31.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032176+0000", 2024-09-18T11:32:31.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1460999999999999e-05 2024-09-18T11:32:31.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032182+0000", 2024-09-18T11:32:31.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0870000000000003e-06 2024-09-18T11:32:31.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032224+0000", 2024-09-18T11:32:31.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1372999999999998e-05 2024-09-18T11:32:31.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032246+0000", 2024-09-18T11:32:31.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2158e-05 2024-09-18T11:32:31.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:31.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037348+0000", 2024-09-18T11:32:31.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051025319999999999 2024-09-18T11:32:31.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037377+0000", 2024-09-18T11:32:31.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8937000000000001e-05 2024-09-18T11:32:31.006 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.006 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.007 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.007 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4178.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-18T11:32:31.007 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:31.007 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 48.998065140999998, 2024-09-18T11:32:31.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048960139999999997, 2024-09-18T11:32:31.007 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.007 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:31.007 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:31.007 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4178", 2024-09-18T11:32:31.007 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/490761693", 2024-09-18T11:32:31.007 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4 2024-09-18T11:32:31.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.008 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:31.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:31.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940819+0000", 2024-09-18T11:32:31.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:31.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940832+0000", 2024-09-18T11:32:31.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3155e-05 2024-09-18T11:32:31.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940848+0000", 2024-09-18T11:32:31.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6288999999999999e-05 2024-09-18T11:32:31.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940856+0000", 2024-09-18T11:32:31.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4919999999999996e-06 2024-09-18T11:32:31.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945613+0000", 2024-09-18T11:32:31.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047574999999999996 2024-09-18T11:32:31.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945717+0000", 2024-09-18T11:32:31.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103553 2024-09-18T11:32:31.011 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.011 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.011 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.011 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3482 4.6 e117/67 4:67c1765a:::smithi11939506-30:head v 117'120, mlcod=115'116)", 2024-09-18T11:32:31.011 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:31.011 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 44.254210917999998, 2024-09-18T11:32:31.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050037479999999997, 2024-09-18T11:32:31.011 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.012 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.012 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:31.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:31.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684674+0000", 2024-09-18T11:32:31.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:31.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684950+0000", 2024-09-18T11:32:31.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027621200000000002 2024-09-18T11:32:31.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684968+0000", 2024-09-18T11:32:31.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8e-05 2024-09-18T11:32:31.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684984+0000", 2024-09-18T11:32:31.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5775000000000001e-05 2024-09-18T11:32:31.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685033+0000", 2024-09-18T11:32:31.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8878000000000002e-05 2024-09-18T11:32:31.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685059+0000", 2024-09-18T11:32:31.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6801999999999999e-05 2024-09-18T11:32:31.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:31.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689640+0000", 2024-09-18T11:32:31.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045809029999999999 2024-09-18T11:32:31.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689679+0000", 2024-09-18T11:32:31.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8507000000000002e-05 2024-09-18T11:32:31.016 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.016 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.016 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.016 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:976 4.4 e132/22 4:2cb3e6a5:::smithi11939506-3:30 v 132'195, mlcod=128'192)", 2024-09-18T11:32:31.016 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:31.016 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.920239847000001, 2024-09-18T11:32:31.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0058526510000000004, 2024-09-18T11:32:31.017 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.017 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.017 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:31.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:31.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:31.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:31.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018661+0000", 2024-09-18T11:32:31.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5565000000000001e-05 2024-09-18T11:32:31.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018672+0000", 2024-09-18T11:32:31.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0210000000000001e-05 2024-09-18T11:32:31.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018678+0000", 2024-09-18T11:32:31.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1539999999999999e-06 2024-09-18T11:32:31.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018716+0000", 2024-09-18T11:32:31.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8389000000000001e-05 2024-09-18T11:32:31.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018737+0000", 2024-09-18T11:32:31.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0611e-05 2024-09-18T11:32:31.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:31.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.024458+0000", 2024-09-18T11:32:31.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057208629999999996 2024-09-18T11:32:31.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.024499+0000", 2024-09-18T11:32:31.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1100999999999998e-05 2024-09-18T11:32:31.022 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.022 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.022 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.022 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:823 4.d e137/22 4:b44229a1:::smithi11939506-42:49 v 137'266, mlcod=132'260)", 2024-09-18T11:32:31.023 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:32:31.023 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.661591291000001, 2024-09-18T11:32:31.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054497770000000003, 2024-09-18T11:32:31.023 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.023 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.024 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:32:31.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:32:31.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277294+0000", 2024-09-18T11:32:31.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:31.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277299+0000", 2024-09-18T11:32:31.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6280000000000004e-06 2024-09-18T11:32:31.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277306+0000", 2024-09-18T11:32:31.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.533e-06 2024-09-18T11:32:31.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277316+0000", 2024-09-18T11:32:31.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0148e-05 2024-09-18T11:32:31.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277868+0000", 2024-09-18T11:32:31.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00055199999999999997 2024-09-18T11:32:31.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277898+0000", 2024-09-18T11:32:31.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0264999999999999e-05 2024-09-18T11:32:31.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:31.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.282715+0000", 2024-09-18T11:32:31.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048165619999999999 2024-09-18T11:32:31.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.282744+0000", 2024-09-18T11:32:31.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9855000000000001e-05 2024-09-18T11:32:31.029 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.029 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.029 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.030 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:163 3.0 e138/21 3:02e8b30e:::4456d047c09ae2a9a25ccebfd83e30cf78df6779d63bfba9365afd7301eaeb6a:head v 138'132, mlcod=134'130)", 2024-09-18T11:32:31.030 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:31.030 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.356728548, 2024-09-18T11:32:31.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051351770000000003, 2024-09-18T11:32:31.030 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.030 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.030 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:31.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:31.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:31.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:31.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582232+0000", 2024-09-18T11:32:31.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5333999999999994e-05 2024-09-18T11:32:31.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582242+0000", 2024-09-18T11:32:31.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5200000000000003e-06 2024-09-18T11:32:31.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582248+0000", 2024-09-18T11:32:31.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8000000000000004e-06 2024-09-18T11:32:31.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582305+0000", 2024-09-18T11:32:31.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7813999999999999e-05 2024-09-18T11:32:31.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582329+0000", 2024-09-18T11:32:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3414999999999999e-05 2024-09-18T11:32:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587257+0000", 2024-09-18T11:32:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049279880000000003 2024-09-18T11:32:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587293+0000", 2024-09-18T11:32:31.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5802e-05 2024-09-18T11:32:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3974 4.d e138/22 4:b5a463bf:::smithi11939506-20:head v 138'270, mlcod=137'266)", 2024-09-18T11:32:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:32:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.352847276999999, 2024-09-18T11:32:31.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011228458, 2024-09-18T11:32:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:32:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:32:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586038+0000", 2024-09-18T11:32:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587638+0000", 2024-09-18T11:32:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001599184 2024-09-18T11:32:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587647+0000", 2024-09-18T11:32:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3719999999999994e-06 2024-09-18T11:32:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587653+0000", 2024-09-18T11:32:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.105e-06 2024-09-18T11:32:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587712+0000", 2024-09-18T11:32:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9252000000000001e-05 2024-09-18T11:32:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587735+0000", 2024-09-18T11:32:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3132000000000001e-05 2024-09-18T11:32:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597228+0000", 2024-09-18T11:32:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0094930090000000002 2024-09-18T11:32:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597267+0000", 2024-09-18T11:32:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8693999999999999e-05 2024-09-18T11:32:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.040 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3975 4.d e138/22 4:b5a463bf:::smithi11939506-20:head v 138'271, mlcod=137'266)", 2024-09-18T11:32:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.346646571000001, 2024-09-18T11:32:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053620450000000002, 2024-09-18T11:32:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592927+0000", 2024-09-18T11:32:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00068878699999999995 2024-09-18T11:32:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592940+0000", 2024-09-18T11:32:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:32:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592951+0000", 2024-09-18T11:32:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1046000000000001e-05 2024-09-18T11:32:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593008+0000", 2024-09-18T11:32:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6875e-05 2024-09-18T11:32:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593033+0000", 2024-09-18T11:32:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5341999999999999e-05 2024-09-18T11:32:31.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597565+0000", 2024-09-18T11:32:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045323150000000003 2024-09-18T11:32:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597601+0000", 2024-09-18T11:32:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6071000000000001e-05 2024-09-18T11:32:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.045 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.046 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.046 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.046 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:31.048 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:32:31.048 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:31.046+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:31.148 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:31.147+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:31.188 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:31.188 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:31.188 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:31.188 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:31.197 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:32:31.281 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:31.280+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:31.335 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:31.335 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:31.335 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:31.335 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:31.336 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:31.344 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:32:31.350 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:31.349+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:31.380 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:31.379+0000 7f3157e90640 -1 osd.4 159 handle_osd_map injecting map crc failure 2024-09-18T11:32:31.383 INFO:tasks.rados.rados.0.smithi119.stdout:1872: done (1 left) 2024-09-18T11:32:31.383 INFO:tasks.rados.rados.0.smithi119.stdout:1873: read oid 2 snap -1 2024-09-18T11:32:31.383 INFO:tasks.rados.rados.0.smithi119.stdout:1873: expect (ObjNum 375 snap 58 seq_num 375) 2024-09-18T11:32:31.383 INFO:tasks.rados.rados.0.smithi119.stdout:1874: write (excl) oid 15 current snap is 60 2024-09-18T11:32:31.384 INFO:tasks.rados.rados.0.smithi119.stdout:1871: finishing write tid 1 to smithi11939506-47 2024-09-18T11:32:31.384 INFO:tasks.rados.rados.0.smithi119.stdout:1874: seq_num 381 ranges {403467=487519,1562110=579120} 2024-09-18T11:32:31.387 INFO:tasks.rados.rados.0.smithi119.stdout:1874: writing smithi11939506-15 from 403467 to 890986 tid 1 2024-09-18T11:32:31.388 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:31.387+0000 7f2d449e8640 -1 osd.1 159 handle_osd_map injecting map crc failure 2024-09-18T11:32:31.397 INFO:tasks.rados.rados.0.smithi119.stdout:1874: writing smithi11939506-15 from 1562110 to 2141230 tid 2 2024-09-18T11:32:31.397 INFO:tasks.rados.rados.0.smithi119.stdout:1875: rollback oid 25 current snap is 60 2024-09-18T11:32:31.397 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 25 to 57 2024-09-18T11:32:31.398 INFO:tasks.rados.rados.0.smithi119.stdout:1876: tier_evict oid 18 2024-09-18T11:32:31.398 INFO:tasks.rados.rados.0.smithi119.stdout:1876: tier_evict oid 18 snap -1 2024-09-18T11:32:31.398 INFO:tasks.rados.rados.0.smithi119.stdout:1877: tier_evict oid 41 2024-09-18T11:32:31.398 INFO:tasks.rados.rados.0.smithi119.stdout:1877: tier_evict oid 41 snap -1 2024-09-18T11:32:31.398 INFO:tasks.rados.rados.0.smithi119.stdout:1871: finishing write tid 2 to smithi11939506-47 2024-09-18T11:32:31.398 INFO:tasks.rados.rados.0.smithi119.stdout:1871: finishing write tid 3 to smithi11939506-47 2024-09-18T11:32:31.398 INFO:tasks.rados.rados.0.smithi119.stdout:1871: finishing write tid 4 to smithi11939506-47 2024-09-18T11:32:31.398 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 178 (ObjNum 380 snap 60 seq_num 380) dirty exists 2024-09-18T11:32:31.398 INFO:tasks.rados.rados.0.smithi119.stdout:1871: left oid 47 (ObjNum 380 snap 60 seq_num 380) 2024-09-18T11:32:31.398 INFO:tasks.rados.rados.0.smithi119.stdout:1873: expect (ObjNum 375 snap 58 seq_num 375) 2024-09-18T11:32:31.451 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:31.449+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:31.490 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:31.490 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:31.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:31.490 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:31.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.490 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:32:31.490 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:31.490 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 140.318179577, 2024-09-18T11:32:31.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:32:31.491 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.491 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:31.491 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:31.491 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:31.491 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:31.491 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:32:31.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.491 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:31.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:31.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:32:31.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:31.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:32:31.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:32:31.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:32:31.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:32:31.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:32:31.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:32:31.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:32:31.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:32:31.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:32:31.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:32:31.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:32:31.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:32:31.495 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.495 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.496 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.496 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1001 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:31.496 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:32:31.496 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 136.32302829899999, 2024-09-18T11:32:31.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007461082, 2024-09-18T11:32:31.496 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.496 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:31.496 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:31.496 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:31.496 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:31.497 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1001 2024-09-18T11:32:31.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.497 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:32:31.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:32:31.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164570+0000", 2024-09-18T11:32:31.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:31.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164574+0000", 2024-09-18T11:32:31.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.822e-06 2024-09-18T11:32:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164582+0000", 2024-09-18T11:32:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3410000000000003e-06 2024-09-18T11:32:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164587+0000", 2024-09-18T11:32:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769999999999999e-06 2024-09-18T11:32:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164702+0000", 2024-09-18T11:32:31.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115705 2024-09-18T11:32:31.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:31.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164709+0000", 2024-09-18T11:32:31.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-09-18T11:32:31.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170296+0000", 2024-09-18T11:32:31.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055876390000000001 2024-09-18T11:32:31.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170418+0000", 2024-09-18T11:32:31.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121549 2024-09-18T11:32:31.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172032+0000", 2024-09-18T11:32:31.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00161407 2024-09-18T11:32:31.501 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.501 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.502 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.502 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:31.502 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:31.502 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 136.32300158300001, 2024-09-18T11:32:31.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:32:31.502 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.502 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:31.502 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:31.502 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:31.503 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:31.503 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:32:31.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.503 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:31.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:31.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:31.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:31.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:32:31.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:32:31.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:32:31.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:32:31.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:32:31.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:32:31.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:32:31.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:32:31.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:31.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:32:31.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:32:31.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:32:31.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:32:31.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:32:31.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:32:31.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:32:31.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:32:31.511 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.511 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.511 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.511 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:31.511 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:31.511 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 136.32298329400001, 2024-09-18T11:32:31.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:32:31.511 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.512 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:31.512 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:31.512 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:31.512 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:31.512 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:32:31.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.512 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:31.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:31.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:32:31.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:31.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:32:31.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:32:31.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:32:31.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:32:31.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:32:31.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:32:31.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:32:31.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:32:31.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:31.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:32:31.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:32:31.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:32:31.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:32:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:32:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:32:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:32:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:32:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.518 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.518 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:31.518 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:31.518 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 136.322922525, 2024-09-18T11:32:31.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:32:31.518 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.518 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:31.518 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:31.518 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:31.518 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:31.519 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:32:31.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.519 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:31.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:31.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:32:31.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:31.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:32:31.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:32:31.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:32:31.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:32:31.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:32:31.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:32:31.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:32:31.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:32:31.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:31.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:32:31.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:32:31.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:32:31.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:32:31.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:32:31.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:32:31.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:32:31.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:32:31.524 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.524 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.524 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.524 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:31.524 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:31.524 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 136.32290020400001, 2024-09-18T11:32:31.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:32:31.525 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.525 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:31.525 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:31.525 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:31.525 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:31.525 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:32:31.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.525 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:31.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:31.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:32:31.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:31.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:32:31.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:32:31.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:32:31.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:32:31.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:32:31.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:32:31.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:32:31.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:32:31.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:31.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:32:31.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:32:31.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:32:31.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:32:31.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:32:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:32:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:32:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:32:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:32:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:31.531 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 131.54177532099999, 2024-09-18T11:32:31.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:32:31.531 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.531 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.531 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:31.531 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:31.531 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:31.531 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:32:31.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.531 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:31.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:31.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:32:31.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:32:31.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:32:31.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:32:31.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:32:31.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:32:31.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:32:31.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:32:31.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:32:31.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:32:31.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:32:31.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:32:31.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:31.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:32:31.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:32:31.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:31.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:32:31.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:32:31.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:31.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:32:31.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:32:31.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:31.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:32:31.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:32:31.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:32:31.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:32:31.537 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.537 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.537 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.538 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1661 4.0 4:019b6e09:::smithi11939506-40:17 [read 0~3445402,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:32:31.538 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:31.538 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.13770261800001, 2024-09-18T11:32:31.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075190880000000002, 2024-09-18T11:32:31.538 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.538 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:31.538 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:31.538 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:31.538 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:31.539 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1661 2024-09-18T11:32:31.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.539 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:31.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:31.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:31.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:31.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349899+0000", 2024-09-18T11:32:31.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9749999999999999e-06 2024-09-18T11:32:31.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349906+0000", 2024-09-18T11:32:31.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9290000000000001e-06 2024-09-18T11:32:31.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349912+0000", 2024-09-18T11:32:31.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9900000000000002e-06 2024-09-18T11:32:31.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355645+0000", 2024-09-18T11:32:31.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057329549999999996 2024-09-18T11:32:31.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355776+0000", 2024-09-18T11:32:31.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130975 2024-09-18T11:32:31.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357416+0000", 2024-09-18T11:32:31.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001639649 2024-09-18T11:32:31.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.543 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.543 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1761 4.0 4:0aec5e1c:::smithi11939506-50:head [write 769787~704463,stat] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:32:31.543 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:32:31.543 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.185210204, 2024-09-18T11:32:31.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074082879999999999, 2024-09-18T11:32:31.544 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.544 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.544 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:31.544 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:31.544 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:31.544 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1761 2024-09-18T11:32:31.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.544 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:32:31.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:32:31.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302386+0000", 2024-09-18T11:32:31.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:31.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302889+0000", 2024-09-18T11:32:31.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050318499999999996 2024-09-18T11:32:31.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302930+0000", 2024-09-18T11:32:31.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0198999999999999e-05 2024-09-18T11:32:31.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302965+0000", 2024-09-18T11:32:31.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5292e-05 2024-09-18T11:32:31.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303046+0000", 2024-09-18T11:32:31.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1254999999999994e-05 2024-09-18T11:32:31.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:32:31.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303054+0000", 2024-09-18T11:32:31.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2800000000000003e-06 2024-09-18T11:32:31.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304658+0000", 2024-09-18T11:32:31.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016039520000000001 2024-09-18T11:32:31.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304724+0000", 2024-09-18T11:32:31.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5888000000000003e-05 2024-09-18T11:32:31.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:31.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304853+0000", 2024-09-18T11:32:31.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012912700000000001 2024-09-18T11:32:31.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:31.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309084+0000", 2024-09-18T11:32:31.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042311229999999998 2024-09-18T11:32:31.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:31.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309738+0000", 2024-09-18T11:32:31.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00065401399999999996 2024-09-18T11:32:31.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:31.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309770+0000", 2024-09-18T11:32:31.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1652000000000001e-05 2024-09-18T11:32:31.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309797+0000", 2024-09-18T11:32:31.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7052000000000001e-05 2024-09-18T11:32:31.551 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.551 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.552 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.552 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2503 3.1 e85/67 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head v 85'117, mlcod=84'115)", 2024-09-18T11:32:31.552 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:31.552 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 82.510376633999996, 2024-09-18T11:32:31.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086527619999999996, 2024-09-18T11:32:31.552 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.552 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.552 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:31.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:31.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977220+0000", 2024-09-18T11:32:31.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:31.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980771+0000", 2024-09-18T11:32:31.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035505200000000002 2024-09-18T11:32:31.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980797+0000", 2024-09-18T11:32:31.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.618e-05 2024-09-18T11:32:31.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980807+0000", 2024-09-18T11:32:31.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0433e-05 2024-09-18T11:32:31.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980861+0000", 2024-09-18T11:32:31.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3415999999999998e-05 2024-09-18T11:32:31.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980885+0000", 2024-09-18T11:32:31.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4212000000000001e-05 2024-09-18T11:32:31.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:31.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985826+0000", 2024-09-18T11:32:31.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049412450000000004 2024-09-18T11:32:31.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985875+0000", 2024-09-18T11:32:31.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9116000000000002e-05 2024-09-18T11:32:31.557 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.557 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.557 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.558 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2648 4.0 4:059d8d95:::smithi11939506-32:head [read 0~3647119,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:31.558 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:32:31.558 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.156288263999997, 2024-09-18T11:32:31.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0076286949999999996, 2024-09-18T11:32:31.558 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.558 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:31.558 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:31.559 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:31.559 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:31.559 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2648 2024-09-18T11:32:31.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.559 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:32:31.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:32:31.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331309+0000", 2024-09-18T11:32:31.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:31.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331321+0000", 2024-09-18T11:32:31.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:32:31.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331340+0000", 2024-09-18T11:32:31.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8227e-05 2024-09-18T11:32:31.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331348+0000", 2024-09-18T11:32:31.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4640000000000006e-06 2024-09-18T11:32:31.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331406+0000", 2024-09-18T11:32:31.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7445000000000003e-05 2024-09-18T11:32:31.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331521+0000", 2024-09-18T11:32:31.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115945 2024-09-18T11:32:31.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338939+0000", 2024-09-18T11:32:31.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074179859999999997 2024-09-18T11:32:31.563 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.563 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.563 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.563 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2649 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:31.563 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:31.564 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.156233264999997, 2024-09-18T11:32:31.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566240000000007, 2024-09-18T11:32:31.564 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.564 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:31.564 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:31.564 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:31.564 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:31.564 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2649 2024-09-18T11:32:31.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.564 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:31.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:31.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331365+0000", 2024-09-18T11:32:31.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:31.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331369+0000", 2024-09-18T11:32:31.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.551e-06 2024-09-18T11:32:31.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331377+0000", 2024-09-18T11:32:31.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8769999999999993e-06 2024-09-18T11:32:31.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331381+0000", 2024-09-18T11:32:31.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.16e-06 2024-09-18T11:32:31.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338973+0000", 2024-09-18T11:32:31.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075918460000000002 2024-09-18T11:32:31.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339048+0000", 2024-09-18T11:32:31.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5813000000000001e-05 2024-09-18T11:32:31.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339322+0000", 2024-09-18T11:32:31.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027399899999999998 2024-09-18T11:32:31.568 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.569 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.569 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.569 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2650 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:31.569 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:31.569 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.156213046999994, 2024-09-18T11:32:31.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084392180000000001, 2024-09-18T11:32:31.569 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.569 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:31.570 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:31.570 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:31.570 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:31.570 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2650 2024-09-18T11:32:31.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.570 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:31.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:31.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:31.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:31.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331388+0000", 2024-09-18T11:32:31.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3429999999999998e-06 2024-09-18T11:32:31.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331394+0000", 2024-09-18T11:32:31.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.19e-06 2024-09-18T11:32:31.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331398+0000", 2024-09-18T11:32:31.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.495e-06 2024-09-18T11:32:31.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339362+0000", 2024-09-18T11:32:31.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079641799999999995 2024-09-18T11:32:31.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339460+0000", 2024-09-18T11:32:31.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8329e-05 2024-09-18T11:32:31.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339825+0000", 2024-09-18T11:32:31.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036510500000000001 2024-09-18T11:32:31.574 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.574 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.574 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.574 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:119 3.2 3:5595a68d:::07854d2fef297a06ba81685e660c332de36d5d18d546927d30daad6d7fda1541:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:31.574 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:32:31.574 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.444061289999993, 2024-09-18T11:32:31.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074145979999999997, 2024-09-18T11:32:31.574 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.575 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.575 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:32:31.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:32:31.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043535+0000", 2024-09-18T11:32:31.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:31.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043970+0000", 2024-09-18T11:32:31.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000435148 2024-09-18T11:32:31.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043993+0000", 2024-09-18T11:32:31.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3113999999999999e-05 2024-09-18T11:32:31.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044003+0000", 2024-09-18T11:32:31.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5310000000000007e-06 2024-09-18T11:32:31.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044062+0000", 2024-09-18T11:32:31.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9487000000000002e-05 2024-09-18T11:32:31.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:32:31.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044071+0000", 2024-09-18T11:32:31.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1610000000000003e-06 2024-09-18T11:32:31.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046423+0000", 2024-09-18T11:32:31.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023523680000000001 2024-09-18T11:32:31.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046502+0000", 2024-09-18T11:32:31.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9339999999999996e-05 2024-09-18T11:32:31.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:31.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046664+0000", 2024-09-18T11:32:31.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016185700000000001 2024-09-18T11:32:31.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:31.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050870+0000", 2024-09-18T11:32:31.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042060070000000003 2024-09-18T11:32:31.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:31.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050878+0000", 2024-09-18T11:32:31.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8960000000000003e-06 2024-09-18T11:32:31.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:31.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050918+0000", 2024-09-18T11:32:31.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0367000000000002e-05 2024-09-18T11:32:31.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050952+0000", 2024-09-18T11:32:31.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3989999999999998e-05 2024-09-18T11:32:31.582 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.582 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.582 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.582 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:127 3.2 3:4d3b277a:::3266c7aaa53a7d2c10c3c54d852d6cbbed49fb041bc0df41a3ac83b56e2b0615:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:31.582 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:31.582 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.443198108000004, 2024-09-18T11:32:31.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093148009999999993, 2024-09-18T11:32:31.582 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.582 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.583 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:31.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:31.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044400+0000", 2024-09-18T11:32:31.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:31.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044442+0000", 2024-09-18T11:32:31.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2247999999999999e-05 2024-09-18T11:32:31.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044453+0000", 2024-09-18T11:32:31.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0617e-05 2024-09-18T11:32:31.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044457+0000", 2024-09-18T11:32:31.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2520000000000001e-06 2024-09-18T11:32:31.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044495+0000", 2024-09-18T11:32:31.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7710000000000003e-05 2024-09-18T11:32:31.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:31.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044499+0000", 2024-09-18T11:32:31.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6269999999999999e-06 2024-09-18T11:32:31.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048239+0000", 2024-09-18T11:32:31.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037396719999999999 2024-09-18T11:32:31.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048351+0000", 2024-09-18T11:32:31.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112342 2024-09-18T11:32:31.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:31.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048545+0000", 2024-09-18T11:32:31.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000193881 2024-09-18T11:32:31.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:31.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.052689+0000", 2024-09-18T11:32:31.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041435780000000002 2024-09-18T11:32:31.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:31.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053654+0000", 2024-09-18T11:32:31.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096558199999999999 2024-09-18T11:32:31.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:31.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053683+0000", 2024-09-18T11:32:31.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8850999999999999e-05 2024-09-18T11:32:31.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053716+0000", 2024-09-18T11:32:31.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2468000000000002e-05 2024-09-18T11:32:31.589 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.589 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.590 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.590 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3209 4.0 4:019b6e09:::smithi11939506-40:head [write 638732~557432] snapc 42=[42,41,3f,3e,3d,3b,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-18T11:32:31.590 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:31.590 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.389516069000003, 2024-09-18T11:32:31.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091774999999999999, 2024-09-18T11:32:31.590 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.590 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.590 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:31.590 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:31.590 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:31.591 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3209 2024-09-18T11:32:31.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.591 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:31.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:31.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098080+0000", 2024-09-18T11:32:31.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:31.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098676+0000", 2024-09-18T11:32:31.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059598099999999998 2024-09-18T11:32:31.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098706+0000", 2024-09-18T11:32:31.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0020000000000001e-05 2024-09-18T11:32:31.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098716+0000", 2024-09-18T11:32:31.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0225e-05 2024-09-18T11:32:31.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098782+0000", 2024-09-18T11:32:31.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5530000000000004e-05 2024-09-18T11:32:31.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098883+0000", 2024-09-18T11:32:31.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101609 2024-09-18T11:32:31.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:31.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.099098+0000", 2024-09-18T11:32:31.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000215052 2024-09-18T11:32:31.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:31.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.102511+0000", 2024-09-18T11:32:31.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034129429999999999 2024-09-18T11:32:31.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:31.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107165+0000", 2024-09-18T11:32:31.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046531480000000002 2024-09-18T11:32:31.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:31.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107217+0000", 2024-09-18T11:32:31.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2058000000000002e-05 2024-09-18T11:32:31.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107260+0000", 2024-09-18T11:32:31.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3865000000000001e-05 2024-09-18T11:32:31.597 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.597 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.597 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.597 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:162 3.1 e138/67 3:8c826a1f:::e682cfc1e82c88472b9acf1cd97978bc105aa035f10b685bb4cace95809b52bf:head v 138'176, mlcod=136'174)", 2024-09-18T11:32:31.597 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:32:31.598 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.904042323999999, 2024-09-18T11:32:31.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010445024000000001, 2024-09-18T11:32:31.598 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.598 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.598 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:32:31.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:32:31.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583556+0000", 2024-09-18T11:32:31.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:31.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584899+0000", 2024-09-18T11:32:31.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001343056 2024-09-18T11:32:31.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584929+0000", 2024-09-18T11:32:31.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0133000000000001e-05 2024-09-18T11:32:31.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584937+0000", 2024-09-18T11:32:31.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6380000000000004e-06 2024-09-18T11:32:31.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584990+0000", 2024-09-18T11:32:31.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3186000000000002e-05 2024-09-18T11:32:31.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585012+0000", 2024-09-18T11:32:31.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1474000000000001e-05 2024-09-18T11:32:31.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:31.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593963+0000", 2024-09-18T11:32:31.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089512859999999993 2024-09-18T11:32:31.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594002+0000", 2024-09-18T11:32:31.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8859999999999997e-05 2024-09-18T11:32:31.602 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.603 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.603 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.603 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:166 3.1 e138/67 3:a73e635b:::b6ee478f0f661ec7bac3d17c2ca2fdaff71a47570f64844a7ba79bbb2a0e8d9d:head v 138'177, mlcod=136'174)", 2024-09-18T11:32:31.603 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:32:31.603 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.90247244, 2024-09-18T11:32:31.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089018680000000003, 2024-09-18T11:32:31.603 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.604 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.604 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:32:31.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:32:31.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585126+0000", 2024-09-18T11:32:31.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:31.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585211+0000", 2024-09-18T11:32:31.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5432999999999993e-05 2024-09-18T11:32:31.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585220+0000", 2024-09-18T11:32:31.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5879999999999998e-06 2024-09-18T11:32:31.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585225+0000", 2024-09-18T11:32:31.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8999999999999997e-06 2024-09-18T11:32:31.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585705+0000", 2024-09-18T11:32:31.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00048042799999999997 2024-09-18T11:32:31.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585724+0000", 2024-09-18T11:32:31.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8352e-05 2024-09-18T11:32:31.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:31.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594008+0000", 2024-09-18T11:32:31.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082840919999999998 2024-09-18T11:32:31.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594028+0000", 2024-09-18T11:32:31.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0869e-05 2024-09-18T11:32:31.608 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.608 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.609 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.609 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:168 3.1 e138/67 3:b64d7bad:::7feb22256bdcec217720fd46184ab43ced8923ed66ec6319c57f078d9f123262:head v 138'178, mlcod=136'174)", 2024-09-18T11:32:31.609 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:31.609 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.901480707000001, 2024-09-18T11:32:31.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079290650000000008, 2024-09-18T11:32:31.609 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.609 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.609 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:31.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:31.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:31.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:31.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586191+0000", 2024-09-18T11:32:31.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3627000000000005e-05 2024-09-18T11:32:31.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586202+0000", 2024-09-18T11:32:31.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0488e-05 2024-09-18T11:32:31.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586207+0000", 2024-09-18T11:32:31.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1719999999999998e-06 2024-09-18T11:32:31.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586239+0000", 2024-09-18T11:32:31.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2570000000000002e-05 2024-09-18T11:32:31.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586271+0000", 2024-09-18T11:32:31.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1341999999999999e-05 2024-09-18T11:32:31.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:31.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594031+0000", 2024-09-18T11:32:31.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077597459999999997 2024-09-18T11:32:31.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594047+0000", 2024-09-18T11:32:31.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6858e-05 2024-09-18T11:32:31.614 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.614 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.614 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.614 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4267 3.1 3:8d8d13e9:::92886a4340d03039afb2c877fad76e3afe7a80ee04388d55256c386022cd5d63:head [write 0~2203648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:31.615 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:32:31.615 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.4304487269999999, 2024-09-18T11:32:31.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0092856740000000007, 2024-09-18T11:32:31.615 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:31.615 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.615 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:31.615 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:31.615 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:31.615 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4267 2024-09-18T11:32:31.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.615 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:31.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:31.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:32:31.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:31.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:32:31.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:31.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:31.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057149+0000", 2024-09-18T11:32:31.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:31.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:31.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058175+0000", 2024-09-18T11:32:31.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001026765 2024-09-18T11:32:31.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:31.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058190+0000", 2024-09-18T11:32:31.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4167e-05 2024-09-18T11:32:31.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058195+0000", 2024-09-18T11:32:31.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.823e-06 2024-09-18T11:32:31.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:31.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058247+0000", 2024-09-18T11:32:31.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1304999999999998e-05 2024-09-18T11:32:31.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:31.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058422+0000", 2024-09-18T11:32:31.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017527100000000001 2024-09-18T11:32:31.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:31.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058609+0000", 2024-09-18T11:32:31.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018714299999999999 2024-09-18T11:32:31.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:31.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.062721+0000", 2024-09-18T11:32:31.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041118869999999998 2024-09-18T11:32:31.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:31.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066291+0000", 2024-09-18T11:32:31.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035694860000000002 2024-09-18T11:32:31.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:31.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066328+0000", 2024-09-18T11:32:31.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7864000000000002e-05 2024-09-18T11:32:31.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:31.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:31.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:31.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066436+0000", 2024-09-18T11:32:31.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010762 2024-09-18T11:32:31.621 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.621 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.621 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.621 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:31.621 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:31.621 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:31.623 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:32:31.626 INFO:tasks.rados.rados.0.smithi119.stdout:1877: got (22) Invalid argument 2024-09-18T11:32:31.626 INFO:tasks.rados.rados.0.smithi119.stdout:1876: got (22) Invalid argument 2024-09-18T11:32:31.626 INFO:tasks.rados.rados.0.smithi119.stdout:1874: finishing write tid 1 to smithi11939506-15 2024-09-18T11:32:31.626 INFO:tasks.rados.rados.0.smithi119.stdout:1871: done (5 left) 2024-09-18T11:32:31.626 INFO:tasks.rados.rados.0.smithi119.stdout:1873: done (4 left) 2024-09-18T11:32:31.626 INFO:tasks.rados.rados.0.smithi119.stdout:1876: done (3 left) 2024-09-18T11:32:31.627 INFO:tasks.rados.rados.0.smithi119.stdout:1877: done (2 left) 2024-09-18T11:32:31.627 INFO:tasks.rados.rados.0.smithi119.stdout:1878: set_chunk oid 43 target oid 2024-09-18T11:32:31.627 INFO:tasks.rados.rados.0.smithi119.stdout:1878: set_chunk oid 43 offset: 439808 length: 228352 target oid 37db6630c16fab6acc12c885828cbb02d9681075d7d4a25a63cfbad18c5beaf2 offset: 0 snap -1 2024-09-18T11:32:31.627 INFO:tasks.rados.rados.0.smithi119.stdout:1879: delete oid 39 current snap is 60 2024-09-18T11:32:31.627 INFO:tasks.rados.rados.0.smithi119.stdout:1875: finishing rollback tid 0 to smithi11939506-25 2024-09-18T11:32:31.627 INFO:tasks.rados.rados.0.smithi119.stdout:1875: finishing rollback tid 1 to smithi11939506-25 2024-09-18T11:32:31.627 INFO:tasks.rados.rados.0.smithi119.stdout:1874: finishing write tid 2 to smithi11939506-15 2024-09-18T11:32:31.627 INFO:tasks.rados.rados.0.smithi119.stdout:1874: finishing write tid 3 to smithi11939506-15 2024-09-18T11:32:31.627 INFO:tasks.rados.rados.0.smithi119.stdout:1875: finishing rollback tid 2 to smithi11939506-25 2024-09-18T11:32:31.627 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 83 (ObjNum 368 snap 57 seq_num 368) dirty exists 2024-09-18T11:32:31.627 INFO:tasks.rados.rados.0.smithi119.stdout:1874: finishing write tid 4 to smithi11939506-15 2024-09-18T11:32:31.628 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 15 v 178 (ObjNum 381 snap 60 seq_num 381) dirty exists 2024-09-18T11:32:31.628 INFO:tasks.rados.rados.0.smithi119.stdout:1874: left oid 15 (ObjNum 381 snap 60 seq_num 381) 2024-09-18T11:32:31.628 INFO:tasks.rados.rados.0.smithi119.stdout:1878: finishing set_chunk to oid 43 2024-09-18T11:32:31.628 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 43 v 254 (ObjNum 346 snap 51 seq_num 346) dirty exists 2024-09-18T11:32:31.628 INFO:tasks.rados.rados.0.smithi119.stdout:1874: done (3 left) 2024-09-18T11:32:31.628 INFO:tasks.rados.rados.0.smithi119.stdout:1875: done (2 left) 2024-09-18T11:32:31.628 INFO:tasks.rados.rados.0.smithi119.stdout:1878: done (1 left) 2024-09-18T11:32:31.628 INFO:tasks.rados.rados.0.smithi119.stdout:1879: done (0 left) 2024-09-18T11:32:31.628 INFO:tasks.rados.rados.0.smithi119.stdout:1880: read oid 31 snap 57 2024-09-18T11:32:31.628 INFO:tasks.rados.rados.0.smithi119.stdout:1880: expect (ObjNum 345 snap 51 seq_num 345) 2024-09-18T11:32:31.628 INFO:tasks.rados.rados.0.smithi119.stdout:1881: rollback oid 27 current snap is 60 2024-09-18T11:32:31.628 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 27 to 43 2024-09-18T11:32:31.629 INFO:tasks.rados.rados.0.smithi119.stdout:1882: read oid 30 snap -1 2024-09-18T11:32:31.629 INFO:tasks.rados.rados.0.smithi119.stdout:1882: expect deleted 2024-09-18T11:32:31.629 INFO:tasks.rados.rados.0.smithi119.stdout:1883: read oid 29 snap 32 2024-09-18T11:32:31.629 INFO:tasks.rados.rados.0.smithi119.stdout:1883: expect (ObjNum 210 snap 23 seq_num 210) 2024-09-18T11:32:31.629 INFO:tasks.rados.rados.0.smithi119.stdout:1884: write (excl) oid 17 current snap is 60 2024-09-18T11:32:31.629 INFO:tasks.rados.rados.0.smithi119.stdout:1884: seq_num 383 ranges {437081=638017,1679074=582789} 2024-09-18T11:32:31.629 INFO:tasks.rados.rados.0.smithi119.stdout:1884: writing smithi11939506-17 from 437081 to 1075098 tid 1 2024-09-18T11:32:31.629 INFO:tasks.rados.rados.0.smithi119.stdout:1884: writing smithi11939506-17 from 1679074 to 2261863 tid 2 2024-09-18T11:32:31.629 INFO:tasks.rados.rados.0.smithi119.stdout:1885: write (excl) oid 7 current snap is 60 2024-09-18T11:32:31.629 INFO:tasks.rados.rados.0.smithi119.stdout:1885: seq_num 384 ranges {453888=713266,1737556=442800} 2024-09-18T11:32:31.630 INFO:tasks.rados.rados.0.smithi119.stdout:1885: writing smithi11939506-7 from 453888 to 1167154 tid 1 2024-09-18T11:32:31.630 INFO:tasks.rados.rados.0.smithi119.stdout:1885: writing smithi11939506-7 from 1737556 to 2180356 tid 2 2024-09-18T11:32:31.630 INFO:tasks.rados.rados.0.smithi119.stdout:1886: read oid 13 snap 55 2024-09-18T11:32:31.630 INFO:tasks.rados.rados.0.smithi119.stdout:1886: expect (ObjNum 309 snap 43 seq_num 309) 2024-09-18T11:32:31.630 INFO:tasks.rados.rados.0.smithi119.stdout:1887: snap_create 2024-09-18T11:32:31.630 INFO:tasks.rados.rados.0.smithi119.stdout:1880: expect (ObjNum 345 snap 51 seq_num 345) 2024-09-18T11:32:31.630 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:31.622+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:31.724 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:31.723+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:31.732 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:31.733 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:31.733 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:31.733 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:31.741 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:32:31.771 INFO:tasks.rados.rados.0.smithi119.stdout:1881: finishing rollback tid 0 to smithi11939506-27 2024-09-18T11:32:31.771 INFO:tasks.rados.rados.0.smithi119.stdout:1881: finishing rollback tid 1 to smithi11939506-27 2024-09-18T11:32:31.771 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 358 (ObjNum 97 snap 0 seq_num 384493840) dirty dne 2024-09-18T11:32:31.771 INFO:tasks.rados.rados.0.smithi119.stdout:1883: expect (ObjNum 210 snap 23 seq_num 210) 2024-09-18T11:32:31.825 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:31.824+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:31.838 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:31.838 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:31.838 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:31.839 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:31.839 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:31.847 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:32:31.926 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:31.925+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:31.946 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:32:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 117.52721882199999, 2024-09-18T11:32:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:32:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:31.946 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:32:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:31.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:32:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:32:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:32:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:31.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:32:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:32:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:32:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:32:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:32:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:32:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:32:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:32:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:32:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:32:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:32:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:32:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:32:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:32:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:32:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:32:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:32:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:32:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:31.951 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:172 4.5 e78/22 4:a087a566:::smithi11939506-46:19 v 78'135, mlcod=77'132)", 2024-09-18T11:32:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:32:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 90.557482168000007, 2024-09-18T11:32:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010938658, 2024-09-18T11:32:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:31.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:32:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:32:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385823+0000", 2024-09-18T11:32:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385830+0000", 2024-09-18T11:32:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2790000000000001e-06 2024-09-18T11:32:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385843+0000", 2024-09-18T11:32:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2785e-05 2024-09-18T11:32:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385848+0000", 2024-09-18T11:32:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5570000000000003e-06 2024-09-18T11:32:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385914+0000", 2024-09-18T11:32:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5168999999999999e-05 2024-09-18T11:32:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385930+0000", 2024-09-18T11:32:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6884999999999998e-05 2024-09-18T11:32:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396721+0000", 2024-09-18T11:32:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010790137 2024-09-18T11:32:31.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396762+0000", 2024-09-18T11:32:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1635999999999999e-05 2024-09-18T11:32:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:32:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 90.556989415999993, 2024-09-18T11:32:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:32:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:31.956 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:32:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:32:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:32:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:32:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:32:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:32:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:32:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:32:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:32:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:32:31.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:32:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:32:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:32:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:32:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:31.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 82.970876809999993, 2024-09-18T11:32:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:32:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:32:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:32:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:31.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:32:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:32:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:32:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:32:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:32:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:32:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:31.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:32:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:32:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:32:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:32:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:32:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:32:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:32:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:32:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:32:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:32:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:32:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:32:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:32:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:32:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.966 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2956 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~2100908,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 67.916080559999997, 2024-09-18T11:32:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011863831, 2024-09-18T11:32:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2956 2024-09-18T11:32:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:31.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027228+0000", 2024-09-18T11:32:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:32:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027235+0000", 2024-09-18T11:32:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:32:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027239+0000", 2024-09-18T11:32:31.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.535e-06 2024-09-18T11:32:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027831+0000", 2024-09-18T11:32:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059244699999999996 2024-09-18T11:32:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027879+0000", 2024-09-18T11:32:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:32:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038098+0000", 2024-09-18T11:32:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010219137 2024-09-18T11:32:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038207+0000", 2024-09-18T11:32:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010951999999999999 2024-09-18T11:32:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039089+0000", 2024-09-18T11:32:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00088151699999999996 2024-09-18T11:32:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:31.971 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2957 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 67.916062776000004, 2024-09-18T11:32:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012041979, 2024-09-18T11:32:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:31.972 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2957 2024-09-18T11:32:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:31.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027245+0000", 2024-09-18T11:32:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:32:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027251+0000", 2024-09-18T11:32:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0090000000000004e-06 2024-09-18T11:32:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027255+0000", 2024-09-18T11:32:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:32:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027895+0000", 2024-09-18T11:32:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064076400000000005 2024-09-18T11:32:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027930+0000", 2024-09-18T11:32:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4334e-05 2024-09-18T11:32:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039111+0000", 2024-09-18T11:32:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181455999999999 2024-09-18T11:32:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039156+0000", 2024-09-18T11:32:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5256000000000002e-05 2024-09-18T11:32:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039285+0000", 2024-09-18T11:32:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012839200000000001 2024-09-18T11:32:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:31.976 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:31.977 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:31.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.977 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:978 4.b e132/67 4:d8d14c58:::smithi11939506-49:30 v 132'417, mlcod=128'414)", 2024-09-18T11:32:31.977 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:31.977 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 28.918031494000001, 2024-09-18T11:32:31.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011942183, 2024-09-18T11:32:31.977 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:31.977 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.977 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:31.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:31.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:31.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025278+0000", 2024-09-18T11:32:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9060000000000004e-06 2024-09-18T11:32:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025288+0000", 2024-09-18T11:32:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.028e-05 2024-09-18T11:32:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025293+0000", 2024-09-18T11:32:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2700000000000004e-06 2024-09-18T11:32:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025346+0000", 2024-09-18T11:32:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2862999999999999e-05 2024-09-18T11:32:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025370+0000", 2024-09-18T11:32:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4005000000000001e-05 2024-09-18T11:32:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037159+0000", 2024-09-18T11:32:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011788343 2024-09-18T11:32:31.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037216+0000", 2024-09-18T11:32:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7695000000000002e-05 2024-09-18T11:32:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3930 4.b e132/67 4:d3ae8215:::smithi11939506-41:head v 132'421, mlcod=128'414)", 2024-09-18T11:32:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 28.914483246, 2024-09-18T11:32:31.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015093554, 2024-09-18T11:32:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028821+0000", 2024-09-18T11:32:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029805+0000", 2024-09-18T11:32:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00098410899999999994 2024-09-18T11:32:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029817+0000", 2024-09-18T11:32:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2058e-05 2024-09-18T11:32:31.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029823+0000", 2024-09-18T11:32:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6450000000000002e-06 2024-09-18T11:32:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032330+0000", 2024-09-18T11:32:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025067710000000001 2024-09-18T11:32:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032349+0000", 2024-09-18T11:32:31.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9729e-05 2024-09-18T11:32:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043878+0000", 2024-09-18T11:32:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011528514 2024-09-18T11:32:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043916+0000", 2024-09-18T11:32:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8049000000000001e-05 2024-09-18T11:32:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:31.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:162 3.1 3:8c826a1f:::e682cfc1e82c88472b9acf1cd97978bc105aa035f10b685bb4cace95809b52bf:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:32:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.362361088, 2024-09-18T11:32:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013234571000000001, 2024-09-18T11:32:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:32:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:32:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580943+0000", 2024-09-18T11:32:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582556+0000", 2024-09-18T11:32:31.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016132970000000001 2024-09-18T11:32:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582572+0000", 2024-09-18T11:32:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5846999999999999e-05 2024-09-18T11:32:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582578+0000", 2024-09-18T11:32:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4810000000000003e-06 2024-09-18T11:32:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582632+0000", 2024-09-18T11:32:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4156000000000001e-05 2024-09-18T11:32:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582819+0000", 2024-09-18T11:32:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000186431 2024-09-18T11:32:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583118+0000", 2024-09-18T11:32:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029880100000000001 2024-09-18T11:32:31.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590676+0000", 2024-09-18T11:32:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075588670000000004 2024-09-18T11:32:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594111+0000", 2024-09-18T11:32:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034348550000000001 2024-09-18T11:32:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594142+0000", 2024-09-18T11:32:31.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.099e-05 2024-09-18T11:32:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594179+0000", 2024-09-18T11:32:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6831e-05 2024-09-18T11:32:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3974 4.d 4:b5a463bf:::smithi11939506-20:head [write 665643~697786,stat] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:32:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.361274405, 2024-09-18T11:32:31.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016621174999999998, 2024-09-18T11:32:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3974 2024-09-18T11:32:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:32:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:31.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:32:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582030+0000", 2024-09-18T11:32:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:31.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585185+0000", 2024-09-18T11:32:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003155644 2024-09-18T11:32:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585208+0000", 2024-09-18T11:32:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3013000000000001e-05 2024-09-18T11:32:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585215+0000", 2024-09-18T11:32:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7569999999999998e-06 2024-09-18T11:32:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585278+0000", 2024-09-18T11:32:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2699000000000004e-05 2024-09-18T11:32:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585374+0000", 2024-09-18T11:32:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.6062000000000003e-05 2024-09-18T11:32:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:32:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585577+0000", 2024-09-18T11:32:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000202926 2024-09-18T11:32:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596544+0000", 2024-09-18T11:32:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010966848 2024-09-18T11:32:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598594+0000", 2024-09-18T11:32:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020502049999999998 2024-09-18T11:32:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598625+0000", 2024-09-18T11:32:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1412999999999999e-05 2024-09-18T11:32:31.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598652+0000", 2024-09-18T11:32:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7166999999999999e-05 2024-09-18T11:32:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:166 3.1 3:a73e635b:::b6ee478f0f661ec7bac3d17c2ca2fdaff71a47570f64844a7ba79bbb2a0e8d9d:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:32:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.360715038999999, 2024-09-18T11:32:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011711037000000001, 2024-09-18T11:32:31.997 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:32:31.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:31.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:32:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582590+0000", 2024-09-18T11:32:32.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582673+0000", 2024-09-18T11:32:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2861000000000005e-05 2024-09-18T11:32:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582683+0000", 2024-09-18T11:32:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0081999999999999e-05 2024-09-18T11:32:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582688+0000", 2024-09-18T11:32:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6870000000000002e-06 2024-09-18T11:32:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584266+0000", 2024-09-18T11:32:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001578423 2024-09-18T11:32:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584407+0000", 2024-09-18T11:32:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014144599999999999 2024-09-18T11:32:32.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584599+0000", 2024-09-18T11:32:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000191881 2024-09-18T11:32:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590680+0000", 2024-09-18T11:32:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060801479999999996 2024-09-18T11:32:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:32.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594224+0000", 2024-09-18T11:32:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035441700000000001 2024-09-18T11:32:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594269+0000", 2024-09-18T11:32:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5297999999999999e-05 2024-09-18T11:32:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594302+0000", 2024-09-18T11:32:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2703000000000002e-05 2024-09-18T11:32:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:32.004 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:168 3.1 3:b64d7bad:::7feb22256bdcec217720fd46184ab43ced8923ed66ec6319c57f078d9f123262:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:32:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.360487587000001, 2024-09-18T11:32:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011589809, 2024-09-18T11:32:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:32:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:32:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582817+0000", 2024-09-18T11:32:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582903+0000", 2024-09-18T11:32:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5915999999999995e-05 2024-09-18T11:32:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582913+0000", 2024-09-18T11:32:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1409999999999994e-06 2024-09-18T11:32:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582917+0000", 2024-09-18T11:32:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6419999999999998e-06 2024-09-18T11:32:32.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585540+0000", 2024-09-18T11:32:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026228940000000002 2024-09-18T11:32:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585721+0000", 2024-09-18T11:32:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018086299999999999 2024-09-18T11:32:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:32.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585918+0000", 2024-09-18T11:32:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000196758 2024-09-18T11:32:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590683+0000", 2024-09-18T11:32:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047657020000000001 2024-09-18T11:32:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594354+0000", 2024-09-18T11:32:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003671068 2024-09-18T11:32:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594386+0000", 2024-09-18T11:32:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1426e-05 2024-09-18T11:32:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594408+0000", 2024-09-18T11:32:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1987000000000001e-05 2024-09-18T11:32:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.010 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:172 3.1 3:8c3b343b:::d73444b0abf5ea0abb7f4bfb783ec9a4695bac9fb180180b88c2eef6a2e03b80:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.583414+0000", 2024-09-18T11:32:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.359891545, 2024-09-18T11:32:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011061609999999999, 2024-09-18T11:32:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583414+0000", 2024-09-18T11:32:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:32.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583414+0000", 2024-09-18T11:32:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583413+0000", 2024-09-18T11:32:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583525+0000", 2024-09-18T11:32:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000111289 2024-09-18T11:32:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583537+0000", 2024-09-18T11:32:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1755e-05 2024-09-18T11:32:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583542+0000", 2024-09-18T11:32:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4399999999999996e-06 2024-09-18T11:32:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.588499+0000", 2024-09-18T11:32:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004957347 2024-09-18T11:32:32.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.588684+0000", 2024-09-18T11:32:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000185002 2024-09-18T11:32:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.588889+0000", 2024-09-18T11:32:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020478200000000001 2024-09-18T11:32:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.591496+0000", 2024-09-18T11:32:32.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026070619999999998 2024-09-18T11:32:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594430+0000", 2024-09-18T11:32:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029342219999999998 2024-09-18T11:32:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594458+0000", 2024-09-18T11:32:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7243e-05 2024-09-18T11:32:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594476+0000", 2024-09-18T11:32:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8139e-05 2024-09-18T11:32:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:175 3.1 3:b5e86b88:::e55cf3d5cf782ed327a67e746b287b76d05f74f58a4f3726e399a564440ceaf6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:32:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.359638646000001, 2024-09-18T11:32:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013709627, 2024-09-18T11:32:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:32.016 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:32:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:32:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:32.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583666+0000", 2024-09-18T11:32:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584041+0000", 2024-09-18T11:32:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037511499999999998 2024-09-18T11:32:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584055+0000", 2024-09-18T11:32:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3295999999999999e-05 2024-09-18T11:32:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584061+0000", 2024-09-18T11:32:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1630000000000001e-06 2024-09-18T11:32:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.589980+0000", 2024-09-18T11:32:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005918998 2024-09-18T11:32:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590117+0000", 2024-09-18T11:32:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013715400000000001 2024-09-18T11:32:32.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590270+0000", 2024-09-18T11:32:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015245600000000001 2024-09-18T11:32:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596513+0000", 2024-09-18T11:32:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062438729999999996 2024-09-18T11:32:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:32.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597328+0000", 2024-09-18T11:32:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081489599999999996 2024-09-18T11:32:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597358+0000", 2024-09-18T11:32:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9291000000000001e-05 2024-09-18T11:32:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597377+0000", 2024-09-18T11:32:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:32:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:32.021 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4061 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:32:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.646459957000001, 2024-09-18T11:32:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013340967, 2024-09-18T11:32:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4061 2024-09-18T11:32:32.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:32:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:32:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296844+0000", 2024-09-18T11:32:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296854+0000", 2024-09-18T11:32:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0037000000000001e-05 2024-09-18T11:32:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296873+0000", 2024-09-18T11:32:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:32:32.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296881+0000", 2024-09-18T11:32:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3790000000000006e-06 2024-09-18T11:32:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296975+0000", 2024-09-18T11:32:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3536999999999996e-05 2024-09-18T11:32:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:32.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296982+0000", 2024-09-18T11:32:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6399999999999997e-06 2024-09-18T11:32:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301933+0000", 2024-09-18T11:32:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049507190000000001 2024-09-18T11:32:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301984+0000", 2024-09-18T11:32:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1439e-05 2024-09-18T11:32:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:32:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082022050000000006 2024-09-18T11:32:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4062 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.646411401999998, 2024-09-18T11:32:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016297147000000001, 2024-09-18T11:32:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:32.027 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4062 2024-09-18T11:32:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:32.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:32.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:32.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:32.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:32.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:32.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:32.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296898+0000", 2024-09-18T11:32:32.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9419999999999997e-06 2024-09-18T11:32:32.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:32.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296906+0000", 2024-09-18T11:32:32.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3520000000000007e-06 2024-09-18T11:32:32.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:32.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296910+0000", 2024-09-18T11:32:32.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4469999999999999e-06 2024-09-18T11:32:32.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:32.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297032+0000", 2024-09-18T11:32:32.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012168900000000001 2024-09-18T11:32:32.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:32.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297038+0000", 2024-09-18T11:32:32.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9320000000000001e-06 2024-09-18T11:32:32.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:32.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310221+0000", 2024-09-18T11:32:32.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013182887000000001 2024-09-18T11:32:32.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:32.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310323+0000", 2024-09-18T11:32:32.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010170000000000001 2024-09-18T11:32:32.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.032 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:32.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313191+0000", 2024-09-18T11:32:32.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002868808 2024-09-18T11:32:32.032 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:32.032 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:32.032 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:32.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.032 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4063 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:32.032 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:32.033 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.64639051, 2024-09-18T11:32:32.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017648712, 2024-09-18T11:32:32.033 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:32.033 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:32.033 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:32.033 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:32.033 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:32.033 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4063 2024-09-18T11:32:32.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.033 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:32.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:32.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:32.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:32.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:32.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:32.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:32.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:32.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:32.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:32.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:32.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296918+0000", 2024-09-18T11:32:32.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8100000000000002e-06 2024-09-18T11:32:32.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:32.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296924+0000", 2024-09-18T11:32:32.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2519999999999996e-06 2024-09-18T11:32:32.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:32.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296928+0000", 2024-09-18T11:32:32.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0500000000000002e-06 2024-09-18T11:32:32.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:32.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297062+0000", 2024-09-18T11:32:32.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013425500000000001 2024-09-18T11:32:32.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.036 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:32.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297069+0000", 2024-09-18T11:32:32.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5239999999999997e-06 2024-09-18T11:32:32.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.036 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:32.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313424+0000", 2024-09-18T11:32:32.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016355277000000001 2024-09-18T11:32:32.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:32.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313514+0000", 2024-09-18T11:32:32.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0581999999999994e-05 2024-09-18T11:32:32.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:32.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.314564+0000", 2024-09-18T11:32:32.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001049384 2024-09-18T11:32:32.037 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:32.037 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:32.037 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:32.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.038 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4065 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:32.038 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:32:32.038 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.646345068999999, 2024-09-18T11:32:32.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013225997, 2024-09-18T11:32:32.038 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:32.038 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:32.038 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:32.038 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:32.038 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:32.038 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4065 2024-09-18T11:32:32.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.039 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:32.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:32.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:32:32.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:32.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:32.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:32:32.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:32.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:32.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296960+0000", 2024-09-18T11:32:32.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:32.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:32.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296963+0000", 2024-09-18T11:32:32.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2270000000000002e-06 2024-09-18T11:32:32.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:32.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296970+0000", 2024-09-18T11:32:32.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6220000000000003e-06 2024-09-18T11:32:32.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:32.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296973+0000", 2024-09-18T11:32:32.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3730000000000001e-06 2024-09-18T11:32:32.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:32.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297097+0000", 2024-09-18T11:32:32.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012352399999999999 2024-09-18T11:32:32.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:32.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297103+0000", 2024-09-18T11:32:32.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7119999999999997e-06 2024-09-18T11:32:32.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:32.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307831+0000", 2024-09-18T11:32:32.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010728794 2024-09-18T11:32:32.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:32.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307924+0000", 2024-09-18T11:32:32.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2099e-05 2024-09-18T11:32:32.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:32.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:32:32.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022630469999999998 2024-09-18T11:32:32.043 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:32.043 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:32.043 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:32.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.043 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4066 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:32.043 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:32:32.043 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.646327994, 2024-09-18T11:32:32.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015656389999999999, 2024-09-18T11:32:32.043 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:32.043 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:32.043 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:32.044 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:32.044 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:32.044 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4066 2024-09-18T11:32:32.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.044 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:32.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.044 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:32.044 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:32:32.044 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:32.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.044 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:32.044 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:32:32.044 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:32.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.045 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:32.045 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296977+0000", 2024-09-18T11:32:32.045 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:32.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.045 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:32.045 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296980+0000", 2024-09-18T11:32:32.045 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8830000000000002e-06 2024-09-18T11:32:32.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.045 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:32.046 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296985+0000", 2024-09-18T11:32:32.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:32:32.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.046 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:32.046 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296988+0000", 2024-09-18T11:32:32.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0419999999999998e-06 2024-09-18T11:32:32.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.046 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:32.046 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297120+0000", 2024-09-18T11:32:32.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131693 2024-09-18T11:32:32.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.047 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:32.047 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297125+0000", 2024-09-18T11:32:32.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1100000000000002e-06 2024-09-18T11:32:32.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.047 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:32.047 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310212+0000", 2024-09-18T11:32:32.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013087395999999999 2024-09-18T11:32:32.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.047 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:32.047 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310292+0000", 2024-09-18T11:32:32.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9980000000000003e-05 2024-09-18T11:32:32.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.048 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:32.048 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312634+0000", 2024-09-18T11:32:32.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002341756 2024-09-18T11:32:32.048 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:32.048 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:32.048 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:32.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.048 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4068 3.1 3:87e79ed0:::3526641c07d3efa989be9684355570f39c331008b8a02c9fa22b38a94b6c6b9f:head [write 0~1015808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:32.048 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:32:32.049 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 19.642551777000001, 2024-09-18T11:32:32.049 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016392648999999999, 2024-09-18T11:32:32.049 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:32.049 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:32.049 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:32.049 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:32.049 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:32.049 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4068 2024-09-18T11:32:32.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.049 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:32.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.049 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:32.049 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:32:32.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:32.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.050 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:32.050 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:32:32.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:32.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.050 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:32.050 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300753+0000", 2024-09-18T11:32:32.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:32.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.050 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:32.051 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301922+0000", 2024-09-18T11:32:32.051 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001169432 2024-09-18T11:32:32.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.051 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:32.051 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301936+0000", 2024-09-18T11:32:32.051 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3574000000000001e-05 2024-09-18T11:32:32.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.051 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:32.051 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301941+0000", 2024-09-18T11:32:32.051 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6069999999999998e-06 2024-09-18T11:32:32.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.052 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:32.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310234+0000", 2024-09-18T11:32:32.052 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082924820000000003 2024-09-18T11:32:32.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.052 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:32.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310337+0000", 2024-09-18T11:32:32.052 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102673 2024-09-18T11:32:32.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.052 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:32.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310469+0000", 2024-09-18T11:32:32.052 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131982 2024-09-18T11:32:32.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.053 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:32.053 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312844+0000", 2024-09-18T11:32:32.053 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023758680000000002 2024-09-18T11:32:32.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.053 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:32.053 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317092+0000", 2024-09-18T11:32:32.053 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042477549999999998 2024-09-18T11:32:32.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.053 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:32.053 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317124+0000", 2024-09-18T11:32:32.054 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1968999999999998e-05 2024-09-18T11:32:32.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:32.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:32.054 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:32.054 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317147+0000", 2024-09-18T11:32:32.054 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2339000000000001e-05 2024-09-18T11:32:32.054 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:32.054 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:32.054 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:32.054 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:32.054 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:32.054 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:32.056 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:32:32.056 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:32.054+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:32.100 INFO:tasks.rados.rados.0.smithi119.stdout:1884: finishing write tid 1 to smithi11939506-17 2024-09-18T11:32:32.100 INFO:tasks.rados.rados.0.smithi119.stdout:1884: finishing write tid 2 to smithi11939506-17 2024-09-18T11:32:32.100 INFO:tasks.rados.rados.0.smithi119.stdout:1884: finishing write tid 3 to smithi11939506-17 2024-09-18T11:32:32.101 INFO:tasks.rados.rados.0.smithi119.stdout:1884: finishing write tid 4 to smithi11939506-17 2024-09-18T11:32:32.101 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 17 v 561 (ObjNum 383 snap 60 seq_num 383) dirty exists 2024-09-18T11:32:32.101 INFO:tasks.rados.rados.0.smithi119.stdout:1884: left oid 17 (ObjNum 383 snap 60 seq_num 383) 2024-09-18T11:32:32.101 INFO:tasks.rados.rados.0.smithi119.stdout:1885: finishing write tid 1 to smithi11939506-7 2024-09-18T11:32:32.101 INFO:tasks.rados.rados.0.smithi119.stdout:1885: finishing write tid 2 to smithi11939506-7 2024-09-18T11:32:32.101 INFO:tasks.rados.rados.0.smithi119.stdout:1885: finishing write tid 3 to smithi11939506-7 2024-09-18T11:32:32.101 INFO:tasks.rados.rados.0.smithi119.stdout:1885: finishing write tid 4 to smithi11939506-7 2024-09-18T11:32:32.101 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 7 v 184 (ObjNum 384 snap 60 seq_num 384) dirty exists 2024-09-18T11:32:32.101 INFO:tasks.rados.rados.0.smithi119.stdout:1885: left oid 7 (ObjNum 384 snap 60 seq_num 384) 2024-09-18T11:32:32.101 INFO:tasks.rados.rados.0.smithi119.stdout:1886: expect (ObjNum 309 snap 43 seq_num 309) 2024-09-18T11:32:32.154 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:32.154 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:32.154 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:32.154 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:32.156 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:32.155+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:32.163 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:32:32.207 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:32:32.258 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:32.256+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:32.301 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:32.301 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:32.301 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:32.301 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:32.301 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:32.309 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:32:32.322 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:32.322 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:32.337 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:32.338 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:32.352 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:32.352 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:32.359 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:32.357+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:32.367 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:32.367 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:32.384 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:32.384 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:32.386 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:32.385+0000 7f3157e90640 -1 osd.4 160 handle_osd_map injecting map crc failure 2024-09-18T11:32:32.391 INFO:tasks.rados.rados.0.smithi119.stdout:1880: done (7 left) 2024-09-18T11:32:32.391 INFO:tasks.rados.rados.0.smithi119.stdout:1881: done (6 left) 2024-09-18T11:32:32.391 INFO:tasks.rados.rados.0.smithi119.stdout:1882: done (5 left) 2024-09-18T11:32:32.391 INFO:tasks.rados.rados.0.smithi119.stdout:1883: done (4 left) 2024-09-18T11:32:32.391 INFO:tasks.rados.rados.0.smithi119.stdout:1884: done (3 left) 2024-09-18T11:32:32.391 INFO:tasks.rados.rados.0.smithi119.stdout:1885: done (2 left) 2024-09-18T11:32:32.391 INFO:tasks.rados.rados.0.smithi119.stdout:1886: done (1 left) 2024-09-18T11:32:32.391 INFO:tasks.rados.rados.0.smithi119.stdout:1887: done (0 left) 2024-09-18T11:32:32.392 INFO:tasks.rados.rados.0.smithi119.stdout:1888: set_chunk oid 33 target oid 2024-09-18T11:32:32.392 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T11:32:32.401 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:32.401 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:32.402 INFO:tasks.rados.rados.0.smithi119.stdout:1888: set_chunk oid 33 offset: 2624512 length: 609280 target oid 217ddd4350e1ff1733b5c902777b4bd4cfbe0d4744762d40c9a9a464b182c2e6 offset: 0 snap -1 2024-09-18T11:32:32.402 INFO:tasks.rados.rados.0.smithi119.stdout:1889: read oid 37 snap -1 2024-09-18T11:32:32.403 INFO:tasks.rados.rados.0.smithi119.stdout:1889: expect deleted 2024-09-18T11:32:32.403 INFO:tasks.rados.rados.0.smithi119.stdout:1890: rollback oid 12 current snap is 61 2024-09-18T11:32:32.403 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 12 to 56 2024-09-18T11:32:32.403 INFO:tasks.rados.rados.0.smithi119.stdout:1891: read oid 14 snap -1 2024-09-18T11:32:32.403 INFO:tasks.rados.rados.0.smithi119.stdout:1891: expect (ObjNum 352 snap 54 seq_num 352) 2024-09-18T11:32:32.403 INFO:tasks.rados.rados.0.smithi119.stdout:1892: read oid 42 snap -1 2024-09-18T11:32:32.403 INFO:tasks.rados.rados.0.smithi119.stdout:1892: expect (ObjNum 338 snap 48 seq_num 338) 2024-09-18T11:32:32.403 INFO:tasks.rados.rados.0.smithi119.stdout:1893: read oid 24 snap -1 2024-09-18T11:32:32.403 INFO:tasks.rados.rados.0.smithi119.stdout:1893: expect (ObjNum 377 snap 58 seq_num 377) 2024-09-18T11:32:32.403 INFO:tasks.rados.rados.0.smithi119.stdout:1894: set_chunk oid 10 target oid 2024-09-18T11:32:32.404 INFO:tasks.rados.rados.0.smithi119.stdout:1894: set_chunk oid 10 offset: 1362944 length: 45056 target oid 1049f390637b120e3fb820f509a33a1fea2018acdde9a991923ed8d4d2266930 offset: 0 snap -1 2024-09-18T11:32:32.404 INFO:tasks.rados.rados.0.smithi119.stdout:1895: set_chunk oid 4 target oid 2024-09-18T11:32:32.416 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:32.416 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:32.418 INFO:tasks.rados.rados.0.smithi119.stdout:1895: set_chunk oid 4 offset: 1710592 length: 580096 target oid c964785f41d8a4d869f473780e3ee8aa3ba4fdb60f670f8e8de624b65e59c485 offset: 0 snap -1 2024-09-18T11:32:32.418 INFO:tasks.rados.rados.0.smithi119.stdout:1896: tier_evict oid 15 2024-09-18T11:32:32.418 INFO:tasks.rados.rados.0.smithi119.stdout:1896: tier_evict oid 15 snap -1 2024-09-18T11:32:32.418 INFO:tasks.rados.rados.0.smithi119.stdout:1888: finishing set_chunk to oid 33 2024-09-18T11:32:32.418 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 33 v 189 (ObjNum 316 snap 43 seq_num 316) dirty exists 2024-09-18T11:32:32.418 INFO:tasks.rados.rados.0.smithi119.stdout:1893: expect (ObjNum 377 snap 58 seq_num 377) 2024-09-18T11:32:32.422 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:32.422 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:32.426 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:32:32.438 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:32.438 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:32.467 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:32.466+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:32.496 INFO:tasks.rados.rados.0.smithi119.stdout:1892: expect (ObjNum 338 snap 48 seq_num 338) 2024-09-18T11:32:32.561 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:32.560+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:32.564 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:32.564 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:32.573 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:32:32.662 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:32.661+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:32.671 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:32.671 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:32.680 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:32:32.763 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:32.762+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:32.770 INFO:tasks.rados.rados.0.smithi119.stdout:1894: finishing set_chunk to oid 10 2024-09-18T11:32:32.771 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 10 v 358 (ObjNum 358 snap 55 seq_num 358) dirty exists 2024-09-18T11:32:32.771 INFO:tasks.rados.rados.0.smithi119.stdout:1890: finishing rollback tid 0 to smithi11939506-12 2024-09-18T11:32:32.771 INFO:tasks.rados.rados.0.smithi119.stdout:1891: expect (ObjNum 352 snap 54 seq_num 352) 2024-09-18T11:32:32.777 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:32.777 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:32.786 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:32:32.870 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:32.869+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:32.925 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:32.925 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:32.933 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:32:32.974 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:32.973+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:33.020 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:33.019+0000 7f3157e90640 -1 osd.4 161 handle_osd_map injecting map crc failure 2024-09-18T11:32:33.022 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is set 2024-09-18T11:32:33.050 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-18T11:32:33.050 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T11:32:33.050 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-18T11:32:33.051 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-18T11:32:33.052 INFO:tasks.rados.rados.0.smithi119.stdout:1890: finishing rollback tid 1 to smithi11939506-12 2024-09-18T11:32:33.052 INFO:tasks.rados.rados.0.smithi119.stdout:1890: finishing rollback tid 2 to smithi11939506-12 2024-09-18T11:32:33.053 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 12 v 135 (ObjNum 283 snap 39 seq_num 283) dirty exists 2024-09-18T11:32:33.053 INFO:tasks.rados.rados.0.smithi119.stdout:1895: finishing set_chunk to oid 4 2024-09-18T11:32:33.053 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 4 set_chunk returned error code -95 offset: 1710592 length: 580096 2024-09-18T11:32:33.053 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 4 v 0 (ObjNum 310 snap 43 seq_num 310) dirty exists 2024-09-18T11:32:33.053 INFO:tasks.rados.rados.0.smithi119.stdout:1888: done (8 left) 2024-09-18T11:32:33.053 INFO:tasks.rados.rados.0.smithi119.stdout:1889: done (7 left) 2024-09-18T11:32:33.053 INFO:tasks.rados.rados.0.smithi119.stdout:1890: done (6 left) 2024-09-18T11:32:33.053 INFO:tasks.rados.rados.0.smithi119.stdout:1891: done (5 left) 2024-09-18T11:32:33.053 INFO:tasks.rados.rados.0.smithi119.stdout:1892: done (4 left) 2024-09-18T11:32:33.053 INFO:tasks.rados.rados.0.smithi119.stdout:1893: done (3 left) 2024-09-18T11:32:33.053 INFO:tasks.rados.rados.0.smithi119.stdout:1894: done (2 left) 2024-09-18T11:32:33.053 INFO:tasks.rados.rados.0.smithi119.stdout:1895: done (1 left) 2024-09-18T11:32:33.054 INFO:tasks.rados.rados.0.smithi119.stdout:1897: snap_remove snap 43 2024-09-18T11:32:33.067 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:33.066+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:33.072 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:33.073 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:33.081 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:32:33.168 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:33.167+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:33.181 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-18T11:32:33.218 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:33.219 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:33.228 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:32:33.309 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:33.308+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:33.352 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-18T11:32:33.366 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:33.367 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:33.376 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:32:33.376 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:33.374+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:33.470 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:33.469+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:33.497 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-18T11:32:33.499 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:33.498+0000 7fd27659f640 -1 osd.6 161 handle_osd_map injecting map crc failure 2024-09-18T11:32:33.517 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:33.517 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:33.526 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:32:33.610 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:33.609+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:33.662 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-18T11:32:33.678 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:33.678 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:33.679 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:33.677+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:33.687 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:32:33.774 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:33.773+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:33.808 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-18T11:32:33.828 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:33.828 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:33.837 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:32:33.878 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:33.877+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:33.958 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-18T11:32:33.976 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:33.975+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:33.978 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:33.978 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:33.988 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:32:34.024 INFO:tasks.rados.rados.0.smithi119.stdout:1897: done (1 left) 2024-09-18T11:32:34.024 INFO:tasks.rados.rados.0.smithi119.stdout:1898: write (excl) oid 32 current snap is 61 2024-09-18T11:32:34.024 INFO:tasks.rados.rados.0.smithi119.stdout:1898: seq_num 385 ranges {470695=788515,1679685=586458} 2024-09-18T11:32:34.028 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:34.027+0000 7f6bf3052640 -1 osd.5 162 handle_osd_map injecting map crc failure 2024-09-18T11:32:34.031 INFO:tasks.rados.rados.0.smithi119.stdout:1898: writing smithi11939506-32 from 470695 to 1259210 tid 1 2024-09-18T11:32:34.038 INFO:tasks.rados.rados.0.smithi119.stdout:1898: writing smithi11939506-32 from 1679685 to 2266143 tid 2 2024-09-18T11:32:34.038 INFO:tasks.rados.rados.0.smithi119.stdout:1899: write (excl) oid 42 current snap is 61 2024-09-18T11:32:34.039 INFO:tasks.rados.rados.0.smithi119.stdout:1896: got (22) Invalid argument 2024-09-18T11:32:34.039 INFO:tasks.rados.rados.0.smithi119.stdout:1899: seq_num 386 ranges {487502=463764,1738167=446469} 2024-09-18T11:32:34.044 INFO:tasks.rados.rados.0.smithi119.stdout:1899: writing smithi11939506-42 from 487502 to 951266 tid 1 2024-09-18T11:32:34.052 INFO:tasks.rados.rados.0.smithi119.stdout:1899: writing smithi11939506-42 from 1738167 to 2184636 tid 2 2024-09-18T11:32:34.052 INFO:tasks.rados.rados.0.smithi119.stdout:1896: done (2 left) 2024-09-18T11:32:34.052 INFO:tasks.rados.rados.0.smithi119.stdout:1900: tier_evict oid 7 2024-09-18T11:32:34.052 INFO:tasks.rados.rados.0.smithi119.stdout:1900: tier_evict oid 7 snap -1 2024-09-18T11:32:34.052 INFO:tasks.rados.rados.0.smithi119.stdout:1898: finishing write tid 1 to smithi11939506-32 2024-09-18T11:32:34.052 INFO:tasks.rados.rados.0.smithi119.stdout:1898: finishing write tid 2 to smithi11939506-32 2024-09-18T11:32:34.052 INFO:tasks.rados.rados.0.smithi119.stdout:1898: finishing write tid 3 to smithi11939506-32 2024-09-18T11:32:34.052 INFO:tasks.rados.rados.0.smithi119.stdout:1898: finishing write tid 4 to smithi11939506-32 2024-09-18T11:32:34.052 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 260 (ObjNum 385 snap 61 seq_num 385) dirty exists 2024-09-18T11:32:34.052 INFO:tasks.rados.rados.0.smithi119.stdout:1898: left oid 32 (ObjNum 385 snap 61 seq_num 385) 2024-09-18T11:32:34.052 INFO:tasks.rados.rados.0.smithi119.stdout:1898: done (2 left) 2024-09-18T11:32:34.052 INFO:tasks.rados.rados.0.smithi119.stdout:1901: snap_create 2024-09-18T11:32:34.053 INFO:tasks.rados.rados.0.smithi119.stdout:1900: got (22) Invalid argument 2024-09-18T11:32:34.055 INFO:tasks.rados.rados.0.smithi119.stdout:1899: finishing write tid 1 to smithi11939506-42 2024-09-18T11:32:34.055 INFO:tasks.rados.rados.0.smithi119.stdout:1899: finishing write tid 2 to smithi11939506-42 2024-09-18T11:32:34.057 INFO:tasks.rados.rados.0.smithi119.stdout:1899: finishing write tid 3 to smithi11939506-42 2024-09-18T11:32:34.057 INFO:tasks.rados.rados.0.smithi119.stdout:1899: finishing write tid 4 to smithi11939506-42 2024-09-18T11:32:34.057 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 368 (ObjNum 386 snap 61 seq_num 386) dirty exists 2024-09-18T11:32:34.057 INFO:tasks.rados.rados.0.smithi119.stdout:1899: left oid 42 (ObjNum 386 snap 61 seq_num 386) 2024-09-18T11:32:34.076 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:34.075+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:34.131 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:34.132 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:34.140 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:32:34.181 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:34.180+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:34.278 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:34.277+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:34.279 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:34.279 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:34.288 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:32:34.379 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:34.378+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:34.429 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:34.429 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:34.437 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:32:34.449 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:32:34.480 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:34.479+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:34.579 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:34.579 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:34.580 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:34.579+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:34.587 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:32:34.593 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:34.594 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:34.609 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:34.610 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:34.626 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:34.627 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:34.642 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:34.642 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:34.658 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:34.658 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:34.674 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:34.674 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:34.681 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:34.680+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:34.690 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:34.690 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:34.694 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:34.694 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:34.694 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:34.694 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:34.703 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:32:34.706 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:34.707 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:34.782 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:34.781+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:34.803 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:34.803 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:34.803 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:34.803 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:34.803 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:34.811 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:32:34.883 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:34.882+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:34.911 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:34.911 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:34.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:34.911 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:34.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.911 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:32:34.911 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:34.911 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 143.739270691, 2024-09-18T11:32:34.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:32:34.911 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:34.912 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:34.912 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:34.912 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:34.912 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:34.912 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:32:34.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.912 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:34.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:34.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:34.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:34.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:34.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:34.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:32:34.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:34.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:34.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:32:34.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:32:34.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:34.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:32:34.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:32:34.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:34.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:32:34.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:32:34.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:32:34.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:32:34.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:34.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:32:34.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:32:34.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:34.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:32:34.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:32:34.915 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.915 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:34.916 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.916 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1001 4.0 4:019b6e09:::smithi11939506-40:6 [read 0~2655473,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:34.916 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:32:34.916 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 139.74411941299999, 2024-09-18T11:32:34.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007461082, 2024-09-18T11:32:34.916 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:34.916 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:34.916 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:34.916 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:34.916 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:34.916 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1001 2024-09-18T11:32:34.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.917 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:34.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:34.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:32:34.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:34.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164571+0000", 2024-09-18T11:32:34.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:34.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164570+0000", 2024-09-18T11:32:34.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:34.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:34.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164574+0000", 2024-09-18T11:32:34.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.822e-06 2024-09-18T11:32:34.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:34.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164582+0000", 2024-09-18T11:32:34.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3410000000000003e-06 2024-09-18T11:32:34.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:34.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164587+0000", 2024-09-18T11:32:34.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2769999999999999e-06 2024-09-18T11:32:34.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164702+0000", 2024-09-18T11:32:34.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115705 2024-09-18T11:32:34.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:34.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164709+0000", 2024-09-18T11:32:34.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1639999999999998e-06 2024-09-18T11:32:34.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170296+0000", 2024-09-18T11:32:34.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055876390000000001 2024-09-18T11:32:34.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:34.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.170418+0000", 2024-09-18T11:32:34.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000121549 2024-09-18T11:32:34.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:34.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172032+0000", 2024-09-18T11:32:34.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00161407 2024-09-18T11:32:34.921 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.921 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:34.921 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.921 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:34.921 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:34.921 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 139.74409269700001, 2024-09-18T11:32:34.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:32:34.922 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:34.922 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:34.922 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:34.922 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:34.922 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:34.922 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:32:34.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.922 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:34.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:34.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:34.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:34.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:34.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:34.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:34.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:34.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:34.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:32:34.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:32:34.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:34.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:32:34.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:32:34.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:34.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:32:34.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:32:34.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:32:34.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:32:34.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:34.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:32:34.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:32:34.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:32:34.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:32:34.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:34.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:32:34.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:32:34.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:34.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:32:34.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:32:34.926 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.926 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:34.926 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.926 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:34.927 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:34.927 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 139.74407440799999, 2024-09-18T11:32:34.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:32:34.927 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:34.927 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:34.927 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:34.927 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:34.927 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:34.927 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:32:34.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.927 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:34.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:34.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:34.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:34.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:34.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:34.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:32:34.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:34.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:34.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:32:34.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:32:34.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:34.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:32:34.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:32:34.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:34.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:32:34.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:32:34.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:32:34.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:32:34.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:34.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:32:34.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:32:34.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:32:34.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:32:34.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:34.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:32:34.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:32:34.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:34.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:32:34.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:32:34.931 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.931 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:34.931 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.932 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:34.932 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:34.932 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 139.744013639, 2024-09-18T11:32:34.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:32:34.932 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:34.932 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:34.932 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:34.932 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:34.932 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:34.932 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:32:34.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.932 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:34.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:34.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:34.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:34.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:34.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:34.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:32:34.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:34.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:34.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:32:34.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:32:34.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:34.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:32:34.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:32:34.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:34.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:32:34.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:32:34.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:32:34.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:32:34.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:34.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:32:34.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:32:34.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:32:34.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:32:34.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:34.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:32:34.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:32:34.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:34.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:32:34.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:32:34.936 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.936 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:34.936 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.937 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:34.937 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:34.937 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 139.74399131800001, 2024-09-18T11:32:34.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:32:34.937 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:34.937 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:34.937 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:34.937 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:34.937 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:34.937 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:32:34.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.937 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:34.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:34.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:34.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:34.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:34.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:34.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:32:34.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:34.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:34.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:32:34.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:32:34.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:34.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:32:34.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:32:34.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:34.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:32:34.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:32:34.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:32:34.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:32:34.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:34.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:32:34.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:32:34.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:32:34.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:32:34.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:34.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:32:34.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:32:34.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:34.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:32:34.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:32:34.941 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.942 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:34.942 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.942 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:32:34.942 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:34.942 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 134.962866435, 2024-09-18T11:32:34.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:32:34.942 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:34.942 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:34.942 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:34.942 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:34.942 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:34.942 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:32:34.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.943 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:34.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:34.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:34.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:34.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:34.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:34.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:32:34.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:32:34.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:34.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:32:34.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:32:34.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:34.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:32:34.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:32:34.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:34.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:32:34.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:32:34.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:32:34.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:32:34.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:34.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:32:34.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:32:34.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:34.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:32:34.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:32:34.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:34.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:32:34.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:32:34.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:34.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:32:34.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:32:34.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:34.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:32:34.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:32:34.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:34.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:32:34.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:32:34.947 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.947 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:34.947 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.948 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1661 4.0 4:019b6e09:::smithi11939506-40:17 [read 0~3445402,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T11:32:34.948 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:34.948 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.558793732, 2024-09-18T11:32:34.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075190880000000002, 2024-09-18T11:32:34.948 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:34.948 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:34.948 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:34.948 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:34.948 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:34.948 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1661 2024-09-18T11:32:34.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.949 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:34.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:34.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:34.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:34.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:34.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:34.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349896+0000", 2024-09-18T11:32:34.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:34.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:34.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349899+0000", 2024-09-18T11:32:34.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9749999999999999e-06 2024-09-18T11:32:34.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:34.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349906+0000", 2024-09-18T11:32:34.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9290000000000001e-06 2024-09-18T11:32:34.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:34.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.349912+0000", 2024-09-18T11:32:34.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9900000000000002e-06 2024-09-18T11:32:34.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355645+0000", 2024-09-18T11:32:34.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057329549999999996 2024-09-18T11:32:34.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:34.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.355776+0000", 2024-09-18T11:32:34.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000130975 2024-09-18T11:32:34.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:34.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:37.357416+0000", 2024-09-18T11:32:34.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001639649 2024-09-18T11:32:34.952 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.952 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:34.952 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.952 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1761 4.0 4:0aec5e1c:::smithi11939506-50:head [write 769787~704463,stat] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:32:34.952 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:32:34.953 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 112.60630131800001, 2024-09-18T11:32:34.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074082879999999999, 2024-09-18T11:32:34.953 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:34.953 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:34.953 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:34.953 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:34.953 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:34.953 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1761 2024-09-18T11:32:34.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.953 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:34.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:34.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:32:34.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:34.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302389+0000", 2024-09-18T11:32:34.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:34.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302386+0000", 2024-09-18T11:32:34.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:34.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:34.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302889+0000", 2024-09-18T11:32:34.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050318499999999996 2024-09-18T11:32:34.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:34.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302930+0000", 2024-09-18T11:32:34.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0198999999999999e-05 2024-09-18T11:32:34.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:34.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.302965+0000", 2024-09-18T11:32:34.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5292e-05 2024-09-18T11:32:34.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303046+0000", 2024-09-18T11:32:34.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1254999999999994e-05 2024-09-18T11:32:34.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:32:34.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.303054+0000", 2024-09-18T11:32:34.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2800000000000003e-06 2024-09-18T11:32:34.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304658+0000", 2024-09-18T11:32:34.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016039520000000001 2024-09-18T11:32:34.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:34.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304724+0000", 2024-09-18T11:32:34.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5888000000000003e-05 2024-09-18T11:32:34.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:34.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.304853+0000", 2024-09-18T11:32:34.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012912700000000001 2024-09-18T11:32:34.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:34.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309084+0000", 2024-09-18T11:32:34.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042311229999999998 2024-09-18T11:32:34.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:34.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309738+0000", 2024-09-18T11:32:34.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00065401399999999996 2024-09-18T11:32:34.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:34.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309770+0000", 2024-09-18T11:32:34.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1652000000000001e-05 2024-09-18T11:32:34.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:34.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:42.309797+0000", 2024-09-18T11:32:34.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7052000000000001e-05 2024-09-18T11:32:34.959 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.959 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:34.959 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.959 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2503 3.1 e85/67 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head v 85'117, mlcod=84'115)", 2024-09-18T11:32:34.959 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:34.959 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.931467748000003, 2024-09-18T11:32:34.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086527619999999996, 2024-09-18T11:32:34.959 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:34.959 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:34.959 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:34.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:34.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:34.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:34.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:34.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:34.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977220+0000", 2024-09-18T11:32:34.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:34.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:34.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980771+0000", 2024-09-18T11:32:34.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035505200000000002 2024-09-18T11:32:34.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:34.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980797+0000", 2024-09-18T11:32:34.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.618e-05 2024-09-18T11:32:34.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:34.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980807+0000", 2024-09-18T11:32:34.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0433e-05 2024-09-18T11:32:34.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980861+0000", 2024-09-18T11:32:34.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3415999999999998e-05 2024-09-18T11:32:34.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:34.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980885+0000", 2024-09-18T11:32:34.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4212000000000001e-05 2024-09-18T11:32:34.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:34.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985826+0000", 2024-09-18T11:32:34.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049412450000000004 2024-09-18T11:32:34.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:34.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985875+0000", 2024-09-18T11:32:34.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9116000000000002e-05 2024-09-18T11:32:34.963 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.963 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:34.963 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.963 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2648 4.0 4:059d8d95:::smithi11939506-32:head [read 0~3647119,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:34.963 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:32:34.963 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 80.577379378000003, 2024-09-18T11:32:34.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0076286949999999996, 2024-09-18T11:32:34.964 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:34.964 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:34.964 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:34.964 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:34.964 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:34.964 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2648 2024-09-18T11:32:34.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.964 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:34.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:34.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:32:34.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:34.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331311+0000", 2024-09-18T11:32:34.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:34.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331309+0000", 2024-09-18T11:32:34.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:34.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:34.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331321+0000", 2024-09-18T11:32:34.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:32:34.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:34.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331340+0000", 2024-09-18T11:32:34.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8227e-05 2024-09-18T11:32:34.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:34.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331348+0000", 2024-09-18T11:32:34.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4640000000000006e-06 2024-09-18T11:32:34.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331406+0000", 2024-09-18T11:32:34.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7445000000000003e-05 2024-09-18T11:32:34.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:34.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331521+0000", 2024-09-18T11:32:34.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000115945 2024-09-18T11:32:34.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:34.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338939+0000", 2024-09-18T11:32:34.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074179859999999997 2024-09-18T11:32:34.968 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.968 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:34.968 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.968 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2649 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:34.968 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:34.968 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 80.577324379000004, 2024-09-18T11:32:34.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566240000000007, 2024-09-18T11:32:34.968 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:34.968 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:34.968 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:34.968 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:34.968 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:34.969 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2649 2024-09-18T11:32:34.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.969 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:34.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:34.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:34.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:34.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:34.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:34.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331365+0000", 2024-09-18T11:32:34.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:34.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:34.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331369+0000", 2024-09-18T11:32:34.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.551e-06 2024-09-18T11:32:34.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:34.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331377+0000", 2024-09-18T11:32:34.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8769999999999993e-06 2024-09-18T11:32:34.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:34.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331381+0000", 2024-09-18T11:32:34.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.16e-06 2024-09-18T11:32:34.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338973+0000", 2024-09-18T11:32:34.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075918460000000002 2024-09-18T11:32:34.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:34.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339048+0000", 2024-09-18T11:32:34.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5813000000000001e-05 2024-09-18T11:32:34.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:34.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339322+0000", 2024-09-18T11:32:34.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027399899999999998 2024-09-18T11:32:34.972 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.972 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:34.972 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.973 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2650 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:34.973 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:34.973 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 80.577304161000001, 2024-09-18T11:32:34.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084392180000000001, 2024-09-18T11:32:34.973 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:34.973 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:34.973 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:34.973 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:34.973 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:34.973 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2650 2024-09-18T11:32:34.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.973 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:34.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:34.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:34.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:34.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:34.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:34.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:34.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:34.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:34.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331388+0000", 2024-09-18T11:32:34.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3429999999999998e-06 2024-09-18T11:32:34.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:34.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331394+0000", 2024-09-18T11:32:34.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.19e-06 2024-09-18T11:32:34.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:34.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331398+0000", 2024-09-18T11:32:34.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.495e-06 2024-09-18T11:32:34.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339362+0000", 2024-09-18T11:32:34.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079641799999999995 2024-09-18T11:32:34.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:34.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339460+0000", 2024-09-18T11:32:34.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8329e-05 2024-09-18T11:32:34.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:34.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339825+0000", 2024-09-18T11:32:34.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036510500000000001 2024-09-18T11:32:34.977 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.977 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:34.977 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.977 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:119 3.2 3:5595a68d:::07854d2fef297a06ba81685e660c332de36d5d18d546927d30daad6d7fda1541:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:34.977 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:32:34.977 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 71.865152404, 2024-09-18T11:32:34.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074145979999999997, 2024-09-18T11:32:34.977 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:34.978 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:34.978 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:34.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:34.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:32:34.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:34.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043538+0000", 2024-09-18T11:32:34.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:34.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043535+0000", 2024-09-18T11:32:34.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:34.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:34.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043970+0000", 2024-09-18T11:32:34.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000435148 2024-09-18T11:32:34.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:34.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.043993+0000", 2024-09-18T11:32:34.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3113999999999999e-05 2024-09-18T11:32:34.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:34.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044003+0000", 2024-09-18T11:32:34.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5310000000000007e-06 2024-09-18T11:32:34.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044062+0000", 2024-09-18T11:32:34.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9487000000000002e-05 2024-09-18T11:32:34.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:32:34.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044071+0000", 2024-09-18T11:32:34.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1610000000000003e-06 2024-09-18T11:32:34.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046423+0000", 2024-09-18T11:32:34.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023523680000000001 2024-09-18T11:32:34.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:34.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046502+0000", 2024-09-18T11:32:34.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9339999999999996e-05 2024-09-18T11:32:34.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:34.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.046664+0000", 2024-09-18T11:32:34.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016185700000000001 2024-09-18T11:32:34.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:34.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050870+0000", 2024-09-18T11:32:34.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042060070000000003 2024-09-18T11:32:34.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:34.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050878+0000", 2024-09-18T11:32:34.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8960000000000003e-06 2024-09-18T11:32:34.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:34.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050918+0000", 2024-09-18T11:32:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0367000000000002e-05 2024-09-18T11:32:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.050952+0000", 2024-09-18T11:32:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3989999999999998e-05 2024-09-18T11:32:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:127 3.2 3:4d3b277a:::3266c7aaa53a7d2c10c3c54d852d6cbbed49fb041bc0df41a3ac83b56e2b0615:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:34.983 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 71.864289221999996, 2024-09-18T11:32:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093148009999999993, 2024-09-18T11:32:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:34.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:34.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:34.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044400+0000", 2024-09-18T11:32:34.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:34.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:34.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044442+0000", 2024-09-18T11:32:34.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2247999999999999e-05 2024-09-18T11:32:34.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:34.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044453+0000", 2024-09-18T11:32:34.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0617e-05 2024-09-18T11:32:34.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:34.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044457+0000", 2024-09-18T11:32:34.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2520000000000001e-06 2024-09-18T11:32:34.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044495+0000", 2024-09-18T11:32:34.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7710000000000003e-05 2024-09-18T11:32:34.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:34.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044499+0000", 2024-09-18T11:32:34.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6269999999999999e-06 2024-09-18T11:32:34.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048239+0000", 2024-09-18T11:32:34.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037396719999999999 2024-09-18T11:32:34.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:34.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048351+0000", 2024-09-18T11:32:34.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112342 2024-09-18T11:32:34.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:34.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048545+0000", 2024-09-18T11:32:34.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000193881 2024-09-18T11:32:34.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:34.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.052689+0000", 2024-09-18T11:32:34.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041435780000000002 2024-09-18T11:32:34.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:34.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053654+0000", 2024-09-18T11:32:34.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096558199999999999 2024-09-18T11:32:34.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:34.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053683+0000", 2024-09-18T11:32:34.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8850999999999999e-05 2024-09-18T11:32:34.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:34.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053716+0000", 2024-09-18T11:32:34.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2468000000000002e-05 2024-09-18T11:32:34.989 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.989 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:34.989 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.989 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3209 4.0 4:019b6e09:::smithi11939506-40:head [write 638732~557432] snapc 42=[42,41,3f,3e,3d,3b,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-18T11:32:34.990 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:34.990 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 57.810607183000002, 2024-09-18T11:32:34.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091774999999999999, 2024-09-18T11:32:34.990 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:34.990 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:34.990 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:34.990 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:34.990 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:34.990 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3209 2024-09-18T11:32:34.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.990 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:34.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:34.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:34.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:34.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:34.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:34.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098080+0000", 2024-09-18T11:32:34.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:34.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:34.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098676+0000", 2024-09-18T11:32:34.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059598099999999998 2024-09-18T11:32:34.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:34.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098706+0000", 2024-09-18T11:32:34.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0020000000000001e-05 2024-09-18T11:32:34.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:34.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098716+0000", 2024-09-18T11:32:34.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0225e-05 2024-09-18T11:32:34.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098782+0000", 2024-09-18T11:32:34.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5530000000000004e-05 2024-09-18T11:32:34.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:34.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098883+0000", 2024-09-18T11:32:34.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101609 2024-09-18T11:32:34.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:34.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.099098+0000", 2024-09-18T11:32:34.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000215052 2024-09-18T11:32:34.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:34.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.102511+0000", 2024-09-18T11:32:34.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034129429999999999 2024-09-18T11:32:34.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:34.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107165+0000", 2024-09-18T11:32:34.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046531480000000002 2024-09-18T11:32:34.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:34.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107217+0000", 2024-09-18T11:32:34.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2058000000000002e-05 2024-09-18T11:32:34.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:34.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107260+0000", 2024-09-18T11:32:34.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3865000000000001e-05 2024-09-18T11:32:34.995 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.995 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:34.995 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.995 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:162 3.1 e138/67 3:8c826a1f:::e682cfc1e82c88472b9acf1cd97978bc105aa035f10b685bb4cace95809b52bf:head v 138'176, mlcod=136'174)", 2024-09-18T11:32:34.995 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:32:34.995 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 25.325133438000002, 2024-09-18T11:32:34.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010445024000000001, 2024-09-18T11:32:34.995 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:34.995 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:34.995 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:34.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:34.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:32:34.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:34.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:32:34.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:34.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:34.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583556+0000", 2024-09-18T11:32:34.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:34.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:34.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584899+0000", 2024-09-18T11:32:34.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001343056 2024-09-18T11:32:34.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:34.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584929+0000", 2024-09-18T11:32:34.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0133000000000001e-05 2024-09-18T11:32:34.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:34.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584937+0000", 2024-09-18T11:32:34.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6380000000000004e-06 2024-09-18T11:32:34.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:34.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584990+0000", 2024-09-18T11:32:34.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3186000000000002e-05 2024-09-18T11:32:34.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:34.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585012+0000", 2024-09-18T11:32:34.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1474000000000001e-05 2024-09-18T11:32:34.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:34.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593963+0000", 2024-09-18T11:32:34.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089512859999999993 2024-09-18T11:32:34.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:34.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594002+0000", 2024-09-18T11:32:34.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8859999999999997e-05 2024-09-18T11:32:34.999 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.999 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:34.999 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:34.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:34.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:34.999 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:166 3.1 e138/67 3:a73e635b:::b6ee478f0f661ec7bac3d17c2ca2fdaff71a47570f64844a7ba79bbb2a0e8d9d:head v 138'177, mlcod=136'174)", 2024-09-18T11:32:34.999 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:32:34.999 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 25.323563554, 2024-09-18T11:32:34.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089018680000000003, 2024-09-18T11:32:35.000 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:35.000 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.000 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:35.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:35.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:32:35.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:35.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:35.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:32:35.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:35.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:35.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585126+0000", 2024-09-18T11:32:35.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:35.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:35.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585211+0000", 2024-09-18T11:32:35.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5432999999999993e-05 2024-09-18T11:32:35.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:35.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585220+0000", 2024-09-18T11:32:35.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5879999999999998e-06 2024-09-18T11:32:35.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585225+0000", 2024-09-18T11:32:35.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8999999999999997e-06 2024-09-18T11:32:35.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:35.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585705+0000", 2024-09-18T11:32:35.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00048042799999999997 2024-09-18T11:32:35.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:35.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585724+0000", 2024-09-18T11:32:35.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8352e-05 2024-09-18T11:32:35.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:35.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594008+0000", 2024-09-18T11:32:35.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082840919999999998 2024-09-18T11:32:35.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:35.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594028+0000", 2024-09-18T11:32:35.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0869e-05 2024-09-18T11:32:35.003 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:35.003 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:35.003 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:35.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.003 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:168 3.1 e138/67 3:b64d7bad:::7feb22256bdcec217720fd46184ab43ced8923ed66ec6319c57f078d9f123262:head v 138'178, mlcod=136'174)", 2024-09-18T11:32:35.003 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:35.003 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 25.322571821, 2024-09-18T11:32:35.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079290650000000008, 2024-09-18T11:32:35.004 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:35.004 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.004 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:35.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:35.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:35.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:35.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:35.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:35.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:35.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:35.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:35.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:35.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:35.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586191+0000", 2024-09-18T11:32:35.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3627000000000005e-05 2024-09-18T11:32:35.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:35.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586202+0000", 2024-09-18T11:32:35.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0488e-05 2024-09-18T11:32:35.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586207+0000", 2024-09-18T11:32:35.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1719999999999998e-06 2024-09-18T11:32:35.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:35.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586239+0000", 2024-09-18T11:32:35.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2570000000000002e-05 2024-09-18T11:32:35.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:35.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586271+0000", 2024-09-18T11:32:35.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1341999999999999e-05 2024-09-18T11:32:35.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:35.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594031+0000", 2024-09-18T11:32:35.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077597459999999997 2024-09-18T11:32:35.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:35.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594047+0000", 2024-09-18T11:32:35.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6858e-05 2024-09-18T11:32:35.007 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:35.007 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:35.007 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:35.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.007 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4267 3.1 3:8d8d13e9:::92886a4340d03039afb2c877fad76e3afe7a80ee04388d55256c386022cd5d63:head [write 0~2203648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:35.008 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:32:35.008 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 7.8515398410000001, 2024-09-18T11:32:35.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0092856740000000007, 2024-09-18T11:32:35.008 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:35.008 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.008 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:35.008 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:35.008 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.008 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4267 2024-09-18T11:32:35.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.008 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:35.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:35.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:32:35.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:35.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:35.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:32:35.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:35.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:35.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057149+0000", 2024-09-18T11:32:35.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:35.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:35.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058175+0000", 2024-09-18T11:32:35.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001026765 2024-09-18T11:32:35.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:35.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058190+0000", 2024-09-18T11:32:35.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4167e-05 2024-09-18T11:32:35.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058195+0000", 2024-09-18T11:32:35.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.823e-06 2024-09-18T11:32:35.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:35.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058247+0000", 2024-09-18T11:32:35.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1304999999999998e-05 2024-09-18T11:32:35.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:35.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058422+0000", 2024-09-18T11:32:35.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017527100000000001 2024-09-18T11:32:35.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:35.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058609+0000", 2024-09-18T11:32:35.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018714299999999999 2024-09-18T11:32:35.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:35.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.062721+0000", 2024-09-18T11:32:35.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041118869999999998 2024-09-18T11:32:35.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:35.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066291+0000", 2024-09-18T11:32:35.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035694860000000002 2024-09-18T11:32:35.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:35.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066328+0000", 2024-09-18T11:32:35.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7864000000000002e-05 2024-09-18T11:32:35.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:35.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:35.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:35.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066436+0000", 2024-09-18T11:32:35.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010762 2024-09-18T11:32:35.013 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:35.013 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:35.013 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:35.013 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:35.014 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:35.014 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:35.015 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:32:35.015 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:35.014+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:35.026 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:35.025+0000 7f2d449e8640 -1 osd.1 163 handle_osd_map injecting map crc failure 2024-09-18T11:32:35.029 INFO:tasks.rados.rados.0.smithi119.stdout:1899: done (2 left) 2024-09-18T11:32:35.029 INFO:tasks.rados.rados.0.smithi119.stdout:1900: done (1 left) 2024-09-18T11:32:35.029 INFO:tasks.rados.rados.0.smithi119.stdout:1901: done (0 left) 2024-09-18T11:32:35.029 INFO:tasks.rados.rados.0.smithi119.stdout:1902: read oid 37 snap -1 2024-09-18T11:32:35.029 INFO:tasks.rados.rados.0.smithi119.stdout:1902: expect deleted 2024-09-18T11:32:35.029 INFO:tasks.rados.rados.0.smithi119.stdout:1903: read oid 7 snap -1 2024-09-18T11:32:35.029 INFO:tasks.rados.rados.0.smithi119.stdout:1903: expect (ObjNum 384 snap 60 seq_num 384) 2024-09-18T11:32:35.029 INFO:tasks.rados.rados.0.smithi119.stdout:1904: read oid 41 snap -1 2024-09-18T11:32:35.029 INFO:tasks.rados.rados.0.smithi119.stdout:1904: expect (ObjNum 347 snap 51 seq_num 347) 2024-09-18T11:32:35.029 INFO:tasks.rados.rados.0.smithi119.stdout:1905: write (excl) oid 1 current snap is 62 2024-09-18T11:32:35.029 INFO:tasks.rados.rados.0.smithi119.stdout:1905: seq_num 387 ranges {504309=539013,1796649=590127} 2024-09-18T11:32:35.030 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:35.028+0000 7fd2785a3640 -1 osd.6 163 handle_osd_map injecting map crc failure 2024-09-18T11:32:35.034 INFO:tasks.rados.rados.0.smithi119.stdout:1905: writing smithi11939506-1 from 504309 to 1043322 tid 1 2024-09-18T11:32:35.035 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T11:32:35.038 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:35.035+0000 7fcb662ad640 -1 osd.7 163 handle_osd_map injecting map crc failure 2024-09-18T11:32:35.042 INFO:tasks.rados.rados.0.smithi119.stdout:1905: writing smithi11939506-1 from 1796649 to 2386776 tid 2 2024-09-18T11:32:35.042 INFO:tasks.rados.rados.0.smithi119.stdout:1906: set_chunk oid 3 target oid 2024-09-18T11:32:35.051 INFO:tasks.rados.rados.0.smithi119.stdout:1906: set_chunk oid 3 offset: 373248 length: 957440 target oid 1fc70b144902afdb653c9753ca2a6eac67b1b1e0e5fa6d04a808b27650495034 offset: 0 snap -1 2024-09-18T11:32:35.051 INFO:tasks.rados.rados.0.smithi119.stdout:1907: snap_remove snap 59 2024-09-18T11:32:35.116 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:35.114+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:35.126 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:35.126 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:35.126 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:35.126 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:35.135 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:32:35.217 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:35.216+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:35.273 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:35.273 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:35.274 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:35.274 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:35.274 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:35.282 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:32:35.318 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:35.317+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:35.382 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:35.382 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:35.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:35.382 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:35.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.382 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:32:35.382 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:35.382 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 120.963562567, 2024-09-18T11:32:35.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:32:35.383 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.383 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.383 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.383 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.383 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.383 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:32:35.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.383 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:35.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:35.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:32:35.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:35.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:32:35.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:32:35.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:32:35.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:32:35.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:32:35.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:32:35.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:32:35.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:32:35.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:32:35.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:32:35.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:35.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:32:35.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:32:35.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:35.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:32:35.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:32:35.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:35.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:32:35.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:32:35.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:35.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:32:35.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:32:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:32:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:32:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:172 4.5 e78/22 4:a087a566:::smithi11939506-46:19 v 78'135, mlcod=77'132)", 2024-09-18T11:32:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:32:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 93.993825912999995, 2024-09-18T11:32:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010938658, 2024-09-18T11:32:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:32:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385824+0000", 2024-09-18T11:32:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385823+0000", 2024-09-18T11:32:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385830+0000", 2024-09-18T11:32:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2790000000000001e-06 2024-09-18T11:32:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385843+0000", 2024-09-18T11:32:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2785e-05 2024-09-18T11:32:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385848+0000", 2024-09-18T11:32:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5570000000000003e-06 2024-09-18T11:32:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385914+0000", 2024-09-18T11:32:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5168999999999999e-05 2024-09-18T11:32:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.385930+0000", 2024-09-18T11:32:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6884999999999998e-05 2024-09-18T11:32:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396721+0000", 2024-09-18T11:32:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010790137 2024-09-18T11:32:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.396762+0000", 2024-09-18T11:32:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1635999999999999e-05 2024-09-18T11:32:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:32:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 93.993333160999995, 2024-09-18T11:32:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:32:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:32:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:32:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:32:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:32:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:32:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:32:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:32:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:32:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:32:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:32:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:32:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:32:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:32:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:32:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 86.407220554999995, 2024-09-18T11:32:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:32:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:32:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:32:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:32:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:32:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:32:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:32:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:32:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:32:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:32:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:32:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:32:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:32:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:32:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:32:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:32:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:32:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:32:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:32:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:32:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:32:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:32:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:32:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2956 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~2100908,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 71.352424305, 2024-09-18T11:32:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011863831, 2024-09-18T11:32:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2956 2024-09-18T11:32:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027228+0000", 2024-09-18T11:32:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:32:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027235+0000", 2024-09-18T11:32:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:32:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027239+0000", 2024-09-18T11:32:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.535e-06 2024-09-18T11:32:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027831+0000", 2024-09-18T11:32:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059244699999999996 2024-09-18T11:32:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027879+0000", 2024-09-18T11:32:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:32:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038098+0000", 2024-09-18T11:32:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010219137 2024-09-18T11:32:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038207+0000", 2024-09-18T11:32:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010951999999999999 2024-09-18T11:32:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039089+0000", 2024-09-18T11:32:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00088151699999999996 2024-09-18T11:32:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2957 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 71.352406521000006, 2024-09-18T11:32:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012041979, 2024-09-18T11:32:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2957 2024-09-18T11:32:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027245+0000", 2024-09-18T11:32:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:32:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027251+0000", 2024-09-18T11:32:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0090000000000004e-06 2024-09-18T11:32:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027255+0000", 2024-09-18T11:32:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:32:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027895+0000", 2024-09-18T11:32:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064076400000000005 2024-09-18T11:32:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:35.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027930+0000", 2024-09-18T11:32:35.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4334e-05 2024-09-18T11:32:35.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039111+0000", 2024-09-18T11:32:35.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181455999999999 2024-09-18T11:32:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039156+0000", 2024-09-18T11:32:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5256000000000002e-05 2024-09-18T11:32:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039285+0000", 2024-09-18T11:32:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012839200000000001 2024-09-18T11:32:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:978 4.b e132/67 4:d8d14c58:::smithi11939506-49:30 v 132'417, mlcod=128'414)", 2024-09-18T11:32:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.354375238999999, 2024-09-18T11:32:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011942183, 2024-09-18T11:32:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025278+0000", 2024-09-18T11:32:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9060000000000004e-06 2024-09-18T11:32:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025288+0000", 2024-09-18T11:32:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.028e-05 2024-09-18T11:32:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025293+0000", 2024-09-18T11:32:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2700000000000004e-06 2024-09-18T11:32:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025346+0000", 2024-09-18T11:32:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2862999999999999e-05 2024-09-18T11:32:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025370+0000", 2024-09-18T11:32:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4005000000000001e-05 2024-09-18T11:32:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037159+0000", 2024-09-18T11:32:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011788343 2024-09-18T11:32:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037216+0000", 2024-09-18T11:32:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7695000000000002e-05 2024-09-18T11:32:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3930 4.b e132/67 4:d3ae8215:::smithi11939506-41:head v 132'421, mlcod=128'414)", 2024-09-18T11:32:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.350826990999998, 2024-09-18T11:32:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015093554, 2024-09-18T11:32:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028821+0000", 2024-09-18T11:32:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029805+0000", 2024-09-18T11:32:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00098410899999999994 2024-09-18T11:32:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029817+0000", 2024-09-18T11:32:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2058e-05 2024-09-18T11:32:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029823+0000", 2024-09-18T11:32:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6450000000000002e-06 2024-09-18T11:32:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032330+0000", 2024-09-18T11:32:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025067710000000001 2024-09-18T11:32:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032349+0000", 2024-09-18T11:32:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9729e-05 2024-09-18T11:32:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043878+0000", 2024-09-18T11:32:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011528514 2024-09-18T11:32:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043916+0000", 2024-09-18T11:32:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8049000000000001e-05 2024-09-18T11:32:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:162 3.1 3:8c826a1f:::e682cfc1e82c88472b9acf1cd97978bc105aa035f10b685bb4cace95809b52bf:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:32:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.798704832999999, 2024-09-18T11:32:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013234571000000001, 2024-09-18T11:32:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:32:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:32:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580943+0000", 2024-09-18T11:32:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582556+0000", 2024-09-18T11:32:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016132970000000001 2024-09-18T11:32:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582572+0000", 2024-09-18T11:32:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5846999999999999e-05 2024-09-18T11:32:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582578+0000", 2024-09-18T11:32:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4810000000000003e-06 2024-09-18T11:32:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582632+0000", 2024-09-18T11:32:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4156000000000001e-05 2024-09-18T11:32:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582819+0000", 2024-09-18T11:32:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000186431 2024-09-18T11:32:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583118+0000", 2024-09-18T11:32:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029880100000000001 2024-09-18T11:32:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590676+0000", 2024-09-18T11:32:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075588670000000004 2024-09-18T11:32:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594111+0000", 2024-09-18T11:32:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034348550000000001 2024-09-18T11:32:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594142+0000", 2024-09-18T11:32:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.099e-05 2024-09-18T11:32:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594179+0000", 2024-09-18T11:32:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6831e-05 2024-09-18T11:32:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3974 4.d 4:b5a463bf:::smithi11939506-20:head [write 665643~697786,stat] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:32:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.797618150000002, 2024-09-18T11:32:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016621174999999998, 2024-09-18T11:32:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3974 2024-09-18T11:32:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:32:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:32:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582030+0000", 2024-09-18T11:32:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585185+0000", 2024-09-18T11:32:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003155644 2024-09-18T11:32:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585208+0000", 2024-09-18T11:32:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3013000000000001e-05 2024-09-18T11:32:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585215+0000", 2024-09-18T11:32:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7569999999999998e-06 2024-09-18T11:32:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585278+0000", 2024-09-18T11:32:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2699000000000004e-05 2024-09-18T11:32:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585374+0000", 2024-09-18T11:32:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.6062000000000003e-05 2024-09-18T11:32:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:32:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585577+0000", 2024-09-18T11:32:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000202926 2024-09-18T11:32:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596544+0000", 2024-09-18T11:32:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010966848 2024-09-18T11:32:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598594+0000", 2024-09-18T11:32:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020502049999999998 2024-09-18T11:32:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598625+0000", 2024-09-18T11:32:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1412999999999999e-05 2024-09-18T11:32:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598652+0000", 2024-09-18T11:32:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7166999999999999e-05 2024-09-18T11:32:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:166 3.1 3:a73e635b:::b6ee478f0f661ec7bac3d17c2ca2fdaff71a47570f64844a7ba79bbb2a0e8d9d:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:32:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.797058784000001, 2024-09-18T11:32:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011711037000000001, 2024-09-18T11:32:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.435 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:32:35.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:32:35.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582590+0000", 2024-09-18T11:32:35.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:35.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582673+0000", 2024-09-18T11:32:35.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2861000000000005e-05 2024-09-18T11:32:35.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582683+0000", 2024-09-18T11:32:35.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0081999999999999e-05 2024-09-18T11:32:35.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582688+0000", 2024-09-18T11:32:35.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6870000000000002e-06 2024-09-18T11:32:35.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584266+0000", 2024-09-18T11:32:35.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001578423 2024-09-18T11:32:35.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584407+0000", 2024-09-18T11:32:35.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014144599999999999 2024-09-18T11:32:35.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:35.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584599+0000", 2024-09-18T11:32:35.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000191881 2024-09-18T11:32:35.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:35.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590680+0000", 2024-09-18T11:32:35.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060801479999999996 2024-09-18T11:32:35.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:35.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594224+0000", 2024-09-18T11:32:35.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035441700000000001 2024-09-18T11:32:35.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:35.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594269+0000", 2024-09-18T11:32:35.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5297999999999999e-05 2024-09-18T11:32:35.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594302+0000", 2024-09-18T11:32:35.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2703000000000002e-05 2024-09-18T11:32:35.439 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.439 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.439 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.440 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:168 3.1 3:b64d7bad:::7feb22256bdcec217720fd46184ab43ced8923ed66ec6319c57f078d9f123262:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:35.440 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:32:35.440 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.796831332, 2024-09-18T11:32:35.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011589809, 2024-09-18T11:32:35.440 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.440 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.440 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:32:35.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:32:35.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582817+0000", 2024-09-18T11:32:35.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:35.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582903+0000", 2024-09-18T11:32:35.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5915999999999995e-05 2024-09-18T11:32:35.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582913+0000", 2024-09-18T11:32:35.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1409999999999994e-06 2024-09-18T11:32:35.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582917+0000", 2024-09-18T11:32:35.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6419999999999998e-06 2024-09-18T11:32:35.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585540+0000", 2024-09-18T11:32:35.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026228940000000002 2024-09-18T11:32:35.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585721+0000", 2024-09-18T11:32:35.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018086299999999999 2024-09-18T11:32:35.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:35.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585918+0000", 2024-09-18T11:32:35.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000196758 2024-09-18T11:32:35.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:35.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590683+0000", 2024-09-18T11:32:35.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047657020000000001 2024-09-18T11:32:35.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:35.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594354+0000", 2024-09-18T11:32:35.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003671068 2024-09-18T11:32:35.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:35.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594386+0000", 2024-09-18T11:32:35.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1426e-05 2024-09-18T11:32:35.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594408+0000", 2024-09-18T11:32:35.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1987000000000001e-05 2024-09-18T11:32:35.445 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.445 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.445 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.445 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:172 3.1 3:8c3b343b:::d73444b0abf5ea0abb7f4bfb783ec9a4695bac9fb180180b88c2eef6a2e03b80:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:35.445 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.583414+0000", 2024-09-18T11:32:35.446 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.796235289999998, 2024-09-18T11:32:35.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011061609999999999, 2024-09-18T11:32:35.446 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.446 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.446 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583414+0000", 2024-09-18T11:32:35.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583414+0000", 2024-09-18T11:32:35.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583413+0000", 2024-09-18T11:32:35.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:35.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583525+0000", 2024-09-18T11:32:35.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000111289 2024-09-18T11:32:35.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583537+0000", 2024-09-18T11:32:35.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1755e-05 2024-09-18T11:32:35.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583542+0000", 2024-09-18T11:32:35.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4399999999999996e-06 2024-09-18T11:32:35.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.588499+0000", 2024-09-18T11:32:35.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004957347 2024-09-18T11:32:35.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.588684+0000", 2024-09-18T11:32:35.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000185002 2024-09-18T11:32:35.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:35.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.588889+0000", 2024-09-18T11:32:35.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020478200000000001 2024-09-18T11:32:35.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:35.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.591496+0000", 2024-09-18T11:32:35.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026070619999999998 2024-09-18T11:32:35.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:35.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594430+0000", 2024-09-18T11:32:35.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029342219999999998 2024-09-18T11:32:35.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:35.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594458+0000", 2024-09-18T11:32:35.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7243e-05 2024-09-18T11:32:35.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594476+0000", 2024-09-18T11:32:35.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8139e-05 2024-09-18T11:32:35.451 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.451 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.451 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.451 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:175 3.1 3:b5e86b88:::e55cf3d5cf782ed327a67e746b287b76d05f74f58a4f3726e399a564440ceaf6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:35.451 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:32:35.451 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 25.795982390999999, 2024-09-18T11:32:35.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013709627, 2024-09-18T11:32:35.451 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.451 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.451 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:32:35.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:32:35.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583666+0000", 2024-09-18T11:32:35.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:35.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584041+0000", 2024-09-18T11:32:35.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037511499999999998 2024-09-18T11:32:35.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584055+0000", 2024-09-18T11:32:35.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3295999999999999e-05 2024-09-18T11:32:35.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584061+0000", 2024-09-18T11:32:35.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1630000000000001e-06 2024-09-18T11:32:35.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.589980+0000", 2024-09-18T11:32:35.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005918998 2024-09-18T11:32:35.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590117+0000", 2024-09-18T11:32:35.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013715400000000001 2024-09-18T11:32:35.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:35.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590270+0000", 2024-09-18T11:32:35.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015245600000000001 2024-09-18T11:32:35.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:35.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596513+0000", 2024-09-18T11:32:35.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062438729999999996 2024-09-18T11:32:35.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:35.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597328+0000", 2024-09-18T11:32:35.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081489599999999996 2024-09-18T11:32:35.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:35.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597358+0000", 2024-09-18T11:32:35.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9291000000000001e-05 2024-09-18T11:32:35.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597377+0000", 2024-09-18T11:32:35.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:32:35.457 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.457 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.457 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.457 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4061 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:35.457 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:32:35.457 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.082803702, 2024-09-18T11:32:35.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013340967, 2024-09-18T11:32:35.457 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.458 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:35.458 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.458 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.458 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.458 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4061 2024-09-18T11:32:35.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.458 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:32:35.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:32:35.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296844+0000", 2024-09-18T11:32:35.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:35.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296854+0000", 2024-09-18T11:32:35.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0037000000000001e-05 2024-09-18T11:32:35.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296873+0000", 2024-09-18T11:32:35.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:32:35.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296881+0000", 2024-09-18T11:32:35.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3790000000000006e-06 2024-09-18T11:32:35.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296975+0000", 2024-09-18T11:32:35.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3536999999999996e-05 2024-09-18T11:32:35.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:35.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296982+0000", 2024-09-18T11:32:35.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6399999999999997e-06 2024-09-18T11:32:35.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301933+0000", 2024-09-18T11:32:35.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049507190000000001 2024-09-18T11:32:35.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301984+0000", 2024-09-18T11:32:35.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1439e-05 2024-09-18T11:32:35.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:32:35.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082022050000000006 2024-09-18T11:32:35.462 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.463 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.463 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.463 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4062 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:35.463 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:35.463 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.082755147, 2024-09-18T11:32:35.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016297147000000001, 2024-09-18T11:32:35.463 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.463 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:35.463 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.463 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.463 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.464 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4062 2024-09-18T11:32:35.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.464 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:35.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:35.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:35.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:35.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296898+0000", 2024-09-18T11:32:35.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9419999999999997e-06 2024-09-18T11:32:35.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296906+0000", 2024-09-18T11:32:35.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3520000000000007e-06 2024-09-18T11:32:35.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296910+0000", 2024-09-18T11:32:35.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4469999999999999e-06 2024-09-18T11:32:35.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297032+0000", 2024-09-18T11:32:35.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012168900000000001 2024-09-18T11:32:35.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:35.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297038+0000", 2024-09-18T11:32:35.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9320000000000001e-06 2024-09-18T11:32:35.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310221+0000", 2024-09-18T11:32:35.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013182887000000001 2024-09-18T11:32:35.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310323+0000", 2024-09-18T11:32:35.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010170000000000001 2024-09-18T11:32:35.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313191+0000", 2024-09-18T11:32:35.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002868808 2024-09-18T11:32:35.468 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.468 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.468 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.468 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4063 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:35.468 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:35.469 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.082734254999998, 2024-09-18T11:32:35.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017648712, 2024-09-18T11:32:35.469 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.469 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:35.469 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.469 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.469 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.469 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4063 2024-09-18T11:32:35.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.469 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:35.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:35.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:35.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:35.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296918+0000", 2024-09-18T11:32:35.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8100000000000002e-06 2024-09-18T11:32:35.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296924+0000", 2024-09-18T11:32:35.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2519999999999996e-06 2024-09-18T11:32:35.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296928+0000", 2024-09-18T11:32:35.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0500000000000002e-06 2024-09-18T11:32:35.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297062+0000", 2024-09-18T11:32:35.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013425500000000001 2024-09-18T11:32:35.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:35.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297069+0000", 2024-09-18T11:32:35.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5239999999999997e-06 2024-09-18T11:32:35.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313424+0000", 2024-09-18T11:32:35.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016355277000000001 2024-09-18T11:32:35.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313514+0000", 2024-09-18T11:32:35.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0581999999999994e-05 2024-09-18T11:32:35.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.314564+0000", 2024-09-18T11:32:35.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001049384 2024-09-18T11:32:35.473 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.474 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.474 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.474 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4065 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:35.474 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:32:35.474 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.082688814000001, 2024-09-18T11:32:35.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013225997, 2024-09-18T11:32:35.474 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.474 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:35.474 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.474 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.474 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.474 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4065 2024-09-18T11:32:35.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.475 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:32:35.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:32:35.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296960+0000", 2024-09-18T11:32:35.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:35.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296963+0000", 2024-09-18T11:32:35.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2270000000000002e-06 2024-09-18T11:32:35.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296970+0000", 2024-09-18T11:32:35.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6220000000000003e-06 2024-09-18T11:32:35.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296973+0000", 2024-09-18T11:32:35.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3730000000000001e-06 2024-09-18T11:32:35.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297097+0000", 2024-09-18T11:32:35.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012352399999999999 2024-09-18T11:32:35.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:35.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297103+0000", 2024-09-18T11:32:35.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7119999999999997e-06 2024-09-18T11:32:35.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307831+0000", 2024-09-18T11:32:35.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010728794 2024-09-18T11:32:35.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307924+0000", 2024-09-18T11:32:35.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2099e-05 2024-09-18T11:32:35.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:32:35.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022630469999999998 2024-09-18T11:32:35.480 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.480 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.480 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.481 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4066 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:35.481 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:32:35.481 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.082671738999998, 2024-09-18T11:32:35.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015656389999999999, 2024-09-18T11:32:35.481 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.481 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:35.481 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.481 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.481 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.481 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4066 2024-09-18T11:32:35.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.481 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:32:35.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:32:35.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296977+0000", 2024-09-18T11:32:35.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:35.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296980+0000", 2024-09-18T11:32:35.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8830000000000002e-06 2024-09-18T11:32:35.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296985+0000", 2024-09-18T11:32:35.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:32:35.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296988+0000", 2024-09-18T11:32:35.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0419999999999998e-06 2024-09-18T11:32:35.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297120+0000", 2024-09-18T11:32:35.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131693 2024-09-18T11:32:35.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:35.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297125+0000", 2024-09-18T11:32:35.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1100000000000002e-06 2024-09-18T11:32:35.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310212+0000", 2024-09-18T11:32:35.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013087395999999999 2024-09-18T11:32:35.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310292+0000", 2024-09-18T11:32:35.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9980000000000003e-05 2024-09-18T11:32:35.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312634+0000", 2024-09-18T11:32:35.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002341756 2024-09-18T11:32:35.485 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.486 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.486 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.486 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4068 3.1 3:87e79ed0:::3526641c07d3efa989be9684355570f39c331008b8a02c9fa22b38a94b6c6b9f:head [write 0~1015808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:35.486 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:32:35.486 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.078895522, 2024-09-18T11:32:35.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016392648999999999, 2024-09-18T11:32:35.486 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.486 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.486 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.486 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.486 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.486 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4068 2024-09-18T11:32:35.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.487 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:32:35.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:32:35.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300753+0000", 2024-09-18T11:32:35.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:35.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301922+0000", 2024-09-18T11:32:35.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001169432 2024-09-18T11:32:35.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301936+0000", 2024-09-18T11:32:35.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3574000000000001e-05 2024-09-18T11:32:35.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301941+0000", 2024-09-18T11:32:35.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6069999999999998e-06 2024-09-18T11:32:35.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310234+0000", 2024-09-18T11:32:35.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082924820000000003 2024-09-18T11:32:35.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310337+0000", 2024-09-18T11:32:35.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102673 2024-09-18T11:32:35.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:35.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310469+0000", 2024-09-18T11:32:35.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131982 2024-09-18T11:32:35.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:35.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312844+0000", 2024-09-18T11:32:35.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023758680000000002 2024-09-18T11:32:35.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:35.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317092+0000", 2024-09-18T11:32:35.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042477549999999998 2024-09-18T11:32:35.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:35.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317124+0000", 2024-09-18T11:32:35.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1968999999999998e-05 2024-09-18T11:32:35.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317147+0000", 2024-09-18T11:32:35.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2339000000000001e-05 2024-09-18T11:32:35.491 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.491 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.491 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.491 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.491 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.492 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:35.493 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:32:35.493 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:35.492+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:35.591 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:35.591 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:35.591 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:35.591 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:35.593 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:35.592+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:35.599 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:32:35.694 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:35.693+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:35.737 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:35.737 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:35.737 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:35.737 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:35.737 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:35.746 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:32:35.796 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:35.795+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:35.885 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:35.885 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:35.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:35.885 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:35.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.885 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1788 4.8 4:1898b15d:::smithi11939506-25:head [write 453822~797061] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:32:35.885 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:35.886 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.966714911, 2024-09-18T11:32:35.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094305159999999999, 2024-09-18T11:32:35.886 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.886 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.886 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.886 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.886 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.886 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1788 2024-09-18T11:32:35.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.886 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:35.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:35.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915811+0000", 2024-09-18T11:32:35.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:35.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916608+0000", 2024-09-18T11:32:35.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079635000000000003 2024-09-18T11:32:35.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916624+0000", 2024-09-18T11:32:35.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6735000000000002e-05 2024-09-18T11:32:35.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916630+0000", 2024-09-18T11:32:35.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5709999999999997e-06 2024-09-18T11:32:35.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916674+0000", 2024-09-18T11:32:35.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4540999999999999e-05 2024-09-18T11:32:35.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916751+0000", 2024-09-18T11:32:35.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6695999999999999e-05 2024-09-18T11:32:35.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:35.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916872+0000", 2024-09-18T11:32:35.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120648 2024-09-18T11:32:35.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:35.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.919076+0000", 2024-09-18T11:32:35.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022046829999999998 2024-09-18T11:32:35.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:35.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925151+0000", 2024-09-18T11:32:35.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060740239999999999 2024-09-18T11:32:35.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:35.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925198+0000", 2024-09-18T11:32:35.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7896999999999999e-05 2024-09-18T11:32:35.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925244+0000", 2024-09-18T11:32:35.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5398999999999997e-05 2024-09-18T11:32:35.892 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.892 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.892 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.893 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2877 4.3 4:cc46532a:::smithi11939506-33:2f [read 0~3680733,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:35.893 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:35.893 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 73.375632455000002, 2024-09-18T11:32:35.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085715170000000007, 2024-09-18T11:32:35.893 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.893 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:35.893 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.893 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.893 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.893 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2877 2024-09-18T11:32:35.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.894 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:35.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:35.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:35.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:35.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506898+0000", 2024-09-18T11:32:35.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4760000000000001e-06 2024-09-18T11:32:35.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506903+0000", 2024-09-18T11:32:35.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:32:35.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506906+0000", 2024-09-18T11:32:35.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1769999999999998e-06 2024-09-18T11:32:35.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507007+0000", 2024-09-18T11:32:35.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101479 2024-09-18T11:32:35.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:35.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507015+0000", 2024-09-18T11:32:35.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8490000000000006e-06 2024-09-18T11:32:35.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513757+0000", 2024-09-18T11:32:35.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0067417800000000002 2024-09-18T11:32:35.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513867+0000", 2024-09-18T11:32:35.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000109987 2024-09-18T11:32:35.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515467+0000", 2024-09-18T11:32:35.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00160037 2024-09-18T11:32:35.899 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.899 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.899 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.899 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2878 4.3 4:cc46532a:::smithi11939506-33:2f [sparse-read 0~1770976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:35.899 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:35.899 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 73.375602000000001, 2024-09-18T11:32:35.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090602640000000002, 2024-09-18T11:32:35.900 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.900 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:35.900 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.900 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.900 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.900 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2878 2024-09-18T11:32:35.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.900 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:35.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:35.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:35.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:35.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506929+0000", 2024-09-18T11:32:35.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.163e-06 2024-09-18T11:32:35.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506936+0000", 2024-09-18T11:32:35.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5519999999999999e-06 2024-09-18T11:32:35.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506941+0000", 2024-09-18T11:32:35.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:32:35.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507036+0000", 2024-09-18T11:32:35.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5070999999999998e-05 2024-09-18T11:32:35.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:35.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507042+0000", 2024-09-18T11:32:35.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3959999999999997e-06 2024-09-18T11:32:35.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515504+0000", 2024-09-18T11:32:35.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084620300000000006 2024-09-18T11:32:35.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515625+0000", 2024-09-18T11:32:35.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012150999999999999 2024-09-18T11:32:35.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515986+0000", 2024-09-18T11:32:35.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036140100000000002 2024-09-18T11:32:35.906 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.906 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.906 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.906 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3963 4.3 4:c70c3162:::smithi11939506-35:head [tier-promote] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:35.906 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:32:35.906 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.506704989999999, 2024-09-18T11:32:35.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010675591999999999, 2024-09-18T11:32:35.906 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.906 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:35.907 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.907 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.907 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.907 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3963 2024-09-18T11:32:35.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.907 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:32:35.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:32:35.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375821+0000", 2024-09-18T11:32:35.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:35.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375829+0000", 2024-09-18T11:32:35.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7989999999999994e-06 2024-09-18T11:32:35.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375843+0000", 2024-09-18T11:32:35.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:32:35.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375849+0000", 2024-09-18T11:32:35.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1419999999999998e-06 2024-09-18T11:32:35.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375889+0000", 2024-09-18T11:32:35.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0197999999999997e-05 2024-09-18T11:32:35.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:35.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375894+0000", 2024-09-18T11:32:35.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0139999999999998e-06 2024-09-18T11:32:35.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.377699+0000", 2024-09-18T11:32:35.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001804688 2024-09-18T11:32:35.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.377776+0000", 2024-09-18T11:32:35.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7081000000000003e-05 2024-09-18T11:32:35.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.386499+0000", 2024-09-18T11:32:35.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008722746 2024-09-18T11:32:35.912 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.912 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.912 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.912 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1025 4.7 e138/22 4:e73ba68f:::smithi11939506-1:head v 138'281, mlcod=137'278)", 2024-09-18T11:32:35.912 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.587274+0000", 2024-09-18T11:32:35.912 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.295254675999999, 2024-09-18T11:32:35.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094231360000000004, 2024-09-18T11:32:35.913 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.913 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.913 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.587274+0000", 2024-09-18T11:32:35.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.587274+0000", 2024-09-18T11:32:35.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.587273+0000", 2024-09-18T11:32:35.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:35.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590258+0000", 2024-09-18T11:32:35.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002984998 2024-09-18T11:32:35.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590273+0000", 2024-09-18T11:32:35.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4999e-05 2024-09-18T11:32:35.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590283+0000", 2024-09-18T11:32:35.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3619999999999995e-06 2024-09-18T11:32:35.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590333+0000", 2024-09-18T11:32:35.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9978999999999997e-05 2024-09-18T11:32:35.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590352+0000", 2024-09-18T11:32:35.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9477999999999999e-05 2024-09-18T11:32:35.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:35.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596664+0000", 2024-09-18T11:32:35.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063120679999999997 2024-09-18T11:32:35.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596697+0000", 2024-09-18T11:32:35.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2456999999999998e-05 2024-09-18T11:32:35.917 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.917 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.917 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.918 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4259 4.b 4:dba5bd39:::smithi11939506-17:head [write 685818~644423] snapc 69=[69,67,61,55,51,4e,4b,45,42,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:35.918 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.040083+0000", 2024-09-18T11:32:35.918 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.842444811, 2024-09-18T11:32:35.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015748635, 2024-09-18T11:32:35.918 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.918 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.918 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.918 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.918 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.919 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4259 2024-09-18T11:32:35.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.919 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.040083+0000", 2024-09-18T11:32:35.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.040083+0000", 2024-09-18T11:32:35.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.040081+0000", 2024-09-18T11:32:35.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:35.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041258+0000", 2024-09-18T11:32:35.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011766540000000001 2024-09-18T11:32:35.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041286+0000", 2024-09-18T11:32:35.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8087000000000001e-05 2024-09-18T11:32:35.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041362+0000", 2024-09-18T11:32:35.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6155999999999994e-05 2024-09-18T11:32:35.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.044285+0000", 2024-09-18T11:32:35.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002922437 2024-09-18T11:32:35.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.044360+0000", 2024-09-18T11:32:35.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5201e-05 2024-09-18T11:32:35.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:35.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.044489+0000", 2024-09-18T11:32:35.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012950599999999999 2024-09-18T11:32:35.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:35.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052904+0000", 2024-09-18T11:32:35.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084149910000000001 2024-09-18T11:32:35.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:35.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055695+0000", 2024-09-18T11:32:35.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027902040000000001 2024-09-18T11:32:35.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:35.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055765+0000", 2024-09-18T11:32:35.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0720000000000001e-05 2024-09-18T11:32:35.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055832+0000", 2024-09-18T11:32:35.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6736999999999995e-05 2024-09-18T11:32:35.925 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.925 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.925 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.925 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.5.0:195 4.b e156/152 ondisk, result = 0)", 2024-09-18T11:32:35.925 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.041863+0000", 2024-09-18T11:32:35.925 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.8406652910000005, 2024-09-18T11:32:35.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010991236, 2024-09-18T11:32:35.926 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.926 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:35.926 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041863+0000", 2024-09-18T11:32:35.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041863+0000", 2024-09-18T11:32:35.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041862+0000", 2024-09-18T11:32:35.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:35.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041868+0000", 2024-09-18T11:32:35.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2979999999999997e-06 2024-09-18T11:32:35.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041879+0000", 2024-09-18T11:32:35.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0522e-05 2024-09-18T11:32:35.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041885+0000", 2024-09-18T11:32:35.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7830000000000004e-06 2024-09-18T11:32:35.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052841+0000", 2024-09-18T11:32:35.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010956604 2024-09-18T11:32:35.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052848+0000", 2024-09-18T11:32:35.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2249999999999997e-06 2024-09-18T11:32:35.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052854+0000", 2024-09-18T11:32:35.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6050000000000003e-06 2024-09-18T11:32:35.930 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.930 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.930 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.930 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.5.0:196 4.b e156/152 ondisk, result = 0)", 2024-09-18T11:32:35.930 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.042994+0000", 2024-09-18T11:32:35.930 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.8395346289999992, 2024-09-18T11:32:35.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098926020000000003, 2024-09-18T11:32:35.931 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.931 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:35.931 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.042994+0000", 2024-09-18T11:32:35.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.042994+0000", 2024-09-18T11:32:35.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.042993+0000", 2024-09-18T11:32:35.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:35.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043000+0000", 2024-09-18T11:32:35.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.525e-06 2024-09-18T11:32:35.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043011+0000", 2024-09-18T11:32:35.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0335e-05 2024-09-18T11:32:35.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043015+0000", 2024-09-18T11:32:35.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2899999999999996e-06 2024-09-18T11:32:35.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052868+0000", 2024-09-18T11:32:35.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098528130000000002 2024-09-18T11:32:35.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052871+0000", 2024-09-18T11:32:35.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5080000000000001e-06 2024-09-18T11:32:35.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052886+0000", 2024-09-18T11:32:35.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4781e-05 2024-09-18T11:32:35.935 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.935 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.935 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.935 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.5.0:194 4.3 e156/67 ondisk, result = 0)", 2024-09-18T11:32:35.935 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.043892+0000", 2024-09-18T11:32:35.935 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.8386358390000002, 2024-09-18T11:32:35.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010237263999999999, 2024-09-18T11:32:35.935 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.935 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:35.936 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043892+0000", 2024-09-18T11:32:35.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043892+0000", 2024-09-18T11:32:35.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043891+0000", 2024-09-18T11:32:35.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:35.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043898+0000", 2024-09-18T11:32:35.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1190000000000002e-06 2024-09-18T11:32:35.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043910+0000", 2024-09-18T11:32:35.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.29e-05 2024-09-18T11:32:35.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043916+0000", 2024-09-18T11:32:35.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9499999999999998e-06 2024-09-18T11:32:35.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052830+0000", 2024-09-18T11:32:35.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089137480000000009 2024-09-18T11:32:35.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052839+0000", 2024-09-18T11:32:35.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9560000000000003e-06 2024-09-18T11:32:35.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054130+0000", 2024-09-18T11:32:35.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012905639999999999 2024-09-18T11:32:35.940 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.940 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.940 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.940 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4260 4.b 4:dba5bd39:::smithi11939506-17:head [write 2069089~216729] snapc 69=[69,67,61,55,51,4e,4b,45,42,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:35.940 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045444+0000", 2024-09-18T11:32:35.940 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.8370843170000004, 2024-09-18T11:32:35.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011166715000000001, 2024-09-18T11:32:35.941 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.941 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.941 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.941 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.941 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.941 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4260 2024-09-18T11:32:35.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.941 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045444+0000", 2024-09-18T11:32:35.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045444+0000", 2024-09-18T11:32:35.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045443+0000", 2024-09-18T11:32:35.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:35.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045649+0000", 2024-09-18T11:32:35.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020605499999999999 2024-09-18T11:32:35.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045662+0000", 2024-09-18T11:32:35.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3386e-05 2024-09-18T11:32:35.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045667+0000", 2024-09-18T11:32:35.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6750000000000001e-06 2024-09-18T11:32:35.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052921+0000", 2024-09-18T11:32:35.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007254613 2024-09-18T11:32:35.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052994+0000", 2024-09-18T11:32:35.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2721999999999999e-05 2024-09-18T11:32:35.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:35.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.053116+0000", 2024-09-18T11:32:35.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121972 2024-09-18T11:32:35.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:35.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055857+0000", 2024-09-18T11:32:35.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027414539999999999 2024-09-18T11:32:35.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:35.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056502+0000", 2024-09-18T11:32:35.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064412499999999995 2024-09-18T11:32:35.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:35.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056555+0000", 2024-09-18T11:32:35.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3467999999999997e-05 2024-09-18T11:32:35.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056611+0000", 2024-09-18T11:32:35.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5577e-05 2024-09-18T11:32:35.947 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.947 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.947 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.948 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4261 4.b 4:dba5bd39:::smithi11939506-17:head [setxattr _header (57) in=7b,truncate 2285818] snapc 69=[69,67,61,55,51,4e,4b,45,42,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:35.948 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045672+0000", 2024-09-18T11:32:35.948 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.8368562189999995, 2024-09-18T11:32:35.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011289933, 2024-09-18T11:32:35.948 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.948 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.948 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.948 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.948 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.948 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4261 2024-09-18T11:32:35.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.949 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045672+0000", 2024-09-18T11:32:35.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045672+0000", 2024-09-18T11:32:35.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045671+0000", 2024-09-18T11:32:35.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:35.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045675+0000", 2024-09-18T11:32:35.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.563e-06 2024-09-18T11:32:35.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045681+0000", 2024-09-18T11:32:35.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5210000000000002e-06 2024-09-18T11:32:35.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045684+0000", 2024-09-18T11:32:35.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4149999999999999e-06 2024-09-18T11:32:35.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054143+0000", 2024-09-18T11:32:35.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084590810000000002 2024-09-18T11:32:35.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054214+0000", 2024-09-18T11:32:35.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1061000000000003e-05 2024-09-18T11:32:35.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:35.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054354+0000", 2024-09-18T11:32:35.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000140035 2024-09-18T11:32:35.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:35.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056208+0000", 2024-09-18T11:32:35.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018539229999999999 2024-09-18T11:32:35.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:35.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056885+0000", 2024-09-18T11:32:35.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00067688000000000004 2024-09-18T11:32:35.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:35.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056922+0000", 2024-09-18T11:32:35.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6693999999999998e-05 2024-09-18T11:32:35.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056962+0000", 2024-09-18T11:32:35.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0299000000000002e-05 2024-09-18T11:32:35.954 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.954 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.954 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.955 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4262 4.b 4:dba5bd39:::smithi11939506-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:35.955 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:32:35.955 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.8368400769999997, 2024-09-18T11:32:35.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011764513000000001, 2024-09-18T11:32:35.955 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.955 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:35.955 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.955 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.955 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.955 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4262 2024-09-18T11:32:35.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.956 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:32:35.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:32:35.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:32:35.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:35.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045690+0000", 2024-09-18T11:32:35.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.745e-06 2024-09-18T11:32:35.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045697+0000", 2024-09-18T11:32:35.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7229999999999998e-06 2024-09-18T11:32:35.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045704+0000", 2024-09-18T11:32:35.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5719999999999999e-06 2024-09-18T11:32:35.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055555+0000", 2024-09-18T11:32:35.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098515430000000008 2024-09-18T11:32:35.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:35.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055675+0000", 2024-09-18T11:32:35.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012000799999999999 2024-09-18T11:32:35.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.057004+0000", 2024-09-18T11:32:35.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013286179999999999 2024-09-18T11:32:35.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.057357+0000", 2024-09-18T11:32:35.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00035355100000000002 2024-09-18T11:32:35.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.057453+0000", 2024-09-18T11:32:35.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5154000000000004e-05 2024-09-18T11:32:35.961 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.961 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.961 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.961 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4264 4.7 4:e0d95e10:::smithi11939506-22:head [read 0~3008486,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:35.961 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:32:35.961 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.8367900900000009, 2024-09-18T11:32:35.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095333889999999998, 2024-09-18T11:32:35.962 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.962 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:35.962 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.962 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.962 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.962 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4264 2024-09-18T11:32:35.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.962 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:32:35.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:32:35.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:32:35.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:35.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045747+0000", 2024-09-18T11:32:35.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2469999999999997e-06 2024-09-18T11:32:35.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045755+0000", 2024-09-18T11:32:35.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1499999999999999e-06 2024-09-18T11:32:35.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045759+0000", 2024-09-18T11:32:35.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0849999999999999e-06 2024-09-18T11:32:35.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045803+0000", 2024-09-18T11:32:35.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3621000000000001e-05 2024-09-18T11:32:35.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:32:35.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045807+0000", 2024-09-18T11:32:35.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7290000000000002e-06 2024-09-18T11:32:35.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.053475+0000", 2024-09-18T11:32:35.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076680100000000003 2024-09-18T11:32:35.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.053644+0000", 2024-09-18T11:32:35.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016971299999999999 2024-09-18T11:32:35.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055272+0000", 2024-09-18T11:32:35.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001627245 2024-09-18T11:32:35.967 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.967 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.967 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.967 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4265 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~2163506] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:35.967 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045792+0000", 2024-09-18T11:32:35.967 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.8367362000000007, 2024-09-18T11:32:35.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010085024, 2024-09-18T11:32:35.968 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.968 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:35.968 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.968 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.968 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.968 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4265 2024-09-18T11:32:35.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.968 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045792+0000", 2024-09-18T11:32:35.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045792+0000", 2024-09-18T11:32:35.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045791+0000", 2024-09-18T11:32:35.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:35.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045797+0000", 2024-09-18T11:32:35.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0209999999999996e-06 2024-09-18T11:32:35.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045805+0000", 2024-09-18T11:32:35.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9349999999999994e-06 2024-09-18T11:32:35.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045811+0000", 2024-09-18T11:32:35.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2240000000000003e-06 2024-09-18T11:32:35.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045858+0000", 2024-09-18T11:32:35.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7565999999999998e-05 2024-09-18T11:32:35.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:32:35.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045869+0000", 2024-09-18T11:32:35.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0852e-05 2024-09-18T11:32:35.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055337+0000", 2024-09-18T11:32:35.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094678729999999999 2024-09-18T11:32:35.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055417+0000", 2024-09-18T11:32:35.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0092000000000005e-05 2024-09-18T11:32:35.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055877+0000", 2024-09-18T11:32:35.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00046001800000000003 2024-09-18T11:32:35.973 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.973 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.973 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.973 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4348 4.b 4:d13decd8:::smithi11939506-2:head [sparse-read 0~1463453,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:32:35.974 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.382848+0000", 2024-09-18T11:32:35.974 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.4996801699999995, 2024-09-18T11:32:35.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0077048719999999998, 2024-09-18T11:32:35.974 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.974 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:35.974 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.974 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.974 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.974 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4348 2024-09-18T11:32:35.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.974 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382848+0000", 2024-09-18T11:32:35.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382848+0000", 2024-09-18T11:32:35.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382846+0000", 2024-09-18T11:32:35.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:35.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382859+0000", 2024-09-18T11:32:35.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2792000000000001e-05 2024-09-18T11:32:35.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382884+0000", 2024-09-18T11:32:35.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4929000000000001e-05 2024-09-18T11:32:35.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382893+0000", 2024-09-18T11:32:35.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8169999999999994e-06 2024-09-18T11:32:35.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382958+0000", 2024-09-18T11:32:35.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4851999999999995e-05 2024-09-18T11:32:35.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:35.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382967+0000", 2024-09-18T11:32:35.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9789999999999999e-06 2024-09-18T11:32:35.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.385505+0000", 2024-09-18T11:32:35.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002537849 2024-09-18T11:32:35.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.385634+0000", 2024-09-18T11:32:35.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000128608 2024-09-18T11:32:35.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.390553+0000", 2024-09-18T11:32:35.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049192979999999999 2024-09-18T11:32:35.979 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.979 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.979 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.980 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4349 4.b 4:d13decd8:::smithi11939506-2:head [read 0~2302625,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:32:35.980 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.382910+0000", 2024-09-18T11:32:35.980 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.4996181359999996, 2024-09-18T11:32:35.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098226930000000004, 2024-09-18T11:32:35.980 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.980 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:35.980 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.980 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.980 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.981 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4349 2024-09-18T11:32:35.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.981 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382910+0000", 2024-09-18T11:32:35.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382910+0000", 2024-09-18T11:32:35.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382909+0000", 2024-09-18T11:32:35.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:35.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382915+0000", 2024-09-18T11:32:35.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6049999999999997e-06 2024-09-18T11:32:35.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382924+0000", 2024-09-18T11:32:35.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6289999999999999e-06 2024-09-18T11:32:35.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382929+0000", 2024-09-18T11:32:35.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0479999999999998e-06 2024-09-18T11:32:35.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383030+0000", 2024-09-18T11:32:35.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101346 2024-09-18T11:32:35.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:35.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383036+0000", 2024-09-18T11:32:35.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2720000000000003e-06 2024-09-18T11:32:35.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.390597+0000", 2024-09-18T11:32:35.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075606880000000003 2024-09-18T11:32:35.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.390696+0000", 2024-09-18T11:32:35.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8740000000000007e-05 2024-09-18T11:32:35.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392733+0000", 2024-09-18T11:32:35.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002037025 2024-09-18T11:32:35.986 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.986 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.986 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.986 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4350 4.b 4:d13decd8:::smithi11939506-2:head [sparse-read 0~1463453] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:32:35.987 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.382978+0000", 2024-09-18T11:32:35.987 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.4995505729999996, 2024-09-18T11:32:35.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01020132, 2024-09-18T11:32:35.987 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.987 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:35.987 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.987 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.987 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.987 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4350 2024-09-18T11:32:35.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.987 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382978+0000", 2024-09-18T11:32:35.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382978+0000", 2024-09-18T11:32:35.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382976+0000", 2024-09-18T11:32:35.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:35.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382984+0000", 2024-09-18T11:32:35.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.328e-06 2024-09-18T11:32:35.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382994+0000", 2024-09-18T11:32:35.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0801999999999999e-05 2024-09-18T11:32:35.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383001+0000", 2024-09-18T11:32:35.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6390000000000003e-06 2024-09-18T11:32:35.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383138+0000", 2024-09-18T11:32:35.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013676899999999999 2024-09-18T11:32:35.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:35.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383144+0000", 2024-09-18T11:32:35.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6359999999999997e-06 2024-09-18T11:32:35.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392791+0000", 2024-09-18T11:32:35.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096461659999999994 2024-09-18T11:32:35.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392897+0000", 2024-09-18T11:32:35.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000106087 2024-09-18T11:32:35.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.393179+0000", 2024-09-18T11:32:35.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00028227500000000002 2024-09-18T11:32:35.992 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.992 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.992 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.992 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4375 4.b 4:dba5bd39:::smithi11939506-17:head [write 437081~638017,stat] snapc 6d=[6d,6c,69,67,61,55,51,4b,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:32:35.992 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.549471+0000", 2024-09-18T11:32:35.993 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.3330568710000001, 2024-09-18T11:32:35.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097774179999999995, 2024-09-18T11:32:35.993 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:35.993 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:35.993 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:35.993 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:35.993 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:35.993 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4375 2024-09-18T11:32:35.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.993 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:35.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:35.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.549471+0000", 2024-09-18T11:32:35.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:35.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.549471+0000", 2024-09-18T11:32:35.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:35.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:35.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.549470+0000", 2024-09-18T11:32:35.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:35.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:35.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553463+0000", 2024-09-18T11:32:35.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039932520000000001 2024-09-18T11:32:35.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:35.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553478+0000", 2024-09-18T11:32:35.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4647999999999999e-05 2024-09-18T11:32:35.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:35.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553484+0000", 2024-09-18T11:32:35.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3799999999999999e-06 2024-09-18T11:32:35.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:35.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553536+0000", 2024-09-18T11:32:35.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1938999999999999e-05 2024-09-18T11:32:35.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:35.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553637+0000", 2024-09-18T11:32:35.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010074899999999999 2024-09-18T11:32:35.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:35.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553824+0000", 2024-09-18T11:32:35.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000187524 2024-09-18T11:32:35.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:35.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.556653+0000", 2024-09-18T11:32:35.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028285879999999999 2024-09-18T11:32:35.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:35.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.559176+0000", 2024-09-18T11:32:35.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025229369999999998 2024-09-18T11:32:35.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:35.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.559212+0000", 2024-09-18T11:32:35.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5750000000000002e-05 2024-09-18T11:32:35.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:35.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.559249+0000", 2024-09-18T11:32:35.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7020999999999997e-05 2024-09-18T11:32:35.999 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.999 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:35.999 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:35.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:35.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:35.999 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4430 4.b 4:d3ae8215:::smithi11939506-41:head [read 0~3832029,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T11:32:35.999 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:32:36.000 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.85401791000000005, 2024-09-18T11:32:36.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010918034, 2024-09-18T11:32:36.000 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.000 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:36.000 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.000 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.000 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.000 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4430 2024-09-18T11:32:36.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.000 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:32:36.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:32:36.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:32:36.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:36.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028520+0000", 2024-09-18T11:32:36.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.005e-05 2024-09-18T11:32:36.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028538+0000", 2024-09-18T11:32:36.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8020999999999999e-05 2024-09-18T11:32:36.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028543+0000", 2024-09-18T11:32:36.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.485e-06 2024-09-18T11:32:36.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028634+0000", 2024-09-18T11:32:36.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1120999999999997e-05 2024-09-18T11:32:36.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:36.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028639+0000", 2024-09-18T11:32:36.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1989999999999997e-06 2024-09-18T11:32:36.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.036054+0000", 2024-09-18T11:32:36.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074147559999999998 2024-09-18T11:32:36.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.036158+0000", 2024-09-18T11:32:36.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103564 2024-09-18T11:32:36.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039428+0000", 2024-09-18T11:32:36.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032706390000000001 2024-09-18T11:32:36.005 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.006 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.006 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.006 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4431 4.b 4:d3ae8215:::smithi11939506-41:head [sparse-read 0~2481209] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T11:32:36.006 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:32:36.006 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.85397912200000003, 2024-09-18T11:32:36.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011399179000000001, 2024-09-18T11:32:36.006 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.006 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:36.006 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.007 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.007 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.007 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4431 2024-09-18T11:32:36.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.007 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:32:36.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:32:36.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:32:36.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028552+0000", 2024-09-18T11:32:36.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5880000000000002e-06 2024-09-18T11:32:36.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028558+0000", 2024-09-18T11:32:36.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.84e-06 2024-09-18T11:32:36.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028562+0000", 2024-09-18T11:32:36.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6600000000000001e-06 2024-09-18T11:32:36.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028676+0000", 2024-09-18T11:32:36.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011408000000000001 2024-09-18T11:32:36.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:36.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028681+0000", 2024-09-18T11:32:36.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5350000000000004e-06 2024-09-18T11:32:36.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039458+0000", 2024-09-18T11:32:36.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010777231 2024-09-18T11:32:36.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039564+0000", 2024-09-18T11:32:36.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000105524 2024-09-18T11:32:36.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039948+0000", 2024-09-18T11:32:36.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00038430099999999998 2024-09-18T11:32:36.012 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.012 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.012 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.012 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.012 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.012 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:36.014 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:32:36.015 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:36.013+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:36.028 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:36.027+0000 7fd27659f640 -1 osd.6 164 handle_osd_map injecting map crc failure 2024-09-18T11:32:36.029 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is set 2024-09-18T11:32:36.031 INFO:tasks.rados.rados.0.smithi119.stdout:1907: done (5 left) 2024-09-18T11:32:36.031 INFO:tasks.rados.rados.0.smithi119.stdout:1908: read oid 23 snap -1 2024-09-18T11:32:36.031 INFO:tasks.rados.rados.0.smithi119.stdout:1908: expect (ObjNum 363 snap 55 seq_num 363) 2024-09-18T11:32:36.031 INFO:tasks.rados.rados.0.smithi119.stdout:1909: read oid 27 snap 56 2024-09-18T11:32:36.031 INFO:tasks.rados.rados.0.smithi119.stdout:1909: expect (ObjNum 328 snap 45 seq_num 328) 2024-09-18T11:32:36.031 INFO:tasks.rados.rados.0.smithi119.stdout:1910: delete oid 12 current snap is 62 2024-09-18T11:32:36.031 INFO:tasks.rados.rados.0.smithi119.stdout:1903: expect (ObjNum 384 snap 60 seq_num 384) 2024-09-18T11:32:36.115 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:36.114+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:36.117 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:36.117 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:36.117 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:36.117 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:36.126 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:32:36.216 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:36.214+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:36.217 INFO:tasks.rados.rados.0.smithi119.stdout:1904: expect (ObjNum 347 snap 51 seq_num 347) 2024-09-18T11:32:36.263 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:36.263 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:36.263 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:36.263 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:36.264 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:36.272 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:32:36.316 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:36.315+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:36.372 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:36.372 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:36.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:36.372 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:36.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.372 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:32:36.372 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:36.373 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 145.25324898, 2024-09-18T11:32:36.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:32:36.373 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.373 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:36.373 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.373 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.373 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.373 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:32:36.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.373 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:36.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:36.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:32:36.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:36.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:32:36.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:32:36.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:32:36.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:32:36.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:32:36.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:32:36.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:32:36.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:32:36.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:36.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:32:36.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:32:36.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:32:36.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:32:36.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:32:36.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:32:36.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:32:36.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:32:36.377 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.377 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.377 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.378 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:32:36.378 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:36.378 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 135.70022357900001, 2024-09-18T11:32:36.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:32:36.378 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.378 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:36.378 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.378 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.378 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.378 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:32:36.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.378 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:36.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:36.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:32:36.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:36.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:32:36.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:32:36.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:32:36.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:32:36.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:32:36.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:32:36.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:32:36.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:32:36.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:32:36.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:32:36.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:32:36.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:32:36.382 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.382 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.382 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.382 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2207 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1410296~761389] snapc 24=[24,1e,1c,1b,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T11:32:36.382 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:32:36.382 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 98.947070916000001, 2024-09-18T11:32:36.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008410743, 2024-09-18T11:32:36.382 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.382 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.383 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.383 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.383 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.383 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2207 2024-09-18T11:32:36.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.383 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:32:36.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:32:36.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422496+0000", 2024-09-18T11:32:36.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:36.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423190+0000", 2024-09-18T11:32:36.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00069405499999999995 2024-09-18T11:32:36.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423204+0000", 2024-09-18T11:32:36.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:32:36.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423208+0000", 2024-09-18T11:32:36.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:32:36.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423293+0000", 2024-09-18T11:32:36.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4524999999999995e-05 2024-09-18T11:32:36.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423394+0000", 2024-09-18T11:32:36.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101395 2024-09-18T11:32:36.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:36.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423579+0000", 2024-09-18T11:32:36.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018461000000000001 2024-09-18T11:32:36.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.428574+0000", 2024-09-18T11:32:36.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049944960000000002 2024-09-18T11:32:36.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430809+0000", 2024-09-18T11:32:36.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022351820000000001 2024-09-18T11:32:36.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430862+0000", 2024-09-18T11:32:36.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3189999999999997e-05 2024-09-18T11:32:36.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430908+0000", 2024-09-18T11:32:36.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5973999999999997e-05 2024-09-18T11:32:36.388 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.388 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.388 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.388 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2339 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:36.389 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:36.389 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.982766287000004, 2024-09-18T11:32:36.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011809992, 2024-09-18T11:32:36.389 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.389 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:36.389 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.389 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.389 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.389 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2339 2024-09-18T11:32:36.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.389 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:36.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:36.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:32:36.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:36.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:32:36.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4710000000000001e-06 2024-09-18T11:32:36.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386824+0000", 2024-09-18T11:32:36.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6104e-05 2024-09-18T11:32:36.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386836+0000", 2024-09-18T11:32:36.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1827999999999999e-05 2024-09-18T11:32:36.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388390+0000", 2024-09-18T11:32:36.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015539499999999999 2024-09-18T11:32:36.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:36.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388443+0000", 2024-09-18T11:32:36.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3785000000000001e-05 2024-09-18T11:32:36.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398238+0000", 2024-09-18T11:32:36.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097947989999999999 2024-09-18T11:32:36.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398315+0000", 2024-09-18T11:32:36.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6909999999999994e-05 2024-09-18T11:32:36.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398612+0000", 2024-09-18T11:32:36.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029671999999999999 2024-09-18T11:32:36.394 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.394 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.394 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.394 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2340 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:36.394 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:36.394 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.982722975000001, 2024-09-18T11:32:36.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012086448, 2024-09-18T11:32:36.394 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.394 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:36.394 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.395 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.395 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.395 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2340 2024-09-18T11:32:36.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.395 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:36.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:36.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:36.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386848+0000", 2024-09-18T11:32:36.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3069999999999998e-06 2024-09-18T11:32:36.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386855+0000", 2024-09-18T11:32:36.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0979999999999996e-06 2024-09-18T11:32:36.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386859+0000", 2024-09-18T11:32:36.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9360000000000001e-06 2024-09-18T11:32:36.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388466+0000", 2024-09-18T11:32:36.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001607154 2024-09-18T11:32:36.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:36.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388507+0000", 2024-09-18T11:32:36.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1093000000000001e-05 2024-09-18T11:32:36.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398647+0000", 2024-09-18T11:32:36.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010139686 2024-09-18T11:32:36.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398718+0000", 2024-09-18T11:32:36.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1138999999999998e-05 2024-09-18T11:32:36.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398931+0000", 2024-09-18T11:32:36.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021356899999999999 2024-09-18T11:32:36.399 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.399 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.399 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.399 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2341 4.5 4:aa575018:::smithi11939506-11:head [read 0~2487436,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:36.399 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:36.399 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.982704752999993, 2024-09-18T11:32:36.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013751648999999999, 2024-09-18T11:32:36.399 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.399 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:36.400 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.400 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.400 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.400 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2341 2024-09-18T11:32:36.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.400 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:36.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:36.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:36.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386866+0000", 2024-09-18T11:32:36.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4249999999999998e-06 2024-09-18T11:32:36.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386872+0000", 2024-09-18T11:32:36.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.003e-06 2024-09-18T11:32:36.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:32:36.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.507e-06 2024-09-18T11:32:36.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388520+0000", 2024-09-18T11:32:36.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001644146 2024-09-18T11:32:36.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:36.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388561+0000", 2024-09-18T11:32:36.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1486999999999998e-05 2024-09-18T11:32:36.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398963+0000", 2024-09-18T11:32:36.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010401353 2024-09-18T11:32:36.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.399046+0000", 2024-09-18T11:32:36.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3141999999999998e-05 2024-09-18T11:32:36.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.400615+0000", 2024-09-18T11:32:36.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015690960000000001 2024-09-18T11:32:36.404 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.404 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.404 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.404 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2530 4.5 4:af4ba75a:::smithi11939506-37:head [rollback 25] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:36.404 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:32:36.404 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 87.196169182999995, 2024-09-18T11:32:36.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080198800000000001, 2024-09-18T11:32:36.404 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.405 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.405 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.405 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.405 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.405 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2530 2024-09-18T11:32:36.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.405 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:32:36.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:32:36.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173398+0000", 2024-09-18T11:32:36.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:36.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173403+0000", 2024-09-18T11:32:36.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4389999999999999e-06 2024-09-18T11:32:36.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173414+0000", 2024-09-18T11:32:36.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0411000000000001e-05 2024-09-18T11:32:36.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173420+0000", 2024-09-18T11:32:36.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9610000000000002e-06 2024-09-18T11:32:36.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176390+0000", 2024-09-18T11:32:36.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029706289999999998 2024-09-18T11:32:36.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176501+0000", 2024-09-18T11:32:36.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011090300000000001 2024-09-18T11:32:36.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:36.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176827+0000", 2024-09-18T11:32:36.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00032526899999999999 2024-09-18T11:32:36.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178958+0000", 2024-09-18T11:32:36.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021314860000000001 2024-09-18T11:32:36.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181318+0000", 2024-09-18T11:32:36.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002359571 2024-09-18T11:32:36.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181372+0000", 2024-09-18T11:32:36.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3884000000000001e-05 2024-09-18T11:32:36.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181419+0000", 2024-09-18T11:32:36.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7148000000000003e-05 2024-09-18T11:32:36.410 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.410 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.410 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.410 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2531 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:36.410 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:36.410 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 87.196143059999997, 2024-09-18T11:32:36.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080687239999999993, 2024-09-18T11:32:36.410 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.410 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.410 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.410 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.410 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.411 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2531 2024-09-18T11:32:36.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.411 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:36.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:36.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:36.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173429+0000", 2024-09-18T11:32:36.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2989999999999998e-06 2024-09-18T11:32:36.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173436+0000", 2024-09-18T11:32:36.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1799999999999999e-06 2024-09-18T11:32:36.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173441+0000", 2024-09-18T11:32:36.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8810000000000004e-06 2024-09-18T11:32:36.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178267+0000", 2024-09-18T11:32:36.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048257969999999997 2024-09-18T11:32:36.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178388+0000", 2024-09-18T11:32:36.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121779 2024-09-18T11:32:36.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:36.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178599+0000", 2024-09-18T11:32:36.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000210094 2024-09-18T11:32:36.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.179339+0000", 2024-09-18T11:32:36.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00074063799999999995 2024-09-18T11:32:36.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181444+0000", 2024-09-18T11:32:36.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021044100000000001 2024-09-18T11:32:36.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181470+0000", 2024-09-18T11:32:36.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6415999999999999e-05 2024-09-18T11:32:36.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181494+0000", 2024-09-18T11:32:36.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3672e-05 2024-09-18T11:32:36.417 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.417 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.417 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.417 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2910 4.5 4:af4ba75a:::smithi11939506-37:head [write 786627~795991,stat] snapc 37=[37,35,34,30,2f,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:36.417 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:32:36.418 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 73.366085231, 2024-09-18T11:32:36.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084093710000000006, 2024-09-18T11:32:36.418 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.418 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.418 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.418 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.418 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.418 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2910 2024-09-18T11:32:36.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.418 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:32:36.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:32:36.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003481+0000", 2024-09-18T11:32:36.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:36.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004300+0000", 2024-09-18T11:32:36.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081934200000000003 2024-09-18T11:32:36.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004324+0000", 2024-09-18T11:32:36.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4618999999999999e-05 2024-09-18T11:32:36.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004335+0000", 2024-09-18T11:32:36.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.092e-05 2024-09-18T11:32:36.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004385+0000", 2024-09-18T11:32:36.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9709999999999997e-05 2024-09-18T11:32:36.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004470+0000", 2024-09-18T11:32:36.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4864000000000006e-05 2024-09-18T11:32:36.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:36.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004612+0000", 2024-09-18T11:32:36.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014241500000000001 2024-09-18T11:32:36.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011193+0000", 2024-09-18T11:32:36.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006580654 2024-09-18T11:32:36.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011788+0000", 2024-09-18T11:32:36.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059533600000000002 2024-09-18T11:32:36.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011842+0000", 2024-09-18T11:32:36.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3180000000000002e-05 2024-09-18T11:32:36.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011892+0000", 2024-09-18T11:32:36.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0618000000000002e-05 2024-09-18T11:32:36.423 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.423 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.423 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.424 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2953 4.a 4:588debc2:::smithi11939506-18:head [assert-version v113,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:36.424 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:32:36.424 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 72.342769712999996, 2024-09-18T11:32:36.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083931279999999997, 2024-09-18T11:32:36.424 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.424 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.424 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.424 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.424 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.424 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2953 2024-09-18T11:32:36.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.424 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:32:36.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:32:36.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026796+0000", 2024-09-18T11:32:36.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:36.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026806+0000", 2024-09-18T11:32:36.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9529999999999994e-06 2024-09-18T11:32:36.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026824+0000", 2024-09-18T11:32:36.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7901999999999999e-05 2024-09-18T11:32:36.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026832+0000", 2024-09-18T11:32:36.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8450000000000003e-06 2024-09-18T11:32:36.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026883+0000", 2024-09-18T11:32:36.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0883000000000001e-05 2024-09-18T11:32:36.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026980+0000", 2024-09-18T11:32:36.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7526000000000001e-05 2024-09-18T11:32:36.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:32:36.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032722+0000", 2024-09-18T11:32:36.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057414689999999999 2024-09-18T11:32:36.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.034230+0000", 2024-09-18T11:32:36.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001508035 2024-09-18T11:32:36.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035090+0000", 2024-09-18T11:32:36.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00086047300000000005 2024-09-18T11:32:36.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035147+0000", 2024-09-18T11:32:36.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6598000000000003e-05 2024-09-18T11:32:36.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035191+0000", 2024-09-18T11:32:36.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4747999999999999e-05 2024-09-18T11:32:36.429 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.429 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.429 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.430 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3105 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 3e=[3e,3d,3b,37,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-18T11:32:36.430 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:36.430 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 63.351357008999997, 2024-09-18T11:32:36.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011735906000000001, 2024-09-18T11:32:36.430 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.430 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:36.430 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.430 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.430 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.430 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3105 2024-09-18T11:32:36.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.430 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:36.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:36.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:36.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018212+0000", 2024-09-18T11:32:36.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6759999999999999e-06 2024-09-18T11:32:36.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018216+0000", 2024-09-18T11:32:36.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-06 2024-09-18T11:32:36.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018218+0000", 2024-09-18T11:32:36.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6110000000000001e-06 2024-09-18T11:32:36.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018246+0000", 2024-09-18T11:32:36.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7557000000000001e-05 2024-09-18T11:32:36.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:36.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018250+0000", 2024-09-18T11:32:36.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4299999999999999e-06 2024-09-18T11:32:36.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021364+0000", 2024-09-18T11:32:36.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031135389999999998 2024-09-18T11:32:36.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021494+0000", 2024-09-18T11:32:36.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000130333 2024-09-18T11:32:36.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.029947+0000", 2024-09-18T11:32:36.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084526150000000001 2024-09-18T11:32:36.435 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.435 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.435 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.435 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3323 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:36.435 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:32:36.435 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.252668964999998, 2024-09-18T11:32:36.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0088051689999999998, 2024-09-18T11:32:36.435 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.435 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.435 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.435 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.435 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.435 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3323 2024-09-18T11:32:36.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.436 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:32:36.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:32:36.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116897+0000", 2024-09-18T11:32:36.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:36.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116908+0000", 2024-09-18T11:32:36.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1511e-05 2024-09-18T11:32:36.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116925+0000", 2024-09-18T11:32:36.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7028e-05 2024-09-18T11:32:36.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116932+0000", 2024-09-18T11:32:36.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5919999999999997e-06 2024-09-18T11:32:36.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116988+0000", 2024-09-18T11:32:36.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5928999999999999e-05 2024-09-18T11:32:36.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:36.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116995+0000", 2024-09-18T11:32:36.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7409999999999992e-06 2024-09-18T11:32:36.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122390+0000", 2024-09-18T11:32:36.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053941889999999998 2024-09-18T11:32:36.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122453+0000", 2024-09-18T11:32:36.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3833999999999999e-05 2024-09-18T11:32:36.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:36.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122581+0000", 2024-09-18T11:32:36.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012751200000000001 2024-09-18T11:32:36.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125034+0000", 2024-09-18T11:32:36.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024525609999999998 2024-09-18T11:32:36.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125607+0000", 2024-09-18T11:32:36.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00057356499999999995 2024-09-18T11:32:36.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125655+0000", 2024-09-18T11:32:36.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8179000000000001e-05 2024-09-18T11:32:36.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125704+0000", 2024-09-18T11:32:36.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8950999999999999e-05 2024-09-18T11:32:36.441 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.441 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.441 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.441 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3324 4.5 4:a087a566:::smithi11939506-46:head [rollback 46] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:36.441 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:36.441 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.252626405999997, 2024-09-18T11:32:36.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010927767999999999, 2024-09-18T11:32:36.442 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.442 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.442 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.442 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.442 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.442 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3324 2024-09-18T11:32:36.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.442 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:36.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:36.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116941+0000", 2024-09-18T11:32:36.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116949+0000", 2024-09-18T11:32:36.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4660000000000002e-06 2024-09-18T11:32:36.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116960+0000", 2024-09-18T11:32:36.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1484999999999999e-05 2024-09-18T11:32:36.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116964+0000", 2024-09-18T11:32:36.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2810000000000002e-06 2024-09-18T11:32:36.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117053+0000", 2024-09-18T11:32:36.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8747999999999998e-05 2024-09-18T11:32:36.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:36.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117059+0000", 2024-09-18T11:32:36.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.246e-06 2024-09-18T11:32:36.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123604+0000", 2024-09-18T11:32:36.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065445319999999996 2024-09-18T11:32:36.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123663+0000", 2024-09-18T11:32:36.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8740999999999999e-05 2024-09-18T11:32:36.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:36.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123803+0000", 2024-09-18T11:32:36.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014049200000000001 2024-09-18T11:32:36.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127600+0000", 2024-09-18T11:32:36.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037971179999999999 2024-09-18T11:32:36.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127814+0000", 2024-09-18T11:32:36.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021333600000000001 2024-09-18T11:32:36.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127847+0000", 2024-09-18T11:32:36.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3188999999999997e-05 2024-09-18T11:32:36.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127869+0000", 2024-09-18T11:32:36.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2515e-05 2024-09-18T11:32:36.448 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.448 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.448 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.448 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3325 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:36.448 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:36.449 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.252579220000001, 2024-09-18T11:32:36.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181030999999999, 2024-09-18T11:32:36.449 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.449 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.449 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.449 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.449 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.449 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3325 2024-09-18T11:32:36.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.449 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:36.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:36.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116988+0000", 2024-09-18T11:32:36.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116991+0000", 2024-09-18T11:32:36.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9890000000000001e-06 2024-09-18T11:32:36.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116998+0000", 2024-09-18T11:32:36.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5309999999999998e-06 2024-09-18T11:32:36.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117005+0000", 2024-09-18T11:32:36.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6420000000000001e-06 2024-09-18T11:32:36.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117091+0000", 2024-09-18T11:32:36.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6835999999999993e-05 2024-09-18T11:32:36.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:36.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117097+0000", 2024-09-18T11:32:36.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:32:36.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124717+0000", 2024-09-18T11:32:36.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076204899999999997 2024-09-18T11:32:36.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124773+0000", 2024-09-18T11:32:36.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5470999999999998e-05 2024-09-18T11:32:36.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:36.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124877+0000", 2024-09-18T11:32:36.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010426500000000001 2024-09-18T11:32:36.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127901+0000", 2024-09-18T11:32:36.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030243449999999999 2024-09-18T11:32:36.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128115+0000", 2024-09-18T11:32:36.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021355300000000001 2024-09-18T11:32:36.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128145+0000", 2024-09-18T11:32:36.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0290999999999998e-05 2024-09-18T11:32:36.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128170+0000", 2024-09-18T11:32:36.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4811e-05 2024-09-18T11:32:36.455 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.455 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.455 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.456 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3474 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1872245~637761] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:36.456 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:36.456 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 49.925802543000003, 2024-09-18T11:32:36.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0092038580000000005, 2024-09-18T11:32:36.456 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.456 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.456 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.456 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.456 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.456 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3474 2024-09-18T11:32:36.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.456 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:36.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:36.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443764+0000", 2024-09-18T11:32:36.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:36.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444359+0000", 2024-09-18T11:32:36.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059519 2024-09-18T11:32:36.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444374+0000", 2024-09-18T11:32:36.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4949e-05 2024-09-18T11:32:36.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444382+0000", 2024-09-18T11:32:36.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1369999999999996e-06 2024-09-18T11:32:36.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444428+0000", 2024-09-18T11:32:36.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6783000000000003e-05 2024-09-18T11:32:36.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444507+0000", 2024-09-18T11:32:36.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8415000000000001e-05 2024-09-18T11:32:36.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:36.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444638+0000", 2024-09-18T11:32:36.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013129 2024-09-18T11:32:36.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.449059+0000", 2024-09-18T11:32:36.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004420719 2024-09-18T11:32:36.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452909+0000", 2024-09-18T11:32:36.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003850773 2024-09-18T11:32:36.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452944+0000", 2024-09-18T11:32:36.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-05 2024-09-18T11:32:36.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452969+0000", 2024-09-18T11:32:36.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5795999999999998e-05 2024-09-18T11:32:36.461 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.461 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.462 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.462 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3475 4.5 4:a56ea1fb:::smithi11939506-48:head [setxattr _header (57) in=7b,truncate 3042100] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:36.462 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:32:36.462 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 49.925173827999998, 2024-09-18T11:32:36.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086700660000000006, 2024-09-18T11:32:36.462 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.462 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.462 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.462 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.462 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.462 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3475 2024-09-18T11:32:36.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.463 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:32:36.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:32:36.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444393+0000", 2024-09-18T11:32:36.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:36.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444398+0000", 2024-09-18T11:32:36.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4909999999999999e-06 2024-09-18T11:32:36.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444405+0000", 2024-09-18T11:32:36.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5329999999999997e-06 2024-09-18T11:32:36.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444410+0000", 2024-09-18T11:32:36.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2470000000000002e-06 2024-09-18T11:32:36.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446046+0000", 2024-09-18T11:32:36.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001636722 2024-09-18T11:32:36.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446121+0000", 2024-09-18T11:32:36.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4604000000000006e-05 2024-09-18T11:32:36.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:36.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446274+0000", 2024-09-18T11:32:36.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015350300000000001 2024-09-18T11:32:36.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452884+0000", 2024-09-18T11:32:36.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066096439999999996 2024-09-18T11:32:36.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453000+0000", 2024-09-18T11:32:36.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000116204 2024-09-18T11:32:36.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453032+0000", 2024-09-18T11:32:36.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1327e-05 2024-09-18T11:32:36.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453064+0000", 2024-09-18T11:32:36.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2645000000000003e-05 2024-09-18T11:32:36.467 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.467 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.467 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.468 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3476 4.5 4:a56ea1fb:::smithi11939506-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:36.468 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:36.468 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 49.925134561, 2024-09-18T11:32:36.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087596800000000006, 2024-09-18T11:32:36.468 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.468 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:36.468 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.468 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.468 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.468 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3476 2024-09-18T11:32:36.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.468 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:36.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:36.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:36.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444436+0000", 2024-09-18T11:32:36.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9330000000000001e-06 2024-09-18T11:32:36.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444443+0000", 2024-09-18T11:32:36.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9369999999999998e-06 2024-09-18T11:32:36.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444447+0000", 2024-09-18T11:32:36.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1309999999999999e-06 2024-09-18T11:32:36.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446599+0000", 2024-09-18T11:32:36.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021521190000000001 2024-09-18T11:32:36.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:36.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446675+0000", 2024-09-18T11:32:36.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6099000000000004e-05 2024-09-18T11:32:36.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453088+0000", 2024-09-18T11:32:36.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006413307 2024-09-18T11:32:36.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453132+0000", 2024-09-18T11:32:36.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3548999999999999e-05 2024-09-18T11:32:36.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453193+0000", 2024-09-18T11:32:36.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1116000000000002e-05 2024-09-18T11:32:36.473 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.473 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.473 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.473 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3928 4.5 4:af4ba75a:::smithi11939506-37:head [sparse-read 0~1543958] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:36.473 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:36.473 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 33.350337650999997, 2024-09-18T11:32:36.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080807250000000004, 2024-09-18T11:32:36.473 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.473 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:36.473 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.473 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.474 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.474 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3928 2024-09-18T11:32:36.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.474 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:36.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:36.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:36.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019232+0000", 2024-09-18T11:32:36.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1050000000000002e-06 2024-09-18T11:32:36.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019236+0000", 2024-09-18T11:32:36.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7910000000000002e-06 2024-09-18T11:32:36.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019238+0000", 2024-09-18T11:32:36.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4339999999999999e-06 2024-09-18T11:32:36.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019326+0000", 2024-09-18T11:32:36.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8097999999999996e-05 2024-09-18T11:32:36.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:36.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019330+0000", 2024-09-18T11:32:36.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9349999999999996e-06 2024-09-18T11:32:36.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027035+0000", 2024-09-18T11:32:36.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0077047269999999998 2024-09-18T11:32:36.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027110+0000", 2024-09-18T11:32:36.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4443000000000005e-05 2024-09-18T11:32:36.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027311+0000", 2024-09-18T11:32:36.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000201528 2024-09-18T11:32:36.478 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.478 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.478 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.478 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3929 4.5 4:af4ba75a:::smithi11939506-37:head [sparse-read 0~1543958] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:36.478 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:36.478 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 33.350307098000002, 2024-09-18T11:32:36.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084586699999999997, 2024-09-18T11:32:36.478 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.479 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:36.479 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.479 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.479 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.479 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3929 2024-09-18T11:32:36.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.479 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:36.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:36.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:36.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019262+0000", 2024-09-18T11:32:36.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8029999999999999e-06 2024-09-18T11:32:36.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019267+0000", 2024-09-18T11:32:36.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1119999999999998e-06 2024-09-18T11:32:36.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019269+0000", 2024-09-18T11:32:36.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4269999999999998e-06 2024-09-18T11:32:36.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019344+0000", 2024-09-18T11:32:36.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4979000000000002e-05 2024-09-18T11:32:36.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:36.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019348+0000", 2024-09-18T11:32:36.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3930000000000001e-06 2024-09-18T11:32:36.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027385+0000", 2024-09-18T11:32:36.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080368409999999994 2024-09-18T11:32:36.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027463+0000", 2024-09-18T11:32:36.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7316000000000003e-05 2024-09-18T11:32:36.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027720+0000", 2024-09-18T11:32:36.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000257068 2024-09-18T11:32:36.483 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.483 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.483 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.484 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4251 4.5 4:af4ba75a:::smithi11939506-37:head [delete] snapc 69=[69,67,61,55,51,4e,4c,4b,45,42,3e,3d] ondisk+write+known_if_redirected+full_force+supports_pool_eio e155)", 2024-09-18T11:32:36.484 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:26.039209+0000", 2024-09-18T11:32:36.484 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 10.330358522999999, 2024-09-18T11:32:36.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010258422, 2024-09-18T11:32:36.484 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.484 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.484 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.484 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:36.484 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:36.484 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4251 2024-09-18T11:32:36.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.484 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039209+0000", 2024-09-18T11:32:36.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039209+0000", 2024-09-18T11:32:36.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039208+0000", 2024-09-18T11:32:36.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:36.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039217+0000", 2024-09-18T11:32:36.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2650000000000002e-06 2024-09-18T11:32:36.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039236+0000", 2024-09-18T11:32:36.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8734000000000001e-05 2024-09-18T11:32:36.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039244+0000", 2024-09-18T11:32:36.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2900000000000002e-06 2024-09-18T11:32:36.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039289+0000", 2024-09-18T11:32:36.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4657999999999998e-05 2024-09-18T11:32:36.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:36.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039294+0000", 2024-09-18T11:32:36.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0719999999999999e-06 2024-09-18T11:32:36.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.042052+0000", 2024-09-18T11:32:36.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027585399999999999 2024-09-18T11:32:36.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.042162+0000", 2024-09-18T11:32:36.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011005699999999999 2024-09-18T11:32:36.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:36.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.042341+0000", 2024-09-18T11:32:36.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000178285 2024-09-18T11:32:36.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.044140+0000", 2024-09-18T11:32:36.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001799092 2024-09-18T11:32:36.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.049396+0000", 2024-09-18T11:32:36.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005256213 2024-09-18T11:32:36.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.049431+0000", 2024-09-18T11:32:36.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.536e-05 2024-09-18T11:32:36.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.049468+0000", 2024-09-18T11:32:36.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6548999999999999e-05 2024-09-18T11:32:36.490 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.490 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.490 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.490 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.490 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.490 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:36.491 INFO:tasks.rados.rados.0.smithi119.stdout:1905: finishing write tid 1 to smithi11939506-1 2024-09-18T11:32:36.491 INFO:tasks.rados.rados.0.smithi119.stdout:1905: finishing write tid 2 to smithi11939506-1 2024-09-18T11:32:36.492 INFO:tasks.rados.rados.0.smithi119.stdout:1905: finishing write tid 3 to smithi11939506-1 2024-09-18T11:32:36.492 INFO:tasks.rados.rados.0.smithi119.stdout:1905: finishing write tid 4 to smithi11939506-1 2024-09-18T11:32:36.492 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 1 v 379 (ObjNum 387 snap 62 seq_num 387) dirty exists 2024-09-18T11:32:36.492 INFO:tasks.rados.rados.0.smithi119.stdout:1905: left oid 1 (ObjNum 387 snap 62 seq_num 387) 2024-09-18T11:32:36.492 INFO:tasks.rados.rados.0.smithi119.stdout:1906: finishing set_chunk to oid 3 2024-09-18T11:32:36.492 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 3 set_chunk returned error code -95 offset: 373248 length: 957440 2024-09-18T11:32:36.492 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 0 (ObjNum 243 snap 28 seq_num 243) dirty exists 2024-09-18T11:32:36.492 INFO:tasks.rados.rados.0.smithi119.stdout:1908: expect (ObjNum 363 snap 55 seq_num 363) 2024-09-18T11:32:36.493 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:32:36.493 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:36.491+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:36.498 INFO:tasks.rados.rados.0.smithi119.stdout:1909: expect (ObjNum 328 snap 45 seq_num 328) 2024-09-18T11:32:36.592 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:36.591+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:36.615 INFO:tasks.rados.rados.0.smithi119.stdout:1902: done (7 left) 2024-09-18T11:32:36.615 INFO:tasks.rados.rados.0.smithi119.stdout:1903: done (6 left) 2024-09-18T11:32:36.615 INFO:tasks.rados.rados.0.smithi119.stdout:1904: done (5 left) 2024-09-18T11:32:36.616 INFO:tasks.rados.rados.0.smithi119.stdout:1905: done (4 left) 2024-09-18T11:32:36.616 INFO:tasks.rados.rados.0.smithi119.stdout:1906: done (3 left) 2024-09-18T11:32:36.616 INFO:tasks.rados.rados.0.smithi119.stdout:1908: done (2 left) 2024-09-18T11:32:36.616 INFO:tasks.rados.rados.0.smithi119.stdout:1909: done (1 left) 2024-09-18T11:32:36.616 INFO:tasks.rados.rados.0.smithi119.stdout:1910: done (0 left) 2024-09-18T11:32:36.616 INFO:tasks.rados.rados.0.smithi119.stdout:1911: tier_promote oid 3 2024-09-18T11:32:36.616 INFO:tasks.rados.rados.0.smithi119.stdout:1912: write (excl) oid 38 current snap is 62 2024-09-18T11:32:36.616 INFO:tasks.rados.rados.0.smithi119.stdout:1912: seq_num 389 ranges {537923=405864,1513613=593796} 2024-09-18T11:32:36.620 INFO:tasks.rados.rados.0.smithi119.stdout:1912: writing smithi11939506-38 from 537923 to 943787 tid 1 2024-09-18T11:32:36.627 INFO:tasks.rados.rados.0.smithi119.stdout:1912: writing smithi11939506-38 from 1513613 to 2107409 tid 2 2024-09-18T11:32:36.628 INFO:tasks.rados.rados.0.smithi119.stdout:1913: read oid 23 snap -1 2024-09-18T11:32:36.628 INFO:tasks.rados.rados.0.smithi119.stdout:1913: expect (ObjNum 363 snap 55 seq_num 363) 2024-09-18T11:32:36.628 INFO:tasks.rados.rados.0.smithi119.stdout:1914: read oid 35 snap -1 2024-09-18T11:32:36.628 INFO:tasks.rados.rados.0.smithi119.stdout:1914: expect (ObjNum 155 snap 20 seq_num 155) 2024-09-18T11:32:36.628 INFO:tasks.rados.rados.0.smithi119.stdout:1915: delete oid 25 current snap is 62 2024-09-18T11:32:36.628 INFO:tasks.rados.rados.0.smithi119.stdout:1911: got (0) Success 2024-09-18T11:32:36.628 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 152 (ObjNum 243 snap 28 seq_num 243) dirty exists 2024-09-18T11:32:36.628 INFO:tasks.rados.rados.0.smithi119.stdout:1912: finishing write tid 1 to smithi11939506-38 2024-09-18T11:32:36.632 INFO:tasks.rados.rados.0.smithi119.stdout:1912: finishing write tid 2 to smithi11939506-38 2024-09-18T11:32:36.633 INFO:tasks.rados.rados.0.smithi119.stdout:1912: finishing write tid 3 to smithi11939506-38 2024-09-18T11:32:36.633 INFO:tasks.rados.rados.0.smithi119.stdout:1912: finishing write tid 4 to smithi11939506-38 2024-09-18T11:32:36.634 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 38 v 290 (ObjNum 389 snap 62 seq_num 389) dirty exists 2024-09-18T11:32:36.634 INFO:tasks.rados.rados.0.smithi119.stdout:1912: left oid 38 (ObjNum 389 snap 62 seq_num 389) 2024-09-18T11:32:36.636 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:36.636 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:36.636 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:36.637 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:36.637 INFO:tasks.rados.rados.0.smithi119.stdout:1911: done (4 left) 2024-09-18T11:32:36.637 INFO:tasks.rados.rados.0.smithi119.stdout:1912: done (3 left) 2024-09-18T11:32:36.637 INFO:tasks.rados.rados.0.smithi119.stdout:1915: done (2 left) 2024-09-18T11:32:36.637 INFO:tasks.rados.rados.0.smithi119.stdout:1916: snap_create 2024-09-18T11:32:36.638 INFO:tasks.rados.rados.0.smithi119.stdout:1913: expect (ObjNum 363 snap 55 seq_num 363) 2024-09-18T11:32:36.644 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:32:36.693 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:36.692+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:36.718 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:32:36.756 INFO:tasks.rados.rados.0.smithi119.stdout:1914: expect (ObjNum 155 snap 20 seq_num 155) 2024-09-18T11:32:36.782 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:36.782 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:36.782 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:36.783 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:36.783 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:36.791 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:32:36.793 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:36.792+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:36.833 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:36.833 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:36.848 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:36.849 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:36.863 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:36.863 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:36.877 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:36.878 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:36.890 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:36.890 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:36.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:36.890 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:36.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.890 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:36.891 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:36.891 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 167.730232414, 2024-09-18T11:32:36.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:32:36.891 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.891 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.891 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.891 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:36.891 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:36.891 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:32:36.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.891 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:36.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:36.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:32:36.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:32:36.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:32:36.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:32:36.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:32:36.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:32:36.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:32:36.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:32:36.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:32:36.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:32:36.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:32:36.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:36.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:32:36.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:32:36.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:32:36.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:32:36.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:32:36.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:32:36.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:32:36.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:32:36.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:32:36.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:32:36.896 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.896 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.896 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.897 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:36.897 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:36.897 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 167.730177484, 2024-09-18T11:32:36.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:32:36.897 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.897 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.897 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.897 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:36.897 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:36.897 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:32:36.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.898 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:36.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:36.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:36.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:32:36.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:32:36.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:32:36.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:32:36.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:32:36.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:32:36.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:32:36.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:32:36.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:32:36.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:32:36.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:36.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:32:36.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:32:36.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:32:36.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:32:36.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:32:36.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:32:36.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:32:36.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:32:36.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:32:36.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:32:36.902 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.902 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.902 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.903 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:36.903 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:36.903 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 167.73016750400001, 2024-09-18T11:32:36.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:32:36.903 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.903 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.903 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.903 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:36.903 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:36.903 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:32:36.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.903 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:36.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:36.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:36.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:36.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:32:36.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:32:36.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:32:36.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:32:36.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:32:36.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:32:36.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:32:36.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:32:36.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:32:36.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:32:36.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:36.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:32:36.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:32:36.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:32:36.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:32:36.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:32:36.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:32:36.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:32:36.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:32:36.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:32:36.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:32:36.908 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.908 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.908 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.909 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:36.909 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:36.909 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 167.730155905, 2024-09-18T11:32:36.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:32:36.909 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.909 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.909 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.909 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:36.909 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:36.909 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:32:36.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.910 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:36.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:36.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:36.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:36.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:32:36.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:32:36.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:32:36.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:32:36.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:32:36.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:32:36.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:32:36.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:32:36.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:32:36.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:32:36.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:36.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:32:36.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:32:36.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:32:36.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:32:36.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:32:36.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:32:36.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:32:36.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:32:36.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:32:36.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:32:36.916 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.916 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.916 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.916 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:36.916 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:36.916 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 167.730099424, 2024-09-18T11:32:36.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:32:36.917 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.917 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.917 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.917 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:36.917 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:36.917 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:32:36.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.917 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:36.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:36.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:32:36.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:32:36.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:32:36.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:32:36.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:32:36.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:32:36.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:32:36.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:32:36.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:32:36.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:32:36.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:32:36.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:36.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:32:36.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:32:36.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:32:36.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:32:36.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:32:36.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:32:36.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:32:36.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:32:36.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:32:36.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:32:36.922 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.922 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.922 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.923 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:36.923 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:36.923 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 167.7300856, 2024-09-18T11:32:36.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:32:36.923 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.923 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.923 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.923 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:36.923 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:36.923 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:32:36.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.923 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:36.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:36.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:36.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:36.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:32:36.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:32:36.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:32:36.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:32:36.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:32:36.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:32:36.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:32:36.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:32:36.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:32:36.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:32:36.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:36.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:32:36.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:32:36.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:32:36.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:32:36.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:32:36.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:32:36.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:32:36.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:32:36.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:32:36.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:32:36.928 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.929 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.929 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.929 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:36.929 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:36.929 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 167.31390669300001, 2024-09-18T11:32:36.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:32:36.929 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.929 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.929 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.929 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:36.930 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:36.930 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:32:36.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.930 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:36.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:36.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:36.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:36.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:32:36.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:32:36.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:32:36.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:32:36.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:32:36.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:32:36.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:32:36.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:32:36.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:32:36.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:32:36.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:36.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:32:36.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:32:36.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:32:36.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:32:36.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:32:36.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:32:36.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:32:36.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:32:36.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:32:36.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:32:36.935 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.935 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.935 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.935 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:36.935 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:36.935 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 167.313883393, 2024-09-18T11:32:36.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:32:36.935 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.935 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.935 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.936 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:36.936 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:36.936 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:32:36.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.936 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:36.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:36.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:32:36.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:32:36.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:32:36.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:32:36.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:32:36.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:32:36.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:32:36.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:32:36.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:32:36.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:32:36.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:32:36.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:36.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:32:36.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:32:36.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:32:36.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:32:36.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:32:36.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:32:36.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:32:36.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:32:36.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:32:36.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:32:36.941 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.941 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.941 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.941 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:36.941 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:36.941 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 167.313797784, 2024-09-18T11:32:36.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:32:36.941 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.941 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.941 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.942 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:36.942 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:36.942 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:32:36.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.942 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:36.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:36.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:36.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:32:36.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:32:36.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:32:36.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:32:36.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:32:36.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:32:36.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:32:36.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:32:36.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:32:36.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:32:36.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:36.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:32:36.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:32:36.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:32:36.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:32:36.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:32:36.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:32:36.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:32:36.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:32:36.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:32:36.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:32:36.947 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.947 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.947 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.947 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:36.947 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:36.947 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 167.313786819, 2024-09-18T11:32:36.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:32:36.947 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.947 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.947 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.947 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:36.948 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:36.948 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:32:36.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.948 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:36.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:36.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:36.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:36.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:32:36.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:32:36.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:32:36.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:32:36.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:32:36.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:32:36.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:32:36.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:32:36.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:32:36.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:32:36.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:36.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:32:36.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:32:36.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:32:36.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:32:36.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:32:36.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:32:36.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:32:36.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:32:36.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:32:36.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:32:36.953 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.953 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.953 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.953 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:36.953 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:36.953 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 167.31377565099999, 2024-09-18T11:32:36.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:32:36.954 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.954 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.954 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.954 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:36.954 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:36.954 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:32:36.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.954 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:36.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:36.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:36.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:36.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:32:36.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:32:36.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:32:36.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:32:36.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:32:36.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:36.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:32:36.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:32:36.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:32:36.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:32:36.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:32:36.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:32:36.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.959 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 167.31371032199999, 2024-09-18T11:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:36.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:36.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:32:36.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:32:36.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:32:36.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:32:36.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:32:36.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:32:36.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:32:36.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:32:36.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:32:36.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:32:36.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:32:36.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:36.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:32:36.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:32:36.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:32:36.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:32:36.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:32:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:32:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:32:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:32:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 167.30354834400001, 2024-09-18T11:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.966 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:36.967 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:36.967 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:32:36.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.967 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:36.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:36.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:32:36.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:32:36.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:32:36.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:32:36.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:32:36.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:32:36.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:32:36.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:32:36.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:32:36.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:32:36.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:32:36.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:36.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:32:36.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:32:36.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:32:36.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:32:36.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:32:36.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:32:36.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:32:36.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:32:36.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:32:36.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:32:36.972 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.972 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.972 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.972 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:36.972 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:36.972 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 167.30345586499999, 2024-09-18T11:32:36.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:32:36.972 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.973 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.973 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:36.973 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:36.973 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:36.973 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:32:36.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.973 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:36.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:36.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:36.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:32:36.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:32:36.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:32:36.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:32:36.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:32:36.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:32:36.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:32:36.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:32:36.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:32:36.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:32:36.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:36.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:32:36.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:32:36.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:36.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:32:36.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:32:36.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:36.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:32:36.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:32:36.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:32:36.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:32:36.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:32:36.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:32:36.978 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.978 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.978 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.978 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3209 4.0 e109/22 4:019b6e09:::smithi11939506-40:head v 109'149, mlcod=106'146)", 2024-09-18T11:32:36.978 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:32:36.978 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 59.788615014000001, 2024-09-18T11:32:36.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063064039999999998, 2024-09-18T11:32:36.979 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.979 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.979 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:32:36.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099386+0000", 2024-09-18T11:32:36.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099385+0000", 2024-09-18T11:32:36.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099951+0000", 2024-09-18T11:32:36.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056606600000000005 2024-09-18T11:32:36.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099964+0000", 2024-09-18T11:32:36.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2918999999999999e-05 2024-09-18T11:32:36.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.099972+0000", 2024-09-18T11:32:36.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2300000000000002e-06 2024-09-18T11:32:36.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.100026+0000", 2024-09-18T11:32:36.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4456999999999997e-05 2024-09-18T11:32:36.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.100046+0000", 2024-09-18T11:32:36.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0417000000000001e-05 2024-09-18T11:32:36.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.105653+0000", 2024-09-18T11:32:36.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005606214 2024-09-18T11:32:36.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:37.105692+0000", 2024-09-18T11:32:36.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9671e-05 2024-09-18T11:32:36.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.982 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.983 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:123 4.3 e132/67 4:cc46532a:::smithi11939506-33:30 v 132'171, mlcod=128'168)", 2024-09-18T11:32:36.983 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:36.983 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 33.860149239999998, 2024-09-18T11:32:36.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087627039999999996, 2024-09-18T11:32:36.983 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.983 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.983 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:36.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:36.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027851+0000", 2024-09-18T11:32:36.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027871+0000", 2024-09-18T11:32:36.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9749e-05 2024-09-18T11:32:36.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027882+0000", 2024-09-18T11:32:36.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1242e-05 2024-09-18T11:32:36.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027889+0000", 2024-09-18T11:32:36.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7649999999999996e-06 2024-09-18T11:32:36.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027929+0000", 2024-09-18T11:32:36.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9984999999999997e-05 2024-09-18T11:32:36.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027951+0000", 2024-09-18T11:32:36.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2047000000000001e-05 2024-09-18T11:32:36.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.036565+0000", 2024-09-18T11:32:36.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086135110000000008 2024-09-18T11:32:36.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.036614+0000", 2024-09-18T11:32:36.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9747000000000003e-05 2024-09-18T11:32:36.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.987 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.987 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:124 4.3 e132/67 4:ca2b2309:::smithi11939506-44:30 v 132'173, mlcod=128'168)", 2024-09-18T11:32:36.987 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:36.987 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 33.859457257999999, 2024-09-18T11:32:36.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0088947309999999995, 2024-09-18T11:32:36.988 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.988 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.988 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:36.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:36.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028543+0000", 2024-09-18T11:32:36.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:36.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028551+0000", 2024-09-18T11:32:36.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8720000000000002e-06 2024-09-18T11:32:36.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028562+0000", 2024-09-18T11:32:36.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1039000000000001e-05 2024-09-18T11:32:36.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028569+0000", 2024-09-18T11:32:36.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0350000000000004e-06 2024-09-18T11:32:36.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028848+0000", 2024-09-18T11:32:36.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00027935600000000003 2024-09-18T11:32:36.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028868+0000", 2024-09-18T11:32:36.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9947999999999999e-05 2024-09-18T11:32:36.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037405+0000", 2024-09-18T11:32:36.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085369920000000002 2024-09-18T11:32:36.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037438+0000", 2024-09-18T11:32:36.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3244000000000002e-05 2024-09-18T11:32:36.991 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.992 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.992 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.992 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:142 4.3 e141/67 4:ca2b2309:::smithi11939506-44:5d v 141'197, mlcod=140'192)", 2024-09-18T11:32:36.992 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.297859+0000", 2024-09-18T11:32:36.992 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 24.590142247999999, 2024-09-18T11:32:36.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063063160000000002, 2024-09-18T11:32:36.992 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.992 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.992 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297859+0000", 2024-09-18T11:32:36.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297859+0000", 2024-09-18T11:32:36.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297858+0000", 2024-09-18T11:32:36.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297863+0000", 2024-09-18T11:32:36.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9450000000000001e-06 2024-09-18T11:32:36.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297870+0000", 2024-09-18T11:32:36.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.43e-06 2024-09-18T11:32:36.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297875+0000", 2024-09-18T11:32:36.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8439999999999997e-06 2024-09-18T11:32:36.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297970+0000", 2024-09-18T11:32:36.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4968000000000003e-05 2024-09-18T11:32:36.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:36.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297991+0000", 2024-09-18T11:32:36.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0726000000000001e-05 2024-09-18T11:32:36.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:36.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.304127+0000", 2024-09-18T11:32:36.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061356700000000002 2024-09-18T11:32:36.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:36.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.304165+0000", 2024-09-18T11:32:36.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8305e-05 2024-09-18T11:32:36.996 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.996 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:36.996 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:36.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.996 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:154 4.3 e147/67 4:ca2b2309:::smithi11939506-44:5d v 147'216, mlcod=145'210)", 2024-09-18T11:32:36.997 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:36.997 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.550670809, 2024-09-18T11:32:36.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066667829999999999, 2024-09-18T11:32:36.997 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:36.997 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:36.997 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:36.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:36.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:36.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:36.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:36.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:36.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:36.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:36.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:36.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:36.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337335+0000", 2024-09-18T11:32:36.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1479999999999997e-06 2024-09-18T11:32:36.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:36.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337344+0000", 2024-09-18T11:32:36.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5640000000000002e-06 2024-09-18T11:32:36.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:36.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337349+0000", 2024-09-18T11:32:36.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9470000000000002e-06 2024-09-18T11:32:36.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:36.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:36.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:36.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337410+0000", 2024-09-18T11:32:37.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0448999999999997e-05 2024-09-18T11:32:37.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:37.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:37.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:37.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337425+0000", 2024-09-18T11:32:37.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.504e-05 2024-09-18T11:32:37.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:37.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:37.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:37.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.343972+0000", 2024-09-18T11:32:37.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065474169999999998 2024-09-18T11:32:37.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:37.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:37.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:37.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.343997+0000", 2024-09-18T11:32:37.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4672999999999999e-05 2024-09-18T11:32:37.001 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:37.001 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:37.001 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:37.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:37.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:37.001 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:4351 4.6 e160/143 4:6f86ffcc:::smithi11939506-15:head v 160'174, mlcod=157'170)", 2024-09-18T11:32:37.001 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.391535+0000", 2024-09-18T11:32:37.001 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.496465615, 2024-09-18T11:32:37.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059655610000000003, 2024-09-18T11:32:37.001 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:37.002 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:37.002 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:37.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:37.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:37.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.391535+0000", 2024-09-18T11:32:37.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:37.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:37.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:37.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:37.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.391535+0000", 2024-09-18T11:32:37.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:37.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:37.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:37.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:37.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.391535+0000", 2024-09-18T11:32:37.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:37.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:37.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:37.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:37.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392591+0000", 2024-09-18T11:32:37.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001056441 2024-09-18T11:32:37.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:37.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:37.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:37.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392605+0000", 2024-09-18T11:32:37.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3668e-05 2024-09-18T11:32:37.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:37.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:37.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:37.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392612+0000", 2024-09-18T11:32:37.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.362e-06 2024-09-18T11:32:37.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:37.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:37.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:37.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392674+0000", 2024-09-18T11:32:37.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1525000000000005e-05 2024-09-18T11:32:37.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:37.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:37.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:37.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392702+0000", 2024-09-18T11:32:37.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8181999999999999e-05 2024-09-18T11:32:37.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:37.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:37.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:37.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.397455+0000", 2024-09-18T11:32:37.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047529210000000002 2024-09-18T11:32:37.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:37.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:37.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:37.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.397501+0000", 2024-09-18T11:32:37.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5812000000000001e-05 2024-09-18T11:32:37.005 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:37.005 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:37.005 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:37.006 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:37.006 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:37.006 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:37.006 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:37.007 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:37.007 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:37.008 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:37.008 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:32:37.009 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:37.009 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:37.009 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:37.010 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:37.010 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:37.006+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:37.108 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:37.107+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:37.147 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:37.147 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:37.156 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:32:37.240 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:37.239+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:37.293 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:37.293 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:37.302 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:32:37.309 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:37.308+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:37.401 INFO:tasks.rados.rados.0.smithi119.stdout:1913: done (2 left) 2024-09-18T11:32:37.401 INFO:tasks.rados.rados.0.smithi119.stdout:1914: done (1 left) 2024-09-18T11:32:37.401 INFO:tasks.rados.rados.0.smithi119.stdout:1916: done (0 left) 2024-09-18T11:32:37.401 INFO:tasks.rados.rados.0.smithi119.stdout:1917: read oid 24 snap -1 2024-09-18T11:32:37.401 INFO:tasks.rados.rados.0.smithi119.stdout:1917: expect (ObjNum 377 snap 58 seq_num 377) 2024-09-18T11:32:37.401 INFO:tasks.rados.rados.0.smithi119.stdout:1918: write oid 30 current snap is 63 2024-09-18T11:32:37.401 INFO:tasks.rados.rados.0.smithi119.stdout:1918: seq_num 391 ranges {571537=556362,1630577=597465} 2024-09-18T11:32:37.407 INFO:tasks.rados.rados.0.smithi119.stdout:1918: writing smithi11939506-30 from 571537 to 1127899 tid 1 2024-09-18T11:32:37.410 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:37.409+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:37.413 INFO:tasks.rados.rados.0.smithi119.stdout:1918: writing smithi11939506-30 from 1630577 to 2228042 tid 2 2024-09-18T11:32:37.414 INFO:tasks.rados.rados.0.smithi119.stdout:1919: read oid 1 snap -1 2024-09-18T11:32:37.414 INFO:tasks.rados.rados.0.smithi119.stdout:1919: expect (ObjNum 387 snap 62 seq_num 387) 2024-09-18T11:32:37.414 INFO:tasks.rados.rados.0.smithi119.stdout:1920: read oid 9 snap -1 2024-09-18T11:32:37.414 INFO:tasks.rados.rados.0.smithi119.stdout:1920: expect (ObjNum 350 snap 54 seq_num 350) 2024-09-18T11:32:37.414 INFO:tasks.rados.rados.0.smithi119.stdout:1921: write oid 14 current snap is 63 2024-09-18T11:32:37.414 INFO:tasks.rados.rados.0.smithi119.stdout:1921: seq_num 392 ranges {588344=631611,1689059=741123} 2024-09-18T11:32:37.420 INFO:tasks.rados.rados.0.smithi119.stdout:1921: writing smithi11939506-14 from 588344 to 1219955 tid 1 2024-09-18T11:32:37.427 INFO:tasks.rados.rados.0.smithi119.stdout:1921: writing smithi11939506-14 from 1689059 to 2430182 tid 2 2024-09-18T11:32:37.427 INFO:tasks.rados.rados.0.smithi119.stdout:1922: set_chunk oid 21 target oid 2024-09-18T11:32:37.431 INFO:tasks.rados.rados.0.smithi119.stdout:1922: set_chunk oid 21 offset: 1888256 length: 91648 target oid 3cf5c9477be9c907a94727753375f7ba1577c143c25173ff8c7ce2cf8532f10d offset: 0 snap -1 2024-09-18T11:32:37.431 INFO:tasks.rados.rados.0.smithi119.stdout:1923: snap_create 2024-09-18T11:32:37.432 INFO:tasks.rados.rados.0.smithi119.stdout:1917: expect (ObjNum 377 snap 58 seq_num 377) 2024-09-18T11:32:37.448 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:37.448 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:37.457 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:32:37.511 INFO:tasks.rados.rados.0.smithi119.stdout:1918: finishing write tid 1 to smithi11939506-30 2024-09-18T11:32:37.511 INFO:tasks.rados.rados.0.smithi119.stdout:1918: finishing write tid 2 to smithi11939506-30 2024-09-18T11:32:37.511 INFO:tasks.rados.rados.0.smithi119.stdout:1918: finishing write tid 3 to smithi11939506-30 2024-09-18T11:32:37.511 INFO:tasks.rados.rados.0.smithi119.stdout:1918: finishing write tid 4 to smithi11939506-30 2024-09-18T11:32:37.511 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 30 v 187 (ObjNum 391 snap 63 seq_num 391) dirty exists 2024-09-18T11:32:37.511 INFO:tasks.rados.rados.0.smithi119.stdout:1918: left oid 30 (ObjNum 391 snap 63 seq_num 391) 2024-09-18T11:32:37.511 INFO:tasks.rados.rados.0.smithi119.stdout:1921: finishing write tid 1 to smithi11939506-14 2024-09-18T11:32:37.511 INFO:tasks.rados.rados.0.smithi119.stdout:1919: expect (ObjNum 387 snap 62 seq_num 387) 2024-09-18T11:32:37.512 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:37.510+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:37.555 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:37.555 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:37.564 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:32:37.612 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:37.611+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:37.662 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:37.662 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:37.666 INFO:tasks.rados.rados.0.smithi119.stdout:1921: finishing write tid 2 to smithi11939506-14 2024-09-18T11:32:37.667 INFO:tasks.rados.rados.0.smithi119.stdout:1921: finishing write tid 3 to smithi11939506-14 2024-09-18T11:32:37.667 INFO:tasks.rados.rados.0.smithi119.stdout:1921: finishing write tid 4 to smithi11939506-14 2024-09-18T11:32:37.667 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 364 (ObjNum 392 snap 63 seq_num 392) dirty exists 2024-09-18T11:32:37.667 INFO:tasks.rados.rados.0.smithi119.stdout:1921: left oid 14 (ObjNum 392 snap 63 seq_num 392) 2024-09-18T11:32:37.667 INFO:tasks.rados.rados.0.smithi119.stdout:1922: finishing set_chunk to oid 21 2024-09-18T11:32:37.667 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 381 (ObjNum 379 snap 59 seq_num 379) dirty exists 2024-09-18T11:32:37.667 INFO:tasks.rados.rados.0.smithi119.stdout:1920: expect (ObjNum 350 snap 54 seq_num 350) 2024-09-18T11:32:37.671 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:32:37.712 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:37.711+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:37.809 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:37.809 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:37.813 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:37.811+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:37.818 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:32:37.913 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:37.912+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:37.956 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:37.957 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:37.965 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:32:38.014 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:38.014+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:38.025 INFO:tasks.rados.rados.0.smithi119.stdout:1917: done (6 left) 2024-09-18T11:32:38.025 INFO:tasks.rados.rados.0.smithi119.stdout:1918: done (5 left) 2024-09-18T11:32:38.025 INFO:tasks.rados.rados.0.smithi119.stdout:1919: done (4 left) 2024-09-18T11:32:38.025 INFO:tasks.rados.rados.0.smithi119.stdout:1920: done (3 left) 2024-09-18T11:32:38.025 INFO:tasks.rados.rados.0.smithi119.stdout:1921: done (2 left) 2024-09-18T11:32:38.025 INFO:tasks.rados.rados.0.smithi119.stdout:1922: done (1 left) 2024-09-18T11:32:38.025 INFO:tasks.rados.rados.0.smithi119.stdout:1923: done (0 left) 2024-09-18T11:32:38.025 INFO:tasks.rados.rados.0.smithi119.stdout:1924: write (excl) oid 9 current snap is 64 2024-09-18T11:32:38.026 INFO:tasks.rados.rados.0.smithi119.stdout:1924: seq_num 393 ranges {605151=706860,2031188=573963} 2024-09-18T11:32:38.032 INFO:tasks.rados.rados.0.smithi119.stdout:1924: writing smithi11939506-9 from 605151 to 1312011 tid 1 2024-09-18T11:32:38.034 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:38.033+0000 7f6bf5056640 -1 osd.5 166 handle_osd_map injecting map crc failure 2024-09-18T11:32:38.040 INFO:tasks.rados.rados.0.smithi119.stdout:1924: writing smithi11939506-9 from 2031188 to 2605151 tid 2 2024-09-18T11:32:38.040 INFO:tasks.rados.rados.0.smithi119.stdout:1925: read oid 10 snap -1 2024-09-18T11:32:38.040 INFO:tasks.rados.rados.0.smithi119.stdout:1925: expect (ObjNum 358 snap 55 seq_num 358) 2024-09-18T11:32:38.040 INFO:tasks.rados.rados.0.smithi119.stdout:1926: write (excl) oid 47 current snap is 64 2024-09-18T11:32:38.040 INFO:tasks.rados.rados.0.smithi119.stdout:1926: seq_num 394 ranges {621958=782109,2089670=532288} 2024-09-18T11:32:38.049 INFO:tasks.rados.rados.0.smithi119.stdout:1926: writing smithi11939506-47 from 621958 to 1404067 tid 1 2024-09-18T11:32:38.050 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T11:32:38.056 INFO:tasks.rados.rados.0.smithi119.stdout:1926: writing smithi11939506-47 from 2089670 to 2621958 tid 2 2024-09-18T11:32:38.056 INFO:tasks.rados.rados.0.smithi119.stdout:1927: read oid 17 snap 60 2024-09-18T11:32:38.056 INFO:tasks.rados.rados.0.smithi119.stdout:1927: expect (ObjNum 383 snap 60 seq_num 383) 2024-09-18T11:32:38.056 INFO:tasks.rados.rados.0.smithi119.stdout:1928: tier_flush oid 34 2024-09-18T11:32:38.057 INFO:tasks.rados.rados.0.smithi119.stdout:1928: tier_flush oid 34 snap -1 2024-09-18T11:32:38.057 INFO:tasks.rados.rados.0.smithi119.stdout:1925: expect (ObjNum 358 snap 55 seq_num 358) 2024-09-18T11:32:38.114 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:38.114 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:38.115 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:38.114+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:38.123 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:32:38.175 INFO:tasks.rados.rados.0.smithi119.stdout:1924: finishing write tid 1 to smithi11939506-9 2024-09-18T11:32:38.175 INFO:tasks.rados.rados.0.smithi119.stdout:1924: finishing write tid 2 to smithi11939506-9 2024-09-18T11:32:38.175 INFO:tasks.rados.rados.0.smithi119.stdout:1925: done (4 left) 2024-09-18T11:32:38.175 INFO:tasks.rados.rados.0.smithi119.stdout:1929: read oid 13 snap 55 2024-09-18T11:32:38.175 INFO:tasks.rados.rados.0.smithi119.stdout:1929: expect (ObjNum 309 snap 43 seq_num 309) 2024-09-18T11:32:38.175 INFO:tasks.rados.rados.0.smithi119.stdout:1930: read oid 49 snap 61 2024-09-18T11:32:38.175 INFO:tasks.rados.rados.0.smithi119.stdout:1930: expect (ObjNum 323 snap 45 seq_num 323) 2024-09-18T11:32:38.175 INFO:tasks.rados.rados.0.smithi119.stdout:1931: read oid 40 snap -1 2024-09-18T11:32:38.176 INFO:tasks.rados.rados.0.smithi119.stdout:1931: expect (ObjNum 287 snap 41 seq_num 287) 2024-09-18T11:32:38.176 INFO:tasks.rados.rados.0.smithi119.stdout:1932: tier_promote oid 45 2024-09-18T11:32:38.176 INFO:tasks.rados.rados.0.smithi119.stdout:1924: finishing write tid 3 to smithi11939506-9 2024-09-18T11:32:38.176 INFO:tasks.rados.rados.0.smithi119.stdout:1924: finishing write tid 4 to smithi11939506-9 2024-09-18T11:32:38.176 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 575 (ObjNum 393 snap 64 seq_num 393) dirty exists 2024-09-18T11:32:38.176 INFO:tasks.rados.rados.0.smithi119.stdout:1924: left oid 9 (ObjNum 393 snap 64 seq_num 393) 2024-09-18T11:32:38.176 INFO:tasks.rados.rados.0.smithi119.stdout:1926: finishing write tid 1 to smithi11939506-47 2024-09-18T11:32:38.176 INFO:tasks.rados.rados.0.smithi119.stdout:1924: done (7 left) 2024-09-18T11:32:38.176 INFO:tasks.rados.rados.0.smithi119.stdout:1933: snap_remove snap 50 2024-09-18T11:32:38.215 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:38.214+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:38.267 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:38.267 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:38.276 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:32:38.317 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:38.316+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:38.417 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:38.417 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:38.418 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:38.416+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:38.426 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:32:38.519 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:38.518+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:38.563 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:38.564 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:38.572 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:32:38.619 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:38.618+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:38.670 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:38.670 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:38.678 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:32:38.720 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:38.719+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:38.790 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:38.790 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:38.799 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:32:38.842 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:38.841+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:38.897 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:38.897 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:38.906 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:32:38.947 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:38.946+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:39.011 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:32:39.035 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is unset 2024-09-18T11:32:39.036 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:39.035+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:39.037 INFO:tasks.rados.rados.0.smithi119.stdout:1933: done (7 left) 2024-09-18T11:32:39.037 INFO:tasks.rados.rados.0.smithi119.stdout:1934: read oid 6 snap -1 2024-09-18T11:32:39.037 INFO:tasks.rados.rados.0.smithi119.stdout:1934: expect (ObjNum 327 snap 45 seq_num 327) 2024-09-18T11:32:39.037 INFO:tasks.rados.rados.0.smithi119.stdout:1935: snap_remove snap 47 2024-09-18T11:32:39.048 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:39.048 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:39.060 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:32:39.099 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-18T11:32:39.099 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T11:32:39.099 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-18T11:32:39.099 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T11:32:39.135 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:39.135 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:39.135 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:39.134+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:39.152 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:39.152 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:39.159 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:39.160 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:39.167 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:39.167 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:39.168 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:32:39.182 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:39.182 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:39.197 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:39.198 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:39.215 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:39.215 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:39.224 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:39.223+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:39.231 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:39.231 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:39.246 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:39.246 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:39.267 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:39.268 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:39.268 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:39.268 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:39.268 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:39.276 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:32:39.360 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:39.359+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:39.370 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:32:39.370 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:39.370 INFO:teuthology.orchestra.run.smithi119.stdout: "epoch": 168, 2024-09-18T11:32:39.371 INFO:teuthology.orchestra.run.smithi119.stdout: "fsid": "f4cca259-b87a-45cc-8995-7e9fca891a9f", 2024-09-18T11:32:39.371 INFO:teuthology.orchestra.run.smithi119.stdout: "created": "2024-09-18T11:29:38.205487+0000", 2024-09-18T11:32:39.371 INFO:teuthology.orchestra.run.smithi119.stdout: "modified": "2024-09-18T11:32:39.030915+0000", 2024-09-18T11:32:39.371 INFO:teuthology.orchestra.run.smithi119.stdout: "last_up_change": "2024-09-18T11:31:49.142260+0000", 2024-09-18T11:32:39.371 INFO:teuthology.orchestra.run.smithi119.stdout: "last_in_change": "2024-09-18T11:30:49.312763+0000", 2024-09-18T11:32:39.371 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T11:32:39.371 INFO:teuthology.orchestra.run.smithi119.stdout: "flags_num": 5804032, 2024-09-18T11:32:39.371 INFO:teuthology.orchestra.run.smithi119.stdout: "flags_set": [ 2024-09-18T11:32:39.371 INFO:teuthology.orchestra.run.smithi119.stdout: "nodeep-scrub", 2024-09-18T11:32:39.371 INFO:teuthology.orchestra.run.smithi119.stdout: "pglog_hardlimit", 2024-09-18T11:32:39.371 INFO:teuthology.orchestra.run.smithi119.stdout: "purged_snapdirs", 2024-09-18T11:32:39.371 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_deletes", 2024-09-18T11:32:39.371 INFO:teuthology.orchestra.run.smithi119.stdout: "sortbitwise" 2024-09-18T11:32:39.372 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-09-18T11:32:39.372 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_version": 6, 2024-09-18T11:32:39.372 INFO:teuthology.orchestra.run.smithi119.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T11:32:39.372 INFO:teuthology.orchestra.run.smithi119.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T11:32:39.372 INFO:teuthology.orchestra.run.smithi119.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T11:32:39.372 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_snapshot": "", 2024-09-18T11:32:39.372 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_max": 4, 2024-09-18T11:32:39.372 INFO:teuthology.orchestra.run.smithi119.stdout: "max_osd": 8, 2024-09-18T11:32:39.372 INFO:teuthology.orchestra.run.smithi119.stdout: "require_min_compat_client": "luminous", 2024-09-18T11:32:39.372 INFO:teuthology.orchestra.run.smithi119.stdout: "min_compat_client": "luminous", 2024-09-18T11:32:39.372 INFO:teuthology.orchestra.run.smithi119.stdout: "require_osd_release": "quincy", 2024-09-18T11:32:39.373 INFO:teuthology.orchestra.run.smithi119.stdout: "pools": [ 2024-09-18T11:32:39.373 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.373 INFO:teuthology.orchestra.run.smithi119.stdout: "pool": 1, 2024-09-18T11:32:39.373 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_name": ".mgr", 2024-09-18T11:32:39.373 INFO:teuthology.orchestra.run.smithi119.stdout: "create_time": "2024-09-18T11:29:47.951262+0000", 2024-09-18T11:32:39.373 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-09-18T11:32:39.373 INFO:teuthology.orchestra.run.smithi119.stdout: "flags_names": "hashpspool", 2024-09-18T11:32:39.373 INFO:teuthology.orchestra.run.smithi119.stdout: "type": 1, 2024-09-18T11:32:39.373 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 2, 2024-09-18T11:32:39.373 INFO:teuthology.orchestra.run.smithi119.stdout: "min_size": 2, 2024-09-18T11:32:39.373 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_rule": 0, 2024-09-18T11:32:39.373 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_count": 0, 2024-09-18T11:32:39.373 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_target": 0, 2024-09-18T11:32:39.374 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T11:32:39.374 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T11:32:39.374 INFO:teuthology.orchestra.run.smithi119.stdout: "object_hash": 2, 2024-09-18T11:32:39.374 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_autoscale_mode": "off", 2024-09-18T11:32:39.374 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num": 1, 2024-09-18T11:32:39.374 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_placement_num": 1, 2024-09-18T11:32:39.374 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_placement_num_target": 1, 2024-09-18T11:32:39.374 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num_target": 1, 2024-09-18T11:32:39.374 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num_pending": 1, 2024-09-18T11:32:39.374 INFO:teuthology.orchestra.run.smithi119.stdout: "last_pg_merge_meta": { 2024-09-18T11:32:39.374 INFO:teuthology.orchestra.run.smithi119.stdout: "source_pgid": "0.0", 2024-09-18T11:32:39.374 INFO:teuthology.orchestra.run.smithi119.stdout: "ready_epoch": 0, 2024-09-18T11:32:39.374 INFO:teuthology.orchestra.run.smithi119.stdout: "last_epoch_started": 0, 2024-09-18T11:32:39.375 INFO:teuthology.orchestra.run.smithi119.stdout: "last_epoch_clean": 0, 2024-09-18T11:32:39.375 INFO:teuthology.orchestra.run.smithi119.stdout: "source_version": "0'0", 2024-09-18T11:32:39.375 INFO:teuthology.orchestra.run.smithi119.stdout: "target_version": "0'0" 2024-09-18T11:32:39.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.375 INFO:teuthology.orchestra.run.smithi119.stdout: "last_change": "16", 2024-09-18T11:32:39.375 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend": "0", 2024-09-18T11:32:39.375 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T11:32:39.375 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T11:32:39.375 INFO:teuthology.orchestra.run.smithi119.stdout: "auid": 0, 2024-09-18T11:32:39.375 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_mode": "selfmanaged", 2024-09-18T11:32:39.375 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_seq": 0, 2024-09-18T11:32:39.375 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_epoch": 0, 2024-09-18T11:32:39.375 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_snaps": [], 2024-09-18T11:32:39.375 INFO:teuthology.orchestra.run.smithi119.stdout: "removed_snaps": "[]", 2024-09-18T11:32:39.376 INFO:teuthology.orchestra.run.smithi119.stdout: "quota_max_bytes": 0, 2024-09-18T11:32:39.376 INFO:teuthology.orchestra.run.smithi119.stdout: "quota_max_objects": 0, 2024-09-18T11:32:39.376 INFO:teuthology.orchestra.run.smithi119.stdout: "tiers": [], 2024-09-18T11:32:39.376 INFO:teuthology.orchestra.run.smithi119.stdout: "tier_of": -1, 2024-09-18T11:32:39.376 INFO:teuthology.orchestra.run.smithi119.stdout: "read_tier": -1, 2024-09-18T11:32:39.376 INFO:teuthology.orchestra.run.smithi119.stdout: "write_tier": -1, 2024-09-18T11:32:39.376 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_mode": "none", 2024-09-18T11:32:39.376 INFO:teuthology.orchestra.run.smithi119.stdout: "target_max_bytes": 0, 2024-09-18T11:32:39.376 INFO:teuthology.orchestra.run.smithi119.stdout: "target_max_objects": 0, 2024-09-18T11:32:39.376 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T11:32:39.376 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T11:32:39.376 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T11:32:39.376 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_min_flush_age": 0, 2024-09-18T11:32:39.376 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_min_evict_age": 0, 2024-09-18T11:32:39.377 INFO:teuthology.orchestra.run.smithi119.stdout: "erasure_code_profile": "", 2024-09-18T11:32:39.377 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_params": { 2024-09-18T11:32:39.377 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "none" 2024-09-18T11:32:39.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.377 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_period": 0, 2024-09-18T11:32:39.377 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_count": 0, 2024-09-18T11:32:39.377 INFO:teuthology.orchestra.run.smithi119.stdout: "use_gmt_hitset": true, 2024-09-18T11:32:39.377 INFO:teuthology.orchestra.run.smithi119.stdout: "min_read_recency_for_promote": 0, 2024-09-18T11:32:39.377 INFO:teuthology.orchestra.run.smithi119.stdout: "min_write_recency_for_promote": 0, 2024-09-18T11:32:39.377 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T11:32:39.377 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_search_last_n": 0, 2024-09-18T11:32:39.377 INFO:teuthology.orchestra.run.smithi119.stdout: "grade_table": [], 2024-09-18T11:32:39.378 INFO:teuthology.orchestra.run.smithi119.stdout: "stripe_width": 0, 2024-09-18T11:32:39.378 INFO:teuthology.orchestra.run.smithi119.stdout: "expected_num_objects": 0, 2024-09-18T11:32:39.378 INFO:teuthology.orchestra.run.smithi119.stdout: "fast_read": false, 2024-09-18T11:32:39.378 INFO:teuthology.orchestra.run.smithi119.stdout: "options": { 2024-09-18T11:32:39.378 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num_max": 32, 2024-09-18T11:32:39.378 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num_min": 1 2024-09-18T11:32:39.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.378 INFO:teuthology.orchestra.run.smithi119.stdout: "application_metadata": { 2024-09-18T11:32:39.378 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": {} 2024-09-18T11:32:39.378 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.378 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.378 INFO:teuthology.orchestra.run.smithi119.stdout: "pool": 2, 2024-09-18T11:32:39.379 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_name": "rbd", 2024-09-18T11:32:39.379 INFO:teuthology.orchestra.run.smithi119.stdout: "create_time": "2024-09-18T11:29:48.866238+0000", 2024-09-18T11:32:39.379 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 8193, 2024-09-18T11:32:39.379 INFO:teuthology.orchestra.run.smithi119.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T11:32:39.379 INFO:teuthology.orchestra.run.smithi119.stdout: "type": 1, 2024-09-18T11:32:39.379 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 2, 2024-09-18T11:32:39.379 INFO:teuthology.orchestra.run.smithi119.stdout: "min_size": 2, 2024-09-18T11:32:39.379 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_rule": 0, 2024-09-18T11:32:39.379 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_count": 0, 2024-09-18T11:32:39.379 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_target": 0, 2024-09-18T11:32:39.379 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T11:32:39.379 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T11:32:39.379 INFO:teuthology.orchestra.run.smithi119.stdout: "object_hash": 2, 2024-09-18T11:32:39.380 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_autoscale_mode": "off", 2024-09-18T11:32:39.380 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num": 8, 2024-09-18T11:32:39.380 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_placement_num": 8, 2024-09-18T11:32:39.380 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_placement_num_target": 8, 2024-09-18T11:32:39.380 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num_target": 8, 2024-09-18T11:32:39.380 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num_pending": 8, 2024-09-18T11:32:39.380 INFO:teuthology.orchestra.run.smithi119.stdout: "last_pg_merge_meta": { 2024-09-18T11:32:39.380 INFO:teuthology.orchestra.run.smithi119.stdout: "source_pgid": "0.0", 2024-09-18T11:32:39.380 INFO:teuthology.orchestra.run.smithi119.stdout: "ready_epoch": 0, 2024-09-18T11:32:39.380 INFO:teuthology.orchestra.run.smithi119.stdout: "last_epoch_started": 0, 2024-09-18T11:32:39.380 INFO:teuthology.orchestra.run.smithi119.stdout: "last_epoch_clean": 0, 2024-09-18T11:32:39.380 INFO:teuthology.orchestra.run.smithi119.stdout: "source_version": "0'0", 2024-09-18T11:32:39.380 INFO:teuthology.orchestra.run.smithi119.stdout: "target_version": "0'0" 2024-09-18T11:32:39.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.381 INFO:teuthology.orchestra.run.smithi119.stdout: "last_change": "18", 2024-09-18T11:32:39.381 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend": "0", 2024-09-18T11:32:39.381 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T11:32:39.381 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T11:32:39.381 INFO:teuthology.orchestra.run.smithi119.stdout: "auid": 0, 2024-09-18T11:32:39.381 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_mode": "selfmanaged", 2024-09-18T11:32:39.381 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_seq": 3, 2024-09-18T11:32:39.381 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_epoch": 18, 2024-09-18T11:32:39.381 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_snaps": [], 2024-09-18T11:32:39.381 INFO:teuthology.orchestra.run.smithi119.stdout: "removed_snaps": "[]", 2024-09-18T11:32:39.381 INFO:teuthology.orchestra.run.smithi119.stdout: "quota_max_bytes": 0, 2024-09-18T11:32:39.381 INFO:teuthology.orchestra.run.smithi119.stdout: "quota_max_objects": 0, 2024-09-18T11:32:39.381 INFO:teuthology.orchestra.run.smithi119.stdout: "tiers": [], 2024-09-18T11:32:39.381 INFO:teuthology.orchestra.run.smithi119.stdout: "tier_of": -1, 2024-09-18T11:32:39.382 INFO:teuthology.orchestra.run.smithi119.stdout: "read_tier": -1, 2024-09-18T11:32:39.382 INFO:teuthology.orchestra.run.smithi119.stdout: "write_tier": -1, 2024-09-18T11:32:39.382 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_mode": "none", 2024-09-18T11:32:39.382 INFO:teuthology.orchestra.run.smithi119.stdout: "target_max_bytes": 0, 2024-09-18T11:32:39.382 INFO:teuthology.orchestra.run.smithi119.stdout: "target_max_objects": 0, 2024-09-18T11:32:39.382 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T11:32:39.382 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T11:32:39.382 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T11:32:39.382 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_min_flush_age": 0, 2024-09-18T11:32:39.382 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_min_evict_age": 0, 2024-09-18T11:32:39.382 INFO:teuthology.orchestra.run.smithi119.stdout: "erasure_code_profile": "", 2024-09-18T11:32:39.382 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_params": { 2024-09-18T11:32:39.382 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "none" 2024-09-18T11:32:39.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.383 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_period": 0, 2024-09-18T11:32:39.383 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_count": 0, 2024-09-18T11:32:39.383 INFO:teuthology.orchestra.run.smithi119.stdout: "use_gmt_hitset": true, 2024-09-18T11:32:39.383 INFO:teuthology.orchestra.run.smithi119.stdout: "min_read_recency_for_promote": 0, 2024-09-18T11:32:39.383 INFO:teuthology.orchestra.run.smithi119.stdout: "min_write_recency_for_promote": 0, 2024-09-18T11:32:39.383 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T11:32:39.383 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_search_last_n": 0, 2024-09-18T11:32:39.383 INFO:teuthology.orchestra.run.smithi119.stdout: "grade_table": [], 2024-09-18T11:32:39.383 INFO:teuthology.orchestra.run.smithi119.stdout: "stripe_width": 0, 2024-09-18T11:32:39.383 INFO:teuthology.orchestra.run.smithi119.stdout: "expected_num_objects": 0, 2024-09-18T11:32:39.383 INFO:teuthology.orchestra.run.smithi119.stdout: "fast_read": false, 2024-09-18T11:32:39.383 INFO:teuthology.orchestra.run.smithi119.stdout: "options": {}, 2024-09-18T11:32:39.383 INFO:teuthology.orchestra.run.smithi119.stdout: "application_metadata": { 2024-09-18T11:32:39.384 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd": {} 2024-09-18T11:32:39.384 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.384 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.384 INFO:teuthology.orchestra.run.smithi119.stdout: "pool": 3, 2024-09-18T11:32:39.384 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_name": "low_tier", 2024-09-18T11:32:39.384 INFO:teuthology.orchestra.run.smithi119.stdout: "create_time": "2024-09-18T11:29:59.343964+0000", 2024-09-18T11:32:39.384 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2024-09-18T11:32:39.384 INFO:teuthology.orchestra.run.smithi119.stdout: "flags_names": "hashpspool", 2024-09-18T11:32:39.384 INFO:teuthology.orchestra.run.smithi119.stdout: "type": 1, 2024-09-18T11:32:39.384 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 2, 2024-09-18T11:32:39.384 INFO:teuthology.orchestra.run.smithi119.stdout: "min_size": 2, 2024-09-18T11:32:39.384 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_rule": 0, 2024-09-18T11:32:39.384 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_count": 0, 2024-09-18T11:32:39.385 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_target": 0, 2024-09-18T11:32:39.385 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T11:32:39.385 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T11:32:39.385 INFO:teuthology.orchestra.run.smithi119.stdout: "object_hash": 2, 2024-09-18T11:32:39.385 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_autoscale_mode": "off", 2024-09-18T11:32:39.385 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num": 4, 2024-09-18T11:32:39.385 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_placement_num": 4, 2024-09-18T11:32:39.385 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_placement_num_target": 4, 2024-09-18T11:32:39.385 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num_target": 4, 2024-09-18T11:32:39.385 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num_pending": 4, 2024-09-18T11:32:39.385 INFO:teuthology.orchestra.run.smithi119.stdout: "last_pg_merge_meta": { 2024-09-18T11:32:39.385 INFO:teuthology.orchestra.run.smithi119.stdout: "source_pgid": "0.0", 2024-09-18T11:32:39.386 INFO:teuthology.orchestra.run.smithi119.stdout: "ready_epoch": 0, 2024-09-18T11:32:39.386 INFO:teuthology.orchestra.run.smithi119.stdout: "last_epoch_started": 0, 2024-09-18T11:32:39.386 INFO:teuthology.orchestra.run.smithi119.stdout: "last_epoch_clean": 0, 2024-09-18T11:32:39.386 INFO:teuthology.orchestra.run.smithi119.stdout: "source_version": "0'0", 2024-09-18T11:32:39.386 INFO:teuthology.orchestra.run.smithi119.stdout: "target_version": "0'0" 2024-09-18T11:32:39.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.386 INFO:teuthology.orchestra.run.smithi119.stdout: "last_change": "23", 2024-09-18T11:32:39.386 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend": "0", 2024-09-18T11:32:39.386 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T11:32:39.386 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T11:32:39.386 INFO:teuthology.orchestra.run.smithi119.stdout: "auid": 0, 2024-09-18T11:32:39.386 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_mode": "selfmanaged", 2024-09-18T11:32:39.387 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_seq": 0, 2024-09-18T11:32:39.387 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_epoch": 0, 2024-09-18T11:32:39.387 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_snaps": [], 2024-09-18T11:32:39.387 INFO:teuthology.orchestra.run.smithi119.stdout: "removed_snaps": "[]", 2024-09-18T11:32:39.387 INFO:teuthology.orchestra.run.smithi119.stdout: "quota_max_bytes": 0, 2024-09-18T11:32:39.387 INFO:teuthology.orchestra.run.smithi119.stdout: "quota_max_objects": 0, 2024-09-18T11:32:39.387 INFO:teuthology.orchestra.run.smithi119.stdout: "tiers": [], 2024-09-18T11:32:39.387 INFO:teuthology.orchestra.run.smithi119.stdout: "tier_of": -1, 2024-09-18T11:32:39.387 INFO:teuthology.orchestra.run.smithi119.stdout: "read_tier": -1, 2024-09-18T11:32:39.387 INFO:teuthology.orchestra.run.smithi119.stdout: "write_tier": -1, 2024-09-18T11:32:39.387 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_mode": "none", 2024-09-18T11:32:39.387 INFO:teuthology.orchestra.run.smithi119.stdout: "target_max_bytes": 0, 2024-09-18T11:32:39.387 INFO:teuthology.orchestra.run.smithi119.stdout: "target_max_objects": 0, 2024-09-18T11:32:39.387 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T11:32:39.387 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T11:32:39.388 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T11:32:39.388 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_min_flush_age": 0, 2024-09-18T11:32:39.388 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_min_evict_age": 0, 2024-09-18T11:32:39.388 INFO:teuthology.orchestra.run.smithi119.stdout: "erasure_code_profile": "", 2024-09-18T11:32:39.388 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_params": { 2024-09-18T11:32:39.388 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "none" 2024-09-18T11:32:39.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.388 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_period": 0, 2024-09-18T11:32:39.388 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_count": 0, 2024-09-18T11:32:39.388 INFO:teuthology.orchestra.run.smithi119.stdout: "use_gmt_hitset": true, 2024-09-18T11:32:39.388 INFO:teuthology.orchestra.run.smithi119.stdout: "min_read_recency_for_promote": 0, 2024-09-18T11:32:39.388 INFO:teuthology.orchestra.run.smithi119.stdout: "min_write_recency_for_promote": 0, 2024-09-18T11:32:39.389 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T11:32:39.389 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_search_last_n": 0, 2024-09-18T11:32:39.389 INFO:teuthology.orchestra.run.smithi119.stdout: "grade_table": [], 2024-09-18T11:32:39.389 INFO:teuthology.orchestra.run.smithi119.stdout: "stripe_width": 0, 2024-09-18T11:32:39.389 INFO:teuthology.orchestra.run.smithi119.stdout: "expected_num_objects": 0, 2024-09-18T11:32:39.389 INFO:teuthology.orchestra.run.smithi119.stdout: "fast_read": false, 2024-09-18T11:32:39.389 INFO:teuthology.orchestra.run.smithi119.stdout: "options": {}, 2024-09-18T11:32:39.389 INFO:teuthology.orchestra.run.smithi119.stdout: "application_metadata": {} 2024-09-18T11:32:39.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.389 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.389 INFO:teuthology.orchestra.run.smithi119.stdout: "pool": 4, 2024-09-18T11:32:39.389 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_name": "unique_pool_0", 2024-09-18T11:32:39.389 INFO:teuthology.orchestra.run.smithi119.stdout: "create_time": "2024-09-18T11:30:00.523818+0000", 2024-09-18T11:32:39.389 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 8193, 2024-09-18T11:32:39.390 INFO:teuthology.orchestra.run.smithi119.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T11:32:39.390 INFO:teuthology.orchestra.run.smithi119.stdout: "type": 1, 2024-09-18T11:32:39.390 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 2, 2024-09-18T11:32:39.390 INFO:teuthology.orchestra.run.smithi119.stdout: "min_size": 2, 2024-09-18T11:32:39.390 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_rule": 0, 2024-09-18T11:32:39.390 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_count": 0, 2024-09-18T11:32:39.390 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_target": 0, 2024-09-18T11:32:39.390 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T11:32:39.390 INFO:teuthology.orchestra.run.smithi119.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T11:32:39.390 INFO:teuthology.orchestra.run.smithi119.stdout: "object_hash": 2, 2024-09-18T11:32:39.390 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_autoscale_mode": "off", 2024-09-18T11:32:39.390 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num": 16, 2024-09-18T11:32:39.390 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_placement_num": 16, 2024-09-18T11:32:39.391 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_placement_num_target": 16, 2024-09-18T11:32:39.391 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num_target": 16, 2024-09-18T11:32:39.391 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num_pending": 16, 2024-09-18T11:32:39.391 INFO:teuthology.orchestra.run.smithi119.stdout: "last_pg_merge_meta": { 2024-09-18T11:32:39.391 INFO:teuthology.orchestra.run.smithi119.stdout: "source_pgid": "0.0", 2024-09-18T11:32:39.391 INFO:teuthology.orchestra.run.smithi119.stdout: "ready_epoch": 0, 2024-09-18T11:32:39.391 INFO:teuthology.orchestra.run.smithi119.stdout: "last_epoch_started": 0, 2024-09-18T11:32:39.391 INFO:teuthology.orchestra.run.smithi119.stdout: "last_epoch_clean": 0, 2024-09-18T11:32:39.391 INFO:teuthology.orchestra.run.smithi119.stdout: "source_version": "0'0", 2024-09-18T11:32:39.391 INFO:teuthology.orchestra.run.smithi119.stdout: "target_version": "0'0" 2024-09-18T11:32:39.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.391 INFO:teuthology.orchestra.run.smithi119.stdout: "last_change": "168", 2024-09-18T11:32:39.392 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend": "0", 2024-09-18T11:32:39.392 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T11:32:39.392 INFO:teuthology.orchestra.run.smithi119.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T11:32:39.392 INFO:teuthology.orchestra.run.smithi119.stdout: "auid": 0, 2024-09-18T11:32:39.392 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_mode": "selfmanaged", 2024-09-18T11:32:39.392 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_seq": 117, 2024-09-18T11:32:39.392 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_epoch": 168, 2024-09-18T11:32:39.392 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_snaps": [], 2024-09-18T11:32:39.392 INFO:teuthology.orchestra.run.smithi119.stdout: "removed_snaps": "[]", 2024-09-18T11:32:39.392 INFO:teuthology.orchestra.run.smithi119.stdout: "quota_max_bytes": 0, 2024-09-18T11:32:39.392 INFO:teuthology.orchestra.run.smithi119.stdout: "quota_max_objects": 0, 2024-09-18T11:32:39.392 INFO:teuthology.orchestra.run.smithi119.stdout: "tiers": [], 2024-09-18T11:32:39.392 INFO:teuthology.orchestra.run.smithi119.stdout: "tier_of": -1, 2024-09-18T11:32:39.393 INFO:teuthology.orchestra.run.smithi119.stdout: "read_tier": -1, 2024-09-18T11:32:39.393 INFO:teuthology.orchestra.run.smithi119.stdout: "write_tier": -1, 2024-09-18T11:32:39.393 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_mode": "none", 2024-09-18T11:32:39.393 INFO:teuthology.orchestra.run.smithi119.stdout: "target_max_bytes": 0, 2024-09-18T11:32:39.393 INFO:teuthology.orchestra.run.smithi119.stdout: "target_max_objects": 0, 2024-09-18T11:32:39.393 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T11:32:39.393 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T11:32:39.393 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T11:32:39.393 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_min_flush_age": 0, 2024-09-18T11:32:39.393 INFO:teuthology.orchestra.run.smithi119.stdout: "cache_min_evict_age": 0, 2024-09-18T11:32:39.393 INFO:teuthology.orchestra.run.smithi119.stdout: "erasure_code_profile": "", 2024-09-18T11:32:39.393 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_params": { 2024-09-18T11:32:39.393 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "none" 2024-09-18T11:32:39.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.394 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_period": 0, 2024-09-18T11:32:39.394 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_count": 0, 2024-09-18T11:32:39.394 INFO:teuthology.orchestra.run.smithi119.stdout: "use_gmt_hitset": true, 2024-09-18T11:32:39.394 INFO:teuthology.orchestra.run.smithi119.stdout: "min_read_recency_for_promote": 0, 2024-09-18T11:32:39.394 INFO:teuthology.orchestra.run.smithi119.stdout: "min_write_recency_for_promote": 0, 2024-09-18T11:32:39.394 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T11:32:39.394 INFO:teuthology.orchestra.run.smithi119.stdout: "hit_set_search_last_n": 0, 2024-09-18T11:32:39.394 INFO:teuthology.orchestra.run.smithi119.stdout: "grade_table": [], 2024-09-18T11:32:39.394 INFO:teuthology.orchestra.run.smithi119.stdout: "stripe_width": 0, 2024-09-18T11:32:39.394 INFO:teuthology.orchestra.run.smithi119.stdout: "expected_num_objects": 0, 2024-09-18T11:32:39.394 INFO:teuthology.orchestra.run.smithi119.stdout: "fast_read": false, 2024-09-18T11:32:39.394 INFO:teuthology.orchestra.run.smithi119.stdout: "options": { 2024-09-18T11:32:39.394 INFO:teuthology.orchestra.run.smithi119.stdout: "dedup_cdc_chunk_size": 131072, 2024-09-18T11:32:39.394 INFO:teuthology.orchestra.run.smithi119.stdout: "dedup_chunk_algorithm": "fastcdc", 2024-09-18T11:32:39.395 INFO:teuthology.orchestra.run.smithi119.stdout: "dedup_tier": 3, 2024-09-18T11:32:39.395 INFO:teuthology.orchestra.run.smithi119.stdout: "fingerprint_algorithm": "sha256" 2024-09-18T11:32:39.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.395 INFO:teuthology.orchestra.run.smithi119.stdout: "application_metadata": { 2024-09-18T11:32:39.395 INFO:teuthology.orchestra.run.smithi119.stdout: "rados": {} 2024-09-18T11:32:39.395 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.395 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.395 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-09-18T11:32:39.395 INFO:teuthology.orchestra.run.smithi119.stdout: "osds": [ 2024-09-18T11:32:39.395 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.395 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 0, 2024-09-18T11:32:39.395 INFO:teuthology.orchestra.run.smithi119.stdout: "uuid": "12faced2-7888-4265-ad12-815efe36466c", 2024-09-18T11:32:39.395 INFO:teuthology.orchestra.run.smithi119.stdout: "up": 1, 2024-09-18T11:32:39.396 INFO:teuthology.orchestra.run.smithi119.stdout: "in": 0, 2024-09-18T11:32:39.396 INFO:teuthology.orchestra.run.smithi119.stdout: "weight": 0, 2024-09-18T11:32:39.396 INFO:teuthology.orchestra.run.smithi119.stdout: "primary_affinity": 1, 2024-09-18T11:32:39.396 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_begin": 13, 2024-09-18T11:32:39.396 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_end": 65, 2024-09-18T11:32:39.396 INFO:teuthology.orchestra.run.smithi119.stdout: "up_from": 119, 2024-09-18T11:32:39.396 INFO:teuthology.orchestra.run.smithi119.stdout: "up_thru": 22, 2024-09-18T11:32:39.396 INFO:teuthology.orchestra.run.smithi119.stdout: "down_at": 66, 2024-09-18T11:32:39.396 INFO:teuthology.orchestra.run.smithi119.stdout: "lost_at": 0, 2024-09-18T11:32:39.396 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addrs": { 2024-09-18T11:32:39.396 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.396 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.396 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.396 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6800", 2024-09-18T11:32:39.397 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 56607 2024-09-18T11:32:39.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.397 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.397 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6801", 2024-09-18T11:32:39.397 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 56607 2024-09-18T11:32:39.397 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.397 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.397 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addrs": { 2024-09-18T11:32:39.397 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.397 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.397 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6802", 2024-09-18T11:32:39.398 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 56607 2024-09-18T11:32:39.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.398 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.398 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6803", 2024-09-18T11:32:39.398 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 56607 2024-09-18T11:32:39.398 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.398 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.398 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addrs": { 2024-09-18T11:32:39.398 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.398 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.398 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6806", 2024-09-18T11:32:39.399 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 56607 2024-09-18T11:32:39.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.399 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.399 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6807", 2024-09-18T11:32:39.399 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 56607 2024-09-18T11:32:39.399 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.399 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.399 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addrs": { 2024-09-18T11:32:39.399 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.399 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.399 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6804", 2024-09-18T11:32:39.400 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 56607 2024-09-18T11:32:39.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.400 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.400 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6805", 2024-09-18T11:32:39.400 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 56607 2024-09-18T11:32:39.400 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.400 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.400 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addr": "172.21.15.119:6801/56607", 2024-09-18T11:32:39.400 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addr": "172.21.15.119:6803/56607", 2024-09-18T11:32:39.400 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addr": "172.21.15.119:6807/56607", 2024-09-18T11:32:39.400 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addr": "172.21.15.119:6805/56607", 2024-09-18T11:32:39.400 INFO:teuthology.orchestra.run.smithi119.stdout: "state": [ 2024-09-18T11:32:39.401 INFO:teuthology.orchestra.run.smithi119.stdout: "exists", 2024-09-18T11:32:39.401 INFO:teuthology.orchestra.run.smithi119.stdout: "up" 2024-09-18T11:32:39.401 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.401 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 1, 2024-09-18T11:32:39.401 INFO:teuthology.orchestra.run.smithi119.stdout: "uuid": "20fee29d-b047-46ae-aa60-88e897fbb0fb", 2024-09-18T11:32:39.401 INFO:teuthology.orchestra.run.smithi119.stdout: "up": 1, 2024-09-18T11:32:39.401 INFO:teuthology.orchestra.run.smithi119.stdout: "in": 1, 2024-09-18T11:32:39.401 INFO:teuthology.orchestra.run.smithi119.stdout: "weight": 0.9000244140625, 2024-09-18T11:32:39.401 INFO:teuthology.orchestra.run.smithi119.stdout: "primary_affinity": 1, 2024-09-18T11:32:39.401 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_begin": 0, 2024-09-18T11:32:39.401 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_end": 0, 2024-09-18T11:32:39.401 INFO:teuthology.orchestra.run.smithi119.stdout: "up_from": 13, 2024-09-18T11:32:39.402 INFO:teuthology.orchestra.run.smithi119.stdout: "up_thru": 157, 2024-09-18T11:32:39.402 INFO:teuthology.orchestra.run.smithi119.stdout: "down_at": 0, 2024-09-18T11:32:39.402 INFO:teuthology.orchestra.run.smithi119.stdout: "lost_at": 0, 2024-09-18T11:32:39.402 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addrs": { 2024-09-18T11:32:39.402 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.402 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.402 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6824", 2024-09-18T11:32:39.402 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33350 2024-09-18T11:32:39.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.402 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.402 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6825", 2024-09-18T11:32:39.402 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33350 2024-09-18T11:32:39.403 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.403 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.403 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addrs": { 2024-09-18T11:32:39.403 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.403 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.403 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6826", 2024-09-18T11:32:39.403 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33350 2024-09-18T11:32:39.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.403 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.403 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6827", 2024-09-18T11:32:39.403 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33350 2024-09-18T11:32:39.403 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.404 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.404 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addrs": { 2024-09-18T11:32:39.404 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.404 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.404 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6830", 2024-09-18T11:32:39.404 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33350 2024-09-18T11:32:39.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.404 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.404 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6831", 2024-09-18T11:32:39.404 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33350 2024-09-18T11:32:39.404 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.405 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.405 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addrs": { 2024-09-18T11:32:39.405 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.405 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.405 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6828", 2024-09-18T11:32:39.405 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33350 2024-09-18T11:32:39.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.405 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.405 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6829", 2024-09-18T11:32:39.405 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33350 2024-09-18T11:32:39.405 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.406 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.406 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addr": "172.21.15.119:6825/33350", 2024-09-18T11:32:39.406 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addr": "172.21.15.119:6827/33350", 2024-09-18T11:32:39.406 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addr": "172.21.15.119:6831/33350", 2024-09-18T11:32:39.406 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addr": "172.21.15.119:6829/33350", 2024-09-18T11:32:39.406 INFO:teuthology.orchestra.run.smithi119.stdout: "state": [ 2024-09-18T11:32:39.406 INFO:teuthology.orchestra.run.smithi119.stdout: "exists", 2024-09-18T11:32:39.406 INFO:teuthology.orchestra.run.smithi119.stdout: "up" 2024-09-18T11:32:39.406 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.406 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 2, 2024-09-18T11:32:39.407 INFO:teuthology.orchestra.run.smithi119.stdout: "uuid": "21a296f3-b259-4841-9a17-4cd21b302b36", 2024-09-18T11:32:39.407 INFO:teuthology.orchestra.run.smithi119.stdout: "up": 1, 2024-09-18T11:32:39.407 INFO:teuthology.orchestra.run.smithi119.stdout: "in": 1, 2024-09-18T11:32:39.407 INFO:teuthology.orchestra.run.smithi119.stdout: "weight": 0.462005615234375, 2024-09-18T11:32:39.407 INFO:teuthology.orchestra.run.smithi119.stdout: "primary_affinity": 0, 2024-09-18T11:32:39.407 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_begin": 0, 2024-09-18T11:32:39.407 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_end": 0, 2024-09-18T11:32:39.407 INFO:teuthology.orchestra.run.smithi119.stdout: "up_from": 13, 2024-09-18T11:32:39.407 INFO:teuthology.orchestra.run.smithi119.stdout: "up_thru": 15, 2024-09-18T11:32:39.407 INFO:teuthology.orchestra.run.smithi119.stdout: "down_at": 0, 2024-09-18T11:32:39.407 INFO:teuthology.orchestra.run.smithi119.stdout: "lost_at": 0, 2024-09-18T11:32:39.407 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addrs": { 2024-09-18T11:32:39.407 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.408 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.408 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6808", 2024-09-18T11:32:39.408 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33351 2024-09-18T11:32:39.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.408 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.408 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6809", 2024-09-18T11:32:39.408 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33351 2024-09-18T11:32:39.408 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.408 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.409 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addrs": { 2024-09-18T11:32:39.409 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.409 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.409 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6811", 2024-09-18T11:32:39.409 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33351 2024-09-18T11:32:39.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.409 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.409 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6813", 2024-09-18T11:32:39.409 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33351 2024-09-18T11:32:39.409 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.410 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.410 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addrs": { 2024-09-18T11:32:39.410 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.410 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.410 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6819", 2024-09-18T11:32:39.410 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33351 2024-09-18T11:32:39.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.410 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.410 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6821", 2024-09-18T11:32:39.410 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33351 2024-09-18T11:32:39.411 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.411 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.411 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addrs": { 2024-09-18T11:32:39.411 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.411 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.411 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6815", 2024-09-18T11:32:39.411 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33351 2024-09-18T11:32:39.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.411 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.411 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6817", 2024-09-18T11:32:39.412 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33351 2024-09-18T11:32:39.412 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.412 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.412 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addr": "172.21.15.119:6809/33351", 2024-09-18T11:32:39.412 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addr": "172.21.15.119:6813/33351", 2024-09-18T11:32:39.412 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addr": "172.21.15.119:6821/33351", 2024-09-18T11:32:39.412 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addr": "172.21.15.119:6817/33351", 2024-09-18T11:32:39.412 INFO:teuthology.orchestra.run.smithi119.stdout: "state": [ 2024-09-18T11:32:39.412 INFO:teuthology.orchestra.run.smithi119.stdout: "exists", 2024-09-18T11:32:39.412 INFO:teuthology.orchestra.run.smithi119.stdout: "up" 2024-09-18T11:32:39.412 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.413 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 3, 2024-09-18T11:32:39.413 INFO:teuthology.orchestra.run.smithi119.stdout: "uuid": "59044f30-32e1-4942-9a8e-202c1def221a", 2024-09-18T11:32:39.413 INFO:teuthology.orchestra.run.smithi119.stdout: "up": 1, 2024-09-18T11:32:39.413 INFO:teuthology.orchestra.run.smithi119.stdout: "in": 1, 2024-09-18T11:32:39.413 INFO:teuthology.orchestra.run.smithi119.stdout: "weight": 1, 2024-09-18T11:32:39.413 INFO:teuthology.orchestra.run.smithi119.stdout: "primary_affinity": 1, 2024-09-18T11:32:39.413 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_begin": 0, 2024-09-18T11:32:39.413 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_end": 0, 2024-09-18T11:32:39.413 INFO:teuthology.orchestra.run.smithi119.stdout: "up_from": 13, 2024-09-18T11:32:39.413 INFO:teuthology.orchestra.run.smithi119.stdout: "up_thru": 155, 2024-09-18T11:32:39.413 INFO:teuthology.orchestra.run.smithi119.stdout: "down_at": 0, 2024-09-18T11:32:39.413 INFO:teuthology.orchestra.run.smithi119.stdout: "lost_at": 0, 2024-09-18T11:32:39.414 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addrs": { 2024-09-18T11:32:39.414 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.414 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.414 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6810", 2024-09-18T11:32:39.414 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33349 2024-09-18T11:32:39.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.414 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.414 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6812", 2024-09-18T11:32:39.414 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33349 2024-09-18T11:32:39.414 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.414 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.415 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addrs": { 2024-09-18T11:32:39.415 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.415 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.415 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6814", 2024-09-18T11:32:39.415 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33349 2024-09-18T11:32:39.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.415 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.415 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6816", 2024-09-18T11:32:39.415 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33349 2024-09-18T11:32:39.415 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.415 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.416 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addrs": { 2024-09-18T11:32:39.416 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.416 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.416 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6822", 2024-09-18T11:32:39.416 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33349 2024-09-18T11:32:39.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.416 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.416 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6823", 2024-09-18T11:32:39.416 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33349 2024-09-18T11:32:39.416 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.416 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.417 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addrs": { 2024-09-18T11:32:39.417 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.417 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.417 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6818", 2024-09-18T11:32:39.417 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33349 2024-09-18T11:32:39.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.417 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.417 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6820", 2024-09-18T11:32:39.417 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33349 2024-09-18T11:32:39.417 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.417 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.418 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addr": "172.21.15.119:6812/33349", 2024-09-18T11:32:39.418 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addr": "172.21.15.119:6816/33349", 2024-09-18T11:32:39.418 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addr": "172.21.15.119:6823/33349", 2024-09-18T11:32:39.418 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addr": "172.21.15.119:6820/33349", 2024-09-18T11:32:39.418 INFO:teuthology.orchestra.run.smithi119.stdout: "state": [ 2024-09-18T11:32:39.418 INFO:teuthology.orchestra.run.smithi119.stdout: "exists", 2024-09-18T11:32:39.418 INFO:teuthology.orchestra.run.smithi119.stdout: "up" 2024-09-18T11:32:39.418 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.418 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 4, 2024-09-18T11:32:39.418 INFO:teuthology.orchestra.run.smithi119.stdout: "uuid": "7d1ae3d0-8d69-4d3c-9e51-24fdd77a22a9", 2024-09-18T11:32:39.418 INFO:teuthology.orchestra.run.smithi119.stdout: "up": 1, 2024-09-18T11:32:39.418 INFO:teuthology.orchestra.run.smithi119.stdout: "in": 1, 2024-09-18T11:32:39.418 INFO:teuthology.orchestra.run.smithi119.stdout: "weight": 0.9000244140625, 2024-09-18T11:32:39.418 INFO:teuthology.orchestra.run.smithi119.stdout: "primary_affinity": 1, 2024-09-18T11:32:39.419 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_begin": 0, 2024-09-18T11:32:39.419 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_end": 0, 2024-09-18T11:32:39.419 INFO:teuthology.orchestra.run.smithi119.stdout: "up_from": 13, 2024-09-18T11:32:39.419 INFO:teuthology.orchestra.run.smithi119.stdout: "up_thru": 143, 2024-09-18T11:32:39.419 INFO:teuthology.orchestra.run.smithi119.stdout: "down_at": 0, 2024-09-18T11:32:39.419 INFO:teuthology.orchestra.run.smithi119.stdout: "lost_at": 0, 2024-09-18T11:32:39.419 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addrs": { 2024-09-18T11:32:39.419 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.419 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.419 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6808", 2024-09-18T11:32:39.419 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33088 2024-09-18T11:32:39.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.419 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.420 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.420 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6809", 2024-09-18T11:32:39.420 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33088 2024-09-18T11:32:39.420 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.420 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.420 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addrs": { 2024-09-18T11:32:39.420 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.420 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.420 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6810", 2024-09-18T11:32:39.420 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33088 2024-09-18T11:32:39.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.421 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.421 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6811", 2024-09-18T11:32:39.421 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33088 2024-09-18T11:32:39.421 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.421 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.421 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addrs": { 2024-09-18T11:32:39.421 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.421 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.421 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6814", 2024-09-18T11:32:39.421 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33088 2024-09-18T11:32:39.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.422 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.422 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6815", 2024-09-18T11:32:39.422 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33088 2024-09-18T11:32:39.422 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.422 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.422 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addrs": { 2024-09-18T11:32:39.422 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.422 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.422 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6812", 2024-09-18T11:32:39.422 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33088 2024-09-18T11:32:39.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.423 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.423 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6813", 2024-09-18T11:32:39.423 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33088 2024-09-18T11:32:39.423 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.423 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.423 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addr": "172.21.15.148:6809/33088", 2024-09-18T11:32:39.423 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addr": "172.21.15.148:6811/33088", 2024-09-18T11:32:39.423 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addr": "172.21.15.148:6815/33088", 2024-09-18T11:32:39.423 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addr": "172.21.15.148:6813/33088", 2024-09-18T11:32:39.423 INFO:teuthology.orchestra.run.smithi119.stdout: "state": [ 2024-09-18T11:32:39.423 INFO:teuthology.orchestra.run.smithi119.stdout: "exists", 2024-09-18T11:32:39.424 INFO:teuthology.orchestra.run.smithi119.stdout: "up" 2024-09-18T11:32:39.424 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.424 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 5, 2024-09-18T11:32:39.424 INFO:teuthology.orchestra.run.smithi119.stdout: "uuid": "b0dd3d30-b273-428c-a4a9-4c238abf45fc", 2024-09-18T11:32:39.424 INFO:teuthology.orchestra.run.smithi119.stdout: "up": 1, 2024-09-18T11:32:39.424 INFO:teuthology.orchestra.run.smithi119.stdout: "in": 1, 2024-09-18T11:32:39.424 INFO:teuthology.orchestra.run.smithi119.stdout: "weight": 1, 2024-09-18T11:32:39.424 INFO:teuthology.orchestra.run.smithi119.stdout: "primary_affinity": 1, 2024-09-18T11:32:39.424 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_begin": 0, 2024-09-18T11:32:39.424 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_end": 0, 2024-09-18T11:32:39.425 INFO:teuthology.orchestra.run.smithi119.stdout: "up_from": 13, 2024-09-18T11:32:39.425 INFO:teuthology.orchestra.run.smithi119.stdout: "up_thru": 157, 2024-09-18T11:32:39.425 INFO:teuthology.orchestra.run.smithi119.stdout: "down_at": 0, 2024-09-18T11:32:39.425 INFO:teuthology.orchestra.run.smithi119.stdout: "lost_at": 0, 2024-09-18T11:32:39.425 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addrs": { 2024-09-18T11:32:39.425 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.425 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.425 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6800", 2024-09-18T11:32:39.425 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33089 2024-09-18T11:32:39.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.425 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.426 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6801", 2024-09-18T11:32:39.426 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33089 2024-09-18T11:32:39.426 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.426 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.426 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addrs": { 2024-09-18T11:32:39.426 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.426 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.426 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6802", 2024-09-18T11:32:39.426 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33089 2024-09-18T11:32:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6803", 2024-09-18T11:32:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33089 2024-09-18T11:32:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addrs": { 2024-09-18T11:32:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6806", 2024-09-18T11:32:39.428 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33089 2024-09-18T11:32:39.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.428 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.428 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6807", 2024-09-18T11:32:39.428 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33089 2024-09-18T11:32:39.428 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.428 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.428 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addrs": { 2024-09-18T11:32:39.428 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.429 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.429 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6804", 2024-09-18T11:32:39.429 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33089 2024-09-18T11:32:39.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.429 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.429 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6805", 2024-09-18T11:32:39.429 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33089 2024-09-18T11:32:39.429 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.429 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.429 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addr": "172.21.15.148:6801/33089", 2024-09-18T11:32:39.429 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addr": "172.21.15.148:6803/33089", 2024-09-18T11:32:39.430 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addr": "172.21.15.148:6807/33089", 2024-09-18T11:32:39.430 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addr": "172.21.15.148:6805/33089", 2024-09-18T11:32:39.430 INFO:teuthology.orchestra.run.smithi119.stdout: "state": [ 2024-09-18T11:32:39.430 INFO:teuthology.orchestra.run.smithi119.stdout: "exists", 2024-09-18T11:32:39.430 INFO:teuthology.orchestra.run.smithi119.stdout: "up" 2024-09-18T11:32:39.430 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.430 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 6, 2024-09-18T11:32:39.430 INFO:teuthology.orchestra.run.smithi119.stdout: "uuid": "40290cb3-2334-4c68-8e6c-59d0d5455171", 2024-09-18T11:32:39.430 INFO:teuthology.orchestra.run.smithi119.stdout: "up": 1, 2024-09-18T11:32:39.430 INFO:teuthology.orchestra.run.smithi119.stdout: "in": 1, 2024-09-18T11:32:39.430 INFO:teuthology.orchestra.run.smithi119.stdout: "weight": 1, 2024-09-18T11:32:39.430 INFO:teuthology.orchestra.run.smithi119.stdout: "primary_affinity": 1, 2024-09-18T11:32:39.431 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_begin": 0, 2024-09-18T11:32:39.431 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_end": 0, 2024-09-18T11:32:39.431 INFO:teuthology.orchestra.run.smithi119.stdout: "up_from": 13, 2024-09-18T11:32:39.431 INFO:teuthology.orchestra.run.smithi119.stdout: "up_thru": 22, 2024-09-18T11:32:39.431 INFO:teuthology.orchestra.run.smithi119.stdout: "down_at": 0, 2024-09-18T11:32:39.431 INFO:teuthology.orchestra.run.smithi119.stdout: "lost_at": 0, 2024-09-18T11:32:39.431 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addrs": { 2024-09-18T11:32:39.431 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.431 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.431 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6816", 2024-09-18T11:32:39.431 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33107 2024-09-18T11:32:39.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.432 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.432 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6817", 2024-09-18T11:32:39.432 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33107 2024-09-18T11:32:39.432 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.432 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.432 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addrs": { 2024-09-18T11:32:39.432 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.432 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.432 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6818", 2024-09-18T11:32:39.432 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33107 2024-09-18T11:32:39.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.433 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.433 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6819", 2024-09-18T11:32:39.433 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33107 2024-09-18T11:32:39.433 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.433 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.433 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addrs": { 2024-09-18T11:32:39.433 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.433 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.433 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6822", 2024-09-18T11:32:39.433 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33107 2024-09-18T11:32:39.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.434 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.434 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6823", 2024-09-18T11:32:39.434 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33107 2024-09-18T11:32:39.434 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.434 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.434 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addrs": { 2024-09-18T11:32:39.434 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.434 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.434 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6820", 2024-09-18T11:32:39.434 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33107 2024-09-18T11:32:39.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.435 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.435 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6821", 2024-09-18T11:32:39.435 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33107 2024-09-18T11:32:39.435 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.435 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.435 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addr": "172.21.15.148:6817/33107", 2024-09-18T11:32:39.435 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addr": "172.21.15.148:6819/33107", 2024-09-18T11:32:39.435 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addr": "172.21.15.148:6823/33107", 2024-09-18T11:32:39.435 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addr": "172.21.15.148:6821/33107", 2024-09-18T11:32:39.435 INFO:teuthology.orchestra.run.smithi119.stdout: "state": [ 2024-09-18T11:32:39.435 INFO:teuthology.orchestra.run.smithi119.stdout: "exists", 2024-09-18T11:32:39.436 INFO:teuthology.orchestra.run.smithi119.stdout: "up" 2024-09-18T11:32:39.436 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.436 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 7, 2024-09-18T11:32:39.436 INFO:teuthology.orchestra.run.smithi119.stdout: "uuid": "54802abe-ea1a-448e-85af-d1f14a45be27", 2024-09-18T11:32:39.436 INFO:teuthology.orchestra.run.smithi119.stdout: "up": 1, 2024-09-18T11:32:39.436 INFO:teuthology.orchestra.run.smithi119.stdout: "in": 1, 2024-09-18T11:32:39.436 INFO:teuthology.orchestra.run.smithi119.stdout: "weight": 1, 2024-09-18T11:32:39.436 INFO:teuthology.orchestra.run.smithi119.stdout: "primary_affinity": 1, 2024-09-18T11:32:39.436 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_begin": 0, 2024-09-18T11:32:39.436 INFO:teuthology.orchestra.run.smithi119.stdout: "last_clean_end": 0, 2024-09-18T11:32:39.436 INFO:teuthology.orchestra.run.smithi119.stdout: "up_from": 13, 2024-09-18T11:32:39.436 INFO:teuthology.orchestra.run.smithi119.stdout: "up_thru": 67, 2024-09-18T11:32:39.437 INFO:teuthology.orchestra.run.smithi119.stdout: "down_at": 0, 2024-09-18T11:32:39.437 INFO:teuthology.orchestra.run.smithi119.stdout: "lost_at": 0, 2024-09-18T11:32:39.437 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addrs": { 2024-09-18T11:32:39.437 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.437 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.437 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6824", 2024-09-18T11:32:39.437 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33119 2024-09-18T11:32:39.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.437 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.437 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6825", 2024-09-18T11:32:39.437 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33119 2024-09-18T11:32:39.437 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.438 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.438 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addrs": { 2024-09-18T11:32:39.438 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.438 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.438 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6826", 2024-09-18T11:32:39.438 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33119 2024-09-18T11:32:39.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.438 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.438 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6827", 2024-09-18T11:32:39.438 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33119 2024-09-18T11:32:39.438 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.438 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.439 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addrs": { 2024-09-18T11:32:39.439 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.439 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.439 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6830", 2024-09-18T11:32:39.439 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33119 2024-09-18T11:32:39.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.439 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.439 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6831", 2024-09-18T11:32:39.439 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33119 2024-09-18T11:32:39.439 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.439 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.440 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addrs": { 2024-09-18T11:32:39.440 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2024-09-18T11:32:39.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.440 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2024-09-18T11:32:39.440 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6828", 2024-09-18T11:32:39.440 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33119 2024-09-18T11:32:39.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.440 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2024-09-18T11:32:39.440 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.148:6829", 2024-09-18T11:32:39.440 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 33119 2024-09-18T11:32:39.440 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.441 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.441 INFO:teuthology.orchestra.run.smithi119.stdout: "public_addr": "172.21.15.148:6825/33119", 2024-09-18T11:32:39.441 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_addr": "172.21.15.148:6827/33119", 2024-09-18T11:32:39.441 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_back_addr": "172.21.15.148:6831/33119", 2024-09-18T11:32:39.441 INFO:teuthology.orchestra.run.smithi119.stdout: "heartbeat_front_addr": "172.21.15.148:6829/33119", 2024-09-18T11:32:39.441 INFO:teuthology.orchestra.run.smithi119.stdout: "state": [ 2024-09-18T11:32:39.441 INFO:teuthology.orchestra.run.smithi119.stdout: "exists", 2024-09-18T11:32:39.441 INFO:teuthology.orchestra.run.smithi119.stdout: "up" 2024-09-18T11:32:39.441 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.441 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.441 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-09-18T11:32:39.441 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_xinfo": [ 2024-09-18T11:32:39.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.442 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 0, 2024-09-18T11:32:39.442 INFO:teuthology.orchestra.run.smithi119.stdout: "down_stamp": "0.000000", 2024-09-18T11:32:39.442 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_probability": 0, 2024-09-18T11:32:39.442 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_interval": 0, 2024-09-18T11:32:39.442 INFO:teuthology.orchestra.run.smithi119.stdout: "features": 4540138320759226367, 2024-09-18T11:32:39.442 INFO:teuthology.orchestra.run.smithi119.stdout: "old_weight": 65536, 2024-09-18T11:32:39.442 INFO:teuthology.orchestra.run.smithi119.stdout: "last_purged_snaps_scrub": "2024-09-18T11:31:49.015287+0000", 2024-09-18T11:32:39.442 INFO:teuthology.orchestra.run.smithi119.stdout: "dead_epoch": 66 2024-09-18T11:32:39.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.442 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 1, 2024-09-18T11:32:39.442 INFO:teuthology.orchestra.run.smithi119.stdout: "down_stamp": "0.000000", 2024-09-18T11:32:39.442 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_probability": 0, 2024-09-18T11:32:39.443 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_interval": 0, 2024-09-18T11:32:39.443 INFO:teuthology.orchestra.run.smithi119.stdout: "features": 4540138320759226367, 2024-09-18T11:32:39.443 INFO:teuthology.orchestra.run.smithi119.stdout: "old_weight": 0, 2024-09-18T11:32:39.443 INFO:teuthology.orchestra.run.smithi119.stdout: "last_purged_snaps_scrub": "2024-09-18T11:32:14.430356+0000", 2024-09-18T11:32:39.443 INFO:teuthology.orchestra.run.smithi119.stdout: "dead_epoch": 0 2024-09-18T11:32:39.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.443 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 2, 2024-09-18T11:32:39.443 INFO:teuthology.orchestra.run.smithi119.stdout: "down_stamp": "0.000000", 2024-09-18T11:32:39.443 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_probability": 0, 2024-09-18T11:32:39.443 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_interval": 0, 2024-09-18T11:32:39.443 INFO:teuthology.orchestra.run.smithi119.stdout: "features": 4540138320759226367, 2024-09-18T11:32:39.443 INFO:teuthology.orchestra.run.smithi119.stdout: "old_weight": 0, 2024-09-18T11:32:39.443 INFO:teuthology.orchestra.run.smithi119.stdout: "last_purged_snaps_scrub": "2024-09-18T11:32:27.493785+0000", 2024-09-18T11:32:39.444 INFO:teuthology.orchestra.run.smithi119.stdout: "dead_epoch": 0 2024-09-18T11:32:39.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.444 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 3, 2024-09-18T11:32:39.444 INFO:teuthology.orchestra.run.smithi119.stdout: "down_stamp": "0.000000", 2024-09-18T11:32:39.444 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_probability": 0, 2024-09-18T11:32:39.444 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_interval": 0, 2024-09-18T11:32:39.444 INFO:teuthology.orchestra.run.smithi119.stdout: "features": 4540138320759226367, 2024-09-18T11:32:39.444 INFO:teuthology.orchestra.run.smithi119.stdout: "old_weight": 0, 2024-09-18T11:32:39.444 INFO:teuthology.orchestra.run.smithi119.stdout: "last_purged_snaps_scrub": "2024-09-18T11:32:12.546041+0000", 2024-09-18T11:32:39.444 INFO:teuthology.orchestra.run.smithi119.stdout: "dead_epoch": 0 2024-09-18T11:32:39.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.444 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 4, 2024-09-18T11:32:39.445 INFO:teuthology.orchestra.run.smithi119.stdout: "down_stamp": "0.000000", 2024-09-18T11:32:39.445 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_probability": 0, 2024-09-18T11:32:39.445 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_interval": 0, 2024-09-18T11:32:39.445 INFO:teuthology.orchestra.run.smithi119.stdout: "features": 4540138320759226367, 2024-09-18T11:32:39.445 INFO:teuthology.orchestra.run.smithi119.stdout: "old_weight": 0, 2024-09-18T11:32:39.445 INFO:teuthology.orchestra.run.smithi119.stdout: "last_purged_snaps_scrub": "2024-09-18T11:32:13.027907+0000", 2024-09-18T11:32:39.445 INFO:teuthology.orchestra.run.smithi119.stdout: "dead_epoch": 0 2024-09-18T11:32:39.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.445 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 5, 2024-09-18T11:32:39.445 INFO:teuthology.orchestra.run.smithi119.stdout: "down_stamp": "0.000000", 2024-09-18T11:32:39.445 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_probability": 0, 2024-09-18T11:32:39.445 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_interval": 0, 2024-09-18T11:32:39.446 INFO:teuthology.orchestra.run.smithi119.stdout: "features": 4540138320759226367, 2024-09-18T11:32:39.446 INFO:teuthology.orchestra.run.smithi119.stdout: "old_weight": 0, 2024-09-18T11:32:39.446 INFO:teuthology.orchestra.run.smithi119.stdout: "last_purged_snaps_scrub": "2024-09-18T11:32:28.500831+0000", 2024-09-18T11:32:39.446 INFO:teuthology.orchestra.run.smithi119.stdout: "dead_epoch": 0 2024-09-18T11:32:39.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.446 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 6, 2024-09-18T11:32:39.446 INFO:teuthology.orchestra.run.smithi119.stdout: "down_stamp": "0.000000", 2024-09-18T11:32:39.446 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_probability": 0, 2024-09-18T11:32:39.446 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_interval": 0, 2024-09-18T11:32:39.446 INFO:teuthology.orchestra.run.smithi119.stdout: "features": 4540138320759226367, 2024-09-18T11:32:39.446 INFO:teuthology.orchestra.run.smithi119.stdout: "old_weight": 0, 2024-09-18T11:32:39.446 INFO:teuthology.orchestra.run.smithi119.stdout: "last_purged_snaps_scrub": "2024-09-18T11:32:14.951013+0000", 2024-09-18T11:32:39.446 INFO:teuthology.orchestra.run.smithi119.stdout: "dead_epoch": 0 2024-09-18T11:32:39.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.447 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": 7, 2024-09-18T11:32:39.447 INFO:teuthology.orchestra.run.smithi119.stdout: "down_stamp": "0.000000", 2024-09-18T11:32:39.447 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_probability": 0, 2024-09-18T11:32:39.447 INFO:teuthology.orchestra.run.smithi119.stdout: "laggy_interval": 0, 2024-09-18T11:32:39.447 INFO:teuthology.orchestra.run.smithi119.stdout: "features": 4540138320759226367, 2024-09-18T11:32:39.447 INFO:teuthology.orchestra.run.smithi119.stdout: "old_weight": 0, 2024-09-18T11:32:39.447 INFO:teuthology.orchestra.run.smithi119.stdout: "last_purged_snaps_scrub": "2024-09-18T11:32:27.373504+0000", 2024-09-18T11:32:39.447 INFO:teuthology.orchestra.run.smithi119.stdout: "dead_epoch": 0 2024-09-18T11:32:39.447 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.447 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-09-18T11:32:39.447 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_upmap": [ 2024-09-18T11:32:39.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.448 INFO:teuthology.orchestra.run.smithi119.stdout: "pgid": "4.b", 2024-09-18T11:32:39.448 INFO:teuthology.orchestra.run.smithi119.stdout: "osds": [ 2024-09-18T11:32:39.448 INFO:teuthology.orchestra.run.smithi119.stdout: 5, 2024-09-18T11:32:39.448 INFO:teuthology.orchestra.run.smithi119.stdout: 3 2024-09-18T11:32:39.448 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.448 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.448 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-09-18T11:32:39.448 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_upmap_items": [], 2024-09-18T11:32:39.448 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_temp": [], 2024-09-18T11:32:39.448 INFO:teuthology.orchestra.run.smithi119.stdout: "primary_temp": [], 2024-09-18T11:32:39.448 INFO:teuthology.orchestra.run.smithi119.stdout: "blocklist": {}, 2024-09-18T11:32:39.448 INFO:teuthology.orchestra.run.smithi119.stdout: "range_blocklist": {}, 2024-09-18T11:32:39.448 INFO:teuthology.orchestra.run.smithi119.stdout: "erasure_code_profiles": { 2024-09-18T11:32:39.449 INFO:teuthology.orchestra.run.smithi119.stdout: "default": { 2024-09-18T11:32:39.449 INFO:teuthology.orchestra.run.smithi119.stdout: "crush-failure-domain": "osd", 2024-09-18T11:32:39.449 INFO:teuthology.orchestra.run.smithi119.stdout: "k": "2", 2024-09-18T11:32:39.449 INFO:teuthology.orchestra.run.smithi119.stdout: "m": "1", 2024-09-18T11:32:39.449 INFO:teuthology.orchestra.run.smithi119.stdout: "plugin": "jerasure", 2024-09-18T11:32:39.449 INFO:teuthology.orchestra.run.smithi119.stdout: "technique": "reed_sol_van" 2024-09-18T11:32:39.449 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.449 INFO:teuthology.orchestra.run.smithi119.stdout: "removed_snaps_queue": [ 2024-09-18T11:32:39.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.449 INFO:teuthology.orchestra.run.smithi119.stdout: "pool": 4, 2024-09-18T11:32:39.449 INFO:teuthology.orchestra.run.smithi119.stdout: "snaps": [ 2024-09-18T11:32:39.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.449 INFO:teuthology.orchestra.run.smithi119.stdout: "begin": 75, 2024-09-18T11:32:39.450 INFO:teuthology.orchestra.run.smithi119.stdout: "length": 1 2024-09-18T11:32:39.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.450 INFO:teuthology.orchestra.run.smithi119.stdout: "begin": 78, 2024-09-18T11:32:39.450 INFO:teuthology.orchestra.run.smithi119.stdout: "length": 1 2024-09-18T11:32:39.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.450 INFO:teuthology.orchestra.run.smithi119.stdout: "begin": 85, 2024-09-18T11:32:39.450 INFO:teuthology.orchestra.run.smithi119.stdout: "length": 1 2024-09-18T11:32:39.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:39.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.450 INFO:teuthology.orchestra.run.smithi119.stdout: "begin": 109, 2024-09-18T11:32:39.450 INFO:teuthology.orchestra.run.smithi119.stdout: "length": 1 2024-09-18T11:32:39.450 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.451 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.451 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.451 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-09-18T11:32:39.451 INFO:teuthology.orchestra.run.smithi119.stdout: "new_removed_snaps": [ 2024-09-18T11:32:39.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.451 INFO:teuthology.orchestra.run.smithi119.stdout: "pool": 4, 2024-09-18T11:32:39.451 INFO:teuthology.orchestra.run.smithi119.stdout: "snaps": [ 2024-09-18T11:32:39.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:39.451 INFO:teuthology.orchestra.run.smithi119.stdout: "begin": 85, 2024-09-18T11:32:39.451 INFO:teuthology.orchestra.run.smithi119.stdout: "length": 1 2024-09-18T11:32:39.451 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.451 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:39.451 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.452 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-09-18T11:32:39.452 INFO:teuthology.orchestra.run.smithi119.stdout: "new_purged_snaps": [], 2024-09-18T11:32:39.452 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_node_flags": {}, 2024-09-18T11:32:39.452 INFO:teuthology.orchestra.run.smithi119.stdout: "device_class_flags": {}, 2024-09-18T11:32:39.452 INFO:teuthology.orchestra.run.smithi119.stdout: "stretch_mode": { 2024-09-18T11:32:39.452 INFO:teuthology.orchestra.run.smithi119.stdout: "stretch_mode_enabled": false, 2024-09-18T11:32:39.452 INFO:teuthology.orchestra.run.smithi119.stdout: "stretch_bucket_count": 0, 2024-09-18T11:32:39.452 INFO:teuthology.orchestra.run.smithi119.stdout: "degraded_stretch_mode": 0, 2024-09-18T11:32:39.452 INFO:teuthology.orchestra.run.smithi119.stdout: "recovering_stretch_mode": 0, 2024-09-18T11:32:39.452 INFO:teuthology.orchestra.run.smithi119.stdout: "stretch_mode_bucket": 0 2024-09-18T11:32:39.452 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:39.452 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:39.455 INFO:tasks.thrashosds.thrasher:j is {'epoch': 168, 'fsid': 'f4cca259-b87a-45cc-8995-7e9fca891a9f', 'created': '2024-09-18T11:29:38.205487+0000', 'modified': '2024-09-18T11:32:39.030915+0000', 'last_up_change': '2024-09-18T11:31:49.142260+0000', 'last_in_change': '2024-09-18T11:30:49.312763+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T11:29:47.951262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T11:29:48.866238+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'low_tier', 'create_time': '2024-09-18T11:29:59.343964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}}, {'pool': 4, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-18T11:30:00.523818+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '168', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 117, 'snap_epoch': 168, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'dedup_cdc_chunk_size': 131072, 'dedup_chunk_algorithm': 'fastcdc', 'dedup_tier': 3, 'fingerprint_algorithm': 'sha256'}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '12faced2-7888-4265-ad12-815efe36466c', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 13, 'last_clean_end': 65, 'up_from': 119, 'up_thru': 22, 'down_at': 66, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6800', 'nonce': 56607}, {'type': 'v1', 'addr': '172.21.15.119:6801', 'nonce': 56607}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6802', 'nonce': 56607}, {'type': 'v1', 'addr': '172.21.15.119:6803', 'nonce': 56607}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6806', 'nonce': 56607}, {'type': 'v1', 'addr': '172.21.15.119:6807', 'nonce': 56607}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6804', 'nonce': 56607}, {'type': 'v1', 'addr': '172.21.15.119:6805', 'nonce': 56607}]}, 'public_addr': '172.21.15.119:6801/56607', 'cluster_addr': '172.21.15.119:6803/56607', 'heartbeat_back_addr': '172.21.15.119:6807/56607', 'heartbeat_front_addr': '172.21.15.119:6805/56607', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '20fee29d-b047-46ae-aa60-88e897fbb0fb', 'up': 1, 'in': 1, 'weight': 0.9000244140625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 157, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6824', 'nonce': 33350}, {'type': 'v1', 'addr': '172.21.15.119:6825', 'nonce': 33350}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6826', 'nonce': 33350}, {'type': 'v1', 'addr': '172.21.15.119:6827', 'nonce': 33350}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6830', 'nonce': 33350}, {'type': 'v1', 'addr': '172.21.15.119:6831', 'nonce': 33350}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6828', 'nonce': 33350}, {'type': 'v1', 'addr': '172.21.15.119:6829', 'nonce': 33350}]}, 'public_addr': '172.21.15.119:6825/33350', 'cluster_addr': '172.21.15.119:6827/33350', 'heartbeat_back_addr': '172.21.15.119:6831/33350', 'heartbeat_front_addr': '172.21.15.119:6829/33350', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '21a296f3-b259-4841-9a17-4cd21b302b36', 'up': 1, 'in': 1, 'weight': 0.462005615234375, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 15, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6808', 'nonce': 33351}, {'type': 'v1', 'addr': '172.21.15.119:6809', 'nonce': 33351}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6811', 'nonce': 33351}, {'type': 'v1', 'addr': '172.21.15.119:6813', 'nonce': 33351}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6819', 'nonce': 33351}, {'type': 'v1', 'addr': '172.21.15.119:6821', 'nonce': 33351}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6815', 'nonce': 33351}, {'type': 'v1', 'addr': '172.21.15.119:6817', 'nonce': 33351}]}, 'public_addr': '172.21.15.119:6809/33351', 'cluster_addr': '172.21.15.119:6813/33351', 'heartbeat_back_addr': '172.21.15.119:6821/33351', 'heartbeat_front_addr': '172.21.15.119:6817/33351', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '59044f30-32e1-4942-9a8e-202c1def221a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 155, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6810', 'nonce': 33349}, {'type': 'v1', 'addr': '172.21.15.119:6812', 'nonce': 33349}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6814', 'nonce': 33349}, {'type': 'v1', 'addr': '172.21.15.119:6816', 'nonce': 33349}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6822', 'nonce': 33349}, {'type': 'v1', 'addr': '172.21.15.119:6823', 'nonce': 33349}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6818', 'nonce': 33349}, {'type': 'v1', 'addr': '172.21.15.119:6820', 'nonce': 33349}]}, 'public_addr': '172.21.15.119:6812/33349', 'cluster_addr': '172.21.15.119:6816/33349', 'heartbeat_back_addr': '172.21.15.119:6823/33349', 'heartbeat_front_addr': '172.21.15.119:6820/33349', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '7d1ae3d0-8d69-4d3c-9e51-24fdd77a22a9', 'up': 1, 'in': 1, 'weight': 0.9000244140625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 143, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6808', 'nonce': 33088}, {'type': 'v1', 'addr': '172.21.15.148:6809', 'nonce': 33088}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6810', 'nonce': 33088}, {'type': 'v1', 'addr': '172.21.15.148:6811', 'nonce': 33088}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6814', 'nonce': 33088}, {'type': 'v1', 'addr': '172.21.15.148:6815', 'nonce': 33088}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6812', 'nonce': 33088}, {'type': 'v1', 'addr': '172.21.15.148:6813', 'nonce': 33088}]}, 'public_addr': '172.21.15.148:6809/33088', 'cluster_addr': '172.21.15.148:6811/33088', 'heartbeat_back_addr': '172.21.15.148:6815/33088', 'heartbeat_front_addr': '172.21.15.148:6813/33088', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b0dd3d30-b273-428c-a4a9-4c238abf45fc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 157, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6800', 'nonce': 33089}, {'type': 'v1', 'addr': '172.21.15.148:6801', 'nonce': 33089}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6802', 'nonce': 33089}, {'type': 'v1', 'addr': '172.21.15.148:6803', 'nonce': 33089}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6806', 'nonce': 33089}, {'type': 'v1', 'addr': '172.21.15.148:6807', 'nonce': 33089}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6804', 'nonce': 33089}, {'type': 'v1', 'addr': '172.21.15.148:6805', 'nonce': 33089}]}, 'public_addr': '172.21.15.148:6801/33089', 'cluster_addr': '172.21.15.148:6803/33089', 'heartbeat_back_addr': '172.21.15.148:6807/33089', 'heartbeat_front_addr': '172.21.15.148:6805/33089', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '40290cb3-2334-4c68-8e6c-59d0d5455171', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6816', 'nonce': 33107}, {'type': 'v1', 'addr': '172.21.15.148:6817', 'nonce': 33107}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6818', 'nonce': 33107}, {'type': 'v1', 'addr': '172.21.15.148:6819', 'nonce': 33107}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6822', 'nonce': 33107}, {'type': 'v1', 'addr': '172.21.15.148:6823', 'nonce': 33107}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6820', 'nonce': 33107}, {'type': 'v1', 'addr': '172.21.15.148:6821', 'nonce': 33107}]}, 'public_addr': '172.21.15.148:6817/33107', 'cluster_addr': '172.21.15.148:6819/33107', 'heartbeat_back_addr': '172.21.15.148:6823/33107', 'heartbeat_front_addr': '172.21.15.148:6821/33107', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '54802abe-ea1a-448e-85af-d1f14a45be27', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 67, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6824', 'nonce': 33119}, {'type': 'v1', 'addr': '172.21.15.148:6825', 'nonce': 33119}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6826', 'nonce': 33119}, {'type': 'v1', 'addr': '172.21.15.148:6827', 'nonce': 33119}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6830', 'nonce': 33119}, {'type': 'v1', 'addr': '172.21.15.148:6831', 'nonce': 33119}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6828', 'nonce': 33119}, {'type': 'v1', 'addr': '172.21.15.148:6829', 'nonce': 33119}]}, 'public_addr': '172.21.15.148:6825/33119', 'cluster_addr': '172.21.15.148:6827/33119', 'heartbeat_back_addr': '172.21.15.148:6831/33119', 'heartbeat_front_addr': '172.21.15.148:6829/33119', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T11:31:49.015287+0000', 'dead_epoch': 66}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T11:32:14.430356+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T11:32:27.493785+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T11:32:12.546041+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T11:32:13.027907+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T11:32:28.500831+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T11:32:14.951013+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T11:32:27.373504+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '4.b', 'osds': [5, 3]}], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 4, 'snaps': [{'begin': 75, 'length': 1}, {'begin': 78, 'length': 1}, {'begin': 85, 'length': 1}, {'begin': 109, 'length': 1}]}], 'new_removed_snaps': [{'pool': 4, 'snaps': [{'begin': 85, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T11:32:39.455 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-18T11:32:39.456 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1788 4.8 4:1898b15d:::smithi11939506-25:head [write 453822~797061] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:32:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 116.499334503, 2024-09-18T11:32:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094305159999999999, 2024-09-18T11:32:39.456 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1788 2024-09-18T11:32:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915811+0000", 2024-09-18T11:32:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916608+0000", 2024-09-18T11:32:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079635000000000003 2024-09-18T11:32:39.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916624+0000", 2024-09-18T11:32:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6735000000000002e-05 2024-09-18T11:32:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916630+0000", 2024-09-18T11:32:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5709999999999997e-06 2024-09-18T11:32:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916674+0000", 2024-09-18T11:32:39.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4540999999999999e-05 2024-09-18T11:32:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916751+0000", 2024-09-18T11:32:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6695999999999999e-05 2024-09-18T11:32:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916872+0000", 2024-09-18T11:32:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120648 2024-09-18T11:32:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.919076+0000", 2024-09-18T11:32:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022046829999999998 2024-09-18T11:32:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925151+0000", 2024-09-18T11:32:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060740239999999999 2024-09-18T11:32:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925198+0000", 2024-09-18T11:32:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7896999999999999e-05 2024-09-18T11:32:39.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925244+0000", 2024-09-18T11:32:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5398999999999997e-05 2024-09-18T11:32:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2877 4.3 4:cc46532a:::smithi11939506-33:2f [read 0~3680733,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 76.908252047000005, 2024-09-18T11:32:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085715170000000007, 2024-09-18T11:32:39.462 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2877 2024-09-18T11:32:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506896+0000", 2024-09-18T11:32:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506898+0000", 2024-09-18T11:32:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4760000000000001e-06 2024-09-18T11:32:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506903+0000", 2024-09-18T11:32:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:32:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506906+0000", 2024-09-18T11:32:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1769999999999998e-06 2024-09-18T11:32:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507007+0000", 2024-09-18T11:32:39.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101479 2024-09-18T11:32:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507015+0000", 2024-09-18T11:32:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8490000000000006e-06 2024-09-18T11:32:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513757+0000", 2024-09-18T11:32:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0067417800000000002 2024-09-18T11:32:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.513867+0000", 2024-09-18T11:32:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000109987 2024-09-18T11:32:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515467+0000", 2024-09-18T11:32:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00160037 2024-09-18T11:32:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2878 4.3 4:cc46532a:::smithi11939506-33:2f [sparse-read 0~1770976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 76.908221592000004, 2024-09-18T11:32:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090602640000000002, 2024-09-18T11:32:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2878 2024-09-18T11:32:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506929+0000", 2024-09-18T11:32:39.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.163e-06 2024-09-18T11:32:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506936+0000", 2024-09-18T11:32:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5519999999999999e-06 2024-09-18T11:32:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506941+0000", 2024-09-18T11:32:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:32:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507036+0000", 2024-09-18T11:32:39.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5070999999999998e-05 2024-09-18T11:32:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507042+0000", 2024-09-18T11:32:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3959999999999997e-06 2024-09-18T11:32:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515504+0000", 2024-09-18T11:32:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084620300000000006 2024-09-18T11:32:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515625+0000", 2024-09-18T11:32:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012150999999999999 2024-09-18T11:32:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515986+0000", 2024-09-18T11:32:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036140100000000002 2024-09-18T11:32:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.472 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3963 4.3 4:c70c3162:::smithi11939506-35:head [tier-promote] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:32:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 30.039324581999999, 2024-09-18T11:32:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010675591999999999, 2024-09-18T11:32:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3963 2024-09-18T11:32:39.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:32:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:32:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375821+0000", 2024-09-18T11:32:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375829+0000", 2024-09-18T11:32:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7989999999999994e-06 2024-09-18T11:32:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375843+0000", 2024-09-18T11:32:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:32:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375849+0000", 2024-09-18T11:32:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1419999999999998e-06 2024-09-18T11:32:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375889+0000", 2024-09-18T11:32:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0197999999999997e-05 2024-09-18T11:32:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375894+0000", 2024-09-18T11:32:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0139999999999998e-06 2024-09-18T11:32:39.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.377699+0000", 2024-09-18T11:32:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001804688 2024-09-18T11:32:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.377776+0000", 2024-09-18T11:32:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7081000000000003e-05 2024-09-18T11:32:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.386499+0000", 2024-09-18T11:32:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008722746 2024-09-18T11:32:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1025 4.7 e138/22 4:e73ba68f:::smithi11939506-1:head v 138'281, mlcod=137'278)", 2024-09-18T11:32:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.587274+0000", 2024-09-18T11:32:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 29.827874267999999, 2024-09-18T11:32:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094231360000000004, 2024-09-18T11:32:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.587274+0000", 2024-09-18T11:32:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.587274+0000", 2024-09-18T11:32:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.587273+0000", 2024-09-18T11:32:39.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590258+0000", 2024-09-18T11:32:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002984998 2024-09-18T11:32:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590273+0000", 2024-09-18T11:32:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4999e-05 2024-09-18T11:32:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590283+0000", 2024-09-18T11:32:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3619999999999995e-06 2024-09-18T11:32:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590333+0000", 2024-09-18T11:32:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9978999999999997e-05 2024-09-18T11:32:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590352+0000", 2024-09-18T11:32:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9477999999999999e-05 2024-09-18T11:32:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596664+0000", 2024-09-18T11:32:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063120679999999997 2024-09-18T11:32:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596697+0000", 2024-09-18T11:32:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2456999999999998e-05 2024-09-18T11:32:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.482 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4259 4.b 4:dba5bd39:::smithi11939506-17:head [write 685818~644423] snapc 69=[69,67,61,55,51,4e,4b,45,42,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.040083+0000", 2024-09-18T11:32:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.375064403, 2024-09-18T11:32:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015748635, 2024-09-18T11:32:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4259 2024-09-18T11:32:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.040083+0000", 2024-09-18T11:32:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.040083+0000", 2024-09-18T11:32:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.040081+0000", 2024-09-18T11:32:39.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041258+0000", 2024-09-18T11:32:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011766540000000001 2024-09-18T11:32:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041286+0000", 2024-09-18T11:32:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8087000000000001e-05 2024-09-18T11:32:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041362+0000", 2024-09-18T11:32:39.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6155999999999994e-05 2024-09-18T11:32:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.044285+0000", 2024-09-18T11:32:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002922437 2024-09-18T11:32:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.044360+0000", 2024-09-18T11:32:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5201e-05 2024-09-18T11:32:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:39.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.044489+0000", 2024-09-18T11:32:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012950599999999999 2024-09-18T11:32:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052904+0000", 2024-09-18T11:32:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084149910000000001 2024-09-18T11:32:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055695+0000", 2024-09-18T11:32:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027902040000000001 2024-09-18T11:32:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055765+0000", 2024-09-18T11:32:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0720000000000001e-05 2024-09-18T11:32:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055832+0000", 2024-09-18T11:32:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6736999999999995e-05 2024-09-18T11:32:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.5.0:195 4.b e156/152 ondisk, result = 0)", 2024-09-18T11:32:39.488 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.041863+0000", 2024-09-18T11:32:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.373284883, 2024-09-18T11:32:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010991236, 2024-09-18T11:32:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041863+0000", 2024-09-18T11:32:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041863+0000", 2024-09-18T11:32:39.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041862+0000", 2024-09-18T11:32:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041868+0000", 2024-09-18T11:32:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2979999999999997e-06 2024-09-18T11:32:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041879+0000", 2024-09-18T11:32:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0522e-05 2024-09-18T11:32:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041885+0000", 2024-09-18T11:32:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7830000000000004e-06 2024-09-18T11:32:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052841+0000", 2024-09-18T11:32:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010956604 2024-09-18T11:32:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052848+0000", 2024-09-18T11:32:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2249999999999997e-06 2024-09-18T11:32:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052854+0000", 2024-09-18T11:32:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6050000000000003e-06 2024-09-18T11:32:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.5.0:196 4.b e156/152 ondisk, result = 0)", 2024-09-18T11:32:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.042994+0000", 2024-09-18T11:32:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.372154221000001, 2024-09-18T11:32:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098926020000000003, 2024-09-18T11:32:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.042994+0000", 2024-09-18T11:32:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.493 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.042994+0000", 2024-09-18T11:32:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.042993+0000", 2024-09-18T11:32:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043000+0000", 2024-09-18T11:32:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.525e-06 2024-09-18T11:32:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043011+0000", 2024-09-18T11:32:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0335e-05 2024-09-18T11:32:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043015+0000", 2024-09-18T11:32:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2899999999999996e-06 2024-09-18T11:32:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052868+0000", 2024-09-18T11:32:39.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098528130000000002 2024-09-18T11:32:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052871+0000", 2024-09-18T11:32:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5080000000000001e-06 2024-09-18T11:32:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052886+0000", 2024-09-18T11:32:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4781e-05 2024-09-18T11:32:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.496 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.5.0:194 4.3 e156/67 ondisk, result = 0)", 2024-09-18T11:32:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.043892+0000", 2024-09-18T11:32:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.371255431, 2024-09-18T11:32:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010237263999999999, 2024-09-18T11:32:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043892+0000", 2024-09-18T11:32:39.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043892+0000", 2024-09-18T11:32:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043891+0000", 2024-09-18T11:32:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043898+0000", 2024-09-18T11:32:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1190000000000002e-06 2024-09-18T11:32:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043910+0000", 2024-09-18T11:32:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.29e-05 2024-09-18T11:32:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043916+0000", 2024-09-18T11:32:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9499999999999998e-06 2024-09-18T11:32:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052830+0000", 2024-09-18T11:32:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089137480000000009 2024-09-18T11:32:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052839+0000", 2024-09-18T11:32:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9560000000000003e-06 2024-09-18T11:32:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054130+0000", 2024-09-18T11:32:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012905639999999999 2024-09-18T11:32:39.500 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4260 4.b 4:dba5bd39:::smithi11939506-17:head [write 2069089~216729] snapc 69=[69,67,61,55,51,4e,4b,45,42,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045444+0000", 2024-09-18T11:32:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.369703909, 2024-09-18T11:32:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011166715000000001, 2024-09-18T11:32:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4260 2024-09-18T11:32:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045444+0000", 2024-09-18T11:32:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045444+0000", 2024-09-18T11:32:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045443+0000", 2024-09-18T11:32:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045649+0000", 2024-09-18T11:32:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020605499999999999 2024-09-18T11:32:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045662+0000", 2024-09-18T11:32:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3386e-05 2024-09-18T11:32:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045667+0000", 2024-09-18T11:32:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6750000000000001e-06 2024-09-18T11:32:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052921+0000", 2024-09-18T11:32:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007254613 2024-09-18T11:32:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052994+0000", 2024-09-18T11:32:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2721999999999999e-05 2024-09-18T11:32:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.053116+0000", 2024-09-18T11:32:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121972 2024-09-18T11:32:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055857+0000", 2024-09-18T11:32:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027414539999999999 2024-09-18T11:32:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056502+0000", 2024-09-18T11:32:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064412499999999995 2024-09-18T11:32:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056555+0000", 2024-09-18T11:32:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3467999999999997e-05 2024-09-18T11:32:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056611+0000", 2024-09-18T11:32:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5577e-05 2024-09-18T11:32:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4261 4.b 4:dba5bd39:::smithi11939506-17:head [setxattr _header (57) in=7b,truncate 2285818] snapc 69=[69,67,61,55,51,4e,4b,45,42,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045672+0000", 2024-09-18T11:32:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.369475810999999, 2024-09-18T11:32:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011289933, 2024-09-18T11:32:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4261 2024-09-18T11:32:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045672+0000", 2024-09-18T11:32:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045672+0000", 2024-09-18T11:32:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045671+0000", 2024-09-18T11:32:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045675+0000", 2024-09-18T11:32:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.563e-06 2024-09-18T11:32:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045681+0000", 2024-09-18T11:32:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5210000000000002e-06 2024-09-18T11:32:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045684+0000", 2024-09-18T11:32:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4149999999999999e-06 2024-09-18T11:32:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054143+0000", 2024-09-18T11:32:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084590810000000002 2024-09-18T11:32:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054214+0000", 2024-09-18T11:32:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1061000000000003e-05 2024-09-18T11:32:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054354+0000", 2024-09-18T11:32:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000140035 2024-09-18T11:32:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056208+0000", 2024-09-18T11:32:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018539229999999999 2024-09-18T11:32:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056885+0000", 2024-09-18T11:32:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00067688000000000004 2024-09-18T11:32:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056922+0000", 2024-09-18T11:32:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6693999999999998e-05 2024-09-18T11:32:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056962+0000", 2024-09-18T11:32:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0299000000000002e-05 2024-09-18T11:32:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4262 4.b 4:dba5bd39:::smithi11939506-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:32:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.369459668999999, 2024-09-18T11:32:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011764513000000001, 2024-09-18T11:32:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4262 2024-09-18T11:32:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:32:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:32:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:32:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045690+0000", 2024-09-18T11:32:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.745e-06 2024-09-18T11:32:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045697+0000", 2024-09-18T11:32:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7229999999999998e-06 2024-09-18T11:32:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045704+0000", 2024-09-18T11:32:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5719999999999999e-06 2024-09-18T11:32:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055555+0000", 2024-09-18T11:32:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098515430000000008 2024-09-18T11:32:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055675+0000", 2024-09-18T11:32:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012000799999999999 2024-09-18T11:32:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.057004+0000", 2024-09-18T11:32:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013286179999999999 2024-09-18T11:32:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.057357+0000", 2024-09-18T11:32:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00035355100000000002 2024-09-18T11:32:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.057453+0000", 2024-09-18T11:32:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5154000000000004e-05 2024-09-18T11:32:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4264 4.7 4:e0d95e10:::smithi11939506-22:head [read 0~3008486,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:32:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.369409682000001, 2024-09-18T11:32:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095333889999999998, 2024-09-18T11:32:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4264 2024-09-18T11:32:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:32:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:32:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:32:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045747+0000", 2024-09-18T11:32:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2469999999999997e-06 2024-09-18T11:32:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045755+0000", 2024-09-18T11:32:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1499999999999999e-06 2024-09-18T11:32:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045759+0000", 2024-09-18T11:32:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0849999999999999e-06 2024-09-18T11:32:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045803+0000", 2024-09-18T11:32:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3621000000000001e-05 2024-09-18T11:32:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:32:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045807+0000", 2024-09-18T11:32:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7290000000000002e-06 2024-09-18T11:32:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.053475+0000", 2024-09-18T11:32:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076680100000000003 2024-09-18T11:32:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.053644+0000", 2024-09-18T11:32:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016971299999999999 2024-09-18T11:32:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055272+0000", 2024-09-18T11:32:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001627245 2024-09-18T11:32:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4265 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~2163506] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045792+0000", 2024-09-18T11:32:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 12.369355792, 2024-09-18T11:32:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010085024, 2024-09-18T11:32:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4265 2024-09-18T11:32:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045792+0000", 2024-09-18T11:32:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045792+0000", 2024-09-18T11:32:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045791+0000", 2024-09-18T11:32:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045797+0000", 2024-09-18T11:32:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0209999999999996e-06 2024-09-18T11:32:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045805+0000", 2024-09-18T11:32:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9349999999999994e-06 2024-09-18T11:32:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045811+0000", 2024-09-18T11:32:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2240000000000003e-06 2024-09-18T11:32:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045858+0000", 2024-09-18T11:32:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7565999999999998e-05 2024-09-18T11:32:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:32:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045869+0000", 2024-09-18T11:32:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0852e-05 2024-09-18T11:32:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055337+0000", 2024-09-18T11:32:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094678729999999999 2024-09-18T11:32:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055417+0000", 2024-09-18T11:32:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0092000000000005e-05 2024-09-18T11:32:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055877+0000", 2024-09-18T11:32:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00046001800000000003 2024-09-18T11:32:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4348 4.b 4:d13decd8:::smithi11939506-2:head [sparse-read 0~1463453,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:32:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.382848+0000", 2024-09-18T11:32:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.0322997619999992, 2024-09-18T11:32:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0077048719999999998, 2024-09-18T11:32:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4348 2024-09-18T11:32:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382848+0000", 2024-09-18T11:32:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382848+0000", 2024-09-18T11:32:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382846+0000", 2024-09-18T11:32:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382859+0000", 2024-09-18T11:32:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2792000000000001e-05 2024-09-18T11:32:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382884+0000", 2024-09-18T11:32:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4929000000000001e-05 2024-09-18T11:32:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382893+0000", 2024-09-18T11:32:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8169999999999994e-06 2024-09-18T11:32:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382958+0000", 2024-09-18T11:32:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4851999999999995e-05 2024-09-18T11:32:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382967+0000", 2024-09-18T11:32:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9789999999999999e-06 2024-09-18T11:32:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.385505+0000", 2024-09-18T11:32:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002537849 2024-09-18T11:32:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.385634+0000", 2024-09-18T11:32:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000128608 2024-09-18T11:32:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.390553+0000", 2024-09-18T11:32:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049192979999999999 2024-09-18T11:32:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4349 4.b 4:d13decd8:::smithi11939506-2:head [read 0~2302625,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:32:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.382910+0000", 2024-09-18T11:32:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.0322377280000001, 2024-09-18T11:32:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098226930000000004, 2024-09-18T11:32:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4349 2024-09-18T11:32:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382910+0000", 2024-09-18T11:32:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382910+0000", 2024-09-18T11:32:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382909+0000", 2024-09-18T11:32:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382915+0000", 2024-09-18T11:32:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6049999999999997e-06 2024-09-18T11:32:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382924+0000", 2024-09-18T11:32:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6289999999999999e-06 2024-09-18T11:32:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382929+0000", 2024-09-18T11:32:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0479999999999998e-06 2024-09-18T11:32:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383030+0000", 2024-09-18T11:32:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101346 2024-09-18T11:32:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383036+0000", 2024-09-18T11:32:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2720000000000003e-06 2024-09-18T11:32:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.390597+0000", 2024-09-18T11:32:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075606880000000003 2024-09-18T11:32:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.390696+0000", 2024-09-18T11:32:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8740000000000007e-05 2024-09-18T11:32:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392733+0000", 2024-09-18T11:32:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002037025 2024-09-18T11:32:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4350 4.b 4:d13decd8:::smithi11939506-2:head [sparse-read 0~1463453] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:32:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.382978+0000", 2024-09-18T11:32:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.0321701650000001, 2024-09-18T11:32:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01020132, 2024-09-18T11:32:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4350 2024-09-18T11:32:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382978+0000", 2024-09-18T11:32:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382978+0000", 2024-09-18T11:32:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382976+0000", 2024-09-18T11:32:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382984+0000", 2024-09-18T11:32:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.328e-06 2024-09-18T11:32:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382994+0000", 2024-09-18T11:32:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0801999999999999e-05 2024-09-18T11:32:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383001+0000", 2024-09-18T11:32:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6390000000000003e-06 2024-09-18T11:32:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383138+0000", 2024-09-18T11:32:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013676899999999999 2024-09-18T11:32:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383144+0000", 2024-09-18T11:32:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6359999999999997e-06 2024-09-18T11:32:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392791+0000", 2024-09-18T11:32:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096461659999999994 2024-09-18T11:32:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392897+0000", 2024-09-18T11:32:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000106087 2024-09-18T11:32:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.393179+0000", 2024-09-18T11:32:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00028227500000000002 2024-09-18T11:32:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4375 4.b 4:dba5bd39:::smithi11939506-17:head [write 437081~638017,stat] snapc 6d=[6d,6c,69,67,61,55,51,4b,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:32:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.549471+0000", 2024-09-18T11:32:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.8656764629999998, 2024-09-18T11:32:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097774179999999995, 2024-09-18T11:32:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4375 2024-09-18T11:32:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.549471+0000", 2024-09-18T11:32:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.549471+0000", 2024-09-18T11:32:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.549470+0000", 2024-09-18T11:32:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553463+0000", 2024-09-18T11:32:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039932520000000001 2024-09-18T11:32:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553478+0000", 2024-09-18T11:32:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4647999999999999e-05 2024-09-18T11:32:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553484+0000", 2024-09-18T11:32:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3799999999999999e-06 2024-09-18T11:32:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553536+0000", 2024-09-18T11:32:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1938999999999999e-05 2024-09-18T11:32:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553637+0000", 2024-09-18T11:32:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010074899999999999 2024-09-18T11:32:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553824+0000", 2024-09-18T11:32:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000187524 2024-09-18T11:32:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.556653+0000", 2024-09-18T11:32:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028285879999999999 2024-09-18T11:32:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.559176+0000", 2024-09-18T11:32:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025229369999999998 2024-09-18T11:32:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.559212+0000", 2024-09-18T11:32:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5750000000000002e-05 2024-09-18T11:32:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.559249+0000", 2024-09-18T11:32:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7020999999999997e-05 2024-09-18T11:32:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4430 4.b 4:d3ae8215:::smithi11939506-41:head [read 0~3832029,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T11:32:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:32:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.3866375020000001, 2024-09-18T11:32:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010918034, 2024-09-18T11:32:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4430 2024-09-18T11:32:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:32:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:32:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:32:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028520+0000", 2024-09-18T11:32:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.005e-05 2024-09-18T11:32:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028538+0000", 2024-09-18T11:32:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8020999999999999e-05 2024-09-18T11:32:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028543+0000", 2024-09-18T11:32:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.485e-06 2024-09-18T11:32:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028634+0000", 2024-09-18T11:32:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1120999999999997e-05 2024-09-18T11:32:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028639+0000", 2024-09-18T11:32:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1989999999999997e-06 2024-09-18T11:32:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.036054+0000", 2024-09-18T11:32:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074147559999999998 2024-09-18T11:32:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.036158+0000", 2024-09-18T11:32:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103564 2024-09-18T11:32:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039428+0000", 2024-09-18T11:32:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032706390000000001 2024-09-18T11:32:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4431 4.b 4:d3ae8215:::smithi11939506-41:head [sparse-read 0~2481209] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T11:32:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:32:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.3865987139999998, 2024-09-18T11:32:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011399179000000001, 2024-09-18T11:32:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4431 2024-09-18T11:32:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:32:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:32:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:32:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028552+0000", 2024-09-18T11:32:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5880000000000002e-06 2024-09-18T11:32:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028558+0000", 2024-09-18T11:32:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.84e-06 2024-09-18T11:32:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028562+0000", 2024-09-18T11:32:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6600000000000001e-06 2024-09-18T11:32:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028676+0000", 2024-09-18T11:32:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011408000000000001 2024-09-18T11:32:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028681+0000", 2024-09-18T11:32:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5350000000000004e-06 2024-09-18T11:32:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039458+0000", 2024-09-18T11:32:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010777231 2024-09-18T11:32:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039564+0000", 2024-09-18T11:32:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000105524 2024-09-18T11:32:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039948+0000", 2024-09-18T11:32:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00038430099999999998 2024-09-18T11:32:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.561 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:39.561 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:32:39.562 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:39.451+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:39.563 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:39.561+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:39.660 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:39.661 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:39.664 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:39.663+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:39.669 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:32:39.766 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:39.765+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:39.767 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:39.767 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:39.767 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:39.767 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:39.767 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:39.775 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:32:39.867 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:39.865+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:39.875 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:39.875 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:39.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:39.875 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:39.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.875 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:32:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 148.75673194399999, 2024-09-18T11:32:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:32:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:32:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:32:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:39.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:32:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:32:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:32:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:32:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:32:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:32:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:32:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:32:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:32:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:32:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.879 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:32:39.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:32:39.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:32:39.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:32:39.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:32:39.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:32:39.880 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.881 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.881 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.881 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:32:39.881 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:39.881 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 139.20370654300001, 2024-09-18T11:32:39.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:32:39.881 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.881 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.881 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.881 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.881 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:32:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:32:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:32:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:32:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:32:39.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:32:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:32:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:32:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:32:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:32:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:32:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:32:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:32:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:32:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2207 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1410296~761389] snapc 24=[24,1e,1c,1b,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-18T11:32:39.885 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:32:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 102.45055388, 2024-09-18T11:32:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008410743, 2024-09-18T11:32:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2207 2024-09-18T11:32:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:32:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422497+0000", 2024-09-18T11:32:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.422496+0000", 2024-09-18T11:32:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423190+0000", 2024-09-18T11:32:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00069405499999999995 2024-09-18T11:32:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423204+0000", 2024-09-18T11:32:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:32:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423208+0000", 2024-09-18T11:32:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:32:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423293+0000", 2024-09-18T11:32:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4524999999999995e-05 2024-09-18T11:32:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423394+0000", 2024-09-18T11:32:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101395 2024-09-18T11:32:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.423579+0000", 2024-09-18T11:32:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018461000000000001 2024-09-18T11:32:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.428574+0000", 2024-09-18T11:32:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049944960000000002 2024-09-18T11:32:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430809+0000", 2024-09-18T11:32:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022351820000000001 2024-09-18T11:32:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430862+0000", 2024-09-18T11:32:39.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3189999999999997e-05 2024-09-18T11:32:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:57.430908+0000", 2024-09-18T11:32:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5973999999999997e-05 2024-09-18T11:32:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2339 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:39.891 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 98.486249251000004, 2024-09-18T11:32:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011809992, 2024-09-18T11:32:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2339 2024-09-18T11:32:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:32:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:32:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4710000000000001e-06 2024-09-18T11:32:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386824+0000", 2024-09-18T11:32:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6104e-05 2024-09-18T11:32:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386836+0000", 2024-09-18T11:32:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1827999999999999e-05 2024-09-18T11:32:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388390+0000", 2024-09-18T11:32:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015539499999999999 2024-09-18T11:32:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388443+0000", 2024-09-18T11:32:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3785000000000001e-05 2024-09-18T11:32:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398238+0000", 2024-09-18T11:32:39.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097947989999999999 2024-09-18T11:32:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398315+0000", 2024-09-18T11:32:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6909999999999994e-05 2024-09-18T11:32:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398612+0000", 2024-09-18T11:32:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029671999999999999 2024-09-18T11:32:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.896 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2340 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 98.486205939000001, 2024-09-18T11:32:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012086448, 2024-09-18T11:32:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2340 2024-09-18T11:32:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.897 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386848+0000", 2024-09-18T11:32:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3069999999999998e-06 2024-09-18T11:32:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386855+0000", 2024-09-18T11:32:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0979999999999996e-06 2024-09-18T11:32:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386859+0000", 2024-09-18T11:32:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9360000000000001e-06 2024-09-18T11:32:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388466+0000", 2024-09-18T11:32:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001607154 2024-09-18T11:32:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:39.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388507+0000", 2024-09-18T11:32:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1093000000000001e-05 2024-09-18T11:32:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398647+0000", 2024-09-18T11:32:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010139686 2024-09-18T11:32:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398718+0000", 2024-09-18T11:32:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1138999999999998e-05 2024-09-18T11:32:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398931+0000", 2024-09-18T11:32:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021356899999999999 2024-09-18T11:32:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2341 4.5 4:aa575018:::smithi11939506-11:head [read 0~2487436,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 98.486187717000007, 2024-09-18T11:32:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013751648999999999, 2024-09-18T11:32:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.902 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2341 2024-09-18T11:32:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386866+0000", 2024-09-18T11:32:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4249999999999998e-06 2024-09-18T11:32:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386872+0000", 2024-09-18T11:32:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.003e-06 2024-09-18T11:32:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:32:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.507e-06 2024-09-18T11:32:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388520+0000", 2024-09-18T11:32:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001644146 2024-09-18T11:32:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388561+0000", 2024-09-18T11:32:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1486999999999998e-05 2024-09-18T11:32:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398963+0000", 2024-09-18T11:32:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010401353 2024-09-18T11:32:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.399046+0000", 2024-09-18T11:32:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3141999999999998e-05 2024-09-18T11:32:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.400615+0000", 2024-09-18T11:32:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015690960000000001 2024-09-18T11:32:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2530 4.5 4:af4ba75a:::smithi11939506-37:head [rollback 25] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:39.907 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:32:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 90.699652146999995, 2024-09-18T11:32:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080198800000000001, 2024-09-18T11:32:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2530 2024-09-18T11:32:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:32:39.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173399+0000", 2024-09-18T11:32:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173398+0000", 2024-09-18T11:32:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173403+0000", 2024-09-18T11:32:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4389999999999999e-06 2024-09-18T11:32:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173414+0000", 2024-09-18T11:32:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0411000000000001e-05 2024-09-18T11:32:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173420+0000", 2024-09-18T11:32:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9610000000000002e-06 2024-09-18T11:32:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176390+0000", 2024-09-18T11:32:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0029706289999999998 2024-09-18T11:32:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176501+0000", 2024-09-18T11:32:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011090300000000001 2024-09-18T11:32:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.176827+0000", 2024-09-18T11:32:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00032526899999999999 2024-09-18T11:32:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178958+0000", 2024-09-18T11:32:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021314860000000001 2024-09-18T11:32:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181318+0000", 2024-09-18T11:32:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002359571 2024-09-18T11:32:39.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181372+0000", 2024-09-18T11:32:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3884000000000001e-05 2024-09-18T11:32:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181419+0000", 2024-09-18T11:32:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7148000000000003e-05 2024-09-18T11:32:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2531 4.5 4:af4ba75a:::smithi11939506-37:head [append 0~0] snapc 2c=[2b,2a,26,25,24,1c,18] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 90.699626023999997, 2024-09-18T11:32:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080687239999999993, 2024-09-18T11:32:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2531 2024-09-18T11:32:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173425+0000", 2024-09-18T11:32:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173429+0000", 2024-09-18T11:32:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2989999999999998e-06 2024-09-18T11:32:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173436+0000", 2024-09-18T11:32:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1799999999999999e-06 2024-09-18T11:32:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.173441+0000", 2024-09-18T11:32:39.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8810000000000004e-06 2024-09-18T11:32:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178267+0000", 2024-09-18T11:32:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0048257969999999997 2024-09-18T11:32:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178388+0000", 2024-09-18T11:32:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121779 2024-09-18T11:32:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:39.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.178599+0000", 2024-09-18T11:32:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000210094 2024-09-18T11:32:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.179339+0000", 2024-09-18T11:32:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00074063799999999995 2024-09-18T11:32:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181444+0000", 2024-09-18T11:32:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021044100000000001 2024-09-18T11:32:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181470+0000", 2024-09-18T11:32:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6415999999999999e-05 2024-09-18T11:32:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:09.181494+0000", 2024-09-18T11:32:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3672e-05 2024-09-18T11:32:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.919 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2910 4.5 4:af4ba75a:::smithi11939506-37:head [write 786627~795991,stat] snapc 37=[37,35,34,30,2f,2a,25,24] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:32:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 76.869568194999999, 2024-09-18T11:32:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084093710000000006, 2024-09-18T11:32:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2910 2024-09-18T11:32:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:32:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003483+0000", 2024-09-18T11:32:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.003481+0000", 2024-09-18T11:32:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:39.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004300+0000", 2024-09-18T11:32:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081934200000000003 2024-09-18T11:32:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004324+0000", 2024-09-18T11:32:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4618999999999999e-05 2024-09-18T11:32:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004335+0000", 2024-09-18T11:32:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.092e-05 2024-09-18T11:32:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004385+0000", 2024-09-18T11:32:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9709999999999997e-05 2024-09-18T11:32:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004470+0000", 2024-09-18T11:32:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4864000000000006e-05 2024-09-18T11:32:39.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.004612+0000", 2024-09-18T11:32:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014241500000000001 2024-09-18T11:32:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011193+0000", 2024-09-18T11:32:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006580654 2024-09-18T11:32:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:39.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011788+0000", 2024-09-18T11:32:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059533600000000002 2024-09-18T11:32:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011842+0000", 2024-09-18T11:32:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3180000000000002e-05 2024-09-18T11:32:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:23.011892+0000", 2024-09-18T11:32:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0618000000000002e-05 2024-09-18T11:32:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.925 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2953 4.a 4:588debc2:::smithi11939506-18:head [assert-version v113,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:32:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 75.846252676999995, 2024-09-18T11:32:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083931279999999997, 2024-09-18T11:32:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2953 2024-09-18T11:32:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.926 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:32:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026798+0000", 2024-09-18T11:32:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026796+0000", 2024-09-18T11:32:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026806+0000", 2024-09-18T11:32:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.9529999999999994e-06 2024-09-18T11:32:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026824+0000", 2024-09-18T11:32:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7901999999999999e-05 2024-09-18T11:32:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026832+0000", 2024-09-18T11:32:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8450000000000003e-06 2024-09-18T11:32:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026883+0000", 2024-09-18T11:32:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0883000000000001e-05 2024-09-18T11:32:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.026980+0000", 2024-09-18T11:32:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.7526000000000001e-05 2024-09-18T11:32:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:32:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.032722+0000", 2024-09-18T11:32:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0057414689999999999 2024-09-18T11:32:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.034230+0000", 2024-09-18T11:32:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001508035 2024-09-18T11:32:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035090+0000", 2024-09-18T11:32:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00086047300000000005 2024-09-18T11:32:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035147+0000", 2024-09-18T11:32:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6598000000000003e-05 2024-09-18T11:32:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.035191+0000", 2024-09-18T11:32:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4747999999999999e-05 2024-09-18T11:32:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3105 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 3e=[3e,3d,3b,37,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-18T11:32:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 66.854839972999997, 2024-09-18T11:32:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011735906000000001, 2024-09-18T11:32:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3105 2024-09-18T11:32:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018212+0000", 2024-09-18T11:32:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6759999999999999e-06 2024-09-18T11:32:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018216+0000", 2024-09-18T11:32:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-06 2024-09-18T11:32:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018218+0000", 2024-09-18T11:32:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6110000000000001e-06 2024-09-18T11:32:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018246+0000", 2024-09-18T11:32:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7557000000000001e-05 2024-09-18T11:32:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018250+0000", 2024-09-18T11:32:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4299999999999999e-06 2024-09-18T11:32:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021364+0000", 2024-09-18T11:32:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031135389999999998 2024-09-18T11:32:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021494+0000", 2024-09-18T11:32:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000130333 2024-09-18T11:32:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.029947+0000", 2024-09-18T11:32:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084526150000000001 2024-09-18T11:32:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3323 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:32:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.756151928999998, 2024-09-18T11:32:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0088051689999999998, 2024-09-18T11:32:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3323 2024-09-18T11:32:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:32:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:32:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116897+0000", 2024-09-18T11:32:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116908+0000", 2024-09-18T11:32:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1511e-05 2024-09-18T11:32:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116925+0000", 2024-09-18T11:32:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7028e-05 2024-09-18T11:32:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116932+0000", 2024-09-18T11:32:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5919999999999997e-06 2024-09-18T11:32:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116988+0000", 2024-09-18T11:32:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5928999999999999e-05 2024-09-18T11:32:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116995+0000", 2024-09-18T11:32:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7409999999999992e-06 2024-09-18T11:32:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122390+0000", 2024-09-18T11:32:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053941889999999998 2024-09-18T11:32:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122453+0000", 2024-09-18T11:32:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3833999999999999e-05 2024-09-18T11:32:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122581+0000", 2024-09-18T11:32:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012751200000000001 2024-09-18T11:32:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125034+0000", 2024-09-18T11:32:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024525609999999998 2024-09-18T11:32:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125607+0000", 2024-09-18T11:32:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00057356499999999995 2024-09-18T11:32:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125655+0000", 2024-09-18T11:32:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8179000000000001e-05 2024-09-18T11:32:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125704+0000", 2024-09-18T11:32:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8950999999999999e-05 2024-09-18T11:32:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3324 4.5 4:a087a566:::smithi11939506-46:head [rollback 46] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.756109369999997, 2024-09-18T11:32:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010927767999999999, 2024-09-18T11:32:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3324 2024-09-18T11:32:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116941+0000", 2024-09-18T11:32:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116949+0000", 2024-09-18T11:32:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4660000000000002e-06 2024-09-18T11:32:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116960+0000", 2024-09-18T11:32:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1484999999999999e-05 2024-09-18T11:32:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116964+0000", 2024-09-18T11:32:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2810000000000002e-06 2024-09-18T11:32:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117053+0000", 2024-09-18T11:32:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8747999999999998e-05 2024-09-18T11:32:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117059+0000", 2024-09-18T11:32:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.246e-06 2024-09-18T11:32:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123604+0000", 2024-09-18T11:32:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065445319999999996 2024-09-18T11:32:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123663+0000", 2024-09-18T11:32:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8740999999999999e-05 2024-09-18T11:32:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123803+0000", 2024-09-18T11:32:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014049200000000001 2024-09-18T11:32:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127600+0000", 2024-09-18T11:32:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037971179999999999 2024-09-18T11:32:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127814+0000", 2024-09-18T11:32:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021333600000000001 2024-09-18T11:32:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127847+0000", 2024-09-18T11:32:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3188999999999997e-05 2024-09-18T11:32:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127869+0000", 2024-09-18T11:32:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2515e-05 2024-09-18T11:32:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3325 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.756062184000001, 2024-09-18T11:32:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181030999999999, 2024-09-18T11:32:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3325 2024-09-18T11:32:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116988+0000", 2024-09-18T11:32:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116991+0000", 2024-09-18T11:32:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9890000000000001e-06 2024-09-18T11:32:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116998+0000", 2024-09-18T11:32:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5309999999999998e-06 2024-09-18T11:32:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117005+0000", 2024-09-18T11:32:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6420000000000001e-06 2024-09-18T11:32:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117091+0000", 2024-09-18T11:32:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6835999999999993e-05 2024-09-18T11:32:39.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:39.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117097+0000", 2024-09-18T11:32:39.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:32:39.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124717+0000", 2024-09-18T11:32:39.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076204899999999997 2024-09-18T11:32:39.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124773+0000", 2024-09-18T11:32:39.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5470999999999998e-05 2024-09-18T11:32:39.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:39.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124877+0000", 2024-09-18T11:32:39.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010426500000000001 2024-09-18T11:32:39.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:39.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127901+0000", 2024-09-18T11:32:39.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030243449999999999 2024-09-18T11:32:39.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:39.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128115+0000", 2024-09-18T11:32:39.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021355300000000001 2024-09-18T11:32:39.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:39.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128145+0000", 2024-09-18T11:32:39.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0290999999999998e-05 2024-09-18T11:32:39.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128170+0000", 2024-09-18T11:32:39.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4811e-05 2024-09-18T11:32:39.957 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.958 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.958 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.958 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3474 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1872245~637761] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:39.958 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:39.958 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.429285507000003, 2024-09-18T11:32:39.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0092038580000000005, 2024-09-18T11:32:39.958 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.958 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:39.958 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.958 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.958 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.958 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3474 2024-09-18T11:32:39.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.959 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:39.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:39.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443764+0000", 2024-09-18T11:32:39.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:39.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444359+0000", 2024-09-18T11:32:39.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059519 2024-09-18T11:32:39.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444374+0000", 2024-09-18T11:32:39.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4949e-05 2024-09-18T11:32:39.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444382+0000", 2024-09-18T11:32:39.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1369999999999996e-06 2024-09-18T11:32:39.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444428+0000", 2024-09-18T11:32:39.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6783000000000003e-05 2024-09-18T11:32:39.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444507+0000", 2024-09-18T11:32:39.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8415000000000001e-05 2024-09-18T11:32:39.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:39.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444638+0000", 2024-09-18T11:32:39.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013129 2024-09-18T11:32:39.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:39.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.449059+0000", 2024-09-18T11:32:39.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004420719 2024-09-18T11:32:39.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:39.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452909+0000", 2024-09-18T11:32:39.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003850773 2024-09-18T11:32:39.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:39.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452944+0000", 2024-09-18T11:32:39.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-05 2024-09-18T11:32:39.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452969+0000", 2024-09-18T11:32:39.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5795999999999998e-05 2024-09-18T11:32:39.963 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.964 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.964 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.964 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3475 4.5 4:a56ea1fb:::smithi11939506-48:head [setxattr _header (57) in=7b,truncate 3042100] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:39.964 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:32:39.964 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.428656791999998, 2024-09-18T11:32:39.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086700660000000006, 2024-09-18T11:32:39.964 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.964 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:39.964 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.964 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.964 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.965 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3475 2024-09-18T11:32:39.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.965 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:32:39.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:32:39.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444393+0000", 2024-09-18T11:32:39.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:39.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444398+0000", 2024-09-18T11:32:39.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4909999999999999e-06 2024-09-18T11:32:39.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444405+0000", 2024-09-18T11:32:39.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5329999999999997e-06 2024-09-18T11:32:39.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444410+0000", 2024-09-18T11:32:39.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2470000000000002e-06 2024-09-18T11:32:39.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446046+0000", 2024-09-18T11:32:39.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001636722 2024-09-18T11:32:39.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446121+0000", 2024-09-18T11:32:39.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4604000000000006e-05 2024-09-18T11:32:39.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:39.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446274+0000", 2024-09-18T11:32:39.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015350300000000001 2024-09-18T11:32:39.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:39.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452884+0000", 2024-09-18T11:32:39.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066096439999999996 2024-09-18T11:32:39.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:39.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453000+0000", 2024-09-18T11:32:39.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000116204 2024-09-18T11:32:39.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:39.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453032+0000", 2024-09-18T11:32:39.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1327e-05 2024-09-18T11:32:39.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453064+0000", 2024-09-18T11:32:39.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2645000000000003e-05 2024-09-18T11:32:39.970 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.970 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.970 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.970 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3476 4.5 4:a56ea1fb:::smithi11939506-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:39.970 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:39.970 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.428617525, 2024-09-18T11:32:39.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087596800000000006, 2024-09-18T11:32:39.970 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.970 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.970 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.971 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.971 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.971 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3476 2024-09-18T11:32:39.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.971 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:39.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:39.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:39.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:39.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444436+0000", 2024-09-18T11:32:39.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9330000000000001e-06 2024-09-18T11:32:39.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444443+0000", 2024-09-18T11:32:39.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9369999999999998e-06 2024-09-18T11:32:39.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444447+0000", 2024-09-18T11:32:39.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1309999999999999e-06 2024-09-18T11:32:39.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446599+0000", 2024-09-18T11:32:39.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021521190000000001 2024-09-18T11:32:39.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:39.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446675+0000", 2024-09-18T11:32:39.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6099000000000004e-05 2024-09-18T11:32:39.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453088+0000", 2024-09-18T11:32:39.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006413307 2024-09-18T11:32:39.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453132+0000", 2024-09-18T11:32:39.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3548999999999999e-05 2024-09-18T11:32:39.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453193+0000", 2024-09-18T11:32:39.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1116000000000002e-05 2024-09-18T11:32:39.976 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.976 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.976 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.977 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3928 4.5 4:af4ba75a:::smithi11939506-37:head [sparse-read 0~1543958] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:39.977 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:39.977 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.853820614999997, 2024-09-18T11:32:39.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080807250000000004, 2024-09-18T11:32:39.977 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.977 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.977 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.977 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.977 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.977 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3928 2024-09-18T11:32:39.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.978 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:39.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:39.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019230+0000", 2024-09-18T11:32:39.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:39.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019232+0000", 2024-09-18T11:32:39.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1050000000000002e-06 2024-09-18T11:32:39.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019236+0000", 2024-09-18T11:32:39.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7910000000000002e-06 2024-09-18T11:32:39.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019238+0000", 2024-09-18T11:32:39.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4339999999999999e-06 2024-09-18T11:32:39.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019326+0000", 2024-09-18T11:32:39.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8097999999999996e-05 2024-09-18T11:32:39.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:39.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019330+0000", 2024-09-18T11:32:39.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9349999999999996e-06 2024-09-18T11:32:39.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027035+0000", 2024-09-18T11:32:39.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0077047269999999998 2024-09-18T11:32:39.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027110+0000", 2024-09-18T11:32:39.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4443000000000005e-05 2024-09-18T11:32:39.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027311+0000", 2024-09-18T11:32:39.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000201528 2024-09-18T11:32:39.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.982 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.982 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3929 4.5 4:af4ba75a:::smithi11939506-37:head [sparse-read 0~1543958] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:39.982 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:39.982 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.853790062000002, 2024-09-18T11:32:39.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084586699999999997, 2024-09-18T11:32:39.982 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.982 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:39.983 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.983 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.983 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.983 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3929 2024-09-18T11:32:39.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.983 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:39.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:39.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:39.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:39.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019262+0000", 2024-09-18T11:32:39.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8029999999999999e-06 2024-09-18T11:32:39.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019267+0000", 2024-09-18T11:32:39.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1119999999999998e-06 2024-09-18T11:32:39.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019269+0000", 2024-09-18T11:32:39.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4269999999999998e-06 2024-09-18T11:32:39.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019344+0000", 2024-09-18T11:32:39.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4979000000000002e-05 2024-09-18T11:32:39.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:39.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019348+0000", 2024-09-18T11:32:39.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3930000000000001e-06 2024-09-18T11:32:39.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027385+0000", 2024-09-18T11:32:39.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080368409999999994 2024-09-18T11:32:39.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027463+0000", 2024-09-18T11:32:39.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7316000000000003e-05 2024-09-18T11:32:39.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027720+0000", 2024-09-18T11:32:39.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000257068 2024-09-18T11:32:39.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.987 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.987 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4251 4.5 4:af4ba75a:::smithi11939506-37:head [delete] snapc 69=[69,67,61,55,51,4e,4c,4b,45,42,3e,3d] ondisk+write+known_if_redirected+full_force+supports_pool_eio e155)", 2024-09-18T11:32:39.988 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:26.039209+0000", 2024-09-18T11:32:39.988 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.833841487000001, 2024-09-18T11:32:39.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010258422, 2024-09-18T11:32:39.988 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:39.988 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:39.988 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:39.988 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:39.988 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:39.988 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4251 2024-09-18T11:32:39.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.988 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:39.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:39.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039209+0000", 2024-09-18T11:32:39.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:39.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039209+0000", 2024-09-18T11:32:39.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:39.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:39.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039208+0000", 2024-09-18T11:32:39.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:39.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:39.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039217+0000", 2024-09-18T11:32:39.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2650000000000002e-06 2024-09-18T11:32:39.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:39.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039236+0000", 2024-09-18T11:32:39.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8734000000000001e-05 2024-09-18T11:32:39.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:39.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039244+0000", 2024-09-18T11:32:39.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2900000000000002e-06 2024-09-18T11:32:39.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039289+0000", 2024-09-18T11:32:39.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4657999999999998e-05 2024-09-18T11:32:39.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:39.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039294+0000", 2024-09-18T11:32:39.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0719999999999999e-06 2024-09-18T11:32:39.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:39.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.042052+0000", 2024-09-18T11:32:39.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027585399999999999 2024-09-18T11:32:39.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:39.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.042162+0000", 2024-09-18T11:32:39.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011005699999999999 2024-09-18T11:32:39.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:39.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.042341+0000", 2024-09-18T11:32:39.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000178285 2024-09-18T11:32:39.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:39.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.044140+0000", 2024-09-18T11:32:39.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001799092 2024-09-18T11:32:39.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:39.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.049396+0000", 2024-09-18T11:32:39.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005256213 2024-09-18T11:32:39.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:39.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.049431+0000", 2024-09-18T11:32:39.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.536e-05 2024-09-18T11:32:39.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:39.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:39.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:39.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.049468+0000", 2024-09-18T11:32:39.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6548999999999999e-05 2024-09-18T11:32:39.994 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.994 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.994 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.994 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:39.994 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:39.994 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:39.996 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:32:39.996 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:39.994+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:40.037 INFO:tasks.rados.rados.0.smithi119.stdout:1935: done (8 left) 2024-09-18T11:32:40.037 INFO:tasks.rados.rados.0.smithi119.stdout:1936: set_chunk oid 38 target oid 2024-09-18T11:32:40.055 INFO:tasks.rados.rados.0.smithi119.stdout:1936: set_chunk oid 38 offset: 2095104 length: 379392 target oid 71947a750bc72352eea280bef91ee72ccad9ebb70f38621011697788c1f35114 offset: 0 snap 62 2024-09-18T11:32:40.055 INFO:tasks.rados.rados.0.smithi119.stdout:1937: read oid 15 snap -1 2024-09-18T11:32:40.055 INFO:tasks.rados.rados.0.smithi119.stdout:1937: expect (ObjNum 381 snap 60 seq_num 381) 2024-09-18T11:32:40.055 INFO:tasks.rados.rados.0.smithi119.stdout:1938: rollback oid 22 current snap is 64 2024-09-18T11:32:40.056 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 22 to 60 2024-09-18T11:32:40.056 INFO:tasks.rados.rados.0.smithi119.stdout:1939: read oid 30 snap -1 2024-09-18T11:32:40.056 INFO:tasks.rados.rados.0.smithi119.stdout:1939: expect (ObjNum 391 snap 63 seq_num 391) 2024-09-18T11:32:40.056 INFO:tasks.rados.rados.0.smithi119.stdout:1940: write (excl) oid 18 current snap is 64 2024-09-18T11:32:40.056 INFO:tasks.rados.rados.0.smithi119.stdout:1940: seq_num 395 ranges {638765=457358,1748152=604803} 2024-09-18T11:32:40.061 INFO:tasks.rados.rados.0.smithi119.stdout:1940: writing smithi11939506-18 from 638765 to 1096123 tid 1 2024-09-18T11:32:40.069 INFO:tasks.rados.rados.0.smithi119.stdout:1940: writing smithi11939506-18 from 1748152 to 2352955 tid 2 2024-09-18T11:32:40.069 INFO:tasks.rados.rados.0.smithi119.stdout:1941: rollback oid 36 current snap is 64 2024-09-18T11:32:40.069 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 36 to 61 2024-09-18T11:32:40.069 INFO:tasks.rados.rados.0.smithi119.stdout:1928: got (0) Success 2024-09-18T11:32:40.069 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 337 (ObjNum 329 snap 48 seq_num 329) dirty exists 2024-09-18T11:32:40.069 INFO:tasks.rados.rados.0.smithi119.stdout:1926: finishing write tid 2 to smithi11939506-47 2024-09-18T11:32:40.070 INFO:tasks.rados.rados.0.smithi119.stdout:1926: finishing write tid 3 to smithi11939506-47 2024-09-18T11:32:40.070 INFO:tasks.rados.rados.0.smithi119.stdout:1926: finishing write tid 4 to smithi11939506-47 2024-09-18T11:32:40.070 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 47 v 196 (ObjNum 394 snap 64 seq_num 394) dirty exists 2024-09-18T11:32:40.070 INFO:tasks.rados.rados.0.smithi119.stdout:1926: left oid 47 (ObjNum 394 snap 64 seq_num 394) 2024-09-18T11:32:40.070 INFO:tasks.rados.rados.0.smithi119.stdout:1927: expect (ObjNum 383 snap 60 seq_num 383) 2024-09-18T11:32:40.096 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:40.095+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:40.141 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:40.142 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:40.142 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:40.142 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:40.150 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:32:40.197 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:40.196+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:40.241 INFO:tasks.rados.rados.0.smithi119.stdout:1932: got (0) Success 2024-09-18T11:32:40.241 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 45 v 91 (ObjNum 254 snap 30 seq_num 254) dirty exists 2024-09-18T11:32:40.242 INFO:tasks.rados.rados.0.smithi119.stdout:1931: expect (ObjNum 287 snap 41 seq_num 287) 2024-09-18T11:32:40.248 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:40.248 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:40.248 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:40.248 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:40.248 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:40.256 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:32:40.301 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:40.299+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:40.356 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:40.356 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:40.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:40.356 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:40.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.356 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:40.356 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:40.356 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 171.19584878800001, 2024-09-18T11:32:40.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:32:40.356 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:40.356 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:40.357 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:40.357 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:40.357 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:40.357 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:32:40.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.357 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:40.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:40.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:40.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:40.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:40.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:40.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:32:40.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:40.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:40.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:32:40.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:32:40.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:40.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:32:40.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:32:40.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:40.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:32:40.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:32:40.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:40.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:32:40.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:32:40.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:40.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:32:40.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:32:40.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:40.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:32:40.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:32:40.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:40.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:32:40.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:32:40.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:40.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:32:40.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:32:40.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.361 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:40.361 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:32:40.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:32:40.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:40.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:32:40.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:32:40.362 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.362 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:40.362 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.362 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:40.362 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:40.362 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 171.195793858, 2024-09-18T11:32:40.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:32:40.363 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:40.363 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:40.363 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:40.363 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:40.363 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:40.363 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:32:40.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.363 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:40.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:40.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:40.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:40.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:40.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:40.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:40.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:40.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:40.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:32:40.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:32:40.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:40.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:32:40.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:32:40.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:40.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:32:40.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:32:40.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:40.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:32:40.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:32:40.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:40.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:32:40.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:32:40.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:40.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:32:40.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:32:40.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:40.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:32:40.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:32:40.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:40.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:32:40.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:32:40.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:40.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:32:40.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:32:40.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:40.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:32:40.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:32:40.368 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.368 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:40.368 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.368 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:40.368 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:40.368 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 171.19578387799999, 2024-09-18T11:32:40.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:32:40.369 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:40.369 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:40.369 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:40.369 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:40.369 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:40.369 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:32:40.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.369 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:40.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:40.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:40.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:40.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:40.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:40.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:40.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:40.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:40.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:32:40.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:32:40.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:40.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:32:40.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:32:40.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:40.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:32:40.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:32:40.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:40.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:32:40.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:32:40.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:40.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:32:40.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:32:40.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:40.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:32:40.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:32:40.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:40.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:32:40.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:32:40.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:40.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:32:40.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:32:40.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:40.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:32:40.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:32:40.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:40.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:32:40.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:32:40.374 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.374 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:40.374 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.374 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:40.374 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:40.374 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 171.19577227900001, 2024-09-18T11:32:40.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:32:40.375 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:40.375 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:40.375 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:40.375 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:40.375 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:40.375 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:32:40.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.375 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:40.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:40.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:40.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:40.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:40.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:40.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:40.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:40.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:40.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:32:40.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:32:40.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:40.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:32:40.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:32:40.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:40.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:32:40.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:32:40.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:40.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:32:40.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:32:40.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:40.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:32:40.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:32:40.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:40.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:32:40.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:32:40.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:40.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:32:40.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:32:40.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:40.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:32:40.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:32:40.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:40.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:32:40.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:32:40.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.380 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:40.380 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:32:40.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:32:40.380 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.380 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:40.380 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.380 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:40.380 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:40.381 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 171.19571579800001, 2024-09-18T11:32:40.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:32:40.381 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:40.381 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:40.381 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:40.381 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:40.381 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:40.381 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:32:40.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.381 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:40.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:40.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:40.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:40.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:40.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:40.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:32:40.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:40.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:40.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:32:40.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:32:40.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:40.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:32:40.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:32:40.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:40.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:32:40.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:32:40.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:40.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:32:40.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:32:40.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:40.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:32:40.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:32:40.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:40.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:32:40.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:32:40.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.386 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:40.386 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:32:40.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:32:40.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:40.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:32:40.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:32:40.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:40.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:32:40.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:32:40.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:40.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:32:40.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:32:40.387 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.388 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:40.388 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.388 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:40.388 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:40.388 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 171.195701974, 2024-09-18T11:32:40.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:32:40.388 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:40.388 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:40.388 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:40.388 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:40.388 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:40.389 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:32:40.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.389 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:40.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:40.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:40.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:40.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:40.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:40.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:40.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:40.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:40.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:32:40.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:32:40.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:40.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:32:40.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:32:40.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:40.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:32:40.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:32:40.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:40.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:32:40.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:32:40.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:40.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:32:40.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:32:40.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:40.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:32:40.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:32:40.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:40.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:32:40.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:32:40.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.392 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:40.392 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:32:40.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:32:40.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:40.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:32:40.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:32:40.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:40.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:32:40.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:32:40.393 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.393 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:40.393 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.394 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:40.394 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:40.394 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 170.77952306700001, 2024-09-18T11:32:40.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:32:40.394 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:40.394 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:40.394 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:40.394 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:40.394 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:40.394 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:32:40.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.394 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:40.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:40.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:40.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:40.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:40.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:40.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:40.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:40.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:40.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:32:40.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:32:40.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:40.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:32:40.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:32:40.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:40.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:32:40.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:32:40.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:40.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:32:40.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:32:40.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:40.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:32:40.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:32:40.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:40.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:32:40.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:32:40.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:40.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:32:40.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:32:40.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.398 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:40.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:32:40.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:32:40.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:40.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:32:40.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:32:40.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:40.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:32:40.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:32:40.399 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.399 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:40.399 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.400 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:40.400 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:40.400 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 170.779499767, 2024-09-18T11:32:40.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:32:40.400 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:40.400 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:40.400 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:40.400 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:40.400 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:40.400 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:32:40.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.400 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:40.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:40.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:40.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:40.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:40.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:40.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:32:40.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:40.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:40.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:32:40.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:32:40.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:40.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:32:40.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:32:40.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:40.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:32:40.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:32:40.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:40.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:32:40.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:32:40.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:40.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:32:40.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:32:40.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:40.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:32:40.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:32:40.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:40.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:32:40.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:32:40.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:40.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:32:40.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:32:40.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:32:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:32:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:32:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:32:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 170.77941415800001, 2024-09-18T11:32:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:32:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:32:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:40.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:40.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:32:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:32:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:32:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:32:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:40.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:32:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:32:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:32:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:32:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:32:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:32:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.409 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:32:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:32:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:32:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:32:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:32:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:32:40.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:32:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:32:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:32:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:32:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 170.77940319300001, 2024-09-18T11:32:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:32:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:40.412 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:32:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:32:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:32:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:32:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:32:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:32:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:32:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:32:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:32:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:32:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:32:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:32:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:32:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:32:40.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:32:40.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:40.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:32:40.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:32:40.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:40.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:32:40.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:32:40.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:40.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:32:40.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:32:40.418 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.418 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:40.418 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.418 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:40.418 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:40.418 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 170.77939202499999, 2024-09-18T11:32:40.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:32:40.418 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:40.418 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:40.419 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:40.419 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:40.419 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:40.419 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:32:40.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.419 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:40.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:40.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:40.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:40.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:40.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:40.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:40.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:40.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:40.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:32:40.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:32:40.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:40.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:32:40.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:32:40.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:40.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:32:40.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:32:40.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:40.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:32:40.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:32:40.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:40.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:32:40.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:32:40.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:40.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:32:40.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:32:40.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:40.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:32:40.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:32:40.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:40.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:32:40.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:32:40.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:40.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:32:40.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:32:40.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:40.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:32:40.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:32:40.424 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.424 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:40.424 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.424 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:40.424 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:40.424 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 170.779326696, 2024-09-18T11:32:40.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:32:40.424 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:40.424 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:40.424 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:40.424 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:40.424 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:40.424 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:32:40.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.425 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:40.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:40.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:40.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:40.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:40.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:40.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:32:40.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:40.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:40.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:32:40.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:32:40.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:40.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:32:40.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:32:40.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:40.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:32:40.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:32:40.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:40.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:32:40.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:32:40.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:40.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:32:40.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:32:40.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:40.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:32:40.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:32:40.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:40.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:32:40.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:32:40.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:40.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:32:40.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:32:40.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:40.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:32:40.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:32:40.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:40.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:32:40.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:32:40.430 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.430 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:40.430 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.430 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:40.430 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:40.430 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 170.76916471800001, 2024-09-18T11:32:40.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:32:40.430 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:40.430 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:40.430 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:40.430 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:40.431 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:40.431 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:32:40.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.431 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:40.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:40.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:40.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:40.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:40.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:40.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:32:40.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:40.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:40.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:32:40.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:32:40.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:40.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:32:40.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:32:40.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:40.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:32:40.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:32:40.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:40.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:32:40.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:32:40.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:40.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:32:40.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:32:40.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:40.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:32:40.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:32:40.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:40.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:32:40.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:32:40.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:40.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:32:40.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:32:40.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:40.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:32:40.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:32:40.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:40.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:32:40.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:32:40.436 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.436 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:40.436 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.436 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:40.436 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:40.436 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 170.769072239, 2024-09-18T11:32:40.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:32:40.436 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:40.436 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:40.437 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:40.437 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:40.437 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:40.437 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:32:40.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.437 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:40.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:40.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:40.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:40.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:40.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:40.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:40.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:40.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:40.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:32:40.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:32:40.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:40.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:32:40.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:32:40.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:40.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:32:40.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:32:40.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:40.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:32:40.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:32:40.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:40.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:32:40.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:32:40.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:40.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:32:40.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:32:40.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:40.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:32:40.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:32:40.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:40.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:32:40.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:32:40.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:40.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:32:40.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:32:40.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:40.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:32:40.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:32:40.442 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.442 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:40.442 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.442 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:123 4.3 e132/67 4:cc46532a:::smithi11939506-33:30 v 132'171, mlcod=128'168)", 2024-09-18T11:32:40.442 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:40.442 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 37.325765613999998, 2024-09-18T11:32:40.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087627039999999996, 2024-09-18T11:32:40.442 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:40.442 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:40.443 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:40.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:40.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:40.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:40.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:40.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:40.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027851+0000", 2024-09-18T11:32:40.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:40.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:40.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027871+0000", 2024-09-18T11:32:40.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9749e-05 2024-09-18T11:32:40.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:40.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027882+0000", 2024-09-18T11:32:40.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1242e-05 2024-09-18T11:32:40.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:40.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027889+0000", 2024-09-18T11:32:40.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7649999999999996e-06 2024-09-18T11:32:40.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:40.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027929+0000", 2024-09-18T11:32:40.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9984999999999997e-05 2024-09-18T11:32:40.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:40.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027951+0000", 2024-09-18T11:32:40.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2047000000000001e-05 2024-09-18T11:32:40.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:40.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.036565+0000", 2024-09-18T11:32:40.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086135110000000008 2024-09-18T11:32:40.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:40.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.036614+0000", 2024-09-18T11:32:40.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9747000000000003e-05 2024-09-18T11:32:40.446 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.447 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:40.447 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.447 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:124 4.3 e132/67 4:ca2b2309:::smithi11939506-44:30 v 132'173, mlcod=128'168)", 2024-09-18T11:32:40.447 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:40.447 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 37.325073631999999, 2024-09-18T11:32:40.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0088947309999999995, 2024-09-18T11:32:40.447 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:40.447 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:40.447 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:40.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:40.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:40.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:40.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:40.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:40.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028543+0000", 2024-09-18T11:32:40.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:40.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:40.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028551+0000", 2024-09-18T11:32:40.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8720000000000002e-06 2024-09-18T11:32:40.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:40.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028562+0000", 2024-09-18T11:32:40.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1039000000000001e-05 2024-09-18T11:32:40.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:40.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028569+0000", 2024-09-18T11:32:40.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0350000000000004e-06 2024-09-18T11:32:40.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:40.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028848+0000", 2024-09-18T11:32:40.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00027935600000000003 2024-09-18T11:32:40.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:40.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028868+0000", 2024-09-18T11:32:40.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9947999999999999e-05 2024-09-18T11:32:40.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:40.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037405+0000", 2024-09-18T11:32:40.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085369920000000002 2024-09-18T11:32:40.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:40.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037438+0000", 2024-09-18T11:32:40.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3244000000000002e-05 2024-09-18T11:32:40.451 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.451 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:40.451 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.452 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:154 4.3 e147/67 4:ca2b2309:::smithi11939506-44:5d v 147'216, mlcod=145'210)", 2024-09-18T11:32:40.452 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:40.452 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.016287182999999, 2024-09-18T11:32:40.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066667829999999999, 2024-09-18T11:32:40.452 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:40.452 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:40.452 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:40.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:40.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:40.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:40.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:40.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:40.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:40.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:40.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:40.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337335+0000", 2024-09-18T11:32:40.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1479999999999997e-06 2024-09-18T11:32:40.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:40.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337344+0000", 2024-09-18T11:32:40.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5640000000000002e-06 2024-09-18T11:32:40.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:40.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337349+0000", 2024-09-18T11:32:40.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9470000000000002e-06 2024-09-18T11:32:40.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:40.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337410+0000", 2024-09-18T11:32:40.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0448999999999997e-05 2024-09-18T11:32:40.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:40.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337425+0000", 2024-09-18T11:32:40.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.504e-05 2024-09-18T11:32:40.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:40.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.343972+0000", 2024-09-18T11:32:40.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065474169999999998 2024-09-18T11:32:40.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:40.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.343997+0000", 2024-09-18T11:32:40.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4672999999999999e-05 2024-09-18T11:32:40.456 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.456 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:40.456 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.456 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1238 4.6 e169/143 4:6f86ffcc:::smithi11939506-15:51 v 169'193, mlcod=166'187)", 2024-09-18T11:32:40.456 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.044089+0000", 2024-09-18T11:32:40.456 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.30952811400000002, 2024-09-18T11:32:40.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072870620000000004, 2024-09-18T11:32:40.457 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:40.457 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:40.457 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:40.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:40.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044089+0000", 2024-09-18T11:32:40.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:40.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044089+0000", 2024-09-18T11:32:40.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:40.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044089+0000", 2024-09-18T11:32:40.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:40.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:40.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044097+0000", 2024-09-18T11:32:40.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9060000000000002e-06 2024-09-18T11:32:40.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:40.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044105+0000", 2024-09-18T11:32:40.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3590000000000008e-06 2024-09-18T11:32:40.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:40.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044112+0000", 2024-09-18T11:32:40.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0639999999999996e-06 2024-09-18T11:32:40.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:40.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044851+0000", 2024-09-18T11:32:40.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00073857800000000004 2024-09-18T11:32:40.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:40.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044871+0000", 2024-09-18T11:32:40.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0140999999999999e-05 2024-09-18T11:32:40.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:40.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.051338+0000", 2024-09-18T11:32:40.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064673760000000004 2024-09-18T11:32:40.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:40.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.051376+0000", 2024-09-18T11:32:40.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8222000000000001e-05 2024-09-18T11:32:40.461 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.461 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:40.461 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.461 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:281 4.3 e169/67 4:ca2b2309:::smithi11939506-44:5d v 169'279, mlcod=168'273)", 2024-09-18T11:32:40.461 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.047201+0000", 2024-09-18T11:32:40.461 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.306415979, 2024-09-18T11:32:40.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007110346, 2024-09-18T11:32:40.461 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:40.461 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:40.461 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:40.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:40.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047201+0000", 2024-09-18T11:32:40.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:40.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047201+0000", 2024-09-18T11:32:40.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:40.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047201+0000", 2024-09-18T11:32:40.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:40.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:40.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047206+0000", 2024-09-18T11:32:40.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7309999999999998e-06 2024-09-18T11:32:40.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:40.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047217+0000", 2024-09-18T11:32:40.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0577000000000001e-05 2024-09-18T11:32:40.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:40.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047221+0000", 2024-09-18T11:32:40.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0509999999999998e-06 2024-09-18T11:32:40.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:40.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.048977+0000", 2024-09-18T11:32:40.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017558179999999999 2024-09-18T11:32:40.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:40.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.048999+0000", 2024-09-18T11:32:40.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.23e-05 2024-09-18T11:32:40.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:40.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054275+0000", 2024-09-18T11:32:40.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0052754150000000003 2024-09-18T11:32:40.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:40.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054312+0000", 2024-09-18T11:32:40.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7163999999999999e-05 2024-09-18T11:32:40.465 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.465 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:40.465 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.465 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:4518 4.7 e169/155 4:e0d95e10:::smithi11939506-22:head v 169'394, mlcod=169'389)", 2024-09-18T11:32:40.466 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.057943+0000", 2024-09-18T11:32:40.466 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.29567472900000003, 2024-09-18T11:32:40.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075088719999999998, 2024-09-18T11:32:40.466 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:40.466 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:40.466 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:40.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:40.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057943+0000", 2024-09-18T11:32:40.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:40.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057943+0000", 2024-09-18T11:32:40.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:40.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:40.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057942+0000", 2024-09-18T11:32:40.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:40.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:40.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057953+0000", 2024-09-18T11:32:40.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1399e-05 2024-09-18T11:32:40.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:40.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057969+0000", 2024-09-18T11:32:40.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5860999999999999e-05 2024-09-18T11:32:40.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:40.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057975+0000", 2024-09-18T11:32:40.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8610000000000003e-06 2024-09-18T11:32:40.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:40.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.058039+0000", 2024-09-18T11:32:40.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4313000000000006e-05 2024-09-18T11:32:40.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:40.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.058059+0000", 2024-09-18T11:32:40.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0205999999999999e-05 2024-09-18T11:32:40.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:40.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065410+0000", 2024-09-18T11:32:40.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073512020000000003 2024-09-18T11:32:40.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:40.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:40.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:40.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065451+0000", 2024-09-18T11:32:40.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1032999999999998e-05 2024-09-18T11:32:40.470 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.470 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:40.470 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.470 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:40.470 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:40.470 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:40.471 INFO:tasks.rados.rados.0.smithi119.stdout:1929: expect (ObjNum 309 snap 43 seq_num 309) 2024-09-18T11:32:40.472 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:32:40.472 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:40.470+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:40.569 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:40.569 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:40.569 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:40.569 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:40.573 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:40.571+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:40.577 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:32:40.609 INFO:tasks.rados.rados.0.smithi119.stdout:1930: expect (ObjNum 323 snap 45 seq_num 323) 2024-09-18T11:32:40.674 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:40.672+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:40.675 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:40.675 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:40.676 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:40.676 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:40.676 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:40.684 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:32:40.774 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:40.773+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:40.823 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:40.823 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:40.823 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:40.823 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [] 2024-09-18T11:32:40.823 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:40.831 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:32:40.849 INFO:tasks.rados.rados.0.smithi119.stdout:1934: expect (ObjNum 327 snap 45 seq_num 327) 2024-09-18T11:32:40.892 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:40.891+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:40.947 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:40.948 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:40.948 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:40.948 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:40.956 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:32:40.965 INFO:tasks.rados.rados.0.smithi119.stdout:1937: expect (ObjNum 381 snap 60 seq_num 381) 2024-09-18T11:32:40.976 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:40.974+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:41.054 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T11:32:41.063 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:41.064 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:41.064 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:41.064 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:41.064 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:41.072 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:32:41.101 INFO:tasks.rados.rados.0.smithi119.stdout:1938: finishing rollback tid 0 to smithi11939506-22 2024-09-18T11:32:41.101 INFO:tasks.rados.rados.0.smithi119.stdout:1939: expect (ObjNum 391 snap 63 seq_num 391) 2024-09-18T11:32:41.102 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:41.100+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:41.177 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:41.176+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:41.191 INFO:tasks.rados.rados.0.smithi119.stdout:1938: finishing rollback tid 1 to smithi11939506-22 2024-09-18T11:32:41.191 INFO:tasks.rados.rados.0.smithi119.stdout:1938: finishing rollback tid 2 to smithi11939506-22 2024-09-18T11:32:41.191 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 395 (ObjNum 298 snap 43 seq_num 298) dirty exists 2024-09-18T11:32:41.191 INFO:tasks.rados.rados.0.smithi119.stdout:1940: finishing write tid 1 to smithi11939506-18 2024-09-18T11:32:41.191 INFO:tasks.rados.rados.0.smithi119.stdout:1926: done (13 left) 2024-09-18T11:32:41.191 INFO:tasks.rados.rados.0.smithi119.stdout:1927: done (12 left) 2024-09-18T11:32:41.191 INFO:tasks.rados.rados.0.smithi119.stdout:1928: done (11 left) 2024-09-18T11:32:41.191 INFO:tasks.rados.rados.0.smithi119.stdout:1929: done (10 left) 2024-09-18T11:32:41.191 INFO:tasks.rados.rados.0.smithi119.stdout:1930: done (9 left) 2024-09-18T11:32:41.191 INFO:tasks.rados.rados.0.smithi119.stdout:1931: done (8 left) 2024-09-18T11:32:41.192 INFO:tasks.rados.rados.0.smithi119.stdout:1932: done (7 left) 2024-09-18T11:32:41.192 INFO:tasks.rados.rados.0.smithi119.stdout:1934: done (6 left) 2024-09-18T11:32:41.192 INFO:tasks.rados.rados.0.smithi119.stdout:1937: done (5 left) 2024-09-18T11:32:41.192 INFO:tasks.rados.rados.0.smithi119.stdout:1938: done (4 left) 2024-09-18T11:32:41.192 INFO:tasks.rados.rados.0.smithi119.stdout:1939: done (3 left) 2024-09-18T11:32:41.192 INFO:tasks.rados.rados.0.smithi119.stdout:1942: set_chunk oid 42 target oid 2024-09-18T11:32:41.203 INFO:tasks.rados.rados.0.smithi119.stdout:1942: set_chunk oid 42 offset: 255488 length: 1630208 target oid d10ff4fdc01068aacf9758eb4ab3f4ec3243e96093eefbe0c74f83c9e42791c6 offset: 0 snap -1 2024-09-18T11:32:41.203 INFO:tasks.rados.rados.0.smithi119.stdout:1943: read oid 48 snap -1 2024-09-18T11:32:41.203 INFO:tasks.rados.rados.0.smithi119.stdout:1943: expect deleted 2024-09-18T11:32:41.203 INFO:tasks.rados.rados.0.smithi119.stdout:1944: read oid 13 snap -1 2024-09-18T11:32:41.203 INFO:tasks.rados.rados.0.smithi119.stdout:1944: expect (ObjNum 309 snap 43 seq_num 309) 2024-09-18T11:32:41.203 INFO:tasks.rados.rados.0.smithi119.stdout:1945: read oid 17 snap -1 2024-09-18T11:32:41.203 INFO:tasks.rados.rados.0.smithi119.stdout:1945: expect (ObjNum 383 snap 60 seq_num 383) 2024-09-18T11:32:41.203 INFO:tasks.rados.rados.0.smithi119.stdout:1946: delete oid 26 current snap is 64 2024-09-18T11:32:41.203 INFO:tasks.rados.rados.0.smithi119.stdout:1936: finishing set_chunk to oid 38 2024-09-18T11:32:41.203 INFO:tasks.rados.rados.0.smithi119.stdout:1941: finishing rollback tid 1 to smithi11939506-36 2024-09-18T11:32:41.204 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 36 v 0 (ObjNum 129 snap 0 seq_num 384649328) dirty dne 2024-09-18T11:32:41.204 INFO:tasks.rados.rados.0.smithi119.stdout:1940: finishing write tid 2 to smithi11939506-18 2024-09-18T11:32:41.204 INFO:tasks.rados.rados.0.smithi119.stdout:1940: finishing write tid 3 to smithi11939506-18 2024-09-18T11:32:41.204 INFO:tasks.rados.rados.0.smithi119.stdout:1940: finishing write tid 4 to smithi11939506-18 2024-09-18T11:32:41.204 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 303 (ObjNum 395 snap 64 seq_num 395) dirty exists 2024-09-18T11:32:41.204 INFO:tasks.rados.rados.0.smithi119.stdout:1940: left oid 18 (ObjNum 395 snap 64 seq_num 395) 2024-09-18T11:32:41.205 INFO:tasks.rados.rados.0.smithi119.stdout:1936: done (7 left) 2024-09-18T11:32:41.205 INFO:tasks.rados.rados.0.smithi119.stdout:1940: done (6 left) 2024-09-18T11:32:41.205 INFO:tasks.rados.rados.0.smithi119.stdout:1941: done (5 left) 2024-09-18T11:32:41.205 INFO:tasks.rados.rados.0.smithi119.stdout:1943: done (4 left) 2024-09-18T11:32:41.205 INFO:tasks.rados.rados.0.smithi119.stdout:1946: done (3 left) 2024-09-18T11:32:41.205 INFO:tasks.rados.rados.0.smithi119.stdout:1947: tier_flush oid 2 2024-09-18T11:32:41.205 INFO:tasks.rados.rados.0.smithi119.stdout:1947: tier_flush oid 2 snap -1 2024-09-18T11:32:41.205 INFO:tasks.rados.rados.0.smithi119.stdout:1948: read oid 14 snap -1 2024-09-18T11:32:41.206 INFO:tasks.rados.rados.0.smithi119.stdout:1948: expect (ObjNum 392 snap 63 seq_num 392) 2024-09-18T11:32:41.206 INFO:tasks.rados.rados.0.smithi119.stdout:1949: read oid 15 snap 58 2024-09-18T11:32:41.206 INFO:tasks.rados.rados.0.smithi119.stdout:1949: expect (ObjNum 334 snap 48 seq_num 334) 2024-09-18T11:32:41.206 INFO:tasks.rados.rados.0.smithi119.stdout:1950: write (excl) oid 6 current snap is 64 2024-09-18T11:32:41.206 INFO:tasks.rados.rados.0.smithi119.stdout:1950: seq_num 397 ranges {672379=724209,1865116=608472} 2024-09-18T11:32:41.212 INFO:tasks.rados.rados.0.smithi119.stdout:1950: writing smithi11939506-6 from 672379 to 1396588 tid 1 2024-09-18T11:32:41.218 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:41.218 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:41.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:41.218 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:41.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:262 4.9 e53/22 4:99feffe9:::smithi11939506-45:13 v 53'17, mlcod=51'14)", 2024-09-18T11:32:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 127.21684672400001, 2024-09-18T11:32:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013878679, 2024-09-18T11:32:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:41.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:41.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:41.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:41.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:41.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:41.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998997+0000", 2024-09-18T11:32:41.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1099999999999999e-06 2024-09-18T11:32:41.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:41.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999008+0000", 2024-09-18T11:32:41.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1321000000000001e-05 2024-09-18T11:32:41.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:41.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999015+0000", 2024-09-18T11:32:41.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3000000000000004e-06 2024-09-18T11:32:41.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999055+0000", 2024-09-18T11:32:41.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0058000000000002e-05 2024-09-18T11:32:41.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:41.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999070+0000", 2024-09-18T11:32:41.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4328000000000001e-05 2024-09-18T11:32:41.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012465+0000", 2024-09-18T11:32:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013395638 2024-09-18T11:32:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012870+0000", 2024-09-18T11:32:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040423400000000001 2024-09-18T11:32:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.224 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:205 4.b e53/22 4:dba5bd39:::smithi11939506-17:13 v 53'77, mlcod=51'74)", 2024-09-18T11:32:41.224 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:41.224 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 127.21679746300001, 2024-09-18T11:32:41.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013359586, 2024-09-18T11:32:41.224 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:41.224 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:41.224 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:41.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:41.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:41.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999048+0000", 2024-09-18T11:32:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3550000000000005e-06 2024-09-18T11:32:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999056+0000", 2024-09-18T11:32:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3259999999999996e-06 2024-09-18T11:32:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999063+0000", 2024-09-18T11:32:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6409999999999996e-06 2024-09-18T11:32:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999098+0000", 2024-09-18T11:32:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5203999999999997e-05 2024-09-18T11:32:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999115+0000", 2024-09-18T11:32:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6478999999999999e-05 2024-09-18T11:32:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012362+0000", 2024-09-18T11:32:41.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01324768 2024-09-18T11:32:41.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:41.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012400+0000", 2024-09-18T11:32:41.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.731e-05 2024-09-18T11:32:41.228 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.228 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:41.228 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.228 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2937 4.7 4:e301aefd:::smithi11939506-19:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:41.228 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 78.180997149999996, 2024-09-18T11:32:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019287667000000001, 2024-09-18T11:32:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2937 2024-09-18T11:32:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:41.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:41.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:41.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:41.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.230 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:41.230 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:41.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:41.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034842+0000", 2024-09-18T11:32:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.372e-06 2024-09-18T11:32:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034846+0000", 2024-09-18T11:32:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3919999999999996e-06 2024-09-18T11:32:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034848+0000", 2024-09-18T11:32:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3539999999999998e-06 2024-09-18T11:32:41.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034893+0000", 2024-09-18T11:32:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4861000000000003e-05 2024-09-18T11:32:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034965+0000", 2024-09-18T11:32:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1860999999999995e-05 2024-09-18T11:32:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:41.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.054128+0000", 2024-09-18T11:32:41.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019163039 2024-09-18T11:32:41.233 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.233 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:41.233 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.233 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3453 4.f 4:f5d7df41:::smithi11939506-16:head [write 1813152~490434] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:41.233 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:32:41.233 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.193085222000001, 2024-09-18T11:32:41.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050685840000000003, 2024-09-18T11:32:41.233 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:41.233 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:41.233 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:41.234 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:41.234 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:41.234 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3453 2024-09-18T11:32:41.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.234 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:41.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:41.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:32:41.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:41.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:32:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022751+0000", 2024-09-18T11:32:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022871+0000", 2024-09-18T11:32:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000119781 2024-09-18T11:32:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022882+0000", 2024-09-18T11:32:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1627000000000001e-05 2024-09-18T11:32:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022887+0000", 2024-09-18T11:32:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2390000000000004e-06 2024-09-18T11:32:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023123+0000", 2024-09-18T11:32:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000236598 2024-09-18T11:32:41.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023207+0000", 2024-09-18T11:32:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4303999999999998e-05 2024-09-18T11:32:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023367+0000", 2024-09-18T11:32:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015983600000000001 2024-09-18T11:32:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073237+0000", 2024-09-18T11:32:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049869400000000001 2024-09-18T11:32:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073352+0000", 2024-09-18T11:32:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011579899999999999 2024-09-18T11:32:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073394+0000", 2024-09-18T11:32:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1445e-05 2024-09-18T11:32:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:41.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073438+0000", 2024-09-18T11:32:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4261999999999997e-05 2024-09-18T11:32:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3454 4.f 4:f5d7df41:::smithi11939506-16:head [setxattr _header (57) in=7b,truncate 2991679] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:32:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.19238678, 2024-09-18T11:32:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050365656000000002, 2024-09-18T11:32:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:41.239 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3454 2024-09-18T11:32:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:32:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:32:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023450+0000", 2024-09-18T11:32:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023456+0000", 2024-09-18T11:32:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8329999999999999e-06 2024-09-18T11:32:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023465+0000", 2024-09-18T11:32:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1130000000000007e-06 2024-09-18T11:32:41.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:41.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023470+0000", 2024-09-18T11:32:41.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9139999999999999e-06 2024-09-18T11:32:41.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072611+0000", 2024-09-18T11:32:41.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141643999999998 2024-09-18T11:32:41.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:41.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072729+0000", 2024-09-18T11:32:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117667 2024-09-18T11:32:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072887+0000", 2024-09-18T11:32:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015760000000000001 2024-09-18T11:32:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073558+0000", 2024-09-18T11:32:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00067171699999999995 2024-09-18T11:32:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073764+0000", 2024-09-18T11:32:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000205749 2024-09-18T11:32:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073790+0000", 2024-09-18T11:32:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5643000000000002e-05 2024-09-18T11:32:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073816+0000", 2024-09-18T11:32:41.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6588e-05 2024-09-18T11:32:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3455 4.f 4:f5d7df41:::smithi11939506-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.192357162, 2024-09-18T11:32:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050460466000000002, 2024-09-18T11:32:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:41.245 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:41.246 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:41.246 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3455 2024-09-18T11:32:41.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.246 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:41.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:41.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:41.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:41.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:41.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023483+0000", 2024-09-18T11:32:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.695e-06 2024-09-18T11:32:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:41.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023489+0000", 2024-09-18T11:32:41.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1079999999999998e-06 2024-09-18T11:32:41.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:41.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023492+0000", 2024-09-18T11:32:41.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.478e-06 2024-09-18T11:32:41.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.248 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073273+0000", 2024-09-18T11:32:41.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049780638000000002 2024-09-18T11:32:41.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:41.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073343+0000", 2024-09-18T11:32:41.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9770000000000005e-05 2024-09-18T11:32:41.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073846+0000", 2024-09-18T11:32:41.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050289499999999995 2024-09-18T11:32:41.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:41.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073886+0000", 2024-09-18T11:32:41.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0658999999999998e-05 2024-09-18T11:32:41.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:41.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073941+0000", 2024-09-18T11:32:41.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4642000000000003e-05 2024-09-18T11:32:41.250 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.250 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:41.250 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.250 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3456 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:41.250 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.192332096999998, 2024-09-18T11:32:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049859120999999999, 2024-09-18T11:32:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3456 2024-09-18T11:32:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:41.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023508+0000", 2024-09-18T11:32:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9490000000000001e-06 2024-09-18T11:32:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023513+0000", 2024-09-18T11:32:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3800000000000002e-06 2024-09-18T11:32:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023517+0000", 2024-09-18T11:32:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5300000000000001e-06 2024-09-18T11:32:41.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073262+0000", 2024-09-18T11:32:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049745194999999999 2024-09-18T11:32:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073365+0000", 2024-09-18T11:32:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102462 2024-09-18T11:32:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.254 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3457 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:32:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.192315000999997, 2024-09-18T11:32:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049928567, 2024-09-18T11:32:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3457 2024-09-18T11:32:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.255 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:41.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:32:41.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:41.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:32:41.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.256 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.256 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:41.256 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023522+0000", 2024-09-18T11:32:41.256 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023524+0000", 2024-09-18T11:32:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8500000000000001e-06 2024-09-18T11:32:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023529+0000", 2024-09-18T11:32:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7509999999999999e-06 2024-09-18T11:32:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:41.257 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023532+0000", 2024-09-18T11:32:41.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.163e-06 2024-09-18T11:32:41.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073386+0000", 2024-09-18T11:32:41.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049853899 2024-09-18T11:32:41.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.258 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.258 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:41.258 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073451+0000", 2024-09-18T11:32:41.258 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5197000000000006e-05 2024-09-18T11:32:41.258 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.258 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:41.259 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.259 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.259 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3458 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:41.259 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:41.259 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.192302443999999, 2024-09-18T11:32:41.259 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.05001659, 2024-09-18T11:32:41.259 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:41.259 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:41.259 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:41.259 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:41.259 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:41.259 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3458 2024-09-18T11:32:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.260 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023537+0000", 2024-09-18T11:32:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.64e-06 2024-09-18T11:32:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023543+0000", 2024-09-18T11:32:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3369999999999996e-06 2024-09-18T11:32:41.261 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:41.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023546+0000", 2024-09-18T11:32:41.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:32:41.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073466+0000", 2024-09-18T11:32:41.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049919719000000001 2024-09-18T11:32:41.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.262 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:41.262 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073552+0000", 2024-09-18T11:32:41.262 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6168999999999994e-05 2024-09-18T11:32:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(client.4590.0:3453 4.f e116/22 ondisk, result = 0)", 2024-09-18T11:32:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:32:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.190697360000001, 2024-09-18T11:32:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048099441, 2024-09-18T11:32:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.263 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:41.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:32:41.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:41.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:32:41.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.264 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.264 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:41.264 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025139+0000", 2024-09-18T11:32:41.264 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:41.264 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025144+0000", 2024-09-18T11:32:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4059999999999998e-06 2024-09-18T11:32:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072212+0000", 2024-09-18T11:32:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047068482000000002 2024-09-18T11:32:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072230+0000", 2024-09-18T11:32:41.265 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7632999999999999e-05 2024-09-18T11:32:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073224+0000", 2024-09-18T11:32:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099436899999999994 2024-09-18T11:32:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073231+0000", 2024-09-18T11:32:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-09-18T11:32:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:41.266 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073240+0000", 2024-09-18T11:32:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5699999999999993e-06 2024-09-18T11:32:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3930 4.b 4:d3ae8215:::smithi11939506-41:head [write 632029~547288] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:32:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.189781680000003, 2024-09-18T11:32:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018962711, 2024-09-18T11:32:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:41.267 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:41.268 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:41.268 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3930 2024-09-18T11:32:41.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.268 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:41.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:41.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:32:41.268 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.268 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.268 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:41.268 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:32:41.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:41.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026055+0000", 2024-09-18T11:32:41.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:41.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.269 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:41.269 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026334+0000", 2024-09-18T11:32:41.269 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000278931 2024-09-18T11:32:41.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.269 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026344+0000", 2024-09-18T11:32:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0716999999999999e-05 2024-09-18T11:32:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026350+0000", 2024-09-18T11:32:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7620000000000001e-06 2024-09-18T11:32:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028429+0000", 2024-09-18T11:32:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020787319999999998 2024-09-18T11:32:41.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028525+0000", 2024-09-18T11:32:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6638999999999994e-05 2024-09-18T11:32:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028720+0000", 2024-09-18T11:32:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000194525 2024-09-18T11:32:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.036996+0000", 2024-09-18T11:32:41.271 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082756140000000006 2024-09-18T11:32:41.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:41.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044952+0000", 2024-09-18T11:32:41.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566740000000004 2024-09-18T11:32:41.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:41.272 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044985+0000", 2024-09-18T11:32:41.272 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3081999999999999e-05 2024-09-18T11:32:41.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.272 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.272 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045019+0000", 2024-09-18T11:32:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3257999999999999e-05 2024-09-18T11:32:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3932 4.b 4:d3ae8215:::smithi11939506-41:head [write 2796249~437859] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:32:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.17777504, 2024-09-18T11:32:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01458329, 2024-09-18T11:32:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:41.273 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:41.274 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:41.274 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:41.274 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3932 2024-09-18T11:32:41.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.274 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:41.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:41.274 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:32:41.274 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.274 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.274 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.274 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:32:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038061+0000", 2024-09-18T11:32:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038341+0000", 2024-09-18T11:32:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027989099999999999 2024-09-18T11:32:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.275 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038353+0000", 2024-09-18T11:32:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2247e-05 2024-09-18T11:32:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038358+0000", 2024-09-18T11:32:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-18T11:32:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043226+0000", 2024-09-18T11:32:41.276 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048673049999999997 2024-09-18T11:32:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043284+0000", 2024-09-18T11:32:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8338999999999998e-05 2024-09-18T11:32:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043367+0000", 2024-09-18T11:32:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3120000000000004e-05 2024-09-18T11:32:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.277 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:41.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.051544+0000", 2024-09-18T11:32:41.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00817734 2024-09-18T11:32:41.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:41.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052528+0000", 2024-09-18T11:32:41.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098318499999999992 2024-09-18T11:32:41.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.278 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.278 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:41.278 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052610+0000", 2024-09-18T11:32:41.278 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2300999999999997e-05 2024-09-18T11:32:41.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.279 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:41.279 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052646+0000", 2024-09-18T11:32:41.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5955999999999999e-05 2024-09-18T11:32:41.279 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.279 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:41.279 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.279 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3933 4.b 4:d3ae8215:::smithi11939506-41:head [write 3653524~178505] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:41.279 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041300+0000", 2024-09-18T11:32:41.279 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.174537143999999, 2024-09-18T11:32:41.279 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01299752, 2024-09-18T11:32:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3933 2024-09-18T11:32:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041300+0000", 2024-09-18T11:32:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041300+0000", 2024-09-18T11:32:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041299+0000", 2024-09-18T11:32:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:41.281 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041369+0000", 2024-09-18T11:32:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9838000000000005e-05 2024-09-18T11:32:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041380+0000", 2024-09-18T11:32:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0910000000000001e-05 2024-09-18T11:32:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041385+0000", 2024-09-18T11:32:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5800000000000002e-06 2024-09-18T11:32:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.282 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044260+0000", 2024-09-18T11:32:41.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028751229999999998 2024-09-18T11:32:41.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:41.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044312+0000", 2024-09-18T11:32:41.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1808000000000003e-05 2024-09-18T11:32:41.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.283 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.283 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:41.283 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044393+0000", 2024-09-18T11:32:41.283 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1806999999999998e-05 2024-09-18T11:32:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052654+0000", 2024-09-18T11:32:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082609320000000003 2024-09-18T11:32:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054207+0000", 2024-09-18T11:32:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001552091 2024-09-18T11:32:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.284 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054256+0000", 2024-09-18T11:32:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9437000000000001e-05 2024-09-18T11:32:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054298+0000", 2024-09-18T11:32:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1993000000000003e-05 2024-09-18T11:32:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3934 4.b 4:d3ae8215:::smithi11939506-41:head [setxattr _header (57) in=7b,truncate 3832029] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:41.285 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.174448523000002, 2024-09-18T11:32:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013018336, 2024-09-18T11:32:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3934 2024-09-18T11:32:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:41.286 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:41.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:41.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:41.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.287 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:41.287 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:41.287 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:41.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.287 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041391+0000", 2024-09-18T11:32:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.26e-06 2024-09-18T11:32:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041395+0000", 2024-09-18T11:32:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8999999999999999e-06 2024-09-18T11:32:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041397+0000", 2024-09-18T11:32:41.288 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3149999999999999e-06 2024-09-18T11:32:41.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045266+0000", 2024-09-18T11:32:41.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003869094 2024-09-18T11:32:41.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.289 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:41.289 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045316+0000", 2024-09-18T11:32:41.289 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0074000000000002e-05 2024-09-18T11:32:41.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.289 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:41.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045403+0000", 2024-09-18T11:32:41.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6737999999999995e-05 2024-09-18T11:32:41.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:41.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052659+0000", 2024-09-18T11:32:41.290 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072558259999999999 2024-09-18T11:32:41.290 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.290 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.290 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:41.290 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054330+0000", 2024-09-18T11:32:41.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016716000000000001 2024-09-18T11:32:41.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:41.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054362+0000", 2024-09-18T11:32:41.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1195000000000002e-05 2024-09-18T11:32:41.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.291 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:41.291 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054407+0000", 2024-09-18T11:32:41.291 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5707000000000001e-05 2024-09-18T11:32:41.292 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.292 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:41.292 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.292 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.292 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.292 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3935 4.b 4:d3ae8215:::smithi11939506-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:41.292 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:32:41.292 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.174437933, 2024-09-18T11:32:41.292 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013142333000000001, 2024-09-18T11:32:41.292 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:41.292 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:41.292 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:41.293 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:41.293 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:41.293 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3935 2024-09-18T11:32:41.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.293 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:41.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:41.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:32:41.293 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.293 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.293 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:41.293 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:32:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041399+0000", 2024-09-18T11:32:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041401+0000", 2024-09-18T11:32:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4890000000000001e-06 2024-09-18T11:32:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.294 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041404+0000", 2024-09-18T11:32:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5240000000000001e-06 2024-09-18T11:32:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041407+0000", 2024-09-18T11:32:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2230000000000001e-06 2024-09-18T11:32:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045699+0000", 2024-09-18T11:32:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042918970000000002 2024-09-18T11:32:41.295 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045750+0000", 2024-09-18T11:32:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1332999999999998e-05 2024-09-18T11:32:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054438+0000", 2024-09-18T11:32:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086882569999999996 2024-09-18T11:32:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:41.296 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054485+0000", 2024-09-18T11:32:41.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7151000000000003e-05 2024-09-18T11:32:41.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.297 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:41.297 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054542+0000", 2024-09-18T11:32:41.297 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6706999999999998e-05 2024-09-18T11:32:41.297 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.297 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:41.297 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.297 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.297 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3969 4.4 4:2cb3e6a5:::smithi11939506-3:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:41.297 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:32:41.298 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.642197668000001, 2024-09-18T11:32:41.298 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.024248042000000001, 2024-09-18T11:32:41.298 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:41.298 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:41.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:41.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:41.298 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:41.298 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3969 2024-09-18T11:32:41.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.298 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:41.298 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:32:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:32:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573639+0000", 2024-09-18T11:32:41.299 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573646+0000", 2024-09-18T11:32:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6569999999999997e-06 2024-09-18T11:32:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573662+0000", 2024-09-18T11:32:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5187999999999999e-05 2024-09-18T11:32:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.300 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:41.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573670+0000", 2024-09-18T11:32:41.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5399999999999996e-06 2024-09-18T11:32:41.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:41.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8592999999999999e-05 2024-09-18T11:32:41.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.301 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.301 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:41.301 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573818+0000", 2024-09-18T11:32:41.301 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9388999999999995e-05 2024-09-18T11:32:41.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.302 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:41.302 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597888+0000", 2024-09-18T11:32:41.302 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.024069859999999998 2024-09-18T11:32:41.302 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.302 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:41.302 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.302 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.302 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3973 4.7 4:e73ba68f:::smithi11939506-1:head [tier-promote] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:41.304 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:41.304 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.642098589, 2024-09-18T11:32:41.304 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.023911297000000001, 2024-09-18T11:32:41.304 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:41.304 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:41.304 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:41.304 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:41.304 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:41.304 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3973 2024-09-18T11:32:41.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.304 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:41.305 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573738+0000", 2024-09-18T11:32:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573743+0000", 2024-09-18T11:32:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0839999999999999e-06 2024-09-18T11:32:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573752+0000", 2024-09-18T11:32:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3080000000000008e-06 2024-09-18T11:32:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.306 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:41.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573758+0000", 2024-09-18T11:32:41.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2659999999999998e-06 2024-09-18T11:32:41.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573803+0000", 2024-09-18T11:32:41.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5243e-05 2024-09-18T11:32:41.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.307 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.307 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:41.307 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573897+0000", 2024-09-18T11:32:41.307 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3398000000000003e-05 2024-09-18T11:32:41.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.308 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:41.308 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597650+0000", 2024-09-18T11:32:41.308 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.023753672999999999 2024-09-18T11:32:41.308 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.308 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:41.308 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.308 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4443 4.d 4:b9bb642b:::smithi11939506-27:67 [read 0~3512696,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T11:32:41.308 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:36.030743+0000", 2024-09-18T11:32:41.309 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.1850949599999998, 2024-09-18T11:32:41.309 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013696302, 2024-09-18T11:32:41.309 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:41.309 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:41.309 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:41.309 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:41.309 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:41.309 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4443 2024-09-18T11:32:41.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.309 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:41.309 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.309 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030743+0000", 2024-09-18T11:32:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030743+0000", 2024-09-18T11:32:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030741+0000", 2024-09-18T11:32:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:41.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:41.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030751+0000", 2024-09-18T11:32:41.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6979999999999993e-06 2024-09-18T11:32:41.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:41.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030763+0000", 2024-09-18T11:32:41.311 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2311e-05 2024-09-18T11:32:41.311 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.311 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.311 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:41.311 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030769+0000", 2024-09-18T11:32:41.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6060000000000002e-06 2024-09-18T11:32:41.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.036777+0000", 2024-09-18T11:32:41.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060079089999999996 2024-09-18T11:32:41.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.312 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.312 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:41.312 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.036945+0000", 2024-09-18T11:32:41.312 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016867100000000001 2024-09-18T11:32:41.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.313 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:41.313 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044439+0000", 2024-09-18T11:32:41.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074937149999999998 2024-09-18T11:32:41.313 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.313 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:41.313 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.313 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.313 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4444 4.d 4:b9bb642b:::smithi11939506-27:67 [sparse-read 0~1262403] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T11:32:41.313 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:32:41.313 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.1850601669999996, 2024-09-18T11:32:41.313 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.014179987, 2024-09-18T11:32:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4444 2024-09-18T11:32:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:32:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:41.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:32:41.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:41.315 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:32:41.315 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:41.315 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.315 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.315 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:41.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030780+0000", 2024-09-18T11:32:41.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.163e-06 2024-09-18T11:32:41.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:41.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030786+0000", 2024-09-18T11:32:41.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879999999999998e-06 2024-09-18T11:32:41.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.316 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.316 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:41.316 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030791+0000", 2024-09-18T11:32:41.316 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2579999999999997e-06 2024-09-18T11:32:41.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044477+0000", 2024-09-18T11:32:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013686188 2024-09-18T11:32:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044614+0000", 2024-09-18T11:32:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013727300000000001 2024-09-18T11:32:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:41.317 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044957+0000", 2024-09-18T11:32:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000343331 2024-09-18T11:32:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4445 4.d 4:b9bb642b:::smithi11939506-27:67 [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T11:32:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:36.030796+0000", 2024-09-18T11:32:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.185041891, 2024-09-18T11:32:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.015916046999999999, 2024-09-18T11:32:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:41.318 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:41.319 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:41.319 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4445 2024-09-18T11:32:41.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.319 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:41.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:41.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030796+0000", 2024-09-18T11:32:41.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.319 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.319 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:41.319 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030796+0000", 2024-09-18T11:32:41.319 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030795+0000", 2024-09-18T11:32:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030799+0000", 2024-09-18T11:32:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5209999999999998e-06 2024-09-18T11:32:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.320 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:41.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030806+0000", 2024-09-18T11:32:41.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7599999999999997e-06 2024-09-18T11:32:41.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:41.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030810+0000", 2024-09-18T11:32:41.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0489999999999997e-06 2024-09-18T11:32:41.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.321 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.321 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:41.321 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.045277+0000", 2024-09-18T11:32:41.321 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.014467683 2024-09-18T11:32:41.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:41.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.045394+0000", 2024-09-18T11:32:41.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116969 2024-09-18T11:32:41.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:41.322 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:41.322 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:41.322 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.046712+0000", 2024-09-18T11:32:41.322 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013174669999999999 2024-09-18T11:32:41.322 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.322 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:41.322 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.323 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:41.323 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:41.323 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:41.323 INFO:tasks.rados.rados.0.smithi119.stdout:1950: writing smithi11939506-6 from 1865116 to 2473588 tid 2 2024-09-18T11:32:41.323 INFO:tasks.rados.rados.0.smithi119.stdout:1951: set_chunk oid 22 target oid 2024-09-18T11:32:41.324 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:32:41.326 INFO:tasks.rados.rados.0.smithi119.stdout:1951: set_chunk oid 22 offset: 845824 length: 1218048 target oid 087f01bdf234032bd85edda653ea76eacea4b4187a7e27f890f16dc46cdce601 offset: 0 snap -1 2024-09-18T11:32:41.326 INFO:tasks.rados.rados.0.smithi119.stdout:1952: write (excl) oid 5 current snap is 64 2024-09-18T11:32:41.326 INFO:tasks.rados.rados.0.smithi119.stdout:1942: finishing set_chunk to oid 42 2024-09-18T11:32:41.326 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 42 v 368 (ObjNum 386 snap 61 seq_num 386) dirty exists 2024-09-18T11:32:41.326 INFO:tasks.rados.rados.0.smithi119.stdout:1952: seq_num 398 ranges {689186=799458,1923598=752130} 2024-09-18T11:32:41.326 INFO:tasks.rados.rados.0.smithi119.stdout:1952: writing smithi11939506-5 from 689186 to 1488644 tid 1 2024-09-18T11:32:41.326 INFO:tasks.rados.rados.0.smithi119.stdout:1952: writing smithi11939506-5 from 1923598 to 2675728 tid 2 2024-09-18T11:32:41.326 INFO:tasks.rados.rados.0.smithi119.stdout:1942: done (8 left) 2024-09-18T11:32:41.326 INFO:tasks.rados.rados.0.smithi119.stdout:1953: set_chunk oid 49 target oid 2024-09-18T11:32:41.326 INFO:tasks.rados.rados.0.smithi119.stdout:1953: set_chunk oid 49 offset: 2135552 length: 147968 target oid 6a44260ec049d9c6f7706a6bccdfc4bc235376e13879175bba1119235472347e offset: 0 snap 62 2024-09-18T11:32:41.326 INFO:tasks.rados.rados.0.smithi119.stdout:1954: snap_create 2024-09-18T11:32:41.327 INFO:tasks.rados.rados.0.smithi119.stdout:1949: expect (ObjNum 334 snap 48 seq_num 334) 2024-09-18T11:32:41.327 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:32:41.328 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:41.326+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:41.420 INFO:tasks.rados.rados.0.smithi119.stdout:1944: expect (ObjNum 309 snap 43 seq_num 309) 2024-09-18T11:32:41.426 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is unset 2024-09-18T11:32:41.428 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:41.426+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:41.468 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:41.468 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:41.468 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:41.469 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:41.480 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:32:41.488 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:41.489 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:41.506 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:41.506 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:41.522 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:41.522 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:41.529 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:41.528+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:41.538 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:41.553 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:41.553 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:41.569 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:41.569 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:41.585 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:41.585 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:41.589 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:41.590 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:41.598 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:32:41.600 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:41.601 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:41.635 INFO:tasks.rados.rados.0.smithi119.stdout:1948: expect (ObjNum 392 snap 63 seq_num 392) 2024-09-18T11:32:41.635 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:41.634+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:41.700 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:41.700 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:41.709 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:32:41.750 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:41.749+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:41.833 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:41.831+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:41.848 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:41.848 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:41.856 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:32:41.884 INFO:tasks.rados.rados.0.smithi119.stdout:1950: finishing write tid 1 to smithi11939506-6 2024-09-18T11:32:41.884 INFO:tasks.rados.rados.0.smithi119.stdout:1945: expect (ObjNum 383 snap 60 seq_num 383) 2024-09-18T11:32:41.934 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:41.932+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:41.958 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:41.958 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:41.967 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:32:42.034 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:42.033+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:42.064 INFO:tasks.rados.rados.0.smithi119.stdout:1947: got (0) Success 2024-09-18T11:32:42.065 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 2 v 537 (ObjNum 375 snap 58 seq_num 375) dirty exists 2024-09-18T11:32:42.065 INFO:tasks.rados.rados.0.smithi119.stdout:1950: finishing write tid 2 to smithi11939506-6 2024-09-18T11:32:42.065 INFO:tasks.rados.rados.0.smithi119.stdout:1950: finishing write tid 3 to smithi11939506-6 2024-09-18T11:32:42.065 INFO:tasks.rados.rados.0.smithi119.stdout:1950: finishing write tid 4 to smithi11939506-6 2024-09-18T11:32:42.065 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 6 v 208 (ObjNum 397 snap 64 seq_num 397) dirty exists 2024-09-18T11:32:42.065 INFO:tasks.rados.rados.0.smithi119.stdout:1950: left oid 6 (ObjNum 397 snap 64 seq_num 397) 2024-09-18T11:32:42.065 INFO:tasks.rados.rados.0.smithi119.stdout:1951: finishing set_chunk to oid 22 2024-09-18T11:32:42.065 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 395 (ObjNum 298 snap 43 seq_num 298) dirty exists 2024-09-18T11:32:42.065 INFO:tasks.rados.rados.0.smithi119.stdout:1952: finishing write tid 1 to smithi11939506-5 2024-09-18T11:32:42.065 INFO:tasks.rados.rados.0.smithi119.stdout:1952: finishing write tid 2 to smithi11939506-5 2024-09-18T11:32:42.065 INFO:tasks.rados.rados.0.smithi119.stdout:1952: finishing write tid 3 to smithi11939506-5 2024-09-18T11:32:42.065 INFO:tasks.rados.rados.0.smithi119.stdout:1952: finishing write tid 4 to smithi11939506-5 2024-09-18T11:32:42.065 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 235 (ObjNum 398 snap 64 seq_num 398) dirty exists 2024-09-18T11:32:42.066 INFO:tasks.rados.rados.0.smithi119.stdout:1952: left oid 5 (ObjNum 398 snap 64 seq_num 398) 2024-09-18T11:32:42.066 INFO:tasks.rados.rados.0.smithi119.stdout:1953: finishing set_chunk to oid 49 2024-09-18T11:32:42.066 INFO:tasks.rados.rados.0.smithi119.stdout:1944: done (9 left) 2024-09-18T11:32:42.066 INFO:tasks.rados.rados.0.smithi119.stdout:1945: done (8 left) 2024-09-18T11:32:42.066 INFO:tasks.rados.rados.0.smithi119.stdout:1947: done (7 left) 2024-09-18T11:32:42.066 INFO:tasks.rados.rados.0.smithi119.stdout:1948: done (6 left) 2024-09-18T11:32:42.066 INFO:tasks.rados.rados.0.smithi119.stdout:1949: done (5 left) 2024-09-18T11:32:42.066 INFO:tasks.rados.rados.0.smithi119.stdout:1950: done (4 left) 2024-09-18T11:32:42.066 INFO:tasks.rados.rados.0.smithi119.stdout:1951: done (3 left) 2024-09-18T11:32:42.066 INFO:tasks.rados.rados.0.smithi119.stdout:1952: done (2 left) 2024-09-18T11:32:42.066 INFO:tasks.rados.rados.0.smithi119.stdout:1953: done (1 left) 2024-09-18T11:32:42.066 INFO:tasks.rados.rados.0.smithi119.stdout:1954: done (0 left) 2024-09-18T11:32:42.066 INFO:tasks.rados.rados.0.smithi119.stdout:1955: write oid 31 current snap is 65 2024-09-18T11:32:42.067 INFO:tasks.rados.rados.0.smithi119.stdout:1955: seq_num 399 ranges {705993=474707,1582080=612141} 2024-09-18T11:32:42.070 INFO:tasks.rados.rados.0.smithi119.stdout:1955: writing smithi11939506-31 from 705993 to 1180700 tid 1 2024-09-18T11:32:42.077 INFO:tasks.rados.rados.0.smithi119.stdout:1955: writing smithi11939506-31 from 1582080 to 2194221 tid 2 2024-09-18T11:32:42.078 INFO:tasks.rados.rados.0.smithi119.stdout:1956: write (excl) oid 11 current snap is 65 2024-09-18T11:32:42.078 INFO:tasks.rados.rados.0.smithi119.stdout:1955: finishing write tid 1 to smithi11939506-31 2024-09-18T11:32:42.078 INFO:tasks.rados.rados.0.smithi119.stdout:1956: seq_num 400 ranges {722800=549956,2040562=682238} 2024-09-18T11:32:42.085 INFO:tasks.rados.rados.0.smithi119.stdout:1956: writing smithi11939506-11 from 722800 to 1272756 tid 1 2024-09-18T11:32:42.095 INFO:tasks.rados.rados.0.smithi119.stdout:1956: writing smithi11939506-11 from 2040562 to 2722800 tid 2 2024-09-18T11:32:42.095 INFO:tasks.rados.rados.0.smithi119.stdout:1957: tier_evict oid 15 2024-09-18T11:32:42.095 INFO:tasks.rados.rados.0.smithi119.stdout:1957: tier_evict oid 15 snap -1 2024-09-18T11:32:42.096 INFO:tasks.rados.rados.0.smithi119.stdout:1955: finishing write tid 2 to smithi11939506-31 2024-09-18T11:32:42.096 INFO:tasks.rados.rados.0.smithi119.stdout:1958: snap_create 2024-09-18T11:32:42.096 INFO:tasks.rados.rados.0.smithi119.stdout:1955: finishing write tid 3 to smithi11939506-31 2024-09-18T11:32:42.096 INFO:tasks.rados.rados.0.smithi119.stdout:1955: finishing write tid 4 to smithi11939506-31 2024-09-18T11:32:42.096 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 412 (ObjNum 399 snap 65 seq_num 399) dirty exists 2024-09-18T11:32:42.096 INFO:tasks.rados.rados.0.smithi119.stdout:1955: left oid 31 (ObjNum 399 snap 65 seq_num 399) 2024-09-18T11:32:42.096 INFO:tasks.rados.rados.0.smithi119.stdout:1956: finishing write tid 1 to smithi11939506-11 2024-09-18T11:32:42.096 INFO:tasks.rados.rados.0.smithi119.stdout:1957: got (22) Invalid argument 2024-09-18T11:32:42.099 INFO:tasks.rados.rados.0.smithi119.stdout:1956: finishing write tid 2 to smithi11939506-11 2024-09-18T11:32:42.099 INFO:tasks.rados.rados.0.smithi119.stdout:1956: finishing write tid 3 to smithi11939506-11 2024-09-18T11:32:42.099 INFO:tasks.rados.rados.0.smithi119.stdout:1956: finishing write tid 4 to smithi11939506-11 2024-09-18T11:32:42.099 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 11 v 393 (ObjNum 400 snap 65 seq_num 400) dirty exists 2024-09-18T11:32:42.099 INFO:tasks.rados.rados.0.smithi119.stdout:1956: left oid 11 (ObjNum 400 snap 65 seq_num 400) 2024-09-18T11:32:42.115 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:42.115 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:42.124 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:32:42.168 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:42.166+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:42.227 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:42.227 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:42.236 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:32:42.237 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:42.235+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:42.337 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:42.336+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:42.374 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:42.374 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:42.383 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:32:42.437 INFO:tasks.rados.rados.0.smithi119.stdout:1955: done (3 left) 2024-09-18T11:32:42.437 INFO:tasks.rados.rados.0.smithi119.stdout:1956: done (2 left) 2024-09-18T11:32:42.437 INFO:tasks.rados.rados.0.smithi119.stdout:1957: done (1 left) 2024-09-18T11:32:42.437 INFO:tasks.rados.rados.0.smithi119.stdout:1958: done (0 left) 2024-09-18T11:32:42.437 INFO:tasks.rados.rados.0.smithi119.stdout:1959: write oid 37 current snap is 66 2024-09-18T11:32:42.437 INFO:tasks.rados.rados.0.smithi119.stdout:1959: seq_num 401 ranges {739607=625205,1982691=499457} 2024-09-18T11:32:42.438 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:42.435+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:42.444 INFO:tasks.rados.rados.0.smithi119.stdout:1959: writing smithi11939506-37 from 739607 to 1364812 tid 1 2024-09-18T11:32:42.445 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:42.443+0000 7fd2785a3640 -1 osd.6 170 handle_osd_map injecting map crc failure 2024-09-18T11:32:42.450 INFO:tasks.rados.rados.0.smithi119.stdout:1959: writing smithi11939506-37 from 1982691 to 2482148 tid 2 2024-09-18T11:32:42.450 INFO:tasks.rados.rados.0.smithi119.stdout:1960: write (excl) oid 13 current snap is 66 2024-09-18T11:32:42.450 INFO:tasks.rados.rados.0.smithi119.stdout:1960: seq_num 402 ranges {756414=700454,2041173=715241} 2024-09-18T11:32:42.457 INFO:tasks.rados.rados.0.smithi119.stdout:1960: writing smithi11939506-13 from 756414 to 1456868 tid 1 2024-09-18T11:32:42.465 INFO:tasks.rados.rados.0.smithi119.stdout:1960: writing smithi11939506-13 from 2041173 to 2756414 tid 2 2024-09-18T11:32:42.465 INFO:tasks.rados.rados.0.smithi119.stdout:1961: tier_promote oid 26 2024-09-18T11:32:42.465 INFO:tasks.rados.rados.0.smithi119.stdout:1959: finishing write tid 1 to smithi11939506-37 2024-09-18T11:32:42.465 INFO:tasks.rados.rados.0.smithi119.stdout:1962: read oid 18 snap 38 2024-09-18T11:32:42.466 INFO:tasks.rados.rados.0.smithi119.stdout:1962: expect (ObjNum 205 snap 23 seq_num 205) 2024-09-18T11:32:42.466 INFO:tasks.rados.rados.0.smithi119.stdout:1963: set_chunk oid 22 target oid 2024-09-18T11:32:42.477 INFO:tasks.rados.rados.0.smithi119.stdout:1963: set_chunk oid 22 offset: 2063360 length: 503808 target oid 0df6897cdaeb57779ec42c5c037bc212a93d93f7eda2d0e902a1ee6aa5083fe7 offset: 0 snap -1 2024-09-18T11:32:42.477 INFO:tasks.rados.rados.0.smithi119.stdout:1964: snap_create 2024-09-18T11:32:42.477 INFO:tasks.rados.rados.0.smithi119.stdout:1959: finishing write tid 2 to smithi11939506-37 2024-09-18T11:32:42.477 INFO:tasks.rados.rados.0.smithi119.stdout:1959: finishing write tid 3 to smithi11939506-37 2024-09-18T11:32:42.477 INFO:tasks.rados.rados.0.smithi119.stdout:1959: finishing write tid 4 to smithi11939506-37 2024-09-18T11:32:42.477 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 37 v 396 (ObjNum 401 snap 66 seq_num 401) dirty exists 2024-09-18T11:32:42.477 INFO:tasks.rados.rados.0.smithi119.stdout:1959: left oid 37 (ObjNum 401 snap 66 seq_num 401) 2024-09-18T11:32:42.477 INFO:tasks.rados.rados.0.smithi119.stdout:1960: finishing write tid 1 to smithi11939506-13 2024-09-18T11:32:42.477 INFO:tasks.rados.rados.0.smithi119.stdout:1961: got (0) Success 2024-09-18T11:32:42.477 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 26 v 378 (ObjNum 177 snap 0 seq_num 384539536) clean dne 2024-09-18T11:32:42.477 INFO:tasks.rados.rados.0.smithi119.stdout:1960: finishing write tid 2 to smithi11939506-13 2024-09-18T11:32:42.478 INFO:tasks.rados.rados.0.smithi119.stdout:1960: finishing write tid 3 to smithi11939506-13 2024-09-18T11:32:42.478 INFO:tasks.rados.rados.0.smithi119.stdout:1960: finishing write tid 4 to smithi11939506-13 2024-09-18T11:32:42.478 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 13 v 592 (ObjNum 402 snap 66 seq_num 402) dirty exists 2024-09-18T11:32:42.478 INFO:tasks.rados.rados.0.smithi119.stdout:1960: left oid 13 (ObjNum 402 snap 66 seq_num 402) 2024-09-18T11:32:42.478 INFO:tasks.rados.rados.0.smithi119.stdout:1962: expect (ObjNum 205 snap 23 seq_num 205) 2024-09-18T11:32:42.489 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:42.489 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:42.498 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:32:42.538 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:42.536+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:42.638 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:42.638 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:42.638 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:42.636+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:42.646 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:32:42.660 INFO:tasks.rados.rados.0.smithi119.stdout:1963: finishing set_chunk to oid 22 2024-09-18T11:32:42.660 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 22 set_chunk returned error code -95 offset: 2063360 length: 503808 2024-09-18T11:32:42.660 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 22 v 0 (ObjNum 298 snap 43 seq_num 298) dirty exists 2024-09-18T11:32:42.739 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:42.737+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:42.784 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:42.784 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:42.792 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:32:42.840 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:42.839+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:42.930 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:42.931 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:42.940 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:32:42.941 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:42.940+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:43.026 INFO:tasks.rados.rados.0.smithi119.stdout:1959: done (5 left) 2024-09-18T11:32:43.026 INFO:tasks.rados.rados.0.smithi119.stdout:1960: done (4 left) 2024-09-18T11:32:43.026 INFO:tasks.rados.rados.0.smithi119.stdout:1961: done (3 left) 2024-09-18T11:32:43.026 INFO:tasks.rados.rados.0.smithi119.stdout:1962: done (2 left) 2024-09-18T11:32:43.027 INFO:tasks.rados.rados.0.smithi119.stdout:1963: done (1 left) 2024-09-18T11:32:43.027 INFO:tasks.rados.rados.0.smithi119.stdout:1964: done (0 left) 2024-09-18T11:32:43.027 INFO:tasks.rados.rados.0.smithi119.stdout:1965: tier_promote oid 27 2024-09-18T11:32:43.027 INFO:tasks.rados.rados.0.smithi119.stdout:1966: set_chunk oid 48 target oid 2024-09-18T11:32:43.027 INFO:tasks.rados.rados.0.smithi119.stdout:1966: set_chunk oid 48 offset: 3074048 length: 640512 target oid smithi11939506- offset: 3074048 snap 62 2024-09-18T11:32:43.027 INFO:tasks.rados.rados.0.smithi119.stdout:1967: write (excl) oid 4 current snap is 67 2024-09-18T11:32:43.027 INFO:tasks.rados.rados.0.smithi119.stdout:1967: seq_num 403 ranges {773221=492056,1816008=503126} 2024-09-18T11:32:43.032 INFO:tasks.rados.rados.0.smithi119.stdout:1967: writing smithi11939506-4 from 773221 to 1265277 tid 1 2024-09-18T11:32:43.039 INFO:tasks.rados.rados.0.smithi119.stdout:1967: writing smithi11939506-4 from 1816008 to 2319134 tid 2 2024-09-18T11:32:43.039 INFO:tasks.rados.rados.0.smithi119.stdout:1968: write oid 21 current snap is 67 2024-09-18T11:32:43.039 INFO:tasks.rados.rados.0.smithi119.stdout:1965: got (0) Success 2024-09-18T11:32:43.039 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 358 (ObjNum 593 snap 0 seq_num 384758312) dirty dne 2024-09-18T11:32:43.039 INFO:tasks.rados.rados.0.smithi119.stdout:1966: finishing set_chunk to oid 48 2024-09-18T11:32:43.039 INFO:tasks.rados.rados.0.smithi119.stdout:1966: got expected ERANGE (src dne) 2024-09-18T11:32:43.039 INFO:tasks.rados.rados.0.smithi119.stdout:1968: seq_num 404 ranges {790028=567305,1874490=763137} 2024-09-18T11:32:43.040 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:43.040 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:43.042 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:43.040+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:43.046 INFO:tasks.rados.rados.0.smithi119.stdout:1968: writing smithi11939506-21 from 790028 to 1357333 tid 1 2024-09-18T11:32:43.052 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:32:43.053 INFO:tasks.rados.rados.0.smithi119.stdout:1968: writing smithi11939506-21 from 1874490 to 2637627 tid 2 2024-09-18T11:32:43.053 INFO:tasks.rados.rados.0.smithi119.stdout:1965: done (3 left) 2024-09-18T11:32:43.053 INFO:tasks.rados.rados.0.smithi119.stdout:1966: done (2 left) 2024-09-18T11:32:43.053 INFO:tasks.rados.rados.0.smithi119.stdout:1969: set_chunk oid 6 target oid 2024-09-18T11:32:43.062 INFO:tasks.rados.rados.0.smithi119.stdout:1969: set_chunk oid 6 offset: 1152512 length: 1024512 target oid c0f116703d947cdb7c03d64cf6bb69531e6316691a7f186c17429ddacc04cb46 offset: 0 snap -1 2024-09-18T11:32:43.062 INFO:tasks.rados.rados.0.smithi119.stdout:1970: set_chunk oid 23 target oid 2024-09-18T11:32:43.070 INFO:tasks.rados.rados.0.smithi119.stdout:1970: set_chunk oid 23 offset: 725504 length: 1074688 target oid 94397cf9dd92b7cfde651a5065b5df1b16fd8ac991413eea1cb03f151d747856 offset: 0 snap -1 2024-09-18T11:32:43.070 INFO:tasks.rados.rados.0.smithi119.stdout:1971: read oid 42 snap -1 2024-09-18T11:32:43.070 INFO:tasks.rados.rados.0.smithi119.stdout:1971: expect (ObjNum 386 snap 61 seq_num 386) 2024-09-18T11:32:43.070 INFO:tasks.rados.rados.0.smithi119.stdout:1972: read oid 2 snap 62 2024-09-18T11:32:43.070 INFO:tasks.rados.rados.0.smithi119.stdout:1972: expect (ObjNum 375 snap 58 seq_num 375) 2024-09-18T11:32:43.070 INFO:tasks.rados.rados.0.smithi119.stdout:1973: tier_promote oid 40 2024-09-18T11:32:43.070 INFO:tasks.rados.rados.0.smithi119.stdout:1967: finishing write tid 1 to smithi11939506-4 2024-09-18T11:32:43.070 INFO:tasks.rados.rados.0.smithi119.stdout:1967: finishing write tid 2 to smithi11939506-4 2024-09-18T11:32:43.071 INFO:tasks.rados.rados.0.smithi119.stdout:1974: read oid 31 snap -1 2024-09-18T11:32:43.071 INFO:tasks.rados.rados.0.smithi119.stdout:1974: expect (ObjNum 399 snap 65 seq_num 399) 2024-09-18T11:32:43.071 INFO:tasks.rados.rados.0.smithi119.stdout:1975: set_chunk oid 8 target oid 2024-09-18T11:32:43.074 INFO:tasks.rados.rados.0.smithi119.stdout:1975: set_chunk oid 8 offset: 380928 length: 123904 target oid d9e785f3a616c5ea29085e905e8216d8e7448d344a77d99a007fea79ade638a8 offset: 0 snap -1 2024-09-18T11:32:43.074 INFO:tasks.rados.rados.0.smithi119.stdout:1976: set_chunk oid 24 target oid 2024-09-18T11:32:43.084 INFO:tasks.rados.rados.0.smithi119.stdout:1976: set_chunk oid 24 offset: 485376 length: 1262592 target oid cb38cfa5ee53b5afd32de2dccce03d671163629c0d505fe5841de37b322b8169 offset: 0 snap 66 2024-09-18T11:32:43.084 INFO:tasks.rados.rados.0.smithi119.stdout:1977: write (excl) oid 12 current snap is 67 2024-09-18T11:32:43.084 INFO:tasks.rados.rados.0.smithi119.stdout:1967: finishing write tid 3 to smithi11939506-4 2024-09-18T11:32:43.084 INFO:tasks.rados.rados.0.smithi119.stdout:1967: finishing write tid 4 to smithi11939506-4 2024-09-18T11:32:43.084 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 4 v 149 (ObjNum 403 snap 67 seq_num 403) dirty exists 2024-09-18T11:32:43.084 INFO:tasks.rados.rados.0.smithi119.stdout:1967: left oid 4 (ObjNum 403 snap 67 seq_num 403) 2024-09-18T11:32:43.084 INFO:tasks.rados.rados.0.smithi119.stdout:1968: finishing write tid 1 to smithi11939506-21 2024-09-18T11:32:43.085 INFO:tasks.rados.rados.0.smithi119.stdout:1968: finishing write tid 2 to smithi11939506-21 2024-09-18T11:32:43.085 INFO:tasks.rados.rados.0.smithi119.stdout:1977: seq_num 405 ranges {406835=642554,1532972=506795,2772312=34523} 2024-09-18T11:32:43.089 INFO:tasks.rados.rados.0.smithi119.stdout:1977: writing smithi11939506-12 from 406835 to 1049389 tid 1 2024-09-18T11:32:43.095 INFO:tasks.rados.rados.0.smithi119.stdout:1977: writing smithi11939506-12 from 1532972 to 2039767 tid 2 2024-09-18T11:32:43.100 INFO:tasks.rados.rados.0.smithi119.stdout:1977: writing smithi11939506-12 from 2772312 to 2806835 tid 3 2024-09-18T11:32:43.100 INFO:tasks.rados.rados.0.smithi119.stdout:1967: done (10 left) 2024-09-18T11:32:43.100 INFO:tasks.rados.rados.0.smithi119.stdout:1978: read oid 43 snap -1 2024-09-18T11:32:43.100 INFO:tasks.rados.rados.0.smithi119.stdout:1978: expect (ObjNum 346 snap 51 seq_num 346) 2024-09-18T11:32:43.100 INFO:tasks.rados.rados.0.smithi119.stdout:1979: read oid 46 snap -1 2024-09-18T11:32:43.100 INFO:tasks.rados.rados.0.smithi119.stdout:1979: expect (ObjNum 320 snap 44 seq_num 320) 2024-09-18T11:32:43.101 INFO:tasks.rados.rados.0.smithi119.stdout:1980: write oid 44 current snap is 67 2024-09-18T11:32:43.101 INFO:tasks.rados.rados.0.smithi119.stdout:1968: finishing write tid 3 to smithi11939506-21 2024-09-18T11:32:43.101 INFO:tasks.rados.rados.0.smithi119.stdout:1968: finishing write tid 4 to smithi11939506-21 2024-09-18T11:32:43.101 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 416 (ObjNum 404 snap 67 seq_num 404) dirty exists 2024-09-18T11:32:43.101 INFO:tasks.rados.rados.0.smithi119.stdout:1968: left oid 21 (ObjNum 404 snap 67 seq_num 404) 2024-09-18T11:32:43.101 INFO:tasks.rados.rados.0.smithi119.stdout:1969: finishing set_chunk to oid 6 2024-09-18T11:32:43.101 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 6 v 208 (ObjNum 397 snap 64 seq_num 397) dirty exists 2024-09-18T11:32:43.101 INFO:tasks.rados.rados.0.smithi119.stdout:1980: seq_num 406 ranges {423642=717803,1875101=766806} 2024-09-18T11:32:43.106 INFO:tasks.rados.rados.0.smithi119.stdout:1980: writing smithi11939506-44 from 423642 to 1141445 tid 1 2024-09-18T11:32:43.115 INFO:tasks.rados.rados.0.smithi119.stdout:1980: writing smithi11939506-44 from 1875101 to 2641907 tid 2 2024-09-18T11:32:43.115 INFO:tasks.rados.rados.0.smithi119.stdout:1968: done (12 left) 2024-09-18T11:32:43.115 INFO:tasks.rados.rados.0.smithi119.stdout:1969: done (11 left) 2024-09-18T11:32:43.115 INFO:tasks.rados.rados.0.smithi119.stdout:1981: read oid 5 snap -1 2024-09-18T11:32:43.115 INFO:tasks.rados.rados.0.smithi119.stdout:1981: expect (ObjNum 398 snap 64 seq_num 398) 2024-09-18T11:32:43.115 INFO:tasks.rados.rados.0.smithi119.stdout:1982: read oid 1 snap -1 2024-09-18T11:32:43.115 INFO:tasks.rados.rados.0.smithi119.stdout:1982: expect (ObjNum 387 snap 62 seq_num 387) 2024-09-18T11:32:43.115 INFO:tasks.rados.rados.0.smithi119.stdout:1983: read oid 49 snap -1 2024-09-18T11:32:43.115 INFO:tasks.rados.rados.0.smithi119.stdout:1983: expect (ObjNum 323 snap 45 seq_num 323) 2024-09-18T11:32:43.115 INFO:tasks.rados.rados.0.smithi119.stdout:1984: write oid 9 current snap is 67 2024-09-18T11:32:43.115 INFO:tasks.rados.rados.0.smithi119.stdout:1970: finishing set_chunk to oid 23 2024-09-18T11:32:43.115 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 23 set_chunk returned error code -95 offset: 725504 length: 1074688 2024-09-18T11:32:43.116 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 23 v 0 (ObjNum 363 snap 55 seq_num 363) dirty exists 2024-09-18T11:32:43.116 INFO:tasks.rados.rados.0.smithi119.stdout:1984: seq_num 407 ranges {440449=793052,1933583=510464} 2024-09-18T11:32:43.121 INFO:tasks.rados.rados.0.smithi119.stdout:1984: writing smithi11939506-9 from 440449 to 1233501 tid 1 2024-09-18T11:32:43.128 INFO:tasks.rados.rados.0.smithi119.stdout:1984: writing smithi11939506-9 from 1933583 to 2444047 tid 2 2024-09-18T11:32:43.128 INFO:tasks.rados.rados.0.smithi119.stdout:1970: done (14 left) 2024-09-18T11:32:43.128 INFO:tasks.rados.rados.0.smithi119.stdout:1985: write (excl) oid 25 current snap is 67 2024-09-18T11:32:43.128 INFO:tasks.rados.rados.0.smithi119.stdout:1971: expect (ObjNum 386 snap 61 seq_num 386) 2024-09-18T11:32:43.143 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:43.142+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:43.192 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:43.193 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:43.201 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:32:43.244 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:43.243+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:43.299 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:43.300 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:43.308 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:32:43.351 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:43.349+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:43.359 INFO:tasks.rados.rados.0.smithi119.stdout:1974: expect (ObjNum 399 snap 65 seq_num 399) 2024-09-18T11:32:43.405 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:43.405 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:43.414 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:32:43.442 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T11:32:43.486 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:43.485+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:43.522 INFO:tasks.rados.rados.0.smithi119.stdout:1973: got (0) Success 2024-09-18T11:32:43.523 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 163 (ObjNum 287 snap 41 seq_num 287) dirty exists 2024-09-18T11:32:43.523 INFO:tasks.rados.rados.0.smithi119.stdout:1972: expect (ObjNum 375 snap 58 seq_num 375) 2024-09-18T11:32:43.547 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:43.546+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:43.552 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:43.552 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:43.561 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:32:43.612 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:32:43.648 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:43.646+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:43.658 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:43.658 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:43.662 INFO:tasks.rados.rados.0.smithi119.stdout:1975: finishing set_chunk to oid 8 2024-09-18T11:32:43.662 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 8 v 305 (ObjNum 362 snap 55 seq_num 362) dirty exists 2024-09-18T11:32:43.662 INFO:tasks.rados.rados.0.smithi119.stdout:1976: finishing set_chunk to oid 24 2024-09-18T11:32:43.662 INFO:tasks.rados.rados.0.smithi119.stdout:1977: finishing write tid 1 to smithi11939506-12 2024-09-18T11:32:43.662 INFO:tasks.rados.rados.0.smithi119.stdout:1985: seq_num 408 ranges {457256=468301,1592065=770475,2854581=2675} 2024-09-18T11:32:43.666 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:32:43.667 INFO:tasks.rados.rados.0.smithi119.stdout:1985: writing smithi11939506-25 from 457256 to 925557 tid 1 2024-09-18T11:32:43.674 INFO:tasks.rados.rados.0.smithi119.stdout:1985: writing smithi11939506-25 from 1592065 to 2362540 tid 2 2024-09-18T11:32:43.678 INFO:tasks.rados.rados.0.smithi119.stdout:1985: writing smithi11939506-25 from 2854581 to 2857256 tid 3 2024-09-18T11:32:43.678 INFO:tasks.rados.rados.0.smithi119.stdout:1971: done (14 left) 2024-09-18T11:32:43.678 INFO:tasks.rados.rados.0.smithi119.stdout:1972: done (13 left) 2024-09-18T11:32:43.678 INFO:tasks.rados.rados.0.smithi119.stdout:1973: done (12 left) 2024-09-18T11:32:43.678 INFO:tasks.rados.rados.0.smithi119.stdout:1974: done (11 left) 2024-09-18T11:32:43.678 INFO:tasks.rados.rados.0.smithi119.stdout:1975: done (10 left) 2024-09-18T11:32:43.678 INFO:tasks.rados.rados.0.smithi119.stdout:1976: done (9 left) 2024-09-18T11:32:43.678 INFO:tasks.rados.rados.0.smithi119.stdout:1986: snap_remove snap 56 2024-09-18T11:32:43.744 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:43.744 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:43.749 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:43.747+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:43.760 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:43.760 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:43.775 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:43.775 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:43.779 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:43.779 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:43.779 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:43.779 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:43.788 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:32:43.790 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:43.791 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:43.807 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:43.808 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:43.823 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:43.823 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:43.839 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:43.839 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:43.849 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:43.848+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:43.854 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:43.855 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:43.890 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:43.890 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:43.890 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:43.890 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:43.890 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:43.898 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:32:43.951 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:43.948+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:44.036 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:44.036 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:44.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:44.037 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [] 2024-09-18T11:32:44.037 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:44.044 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:32:44.088 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:44.087+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:44.142 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:44.143 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:44.143 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:44.143 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:44.151 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:32:44.152 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:44.151+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:44.254 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:44.251+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:44.289 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:44.289 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:44.289 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:44.289 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:44.289 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:44.298 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:32:44.355 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:44.353+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:44.397 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:44.397 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:44.397 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:44.397 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:44.397 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.397 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:262 4.9 e53/22 4:99feffe9:::smithi11939506-45:13 v 53'17, mlcod=51'14)", 2024-09-18T11:32:44.397 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:44.397 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 130.39575664399999, 2024-09-18T11:32:44.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013878679, 2024-09-18T11:32:44.398 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.398 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.398 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:44.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.398 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.398 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.398 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:44.398 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:44.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:44.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.399 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998997+0000", 2024-09-18T11:32:44.399 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1099999999999999e-06 2024-09-18T11:32:44.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.399 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.399 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999008+0000", 2024-09-18T11:32:44.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1321000000000001e-05 2024-09-18T11:32:44.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999015+0000", 2024-09-18T11:32:44.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3000000000000004e-06 2024-09-18T11:32:44.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.400 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.400 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.400 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999055+0000", 2024-09-18T11:32:44.400 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0058000000000002e-05 2024-09-18T11:32:44.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999070+0000", 2024-09-18T11:32:44.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4328000000000001e-05 2024-09-18T11:32:44.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.401 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012465+0000", 2024-09-18T11:32:44.401 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013395638 2024-09-18T11:32:44.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.401 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.401 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.402 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012870+0000", 2024-09-18T11:32:44.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040423400000000001 2024-09-18T11:32:44.402 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.402 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.402 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.402 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:205 4.b e53/22 4:dba5bd39:::smithi11939506-17:13 v 53'77, mlcod=51'74)", 2024-09-18T11:32:44.402 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:44.402 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 130.395707383, 2024-09-18T11:32:44.402 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013359586, 2024-09-18T11:32:44.402 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.402 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.402 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.402 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:44.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:44.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.403 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.403 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.403 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:44.403 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:44.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999048+0000", 2024-09-18T11:32:44.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3550000000000005e-06 2024-09-18T11:32:44.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.404 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.404 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999056+0000", 2024-09-18T11:32:44.404 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3259999999999996e-06 2024-09-18T11:32:44.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.404 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999063+0000", 2024-09-18T11:32:44.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6409999999999996e-06 2024-09-18T11:32:44.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999098+0000", 2024-09-18T11:32:44.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5203999999999997e-05 2024-09-18T11:32:44.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.405 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.405 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.405 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999115+0000", 2024-09-18T11:32:44.405 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6478999999999999e-05 2024-09-18T11:32:44.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012362+0000", 2024-09-18T11:32:44.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01324768 2024-09-18T11:32:44.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.406 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.406 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012400+0000", 2024-09-18T11:32:44.406 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.731e-05 2024-09-18T11:32:44.406 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.406 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.406 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.407 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.407 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2937 4.7 4:e301aefd:::smithi11939506-19:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:44.407 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:44.407 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 81.359907070000006, 2024-09-18T11:32:44.407 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019287667000000001, 2024-09-18T11:32:44.407 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.407 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:44.407 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:44.407 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:44.407 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:44.407 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2937 2024-09-18T11:32:44.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.408 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:44.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:44.408 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.408 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.408 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.408 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:44.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:44.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034842+0000", 2024-09-18T11:32:44.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.372e-06 2024-09-18T11:32:44.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.409 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.409 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034846+0000", 2024-09-18T11:32:44.409 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3919999999999996e-06 2024-09-18T11:32:44.409 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.409 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034848+0000", 2024-09-18T11:32:44.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3539999999999998e-06 2024-09-18T11:32:44.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034893+0000", 2024-09-18T11:32:44.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4861000000000003e-05 2024-09-18T11:32:44.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.410 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.410 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.410 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034965+0000", 2024-09-18T11:32:44.410 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1860999999999995e-05 2024-09-18T11:32:44.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.411 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.411 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.054128+0000", 2024-09-18T11:32:44.411 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019163039 2024-09-18T11:32:44.411 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.411 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.411 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.411 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.411 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.411 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3453 4.f 4:f5d7df41:::smithi11939506-16:head [write 1813152~490434] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:44.412 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:32:44.412 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.371995142000003, 2024-09-18T11:32:44.412 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050685840000000003, 2024-09-18T11:32:44.412 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.412 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.412 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:44.412 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:44.412 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:44.412 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3453 2024-09-18T11:32:44.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.412 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.412 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:32:44.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:32:44.413 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.413 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.413 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.413 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022751+0000", 2024-09-18T11:32:44.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:44.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022871+0000", 2024-09-18T11:32:44.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000119781 2024-09-18T11:32:44.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.414 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.414 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.414 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022882+0000", 2024-09-18T11:32:44.414 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1627000000000001e-05 2024-09-18T11:32:44.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022887+0000", 2024-09-18T11:32:44.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2390000000000004e-06 2024-09-18T11:32:44.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.415 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.415 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023123+0000", 2024-09-18T11:32:44.415 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000236598 2024-09-18T11:32:44.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.415 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023207+0000", 2024-09-18T11:32:44.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4303999999999998e-05 2024-09-18T11:32:44.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:44.416 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023367+0000", 2024-09-18T11:32:44.416 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015983600000000001 2024-09-18T11:32:44.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.416 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.416 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:44.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073237+0000", 2024-09-18T11:32:44.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049869400000000001 2024-09-18T11:32:44.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:44.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073352+0000", 2024-09-18T11:32:44.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011579899999999999 2024-09-18T11:32:44.417 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.417 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.417 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.417 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073394+0000", 2024-09-18T11:32:44.417 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1445e-05 2024-09-18T11:32:44.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.418 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.418 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073438+0000", 2024-09-18T11:32:44.418 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4261999999999997e-05 2024-09-18T11:32:44.418 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.418 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.418 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.419 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3454 4.f 4:f5d7df41:::smithi11939506-16:head [setxattr _header (57) in=7b,truncate 2991679] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:44.419 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:32:44.419 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.371296700000002, 2024-09-18T11:32:44.419 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050365656000000002, 2024-09-18T11:32:44.419 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.419 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.419 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:44.419 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:44.419 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:44.419 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3454 2024-09-18T11:32:44.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.420 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:32:44.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.420 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.420 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:32:44.420 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.420 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.420 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023450+0000", 2024-09-18T11:32:44.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:44.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023456+0000", 2024-09-18T11:32:44.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8329999999999999e-06 2024-09-18T11:32:44.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.421 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.421 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.421 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023465+0000", 2024-09-18T11:32:44.421 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1130000000000007e-06 2024-09-18T11:32:44.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023470+0000", 2024-09-18T11:32:44.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9139999999999999e-06 2024-09-18T11:32:44.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.422 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072611+0000", 2024-09-18T11:32:44.422 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141643999999998 2024-09-18T11:32:44.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.422 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.422 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072729+0000", 2024-09-18T11:32:44.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117667 2024-09-18T11:32:44.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:44.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072887+0000", 2024-09-18T11:32:44.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015760000000000001 2024-09-18T11:32:44.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.423 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.423 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:44.423 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073558+0000", 2024-09-18T11:32:44.423 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00067171699999999995 2024-09-18T11:32:44.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:44.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073764+0000", 2024-09-18T11:32:44.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000205749 2024-09-18T11:32:44.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073790+0000", 2024-09-18T11:32:44.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5643000000000002e-05 2024-09-18T11:32:44.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.424 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.424 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.424 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073816+0000", 2024-09-18T11:32:44.424 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6588e-05 2024-09-18T11:32:44.425 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.425 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.425 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.425 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3455 4.f 4:f5d7df41:::smithi11939506-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:44.425 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:44.425 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.371267082000003, 2024-09-18T11:32:44.425 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050460466000000002, 2024-09-18T11:32:44.425 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.425 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:44.425 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:44.425 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:44.425 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:44.426 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3455 2024-09-18T11:32:44.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.426 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:44.426 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.426 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.426 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:44.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:44.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:44.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.427 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.427 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.427 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023483+0000", 2024-09-18T11:32:44.427 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.695e-06 2024-09-18T11:32:44.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023489+0000", 2024-09-18T11:32:44.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1079999999999998e-06 2024-09-18T11:32:44.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.428 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.428 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023492+0000", 2024-09-18T11:32:44.428 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.478e-06 2024-09-18T11:32:44.428 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073273+0000", 2024-09-18T11:32:44.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049780638000000002 2024-09-18T11:32:44.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:44.429 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073343+0000", 2024-09-18T11:32:44.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9770000000000005e-05 2024-09-18T11:32:44.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.429 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073846+0000", 2024-09-18T11:32:44.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050289499999999995 2024-09-18T11:32:44.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073886+0000", 2024-09-18T11:32:44.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0658999999999998e-05 2024-09-18T11:32:44.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073941+0000", 2024-09-18T11:32:44.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4642000000000003e-05 2024-09-18T11:32:44.430 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.430 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.431 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.431 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3456 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:44.431 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:44.431 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.371242017, 2024-09-18T11:32:44.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049859120999999999, 2024-09-18T11:32:44.431 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.431 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:44.431 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:44.431 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:44.431 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:44.431 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3456 2024-09-18T11:32:44.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.432 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:44.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:44.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:44.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:44.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023508+0000", 2024-09-18T11:32:44.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9490000000000001e-06 2024-09-18T11:32:44.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023513+0000", 2024-09-18T11:32:44.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3800000000000002e-06 2024-09-18T11:32:44.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023517+0000", 2024-09-18T11:32:44.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5300000000000001e-06 2024-09-18T11:32:44.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073262+0000", 2024-09-18T11:32:44.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049745194999999999 2024-09-18T11:32:44.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.434 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.434 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073365+0000", 2024-09-18T11:32:44.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102462 2024-09-18T11:32:44.435 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.435 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.435 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.435 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3457 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:44.435 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:32:44.435 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.371224921, 2024-09-18T11:32:44.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049928567, 2024-09-18T11:32:44.435 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.435 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:44.435 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:44.435 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:44.435 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:44.436 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3457 2024-09-18T11:32:44.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.436 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:32:44.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:32:44.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023522+0000", 2024-09-18T11:32:44.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:44.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023524+0000", 2024-09-18T11:32:44.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8500000000000001e-06 2024-09-18T11:32:44.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023529+0000", 2024-09-18T11:32:44.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7509999999999999e-06 2024-09-18T11:32:44.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023532+0000", 2024-09-18T11:32:44.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.163e-06 2024-09-18T11:32:44.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073386+0000", 2024-09-18T11:32:44.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049853899 2024-09-18T11:32:44.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073451+0000", 2024-09-18T11:32:44.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5197000000000006e-05 2024-09-18T11:32:44.439 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.439 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.439 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.439 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3458 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:44.440 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:44.440 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.371212364000002, 2024-09-18T11:32:44.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.05001659, 2024-09-18T11:32:44.440 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.440 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:44.440 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:44.440 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:44.440 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:44.440 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3458 2024-09-18T11:32:44.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.440 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.440 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:44.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:44.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:44.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:44.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023537+0000", 2024-09-18T11:32:44.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.64e-06 2024-09-18T11:32:44.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023543+0000", 2024-09-18T11:32:44.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3369999999999996e-06 2024-09-18T11:32:44.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023546+0000", 2024-09-18T11:32:44.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:32:44.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073466+0000", 2024-09-18T11:32:44.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049919719000000001 2024-09-18T11:32:44.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073552+0000", 2024-09-18T11:32:44.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6168999999999994e-05 2024-09-18T11:32:44.443 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.443 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.443 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.444 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(client.4590.0:3453 4.f e116/22 ondisk, result = 0)", 2024-09-18T11:32:44.444 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:32:44.444 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.369607279999997, 2024-09-18T11:32:44.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048099441, 2024-09-18T11:32:44.444 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.444 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:44.444 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:32:44.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:32:44.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.445 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025139+0000", 2024-09-18T11:32:44.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:44.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.445 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025144+0000", 2024-09-18T11:32:44.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4059999999999998e-06 2024-09-18T11:32:44.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072212+0000", 2024-09-18T11:32:44.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047068482000000002 2024-09-18T11:32:44.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072230+0000", 2024-09-18T11:32:44.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7632999999999999e-05 2024-09-18T11:32:44.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073224+0000", 2024-09-18T11:32:44.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099436899999999994 2024-09-18T11:32:44.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073231+0000", 2024-09-18T11:32:44.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-09-18T11:32:44.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073240+0000", 2024-09-18T11:32:44.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5699999999999993e-06 2024-09-18T11:32:44.448 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.448 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.448 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.448 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3930 4.b 4:d3ae8215:::smithi11939506-41:head [write 632029~547288] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:44.448 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:32:44.448 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.368691599999998, 2024-09-18T11:32:44.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018962711, 2024-09-18T11:32:44.448 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.448 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.448 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:44.448 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:44.449 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:44.449 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3930 2024-09-18T11:32:44.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.449 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:32:44.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:32:44.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026055+0000", 2024-09-18T11:32:44.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:44.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026334+0000", 2024-09-18T11:32:44.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000278931 2024-09-18T11:32:44.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026344+0000", 2024-09-18T11:32:44.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0716999999999999e-05 2024-09-18T11:32:44.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026350+0000", 2024-09-18T11:32:44.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7620000000000001e-06 2024-09-18T11:32:44.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028429+0000", 2024-09-18T11:32:44.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020787319999999998 2024-09-18T11:32:44.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028525+0000", 2024-09-18T11:32:44.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6638999999999994e-05 2024-09-18T11:32:44.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:44.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028720+0000", 2024-09-18T11:32:44.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000194525 2024-09-18T11:32:44.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:44.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.036996+0000", 2024-09-18T11:32:44.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082756140000000006 2024-09-18T11:32:44.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:44.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044952+0000", 2024-09-18T11:32:44.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566740000000004 2024-09-18T11:32:44.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044985+0000", 2024-09-18T11:32:44.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3081999999999999e-05 2024-09-18T11:32:44.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045019+0000", 2024-09-18T11:32:44.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3257999999999999e-05 2024-09-18T11:32:44.453 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.454 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.454 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.454 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3932 4.b 4:d3ae8215:::smithi11939506-41:head [write 2796249~437859] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:44.454 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:32:44.454 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.356684960000003, 2024-09-18T11:32:44.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01458329, 2024-09-18T11:32:44.454 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.454 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.454 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:44.454 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:44.454 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:44.455 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3932 2024-09-18T11:32:44.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.455 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:32:44.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:32:44.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038061+0000", 2024-09-18T11:32:44.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:44.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038341+0000", 2024-09-18T11:32:44.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027989099999999999 2024-09-18T11:32:44.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038353+0000", 2024-09-18T11:32:44.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2247e-05 2024-09-18T11:32:44.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038358+0000", 2024-09-18T11:32:44.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-18T11:32:44.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043226+0000", 2024-09-18T11:32:44.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048673049999999997 2024-09-18T11:32:44.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043284+0000", 2024-09-18T11:32:44.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8338999999999998e-05 2024-09-18T11:32:44.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:44.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043367+0000", 2024-09-18T11:32:44.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3120000000000004e-05 2024-09-18T11:32:44.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:44.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.051544+0000", 2024-09-18T11:32:44.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00817734 2024-09-18T11:32:44.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:44.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052528+0000", 2024-09-18T11:32:44.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098318499999999992 2024-09-18T11:32:44.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052610+0000", 2024-09-18T11:32:44.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2300999999999997e-05 2024-09-18T11:32:44.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052646+0000", 2024-09-18T11:32:44.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5955999999999999e-05 2024-09-18T11:32:44.459 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.459 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.460 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.460 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3933 4.b 4:d3ae8215:::smithi11939506-41:head [write 3653524~178505] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:44.460 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041300+0000", 2024-09-18T11:32:44.460 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.353447064000001, 2024-09-18T11:32:44.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01299752, 2024-09-18T11:32:44.460 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.460 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.460 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:44.460 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:44.460 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:44.460 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3933 2024-09-18T11:32:44.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.461 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041300+0000", 2024-09-18T11:32:44.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041300+0000", 2024-09-18T11:32:44.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041299+0000", 2024-09-18T11:32:44.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:44.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041369+0000", 2024-09-18T11:32:44.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9838000000000005e-05 2024-09-18T11:32:44.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041380+0000", 2024-09-18T11:32:44.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0910000000000001e-05 2024-09-18T11:32:44.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041385+0000", 2024-09-18T11:32:44.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5800000000000002e-06 2024-09-18T11:32:44.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044260+0000", 2024-09-18T11:32:44.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028751229999999998 2024-09-18T11:32:44.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044312+0000", 2024-09-18T11:32:44.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1808000000000003e-05 2024-09-18T11:32:44.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:44.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044393+0000", 2024-09-18T11:32:44.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.1806999999999998e-05 2024-09-18T11:32:44.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:44.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052654+0000", 2024-09-18T11:32:44.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082609320000000003 2024-09-18T11:32:44.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:44.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054207+0000", 2024-09-18T11:32:44.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001552091 2024-09-18T11:32:44.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054256+0000", 2024-09-18T11:32:44.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9437000000000001e-05 2024-09-18T11:32:44.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054298+0000", 2024-09-18T11:32:44.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1993000000000003e-05 2024-09-18T11:32:44.466 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.466 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.466 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.466 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3934 4.b 4:d3ae8215:::smithi11939506-41:head [setxattr _header (57) in=7b,truncate 3832029] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:44.466 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:44.466 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.353358442999998, 2024-09-18T11:32:44.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013018336, 2024-09-18T11:32:44.466 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.466 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.466 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:44.466 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:44.467 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:44.467 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3934 2024-09-18T11:32:44.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.467 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:44.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:44.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041389+0000", 2024-09-18T11:32:44.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:44.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041391+0000", 2024-09-18T11:32:44.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.26e-06 2024-09-18T11:32:44.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041395+0000", 2024-09-18T11:32:44.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8999999999999999e-06 2024-09-18T11:32:44.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041397+0000", 2024-09-18T11:32:44.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3149999999999999e-06 2024-09-18T11:32:44.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045266+0000", 2024-09-18T11:32:44.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003869094 2024-09-18T11:32:44.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045316+0000", 2024-09-18T11:32:44.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0074000000000002e-05 2024-09-18T11:32:44.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:44.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045403+0000", 2024-09-18T11:32:44.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6737999999999995e-05 2024-09-18T11:32:44.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:44.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052659+0000", 2024-09-18T11:32:44.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0072558259999999999 2024-09-18T11:32:44.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:44.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054330+0000", 2024-09-18T11:32:44.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0016716000000000001 2024-09-18T11:32:44.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.471 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054362+0000", 2024-09-18T11:32:44.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1195000000000002e-05 2024-09-18T11:32:44.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054407+0000", 2024-09-18T11:32:44.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5707000000000001e-05 2024-09-18T11:32:44.472 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.472 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.472 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.472 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3935 4.b 4:d3ae8215:::smithi11939506-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:44.472 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:32:44.472 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.353347853000002, 2024-09-18T11:32:44.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013142333000000001, 2024-09-18T11:32:44.472 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.472 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:44.472 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:44.473 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:44.473 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:44.473 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3935 2024-09-18T11:32:44.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.473 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:32:44.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:32:44.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041399+0000", 2024-09-18T11:32:44.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:44.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041401+0000", 2024-09-18T11:32:44.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4890000000000001e-06 2024-09-18T11:32:44.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041404+0000", 2024-09-18T11:32:44.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5240000000000001e-06 2024-09-18T11:32:44.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041407+0000", 2024-09-18T11:32:44.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2230000000000001e-06 2024-09-18T11:32:44.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045699+0000", 2024-09-18T11:32:44.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042918970000000002 2024-09-18T11:32:44.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:44.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045750+0000", 2024-09-18T11:32:44.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1332999999999998e-05 2024-09-18T11:32:44.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054438+0000", 2024-09-18T11:32:44.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086882569999999996 2024-09-18T11:32:44.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054485+0000", 2024-09-18T11:32:44.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7151000000000003e-05 2024-09-18T11:32:44.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054542+0000", 2024-09-18T11:32:44.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6706999999999998e-05 2024-09-18T11:32:44.477 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.477 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.477 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.477 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3969 4.4 4:2cb3e6a5:::smithi11939506-3:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:44.477 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:32:44.478 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 34.821107587999997, 2024-09-18T11:32:44.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.024248042000000001, 2024-09-18T11:32:44.478 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.478 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:44.478 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:44.478 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:44.478 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:44.478 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3969 2024-09-18T11:32:44.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.478 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:32:44.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:32:44.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573639+0000", 2024-09-18T11:32:44.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:44.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573646+0000", 2024-09-18T11:32:44.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6569999999999997e-06 2024-09-18T11:32:44.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573662+0000", 2024-09-18T11:32:44.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5187999999999999e-05 2024-09-18T11:32:44.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573670+0000", 2024-09-18T11:32:44.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5399999999999996e-06 2024-09-18T11:32:44.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:44.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8592999999999999e-05 2024-09-18T11:32:44.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573818+0000", 2024-09-18T11:32:44.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9388999999999995e-05 2024-09-18T11:32:44.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597888+0000", 2024-09-18T11:32:44.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.024069859999999998 2024-09-18T11:32:44.482 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.482 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.482 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.482 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3973 4.7 4:e73ba68f:::smithi11939506-1:head [tier-promote] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:44.482 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:44.482 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 34.821008509000002, 2024-09-18T11:32:44.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.023911297000000001, 2024-09-18T11:32:44.482 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.482 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:44.482 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:44.484 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:44.484 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:44.484 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3973 2024-09-18T11:32:44.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.484 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:44.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:44.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573738+0000", 2024-09-18T11:32:44.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:44.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573743+0000", 2024-09-18T11:32:44.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0839999999999999e-06 2024-09-18T11:32:44.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573752+0000", 2024-09-18T11:32:44.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3080000000000008e-06 2024-09-18T11:32:44.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573758+0000", 2024-09-18T11:32:44.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2659999999999998e-06 2024-09-18T11:32:44.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573803+0000", 2024-09-18T11:32:44.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5243e-05 2024-09-18T11:32:44.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573897+0000", 2024-09-18T11:32:44.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3398000000000003e-05 2024-09-18T11:32:44.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597650+0000", 2024-09-18T11:32:44.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.023753672999999999 2024-09-18T11:32:44.488 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.488 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.488 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.488 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4443 4.d 4:b9bb642b:::smithi11939506-27:67 [read 0~3512696,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T11:32:44.488 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:36.030743+0000", 2024-09-18T11:32:44.488 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3640048799999995, 2024-09-18T11:32:44.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013696302, 2024-09-18T11:32:44.488 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.489 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:44.489 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:44.489 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:44.489 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:44.489 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4443 2024-09-18T11:32:44.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.489 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030743+0000", 2024-09-18T11:32:44.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030743+0000", 2024-09-18T11:32:44.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030741+0000", 2024-09-18T11:32:44.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:44.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030751+0000", 2024-09-18T11:32:44.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6979999999999993e-06 2024-09-18T11:32:44.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030763+0000", 2024-09-18T11:32:44.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2311e-05 2024-09-18T11:32:44.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030769+0000", 2024-09-18T11:32:44.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6060000000000002e-06 2024-09-18T11:32:44.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.036777+0000", 2024-09-18T11:32:44.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060079089999999996 2024-09-18T11:32:44.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.036945+0000", 2024-09-18T11:32:44.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016867100000000001 2024-09-18T11:32:44.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044439+0000", 2024-09-18T11:32:44.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074937149999999998 2024-09-18T11:32:44.492 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.492 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.492 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.493 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4444 4.d 4:b9bb642b:::smithi11939506-27:67 [sparse-read 0~1262403] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T11:32:44.493 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:32:44.493 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3639700870000002, 2024-09-18T11:32:44.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.014179987, 2024-09-18T11:32:44.493 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.493 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:44.493 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:44.493 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:44.493 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:44.493 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4444 2024-09-18T11:32:44.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.494 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:32:44.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:32:44.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:32:44.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:44.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030780+0000", 2024-09-18T11:32:44.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.163e-06 2024-09-18T11:32:44.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030786+0000", 2024-09-18T11:32:44.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879999999999998e-06 2024-09-18T11:32:44.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030791+0000", 2024-09-18T11:32:44.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2579999999999997e-06 2024-09-18T11:32:44.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044477+0000", 2024-09-18T11:32:44.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013686188 2024-09-18T11:32:44.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044614+0000", 2024-09-18T11:32:44.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013727300000000001 2024-09-18T11:32:44.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044957+0000", 2024-09-18T11:32:44.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000343331 2024-09-18T11:32:44.497 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.497 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.497 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.498 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4445 4.d 4:b9bb642b:::smithi11939506-27:67 [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T11:32:44.498 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:36.030796+0000", 2024-09-18T11:32:44.498 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.3639518109999997, 2024-09-18T11:32:44.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.015916046999999999, 2024-09-18T11:32:44.498 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.498 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:44.498 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:44.498 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:44.498 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:44.498 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4445 2024-09-18T11:32:44.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.498 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030796+0000", 2024-09-18T11:32:44.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030796+0000", 2024-09-18T11:32:44.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030795+0000", 2024-09-18T11:32:44.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:44.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030799+0000", 2024-09-18T11:32:44.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5209999999999998e-06 2024-09-18T11:32:44.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030806+0000", 2024-09-18T11:32:44.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7599999999999997e-06 2024-09-18T11:32:44.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030810+0000", 2024-09-18T11:32:44.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0489999999999997e-06 2024-09-18T11:32:44.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.045277+0000", 2024-09-18T11:32:44.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.014467683 2024-09-18T11:32:44.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.045394+0000", 2024-09-18T11:32:44.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116969 2024-09-18T11:32:44.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.046712+0000", 2024-09-18T11:32:44.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013174669999999999 2024-09-18T11:32:44.502 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.502 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.502 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.502 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.502 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.502 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:44.503 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-18T11:32:44.503 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T11:32:44.503 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-18T11:32:44.503 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-18T11:32:44.505 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:32:44.505 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is set 2024-09-18T11:32:44.505 INFO:tasks.rados.rados.0.smithi119.stdout:1986: done (9 left) 2024-09-18T11:32:44.505 INFO:tasks.rados.rados.0.smithi119.stdout:1987: write oid 27 current snap is 67 2024-09-18T11:32:44.506 INFO:tasks.rados.rados.0.smithi119.stdout:1987: seq_num 409 ranges {474063=543550,1650547=514133,2765651=108412} 2024-09-18T11:32:44.506 INFO:tasks.rados.rados.0.smithi119.stdout:1987: writing smithi11939506-27 from 474063 to 1017613 tid 1 2024-09-18T11:32:44.506 INFO:tasks.rados.rados.0.smithi119.stdout:1987: writing smithi11939506-27 from 1650547 to 2164680 tid 2 2024-09-18T11:32:44.507 INFO:tasks.rados.rados.0.smithi119.stdout:1987: writing smithi11939506-27 from 2765651 to 2874063 tid 3 2024-09-18T11:32:44.507 INFO:tasks.rados.rados.0.smithi119.stdout:1988: delete oid 35 current snap is 67 2024-09-18T11:32:44.507 INFO:tasks.rados.rados.0.smithi119.stdout:1977: finishing write tid 2 to smithi11939506-12 2024-09-18T11:32:44.507 INFO:tasks.rados.rados.0.smithi119.stdout:1977: finishing write tid 3 to smithi11939506-12 2024-09-18T11:32:44.507 INFO:tasks.rados.rados.0.smithi119.stdout:1977: finishing write tid 4 to smithi11939506-12 2024-09-18T11:32:44.507 INFO:tasks.rados.rados.0.smithi119.stdout:1977: finishing write tid 5 to smithi11939506-12 2024-09-18T11:32:44.507 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 12 v 153 (ObjNum 405 snap 67 seq_num 405) dirty exists 2024-09-18T11:32:44.507 INFO:tasks.rados.rados.0.smithi119.stdout:1977: left oid 12 (ObjNum 405 snap 67 seq_num 405) 2024-09-18T11:32:44.507 INFO:tasks.rados.rados.0.smithi119.stdout:1978: expect (ObjNum 346 snap 51 seq_num 346) 2024-09-18T11:32:44.508 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:44.502+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:44.604 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:44.603+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:44.642 INFO:tasks.rados.rados.0.smithi119.stdout:1979: expect (ObjNum 320 snap 44 seq_num 320) 2024-09-18T11:32:44.645 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:44.645 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:44.645 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:44.646 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:44.655 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:32:44.705 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:44.704+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:44.755 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:44.755 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:44.755 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:44.755 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:44.755 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:44.764 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:32:44.806 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:44.804+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:44.828 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-09-18T11:32:44.827+0000 7ff6f6bf1640 -1 mon.a@0(leader).osd e173 definitely_dead 0 2024-09-18T11:32:44.835 INFO:tasks.rados.rados.0.smithi119.stdout:1980: finishing write tid 1 to smithi11939506-44 2024-09-18T11:32:44.836 INFO:tasks.rados.rados.0.smithi119.stdout:1981: expect (ObjNum 398 snap 64 seq_num 398) 2024-09-18T11:32:44.905 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:44.905 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:44.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:44.905 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:44.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.906 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:32:44.906 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:44.906 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 144.91856991, 2024-09-18T11:32:44.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:32:44.906 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.906 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.906 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:44.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:44.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:32:44.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:44.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:32:44.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:32:44.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:32:44.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:32:44.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:32:44.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:32:44.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:32:44.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:32:44.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:32:44.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:32:44.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.909 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:32:44.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:32:44.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:32:44.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:32:44.910 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.910 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.910 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.910 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:32:44.910 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:44.910 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 135.683738957, 2024-09-18T11:32:44.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:32:44.910 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.911 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.911 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:44.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:44.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:44.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:44.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:32:44.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:32:44.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:32:44.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:32:44.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:32:44.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:32:44.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.913 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:32:44.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:32:44.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.913 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:32:44.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:32:44.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:32:44.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:32:44.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:32:44.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:32:44.914 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.914 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.915 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.915 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:32:44.915 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:44.915 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 130.50983267199999, 2024-09-18T11:32:44.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:32:44.915 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.915 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.915 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:44.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:44.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:32:44.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:44.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:32:44.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:32:44.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:32:44.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:32:44.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.917 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.917 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:32:44.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:32:44.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:32:44.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:32:44.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:32:44.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:32:44.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:32:44.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:32:44.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:32:44.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:32:44.919 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.919 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.919 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.919 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:304 4.6 e69/67 4:67c1765a:::smithi11939506-30:17 v 69'35, mlcod=58'31)", 2024-09-18T11:32:44.919 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:44.919 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.555991578, 2024-09-18T11:32:44.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006377611, 2024-09-18T11:32:44.920 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.920 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.920 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:44.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:44.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:44.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:44.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345961+0000", 2024-09-18T11:32:44.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2050000000000008e-06 2024-09-18T11:32:44.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345969+0000", 2024-09-18T11:32:44.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0339999999999997e-06 2024-09-18T11:32:44.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345976+0000", 2024-09-18T11:32:44.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-18T11:32:44.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346026+0000", 2024-09-18T11:32:44.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9286999999999998e-05 2024-09-18T11:32:44.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346050+0000", 2024-09-18T11:32:44.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4397e-05 2024-09-18T11:32:44.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352293+0000", 2024-09-18T11:32:44.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062427170000000001 2024-09-18T11:32:44.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352330+0000", 2024-09-18T11:32:44.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7348999999999998e-05 2024-09-18T11:32:44.923 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.924 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.924 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.924 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:347 4.d e69/22 4:b44229a1:::smithi11939506-42:19 v 69'99, mlcod=65'93)", 2024-09-18T11:32:44.924 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:44.924 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.550258383, 2024-09-18T11:32:44.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056998760000000004, 2024-09-18T11:32:44.924 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.924 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.924 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:44.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:44.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351684+0000", 2024-09-18T11:32:44.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:44.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351693+0000", 2024-09-18T11:32:44.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-18T11:32:44.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351704+0000", 2024-09-18T11:32:44.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1077000000000001e-05 2024-09-18T11:32:44.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351709+0000", 2024-09-18T11:32:44.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5419999999999999e-06 2024-09-18T11:32:44.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351956+0000", 2024-09-18T11:32:44.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000247252 2024-09-18T11:32:44.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351975+0000", 2024-09-18T11:32:44.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9201000000000001e-05 2024-09-18T11:32:44.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357334+0000", 2024-09-18T11:32:44.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053588749999999999 2024-09-18T11:32:44.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357385+0000", 2024-09-18T11:32:44.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1406999999999998e-05 2024-09-18T11:32:44.928 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.928 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.928 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.929 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1956 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'37, mlcod=58'31)", 2024-09-18T11:32:44.929 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:44.929 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.54931347, 2024-09-18T11:32:44.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010958842, 2024-09-18T11:32:44.929 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.929 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.929 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.929 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.929 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:44.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.929 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:44.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:44.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:44.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352996+0000", 2024-09-18T11:32:44.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036622 2024-09-18T11:32:44.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353011+0000", 2024-09-18T11:32:44.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4849e-05 2024-09-18T11:32:44.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353017+0000", 2024-09-18T11:32:44.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:32:44.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353068+0000", 2024-09-18T11:32:44.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0695000000000002e-05 2024-09-18T11:32:44.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353092+0000", 2024-09-18T11:32:44.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4260000000000002e-05 2024-09-18T11:32:44.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363551+0000", 2024-09-18T11:32:44.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010459154999999999 2024-09-18T11:32:44.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363589+0000", 2024-09-18T11:32:44.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8161000000000003e-05 2024-09-18T11:32:44.933 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.933 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.933 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.933 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1957 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'38, mlcod=65'33)", 2024-09-18T11:32:44.933 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:44.933 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.542897603, 2024-09-18T11:32:44.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050499400000000002, 2024-09-18T11:32:44.934 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.934 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.934 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:44.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:44.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:44.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:44.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359214+0000", 2024-09-18T11:32:44.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000168047 2024-09-18T11:32:44.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359227+0000", 2024-09-18T11:32:44.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3124e-05 2024-09-18T11:32:44.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359235+0000", 2024-09-18T11:32:44.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0760000000000003e-06 2024-09-18T11:32:44.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359287+0000", 2024-09-18T11:32:44.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.151e-05 2024-09-18T11:32:44.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359310+0000", 2024-09-18T11:32:44.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-09-18T11:32:44.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364065+0000", 2024-09-18T11:32:44.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047549439999999997 2024-09-18T11:32:44.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.937 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.937 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364096+0000", 2024-09-18T11:32:44.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1423000000000001e-05 2024-09-18T11:32:44.938 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.938 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.938 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.938 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2300 4.4 e77/22 4:2c93196f:::smithi11939506-28:head v 77'94, mlcod=77'90)", 2024-09-18T11:32:44.938 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:44.938 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 103.913835385, 2024-09-18T11:32:44.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050425380000000001, 2024-09-18T11:32:44.938 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.938 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.938 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:44.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:44.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988108+0000", 2024-09-18T11:32:44.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:44.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988388+0000", 2024-09-18T11:32:44.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028087700000000001 2024-09-18T11:32:44.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988405+0000", 2024-09-18T11:32:44.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6565000000000001e-05 2024-09-18T11:32:44.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988412+0000", 2024-09-18T11:32:44.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9709999999999998e-06 2024-09-18T11:32:44.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988464+0000", 2024-09-18T11:32:44.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1949e-05 2024-09-18T11:32:44.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988490+0000", 2024-09-18T11:32:44.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5780999999999999e-05 2024-09-18T11:32:44.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993112+0000", 2024-09-18T11:32:44.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046228399999999996 2024-09-18T11:32:44.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993151+0000", 2024-09-18T11:32:44.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8587999999999997e-05 2024-09-18T11:32:44.942 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.942 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.942 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.943 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.13:63 3.0 e78/21 3:24ebb740:::smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 78'80, mlcod=73'78)", 2024-09-18T11:32:44.943 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:44.943 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 103.49684596900001, 2024-09-18T11:32:44.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052262230000000003, 2024-09-18T11:32:44.943 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.943 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.943 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:44.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:44.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.944 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.944 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405097+0000", 2024-09-18T11:32:44.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:44.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405103+0000", 2024-09-18T11:32:44.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8780000000000003e-06 2024-09-18T11:32:44.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405117+0000", 2024-09-18T11:32:44.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3441000000000001e-05 2024-09-18T11:32:44.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405121+0000", 2024-09-18T11:32:44.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7099999999999998e-06 2024-09-18T11:32:44.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405158+0000", 2024-09-18T11:32:44.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6477999999999999e-05 2024-09-18T11:32:44.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405173+0000", 2024-09-18T11:32:44.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4796e-05 2024-09-18T11:32:44.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410285+0000", 2024-09-18T11:32:44.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051118919999999998 2024-09-18T11:32:44.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410324+0000", 2024-09-18T11:32:44.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9616999999999997e-05 2024-09-18T11:32:44.947 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.947 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.947 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.947 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.13:18 3.0 e91/21 3:35e3c617:::2e64dd273da7619bd64b294f52905c1b9405a1ced2a8fd3c0b5bc481bc9732a6:head v 91'89, mlcod=85'87)", 2024-09-18T11:32:44.947 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:44.948 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 90.880070558, 2024-09-18T11:32:44.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048558539999999997, 2024-09-18T11:32:44.948 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.948 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.948 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:44.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:44.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:44.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:44.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021880+0000", 2024-09-18T11:32:44.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5860000000000001e-06 2024-09-18T11:32:44.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021890+0000", 2024-09-18T11:32:44.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8290000000000003e-06 2024-09-18T11:32:44.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021895+0000", 2024-09-18T11:32:44.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2480000000000004e-06 2024-09-18T11:32:44.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022166+0000", 2024-09-18T11:32:44.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027073499999999998 2024-09-18T11:32:44.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022184+0000", 2024-09-18T11:32:44.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8082000000000001e-05 2024-09-18T11:32:44.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026693+0000", 2024-09-18T11:32:44.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045093479999999998 2024-09-18T11:32:44.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026729+0000", 2024-09-18T11:32:44.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6282999999999998e-05 2024-09-18T11:32:44.952 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.952 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.952 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.952 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:573 4.4 e91/22 4:2c93196f:::smithi11939506-28:26 v 91'135, mlcod=90'131)", 2024-09-18T11:32:44.952 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:44.952 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 90.877603969000006, 2024-09-18T11:32:44.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050722739999999999, 2024-09-18T11:32:44.952 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.953 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.953 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:44.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:44.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024339+0000", 2024-09-18T11:32:44.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:44.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024349+0000", 2024-09-18T11:32:44.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6339999999999996e-06 2024-09-18T11:32:44.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024362+0000", 2024-09-18T11:32:44.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-18T11:32:44.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024368+0000", 2024-09-18T11:32:44.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8649999999999998e-06 2024-09-18T11:32:44.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024431+0000", 2024-09-18T11:32:44.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2780999999999994e-05 2024-09-18T11:32:44.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024448+0000", 2024-09-18T11:32:44.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7388999999999999e-05 2024-09-18T11:32:44.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029385+0000", 2024-09-18T11:32:44.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004936523 2024-09-18T11:32:44.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.956 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029412+0000", 2024-09-18T11:32:44.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7702999999999999e-05 2024-09-18T11:32:44.956 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.956 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.956 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.957 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:574 4.4 e91/22 4:2cb3e6a5:::smithi11939506-3:1e v 91'137, mlcod=90'131)", 2024-09-18T11:32:44.957 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:44.957 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 90.877165926000004, 2024-09-18T11:32:44.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070802850000000004, 2024-09-18T11:32:44.957 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.957 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.957 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:44.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:44.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024777+0000", 2024-09-18T11:32:44.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:44.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024785+0000", 2024-09-18T11:32:44.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8099999999999998e-06 2024-09-18T11:32:44.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024794+0000", 2024-09-18T11:32:44.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5839999999999995e-06 2024-09-18T11:32:44.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024798+0000", 2024-09-18T11:32:44.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.031e-06 2024-09-18T11:32:44.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024830+0000", 2024-09-18T11:32:44.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2268999999999999e-05 2024-09-18T11:32:44.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024847+0000", 2024-09-18T11:32:44.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7306999999999999e-05 2024-09-18T11:32:44.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031834+0000", 2024-09-18T11:32:44.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069868380000000004 2024-09-18T11:32:44.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031858+0000", 2024-09-18T11:32:44.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4082000000000001e-05 2024-09-18T11:32:44.961 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.961 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.961 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.961 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:541 4.d e91/22 4:b9bb642b:::smithi11939506-27:19 v 91'158, mlcod=91'154)", 2024-09-18T11:32:44.961 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:44.961 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 90.869785234000005, 2024-09-18T11:32:44.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052185, 2024-09-18T11:32:44.962 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.962 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.962 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:44.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:44.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032158+0000", 2024-09-18T11:32:44.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:44.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032165+0000", 2024-09-18T11:32:44.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-09-18T11:32:44.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032176+0000", 2024-09-18T11:32:44.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1460999999999999e-05 2024-09-18T11:32:44.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032182+0000", 2024-09-18T11:32:44.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0870000000000003e-06 2024-09-18T11:32:44.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032224+0000", 2024-09-18T11:32:44.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1372999999999998e-05 2024-09-18T11:32:44.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032246+0000", 2024-09-18T11:32:44.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2158e-05 2024-09-18T11:32:44.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037348+0000", 2024-09-18T11:32:44.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051025319999999999 2024-09-18T11:32:44.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037377+0000", 2024-09-18T11:32:44.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8937000000000001e-05 2024-09-18T11:32:44.966 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.966 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.966 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.966 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4178.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-18T11:32:44.966 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:44.966 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 62.961123168999997, 2024-09-18T11:32:44.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048960139999999997, 2024-09-18T11:32:44.966 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.966 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:44.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:44.966 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4178", 2024-09-18T11:32:44.967 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/490761693", 2024-09-18T11:32:44.967 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4 2024-09-18T11:32:44.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.967 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:44.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:44.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940819+0000", 2024-09-18T11:32:44.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:44.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940832+0000", 2024-09-18T11:32:44.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3155e-05 2024-09-18T11:32:44.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940848+0000", 2024-09-18T11:32:44.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6288999999999999e-05 2024-09-18T11:32:44.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940856+0000", 2024-09-18T11:32:44.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4919999999999996e-06 2024-09-18T11:32:44.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945613+0000", 2024-09-18T11:32:44.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047574999999999996 2024-09-18T11:32:44.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945717+0000", 2024-09-18T11:32:44.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103553 2024-09-18T11:32:44.970 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.970 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.970 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.970 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3482 4.6 e117/67 4:67c1765a:::smithi11939506-30:head v 117'120, mlcod=115'116)", 2024-09-18T11:32:44.970 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:44.970 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.217268945999997, 2024-09-18T11:32:44.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050037479999999997, 2024-09-18T11:32:44.971 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.971 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.971 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:44.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:44.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684674+0000", 2024-09-18T11:32:44.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:44.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684950+0000", 2024-09-18T11:32:44.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027621200000000002 2024-09-18T11:32:44.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684968+0000", 2024-09-18T11:32:44.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8e-05 2024-09-18T11:32:44.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684984+0000", 2024-09-18T11:32:44.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5775000000000001e-05 2024-09-18T11:32:44.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685033+0000", 2024-09-18T11:32:44.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8878000000000002e-05 2024-09-18T11:32:44.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685059+0000", 2024-09-18T11:32:44.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6801999999999999e-05 2024-09-18T11:32:44.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689640+0000", 2024-09-18T11:32:44.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045809029999999999 2024-09-18T11:32:44.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689679+0000", 2024-09-18T11:32:44.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8507000000000002e-05 2024-09-18T11:32:44.974 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.974 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.974 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.975 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:976 4.4 e132/22 4:2cb3e6a5:::smithi11939506-3:30 v 132'195, mlcod=128'192)", 2024-09-18T11:32:44.975 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:44.975 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 41.883297874999997, 2024-09-18T11:32:44.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0058526510000000004, 2024-09-18T11:32:44.975 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.975 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.975 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:44.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:44.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:44.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:44.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018661+0000", 2024-09-18T11:32:44.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5565000000000001e-05 2024-09-18T11:32:44.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018672+0000", 2024-09-18T11:32:44.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0210000000000001e-05 2024-09-18T11:32:44.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018678+0000", 2024-09-18T11:32:44.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1539999999999999e-06 2024-09-18T11:32:44.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018716+0000", 2024-09-18T11:32:44.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8389000000000001e-05 2024-09-18T11:32:44.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018737+0000", 2024-09-18T11:32:44.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0611e-05 2024-09-18T11:32:44.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.024458+0000", 2024-09-18T11:32:44.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057208629999999996 2024-09-18T11:32:44.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.024499+0000", 2024-09-18T11:32:44.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1100999999999998e-05 2024-09-18T11:32:44.979 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.979 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.979 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.979 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:823 4.d e137/22 4:b44229a1:::smithi11939506-42:49 v 137'266, mlcod=132'260)", 2024-09-18T11:32:44.979 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:32:44.979 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.624649319, 2024-09-18T11:32:44.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054497770000000003, 2024-09-18T11:32:44.980 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.980 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.980 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:32:44.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:32:44.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277294+0000", 2024-09-18T11:32:44.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:44.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277299+0000", 2024-09-18T11:32:44.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6280000000000004e-06 2024-09-18T11:32:44.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277306+0000", 2024-09-18T11:32:44.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.533e-06 2024-09-18T11:32:44.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277316+0000", 2024-09-18T11:32:44.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0148e-05 2024-09-18T11:32:44.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277868+0000", 2024-09-18T11:32:44.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00055199999999999997 2024-09-18T11:32:44.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277898+0000", 2024-09-18T11:32:44.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0264999999999999e-05 2024-09-18T11:32:44.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.282715+0000", 2024-09-18T11:32:44.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048165619999999999 2024-09-18T11:32:44.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.282744+0000", 2024-09-18T11:32:44.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9855000000000001e-05 2024-09-18T11:32:44.983 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.983 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.984 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.984 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:163 3.0 e138/21 3:02e8b30e:::4456d047c09ae2a9a25ccebfd83e30cf78df6779d63bfba9365afd7301eaeb6a:head v 138'132, mlcod=134'130)", 2024-09-18T11:32:44.984 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:44.984 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.319786575999998, 2024-09-18T11:32:44.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051351770000000003, 2024-09-18T11:32:44.984 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.984 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.984 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:44.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:44.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:44.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:44.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582232+0000", 2024-09-18T11:32:44.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5333999999999994e-05 2024-09-18T11:32:44.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582242+0000", 2024-09-18T11:32:44.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5200000000000003e-06 2024-09-18T11:32:44.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582248+0000", 2024-09-18T11:32:44.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8000000000000004e-06 2024-09-18T11:32:44.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582305+0000", 2024-09-18T11:32:44.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7813999999999999e-05 2024-09-18T11:32:44.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582329+0000", 2024-09-18T11:32:44.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3414999999999999e-05 2024-09-18T11:32:44.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587257+0000", 2024-09-18T11:32:44.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049279880000000003 2024-09-18T11:32:44.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587293+0000", 2024-09-18T11:32:44.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5802e-05 2024-09-18T11:32:44.988 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.988 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.988 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.988 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3974 4.d e138/22 4:b5a463bf:::smithi11939506-20:head v 138'270, mlcod=137'266)", 2024-09-18T11:32:44.988 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:32:44.989 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.315905305000001, 2024-09-18T11:32:44.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011228458, 2024-09-18T11:32:44.989 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.989 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.989 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:32:44.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:32:44.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586038+0000", 2024-09-18T11:32:44.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:44.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587638+0000", 2024-09-18T11:32:44.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001599184 2024-09-18T11:32:44.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587647+0000", 2024-09-18T11:32:44.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3719999999999994e-06 2024-09-18T11:32:44.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587653+0000", 2024-09-18T11:32:44.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.105e-06 2024-09-18T11:32:44.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587712+0000", 2024-09-18T11:32:44.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9252000000000001e-05 2024-09-18T11:32:44.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587735+0000", 2024-09-18T11:32:44.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3132000000000001e-05 2024-09-18T11:32:44.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597228+0000", 2024-09-18T11:32:44.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0094930090000000002 2024-09-18T11:32:44.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597267+0000", 2024-09-18T11:32:44.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8693999999999999e-05 2024-09-18T11:32:44.992 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.993 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.993 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.993 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3975 4.d e138/22 4:b5a463bf:::smithi11939506-20:head v 138'271, mlcod=137'266)", 2024-09-18T11:32:44.993 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:44.993 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.309704599, 2024-09-18T11:32:44.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053620450000000002, 2024-09-18T11:32:44.993 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:44.993 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:44.993 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:44.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:44.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:44.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:44.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:44.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:44.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:44.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592927+0000", 2024-09-18T11:32:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00068878699999999995 2024-09-18T11:32:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592940+0000", 2024-09-18T11:32:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:32:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592951+0000", 2024-09-18T11:32:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1046000000000001e-05 2024-09-18T11:32:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593008+0000", 2024-09-18T11:32:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6875e-05 2024-09-18T11:32:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593033+0000", 2024-09-18T11:32:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5341999999999999e-05 2024-09-18T11:32:44.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597565+0000", 2024-09-18T11:32:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045323150000000003 2024-09-18T11:32:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597601+0000", 2024-09-18T11:32:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6071000000000001e-05 2024-09-18T11:32:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.997 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:44.998 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:44.998 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:44.999 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:32:45.000 INFO:tasks.rados.rados.0.smithi119.stdout:1980: finishing write tid 2 to smithi11939506-44 2024-09-18T11:32:45.000 INFO:tasks.rados.rados.0.smithi119.stdout:1980: finishing write tid 3 to smithi11939506-44 2024-09-18T11:32:45.000 INFO:tasks.rados.rados.0.smithi119.stdout:1977: done (10 left) 2024-09-18T11:32:45.001 INFO:tasks.rados.rados.0.smithi119.stdout:1978: done (9 left) 2024-09-18T11:32:45.001 INFO:tasks.rados.rados.0.smithi119.stdout:1979: done (8 left) 2024-09-18T11:32:45.001 INFO:tasks.rados.rados.0.smithi119.stdout:1981: done (7 left) 2024-09-18T11:32:45.001 INFO:tasks.rados.rados.0.smithi119.stdout:1988: done (6 left) 2024-09-18T11:32:45.001 INFO:tasks.rados.rados.0.smithi119.stdout:1989: rollback oid 21 current snap is 67 2024-09-18T11:32:45.001 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 21 to 55 2024-09-18T11:32:45.001 INFO:tasks.rados.rados.0.smithi119.stdout:1980: finishing write tid 4 to smithi11939506-44 2024-09-18T11:32:45.001 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 44 v 285 (ObjNum 406 snap 67 seq_num 406) dirty exists 2024-09-18T11:32:45.001 INFO:tasks.rados.rados.0.smithi119.stdout:1980: left oid 44 (ObjNum 406 snap 67 seq_num 406) 2024-09-18T11:32:45.001 INFO:tasks.rados.rados.0.smithi119.stdout:1982: expect (ObjNum 387 snap 62 seq_num 387) 2024-09-18T11:32:45.002 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:44.997+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:45.074 INFO:tasks.rados.rados.0.smithi119.stdout:1983: expect (ObjNum 323 snap 45 seq_num 323) 2024-09-18T11:32:45.099 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:45.098+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:45.138 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:45.138 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:45.138 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:45.138 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:45.146 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:32:45.201 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:45.199+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:45.285 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:45.286 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:45.286 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:45.286 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:45.286 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:45.294 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:32:45.302 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:45.300+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:45.317 INFO:tasks.rados.rados.0.smithi119.stdout:1984: finishing write tid 1 to smithi11939506-9 2024-09-18T11:32:45.317 INFO:tasks.rados.rados.0.smithi119.stdout:1984: finishing write tid 2 to smithi11939506-9 2024-09-18T11:32:45.317 INFO:tasks.rados.rados.0.smithi119.stdout:1984: finishing write tid 3 to smithi11939506-9 2024-09-18T11:32:45.317 INFO:tasks.rados.rados.0.smithi119.stdout:1980: done (6 left) 2024-09-18T11:32:45.317 INFO:tasks.rados.rados.0.smithi119.stdout:1982: done (5 left) 2024-09-18T11:32:45.317 INFO:tasks.rados.rados.0.smithi119.stdout:1983: done (4 left) 2024-09-18T11:32:45.318 INFO:tasks.rados.rados.0.smithi119.stdout:1990: rollback oid 18 current snap is 67 2024-09-18T11:32:45.318 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 18 to 61 2024-09-18T11:32:45.318 INFO:tasks.rados.rados.0.smithi119.stdout:1984: finishing write tid 4 to smithi11939506-9 2024-09-18T11:32:45.318 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 596 (ObjNum 407 snap 67 seq_num 407) dirty exists 2024-09-18T11:32:45.318 INFO:tasks.rados.rados.0.smithi119.stdout:1984: left oid 9 (ObjNum 407 snap 67 seq_num 407) 2024-09-18T11:32:45.318 INFO:tasks.rados.rados.0.smithi119.stdout:1985: finishing write tid 1 to smithi11939506-25 2024-09-18T11:32:45.318 INFO:tasks.rados.rados.0.smithi119.stdout:1985: finishing write tid 2 to smithi11939506-25 2024-09-18T11:32:45.318 INFO:tasks.rados.rados.0.smithi119.stdout:1985: finishing write tid 3 to smithi11939506-25 2024-09-18T11:32:45.318 INFO:tasks.rados.rados.0.smithi119.stdout:1985: finishing write tid 4 to smithi11939506-25 2024-09-18T11:32:45.318 INFO:tasks.rados.rados.0.smithi119.stdout:1985: finishing write tid 5 to smithi11939506-25 2024-09-18T11:32:45.318 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 25 v 95 (ObjNum 408 snap 67 seq_num 408) dirty exists 2024-09-18T11:32:45.318 INFO:tasks.rados.rados.0.smithi119.stdout:1985: left oid 25 (ObjNum 408 snap 67 seq_num 408) 2024-09-18T11:32:45.319 INFO:tasks.rados.rados.0.smithi119.stdout:1987: finishing write tid 1 to smithi11939506-27 2024-09-18T11:32:45.319 INFO:tasks.rados.rados.0.smithi119.stdout:1987: finishing write tid 2 to smithi11939506-27 2024-09-18T11:32:45.319 INFO:tasks.rados.rados.0.smithi119.stdout:1987: finishing write tid 3 to smithi11939506-27 2024-09-18T11:32:45.319 INFO:tasks.rados.rados.0.smithi119.stdout:1987: finishing write tid 4 to smithi11939506-27 2024-09-18T11:32:45.319 INFO:tasks.rados.rados.0.smithi119.stdout:1987: finishing write tid 5 to smithi11939506-27 2024-09-18T11:32:45.319 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 27 v 402 (ObjNum 409 snap 67 seq_num 409) dirty exists 2024-09-18T11:32:45.319 INFO:tasks.rados.rados.0.smithi119.stdout:1987: left oid 27 (ObjNum 409 snap 67 seq_num 409) 2024-09-18T11:32:45.319 INFO:tasks.rados.rados.0.smithi119.stdout:1989: finishing rollback tid 0 to smithi11939506-21 2024-09-18T11:32:45.319 INFO:tasks.rados.rados.0.smithi119.stdout:1989: finishing rollback tid 1 to smithi11939506-21 2024-09-18T11:32:45.319 INFO:tasks.rados.rados.0.smithi119.stdout:1989: finishing rollback tid 2 to smithi11939506-21 2024-09-18T11:32:45.319 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 21 v 424 (ObjNum 311 snap 43 seq_num 311) dirty exists 2024-09-18T11:32:45.319 INFO:tasks.rados.rados.0.smithi119.stdout:1984: done (4 left) 2024-09-18T11:32:45.319 INFO:tasks.rados.rados.0.smithi119.stdout:1985: done (3 left) 2024-09-18T11:32:45.319 INFO:tasks.rados.rados.0.smithi119.stdout:1987: done (2 left) 2024-09-18T11:32:45.320 INFO:tasks.rados.rados.0.smithi119.stdout:1989: done (1 left) 2024-09-18T11:32:45.320 INFO:tasks.rados.rados.0.smithi119.stdout:1991: set_chunk oid 3 target oid 2024-09-18T11:32:45.321 INFO:tasks.rados.rados.0.smithi119.stdout:1991: set_chunk oid 3 offset: 160256 length: 18432 target oid f7b586904e3678145aa47e4232587c913139cef0102d6d8e9276fc80c35cbad3 offset: 0 snap -1 2024-09-18T11:32:45.321 INFO:tasks.rados.rados.0.smithi119.stdout:1992: rollback oid 14 current snap is 67 2024-09-18T11:32:45.321 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 14 to 60 2024-09-18T11:32:45.321 INFO:tasks.rados.rados.0.smithi119.stdout:1993: read oid 42 snap -1 2024-09-18T11:32:45.321 INFO:tasks.rados.rados.0.smithi119.stdout:1993: expect (ObjNum 386 snap 61 seq_num 386) 2024-09-18T11:32:45.321 INFO:tasks.rados.rados.0.smithi119.stdout:1994: read oid 40 snap -1 2024-09-18T11:32:45.321 INFO:tasks.rados.rados.0.smithi119.stdout:1994: expect (ObjNum 287 snap 41 seq_num 287) 2024-09-18T11:32:45.321 INFO:tasks.rados.rados.0.smithi119.stdout:1995: read oid 34 snap 66 2024-09-18T11:32:45.322 INFO:tasks.rados.rados.0.smithi119.stdout:1995: expect (ObjNum 329 snap 48 seq_num 329) 2024-09-18T11:32:45.322 INFO:tasks.rados.rados.0.smithi119.stdout:1996: read oid 20 snap -1 2024-09-18T11:32:45.322 INFO:tasks.rados.rados.0.smithi119.stdout:1996: expect (ObjNum 349 snap 54 seq_num 349) 2024-09-18T11:32:45.322 INFO:tasks.rados.rados.0.smithi119.stdout:1997: read oid 29 snap -1 2024-09-18T11:32:45.322 INFO:tasks.rados.rados.0.smithi119.stdout:1997: expect (ObjNum 355 snap 55 seq_num 355) 2024-09-18T11:32:45.322 INFO:tasks.rados.rados.0.smithi119.stdout:1998: read oid 11 snap -1 2024-09-18T11:32:45.322 INFO:tasks.rados.rados.0.smithi119.stdout:1998: expect (ObjNum 400 snap 65 seq_num 400) 2024-09-18T11:32:45.322 INFO:tasks.rados.rados.0.smithi119.stdout:1999: write oid 5 current snap is 67 2024-09-18T11:32:45.322 INFO:tasks.rados.rados.0.smithi119.stdout:1999: seq_num 411 ranges {507677=410401,1367511=517802,2420497=487180} 2024-09-18T11:32:45.325 INFO:tasks.rados.rados.0.smithi119.stdout:1999: writing smithi11939506-5 from 507677 to 918078 tid 1 2024-09-18T11:32:45.330 INFO:tasks.rados.rados.0.smithi119.stdout:1999: writing smithi11939506-5 from 1367511 to 1885313 tid 2 2024-09-18T11:32:45.337 INFO:tasks.rados.rados.0.smithi119.stdout:1999: writing smithi11939506-5 from 2420497 to 2907677 tid 3 2024-09-18T11:32:45.337 INFO:tasks.rados.rados.0.smithi119.stdout:2000: rollback oid 31 current snap is 67 2024-09-18T11:32:45.337 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 31 to 60 2024-09-18T11:32:45.338 INFO:tasks.rados.rados.0.smithi119.stdout:1991: finishing set_chunk to oid 3 2024-09-18T11:32:45.338 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 3 set_chunk returned error code -95 offset: 160256 length: 18432 2024-09-18T11:32:45.338 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 3 v 0 (ObjNum 243 snap 28 seq_num 243) dirty exists 2024-09-18T11:32:45.338 INFO:tasks.rados.rados.0.smithi119.stdout:1990: finishing rollback tid 0 to smithi11939506-18 2024-09-18T11:32:45.338 INFO:tasks.rados.rados.0.smithi119.stdout:1992: finishing rollback tid 0 to smithi11939506-14 2024-09-18T11:32:45.338 INFO:tasks.rados.rados.0.smithi119.stdout:1993: expect (ObjNum 386 snap 61 seq_num 386) 2024-09-18T11:32:45.402 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:45.401+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:45.428 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:32:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 154.256946494, 2024-09-18T11:32:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:32:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:45.429 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:32:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:32:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:45.431 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:32:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:32:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:32:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:32:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:32:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:32:45.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:32:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:32:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:32:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:32:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:45.433 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:32:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:32:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 150.26176849999999, 2024-09-18T11:32:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:32:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:45.434 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:32:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:32:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:32:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:32:45.436 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:32:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:32:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:32:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:32:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:32:45.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:32:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:32:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:32:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:32:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:32:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:32:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:32:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:32:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 150.26175021099999, 2024-09-18T11:32:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:32:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:32:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.440 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:32:45.441 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:32:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:32:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:32:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:32:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:32:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:32:45.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:32:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:32:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:32:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:32:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.443 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:32:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:32:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:32:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:32:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:32:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:32:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.444 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 150.26168944200001, 2024-09-18T11:32:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:32:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:32:45.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.446 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:32:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:32:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:32:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:32:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:32:45.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:32:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:32:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:32:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:32:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:45.448 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:32:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:32:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:32:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:32:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:32:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:32:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:32:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:32:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 150.26166712099999, 2024-09-18T11:32:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:32:45.450 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:32:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:32:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:32:45.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:32:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:32:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:32:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:32:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:32:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:32:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:32:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:32:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:32:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:32:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:32:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:45.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:32:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:32:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:32:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:32:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:32:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 145.480542238, 2024-09-18T11:32:45.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:32:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:32:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:32:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:32:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:32:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:32:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:32:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:32:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:32:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:32:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:32:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:32:45.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:32:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:32:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:32:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:32:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:45.459 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:32:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:32:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:32:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:32:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:32:45.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:32:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:32:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:32:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2503 3.1 e85/67 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head v 85'117, mlcod=84'115)", 2024-09-18T11:32:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.449143551000006, 2024-09-18T11:32:45.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086527619999999996, 2024-09-18T11:32:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977220+0000", 2024-09-18T11:32:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980771+0000", 2024-09-18T11:32:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035505200000000002 2024-09-18T11:32:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980797+0000", 2024-09-18T11:32:45.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.618e-05 2024-09-18T11:32:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980807+0000", 2024-09-18T11:32:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0433e-05 2024-09-18T11:32:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980861+0000", 2024-09-18T11:32:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3415999999999998e-05 2024-09-18T11:32:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980885+0000", 2024-09-18T11:32:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4212000000000001e-05 2024-09-18T11:32:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985826+0000", 2024-09-18T11:32:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049412450000000004 2024-09-18T11:32:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985875+0000", 2024-09-18T11:32:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9116000000000002e-05 2024-09-18T11:32:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.465 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2649 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.095000182000007, 2024-09-18T11:32:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566240000000007, 2024-09-18T11:32:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:45.466 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2649 2024-09-18T11:32:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331365+0000", 2024-09-18T11:32:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331369+0000", 2024-09-18T11:32:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.551e-06 2024-09-18T11:32:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331377+0000", 2024-09-18T11:32:45.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8769999999999993e-06 2024-09-18T11:32:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331381+0000", 2024-09-18T11:32:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.16e-06 2024-09-18T11:32:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338973+0000", 2024-09-18T11:32:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075918460000000002 2024-09-18T11:32:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339048+0000", 2024-09-18T11:32:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5813000000000001e-05 2024-09-18T11:32:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339322+0000", 2024-09-18T11:32:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027399899999999998 2024-09-18T11:32:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:45.470 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2650 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 91.094979964000004, 2024-09-18T11:32:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084392180000000001, 2024-09-18T11:32:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:45.471 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2650 2024-09-18T11:32:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331388+0000", 2024-09-18T11:32:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3429999999999998e-06 2024-09-18T11:32:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331394+0000", 2024-09-18T11:32:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.19e-06 2024-09-18T11:32:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331398+0000", 2024-09-18T11:32:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.495e-06 2024-09-18T11:32:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339362+0000", 2024-09-18T11:32:45.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079641799999999995 2024-09-18T11:32:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339460+0000", 2024-09-18T11:32:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8329e-05 2024-09-18T11:32:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339825+0000", 2024-09-18T11:32:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036510500000000001 2024-09-18T11:32:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:127 3.2 3:4d3b277a:::3266c7aaa53a7d2c10c3c54d852d6cbbed49fb041bc0df41a3ac83b56e2b0615:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 82.381965025, 2024-09-18T11:32:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093148009999999993, 2024-09-18T11:32:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044400+0000", 2024-09-18T11:32:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044442+0000", 2024-09-18T11:32:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2247999999999999e-05 2024-09-18T11:32:45.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044453+0000", 2024-09-18T11:32:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0617e-05 2024-09-18T11:32:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044457+0000", 2024-09-18T11:32:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2520000000000001e-06 2024-09-18T11:32:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044495+0000", 2024-09-18T11:32:45.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7710000000000003e-05 2024-09-18T11:32:45.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:45.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044499+0000", 2024-09-18T11:32:45.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6269999999999999e-06 2024-09-18T11:32:45.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048239+0000", 2024-09-18T11:32:45.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037396719999999999 2024-09-18T11:32:45.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:45.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048351+0000", 2024-09-18T11:32:45.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112342 2024-09-18T11:32:45.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:45.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048545+0000", 2024-09-18T11:32:45.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000193881 2024-09-18T11:32:45.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:45.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.052689+0000", 2024-09-18T11:32:45.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041435780000000002 2024-09-18T11:32:45.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:45.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053654+0000", 2024-09-18T11:32:45.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096558199999999999 2024-09-18T11:32:45.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:45.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053683+0000", 2024-09-18T11:32:45.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8850999999999999e-05 2024-09-18T11:32:45.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:45.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053716+0000", 2024-09-18T11:32:45.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2468000000000002e-05 2024-09-18T11:32:45.482 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.482 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:45.482 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.482 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3209 4.0 4:019b6e09:::smithi11939506-40:head [write 638732~557432] snapc 42=[42,41,3f,3e,3d,3b,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-18T11:32:45.482 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:45.482 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 68.328282986000005, 2024-09-18T11:32:45.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091774999999999999, 2024-09-18T11:32:45.482 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:45.482 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:45.482 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:45.483 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:45.483 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:45.483 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3209 2024-09-18T11:32:45.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.483 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:45.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:45.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:45.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:45.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:45.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:45.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098080+0000", 2024-09-18T11:32:45.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:45.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:45.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098676+0000", 2024-09-18T11:32:45.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059598099999999998 2024-09-18T11:32:45.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:45.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098706+0000", 2024-09-18T11:32:45.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0020000000000001e-05 2024-09-18T11:32:45.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098716+0000", 2024-09-18T11:32:45.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0225e-05 2024-09-18T11:32:45.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098782+0000", 2024-09-18T11:32:45.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5530000000000004e-05 2024-09-18T11:32:45.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:45.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098883+0000", 2024-09-18T11:32:45.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101609 2024-09-18T11:32:45.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:45.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.099098+0000", 2024-09-18T11:32:45.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000215052 2024-09-18T11:32:45.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:45.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.102511+0000", 2024-09-18T11:32:45.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034129429999999999 2024-09-18T11:32:45.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:45.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107165+0000", 2024-09-18T11:32:45.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046531480000000002 2024-09-18T11:32:45.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:45.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107217+0000", 2024-09-18T11:32:45.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2058000000000002e-05 2024-09-18T11:32:45.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:45.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107260+0000", 2024-09-18T11:32:45.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3865000000000001e-05 2024-09-18T11:32:45.488 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.488 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:45.488 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.488 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:162 3.1 e138/67 3:8c826a1f:::e682cfc1e82c88472b9acf1cd97978bc105aa035f10b685bb4cace95809b52bf:head v 138'176, mlcod=136'174)", 2024-09-18T11:32:45.488 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:32:45.488 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.842809240999998, 2024-09-18T11:32:45.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010445024000000001, 2024-09-18T11:32:45.488 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:45.488 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:45.488 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:45.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:45.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:32:45.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:45.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:32:45.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:45.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583556+0000", 2024-09-18T11:32:45.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:45.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:45.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584899+0000", 2024-09-18T11:32:45.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001343056 2024-09-18T11:32:45.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:45.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584929+0000", 2024-09-18T11:32:45.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0133000000000001e-05 2024-09-18T11:32:45.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584937+0000", 2024-09-18T11:32:45.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6380000000000004e-06 2024-09-18T11:32:45.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584990+0000", 2024-09-18T11:32:45.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3186000000000002e-05 2024-09-18T11:32:45.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:45.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585012+0000", 2024-09-18T11:32:45.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1474000000000001e-05 2024-09-18T11:32:45.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:45.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593963+0000", 2024-09-18T11:32:45.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089512859999999993 2024-09-18T11:32:45.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:45.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594002+0000", 2024-09-18T11:32:45.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8859999999999997e-05 2024-09-18T11:32:45.492 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.492 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:45.492 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.492 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:166 3.1 e138/67 3:a73e635b:::b6ee478f0f661ec7bac3d17c2ca2fdaff71a47570f64844a7ba79bbb2a0e8d9d:head v 138'177, mlcod=136'174)", 2024-09-18T11:32:45.493 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:32:45.493 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.841239356999999, 2024-09-18T11:32:45.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089018680000000003, 2024-09-18T11:32:45.493 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:45.493 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:45.493 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:45.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:45.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:32:45.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:45.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:32:45.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:45.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585126+0000", 2024-09-18T11:32:45.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:45.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:45.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585211+0000", 2024-09-18T11:32:45.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5432999999999993e-05 2024-09-18T11:32:45.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:45.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585220+0000", 2024-09-18T11:32:45.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5879999999999998e-06 2024-09-18T11:32:45.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585225+0000", 2024-09-18T11:32:45.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8999999999999997e-06 2024-09-18T11:32:45.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585705+0000", 2024-09-18T11:32:45.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00048042799999999997 2024-09-18T11:32:45.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:45.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585724+0000", 2024-09-18T11:32:45.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8352e-05 2024-09-18T11:32:45.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:45.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594008+0000", 2024-09-18T11:32:45.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082840919999999998 2024-09-18T11:32:45.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:45.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594028+0000", 2024-09-18T11:32:45.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0869e-05 2024-09-18T11:32:45.497 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.497 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:45.497 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.497 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:168 3.1 e138/67 3:b64d7bad:::7feb22256bdcec217720fd46184ab43ced8923ed66ec6319c57f078d9f123262:head v 138'178, mlcod=136'174)", 2024-09-18T11:32:45.497 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:45.497 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.840247624, 2024-09-18T11:32:45.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079290650000000008, 2024-09-18T11:32:45.498 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:45.498 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:45.498 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:45.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:45.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:45.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:45.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:45.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586191+0000", 2024-09-18T11:32:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3627000000000005e-05 2024-09-18T11:32:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586202+0000", 2024-09-18T11:32:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0488e-05 2024-09-18T11:32:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586207+0000", 2024-09-18T11:32:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1719999999999998e-06 2024-09-18T11:32:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586239+0000", 2024-09-18T11:32:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2570000000000002e-05 2024-09-18T11:32:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586271+0000", 2024-09-18T11:32:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1341999999999999e-05 2024-09-18T11:32:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594031+0000", 2024-09-18T11:32:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077597459999999997 2024-09-18T11:32:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594047+0000", 2024-09-18T11:32:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6858e-05 2024-09-18T11:32:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.501 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4267 3.1 3:8d8d13e9:::92886a4340d03039afb2c877fad76e3afe7a80ee04388d55256c386022cd5d63:head [write 0~2203648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:32:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.369215644000001, 2024-09-18T11:32:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0092856740000000007, 2024-09-18T11:32:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4267 2024-09-18T11:32:45.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:32:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:32:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:45.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057149+0000", 2024-09-18T11:32:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058175+0000", 2024-09-18T11:32:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001026765 2024-09-18T11:32:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058190+0000", 2024-09-18T11:32:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4167e-05 2024-09-18T11:32:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058195+0000", 2024-09-18T11:32:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.823e-06 2024-09-18T11:32:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058247+0000", 2024-09-18T11:32:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1304999999999998e-05 2024-09-18T11:32:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058422+0000", 2024-09-18T11:32:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017527100000000001 2024-09-18T11:32:45.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058609+0000", 2024-09-18T11:32:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018714299999999999 2024-09-18T11:32:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.062721+0000", 2024-09-18T11:32:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041118869999999998 2024-09-18T11:32:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066291+0000", 2024-09-18T11:32:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035694860000000002 2024-09-18T11:32:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066328+0000", 2024-09-18T11:32:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7864000000000002e-05 2024-09-18T11:32:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066436+0000", 2024-09-18T11:32:45.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010762 2024-09-18T11:32:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:90 3.1 3:a224eaa1:::22b064a083114a2756d85a07d2762a393d3e92dd898a5334bc045c9031244d76:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:32:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:40.057559+0000", 2024-09-18T11:32:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.3688068649999998, 2024-09-18T11:32:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085024620000000006, 2024-09-18T11:32:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:45.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:45.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057559+0000", 2024-09-18T11:32:45.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:45.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057559+0000", 2024-09-18T11:32:45.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:45.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057558+0000", 2024-09-18T11:32:45.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:45.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:45.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057563+0000", 2024-09-18T11:32:45.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7990000000000001e-06 2024-09-18T11:32:45.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:45.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057572+0000", 2024-09-18T11:32:45.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0510000000000002e-06 2024-09-18T11:32:45.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057576+0000", 2024-09-18T11:32:45.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2509999999999996e-06 2024-09-18T11:32:45.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057624+0000", 2024-09-18T11:32:45.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7636000000000003e-05 2024-09-18T11:32:45.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:45.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057722+0000", 2024-09-18T11:32:45.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7676999999999993e-05 2024-09-18T11:32:45.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:45.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057883+0000", 2024-09-18T11:32:45.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000160922 2024-09-18T11:32:45.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:45.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059927+0000", 2024-09-18T11:32:45.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020447350000000002 2024-09-18T11:32:45.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:45.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.065977+0000", 2024-09-18T11:32:45.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060497679999999996 2024-09-18T11:32:45.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:45.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066023+0000", 2024-09-18T11:32:45.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6041000000000002e-05 2024-09-18T11:32:45.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:45.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066062+0000", 2024-09-18T11:32:45.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8469e-05 2024-09-18T11:32:45.513 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.513 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:45.513 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.514 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.5.13:42 3.1 3:8372521c:::d4b4fff22a71c591f3e871de3fede050f6721df41a8c639a614c0b94ba6762c2:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:32:45.514 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:40.058327+0000", 2024-09-18T11:32:45.514 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.3680391920000003, 2024-09-18T11:32:45.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079034699999999992, 2024-09-18T11:32:45.514 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:45.514 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:45.514 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:45.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:45.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058327+0000", 2024-09-18T11:32:45.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:45.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058327+0000", 2024-09-18T11:32:45.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:45.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058325+0000", 2024-09-18T11:32:45.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:45.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:45.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058339+0000", 2024-09-18T11:32:45.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3337e-05 2024-09-18T11:32:45.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:45.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058357+0000", 2024-09-18T11:32:45.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8615e-05 2024-09-18T11:32:45.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058366+0000", 2024-09-18T11:32:45.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9460000000000004e-06 2024-09-18T11:32:45.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058658+0000", 2024-09-18T11:32:45.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029212200000000002 2024-09-18T11:32:45.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:45.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058757+0000", 2024-09-18T11:32:45.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9025999999999997e-05 2024-09-18T11:32:45.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:45.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058914+0000", 2024-09-18T11:32:45.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000156236 2024-09-18T11:32:45.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:45.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.060326+0000", 2024-09-18T11:32:45.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014118710000000001 2024-09-18T11:32:45.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:45.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066122+0000", 2024-09-18T11:32:45.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057962029999999998 2024-09-18T11:32:45.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:45.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066192+0000", 2024-09-18T11:32:45.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0533999999999999e-05 2024-09-18T11:32:45.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:45.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066230+0000", 2024-09-18T11:32:45.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7914000000000003e-05 2024-09-18T11:32:45.520 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.520 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:45.520 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.520 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.5.13:43 3.1 3:b44664f3:::7f31751f088e4f7577561c03b406e8a8bc48ebfdd310b40206061655307aebe3:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:32:45.520 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:40.058386+0000", 2024-09-18T11:32:45.520 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.3679801640000004, 2024-09-18T11:32:45.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079575740000000002, 2024-09-18T11:32:45.520 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:45.521 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:45.521 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:45.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:45.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058386+0000", 2024-09-18T11:32:45.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:45.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058386+0000", 2024-09-18T11:32:45.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:45.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058385+0000", 2024-09-18T11:32:45.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:45.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:45.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058392+0000", 2024-09-18T11:32:45.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6200000000000001e-06 2024-09-18T11:32:45.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:45.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058400+0000", 2024-09-18T11:32:45.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7849999999999997e-06 2024-09-18T11:32:45.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058408+0000", 2024-09-18T11:32:45.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1770000000000001e-06 2024-09-18T11:32:45.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059302+0000", 2024-09-18T11:32:45.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000894851 2024-09-18T11:32:45.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:45.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059400+0000", 2024-09-18T11:32:45.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7460000000000005e-05 2024-09-18T11:32:45.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:45.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059520+0000", 2024-09-18T11:32:45.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000120457 2024-09-18T11:32:45.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:45.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.060333+0000", 2024-09-18T11:32:45.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00081255599999999998 2024-09-18T11:32:45.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:45.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066274+0000", 2024-09-18T11:32:45.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059409700000000003 2024-09-18T11:32:45.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:45.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066306+0000", 2024-09-18T11:32:45.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2020000000000002e-05 2024-09-18T11:32:45.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:45.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066343+0000", 2024-09-18T11:32:45.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7438999999999998e-05 2024-09-18T11:32:45.525 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.526 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:45.526 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.526 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:1265 4.f e173/22 4:f3c51b3f:::smithi11939506-21:6c v 173'421, mlcod=172'416)", 2024-09-18T11:32:45.526 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:44.453434+0000", 2024-09-18T11:32:45.526 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.97293180800000001, 2024-09-18T11:32:45.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007737893, 2024-09-18T11:32:45.526 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:45.526 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:45.526 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:45.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:45.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:44.453434+0000", 2024-09-18T11:32:45.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:45.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:44.453434+0000", 2024-09-18T11:32:45.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:45.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:44.453434+0000", 2024-09-18T11:32:45.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:45.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:45.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:44.453439+0000", 2024-09-18T11:32:45.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8069999999999999e-06 2024-09-18T11:32:45.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:45.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:44.453445+0000", 2024-09-18T11:32:45.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6460000000000004e-06 2024-09-18T11:32:45.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:44.453448+0000", 2024-09-18T11:32:45.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9299999999999999e-06 2024-09-18T11:32:45.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:44.455833+0000", 2024-09-18T11:32:45.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023846879999999998 2024-09-18T11:32:45.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:45.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:44.455855+0000", 2024-09-18T11:32:45.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1747999999999999e-05 2024-09-18T11:32:45.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:45.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:44.461134+0000", 2024-09-18T11:32:45.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052790659999999998 2024-09-18T11:32:45.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:45.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:44.461172+0000", 2024-09-18T11:32:45.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8452999999999999e-05 2024-09-18T11:32:45.530 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.530 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:45.531 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.531 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.4.13:145 3.2 3:435dd00a:::773f1f20723e27798deecfe7048ee8e76eeec814e6d6ad791ed305ff28c69be2:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:32:45.531 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:45.321055+0000", 2024-09-18T11:32:45.531 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 0.10531135899999999, 2024-09-18T11:32:45.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079066459999999998, 2024-09-18T11:32:45.531 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:45.531 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:45.531 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:45.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:45.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321055+0000", 2024-09-18T11:32:45.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:45.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321055+0000", 2024-09-18T11:32:45.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:45.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:45.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321053+0000", 2024-09-18T11:32:45.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:45.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:45.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321062+0000", 2024-09-18T11:32:45.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0399999999999998e-06 2024-09-18T11:32:45.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:45.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321076+0000", 2024-09-18T11:32:45.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3869999999999999e-05 2024-09-18T11:32:45.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321083+0000", 2024-09-18T11:32:45.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6159999999999998e-06 2024-09-18T11:32:45.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:45.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321156+0000", 2024-09-18T11:32:45.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3093e-05 2024-09-18T11:32:45.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:45.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321274+0000", 2024-09-18T11:32:45.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000118353 2024-09-18T11:32:45.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:45.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321443+0000", 2024-09-18T11:32:45.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016836600000000001 2024-09-18T11:32:45.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:45.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.324281+0000", 2024-09-18T11:32:45.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028381270000000002 2024-09-18T11:32:45.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:45.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.328785+0000", 2024-09-18T11:32:45.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045048370000000003 2024-09-18T11:32:45.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:45.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.328910+0000", 2024-09-18T11:32:45.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012438600000000001 2024-09-18T11:32:45.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:45.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:45.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:45.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.328961+0000", 2024-09-18T11:32:45.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1375000000000002e-05 2024-09-18T11:32:45.536 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.536 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:45.537 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.537 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:45.537 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:45.537 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:45.538 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:45.444+0000 7f6bf5056640 -1 osd.5 173 handle_osd_map injecting map crc failure 2024-09-18T11:32:45.539 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:32:45.539 INFO:teuthology.orchestra.run.smithi119.stderr:marked in osd.0. 2024-09-18T11:32:45.539 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:45.452+0000 7f2d429e4640 -1 osd.1 173 handle_osd_map injecting map crc failure 2024-09-18T11:32:45.540 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-18T11:32:45.540 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:45.536+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:45.589 INFO:tasks.rados.rados.0.smithi119.stdout:1995: expect (ObjNum 329 snap 48 seq_num 329) 2024-09-18T11:32:45.638 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:45.636+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:45.649 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:45.649 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:45.650 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:45.650 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:45.658 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:32:45.702 INFO:tasks.rados.rados.0.smithi119.stdout:1994: expect (ObjNum 287 snap 41 seq_num 287) 2024-09-18T11:32:45.742 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:45.740+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:45.797 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:45.797 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:45.797 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:45.798 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:45.798 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:45.806 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:32:45.840 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:45.839+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:45.864 INFO:tasks.rados.rados.0.smithi119.stdout:1992: finishing rollback tid 1 to smithi11939506-14 2024-09-18T11:32:45.864 INFO:tasks.rados.rados.0.smithi119.stdout:1992: finishing rollback tid 2 to smithi11939506-14 2024-09-18T11:32:45.864 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 14 v 389 (ObjNum 352 snap 54 seq_num 352) dirty exists 2024-09-18T11:32:45.864 INFO:tasks.rados.rados.0.smithi119.stdout:1999: finishing write tid 1 to smithi11939506-5 2024-09-18T11:32:45.864 INFO:tasks.rados.rados.0.smithi119.stdout:1999: finishing write tid 2 to smithi11939506-5 2024-09-18T11:32:45.864 INFO:tasks.rados.rados.0.smithi119.stdout:1996: expect (ObjNum 349 snap 54 seq_num 349) 2024-09-18T11:32:45.866 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:32:45.941 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:45.939+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:45.946 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:45.946 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:45.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:45.947 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:45.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.947 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:32:45.947 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:45.947 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 131.528058496, 2024-09-18T11:32:45.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:32:45.947 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:45.947 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:45.947 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:45.947 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:45.947 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:45.947 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:32:45.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.948 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:45.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:45.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:45.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:45.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:45.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:45.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:45.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:45.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:32:45.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:45.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:45.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:32:45.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:32:45.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:45.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:32:45.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:32:45.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:32:45.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:32:45.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:45.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:32:45.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:32:45.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:45.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:32:45.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:32:45.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:45.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:32:45.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:32:45.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:45.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:32:45.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:32:45.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:45.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:32:45.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:32:45.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:45.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:32:45.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:32:45.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:45.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:32:45.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:32:45.952 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:45.952 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:45.952 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:45.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.953 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:32:45.953 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:45.953 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 104.55782909, 2024-09-18T11:32:45.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:32:45.953 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:45.953 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:45.953 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:45.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:45.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:45.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:45.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:45.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:45.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:45.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:45.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:45.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:45.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:45.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:32:45.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:32:45.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:45.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:32:45.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:32:45.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:32:45.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:32:45.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:45.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:32:45.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:32:45.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:45.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:32:45.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:32:45.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:45.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:32:45.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:32:45.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:45.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:32:45.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:32:45.957 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:45.957 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:45.957 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:45.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.957 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:45.957 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:45.957 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 96.971716483999998, 2024-09-18T11:32:45.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:32:45.957 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:45.958 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:45.958 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:45.958 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:45.958 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:45.958 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:32:45.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.958 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:45.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:45.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:45.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:45.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:45.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:45.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:45.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:45.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:32:45.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:45.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:45.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:32:45.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:32:45.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:45.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:32:45.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:32:45.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:32:45.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:32:45.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:45.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:32:45.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:32:45.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:45.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:32:45.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:32:45.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:45.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:32:45.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:32:45.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:45.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:32:45.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:32:45.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:45.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:32:45.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:32:45.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:45.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:32:45.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:32:45.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:45.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:32:45.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:32:45.963 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:45.963 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:45.963 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:45.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.963 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2956 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~2100908,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:45.963 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:45.963 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 81.916920234000003, 2024-09-18T11:32:45.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011863831, 2024-09-18T11:32:45.964 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:45.964 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:45.964 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:45.964 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:45.964 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:45.964 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2956 2024-09-18T11:32:45.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.964 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:45.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:45.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:45.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:45.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:45.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:45.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:45.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:45.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:45.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:45.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:45.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027228+0000", 2024-09-18T11:32:45.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:32:45.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:45.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027235+0000", 2024-09-18T11:32:45.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:32:45.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027239+0000", 2024-09-18T11:32:45.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.535e-06 2024-09-18T11:32:45.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:45.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027831+0000", 2024-09-18T11:32:45.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059244699999999996 2024-09-18T11:32:45.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:45.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027879+0000", 2024-09-18T11:32:45.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:32:45.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:45.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038098+0000", 2024-09-18T11:32:45.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010219137 2024-09-18T11:32:45.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:45.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038207+0000", 2024-09-18T11:32:45.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010951999999999999 2024-09-18T11:32:45.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:45.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039089+0000", 2024-09-18T11:32:45.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00088151699999999996 2024-09-18T11:32:45.968 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:45.968 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:45.968 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:45.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.968 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2957 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:45.969 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:45.969 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 81.916902449999995, 2024-09-18T11:32:45.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012041979, 2024-09-18T11:32:45.969 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:45.969 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:45.969 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:45.969 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:45.969 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:45.969 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2957 2024-09-18T11:32:45.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.969 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:45.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:45.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:45.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:45.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:45.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:45.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:45.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:45.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:45.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:45.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:45.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027245+0000", 2024-09-18T11:32:45.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:32:45.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:45.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027251+0000", 2024-09-18T11:32:45.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0090000000000004e-06 2024-09-18T11:32:45.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027255+0000", 2024-09-18T11:32:45.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:32:45.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:45.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027895+0000", 2024-09-18T11:32:45.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064076400000000005 2024-09-18T11:32:45.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:45.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027930+0000", 2024-09-18T11:32:45.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4334e-05 2024-09-18T11:32:45.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:45.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039111+0000", 2024-09-18T11:32:45.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181455999999999 2024-09-18T11:32:45.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:45.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039156+0000", 2024-09-18T11:32:45.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5256000000000002e-05 2024-09-18T11:32:45.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:45.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039285+0000", 2024-09-18T11:32:45.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012839200000000001 2024-09-18T11:32:45.974 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:45.974 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:45.974 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:45.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.974 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:978 4.b e132/67 4:d8d14c58:::smithi11939506-49:30 v 132'417, mlcod=128'414)", 2024-09-18T11:32:45.974 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:45.974 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 42.918871168000003, 2024-09-18T11:32:45.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011942183, 2024-09-18T11:32:45.974 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:45.974 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:45.974 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:45.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:45.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:45.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:45.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:45.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:45.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:45.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:45.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:45.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:45.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:45.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025278+0000", 2024-09-18T11:32:45.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9060000000000004e-06 2024-09-18T11:32:45.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:45.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025288+0000", 2024-09-18T11:32:45.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.028e-05 2024-09-18T11:32:45.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025293+0000", 2024-09-18T11:32:45.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2700000000000004e-06 2024-09-18T11:32:45.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:45.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025346+0000", 2024-09-18T11:32:45.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2862999999999999e-05 2024-09-18T11:32:45.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:45.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025370+0000", 2024-09-18T11:32:45.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4005000000000001e-05 2024-09-18T11:32:45.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:45.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037159+0000", 2024-09-18T11:32:45.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011788343 2024-09-18T11:32:45.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:45.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037216+0000", 2024-09-18T11:32:45.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7695000000000002e-05 2024-09-18T11:32:45.978 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:45.978 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:45.978 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:45.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.978 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3930 4.b e132/67 4:d3ae8215:::smithi11939506-41:head v 132'421, mlcod=128'414)", 2024-09-18T11:32:45.978 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:45.978 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 42.915322920000001, 2024-09-18T11:32:45.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015093554, 2024-09-18T11:32:45.979 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:45.979 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:45.979 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:45.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:45.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:45.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:45.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:45.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:45.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:45.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:45.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028821+0000", 2024-09-18T11:32:45.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:45.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:45.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029805+0000", 2024-09-18T11:32:45.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00098410899999999994 2024-09-18T11:32:45.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:45.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029817+0000", 2024-09-18T11:32:45.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2058e-05 2024-09-18T11:32:45.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029823+0000", 2024-09-18T11:32:45.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6450000000000002e-06 2024-09-18T11:32:45.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:45.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032330+0000", 2024-09-18T11:32:45.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025067710000000001 2024-09-18T11:32:45.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:45.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032349+0000", 2024-09-18T11:32:45.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9729e-05 2024-09-18T11:32:45.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:45.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043878+0000", 2024-09-18T11:32:45.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011528514 2024-09-18T11:32:45.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:45.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043916+0000", 2024-09-18T11:32:45.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8049000000000001e-05 2024-09-18T11:32:45.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:45.982 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:45.983 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:45.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.983 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:162 3.1 3:8c826a1f:::e682cfc1e82c88472b9acf1cd97978bc105aa035f10b685bb4cace95809b52bf:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:45.983 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:32:45.983 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.363200761999998, 2024-09-18T11:32:45.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013234571000000001, 2024-09-18T11:32:45.983 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:45.983 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:45.983 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:45.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:45.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:32:45.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:45.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:45.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:32:45.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:45.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:45.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580943+0000", 2024-09-18T11:32:45.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:45.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:45.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582556+0000", 2024-09-18T11:32:45.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016132970000000001 2024-09-18T11:32:45.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:45.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582572+0000", 2024-09-18T11:32:45.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5846999999999999e-05 2024-09-18T11:32:45.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582578+0000", 2024-09-18T11:32:45.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4810000000000003e-06 2024-09-18T11:32:45.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:45.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582632+0000", 2024-09-18T11:32:45.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4156000000000001e-05 2024-09-18T11:32:45.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:45.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582819+0000", 2024-09-18T11:32:45.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000186431 2024-09-18T11:32:45.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:45.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583118+0000", 2024-09-18T11:32:45.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029880100000000001 2024-09-18T11:32:45.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:45.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590676+0000", 2024-09-18T11:32:45.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075588670000000004 2024-09-18T11:32:45.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:45.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594111+0000", 2024-09-18T11:32:45.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034348550000000001 2024-09-18T11:32:45.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:45.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594142+0000", 2024-09-18T11:32:45.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.099e-05 2024-09-18T11:32:45.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:45.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594179+0000", 2024-09-18T11:32:45.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6831e-05 2024-09-18T11:32:45.988 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:45.988 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:45.988 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:45.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.989 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3974 4.d 4:b5a463bf:::smithi11939506-20:head [write 665643~697786,stat] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:45.989 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:32:45.989 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.362114079000001, 2024-09-18T11:32:45.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016621174999999998, 2024-09-18T11:32:45.989 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:45.989 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:45.989 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:45.989 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:45.989 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:45.989 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3974 2024-09-18T11:32:45.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.989 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:45.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:45.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:32:45.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:45.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:45.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:32:45.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:45.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:45.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582030+0000", 2024-09-18T11:32:45.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:45.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:45.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585185+0000", 2024-09-18T11:32:45.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003155644 2024-09-18T11:32:45.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:45.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585208+0000", 2024-09-18T11:32:45.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3013000000000001e-05 2024-09-18T11:32:45.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585215+0000", 2024-09-18T11:32:45.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7569999999999998e-06 2024-09-18T11:32:45.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:45.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585278+0000", 2024-09-18T11:32:45.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2699000000000004e-05 2024-09-18T11:32:45.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:45.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585374+0000", 2024-09-18T11:32:45.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.6062000000000003e-05 2024-09-18T11:32:45.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:32:45.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585577+0000", 2024-09-18T11:32:45.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000202926 2024-09-18T11:32:45.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:45.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596544+0000", 2024-09-18T11:32:45.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010966848 2024-09-18T11:32:45.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:45.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598594+0000", 2024-09-18T11:32:45.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020502049999999998 2024-09-18T11:32:45.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:45.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598625+0000", 2024-09-18T11:32:45.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1412999999999999e-05 2024-09-18T11:32:45.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:45.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598652+0000", 2024-09-18T11:32:45.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7166999999999999e-05 2024-09-18T11:32:45.994 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:45.994 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:45.994 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:45.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.994 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:166 3.1 3:a73e635b:::b6ee478f0f661ec7bac3d17c2ca2fdaff71a47570f64844a7ba79bbb2a0e8d9d:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:45.995 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:32:45.995 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.361554712999997, 2024-09-18T11:32:45.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011711037000000001, 2024-09-18T11:32:45.995 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:45.995 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:45.995 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:45.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:45.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:32:45.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:45.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:45.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:32:45.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:45.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:45.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582590+0000", 2024-09-18T11:32:45.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:45.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:45.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582673+0000", 2024-09-18T11:32:45.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2861000000000005e-05 2024-09-18T11:32:45.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:45.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582683+0000", 2024-09-18T11:32:45.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0081999999999999e-05 2024-09-18T11:32:45.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:45.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582688+0000", 2024-09-18T11:32:45.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6870000000000002e-06 2024-09-18T11:32:45.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:45.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584266+0000", 2024-09-18T11:32:45.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001578423 2024-09-18T11:32:45.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:45.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584407+0000", 2024-09-18T11:32:45.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014144599999999999 2024-09-18T11:32:45.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:45.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584599+0000", 2024-09-18T11:32:45.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000191881 2024-09-18T11:32:45.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:45.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590680+0000", 2024-09-18T11:32:45.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060801479999999996 2024-09-18T11:32:45.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:45.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594224+0000", 2024-09-18T11:32:45.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035441700000000001 2024-09-18T11:32:45.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:45.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594269+0000", 2024-09-18T11:32:45.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5297999999999999e-05 2024-09-18T11:32:45.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:45.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:45.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:46.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594302+0000", 2024-09-18T11:32:46.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2703000000000002e-05 2024-09-18T11:32:46.000 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.000 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:46.000 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.000 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:168 3.1 3:b64d7bad:::7feb22256bdcec217720fd46184ab43ced8923ed66ec6319c57f078d9f123262:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:46.000 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:32:46.000 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.361327261, 2024-09-18T11:32:46.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011589809, 2024-09-18T11:32:46.000 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:46.000 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:46.001 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:46.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:46.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:32:46.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:46.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:46.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:32:46.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:46.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:46.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582817+0000", 2024-09-18T11:32:46.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:46.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:46.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582903+0000", 2024-09-18T11:32:46.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5915999999999995e-05 2024-09-18T11:32:46.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:46.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582913+0000", 2024-09-18T11:32:46.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1409999999999994e-06 2024-09-18T11:32:46.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:46.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582917+0000", 2024-09-18T11:32:46.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6419999999999998e-06 2024-09-18T11:32:46.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:46.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585540+0000", 2024-09-18T11:32:46.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026228940000000002 2024-09-18T11:32:46.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:46.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585721+0000", 2024-09-18T11:32:46.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018086299999999999 2024-09-18T11:32:46.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:46.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585918+0000", 2024-09-18T11:32:46.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000196758 2024-09-18T11:32:46.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:46.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590683+0000", 2024-09-18T11:32:46.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047657020000000001 2024-09-18T11:32:46.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:46.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594354+0000", 2024-09-18T11:32:46.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003671068 2024-09-18T11:32:46.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:46.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594386+0000", 2024-09-18T11:32:46.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1426e-05 2024-09-18T11:32:46.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:46.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594408+0000", 2024-09-18T11:32:46.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1987000000000001e-05 2024-09-18T11:32:46.005 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.005 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:46.005 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.006 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:175 3.1 3:b5e86b88:::e55cf3d5cf782ed327a67e746b287b76d05f74f58a4f3726e399a564440ceaf6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:46.006 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:32:46.006 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.360478319999999, 2024-09-18T11:32:46.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013709627, 2024-09-18T11:32:46.006 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:46.006 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:46.006 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:46.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:46.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:32:46.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:46.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:46.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:32:46.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:46.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:46.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583666+0000", 2024-09-18T11:32:46.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:46.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:46.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584041+0000", 2024-09-18T11:32:46.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037511499999999998 2024-09-18T11:32:46.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:46.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584055+0000", 2024-09-18T11:32:46.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3295999999999999e-05 2024-09-18T11:32:46.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:46.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584061+0000", 2024-09-18T11:32:46.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1630000000000001e-06 2024-09-18T11:32:46.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:46.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.589980+0000", 2024-09-18T11:32:46.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005918998 2024-09-18T11:32:46.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:46.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590117+0000", 2024-09-18T11:32:46.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013715400000000001 2024-09-18T11:32:46.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:46.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590270+0000", 2024-09-18T11:32:46.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015245600000000001 2024-09-18T11:32:46.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:46.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596513+0000", 2024-09-18T11:32:46.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062438729999999996 2024-09-18T11:32:46.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:46.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597328+0000", 2024-09-18T11:32:46.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081489599999999996 2024-09-18T11:32:46.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:46.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597358+0000", 2024-09-18T11:32:46.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9291000000000001e-05 2024-09-18T11:32:46.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:46.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597377+0000", 2024-09-18T11:32:46.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:32:46.011 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.011 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:46.011 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.011 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4061 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:46.011 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:32:46.011 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 33.647299631000003, 2024-09-18T11:32:46.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013340967, 2024-09-18T11:32:46.011 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:46.012 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:46.012 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:46.012 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:46.012 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:46.012 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4061 2024-09-18T11:32:46.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.012 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:46.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:46.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:32:46.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:46.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:46.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:32:46.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:46.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:46.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296844+0000", 2024-09-18T11:32:46.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:46.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:46.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296854+0000", 2024-09-18T11:32:46.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0037000000000001e-05 2024-09-18T11:32:46.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:46.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296873+0000", 2024-09-18T11:32:46.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:32:46.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:46.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296881+0000", 2024-09-18T11:32:46.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3790000000000006e-06 2024-09-18T11:32:46.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:46.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296975+0000", 2024-09-18T11:32:46.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3536999999999996e-05 2024-09-18T11:32:46.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:46.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296982+0000", 2024-09-18T11:32:46.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6399999999999997e-06 2024-09-18T11:32:46.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:46.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301933+0000", 2024-09-18T11:32:46.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049507190000000001 2024-09-18T11:32:46.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:46.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301984+0000", 2024-09-18T11:32:46.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1439e-05 2024-09-18T11:32:46.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:46.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:32:46.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082022050000000006 2024-09-18T11:32:46.016 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.016 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:46.016 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.016 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4062 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:46.016 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:46.016 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 33.647251076000003, 2024-09-18T11:32:46.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016297147000000001, 2024-09-18T11:32:46.016 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:46.017 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:46.017 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:46.017 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:46.017 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:46.017 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4062 2024-09-18T11:32:46.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.017 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:46.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:46.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:46.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:46.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:46.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:46.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:46.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:46.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:46.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:46.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:46.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296898+0000", 2024-09-18T11:32:46.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9419999999999997e-06 2024-09-18T11:32:46.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:46.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296906+0000", 2024-09-18T11:32:46.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3520000000000007e-06 2024-09-18T11:32:46.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:46.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296910+0000", 2024-09-18T11:32:46.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4469999999999999e-06 2024-09-18T11:32:46.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:46.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297032+0000", 2024-09-18T11:32:46.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012168900000000001 2024-09-18T11:32:46.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:46.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297038+0000", 2024-09-18T11:32:46.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9320000000000001e-06 2024-09-18T11:32:46.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:46.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310221+0000", 2024-09-18T11:32:46.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013182887000000001 2024-09-18T11:32:46.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:46.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310323+0000", 2024-09-18T11:32:46.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010170000000000001 2024-09-18T11:32:46.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:46.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313191+0000", 2024-09-18T11:32:46.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002868808 2024-09-18T11:32:46.021 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.021 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:46.021 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.022 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4063 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:46.022 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:46.022 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 33.647230184000001, 2024-09-18T11:32:46.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017648712, 2024-09-18T11:32:46.022 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:46.022 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:46.022 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:46.022 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:46.022 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:46.022 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4063 2024-09-18T11:32:46.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.022 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:46.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.022 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:46.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:46.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:46.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:46.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:46.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:46.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:46.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:46.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:46.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:46.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296918+0000", 2024-09-18T11:32:46.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8100000000000002e-06 2024-09-18T11:32:46.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:46.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296924+0000", 2024-09-18T11:32:46.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2519999999999996e-06 2024-09-18T11:32:46.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:46.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296928+0000", 2024-09-18T11:32:46.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0500000000000002e-06 2024-09-18T11:32:46.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:46.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297062+0000", 2024-09-18T11:32:46.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013425500000000001 2024-09-18T11:32:46.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:46.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297069+0000", 2024-09-18T11:32:46.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5239999999999997e-06 2024-09-18T11:32:46.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:46.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313424+0000", 2024-09-18T11:32:46.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016355277000000001 2024-09-18T11:32:46.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:46.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313514+0000", 2024-09-18T11:32:46.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0581999999999994e-05 2024-09-18T11:32:46.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:46.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.314564+0000", 2024-09-18T11:32:46.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001049384 2024-09-18T11:32:46.026 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.027 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:46.027 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.027 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4065 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:46.027 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:32:46.027 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 33.647184742999997, 2024-09-18T11:32:46.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013225997, 2024-09-18T11:32:46.027 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:46.027 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:46.027 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:46.027 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:46.027 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:46.027 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4065 2024-09-18T11:32:46.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.028 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:46.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:46.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:32:46.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:46.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:46.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:32:46.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:46.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:46.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296960+0000", 2024-09-18T11:32:46.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:46.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:46.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296963+0000", 2024-09-18T11:32:46.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2270000000000002e-06 2024-09-18T11:32:46.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:46.029 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296970+0000", 2024-09-18T11:32:46.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6220000000000003e-06 2024-09-18T11:32:46.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:46.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296973+0000", 2024-09-18T11:32:46.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3730000000000001e-06 2024-09-18T11:32:46.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:46.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297097+0000", 2024-09-18T11:32:46.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012352399999999999 2024-09-18T11:32:46.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:46.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297103+0000", 2024-09-18T11:32:46.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7119999999999997e-06 2024-09-18T11:32:46.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:46.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307831+0000", 2024-09-18T11:32:46.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010728794 2024-09-18T11:32:46.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:46.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307924+0000", 2024-09-18T11:32:46.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2099e-05 2024-09-18T11:32:46.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.032 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:46.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:32:46.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022630469999999998 2024-09-18T11:32:46.032 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.032 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:46.032 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.032 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4066 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:46.032 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:32:46.032 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 33.647167668000002, 2024-09-18T11:32:46.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015656389999999999, 2024-09-18T11:32:46.032 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:46.033 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:46.033 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:46.033 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:46.033 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:46.033 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4066 2024-09-18T11:32:46.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.033 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:46.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:46.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:32:46.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:46.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:46.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:32:46.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:46.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:46.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296977+0000", 2024-09-18T11:32:46.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:46.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:46.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296980+0000", 2024-09-18T11:32:46.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8830000000000002e-06 2024-09-18T11:32:46.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:46.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296985+0000", 2024-09-18T11:32:46.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:32:46.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:46.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296988+0000", 2024-09-18T11:32:46.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0419999999999998e-06 2024-09-18T11:32:46.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.036 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:46.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297120+0000", 2024-09-18T11:32:46.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131693 2024-09-18T11:32:46.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.036 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:46.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297125+0000", 2024-09-18T11:32:46.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1100000000000002e-06 2024-09-18T11:32:46.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.036 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:46.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310212+0000", 2024-09-18T11:32:46.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013087395999999999 2024-09-18T11:32:46.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:46.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310292+0000", 2024-09-18T11:32:46.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9980000000000003e-05 2024-09-18T11:32:46.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:46.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312634+0000", 2024-09-18T11:32:46.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002341756 2024-09-18T11:32:46.037 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.037 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:46.037 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.038 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4068 3.1 3:87e79ed0:::3526641c07d3efa989be9684355570f39c331008b8a02c9fa22b38a94b6c6b9f:head [write 0~1015808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:46.038 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:32:46.038 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 33.643391450999999, 2024-09-18T11:32:46.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016392648999999999, 2024-09-18T11:32:46.038 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:46.038 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:46.038 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:46.038 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:46.038 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:46.038 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4068 2024-09-18T11:32:46.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.038 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:46.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:46.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:32:46.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:46.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:46.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:32:46.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:46.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:46.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300753+0000", 2024-09-18T11:32:46.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:46.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:46.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301922+0000", 2024-09-18T11:32:46.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001169432 2024-09-18T11:32:46.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:46.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301936+0000", 2024-09-18T11:32:46.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3574000000000001e-05 2024-09-18T11:32:46.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:46.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301941+0000", 2024-09-18T11:32:46.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6069999999999998e-06 2024-09-18T11:32:46.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:46.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310234+0000", 2024-09-18T11:32:46.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082924820000000003 2024-09-18T11:32:46.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:46.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310337+0000", 2024-09-18T11:32:46.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102673 2024-09-18T11:32:46.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:46.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310469+0000", 2024-09-18T11:32:46.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131982 2024-09-18T11:32:46.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:46.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312844+0000", 2024-09-18T11:32:46.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023758680000000002 2024-09-18T11:32:46.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:46.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317092+0000", 2024-09-18T11:32:46.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042477549999999998 2024-09-18T11:32:46.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:46.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317124+0000", 2024-09-18T11:32:46.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1968999999999998e-05 2024-09-18T11:32:46.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.043 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:46.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317147+0000", 2024-09-18T11:32:46.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2339000000000001e-05 2024-09-18T11:32:46.043 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.043 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:46.043 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.043 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4671 4.1 4:87078dfe:::smithi11939506-14:head [rollback 6f] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:32:46.043 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:32:46.043 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.62542907599999997, 2024-09-18T11:32:46.044 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013540905000000001, 2024-09-18T11:32:46.044 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:46.044 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:46.044 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:46.044 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:46.044 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:46.044 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4671 2024-09-18T11:32:46.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.044 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:46.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.044 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:46.044 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:32:46.044 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:46.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.045 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:46.045 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:32:46.045 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:46.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.045 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:46.045 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:32:46.045 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:46.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.045 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:46.045 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318718+0000", 2024-09-18T11:32:46.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3300000000000001e-06 2024-09-18T11:32:46.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.046 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:46.046 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318723+0000", 2024-09-18T11:32:46.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1590000000000001e-06 2024-09-18T11:32:46.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.046 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:46.046 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318727+0000", 2024-09-18T11:32:46.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3529999999999999e-06 2024-09-18T11:32:46.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.046 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:46.046 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.320735+0000", 2024-09-18T11:32:46.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020079730000000001 2024-09-18T11:32:46.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.047 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:46.047 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.320843+0000", 2024-09-18T11:32:46.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000108488 2024-09-18T11:32:46.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.047 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:46.047 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.329522+0000", 2024-09-18T11:32:46.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086787979999999997 2024-09-18T11:32:46.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.047 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:46.048 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331769+0000", 2024-09-18T11:32:46.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022471269999999998 2024-09-18T11:32:46.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.048 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:46.048 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332189+0000", 2024-09-18T11:32:46.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00041940400000000001 2024-09-18T11:32:46.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.048 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:46.048 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332228+0000", 2024-09-18T11:32:46.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9298000000000002e-05 2024-09-18T11:32:46.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.049 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:46.049 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332257+0000", 2024-09-18T11:32:46.049 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.934e-05 2024-09-18T11:32:46.049 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.049 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:46.049 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.049 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4672 4.1 4:87078dfe:::smithi11939506-14:head [append 0~0] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:32:46.049 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.318763+0000", 2024-09-18T11:32:46.049 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.62538267299999994, 2024-09-18T11:32:46.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016099992, 2024-09-18T11:32:46.050 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:46.050 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:46.050 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:46.050 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:46.050 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:46.050 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4672 2024-09-18T11:32:46.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.050 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:46.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.050 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:46.050 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318763+0000", 2024-09-18T11:32:46.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:46.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.051 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:46.051 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318763+0000", 2024-09-18T11:32:46.051 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:46.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.051 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:46.051 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318762+0000", 2024-09-18T11:32:46.051 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:46.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.051 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:46.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318765+0000", 2024-09-18T11:32:46.052 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.43e-06 2024-09-18T11:32:46.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.052 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:46.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318770+0000", 2024-09-18T11:32:46.052 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1479999999999997e-06 2024-09-18T11:32:46.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.052 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:46.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318773+0000", 2024-09-18T11:32:46.052 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.18e-06 2024-09-18T11:32:46.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.053 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:46.053 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.321043+0000", 2024-09-18T11:32:46.053 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022696819999999999 2024-09-18T11:32:46.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.053 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for blocked object", 2024-09-18T11:32:46.053 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.321104+0000", 2024-09-18T11:32:46.053 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1050999999999995e-05 2024-09-18T11:32:46.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.053 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:46.053 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331070+0000", 2024-09-18T11:32:46.053 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0099659610000000006 2024-09-18T11:32:46.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.054 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:46.054 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331176+0000", 2024-09-18T11:32:46.054 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010582 2024-09-18T11:32:46.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.054 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:46.054 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331377+0000", 2024-09-18T11:32:46.054 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000201208 2024-09-18T11:32:46.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.054 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:46.055 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332278+0000", 2024-09-18T11:32:46.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00090079999999999999 2024-09-18T11:32:46.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:46.055 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.334798+0000", 2024-09-18T11:32:46.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025200190000000001 2024-09-18T11:32:46.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:46.055 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.334833+0000", 2024-09-18T11:32:46.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5678e-05 2024-09-18T11:32:46.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:46.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:46.056 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:46.056 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.334863+0000", 2024-09-18T11:32:46.056 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9374e-05 2024-09-18T11:32:46.056 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.056 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:46.056 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.056 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:46.056 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:46.056 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:46.057 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:46.057 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:46.058 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:46.058 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:46.059 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:32:46.059 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:46.059 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:46.060 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:46.060 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:46.060 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:46.056+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:46.061 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:46.061 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:46.074 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:46.074 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:46.090 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:46.090 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:46.105 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:46.105 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:46.142 INFO:tasks.rados.rados.0.smithi119.stdout:1999: finishing write tid 3 to smithi11939506-5 2024-09-18T11:32:46.143 INFO:tasks.rados.rados.0.smithi119.stdout:1999: finishing write tid 4 to smithi11939506-5 2024-09-18T11:32:46.143 INFO:tasks.rados.rados.0.smithi119.stdout:1999: finishing write tid 5 to smithi11939506-5 2024-09-18T11:32:46.143 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 5 v 240 (ObjNum 411 snap 67 seq_num 411) dirty exists 2024-09-18T11:32:46.143 INFO:tasks.rados.rados.0.smithi119.stdout:1999: left oid 5 (ObjNum 411 snap 67 seq_num 411) 2024-09-18T11:32:46.143 INFO:tasks.rados.rados.0.smithi119.stdout:1998: expect (ObjNum 400 snap 65 seq_num 400) 2024-09-18T11:32:46.158 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:46.156+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:46.197 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:46.198 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:46.206 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:32:46.290 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:46.288+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:46.293 INFO:tasks.rados.rados.0.smithi119.stdout:1990: finishing rollback tid 1 to smithi11939506-18 2024-09-18T11:32:46.293 INFO:tasks.rados.rados.0.smithi119.stdout:1990: finishing rollback tid 2 to smithi11939506-18 2024-09-18T11:32:46.293 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 18 v 311 (ObjNum 205 snap 23 seq_num 205) dirty exists 2024-09-18T11:32:46.293 INFO:tasks.rados.rados.0.smithi119.stdout:2000: finishing rollback tid 0 to smithi11939506-31 2024-09-18T11:32:46.293 INFO:tasks.rados.rados.0.smithi119.stdout:2000: finishing rollback tid 1 to smithi11939506-31 2024-09-18T11:32:46.293 INFO:tasks.rados.rados.0.smithi119.stdout:2000: finishing rollback tid 2 to smithi11939506-31 2024-09-18T11:32:46.293 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 428 (ObjNum 345 snap 51 seq_num 345) dirty exists 2024-09-18T11:32:46.294 INFO:tasks.rados.rados.0.smithi119.stdout:1990: done (10 left) 2024-09-18T11:32:46.294 INFO:tasks.rados.rados.0.smithi119.stdout:1991: done (9 left) 2024-09-18T11:32:46.294 INFO:tasks.rados.rados.0.smithi119.stdout:1992: done (8 left) 2024-09-18T11:32:46.294 INFO:tasks.rados.rados.0.smithi119.stdout:1993: done (7 left) 2024-09-18T11:32:46.294 INFO:tasks.rados.rados.0.smithi119.stdout:1994: done (6 left) 2024-09-18T11:32:46.294 INFO:tasks.rados.rados.0.smithi119.stdout:1995: done (5 left) 2024-09-18T11:32:46.294 INFO:tasks.rados.rados.0.smithi119.stdout:1996: done (4 left) 2024-09-18T11:32:46.294 INFO:tasks.rados.rados.0.smithi119.stdout:1998: done (3 left) 2024-09-18T11:32:46.294 INFO:tasks.rados.rados.0.smithi119.stdout:1999: done (2 left) 2024-09-18T11:32:46.294 INFO:tasks.rados.rados.0.smithi119.stdout:2000: done (1 left) 2024-09-18T11:32:46.294 INFO:tasks.rados.rados.0.smithi119.stdout:2001: read oid 41 snap -1 2024-09-18T11:32:46.294 INFO:tasks.rados.rados.0.smithi119.stdout:2001: expect (ObjNum 347 snap 51 seq_num 347) 2024-09-18T11:32:46.294 INFO:tasks.rados.rados.0.smithi119.stdout:2002: set_chunk oid 13 target oid 2024-09-18T11:32:46.317 INFO:tasks.rados.rados.0.smithi119.stdout:2002: set_chunk oid 13 offset: 2104832 length: 825856 target oid 3076454a806d68c2d26a0c7d57c6cb9d791f9f292de24e77ba3b8e771422874f offset: 0 snap 58 2024-09-18T11:32:46.317 INFO:tasks.rados.rados.0.smithi119.stdout:1997: expect (ObjNum 355 snap 55 seq_num 355) 2024-09-18T11:32:46.349 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:46.350 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:46.358 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:32:46.399 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:46.397+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:46.448 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:46.446+0000 7f2d449e8640 -1 osd.1 174 handle_osd_map injecting map crc failure 2024-09-18T11:32:46.449 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:46.447+0000 7f6bf3052640 -1 osd.5 174 handle_osd_map injecting map crc failure 2024-09-18T11:32:46.460 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:46.459+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:46.503 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:46.504 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:46.507 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T11:32:46.512 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:32:46.562 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:46.559+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:46.619 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:46.619 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:46.628 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:32:46.669 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:46.667+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:46.673 INFO:tasks.rados.rados.0.smithi119.stdout:2001: expect (ObjNum 347 snap 51 seq_num 347) 2024-09-18T11:32:46.763 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:46.761+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:46.767 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:46.767 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:46.775 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:32:46.846 INFO:tasks.rados.rados.0.smithi119.stdout:2002: finishing set_chunk to oid 13 2024-09-18T11:32:46.846 INFO:tasks.rados.rados.0.smithi119.stdout:2003: read oid 10 snap -1 2024-09-18T11:32:46.846 INFO:tasks.rados.rados.0.smithi119.stdout:2003: expect (ObjNum 358 snap 55 seq_num 358) 2024-09-18T11:32:46.846 INFO:tasks.rados.rados.0.smithi119.stdout:1997: done (3 left) 2024-09-18T11:32:46.846 INFO:tasks.rados.rados.0.smithi119.stdout:2001: done (2 left) 2024-09-18T11:32:46.846 INFO:tasks.rados.rados.0.smithi119.stdout:2002: done (1 left) 2024-09-18T11:32:46.846 INFO:tasks.rados.rados.0.smithi119.stdout:2004: snap_create 2024-09-18T11:32:46.854 INFO:tasks.rados.rados.0.smithi119.stdout:2003: expect (ObjNum 358 snap 55 seq_num 358) 2024-09-18T11:32:46.864 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:46.863+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:46.915 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:46.915 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:46.924 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:32:46.968 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:46.966+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:47.022 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:47.022 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:47.031 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:32:47.072 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:47.070+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:47.167 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:47.166+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:47.168 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:47.168 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:47.176 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:32:47.268 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:47.267+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:47.273 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:47.274 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:47.282 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:32:47.369 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:47.367+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:47.381 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:47.381 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:47.389 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:32:47.457 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is set 2024-09-18T11:32:47.457 INFO:tasks.rados.rados.0.smithi119.stdout:2003: done (1 left) 2024-09-18T11:32:47.457 INFO:tasks.rados.rados.0.smithi119.stdout:2004: done (0 left) 2024-09-18T11:32:47.457 INFO:tasks.rados.rados.0.smithi119.stdout:2005: read oid 20 snap -1 2024-09-18T11:32:47.457 INFO:tasks.rados.rados.0.smithi119.stdout:2005: expect (ObjNum 349 snap 54 seq_num 349) 2024-09-18T11:32:47.458 INFO:tasks.rados.rados.0.smithi119.stdout:2006: tier_evict oid 45 2024-09-18T11:32:47.458 INFO:tasks.rados.rados.0.smithi119.stdout:2006: tier_evict oid 45 snap -1 2024-09-18T11:32:47.458 INFO:tasks.rados.rados.0.smithi119.stdout:2007: read oid 40 snap -1 2024-09-18T11:32:47.458 INFO:tasks.rados.rados.0.smithi119.stdout:2007: expect (ObjNum 287 snap 41 seq_num 287) 2024-09-18T11:32:47.458 INFO:tasks.rados.rados.0.smithi119.stdout:2008: write (excl) oid 9 current snap is 68 2024-09-18T11:32:47.458 INFO:tasks.rados.rados.0.smithi119.stdout:2008: seq_num 412 ranges {524484=485650,1425993=661460,2847920=76564} 2024-09-18T11:32:47.463 INFO:tasks.rados.rados.0.smithi119.stdout:2008: writing smithi11939506-9 from 524484 to 1010134 tid 1 2024-09-18T11:32:47.470 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:47.469+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:47.471 INFO:tasks.rados.rados.0.smithi119.stdout:2008: writing smithi11939506-9 from 1425993 to 2087453 tid 2 2024-09-18T11:32:47.476 INFO:tasks.rados.rados.0.smithi119.stdout:2008: writing smithi11939506-9 from 2847920 to 2924484 tid 3 2024-09-18T11:32:47.476 INFO:tasks.rados.rados.0.smithi119.stdout:2009: set_chunk oid 7 target oid 2024-09-18T11:32:47.496 INFO:tasks.rados.rados.0.smithi119.stdout:2009: set_chunk oid 7 offset: 15360 length: 2396672 target oid b21cbc7e46fd2e27c0b3d80d14d4b26dd3095959b9d8d6bd1250f651386fd7b8 offset: 0 snap -1 2024-09-18T11:32:47.497 INFO:tasks.rados.rados.0.smithi119.stdout:2010: delete oid 44 current snap is 68 2024-09-18T11:32:47.497 INFO:tasks.rados.rados.0.smithi119.stdout:2006: got (22) Invalid argument 2024-09-18T11:32:47.497 INFO:tasks.rados.rados.0.smithi119.stdout:2007: expect (ObjNum 287 snap 41 seq_num 287) 2024-09-18T11:32:47.509 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:47.509 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:47.522 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:32:47.607 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:47.606+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:47.671 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:47.670+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:47.678 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:47.678 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:47.685 INFO:tasks.rados.rados.0.smithi119.stdout:2005: expect (ObjNum 349 snap 54 seq_num 349) 2024-09-18T11:32:47.688 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:32:47.773 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:47.772+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:47.829 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:47.829 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:47.837 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:32:47.879 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:47.878+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:47.959 INFO:tasks.rados.rados.0.smithi119.stdout:2008: finishing write tid 1 to smithi11939506-9 2024-09-18T11:32:47.959 INFO:tasks.rados.rados.0.smithi119.stdout:2008: finishing write tid 2 to smithi11939506-9 2024-09-18T11:32:47.959 INFO:tasks.rados.rados.0.smithi119.stdout:2008: finishing write tid 3 to smithi11939506-9 2024-09-18T11:32:47.959 INFO:tasks.rados.rados.0.smithi119.stdout:2008: finishing write tid 4 to smithi11939506-9 2024-09-18T11:32:47.959 INFO:tasks.rados.rados.0.smithi119.stdout:2008: finishing write tid 5 to smithi11939506-9 2024-09-18T11:32:47.959 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 9 v 610 (ObjNum 412 snap 68 seq_num 412) dirty exists 2024-09-18T11:32:47.959 INFO:tasks.rados.rados.0.smithi119.stdout:2008: left oid 9 (ObjNum 412 snap 68 seq_num 412) 2024-09-18T11:32:47.959 INFO:tasks.rados.rados.0.smithi119.stdout:2009: finishing set_chunk to oid 7 2024-09-18T11:32:47.959 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 7 v 184 (ObjNum 384 snap 60 seq_num 384) dirty exists 2024-09-18T11:32:47.959 INFO:tasks.rados.rados.0.smithi119.stdout:2005: done (5 left) 2024-09-18T11:32:47.959 INFO:tasks.rados.rados.0.smithi119.stdout:2006: done (4 left) 2024-09-18T11:32:47.959 INFO:tasks.rados.rados.0.smithi119.stdout:2007: done (3 left) 2024-09-18T11:32:47.960 INFO:tasks.rados.rados.0.smithi119.stdout:2008: done (2 left) 2024-09-18T11:32:47.960 INFO:tasks.rados.rados.0.smithi119.stdout:2009: done (1 left) 2024-09-18T11:32:47.960 INFO:tasks.rados.rados.0.smithi119.stdout:2010: done (0 left) 2024-09-18T11:32:47.960 INFO:tasks.rados.rados.0.smithi119.stdout:2011: read oid 48 snap 57 2024-09-18T11:32:47.960 INFO:tasks.rados.rados.0.smithi119.stdout:2011: expect deleted 2024-09-18T11:32:47.960 INFO:tasks.rados.rados.0.smithi119.stdout:2012: read oid 7 snap -1 2024-09-18T11:32:47.960 INFO:tasks.rados.rados.0.smithi119.stdout:2012: expect (ObjNum 384 snap 60 seq_num 384) 2024-09-18T11:32:47.960 INFO:tasks.rados.rados.0.smithi119.stdout:2013: snap_create 2024-09-18T11:32:47.964 INFO:tasks.rados.rados.0.smithi119.stdout:2012: expect (ObjNum 384 snap 60 seq_num 384) 2024-09-18T11:32:47.973 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:47.972+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:47.980 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:47.980 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:47.989 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:32:48.074 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:48.073+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:48.116 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:32:48.127 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:48.128 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:48.136 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:32:48.154 INFO:tasks.rados.rados.0.smithi119.stdout:2011: done (2 left) 2024-09-18T11:32:48.154 INFO:tasks.rados.rados.0.smithi119.stdout:2012: done (1 left) 2024-09-18T11:32:48.154 INFO:tasks.rados.rados.0.smithi119.stdout:2013: done (0 left) 2024-09-18T11:32:48.154 INFO:tasks.rados.rados.0.smithi119.stdout:2014: write oid 48 current snap is 69 2024-09-18T11:32:48.154 INFO:tasks.rados.rados.0.smithi119.stdout:2014: seq_num 414 ranges {558098=636148,1826604=665129} 2024-09-18T11:32:48.160 INFO:tasks.rados.rados.0.smithi119.stdout:2014: writing smithi11939506-48 from 558098 to 1194246 tid 1 2024-09-18T11:32:48.167 INFO:tasks.rados.rados.0.smithi119.stdout:2014: writing smithi11939506-48 from 1826604 to 2491733 tid 2 2024-09-18T11:32:48.167 INFO:tasks.rados.rados.0.smithi119.stdout:2015: write (excl) oid 35 current snap is 69 2024-09-18T11:32:48.168 INFO:tasks.rados.rados.0.smithi119.stdout:2015: seq_num 415 ranges {574905=711397,1885086=525140,2930189=44716} 2024-09-18T11:32:48.174 INFO:tasks.rados.rados.0.smithi119.stdout:2015: writing smithi11939506-35 from 574905 to 1286302 tid 1 2024-09-18T11:32:48.175 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:48.173+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:48.180 INFO:tasks.rados.rados.0.smithi119.stdout:2015: writing smithi11939506-35 from 1885086 to 2410226 tid 2 2024-09-18T11:32:48.184 INFO:tasks.rados.rados.0.smithi119.stdout:2015: writing smithi11939506-35 from 2930189 to 2974905 tid 3 2024-09-18T11:32:48.184 INFO:tasks.rados.rados.0.smithi119.stdout:2016: read oid 31 snap -1 2024-09-18T11:32:48.184 INFO:tasks.rados.rados.0.smithi119.stdout:2016: expect (ObjNum 345 snap 51 seq_num 345) 2024-09-18T11:32:48.184 INFO:tasks.rados.rados.0.smithi119.stdout:2017: set_chunk oid 26 target oid 2024-09-18T11:32:48.184 INFO:tasks.rados.rados.0.smithi119.stdout:2017: set_chunk oid 26 offset: 0 length: 4000000 target oid smithi11939506- offset: 0 snap -1 2024-09-18T11:32:48.184 INFO:tasks.rados.rados.0.smithi119.stdout:2018: snap_remove snap 66 2024-09-18T11:32:48.249 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:48.249 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:48.267 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:48.267 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:48.268 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:48.268 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:48.276 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:48.275+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:48.277 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:32:48.282 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:48.282 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:48.297 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:48.297 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:48.313 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:48.314 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:48.330 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:48.330 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:48.346 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:48.346 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:48.361 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:48.361 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:48.377 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:48.376+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:48.382 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:48.382 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:48.382 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:48.382 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:48.382 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:48.391 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:32:48.479 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:48.477+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:48.533 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:48.533 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:48.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:48.533 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:48.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.534 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:32:48.534 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:48.534 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 148.547771297, 2024-09-18T11:32:48.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:32:48.534 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:48.534 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:48.534 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:48.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:48.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:48.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:48.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:48.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:48.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:32:48.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:48.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:48.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:32:48.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:32:48.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:48.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:32:48.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:32:48.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:48.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:32:48.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:32:48.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:48.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:32:48.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:32:48.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:48.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:32:48.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:32:48.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:48.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:32:48.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:32:48.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:48.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:32:48.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:32:48.538 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.538 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:48.538 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.538 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:32:48.538 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:48.538 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 139.312940344, 2024-09-18T11:32:48.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:32:48.539 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:48.539 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:48.539 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:48.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:48.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:48.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:48.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:48.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:48.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:48.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:48.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:48.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:32:48.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:32:48.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:48.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:32:48.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:32:48.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:48.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:32:48.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:32:48.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:48.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:32:48.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:32:48.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:48.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:32:48.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:32:48.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:48.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:32:48.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:32:48.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:48.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:32:48.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:32:48.542 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.543 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:48.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.543 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:32:48.543 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:48.543 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 134.13903405900001, 2024-09-18T11:32:48.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:32:48.543 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:48.543 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:48.543 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:48.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:48.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:48.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:48.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:48.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:48.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:32:48.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:48.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:48.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:32:48.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:32:48.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:48.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:32:48.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:32:48.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:48.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:32:48.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:32:48.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:48.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:32:48.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:32:48.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:48.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:32:48.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:32:48.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:48.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:32:48.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:32:48.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:48.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:32:48.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:32:48.547 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.547 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:48.547 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.548 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:304 4.6 e69/67 4:67c1765a:::smithi11939506-30:17 v 69'35, mlcod=58'31)", 2024-09-18T11:32:48.548 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:48.548 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.185192965, 2024-09-18T11:32:48.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006377611, 2024-09-18T11:32:48.548 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:48.548 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:48.548 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:48.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:48.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:48.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:48.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:48.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:48.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:48.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:48.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:48.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345961+0000", 2024-09-18T11:32:48.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2050000000000008e-06 2024-09-18T11:32:48.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:48.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345969+0000", 2024-09-18T11:32:48.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0339999999999997e-06 2024-09-18T11:32:48.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:48.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345976+0000", 2024-09-18T11:32:48.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-18T11:32:48.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:48.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346026+0000", 2024-09-18T11:32:48.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9286999999999998e-05 2024-09-18T11:32:48.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:48.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346050+0000", 2024-09-18T11:32:48.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4397e-05 2024-09-18T11:32:48.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:48.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352293+0000", 2024-09-18T11:32:48.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062427170000000001 2024-09-18T11:32:48.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:48.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352330+0000", 2024-09-18T11:32:48.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7348999999999998e-05 2024-09-18T11:32:48.552 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.552 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:48.552 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.552 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:347 4.d e69/22 4:b44229a1:::smithi11939506-42:19 v 69'99, mlcod=65'93)", 2024-09-18T11:32:48.552 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:48.552 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.17945976999999, 2024-09-18T11:32:48.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056998760000000004, 2024-09-18T11:32:48.552 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:48.552 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:48.552 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:48.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:48.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:48.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:48.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:48.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:48.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351684+0000", 2024-09-18T11:32:48.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:48.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:48.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351693+0000", 2024-09-18T11:32:48.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-18T11:32:48.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:48.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351704+0000", 2024-09-18T11:32:48.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1077000000000001e-05 2024-09-18T11:32:48.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:48.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351709+0000", 2024-09-18T11:32:48.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5419999999999999e-06 2024-09-18T11:32:48.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:48.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351956+0000", 2024-09-18T11:32:48.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000247252 2024-09-18T11:32:48.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:48.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351975+0000", 2024-09-18T11:32:48.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9201000000000001e-05 2024-09-18T11:32:48.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:48.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357334+0000", 2024-09-18T11:32:48.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053588749999999999 2024-09-18T11:32:48.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:48.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357385+0000", 2024-09-18T11:32:48.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1406999999999998e-05 2024-09-18T11:32:48.556 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.556 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:48.556 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.557 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1956 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'37, mlcod=58'31)", 2024-09-18T11:32:48.557 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:48.557 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.178514857, 2024-09-18T11:32:48.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010958842, 2024-09-18T11:32:48.557 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:48.557 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:48.557 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:48.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:48.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:48.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:48.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:48.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:48.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:48.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:48.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:48.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352996+0000", 2024-09-18T11:32:48.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036622 2024-09-18T11:32:48.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:48.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353011+0000", 2024-09-18T11:32:48.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4849e-05 2024-09-18T11:32:48.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:48.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353017+0000", 2024-09-18T11:32:48.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:32:48.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:48.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353068+0000", 2024-09-18T11:32:48.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0695000000000002e-05 2024-09-18T11:32:48.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:48.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353092+0000", 2024-09-18T11:32:48.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4260000000000002e-05 2024-09-18T11:32:48.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:48.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363551+0000", 2024-09-18T11:32:48.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010459154999999999 2024-09-18T11:32:48.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:48.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363589+0000", 2024-09-18T11:32:48.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8161000000000003e-05 2024-09-18T11:32:48.561 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.561 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:48.561 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.561 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1957 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'38, mlcod=65'33)", 2024-09-18T11:32:48.561 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:48.561 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.17209898999999, 2024-09-18T11:32:48.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050499400000000002, 2024-09-18T11:32:48.561 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:48.561 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:48.561 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:48.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:48.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:48.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:48.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:48.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:48.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:48.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:48.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:48.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359214+0000", 2024-09-18T11:32:48.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000168047 2024-09-18T11:32:48.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:48.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359227+0000", 2024-09-18T11:32:48.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3124e-05 2024-09-18T11:32:48.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:48.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359235+0000", 2024-09-18T11:32:48.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0760000000000003e-06 2024-09-18T11:32:48.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:48.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359287+0000", 2024-09-18T11:32:48.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.151e-05 2024-09-18T11:32:48.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:48.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359310+0000", 2024-09-18T11:32:48.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-09-18T11:32:48.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:48.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364065+0000", 2024-09-18T11:32:48.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047549439999999997 2024-09-18T11:32:48.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:48.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364096+0000", 2024-09-18T11:32:48.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1423000000000001e-05 2024-09-18T11:32:48.565 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.565 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:48.565 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.565 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2300 4.4 e77/22 4:2c93196f:::smithi11939506-28:head v 77'94, mlcod=77'90)", 2024-09-18T11:32:48.565 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:48.565 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 107.54303677199999, 2024-09-18T11:32:48.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050425380000000001, 2024-09-18T11:32:48.565 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:48.565 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:48.566 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:48.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:48.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:48.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:48.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:48.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:48.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988108+0000", 2024-09-18T11:32:48.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:48.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:48.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988388+0000", 2024-09-18T11:32:48.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028087700000000001 2024-09-18T11:32:48.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:48.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988405+0000", 2024-09-18T11:32:48.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6565000000000001e-05 2024-09-18T11:32:48.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:48.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988412+0000", 2024-09-18T11:32:48.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9709999999999998e-06 2024-09-18T11:32:48.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:48.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988464+0000", 2024-09-18T11:32:48.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1949e-05 2024-09-18T11:32:48.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:48.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988490+0000", 2024-09-18T11:32:48.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5780999999999999e-05 2024-09-18T11:32:48.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:48.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993112+0000", 2024-09-18T11:32:48.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046228399999999996 2024-09-18T11:32:48.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:48.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993151+0000", 2024-09-18T11:32:48.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8587999999999997e-05 2024-09-18T11:32:48.569 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.569 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:48.569 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.570 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.13:63 3.0 e78/21 3:24ebb740:::smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 78'80, mlcod=73'78)", 2024-09-18T11:32:48.570 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:48.570 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 107.126047356, 2024-09-18T11:32:48.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052262230000000003, 2024-09-18T11:32:48.570 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:48.570 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:48.570 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:48.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.570 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:48.570 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:48.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:48.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:48.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:48.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405097+0000", 2024-09-18T11:32:48.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:48.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:48.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405103+0000", 2024-09-18T11:32:48.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8780000000000003e-06 2024-09-18T11:32:48.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:48.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405117+0000", 2024-09-18T11:32:48.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3441000000000001e-05 2024-09-18T11:32:48.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:48.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405121+0000", 2024-09-18T11:32:48.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7099999999999998e-06 2024-09-18T11:32:48.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:48.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405158+0000", 2024-09-18T11:32:48.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6477999999999999e-05 2024-09-18T11:32:48.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:48.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405173+0000", 2024-09-18T11:32:48.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4796e-05 2024-09-18T11:32:48.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:48.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410285+0000", 2024-09-18T11:32:48.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051118919999999998 2024-09-18T11:32:48.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:48.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410324+0000", 2024-09-18T11:32:48.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9616999999999997e-05 2024-09-18T11:32:48.574 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.574 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:48.574 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.574 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.13:18 3.0 e91/21 3:35e3c617:::2e64dd273da7619bd64b294f52905c1b9405a1ced2a8fd3c0b5bc481bc9732a6:head v 91'89, mlcod=85'87)", 2024-09-18T11:32:48.574 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:48.574 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 94.509271944999995, 2024-09-18T11:32:48.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048558539999999997, 2024-09-18T11:32:48.574 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:48.574 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:48.574 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:48.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:48.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:48.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:48.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:48.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:48.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:48.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:48.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:48.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021880+0000", 2024-09-18T11:32:48.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5860000000000001e-06 2024-09-18T11:32:48.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:48.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021890+0000", 2024-09-18T11:32:48.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8290000000000003e-06 2024-09-18T11:32:48.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.576 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:48.576 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021895+0000", 2024-09-18T11:32:48.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2480000000000004e-06 2024-09-18T11:32:48.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:48.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022166+0000", 2024-09-18T11:32:48.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027073499999999998 2024-09-18T11:32:48.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:48.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022184+0000", 2024-09-18T11:32:48.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8082000000000001e-05 2024-09-18T11:32:48.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:48.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026693+0000", 2024-09-18T11:32:48.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045093479999999998 2024-09-18T11:32:48.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:48.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026729+0000", 2024-09-18T11:32:48.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6282999999999998e-05 2024-09-18T11:32:48.578 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.578 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:48.578 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.578 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:573 4.4 e91/22 4:2c93196f:::smithi11939506-28:26 v 91'135, mlcod=90'131)", 2024-09-18T11:32:48.578 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:48.578 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 94.506805356000001, 2024-09-18T11:32:48.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050722739999999999, 2024-09-18T11:32:48.578 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:48.578 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:48.579 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:48.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:48.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:48.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:48.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:48.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:48.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024339+0000", 2024-09-18T11:32:48.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:48.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:48.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024349+0000", 2024-09-18T11:32:48.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6339999999999996e-06 2024-09-18T11:32:48.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:48.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024362+0000", 2024-09-18T11:32:48.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-18T11:32:48.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:48.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024368+0000", 2024-09-18T11:32:48.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8649999999999998e-06 2024-09-18T11:32:48.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:48.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024431+0000", 2024-09-18T11:32:48.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2780999999999994e-05 2024-09-18T11:32:48.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:48.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024448+0000", 2024-09-18T11:32:48.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7388999999999999e-05 2024-09-18T11:32:48.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:48.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029385+0000", 2024-09-18T11:32:48.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004936523 2024-09-18T11:32:48.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.582 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:48.582 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029412+0000", 2024-09-18T11:32:48.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7702999999999999e-05 2024-09-18T11:32:48.582 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.582 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:48.582 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.583 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:574 4.4 e91/22 4:2cb3e6a5:::smithi11939506-3:1e v 91'137, mlcod=90'131)", 2024-09-18T11:32:48.583 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:48.583 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 94.506367312999998, 2024-09-18T11:32:48.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070802850000000004, 2024-09-18T11:32:48.583 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:48.583 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:48.583 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:48.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:48.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:48.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:48.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:48.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:48.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024777+0000", 2024-09-18T11:32:48.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:48.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:48.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024785+0000", 2024-09-18T11:32:48.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8099999999999998e-06 2024-09-18T11:32:48.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:48.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024794+0000", 2024-09-18T11:32:48.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5839999999999995e-06 2024-09-18T11:32:48.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:48.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024798+0000", 2024-09-18T11:32:48.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.031e-06 2024-09-18T11:32:48.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:48.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024830+0000", 2024-09-18T11:32:48.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2268999999999999e-05 2024-09-18T11:32:48.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:48.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024847+0000", 2024-09-18T11:32:48.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7306999999999999e-05 2024-09-18T11:32:48.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:48.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031834+0000", 2024-09-18T11:32:48.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069868380000000004 2024-09-18T11:32:48.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:48.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031858+0000", 2024-09-18T11:32:48.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4082000000000001e-05 2024-09-18T11:32:48.587 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.587 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:48.587 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.587 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:541 4.d e91/22 4:b9bb642b:::smithi11939506-27:19 v 91'158, mlcod=91'154)", 2024-09-18T11:32:48.587 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:48.587 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 94.498986621, 2024-09-18T11:32:48.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052185, 2024-09-18T11:32:48.587 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:48.587 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:48.588 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:48.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:48.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:48.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:48.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:48.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:48.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032158+0000", 2024-09-18T11:32:48.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:48.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:48.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032165+0000", 2024-09-18T11:32:48.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-09-18T11:32:48.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:48.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032176+0000", 2024-09-18T11:32:48.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1460999999999999e-05 2024-09-18T11:32:48.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:48.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032182+0000", 2024-09-18T11:32:48.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0870000000000003e-06 2024-09-18T11:32:48.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:48.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032224+0000", 2024-09-18T11:32:48.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1372999999999998e-05 2024-09-18T11:32:48.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:48.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032246+0000", 2024-09-18T11:32:48.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2158e-05 2024-09-18T11:32:48.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:48.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037348+0000", 2024-09-18T11:32:48.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051025319999999999 2024-09-18T11:32:48.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.591 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:48.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037377+0000", 2024-09-18T11:32:48.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8937000000000001e-05 2024-09-18T11:32:48.591 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.591 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:48.591 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.592 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4178.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-18T11:32:48.592 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:48.592 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 66.590324555999999, 2024-09-18T11:32:48.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048960139999999997, 2024-09-18T11:32:48.592 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:48.592 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:48.592 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:48.592 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4178", 2024-09-18T11:32:48.592 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/490761693", 2024-09-18T11:32:48.592 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4 2024-09-18T11:32:48.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.592 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:48.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:48.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:48.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:48.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:48.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:48.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940819+0000", 2024-09-18T11:32:48.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:48.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:48.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940832+0000", 2024-09-18T11:32:48.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3155e-05 2024-09-18T11:32:48.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:48.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940848+0000", 2024-09-18T11:32:48.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6288999999999999e-05 2024-09-18T11:32:48.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:48.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940856+0000", 2024-09-18T11:32:48.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4919999999999996e-06 2024-09-18T11:32:48.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:48.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945613+0000", 2024-09-18T11:32:48.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047574999999999996 2024-09-18T11:32:48.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:48.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945717+0000", 2024-09-18T11:32:48.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103553 2024-09-18T11:32:48.595 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.595 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:48.596 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.596 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3482 4.6 e117/67 4:67c1765a:::smithi11939506-30:head v 117'120, mlcod=115'116)", 2024-09-18T11:32:48.596 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:48.596 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 61.846470332999999, 2024-09-18T11:32:48.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050037479999999997, 2024-09-18T11:32:48.596 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:48.596 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:48.596 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:48.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:48.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:48.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:48.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:48.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:48.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684674+0000", 2024-09-18T11:32:48.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:48.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:48.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684950+0000", 2024-09-18T11:32:48.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027621200000000002 2024-09-18T11:32:48.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:48.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684968+0000", 2024-09-18T11:32:48.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8e-05 2024-09-18T11:32:48.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:48.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684984+0000", 2024-09-18T11:32:48.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5775000000000001e-05 2024-09-18T11:32:48.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:48.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685033+0000", 2024-09-18T11:32:48.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8878000000000002e-05 2024-09-18T11:32:48.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:48.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685059+0000", 2024-09-18T11:32:48.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6801999999999999e-05 2024-09-18T11:32:48.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:48.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689640+0000", 2024-09-18T11:32:48.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045809029999999999 2024-09-18T11:32:48.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:48.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689679+0000", 2024-09-18T11:32:48.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8507000000000002e-05 2024-09-18T11:32:48.599 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.600 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:48.600 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.600 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:976 4.4 e132/22 4:2cb3e6a5:::smithi11939506-3:30 v 132'195, mlcod=128'192)", 2024-09-18T11:32:48.600 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:48.600 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 45.512499261999999, 2024-09-18T11:32:48.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0058526510000000004, 2024-09-18T11:32:48.600 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:48.600 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:48.600 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:48.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:48.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:48.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:48.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:48.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:48.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:48.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:48.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:48.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018661+0000", 2024-09-18T11:32:48.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5565000000000001e-05 2024-09-18T11:32:48.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:48.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018672+0000", 2024-09-18T11:32:48.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0210000000000001e-05 2024-09-18T11:32:48.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:48.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018678+0000", 2024-09-18T11:32:48.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1539999999999999e-06 2024-09-18T11:32:48.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:48.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018716+0000", 2024-09-18T11:32:48.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8389000000000001e-05 2024-09-18T11:32:48.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:48.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018737+0000", 2024-09-18T11:32:48.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0611e-05 2024-09-18T11:32:48.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:48.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.024458+0000", 2024-09-18T11:32:48.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057208629999999996 2024-09-18T11:32:48.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.603 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:48.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.024499+0000", 2024-09-18T11:32:48.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1100999999999998e-05 2024-09-18T11:32:48.604 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.604 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:48.604 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.604 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:823 4.d e137/22 4:b44229a1:::smithi11939506-42:49 v 137'266, mlcod=132'260)", 2024-09-18T11:32:48.604 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:32:48.604 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 40.253850706000001, 2024-09-18T11:32:48.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054497770000000003, 2024-09-18T11:32:48.604 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:48.604 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:48.605 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:48.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:48.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:32:48.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:48.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:32:48.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:48.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277294+0000", 2024-09-18T11:32:48.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:48.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:48.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277299+0000", 2024-09-18T11:32:48.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6280000000000004e-06 2024-09-18T11:32:48.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:48.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277306+0000", 2024-09-18T11:32:48.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.533e-06 2024-09-18T11:32:48.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:48.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277316+0000", 2024-09-18T11:32:48.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0148e-05 2024-09-18T11:32:48.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:48.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277868+0000", 2024-09-18T11:32:48.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00055199999999999997 2024-09-18T11:32:48.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.607 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:48.607 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277898+0000", 2024-09-18T11:32:48.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0264999999999999e-05 2024-09-18T11:32:48.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:48.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.282715+0000", 2024-09-18T11:32:48.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048165619999999999 2024-09-18T11:32:48.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:48.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.282744+0000", 2024-09-18T11:32:48.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9855000000000001e-05 2024-09-18T11:32:48.608 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.608 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:48.608 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.609 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:163 3.0 e138/21 3:02e8b30e:::4456d047c09ae2a9a25ccebfd83e30cf78df6779d63bfba9365afd7301eaeb6a:head v 138'132, mlcod=134'130)", 2024-09-18T11:32:48.609 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:48.609 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.948987963, 2024-09-18T11:32:48.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051351770000000003, 2024-09-18T11:32:48.609 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:48.609 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:48.609 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:48.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:48.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:48.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:48.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:48.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:48.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:48.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:48.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:48.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582232+0000", 2024-09-18T11:32:48.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5333999999999994e-05 2024-09-18T11:32:48.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:48.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582242+0000", 2024-09-18T11:32:48.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5200000000000003e-06 2024-09-18T11:32:48.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:48.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582248+0000", 2024-09-18T11:32:48.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8000000000000004e-06 2024-09-18T11:32:48.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:48.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582305+0000", 2024-09-18T11:32:48.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7813999999999999e-05 2024-09-18T11:32:48.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:48.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582329+0000", 2024-09-18T11:32:48.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3414999999999999e-05 2024-09-18T11:32:48.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:48.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587257+0000", 2024-09-18T11:32:48.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049279880000000003 2024-09-18T11:32:48.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:48.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587293+0000", 2024-09-18T11:32:48.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5802e-05 2024-09-18T11:32:48.613 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.613 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:48.613 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.613 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3974 4.d e138/22 4:b5a463bf:::smithi11939506-20:head v 138'270, mlcod=137'266)", 2024-09-18T11:32:48.613 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:32:48.614 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.945106692000003, 2024-09-18T11:32:48.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011228458, 2024-09-18T11:32:48.614 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:48.614 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:48.614 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:48.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:48.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:32:48.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:48.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:32:48.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:48.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586038+0000", 2024-09-18T11:32:48.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:48.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:48.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587638+0000", 2024-09-18T11:32:48.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001599184 2024-09-18T11:32:48.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:48.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587647+0000", 2024-09-18T11:32:48.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3719999999999994e-06 2024-09-18T11:32:48.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:48.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587653+0000", 2024-09-18T11:32:48.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.105e-06 2024-09-18T11:32:48.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:48.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587712+0000", 2024-09-18T11:32:48.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9252000000000001e-05 2024-09-18T11:32:48.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:48.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587735+0000", 2024-09-18T11:32:48.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3132000000000001e-05 2024-09-18T11:32:48.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:48.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597228+0000", 2024-09-18T11:32:48.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0094930090000000002 2024-09-18T11:32:48.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:48.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597267+0000", 2024-09-18T11:32:48.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8693999999999999e-05 2024-09-18T11:32:48.617 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.617 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:48.617 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.618 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3975 4.d e138/22 4:b5a463bf:::smithi11939506-20:head v 138'271, mlcod=137'266)", 2024-09-18T11:32:48.618 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:48.618 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 38.938905986000002, 2024-09-18T11:32:48.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053620450000000002, 2024-09-18T11:32:48.618 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:48.618 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:48.618 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:48.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:48.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:48.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:48.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:48.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:48.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:48.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:48.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:48.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:48.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592927+0000", 2024-09-18T11:32:48.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00068878699999999995 2024-09-18T11:32:48.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:48.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592940+0000", 2024-09-18T11:32:48.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:32:48.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:48.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592951+0000", 2024-09-18T11:32:48.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1046000000000001e-05 2024-09-18T11:32:48.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:48.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593008+0000", 2024-09-18T11:32:48.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6875e-05 2024-09-18T11:32:48.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593033+0000", 2024-09-18T11:32:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5341999999999999e-05 2024-09-18T11:32:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597565+0000", 2024-09-18T11:32:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045323150000000003 2024-09-18T11:32:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597601+0000", 2024-09-18T11:32:48.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6071000000000001e-05 2024-09-18T11:32:48.622 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.622 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:48.622 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.622 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:48.622 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:48.622 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:48.623 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:32:48.624 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:48.621+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:48.721 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:48.721 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:48.721 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:48.721 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:48.724 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:48.723+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:48.729 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:32:48.825 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:48.823+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:48.866 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:48.866 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:48.866 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:48.866 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:48.866 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:48.874 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:32:48.926 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:48.924+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:49.015 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:49.015 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:49.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:49.015 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:49.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.015 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:32:49.015 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:49.015 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 157.84327398100001, 2024-09-18T11:32:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:32:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:32:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:49.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:32:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:49.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:32:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:32:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:32:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:32:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:32:49.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:32:49.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:32:49.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:32:49.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:49.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:32:49.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:32:49.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:49.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:32:49.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:32:49.020 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.020 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:49.020 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.020 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:49.020 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:49.020 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 153.84809598699999, 2024-09-18T11:32:49.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:32:49.020 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:49.021 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:49.021 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:49.021 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:49.021 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.021 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:32:49.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.021 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:49.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:49.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:49.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:49.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:49.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:49.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:49.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:49.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:49.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:32:49.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:32:49.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:49.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:32:49.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:32:49.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:32:49.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:32:49.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:32:49.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:32:49.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:49.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:32:49.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:32:49.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:32:49.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:32:49.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:49.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:32:49.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:32:49.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:49.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:32:49.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:32:49.025 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.025 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:49.025 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.026 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:49.026 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:49.026 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 153.848077698, 2024-09-18T11:32:49.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:32:49.026 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:49.026 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:49.026 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:49.026 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:49.026 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.026 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:32:49.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.026 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:49.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:49.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:49.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:49.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:49.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:49.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:32:49.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:49.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:49.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:32:49.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:32:49.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:49.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:32:49.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:32:49.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:32:49.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:32:49.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:32:49.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:32:49.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:49.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:32:49.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:32:49.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:32:49.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:32:49.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:49.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:32:49.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:32:49.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:49.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:32:49.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:32:49.031 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.031 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:49.031 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.031 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:49.031 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:49.031 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 153.84801692900001, 2024-09-18T11:32:49.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:32:49.031 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:49.031 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:49.031 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:49.031 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:49.032 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.032 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:32:49.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.032 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:49.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:49.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:49.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:49.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:49.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:49.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:32:49.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:49.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:49.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:32:49.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:32:49.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:49.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:32:49.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:32:49.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:32:49.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:32:49.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:32:49.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:32:49.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:49.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:32:49.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:32:49.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:32:49.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:32:49.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:49.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:32:49.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:32:49.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:49.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:32:49.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:32:49.036 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.036 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:49.036 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.036 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:49.037 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:49.037 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 153.84799460799999, 2024-09-18T11:32:49.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:32:49.037 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:49.037 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:49.037 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:49.037 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:49.037 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.037 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:32:49.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.037 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:49.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:49.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:49.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:49.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:49.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:49.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:32:49.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:49.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:49.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:32:49.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:32:49.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:49.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:32:49.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:32:49.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:32:49.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:32:49.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:32:49.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:32:49.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:49.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:32:49.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:32:49.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:32:49.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:32:49.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:49.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:32:49.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:32:49.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:49.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:32:49.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:32:49.041 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.041 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:49.041 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.042 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:32:49.042 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:49.042 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 149.066869725, 2024-09-18T11:32:49.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:32:49.042 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:49.042 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.042 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:49.042 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:49.042 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.042 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:32:49.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.043 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:49.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:49.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:49.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:49.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:49.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:49.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:32:49.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:32:49.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:49.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:32:49.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:32:49.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:49.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:32:49.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:32:49.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:32:49.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:32:49.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:32:49.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:32:49.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:49.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:32:49.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:32:49.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:49.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:32:49.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:32:49.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:49.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:32:49.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:32:49.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:49.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:32:49.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:32:49.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:49.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:32:49.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:32:49.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:49.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:32:49.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:32:49.048 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.048 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:49.048 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.048 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2503 3.1 e85/67 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head v 85'117, mlcod=84'115)", 2024-09-18T11:32:49.048 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:49.048 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 100.035471038, 2024-09-18T11:32:49.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086527619999999996, 2024-09-18T11:32:49.048 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:49.048 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.048 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:49.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:49.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:49.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:49.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:49.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:49.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977220+0000", 2024-09-18T11:32:49.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:49.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:49.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980771+0000", 2024-09-18T11:32:49.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035505200000000002 2024-09-18T11:32:49.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:49.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980797+0000", 2024-09-18T11:32:49.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.618e-05 2024-09-18T11:32:49.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980807+0000", 2024-09-18T11:32:49.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0433e-05 2024-09-18T11:32:49.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980861+0000", 2024-09-18T11:32:49.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3415999999999998e-05 2024-09-18T11:32:49.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:49.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980885+0000", 2024-09-18T11:32:49.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4212000000000001e-05 2024-09-18T11:32:49.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:49.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985826+0000", 2024-09-18T11:32:49.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049412450000000004 2024-09-18T11:32:49.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:49.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985875+0000", 2024-09-18T11:32:49.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9116000000000002e-05 2024-09-18T11:32:49.052 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.052 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:49.052 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.053 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2649 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:49.053 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:49.053 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 94.681327668999998, 2024-09-18T11:32:49.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566240000000007, 2024-09-18T11:32:49.053 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:49.053 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:49.053 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:49.053 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:49.053 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.053 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2649 2024-09-18T11:32:49.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.054 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:49.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:49.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:49.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:49.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:49.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:49.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331365+0000", 2024-09-18T11:32:49.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:49.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:49.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331369+0000", 2024-09-18T11:32:49.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.551e-06 2024-09-18T11:32:49.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:49.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331377+0000", 2024-09-18T11:32:49.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8769999999999993e-06 2024-09-18T11:32:49.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331381+0000", 2024-09-18T11:32:49.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.16e-06 2024-09-18T11:32:49.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338973+0000", 2024-09-18T11:32:49.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075918460000000002 2024-09-18T11:32:49.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:49.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339048+0000", 2024-09-18T11:32:49.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5813000000000001e-05 2024-09-18T11:32:49.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:49.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339322+0000", 2024-09-18T11:32:49.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027399899999999998 2024-09-18T11:32:49.057 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.057 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:49.057 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.057 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2650 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:49.057 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:49.058 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 94.681307450999995, 2024-09-18T11:32:49.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084392180000000001, 2024-09-18T11:32:49.058 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:49.058 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:49.058 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:49.058 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:49.058 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.058 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2650 2024-09-18T11:32:49.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.058 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:49.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:49.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:49.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:49.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:49.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:49.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:49.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:49.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:49.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331388+0000", 2024-09-18T11:32:49.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3429999999999998e-06 2024-09-18T11:32:49.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:49.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331394+0000", 2024-09-18T11:32:49.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.19e-06 2024-09-18T11:32:49.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331398+0000", 2024-09-18T11:32:49.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.495e-06 2024-09-18T11:32:49.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339362+0000", 2024-09-18T11:32:49.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079641799999999995 2024-09-18T11:32:49.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:49.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339460+0000", 2024-09-18T11:32:49.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8329e-05 2024-09-18T11:32:49.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:49.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339825+0000", 2024-09-18T11:32:49.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036510500000000001 2024-09-18T11:32:49.062 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.062 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:49.062 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.062 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:127 3.2 3:4d3b277a:::3266c7aaa53a7d2c10c3c54d852d6cbbed49fb041bc0df41a3ac83b56e2b0615:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:49.062 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:49.062 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.968292512000005, 2024-09-18T11:32:49.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093148009999999993, 2024-09-18T11:32:49.063 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:49.063 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.063 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:49.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:49.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:49.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:49.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:49.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:49.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044400+0000", 2024-09-18T11:32:49.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:49.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:49.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044442+0000", 2024-09-18T11:32:49.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2247999999999999e-05 2024-09-18T11:32:49.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:49.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044453+0000", 2024-09-18T11:32:49.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0617e-05 2024-09-18T11:32:49.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044457+0000", 2024-09-18T11:32:49.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2520000000000001e-06 2024-09-18T11:32:49.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044495+0000", 2024-09-18T11:32:49.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7710000000000003e-05 2024-09-18T11:32:49.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:49.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044499+0000", 2024-09-18T11:32:49.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6269999999999999e-06 2024-09-18T11:32:49.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048239+0000", 2024-09-18T11:32:49.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037396719999999999 2024-09-18T11:32:49.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:49.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048351+0000", 2024-09-18T11:32:49.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112342 2024-09-18T11:32:49.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:49.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048545+0000", 2024-09-18T11:32:49.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000193881 2024-09-18T11:32:49.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:49.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.052689+0000", 2024-09-18T11:32:49.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041435780000000002 2024-09-18T11:32:49.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:49.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053654+0000", 2024-09-18T11:32:49.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096558199999999999 2024-09-18T11:32:49.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:49.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053683+0000", 2024-09-18T11:32:49.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8850999999999999e-05 2024-09-18T11:32:49.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:49.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053716+0000", 2024-09-18T11:32:49.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2468000000000002e-05 2024-09-18T11:32:49.068 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.068 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:49.068 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.069 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3209 4.0 4:019b6e09:::smithi11939506-40:head [write 638732~557432] snapc 42=[42,41,3f,3e,3d,3b,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-18T11:32:49.069 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:49.069 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 71.914610472999996, 2024-09-18T11:32:49.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091774999999999999, 2024-09-18T11:32:49.069 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:49.069 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.069 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:49.069 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:49.069 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.069 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3209 2024-09-18T11:32:49.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.070 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:49.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:49.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:49.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:49.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:49.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:49.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098080+0000", 2024-09-18T11:32:49.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:49.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:49.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098676+0000", 2024-09-18T11:32:49.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059598099999999998 2024-09-18T11:32:49.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:49.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098706+0000", 2024-09-18T11:32:49.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0020000000000001e-05 2024-09-18T11:32:49.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098716+0000", 2024-09-18T11:32:49.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0225e-05 2024-09-18T11:32:49.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098782+0000", 2024-09-18T11:32:49.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5530000000000004e-05 2024-09-18T11:32:49.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:49.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098883+0000", 2024-09-18T11:32:49.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101609 2024-09-18T11:32:49.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:49.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.099098+0000", 2024-09-18T11:32:49.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000215052 2024-09-18T11:32:49.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:49.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.102511+0000", 2024-09-18T11:32:49.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034129429999999999 2024-09-18T11:32:49.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:49.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107165+0000", 2024-09-18T11:32:49.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046531480000000002 2024-09-18T11:32:49.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:49.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107217+0000", 2024-09-18T11:32:49.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2058000000000002e-05 2024-09-18T11:32:49.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:49.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107260+0000", 2024-09-18T11:32:49.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3865000000000001e-05 2024-09-18T11:32:49.074 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.075 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:49.075 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.075 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:162 3.1 e138/67 3:8c826a1f:::e682cfc1e82c88472b9acf1cd97978bc105aa035f10b685bb4cace95809b52bf:head v 138'176, mlcod=136'174)", 2024-09-18T11:32:49.075 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:32:49.075 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.429136728000003, 2024-09-18T11:32:49.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010445024000000001, 2024-09-18T11:32:49.075 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:49.075 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.075 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:49.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:49.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:32:49.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:49.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:32:49.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:49.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583556+0000", 2024-09-18T11:32:49.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:49.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:49.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584899+0000", 2024-09-18T11:32:49.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001343056 2024-09-18T11:32:49.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:49.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584929+0000", 2024-09-18T11:32:49.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0133000000000001e-05 2024-09-18T11:32:49.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584937+0000", 2024-09-18T11:32:49.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6380000000000004e-06 2024-09-18T11:32:49.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584990+0000", 2024-09-18T11:32:49.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3186000000000002e-05 2024-09-18T11:32:49.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:49.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585012+0000", 2024-09-18T11:32:49.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1474000000000001e-05 2024-09-18T11:32:49.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:49.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593963+0000", 2024-09-18T11:32:49.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089512859999999993 2024-09-18T11:32:49.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:49.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594002+0000", 2024-09-18T11:32:49.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8859999999999997e-05 2024-09-18T11:32:49.079 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.079 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:49.079 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.080 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:166 3.1 e138/67 3:a73e635b:::b6ee478f0f661ec7bac3d17c2ca2fdaff71a47570f64844a7ba79bbb2a0e8d9d:head v 138'177, mlcod=136'174)", 2024-09-18T11:32:49.080 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:32:49.080 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.427566843999998, 2024-09-18T11:32:49.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089018680000000003, 2024-09-18T11:32:49.080 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:49.080 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.080 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:49.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:49.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:32:49.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:49.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:32:49.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:49.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585126+0000", 2024-09-18T11:32:49.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:49.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:49.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585211+0000", 2024-09-18T11:32:49.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5432999999999993e-05 2024-09-18T11:32:49.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:49.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585220+0000", 2024-09-18T11:32:49.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5879999999999998e-06 2024-09-18T11:32:49.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585225+0000", 2024-09-18T11:32:49.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8999999999999997e-06 2024-09-18T11:32:49.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585705+0000", 2024-09-18T11:32:49.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00048042799999999997 2024-09-18T11:32:49.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:49.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585724+0000", 2024-09-18T11:32:49.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8352e-05 2024-09-18T11:32:49.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:49.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594008+0000", 2024-09-18T11:32:49.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082840919999999998 2024-09-18T11:32:49.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:49.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594028+0000", 2024-09-18T11:32:49.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0869e-05 2024-09-18T11:32:49.084 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.084 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:49.084 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.084 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:168 3.1 e138/67 3:b64d7bad:::7feb22256bdcec217720fd46184ab43ced8923ed66ec6319c57f078d9f123262:head v 138'178, mlcod=136'174)", 2024-09-18T11:32:49.084 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:49.084 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 39.426575110999998, 2024-09-18T11:32:49.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079290650000000008, 2024-09-18T11:32:49.085 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:49.085 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.085 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:49.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:49.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:49.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:49.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:49.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:49.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:49.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:49.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:49.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586191+0000", 2024-09-18T11:32:49.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3627000000000005e-05 2024-09-18T11:32:49.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:49.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586202+0000", 2024-09-18T11:32:49.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0488e-05 2024-09-18T11:32:49.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586207+0000", 2024-09-18T11:32:49.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1719999999999998e-06 2024-09-18T11:32:49.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586239+0000", 2024-09-18T11:32:49.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2570000000000002e-05 2024-09-18T11:32:49.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:49.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586271+0000", 2024-09-18T11:32:49.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1341999999999999e-05 2024-09-18T11:32:49.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:49.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594031+0000", 2024-09-18T11:32:49.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077597459999999997 2024-09-18T11:32:49.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:49.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594047+0000", 2024-09-18T11:32:49.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6858e-05 2024-09-18T11:32:49.089 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.089 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:49.089 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.089 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4267 3.1 3:8d8d13e9:::92886a4340d03039afb2c877fad76e3afe7a80ee04388d55256c386022cd5d63:head [write 0~2203648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:49.089 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:32:49.089 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.955543130999999, 2024-09-18T11:32:49.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0092856740000000007, 2024-09-18T11:32:49.089 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:49.089 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.089 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:49.089 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:49.090 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.090 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4267 2024-09-18T11:32:49.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.090 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:49.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:49.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:32:49.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:49.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:32:49.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:49.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057149+0000", 2024-09-18T11:32:49.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:49.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:49.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058175+0000", 2024-09-18T11:32:49.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001026765 2024-09-18T11:32:49.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:49.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058190+0000", 2024-09-18T11:32:49.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4167e-05 2024-09-18T11:32:49.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058195+0000", 2024-09-18T11:32:49.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.823e-06 2024-09-18T11:32:49.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058247+0000", 2024-09-18T11:32:49.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1304999999999998e-05 2024-09-18T11:32:49.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:49.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058422+0000", 2024-09-18T11:32:49.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017527100000000001 2024-09-18T11:32:49.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:49.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058609+0000", 2024-09-18T11:32:49.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018714299999999999 2024-09-18T11:32:49.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:49.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.062721+0000", 2024-09-18T11:32:49.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041118869999999998 2024-09-18T11:32:49.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:49.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066291+0000", 2024-09-18T11:32:49.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035694860000000002 2024-09-18T11:32:49.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:49.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066328+0000", 2024-09-18T11:32:49.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7864000000000002e-05 2024-09-18T11:32:49.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:49.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066436+0000", 2024-09-18T11:32:49.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010762 2024-09-18T11:32:49.095 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.095 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:49.095 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.095 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:90 3.1 3:a224eaa1:::22b064a083114a2756d85a07d2762a393d3e92dd898a5334bc045c9031244d76:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:32:49.095 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:40.057559+0000", 2024-09-18T11:32:49.095 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.955134352, 2024-09-18T11:32:49.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085024620000000006, 2024-09-18T11:32:49.095 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:49.096 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.096 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:49.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:49.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057559+0000", 2024-09-18T11:32:49.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:49.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057559+0000", 2024-09-18T11:32:49.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:49.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057558+0000", 2024-09-18T11:32:49.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:49.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:49.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057563+0000", 2024-09-18T11:32:49.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7990000000000001e-06 2024-09-18T11:32:49.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:49.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057572+0000", 2024-09-18T11:32:49.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0510000000000002e-06 2024-09-18T11:32:49.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057576+0000", 2024-09-18T11:32:49.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2509999999999996e-06 2024-09-18T11:32:49.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057624+0000", 2024-09-18T11:32:49.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7636000000000003e-05 2024-09-18T11:32:49.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:49.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057722+0000", 2024-09-18T11:32:49.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7676999999999993e-05 2024-09-18T11:32:49.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:49.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057883+0000", 2024-09-18T11:32:49.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000160922 2024-09-18T11:32:49.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:49.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059927+0000", 2024-09-18T11:32:49.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020447350000000002 2024-09-18T11:32:49.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:49.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.065977+0000", 2024-09-18T11:32:49.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060497679999999996 2024-09-18T11:32:49.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:49.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066023+0000", 2024-09-18T11:32:49.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6041000000000002e-05 2024-09-18T11:32:49.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:49.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066062+0000", 2024-09-18T11:32:49.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8469e-05 2024-09-18T11:32:49.101 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.101 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:49.101 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.101 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.5.13:42 3.1 3:8372521c:::d4b4fff22a71c591f3e871de3fede050f6721df41a8c639a614c0b94ba6762c2:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:32:49.101 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:40.058327+0000", 2024-09-18T11:32:49.101 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.9543666789999996, 2024-09-18T11:32:49.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079034699999999992, 2024-09-18T11:32:49.101 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:49.101 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.102 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:49.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:49.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058327+0000", 2024-09-18T11:32:49.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:49.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058327+0000", 2024-09-18T11:32:49.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:49.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058325+0000", 2024-09-18T11:32:49.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:49.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:49.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058339+0000", 2024-09-18T11:32:49.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3337e-05 2024-09-18T11:32:49.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:49.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058357+0000", 2024-09-18T11:32:49.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8615e-05 2024-09-18T11:32:49.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058366+0000", 2024-09-18T11:32:49.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9460000000000004e-06 2024-09-18T11:32:49.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058658+0000", 2024-09-18T11:32:49.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029212200000000002 2024-09-18T11:32:49.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:49.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058757+0000", 2024-09-18T11:32:49.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9025999999999997e-05 2024-09-18T11:32:49.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:49.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058914+0000", 2024-09-18T11:32:49.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000156236 2024-09-18T11:32:49.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:49.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.060326+0000", 2024-09-18T11:32:49.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014118710000000001 2024-09-18T11:32:49.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:49.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066122+0000", 2024-09-18T11:32:49.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057962029999999998 2024-09-18T11:32:49.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:49.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066192+0000", 2024-09-18T11:32:49.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0533999999999999e-05 2024-09-18T11:32:49.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:49.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066230+0000", 2024-09-18T11:32:49.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7914000000000003e-05 2024-09-18T11:32:49.107 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.107 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:49.107 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.107 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.5.13:43 3.1 3:b44664f3:::7f31751f088e4f7577561c03b406e8a8bc48ebfdd310b40206061655307aebe3:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:32:49.107 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:40.058386+0000", 2024-09-18T11:32:49.107 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.9543076510000006, 2024-09-18T11:32:49.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079575740000000002, 2024-09-18T11:32:49.107 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:49.107 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.107 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:49.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:49.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058386+0000", 2024-09-18T11:32:49.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:49.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058386+0000", 2024-09-18T11:32:49.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:49.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058385+0000", 2024-09-18T11:32:49.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:49.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:49.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058392+0000", 2024-09-18T11:32:49.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6200000000000001e-06 2024-09-18T11:32:49.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:49.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058400+0000", 2024-09-18T11:32:49.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7849999999999997e-06 2024-09-18T11:32:49.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058408+0000", 2024-09-18T11:32:49.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1770000000000001e-06 2024-09-18T11:32:49.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059302+0000", 2024-09-18T11:32:49.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000894851 2024-09-18T11:32:49.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:49.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059400+0000", 2024-09-18T11:32:49.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7460000000000005e-05 2024-09-18T11:32:49.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:49.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059520+0000", 2024-09-18T11:32:49.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000120457 2024-09-18T11:32:49.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:49.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.060333+0000", 2024-09-18T11:32:49.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00081255599999999998 2024-09-18T11:32:49.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:49.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066274+0000", 2024-09-18T11:32:49.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059409700000000003 2024-09-18T11:32:49.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:49.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066306+0000", 2024-09-18T11:32:49.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2020000000000002e-05 2024-09-18T11:32:49.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:49.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066343+0000", 2024-09-18T11:32:49.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7438999999999998e-05 2024-09-18T11:32:49.112 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.112 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:49.112 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.113 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:1265 4.f e173/22 4:f3c51b3f:::smithi11939506-21:6c v 173'421, mlcod=172'416)", 2024-09-18T11:32:49.113 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:44.453434+0000", 2024-09-18T11:32:49.113 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.5592592950000004, 2024-09-18T11:32:49.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.007737893, 2024-09-18T11:32:49.113 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:49.113 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.113 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:49.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:49.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:44.453434+0000", 2024-09-18T11:32:49.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:49.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:44.453434+0000", 2024-09-18T11:32:49.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:49.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:44.453434+0000", 2024-09-18T11:32:49.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:49.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:49.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:44.453439+0000", 2024-09-18T11:32:49.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8069999999999999e-06 2024-09-18T11:32:49.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:49.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:44.453445+0000", 2024-09-18T11:32:49.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6460000000000004e-06 2024-09-18T11:32:49.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:44.453448+0000", 2024-09-18T11:32:49.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9299999999999999e-06 2024-09-18T11:32:49.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:44.455833+0000", 2024-09-18T11:32:49.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023846879999999998 2024-09-18T11:32:49.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:49.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:44.455855+0000", 2024-09-18T11:32:49.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1747999999999999e-05 2024-09-18T11:32:49.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:49.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:44.461134+0000", 2024-09-18T11:32:49.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052790659999999998 2024-09-18T11:32:49.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:49.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:44.461172+0000", 2024-09-18T11:32:49.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8452999999999999e-05 2024-09-18T11:32:49.117 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.117 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:49.117 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.117 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.4.13:145 3.2 3:435dd00a:::773f1f20723e27798deecfe7048ee8e76eeec814e6d6ad791ed305ff28c69be2:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:32:49.117 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:45.321055+0000", 2024-09-18T11:32:49.117 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 3.691638846, 2024-09-18T11:32:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079066459999999998, 2024-09-18T11:32:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321055+0000", 2024-09-18T11:32:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321055+0000", 2024-09-18T11:32:49.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321053+0000", 2024-09-18T11:32:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321062+0000", 2024-09-18T11:32:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0399999999999998e-06 2024-09-18T11:32:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:49.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321076+0000", 2024-09-18T11:32:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3869999999999999e-05 2024-09-18T11:32:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321083+0000", 2024-09-18T11:32:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6159999999999998e-06 2024-09-18T11:32:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321156+0000", 2024-09-18T11:32:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3093e-05 2024-09-18T11:32:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321274+0000", 2024-09-18T11:32:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000118353 2024-09-18T11:32:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321443+0000", 2024-09-18T11:32:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016836600000000001 2024-09-18T11:32:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.324281+0000", 2024-09-18T11:32:49.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028381270000000002 2024-09-18T11:32:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.328785+0000", 2024-09-18T11:32:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045048370000000003 2024-09-18T11:32:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.328910+0000", 2024-09-18T11:32:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012438600000000001 2024-09-18T11:32:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:49.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:49.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.328961+0000", 2024-09-18T11:32:49.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1375000000000002e-05 2024-09-18T11:32:49.123 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.123 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:49.123 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.123 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:49.123 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:49.123 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:49.125 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:32:49.125 INFO:tasks.rados.rados.0.smithi119.stdout:2014: finishing write tid 1 to smithi11939506-48 2024-09-18T11:32:49.125 INFO:tasks.rados.rados.0.smithi119.stdout:2014: finishing write tid 2 to smithi11939506-48 2024-09-18T11:32:49.125 INFO:tasks.rados.rados.0.smithi119.stdout:2014: finishing write tid 3 to smithi11939506-48 2024-09-18T11:32:49.125 INFO:tasks.rados.rados.0.smithi119.stdout:2014: finishing write tid 4 to smithi11939506-48 2024-09-18T11:32:49.125 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 48 v 403 (ObjNum 414 snap 69 seq_num 414) dirty exists 2024-09-18T11:32:49.125 INFO:tasks.rados.rados.0.smithi119.stdout:2014: left oid 48 (ObjNum 414 snap 69 seq_num 414) 2024-09-18T11:32:49.125 INFO:tasks.rados.rados.0.smithi119.stdout:2017: finishing set_chunk to oid 26 2024-09-18T11:32:49.125 INFO:tasks.rados.rados.0.smithi119.stdout:2017: got expected ERANGE (src dne) 2024-09-18T11:32:49.126 INFO:tasks.rados.rados.0.smithi119.stdout:2016: expect (ObjNum 345 snap 51 seq_num 345) 2024-09-18T11:32:49.126 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:49.123+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:49.225 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:49.224+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:49.236 INFO:tasks.rados.rados.0.smithi119.stdout:2015: finishing write tid 1 to smithi11939506-35 2024-09-18T11:32:49.236 INFO:tasks.rados.rados.0.smithi119.stdout:2015: finishing write tid 2 to smithi11939506-35 2024-09-18T11:32:49.236 INFO:tasks.rados.rados.0.smithi119.stdout:2015: finishing write tid 3 to smithi11939506-35 2024-09-18T11:32:49.236 INFO:tasks.rados.rados.0.smithi119.stdout:2015: finishing write tid 4 to smithi11939506-35 2024-09-18T11:32:49.237 INFO:tasks.rados.rados.0.smithi119.stdout:2015: finishing write tid 5 to smithi11939506-35 2024-09-18T11:32:49.237 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 35 v 295 (ObjNum 415 snap 69 seq_num 415) dirty exists 2024-09-18T11:32:49.237 INFO:tasks.rados.rados.0.smithi119.stdout:2015: left oid 35 (ObjNum 415 snap 69 seq_num 415) 2024-09-18T11:32:49.237 INFO:tasks.rados.rados.0.smithi119.stdout:2014: done (4 left) 2024-09-18T11:32:49.237 INFO:tasks.rados.rados.0.smithi119.stdout:2015: done (3 left) 2024-09-18T11:32:49.238 INFO:tasks.rados.rados.0.smithi119.stdout:2016: done (2 left) 2024-09-18T11:32:49.238 INFO:tasks.rados.rados.0.smithi119.stdout:2017: done (1 left) 2024-09-18T11:32:49.238 INFO:tasks.rados.rados.0.smithi119.stdout:2018: done (0 left) 2024-09-18T11:32:49.238 INFO:tasks.rados.rados.0.smithi119.stdout:2019: read oid 16 snap 58 2024-09-18T11:32:49.238 INFO:tasks.rados.rados.0.smithi119.stdout:2019: expect (ObjNum 359 snap 55 seq_num 359) 2024-09-18T11:32:49.238 INFO:tasks.rados.rados.0.smithi119.stdout:2020: read oid 24 snap 67 2024-09-18T11:32:49.239 INFO:tasks.rados.rados.0.smithi119.stdout:2020: expect (ObjNum 377 snap 58 seq_num 377) 2024-09-18T11:32:49.239 INFO:tasks.rados.rados.0.smithi119.stdout:2021: read oid 37 snap -1 2024-09-18T11:32:49.239 INFO:tasks.rados.rados.0.smithi119.stdout:2021: expect (ObjNum 401 snap 66 seq_num 401) 2024-09-18T11:32:49.239 INFO:tasks.rados.rados.0.smithi119.stdout:2022: snap_create 2024-09-18T11:32:49.240 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:49.233+0000 7fd2785a3640 -1 osd.6 177 handle_osd_map injecting map crc failure 2024-09-18T11:32:49.240 INFO:tasks.rados.rados.0.smithi119.stdout:2019: expect (ObjNum 359 snap 55 seq_num 359) 2024-09-18T11:32:49.241 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:49.241 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:49.241 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:49.241 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:49.249 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:32:49.326 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:49.324+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:49.348 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:49.348 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:49.348 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:49.349 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:49.349 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:49.351 INFO:tasks.rados.rados.0.smithi119.stdout:2020: expect (ObjNum 377 snap 58 seq_num 377) 2024-09-18T11:32:49.357 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:32:49.425 INFO:tasks.rados.rados.0.smithi119.stdout:2021: expect (ObjNum 401 snap 66 seq_num 401) 2024-09-18T11:32:49.441 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:49.440+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:49.485 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T11:32:49.497 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:49.497 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:49.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:49.497 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:49.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.497 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:32:49.497 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:49.498 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 135.07872715600001, 2024-09-18T11:32:49.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:32:49.498 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:49.498 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.498 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:49.498 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:49.498 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.498 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:32:49.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.498 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:49.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:49.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:49.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:49.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:49.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:49.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:32:49.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:49.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:49.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:32:49.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:32:49.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:49.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:32:49.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:32:49.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:32:49.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:32:49.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:32:49.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:32:49.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:49.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:32:49.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:32:49.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:49.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:32:49.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:32:49.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:49.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:32:49.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:32:49.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:49.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:32:49.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:32:49.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:49.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:32:49.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:32:49.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:49.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:32:49.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:32:49.503 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.503 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:49.503 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.504 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:32:49.504 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:49.504 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 108.10849775, 2024-09-18T11:32:49.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:32:49.504 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:49.504 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.504 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:49.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.504 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:49.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:49.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:49.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:49.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:49.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:49.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:49.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:49.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:32:49.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:32:49.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:49.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:32:49.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:32:49.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:32:49.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:32:49.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:32:49.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:32:49.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:49.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:32:49.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:32:49.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:49.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:32:49.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:32:49.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:49.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:32:49.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:32:49.508 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.508 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:49.508 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.508 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:49.508 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:49.508 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 100.522385144, 2024-09-18T11:32:49.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:32:49.509 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:49.509 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.509 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:49.509 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:49.509 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.509 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:32:49.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.509 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:49.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:49.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:49.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:49.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:49.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:49.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:32:49.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:49.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:49.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:32:49.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:32:49.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:32:49.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:49.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:32:49.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:32:49.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:49.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:32:49.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:32:49.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:49.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:32:49.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:32:49.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:49.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:32:49.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:32:49.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:49.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:32:49.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:32:49.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:49.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:32:49.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:32:49.514 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.514 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:49.514 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.514 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2956 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~2100908,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:49.514 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:49.515 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.467588894000002, 2024-09-18T11:32:49.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011863831, 2024-09-18T11:32:49.515 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:49.515 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:49.515 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:49.515 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:49.515 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.515 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2956 2024-09-18T11:32:49.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.515 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:49.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:49.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:49.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:49.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:49.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:49.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:49.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:49.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:49.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027228+0000", 2024-09-18T11:32:49.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:32:49.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:49.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027235+0000", 2024-09-18T11:32:49.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:32:49.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027239+0000", 2024-09-18T11:32:49.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.535e-06 2024-09-18T11:32:49.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027831+0000", 2024-09-18T11:32:49.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059244699999999996 2024-09-18T11:32:49.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:49.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027879+0000", 2024-09-18T11:32:49.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:32:49.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.519 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038098+0000", 2024-09-18T11:32:49.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010219137 2024-09-18T11:32:49.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.519 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:49.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038207+0000", 2024-09-18T11:32:49.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010951999999999999 2024-09-18T11:32:49.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.519 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:49.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039089+0000", 2024-09-18T11:32:49.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00088151699999999996 2024-09-18T11:32:49.519 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.520 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:49.520 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.520 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2957 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:49.520 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:49.520 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.467571109999994, 2024-09-18T11:32:49.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012041979, 2024-09-18T11:32:49.520 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:49.520 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:49.520 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:49.520 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:49.520 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.521 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2957 2024-09-18T11:32:49.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.521 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:49.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:49.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:49.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:49.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:49.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:49.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:49.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:49.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:49.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027245+0000", 2024-09-18T11:32:49.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:32:49.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:49.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027251+0000", 2024-09-18T11:32:49.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0090000000000004e-06 2024-09-18T11:32:49.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027255+0000", 2024-09-18T11:32:49.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:32:49.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027895+0000", 2024-09-18T11:32:49.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064076400000000005 2024-09-18T11:32:49.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:49.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027930+0000", 2024-09-18T11:32:49.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4334e-05 2024-09-18T11:32:49.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.524 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039111+0000", 2024-09-18T11:32:49.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181455999999999 2024-09-18T11:32:49.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.524 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:49.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039156+0000", 2024-09-18T11:32:49.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5256000000000002e-05 2024-09-18T11:32:49.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.524 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:49.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039285+0000", 2024-09-18T11:32:49.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012839200000000001 2024-09-18T11:32:49.525 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.525 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:49.525 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.525 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:978 4.b e132/67 4:d8d14c58:::smithi11939506-49:30 v 132'417, mlcod=128'414)", 2024-09-18T11:32:49.525 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:49.525 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 46.469539828000002, 2024-09-18T11:32:49.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011942183, 2024-09-18T11:32:49.525 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:49.525 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.525 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:49.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:49.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:49.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:49.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:49.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:49.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:49.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:49.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:49.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025278+0000", 2024-09-18T11:32:49.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9060000000000004e-06 2024-09-18T11:32:49.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:49.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025288+0000", 2024-09-18T11:32:49.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.028e-05 2024-09-18T11:32:49.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025293+0000", 2024-09-18T11:32:49.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2700000000000004e-06 2024-09-18T11:32:49.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025346+0000", 2024-09-18T11:32:49.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2862999999999999e-05 2024-09-18T11:32:49.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:49.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025370+0000", 2024-09-18T11:32:49.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4005000000000001e-05 2024-09-18T11:32:49.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:49.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037159+0000", 2024-09-18T11:32:49.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011788343 2024-09-18T11:32:49.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:49.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037216+0000", 2024-09-18T11:32:49.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7695000000000002e-05 2024-09-18T11:32:49.529 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.529 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:49.529 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.530 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3930 4.b e132/67 4:d3ae8215:::smithi11939506-41:head v 132'421, mlcod=128'414)", 2024-09-18T11:32:49.530 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:49.530 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 46.465991580000001, 2024-09-18T11:32:49.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015093554, 2024-09-18T11:32:49.530 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:49.530 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.530 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:49.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.530 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:49.530 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:49.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:49.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:49.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:49.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028821+0000", 2024-09-18T11:32:49.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:49.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:49.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029805+0000", 2024-09-18T11:32:49.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00098410899999999994 2024-09-18T11:32:49.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:49.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029817+0000", 2024-09-18T11:32:49.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2058e-05 2024-09-18T11:32:49.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029823+0000", 2024-09-18T11:32:49.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6450000000000002e-06 2024-09-18T11:32:49.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032330+0000", 2024-09-18T11:32:49.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025067710000000001 2024-09-18T11:32:49.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:49.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032349+0000", 2024-09-18T11:32:49.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9729e-05 2024-09-18T11:32:49.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:49.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043878+0000", 2024-09-18T11:32:49.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011528514 2024-09-18T11:32:49.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:49.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043916+0000", 2024-09-18T11:32:49.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8049000000000001e-05 2024-09-18T11:32:49.534 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.534 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:49.534 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.534 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:162 3.1 3:8c826a1f:::e682cfc1e82c88472b9acf1cd97978bc105aa035f10b685bb4cace95809b52bf:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:49.534 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:32:49.534 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 39.913869421999998, 2024-09-18T11:32:49.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013234571000000001, 2024-09-18T11:32:49.534 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:49.534 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.534 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:49.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.535 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:49.535 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:32:49.535 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.535 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:49.535 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:32:49.535 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.535 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:49.535 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580943+0000", 2024-09-18T11:32:49.535 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:49.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.536 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:49.536 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582556+0000", 2024-09-18T11:32:49.536 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016132970000000001 2024-09-18T11:32:49.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.536 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:49.536 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582572+0000", 2024-09-18T11:32:49.536 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5846999999999999e-05 2024-09-18T11:32:49.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.536 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582578+0000", 2024-09-18T11:32:49.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4810000000000003e-06 2024-09-18T11:32:49.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.537 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582632+0000", 2024-09-18T11:32:49.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4156000000000001e-05 2024-09-18T11:32:49.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.537 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:49.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582819+0000", 2024-09-18T11:32:49.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000186431 2024-09-18T11:32:49.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.538 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:49.538 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583118+0000", 2024-09-18T11:32:49.538 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029880100000000001 2024-09-18T11:32:49.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.538 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:49.538 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590676+0000", 2024-09-18T11:32:49.538 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075588670000000004 2024-09-18T11:32:49.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.538 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:49.538 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594111+0000", 2024-09-18T11:32:49.538 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034348550000000001 2024-09-18T11:32:49.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:49.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594142+0000", 2024-09-18T11:32:49.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.099e-05 2024-09-18T11:32:49.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:49.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594179+0000", 2024-09-18T11:32:49.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6831e-05 2024-09-18T11:32:49.539 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.539 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:49.539 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.540 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3974 4.d 4:b5a463bf:::smithi11939506-20:head [write 665643~697786,stat] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:49.540 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:32:49.540 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 39.912782739000001, 2024-09-18T11:32:49.540 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016621174999999998, 2024-09-18T11:32:49.540 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:49.540 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.540 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:49.540 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:49.540 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.540 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3974 2024-09-18T11:32:49.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.540 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:49.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:49.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:32:49.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:49.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:32:49.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:49.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582030+0000", 2024-09-18T11:32:49.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:49.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.542 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:49.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585185+0000", 2024-09-18T11:32:49.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003155644 2024-09-18T11:32:49.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.542 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:49.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585208+0000", 2024-09-18T11:32:49.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3013000000000001e-05 2024-09-18T11:32:49.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585215+0000", 2024-09-18T11:32:49.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7569999999999998e-06 2024-09-18T11:32:49.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585278+0000", 2024-09-18T11:32:49.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2699000000000004e-05 2024-09-18T11:32:49.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:49.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585374+0000", 2024-09-18T11:32:49.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.6062000000000003e-05 2024-09-18T11:32:49.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:32:49.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585577+0000", 2024-09-18T11:32:49.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000202926 2024-09-18T11:32:49.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:49.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596544+0000", 2024-09-18T11:32:49.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010966848 2024-09-18T11:32:49.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:49.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598594+0000", 2024-09-18T11:32:49.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020502049999999998 2024-09-18T11:32:49.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.545 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:49.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598625+0000", 2024-09-18T11:32:49.545 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1412999999999999e-05 2024-09-18T11:32:49.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.545 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:49.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598652+0000", 2024-09-18T11:32:49.545 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7166999999999999e-05 2024-09-18T11:32:49.545 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.545 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:49.545 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.546 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:166 3.1 3:a73e635b:::b6ee478f0f661ec7bac3d17c2ca2fdaff71a47570f64844a7ba79bbb2a0e8d9d:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:49.546 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:32:49.546 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 39.912223373000003, 2024-09-18T11:32:49.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011711037000000001, 2024-09-18T11:32:49.546 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:49.546 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.546 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:49.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.546 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:49.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:32:49.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.547 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:49.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:32:49.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.547 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:49.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582590+0000", 2024-09-18T11:32:49.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:49.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.547 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:49.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582673+0000", 2024-09-18T11:32:49.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2861000000000005e-05 2024-09-18T11:32:49.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:49.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582683+0000", 2024-09-18T11:32:49.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0081999999999999e-05 2024-09-18T11:32:49.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582688+0000", 2024-09-18T11:32:49.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6870000000000002e-06 2024-09-18T11:32:49.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584266+0000", 2024-09-18T11:32:49.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001578423 2024-09-18T11:32:49.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:49.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584407+0000", 2024-09-18T11:32:49.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014144599999999999 2024-09-18T11:32:49.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:49.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584599+0000", 2024-09-18T11:32:49.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000191881 2024-09-18T11:32:49.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:49.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590680+0000", 2024-09-18T11:32:49.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060801479999999996 2024-09-18T11:32:49.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:49.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594224+0000", 2024-09-18T11:32:49.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035441700000000001 2024-09-18T11:32:49.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:49.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594269+0000", 2024-09-18T11:32:49.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5297999999999999e-05 2024-09-18T11:32:49.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.551 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:49.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594302+0000", 2024-09-18T11:32:49.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2703000000000002e-05 2024-09-18T11:32:49.551 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.551 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:49.551 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.551 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:168 3.1 3:b64d7bad:::7feb22256bdcec217720fd46184ab43ced8923ed66ec6319c57f078d9f123262:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:49.551 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:32:49.551 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 39.911995920999999, 2024-09-18T11:32:49.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011589809, 2024-09-18T11:32:49.552 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:49.552 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.552 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:49.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.552 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:49.552 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:32:49.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.552 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:49.552 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:32:49.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:49.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582817+0000", 2024-09-18T11:32:49.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:49.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:49.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582903+0000", 2024-09-18T11:32:49.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5915999999999995e-05 2024-09-18T11:32:49.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:49.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582913+0000", 2024-09-18T11:32:49.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1409999999999994e-06 2024-09-18T11:32:49.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.554 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582917+0000", 2024-09-18T11:32:49.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6419999999999998e-06 2024-09-18T11:32:49.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.554 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585540+0000", 2024-09-18T11:32:49.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026228940000000002 2024-09-18T11:32:49.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.554 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:49.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585721+0000", 2024-09-18T11:32:49.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018086299999999999 2024-09-18T11:32:49.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:49.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585918+0000", 2024-09-18T11:32:49.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000196758 2024-09-18T11:32:49.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:49.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590683+0000", 2024-09-18T11:32:49.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047657020000000001 2024-09-18T11:32:49.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:49.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594354+0000", 2024-09-18T11:32:49.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003671068 2024-09-18T11:32:49.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.556 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:49.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594386+0000", 2024-09-18T11:32:49.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1426e-05 2024-09-18T11:32:49.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.556 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:49.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594408+0000", 2024-09-18T11:32:49.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1987000000000001e-05 2024-09-18T11:32:49.556 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.556 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:49.557 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.557 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:175 3.1 3:b5e86b88:::e55cf3d5cf782ed327a67e746b287b76d05f74f58a4f3726e399a564440ceaf6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:49.557 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:32:49.557 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 39.911146979999998, 2024-09-18T11:32:49.557 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013709627, 2024-09-18T11:32:49.557 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:49.557 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.557 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:49.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.557 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:49.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:32:49.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.558 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:49.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:32:49.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.558 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:49.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583666+0000", 2024-09-18T11:32:49.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:49.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.558 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:49.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584041+0000", 2024-09-18T11:32:49.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037511499999999998 2024-09-18T11:32:49.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:49.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584055+0000", 2024-09-18T11:32:49.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3295999999999999e-05 2024-09-18T11:32:49.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584061+0000", 2024-09-18T11:32:49.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1630000000000001e-06 2024-09-18T11:32:49.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.589980+0000", 2024-09-18T11:32:49.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005918998 2024-09-18T11:32:49.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:49.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590117+0000", 2024-09-18T11:32:49.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013715400000000001 2024-09-18T11:32:49.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:49.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590270+0000", 2024-09-18T11:32:49.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015245600000000001 2024-09-18T11:32:49.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.561 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:49.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596513+0000", 2024-09-18T11:32:49.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062438729999999996 2024-09-18T11:32:49.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.561 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:49.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597328+0000", 2024-09-18T11:32:49.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081489599999999996 2024-09-18T11:32:49.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.561 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:49.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597358+0000", 2024-09-18T11:32:49.562 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9291000000000001e-05 2024-09-18T11:32:49.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.562 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:49.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597377+0000", 2024-09-18T11:32:49.562 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:32:49.562 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.562 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:49.562 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.562 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4061 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:49.563 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:32:49.563 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 37.197968291000002, 2024-09-18T11:32:49.563 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013340967, 2024-09-18T11:32:49.563 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:49.563 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:49.563 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:49.563 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:49.563 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.563 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4061 2024-09-18T11:32:49.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.563 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:49.563 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.563 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:49.564 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:32:49.564 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.564 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.564 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:49.564 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:32:49.564 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.564 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.564 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:49.564 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296844+0000", 2024-09-18T11:32:49.564 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:49.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.564 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.565 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:49.565 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296854+0000", 2024-09-18T11:32:49.565 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0037000000000001e-05 2024-09-18T11:32:49.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.565 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.565 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:49.565 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296873+0000", 2024-09-18T11:32:49.565 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:32:49.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.565 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.565 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.565 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296881+0000", 2024-09-18T11:32:49.565 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3790000000000006e-06 2024-09-18T11:32:49.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.566 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.566 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296975+0000", 2024-09-18T11:32:49.566 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3536999999999996e-05 2024-09-18T11:32:49.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.566 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:49.566 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296982+0000", 2024-09-18T11:32:49.566 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6399999999999997e-06 2024-09-18T11:32:49.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.566 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.567 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301933+0000", 2024-09-18T11:32:49.567 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049507190000000001 2024-09-18T11:32:49.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.567 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.567 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:49.567 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301984+0000", 2024-09-18T11:32:49.567 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1439e-05 2024-09-18T11:32:49.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.567 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.567 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:49.567 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:32:49.567 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082022050000000006 2024-09-18T11:32:49.567 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.567 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:49.568 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.568 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.568 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4062 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:49.568 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:49.568 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 37.197919736000003, 2024-09-18T11:32:49.568 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016297147000000001, 2024-09-18T11:32:49.568 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:49.568 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:49.568 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:49.568 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:49.568 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.568 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4062 2024-09-18T11:32:49.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.569 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:49.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.569 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:49.569 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:49.569 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.569 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:49.569 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:49.569 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.570 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:49.570 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:49.570 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:49.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.570 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.570 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:49.570 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296898+0000", 2024-09-18T11:32:49.570 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9419999999999997e-06 2024-09-18T11:32:49.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.570 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.570 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:49.570 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296906+0000", 2024-09-18T11:32:49.570 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3520000000000007e-06 2024-09-18T11:32:49.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.571 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.571 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.571 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296910+0000", 2024-09-18T11:32:49.571 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4469999999999999e-06 2024-09-18T11:32:49.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.571 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.571 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.571 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297032+0000", 2024-09-18T11:32:49.571 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012168900000000001 2024-09-18T11:32:49.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.571 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.571 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:49.571 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297038+0000", 2024-09-18T11:32:49.572 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9320000000000001e-06 2024-09-18T11:32:49.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.572 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.572 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.572 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310221+0000", 2024-09-18T11:32:49.572 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013182887000000001 2024-09-18T11:32:49.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.572 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.572 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:49.572 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310323+0000", 2024-09-18T11:32:49.572 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010170000000000001 2024-09-18T11:32:49.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.572 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.573 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:49.573 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313191+0000", 2024-09-18T11:32:49.573 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002868808 2024-09-18T11:32:49.573 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.573 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:49.573 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.573 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.573 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4063 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:49.573 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:49.573 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 37.197898844000001, 2024-09-18T11:32:49.573 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017648712, 2024-09-18T11:32:49.573 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:49.574 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:49.574 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:49.574 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:49.574 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.574 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4063 2024-09-18T11:32:49.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.574 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:49.574 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.574 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:49.574 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:49.574 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.574 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.575 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:49.575 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:49.575 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.575 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.575 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:49.575 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:49.575 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:49.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.575 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.575 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:49.575 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296918+0000", 2024-09-18T11:32:49.575 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8100000000000002e-06 2024-09-18T11:32:49.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.575 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.576 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:49.576 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296924+0000", 2024-09-18T11:32:49.576 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2519999999999996e-06 2024-09-18T11:32:49.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.576 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.576 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296928+0000", 2024-09-18T11:32:49.576 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0500000000000002e-06 2024-09-18T11:32:49.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.576 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.576 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297062+0000", 2024-09-18T11:32:49.576 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013425500000000001 2024-09-18T11:32:49.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.577 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:49.577 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297069+0000", 2024-09-18T11:32:49.577 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5239999999999997e-06 2024-09-18T11:32:49.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.577 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.577 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313424+0000", 2024-09-18T11:32:49.577 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016355277000000001 2024-09-18T11:32:49.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.577 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:49.577 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313514+0000", 2024-09-18T11:32:49.578 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0581999999999994e-05 2024-09-18T11:32:49.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.578 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.578 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:49.578 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.314564+0000", 2024-09-18T11:32:49.578 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001049384 2024-09-18T11:32:49.578 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.578 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:49.578 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.578 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.578 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4065 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:49.578 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:32:49.578 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 37.197853403000003, 2024-09-18T11:32:49.579 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013225997, 2024-09-18T11:32:49.579 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:49.579 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:49.579 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:49.579 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:49.579 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.579 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4065 2024-09-18T11:32:49.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.579 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:49.579 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.579 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:49.579 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:32:49.579 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.580 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.580 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:49.580 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:32:49.580 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.580 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.580 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:49.580 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296960+0000", 2024-09-18T11:32:49.580 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:49.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.580 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.580 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:49.580 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296963+0000", 2024-09-18T11:32:49.581 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2270000000000002e-06 2024-09-18T11:32:49.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.581 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.581 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:49.581 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296970+0000", 2024-09-18T11:32:49.581 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6220000000000003e-06 2024-09-18T11:32:49.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.581 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.581 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.581 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296973+0000", 2024-09-18T11:32:49.581 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3730000000000001e-06 2024-09-18T11:32:49.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.581 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.581 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.582 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297097+0000", 2024-09-18T11:32:49.582 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012352399999999999 2024-09-18T11:32:49.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.582 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.582 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:49.582 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297103+0000", 2024-09-18T11:32:49.582 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7119999999999997e-06 2024-09-18T11:32:49.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.582 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.582 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.582 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307831+0000", 2024-09-18T11:32:49.582 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010728794 2024-09-18T11:32:49.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.583 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.583 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:49.583 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307924+0000", 2024-09-18T11:32:49.583 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2099e-05 2024-09-18T11:32:49.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.583 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.583 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:49.583 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:32:49.583 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022630469999999998 2024-09-18T11:32:49.583 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.583 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:49.583 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.583 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.584 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4066 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:49.584 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:32:49.584 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 37.197836328000001, 2024-09-18T11:32:49.584 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015656389999999999, 2024-09-18T11:32:49.584 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:49.584 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:49.584 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:49.584 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:49.584 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.584 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4066 2024-09-18T11:32:49.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.584 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:49.584 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.585 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:49.585 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:32:49.585 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.585 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.585 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:49.585 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:32:49.585 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.585 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.585 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:49.585 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296977+0000", 2024-09-18T11:32:49.585 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:49.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.586 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.586 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:49.586 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296980+0000", 2024-09-18T11:32:49.586 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8830000000000002e-06 2024-09-18T11:32:49.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.586 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.586 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:49.586 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296985+0000", 2024-09-18T11:32:49.586 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:32:49.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.586 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.586 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.586 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296988+0000", 2024-09-18T11:32:49.587 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0419999999999998e-06 2024-09-18T11:32:49.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.587 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.587 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297120+0000", 2024-09-18T11:32:49.587 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131693 2024-09-18T11:32:49.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.587 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:49.587 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297125+0000", 2024-09-18T11:32:49.587 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1100000000000002e-06 2024-09-18T11:32:49.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.587 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.588 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310212+0000", 2024-09-18T11:32:49.588 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013087395999999999 2024-09-18T11:32:49.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.588 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.588 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:49.588 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310292+0000", 2024-09-18T11:32:49.588 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9980000000000003e-05 2024-09-18T11:32:49.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.588 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.588 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:49.588 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312634+0000", 2024-09-18T11:32:49.588 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002341756 2024-09-18T11:32:49.589 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.589 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:49.589 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.589 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4068 3.1 3:87e79ed0:::3526641c07d3efa989be9684355570f39c331008b8a02c9fa22b38a94b6c6b9f:head [write 0~1015808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:49.589 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:32:49.589 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 37.194060110999999, 2024-09-18T11:32:49.589 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016392648999999999, 2024-09-18T11:32:49.589 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:49.589 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.589 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:49.589 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:49.589 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.590 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4068 2024-09-18T11:32:49.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.590 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:49.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.590 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:49.590 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:32:49.590 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.590 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:49.590 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:32:49.590 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.591 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:49.591 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300753+0000", 2024-09-18T11:32:49.591 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:49.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.591 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.591 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:49.591 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301922+0000", 2024-09-18T11:32:49.591 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001169432 2024-09-18T11:32:49.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.591 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.591 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:49.591 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301936+0000", 2024-09-18T11:32:49.591 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3574000000000001e-05 2024-09-18T11:32:49.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.592 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.592 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.592 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301941+0000", 2024-09-18T11:32:49.592 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6069999999999998e-06 2024-09-18T11:32:49.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.592 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.592 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.592 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310234+0000", 2024-09-18T11:32:49.592 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082924820000000003 2024-09-18T11:32:49.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.592 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.592 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:49.592 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310337+0000", 2024-09-18T11:32:49.592 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102673 2024-09-18T11:32:49.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.593 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.593 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:49.593 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310469+0000", 2024-09-18T11:32:49.593 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131982 2024-09-18T11:32:49.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.593 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.593 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:49.593 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312844+0000", 2024-09-18T11:32:49.593 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023758680000000002 2024-09-18T11:32:49.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.593 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.593 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:49.594 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317092+0000", 2024-09-18T11:32:49.594 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042477549999999998 2024-09-18T11:32:49.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.594 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.594 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:49.594 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317124+0000", 2024-09-18T11:32:49.594 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1968999999999998e-05 2024-09-18T11:32:49.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.594 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.594 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:49.594 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317147+0000", 2024-09-18T11:32:49.594 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2339000000000001e-05 2024-09-18T11:32:49.594 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.594 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:49.595 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.595 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.595 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4671 4.1 4:87078dfe:::smithi11939506-14:head [rollback 6f] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:32:49.595 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:32:49.595 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.176097736, 2024-09-18T11:32:49.595 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013540905000000001, 2024-09-18T11:32:49.595 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:49.595 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.595 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:49.595 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:49.595 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.595 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4671 2024-09-18T11:32:49.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.596 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:49.596 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.596 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:49.596 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:32:49.596 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.596 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.596 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:49.596 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:32:49.596 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.596 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.596 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:49.597 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:32:49.597 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:49.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.597 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.597 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:49.597 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318718+0000", 2024-09-18T11:32:49.597 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3300000000000001e-06 2024-09-18T11:32:49.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.597 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.597 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:49.597 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318723+0000", 2024-09-18T11:32:49.597 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1590000000000001e-06 2024-09-18T11:32:49.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.598 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.598 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.598 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318727+0000", 2024-09-18T11:32:49.598 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3529999999999999e-06 2024-09-18T11:32:49.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.598 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.598 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.598 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.320735+0000", 2024-09-18T11:32:49.598 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020079730000000001 2024-09-18T11:32:49.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.598 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.598 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:49.598 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.320843+0000", 2024-09-18T11:32:49.598 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000108488 2024-09-18T11:32:49.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.599 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.599 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:49.599 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.329522+0000", 2024-09-18T11:32:49.599 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086787979999999997 2024-09-18T11:32:49.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.599 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.599 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:49.599 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331769+0000", 2024-09-18T11:32:49.599 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022471269999999998 2024-09-18T11:32:49.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.599 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.599 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:49.600 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332189+0000", 2024-09-18T11:32:49.600 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00041940400000000001 2024-09-18T11:32:49.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.600 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.600 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:49.600 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332228+0000", 2024-09-18T11:32:49.600 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9298000000000002e-05 2024-09-18T11:32:49.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.600 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.600 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:49.600 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332257+0000", 2024-09-18T11:32:49.600 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.934e-05 2024-09-18T11:32:49.600 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.600 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:49.601 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.601 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.601 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4672 4.1 4:87078dfe:::smithi11939506-14:head [append 0~0] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:32:49.601 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.318763+0000", 2024-09-18T11:32:49.601 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.1760513330000002, 2024-09-18T11:32:49.601 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016099992, 2024-09-18T11:32:49.601 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:49.601 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:49.601 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:49.601 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:49.601 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:49.601 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4672 2024-09-18T11:32:49.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.602 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:49.602 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.602 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:49.602 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318763+0000", 2024-09-18T11:32:49.602 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.602 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.602 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:49.602 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318763+0000", 2024-09-18T11:32:49.602 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:49.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.602 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.602 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:49.603 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318762+0000", 2024-09-18T11:32:49.603 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:49.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.603 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:49.603 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318765+0000", 2024-09-18T11:32:49.603 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.43e-06 2024-09-18T11:32:49.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.603 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:49.603 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318770+0000", 2024-09-18T11:32:49.603 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1479999999999997e-06 2024-09-18T11:32:49.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.604 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:49.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318773+0000", 2024-09-18T11:32:49.604 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.18e-06 2024-09-18T11:32:49.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.604 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.604 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.321043+0000", 2024-09-18T11:32:49.604 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022696819999999999 2024-09-18T11:32:49.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.604 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.604 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for blocked object", 2024-09-18T11:32:49.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.321104+0000", 2024-09-18T11:32:49.605 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1050999999999995e-05 2024-09-18T11:32:49.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.605 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:49.605 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331070+0000", 2024-09-18T11:32:49.605 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0099659610000000006 2024-09-18T11:32:49.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.605 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:49.605 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331176+0000", 2024-09-18T11:32:49.605 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010582 2024-09-18T11:32:49.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.605 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:49.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331377+0000", 2024-09-18T11:32:49.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000201208 2024-09-18T11:32:49.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.606 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:49.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332278+0000", 2024-09-18T11:32:49.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00090079999999999999 2024-09-18T11:32:49.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.606 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:49.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.334798+0000", 2024-09-18T11:32:49.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025200190000000001 2024-09-18T11:32:49.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.607 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:49.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.334833+0000", 2024-09-18T11:32:49.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5678e-05 2024-09-18T11:32:49.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:49.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:49.607 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:49.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.334863+0000", 2024-09-18T11:32:49.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9374e-05 2024-09-18T11:32:49.607 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.607 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:49.607 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.607 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:49.608 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:49.608 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:49.609 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:32:49.610 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:49.607+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:49.709 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:49.707+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:49.748 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:49.748 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:49.748 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:49.748 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:49.756 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:32:49.840 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:49.839+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:49.893 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:49.893 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:49.893 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:49.893 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:49.893 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:49.902 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:32:49.911 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:49.911+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:50.013 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:50.012+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:50.029 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is unset 2024-09-18T11:32:50.029 INFO:tasks.rados.rados.0.smithi119.stdout:2019: done (3 left) 2024-09-18T11:32:50.029 INFO:tasks.rados.rados.0.smithi119.stdout:2020: done (2 left) 2024-09-18T11:32:50.029 INFO:tasks.rados.rados.0.smithi119.stdout:2021: done (1 left) 2024-09-18T11:32:50.029 INFO:tasks.rados.rados.0.smithi119.stdout:2022: done (0 left) 2024-09-18T11:32:50.029 INFO:tasks.rados.rados.0.smithi119.stdout:2023: write (excl) oid 19 current snap is 70 2024-09-18T11:32:50.030 INFO:tasks.rados.rados.0.smithi119.stdout:2023: seq_num 416 ranges {591712=786646,1943568=668798} 2024-09-18T11:32:50.036 INFO:tasks.rados.rados.0.smithi119.stdout:2023: writing smithi11939506-19 from 591712 to 1378358 tid 1 2024-09-18T11:32:50.043 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:50.043 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:50.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:50.043 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:50.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.044 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1788 4.8 4:1898b15d:::smithi11939506-25:head [write 453822~797061] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:32:50.044 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:50.044 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 127.124978101, 2024-09-18T11:32:50.044 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094305159999999999, 2024-09-18T11:32:50.044 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.044 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:50.044 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.044 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.044 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.044 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1788 2024-09-18T11:32:50.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.045 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.045 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.045 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:50.045 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.045 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.045 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:50.045 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.045 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.046 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915811+0000", 2024-09-18T11:32:50.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:50.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.046 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.046 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916608+0000", 2024-09-18T11:32:50.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079635000000000003 2024-09-18T11:32:50.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.046 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.046 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916624+0000", 2024-09-18T11:32:50.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6735000000000002e-05 2024-09-18T11:32:50.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.047 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.047 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916630+0000", 2024-09-18T11:32:50.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5709999999999997e-06 2024-09-18T11:32:50.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.047 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.048 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916674+0000", 2024-09-18T11:32:50.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4540999999999999e-05 2024-09-18T11:32:50.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.048 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.048 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916751+0000", 2024-09-18T11:32:50.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6695999999999999e-05 2024-09-18T11:32:50.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.048 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:50.049 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916872+0000", 2024-09-18T11:32:50.049 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120648 2024-09-18T11:32:50.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.049 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:50.049 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.919076+0000", 2024-09-18T11:32:50.049 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022046829999999998 2024-09-18T11:32:50.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.050 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:50.050 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925151+0000", 2024-09-18T11:32:50.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060740239999999999 2024-09-18T11:32:50.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.050 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:50.050 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925198+0000", 2024-09-18T11:32:50.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7896999999999999e-05 2024-09-18T11:32:50.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.051 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.051 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925244+0000", 2024-09-18T11:32:50.051 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5398999999999997e-05 2024-09-18T11:32:50.051 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.051 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.051 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.052 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2878 4.3 4:cc46532a:::smithi11939506-33:2f [sparse-read 0~1770976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-18T11:32:50.052 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:50.052 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 87.53386519, 2024-09-18T11:32:50.052 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090602640000000002, 2024-09-18T11:32:50.052 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.053 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:50.053 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.053 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.053 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.053 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2878 2024-09-18T11:32:50.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.054 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.054 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.054 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:50.054 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.055 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:50.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.055 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506926+0000", 2024-09-18T11:32:50.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:50.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.056 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.056 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506929+0000", 2024-09-18T11:32:50.056 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.163e-06 2024-09-18T11:32:50.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.057 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.057 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506936+0000", 2024-09-18T11:32:50.057 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5519999999999999e-06 2024-09-18T11:32:50.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.057 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.057 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.506941+0000", 2024-09-18T11:32:50.057 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6550000000000003e-06 2024-09-18T11:32:50.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.057 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.058 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507036+0000", 2024-09-18T11:32:50.058 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5070999999999998e-05 2024-09-18T11:32:50.058 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.058 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:50.058 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.507042+0000", 2024-09-18T11:32:50.058 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3959999999999997e-06 2024-09-18T11:32:50.058 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.058 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.058 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515504+0000", 2024-09-18T11:32:50.059 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084620300000000006 2024-09-18T11:32:50.059 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.059 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.059 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.059 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515625+0000", 2024-09-18T11:32:50.059 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012150999999999999 2024-09-18T11:32:50.060 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.060 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.060 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.060 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:22.515986+0000", 2024-09-18T11:32:50.060 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00036140100000000002 2024-09-18T11:32:50.061 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.061 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.061 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.061 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.061 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.062 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3963 4.3 4:c70c3162:::smithi11939506-35:head [tier-promote] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:50.062 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:32:50.062 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.664968180000002, 2024-09-18T11:32:50.062 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010675591999999999, 2024-09-18T11:32:50.062 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.062 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:50.063 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.063 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.063 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.063 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3963 2024-09-18T11:32:50.063 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.064 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.064 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.064 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:32:50.064 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.065 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.065 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.065 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:32:50.065 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.066 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.066 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375821+0000", 2024-09-18T11:32:50.066 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:50.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.067 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.067 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375829+0000", 2024-09-18T11:32:50.067 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7989999999999994e-06 2024-09-18T11:32:50.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.067 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.068 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375843+0000", 2024-09-18T11:32:50.068 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:32:50.068 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.068 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.068 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.068 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375849+0000", 2024-09-18T11:32:50.068 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1419999999999998e-06 2024-09-18T11:32:50.068 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.069 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.069 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.069 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375889+0000", 2024-09-18T11:32:50.069 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0197999999999997e-05 2024-09-18T11:32:50.069 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.069 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.069 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:50.069 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375894+0000", 2024-09-18T11:32:50.069 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0139999999999998e-06 2024-09-18T11:32:50.069 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.069 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.069 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.070 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.377699+0000", 2024-09-18T11:32:50.070 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001804688 2024-09-18T11:32:50.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.070 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.070 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.377776+0000", 2024-09-18T11:32:50.070 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7081000000000003e-05 2024-09-18T11:32:50.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.070 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.070 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.386499+0000", 2024-09-18T11:32:50.070 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008722746 2024-09-18T11:32:50.071 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.071 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.071 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.071 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.071 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1025 4.7 e138/22 4:e73ba68f:::smithi11939506-1:head v 138'281, mlcod=137'278)", 2024-09-18T11:32:50.071 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.587274+0000", 2024-09-18T11:32:50.071 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.453517865999999, 2024-09-18T11:32:50.071 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094231360000000004, 2024-09-18T11:32:50.071 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.071 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:50.071 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.072 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.072 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.072 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.587274+0000", 2024-09-18T11:32:50.072 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.072 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.072 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.072 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.072 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.587274+0000", 2024-09-18T11:32:50.072 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.073 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.073 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.073 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.587273+0000", 2024-09-18T11:32:50.073 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:50.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.074 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.074 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590258+0000", 2024-09-18T11:32:50.074 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002984998 2024-09-18T11:32:50.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.074 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.074 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590273+0000", 2024-09-18T11:32:50.074 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4999e-05 2024-09-18T11:32:50.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.074 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.074 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590283+0000", 2024-09-18T11:32:50.075 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3619999999999995e-06 2024-09-18T11:32:50.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.075 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.075 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590333+0000", 2024-09-18T11:32:50.075 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9978999999999997e-05 2024-09-18T11:32:50.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.075 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.075 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590352+0000", 2024-09-18T11:32:50.075 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9477999999999999e-05 2024-09-18T11:32:50.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.076 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:50.076 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596664+0000", 2024-09-18T11:32:50.076 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063120679999999997 2024-09-18T11:32:50.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.076 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.076 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596697+0000", 2024-09-18T11:32:50.076 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2456999999999998e-05 2024-09-18T11:32:50.076 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.076 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.076 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.077 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4259 4.b 4:dba5bd39:::smithi11939506-17:head [write 685818~644423] snapc 69=[69,67,61,55,51,4e,4b,45,42,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:50.077 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.040083+0000", 2024-09-18T11:32:50.077 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.000708001, 2024-09-18T11:32:50.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015748635, 2024-09-18T11:32:50.077 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.077 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:50.077 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.077 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.077 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.077 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4259 2024-09-18T11:32:50.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.077 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.040083+0000", 2024-09-18T11:32:50.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.040083+0000", 2024-09-18T11:32:50.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.040081+0000", 2024-09-18T11:32:50.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:50.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.079 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041258+0000", 2024-09-18T11:32:50.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011766540000000001 2024-09-18T11:32:50.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.079 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041286+0000", 2024-09-18T11:32:50.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8087000000000001e-05 2024-09-18T11:32:50.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.079 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041362+0000", 2024-09-18T11:32:50.080 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6155999999999994e-05 2024-09-18T11:32:50.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.080 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.044285+0000", 2024-09-18T11:32:50.080 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002922437 2024-09-18T11:32:50.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.080 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.044360+0000", 2024-09-18T11:32:50.080 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5201e-05 2024-09-18T11:32:50.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.081 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:50.081 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.044489+0000", 2024-09-18T11:32:50.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012950599999999999 2024-09-18T11:32:50.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.081 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:50.081 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052904+0000", 2024-09-18T11:32:50.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084149910000000001 2024-09-18T11:32:50.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.081 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:50.081 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055695+0000", 2024-09-18T11:32:50.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027902040000000001 2024-09-18T11:32:50.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.082 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:50.082 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055765+0000", 2024-09-18T11:32:50.082 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0720000000000001e-05 2024-09-18T11:32:50.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.082 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.082 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055832+0000", 2024-09-18T11:32:50.082 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6736999999999995e-05 2024-09-18T11:32:50.082 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.083 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.083 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.083 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.5.0:195 4.b e156/152 ondisk, result = 0)", 2024-09-18T11:32:50.083 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.041863+0000", 2024-09-18T11:32:50.083 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.998928481, 2024-09-18T11:32:50.083 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010991236, 2024-09-18T11:32:50.083 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.083 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:50.083 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.083 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.083 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041863+0000", 2024-09-18T11:32:50.084 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.084 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.084 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041863+0000", 2024-09-18T11:32:50.084 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.084 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.084 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041862+0000", 2024-09-18T11:32:50.084 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:50.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041868+0000", 2024-09-18T11:32:50.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2979999999999997e-06 2024-09-18T11:32:50.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041879+0000", 2024-09-18T11:32:50.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0522e-05 2024-09-18T11:32:50.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041885+0000", 2024-09-18T11:32:50.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7830000000000004e-06 2024-09-18T11:32:50.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.086 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.086 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052841+0000", 2024-09-18T11:32:50.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010956604 2024-09-18T11:32:50.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.086 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.086 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052848+0000", 2024-09-18T11:32:50.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2249999999999997e-06 2024-09-18T11:32:50.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.086 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.087 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052854+0000", 2024-09-18T11:32:50.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6050000000000003e-06 2024-09-18T11:32:50.087 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.087 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.087 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.087 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.5.0:196 4.b e156/152 ondisk, result = 0)", 2024-09-18T11:32:50.087 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.042994+0000", 2024-09-18T11:32:50.087 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.997797818999999, 2024-09-18T11:32:50.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098926020000000003, 2024-09-18T11:32:50.087 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.087 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:50.087 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.088 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.088 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.042994+0000", 2024-09-18T11:32:50.088 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.088 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.088 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.042994+0000", 2024-09-18T11:32:50.088 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.088 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.088 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.042993+0000", 2024-09-18T11:32:50.089 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:50.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.089 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.089 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043000+0000", 2024-09-18T11:32:50.089 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.525e-06 2024-09-18T11:32:50.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.089 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.089 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043011+0000", 2024-09-18T11:32:50.089 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0335e-05 2024-09-18T11:32:50.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043015+0000", 2024-09-18T11:32:50.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2899999999999996e-06 2024-09-18T11:32:50.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052868+0000", 2024-09-18T11:32:50.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098528130000000002 2024-09-18T11:32:50.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052871+0000", 2024-09-18T11:32:50.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5080000000000001e-06 2024-09-18T11:32:50.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052886+0000", 2024-09-18T11:32:50.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4781e-05 2024-09-18T11:32:50.091 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.091 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.091 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.092 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.5.0:194 4.3 e156/67 ondisk, result = 0)", 2024-09-18T11:32:50.092 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.043892+0000", 2024-09-18T11:32:50.092 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.996899029000001, 2024-09-18T11:32:50.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010237263999999999, 2024-09-18T11:32:50.092 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.092 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:50.092 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.092 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.092 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043892+0000", 2024-09-18T11:32:50.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.093 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.093 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043892+0000", 2024-09-18T11:32:50.093 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.093 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.093 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043891+0000", 2024-09-18T11:32:50.093 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:50.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.093 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.093 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043898+0000", 2024-09-18T11:32:50.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1190000000000002e-06 2024-09-18T11:32:50.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.094 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.094 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043910+0000", 2024-09-18T11:32:50.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.29e-05 2024-09-18T11:32:50.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.094 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.094 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043916+0000", 2024-09-18T11:32:50.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9499999999999998e-06 2024-09-18T11:32:50.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.095 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.095 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052830+0000", 2024-09-18T11:32:50.095 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089137480000000009 2024-09-18T11:32:50.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.095 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.095 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052839+0000", 2024-09-18T11:32:50.095 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9560000000000003e-06 2024-09-18T11:32:50.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.095 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.095 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054130+0000", 2024-09-18T11:32:50.095 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012905639999999999 2024-09-18T11:32:50.096 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.096 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.096 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.096 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4260 4.b 4:dba5bd39:::smithi11939506-17:head [write 2069089~216729] snapc 69=[69,67,61,55,51,4e,4b,45,42,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:50.096 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045444+0000", 2024-09-18T11:32:50.096 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.995347506999998, 2024-09-18T11:32:50.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011166715000000001, 2024-09-18T11:32:50.096 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.096 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:50.097 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.097 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.097 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.097 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4260 2024-09-18T11:32:50.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.097 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.097 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045444+0000", 2024-09-18T11:32:50.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045444+0000", 2024-09-18T11:32:50.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.098 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045443+0000", 2024-09-18T11:32:50.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:50.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.098 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045649+0000", 2024-09-18T11:32:50.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020605499999999999 2024-09-18T11:32:50.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045662+0000", 2024-09-18T11:32:50.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3386e-05 2024-09-18T11:32:50.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045667+0000", 2024-09-18T11:32:50.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6750000000000001e-06 2024-09-18T11:32:50.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.099 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.099 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052921+0000", 2024-09-18T11:32:50.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007254613 2024-09-18T11:32:50.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052994+0000", 2024-09-18T11:32:50.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2721999999999999e-05 2024-09-18T11:32:50.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:50.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.053116+0000", 2024-09-18T11:32:50.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121972 2024-09-18T11:32:50.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.101 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:50.101 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055857+0000", 2024-09-18T11:32:50.101 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027414539999999999 2024-09-18T11:32:50.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.101 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:50.101 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056502+0000", 2024-09-18T11:32:50.101 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064412499999999995 2024-09-18T11:32:50.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.101 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:50.101 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056555+0000", 2024-09-18T11:32:50.101 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3467999999999997e-05 2024-09-18T11:32:50.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.102 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056611+0000", 2024-09-18T11:32:50.102 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5577e-05 2024-09-18T11:32:50.102 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.102 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.102 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.103 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4261 4.b 4:dba5bd39:::smithi11939506-17:head [setxattr _header (57) in=7b,truncate 2285818] snapc 69=[69,67,61,55,51,4e,4b,45,42,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:50.103 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045672+0000", 2024-09-18T11:32:50.103 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.995119409000001, 2024-09-18T11:32:50.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011289933, 2024-09-18T11:32:50.103 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.103 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:50.103 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.103 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.103 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.103 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4261 2024-09-18T11:32:50.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.103 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045672+0000", 2024-09-18T11:32:50.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045672+0000", 2024-09-18T11:32:50.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045671+0000", 2024-09-18T11:32:50.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:50.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.105 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.105 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045675+0000", 2024-09-18T11:32:50.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.563e-06 2024-09-18T11:32:50.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.105 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.105 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045681+0000", 2024-09-18T11:32:50.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5210000000000002e-06 2024-09-18T11:32:50.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045684+0000", 2024-09-18T11:32:50.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4149999999999999e-06 2024-09-18T11:32:50.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054143+0000", 2024-09-18T11:32:50.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084590810000000002 2024-09-18T11:32:50.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054214+0000", 2024-09-18T11:32:50.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1061000000000003e-05 2024-09-18T11:32:50.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:50.107 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054354+0000", 2024-09-18T11:32:50.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000140035 2024-09-18T11:32:50.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:50.107 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056208+0000", 2024-09-18T11:32:50.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018539229999999999 2024-09-18T11:32:50.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:50.107 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056885+0000", 2024-09-18T11:32:50.108 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00067688000000000004 2024-09-18T11:32:50.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.108 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:50.108 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056922+0000", 2024-09-18T11:32:50.108 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6693999999999998e-05 2024-09-18T11:32:50.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.108 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.108 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056962+0000", 2024-09-18T11:32:50.108 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0299000000000002e-05 2024-09-18T11:32:50.108 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.109 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.109 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.109 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4262 4.b 4:dba5bd39:::smithi11939506-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:50.109 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:32:50.109 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.995103267000001, 2024-09-18T11:32:50.109 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011764513000000001, 2024-09-18T11:32:50.109 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.109 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:50.109 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.109 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.109 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.109 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4262 2024-09-18T11:32:50.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.110 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.110 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.110 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:32:50.110 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.110 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.110 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:32:50.110 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.111 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.111 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:32:50.111 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:50.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.111 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.111 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045690+0000", 2024-09-18T11:32:50.111 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.745e-06 2024-09-18T11:32:50.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.112 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.112 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045697+0000", 2024-09-18T11:32:50.112 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7229999999999998e-06 2024-09-18T11:32:50.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.113 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.113 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045704+0000", 2024-09-18T11:32:50.113 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5719999999999999e-06 2024-09-18T11:32:50.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.113 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.113 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055555+0000", 2024-09-18T11:32:50.113 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098515430000000008 2024-09-18T11:32:50.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.114 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:50.114 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055675+0000", 2024-09-18T11:32:50.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012000799999999999 2024-09-18T11:32:50.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.114 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.114 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.057004+0000", 2024-09-18T11:32:50.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013286179999999999 2024-09-18T11:32:50.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.057357+0000", 2024-09-18T11:32:50.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00035355100000000002 2024-09-18T11:32:50.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.057453+0000", 2024-09-18T11:32:50.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5154000000000004e-05 2024-09-18T11:32:50.115 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.115 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.116 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.116 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4264 4.7 4:e0d95e10:::smithi11939506-22:head [read 0~3008486,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:50.116 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:32:50.116 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.99505328, 2024-09-18T11:32:50.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095333889999999998, 2024-09-18T11:32:50.116 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.116 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:50.116 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.116 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.116 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.117 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4264 2024-09-18T11:32:50.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.117 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:32:50.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:32:50.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.118 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:32:50.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:50.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.118 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045747+0000", 2024-09-18T11:32:50.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2469999999999997e-06 2024-09-18T11:32:50.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.118 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.119 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045755+0000", 2024-09-18T11:32:50.119 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1499999999999999e-06 2024-09-18T11:32:50.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.119 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.119 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045759+0000", 2024-09-18T11:32:50.119 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0849999999999999e-06 2024-09-18T11:32:50.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.119 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045803+0000", 2024-09-18T11:32:50.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3621000000000001e-05 2024-09-18T11:32:50.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:32:50.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045807+0000", 2024-09-18T11:32:50.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7290000000000002e-06 2024-09-18T11:32:50.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.053475+0000", 2024-09-18T11:32:50.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076680100000000003 2024-09-18T11:32:50.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.053644+0000", 2024-09-18T11:32:50.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016971299999999999 2024-09-18T11:32:50.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055272+0000", 2024-09-18T11:32:50.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001627245 2024-09-18T11:32:50.121 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.122 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.122 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.122 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4265 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~2163506] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:50.122 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045792+0000", 2024-09-18T11:32:50.122 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.99499939, 2024-09-18T11:32:50.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010085024, 2024-09-18T11:32:50.122 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.122 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:50.122 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.122 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.122 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.123 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4265 2024-09-18T11:32:50.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.123 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045792+0000", 2024-09-18T11:32:50.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045792+0000", 2024-09-18T11:32:50.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.124 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.124 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045791+0000", 2024-09-18T11:32:50.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:50.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.124 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.124 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045797+0000", 2024-09-18T11:32:50.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0209999999999996e-06 2024-09-18T11:32:50.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.125 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.125 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045805+0000", 2024-09-18T11:32:50.125 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9349999999999994e-06 2024-09-18T11:32:50.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.125 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.125 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045811+0000", 2024-09-18T11:32:50.125 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2240000000000003e-06 2024-09-18T11:32:50.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.125 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.126 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045858+0000", 2024-09-18T11:32:50.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7565999999999998e-05 2024-09-18T11:32:50.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.126 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:32:50.126 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045869+0000", 2024-09-18T11:32:50.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0852e-05 2024-09-18T11:32:50.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.126 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.126 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055337+0000", 2024-09-18T11:32:50.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094678729999999999 2024-09-18T11:32:50.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.127 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055417+0000", 2024-09-18T11:32:50.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0092000000000005e-05 2024-09-18T11:32:50.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.127 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055877+0000", 2024-09-18T11:32:50.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00046001800000000003 2024-09-18T11:32:50.127 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.127 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.127 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4349 4.b 4:d13decd8:::smithi11939506-2:head [read 0~2302625,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.382910+0000", 2024-09-18T11:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.657881326000002, 2024-09-18T11:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098226930000000004, 2024-09-18T11:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4349 2024-09-18T11:32:50.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.129 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382910+0000", 2024-09-18T11:32:50.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382910+0000", 2024-09-18T11:32:50.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382909+0000", 2024-09-18T11:32:50.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:50.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.130 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382915+0000", 2024-09-18T11:32:50.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6049999999999997e-06 2024-09-18T11:32:50.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.130 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382924+0000", 2024-09-18T11:32:50.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6289999999999999e-06 2024-09-18T11:32:50.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.131 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.131 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382929+0000", 2024-09-18T11:32:50.131 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0479999999999998e-06 2024-09-18T11:32:50.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.131 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.131 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383030+0000", 2024-09-18T11:32:50.131 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101346 2024-09-18T11:32:50.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.131 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:50.131 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383036+0000", 2024-09-18T11:32:50.131 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2720000000000003e-06 2024-09-18T11:32:50.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.132 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.132 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.390597+0000", 2024-09-18T11:32:50.132 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075606880000000003 2024-09-18T11:32:50.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.132 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.132 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.390696+0000", 2024-09-18T11:32:50.132 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8740000000000007e-05 2024-09-18T11:32:50.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.132 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.133 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392733+0000", 2024-09-18T11:32:50.133 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002037025 2024-09-18T11:32:50.133 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.133 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.133 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.133 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.133 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4350 4.b 4:d13decd8:::smithi11939506-2:head [sparse-read 0~1463453] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:32:50.133 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.382978+0000", 2024-09-18T11:32:50.133 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.657813763, 2024-09-18T11:32:50.133 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01020132, 2024-09-18T11:32:50.133 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.133 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:50.134 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.134 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.134 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.134 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4350 2024-09-18T11:32:50.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.134 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382978+0000", 2024-09-18T11:32:50.134 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382978+0000", 2024-09-18T11:32:50.134 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.135 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.135 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382976+0000", 2024-09-18T11:32:50.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:50.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.135 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.135 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382984+0000", 2024-09-18T11:32:50.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.328e-06 2024-09-18T11:32:50.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.135 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.136 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382994+0000", 2024-09-18T11:32:50.136 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0801999999999999e-05 2024-09-18T11:32:50.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.136 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.136 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383001+0000", 2024-09-18T11:32:50.136 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6390000000000003e-06 2024-09-18T11:32:50.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.136 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.136 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383138+0000", 2024-09-18T11:32:50.136 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013676899999999999 2024-09-18T11:32:50.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.137 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.137 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:50.137 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383144+0000", 2024-09-18T11:32:50.137 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6359999999999997e-06 2024-09-18T11:32:50.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.137 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.137 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.137 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392791+0000", 2024-09-18T11:32:50.137 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096461659999999994 2024-09-18T11:32:50.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.137 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.138 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392897+0000", 2024-09-18T11:32:50.138 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000106087 2024-09-18T11:32:50.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.138 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.393179+0000", 2024-09-18T11:32:50.138 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00028227500000000002 2024-09-18T11:32:50.138 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.138 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.138 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.139 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4375 4.b 4:dba5bd39:::smithi11939506-17:head [write 437081~638017,stat] snapc 6d=[6d,6c,69,67,61,55,51,4b,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:32:50.139 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.549471+0000", 2024-09-18T11:32:50.139 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.491320061, 2024-09-18T11:32:50.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097774179999999995, 2024-09-18T11:32:50.139 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.139 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:50.139 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.139 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.139 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.139 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4375 2024-09-18T11:32:50.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.139 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.140 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.140 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.549471+0000", 2024-09-18T11:32:50.140 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.140 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.140 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.549471+0000", 2024-09-18T11:32:50.140 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.140 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.140 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.549470+0000", 2024-09-18T11:32:50.140 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:50.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.141 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.141 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553463+0000", 2024-09-18T11:32:50.141 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039932520000000001 2024-09-18T11:32:50.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.141 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.141 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553478+0000", 2024-09-18T11:32:50.141 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4647999999999999e-05 2024-09-18T11:32:50.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.141 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553484+0000", 2024-09-18T11:32:50.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3799999999999999e-06 2024-09-18T11:32:50.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.142 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553536+0000", 2024-09-18T11:32:50.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1938999999999999e-05 2024-09-18T11:32:50.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.142 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553637+0000", 2024-09-18T11:32:50.143 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010074899999999999 2024-09-18T11:32:50.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.143 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:50.143 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553824+0000", 2024-09-18T11:32:50.143 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000187524 2024-09-18T11:32:50.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.143 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:50.143 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.556653+0000", 2024-09-18T11:32:50.143 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028285879999999999 2024-09-18T11:32:50.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.144 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:50.144 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.559176+0000", 2024-09-18T11:32:50.144 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025229369999999998 2024-09-18T11:32:50.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.144 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:50.144 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.559212+0000", 2024-09-18T11:32:50.144 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5750000000000002e-05 2024-09-18T11:32:50.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.144 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.144 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.559249+0000", 2024-09-18T11:32:50.145 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7020999999999997e-05 2024-09-18T11:32:50.145 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.145 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.145 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.145 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4430 4.b 4:d3ae8215:::smithi11939506-41:head [read 0~3832029,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T11:32:50.145 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:32:50.145 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 15.012281099999999, 2024-09-18T11:32:50.145 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010918034, 2024-09-18T11:32:50.145 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.145 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:50.145 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.145 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.146 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.146 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4430 2024-09-18T11:32:50.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.146 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.146 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.146 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.146 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:32:50.146 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.146 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.146 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.146 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:32:50.146 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.147 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.147 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:32:50.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:50.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.147 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.147 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028520+0000", 2024-09-18T11:32:50.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.005e-05 2024-09-18T11:32:50.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028538+0000", 2024-09-18T11:32:50.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8020999999999999e-05 2024-09-18T11:32:50.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028543+0000", 2024-09-18T11:32:50.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.485e-06 2024-09-18T11:32:50.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.149 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028634+0000", 2024-09-18T11:32:50.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1120999999999997e-05 2024-09-18T11:32:50.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.149 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:50.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028639+0000", 2024-09-18T11:32:50.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1989999999999997e-06 2024-09-18T11:32:50.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.149 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.036054+0000", 2024-09-18T11:32:50.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074147559999999998 2024-09-18T11:32:50.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.150 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.150 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.150 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.036158+0000", 2024-09-18T11:32:50.150 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103564 2024-09-18T11:32:50.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.150 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.150 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.150 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039428+0000", 2024-09-18T11:32:50.151 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032706390000000001 2024-09-18T11:32:50.151 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.151 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.151 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.151 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.151 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4431 4.b 4:d3ae8215:::smithi11939506-41:head [sparse-read 0~2481209] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T11:32:50.151 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:32:50.151 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 15.012242312, 2024-09-18T11:32:50.151 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011399179000000001, 2024-09-18T11:32:50.151 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.151 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:50.151 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.152 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.152 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.152 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4431 2024-09-18T11:32:50.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.152 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.152 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.152 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.152 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:32:50.152 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.152 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.152 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.152 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:32:50.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.153 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:32:50.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:50.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.153 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028552+0000", 2024-09-18T11:32:50.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5880000000000002e-06 2024-09-18T11:32:50.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028558+0000", 2024-09-18T11:32:50.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.84e-06 2024-09-18T11:32:50.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028562+0000", 2024-09-18T11:32:50.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6600000000000001e-06 2024-09-18T11:32:50.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.155 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028676+0000", 2024-09-18T11:32:50.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011408000000000001 2024-09-18T11:32:50.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.155 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:50.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028681+0000", 2024-09-18T11:32:50.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5350000000000004e-06 2024-09-18T11:32:50.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.155 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039458+0000", 2024-09-18T11:32:50.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010777231 2024-09-18T11:32:50.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.156 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039564+0000", 2024-09-18T11:32:50.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000105524 2024-09-18T11:32:50.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.156 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039948+0000", 2024-09-18T11:32:50.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00038430099999999998 2024-09-18T11:32:50.156 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.156 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.157 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.157 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4518 4.7 4:e0d95e10:::smithi11939506-22:head [rollback 6f] snapc 74=[74,73,71,6f,6c,69,67,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:32:50.157 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.055692+0000", 2024-09-18T11:32:50.157 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.9850990510000006, 2024-09-18T11:32:50.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010093540999999999, 2024-09-18T11:32:50.157 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.157 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:50.157 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.157 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.157 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.157 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4518 2024-09-18T11:32:50.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.158 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.158 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.158 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055692+0000", 2024-09-18T11:32:50.158 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.158 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.158 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055692+0000", 2024-09-18T11:32:50.158 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.159 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055692+0000", 2024-09-18T11:32:50.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:50.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.159 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055697+0000", 2024-09-18T11:32:50.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.101e-06 2024-09-18T11:32:50.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.160 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.160 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055707+0000", 2024-09-18T11:32:50.160 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0316000000000001e-05 2024-09-18T11:32:50.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.160 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.160 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055713+0000", 2024-09-18T11:32:50.160 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8900000000000004e-06 2024-09-18T11:32:50.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.160 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.160 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057453+0000", 2024-09-18T11:32:50.160 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001739953 2024-09-18T11:32:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057537+0000", 2024-09-18T11:32:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4287000000000001e-05 2024-09-18T11:32:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057760+0000", 2024-09-18T11:32:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022294700000000001 2024-09-18T11:32:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065674+0000", 2024-09-18T11:32:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079140110000000003 2024-09-18T11:32:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065681+0000", 2024-09-18T11:32:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2289999999999997e-06 2024-09-18T11:32:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065734+0000", 2024-09-18T11:32:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3136000000000001e-05 2024-09-18T11:32:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065786+0000", 2024-09-18T11:32:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1419999999999999e-05 2024-09-18T11:32:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4519 4.7 4:e0d95e10:::smithi11939506-22:head [append 0~0] snapc 74=[74,73,71,6f,6c,69,67,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:32:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.055719+0000", 2024-09-18T11:32:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 9.9850728889999996, 2024-09-18T11:32:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010210589000000001, 2024-09-18T11:32:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4519 2024-09-18T11:32:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055719+0000", 2024-09-18T11:32:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055719+0000", 2024-09-18T11:32:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055718+0000", 2024-09-18T11:32:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055721+0000", 2024-09-18T11:32:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7039999999999999e-06 2024-09-18T11:32:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055728+0000", 2024-09-18T11:32:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1620000000000002e-06 2024-09-18T11:32:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055732+0000", 2024-09-18T11:32:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8410000000000002e-06 2024-09-18T11:32:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.059646+0000", 2024-09-18T11:32:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039138489999999996 2024-09-18T11:32:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.059733+0000", 2024-09-18T11:32:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7163999999999994e-05 2024-09-18T11:32:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.059891+0000", 2024-09-18T11:32:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000158102 2024-09-18T11:32:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065798+0000", 2024-09-18T11:32:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059067649999999996 2024-09-18T11:32:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065840+0000", 2024-09-18T11:32:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2515999999999997e-05 2024-09-18T11:32:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065890+0000", 2024-09-18T11:32:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0055000000000001e-05 2024-09-18T11:32:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065929+0000", 2024-09-18T11:32:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8831000000000001e-05 2024-09-18T11:32:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.169 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:50.172 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:32:50.172 INFO:tasks.rados.rados.0.smithi119.stdout:2023: writing smithi11939506-19 from 1943568 to 2612366 tid 2 2024-09-18T11:32:50.173 INFO:tasks.rados.rados.0.smithi119.stdout:2024: read oid 47 snap 60 2024-09-18T11:32:50.173 INFO:tasks.rados.rados.0.smithi119.stdout:2024: expect (ObjNum 380 snap 60 seq_num 380) 2024-09-18T11:32:50.173 INFO:tasks.rados.rados.0.smithi119.stdout:2025: set_chunk oid 29 target oid 2024-09-18T11:32:50.173 INFO:tasks.rados.rados.0.smithi119.stdout:2025: set_chunk oid 29 offset: 238592 length: 2814976 target oid 259b372f30a46a47845d5d9c58303dcbeccaeb2db18e8ed0152f1c66341c4df2 offset: 0 snap 60 2024-09-18T11:32:50.173 INFO:tasks.rados.rados.0.smithi119.stdout:2026: tier_promote oid 34 2024-09-18T11:32:50.173 INFO:tasks.rados.rados.0.smithi119.stdout:2023: finishing write tid 1 to smithi11939506-19 2024-09-18T11:32:50.173 INFO:tasks.rados.rados.0.smithi119.stdout:2023: finishing write tid 2 to smithi11939506-19 2024-09-18T11:32:50.173 INFO:tasks.rados.rados.0.smithi119.stdout:2023: finishing write tid 3 to smithi11939506-19 2024-09-18T11:32:50.173 INFO:tasks.rados.rados.0.smithi119.stdout:2023: finishing write tid 4 to smithi11939506-19 2024-09-18T11:32:50.173 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 19 v 407 (ObjNum 416 snap 70 seq_num 416) dirty exists 2024-09-18T11:32:50.173 INFO:tasks.rados.rados.0.smithi119.stdout:2023: left oid 19 (ObjNum 416 snap 70 seq_num 416) 2024-09-18T11:32:50.173 INFO:tasks.rados.rados.0.smithi119.stdout:2024: expect (ObjNum 380 snap 60 seq_num 380) 2024-09-18T11:32:50.174 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:50.169+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:50.213 INFO:tasks.rados.rados.0.smithi119.stdout:2025: finishing set_chunk to oid 29 2024-09-18T11:32:50.213 INFO:tasks.rados.rados.0.smithi119.stdout:2026: got (0) Success 2024-09-18T11:32:50.213 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 337 (ObjNum 329 snap 48 seq_num 329) dirty exists 2024-09-18T11:32:50.213 INFO:tasks.rados.rados.0.smithi119.stdout:2023: done (3 left) 2024-09-18T11:32:50.213 INFO:tasks.rados.rados.0.smithi119.stdout:2024: done (2 left) 2024-09-18T11:32:50.214 INFO:tasks.rados.rados.0.smithi119.stdout:2025: done (1 left) 2024-09-18T11:32:50.214 INFO:tasks.rados.rados.0.smithi119.stdout:2026: done (0 left) 2024-09-18T11:32:50.214 INFO:tasks.rados.rados.0.smithi119.stdout:2027: rollback oid 43 current snap is 70 2024-09-18T11:32:50.214 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 43 to 62 2024-09-18T11:32:50.214 INFO:tasks.rados.rados.0.smithi119.stdout:2028: read oid 46 snap -1 2024-09-18T11:32:50.214 INFO:tasks.rados.rados.0.smithi119.stdout:2028: expect (ObjNum 320 snap 44 seq_num 320) 2024-09-18T11:32:50.214 INFO:tasks.rados.rados.0.smithi119.stdout:2029: read oid 7 snap -1 2024-09-18T11:32:50.214 INFO:tasks.rados.rados.0.smithi119.stdout:2029: expect (ObjNum 384 snap 60 seq_num 384) 2024-09-18T11:32:50.214 INFO:tasks.rados.rados.0.smithi119.stdout:2030: delete oid 13 current snap is 70 2024-09-18T11:32:50.216 INFO:tasks.rados.rados.0.smithi119.stdout:2030: done (3 left) 2024-09-18T11:32:50.216 INFO:tasks.rados.rados.0.smithi119.stdout:2031: read oid 3 snap -1 2024-09-18T11:32:50.216 INFO:tasks.rados.rados.0.smithi119.stdout:2031: expect (ObjNum 243 snap 28 seq_num 243) 2024-09-18T11:32:50.216 INFO:tasks.rados.rados.0.smithi119.stdout:2032: snap_create 2024-09-18T11:32:50.221 INFO:tasks.rados.rados.0.smithi119.stdout:2029: expect (ObjNum 384 snap 60 seq_num 384) 2024-09-18T11:32:50.273 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:50.271+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:50.276 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:50.276 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:50.276 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:50.276 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:50.284 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:32:50.372 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:32:50.373 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:50.371+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:50.382 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:50.382 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:50.383 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:50.383 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:50.383 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:50.391 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:32:50.404 INFO:tasks.rados.rados.0.smithi119.stdout:2031: expect (ObjNum 243 snap 28 seq_num 243) 2024-09-18T11:32:50.474 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:50.472+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:50.492 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:50.492 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:50.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:50.492 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:50.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.492 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:32:50.492 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:50.492 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 159.37336620900001, 2024-09-18T11:32:50.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:32:50.493 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.493 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:50.493 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.493 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.493 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.493 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:32:50.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.493 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.493 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:50.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:50.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:32:50.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:50.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:32:50.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:32:50.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:32:50.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:32:50.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:32:50.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:32:50.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:32:50.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:32:50.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:50.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:32:50.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:32:50.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:32:50.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:32:50.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:32:50.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:32:50.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:32:50.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:32:50.498 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.498 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.498 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.498 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:32:50.498 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:50.498 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 149.820340808, 2024-09-18T11:32:50.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:32:50.498 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.499 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:50.499 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.499 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.499 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.499 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:32:50.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.499 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:50.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:50.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:32:50.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:50.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:32:50.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:32:50.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:32:50.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:32:50.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:32:50.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:32:50.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:32:50.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:32:50.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:32:50.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:32:50.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:32:50.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:32:50.503 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.503 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.503 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.503 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2339 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:50.503 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:50.503 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 109.10288351600001, 2024-09-18T11:32:50.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011809992, 2024-09-18T11:32:50.503 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.504 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:50.504 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.504 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.504 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.504 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2339 2024-09-18T11:32:50.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.504 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:50.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:50.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:32:50.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:50.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:32:50.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4710000000000001e-06 2024-09-18T11:32:50.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386824+0000", 2024-09-18T11:32:50.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6104e-05 2024-09-18T11:32:50.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386836+0000", 2024-09-18T11:32:50.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1827999999999999e-05 2024-09-18T11:32:50.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388390+0000", 2024-09-18T11:32:50.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015539499999999999 2024-09-18T11:32:50.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:50.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388443+0000", 2024-09-18T11:32:50.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3785000000000001e-05 2024-09-18T11:32:50.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398238+0000", 2024-09-18T11:32:50.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097947989999999999 2024-09-18T11:32:50.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398315+0000", 2024-09-18T11:32:50.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6909999999999994e-05 2024-09-18T11:32:50.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398612+0000", 2024-09-18T11:32:50.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029671999999999999 2024-09-18T11:32:50.509 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.509 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.509 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.509 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2340 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:50.509 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:50.510 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 109.102840204, 2024-09-18T11:32:50.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012086448, 2024-09-18T11:32:50.510 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.510 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:50.510 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.510 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.510 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.510 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2340 2024-09-18T11:32:50.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.511 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:50.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:50.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:50.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:50.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386848+0000", 2024-09-18T11:32:50.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3069999999999998e-06 2024-09-18T11:32:50.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386855+0000", 2024-09-18T11:32:50.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0979999999999996e-06 2024-09-18T11:32:50.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386859+0000", 2024-09-18T11:32:50.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9360000000000001e-06 2024-09-18T11:32:50.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388466+0000", 2024-09-18T11:32:50.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001607154 2024-09-18T11:32:50.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:50.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388507+0000", 2024-09-18T11:32:50.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1093000000000001e-05 2024-09-18T11:32:50.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398647+0000", 2024-09-18T11:32:50.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010139686 2024-09-18T11:32:50.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398718+0000", 2024-09-18T11:32:50.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1138999999999998e-05 2024-09-18T11:32:50.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398931+0000", 2024-09-18T11:32:50.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021356899999999999 2024-09-18T11:32:50.516 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.516 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.516 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.516 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2341 4.5 4:aa575018:::smithi11939506-11:head [read 0~2487436,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:50.516 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:50.516 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 109.10282198199999, 2024-09-18T11:32:50.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013751648999999999, 2024-09-18T11:32:50.516 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.516 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:50.517 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.517 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.517 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.517 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2341 2024-09-18T11:32:50.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.517 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:50.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:50.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:50.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:50.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386866+0000", 2024-09-18T11:32:50.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4249999999999998e-06 2024-09-18T11:32:50.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.519 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386872+0000", 2024-09-18T11:32:50.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.003e-06 2024-09-18T11:32:50.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.519 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:32:50.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.507e-06 2024-09-18T11:32:50.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.519 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388520+0000", 2024-09-18T11:32:50.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001644146 2024-09-18T11:32:50.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:50.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388561+0000", 2024-09-18T11:32:50.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1486999999999998e-05 2024-09-18T11:32:50.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398963+0000", 2024-09-18T11:32:50.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010401353 2024-09-18T11:32:50.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.399046+0000", 2024-09-18T11:32:50.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3141999999999998e-05 2024-09-18T11:32:50.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.400615+0000", 2024-09-18T11:32:50.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015690960000000001 2024-09-18T11:32:50.522 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.522 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.522 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.523 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3105 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 3e=[3e,3d,3b,37,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-18T11:32:50.523 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:50.523 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 77.471474237999999, 2024-09-18T11:32:50.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011735906000000001, 2024-09-18T11:32:50.523 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.523 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:50.523 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.523 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.523 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.523 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3105 2024-09-18T11:32:50.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.523 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.524 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:50.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.524 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:50.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.524 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:50.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:50.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018212+0000", 2024-09-18T11:32:50.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6759999999999999e-06 2024-09-18T11:32:50.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018216+0000", 2024-09-18T11:32:50.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-06 2024-09-18T11:32:50.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018218+0000", 2024-09-18T11:32:50.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6110000000000001e-06 2024-09-18T11:32:50.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018246+0000", 2024-09-18T11:32:50.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7557000000000001e-05 2024-09-18T11:32:50.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:50.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018250+0000", 2024-09-18T11:32:50.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4299999999999999e-06 2024-09-18T11:32:50.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021364+0000", 2024-09-18T11:32:50.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031135389999999998 2024-09-18T11:32:50.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021494+0000", 2024-09-18T11:32:50.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000130333 2024-09-18T11:32:50.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.029947+0000", 2024-09-18T11:32:50.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084526150000000001 2024-09-18T11:32:50.529 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.529 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.530 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.530 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3323 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:50.530 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:32:50.530 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 68.372786194, 2024-09-18T11:32:50.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0088051689999999998, 2024-09-18T11:32:50.530 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.530 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:50.530 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.530 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.530 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.530 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3323 2024-09-18T11:32:50.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.531 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:32:50.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116899+0000", 2024-09-18T11:32:50.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116897+0000", 2024-09-18T11:32:50.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:50.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116908+0000", 2024-09-18T11:32:50.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1511e-05 2024-09-18T11:32:50.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116925+0000", 2024-09-18T11:32:50.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7028e-05 2024-09-18T11:32:50.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116932+0000", 2024-09-18T11:32:50.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5919999999999997e-06 2024-09-18T11:32:50.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116988+0000", 2024-09-18T11:32:50.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5928999999999999e-05 2024-09-18T11:32:50.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:50.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116995+0000", 2024-09-18T11:32:50.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7409999999999992e-06 2024-09-18T11:32:50.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122390+0000", 2024-09-18T11:32:50.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0053941889999999998 2024-09-18T11:32:50.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122453+0000", 2024-09-18T11:32:50.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3833999999999999e-05 2024-09-18T11:32:50.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.535 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:50.535 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.122581+0000", 2024-09-18T11:32:50.535 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012751200000000001 2024-09-18T11:32:50.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.535 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:50.535 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125034+0000", 2024-09-18T11:32:50.535 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024525609999999998 2024-09-18T11:32:50.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.535 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:50.536 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125607+0000", 2024-09-18T11:32:50.536 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00057356499999999995 2024-09-18T11:32:50.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.536 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:50.536 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125655+0000", 2024-09-18T11:32:50.536 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8179000000000001e-05 2024-09-18T11:32:50.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.536 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.536 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.125704+0000", 2024-09-18T11:32:50.536 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8950999999999999e-05 2024-09-18T11:32:50.537 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.537 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.537 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.537 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3324 4.5 4:a087a566:::smithi11939506-46:head [rollback 46] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:50.537 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:50.537 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 68.372743635000006, 2024-09-18T11:32:50.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010927767999999999, 2024-09-18T11:32:50.537 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.537 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:50.537 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.538 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.538 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.538 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3324 2024-09-18T11:32:50.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.538 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.538 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.538 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:50.538 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.538 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:50.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116941+0000", 2024-09-18T11:32:50.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:50.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116949+0000", 2024-09-18T11:32:50.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4660000000000002e-06 2024-09-18T11:32:50.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.540 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.540 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116960+0000", 2024-09-18T11:32:50.540 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1484999999999999e-05 2024-09-18T11:32:50.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.540 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.540 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116964+0000", 2024-09-18T11:32:50.540 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2810000000000002e-06 2024-09-18T11:32:50.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.540 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117053+0000", 2024-09-18T11:32:50.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8747999999999998e-05 2024-09-18T11:32:50.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:50.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117059+0000", 2024-09-18T11:32:50.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.246e-06 2024-09-18T11:32:50.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123604+0000", 2024-09-18T11:32:50.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065445319999999996 2024-09-18T11:32:50.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.542 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123663+0000", 2024-09-18T11:32:50.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8740999999999999e-05 2024-09-18T11:32:50.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.542 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:50.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123803+0000", 2024-09-18T11:32:50.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014049200000000001 2024-09-18T11:32:50.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.542 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:50.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127600+0000", 2024-09-18T11:32:50.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037971179999999999 2024-09-18T11:32:50.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:50.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127814+0000", 2024-09-18T11:32:50.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021333600000000001 2024-09-18T11:32:50.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:50.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127847+0000", 2024-09-18T11:32:50.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3188999999999997e-05 2024-09-18T11:32:50.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127869+0000", 2024-09-18T11:32:50.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2515e-05 2024-09-18T11:32:50.544 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.544 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.544 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.545 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3325 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:50.545 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:50.545 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 68.372696449000003, 2024-09-18T11:32:50.545 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181030999999999, 2024-09-18T11:32:50.545 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.545 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:50.545 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.545 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.545 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.546 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3325 2024-09-18T11:32:50.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.546 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.546 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:50.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.546 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:50.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.547 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116988+0000", 2024-09-18T11:32:50.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:50.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.547 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116991+0000", 2024-09-18T11:32:50.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9890000000000001e-06 2024-09-18T11:32:50.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116998+0000", 2024-09-18T11:32:50.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5309999999999998e-06 2024-09-18T11:32:50.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117005+0000", 2024-09-18T11:32:50.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6420000000000001e-06 2024-09-18T11:32:50.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117091+0000", 2024-09-18T11:32:50.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6835999999999993e-05 2024-09-18T11:32:50.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:50.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117097+0000", 2024-09-18T11:32:50.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:32:50.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124717+0000", 2024-09-18T11:32:50.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076204899999999997 2024-09-18T11:32:50.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124773+0000", 2024-09-18T11:32:50.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5470999999999998e-05 2024-09-18T11:32:50.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:50.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124877+0000", 2024-09-18T11:32:50.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010426500000000001 2024-09-18T11:32:50.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.551 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:50.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127901+0000", 2024-09-18T11:32:50.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030243449999999999 2024-09-18T11:32:50.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.551 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:50.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128115+0000", 2024-09-18T11:32:50.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021355300000000001 2024-09-18T11:32:50.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.552 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:50.552 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128145+0000", 2024-09-18T11:32:50.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0290999999999998e-05 2024-09-18T11:32:50.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.552 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.552 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128170+0000", 2024-09-18T11:32:50.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4811e-05 2024-09-18T11:32:50.553 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.553 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.553 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.553 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3474 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1872245~637761] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:50.553 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:50.553 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 64.045919772000005, 2024-09-18T11:32:50.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0092038580000000005, 2024-09-18T11:32:50.553 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.554 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:50.554 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.554 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.554 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.554 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3474 2024-09-18T11:32:50.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.554 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:50.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.556 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:50.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.557 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443764+0000", 2024-09-18T11:32:50.557 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:50.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.557 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444359+0000", 2024-09-18T11:32:50.557 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059519 2024-09-18T11:32:50.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.558 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444374+0000", 2024-09-18T11:32:50.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4949e-05 2024-09-18T11:32:50.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.558 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444382+0000", 2024-09-18T11:32:50.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1369999999999996e-06 2024-09-18T11:32:50.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444428+0000", 2024-09-18T11:32:50.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6783000000000003e-05 2024-09-18T11:32:50.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444507+0000", 2024-09-18T11:32:50.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8415000000000001e-05 2024-09-18T11:32:50.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:50.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444638+0000", 2024-09-18T11:32:50.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013129 2024-09-18T11:32:50.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:50.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.449059+0000", 2024-09-18T11:32:50.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004420719 2024-09-18T11:32:50.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.561 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:50.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452909+0000", 2024-09-18T11:32:50.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003850773 2024-09-18T11:32:50.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.561 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:50.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452944+0000", 2024-09-18T11:32:50.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-05 2024-09-18T11:32:50.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.562 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452969+0000", 2024-09-18T11:32:50.562 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5795999999999998e-05 2024-09-18T11:32:50.562 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.563 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.563 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.563 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.563 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3475 4.5 4:a56ea1fb:::smithi11939506-48:head [setxattr _header (57) in=7b,truncate 3042100] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:50.563 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:32:50.563 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 64.045291057, 2024-09-18T11:32:50.563 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086700660000000006, 2024-09-18T11:32:50.563 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.563 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:50.564 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.564 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.564 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.564 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3475 2024-09-18T11:32:50.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.564 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.565 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.565 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.565 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:32:50.565 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.565 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.565 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.565 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444394+0000", 2024-09-18T11:32:50.565 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.565 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.565 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.566 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444393+0000", 2024-09-18T11:32:50.566 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:50.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.566 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.566 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444398+0000", 2024-09-18T11:32:50.566 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4909999999999999e-06 2024-09-18T11:32:50.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.567 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.567 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.567 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444405+0000", 2024-09-18T11:32:50.567 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5329999999999997e-06 2024-09-18T11:32:50.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.568 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.568 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.568 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444410+0000", 2024-09-18T11:32:50.568 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2470000000000002e-06 2024-09-18T11:32:50.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.568 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.569 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.569 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446046+0000", 2024-09-18T11:32:50.569 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001636722 2024-09-18T11:32:50.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.569 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.569 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446121+0000", 2024-09-18T11:32:50.569 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4604000000000006e-05 2024-09-18T11:32:50.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.569 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:50.569 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446274+0000", 2024-09-18T11:32:50.570 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015350300000000001 2024-09-18T11:32:50.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.570 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.570 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:50.570 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452884+0000", 2024-09-18T11:32:50.570 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066096439999999996 2024-09-18T11:32:50.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.570 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.571 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:50.571 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453000+0000", 2024-09-18T11:32:50.571 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000116204 2024-09-18T11:32:50.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.571 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.572 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:50.572 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453032+0000", 2024-09-18T11:32:50.572 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1327e-05 2024-09-18T11:32:50.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.572 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.573 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.573 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453064+0000", 2024-09-18T11:32:50.573 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2645000000000003e-05 2024-09-18T11:32:50.573 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.573 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.573 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.574 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.574 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3476 4.5 4:a56ea1fb:::smithi11939506-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:50.574 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:50.574 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 64.045251789999995, 2024-09-18T11:32:50.574 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087596800000000006, 2024-09-18T11:32:50.574 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.574 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:50.574 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.575 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.575 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.575 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3476 2024-09-18T11:32:50.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.575 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.575 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.575 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.576 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:50.576 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.576 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.576 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:50.576 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.576 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.577 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444433+0000", 2024-09-18T11:32:50.577 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:50.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.577 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.577 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444436+0000", 2024-09-18T11:32:50.577 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9330000000000001e-06 2024-09-18T11:32:50.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.577 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.577 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444443+0000", 2024-09-18T11:32:50.577 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9369999999999998e-06 2024-09-18T11:32:50.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.578 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.578 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.578 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444447+0000", 2024-09-18T11:32:50.578 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1309999999999999e-06 2024-09-18T11:32:50.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.578 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.578 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.578 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446599+0000", 2024-09-18T11:32:50.578 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0021521190000000001 2024-09-18T11:32:50.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.578 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.578 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:50.578 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.446675+0000", 2024-09-18T11:32:50.578 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6099000000000004e-05 2024-09-18T11:32:50.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.579 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.579 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.579 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453088+0000", 2024-09-18T11:32:50.579 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.006413307 2024-09-18T11:32:50.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.579 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.579 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.579 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453132+0000", 2024-09-18T11:32:50.579 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3548999999999999e-05 2024-09-18T11:32:50.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.579 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.579 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.580 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.453193+0000", 2024-09-18T11:32:50.580 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1116000000000002e-05 2024-09-18T11:32:50.580 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.580 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.580 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.580 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.580 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3929 4.5 4:af4ba75a:::smithi11939506-37:head [sparse-read 0~1543958] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:50.580 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:50.580 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 47.470424327000003, 2024-09-18T11:32:50.580 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084586699999999997, 2024-09-18T11:32:50.580 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.581 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:50.581 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.581 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.581 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.581 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3929 2024-09-18T11:32:50.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.581 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.581 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.581 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.581 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:50.581 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.581 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.581 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.582 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:50.582 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.582 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.582 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.582 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019261+0000", 2024-09-18T11:32:50.582 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:50.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.582 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.582 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.583 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019262+0000", 2024-09-18T11:32:50.583 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8029999999999999e-06 2024-09-18T11:32:50.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.583 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.583 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.583 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019267+0000", 2024-09-18T11:32:50.583 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1119999999999998e-06 2024-09-18T11:32:50.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.583 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.583 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.583 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019269+0000", 2024-09-18T11:32:50.583 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4269999999999998e-06 2024-09-18T11:32:50.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.584 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.584 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.584 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019344+0000", 2024-09-18T11:32:50.584 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4979000000000002e-05 2024-09-18T11:32:50.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.584 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.584 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:50.584 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.019348+0000", 2024-09-18T11:32:50.584 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3930000000000001e-06 2024-09-18T11:32:50.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.584 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.585 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.585 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027385+0000", 2024-09-18T11:32:50.585 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080368409999999994 2024-09-18T11:32:50.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.585 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.585 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.585 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027463+0000", 2024-09-18T11:32:50.585 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7316000000000003e-05 2024-09-18T11:32:50.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.585 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.585 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.586 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027720+0000", 2024-09-18T11:32:50.586 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000257068 2024-09-18T11:32:50.586 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.586 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.586 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.586 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.586 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4251 4.5 4:af4ba75a:::smithi11939506-37:head [delete] snapc 69=[69,67,61,55,51,4e,4c,4b,45,42,3e,3d] ondisk+write+known_if_redirected+full_force+supports_pool_eio e155)", 2024-09-18T11:32:50.586 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:26.039209+0000", 2024-09-18T11:32:50.586 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 24.450475751999999, 2024-09-18T11:32:50.586 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010258422, 2024-09-18T11:32:50.586 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.586 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:50.587 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.587 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.587 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.587 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4251 2024-09-18T11:32:50.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.587 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.587 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.587 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039209+0000", 2024-09-18T11:32:50.587 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.587 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.588 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039209+0000", 2024-09-18T11:32:50.588 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.588 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.588 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.588 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039208+0000", 2024-09-18T11:32:50.588 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:50.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.588 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.588 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.588 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039217+0000", 2024-09-18T11:32:50.588 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2650000000000002e-06 2024-09-18T11:32:50.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.589 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.589 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039236+0000", 2024-09-18T11:32:50.589 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8734000000000001e-05 2024-09-18T11:32:50.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.589 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.589 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039244+0000", 2024-09-18T11:32:50.589 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2900000000000002e-06 2024-09-18T11:32:50.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.590 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.590 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039289+0000", 2024-09-18T11:32:50.590 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4657999999999998e-05 2024-09-18T11:32:50.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.590 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:50.591 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039294+0000", 2024-09-18T11:32:50.591 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0719999999999999e-06 2024-09-18T11:32:50.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.591 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.591 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.591 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.042052+0000", 2024-09-18T11:32:50.591 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027585399999999999 2024-09-18T11:32:50.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.591 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.592 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.592 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.042162+0000", 2024-09-18T11:32:50.592 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011005699999999999 2024-09-18T11:32:50.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.592 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.592 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:50.592 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.042341+0000", 2024-09-18T11:32:50.592 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000178285 2024-09-18T11:32:50.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.593 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.593 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:50.593 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.044140+0000", 2024-09-18T11:32:50.593 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001799092 2024-09-18T11:32:50.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.593 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.593 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:50.594 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.049396+0000", 2024-09-18T11:32:50.594 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005256213 2024-09-18T11:32:50.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.594 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.594 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:50.594 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.049431+0000", 2024-09-18T11:32:50.594 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.536e-05 2024-09-18T11:32:50.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.594 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.595 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.595 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.049468+0000", 2024-09-18T11:32:50.595 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6548999999999999e-05 2024-09-18T11:32:50.595 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.595 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.595 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.595 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.596 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1241 4.d e169/157 4:b44229a1:::smithi11939506-42:51 v 169'384, mlcod=168'380)", 2024-09-18T11:32:50.596 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.053467+0000", 2024-09-18T11:32:50.596 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 10.436217925999999, 2024-09-18T11:32:50.596 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010417658, 2024-09-18T11:32:50.596 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.596 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:50.596 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.597 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.597 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.597 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053467+0000", 2024-09-18T11:32:50.597 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.597 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.597 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.597 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053467+0000", 2024-09-18T11:32:50.597 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.598 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.598 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.598 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053467+0000", 2024-09-18T11:32:50.598 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:50.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.598 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.598 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.599 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053474+0000", 2024-09-18T11:32:50.599 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9430000000000003e-06 2024-09-18T11:32:50.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.599 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.599 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.599 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053483+0000", 2024-09-18T11:32:50.599 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3819999999999993e-06 2024-09-18T11:32:50.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.600 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.600 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.600 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053494+0000", 2024-09-18T11:32:50.600 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0609e-05 2024-09-18T11:32:50.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.600 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.600 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.601 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053532+0000", 2024-09-18T11:32:50.601 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8405000000000002e-05 2024-09-18T11:32:50.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.601 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.601 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.601 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053554+0000", 2024-09-18T11:32:50.601 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1758000000000001e-05 2024-09-18T11:32:50.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.602 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.602 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:50.602 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.063852+0000", 2024-09-18T11:32:50.602 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010298185 2024-09-18T11:32:50.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.602 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.602 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.602 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.063885+0000", 2024-09-18T11:32:50.603 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2838e-05 2024-09-18T11:32:50.603 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.603 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.603 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.603 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4513 4.a 4:552523b6:::smithi11939506-38:73 [set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:32:50.603 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.054905+0000", 2024-09-18T11:32:50.603 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 10.434780049, 2024-09-18T11:32:50.603 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.014365329, 2024-09-18T11:32:50.603 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.603 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:50.603 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.603 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.604 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.604 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4513 2024-09-18T11:32:50.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.604 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.604 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.604 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054905+0000", 2024-09-18T11:32:50.604 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.605 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.605 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054905+0000", 2024-09-18T11:32:50.605 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.605 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.605 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054904+0000", 2024-09-18T11:32:50.605 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:50.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.606 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054912+0000", 2024-09-18T11:32:50.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4300000000000006e-06 2024-09-18T11:32:50.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.606 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054927+0000", 2024-09-18T11:32:50.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4588e-05 2024-09-18T11:32:50.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.606 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054933+0000", 2024-09-18T11:32:50.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9009999999999999e-06 2024-09-18T11:32:50.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.607 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054975+0000", 2024-09-18T11:32:50.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2588e-05 2024-09-18T11:32:50.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.607 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055079+0000", 2024-09-18T11:32:50.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103508 2024-09-18T11:32:50.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.608 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:32:50.608 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.064787+0000", 2024-09-18T11:32:50.608 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097086719999999998 2024-09-18T11:32:50.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.608 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:50.608 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.066147+0000", 2024-09-18T11:32:50.608 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013597660000000001 2024-09-18T11:32:50.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.608 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:50.609 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.069210+0000", 2024-09-18T11:32:50.609 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030630430000000001 2024-09-18T11:32:50.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.609 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:50.609 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.069240+0000", 2024-09-18T11:32:50.609 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9628000000000002e-05 2024-09-18T11:32:50.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.609 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.609 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.069271+0000", 2024-09-18T11:32:50.610 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0682000000000002e-05 2024-09-18T11:32:50.610 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.610 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.610 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.610 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.610 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1242 4.d e169/157 4:b9bb642b:::smithi11939506-27:67 v 169'386, mlcod=168'380)", 2024-09-18T11:32:50.610 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.054970+0000", 2024-09-18T11:32:50.610 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 10.43471532, 2024-09-18T11:32:50.610 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089404059999999997, 2024-09-18T11:32:50.610 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.610 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:50.610 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.610 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054970+0000", 2024-09-18T11:32:50.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054970+0000", 2024-09-18T11:32:50.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054969+0000", 2024-09-18T11:32:50.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:50.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.612 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054977+0000", 2024-09-18T11:32:50.612 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6240000000000002e-06 2024-09-18T11:32:50.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.612 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054987+0000", 2024-09-18T11:32:50.612 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0186e-05 2024-09-18T11:32:50.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.612 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054993+0000", 2024-09-18T11:32:50.613 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5160000000000002e-06 2024-09-18T11:32:50.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.613 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.613 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055563+0000", 2024-09-18T11:32:50.613 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00057030399999999999 2024-09-18T11:32:50.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.613 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.613 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055576+0000", 2024-09-18T11:32:50.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3366e-05 2024-09-18T11:32:50.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.614 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:50.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.063887+0000", 2024-09-18T11:32:50.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083112240000000007 2024-09-18T11:32:50.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.614 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.063910+0000", 2024-09-18T11:32:50.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2932999999999999e-05 2024-09-18T11:32:50.614 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.614 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.614 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.615 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4665 4.a 4:588debc2:::smithi11939506-18:head [rollback 71] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:32:50.615 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.316572+0000", 2024-09-18T11:32:50.615 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.1731130619999997, 2024-09-18T11:32:50.615 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012421599, 2024-09-18T11:32:50.615 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.615 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:50.615 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.615 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.615 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.615 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4665 2024-09-18T11:32:50.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.615 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.615 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316572+0000", 2024-09-18T11:32:50.616 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.616 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316572+0000", 2024-09-18T11:32:50.616 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.616 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316572+0000", 2024-09-18T11:32:50.616 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:50.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.616 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316574+0000", 2024-09-18T11:32:50.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3599999999999999e-06 2024-09-18T11:32:50.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.617 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316578+0000", 2024-09-18T11:32:50.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0840000000000002e-06 2024-09-18T11:32:50.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.617 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316581+0000", 2024-09-18T11:32:50.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8739999999999999e-06 2024-09-18T11:32:50.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318695+0000", 2024-09-18T11:32:50.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002113382 2024-09-18T11:32:50.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318805+0000", 2024-09-18T11:32:50.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000110511 2024-09-18T11:32:50.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:32:50.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.321575+0000", 2024-09-18T11:32:50.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027700730000000001 2024-09-18T11:32:50.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.619 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:50.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.324576+0000", 2024-09-18T11:32:50.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030004599999999999 2024-09-18T11:32:50.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.619 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:50.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.328914+0000", 2024-09-18T11:32:50.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043382159999999998 2024-09-18T11:32:50.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.619 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:50.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.328951+0000", 2024-09-18T11:32:50.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7036999999999998e-05 2024-09-18T11:32:50.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.620 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.620 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.328994+0000", 2024-09-18T11:32:50.620 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2979999999999998e-05 2024-09-18T11:32:50.620 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.620 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.620 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.620 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4666 4.a 4:588debc2:::smithi11939506-18:head [append 0~0] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:32:50.620 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.316584+0000", 2024-09-18T11:32:50.621 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 5.1731014320000002, 2024-09-18T11:32:50.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012596499000000001, 2024-09-18T11:32:50.621 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.621 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:50.621 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.621 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.621 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.621 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4666 2024-09-18T11:32:50.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.621 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316584+0000", 2024-09-18T11:32:50.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316584+0000", 2024-09-18T11:32:50.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316584+0000", 2024-09-18T11:32:50.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:50.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316585+0000", 2024-09-18T11:32:50.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8109999999999999e-06 2024-09-18T11:32:50.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.623 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316588+0000", 2024-09-18T11:32:50.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0769999999999999e-06 2024-09-18T11:32:50.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.623 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316591+0000", 2024-09-18T11:32:50.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1320000000000001e-06 2024-09-18T11:32:50.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.624 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.319126+0000", 2024-09-18T11:32:50.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002535266 2024-09-18T11:32:50.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.624 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for blocked object", 2024-09-18T11:32:50.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.319180+0000", 2024-09-18T11:32:50.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4457999999999999e-05 2024-09-18T11:32:50.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.624 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.625 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.323839+0000", 2024-09-18T11:32:50.625 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046581720000000004 2024-09-18T11:32:50.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.625 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.625 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.324024+0000", 2024-09-18T11:32:50.625 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018519599999999999 2024-09-18T11:32:50.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.625 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:32:50.625 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.324246+0000", 2024-09-18T11:32:50.625 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022207800000000001 2024-09-18T11:32:50.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:50.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.325067+0000", 2024-09-18T11:32:50.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00082141699999999996 2024-09-18T11:32:50.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:50.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.329117+0000", 2024-09-18T11:32:50.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040501210000000003 2024-09-18T11:32:50.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:50.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.329151+0000", 2024-09-18T11:32:50.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4133e-05 2024-09-18T11:32:50.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.329180+0000", 2024-09-18T11:32:50.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.887e-05 2024-09-18T11:32:50.627 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.627 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.627 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.627 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4754 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2739607,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:32:50.628 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:49.234755+0000", 2024-09-18T11:32:50.628 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 1.254929773, 2024-09-18T11:32:50.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0093043659999999997, 2024-09-18T11:32:50.628 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:50.628 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:50.628 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:50.628 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:50.628 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:50.628 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4754 2024-09-18T11:32:50.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.628 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:50.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:50.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234755+0000", 2024-09-18T11:32:50.629 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:50.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234755+0000", 2024-09-18T11:32:50.629 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:50.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:50.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234755+0000", 2024-09-18T11:32:50.629 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:50.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:50.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234760+0000", 2024-09-18T11:32:50.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6380000000000003e-06 2024-09-18T11:32:50.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:50.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234766+0000", 2024-09-18T11:32:50.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0159999999999997e-06 2024-09-18T11:32:50.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:50.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234771+0000", 2024-09-18T11:32:50.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1399999999999999e-06 2024-09-18T11:32:50.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234846+0000", 2024-09-18T11:32:50.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5308999999999993e-05 2024-09-18T11:32:50.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:50.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234848+0000", 2024-09-18T11:32:50.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4260000000000002e-06 2024-09-18T11:32:50.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.632 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:50.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.241890+0000", 2024-09-18T11:32:50.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070411859999999996 2024-09-18T11:32:50.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.632 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:50.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.241983+0000", 2024-09-18T11:32:50.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3843000000000003e-05 2024-09-18T11:32:50.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:50.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:50.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:50.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.244060+0000", 2024-09-18T11:32:50.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020763679999999999 2024-09-18T11:32:50.633 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.633 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.633 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.633 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:50.633 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:50.633 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:50.634 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-18T11:32:50.634 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T11:32:50.634 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-18T11:32:50.634 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T11:32:50.634 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T11:32:50.634 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-18T11:32:50.635 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:50.636 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:50.636 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:50.637 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:50.637 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:32:50.638 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:50.638 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:50.638 INFO:tasks.rados.rados.0.smithi119.stdout:2027: finishing rollback tid 0 to smithi11939506-43 2024-09-18T11:32:50.638 INFO:tasks.rados.rados.0.smithi119.stdout:2027: finishing rollback tid 1 to smithi11939506-43 2024-09-18T11:32:50.638 INFO:tasks.rados.rados.0.smithi119.stdout:2027: finishing rollback tid 2 to smithi11939506-43 2024-09-18T11:32:50.639 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 43 v 407 (ObjNum 346 snap 51 seq_num 346) dirty exists 2024-09-18T11:32:50.639 INFO:tasks.rados.rados.0.smithi119.stdout:2028: expect (ObjNum 320 snap 44 seq_num 320) 2024-09-18T11:32:50.639 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:50.639 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:50.639 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:50.640 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:50.640 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:50.640 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:50.641 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:50.641 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:50.641 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:50.642 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:50.643 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:50.634+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:50.736 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:50.735+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:50.741 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:50.742 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:50.750 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:32:50.779 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:50.779 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T11:32:50.779 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:50.837 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:50.836+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:50.847 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:50.847 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:50.856 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:32:50.938 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:50.938+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:50.995 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:50.995 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:51.004 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:32:51.031 INFO:tasks.rados.rados.0.smithi119.stdout:2027: done (4 left) 2024-09-18T11:32:51.031 INFO:tasks.rados.rados.0.smithi119.stdout:2028: done (3 left) 2024-09-18T11:32:51.031 INFO:tasks.rados.rados.0.smithi119.stdout:2029: done (2 left) 2024-09-18T11:32:51.032 INFO:tasks.rados.rados.0.smithi119.stdout:2031: done (1 left) 2024-09-18T11:32:51.032 INFO:tasks.rados.rados.0.smithi119.stdout:2032: done (0 left) 2024-09-18T11:32:51.032 INFO:tasks.rados.rados.0.smithi119.stdout:2033: snap_remove snap 67 2024-09-18T11:32:51.039 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:51.038+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:51.102 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:51.103 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:51.111 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:32:51.152 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:51.151+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:51.240 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:51.239+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:51.249 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:51.249 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:51.257 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:32:51.342 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:51.340+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:51.354 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:51.354 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:51.363 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:32:51.443 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:51.441+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:51.459 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:51.459 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:51.468 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:32:51.543 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:51.542+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:51.566 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:51.566 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:51.574 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:32:51.645 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:51.643+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:51.671 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:51.671 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:51.680 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:32:51.746 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:51.744+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:51.778 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:51.778 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:51.786 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:32:51.871 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:51.869+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:51.924 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:51.924 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:51.932 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:32:51.949 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:51.948+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:52.032 INFO:tasks.rados.rados.0.smithi119.stdout:2033: done (0 left) 2024-09-18T11:32:52.032 INFO:tasks.rados.rados.0.smithi119.stdout:2034: read oid 17 snap -1 2024-09-18T11:32:52.032 INFO:tasks.rados.rados.0.smithi119.stdout:2034: expect (ObjNum 383 snap 60 seq_num 383) 2024-09-18T11:32:52.032 INFO:tasks.rados.rados.0.smithi119.stdout:2035: read oid 25 snap 63 2024-09-18T11:32:52.032 INFO:tasks.rados.rados.0.smithi119.stdout:2035: expect deleted 2024-09-18T11:32:52.032 INFO:tasks.rados.rados.0.smithi119.stdout:2036: snap_remove snap 58 2024-09-18T11:32:52.043 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T11:32:52.049 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:52.048+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:52.075 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:52.075 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:52.084 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:32:52.168 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:52.166+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:52.225 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:52.225 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:52.234 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:32:52.275 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:52.273+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:52.353 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:52.352+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:52.371 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:52.371 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:52.379 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:32:52.453 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:52.451+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:52.478 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:52.478 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:52.487 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:32:52.554 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:52.553+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:52.597 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:52.597 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:52.606 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:32:52.643 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:32:52.655 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:52.654+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:52.703 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:52.704 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:52.712 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:32:52.757 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:52.757 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:52.772 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:52.773 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:52.787 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:52.787 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:52.796 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:52.795+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:52.802 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:52.802 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:52.817 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:52.817 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:52.833 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:52.834 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:52.848 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:52.849 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:52.854 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:52.854 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:52.854 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:52.854 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:52.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.855 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:32:52.855 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:52.855 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 138.43577028600001, 2024-09-18T11:32:52.855 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:32:52.855 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:52.855 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:52.855 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:52.855 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:52.855 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:52.855 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:32:52.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.856 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:52.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:52.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:52.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:52.856 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:32:52.856 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.856 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:52.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:32:52.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:52.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:52.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:32:52.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:32:52.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.857 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:52.857 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:32:52.857 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:32:52.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:52.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:32:52.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:32:52.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:32:52.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:32:52.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.858 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:52.858 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:32:52.858 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:32:52.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:52.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:32:52.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:32:52.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.859 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:52.859 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:32:52.859 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:32:52.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.859 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:52.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:32:52.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:32:52.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:52.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:32:52.860 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:32:52.860 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.860 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:52.860 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:32:52.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:32:52.861 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.861 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:52.861 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.861 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:32:52.861 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:52.861 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 111.46554088000001, 2024-09-18T11:32:52.861 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:32:52.861 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:52.861 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:52.862 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:52.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:52.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:52.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:52.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:52.862 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.862 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.862 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:52.862 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:32:52.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:52.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:52.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:32:52.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:32:52.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.863 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:52.863 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:32:52.863 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:32:52.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:52.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:32:52.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:32:52.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.864 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:32:52.864 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:32:52.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.864 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:52.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:32:52.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:32:52.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:52.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:32:52.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:32:52.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.865 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:52.865 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:32:52.865 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:32:52.866 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.866 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:52.866 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.866 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:32:52.866 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:52.866 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 103.87942827400001, 2024-09-18T11:32:52.866 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:32:52.866 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:52.866 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:52.866 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:52.867 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:52.867 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:52.867 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:32:52.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.867 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:52.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:52.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:52.867 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.867 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:52.867 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:32:52.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:52.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:32:52.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:52.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.868 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:52.868 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:32:52.868 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:32:52.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:52.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:32:52.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:32:52.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:52.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:32:52.869 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:32:52.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.869 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.869 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:32:52.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:32:52.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:52.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:32:52.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:32:52.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.870 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:52.870 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:32:52.870 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:32:52.870 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:52.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:32:52.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:32:52.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:52.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:32:52.871 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:32:52.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.871 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:52.871 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:32:52.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:32:52.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.872 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:52.872 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:32:52.872 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:32:52.872 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.872 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:52.872 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.872 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.872 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2956 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~2100908,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:52.873 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:52.873 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.824632023999996, 2024-09-18T11:32:52.873 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011863831, 2024-09-18T11:32:52.873 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:52.873 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:52.873 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:52.873 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:52.873 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:52.873 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2956 2024-09-18T11:32:52.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.873 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:52.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.873 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:52.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:52.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:52.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:52.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.874 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:52.874 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:32:52.874 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:52.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:52.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027228+0000", 2024-09-18T11:32:52.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:32:52.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:52.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027235+0000", 2024-09-18T11:32:52.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:32:52.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.875 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:52.875 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027239+0000", 2024-09-18T11:32:52.875 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.535e-06 2024-09-18T11:32:52.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027831+0000", 2024-09-18T11:32:52.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059244699999999996 2024-09-18T11:32:52.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:52.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027879+0000", 2024-09-18T11:32:52.876 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:32:52.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.876 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.876 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038098+0000", 2024-09-18T11:32:52.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010219137 2024-09-18T11:32:52.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:52.877 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038207+0000", 2024-09-18T11:32:52.877 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010951999999999999 2024-09-18T11:32:52.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.877 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:52.878 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039089+0000", 2024-09-18T11:32:52.878 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00088151699999999996 2024-09-18T11:32:52.878 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.878 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:52.878 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.879 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2957 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:32:52.879 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:52.880 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 88.824614240000002, 2024-09-18T11:32:52.880 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012041979, 2024-09-18T11:32:52.880 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:52.880 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:52.880 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:52.880 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:52.880 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:52.880 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2957 2024-09-18T11:32:52.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.880 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:52.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.880 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:52.880 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:52.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:52.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:52.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.881 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:52.881 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:32:52.881 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:52.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:52.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027245+0000", 2024-09-18T11:32:52.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:32:52.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:52.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027251+0000", 2024-09-18T11:32:52.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0090000000000004e-06 2024-09-18T11:32:52.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.882 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:52.882 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027255+0000", 2024-09-18T11:32:52.882 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:32:52.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027895+0000", 2024-09-18T11:32:52.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064076400000000005 2024-09-18T11:32:52.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:52.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027930+0000", 2024-09-18T11:32:52.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4334e-05 2024-09-18T11:32:52.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.883 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.883 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039111+0000", 2024-09-18T11:32:52.883 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181455999999999 2024-09-18T11:32:52.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:52.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039156+0000", 2024-09-18T11:32:52.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5256000000000002e-05 2024-09-18T11:32:52.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.884 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:52.884 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039285+0000", 2024-09-18T11:32:52.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012839200000000001 2024-09-18T11:32:52.884 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.884 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:52.884 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.885 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:978 4.b e132/67 4:d8d14c58:::smithi11939506-49:30 v 132'417, mlcod=128'414)", 2024-09-18T11:32:52.885 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:52.885 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 49.826582958000003, 2024-09-18T11:32:52.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011942183, 2024-09-18T11:32:52.885 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:52.885 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:52.885 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:52.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:52.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:52.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:52.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:52.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:52.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:32:52.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:52.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:52.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025278+0000", 2024-09-18T11:32:52.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9060000000000004e-06 2024-09-18T11:32:52.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:52.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025288+0000", 2024-09-18T11:32:52.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.028e-05 2024-09-18T11:32:52.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:52.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025293+0000", 2024-09-18T11:32:52.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2700000000000004e-06 2024-09-18T11:32:52.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025346+0000", 2024-09-18T11:32:52.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2862999999999999e-05 2024-09-18T11:32:52.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:52.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025370+0000", 2024-09-18T11:32:52.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4005000000000001e-05 2024-09-18T11:32:52.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:52.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037159+0000", 2024-09-18T11:32:52.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011788343 2024-09-18T11:32:52.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:52.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037216+0000", 2024-09-18T11:32:52.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7695000000000002e-05 2024-09-18T11:32:52.889 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.889 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:52.889 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.890 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3930 4.b e132/67 4:d3ae8215:::smithi11939506-41:head v 132'421, mlcod=128'414)", 2024-09-18T11:32:52.890 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:52.890 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 49.823034710000002, 2024-09-18T11:32:52.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015093554, 2024-09-18T11:32:52.890 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:52.890 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:52.890 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:52.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:52.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:52.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:52.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:32:52.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:52.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028821+0000", 2024-09-18T11:32:52.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:52.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:52.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029805+0000", 2024-09-18T11:32:52.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00098410899999999994 2024-09-18T11:32:52.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:52.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029817+0000", 2024-09-18T11:32:52.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2058e-05 2024-09-18T11:32:52.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:52.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029823+0000", 2024-09-18T11:32:52.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6450000000000002e-06 2024-09-18T11:32:52.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032330+0000", 2024-09-18T11:32:52.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025067710000000001 2024-09-18T11:32:52.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:52.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032349+0000", 2024-09-18T11:32:52.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9729e-05 2024-09-18T11:32:52.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:52.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043878+0000", 2024-09-18T11:32:52.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011528514 2024-09-18T11:32:52.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:52.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043916+0000", 2024-09-18T11:32:52.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8049000000000001e-05 2024-09-18T11:32:52.893 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.894 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:52.894 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.894 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:162 3.1 3:8c826a1f:::e682cfc1e82c88472b9acf1cd97978bc105aa035f10b685bb4cace95809b52bf:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:52.894 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:32:52.894 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 43.270912551999999, 2024-09-18T11:32:52.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013234571000000001, 2024-09-18T11:32:52.894 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:52.894 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:52.894 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:52.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:52.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:32:52.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:52.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:32:52.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:52.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580943+0000", 2024-09-18T11:32:52.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:52.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:52.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582556+0000", 2024-09-18T11:32:52.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016132970000000001 2024-09-18T11:32:52.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:52.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582572+0000", 2024-09-18T11:32:52.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5846999999999999e-05 2024-09-18T11:32:52.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:52.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582578+0000", 2024-09-18T11:32:52.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4810000000000003e-06 2024-09-18T11:32:52.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582632+0000", 2024-09-18T11:32:52.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4156000000000001e-05 2024-09-18T11:32:52.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:52.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582819+0000", 2024-09-18T11:32:52.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000186431 2024-09-18T11:32:52.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:52.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583118+0000", 2024-09-18T11:32:52.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029880100000000001 2024-09-18T11:32:52.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:52.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590676+0000", 2024-09-18T11:32:52.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075588670000000004 2024-09-18T11:32:52.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:52.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594111+0000", 2024-09-18T11:32:52.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034348550000000001 2024-09-18T11:32:52.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:52.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594142+0000", 2024-09-18T11:32:52.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.099e-05 2024-09-18T11:32:52.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:52.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594179+0000", 2024-09-18T11:32:52.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6831e-05 2024-09-18T11:32:52.899 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.899 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:52.899 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.899 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3974 4.d 4:b5a463bf:::smithi11939506-20:head [write 665643~697786,stat] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:52.899 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:32:52.899 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 43.269825869000002, 2024-09-18T11:32:52.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016621174999999998, 2024-09-18T11:32:52.900 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:52.900 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:52.900 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:52.900 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:52.900 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:52.900 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3974 2024-09-18T11:32:52.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.900 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:52.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:52.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:32:52.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:52.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:32:52.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:52.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582030+0000", 2024-09-18T11:32:52.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:52.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:52.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585185+0000", 2024-09-18T11:32:52.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003155644 2024-09-18T11:32:52.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:52.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585208+0000", 2024-09-18T11:32:52.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3013000000000001e-05 2024-09-18T11:32:52.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:52.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585215+0000", 2024-09-18T11:32:52.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7569999999999998e-06 2024-09-18T11:32:52.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585278+0000", 2024-09-18T11:32:52.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2699000000000004e-05 2024-09-18T11:32:52.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:52.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585374+0000", 2024-09-18T11:32:52.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.6062000000000003e-05 2024-09-18T11:32:52.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:32:52.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585577+0000", 2024-09-18T11:32:52.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000202926 2024-09-18T11:32:52.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:52.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596544+0000", 2024-09-18T11:32:52.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010966848 2024-09-18T11:32:52.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:52.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598594+0000", 2024-09-18T11:32:52.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020502049999999998 2024-09-18T11:32:52.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:52.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598625+0000", 2024-09-18T11:32:52.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1412999999999999e-05 2024-09-18T11:32:52.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:52.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598652+0000", 2024-09-18T11:32:52.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7166999999999999e-05 2024-09-18T11:32:52.905 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.905 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:52.905 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.905 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:166 3.1 3:a73e635b:::b6ee478f0f661ec7bac3d17c2ca2fdaff71a47570f64844a7ba79bbb2a0e8d9d:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:52.906 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:32:52.906 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 43.269266502999997, 2024-09-18T11:32:52.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011711037000000001, 2024-09-18T11:32:52.906 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:52.906 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:52.906 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:52.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:52.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:32:52.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:52.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:32:52.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:52.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582590+0000", 2024-09-18T11:32:52.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:52.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:52.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582673+0000", 2024-09-18T11:32:52.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2861000000000005e-05 2024-09-18T11:32:52.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:52.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582683+0000", 2024-09-18T11:32:52.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0081999999999999e-05 2024-09-18T11:32:52.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:52.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582688+0000", 2024-09-18T11:32:52.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6870000000000002e-06 2024-09-18T11:32:52.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584266+0000", 2024-09-18T11:32:52.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001578423 2024-09-18T11:32:52.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:52.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584407+0000", 2024-09-18T11:32:52.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014144599999999999 2024-09-18T11:32:52.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:52.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584599+0000", 2024-09-18T11:32:52.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000191881 2024-09-18T11:32:52.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:52.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590680+0000", 2024-09-18T11:32:52.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060801479999999996 2024-09-18T11:32:52.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:52.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594224+0000", 2024-09-18T11:32:52.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035441700000000001 2024-09-18T11:32:52.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:52.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594269+0000", 2024-09-18T11:32:52.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5297999999999999e-05 2024-09-18T11:32:52.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:52.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594302+0000", 2024-09-18T11:32:52.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2703000000000002e-05 2024-09-18T11:32:52.911 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.911 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:52.911 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.911 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:168 3.1 3:b64d7bad:::7feb22256bdcec217720fd46184ab43ced8923ed66ec6319c57f078d9f123262:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:52.911 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:32:52.911 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 43.269039051, 2024-09-18T11:32:52.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011589809, 2024-09-18T11:32:52.912 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:52.912 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:52.912 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:52.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:52.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:32:52.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:52.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:32:52.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:52.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582817+0000", 2024-09-18T11:32:52.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:52.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:52.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582903+0000", 2024-09-18T11:32:52.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5915999999999995e-05 2024-09-18T11:32:52.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:52.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582913+0000", 2024-09-18T11:32:52.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1409999999999994e-06 2024-09-18T11:32:52.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:52.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582917+0000", 2024-09-18T11:32:52.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6419999999999998e-06 2024-09-18T11:32:52.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585540+0000", 2024-09-18T11:32:52.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026228940000000002 2024-09-18T11:32:52.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:52.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585721+0000", 2024-09-18T11:32:52.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018086299999999999 2024-09-18T11:32:52.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:52.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585918+0000", 2024-09-18T11:32:52.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000196758 2024-09-18T11:32:52.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:52.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590683+0000", 2024-09-18T11:32:52.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047657020000000001 2024-09-18T11:32:52.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:52.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594354+0000", 2024-09-18T11:32:52.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003671068 2024-09-18T11:32:52.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:52.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594386+0000", 2024-09-18T11:32:52.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1426e-05 2024-09-18T11:32:52.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:52.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594408+0000", 2024-09-18T11:32:52.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1987000000000001e-05 2024-09-18T11:32:52.917 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.917 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:52.917 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.917 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:175 3.1 3:b5e86b88:::e55cf3d5cf782ed327a67e746b287b76d05f74f58a4f3726e399a564440ceaf6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:52.917 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:32:52.917 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 43.268190109999999, 2024-09-18T11:32:52.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013709627, 2024-09-18T11:32:52.917 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:52.917 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:52.917 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:52.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:52.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:32:52.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:52.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:32:52.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:52.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583666+0000", 2024-09-18T11:32:52.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:52.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:52.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584041+0000", 2024-09-18T11:32:52.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037511499999999998 2024-09-18T11:32:52.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:52.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584055+0000", 2024-09-18T11:32:52.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3295999999999999e-05 2024-09-18T11:32:52.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:52.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584061+0000", 2024-09-18T11:32:52.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1630000000000001e-06 2024-09-18T11:32:52.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.589980+0000", 2024-09-18T11:32:52.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005918998 2024-09-18T11:32:52.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:52.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590117+0000", 2024-09-18T11:32:52.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013715400000000001 2024-09-18T11:32:52.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:52.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590270+0000", 2024-09-18T11:32:52.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015245600000000001 2024-09-18T11:32:52.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:52.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596513+0000", 2024-09-18T11:32:52.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062438729999999996 2024-09-18T11:32:52.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:52.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597328+0000", 2024-09-18T11:32:52.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081489599999999996 2024-09-18T11:32:52.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:52.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597358+0000", 2024-09-18T11:32:52.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9291000000000001e-05 2024-09-18T11:32:52.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:52.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597377+0000", 2024-09-18T11:32:52.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:32:52.922 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.922 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:52.922 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.923 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4061 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:52.923 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:32:52.923 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.555011421000003, 2024-09-18T11:32:52.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013340967, 2024-09-18T11:32:52.923 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:52.923 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:52.923 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:52.923 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:52.923 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:52.923 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4061 2024-09-18T11:32:52.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.923 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:52.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:52.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:32:52.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:52.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:32:52.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:52.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296844+0000", 2024-09-18T11:32:52.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:52.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:52.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296854+0000", 2024-09-18T11:32:52.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0037000000000001e-05 2024-09-18T11:32:52.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:52.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296873+0000", 2024-09-18T11:32:52.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:32:52.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:52.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296881+0000", 2024-09-18T11:32:52.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3790000000000006e-06 2024-09-18T11:32:52.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296975+0000", 2024-09-18T11:32:52.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3536999999999996e-05 2024-09-18T11:32:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296982+0000", 2024-09-18T11:32:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6399999999999997e-06 2024-09-18T11:32:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301933+0000", 2024-09-18T11:32:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049507190000000001 2024-09-18T11:32:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301984+0000", 2024-09-18T11:32:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1439e-05 2024-09-18T11:32:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:32:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082022050000000006 2024-09-18T11:32:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4062 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.554962865999997, 2024-09-18T11:32:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016297147000000001, 2024-09-18T11:32:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4062 2024-09-18T11:32:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:32:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296898+0000", 2024-09-18T11:32:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9419999999999997e-06 2024-09-18T11:32:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296906+0000", 2024-09-18T11:32:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3520000000000007e-06 2024-09-18T11:32:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296910+0000", 2024-09-18T11:32:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4469999999999999e-06 2024-09-18T11:32:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297032+0000", 2024-09-18T11:32:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012168900000000001 2024-09-18T11:32:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297038+0000", 2024-09-18T11:32:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9320000000000001e-06 2024-09-18T11:32:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310221+0000", 2024-09-18T11:32:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013182887000000001 2024-09-18T11:32:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310323+0000", 2024-09-18T11:32:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010170000000000001 2024-09-18T11:32:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313191+0000", 2024-09-18T11:32:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002868808 2024-09-18T11:32:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4063 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.554941974000002, 2024-09-18T11:32:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017648712, 2024-09-18T11:32:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4063 2024-09-18T11:32:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:32:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296918+0000", 2024-09-18T11:32:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8100000000000002e-06 2024-09-18T11:32:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296924+0000", 2024-09-18T11:32:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2519999999999996e-06 2024-09-18T11:32:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296928+0000", 2024-09-18T11:32:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0500000000000002e-06 2024-09-18T11:32:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297062+0000", 2024-09-18T11:32:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013425500000000001 2024-09-18T11:32:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297069+0000", 2024-09-18T11:32:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5239999999999997e-06 2024-09-18T11:32:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313424+0000", 2024-09-18T11:32:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016355277000000001 2024-09-18T11:32:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313514+0000", 2024-09-18T11:32:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0581999999999994e-05 2024-09-18T11:32:52.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:52.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.314564+0000", 2024-09-18T11:32:52.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001049384 2024-09-18T11:32:52.939 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.939 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:52.939 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4065 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:32:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.554896532999997, 2024-09-18T11:32:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013225997, 2024-09-18T11:32:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4065 2024-09-18T11:32:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:32:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:32:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296960+0000", 2024-09-18T11:32:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296963+0000", 2024-09-18T11:32:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2270000000000002e-06 2024-09-18T11:32:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296970+0000", 2024-09-18T11:32:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6220000000000003e-06 2024-09-18T11:32:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296973+0000", 2024-09-18T11:32:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3730000000000001e-06 2024-09-18T11:32:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297097+0000", 2024-09-18T11:32:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012352399999999999 2024-09-18T11:32:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297103+0000", 2024-09-18T11:32:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7119999999999997e-06 2024-09-18T11:32:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307831+0000", 2024-09-18T11:32:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010728794 2024-09-18T11:32:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307924+0000", 2024-09-18T11:32:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2099e-05 2024-09-18T11:32:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:32:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022630469999999998 2024-09-18T11:32:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4066 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:32:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.554879458000002, 2024-09-18T11:32:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015656389999999999, 2024-09-18T11:32:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4066 2024-09-18T11:32:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:32:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:32:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:52.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296977+0000", 2024-09-18T11:32:52.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:52.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:52.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296980+0000", 2024-09-18T11:32:52.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8830000000000002e-06 2024-09-18T11:32:52.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:52.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296985+0000", 2024-09-18T11:32:52.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:32:52.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:52.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296988+0000", 2024-09-18T11:32:52.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0419999999999998e-06 2024-09-18T11:32:52.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297120+0000", 2024-09-18T11:32:52.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131693 2024-09-18T11:32:52.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:52.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297125+0000", 2024-09-18T11:32:52.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1100000000000002e-06 2024-09-18T11:32:52.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310212+0000", 2024-09-18T11:32:52.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013087395999999999 2024-09-18T11:32:52.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:52.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310292+0000", 2024-09-18T11:32:52.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9980000000000003e-05 2024-09-18T11:32:52.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:52.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312634+0000", 2024-09-18T11:32:52.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002341756 2024-09-18T11:32:52.950 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.950 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:52.950 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.950 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4068 3.1 3:87e79ed0:::3526641c07d3efa989be9684355570f39c331008b8a02c9fa22b38a94b6c6b9f:head [write 0~1015808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:32:52.950 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:32:52.950 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.551103241, 2024-09-18T11:32:52.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016392648999999999, 2024-09-18T11:32:52.951 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:52.951 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:52.951 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:52.951 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:52.951 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:52.951 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4068 2024-09-18T11:32:52.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.951 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:52.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:52.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:32:52.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:52.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:32:52.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:52.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300753+0000", 2024-09-18T11:32:52.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:52.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:52.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301922+0000", 2024-09-18T11:32:52.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001169432 2024-09-18T11:32:52.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:52.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301936+0000", 2024-09-18T11:32:52.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3574000000000001e-05 2024-09-18T11:32:52.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:52.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301941+0000", 2024-09-18T11:32:52.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6069999999999998e-06 2024-09-18T11:32:52.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310234+0000", 2024-09-18T11:32:52.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082924820000000003 2024-09-18T11:32:52.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:52.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310337+0000", 2024-09-18T11:32:52.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102673 2024-09-18T11:32:52.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:52.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310469+0000", 2024-09-18T11:32:52.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131982 2024-09-18T11:32:52.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:52.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312844+0000", 2024-09-18T11:32:52.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023758680000000002 2024-09-18T11:32:52.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:52.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317092+0000", 2024-09-18T11:32:52.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042477549999999998 2024-09-18T11:32:52.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:52.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317124+0000", 2024-09-18T11:32:52.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1968999999999998e-05 2024-09-18T11:32:52.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:52.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317147+0000", 2024-09-18T11:32:52.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2339000000000001e-05 2024-09-18T11:32:52.956 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.956 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:52.956 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.956 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4671 4.1 4:87078dfe:::smithi11939506-14:head [rollback 6f] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:32:52.956 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:32:52.956 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.5331408660000001, 2024-09-18T11:32:52.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013540905000000001, 2024-09-18T11:32:52.957 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:52.957 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:52.957 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:52.957 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:52.957 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:52.957 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4671 2024-09-18T11:32:52.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.957 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:52.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:52.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:32:52.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:52.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:32:52.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:52.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:32:52.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:52.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:52.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318718+0000", 2024-09-18T11:32:52.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3300000000000001e-06 2024-09-18T11:32:52.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:52.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318723+0000", 2024-09-18T11:32:52.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1590000000000001e-06 2024-09-18T11:32:52.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:52.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318727+0000", 2024-09-18T11:32:52.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3529999999999999e-06 2024-09-18T11:32:52.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.320735+0000", 2024-09-18T11:32:52.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020079730000000001 2024-09-18T11:32:52.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:52.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.320843+0000", 2024-09-18T11:32:52.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000108488 2024-09-18T11:32:52.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:52.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.329522+0000", 2024-09-18T11:32:52.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086787979999999997 2024-09-18T11:32:52.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:52.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331769+0000", 2024-09-18T11:32:52.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022471269999999998 2024-09-18T11:32:52.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:52.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332189+0000", 2024-09-18T11:32:52.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00041940400000000001 2024-09-18T11:32:52.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:52.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332228+0000", 2024-09-18T11:32:52.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9298000000000002e-05 2024-09-18T11:32:52.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:52.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332257+0000", 2024-09-18T11:32:52.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.934e-05 2024-09-18T11:32:52.962 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.962 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:52.962 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.962 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4672 4.1 4:87078dfe:::smithi11939506-14:head [append 0~0] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:32:52.962 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.318763+0000", 2024-09-18T11:32:52.962 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 7.5330944630000003, 2024-09-18T11:32:52.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016099992, 2024-09-18T11:32:52.962 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:52.963 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:52.963 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:52.963 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:52.963 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:52.963 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4672 2024-09-18T11:32:52.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.963 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:52.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:52.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318763+0000", 2024-09-18T11:32:52.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:52.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318763+0000", 2024-09-18T11:32:52.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:52.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:52.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318762+0000", 2024-09-18T11:32:52.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:52.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:52.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318765+0000", 2024-09-18T11:32:52.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.43e-06 2024-09-18T11:32:52.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:52.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318770+0000", 2024-09-18T11:32:52.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1479999999999997e-06 2024-09-18T11:32:52.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:52.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318773+0000", 2024-09-18T11:32:52.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.18e-06 2024-09-18T11:32:52.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.321043+0000", 2024-09-18T11:32:52.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022696819999999999 2024-09-18T11:32:52.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for blocked object", 2024-09-18T11:32:52.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.321104+0000", 2024-09-18T11:32:52.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1050999999999995e-05 2024-09-18T11:32:52.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:52.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331070+0000", 2024-09-18T11:32:52.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0099659610000000006 2024-09-18T11:32:52.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:52.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331176+0000", 2024-09-18T11:32:52.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010582 2024-09-18T11:32:52.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:52.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331377+0000", 2024-09-18T11:32:52.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000201208 2024-09-18T11:32:52.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:52.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332278+0000", 2024-09-18T11:32:52.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00090079999999999999 2024-09-18T11:32:52.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:52.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.334798+0000", 2024-09-18T11:32:52.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025200190000000001 2024-09-18T11:32:52.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:52.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.334833+0000", 2024-09-18T11:32:52.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5678e-05 2024-09-18T11:32:52.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:52.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:52.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:52.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.334863+0000", 2024-09-18T11:32:52.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9374e-05 2024-09-18T11:32:52.968 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.969 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:52.969 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.969 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:52.969 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:52.969 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:52.970 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:52.971 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:32:52.971 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:52.972 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:52.968+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:53.027 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:53.026+0000 7fcb682b1640 -1 osd.7 181 handle_osd_map injecting map crc failure 2024-09-18T11:32:53.031 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is unset 2024-09-18T11:32:53.032 INFO:tasks.rados.rados.0.smithi119.stdout:2036: done (2 left) 2024-09-18T11:32:53.032 INFO:tasks.rados.rados.0.smithi119.stdout:2037: set_chunk oid 31 target oid 2024-09-18T11:32:53.054 INFO:tasks.rados.rados.0.smithi119.stdout:2037: set_chunk oid 31 offset: 145920 length: 2177536 target oid 2f02c59c01a730562a65f21445c8af0e4a8aab3b969b8a3abf39e7b1bc84f918 offset: 0 snap -1 2024-09-18T11:32:53.054 INFO:tasks.rados.rados.0.smithi119.stdout:2038: read oid 30 snap -1 2024-09-18T11:32:53.054 INFO:tasks.rados.rados.0.smithi119.stdout:2038: expect (ObjNum 391 snap 63 seq_num 391) 2024-09-18T11:32:53.054 INFO:tasks.rados.rados.0.smithi119.stdout:2039: snap_create 2024-09-18T11:32:53.054 INFO:tasks.rados.rados.0.smithi119.stdout:2034: expect (ObjNum 383 snap 60 seq_num 383) 2024-09-18T11:32:53.071 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:53.069+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:53.116 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:53.116 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:53.116 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:53.116 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:53.124 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:32:53.208 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:53.207+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:53.254 INFO:tasks.rados.rados.0.smithi119.stdout:2037: finishing set_chunk to oid 31 2024-09-18T11:32:53.254 INFO:tasks.rados.rados.0.smithi119.stdout:Range is overlapped: oid 31 set_chunk returned error code -95 offset: 145920 length: 2177536 2024-09-18T11:32:53.254 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 31 v 0 (ObjNum 345 snap 51 seq_num 345) dirty exists 2024-09-18T11:32:53.254 INFO:tasks.rados.rados.0.smithi119.stdout:2038: expect (ObjNum 391 snap 63 seq_num 391) 2024-09-18T11:32:53.262 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:53.262 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:53.262 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:53.262 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:53.262 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:53.271 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:32:53.273 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:53.271+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:53.374 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:53.372+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:53.410 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:53.411 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:53.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:53.411 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:53.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.411 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1788 4.8 4:1898b15d:::smithi11939506-25:head [write 453822~797061] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:32:53.411 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:53.411 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 130.492239614, 2024-09-18T11:32:53.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094305159999999999, 2024-09-18T11:32:53.411 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.412 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.412 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.412 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.412 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.412 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1788 2024-09-18T11:32:53.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.412 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:53.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:32:53.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915811+0000", 2024-09-18T11:32:53.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:53.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916608+0000", 2024-09-18T11:32:53.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079635000000000003 2024-09-18T11:32:53.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916624+0000", 2024-09-18T11:32:53.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6735000000000002e-05 2024-09-18T11:32:53.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916630+0000", 2024-09-18T11:32:53.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5709999999999997e-06 2024-09-18T11:32:53.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916674+0000", 2024-09-18T11:32:53.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4540999999999999e-05 2024-09-18T11:32:53.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916751+0000", 2024-09-18T11:32:53.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6695999999999999e-05 2024-09-18T11:32:53.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:53.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916872+0000", 2024-09-18T11:32:53.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120648 2024-09-18T11:32:53.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:53.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.919076+0000", 2024-09-18T11:32:53.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022046829999999998 2024-09-18T11:32:53.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:53.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925151+0000", 2024-09-18T11:32:53.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060740239999999999 2024-09-18T11:32:53.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:53.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925198+0000", 2024-09-18T11:32:53.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7896999999999999e-05 2024-09-18T11:32:53.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.418 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925244+0000", 2024-09-18T11:32:53.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5398999999999997e-05 2024-09-18T11:32:53.418 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.418 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.418 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.418 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3963 4.3 4:c70c3162:::smithi11939506-35:head [tier-promote] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:53.418 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:32:53.418 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 44.032229692999998, 2024-09-18T11:32:53.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010675591999999999, 2024-09-18T11:32:53.418 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.419 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:53.419 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.419 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.419 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.419 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3963 2024-09-18T11:32:53.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.419 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:32:53.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:32:53.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375821+0000", 2024-09-18T11:32:53.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:53.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375829+0000", 2024-09-18T11:32:53.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7989999999999994e-06 2024-09-18T11:32:53.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375843+0000", 2024-09-18T11:32:53.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:32:53.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375849+0000", 2024-09-18T11:32:53.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1419999999999998e-06 2024-09-18T11:32:53.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375889+0000", 2024-09-18T11:32:53.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0197999999999997e-05 2024-09-18T11:32:53.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:53.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375894+0000", 2024-09-18T11:32:53.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0139999999999998e-06 2024-09-18T11:32:53.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.377699+0000", 2024-09-18T11:32:53.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001804688 2024-09-18T11:32:53.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.377776+0000", 2024-09-18T11:32:53.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7081000000000003e-05 2024-09-18T11:32:53.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.386499+0000", 2024-09-18T11:32:53.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008722746 2024-09-18T11:32:53.423 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.423 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.423 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.424 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4259 4.b 4:dba5bd39:::smithi11939506-17:head [write 685818~644423] snapc 69=[69,67,61,55,51,4e,4b,45,42,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:53.424 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.040083+0000", 2024-09-18T11:32:53.424 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.367969513999999, 2024-09-18T11:32:53.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015748635, 2024-09-18T11:32:53.424 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.424 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.424 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.424 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.424 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.424 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4259 2024-09-18T11:32:53.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.424 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.040083+0000", 2024-09-18T11:32:53.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.040083+0000", 2024-09-18T11:32:53.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.040081+0000", 2024-09-18T11:32:53.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:53.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041258+0000", 2024-09-18T11:32:53.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011766540000000001 2024-09-18T11:32:53.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041286+0000", 2024-09-18T11:32:53.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8087000000000001e-05 2024-09-18T11:32:53.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041362+0000", 2024-09-18T11:32:53.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6155999999999994e-05 2024-09-18T11:32:53.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.044285+0000", 2024-09-18T11:32:53.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002922437 2024-09-18T11:32:53.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.044360+0000", 2024-09-18T11:32:53.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5201e-05 2024-09-18T11:32:53.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:53.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.044489+0000", 2024-09-18T11:32:53.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012950599999999999 2024-09-18T11:32:53.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:53.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052904+0000", 2024-09-18T11:32:53.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084149910000000001 2024-09-18T11:32:53.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:53.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055695+0000", 2024-09-18T11:32:53.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027902040000000001 2024-09-18T11:32:53.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:53.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055765+0000", 2024-09-18T11:32:53.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0720000000000001e-05 2024-09-18T11:32:53.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055832+0000", 2024-09-18T11:32:53.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6736999999999995e-05 2024-09-18T11:32:53.430 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.430 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.430 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.431 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.5.0:195 4.b e156/152 ondisk, result = 0)", 2024-09-18T11:32:53.431 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.041863+0000", 2024-09-18T11:32:53.431 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.366189993999999, 2024-09-18T11:32:53.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010991236, 2024-09-18T11:32:53.431 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.431 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:53.431 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041863+0000", 2024-09-18T11:32:53.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041863+0000", 2024-09-18T11:32:53.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041862+0000", 2024-09-18T11:32:53.432 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:53.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.432 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.432 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041868+0000", 2024-09-18T11:32:53.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2979999999999997e-06 2024-09-18T11:32:53.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041879+0000", 2024-09-18T11:32:53.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0522e-05 2024-09-18T11:32:53.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.433 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.433 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041885+0000", 2024-09-18T11:32:53.433 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7830000000000004e-06 2024-09-18T11:32:53.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052841+0000", 2024-09-18T11:32:53.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010956604 2024-09-18T11:32:53.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.434 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.434 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052848+0000", 2024-09-18T11:32:53.434 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2249999999999997e-06 2024-09-18T11:32:53.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.435 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.435 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052854+0000", 2024-09-18T11:32:53.435 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6050000000000003e-06 2024-09-18T11:32:53.435 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.435 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.435 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.435 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.5.0:196 4.b e156/152 ondisk, result = 0)", 2024-09-18T11:32:53.435 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.042994+0000", 2024-09-18T11:32:53.435 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.365059332000001, 2024-09-18T11:32:53.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098926020000000003, 2024-09-18T11:32:53.436 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.436 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:53.436 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.042994+0000", 2024-09-18T11:32:53.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.436 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.436 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.042994+0000", 2024-09-18T11:32:53.436 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.042993+0000", 2024-09-18T11:32:53.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:53.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043000+0000", 2024-09-18T11:32:53.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.525e-06 2024-09-18T11:32:53.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.437 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.437 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043011+0000", 2024-09-18T11:32:53.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0335e-05 2024-09-18T11:32:53.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043015+0000", 2024-09-18T11:32:53.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2899999999999996e-06 2024-09-18T11:32:53.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052868+0000", 2024-09-18T11:32:53.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098528130000000002 2024-09-18T11:32:53.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052871+0000", 2024-09-18T11:32:53.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5080000000000001e-06 2024-09-18T11:32:53.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052886+0000", 2024-09-18T11:32:53.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4781e-05 2024-09-18T11:32:53.439 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.439 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.439 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.440 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.5.0:194 4.3 e156/67 ondisk, result = 0)", 2024-09-18T11:32:53.440 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.043892+0000", 2024-09-18T11:32:53.440 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.364160542, 2024-09-18T11:32:53.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010237263999999999, 2024-09-18T11:32:53.440 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.440 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:53.440 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043892+0000", 2024-09-18T11:32:53.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043892+0000", 2024-09-18T11:32:53.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043891+0000", 2024-09-18T11:32:53.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:53.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043898+0000", 2024-09-18T11:32:53.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1190000000000002e-06 2024-09-18T11:32:53.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043910+0000", 2024-09-18T11:32:53.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.29e-05 2024-09-18T11:32:53.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043916+0000", 2024-09-18T11:32:53.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9499999999999998e-06 2024-09-18T11:32:53.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052830+0000", 2024-09-18T11:32:53.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089137480000000009 2024-09-18T11:32:53.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052839+0000", 2024-09-18T11:32:53.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9560000000000003e-06 2024-09-18T11:32:53.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054130+0000", 2024-09-18T11:32:53.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012905639999999999 2024-09-18T11:32:53.444 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.444 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.444 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.445 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4260 4.b 4:dba5bd39:::smithi11939506-17:head [write 2069089~216729] snapc 69=[69,67,61,55,51,4e,4b,45,42,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:53.445 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045444+0000", 2024-09-18T11:32:53.445 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.362609020000001, 2024-09-18T11:32:53.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011166715000000001, 2024-09-18T11:32:53.445 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.445 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.445 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.445 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.445 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.445 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4260 2024-09-18T11:32:53.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.445 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045444+0000", 2024-09-18T11:32:53.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045444+0000", 2024-09-18T11:32:53.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045443+0000", 2024-09-18T11:32:53.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:53.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045649+0000", 2024-09-18T11:32:53.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020605499999999999 2024-09-18T11:32:53.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045662+0000", 2024-09-18T11:32:53.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3386e-05 2024-09-18T11:32:53.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045667+0000", 2024-09-18T11:32:53.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6750000000000001e-06 2024-09-18T11:32:53.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052921+0000", 2024-09-18T11:32:53.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007254613 2024-09-18T11:32:53.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052994+0000", 2024-09-18T11:32:53.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2721999999999999e-05 2024-09-18T11:32:53.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:53.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.053116+0000", 2024-09-18T11:32:53.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121972 2024-09-18T11:32:53.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:53.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055857+0000", 2024-09-18T11:32:53.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027414539999999999 2024-09-18T11:32:53.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:53.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056502+0000", 2024-09-18T11:32:53.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064412499999999995 2024-09-18T11:32:53.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:53.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056555+0000", 2024-09-18T11:32:53.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3467999999999997e-05 2024-09-18T11:32:53.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056611+0000", 2024-09-18T11:32:53.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5577e-05 2024-09-18T11:32:53.451 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.451 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.451 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.451 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4261 4.b 4:dba5bd39:::smithi11939506-17:head [setxattr _header (57) in=7b,truncate 2285818] snapc 69=[69,67,61,55,51,4e,4b,45,42,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:53.452 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045672+0000", 2024-09-18T11:32:53.452 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.362380922, 2024-09-18T11:32:53.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011289933, 2024-09-18T11:32:53.452 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.452 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.452 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.452 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.452 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.452 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4261 2024-09-18T11:32:53.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.452 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045672+0000", 2024-09-18T11:32:53.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045672+0000", 2024-09-18T11:32:53.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045671+0000", 2024-09-18T11:32:53.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:53.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045675+0000", 2024-09-18T11:32:53.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.563e-06 2024-09-18T11:32:53.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045681+0000", 2024-09-18T11:32:53.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5210000000000002e-06 2024-09-18T11:32:53.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045684+0000", 2024-09-18T11:32:53.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4149999999999999e-06 2024-09-18T11:32:53.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054143+0000", 2024-09-18T11:32:53.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084590810000000002 2024-09-18T11:32:53.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054214+0000", 2024-09-18T11:32:53.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1061000000000003e-05 2024-09-18T11:32:53.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:53.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054354+0000", 2024-09-18T11:32:53.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000140035 2024-09-18T11:32:53.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:53.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056208+0000", 2024-09-18T11:32:53.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018539229999999999 2024-09-18T11:32:53.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:53.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056885+0000", 2024-09-18T11:32:53.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00067688000000000004 2024-09-18T11:32:53.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:53.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056922+0000", 2024-09-18T11:32:53.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6693999999999998e-05 2024-09-18T11:32:53.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056962+0000", 2024-09-18T11:32:53.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0299000000000002e-05 2024-09-18T11:32:53.458 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.458 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.458 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.458 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4262 4.b 4:dba5bd39:::smithi11939506-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:53.458 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:32:53.458 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.36236478, 2024-09-18T11:32:53.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011764513000000001, 2024-09-18T11:32:53.459 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.459 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:53.459 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.459 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.459 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.459 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4262 2024-09-18T11:32:53.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.459 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:32:53.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:32:53.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:32:53.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:53.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045690+0000", 2024-09-18T11:32:53.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.745e-06 2024-09-18T11:32:53.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045697+0000", 2024-09-18T11:32:53.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7229999999999998e-06 2024-09-18T11:32:53.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045704+0000", 2024-09-18T11:32:53.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5719999999999999e-06 2024-09-18T11:32:53.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055555+0000", 2024-09-18T11:32:53.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098515430000000008 2024-09-18T11:32:53.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:53.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055675+0000", 2024-09-18T11:32:53.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012000799999999999 2024-09-18T11:32:53.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.057004+0000", 2024-09-18T11:32:53.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013286179999999999 2024-09-18T11:32:53.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.057357+0000", 2024-09-18T11:32:53.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00035355100000000002 2024-09-18T11:32:53.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.057453+0000", 2024-09-18T11:32:53.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5154000000000004e-05 2024-09-18T11:32:53.464 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.464 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.464 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.464 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4264 4.7 4:e0d95e10:::smithi11939506-22:head [read 0~3008486,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:53.464 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:32:53.464 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.362314792999999, 2024-09-18T11:32:53.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095333889999999998, 2024-09-18T11:32:53.464 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.464 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:53.465 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.465 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.465 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.465 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4264 2024-09-18T11:32:53.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.465 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:32:53.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:32:53.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:32:53.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:53.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045747+0000", 2024-09-18T11:32:53.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2469999999999997e-06 2024-09-18T11:32:53.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045755+0000", 2024-09-18T11:32:53.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1499999999999999e-06 2024-09-18T11:32:53.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045759+0000", 2024-09-18T11:32:53.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0849999999999999e-06 2024-09-18T11:32:53.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045803+0000", 2024-09-18T11:32:53.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3621000000000001e-05 2024-09-18T11:32:53.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:32:53.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045807+0000", 2024-09-18T11:32:53.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7290000000000002e-06 2024-09-18T11:32:53.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.053475+0000", 2024-09-18T11:32:53.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076680100000000003 2024-09-18T11:32:53.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.053644+0000", 2024-09-18T11:32:53.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016971299999999999 2024-09-18T11:32:53.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055272+0000", 2024-09-18T11:32:53.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001627245 2024-09-18T11:32:53.469 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.469 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.469 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.470 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4265 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~2163506] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:53.470 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045792+0000", 2024-09-18T11:32:53.470 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 26.362260902999999, 2024-09-18T11:32:53.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010085024, 2024-09-18T11:32:53.470 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.470 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:53.470 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.470 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.470 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.470 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4265 2024-09-18T11:32:53.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.470 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045792+0000", 2024-09-18T11:32:53.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045792+0000", 2024-09-18T11:32:53.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045791+0000", 2024-09-18T11:32:53.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:53.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045797+0000", 2024-09-18T11:32:53.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0209999999999996e-06 2024-09-18T11:32:53.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045805+0000", 2024-09-18T11:32:53.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9349999999999994e-06 2024-09-18T11:32:53.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045811+0000", 2024-09-18T11:32:53.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2240000000000003e-06 2024-09-18T11:32:53.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045858+0000", 2024-09-18T11:32:53.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7565999999999998e-05 2024-09-18T11:32:53.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:32:53.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045869+0000", 2024-09-18T11:32:53.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0852e-05 2024-09-18T11:32:53.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055337+0000", 2024-09-18T11:32:53.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094678729999999999 2024-09-18T11:32:53.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055417+0000", 2024-09-18T11:32:53.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0092000000000005e-05 2024-09-18T11:32:53.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055877+0000", 2024-09-18T11:32:53.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00046001800000000003 2024-09-18T11:32:53.475 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.475 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.475 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.475 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4349 4.b 4:d13decd8:::smithi11939506-2:head [read 0~2302625,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:32:53.475 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.382910+0000", 2024-09-18T11:32:53.475 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.025142839000001, 2024-09-18T11:32:53.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098226930000000004, 2024-09-18T11:32:53.475 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.475 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:53.475 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.476 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.476 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.476 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4349 2024-09-18T11:32:53.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.476 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382910+0000", 2024-09-18T11:32:53.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382910+0000", 2024-09-18T11:32:53.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382909+0000", 2024-09-18T11:32:53.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:53.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382915+0000", 2024-09-18T11:32:53.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6049999999999997e-06 2024-09-18T11:32:53.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382924+0000", 2024-09-18T11:32:53.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6289999999999999e-06 2024-09-18T11:32:53.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382929+0000", 2024-09-18T11:32:53.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0479999999999998e-06 2024-09-18T11:32:53.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383030+0000", 2024-09-18T11:32:53.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101346 2024-09-18T11:32:53.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:53.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383036+0000", 2024-09-18T11:32:53.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2720000000000003e-06 2024-09-18T11:32:53.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.390597+0000", 2024-09-18T11:32:53.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075606880000000003 2024-09-18T11:32:53.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.390696+0000", 2024-09-18T11:32:53.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8740000000000007e-05 2024-09-18T11:32:53.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392733+0000", 2024-09-18T11:32:53.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002037025 2024-09-18T11:32:53.480 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.480 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.480 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.481 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4350 4.b 4:d13decd8:::smithi11939506-2:head [sparse-read 0~1463453] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:32:53.481 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.382978+0000", 2024-09-18T11:32:53.481 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.025075275999999, 2024-09-18T11:32:53.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01020132, 2024-09-18T11:32:53.481 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.481 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:53.481 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.481 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.481 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.481 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4350 2024-09-18T11:32:53.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.481 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382978+0000", 2024-09-18T11:32:53.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382978+0000", 2024-09-18T11:32:53.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382976+0000", 2024-09-18T11:32:53.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:53.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382984+0000", 2024-09-18T11:32:53.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.328e-06 2024-09-18T11:32:53.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382994+0000", 2024-09-18T11:32:53.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0801999999999999e-05 2024-09-18T11:32:53.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383001+0000", 2024-09-18T11:32:53.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6390000000000003e-06 2024-09-18T11:32:53.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383138+0000", 2024-09-18T11:32:53.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013676899999999999 2024-09-18T11:32:53.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:53.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383144+0000", 2024-09-18T11:32:53.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6359999999999997e-06 2024-09-18T11:32:53.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392791+0000", 2024-09-18T11:32:53.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096461659999999994 2024-09-18T11:32:53.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392897+0000", 2024-09-18T11:32:53.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000106087 2024-09-18T11:32:53.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.393179+0000", 2024-09-18T11:32:53.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00028227500000000002 2024-09-18T11:32:53.485 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.486 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.486 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.486 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4375 4.b 4:dba5bd39:::smithi11939506-17:head [write 437081~638017,stat] snapc 6d=[6d,6c,69,67,61,55,51,4b,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:32:53.486 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.549471+0000", 2024-09-18T11:32:53.486 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.858581573999999, 2024-09-18T11:32:53.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097774179999999995, 2024-09-18T11:32:53.486 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.486 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.486 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.486 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.486 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.487 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4375 2024-09-18T11:32:53.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.487 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.549471+0000", 2024-09-18T11:32:53.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.549471+0000", 2024-09-18T11:32:53.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.549470+0000", 2024-09-18T11:32:53.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:53.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553463+0000", 2024-09-18T11:32:53.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039932520000000001 2024-09-18T11:32:53.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553478+0000", 2024-09-18T11:32:53.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4647999999999999e-05 2024-09-18T11:32:53.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553484+0000", 2024-09-18T11:32:53.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3799999999999999e-06 2024-09-18T11:32:53.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553536+0000", 2024-09-18T11:32:53.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1938999999999999e-05 2024-09-18T11:32:53.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553637+0000", 2024-09-18T11:32:53.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010074899999999999 2024-09-18T11:32:53.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:53.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553824+0000", 2024-09-18T11:32:53.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000187524 2024-09-18T11:32:53.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:53.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.556653+0000", 2024-09-18T11:32:53.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028285879999999999 2024-09-18T11:32:53.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:53.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.559176+0000", 2024-09-18T11:32:53.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025229369999999998 2024-09-18T11:32:53.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.493 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:53.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.559212+0000", 2024-09-18T11:32:53.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5750000000000002e-05 2024-09-18T11:32:53.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.493 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.559249+0000", 2024-09-18T11:32:53.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7020999999999997e-05 2024-09-18T11:32:53.493 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.493 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.493 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.494 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4430 4.b 4:d3ae8215:::smithi11939506-41:head [read 0~3832029,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T11:32:53.494 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:32:53.494 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.379542613000002, 2024-09-18T11:32:53.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010918034, 2024-09-18T11:32:53.494 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.494 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:53.494 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.494 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.494 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.494 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4430 2024-09-18T11:32:53.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.495 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:32:53.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:32:53.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:32:53.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:53.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028520+0000", 2024-09-18T11:32:53.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.005e-05 2024-09-18T11:32:53.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028538+0000", 2024-09-18T11:32:53.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8020999999999999e-05 2024-09-18T11:32:53.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028543+0000", 2024-09-18T11:32:53.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.485e-06 2024-09-18T11:32:53.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028634+0000", 2024-09-18T11:32:53.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1120999999999997e-05 2024-09-18T11:32:53.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:53.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028639+0000", 2024-09-18T11:32:53.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1989999999999997e-06 2024-09-18T11:32:53.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.036054+0000", 2024-09-18T11:32:53.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074147559999999998 2024-09-18T11:32:53.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.036158+0000", 2024-09-18T11:32:53.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103564 2024-09-18T11:32:53.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039428+0000", 2024-09-18T11:32:53.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032706390000000001 2024-09-18T11:32:53.499 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.499 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.499 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.499 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4431 4.b 4:d3ae8215:::smithi11939506-41:head [sparse-read 0~2481209] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T11:32:53.499 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:32:53.499 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.379503825, 2024-09-18T11:32:53.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011399179000000001, 2024-09-18T11:32:53.499 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.499 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:53.500 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.500 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.500 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.500 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4431 2024-09-18T11:32:53.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.500 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:32:53.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:32:53.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:32:53.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:53.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028552+0000", 2024-09-18T11:32:53.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5880000000000002e-06 2024-09-18T11:32:53.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028558+0000", 2024-09-18T11:32:53.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.84e-06 2024-09-18T11:32:53.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028562+0000", 2024-09-18T11:32:53.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6600000000000001e-06 2024-09-18T11:32:53.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028676+0000", 2024-09-18T11:32:53.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011408000000000001 2024-09-18T11:32:53.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:53.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028681+0000", 2024-09-18T11:32:53.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5350000000000004e-06 2024-09-18T11:32:53.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039458+0000", 2024-09-18T11:32:53.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010777231 2024-09-18T11:32:53.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.504 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039564+0000", 2024-09-18T11:32:53.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000105524 2024-09-18T11:32:53.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.504 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039948+0000", 2024-09-18T11:32:53.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00038430099999999998 2024-09-18T11:32:53.504 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.504 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.504 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.505 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4518 4.7 4:e0d95e10:::smithi11939506-22:head [rollback 6f] snapc 74=[74,73,71,6f,6c,69,67,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:32:53.505 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.055692+0000", 2024-09-18T11:32:53.505 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.352360564, 2024-09-18T11:32:53.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010093540999999999, 2024-09-18T11:32:53.505 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.505 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.505 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.505 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.506 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.506 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4518 2024-09-18T11:32:53.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.506 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055692+0000", 2024-09-18T11:32:53.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055692+0000", 2024-09-18T11:32:53.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055692+0000", 2024-09-18T11:32:53.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:53.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055697+0000", 2024-09-18T11:32:53.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.101e-06 2024-09-18T11:32:53.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055707+0000", 2024-09-18T11:32:53.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0316000000000001e-05 2024-09-18T11:32:53.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055713+0000", 2024-09-18T11:32:53.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8900000000000004e-06 2024-09-18T11:32:53.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057453+0000", 2024-09-18T11:32:53.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001739953 2024-09-18T11:32:53.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057537+0000", 2024-09-18T11:32:53.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4287000000000001e-05 2024-09-18T11:32:53.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:53.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057760+0000", 2024-09-18T11:32:53.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022294700000000001 2024-09-18T11:32:53.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:53.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065674+0000", 2024-09-18T11:32:53.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079140110000000003 2024-09-18T11:32:53.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:53.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065681+0000", 2024-09-18T11:32:53.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2289999999999997e-06 2024-09-18T11:32:53.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:53.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065734+0000", 2024-09-18T11:32:53.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3136000000000001e-05 2024-09-18T11:32:53.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065786+0000", 2024-09-18T11:32:53.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1419999999999999e-05 2024-09-18T11:32:53.512 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.512 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.512 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.512 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4519 4.7 4:e0d95e10:::smithi11939506-22:head [append 0~0] snapc 74=[74,73,71,6f,6c,69,67,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:32:53.512 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.055719+0000", 2024-09-18T11:32:53.512 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.352334402, 2024-09-18T11:32:53.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010210589000000001, 2024-09-18T11:32:53.513 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.513 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.513 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.513 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.513 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.513 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4519 2024-09-18T11:32:53.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.513 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055719+0000", 2024-09-18T11:32:53.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055719+0000", 2024-09-18T11:32:53.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055718+0000", 2024-09-18T11:32:53.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:53.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055721+0000", 2024-09-18T11:32:53.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7039999999999999e-06 2024-09-18T11:32:53.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055728+0000", 2024-09-18T11:32:53.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1620000000000002e-06 2024-09-18T11:32:53.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055732+0000", 2024-09-18T11:32:53.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8410000000000002e-06 2024-09-18T11:32:53.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.059646+0000", 2024-09-18T11:32:53.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039138489999999996 2024-09-18T11:32:53.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.059733+0000", 2024-09-18T11:32:53.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7163999999999994e-05 2024-09-18T11:32:53.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:53.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.059891+0000", 2024-09-18T11:32:53.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000158102 2024-09-18T11:32:53.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:53.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065798+0000", 2024-09-18T11:32:53.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059067649999999996 2024-09-18T11:32:53.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:53.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065840+0000", 2024-09-18T11:32:53.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2515999999999997e-05 2024-09-18T11:32:53.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:53.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065890+0000", 2024-09-18T11:32:53.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0055000000000001e-05 2024-09-18T11:32:53.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065929+0000", 2024-09-18T11:32:53.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8831000000000001e-05 2024-09-18T11:32:53.519 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.519 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.519 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.519 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4756 4.7 4:e301aefd:::smithi11939506-19:head [write 591712~786646,stat] snapc 7e=[7e,7c,7b,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e179)", 2024-09-18T11:32:53.519 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:50.036231+0000", 2024-09-18T11:32:53.519 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 3.3718217579999998, 2024-09-18T11:32:53.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010177099, 2024-09-18T11:32:53.519 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.519 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.519 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.520 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.520 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.520 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4756 2024-09-18T11:32:53.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.520 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.036231+0000", 2024-09-18T11:32:53.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.036231+0000", 2024-09-18T11:32:53.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.036229+0000", 2024-09-18T11:32:53.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:53.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.037165+0000", 2024-09-18T11:32:53.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00093574399999999996 2024-09-18T11:32:53.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.037183+0000", 2024-09-18T11:32:53.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7431999999999998e-05 2024-09-18T11:32:53.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.037189+0000", 2024-09-18T11:32:53.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4590000000000003e-06 2024-09-18T11:32:53.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.037243+0000", 2024-09-18T11:32:53.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4237999999999998e-05 2024-09-18T11:32:53.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:53.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.037249+0000", 2024-09-18T11:32:53.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.699e-06 2024-09-18T11:32:53.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.038725+0000", 2024-09-18T11:32:53.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014756420000000001 2024-09-18T11:32:53.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.038797+0000", 2024-09-18T11:32:53.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2446000000000004e-05 2024-09-18T11:32:53.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.524 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:53.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.038915+0000", 2024-09-18T11:32:53.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011769000000000001 2024-09-18T11:32:53.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.524 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:53.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.041337+0000", 2024-09-18T11:32:53.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024226339999999999 2024-09-18T11:32:53.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:53.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.046311+0000", 2024-09-18T11:32:53.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049738359999999997 2024-09-18T11:32:53.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:53.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.046359+0000", 2024-09-18T11:32:53.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7586999999999997e-05 2024-09-18T11:32:53.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.046408+0000", 2024-09-18T11:32:53.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9478000000000003e-05 2024-09-18T11:32:53.526 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.526 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.526 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.526 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:4790 3.2 e182/21 3:79129b11:::2f02c59c01a730562a65f21445c8af0e4a8aab3b969b8a3abf39e7b1bc84f918:head v 182'203, mlcod=176'201)", 2024-09-18T11:32:53.526 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:53.043420+0000", 2024-09-18T11:32:53.526 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.36463291399999997, 2024-09-18T11:32:53.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096982140000000001, 2024-09-18T11:32:53.526 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.526 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.526 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.043420+0000", 2024-09-18T11:32:53.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.043420+0000", 2024-09-18T11:32:53.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.043420+0000", 2024-09-18T11:32:53.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:53.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.046026+0000", 2024-09-18T11:32:53.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026064590000000002 2024-09-18T11:32:53.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.046040+0000", 2024-09-18T11:32:53.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.364e-05 2024-09-18T11:32:53.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.046045+0000", 2024-09-18T11:32:53.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6010000000000002e-06 2024-09-18T11:32:53.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.046080+0000", 2024-09-18T11:32:53.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4702000000000001e-05 2024-09-18T11:32:53.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.046095+0000", 2024-09-18T11:32:53.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5265e-05 2024-09-18T11:32:53.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.530 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:53.530 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.053085+0000", 2024-09-18T11:32:53.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0069894559999999998 2024-09-18T11:32:53.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.530 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.530 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.053118+0000", 2024-09-18T11:32:53.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3565999999999997e-05 2024-09-18T11:32:53.530 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.530 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.530 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.530 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.530 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.530 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:53.532 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:32:53.533 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:53.530+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:53.630 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:53.631 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:53.631 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:53.631 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:53.633 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:53.632+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:53.639 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:32:53.734 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:53.732+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:53.776 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:53.776 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:53.776 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:53.776 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:53.777 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:53.784 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:32:53.835 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:53.834+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:53.884 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:53.884 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:53.884 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:53.884 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:53.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.884 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:32:53.884 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:53.884 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 162.76547405100001, 2024-09-18T11:32:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:32:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:32:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.885 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:32:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:32:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.886 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:32:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:32:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:32:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:32:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:32:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:32:53.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:32:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:32:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:32:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:32:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.888 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:32:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:32:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:32:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:32:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:32:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:32:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.889 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:32:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 153.21244865, 2024-09-18T11:32:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:32:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.890 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:32:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:32:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:32:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:32:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:32:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:32:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:32:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:32:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:32:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:32:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:32:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.893 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:32:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:32:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:32:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:32:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.894 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2339 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.49499135799999, 2024-09-18T11:32:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011809992, 2024-09-18T11:32:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2339 2024-09-18T11:32:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:32:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:32:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:32:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4710000000000001e-06 2024-09-18T11:32:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386824+0000", 2024-09-18T11:32:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6104e-05 2024-09-18T11:32:53.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386836+0000", 2024-09-18T11:32:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1827999999999999e-05 2024-09-18T11:32:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.898 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388390+0000", 2024-09-18T11:32:53.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015539499999999999 2024-09-18T11:32:53.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:53.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388443+0000", 2024-09-18T11:32:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3785000000000001e-05 2024-09-18T11:32:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398238+0000", 2024-09-18T11:32:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097947989999999999 2024-09-18T11:32:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398315+0000", 2024-09-18T11:32:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6909999999999994e-05 2024-09-18T11:32:53.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398612+0000", 2024-09-18T11:32:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029671999999999999 2024-09-18T11:32:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2340 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.494948046, 2024-09-18T11:32:53.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012086448, 2024-09-18T11:32:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2340 2024-09-18T11:32:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:53.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:32:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386848+0000", 2024-09-18T11:32:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3069999999999998e-06 2024-09-18T11:32:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386855+0000", 2024-09-18T11:32:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0979999999999996e-06 2024-09-18T11:32:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386859+0000", 2024-09-18T11:32:53.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9360000000000001e-06 2024-09-18T11:32:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388466+0000", 2024-09-18T11:32:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001607154 2024-09-18T11:32:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388507+0000", 2024-09-18T11:32:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1093000000000001e-05 2024-09-18T11:32:53.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398647+0000", 2024-09-18T11:32:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010139686 2024-09-18T11:32:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398718+0000", 2024-09-18T11:32:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1138999999999998e-05 2024-09-18T11:32:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398931+0000", 2024-09-18T11:32:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021356899999999999 2024-09-18T11:32:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2341 4.5 4:aa575018:::smithi11939506-11:head [read 0~2487436,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:32:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.494929824, 2024-09-18T11:32:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013751648999999999, 2024-09-18T11:32:53.908 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2341 2024-09-18T11:32:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:53.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:32:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386866+0000", 2024-09-18T11:32:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4249999999999998e-06 2024-09-18T11:32:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386872+0000", 2024-09-18T11:32:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.003e-06 2024-09-18T11:32:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:32:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.507e-06 2024-09-18T11:32:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388520+0000", 2024-09-18T11:32:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001644146 2024-09-18T11:32:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388561+0000", 2024-09-18T11:32:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1486999999999998e-05 2024-09-18T11:32:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398963+0000", 2024-09-18T11:32:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010401353 2024-09-18T11:32:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.399046+0000", 2024-09-18T11:32:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3141999999999998e-05 2024-09-18T11:32:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.400615+0000", 2024-09-18T11:32:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015690960000000001 2024-09-18T11:32:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.913 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3105 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 3e=[3e,3d,3b,37,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-18T11:32:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 80.86358208, 2024-09-18T11:32:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011735906000000001, 2024-09-18T11:32:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3105 2024-09-18T11:32:53.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:32:53.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:53.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018212+0000", 2024-09-18T11:32:53.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6759999999999999e-06 2024-09-18T11:32:53.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018216+0000", 2024-09-18T11:32:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-06 2024-09-18T11:32:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018218+0000", 2024-09-18T11:32:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6110000000000001e-06 2024-09-18T11:32:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018246+0000", 2024-09-18T11:32:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7557000000000001e-05 2024-09-18T11:32:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018250+0000", 2024-09-18T11:32:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4299999999999999e-06 2024-09-18T11:32:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021364+0000", 2024-09-18T11:32:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031135389999999998 2024-09-18T11:32:53.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021494+0000", 2024-09-18T11:32:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000130333 2024-09-18T11:32:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.029947+0000", 2024-09-18T11:32:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084526150000000001 2024-09-18T11:32:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.919 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3324 4.5 4:a087a566:::smithi11939506-46:head [rollback 46] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 71.764851476999993, 2024-09-18T11:32:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010927767999999999, 2024-09-18T11:32:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.920 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3324 2024-09-18T11:32:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.921 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:32:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116941+0000", 2024-09-18T11:32:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116949+0000", 2024-09-18T11:32:53.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4660000000000002e-06 2024-09-18T11:32:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116960+0000", 2024-09-18T11:32:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1484999999999999e-05 2024-09-18T11:32:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116964+0000", 2024-09-18T11:32:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2810000000000002e-06 2024-09-18T11:32:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117053+0000", 2024-09-18T11:32:53.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8747999999999998e-05 2024-09-18T11:32:53.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:53.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117059+0000", 2024-09-18T11:32:53.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.246e-06 2024-09-18T11:32:53.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123604+0000", 2024-09-18T11:32:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065445319999999996 2024-09-18T11:32:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123663+0000", 2024-09-18T11:32:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8740999999999999e-05 2024-09-18T11:32:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123803+0000", 2024-09-18T11:32:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014049200000000001 2024-09-18T11:32:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127600+0000", 2024-09-18T11:32:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037971179999999999 2024-09-18T11:32:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127814+0000", 2024-09-18T11:32:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021333600000000001 2024-09-18T11:32:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127847+0000", 2024-09-18T11:32:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3188999999999997e-05 2024-09-18T11:32:53.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127869+0000", 2024-09-18T11:32:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2515e-05 2024-09-18T11:32:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3325 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:32:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 71.764804291000004, 2024-09-18T11:32:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181030999999999, 2024-09-18T11:32:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3325 2024-09-18T11:32:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:53.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:32:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116988+0000", 2024-09-18T11:32:53.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116991+0000", 2024-09-18T11:32:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9890000000000001e-06 2024-09-18T11:32:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116998+0000", 2024-09-18T11:32:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5309999999999998e-06 2024-09-18T11:32:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117005+0000", 2024-09-18T11:32:53.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6420000000000001e-06 2024-09-18T11:32:53.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117091+0000", 2024-09-18T11:32:53.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6835999999999993e-05 2024-09-18T11:32:53.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:53.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117097+0000", 2024-09-18T11:32:53.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:32:53.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124717+0000", 2024-09-18T11:32:53.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076204899999999997 2024-09-18T11:32:53.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124773+0000", 2024-09-18T11:32:53.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5470999999999998e-05 2024-09-18T11:32:53.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:53.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124877+0000", 2024-09-18T11:32:53.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010426500000000001 2024-09-18T11:32:53.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:53.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127901+0000", 2024-09-18T11:32:53.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030243449999999999 2024-09-18T11:32:53.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:53.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128115+0000", 2024-09-18T11:32:53.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021355300000000001 2024-09-18T11:32:53.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:53.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128145+0000", 2024-09-18T11:32:53.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0290999999999998e-05 2024-09-18T11:32:53.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128170+0000", 2024-09-18T11:32:53.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4811e-05 2024-09-18T11:32:53.935 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.935 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.935 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.935 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3474 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1872245~637761] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:32:53.935 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:53.936 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 67.438027614000006, 2024-09-18T11:32:53.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0092038580000000005, 2024-09-18T11:32:53.936 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.936 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.936 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.936 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.936 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.936 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3474 2024-09-18T11:32:53.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.936 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.936 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.936 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:53.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:32:53.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443764+0000", 2024-09-18T11:32:53.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:53.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444359+0000", 2024-09-18T11:32:53.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059519 2024-09-18T11:32:53.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444374+0000", 2024-09-18T11:32:53.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4949e-05 2024-09-18T11:32:53.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444382+0000", 2024-09-18T11:32:53.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1369999999999996e-06 2024-09-18T11:32:53.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444428+0000", 2024-09-18T11:32:53.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6783000000000003e-05 2024-09-18T11:32:53.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444507+0000", 2024-09-18T11:32:53.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8415000000000001e-05 2024-09-18T11:32:53.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:53.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444638+0000", 2024-09-18T11:32:53.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013129 2024-09-18T11:32:53.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:53.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.449059+0000", 2024-09-18T11:32:53.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004420719 2024-09-18T11:32:53.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:53.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452909+0000", 2024-09-18T11:32:53.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003850773 2024-09-18T11:32:53.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:53.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452944+0000", 2024-09-18T11:32:53.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-05 2024-09-18T11:32:53.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452969+0000", 2024-09-18T11:32:53.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5795999999999998e-05 2024-09-18T11:32:53.941 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.941 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.941 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.942 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4251 4.5 4:af4ba75a:::smithi11939506-37:head [delete] snapc 69=[69,67,61,55,51,4e,4c,4b,45,42,3e,3d] ondisk+write+known_if_redirected+full_force+supports_pool_eio e155)", 2024-09-18T11:32:53.942 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:26.039209+0000", 2024-09-18T11:32:53.942 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.842583594000001, 2024-09-18T11:32:53.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010258422, 2024-09-18T11:32:53.942 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.942 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.942 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.942 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.942 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.942 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4251 2024-09-18T11:32:53.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.942 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039209+0000", 2024-09-18T11:32:53.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039209+0000", 2024-09-18T11:32:53.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039208+0000", 2024-09-18T11:32:53.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:53.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039217+0000", 2024-09-18T11:32:53.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2650000000000002e-06 2024-09-18T11:32:53.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039236+0000", 2024-09-18T11:32:53.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8734000000000001e-05 2024-09-18T11:32:53.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039244+0000", 2024-09-18T11:32:53.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2900000000000002e-06 2024-09-18T11:32:53.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039289+0000", 2024-09-18T11:32:53.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4657999999999998e-05 2024-09-18T11:32:53.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:32:53.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039294+0000", 2024-09-18T11:32:53.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0719999999999999e-06 2024-09-18T11:32:53.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.042052+0000", 2024-09-18T11:32:53.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027585399999999999 2024-09-18T11:32:53.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.042162+0000", 2024-09-18T11:32:53.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011005699999999999 2024-09-18T11:32:53.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:53.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.042341+0000", 2024-09-18T11:32:53.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000178285 2024-09-18T11:32:53.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:53.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.044140+0000", 2024-09-18T11:32:53.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001799092 2024-09-18T11:32:53.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:53.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.049396+0000", 2024-09-18T11:32:53.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005256213 2024-09-18T11:32:53.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:53.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.049431+0000", 2024-09-18T11:32:53.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.536e-05 2024-09-18T11:32:53.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.049468+0000", 2024-09-18T11:32:53.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6548999999999999e-05 2024-09-18T11:32:53.948 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.948 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.948 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.948 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1241 4.d e169/157 4:b44229a1:::smithi11939506-42:51 v 169'384, mlcod=168'380)", 2024-09-18T11:32:53.949 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.053467+0000", 2024-09-18T11:32:53.949 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.828325767999999, 2024-09-18T11:32:53.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010417658, 2024-09-18T11:32:53.949 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.949 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.949 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053467+0000", 2024-09-18T11:32:53.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.949 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053467+0000", 2024-09-18T11:32:53.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053467+0000", 2024-09-18T11:32:53.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:53.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053474+0000", 2024-09-18T11:32:53.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9430000000000003e-06 2024-09-18T11:32:53.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053483+0000", 2024-09-18T11:32:53.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3819999999999993e-06 2024-09-18T11:32:53.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053494+0000", 2024-09-18T11:32:53.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0609e-05 2024-09-18T11:32:53.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053532+0000", 2024-09-18T11:32:53.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8405000000000002e-05 2024-09-18T11:32:53.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053554+0000", 2024-09-18T11:32:53.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1758000000000001e-05 2024-09-18T11:32:53.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:53.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.063852+0000", 2024-09-18T11:32:53.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010298185 2024-09-18T11:32:53.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.063885+0000", 2024-09-18T11:32:53.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2838e-05 2024-09-18T11:32:53.953 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.953 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.953 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.953 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4513 4.a 4:552523b6:::smithi11939506-38:73 [set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:32:53.953 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.054905+0000", 2024-09-18T11:32:53.953 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.826887891, 2024-09-18T11:32:53.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.014365329, 2024-09-18T11:32:53.953 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.953 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.953 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.954 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.954 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.954 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4513 2024-09-18T11:32:53.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.954 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054905+0000", 2024-09-18T11:32:53.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054905+0000", 2024-09-18T11:32:53.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054904+0000", 2024-09-18T11:32:53.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:53.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054912+0000", 2024-09-18T11:32:53.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4300000000000006e-06 2024-09-18T11:32:53.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054927+0000", 2024-09-18T11:32:53.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4588e-05 2024-09-18T11:32:53.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054933+0000", 2024-09-18T11:32:53.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9009999999999999e-06 2024-09-18T11:32:53.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054975+0000", 2024-09-18T11:32:53.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2588e-05 2024-09-18T11:32:53.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055079+0000", 2024-09-18T11:32:53.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103508 2024-09-18T11:32:53.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:32:53.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.064787+0000", 2024-09-18T11:32:53.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097086719999999998 2024-09-18T11:32:53.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:53.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.066147+0000", 2024-09-18T11:32:53.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013597660000000001 2024-09-18T11:32:53.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:53.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.069210+0000", 2024-09-18T11:32:53.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030630430000000001 2024-09-18T11:32:53.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:53.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.069240+0000", 2024-09-18T11:32:53.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9628000000000002e-05 2024-09-18T11:32:53.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.069271+0000", 2024-09-18T11:32:53.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0682000000000002e-05 2024-09-18T11:32:53.959 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.959 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.959 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.959 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1242 4.d e169/157 4:b9bb642b:::smithi11939506-27:67 v 169'386, mlcod=168'380)", 2024-09-18T11:32:53.959 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.054970+0000", 2024-09-18T11:32:53.959 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.826823162, 2024-09-18T11:32:53.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089404059999999997, 2024-09-18T11:32:53.959 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.959 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.960 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054970+0000", 2024-09-18T11:32:53.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054970+0000", 2024-09-18T11:32:53.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054969+0000", 2024-09-18T11:32:53.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:53.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054977+0000", 2024-09-18T11:32:53.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6240000000000002e-06 2024-09-18T11:32:53.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054987+0000", 2024-09-18T11:32:53.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0186e-05 2024-09-18T11:32:53.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054993+0000", 2024-09-18T11:32:53.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5160000000000002e-06 2024-09-18T11:32:53.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055563+0000", 2024-09-18T11:32:53.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00057030399999999999 2024-09-18T11:32:53.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055576+0000", 2024-09-18T11:32:53.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3366e-05 2024-09-18T11:32:53.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:53.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.063887+0000", 2024-09-18T11:32:53.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083112240000000007 2024-09-18T11:32:53.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.063910+0000", 2024-09-18T11:32:53.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2932999999999999e-05 2024-09-18T11:32:53.963 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.963 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.963 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.964 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4665 4.a 4:588debc2:::smithi11939506-18:head [rollback 71] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:32:53.964 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.316572+0000", 2024-09-18T11:32:53.964 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.5652209040000002, 2024-09-18T11:32:53.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012421599, 2024-09-18T11:32:53.964 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.964 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.964 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.964 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.964 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.964 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4665 2024-09-18T11:32:53.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.965 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316572+0000", 2024-09-18T11:32:53.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316572+0000", 2024-09-18T11:32:53.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316572+0000", 2024-09-18T11:32:53.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:53.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316574+0000", 2024-09-18T11:32:53.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3599999999999999e-06 2024-09-18T11:32:53.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316578+0000", 2024-09-18T11:32:53.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0840000000000002e-06 2024-09-18T11:32:53.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316581+0000", 2024-09-18T11:32:53.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8739999999999999e-06 2024-09-18T11:32:53.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318695+0000", 2024-09-18T11:32:53.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002113382 2024-09-18T11:32:53.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318805+0000", 2024-09-18T11:32:53.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000110511 2024-09-18T11:32:53.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:32:53.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.321575+0000", 2024-09-18T11:32:53.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027700730000000001 2024-09-18T11:32:53.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:53.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.324576+0000", 2024-09-18T11:32:53.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030004599999999999 2024-09-18T11:32:53.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:53.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.328914+0000", 2024-09-18T11:32:53.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043382159999999998 2024-09-18T11:32:53.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:53.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.328951+0000", 2024-09-18T11:32:53.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7036999999999998e-05 2024-09-18T11:32:53.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.328994+0000", 2024-09-18T11:32:53.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2979999999999998e-05 2024-09-18T11:32:53.970 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.970 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.970 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.970 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4666 4.a 4:588debc2:::smithi11939506-18:head [append 0~0] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:32:53.970 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.316584+0000", 2024-09-18T11:32:53.970 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 8.5652092740000008, 2024-09-18T11:32:53.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012596499000000001, 2024-09-18T11:32:53.970 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.970 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.970 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.970 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.970 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.971 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4666 2024-09-18T11:32:53.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.971 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316584+0000", 2024-09-18T11:32:53.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316584+0000", 2024-09-18T11:32:53.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316584+0000", 2024-09-18T11:32:53.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:53.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316585+0000", 2024-09-18T11:32:53.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8109999999999999e-06 2024-09-18T11:32:53.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316588+0000", 2024-09-18T11:32:53.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0769999999999999e-06 2024-09-18T11:32:53.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316591+0000", 2024-09-18T11:32:53.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1320000000000001e-06 2024-09-18T11:32:53.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.319126+0000", 2024-09-18T11:32:53.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002535266 2024-09-18T11:32:53.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for blocked object", 2024-09-18T11:32:53.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.319180+0000", 2024-09-18T11:32:53.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4457999999999999e-05 2024-09-18T11:32:53.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.323839+0000", 2024-09-18T11:32:53.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046581720000000004 2024-09-18T11:32:53.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.324024+0000", 2024-09-18T11:32:53.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018519599999999999 2024-09-18T11:32:53.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:32:53.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.324246+0000", 2024-09-18T11:32:53.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022207800000000001 2024-09-18T11:32:53.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:53.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.325067+0000", 2024-09-18T11:32:53.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00082141699999999996 2024-09-18T11:32:53.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:53.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.329117+0000", 2024-09-18T11:32:53.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040501210000000003 2024-09-18T11:32:53.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:53.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.329151+0000", 2024-09-18T11:32:53.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4133e-05 2024-09-18T11:32:53.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.329180+0000", 2024-09-18T11:32:53.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.887e-05 2024-09-18T11:32:53.976 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.976 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.976 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.977 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4754 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2739607,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:32:53.977 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:49.234755+0000", 2024-09-18T11:32:53.977 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 4.6470376150000003, 2024-09-18T11:32:53.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0093043659999999997, 2024-09-18T11:32:53.977 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.977 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:32:53.977 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:53.977 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:32:53.977 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:53.977 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4754 2024-09-18T11:32:53.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.978 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234755+0000", 2024-09-18T11:32:53.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234755+0000", 2024-09-18T11:32:53.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234755+0000", 2024-09-18T11:32:53.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:53.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234760+0000", 2024-09-18T11:32:53.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6380000000000003e-06 2024-09-18T11:32:53.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234766+0000", 2024-09-18T11:32:53.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0159999999999997e-06 2024-09-18T11:32:53.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234771+0000", 2024-09-18T11:32:53.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1399999999999999e-06 2024-09-18T11:32:53.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234846+0000", 2024-09-18T11:32:53.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5308999999999993e-05 2024-09-18T11:32:53.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:53.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234848+0000", 2024-09-18T11:32:53.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4260000000000002e-06 2024-09-18T11:32:53.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.241890+0000", 2024-09-18T11:32:53.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070411859999999996 2024-09-18T11:32:53.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.241983+0000", 2024-09-18T11:32:53.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3843000000000003e-05 2024-09-18T11:32:53.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.244060+0000", 2024-09-18T11:32:53.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020763679999999999 2024-09-18T11:32:53.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.982 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.982 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1293 4.d e182/157 4:b9bb642b:::smithi11939506-27:6d v 182'415, mlcod=182'411)", 2024-09-18T11:32:53.982 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:53.041205+0000", 2024-09-18T11:32:53.982 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.84058835200000004, 2024-09-18T11:32:53.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0091437189999999998, 2024-09-18T11:32:53.982 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.982 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.983 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041205+0000", 2024-09-18T11:32:53.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041205+0000", 2024-09-18T11:32:53.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041204+0000", 2024-09-18T11:32:53.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:53.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041210+0000", 2024-09-18T11:32:53.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3280000000000002e-06 2024-09-18T11:32:53.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041222+0000", 2024-09-18T11:32:53.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1686e-05 2024-09-18T11:32:53.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041227+0000", 2024-09-18T11:32:53.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2680000000000002e-06 2024-09-18T11:32:53.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041257+0000", 2024-09-18T11:32:53.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9869000000000002e-05 2024-09-18T11:32:53.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041279+0000", 2024-09-18T11:32:53.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1651999999999999e-05 2024-09-18T11:32:53.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:53.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.050305+0000", 2024-09-18T11:32:53.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090263770000000004 2024-09-18T11:32:53.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.050348+0000", 2024-09-18T11:32:53.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3173999999999997e-05 2024-09-18T11:32:53.986 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.986 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.987 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1294 4.d e182/157 4:b30010d9:::smithi11939506-43:7e v 182'417, mlcod=182'411)", 2024-09-18T11:32:53.987 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:53.041747+0000", 2024-09-18T11:32:53.987 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.84004611900000004, 2024-09-18T11:32:53.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011293531000000001, 2024-09-18T11:32:53.987 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.987 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.987 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041747+0000", 2024-09-18T11:32:53.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041747+0000", 2024-09-18T11:32:53.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041746+0000", 2024-09-18T11:32:53.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:53.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041753+0000", 2024-09-18T11:32:53.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0010000000000004e-06 2024-09-18T11:32:53.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041766+0000", 2024-09-18T11:32:53.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.291e-05 2024-09-18T11:32:53.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041772+0000", 2024-09-18T11:32:53.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8390000000000003e-06 2024-09-18T11:32:53.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041810+0000", 2024-09-18T11:32:53.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7864000000000002e-05 2024-09-18T11:32:53.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041828+0000", 2024-09-18T11:32:53.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8748000000000001e-05 2024-09-18T11:32:53.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:53.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.053002+0000", 2024-09-18T11:32:53.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011173819 2024-09-18T11:32:53.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.053041+0000", 2024-09-18T11:32:53.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8260000000000003e-05 2024-09-18T11:32:53.991 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.991 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.991 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.991 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.4.0:1048 4.e e182/22 4:78eefc4b:::smithi11939506-7:6d v 182'218, mlcod=173'215)", 2024-09-18T11:32:53.992 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:53.042298+0000", 2024-09-18T11:32:53.992 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.83949500600000004, 2024-09-18T11:32:53.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010780692999999999, 2024-09-18T11:32:53.992 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.992 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.992 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042298+0000", 2024-09-18T11:32:53.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042298+0000", 2024-09-18T11:32:53.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042297+0000", 2024-09-18T11:32:53.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:53.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042304+0000", 2024-09-18T11:32:53.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3249999999999996e-06 2024-09-18T11:32:53.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042316+0000", 2024-09-18T11:32:53.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2108e-05 2024-09-18T11:32:53.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042322+0000", 2024-09-18T11:32:53.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8490000000000002e-06 2024-09-18T11:32:53.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042678+0000", 2024-09-18T11:32:53.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00035671900000000002 2024-09-18T11:32:53.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042709+0000", 2024-09-18T11:32:53.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0518000000000002e-05 2024-09-18T11:32:53.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:53.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.053041+0000", 2024-09-18T11:32:53.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010331965 2024-09-18T11:32:53.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:53.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.053079+0000", 2024-09-18T11:32:53.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7805000000000001e-05 2024-09-18T11:32:53.996 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.996 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:53.996 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:53.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.996 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.4.0:1049 4.e e182/22 4:7410c0e0:::smithi11939506-47:6d v 182'220, mlcod=173'215)", 2024-09-18T11:32:53.996 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:53.042878+0000", 2024-09-18T11:32:53.996 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 0.83891474200000005, 2024-09-18T11:32:53.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012621894999999999, 2024-09-18T11:32:53.997 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:53.997 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:53.997 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:53.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:53.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042878+0000", 2024-09-18T11:32:53.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:53.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042878+0000", 2024-09-18T11:32:53.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:53.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:53.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042878+0000", 2024-09-18T11:32:53.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:53.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:53.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042883+0000", 2024-09-18T11:32:53.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7699999999999998e-06 2024-09-18T11:32:53.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:53.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042896+0000", 2024-09-18T11:32:53.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.296e-05 2024-09-18T11:32:53.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:53.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042902+0000", 2024-09-18T11:32:53.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4600000000000002e-06 2024-09-18T11:32:53.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:53.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.047389+0000", 2024-09-18T11:32:53.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0044867010000000001 2024-09-18T11:32:53.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:53.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:53.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:53.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.047411+0000", 2024-09-18T11:32:53.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2076e-05 2024-09-18T11:32:54.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:54.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.055464+0000", 2024-09-18T11:32:54.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080533370000000007 2024-09-18T11:32:54.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:54.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.055500+0000", 2024-09-18T11:32:54.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6251000000000002e-05 2024-09-18T11:32:54.000 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.000 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.000 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.001 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.001 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.001 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:54.002 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:32:54.003 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:54.001+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:54.028 INFO:tasks.rados.rados.0.smithi119.stdout:2034: done (4 left) 2024-09-18T11:32:54.029 INFO:tasks.rados.rados.0.smithi119.stdout:2035: done (3 left) 2024-09-18T11:32:54.029 INFO:tasks.rados.rados.0.smithi119.stdout:2037: done (2 left) 2024-09-18T11:32:54.029 INFO:tasks.rados.rados.0.smithi119.stdout:2038: done (1 left) 2024-09-18T11:32:54.029 INFO:tasks.rados.rados.0.smithi119.stdout:2039: done (0 left) 2024-09-18T11:32:54.029 INFO:tasks.rados.rados.0.smithi119.stdout:2040: read oid 50 snap 61 2024-09-18T11:32:54.029 INFO:tasks.rados.rados.0.smithi119.stdout:2040: expect (ObjNum 376 snap 58 seq_num 376) 2024-09-18T11:32:54.029 INFO:tasks.rados.rados.0.smithi119.stdout:2041: snap_create 2024-09-18T11:32:54.034 INFO:tasks.rados.rados.0.smithi119.stdout:2040: expect (ObjNum 376 snap 58 seq_num 376) 2024-09-18T11:32:54.105 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:54.103+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:54.142 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:54.142 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:54.142 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:32:54.142 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:54.151 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:32:54.235 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:54.234+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:54.289 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:54.289 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:32:54.289 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:32:54.289 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:32:54.289 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:54.297 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:32:54.338 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:54.337+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:54.351 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:54.350+0000 7fd27659f640 -1 osd.6 182 handle_osd_map injecting map crc failure 2024-09-18T11:32:54.407 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:54.406+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:54.437 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 185.27594594999999, 2024-09-18T11:32:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:32:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:54.437 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:32:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:32:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:54.439 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:32:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:32:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:32:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:32:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:32:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:32:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:32:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:32:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:32:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:32:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:54.441 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:32:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:32:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:32:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:32:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:32:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:32:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:32:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:32:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:32:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:32:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:54.443 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:54.444 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 185.27589101999999, 2024-09-18T11:32:54.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:32:54.444 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:54.444 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.444 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:54.444 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:54.444 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:54.444 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:32:54.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.444 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:54.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:54.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:54.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:54.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:54.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:54.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:32:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:32:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:32:54.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:32:54.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:32:54.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:32:54.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:54.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:32:54.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:32:54.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:32:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:32:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:32:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:32:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:32:54.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:32:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:32:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:32:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:32:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:32:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:32:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:32:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 185.27588104, 2024-09-18T11:32:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:32:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:32:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:32:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:32:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:32:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:32:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:32:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:32:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:32:54.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:32:54.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:54.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:32:54.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:32:54.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:54.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:32:54.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:32:54.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:54.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:32:54.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:32:54.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:54.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:32:54.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:32:54.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:32:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:32:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:32:54.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:32:54.458 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.458 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.458 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.458 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:54.458 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:54.458 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 185.275869441, 2024-09-18T11:32:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:32:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:32:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:54.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:32:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:32:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:32:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:32:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:32:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:32:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:32:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:32:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:32:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:32:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:32:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:32:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:32:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:32:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:54.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:32:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:32:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:32:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:32:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:32:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:32:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 185.27581296, 2024-09-18T11:32:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:32:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:54.465 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:32:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:32:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:32:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:32:54.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:32:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:32:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:32:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:32:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:54.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:32:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:32:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:32:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:32:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:32:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:32:54.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:32:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:32:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:32:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:32:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:32:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:32:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:32:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:32:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.471 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 185.27579913599999, 2024-09-18T11:32:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:32:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:54.472 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:32:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:32:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:32:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:32:54.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:32:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:32:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:32:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:32:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:32:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:32:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:32:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:32:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:32:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:32:54.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:32:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:32:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:32:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:32:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:32:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:32:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:32:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 184.859620229, 2024-09-18T11:32:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:32:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:54.478 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:32:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:54.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:32:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:32:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:32:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:32:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:32:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:32:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:32:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:32:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:32:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:32:54.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:32:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:32:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:32:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:32:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:32:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:32:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:32:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:32:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:54.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:32:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:32:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 184.85959692899999, 2024-09-18T11:32:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:32:54.485 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:32:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:32:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:54.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:32:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:32:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:32:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:32:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:32:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:32:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:32:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:32:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:32:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:32:54.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:54.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:32:54.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:32:54.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:54.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:32:54.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:32:54.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:54.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:32:54.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:32:54.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:54.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:32:54.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:32:54.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.493 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:54.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:32:54.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:32:54.493 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.494 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.494 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.494 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:54.494 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:54.494 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 184.85951132, 2024-09-18T11:32:54.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:32:54.495 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:54.495 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.495 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:54.495 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:54.495 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:54.495 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:32:54.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.496 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:54.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:54.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:54.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:54.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:54.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:54.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:54.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:54.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:54.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:32:54.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:32:54.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:54.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:32:54.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:32:54.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:32:54.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:32:54.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:54.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:32:54.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:32:54.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:54.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:32:54.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:32:54.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:54.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:32:54.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:32:54.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:54.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:32:54.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:32:54.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:54.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:32:54.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:32:54.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:54.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:32:54.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:32:54.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:54.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:32:54.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:32:54.503 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.503 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.503 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.503 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:54.503 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:54.503 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 184.85950035499999, 2024-09-18T11:32:54.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:32:54.504 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:54.504 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.504 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:54.504 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:54.504 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:54.504 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:32:54.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.504 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:54.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.504 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:54.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:54.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:54.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:54.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:54.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:54.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:54.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:54.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:32:54.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:32:54.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:54.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:32:54.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:32:54.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:32:54.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:32:54.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:54.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:32:54.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:32:54.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:54.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:32:54.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:32:54.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:54.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:32:54.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:32:54.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:54.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:32:54.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:32:54.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.508 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:54.508 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:32:54.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:32:54.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:54.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:32:54.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:32:54.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:54.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:32:54.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:32:54.509 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.509 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.510 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.510 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:54.510 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:54.510 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 184.85948918700001, 2024-09-18T11:32:54.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:32:54.510 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:54.510 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.510 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:54.510 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:54.510 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:54.511 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:32:54.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.511 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:54.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:54.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:54.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:54.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:54.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:54.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:54.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:54.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:54.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:32:54.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:32:54.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:54.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:32:54.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:32:54.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:32:54.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:32:54.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:54.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:32:54.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:32:54.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:54.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:32:54.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:32:54.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:54.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:32:54.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:32:54.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:54.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:32:54.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:32:54.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:54.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:32:54.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:32:54.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:54.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:32:54.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:32:54.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.519 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:54.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:32:54.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:32:54.519 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.519 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.519 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.519 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:54.520 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:54.520 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 184.85942385800001, 2024-09-18T11:32:54.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:32:54.520 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:54.520 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.520 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:54.520 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:54.520 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:54.520 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:32:54.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.520 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:54.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:54.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:54.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:54.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:54.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:54.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:32:54.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:54.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:54.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:32:54.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:32:54.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:54.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:32:54.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:32:54.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:32:54.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:32:54.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:54.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:32:54.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:32:54.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:54.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:32:54.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:32:54.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:54.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:32:54.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:32:54.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.524 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:54.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:32:54.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:32:54.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.524 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:54.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:32:54.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:32:54.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:54.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:32:54.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:32:54.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:54.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:32:54.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:32:54.525 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.525 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.525 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.526 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:54.526 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:54.526 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 184.84926188, 2024-09-18T11:32:54.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:32:54.526 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:54.526 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.526 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:54.526 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:54.526 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:54.526 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:32:54.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.526 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:54.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:54.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:54.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:54.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:54.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:54.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:32:54.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:54.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:54.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:32:54.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:32:54.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:54.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:32:54.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:32:54.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:32:54.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:32:54.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:54.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:32:54.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:32:54.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:54.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:32:54.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:32:54.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.530 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:54.530 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:32:54.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:32:54.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.530 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:54.530 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:32:54.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:32:54.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.530 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:54.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:32:54.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:32:54.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:54.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:32:54.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:32:54.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:54.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:32:54.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:32:54.531 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.532 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.532 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.532 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:54.532 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:54.532 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 184.84916940100001, 2024-09-18T11:32:54.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:32:54.532 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:54.532 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.532 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:54.532 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:54.533 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:54.533 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:32:54.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.533 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:54.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:54.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:54.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:54.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:54.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:54.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:54.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:54.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:54.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:32:54.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:32:54.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:54.535 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:32:54.535 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:32:54.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.535 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.535 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:32:54.535 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:32:54.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.535 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:54.535 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:32:54.535 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:32:54.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.536 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:54.536 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:32:54.536 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:32:54.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.536 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:54.536 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:32:54.536 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:32:54.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.536 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:54.536 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:32:54.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:32:54.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.537 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:54.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:32:54.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:32:54.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.537 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:54.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:32:54.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:32:54.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.538 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:54.538 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:32:54.538 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:32:54.538 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.538 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.538 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.538 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:123 4.3 e132/67 4:cc46532a:::smithi11939506-33:30 v 132'171, mlcod=128'168)", 2024-09-18T11:32:54.538 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:54.538 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 51.405862775999999, 2024-09-18T11:32:54.538 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087627039999999996, 2024-09-18T11:32:54.538 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:54.538 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.539 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:54.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:54.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:54.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:54.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:54.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:54.540 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027851+0000", 2024-09-18T11:32:54.540 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:54.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.540 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:54.540 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027871+0000", 2024-09-18T11:32:54.540 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9749e-05 2024-09-18T11:32:54.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.540 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:54.540 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027882+0000", 2024-09-18T11:32:54.540 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1242e-05 2024-09-18T11:32:54.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027889+0000", 2024-09-18T11:32:54.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7649999999999996e-06 2024-09-18T11:32:54.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:54.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027929+0000", 2024-09-18T11:32:54.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9984999999999997e-05 2024-09-18T11:32:54.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:54.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027951+0000", 2024-09-18T11:32:54.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2047000000000001e-05 2024-09-18T11:32:54.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.542 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:54.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.036565+0000", 2024-09-18T11:32:54.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086135110000000008 2024-09-18T11:32:54.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.542 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:54.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.036614+0000", 2024-09-18T11:32:54.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9747000000000003e-05 2024-09-18T11:32:54.542 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.542 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.543 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.543 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:124 4.3 e132/67 4:ca2b2309:::smithi11939506-44:30 v 132'173, mlcod=128'168)", 2024-09-18T11:32:54.543 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:54.543 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 51.405170794, 2024-09-18T11:32:54.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0088947309999999995, 2024-09-18T11:32:54.543 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:54.543 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.543 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:54.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:54.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:54.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:54.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:54.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:54.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028543+0000", 2024-09-18T11:32:54.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:54.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:54.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028551+0000", 2024-09-18T11:32:54.545 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8720000000000002e-06 2024-09-18T11:32:54.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.545 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:54.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028562+0000", 2024-09-18T11:32:54.545 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1039000000000001e-05 2024-09-18T11:32:54.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.545 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028569+0000", 2024-09-18T11:32:54.545 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0350000000000004e-06 2024-09-18T11:32:54.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.546 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:54.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028848+0000", 2024-09-18T11:32:54.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00027935600000000003 2024-09-18T11:32:54.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.546 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:54.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028868+0000", 2024-09-18T11:32:54.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9947999999999999e-05 2024-09-18T11:32:54.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.546 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:54.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037405+0000", 2024-09-18T11:32:54.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085369920000000002 2024-09-18T11:32:54.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.547 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:54.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037438+0000", 2024-09-18T11:32:54.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3244000000000002e-05 2024-09-18T11:32:54.547 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.547 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.547 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.547 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:154 4.3 e147/67 4:ca2b2309:::smithi11939506-44:5d v 147'216, mlcod=145'210)", 2024-09-18T11:32:54.547 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:54.548 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.096384344999997, 2024-09-18T11:32:54.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066667829999999999, 2024-09-18T11:32:54.548 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:54.548 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.548 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:54.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:54.548 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:54.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.548 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:54.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:54.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:54.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:54.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:54.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:54.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337335+0000", 2024-09-18T11:32:54.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1479999999999997e-06 2024-09-18T11:32:54.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:54.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337344+0000", 2024-09-18T11:32:54.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5640000000000002e-06 2024-09-18T11:32:54.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337349+0000", 2024-09-18T11:32:54.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9470000000000002e-06 2024-09-18T11:32:54.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:54.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337410+0000", 2024-09-18T11:32:54.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0448999999999997e-05 2024-09-18T11:32:54.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.551 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:54.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337425+0000", 2024-09-18T11:32:54.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.504e-05 2024-09-18T11:32:54.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.551 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:54.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.343972+0000", 2024-09-18T11:32:54.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065474169999999998 2024-09-18T11:32:54.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.552 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:54.552 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.343997+0000", 2024-09-18T11:32:54.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4672999999999999e-05 2024-09-18T11:32:54.552 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.552 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.552 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.552 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1238 4.6 e169/143 4:6f86ffcc:::smithi11939506-15:51 v 169'193, mlcod=166'187)", 2024-09-18T11:32:54.552 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.044089+0000", 2024-09-18T11:32:54.552 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.389625276, 2024-09-18T11:32:54.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072870620000000004, 2024-09-18T11:32:54.552 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:54.553 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.553 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:54.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:54.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044089+0000", 2024-09-18T11:32:54.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:54.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044089+0000", 2024-09-18T11:32:54.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.554 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:54.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044089+0000", 2024-09-18T11:32:54.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:54.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.554 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:54.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044097+0000", 2024-09-18T11:32:54.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9060000000000002e-06 2024-09-18T11:32:54.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.554 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:54.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044105+0000", 2024-09-18T11:32:54.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3590000000000008e-06 2024-09-18T11:32:54.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044112+0000", 2024-09-18T11:32:54.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0639999999999996e-06 2024-09-18T11:32:54.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:54.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044851+0000", 2024-09-18T11:32:54.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00073857800000000004 2024-09-18T11:32:54.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:54.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044871+0000", 2024-09-18T11:32:54.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0140999999999999e-05 2024-09-18T11:32:54.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.556 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:54.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.051338+0000", 2024-09-18T11:32:54.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064673760000000004 2024-09-18T11:32:54.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.556 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:54.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.051376+0000", 2024-09-18T11:32:54.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8222000000000001e-05 2024-09-18T11:32:54.556 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.557 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.557 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.557 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:281 4.3 e169/67 4:ca2b2309:::smithi11939506-44:5d v 169'279, mlcod=168'273)", 2024-09-18T11:32:54.557 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.047201+0000", 2024-09-18T11:32:54.557 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.386513141, 2024-09-18T11:32:54.557 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007110346, 2024-09-18T11:32:54.557 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:54.557 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.557 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:54.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.557 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:54.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047201+0000", 2024-09-18T11:32:54.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.558 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:54.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047201+0000", 2024-09-18T11:32:54.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.558 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:54.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047201+0000", 2024-09-18T11:32:54.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:54.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:54.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047206+0000", 2024-09-18T11:32:54.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7309999999999998e-06 2024-09-18T11:32:54.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:54.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047217+0000", 2024-09-18T11:32:54.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0577000000000001e-05 2024-09-18T11:32:54.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047221+0000", 2024-09-18T11:32:54.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0509999999999998e-06 2024-09-18T11:32:54.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:54.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.048977+0000", 2024-09-18T11:32:54.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017558179999999999 2024-09-18T11:32:54.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:54.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.048999+0000", 2024-09-18T11:32:54.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.23e-05 2024-09-18T11:32:54.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:54.560 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054275+0000", 2024-09-18T11:32:54.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0052754150000000003 2024-09-18T11:32:54.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.561 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:54.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054312+0000", 2024-09-18T11:32:54.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7163999999999999e-05 2024-09-18T11:32:54.561 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.561 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.561 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.561 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:4518 4.7 e169/155 4:e0d95e10:::smithi11939506-22:head v 169'394, mlcod=169'389)", 2024-09-18T11:32:54.561 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.057943+0000", 2024-09-18T11:32:54.562 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.375771890999999, 2024-09-18T11:32:54.562 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075088719999999998, 2024-09-18T11:32:54.562 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:54.562 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.562 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:54.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.562 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:54.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057943+0000", 2024-09-18T11:32:54.562 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.562 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:54.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057943+0000", 2024-09-18T11:32:54.562 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:54.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.563 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.563 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:54.563 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057942+0000", 2024-09-18T11:32:54.563 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:54.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.563 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.563 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:54.563 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057953+0000", 2024-09-18T11:32:54.563 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1399e-05 2024-09-18T11:32:54.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.563 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.563 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:54.563 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057969+0000", 2024-09-18T11:32:54.564 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5860999999999999e-05 2024-09-18T11:32:54.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.564 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.564 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.564 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057975+0000", 2024-09-18T11:32:54.564 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8610000000000003e-06 2024-09-18T11:32:54.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.564 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.564 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:54.564 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.058039+0000", 2024-09-18T11:32:54.564 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4313000000000006e-05 2024-09-18T11:32:54.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.564 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.564 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:54.565 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.058059+0000", 2024-09-18T11:32:54.565 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0205999999999999e-05 2024-09-18T11:32:54.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.565 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.565 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:54.565 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065410+0000", 2024-09-18T11:32:54.565 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073512020000000003 2024-09-18T11:32:54.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:54.565 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:54.565 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:54.565 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065451+0000", 2024-09-18T11:32:54.565 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1032999999999998e-05 2024-09-18T11:32:54.565 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.566 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.566 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.566 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:54.566 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:54.566 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:54.567 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:32:54.568 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:54.566+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:54.664 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:54.665 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:54.665 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:54.665 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:54.668 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:54.666+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:54.673 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:32:54.770 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:54.769+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:54.811 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:54.812 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:54.812 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:54.812 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:54.812 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:54.820 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:32:54.870 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:54.869+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:54.919 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:54.919 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:54.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:54.919 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:54.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.919 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4736 4.3 4:c70c3162:::smithi11939506-35:head [write 574905~711397,stat] snapc 7c=[7c,7b,79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:32:54.919 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.031772+0000", 2024-09-18T11:32:54.919 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.884710192, 2024-09-18T11:32:54.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0092375840000000001, 2024-09-18T11:32:54.919 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:54.919 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.920 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:54.920 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:54.920 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:54.920 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4736 2024-09-18T11:32:54.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.920 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:54.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:54.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.031772+0000", 2024-09-18T11:32:54.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:54.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.031772+0000", 2024-09-18T11:32:54.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:54.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.031771+0000", 2024-09-18T11:32:54.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:54.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:54.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032335+0000", 2024-09-18T11:32:54.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00056435800000000005 2024-09-18T11:32:54.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:54.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032349+0000", 2024-09-18T11:32:54.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4353e-05 2024-09-18T11:32:54.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032356+0000", 2024-09-18T11:32:54.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3550000000000001e-06 2024-09-18T11:32:54.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:54.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032487+0000", 2024-09-18T11:32:54.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013154900000000001 2024-09-18T11:32:54.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:32:54.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032495+0000", 2024-09-18T11:32:54.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7519999999999996e-06 2024-09-18T11:32:54.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:54.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.035010+0000", 2024-09-18T11:32:54.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025153530000000001 2024-09-18T11:32:54.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:54.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.035142+0000", 2024-09-18T11:32:54.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013133599999999999 2024-09-18T11:32:54.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:54.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.035330+0000", 2024-09-18T11:32:54.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018873499999999999 2024-09-18T11:32:54.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:54.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040882+0000", 2024-09-18T11:32:54.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055514689999999998 2024-09-18T11:32:54.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:54.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040917+0000", 2024-09-18T11:32:54.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5305999999999997e-05 2024-09-18T11:32:54.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:54.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040968+0000", 2024-09-18T11:32:54.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0720000000000002e-05 2024-09-18T11:32:54.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:54.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041010+0000", 2024-09-18T11:32:54.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1893e-05 2024-09-18T11:32:54.928 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.928 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:54.928 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.929 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4737 4.3 4:c70c3162:::smithi11939506-35:head [write 1885086~525140] snapc 7c=[7c,7b,79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:32:54.929 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.032373+0000", 2024-09-18T11:32:54.929 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.8841091399999996, 2024-09-18T11:32:54.929 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0088738949999999997, 2024-09-18T11:32:54.929 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:54.929 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.929 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:54.929 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:54.929 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:54.929 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4737 2024-09-18T11:32:54.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.930 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:54.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:54.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032373+0000", 2024-09-18T11:32:54.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:54.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032373+0000", 2024-09-18T11:32:54.930 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.930 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.930 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:54.930 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032372+0000", 2024-09-18T11:32:54.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:54.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:54.931 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032651+0000", 2024-09-18T11:32:54.931 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027865499999999999 2024-09-18T11:32:54.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.931 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:54.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032661+0000", 2024-09-18T11:32:54.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5440000000000004e-06 2024-09-18T11:32:54.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.932 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.932 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032665+0000", 2024-09-18T11:32:54.932 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9899999999999999e-06 2024-09-18T11:32:54.932 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.932 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:54.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032701+0000", 2024-09-18T11:32:54.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6587000000000001e-05 2024-09-18T11:32:54.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.933 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:32:54.933 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032708+0000", 2024-09-18T11:32:54.933 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.742e-06 2024-09-18T11:32:54.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:54.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.037089+0000", 2024-09-18T11:32:54.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043810639999999996 2024-09-18T11:32:54.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:54.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.037176+0000", 2024-09-18T11:32:54.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7266000000000001e-05 2024-09-18T11:32:54.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.934 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.934 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:54.934 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.037317+0000", 2024-09-18T11:32:54.934 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014054200000000001 2024-09-18T11:32:54.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:54.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041074+0000", 2024-09-18T11:32:54.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037575709999999999 2024-09-18T11:32:54.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.935 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:54.935 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041165+0000", 2024-09-18T11:32:54.935 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0785999999999994e-05 2024-09-18T11:32:54.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.935 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:54.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041208+0000", 2024-09-18T11:32:54.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2611999999999998e-05 2024-09-18T11:32:54.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.936 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.936 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:54.936 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041247+0000", 2024-09-18T11:32:54.936 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9538e-05 2024-09-18T11:32:54.936 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.936 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:54.936 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.937 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4738 4.3 4:c70c3162:::smithi11939506-35:head [write 2930189~44716] snapc 7c=[7c,7b,79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:32:54.937 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.032671+0000", 2024-09-18T11:32:54.937 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.8838116459999998, 2024-09-18T11:32:54.937 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086678190000000002, 2024-09-18T11:32:54.937 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:54.937 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.937 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:54.937 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:54.937 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:54.937 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4738 2024-09-18T11:32:54.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.937 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:54.937 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:54.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032671+0000", 2024-09-18T11:32:54.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.938 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:54.938 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032671+0000", 2024-09-18T11:32:54.938 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:54.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032670+0000", 2024-09-18T11:32:54.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:54.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:54.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032908+0000", 2024-09-18T11:32:54.939 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023724099999999999 2024-09-18T11:32:54.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.939 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.939 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:54.939 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032917+0000", 2024-09-18T11:32:54.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1719999999999996e-06 2024-09-18T11:32:54.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032920+0000", 2024-09-18T11:32:54.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9270000000000001e-06 2024-09-18T11:32:54.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:54.940 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:32:54.940 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9938e-05 2024-09-18T11:32:54.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.940 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.940 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:32:54.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032957+0000", 2024-09-18T11:32:54.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7789999999999996e-06 2024-09-18T11:32:54.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:54.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.038135+0000", 2024-09-18T11:32:54.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051778789999999998 2024-09-18T11:32:54.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.941 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:54.941 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.038219+0000", 2024-09-18T11:32:54.941 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4047000000000004e-05 2024-09-18T11:32:54.941 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.941 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:54.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.038413+0000", 2024-09-18T11:32:54.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019343600000000001 2024-09-18T11:32:54.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:54.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041104+0000", 2024-09-18T11:32:54.942 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026916190000000001 2024-09-18T11:32:54.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.942 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.942 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:54.942 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041256+0000", 2024-09-18T11:32:54.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015182100000000001 2024-09-18T11:32:54.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:54.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041300+0000", 2024-09-18T11:32:54.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3915999999999998e-05 2024-09-18T11:32:54.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.943 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.943 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:54.943 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041339+0000", 2024-09-18T11:32:54.943 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8510000000000002e-05 2024-09-18T11:32:54.943 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.943 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:54.943 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.943 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.944 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4739 4.3 4:c70c3162:::smithi11939506-35:head [setxattr _header (57) in=7b,truncate 2974905] snapc 7c=[7c,7b,79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:32:54.944 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.032924+0000", 2024-09-18T11:32:54.944 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.8835580189999996, 2024-09-18T11:32:54.944 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085165610000000006, 2024-09-18T11:32:54.944 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:54.944 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.944 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:54.944 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:54.944 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:54.944 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4739 2024-09-18T11:32:54.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.944 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:54.944 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:54.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032924+0000", 2024-09-18T11:32:54.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:54.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032924+0000", 2024-09-18T11:32:54.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.945 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.945 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:54.945 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032924+0000", 2024-09-18T11:32:54.945 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:54.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:54.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032926+0000", 2024-09-18T11:32:54.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0619999999999999e-06 2024-09-18T11:32:54.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:54.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032935+0000", 2024-09-18T11:32:54.946 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.371e-06 2024-09-18T11:32:54.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.946 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.946 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032937+0000", 2024-09-18T11:32:54.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6769999999999999e-06 2024-09-18T11:32:54.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:54.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032979+0000", 2024-09-18T11:32:54.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1810000000000001e-05 2024-09-18T11:32:54.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:32:54.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032984+0000", 2024-09-18T11:32:54.947 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1660000000000002e-06 2024-09-18T11:32:54.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.947 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:54.947 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039042+0000", 2024-09-18T11:32:54.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060573399999999996 2024-09-18T11:32:54.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:54.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039127+0000", 2024-09-18T11:32:54.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5494999999999994e-05 2024-09-18T11:32:54.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:54.948 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039293+0000", 2024-09-18T11:32:54.948 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016590999999999999 2024-09-18T11:32:54.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.948 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.948 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:54.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041146+0000", 2024-09-18T11:32:54.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018531369999999999 2024-09-18T11:32:54.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:54.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041345+0000", 2024-09-18T11:32:54.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000199218 2024-09-18T11:32:54.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.949 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:54.949 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041390+0000", 2024-09-18T11:32:54.949 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4892000000000003e-05 2024-09-18T11:32:54.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.949 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.950 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:54.950 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041441+0000", 2024-09-18T11:32:54.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0815000000000001e-05 2024-09-18T11:32:54.950 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.950 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:54.950 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.950 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.950 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4740 4.3 4:c70c3162:::smithi11939506-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:32:54.950 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:32:54.950 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.8835323580000001, 2024-09-18T11:32:54.950 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0087685269999999999, 2024-09-18T11:32:54.950 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:54.951 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:54.951 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:54.951 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:54.951 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:54.951 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4740 2024-09-18T11:32:54.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.951 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:54.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.951 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:54.951 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:32:54.951 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.951 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:54.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:32:54.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:54.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:32:54.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:54.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.952 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.952 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:54.952 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032952+0000", 2024-09-18T11:32:54.952 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.215e-06 2024-09-18T11:32:54.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:54.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032956+0000", 2024-09-18T11:32:54.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8850000000000001e-06 2024-09-18T11:32:54.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032958+0000", 2024-09-18T11:32:54.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1840000000000002e-06 2024-09-18T11:32:54.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.953 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.953 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:54.953 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032997+0000", 2024-09-18T11:32:54.953 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9161e-05 2024-09-18T11:32:54.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:32:54.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.033000+0000", 2024-09-18T11:32:54.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1030000000000002e-06 2024-09-18T11:32:54.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:54.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039536+0000", 2024-09-18T11:32:54.954 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006535201 2024-09-18T11:32:54.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.954 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:54.954 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039616+0000", 2024-09-18T11:32:54.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0684000000000002e-05 2024-09-18T11:32:54.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:54.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041536+0000", 2024-09-18T11:32:54.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019196440000000001 2024-09-18T11:32:54.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:54.955 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041610+0000", 2024-09-18T11:32:54.955 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4295999999999995e-05 2024-09-18T11:32:54.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.955 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:54.956 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041719+0000", 2024-09-18T11:32:54.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108465 2024-09-18T11:32:54.956 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.956 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:54.956 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.956 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.956 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(client.4590.0:4736 4.3 e178/177 ondisk, result = 0)", 2024-09-18T11:32:54.956 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.040857+0000", 2024-09-18T11:32:54.956 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.8756255209999999, 2024-09-18T11:32:54.956 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015965400000000001, 2024-09-18T11:32:54.956 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:54.957 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:54.957 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:54.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:54.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040857+0000", 2024-09-18T11:32:54.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:54.957 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040857+0000", 2024-09-18T11:32:54.957 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.957 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.957 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:54.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040856+0000", 2024-09-18T11:32:54.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:54.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:54.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040861+0000", 2024-09-18T11:32:54.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2989999999999996e-06 2024-09-18T11:32:54.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.958 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.958 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:54.958 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040872+0000", 2024-09-18T11:32:54.958 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0416e-05 2024-09-18T11:32:54.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040878+0000", 2024-09-18T11:32:54.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.049e-06 2024-09-18T11:32:54.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:54.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040904+0000", 2024-09-18T11:32:54.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5950999999999999e-05 2024-09-18T11:32:54.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.959 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:54.959 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040910+0000", 2024-09-18T11:32:54.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5420000000000002e-06 2024-09-18T11:32:54.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:54.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041017+0000", 2024-09-18T11:32:54.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010631 2024-09-18T11:32:54.960 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.960 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:54.960 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.960 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(osd.0.0:4 4.3 e178/177 ondisk, result = 0)", 2024-09-18T11:32:54.960 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.043400+0000", 2024-09-18T11:32:54.961 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.8730821070000001, 2024-09-18T11:32:54.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000155208, 2024-09-18T11:32:54.961 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:54.961 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:54.961 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:54.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:54.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.043400+0000", 2024-09-18T11:32:54.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:54.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.043400+0000", 2024-09-18T11:32:54.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:54.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.043400+0000", 2024-09-18T11:32:54.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:54.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:54.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.043404+0000", 2024-09-18T11:32:54.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.138e-06 2024-09-18T11:32:54.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:54.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.043413+0000", 2024-09-18T11:32:54.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3719999999999994e-06 2024-09-18T11:32:54.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.043417+0000", 2024-09-18T11:32:54.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.709e-06 2024-09-18T11:32:54.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:54.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.043446+0000", 2024-09-18T11:32:54.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8677999999999999e-05 2024-09-18T11:32:54.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:54.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.043452+0000", 2024-09-18T11:32:54.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8140000000000003e-06 2024-09-18T11:32:54.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:54.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.043556+0000", 2024-09-18T11:32:54.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103106 2024-09-18T11:32:54.964 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.964 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:54.964 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.965 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(osd.0.0:6 4.3 e178/177 ondisk, result = 0)", 2024-09-18T11:32:54.965 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.044547+0000", 2024-09-18T11:32:54.965 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.8719356149999999, 2024-09-18T11:32:54.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016644500000000001, 2024-09-18T11:32:54.965 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:54.965 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:54.965 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:54.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:54.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.044547+0000", 2024-09-18T11:32:54.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:54.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.044547+0000", 2024-09-18T11:32:54.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:54.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.044546+0000", 2024-09-18T11:32:54.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:54.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:54.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.044550+0000", 2024-09-18T11:32:54.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2300000000000002e-06 2024-09-18T11:32:54.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:54.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.044560+0000", 2024-09-18T11:32:54.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8500000000000006e-06 2024-09-18T11:32:54.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.044564+0000", 2024-09-18T11:32:54.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.743e-06 2024-09-18T11:32:54.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:54.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.044593+0000", 2024-09-18T11:32:54.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8717e-05 2024-09-18T11:32:54.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:54.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.044600+0000", 2024-09-18T11:32:54.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.267e-06 2024-09-18T11:32:54.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:54.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.044713+0000", 2024-09-18T11:32:54.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113248 2024-09-18T11:32:54.969 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.969 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:54.969 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.969 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:4756 4.7 e179/174 4:e301aefd:::smithi11939506-19:head v 179'405, mlcod=173'401)", 2024-09-18T11:32:54.969 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:50.039068+0000", 2024-09-18T11:32:54.969 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.877414259, 2024-09-18T11:32:54.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055292249999999996, 2024-09-18T11:32:54.970 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:54.970 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.970 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:54.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:54.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.039068+0000", 2024-09-18T11:32:54.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:54.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.039068+0000", 2024-09-18T11:32:54.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:54.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.039068+0000", 2024-09-18T11:32:54.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:54.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:54.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040038+0000", 2024-09-18T11:32:54.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096998500000000003 2024-09-18T11:32:54.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:54.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040053+0000", 2024-09-18T11:32:54.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5481999999999998e-05 2024-09-18T11:32:54.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040062+0000", 2024-09-18T11:32:54.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7080000000000003e-06 2024-09-18T11:32:54.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:54.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040109+0000", 2024-09-18T11:32:54.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6752000000000002e-05 2024-09-18T11:32:54.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:54.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040124+0000", 2024-09-18T11:32:54.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5715000000000001e-05 2024-09-18T11:32:54.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:54.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044568+0000", 2024-09-18T11:32:54.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044430600000000004 2024-09-18T11:32:54.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:54.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044597+0000", 2024-09-18T11:32:54.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9887999999999999e-05 2024-09-18T11:32:54.973 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.973 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:54.974 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.974 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:4757 4.7 e179/174 4:e301aefd:::smithi11939506-19:head v 179'406, mlcod=173'401)", 2024-09-18T11:32:54.974 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:50.044255+0000", 2024-09-18T11:32:54.974 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.8722276979999997, 2024-09-18T11:32:54.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002073166, 2024-09-18T11:32:54.974 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:54.974 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.974 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:54.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:54.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044255+0000", 2024-09-18T11:32:54.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:54.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044255+0000", 2024-09-18T11:32:54.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:54.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044254+0000", 2024-09-18T11:32:54.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:54.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:54.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044875+0000", 2024-09-18T11:32:54.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00062136400000000001 2024-09-18T11:32:54.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:54.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044886+0000", 2024-09-18T11:32:54.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0709e-05 2024-09-18T11:32:54.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044891+0000", 2024-09-18T11:32:54.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8019999999999999e-06 2024-09-18T11:32:54.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:54.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044911+0000", 2024-09-18T11:32:54.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0729000000000001e-05 2024-09-18T11:32:54.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:54.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044930+0000", 2024-09-18T11:32:54.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8890999999999999e-05 2024-09-18T11:32:54.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:54.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046295+0000", 2024-09-18T11:32:54.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013645420000000001 2024-09-18T11:32:54.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:54.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046328+0000", 2024-09-18T11:32:54.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3089000000000001e-05 2024-09-18T11:32:54.978 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.979 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:54.979 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.979 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:4758 4.7 e179/174 4:e301aefd:::smithi11939506-19:head v 179'407, mlcod=173'401)", 2024-09-18T11:32:54.979 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:50.046040+0000", 2024-09-18T11:32:54.979 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.8704420710000003, 2024-09-18T11:32:54.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00053415600000000004, 2024-09-18T11:32:54.979 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:54.979 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.979 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:54.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:54.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046040+0000", 2024-09-18T11:32:54.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:54.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046040+0000", 2024-09-18T11:32:54.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:54.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046040+0000", 2024-09-18T11:32:54.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:54.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:54.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046045+0000", 2024-09-18T11:32:54.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7620000000000003e-06 2024-09-18T11:32:54.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:54.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046052+0000", 2024-09-18T11:32:54.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4699999999999996e-06 2024-09-18T11:32:54.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046056+0000", 2024-09-18T11:32:54.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0380000000000001e-06 2024-09-18T11:32:54.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:54.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046083+0000", 2024-09-18T11:32:54.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6957999999999998e-05 2024-09-18T11:32:54.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:54.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046113+0000", 2024-09-18T11:32:54.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9901999999999999e-05 2024-09-18T11:32:54.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:54.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046555+0000", 2024-09-18T11:32:54.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00044173699999999999 2024-09-18T11:32:54.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:54.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046575+0000", 2024-09-18T11:32:54.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9885999999999999e-05 2024-09-18T11:32:54.983 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.983 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:54.983 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.984 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:4764 3.3 e179/174 3:ec4378e2:::259b372f30a46a47845d5d9c58303dcbeccaeb2db18e8ed0152f1c66341c4df2:head v 179'156, mlcod=172'154)", 2024-09-18T11:32:54.984 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:50.059675+0000", 2024-09-18T11:32:54.984 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.8568073460000001, 2024-09-18T11:32:54.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056090439999999997, 2024-09-18T11:32:54.984 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:54.984 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.984 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:54.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:54.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.059675+0000", 2024-09-18T11:32:54.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:54.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.059675+0000", 2024-09-18T11:32:54.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:54.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.059675+0000", 2024-09-18T11:32:54.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:54.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:54.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062252+0000", 2024-09-18T11:32:54.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025772859999999998 2024-09-18T11:32:54.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:54.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062265+0000", 2024-09-18T11:32:54.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3059e-05 2024-09-18T11:32:54.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062272+0000", 2024-09-18T11:32:54.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3589999999999996e-06 2024-09-18T11:32:54.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:54.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062317+0000", 2024-09-18T11:32:54.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5188999999999997e-05 2024-09-18T11:32:54.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:54.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062334+0000", 2024-09-18T11:32:54.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.73e-05 2024-09-18T11:32:54.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:54.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.065251+0000", 2024-09-18T11:32:54.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029169539999999998 2024-09-18T11:32:54.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:54.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.065284+0000", 2024-09-18T11:32:54.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3206e-05 2024-09-18T11:32:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.13:48 3.3 e179/174 3:ec4378e2:::259b372f30a46a47845d5d9c58303dcbeccaeb2db18e8ed0152f1c66341c4df2:head v 179'157, mlcod=173'155)", 2024-09-18T11:32:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:50.066418+0000", 2024-09-18T11:32:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 4.8500643889999999, 2024-09-18T11:32:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059943099999999999, 2024-09-18T11:32:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:54.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.066418+0000", 2024-09-18T11:32:54.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:54.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.066418+0000", 2024-09-18T11:32:54.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:54.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.066417+0000", 2024-09-18T11:32:54.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:54.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:54.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.066423+0000", 2024-09-18T11:32:54.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9279999999999998e-06 2024-09-18T11:32:54.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:54.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.066433+0000", 2024-09-18T11:32:54.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4190000000000008e-06 2024-09-18T11:32:54.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.066438+0000", 2024-09-18T11:32:54.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4600000000000002e-06 2024-09-18T11:32:54.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:54.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.066478+0000", 2024-09-18T11:32:54.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9895999999999999e-05 2024-09-18T11:32:54.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:54.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.066500+0000", 2024-09-18T11:32:54.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2347000000000001e-05 2024-09-18T11:32:54.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:54.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.066984+0000", 2024-09-18T11:32:54.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00048345799999999998 2024-09-18T11:32:54.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:54.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.067018+0000", 2024-09-18T11:32:54.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3640999999999999e-05 2024-09-18T11:32:54.992 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.992 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:54.992 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.993 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4792 4.6 4:67c1765a:::smithi11939506-30:head [sparse-read 0~1153827,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-18T11:32:54.993 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.053543+0000", 2024-09-18T11:32:54.993 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.8629396250000001, 2024-09-18T11:32:54.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003529337, 2024-09-18T11:32:54.993 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:54.993 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:54.993 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:54.993 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:54.993 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:54.993 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4792 2024-09-18T11:32:54.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.993 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:54.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:54.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053543+0000", 2024-09-18T11:32:54.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:54.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053543+0000", 2024-09-18T11:32:54.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:54.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053541+0000", 2024-09-18T11:32:54.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:54.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:54.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053550+0000", 2024-09-18T11:32:54.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2070000000000001e-06 2024-09-18T11:32:54.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:54.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053566+0000", 2024-09-18T11:32:54.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6098000000000001e-05 2024-09-18T11:32:54.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:54.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053575+0000", 2024-09-18T11:32:54.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3769999999999996e-06 2024-09-18T11:32:54.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:54.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053620+0000", 2024-09-18T11:32:54.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5404000000000001e-05 2024-09-18T11:32:54.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:32:54.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053634+0000", 2024-09-18T11:32:54.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4161e-05 2024-09-18T11:32:54.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:54.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.055502+0000", 2024-09-18T11:32:54.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018678810000000001 2024-09-18T11:32:54.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:54.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.055629+0000", 2024-09-18T11:32:54.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012652 2024-09-18T11:32:54.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:54.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057072+0000", 2024-09-18T11:32:54.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001443567 2024-09-18T11:32:54.997 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.997 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:54.997 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:54.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.997 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4793 4.6 4:67c1765a:::smithi11939506-30:head [sparse-read 0~1153827] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-18T11:32:54.998 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.053593+0000", 2024-09-18T11:32:54.998 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.862889759, 2024-09-18T11:32:54.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037961539999999999, 2024-09-18T11:32:54.998 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:54.998 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:54.998 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:54.998 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:54.998 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:54.998 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4793 2024-09-18T11:32:54.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.998 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:54.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:54.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053593+0000", 2024-09-18T11:32:54.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:54.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053593+0000", 2024-09-18T11:32:54.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:54.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:54.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053592+0000", 2024-09-18T11:32:54.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:54.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:54.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:54.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:54.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053595+0000", 2024-09-18T11:32:55.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2179999999999999e-06 2024-09-18T11:32:55.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:55.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053603+0000", 2024-09-18T11:32:55.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.576e-06 2024-09-18T11:32:55.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:55.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053607+0000", 2024-09-18T11:32:55.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2309999999999998e-06 2024-09-18T11:32:55.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:55.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053696+0000", 2024-09-18T11:32:55.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9130999999999998e-05 2024-09-18T11:32:55.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:55.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053702+0000", 2024-09-18T11:32:55.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6370000000000004e-06 2024-09-18T11:32:55.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:55.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057115+0000", 2024-09-18T11:32:55.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034128100000000001 2024-09-18T11:32:55.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:55.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057176+0000", 2024-09-18T11:32:55.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.143e-05 2024-09-18T11:32:55.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:55.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057389+0000", 2024-09-18T11:32:55.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021267700000000001 2024-09-18T11:32:55.002 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:55.002 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:55.002 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:55.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.003 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4794 4.6 4:67c1765a:::smithi11939506-30:head [sparse-read 0~1153827] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-18T11:32:55.003 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.053613+0000", 2024-09-18T11:32:55.003 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.8628695070000001, 2024-09-18T11:32:55.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040685349999999999, 2024-09-18T11:32:55.003 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:55.003 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:55.003 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:55.003 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:55.003 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:55.003 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4794 2024-09-18T11:32:55.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.003 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:55.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:55.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053613+0000", 2024-09-18T11:32:55.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:55.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:55.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053613+0000", 2024-09-18T11:32:55.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:55.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:55.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053612+0000", 2024-09-18T11:32:55.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:55.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:55.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053615+0000", 2024-09-18T11:32:55.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5730000000000002e-06 2024-09-18T11:32:55.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:55.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053628+0000", 2024-09-18T11:32:55.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3084e-05 2024-09-18T11:32:55.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:55.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053632+0000", 2024-09-18T11:32:55.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9620000000000004e-06 2024-09-18T11:32:55.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:55.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053731+0000", 2024-09-18T11:32:55.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8547000000000004e-05 2024-09-18T11:32:55.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:55.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053736+0000", 2024-09-18T11:32:55.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.767e-06 2024-09-18T11:32:55.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:55.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057461+0000", 2024-09-18T11:32:55.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037242780000000001 2024-09-18T11:32:55.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:55.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057538+0000", 2024-09-18T11:32:55.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7008999999999994e-05 2024-09-18T11:32:55.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:55.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057681+0000", 2024-09-18T11:32:55.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000143786 2024-09-18T11:32:55.007 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:55.007 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:55.008 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:55.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.008 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.0:332 4.7 e182/174 4:e0d95e10:::smithi11939506-22:74 v 182'411, mlcod=179'407)", 2024-09-18T11:32:55.008 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.053724+0000", 2024-09-18T11:32:55.008 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.862758779, 2024-09-18T11:32:55.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096596200000000001, 2024-09-18T11:32:55.008 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:55.008 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:55.008 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:55.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:55.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053724+0000", 2024-09-18T11:32:55.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:55.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:55.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053724+0000", 2024-09-18T11:32:55.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:55.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:55.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053723+0000", 2024-09-18T11:32:55.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:55.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:55.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053728+0000", 2024-09-18T11:32:55.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7060000000000003e-06 2024-09-18T11:32:55.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:55.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053737+0000", 2024-09-18T11:32:55.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9039999999999998e-06 2024-09-18T11:32:55.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:55.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053741+0000", 2024-09-18T11:32:55.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3889999999999998e-06 2024-09-18T11:32:55.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:55.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053786+0000", 2024-09-18T11:32:55.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4291e-05 2024-09-18T11:32:55.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:55.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053807+0000", 2024-09-18T11:32:55.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1389000000000001e-05 2024-09-18T11:32:55.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:55.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054661+0000", 2024-09-18T11:32:55.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00085419499999999998 2024-09-18T11:32:55.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:55.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054690+0000", 2024-09-18T11:32:55.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8462999999999998e-05 2024-09-18T11:32:55.012 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:55.012 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:55.012 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:55.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.012 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.0:333 4.7 e182/174 4:e301aefd:::smithi11939506-19:7e v 182'413, mlcod=179'407)", 2024-09-18T11:32:55.012 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.054168+0000", 2024-09-18T11:32:55.013 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.862314641, 2024-09-18T11:32:55.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00082222999999999997, 2024-09-18T11:32:55.013 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:55.013 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:55.013 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:55.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:55.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054168+0000", 2024-09-18T11:32:55.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:55.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:55.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054168+0000", 2024-09-18T11:32:55.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:55.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:55.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054167+0000", 2024-09-18T11:32:55.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:55.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:55.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054184+0000", 2024-09-18T11:32:55.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6906e-05 2024-09-18T11:32:55.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:55.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054193+0000", 2024-09-18T11:32:55.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1940000000000004e-06 2024-09-18T11:32:55.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:55.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054203+0000", 2024-09-18T11:32:55.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5429999999999999e-06 2024-09-18T11:32:55.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:55.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054356+0000", 2024-09-18T11:32:55.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015282499999999999 2024-09-18T11:32:55.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:55.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054368+0000", 2024-09-18T11:32:55.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2813000000000001e-05 2024-09-18T11:32:55.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:55.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054969+0000", 2024-09-18T11:32:55.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00060045300000000003 2024-09-18T11:32:55.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:55.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054990+0000", 2024-09-18T11:32:55.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1149e-05 2024-09-18T11:32:55.017 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:55.017 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:55.017 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:55.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.017 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.0:335 4.7 e182/174 4:e73ba68f:::smithi11939506-1:71 v 182'415, mlcod=182'411)", 2024-09-18T11:32:55.017 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.061501+0000", 2024-09-18T11:32:55.017 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.8549817019999999, 2024-09-18T11:32:55.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00069536999999999995, 2024-09-18T11:32:55.017 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:55.017 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:55.018 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:55.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:55.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.061501+0000", 2024-09-18T11:32:55.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:55.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:55.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.061501+0000", 2024-09-18T11:32:55.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:55.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:55.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.061500+0000", 2024-09-18T11:32:55.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:55.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:55.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.061506+0000", 2024-09-18T11:32:55.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3720000000000001e-06 2024-09-18T11:32:55.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:55.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.061516+0000", 2024-09-18T11:32:55.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.893e-06 2024-09-18T11:32:55.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:55.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.061522+0000", 2024-09-18T11:32:55.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5269999999999998e-06 2024-09-18T11:32:55.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:55.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.061565+0000", 2024-09-18T11:32:55.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2826e-05 2024-09-18T11:32:55.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:55.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.061586+0000", 2024-09-18T11:32:55.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1576000000000001e-05 2024-09-18T11:32:55.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:55.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.062170+0000", 2024-09-18T11:32:55.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00058380699999999999 2024-09-18T11:32:55.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:55.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.062196+0000", 2024-09-18T11:32:55.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6069000000000001e-05 2024-09-18T11:32:55.021 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:55.021 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:55.021 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:55.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.022 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(osd.0.0:10 4.3 e182/177 ondisk, result = 0)", 2024-09-18T11:32:55.022 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.063855+0000", 2024-09-18T11:32:55.022 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 1.8526278189999998, 2024-09-18T11:32:55.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016419399999999999, 2024-09-18T11:32:55.022 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:55.022 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:55.022 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:55.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:55.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.063855+0000", 2024-09-18T11:32:55.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:55.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:55.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.063855+0000", 2024-09-18T11:32:55.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:55.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:55.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.063854+0000", 2024-09-18T11:32:55.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:55.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:55.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.063857+0000", 2024-09-18T11:32:55.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1319999999999998e-06 2024-09-18T11:32:55.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:55.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.063867+0000", 2024-09-18T11:32:55.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5889999999999992e-06 2024-09-18T11:32:55.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:55.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.063870+0000", 2024-09-18T11:32:55.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4960000000000001e-06 2024-09-18T11:32:55.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:55.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.063904+0000", 2024-09-18T11:32:55.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3399999999999999e-05 2024-09-18T11:32:55.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:55.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.063912+0000", 2024-09-18T11:32:55.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0679999999999997e-06 2024-09-18T11:32:55.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:55.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:55.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:55.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.064019+0000", 2024-09-18T11:32:55.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107017 2024-09-18T11:32:55.025 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:55.025 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:55.025 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:55.026 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:55.026 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:55.026 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:55.026 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:32:55.028 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:55.025+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:55.028 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:32:55.029 INFO:tasks.rados.rados.0.smithi119.stdout:2040: done (1 left) 2024-09-18T11:32:55.029 INFO:tasks.rados.rados.0.smithi119.stdout:2041: done (0 left) 2024-09-18T11:32:55.030 INFO:tasks.rados.rados.0.smithi119.stdout:2042: write (excl) oid 49 current snap is 73 2024-09-18T11:32:55.030 INFO:tasks.rados.rados.0.smithi119.stdout:2042: seq_num 418 ranges {625326=537144,1944179=672467} 2024-09-18T11:32:55.036 INFO:tasks.rados.rados.0.smithi119.stdout:2042: writing smithi11939506-49 from 625326 to 1162470 tid 1 2024-09-18T11:32:55.044 INFO:tasks.rados.rados.0.smithi119.stdout:2042: writing smithi11939506-49 from 1944179 to 2616646 tid 2 2024-09-18T11:32:55.044 INFO:tasks.rados.rados.0.smithi119.stdout:2043: write oid 32 current snap is 73 2024-09-18T11:32:55.044 INFO:tasks.rados.rados.0.smithi119.stdout:2042: finishing write tid 1 to smithi11939506-49 2024-09-18T11:32:55.044 INFO:tasks.rados.rados.0.smithi119.stdout:2043: seq_num 419 ranges {642133=728746,2119014=532478} 2024-09-18T11:32:55.047 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T11:32:55.051 INFO:tasks.rados.rados.0.smithi119.stdout:2043: writing smithi11939506-32 from 642133 to 1370879 tid 1 2024-09-18T11:32:55.059 INFO:tasks.rados.rados.0.smithi119.stdout:2043: writing smithi11939506-32 from 2119014 to 2651492 tid 2 2024-09-18T11:32:55.059 INFO:tasks.rados.rados.0.smithi119.stdout:2044: read oid 39 snap -1 2024-09-18T11:32:55.060 INFO:tasks.rados.rados.0.smithi119.stdout:2044: expect deleted 2024-09-18T11:32:55.060 INFO:tasks.rados.rados.0.smithi119.stdout:2045: rollback oid 40 current snap is 73 2024-09-18T11:32:55.060 INFO:tasks.rados.rados.0.smithi119.stdout:rollback oid 40 to 72 2024-09-18T11:32:55.060 INFO:tasks.rados.rados.0.smithi119.stdout:2046: write oid 34 current snap is 73 2024-09-18T11:32:55.060 INFO:tasks.rados.rados.0.smithi119.stdout:2046: seq_num 420 ranges {658940=403995,1777496=676136,2950951=107989} 2024-09-18T11:32:55.065 INFO:tasks.rados.rados.0.smithi119.stdout:2046: writing smithi11939506-34 from 658940 to 1062935 tid 1 2024-09-18T11:32:55.073 INFO:tasks.rados.rados.0.smithi119.stdout:2046: writing smithi11939506-34 from 1777496 to 2453632 tid 2 2024-09-18T11:32:55.078 INFO:tasks.rados.rados.0.smithi119.stdout:2046: writing smithi11939506-34 from 2950951 to 3058940 tid 3 2024-09-18T11:32:55.078 INFO:tasks.rados.rados.0.smithi119.stdout:2047: read oid 25 snap -1 2024-09-18T11:32:55.078 INFO:tasks.rados.rados.0.smithi119.stdout:2047: expect (ObjNum 408 snap 67 seq_num 408) 2024-09-18T11:32:55.078 INFO:tasks.rados.rados.0.smithi119.stdout:2048: read oid 12 snap 68 2024-09-18T11:32:55.078 INFO:tasks.rados.rados.0.smithi119.stdout:2048: expect (ObjNum 405 snap 67 seq_num 405) 2024-09-18T11:32:55.078 INFO:tasks.rados.rados.0.smithi119.stdout:2049: read oid 9 snap 38 2024-09-18T11:32:55.078 INFO:tasks.rados.rados.0.smithi119.stdout:2049: expect (ObjNum 199 snap 23 seq_num 199) 2024-09-18T11:32:55.078 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 8 2024-09-18T11:32:55.078 INFO:tasks.rados.rados.0.smithi119.stdout:2042: finishing write tid 2 to smithi11939506-49 2024-09-18T11:32:55.078 INFO:tasks.rados.rados.0.smithi119.stdout:2042: finishing write tid 3 to smithi11939506-49 2024-09-18T11:32:55.078 INFO:tasks.rados.rados.0.smithi119.stdout:2042: finishing write tid 4 to smithi11939506-49 2024-09-18T11:32:55.078 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 49 v 628 (ObjNum 418 snap 73 seq_num 418) dirty exists 2024-09-18T11:32:55.078 INFO:tasks.rados.rados.0.smithi119.stdout:2042: left oid 49 (ObjNum 418 snap 73 seq_num 418) 2024-09-18T11:32:55.078 INFO:tasks.rados.rados.0.smithi119.stdout:2043: finishing write tid 1 to smithi11939506-32 2024-09-18T11:32:55.079 INFO:tasks.rados.rados.0.smithi119.stdout:2042: done (7 left) 2024-09-18T11:32:55.079 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 7 2024-09-18T11:32:55.079 INFO:tasks.rados.rados.0.smithi119.stdout:2043: finishing write tid 2 to smithi11939506-32 2024-09-18T11:32:55.079 INFO:tasks.rados.rados.0.smithi119.stdout:2043: finishing write tid 3 to smithi11939506-32 2024-09-18T11:32:55.079 INFO:tasks.rados.rados.0.smithi119.stdout:2045: finishing rollback tid 0 to smithi11939506-40 2024-09-18T11:32:55.079 INFO:tasks.rados.rados.0.smithi119.stdout:2045: finishing rollback tid 1 to smithi11939506-40 2024-09-18T11:32:55.079 INFO:tasks.rados.rados.0.smithi119.stdout:2044: done (6 left) 2024-09-18T11:32:55.079 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 6 2024-09-18T11:32:55.079 INFO:tasks.rados.rados.0.smithi119.stdout:2043: finishing write tid 4 to smithi11939506-32 2024-09-18T11:32:55.079 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 32 v 282 (ObjNum 419 snap 73 seq_num 419) dirty exists 2024-09-18T11:32:55.079 INFO:tasks.rados.rados.0.smithi119.stdout:2043: left oid 32 (ObjNum 419 snap 73 seq_num 419) 2024-09-18T11:32:55.079 INFO:tasks.rados.rados.0.smithi119.stdout:2045: finishing rollback tid 2 to smithi11939506-40 2024-09-18T11:32:55.079 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 40 v 286 (ObjNum 287 snap 41 seq_num 287) dirty exists 2024-09-18T11:32:55.079 INFO:tasks.rados.rados.0.smithi119.stdout:2046: finishing write tid 1 to smithi11939506-34 2024-09-18T11:32:55.079 INFO:tasks.rados.rados.0.smithi119.stdout:2043: done (5 left) 2024-09-18T11:32:55.080 INFO:tasks.rados.rados.0.smithi119.stdout:2045: done (4 left) 2024-09-18T11:32:55.080 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 4 2024-09-18T11:32:55.080 INFO:tasks.rados.rados.0.smithi119.stdout:2046: finishing write tid 2 to smithi11939506-34 2024-09-18T11:32:55.081 INFO:tasks.rados.rados.0.smithi119.stdout:2046: finishing write tid 3 to smithi11939506-34 2024-09-18T11:32:55.082 INFO:tasks.rados.rados.0.smithi119.stdout:2046: finishing write tid 4 to smithi11939506-34 2024-09-18T11:32:55.082 INFO:tasks.rados.rados.0.smithi119.stdout:2046: finishing write tid 5 to smithi11939506-34 2024-09-18T11:32:55.082 INFO:tasks.rados.rados.0.smithi119.stdout:update_object_version oid 34 v 422 (ObjNum 420 snap 73 seq_num 420) dirty exists 2024-09-18T11:32:55.083 INFO:tasks.rados.rados.0.smithi119.stdout:2046: left oid 34 (ObjNum 420 snap 73 seq_num 420) 2024-09-18T11:32:55.083 INFO:tasks.rados.rados.0.smithi119.stdout:2046: done (3 left) 2024-09-18T11:32:55.083 INFO:tasks.rados.rados.0.smithi119.stdout: waiting on 3 2024-09-18T11:32:55.084 INFO:tasks.rados.rados.0.smithi119.stdout:2048: expect (ObjNum 405 snap 67 seq_num 405) 2024-09-18T11:32:55.128 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:55.126+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:55.149 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:55.150 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:55.150 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:55.150 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:55.164 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:32:55.168 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:55.168 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:55.190 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:55.190 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:55.190 INFO:tasks.rados.rados.0.smithi119.stdout:2047: expect (ObjNum 408 snap 67 seq_num 408) 2024-09-18T11:32:55.206 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:55.207 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:55.223 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:55.223 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:55.228 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:55.227+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:55.239 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:55.239 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:55.255 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:55.255 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:55.269 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:55.269 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:55.270 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:55.271 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:55.279 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:32:55.286 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:55.286 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:55.288 INFO:tasks.rados.rados.0.smithi119.stdout:2049: expect (ObjNum 199 snap 23 seq_num 199) 2024-09-18T11:32:55.329 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:55.328+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:55.385 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:55.385 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:55.393 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:32:55.394 INFO:tasks.rados.rados.0.smithi119.stdout:2047: done (2 left) 2024-09-18T11:32:55.394 INFO:tasks.rados.rados.0.smithi119.stdout:2048: done (1 left) 2024-09-18T11:32:55.394 INFO:tasks.rados.rados.0.smithi119.stdout:2049: done (0 left) 2024-09-18T11:32:55.431 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:55.429+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:55.492 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:55.492 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:55.501 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:32:55.532 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:55.530+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:55.598 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:55.598 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:55.606 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:32:55.632 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:55.631+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:55.703 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:55.703 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:55.712 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:32:55.733 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:55.732+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:55.789 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-18T11:32:55.789 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T11:32:55.811 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:55.811 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:55.819 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:32:55.834 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:55.833+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:55.935 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:55.934+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:55.964 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:55.964 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:55.973 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:32:56.031 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is set 2024-09-18T11:32:56.036 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:56.035+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:56.074 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:56.074 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:56.083 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:32:56.137 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:56.136+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:56.191 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:56.191 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:56.200 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:32:56.238 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:56.236+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:56.339 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:56.339 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:56.339 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:56.338+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:56.347 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:32:56.440 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:56.438+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:56.485 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:56.486 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:56.494 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:32:56.541 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:56.538+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:56.592 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:56.592 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:56.600 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:32:56.642 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:56.640+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:56.697 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:56.697 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:56.706 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:32:56.749 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:56.748+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:56.804 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:56.805 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:56.813 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:32:56.854 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:56.853+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:56.944 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:56.943+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:56.952 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:56.952 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:56.961 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:32:57.033 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T11:32:57.045 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:57.044+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:57.048 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T11:32:57.099 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:57.099 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:57.108 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:32:57.129 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:57.128+0000 7f3155e8c640 -1 osd.4 185 handle_osd_map injecting map crc failure 2024-09-18T11:32:57.172 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:57.171+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:57.227 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:57.227 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:57.236 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:32:57.277 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:57.276+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:57.299 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:32:57.347 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:57.345+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:57.375 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:57.375 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:57.385 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:32:57.414 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:57.414 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:57.429 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:57.429 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:57.444 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:57.444 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:57.448 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:57.447+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:57.458 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:57.459 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:57.473 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:57.474 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:57.484 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:57.484 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:57.490 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:57.490 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:57.493 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:32:57.506 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:57.506 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:57.521 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:57.521 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:57.573 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:57.571+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:57.633 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:32:57.634 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:32:57.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:32:57.634 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:32:57.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.634 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:57.634 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:57.634 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 188.47344700900001, 2024-09-18T11:32:57.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:32:57.634 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:57.634 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:57.634 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:57.634 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:57.634 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:57.634 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:32:57.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.635 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:57.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.635 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:57.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:57.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.635 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:57.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:32:57.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.636 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:57.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:32:57.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:57.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.636 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:57.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:32:57.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:32:57.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.636 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:57.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:32:57.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:32:57.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:57.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:32:57.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:32:57.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:57.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:32:57.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:32:57.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:57.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:32:57.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:32:57.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:57.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:32:57.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:32:57.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:57.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:32:57.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:32:57.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.639 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:57.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:32:57.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:32:57.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.639 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:57.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:32:57.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:32:57.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.640 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:57.640 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:32:57.640 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:32:57.640 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.640 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:57.640 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.640 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:57.640 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:57.640 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 188.47339207900001, 2024-09-18T11:32:57.641 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:32:57.641 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:57.641 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:57.641 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:57.641 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:57.641 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:57.641 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:32:57.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.641 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:57.641 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.641 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:57.641 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:57.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:57.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:57.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:57.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:32:57.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:57.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:57.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:32:57.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:32:57.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:57.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:32:57.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:32:57.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:57.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:32:57.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:32:57.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.644 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:57.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:32:57.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:32:57.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.644 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:57.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:32:57.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:32:57.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.644 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:57.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:32:57.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:32:57.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.645 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:57.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:32:57.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:32:57.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.645 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:57.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:32:57.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:32:57.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.646 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:57.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:32:57.646 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:32:57.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.646 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:57.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:32:57.646 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:32:57.646 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.646 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:57.646 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.647 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:57.647 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:57.647 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 188.47338209899999, 2024-09-18T11:32:57.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:32:57.647 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:57.647 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:57.647 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:57.647 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:57.647 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:57.647 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:32:57.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.647 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:57.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:57.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:57.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:57.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:57.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:57.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:32:57.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:57.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.649 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:57.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:32:57.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:32:57.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.649 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:57.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:32:57.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:32:57.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.650 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:57.650 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:32:57.650 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:32:57.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.650 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:57.650 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:32:57.650 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:32:57.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.650 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:57.650 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:32:57.650 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:32:57.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.651 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:57.651 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:32:57.651 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:32:57.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.651 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:57.651 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:32:57.651 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:32:57.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.651 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:57.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:32:57.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:32:57.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:57.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:32:57.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:32:57.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:57.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:32:57.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:32:57.652 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.652 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:57.653 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.653 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:57.653 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:57.653 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 188.47337049999999, 2024-09-18T11:32:57.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:32:57.653 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:57.653 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:57.653 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:57.653 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:57.653 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:57.654 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:32:57.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.654 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:57.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:57.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:57.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:57.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:57.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:57.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:32:57.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:57.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:57.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:32:57.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:32:57.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:57.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:32:57.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:32:57.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.656 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:57.656 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:32:57.656 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:32:57.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.656 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:57.656 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:32:57.656 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:32:57.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.656 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:57.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:32:57.657 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:32:57.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.657 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:57.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:32:57.657 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:32:57.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.657 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:57.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:32:57.657 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:32:57.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:57.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:32:57.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:32:57.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:57.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:32:57.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:32:57.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:57.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:32:57.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:32:57.659 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.659 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:57.659 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.659 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:57.659 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:57.659 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 188.47331401899999, 2024-09-18T11:32:57.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:32:57.659 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:57.659 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:57.659 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:57.659 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:57.660 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:57.660 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:32:57.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.660 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:57.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:57.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:57.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:57.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:32:57.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:57.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:32:57.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:57.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:57.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:32:57.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:32:57.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:57.662 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:32:57.662 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:32:57.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.662 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:57.662 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:32:57.662 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:32:57.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.662 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:57.662 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:32:57.662 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:32:57.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.663 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:57.663 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:32:57.663 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:32:57.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.663 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.663 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:57.663 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:32:57.663 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:32:57.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.663 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.663 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:57.663 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:32:57.663 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:32:57.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:57.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:32:57.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:32:57.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:57.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:32:57.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:32:57.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:57.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:32:57.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:32:57.664 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.665 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:57.665 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.665 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:57.665 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:57.665 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 188.47330019500001, 2024-09-18T11:32:57.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:32:57.665 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:57.665 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:57.665 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:57.665 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:57.665 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:57.665 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:32:57.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.666 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:57.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:57.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:57.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:57.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:57.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.667 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:57.667 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:32:57.667 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:57.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.667 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.667 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:57.667 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:32:57.667 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:32:57.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.667 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.667 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:57.667 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:32:57.667 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:32:57.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.668 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:57.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:32:57.668 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:32:57.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.668 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:57.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:32:57.668 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:32:57.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.668 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:57.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:32:57.669 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:32:57.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.669 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.669 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:57.669 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:32:57.669 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:32:57.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.669 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.669 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:57.669 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:32:57.669 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:32:57.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.669 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.669 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:57.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:32:57.670 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:32:57.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.670 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:57.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:32:57.670 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:32:57.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.670 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:57.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:32:57.670 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:32:57.671 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.671 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:57.671 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.671 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:57.671 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:57.671 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 188.05712128799999, 2024-09-18T11:32:57.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:32:57.671 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:57.671 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:57.671 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:57.672 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:57.672 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:57.672 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:32:57.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.672 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:57.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:57.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:57.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:57.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:57.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.673 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.673 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:57.673 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:32:57.673 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:57.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.673 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.673 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:57.673 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:32:57.673 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:32:57.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.673 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.673 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:57.674 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:32:57.674 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:32:57.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.674 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:57.674 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:32:57.674 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:32:57.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.674 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:57.674 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:32:57.674 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:32:57.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:57.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:32:57.675 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:32:57.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:57.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:32:57.675 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:32:57.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:57.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:32:57.675 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:32:57.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.676 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:57.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:32:57.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:32:57.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.676 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:57.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:32:57.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:32:57.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.676 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:57.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:32:57.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:32:57.677 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.677 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:57.677 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.677 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:57.677 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:57.677 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 188.05709798800001, 2024-09-18T11:32:57.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:32:57.677 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:57.678 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:57.678 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:57.678 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:57.678 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:57.678 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:32:57.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.678 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:57.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.678 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:57.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:57.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.678 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:57.679 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:32:57.679 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.679 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:57.679 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:32:57.679 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:57.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.679 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:57.679 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:32:57.679 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:32:57.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.680 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:57.680 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:32:57.680 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:32:57.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.680 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:57.680 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:32:57.680 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:32:57.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.680 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:57.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:32:57.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:32:57.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.681 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:57.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:32:57.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:32:57.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.681 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:57.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:32:57.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:32:57.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.682 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:57.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:32:57.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:32:57.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.682 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:57.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:32:57.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:32:57.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.682 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:57.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:32:57.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:32:57.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.683 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:57.683 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:32:57.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:32:57.683 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.683 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:57.683 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.683 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:57.683 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:57.683 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 188.05701237900001, 2024-09-18T11:32:57.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:32:57.684 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:57.684 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:57.684 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:57.684 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:57.684 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:57.684 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:32:57.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.684 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:57.684 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.684 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:57.684 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:57.684 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.685 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.685 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:57.685 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:57.685 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.685 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.685 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:57.685 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:32:57.685 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:57.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.685 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.685 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:57.685 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:32:57.685 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:32:57.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:57.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:32:57.686 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:32:57.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:57.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:32:57.686 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:32:57.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.687 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:57.687 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:32:57.687 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:32:57.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.687 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:57.687 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:32:57.687 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:32:57.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.687 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:57.687 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:32:57.687 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:32:57.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.688 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:57.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:32:57.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:32:57.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.688 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:57.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:32:57.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:32:57.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.689 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:57.689 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:32:57.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:32:57.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.689 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:57.689 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:32:57.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:32:57.689 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.689 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:57.689 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.690 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:57.690 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:57.690 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 188.05700141400001, 2024-09-18T11:32:57.690 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:32:57.690 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:57.690 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:57.690 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:57.690 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:57.690 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:57.690 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:32:57.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.690 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:57.690 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.690 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:57.691 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:57.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.691 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:57.691 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:57.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.691 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:57.691 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:32:57.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:57.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.691 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:57.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:32:57.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:32:57.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:57.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:32:57.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:32:57.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:57.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:32:57.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:32:57.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.693 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:57.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:32:57.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:32:57.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.693 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:57.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:32:57.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:32:57.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.693 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:57.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:32:57.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:32:57.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.694 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:57.694 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:32:57.694 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:32:57.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.694 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:57.694 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:32:57.694 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:32:57.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.694 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:57.694 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:32:57.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:32:57.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:57.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:32:57.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:32:57.695 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.695 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:57.695 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.695 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:57.695 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:57.695 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 188.056990246, 2024-09-18T11:32:57.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:32:57.696 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:57.696 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:57.696 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:57.696 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:57.696 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:57.696 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:32:57.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.696 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:57.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.697 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:57.697 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:57.697 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.697 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:57.697 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:57.697 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.697 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:57.697 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:32:57.698 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:32:57.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.698 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.698 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:57.698 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:32:57.698 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:32:57.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.698 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.698 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:57.698 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:32:57.698 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:32:57.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.698 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.699 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:57.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:32:57.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:32:57.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.699 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.699 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:57.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:32:57.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:32:57.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.699 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.699 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:57.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:32:57.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:32:57.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.700 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:57.700 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:32:57.700 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:32:57.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.700 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:57.700 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:32:57.700 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:32:57.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.700 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:57.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:32:57.701 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:32:57.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.701 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:57.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:32:57.701 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:32:57.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.701 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:57.702 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:32:57.702 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:32:57.702 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.702 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:57.702 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.702 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.702 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:57.702 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:57.702 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 188.056924917, 2024-09-18T11:32:57.702 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:32:57.702 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:57.702 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:57.702 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:57.703 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:57.703 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:57.703 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:32:57.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.703 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:57.703 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.703 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:57.703 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:57.703 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.703 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.703 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:57.703 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:32:57.703 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.704 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.704 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:57.704 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:32:57.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:57.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.704 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.704 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:57.704 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:32:57.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:32:57.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.704 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.705 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:57.705 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:32:57.705 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:32:57.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.705 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:57.705 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:32:57.705 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:32:57.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.705 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:57.705 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:32:57.705 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:32:57.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:57.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:32:57.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:32:57.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:57.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:32:57.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:32:57.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:57.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:32:57.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:32:57.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:57.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:32:57.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:32:57.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:57.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:32:57.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:32:57.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.708 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:57.708 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:32:57.708 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:32:57.708 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.708 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:57.708 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.708 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.708 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:57.708 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:57.708 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 188.04676293899999, 2024-09-18T11:32:57.708 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:32:57.708 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:57.709 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:57.709 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:57.709 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:57.709 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:57.709 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:32:57.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.709 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:57.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.709 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:57.709 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:57.709 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.710 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:57.710 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:32:57.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.710 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:57.710 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:32:57.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:57.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.710 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:57.710 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:32:57.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:32:57.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:57.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:32:57.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:32:57.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:57.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:32:57.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:32:57.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:57.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:32:57.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:32:57.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.712 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:57.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:32:57.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:32:57.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.712 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:57.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:32:57.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:32:57.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:57.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:32:57.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:32:57.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:57.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:32:57.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:32:57.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:57.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:32:57.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:32:57.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.714 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:57.714 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:32:57.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:32:57.714 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.714 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:57.714 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.714 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:32:57.714 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:57.715 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 188.04667046, 2024-09-18T11:32:57.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:32:57.715 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:57.715 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:57.715 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:32:57.715 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:32:57.715 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:32:57.715 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:32:57.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.715 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:57.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.715 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:57.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:57.716 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.716 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:57.716 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:57.716 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.716 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:57.716 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:32:57.716 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:57.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.717 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:57.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:32:57.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:32:57.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.717 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:57.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:32:57.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:32:57.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.717 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:57.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:32:57.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:32:57.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:57.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:32:57.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:32:57.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:57.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:32:57.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:32:57.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:32:57.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:32:57.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:32:57.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:32:57.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:32:57.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:32:57.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:57.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:32:57.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:32:57.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.720 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:57.720 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:32:57.720 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:32:57.720 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.720 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.720 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:57.720 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:32:57.720 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:32:57.720 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.720 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:57.720 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.720 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.720 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.720 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:123 4.3 e132/67 4:cc46532a:::smithi11939506-33:30 v 132'171, mlcod=128'168)", 2024-09-18T11:32:57.720 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:57.721 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.603363835000003, 2024-09-18T11:32:57.721 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087627039999999996, 2024-09-18T11:32:57.721 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:57.721 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:57.721 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:57.721 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.721 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:57.721 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:57.721 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.721 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.721 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.721 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:57.721 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:32:57.721 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.722 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.722 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.722 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:57.722 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027851+0000", 2024-09-18T11:32:57.722 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:57.722 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.722 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.722 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:57.722 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027871+0000", 2024-09-18T11:32:57.722 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9749e-05 2024-09-18T11:32:57.722 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.722 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.722 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:57.722 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027882+0000", 2024-09-18T11:32:57.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1242e-05 2024-09-18T11:32:57.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.723 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:57.723 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027889+0000", 2024-09-18T11:32:57.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7649999999999996e-06 2024-09-18T11:32:57.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.723 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:57.723 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027929+0000", 2024-09-18T11:32:57.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9984999999999997e-05 2024-09-18T11:32:57.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.723 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:57.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027951+0000", 2024-09-18T11:32:57.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2047000000000001e-05 2024-09-18T11:32:57.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:57.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.036565+0000", 2024-09-18T11:32:57.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086135110000000008 2024-09-18T11:32:57.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:57.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.036614+0000", 2024-09-18T11:32:57.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9747000000000003e-05 2024-09-18T11:32:57.724 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.725 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:57.725 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.725 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:124 4.3 e132/67 4:ca2b2309:::smithi11939506-44:30 v 132'173, mlcod=128'168)", 2024-09-18T11:32:57.725 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:57.725 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.602671852999997, 2024-09-18T11:32:57.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0088947309999999995, 2024-09-18T11:32:57.725 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:57.725 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:57.725 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:57.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:57.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:57.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.726 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:57.726 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:32:57.726 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.726 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:57.726 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028543+0000", 2024-09-18T11:32:57.726 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:57.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.726 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:57.726 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028551+0000", 2024-09-18T11:32:57.727 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8720000000000002e-06 2024-09-18T11:32:57.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.727 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:57.727 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028562+0000", 2024-09-18T11:32:57.727 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1039000000000001e-05 2024-09-18T11:32:57.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.727 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:57.727 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028569+0000", 2024-09-18T11:32:57.727 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0350000000000004e-06 2024-09-18T11:32:57.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.727 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:57.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028848+0000", 2024-09-18T11:32:57.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00027935600000000003 2024-09-18T11:32:57.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.728 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:57.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028868+0000", 2024-09-18T11:32:57.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9947999999999999e-05 2024-09-18T11:32:57.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.728 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:57.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037405+0000", 2024-09-18T11:32:57.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085369920000000002 2024-09-18T11:32:57.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.729 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:57.729 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037438+0000", 2024-09-18T11:32:57.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3244000000000002e-05 2024-09-18T11:32:57.729 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.729 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:57.729 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.729 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:154 4.3 e147/67 4:ca2b2309:::smithi11939506-44:5d v 147'216, mlcod=145'210)", 2024-09-18T11:32:57.729 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:57.729 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 39.293885404000001, 2024-09-18T11:32:57.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066667829999999999, 2024-09-18T11:32:57.729 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:57.729 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:57.730 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:57.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.730 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:57.730 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:57.730 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.730 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:57.730 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:57.730 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.731 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:57.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:32:57.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:57.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.731 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:57.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337335+0000", 2024-09-18T11:32:57.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1479999999999997e-06 2024-09-18T11:32:57.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.731 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:57.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337344+0000", 2024-09-18T11:32:57.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5640000000000002e-06 2024-09-18T11:32:57.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.732 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:57.732 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337349+0000", 2024-09-18T11:32:57.732 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9470000000000002e-06 2024-09-18T11:32:57.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.732 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:57.732 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337410+0000", 2024-09-18T11:32:57.732 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0448999999999997e-05 2024-09-18T11:32:57.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.732 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:57.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337425+0000", 2024-09-18T11:32:57.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.504e-05 2024-09-18T11:32:57.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.733 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:57.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.343972+0000", 2024-09-18T11:32:57.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065474169999999998 2024-09-18T11:32:57.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.733 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:57.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.343997+0000", 2024-09-18T11:32:57.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4672999999999999e-05 2024-09-18T11:32:57.734 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.734 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:57.734 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.734 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1238 4.6 e169/143 4:6f86ffcc:::smithi11939506-15:51 v 169'193, mlcod=166'187)", 2024-09-18T11:32:57.734 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.044089+0000", 2024-09-18T11:32:57.734 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.587126335000001, 2024-09-18T11:32:57.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072870620000000004, 2024-09-18T11:32:57.734 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:57.734 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:57.734 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:57.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.734 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:57.735 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044089+0000", 2024-09-18T11:32:57.735 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.735 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.735 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.735 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:57.735 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044089+0000", 2024-09-18T11:32:57.735 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.735 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.735 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.735 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:57.735 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044089+0000", 2024-09-18T11:32:57.735 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:57.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.736 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:57.736 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044097+0000", 2024-09-18T11:32:57.736 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9060000000000002e-06 2024-09-18T11:32:57.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.736 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:57.736 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044105+0000", 2024-09-18T11:32:57.736 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3590000000000008e-06 2024-09-18T11:32:57.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:57.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044112+0000", 2024-09-18T11:32:57.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0639999999999996e-06 2024-09-18T11:32:57.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.737 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:57.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044851+0000", 2024-09-18T11:32:57.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00073857800000000004 2024-09-18T11:32:57.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.737 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.737 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:57.737 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044871+0000", 2024-09-18T11:32:57.737 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0140999999999999e-05 2024-09-18T11:32:57.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:57.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.051338+0000", 2024-09-18T11:32:57.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064673760000000004 2024-09-18T11:32:57.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.738 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:57.738 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.051376+0000", 2024-09-18T11:32:57.738 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8222000000000001e-05 2024-09-18T11:32:57.738 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.738 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:57.738 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.739 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.739 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:281 4.3 e169/67 4:ca2b2309:::smithi11939506-44:5d v 169'279, mlcod=168'273)", 2024-09-18T11:32:57.739 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.047201+0000", 2024-09-18T11:32:57.739 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.584014199999999, 2024-09-18T11:32:57.739 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007110346, 2024-09-18T11:32:57.739 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:57.739 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:57.739 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:57.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.740 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:57.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047201+0000", 2024-09-18T11:32:57.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.740 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:57.740 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047201+0000", 2024-09-18T11:32:57.740 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.740 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:57.741 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047201+0000", 2024-09-18T11:32:57.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:57.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.741 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:57.741 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047206+0000", 2024-09-18T11:32:57.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7309999999999998e-06 2024-09-18T11:32:57.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.741 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:57.741 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047217+0000", 2024-09-18T11:32:57.741 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0577000000000001e-05 2024-09-18T11:32:57.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:57.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047221+0000", 2024-09-18T11:32:57.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0509999999999998e-06 2024-09-18T11:32:57.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.742 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:57.742 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.048977+0000", 2024-09-18T11:32:57.742 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017558179999999999 2024-09-18T11:32:57.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:57.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.048999+0000", 2024-09-18T11:32:57.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.23e-05 2024-09-18T11:32:57.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:57.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054275+0000", 2024-09-18T11:32:57.743 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0052754150000000003 2024-09-18T11:32:57.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.743 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:57.743 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054312+0000", 2024-09-18T11:32:57.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7163999999999999e-05 2024-09-18T11:32:57.744 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.744 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:57.744 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.744 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:4518 4.7 e169/155 4:e0d95e10:::smithi11939506-22:head v 169'394, mlcod=169'389)", 2024-09-18T11:32:57.744 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.057943+0000", 2024-09-18T11:32:57.744 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.57327295, 2024-09-18T11:32:57.744 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075088719999999998, 2024-09-18T11:32:57.744 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:32:57.744 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:57.745 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:32:57.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:32:57.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057943+0000", 2024-09-18T11:32:57.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:32:57.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057943+0000", 2024-09-18T11:32:57.745 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:32:57.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.745 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:32:57.745 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057942+0000", 2024-09-18T11:32:57.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:32:57.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.746 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:32:57.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057953+0000", 2024-09-18T11:32:57.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1399e-05 2024-09-18T11:32:57.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.746 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:32:57.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057969+0000", 2024-09-18T11:32:57.746 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5860999999999999e-05 2024-09-18T11:32:57.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.746 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:32:57.746 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057975+0000", 2024-09-18T11:32:57.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8610000000000003e-06 2024-09-18T11:32:57.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:32:57.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.058039+0000", 2024-09-18T11:32:57.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4313000000000006e-05 2024-09-18T11:32:57.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.747 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:32:57.747 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.058059+0000", 2024-09-18T11:32:57.747 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0205999999999999e-05 2024-09-18T11:32:57.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:32:57.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065410+0000", 2024-09-18T11:32:57.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073512020000000003 2024-09-18T11:32:57.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:32:57.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:32:57.748 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:32:57.748 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065451+0000", 2024-09-18T11:32:57.748 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1032999999999998e-05 2024-09-18T11:32:57.748 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.748 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:57.748 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.748 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:32:57.749 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:32:57.749 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:32:57.750 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:57.748+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:57.751 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:32:57.849 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:57.849 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:57.849 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:57.850 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:57.851 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:57.849+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:57.858 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:32:57.951 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:57.949+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:57.956 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:57.956 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:57.956 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:57.956 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:57.956 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:57.965 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:32:58.029 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T11:32:58.033 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:58.032+0000 7fd27659f640 -1 osd.6 186 handle_osd_map injecting map crc failure 2024-09-18T11:32:58.045 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T11:32:58.052 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:58.051+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:58.054 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T11:32:58.116 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:58.117 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:58.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:58.117 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:58.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.117 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4736 4.3 4:c70c3162:::smithi11939506-35:head [write 574905~711397,stat] snapc 7c=[7c,7b,79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:32:58.117 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.031772+0000", 2024-09-18T11:32:58.117 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0822436680000003, 2024-09-18T11:32:58.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0092375840000000001, 2024-09-18T11:32:58.117 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.117 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:58.117 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.117 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.118 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.118 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4736 2024-09-18T11:32:58.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.118 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.031772+0000", 2024-09-18T11:32:58.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.031772+0000", 2024-09-18T11:32:58.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.031771+0000", 2024-09-18T11:32:58.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:58.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032335+0000", 2024-09-18T11:32:58.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00056435800000000005 2024-09-18T11:32:58.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032349+0000", 2024-09-18T11:32:58.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4353e-05 2024-09-18T11:32:58.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032356+0000", 2024-09-18T11:32:58.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3550000000000001e-06 2024-09-18T11:32:58.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032487+0000", 2024-09-18T11:32:58.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013154900000000001 2024-09-18T11:32:58.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:32:58.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032495+0000", 2024-09-18T11:32:58.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7519999999999996e-06 2024-09-18T11:32:58.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.035010+0000", 2024-09-18T11:32:58.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025153530000000001 2024-09-18T11:32:58.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.035142+0000", 2024-09-18T11:32:58.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013133599999999999 2024-09-18T11:32:58.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:58.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.035330+0000", 2024-09-18T11:32:58.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018873499999999999 2024-09-18T11:32:58.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:58.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040882+0000", 2024-09-18T11:32:58.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055514689999999998 2024-09-18T11:32:58.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:58.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040917+0000", 2024-09-18T11:32:58.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5305999999999997e-05 2024-09-18T11:32:58.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:58.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040968+0000", 2024-09-18T11:32:58.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0720000000000002e-05 2024-09-18T11:32:58.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041010+0000", 2024-09-18T11:32:58.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1893e-05 2024-09-18T11:32:58.123 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.123 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.124 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.124 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4737 4.3 4:c70c3162:::smithi11939506-35:head [write 1885086~525140] snapc 7c=[7c,7b,79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:32:58.124 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.032373+0000", 2024-09-18T11:32:58.124 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0816426159999999, 2024-09-18T11:32:58.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0088738949999999997, 2024-09-18T11:32:58.124 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.124 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:58.124 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.124 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.124 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.124 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4737 2024-09-18T11:32:58.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.125 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032373+0000", 2024-09-18T11:32:58.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032373+0000", 2024-09-18T11:32:58.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032372+0000", 2024-09-18T11:32:58.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:58.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032651+0000", 2024-09-18T11:32:58.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027865499999999999 2024-09-18T11:32:58.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032661+0000", 2024-09-18T11:32:58.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5440000000000004e-06 2024-09-18T11:32:58.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032665+0000", 2024-09-18T11:32:58.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9899999999999999e-06 2024-09-18T11:32:58.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032701+0000", 2024-09-18T11:32:58.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6587000000000001e-05 2024-09-18T11:32:58.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:32:58.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032708+0000", 2024-09-18T11:32:58.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.742e-06 2024-09-18T11:32:58.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.037089+0000", 2024-09-18T11:32:58.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043810639999999996 2024-09-18T11:32:58.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.037176+0000", 2024-09-18T11:32:58.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7266000000000001e-05 2024-09-18T11:32:58.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:58.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.037317+0000", 2024-09-18T11:32:58.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014054200000000001 2024-09-18T11:32:58.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:58.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041074+0000", 2024-09-18T11:32:58.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037575709999999999 2024-09-18T11:32:58.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:58.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041165+0000", 2024-09-18T11:32:58.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0785999999999994e-05 2024-09-18T11:32:58.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:58.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041208+0000", 2024-09-18T11:32:58.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2611999999999998e-05 2024-09-18T11:32:58.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041247+0000", 2024-09-18T11:32:58.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9538e-05 2024-09-18T11:32:58.131 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.131 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.131 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.131 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4738 4.3 4:c70c3162:::smithi11939506-35:head [write 2930189~44716] snapc 7c=[7c,7b,79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:32:58.131 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.032671+0000", 2024-09-18T11:32:58.131 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0813451220000001, 2024-09-18T11:32:58.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086678190000000002, 2024-09-18T11:32:58.131 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.131 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:58.131 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.131 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.132 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.132 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4738 2024-09-18T11:32:58.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.132 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032671+0000", 2024-09-18T11:32:58.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032671+0000", 2024-09-18T11:32:58.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032670+0000", 2024-09-18T11:32:58.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032908+0000", 2024-09-18T11:32:58.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023724099999999999 2024-09-18T11:32:58.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032917+0000", 2024-09-18T11:32:58.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1719999999999996e-06 2024-09-18T11:32:58.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032920+0000", 2024-09-18T11:32:58.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9270000000000001e-06 2024-09-18T11:32:58.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:32:58.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9938e-05 2024-09-18T11:32:58.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:32:58.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032957+0000", 2024-09-18T11:32:58.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7789999999999996e-06 2024-09-18T11:32:58.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.038135+0000", 2024-09-18T11:32:58.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051778789999999998 2024-09-18T11:32:58.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.038219+0000", 2024-09-18T11:32:58.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4047000000000004e-05 2024-09-18T11:32:58.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:58.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.038413+0000", 2024-09-18T11:32:58.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019343600000000001 2024-09-18T11:32:58.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:58.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041104+0000", 2024-09-18T11:32:58.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026916190000000001 2024-09-18T11:32:58.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:58.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041256+0000", 2024-09-18T11:32:58.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015182100000000001 2024-09-18T11:32:58.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:58.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041300+0000", 2024-09-18T11:32:58.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3915999999999998e-05 2024-09-18T11:32:58.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041339+0000", 2024-09-18T11:32:58.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8510000000000002e-05 2024-09-18T11:32:58.137 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.137 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.138 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.138 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4739 4.3 4:c70c3162:::smithi11939506-35:head [setxattr _header (57) in=7b,truncate 2974905] snapc 7c=[7c,7b,79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:32:58.138 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.032924+0000", 2024-09-18T11:32:58.138 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0810914950000008, 2024-09-18T11:32:58.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085165610000000006, 2024-09-18T11:32:58.138 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.138 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:58.138 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.138 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.138 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.138 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4739 2024-09-18T11:32:58.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.139 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032924+0000", 2024-09-18T11:32:58.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032924+0000", 2024-09-18T11:32:58.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032924+0000", 2024-09-18T11:32:58.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032926+0000", 2024-09-18T11:32:58.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0619999999999999e-06 2024-09-18T11:32:58.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032935+0000", 2024-09-18T11:32:58.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.371e-06 2024-09-18T11:32:58.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032937+0000", 2024-09-18T11:32:58.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6769999999999999e-06 2024-09-18T11:32:58.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032979+0000", 2024-09-18T11:32:58.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1810000000000001e-05 2024-09-18T11:32:58.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:32:58.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032984+0000", 2024-09-18T11:32:58.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1660000000000002e-06 2024-09-18T11:32:58.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039042+0000", 2024-09-18T11:32:58.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060573399999999996 2024-09-18T11:32:58.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039127+0000", 2024-09-18T11:32:58.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5494999999999994e-05 2024-09-18T11:32:58.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:58.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039293+0000", 2024-09-18T11:32:58.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016590999999999999 2024-09-18T11:32:58.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:58.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041146+0000", 2024-09-18T11:32:58.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018531369999999999 2024-09-18T11:32:58.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:58.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041345+0000", 2024-09-18T11:32:58.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000199218 2024-09-18T11:32:58.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:58.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041390+0000", 2024-09-18T11:32:58.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4892000000000003e-05 2024-09-18T11:32:58.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041441+0000", 2024-09-18T11:32:58.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0815000000000001e-05 2024-09-18T11:32:58.144 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.144 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.145 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.145 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4740 4.3 4:c70c3162:::smithi11939506-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:32:58.145 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:32:58.145 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0810658340000003, 2024-09-18T11:32:58.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0087685269999999999, 2024-09-18T11:32:58.145 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.145 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:58.145 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.145 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.145 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.145 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4740 2024-09-18T11:32:58.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.146 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:32:58.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:32:58.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:32:58.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032952+0000", 2024-09-18T11:32:58.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.215e-06 2024-09-18T11:32:58.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032956+0000", 2024-09-18T11:32:58.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8850000000000001e-06 2024-09-18T11:32:58.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032958+0000", 2024-09-18T11:32:58.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1840000000000002e-06 2024-09-18T11:32:58.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032997+0000", 2024-09-18T11:32:58.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9161e-05 2024-09-18T11:32:58.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:32:58.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.033000+0000", 2024-09-18T11:32:58.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1030000000000002e-06 2024-09-18T11:32:58.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039536+0000", 2024-09-18T11:32:58.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006535201 2024-09-18T11:32:58.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:58.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039616+0000", 2024-09-18T11:32:58.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0684000000000002e-05 2024-09-18T11:32:58.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041536+0000", 2024-09-18T11:32:58.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019196440000000001 2024-09-18T11:32:58.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041610+0000", 2024-09-18T11:32:58.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4295999999999995e-05 2024-09-18T11:32:58.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041719+0000", 2024-09-18T11:32:58.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108465 2024-09-18T11:32:58.151 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.151 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.151 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.151 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(client.4590.0:4736 4.3 e178/177 ondisk, result = 0)", 2024-09-18T11:32:58.151 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.040857+0000", 2024-09-18T11:32:58.152 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0731589970000002, 2024-09-18T11:32:58.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015965400000000001, 2024-09-18T11:32:58.152 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.152 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:58.152 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040857+0000", 2024-09-18T11:32:58.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040857+0000", 2024-09-18T11:32:58.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040856+0000", 2024-09-18T11:32:58.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:58.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040861+0000", 2024-09-18T11:32:58.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2989999999999996e-06 2024-09-18T11:32:58.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040872+0000", 2024-09-18T11:32:58.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0416e-05 2024-09-18T11:32:58.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040878+0000", 2024-09-18T11:32:58.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.049e-06 2024-09-18T11:32:58.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040904+0000", 2024-09-18T11:32:58.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5950999999999999e-05 2024-09-18T11:32:58.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040910+0000", 2024-09-18T11:32:58.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5420000000000002e-06 2024-09-18T11:32:58.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041017+0000", 2024-09-18T11:32:58.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010631 2024-09-18T11:32:58.155 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.155 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.156 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.156 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(osd.0.0:4 4.3 e178/177 ondisk, result = 0)", 2024-09-18T11:32:58.156 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.043400+0000", 2024-09-18T11:32:58.156 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0706155830000004, 2024-09-18T11:32:58.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000155208, 2024-09-18T11:32:58.156 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.156 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:58.156 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.043400+0000", 2024-09-18T11:32:58.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.043400+0000", 2024-09-18T11:32:58.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.043400+0000", 2024-09-18T11:32:58.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.043404+0000", 2024-09-18T11:32:58.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.138e-06 2024-09-18T11:32:58.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.043413+0000", 2024-09-18T11:32:58.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3719999999999994e-06 2024-09-18T11:32:58.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.043417+0000", 2024-09-18T11:32:58.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.709e-06 2024-09-18T11:32:58.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.043446+0000", 2024-09-18T11:32:58.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8677999999999999e-05 2024-09-18T11:32:58.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.043452+0000", 2024-09-18T11:32:58.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8140000000000003e-06 2024-09-18T11:32:58.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.043556+0000", 2024-09-18T11:32:58.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103106 2024-09-18T11:32:58.160 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.160 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.160 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.160 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(osd.0.0:6 4.3 e178/177 ondisk, result = 0)", 2024-09-18T11:32:58.160 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.044547+0000", 2024-09-18T11:32:58.160 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.0694690910000002, 2024-09-18T11:32:58.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016644500000000001, 2024-09-18T11:32:58.160 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.160 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:58.160 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.044547+0000", 2024-09-18T11:32:58.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.044547+0000", 2024-09-18T11:32:58.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.044546+0000", 2024-09-18T11:32:58.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.044550+0000", 2024-09-18T11:32:58.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2300000000000002e-06 2024-09-18T11:32:58.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.044560+0000", 2024-09-18T11:32:58.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8500000000000006e-06 2024-09-18T11:32:58.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.044564+0000", 2024-09-18T11:32:58.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.743e-06 2024-09-18T11:32:58.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.044593+0000", 2024-09-18T11:32:58.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8717e-05 2024-09-18T11:32:58.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.044600+0000", 2024-09-18T11:32:58.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.267e-06 2024-09-18T11:32:58.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.044713+0000", 2024-09-18T11:32:58.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000113248 2024-09-18T11:32:58.164 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.164 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.164 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.164 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:4756 4.7 e179/174 4:e301aefd:::smithi11939506-19:head v 179'405, mlcod=173'401)", 2024-09-18T11:32:58.164 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:50.039068+0000", 2024-09-18T11:32:58.164 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.0749477350000003, 2024-09-18T11:32:58.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055292249999999996, 2024-09-18T11:32:58.165 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.165 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:58.165 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.039068+0000", 2024-09-18T11:32:58.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.039068+0000", 2024-09-18T11:32:58.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.039068+0000", 2024-09-18T11:32:58.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040038+0000", 2024-09-18T11:32:58.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096998500000000003 2024-09-18T11:32:58.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040053+0000", 2024-09-18T11:32:58.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5481999999999998e-05 2024-09-18T11:32:58.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040062+0000", 2024-09-18T11:32:58.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7080000000000003e-06 2024-09-18T11:32:58.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040109+0000", 2024-09-18T11:32:58.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6752000000000002e-05 2024-09-18T11:32:58.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040124+0000", 2024-09-18T11:32:58.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5715000000000001e-05 2024-09-18T11:32:58.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:58.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044568+0000", 2024-09-18T11:32:58.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044430600000000004 2024-09-18T11:32:58.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044597+0000", 2024-09-18T11:32:58.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9887999999999999e-05 2024-09-18T11:32:58.169 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.169 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.169 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.169 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:4757 4.7 e179/174 4:e301aefd:::smithi11939506-19:head v 179'406, mlcod=173'401)", 2024-09-18T11:32:58.169 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:50.044255+0000", 2024-09-18T11:32:58.169 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.0697611739999999, 2024-09-18T11:32:58.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002073166, 2024-09-18T11:32:58.169 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.169 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:58.169 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044255+0000", 2024-09-18T11:32:58.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044255+0000", 2024-09-18T11:32:58.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044254+0000", 2024-09-18T11:32:58.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:58.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044875+0000", 2024-09-18T11:32:58.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00062136400000000001 2024-09-18T11:32:58.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044886+0000", 2024-09-18T11:32:58.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0709e-05 2024-09-18T11:32:58.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044891+0000", 2024-09-18T11:32:58.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8019999999999999e-06 2024-09-18T11:32:58.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044911+0000", 2024-09-18T11:32:58.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0729000000000001e-05 2024-09-18T11:32:58.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044930+0000", 2024-09-18T11:32:58.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8890999999999999e-05 2024-09-18T11:32:58.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:58.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046295+0000", 2024-09-18T11:32:58.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013645420000000001 2024-09-18T11:32:58.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046328+0000", 2024-09-18T11:32:58.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3089000000000001e-05 2024-09-18T11:32:58.173 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.173 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.173 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.173 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:4758 4.7 e179/174 4:e301aefd:::smithi11939506-19:head v 179'407, mlcod=173'401)", 2024-09-18T11:32:58.174 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:50.046040+0000", 2024-09-18T11:32:58.174 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.0679755469999996, 2024-09-18T11:32:58.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00053415600000000004, 2024-09-18T11:32:58.174 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.174 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:58.174 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046040+0000", 2024-09-18T11:32:58.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046040+0000", 2024-09-18T11:32:58.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046040+0000", 2024-09-18T11:32:58.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046045+0000", 2024-09-18T11:32:58.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7620000000000003e-06 2024-09-18T11:32:58.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046052+0000", 2024-09-18T11:32:58.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4699999999999996e-06 2024-09-18T11:32:58.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046056+0000", 2024-09-18T11:32:58.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0380000000000001e-06 2024-09-18T11:32:58.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046083+0000", 2024-09-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6957999999999998e-05 2024-09-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046113+0000", 2024-09-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9901999999999999e-05 2024-09-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046555+0000", 2024-09-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00044173699999999999 2024-09-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046575+0000", 2024-09-18T11:32:58.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9885999999999999e-05 2024-09-18T11:32:58.178 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.178 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.178 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.178 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:4764 3.3 e179/174 3:ec4378e2:::259b372f30a46a47845d5d9c58303dcbeccaeb2db18e8ed0152f1c66341c4df2:head v 179'156, mlcod=172'154)", 2024-09-18T11:32:58.178 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:50.059675+0000", 2024-09-18T11:32:58.178 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.0543408220000003, 2024-09-18T11:32:58.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056090439999999997, 2024-09-18T11:32:58.178 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.178 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:58.178 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.059675+0000", 2024-09-18T11:32:58.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.059675+0000", 2024-09-18T11:32:58.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.059675+0000", 2024-09-18T11:32:58.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062252+0000", 2024-09-18T11:32:58.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025772859999999998 2024-09-18T11:32:58.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062265+0000", 2024-09-18T11:32:58.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3059e-05 2024-09-18T11:32:58.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062272+0000", 2024-09-18T11:32:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3589999999999996e-06 2024-09-18T11:32:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062317+0000", 2024-09-18T11:32:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5188999999999997e-05 2024-09-18T11:32:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062334+0000", 2024-09-18T11:32:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.73e-05 2024-09-18T11:32:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:58.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.065251+0000", 2024-09-18T11:32:58.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029169539999999998 2024-09-18T11:32:58.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.065284+0000", 2024-09-18T11:32:58.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3206e-05 2024-09-18T11:32:58.182 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.182 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.182 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.183 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.13:48 3.3 e179/174 3:ec4378e2:::259b372f30a46a47845d5d9c58303dcbeccaeb2db18e8ed0152f1c66341c4df2:head v 179'157, mlcod=173'155)", 2024-09-18T11:32:58.183 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:50.066418+0000", 2024-09-18T11:32:58.183 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 8.0475978650000002, 2024-09-18T11:32:58.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059943099999999999, 2024-09-18T11:32:58.183 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.183 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:58.183 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.066418+0000", 2024-09-18T11:32:58.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.066418+0000", 2024-09-18T11:32:58.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.066417+0000", 2024-09-18T11:32:58.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:58.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.066423+0000", 2024-09-18T11:32:58.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9279999999999998e-06 2024-09-18T11:32:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.066433+0000", 2024-09-18T11:32:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4190000000000008e-06 2024-09-18T11:32:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.066438+0000", 2024-09-18T11:32:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4600000000000002e-06 2024-09-18T11:32:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.066478+0000", 2024-09-18T11:32:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9895999999999999e-05 2024-09-18T11:32:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.066500+0000", 2024-09-18T11:32:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2347000000000001e-05 2024-09-18T11:32:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.066984+0000", 2024-09-18T11:32:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00048345799999999998 2024-09-18T11:32:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.067018+0000", 2024-09-18T11:32:58.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3640999999999999e-05 2024-09-18T11:32:58.187 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.187 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.187 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.187 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4792 4.6 4:67c1765a:::smithi11939506-30:head [sparse-read 0~1153827,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-18T11:32:58.187 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.053543+0000", 2024-09-18T11:32:58.187 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.0604731010000004, 2024-09-18T11:32:58.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003529337, 2024-09-18T11:32:58.187 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.187 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:58.187 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.188 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.188 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.188 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4792 2024-09-18T11:32:58.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.188 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053543+0000", 2024-09-18T11:32:58.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053543+0000", 2024-09-18T11:32:58.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053541+0000", 2024-09-18T11:32:58.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:58.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053550+0000", 2024-09-18T11:32:58.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2070000000000001e-06 2024-09-18T11:32:58.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053566+0000", 2024-09-18T11:32:58.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6098000000000001e-05 2024-09-18T11:32:58.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053575+0000", 2024-09-18T11:32:58.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3769999999999996e-06 2024-09-18T11:32:58.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053620+0000", 2024-09-18T11:32:58.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5404000000000001e-05 2024-09-18T11:32:58.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:32:58.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053634+0000", 2024-09-18T11:32:58.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4161e-05 2024-09-18T11:32:58.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.055502+0000", 2024-09-18T11:32:58.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018678810000000001 2024-09-18T11:32:58.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.055629+0000", 2024-09-18T11:32:58.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012652 2024-09-18T11:32:58.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057072+0000", 2024-09-18T11:32:58.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001443567 2024-09-18T11:32:58.192 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.192 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.192 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.193 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4793 4.6 4:67c1765a:::smithi11939506-30:head [sparse-read 0~1153827] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-18T11:32:58.193 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.053593+0000", 2024-09-18T11:32:58.193 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.060423235, 2024-09-18T11:32:58.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037961539999999999, 2024-09-18T11:32:58.193 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.193 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:58.193 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.193 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.193 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.193 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4793 2024-09-18T11:32:58.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.194 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053593+0000", 2024-09-18T11:32:58.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053593+0000", 2024-09-18T11:32:58.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053592+0000", 2024-09-18T11:32:58.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053595+0000", 2024-09-18T11:32:58.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2179999999999999e-06 2024-09-18T11:32:58.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053603+0000", 2024-09-18T11:32:58.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.576e-06 2024-09-18T11:32:58.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053607+0000", 2024-09-18T11:32:58.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2309999999999998e-06 2024-09-18T11:32:58.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053696+0000", 2024-09-18T11:32:58.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9130999999999998e-05 2024-09-18T11:32:58.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:58.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053702+0000", 2024-09-18T11:32:58.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6370000000000004e-06 2024-09-18T11:32:58.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057115+0000", 2024-09-18T11:32:58.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034128100000000001 2024-09-18T11:32:58.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057176+0000", 2024-09-18T11:32:58.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.143e-05 2024-09-18T11:32:58.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057389+0000", 2024-09-18T11:32:58.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021267700000000001 2024-09-18T11:32:58.198 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.198 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.198 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.198 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4794 4.6 4:67c1765a:::smithi11939506-30:head [sparse-read 0~1153827] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-18T11:32:58.198 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.053613+0000", 2024-09-18T11:32:58.198 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.0604029830000004, 2024-09-18T11:32:58.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040685349999999999, 2024-09-18T11:32:58.198 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.199 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:58.199 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.199 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.199 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.199 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4794 2024-09-18T11:32:58.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.199 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053613+0000", 2024-09-18T11:32:58.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053613+0000", 2024-09-18T11:32:58.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053612+0000", 2024-09-18T11:32:58.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053615+0000", 2024-09-18T11:32:58.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5730000000000002e-06 2024-09-18T11:32:58.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053628+0000", 2024-09-18T11:32:58.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3084e-05 2024-09-18T11:32:58.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053632+0000", 2024-09-18T11:32:58.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9620000000000004e-06 2024-09-18T11:32:58.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053731+0000", 2024-09-18T11:32:58.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8547000000000004e-05 2024-09-18T11:32:58.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:58.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053736+0000", 2024-09-18T11:32:58.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.767e-06 2024-09-18T11:32:58.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057461+0000", 2024-09-18T11:32:58.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037242780000000001 2024-09-18T11:32:58.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057538+0000", 2024-09-18T11:32:58.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7008999999999994e-05 2024-09-18T11:32:58.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057681+0000", 2024-09-18T11:32:58.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000143786 2024-09-18T11:32:58.203 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.203 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.203 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.204 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.0:332 4.7 e182/174 4:e0d95e10:::smithi11939506-22:74 v 182'411, mlcod=179'407)", 2024-09-18T11:32:58.204 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.053724+0000", 2024-09-18T11:32:58.204 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.0602922550000002, 2024-09-18T11:32:58.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096596200000000001, 2024-09-18T11:32:58.204 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.204 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:58.204 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.204 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.204 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053724+0000", 2024-09-18T11:32:58.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053724+0000", 2024-09-18T11:32:58.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053723+0000", 2024-09-18T11:32:58.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053728+0000", 2024-09-18T11:32:58.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7060000000000003e-06 2024-09-18T11:32:58.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053737+0000", 2024-09-18T11:32:58.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9039999999999998e-06 2024-09-18T11:32:58.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053741+0000", 2024-09-18T11:32:58.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3889999999999998e-06 2024-09-18T11:32:58.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053786+0000", 2024-09-18T11:32:58.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4291e-05 2024-09-18T11:32:58.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053807+0000", 2024-09-18T11:32:58.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1389000000000001e-05 2024-09-18T11:32:58.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:58.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054661+0000", 2024-09-18T11:32:58.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00085419499999999998 2024-09-18T11:32:58.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054690+0000", 2024-09-18T11:32:58.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8462999999999998e-05 2024-09-18T11:32:58.208 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.208 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.208 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.208 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.0:333 4.7 e182/174 4:e301aefd:::smithi11939506-19:7e v 182'413, mlcod=179'407)", 2024-09-18T11:32:58.208 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.054168+0000", 2024-09-18T11:32:58.209 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.0598481169999996, 2024-09-18T11:32:58.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00082222999999999997, 2024-09-18T11:32:58.209 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.209 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:58.209 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054168+0000", 2024-09-18T11:32:58.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054168+0000", 2024-09-18T11:32:58.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054167+0000", 2024-09-18T11:32:58.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054184+0000", 2024-09-18T11:32:58.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6906e-05 2024-09-18T11:32:58.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054193+0000", 2024-09-18T11:32:58.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1940000000000004e-06 2024-09-18T11:32:58.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054203+0000", 2024-09-18T11:32:58.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5429999999999999e-06 2024-09-18T11:32:58.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054356+0000", 2024-09-18T11:32:58.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015282499999999999 2024-09-18T11:32:58.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054368+0000", 2024-09-18T11:32:58.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2813000000000001e-05 2024-09-18T11:32:58.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:58.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054969+0000", 2024-09-18T11:32:58.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00060045300000000003 2024-09-18T11:32:58.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.054990+0000", 2024-09-18T11:32:58.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1149e-05 2024-09-18T11:32:58.213 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.213 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.213 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.213 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.0:335 4.7 e182/174 4:e73ba68f:::smithi11939506-1:71 v 182'415, mlcod=182'411)", 2024-09-18T11:32:58.213 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.061501+0000", 2024-09-18T11:32:58.213 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.0525151780000002, 2024-09-18T11:32:58.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00069536999999999995, 2024-09-18T11:32:58.213 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.213 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:58.213 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.061501+0000", 2024-09-18T11:32:58.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.061501+0000", 2024-09-18T11:32:58.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.061500+0000", 2024-09-18T11:32:58.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.061506+0000", 2024-09-18T11:32:58.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3720000000000001e-06 2024-09-18T11:32:58.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.061516+0000", 2024-09-18T11:32:58.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.893e-06 2024-09-18T11:32:58.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.061522+0000", 2024-09-18T11:32:58.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5269999999999998e-06 2024-09-18T11:32:58.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.061565+0000", 2024-09-18T11:32:58.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2826e-05 2024-09-18T11:32:58.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.061586+0000", 2024-09-18T11:32:58.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1576000000000001e-05 2024-09-18T11:32:58.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:58.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.062170+0000", 2024-09-18T11:32:58.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00058380699999999999 2024-09-18T11:32:58.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.062196+0000", 2024-09-18T11:32:58.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6069000000000001e-05 2024-09-18T11:32:58.217 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.217 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.218 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.218 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(osd.0.0:10 4.3 e182/177 ondisk, result = 0)", 2024-09-18T11:32:58.218 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.063855+0000", 2024-09-18T11:32:58.218 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 5.0501612949999997, 2024-09-18T11:32:58.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016419399999999999, 2024-09-18T11:32:58.218 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.218 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:58.218 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.063855+0000", 2024-09-18T11:32:58.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.063855+0000", 2024-09-18T11:32:58.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.063854+0000", 2024-09-18T11:32:58.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.063857+0000", 2024-09-18T11:32:58.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1319999999999998e-06 2024-09-18T11:32:58.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.063867+0000", 2024-09-18T11:32:58.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5889999999999992e-06 2024-09-18T11:32:58.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.063870+0000", 2024-09-18T11:32:58.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.4960000000000001e-06 2024-09-18T11:32:58.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.063904+0000", 2024-09-18T11:32:58.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3399999999999999e-05 2024-09-18T11:32:58.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.063912+0000", 2024-09-18T11:32:58.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0679999999999997e-06 2024-09-18T11:32:58.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.064019+0000", 2024-09-18T11:32:58.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000107017 2024-09-18T11:32:58.222 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.222 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.222 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.222 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.222 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.222 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:58.223 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:32:58.224 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:58.222+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:58.324 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:58.322+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:58.363 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:58.363 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:58.364 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:58.364 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:58.372 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:32:58.456 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:58.454+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:58.510 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:58.510 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:58.510 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:58.511 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:58.511 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:58.518 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:32:58.560 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:58.558+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:58.627 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:58.626+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:58.658 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:262 4.9 e53/22 4:99feffe9:::smithi11939506-45:13 v 53'17, mlcod=51'14)", 2024-09-18T11:32:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 144.65751665799999, 2024-09-18T11:32:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013878679, 2024-09-18T11:32:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:58.659 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:32:58.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998997+0000", 2024-09-18T11:32:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1099999999999999e-06 2024-09-18T11:32:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999008+0000", 2024-09-18T11:32:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1321000000000001e-05 2024-09-18T11:32:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999015+0000", 2024-09-18T11:32:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3000000000000004e-06 2024-09-18T11:32:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999055+0000", 2024-09-18T11:32:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0058000000000002e-05 2024-09-18T11:32:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999070+0000", 2024-09-18T11:32:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4328000000000001e-05 2024-09-18T11:32:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012465+0000", 2024-09-18T11:32:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013395638 2024-09-18T11:32:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012870+0000", 2024-09-18T11:32:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040423400000000001 2024-09-18T11:32:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:205 4.b e53/22 4:dba5bd39:::smithi11939506-17:13 v 53'77, mlcod=51'74)", 2024-09-18T11:32:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 144.657467397, 2024-09-18T11:32:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013359586, 2024-09-18T11:32:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:32:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999048+0000", 2024-09-18T11:32:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3550000000000005e-06 2024-09-18T11:32:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999056+0000", 2024-09-18T11:32:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3259999999999996e-06 2024-09-18T11:32:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999063+0000", 2024-09-18T11:32:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6409999999999996e-06 2024-09-18T11:32:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999098+0000", 2024-09-18T11:32:58.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5203999999999997e-05 2024-09-18T11:32:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999115+0000", 2024-09-18T11:32:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6478999999999999e-05 2024-09-18T11:32:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012362+0000", 2024-09-18T11:32:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01324768 2024-09-18T11:32:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012400+0000", 2024-09-18T11:32:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.731e-05 2024-09-18T11:32:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2937 4.7 4:e301aefd:::smithi11939506-19:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:58.669 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.621667083999995, 2024-09-18T11:32:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019287667000000001, 2024-09-18T11:32:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2937 2024-09-18T11:32:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:32:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034842+0000", 2024-09-18T11:32:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.372e-06 2024-09-18T11:32:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034846+0000", 2024-09-18T11:32:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3919999999999996e-06 2024-09-18T11:32:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034848+0000", 2024-09-18T11:32:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3539999999999998e-06 2024-09-18T11:32:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034893+0000", 2024-09-18T11:32:58.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4861000000000003e-05 2024-09-18T11:32:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034965+0000", 2024-09-18T11:32:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1860999999999995e-05 2024-09-18T11:32:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.054128+0000", 2024-09-18T11:32:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019163039 2024-09-18T11:32:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3453 4.f 4:f5d7df41:::smithi11939506-16:head [write 1813152~490434] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:32:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.633755156000007, 2024-09-18T11:32:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050685840000000003, 2024-09-18T11:32:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3453 2024-09-18T11:32:58.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:32:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:32:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022751+0000", 2024-09-18T11:32:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022871+0000", 2024-09-18T11:32:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000119781 2024-09-18T11:32:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022882+0000", 2024-09-18T11:32:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1627000000000001e-05 2024-09-18T11:32:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022887+0000", 2024-09-18T11:32:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2390000000000004e-06 2024-09-18T11:32:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023123+0000", 2024-09-18T11:32:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000236598 2024-09-18T11:32:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023207+0000", 2024-09-18T11:32:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4303999999999998e-05 2024-09-18T11:32:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023367+0000", 2024-09-18T11:32:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015983600000000001 2024-09-18T11:32:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073237+0000", 2024-09-18T11:32:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049869400000000001 2024-09-18T11:32:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073352+0000", 2024-09-18T11:32:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011579899999999999 2024-09-18T11:32:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073394+0000", 2024-09-18T11:32:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1445e-05 2024-09-18T11:32:58.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073438+0000", 2024-09-18T11:32:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4261999999999997e-05 2024-09-18T11:32:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3454 4.f 4:f5d7df41:::smithi11939506-16:head [setxattr _header (57) in=7b,truncate 2991679] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:32:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.633056714000006, 2024-09-18T11:32:58.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050365656000000002, 2024-09-18T11:32:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3454 2024-09-18T11:32:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:32:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:32:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023450+0000", 2024-09-18T11:32:58.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023456+0000", 2024-09-18T11:32:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8329999999999999e-06 2024-09-18T11:32:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023465+0000", 2024-09-18T11:32:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1130000000000007e-06 2024-09-18T11:32:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023470+0000", 2024-09-18T11:32:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9139999999999999e-06 2024-09-18T11:32:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072611+0000", 2024-09-18T11:32:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141643999999998 2024-09-18T11:32:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072729+0000", 2024-09-18T11:32:58.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117667 2024-09-18T11:32:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:32:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072887+0000", 2024-09-18T11:32:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015760000000000001 2024-09-18T11:32:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073558+0000", 2024-09-18T11:32:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00067171699999999995 2024-09-18T11:32:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073764+0000", 2024-09-18T11:32:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000205749 2024-09-18T11:32:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073790+0000", 2024-09-18T11:32:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5643000000000002e-05 2024-09-18T11:32:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073816+0000", 2024-09-18T11:32:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6588e-05 2024-09-18T11:32:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.686 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3455 4.f 4:f5d7df41:::smithi11939506-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.633027096000006, 2024-09-18T11:32:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050460466000000002, 2024-09-18T11:32:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.687 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3455 2024-09-18T11:32:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:32:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023483+0000", 2024-09-18T11:32:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.695e-06 2024-09-18T11:32:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023489+0000", 2024-09-18T11:32:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1079999999999998e-06 2024-09-18T11:32:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023492+0000", 2024-09-18T11:32:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.478e-06 2024-09-18T11:32:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073273+0000", 2024-09-18T11:32:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049780638000000002 2024-09-18T11:32:58.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073343+0000", 2024-09-18T11:32:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9770000000000005e-05 2024-09-18T11:32:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073846+0000", 2024-09-18T11:32:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050289499999999995 2024-09-18T11:32:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073886+0000", 2024-09-18T11:32:58.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0658999999999998e-05 2024-09-18T11:32:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073941+0000", 2024-09-18T11:32:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4642000000000003e-05 2024-09-18T11:32:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3456 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.633002031000004, 2024-09-18T11:32:58.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049859120999999999, 2024-09-18T11:32:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3456 2024-09-18T11:32:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:32:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023508+0000", 2024-09-18T11:32:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9490000000000001e-06 2024-09-18T11:32:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023513+0000", 2024-09-18T11:32:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3800000000000002e-06 2024-09-18T11:32:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023517+0000", 2024-09-18T11:32:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5300000000000001e-06 2024-09-18T11:32:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073262+0000", 2024-09-18T11:32:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049745194999999999 2024-09-18T11:32:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073365+0000", 2024-09-18T11:32:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102462 2024-09-18T11:32:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.696 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3457 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:32:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.632984934999996, 2024-09-18T11:32:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049928567, 2024-09-18T11:32:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.697 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3457 2024-09-18T11:32:58.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.698 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:32:58.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:32:58.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023522+0000", 2024-09-18T11:32:58.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023524+0000", 2024-09-18T11:32:58.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8500000000000001e-06 2024-09-18T11:32:58.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023529+0000", 2024-09-18T11:32:58.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7509999999999999e-06 2024-09-18T11:32:58.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023532+0000", 2024-09-18T11:32:58.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.163e-06 2024-09-18T11:32:58.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073386+0000", 2024-09-18T11:32:58.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049853899 2024-09-18T11:32:58.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073451+0000", 2024-09-18T11:32:58.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5197000000000006e-05 2024-09-18T11:32:58.701 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.701 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.701 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.701 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3458 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:32:58.701 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:58.701 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.632972378000005, 2024-09-18T11:32:58.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.05001659, 2024-09-18T11:32:58.702 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.702 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:58.702 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.702 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.702 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.702 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3458 2024-09-18T11:32:58.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.702 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:58.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:58.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:32:58.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023537+0000", 2024-09-18T11:32:58.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.64e-06 2024-09-18T11:32:58.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023543+0000", 2024-09-18T11:32:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3369999999999996e-06 2024-09-18T11:32:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023546+0000", 2024-09-18T11:32:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:32:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073466+0000", 2024-09-18T11:32:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049919719000000001 2024-09-18T11:32:58.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073552+0000", 2024-09-18T11:32:58.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6168999999999994e-05 2024-09-18T11:32:58.705 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.705 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.705 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.706 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(client.4590.0:3453 4.f e116/22 ondisk, result = 0)", 2024-09-18T11:32:58.706 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:32:58.706 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.631367294, 2024-09-18T11:32:58.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048099441, 2024-09-18T11:32:58.706 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.706 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:58.706 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:32:58.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:32:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025139+0000", 2024-09-18T11:32:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025144+0000", 2024-09-18T11:32:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4059999999999998e-06 2024-09-18T11:32:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072212+0000", 2024-09-18T11:32:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047068482000000002 2024-09-18T11:32:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072230+0000", 2024-09-18T11:32:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7632999999999999e-05 2024-09-18T11:32:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073224+0000", 2024-09-18T11:32:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099436899999999994 2024-09-18T11:32:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073231+0000", 2024-09-18T11:32:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-09-18T11:32:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073240+0000", 2024-09-18T11:32:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5699999999999993e-06 2024-09-18T11:32:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3930 4.b 4:d3ae8215:::smithi11939506-41:head [write 632029~547288] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:32:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.630451614000002, 2024-09-18T11:32:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018962711, 2024-09-18T11:32:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3930 2024-09-18T11:32:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:32:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:32:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026055+0000", 2024-09-18T11:32:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026334+0000", 2024-09-18T11:32:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000278931 2024-09-18T11:32:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026344+0000", 2024-09-18T11:32:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0716999999999999e-05 2024-09-18T11:32:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026350+0000", 2024-09-18T11:32:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7620000000000001e-06 2024-09-18T11:32:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028429+0000", 2024-09-18T11:32:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020787319999999998 2024-09-18T11:32:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028525+0000", 2024-09-18T11:32:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6638999999999994e-05 2024-09-18T11:32:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028720+0000", 2024-09-18T11:32:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000194525 2024-09-18T11:32:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.036996+0000", 2024-09-18T11:32:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082756140000000006 2024-09-18T11:32:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044952+0000", 2024-09-18T11:32:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566740000000004 2024-09-18T11:32:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044985+0000", 2024-09-18T11:32:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3081999999999999e-05 2024-09-18T11:32:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045019+0000", 2024-09-18T11:32:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3257999999999999e-05 2024-09-18T11:32:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3932 4.b 4:d3ae8215:::smithi11939506-41:head [write 2796249~437859] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:32:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.618444973999999, 2024-09-18T11:32:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01458329, 2024-09-18T11:32:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3932 2024-09-18T11:32:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:32:58.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:32:58.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038061+0000", 2024-09-18T11:32:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038341+0000", 2024-09-18T11:32:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027989099999999999 2024-09-18T11:32:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038353+0000", 2024-09-18T11:32:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2247e-05 2024-09-18T11:32:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038358+0000", 2024-09-18T11:32:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-18T11:32:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043226+0000", 2024-09-18T11:32:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048673049999999997 2024-09-18T11:32:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043284+0000", 2024-09-18T11:32:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8338999999999998e-05 2024-09-18T11:32:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:32:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043367+0000", 2024-09-18T11:32:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3120000000000004e-05 2024-09-18T11:32:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.051544+0000", 2024-09-18T11:32:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00817734 2024-09-18T11:32:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:58.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052528+0000", 2024-09-18T11:32:58.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098318499999999992 2024-09-18T11:32:58.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:58.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052610+0000", 2024-09-18T11:32:58.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2300999999999997e-05 2024-09-18T11:32:58.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052646+0000", 2024-09-18T11:32:58.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5955999999999999e-05 2024-09-18T11:32:58.722 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.723 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.723 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.723 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3935 4.b 4:d3ae8215:::smithi11939506-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:32:58.723 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:32:58.723 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 55.615107866999999, 2024-09-18T11:32:58.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013142333000000001, 2024-09-18T11:32:58.723 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.723 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:58.723 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.723 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.723 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.723 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3935 2024-09-18T11:32:58.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.724 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:32:58.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:32:58.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041399+0000", 2024-09-18T11:32:58.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041401+0000", 2024-09-18T11:32:58.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4890000000000001e-06 2024-09-18T11:32:58.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041404+0000", 2024-09-18T11:32:58.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5240000000000001e-06 2024-09-18T11:32:58.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041407+0000", 2024-09-18T11:32:58.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2230000000000001e-06 2024-09-18T11:32:58.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045699+0000", 2024-09-18T11:32:58.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042918970000000002 2024-09-18T11:32:58.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:32:58.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045750+0000", 2024-09-18T11:32:58.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1332999999999998e-05 2024-09-18T11:32:58.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054438+0000", 2024-09-18T11:32:58.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086882569999999996 2024-09-18T11:32:58.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054485+0000", 2024-09-18T11:32:58.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7151000000000003e-05 2024-09-18T11:32:58.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054542+0000", 2024-09-18T11:32:58.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6706999999999998e-05 2024-09-18T11:32:58.728 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.728 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.728 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.728 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3969 4.4 4:2cb3e6a5:::smithi11939506-3:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:58.728 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:32:58.728 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 49.082867602, 2024-09-18T11:32:58.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.024248042000000001, 2024-09-18T11:32:58.728 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.728 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:58.728 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.729 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.729 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.729 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3969 2024-09-18T11:32:58.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.729 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:32:58.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:32:58.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573639+0000", 2024-09-18T11:32:58.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:58.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573646+0000", 2024-09-18T11:32:58.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6569999999999997e-06 2024-09-18T11:32:58.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573662+0000", 2024-09-18T11:32:58.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5187999999999999e-05 2024-09-18T11:32:58.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573670+0000", 2024-09-18T11:32:58.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5399999999999996e-06 2024-09-18T11:32:58.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:58.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8592999999999999e-05 2024-09-18T11:32:58.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573818+0000", 2024-09-18T11:32:58.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9388999999999995e-05 2024-09-18T11:32:58.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597888+0000", 2024-09-18T11:32:58.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.024069859999999998 2024-09-18T11:32:58.732 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.732 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.732 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.733 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3973 4.7 4:e73ba68f:::smithi11939506-1:head [tier-promote] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:32:58.733 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:58.733 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 49.082768522999999, 2024-09-18T11:32:58.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.023911297000000001, 2024-09-18T11:32:58.733 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.733 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:58.733 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.733 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.733 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.733 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3973 2024-09-18T11:32:58.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.734 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:58.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:32:58.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573738+0000", 2024-09-18T11:32:58.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573743+0000", 2024-09-18T11:32:58.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0839999999999999e-06 2024-09-18T11:32:58.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573752+0000", 2024-09-18T11:32:58.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3080000000000008e-06 2024-09-18T11:32:58.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573758+0000", 2024-09-18T11:32:58.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2659999999999998e-06 2024-09-18T11:32:58.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573803+0000", 2024-09-18T11:32:58.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5243e-05 2024-09-18T11:32:58.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573897+0000", 2024-09-18T11:32:58.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3398000000000003e-05 2024-09-18T11:32:58.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.737 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.737 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597650+0000", 2024-09-18T11:32:58.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.023753672999999999 2024-09-18T11:32:58.737 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.737 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.737 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.737 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4443 4.d 4:b9bb642b:::smithi11939506-27:67 [read 0~3512696,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T11:32:58.737 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:36.030743+0000", 2024-09-18T11:32:58.737 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.625764894, 2024-09-18T11:32:58.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013696302, 2024-09-18T11:32:58.737 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.738 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:58.738 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.738 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.738 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.738 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4443 2024-09-18T11:32:58.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.738 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030743+0000", 2024-09-18T11:32:58.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030743+0000", 2024-09-18T11:32:58.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030741+0000", 2024-09-18T11:32:58.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:58.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030751+0000", 2024-09-18T11:32:58.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6979999999999993e-06 2024-09-18T11:32:58.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030763+0000", 2024-09-18T11:32:58.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2311e-05 2024-09-18T11:32:58.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030769+0000", 2024-09-18T11:32:58.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6060000000000002e-06 2024-09-18T11:32:58.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.036777+0000", 2024-09-18T11:32:58.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060079089999999996 2024-09-18T11:32:58.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.036945+0000", 2024-09-18T11:32:58.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016867100000000001 2024-09-18T11:32:58.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044439+0000", 2024-09-18T11:32:58.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074937149999999998 2024-09-18T11:32:58.742 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.742 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.742 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.742 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4444 4.d 4:b9bb642b:::smithi11939506-27:67 [sparse-read 0~1262403] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T11:32:58.742 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:32:58.742 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.625730100999998, 2024-09-18T11:32:58.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.014179987, 2024-09-18T11:32:58.742 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.742 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:58.742 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.743 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.743 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.743 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4444 2024-09-18T11:32:58.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.743 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:32:58.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.743 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:32:58.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:32:58.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030780+0000", 2024-09-18T11:32:58.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.163e-06 2024-09-18T11:32:58.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030786+0000", 2024-09-18T11:32:58.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879999999999998e-06 2024-09-18T11:32:58.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030791+0000", 2024-09-18T11:32:58.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2579999999999997e-06 2024-09-18T11:32:58.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044477+0000", 2024-09-18T11:32:58.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013686188 2024-09-18T11:32:58.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044614+0000", 2024-09-18T11:32:58.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013727300000000001 2024-09-18T11:32:58.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044957+0000", 2024-09-18T11:32:58.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000343331 2024-09-18T11:32:58.746 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.746 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.746 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.747 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4445 4.d 4:b9bb642b:::smithi11939506-27:67 [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T11:32:58.747 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:36.030796+0000", 2024-09-18T11:32:58.747 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.625711825, 2024-09-18T11:32:58.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.015916046999999999, 2024-09-18T11:32:58.747 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.747 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:58.747 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.747 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.747 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.747 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4445 2024-09-18T11:32:58.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.747 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030796+0000", 2024-09-18T11:32:58.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030796+0000", 2024-09-18T11:32:58.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030795+0000", 2024-09-18T11:32:58.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030799+0000", 2024-09-18T11:32:58.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5209999999999998e-06 2024-09-18T11:32:58.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.749 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030806+0000", 2024-09-18T11:32:58.749 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7599999999999997e-06 2024-09-18T11:32:58.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.749 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030810+0000", 2024-09-18T11:32:58.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0489999999999997e-06 2024-09-18T11:32:58.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.045277+0000", 2024-09-18T11:32:58.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.014467683 2024-09-18T11:32:58.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.750 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.045394+0000", 2024-09-18T11:32:58.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116969 2024-09-18T11:32:58.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.046712+0000", 2024-09-18T11:32:58.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013174669999999999 2024-09-18T11:32:58.751 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.751 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.751 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.751 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4683 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865643,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:32:58.751 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:45.318891+0000", 2024-09-18T11:32:58.751 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.337616208, 2024-09-18T11:32:58.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.017494045999999999, 2024-09-18T11:32:58.751 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.752 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:58.752 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.752 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.752 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.752 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4683 2024-09-18T11:32:58.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.752 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318891+0000", 2024-09-18T11:32:58.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318891+0000", 2024-09-18T11:32:58.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318891+0000", 2024-09-18T11:32:58.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318894+0000", 2024-09-18T11:32:58.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1379999999999999e-06 2024-09-18T11:32:58.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318900+0000", 2024-09-18T11:32:58.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6239999999999999e-06 2024-09-18T11:32:58.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318903+0000", 2024-09-18T11:32:58.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8249999999999998e-06 2024-09-18T11:32:58.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.330916+0000", 2024-09-18T11:32:58.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012012063999999999 2024-09-18T11:32:58.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.331011+0000", 2024-09-18T11:32:58.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5388000000000002e-05 2024-09-18T11:32:58.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.336385+0000", 2024-09-18T11:32:58.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005374429 2024-09-18T11:32:58.756 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.756 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.756 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.756 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4684 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865643,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:32:58.756 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:45.318908+0000", 2024-09-18T11:32:58.756 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 13.337599902999999, 2024-09-18T11:32:58.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.021211049999999999, 2024-09-18T11:32:58.756 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:58.756 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:58.757 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:58.757 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:58.757 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:58.757 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4684 2024-09-18T11:32:58.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.757 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:58.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:58.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318908+0000", 2024-09-18T11:32:58.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:58.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318908+0000", 2024-09-18T11:32:58.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:58.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:58.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318907+0000", 2024-09-18T11:32:58.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:58.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:58.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318910+0000", 2024-09-18T11:32:58.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.79e-06 2024-09-18T11:32:58.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:58.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318917+0000", 2024-09-18T11:32:58.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5740000000000001e-06 2024-09-18T11:32:58.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:58.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318920+0000", 2024-09-18T11:32:58.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5729999999999999e-06 2024-09-18T11:32:58.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:58.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.336370+0000", 2024-09-18T11:32:58.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01745025 2024-09-18T11:32:58.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:58.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.336506+0000", 2024-09-18T11:32:58.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013603700000000001 2024-09-18T11:32:58.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:58.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:58.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:58.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.340119+0000", 2024-09-18T11:32:58.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00361218 2024-09-18T11:32:58.760 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.760 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.760 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.761 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:58.761 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:58.761 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:58.762 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:32:58.763 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:58.761+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:58.860 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:58.860 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:58.860 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:58.860 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:58.863 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:58.861+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:58.868 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:32:58.964 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:32:58.963+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:32:59.006 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:59.007 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:59.007 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:59.007 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:59.007 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:59.015 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:32:59.033 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is set 2024-09-18T11:32:59.034 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T11:32:59.054 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T11:32:59.096 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:32:59.094+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:32:59.135 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:59.135 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:59.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:59.135 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:59.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.135 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:32:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 159.149843538, 2024-09-18T11:32:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:32:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:32:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:32:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:32:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:32:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:32:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:32:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:32:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:32:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:32:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:32:59.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:32:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:32:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:32:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:32:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:32:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:32:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:32:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 149.915012585, 2024-09-18T11:32:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:32:59.140 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:32:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:32:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:32:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:32:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:32:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:32:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:32:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:32:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:32:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:32:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:32:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:32:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:32:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:32:59.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:32:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:32:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 144.74110630000001, 2024-09-18T11:32:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:32:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:32:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:32:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:32:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:32:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:32:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:32:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:32:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:32:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:32:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:32:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:32:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:32:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:32:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:32:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:32:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:32:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:304 4.6 e69/67 4:67c1765a:::smithi11939506-30:17 v 69'35, mlcod=58'31)", 2024-09-18T11:32:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 127.787265206, 2024-09-18T11:32:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006377611, 2024-09-18T11:32:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:32:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345961+0000", 2024-09-18T11:32:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2050000000000008e-06 2024-09-18T11:32:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345969+0000", 2024-09-18T11:32:59.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0339999999999997e-06 2024-09-18T11:32:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345976+0000", 2024-09-18T11:32:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-18T11:32:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346026+0000", 2024-09-18T11:32:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9286999999999998e-05 2024-09-18T11:32:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346050+0000", 2024-09-18T11:32:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4397e-05 2024-09-18T11:32:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352293+0000", 2024-09-18T11:32:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062427170000000001 2024-09-18T11:32:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352330+0000", 2024-09-18T11:32:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7348999999999998e-05 2024-09-18T11:32:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.153 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:347 4.d e69/22 4:b44229a1:::smithi11939506-42:19 v 69'99, mlcod=65'93)", 2024-09-18T11:32:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 127.781532011, 2024-09-18T11:32:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056998760000000004, 2024-09-18T11:32:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:32:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351684+0000", 2024-09-18T11:32:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351693+0000", 2024-09-18T11:32:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-18T11:32:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351704+0000", 2024-09-18T11:32:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1077000000000001e-05 2024-09-18T11:32:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351709+0000", 2024-09-18T11:32:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5419999999999999e-06 2024-09-18T11:32:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351956+0000", 2024-09-18T11:32:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000247252 2024-09-18T11:32:59.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351975+0000", 2024-09-18T11:32:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9201000000000001e-05 2024-09-18T11:32:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357334+0000", 2024-09-18T11:32:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053588749999999999 2024-09-18T11:32:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357385+0000", 2024-09-18T11:32:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1406999999999998e-05 2024-09-18T11:32:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1956 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'37, mlcod=58'31)", 2024-09-18T11:32:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 127.780587098, 2024-09-18T11:32:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010958842, 2024-09-18T11:32:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:32:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352996+0000", 2024-09-18T11:32:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036622 2024-09-18T11:32:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353011+0000", 2024-09-18T11:32:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4849e-05 2024-09-18T11:32:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353017+0000", 2024-09-18T11:32:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:32:59.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353068+0000", 2024-09-18T11:32:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0695000000000002e-05 2024-09-18T11:32:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353092+0000", 2024-09-18T11:32:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4260000000000002e-05 2024-09-18T11:32:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363551+0000", 2024-09-18T11:32:59.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010459154999999999 2024-09-18T11:32:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363589+0000", 2024-09-18T11:32:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8161000000000003e-05 2024-09-18T11:32:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1957 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'38, mlcod=65'33)", 2024-09-18T11:32:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:59.162 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 127.774171231, 2024-09-18T11:32:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050499400000000002, 2024-09-18T11:32:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:32:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359214+0000", 2024-09-18T11:32:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000168047 2024-09-18T11:32:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359227+0000", 2024-09-18T11:32:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3124e-05 2024-09-18T11:32:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359235+0000", 2024-09-18T11:32:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0760000000000003e-06 2024-09-18T11:32:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359287+0000", 2024-09-18T11:32:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.151e-05 2024-09-18T11:32:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359310+0000", 2024-09-18T11:32:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-09-18T11:32:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364065+0000", 2024-09-18T11:32:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047549439999999997 2024-09-18T11:32:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364096+0000", 2024-09-18T11:32:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1423000000000001e-05 2024-09-18T11:32:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.166 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2300 4.4 e77/22 4:2c93196f:::smithi11939506-28:head v 77'94, mlcod=77'90)", 2024-09-18T11:32:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.145109013, 2024-09-18T11:32:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050425380000000001, 2024-09-18T11:32:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:32:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988108+0000", 2024-09-18T11:32:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988388+0000", 2024-09-18T11:32:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028087700000000001 2024-09-18T11:32:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988405+0000", 2024-09-18T11:32:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6565000000000001e-05 2024-09-18T11:32:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988412+0000", 2024-09-18T11:32:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9709999999999998e-06 2024-09-18T11:32:59.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988464+0000", 2024-09-18T11:32:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1949e-05 2024-09-18T11:32:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988490+0000", 2024-09-18T11:32:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5780999999999999e-05 2024-09-18T11:32:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993112+0000", 2024-09-18T11:32:59.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046228399999999996 2024-09-18T11:32:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993151+0000", 2024-09-18T11:32:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8587999999999997e-05 2024-09-18T11:32:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.13:63 3.0 e78/21 3:24ebb740:::smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 78'80, mlcod=73'78)", 2024-09-18T11:32:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:59.171 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 117.728119597, 2024-09-18T11:32:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052262230000000003, 2024-09-18T11:32:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:32:59.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405097+0000", 2024-09-18T11:32:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405103+0000", 2024-09-18T11:32:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8780000000000003e-06 2024-09-18T11:32:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405117+0000", 2024-09-18T11:32:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3441000000000001e-05 2024-09-18T11:32:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405121+0000", 2024-09-18T11:32:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7099999999999998e-06 2024-09-18T11:32:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405158+0000", 2024-09-18T11:32:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6477999999999999e-05 2024-09-18T11:32:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405173+0000", 2024-09-18T11:32:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4796e-05 2024-09-18T11:32:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410285+0000", 2024-09-18T11:32:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051118919999999998 2024-09-18T11:32:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410324+0000", 2024-09-18T11:32:59.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9616999999999997e-05 2024-09-18T11:32:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.13:18 3.0 e91/21 3:35e3c617:::2e64dd273da7619bd64b294f52905c1b9405a1ced2a8fd3c0b5bc481bc9732a6:head v 91'89, mlcod=85'87)", 2024-09-18T11:32:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.111344186, 2024-09-18T11:32:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048558539999999997, 2024-09-18T11:32:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:32:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021880+0000", 2024-09-18T11:32:59.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5860000000000001e-06 2024-09-18T11:32:59.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021890+0000", 2024-09-18T11:32:59.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8290000000000003e-06 2024-09-18T11:32:59.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021895+0000", 2024-09-18T11:32:59.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2480000000000004e-06 2024-09-18T11:32:59.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022166+0000", 2024-09-18T11:32:59.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027073499999999998 2024-09-18T11:32:59.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022184+0000", 2024-09-18T11:32:59.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8082000000000001e-05 2024-09-18T11:32:59.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026693+0000", 2024-09-18T11:32:59.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045093479999999998 2024-09-18T11:32:59.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026729+0000", 2024-09-18T11:32:59.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6282999999999998e-05 2024-09-18T11:32:59.180 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.180 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.180 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.180 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:573 4.4 e91/22 4:2c93196f:::smithi11939506-28:26 v 91'135, mlcod=90'131)", 2024-09-18T11:32:59.180 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:59.181 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.108877597, 2024-09-18T11:32:59.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050722739999999999, 2024-09-18T11:32:59.181 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.181 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.181 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:59.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:32:59.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024339+0000", 2024-09-18T11:32:59.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:59.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024349+0000", 2024-09-18T11:32:59.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6339999999999996e-06 2024-09-18T11:32:59.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024362+0000", 2024-09-18T11:32:59.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-18T11:32:59.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024368+0000", 2024-09-18T11:32:59.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8649999999999998e-06 2024-09-18T11:32:59.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024431+0000", 2024-09-18T11:32:59.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2780999999999994e-05 2024-09-18T11:32:59.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024448+0000", 2024-09-18T11:32:59.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7388999999999999e-05 2024-09-18T11:32:59.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029385+0000", 2024-09-18T11:32:59.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004936523 2024-09-18T11:32:59.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029412+0000", 2024-09-18T11:32:59.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7702999999999999e-05 2024-09-18T11:32:59.185 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.185 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.185 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.185 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:574 4.4 e91/22 4:2cb3e6a5:::smithi11939506-3:1e v 91'137, mlcod=90'131)", 2024-09-18T11:32:59.185 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:59.185 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.108439554, 2024-09-18T11:32:59.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070802850000000004, 2024-09-18T11:32:59.186 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.186 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.186 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:59.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:32:59.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024777+0000", 2024-09-18T11:32:59.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:59.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024785+0000", 2024-09-18T11:32:59.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8099999999999998e-06 2024-09-18T11:32:59.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024794+0000", 2024-09-18T11:32:59.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5839999999999995e-06 2024-09-18T11:32:59.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024798+0000", 2024-09-18T11:32:59.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.031e-06 2024-09-18T11:32:59.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024830+0000", 2024-09-18T11:32:59.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2268999999999999e-05 2024-09-18T11:32:59.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024847+0000", 2024-09-18T11:32:59.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7306999999999999e-05 2024-09-18T11:32:59.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031834+0000", 2024-09-18T11:32:59.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069868380000000004 2024-09-18T11:32:59.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031858+0000", 2024-09-18T11:32:59.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4082000000000001e-05 2024-09-18T11:32:59.190 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.190 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.190 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.190 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:541 4.d e91/22 4:b9bb642b:::smithi11939506-27:19 v 91'158, mlcod=91'154)", 2024-09-18T11:32:59.190 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:59.190 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.101058862, 2024-09-18T11:32:59.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052185, 2024-09-18T11:32:59.190 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.190 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.190 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:59.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:32:59.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032158+0000", 2024-09-18T11:32:59.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:59.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032165+0000", 2024-09-18T11:32:59.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-09-18T11:32:59.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032176+0000", 2024-09-18T11:32:59.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1460999999999999e-05 2024-09-18T11:32:59.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032182+0000", 2024-09-18T11:32:59.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0870000000000003e-06 2024-09-18T11:32:59.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032224+0000", 2024-09-18T11:32:59.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1372999999999998e-05 2024-09-18T11:32:59.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032246+0000", 2024-09-18T11:32:59.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2158e-05 2024-09-18T11:32:59.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037348+0000", 2024-09-18T11:32:59.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051025319999999999 2024-09-18T11:32:59.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037377+0000", 2024-09-18T11:32:59.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8937000000000001e-05 2024-09-18T11:32:59.194 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.194 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.194 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.195 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4178.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-18T11:32:59.195 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:59.195 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 77.192396797000001, 2024-09-18T11:32:59.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048960139999999997, 2024-09-18T11:32:59.195 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.195 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:32:59.195 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:59.195 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4178", 2024-09-18T11:32:59.195 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/490761693", 2024-09-18T11:32:59.195 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4 2024-09-18T11:32:59.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.196 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:59.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:32:59.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940819+0000", 2024-09-18T11:32:59.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:59.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940832+0000", 2024-09-18T11:32:59.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3155e-05 2024-09-18T11:32:59.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.197 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.197 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940848+0000", 2024-09-18T11:32:59.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6288999999999999e-05 2024-09-18T11:32:59.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940856+0000", 2024-09-18T11:32:59.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4919999999999996e-06 2024-09-18T11:32:59.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945613+0000", 2024-09-18T11:32:59.198 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047574999999999996 2024-09-18T11:32:59.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.198 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945717+0000", 2024-09-18T11:32:59.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103553 2024-09-18T11:32:59.199 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.199 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.199 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.199 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3482 4.6 e117/67 4:67c1765a:::smithi11939506-30:head v 117'120, mlcod=115'116)", 2024-09-18T11:32:59.199 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:59.199 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.448542574000001, 2024-09-18T11:32:59.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050037479999999997, 2024-09-18T11:32:59.199 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.199 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.199 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:59.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:32:59.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684674+0000", 2024-09-18T11:32:59.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:59.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684950+0000", 2024-09-18T11:32:59.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027621200000000002 2024-09-18T11:32:59.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684968+0000", 2024-09-18T11:32:59.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8e-05 2024-09-18T11:32:59.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684984+0000", 2024-09-18T11:32:59.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5775000000000001e-05 2024-09-18T11:32:59.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685033+0000", 2024-09-18T11:32:59.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8878000000000002e-05 2024-09-18T11:32:59.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685059+0000", 2024-09-18T11:32:59.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6801999999999999e-05 2024-09-18T11:32:59.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689640+0000", 2024-09-18T11:32:59.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045809029999999999 2024-09-18T11:32:59.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689679+0000", 2024-09-18T11:32:59.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8507000000000002e-05 2024-09-18T11:32:59.204 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.204 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.204 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.204 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:976 4.4 e132/22 4:2cb3e6a5:::smithi11939506-3:30 v 132'195, mlcod=128'192)", 2024-09-18T11:32:59.204 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:59.204 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 56.114571503000001, 2024-09-18T11:32:59.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0058526510000000004, 2024-09-18T11:32:59.204 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.204 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.204 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:59.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:59.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:32:59.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:59.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018661+0000", 2024-09-18T11:32:59.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5565000000000001e-05 2024-09-18T11:32:59.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018672+0000", 2024-09-18T11:32:59.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0210000000000001e-05 2024-09-18T11:32:59.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018678+0000", 2024-09-18T11:32:59.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1539999999999999e-06 2024-09-18T11:32:59.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018716+0000", 2024-09-18T11:32:59.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8389000000000001e-05 2024-09-18T11:32:59.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018737+0000", 2024-09-18T11:32:59.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0611e-05 2024-09-18T11:32:59.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.024458+0000", 2024-09-18T11:32:59.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057208629999999996 2024-09-18T11:32:59.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.024499+0000", 2024-09-18T11:32:59.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1100999999999998e-05 2024-09-18T11:32:59.208 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.208 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.208 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.208 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:823 4.d e137/22 4:b44229a1:::smithi11939506-42:49 v 137'266, mlcod=132'260)", 2024-09-18T11:32:59.208 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:32:59.208 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 50.855922947000003, 2024-09-18T11:32:59.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054497770000000003, 2024-09-18T11:32:59.209 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.209 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.209 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:32:59.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:32:59.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277294+0000", 2024-09-18T11:32:59.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:32:59.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277299+0000", 2024-09-18T11:32:59.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6280000000000004e-06 2024-09-18T11:32:59.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.210 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277306+0000", 2024-09-18T11:32:59.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.533e-06 2024-09-18T11:32:59.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277316+0000", 2024-09-18T11:32:59.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0148e-05 2024-09-18T11:32:59.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.211 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.211 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277868+0000", 2024-09-18T11:32:59.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00055199999999999997 2024-09-18T11:32:59.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277898+0000", 2024-09-18T11:32:59.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0264999999999999e-05 2024-09-18T11:32:59.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.282715+0000", 2024-09-18T11:32:59.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048165619999999999 2024-09-18T11:32:59.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.212 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.212 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.282744+0000", 2024-09-18T11:32:59.212 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9855000000000001e-05 2024-09-18T11:32:59.213 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.213 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.213 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.213 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:163 3.0 e138/21 3:02e8b30e:::4456d047c09ae2a9a25ccebfd83e30cf78df6779d63bfba9365afd7301eaeb6a:head v 138'132, mlcod=134'130)", 2024-09-18T11:32:59.213 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:59.213 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 49.551060204000002, 2024-09-18T11:32:59.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051351770000000003, 2024-09-18T11:32:59.213 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.213 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.213 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:59.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:59.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:32:59.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:59.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582232+0000", 2024-09-18T11:32:59.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5333999999999994e-05 2024-09-18T11:32:59.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582242+0000", 2024-09-18T11:32:59.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5200000000000003e-06 2024-09-18T11:32:59.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582248+0000", 2024-09-18T11:32:59.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8000000000000004e-06 2024-09-18T11:32:59.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582305+0000", 2024-09-18T11:32:59.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7813999999999999e-05 2024-09-18T11:32:59.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582329+0000", 2024-09-18T11:32:59.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3414999999999999e-05 2024-09-18T11:32:59.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587257+0000", 2024-09-18T11:32:59.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049279880000000003 2024-09-18T11:32:59.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587293+0000", 2024-09-18T11:32:59.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5802e-05 2024-09-18T11:32:59.218 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.218 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.218 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.218 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3974 4.d e138/22 4:b5a463bf:::smithi11939506-20:head v 138'270, mlcod=137'266)", 2024-09-18T11:32:59.218 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:32:59.218 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 49.547178932999998, 2024-09-18T11:32:59.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011228458, 2024-09-18T11:32:59.218 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.219 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.219 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:32:59.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:32:59.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586038+0000", 2024-09-18T11:32:59.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:59.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587638+0000", 2024-09-18T11:32:59.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001599184 2024-09-18T11:32:59.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587647+0000", 2024-09-18T11:32:59.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3719999999999994e-06 2024-09-18T11:32:59.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587653+0000", 2024-09-18T11:32:59.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.105e-06 2024-09-18T11:32:59.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.221 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.221 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587712+0000", 2024-09-18T11:32:59.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9252000000000001e-05 2024-09-18T11:32:59.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587735+0000", 2024-09-18T11:32:59.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3132000000000001e-05 2024-09-18T11:32:59.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597228+0000", 2024-09-18T11:32:59.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0094930090000000002 2024-09-18T11:32:59.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597267+0000", 2024-09-18T11:32:59.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8693999999999999e-05 2024-09-18T11:32:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3975 4.d e138/22 4:b5a463bf:::smithi11939506-20:head v 138'271, mlcod=137'266)", 2024-09-18T11:32:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 49.540978226999997, 2024-09-18T11:32:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053620450000000002, 2024-09-18T11:32:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:59.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:59.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:32:59.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:59.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592927+0000", 2024-09-18T11:32:59.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00068878699999999995 2024-09-18T11:32:59.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592940+0000", 2024-09-18T11:32:59.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:32:59.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592951+0000", 2024-09-18T11:32:59.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1046000000000001e-05 2024-09-18T11:32:59.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593008+0000", 2024-09-18T11:32:59.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6875e-05 2024-09-18T11:32:59.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593033+0000", 2024-09-18T11:32:59.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5341999999999999e-05 2024-09-18T11:32:59.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597565+0000", 2024-09-18T11:32:59.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045323150000000003 2024-09-18T11:32:59.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597601+0000", 2024-09-18T11:32:59.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6071000000000001e-05 2024-09-18T11:32:59.227 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.227 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.227 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.227 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.227 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.227 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:59.229 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:32:59.233 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:59.227+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:59.329 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:32:59.328+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:32:59.368 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:59.368 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:59.368 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:32:59.368 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:59.377 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:32:59.430 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:32:59.429+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:32:59.469 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:59.467+0000 7fb6950ff640 -1 osd.2 187 handle_osd_map injecting map crc failure 2024-09-18T11:32:59.473 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:59.473 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:32:59.474 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:32:59.474 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:32:59.474 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:59.485 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:32:59.531 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:32:59.529+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:32:59.532 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:32:59.608 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:32:59.608 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:32:59.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:32:59.608 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:32:59.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.608 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:32:59.608 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:59.608 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 168.43538302100001, 2024-09-18T11:32:59.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:32:59.608 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.608 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:59.609 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:59.609 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:59.609 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:59.609 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:32:59.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.609 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:59.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:32:59.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:32:59.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:59.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:32:59.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:32:59.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:32:59.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:32:59.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:32:59.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:32:59.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:32:59.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:32:59.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:32:59.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:32:59.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:32:59.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:32:59.612 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.612 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.612 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.613 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:59.613 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:59.613 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 164.44020502699999, 2024-09-18T11:32:59.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:32:59.613 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.613 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:59.613 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:59.613 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:59.613 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:59.613 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:32:59.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.614 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:59.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:59.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:32:59.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:59.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:32:59.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:32:59.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.615 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:32:59.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:32:59.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:32:59.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:32:59.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:32:59.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:32:59.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:59.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:32:59.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:32:59.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:32:59.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:32:59.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:32:59.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:32:59.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:32:59.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:32:59.618 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.618 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.618 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.618 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:59.618 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:59.618 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 164.44018673799999, 2024-09-18T11:32:59.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:32:59.619 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.619 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:59.619 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:59.619 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:59.619 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:59.619 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:32:59.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.619 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.619 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:59.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:32:59.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:32:59.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:59.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:32:59.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:32:59.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:32:59.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:32:59.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:32:59.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:32:59.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:32:59.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:32:59.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:59.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:32:59.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:32:59.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:32:59.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:32:59.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:32:59.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:32:59.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:32:59.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:32:59.623 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.623 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.623 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.623 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 164.44012596900001, 2024-09-18T11:32:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:32:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:32:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.624 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:32:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:32:59.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:32:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:32:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:32:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:32:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:32:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:32:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:32:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:32:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:32:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:32:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:32:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:32:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:32:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:32:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:32:59.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:32:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:32:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 164.44010364799999, 2024-09-18T11:32:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:32:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:59.629 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:32:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:32:59.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:32:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:32:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:32:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:32:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:32:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:32:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:32:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:32:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:32:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:32:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:32:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:32:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:32:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:32:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:32:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:32:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:32:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.634 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:32:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 159.658978765, 2024-09-18T11:32:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:32:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:32:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:59.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:32:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:32:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:32:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:32:59.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:32:59.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:32:59.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:32:59.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:32:59.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:32:59.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:32:59.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:32:59.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:32:59.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:32:59.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:59.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:32:59.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:32:59.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:59.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:32:59.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:32:59.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:59.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:32:59.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:32:59.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:32:59.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:32:59.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:32:59.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:32:59.642 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.642 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.642 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.643 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2503 3.1 e85/67 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head v 85'117, mlcod=84'115)", 2024-09-18T11:32:59.643 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:59.643 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.62758007799999, 2024-09-18T11:32:59.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086527619999999996, 2024-09-18T11:32:59.643 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.643 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.643 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:59.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:32:59.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977220+0000", 2024-09-18T11:32:59.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:32:59.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980771+0000", 2024-09-18T11:32:59.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035505200000000002 2024-09-18T11:32:59.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980797+0000", 2024-09-18T11:32:59.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.618e-05 2024-09-18T11:32:59.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980807+0000", 2024-09-18T11:32:59.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0433e-05 2024-09-18T11:32:59.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980861+0000", 2024-09-18T11:32:59.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3415999999999998e-05 2024-09-18T11:32:59.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980885+0000", 2024-09-18T11:32:59.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4212000000000001e-05 2024-09-18T11:32:59.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985826+0000", 2024-09-18T11:32:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049412450000000004 2024-09-18T11:32:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985875+0000", 2024-09-18T11:32:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9116000000000002e-05 2024-09-18T11:32:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.648 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2649 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:59.648 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:59.648 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.27343670899999, 2024-09-18T11:32:59.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566240000000007, 2024-09-18T11:32:59.648 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.648 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:59.648 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:59.648 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:59.648 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:59.648 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2649 2024-09-18T11:32:59.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:32:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331365+0000", 2024-09-18T11:32:59.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331369+0000", 2024-09-18T11:32:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.551e-06 2024-09-18T11:32:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331377+0000", 2024-09-18T11:32:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8769999999999993e-06 2024-09-18T11:32:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331381+0000", 2024-09-18T11:32:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.16e-06 2024-09-18T11:32:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338973+0000", 2024-09-18T11:32:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075918460000000002 2024-09-18T11:32:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339048+0000", 2024-09-18T11:32:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5813000000000001e-05 2024-09-18T11:32:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339322+0000", 2024-09-18T11:32:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027399899999999998 2024-09-18T11:32:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2650 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:32:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.27341649100001, 2024-09-18T11:32:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084392180000000001, 2024-09-18T11:32:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:32:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2650 2024-09-18T11:32:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:59.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:32:59.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331388+0000", 2024-09-18T11:32:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3429999999999998e-06 2024-09-18T11:32:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331394+0000", 2024-09-18T11:32:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.19e-06 2024-09-18T11:32:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331398+0000", 2024-09-18T11:32:59.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.495e-06 2024-09-18T11:32:59.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339362+0000", 2024-09-18T11:32:59.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079641799999999995 2024-09-18T11:32:59.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339460+0000", 2024-09-18T11:32:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8329e-05 2024-09-18T11:32:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339825+0000", 2024-09-18T11:32:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036510500000000001 2024-09-18T11:32:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:127 3.2 3:4d3b277a:::3266c7aaa53a7d2c10c3c54d852d6cbbed49fb041bc0df41a3ac83b56e2b0615:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:32:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 96.560401552000002, 2024-09-18T11:32:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093148009999999993, 2024-09-18T11:32:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:59.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:32:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044400+0000", 2024-09-18T11:32:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044442+0000", 2024-09-18T11:32:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2247999999999999e-05 2024-09-18T11:32:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044453+0000", 2024-09-18T11:32:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0617e-05 2024-09-18T11:32:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044457+0000", 2024-09-18T11:32:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2520000000000001e-06 2024-09-18T11:32:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044495+0000", 2024-09-18T11:32:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7710000000000003e-05 2024-09-18T11:32:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:32:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044499+0000", 2024-09-18T11:32:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6269999999999999e-06 2024-09-18T11:32:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048239+0000", 2024-09-18T11:32:59.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037396719999999999 2024-09-18T11:32:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048351+0000", 2024-09-18T11:32:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112342 2024-09-18T11:32:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048545+0000", 2024-09-18T11:32:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000193881 2024-09-18T11:32:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.052689+0000", 2024-09-18T11:32:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041435780000000002 2024-09-18T11:32:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053654+0000", 2024-09-18T11:32:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096558199999999999 2024-09-18T11:32:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053683+0000", 2024-09-18T11:32:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8850999999999999e-05 2024-09-18T11:32:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053716+0000", 2024-09-18T11:32:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2468000000000002e-05 2024-09-18T11:32:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3209 4.0 4:019b6e09:::smithi11939506-40:head [write 638732~557432] snapc 42=[42,41,3f,3e,3d,3b,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-18T11:32:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 82.506719512999993, 2024-09-18T11:32:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091774999999999999, 2024-09-18T11:32:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3209 2024-09-18T11:32:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:32:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098080+0000", 2024-09-18T11:32:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:32:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098676+0000", 2024-09-18T11:32:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059598099999999998 2024-09-18T11:32:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098706+0000", 2024-09-18T11:32:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0020000000000001e-05 2024-09-18T11:32:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098716+0000", 2024-09-18T11:32:59.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0225e-05 2024-09-18T11:32:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098782+0000", 2024-09-18T11:32:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5530000000000004e-05 2024-09-18T11:32:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098883+0000", 2024-09-18T11:32:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101609 2024-09-18T11:32:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:32:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.099098+0000", 2024-09-18T11:32:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000215052 2024-09-18T11:32:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.102511+0000", 2024-09-18T11:32:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034129429999999999 2024-09-18T11:32:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:59.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107165+0000", 2024-09-18T11:32:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046531480000000002 2024-09-18T11:32:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107217+0000", 2024-09-18T11:32:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2058000000000002e-05 2024-09-18T11:32:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107260+0000", 2024-09-18T11:32:59.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3865000000000001e-05 2024-09-18T11:32:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:162 3.1 e138/67 3:8c826a1f:::e682cfc1e82c88472b9acf1cd97978bc105aa035f10b685bb4cace95809b52bf:head v 138'176, mlcod=136'174)", 2024-09-18T11:32:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:32:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 50.021245768, 2024-09-18T11:32:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010445024000000001, 2024-09-18T11:32:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.671 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:32:59.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:32:59.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583556+0000", 2024-09-18T11:32:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:32:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584899+0000", 2024-09-18T11:32:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001343056 2024-09-18T11:32:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584929+0000", 2024-09-18T11:32:59.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0133000000000001e-05 2024-09-18T11:32:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584937+0000", 2024-09-18T11:32:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6380000000000004e-06 2024-09-18T11:32:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584990+0000", 2024-09-18T11:32:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3186000000000002e-05 2024-09-18T11:32:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585012+0000", 2024-09-18T11:32:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1474000000000001e-05 2024-09-18T11:32:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593963+0000", 2024-09-18T11:32:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089512859999999993 2024-09-18T11:32:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594002+0000", 2024-09-18T11:32:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8859999999999997e-05 2024-09-18T11:32:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:166 3.1 e138/67 3:a73e635b:::b6ee478f0f661ec7bac3d17c2ca2fdaff71a47570f64844a7ba79bbb2a0e8d9d:head v 138'177, mlcod=136'174)", 2024-09-18T11:32:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:32:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 50.019675884000002, 2024-09-18T11:32:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089018680000000003, 2024-09-18T11:32:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:32:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:32:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585126+0000", 2024-09-18T11:32:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585211+0000", 2024-09-18T11:32:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5432999999999993e-05 2024-09-18T11:32:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585220+0000", 2024-09-18T11:32:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5879999999999998e-06 2024-09-18T11:32:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585225+0000", 2024-09-18T11:32:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8999999999999997e-06 2024-09-18T11:32:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585705+0000", 2024-09-18T11:32:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00048042799999999997 2024-09-18T11:32:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585724+0000", 2024-09-18T11:32:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8352e-05 2024-09-18T11:32:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594008+0000", 2024-09-18T11:32:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082840919999999998 2024-09-18T11:32:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594028+0000", 2024-09-18T11:32:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0869e-05 2024-09-18T11:32:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:168 3.1 e138/67 3:b64d7bad:::7feb22256bdcec217720fd46184ab43ced8923ed66ec6319c57f078d9f123262:head v 138'178, mlcod=136'174)", 2024-09-18T11:32:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 50.018684151000002, 2024-09-18T11:32:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079290650000000008, 2024-09-18T11:32:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.680 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:32:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:59.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586191+0000", 2024-09-18T11:32:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3627000000000005e-05 2024-09-18T11:32:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586202+0000", 2024-09-18T11:32:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0488e-05 2024-09-18T11:32:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586207+0000", 2024-09-18T11:32:59.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1719999999999998e-06 2024-09-18T11:32:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586239+0000", 2024-09-18T11:32:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2570000000000002e-05 2024-09-18T11:32:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586271+0000", 2024-09-18T11:32:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1341999999999999e-05 2024-09-18T11:32:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594031+0000", 2024-09-18T11:32:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077597459999999997 2024-09-18T11:32:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594047+0000", 2024-09-18T11:32:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6858e-05 2024-09-18T11:32:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.684 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4267 3.1 3:8d8d13e9:::92886a4340d03039afb2c877fad76e3afe7a80ee04388d55256c386022cd5d63:head [write 0~2203648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:32:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:32:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 32.547652171000003, 2024-09-18T11:32:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0092856740000000007, 2024-09-18T11:32:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4267 2024-09-18T11:32:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:32:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:32:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057149+0000", 2024-09-18T11:32:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058175+0000", 2024-09-18T11:32:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001026765 2024-09-18T11:32:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058190+0000", 2024-09-18T11:32:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4167e-05 2024-09-18T11:32:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058195+0000", 2024-09-18T11:32:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.823e-06 2024-09-18T11:32:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058247+0000", 2024-09-18T11:32:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1304999999999998e-05 2024-09-18T11:32:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058422+0000", 2024-09-18T11:32:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017527100000000001 2024-09-18T11:32:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:59.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058609+0000", 2024-09-18T11:32:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018714299999999999 2024-09-18T11:32:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.062721+0000", 2024-09-18T11:32:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041118869999999998 2024-09-18T11:32:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066291+0000", 2024-09-18T11:32:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035694860000000002 2024-09-18T11:32:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066328+0000", 2024-09-18T11:32:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7864000000000002e-05 2024-09-18T11:32:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066436+0000", 2024-09-18T11:32:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010762 2024-09-18T11:32:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.690 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:90 3.1 3:a224eaa1:::22b064a083114a2756d85a07d2762a393d3e92dd898a5334bc045c9031244d76:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:32:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:40.057559+0000", 2024-09-18T11:32:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.547243391999999, 2024-09-18T11:32:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085024620000000006, 2024-09-18T11:32:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057559+0000", 2024-09-18T11:32:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057559+0000", 2024-09-18T11:32:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057558+0000", 2024-09-18T11:32:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057563+0000", 2024-09-18T11:32:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7990000000000001e-06 2024-09-18T11:32:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057572+0000", 2024-09-18T11:32:59.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0510000000000002e-06 2024-09-18T11:32:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057576+0000", 2024-09-18T11:32:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2509999999999996e-06 2024-09-18T11:32:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057624+0000", 2024-09-18T11:32:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7636000000000003e-05 2024-09-18T11:32:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057722+0000", 2024-09-18T11:32:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7676999999999993e-05 2024-09-18T11:32:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057883+0000", 2024-09-18T11:32:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000160922 2024-09-18T11:32:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059927+0000", 2024-09-18T11:32:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020447350000000002 2024-09-18T11:32:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.065977+0000", 2024-09-18T11:32:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060497679999999996 2024-09-18T11:32:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066023+0000", 2024-09-18T11:32:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6041000000000002e-05 2024-09-18T11:32:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066062+0000", 2024-09-18T11:32:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8469e-05 2024-09-18T11:32:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.695 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.5.13:42 3.1 3:8372521c:::d4b4fff22a71c591f3e871de3fede050f6721df41a8c639a614c0b94ba6762c2:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:32:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:40.058327+0000", 2024-09-18T11:32:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.546475719, 2024-09-18T11:32:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079034699999999992, 2024-09-18T11:32:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058327+0000", 2024-09-18T11:32:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058327+0000", 2024-09-18T11:32:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058325+0000", 2024-09-18T11:32:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058339+0000", 2024-09-18T11:32:59.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3337e-05 2024-09-18T11:32:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058357+0000", 2024-09-18T11:32:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8615e-05 2024-09-18T11:32:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058366+0000", 2024-09-18T11:32:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9460000000000004e-06 2024-09-18T11:32:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058658+0000", 2024-09-18T11:32:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029212200000000002 2024-09-18T11:32:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058757+0000", 2024-09-18T11:32:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9025999999999997e-05 2024-09-18T11:32:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058914+0000", 2024-09-18T11:32:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000156236 2024-09-18T11:32:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:59.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.060326+0000", 2024-09-18T11:32:59.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014118710000000001 2024-09-18T11:32:59.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:59.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066122+0000", 2024-09-18T11:32:59.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057962029999999998 2024-09-18T11:32:59.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066192+0000", 2024-09-18T11:32:59.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0533999999999999e-05 2024-09-18T11:32:59.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066230+0000", 2024-09-18T11:32:59.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7914000000000003e-05 2024-09-18T11:32:59.701 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.701 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.701 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.701 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.5.13:43 3.1 3:b44664f3:::7f31751f088e4f7577561c03b406e8a8bc48ebfdd310b40206061655307aebe3:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:32:59.701 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:40.058386+0000", 2024-09-18T11:32:59.701 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.546416691000001, 2024-09-18T11:32:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079575740000000002, 2024-09-18T11:32:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058386+0000", 2024-09-18T11:32:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058386+0000", 2024-09-18T11:32:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058385+0000", 2024-09-18T11:32:59.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:32:59.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058392+0000", 2024-09-18T11:32:59.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6200000000000001e-06 2024-09-18T11:32:59.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058400+0000", 2024-09-18T11:32:59.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7849999999999997e-06 2024-09-18T11:32:59.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058408+0000", 2024-09-18T11:32:59.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1770000000000001e-06 2024-09-18T11:32:59.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059302+0000", 2024-09-18T11:32:59.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000894851 2024-09-18T11:32:59.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059400+0000", 2024-09-18T11:32:59.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7460000000000005e-05 2024-09-18T11:32:59.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:59.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059520+0000", 2024-09-18T11:32:59.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000120457 2024-09-18T11:32:59.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:59.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.060333+0000", 2024-09-18T11:32:59.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00081255599999999998 2024-09-18T11:32:59.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:59.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066274+0000", 2024-09-18T11:32:59.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059409700000000003 2024-09-18T11:32:59.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066306+0000", 2024-09-18T11:32:59.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2020000000000002e-05 2024-09-18T11:32:59.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066343+0000", 2024-09-18T11:32:59.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7438999999999998e-05 2024-09-18T11:32:59.707 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.707 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.707 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.707 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.4.13:145 3.2 3:435dd00a:::773f1f20723e27798deecfe7048ee8e76eeec814e6d6ad791ed305ff28c69be2:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:32:59.707 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:45.321055+0000", 2024-09-18T11:32:59.707 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 14.283747886, 2024-09-18T11:32:59.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079066459999999998, 2024-09-18T11:32:59.707 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.707 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.708 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321055+0000", 2024-09-18T11:32:59.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321055+0000", 2024-09-18T11:32:59.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321053+0000", 2024-09-18T11:32:59.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:32:59.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321062+0000", 2024-09-18T11:32:59.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0399999999999998e-06 2024-09-18T11:32:59.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321076+0000", 2024-09-18T11:32:59.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3869999999999999e-05 2024-09-18T11:32:59.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321083+0000", 2024-09-18T11:32:59.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6159999999999998e-06 2024-09-18T11:32:59.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321156+0000", 2024-09-18T11:32:59.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3093e-05 2024-09-18T11:32:59.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321274+0000", 2024-09-18T11:32:59.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000118353 2024-09-18T11:32:59.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:59.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321443+0000", 2024-09-18T11:32:59.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016836600000000001 2024-09-18T11:32:59.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:59.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.324281+0000", 2024-09-18T11:32:59.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028381270000000002 2024-09-18T11:32:59.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:59.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.328785+0000", 2024-09-18T11:32:59.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045048370000000003 2024-09-18T11:32:59.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.328910+0000", 2024-09-18T11:32:59.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012438600000000001 2024-09-18T11:32:59.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.328961+0000", 2024-09-18T11:32:59.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1375000000000002e-05 2024-09-18T11:32:59.713 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.713 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.713 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.713 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4790 3.2 3:79129b11:::2f02c59c01a730562a65f21445c8af0e4a8aab3b969b8a3abf39e7b1bc84f918:head [write 0~2177536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-18T11:32:59.713 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.041705+0000", 2024-09-18T11:32:59.713 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 6.5630972730000003, 2024-09-18T11:32:59.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011619633000000001, 2024-09-18T11:32:59.713 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:32:59.714 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:32:59.714 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:32:59.714 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:32:59.714 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:32:59.714 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4790 2024-09-18T11:32:59.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.714 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:32:59.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:32:59.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.041705+0000", 2024-09-18T11:32:59.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:32:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.041705+0000", 2024-09-18T11:32:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:32:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:32:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.041703+0000", 2024-09-18T11:32:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:32:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:32:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.042939+0000", 2024-09-18T11:32:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012361570000000001 2024-09-18T11:32:59.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:32:59.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.042961+0000", 2024-09-18T11:32:59.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1531000000000001e-05 2024-09-18T11:32:59.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:32:59.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.042969+0000", 2024-09-18T11:32:59.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.892e-06 2024-09-18T11:32:59.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:32:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.043013+0000", 2024-09-18T11:32:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3921000000000002e-05 2024-09-18T11:32:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:32:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.043147+0000", 2024-09-18T11:32:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013394599999999999 2024-09-18T11:32:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:32:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.043291+0000", 2024-09-18T11:32:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014399500000000001 2024-09-18T11:32:59.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:32:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.048224+0000", 2024-09-18T11:32:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049331160000000004 2024-09-18T11:32:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:32:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053239+0000", 2024-09-18T11:32:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050155720000000003 2024-09-18T11:32:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:32:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053287+0000", 2024-09-18T11:32:59.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8126e-05 2024-09-18T11:32:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:32:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:32:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:32:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053325+0000", 2024-09-18T11:32:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7499999999999997e-05 2024-09-18T11:32:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:32:59.719 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:32:59.719 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:32:59.721 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:32:59.721 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:32:59.722 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:32:59.722 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:32:59.723 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:32:59.723 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:32:59.723 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:32:59.724 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:32:59.724 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:32:59.724 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:32:59.719+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:32:59.738 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:32:59.739 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:32:59.754 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:32:59.755 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:32:59.769 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:32:59.770 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:32:59.786 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:32:59.786 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:32:59.832 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:59.833 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:59.838 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:32:59.836+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:32:59.842 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:32:59.922 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:32:59.921+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:32:59.940 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:32:59.940 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:32:59.949 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:33:00.023 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:00.021+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:33:00.048 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T11:33:00.049 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:00.049 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:00.053 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:33:00.052+0000 7f6bf3052640 -1 osd.5 188 handle_osd_map injecting map crc failure 2024-09-18T11:33:00.059 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:33:00.063 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-18T11:33:00.124 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:33:00.123+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:33:00.156 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:00.156 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:00.164 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:33:00.225 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:00.223+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:33:00.301 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:00.301 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:00.309 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:33:00.326 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:33:00.324+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:33:00.407 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:00.408 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:00.416 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:33:00.459 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:00.458+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:00.515 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:00.515 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:00.523 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:33:00.528 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:00.526+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:33:00.628 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:00.627+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:33:00.660 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:00.661 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:00.669 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:33:00.729 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:33:00.728+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:33:00.767 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:00.767 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:00.775 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:33:00.830 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:00.828+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:33:00.916 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:00.916 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:00.924 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:33:00.968 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:00.967+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:01.022 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:01.022 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:01.031 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:33:01.054 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.5 to 1 (10000) 2024-09-18T11:33:01.055 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:01.052+0000 7f697aead640 -1 osd.3 189 handle_osd_map injecting map crc failure 2024-09-18T11:33:01.055 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T11:33:01.073 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:01.072+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:01.074 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-18T11:33:01.158 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:01.156+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:33:01.180 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:01.181 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:01.190 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:33:01.233 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:01.232+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:01.295 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:01.296 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:01.305 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:33:01.348 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:01.346+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:33:01.404 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:01.405 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:01.413 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:33:01.454 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:01.452+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:33:01.535 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:33:01.533+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:33:01.550 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:01.550 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:01.559 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:33:01.643 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:01.641+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:33:01.696 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:01.697 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:01.705 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:33:01.746 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:01.744+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:33:01.797 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:33:01.838 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:33:01.837+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:33:01.845 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:01.845 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:01.854 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:33:01.914 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:33:01.914 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:33:01.930 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:33:01.930 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:33:01.939 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:33:01.937+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:33:01.944 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:33:01.945 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:33:01.959 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:01.959 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:33:01.959 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:33:01.959 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:33:01.959 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.959 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:262 4.9 e53/22 4:99feffe9:::smithi11939506-45:13 v 53'17, mlcod=51'14)", 2024-09-18T11:33:01.959 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:33:01.960 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 147.957255812, 2024-09-18T11:33:01.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013878679, 2024-09-18T11:33:01.960 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:01.960 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:01.960 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:01.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.960 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:01.960 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:33:01.960 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:01.960 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.960 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:01.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:33:01.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:01.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:01.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:33:01.961 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:01.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.961 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.961 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:01.961 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998997+0000", 2024-09-18T11:33:01.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1099999999999999e-06 2024-09-18T11:33:01.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:01.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999008+0000", 2024-09-18T11:33:01.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1321000000000001e-05 2024-09-18T11:33:01.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:01.962 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999015+0000", 2024-09-18T11:33:01.962 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3000000000000004e-06 2024-09-18T11:33:01.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.962 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.962 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:01.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999055+0000", 2024-09-18T11:33:01.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0058000000000002e-05 2024-09-18T11:33:01.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:01.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999070+0000", 2024-09-18T11:33:01.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4328000000000001e-05 2024-09-18T11:33:01.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.963 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:01.963 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012465+0000", 2024-09-18T11:33:01.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013395638 2024-09-18T11:33:01.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:01.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012870+0000", 2024-09-18T11:33:01.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040423400000000001 2024-09-18T11:33:01.964 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:01.964 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:01.964 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:01.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.964 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:205 4.b e53/22 4:dba5bd39:::smithi11939506-17:13 v 53'77, mlcod=51'74)", 2024-09-18T11:33:01.964 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:33:01.964 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 147.95720655100001, 2024-09-18T11:33:01.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013359586, 2024-09-18T11:33:01.964 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:01.965 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:01.965 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:01.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:01.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:33:01.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:01.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:01.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:33:01.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:01.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:01.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:33:01.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:01.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:01.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999048+0000", 2024-09-18T11:33:01.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3550000000000005e-06 2024-09-18T11:33:01.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:01.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999056+0000", 2024-09-18T11:33:01.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3259999999999996e-06 2024-09-18T11:33:01.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:01.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999063+0000", 2024-09-18T11:33:01.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6409999999999996e-06 2024-09-18T11:33:01.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:01.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999098+0000", 2024-09-18T11:33:01.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5203999999999997e-05 2024-09-18T11:33:01.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:01.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999115+0000", 2024-09-18T11:33:01.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6478999999999999e-05 2024-09-18T11:33:01.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:01.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012362+0000", 2024-09-18T11:33:01.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01324768 2024-09-18T11:33:01.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.968 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:01.968 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012400+0000", 2024-09-18T11:33:01.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.731e-05 2024-09-18T11:33:01.968 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:01.968 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:01.968 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:01.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.968 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2937 4.7 4:e301aefd:::smithi11939506-19:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:33:01.968 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:33:01.969 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 98.921406238000003, 2024-09-18T11:33:01.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019287667000000001, 2024-09-18T11:33:01.969 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:01.969 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:01.969 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:01.969 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:01.969 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:01.969 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2937 2024-09-18T11:33:01.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.969 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:01.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:01.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:33:01.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:01.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:01.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:33:01.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:01.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:01.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:33:01.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:33:01.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:01.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034842+0000", 2024-09-18T11:33:01.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.372e-06 2024-09-18T11:33:01.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:01.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034846+0000", 2024-09-18T11:33:01.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3919999999999996e-06 2024-09-18T11:33:01.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:01.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034848+0000", 2024-09-18T11:33:01.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3539999999999998e-06 2024-09-18T11:33:01.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:01.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034893+0000", 2024-09-18T11:33:01.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4861000000000003e-05 2024-09-18T11:33:01.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:01.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034965+0000", 2024-09-18T11:33:01.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1860999999999995e-05 2024-09-18T11:33:01.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:01.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.054128+0000", 2024-09-18T11:33:01.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019163039 2024-09-18T11:33:01.973 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:01.973 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:01.973 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:01.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.973 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3453 4.f 4:f5d7df41:::smithi11939506-16:head [write 1813152~490434] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:33:01.973 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:33:01.973 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 75.93349431, 2024-09-18T11:33:01.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050685840000000003, 2024-09-18T11:33:01.973 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:01.973 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:01.973 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:01.974 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:01.974 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:01.974 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3453 2024-09-18T11:33:01.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.974 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:01.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:01.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:33:01.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:01.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:01.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:33:01.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:01.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:01.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022751+0000", 2024-09-18T11:33:01.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:01.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:01.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022871+0000", 2024-09-18T11:33:01.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000119781 2024-09-18T11:33:01.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:01.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022882+0000", 2024-09-18T11:33:01.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1627000000000001e-05 2024-09-18T11:33:01.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:01.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022887+0000", 2024-09-18T11:33:01.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2390000000000004e-06 2024-09-18T11:33:01.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:01.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023123+0000", 2024-09-18T11:33:01.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000236598 2024-09-18T11:33:01.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:01.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023207+0000", 2024-09-18T11:33:01.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4303999999999998e-05 2024-09-18T11:33:01.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:01.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023367+0000", 2024-09-18T11:33:01.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015983600000000001 2024-09-18T11:33:01.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:01.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073237+0000", 2024-09-18T11:33:01.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049869400000000001 2024-09-18T11:33:01.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:01.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073352+0000", 2024-09-18T11:33:01.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011579899999999999 2024-09-18T11:33:01.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:01.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073394+0000", 2024-09-18T11:33:01.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1445e-05 2024-09-18T11:33:01.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:01.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073438+0000", 2024-09-18T11:33:01.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4261999999999997e-05 2024-09-18T11:33:01.979 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:01.979 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:01.979 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:01.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.979 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3454 4.f 4:f5d7df41:::smithi11939506-16:head [setxattr _header (57) in=7b,truncate 2991679] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:33:01.980 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:33:01.980 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 75.932795867999999, 2024-09-18T11:33:01.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050365656000000002, 2024-09-18T11:33:01.980 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:01.980 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:01.980 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:01.980 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:01.980 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:01.980 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3454 2024-09-18T11:33:01.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.980 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:01.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:01.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:33:01.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:01.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:01.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:33:01.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:01.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:01.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023450+0000", 2024-09-18T11:33:01.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:01.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:01.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023456+0000", 2024-09-18T11:33:01.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8329999999999999e-06 2024-09-18T11:33:01.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:01.982 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023465+0000", 2024-09-18T11:33:01.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1130000000000007e-06 2024-09-18T11:33:01.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.982 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:01.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023470+0000", 2024-09-18T11:33:01.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9139999999999999e-06 2024-09-18T11:33:01.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:01.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072611+0000", 2024-09-18T11:33:01.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141643999999998 2024-09-18T11:33:01.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:01.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072729+0000", 2024-09-18T11:33:01.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117667 2024-09-18T11:33:01.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:01.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072887+0000", 2024-09-18T11:33:01.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015760000000000001 2024-09-18T11:33:01.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:01.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073558+0000", 2024-09-18T11:33:01.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00067171699999999995 2024-09-18T11:33:01.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:01.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073764+0000", 2024-09-18T11:33:01.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000205749 2024-09-18T11:33:01.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:01.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073790+0000", 2024-09-18T11:33:01.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5643000000000002e-05 2024-09-18T11:33:01.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:01.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073816+0000", 2024-09-18T11:33:01.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6588e-05 2024-09-18T11:33:01.985 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:01.985 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:01.985 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:01.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.986 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3455 4.f 4:f5d7df41:::smithi11939506-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:33:01.986 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:33:01.986 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 75.93276625, 2024-09-18T11:33:01.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050460466000000002, 2024-09-18T11:33:01.986 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:01.986 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:01.986 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:01.986 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:01.986 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:01.986 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3455 2024-09-18T11:33:01.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.986 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:01.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:01.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:33:01.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:01.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:01.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:33:01.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:01.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:01.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:33:01.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:01.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:01.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023483+0000", 2024-09-18T11:33:01.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.695e-06 2024-09-18T11:33:01.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:01.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023489+0000", 2024-09-18T11:33:01.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1079999999999998e-06 2024-09-18T11:33:01.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:01.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023492+0000", 2024-09-18T11:33:01.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.478e-06 2024-09-18T11:33:01.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:01.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073273+0000", 2024-09-18T11:33:01.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049780638000000002 2024-09-18T11:33:01.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:01.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073343+0000", 2024-09-18T11:33:01.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9770000000000005e-05 2024-09-18T11:33:01.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:01.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073846+0000", 2024-09-18T11:33:01.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050289499999999995 2024-09-18T11:33:01.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:01.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073886+0000", 2024-09-18T11:33:01.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0658999999999998e-05 2024-09-18T11:33:01.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:01.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073941+0000", 2024-09-18T11:33:01.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4642000000000003e-05 2024-09-18T11:33:01.990 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:01.990 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:01.990 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:01.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.990 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3456 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:33:01.990 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:33:01.991 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 75.932741184999998, 2024-09-18T11:33:01.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049859120999999999, 2024-09-18T11:33:01.991 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:01.991 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:01.991 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:01.991 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:01.991 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:01.991 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3456 2024-09-18T11:33:01.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.991 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:01.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:01.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:33:01.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:01.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:01.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:33:01.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:01.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:01.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:33:01.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:01.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:01.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023508+0000", 2024-09-18T11:33:01.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9490000000000001e-06 2024-09-18T11:33:01.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:01.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023513+0000", 2024-09-18T11:33:01.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3800000000000002e-06 2024-09-18T11:33:01.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:01.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023517+0000", 2024-09-18T11:33:01.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5300000000000001e-06 2024-09-18T11:33:01.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:01.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073262+0000", 2024-09-18T11:33:01.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049745194999999999 2024-09-18T11:33:01.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:01.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073365+0000", 2024-09-18T11:33:01.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102462 2024-09-18T11:33:01.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:01.994 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:01.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:01.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.994 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3457 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:33:01.994 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:33:01.994 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 75.932724089000004, 2024-09-18T11:33:01.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049928567, 2024-09-18T11:33:01.995 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:01.995 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:01.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:01.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:01.995 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:01.995 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3457 2024-09-18T11:33:01.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.995 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:01.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:01.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:33:01.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:01.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:01.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:33:01.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:01.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:01.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023522+0000", 2024-09-18T11:33:01.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:01.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:01.996 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023524+0000", 2024-09-18T11:33:01.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8500000000000001e-06 2024-09-18T11:33:01.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.996 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:01.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023529+0000", 2024-09-18T11:33:01.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7509999999999999e-06 2024-09-18T11:33:01.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:01.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023532+0000", 2024-09-18T11:33:01.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.163e-06 2024-09-18T11:33:01.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:01.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073386+0000", 2024-09-18T11:33:01.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049853899 2024-09-18T11:33:01.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073451+0000", 2024-09-18T11:33:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5197000000000006e-05 2024-09-18T11:33:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3458 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:33:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:33:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 75.932711531999999, 2024-09-18T11:33:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.05001659, 2024-09-18T11:33:01.998 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3458 2024-09-18T11:33:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:33:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:01.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:33:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:33:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023537+0000", 2024-09-18T11:33:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.64e-06 2024-09-18T11:33:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023543+0000", 2024-09-18T11:33:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3369999999999996e-06 2024-09-18T11:33:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023546+0000", 2024-09-18T11:33:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:33:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073466+0000", 2024-09-18T11:33:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049919719000000001 2024-09-18T11:33:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073552+0000", 2024-09-18T11:33:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6168999999999994e-05 2024-09-18T11:33:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(client.4590.0:3453 4.f e116/22 ondisk, result = 0)", 2024-09-18T11:33:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:33:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 75.931106447999994, 2024-09-18T11:33:02.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048099441, 2024-09-18T11:33:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:33:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:33:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025139+0000", 2024-09-18T11:33:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025144+0000", 2024-09-18T11:33:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4059999999999998e-06 2024-09-18T11:33:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072212+0000", 2024-09-18T11:33:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047068482000000002 2024-09-18T11:33:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072230+0000", 2024-09-18T11:33:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7632999999999999e-05 2024-09-18T11:33:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073224+0000", 2024-09-18T11:33:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099436899999999994 2024-09-18T11:33:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073231+0000", 2024-09-18T11:33:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-09-18T11:33:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073240+0000", 2024-09-18T11:33:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5699999999999993e-06 2024-09-18T11:33:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.006 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3930 4.b 4:d3ae8215:::smithi11939506-41:head [write 632029~547288] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:33:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:33:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.930190768000003, 2024-09-18T11:33:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018962711, 2024-09-18T11:33:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3930 2024-09-18T11:33:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:33:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:33:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026055+0000", 2024-09-18T11:33:02.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026334+0000", 2024-09-18T11:33:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000278931 2024-09-18T11:33:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026344+0000", 2024-09-18T11:33:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0716999999999999e-05 2024-09-18T11:33:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026350+0000", 2024-09-18T11:33:02.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7620000000000001e-06 2024-09-18T11:33:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028429+0000", 2024-09-18T11:33:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020787319999999998 2024-09-18T11:33:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028525+0000", 2024-09-18T11:33:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6638999999999994e-05 2024-09-18T11:33:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:33:02.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028720+0000", 2024-09-18T11:33:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000194525 2024-09-18T11:33:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.036996+0000", 2024-09-18T11:33:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082756140000000006 2024-09-18T11:33:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044952+0000", 2024-09-18T11:33:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566740000000004 2024-09-18T11:33:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.011 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044985+0000", 2024-09-18T11:33:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3081999999999999e-05 2024-09-18T11:33:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045019+0000", 2024-09-18T11:33:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3257999999999999e-05 2024-09-18T11:33:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.012 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3932 4.b 4:d3ae8215:::smithi11939506-41:head [write 2796249~437859] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:33:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:33:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.918184128, 2024-09-18T11:33:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01458329, 2024-09-18T11:33:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:02.013 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3932 2024-09-18T11:33:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:33:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:33:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038061+0000", 2024-09-18T11:33:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038341+0000", 2024-09-18T11:33:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027989099999999999 2024-09-18T11:33:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038353+0000", 2024-09-18T11:33:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2247e-05 2024-09-18T11:33:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038358+0000", 2024-09-18T11:33:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-18T11:33:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043226+0000", 2024-09-18T11:33:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048673049999999997 2024-09-18T11:33:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043284+0000", 2024-09-18T11:33:02.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8338999999999998e-05 2024-09-18T11:33:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:33:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043367+0000", 2024-09-18T11:33:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3120000000000004e-05 2024-09-18T11:33:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.051544+0000", 2024-09-18T11:33:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00817734 2024-09-18T11:33:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052528+0000", 2024-09-18T11:33:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098318499999999992 2024-09-18T11:33:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052610+0000", 2024-09-18T11:33:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2300999999999997e-05 2024-09-18T11:33:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052646+0000", 2024-09-18T11:33:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5955999999999999e-05 2024-09-18T11:33:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3935 4.b 4:d3ae8215:::smithi11939506-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:33:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:33:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 58.914847021, 2024-09-18T11:33:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013142333000000001, 2024-09-18T11:33:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3935 2024-09-18T11:33:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:33:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:33:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041399+0000", 2024-09-18T11:33:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041401+0000", 2024-09-18T11:33:02.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4890000000000001e-06 2024-09-18T11:33:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041404+0000", 2024-09-18T11:33:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5240000000000001e-06 2024-09-18T11:33:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041407+0000", 2024-09-18T11:33:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2230000000000001e-06 2024-09-18T11:33:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045699+0000", 2024-09-18T11:33:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042918970000000002 2024-09-18T11:33:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045750+0000", 2024-09-18T11:33:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1332999999999998e-05 2024-09-18T11:33:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054438+0000", 2024-09-18T11:33:02.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086882569999999996 2024-09-18T11:33:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054485+0000", 2024-09-18T11:33:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7151000000000003e-05 2024-09-18T11:33:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054542+0000", 2024-09-18T11:33:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6706999999999998e-05 2024-09-18T11:33:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3969 4.4 4:2cb3e6a5:::smithi11939506-3:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:33:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.382606756000001, 2024-09-18T11:33:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.024248042000000001, 2024-09-18T11:33:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3969 2024-09-18T11:33:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:33:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:33:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573639+0000", 2024-09-18T11:33:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573646+0000", 2024-09-18T11:33:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6569999999999997e-06 2024-09-18T11:33:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573662+0000", 2024-09-18T11:33:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5187999999999999e-05 2024-09-18T11:33:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573670+0000", 2024-09-18T11:33:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5399999999999996e-06 2024-09-18T11:33:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:33:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8592999999999999e-05 2024-09-18T11:33:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573818+0000", 2024-09-18T11:33:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9388999999999995e-05 2024-09-18T11:33:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597888+0000", 2024-09-18T11:33:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.024069859999999998 2024-09-18T11:33:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3973 4.7 4:e73ba68f:::smithi11939506-1:head [tier-promote] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:33:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.382507677, 2024-09-18T11:33:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.023911297000000001, 2024-09-18T11:33:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3973 2024-09-18T11:33:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:33:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:33:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573738+0000", 2024-09-18T11:33:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573743+0000", 2024-09-18T11:33:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0839999999999999e-06 2024-09-18T11:33:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573752+0000", 2024-09-18T11:33:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3080000000000008e-06 2024-09-18T11:33:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573758+0000", 2024-09-18T11:33:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2659999999999998e-06 2024-09-18T11:33:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573803+0000", 2024-09-18T11:33:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5243e-05 2024-09-18T11:33:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573897+0000", 2024-09-18T11:33:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3398000000000003e-05 2024-09-18T11:33:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597650+0000", 2024-09-18T11:33:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.023753672999999999 2024-09-18T11:33:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4443 4.d 4:b9bb642b:::smithi11939506-27:67 [read 0~3512696,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T11:33:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:36.030743+0000", 2024-09-18T11:33:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 25.925504048000001, 2024-09-18T11:33:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013696302, 2024-09-18T11:33:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4443 2024-09-18T11:33:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030743+0000", 2024-09-18T11:33:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030743+0000", 2024-09-18T11:33:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030741+0000", 2024-09-18T11:33:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030751+0000", 2024-09-18T11:33:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6979999999999993e-06 2024-09-18T11:33:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030763+0000", 2024-09-18T11:33:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2311e-05 2024-09-18T11:33:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030769+0000", 2024-09-18T11:33:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6060000000000002e-06 2024-09-18T11:33:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.036777+0000", 2024-09-18T11:33:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060079089999999996 2024-09-18T11:33:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.036945+0000", 2024-09-18T11:33:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016867100000000001 2024-09-18T11:33:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044439+0000", 2024-09-18T11:33:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074937149999999998 2024-09-18T11:33:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4444 4.d 4:b9bb642b:::smithi11939506-27:67 [sparse-read 0~1262403] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T11:33:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:33:02.038 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 25.925469254999999, 2024-09-18T11:33:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.014179987, 2024-09-18T11:33:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4444 2024-09-18T11:33:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:33:02.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:33:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:33:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030780+0000", 2024-09-18T11:33:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.163e-06 2024-09-18T11:33:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030786+0000", 2024-09-18T11:33:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879999999999998e-06 2024-09-18T11:33:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030791+0000", 2024-09-18T11:33:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2579999999999997e-06 2024-09-18T11:33:02.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044477+0000", 2024-09-18T11:33:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013686188 2024-09-18T11:33:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044614+0000", 2024-09-18T11:33:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013727300000000001 2024-09-18T11:33:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044957+0000", 2024-09-18T11:33:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000343331 2024-09-18T11:33:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4445 4.d 4:b9bb642b:::smithi11939506-27:67 [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T11:33:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:36.030796+0000", 2024-09-18T11:33:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 25.925450979000001, 2024-09-18T11:33:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.015916046999999999, 2024-09-18T11:33:02.043 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.044 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:02.044 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:02.044 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:02.044 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:02.044 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4445 2024-09-18T11:33:02.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.044 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030796+0000", 2024-09-18T11:33:02.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030796+0000", 2024-09-18T11:33:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030795+0000", 2024-09-18T11:33:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030799+0000", 2024-09-18T11:33:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5209999999999998e-06 2024-09-18T11:33:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030806+0000", 2024-09-18T11:33:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7599999999999997e-06 2024-09-18T11:33:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030810+0000", 2024-09-18T11:33:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0489999999999997e-06 2024-09-18T11:33:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.045277+0000", 2024-09-18T11:33:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.014467683 2024-09-18T11:33:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.045394+0000", 2024-09-18T11:33:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116969 2024-09-18T11:33:02.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.046712+0000", 2024-09-18T11:33:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013174669999999999 2024-09-18T11:33:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4683 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865643,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:33:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:45.318891+0000", 2024-09-18T11:33:02.048 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 16.637355362000001, 2024-09-18T11:33:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.017494045999999999, 2024-09-18T11:33:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4683 2024-09-18T11:33:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318891+0000", 2024-09-18T11:33:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318891+0000", 2024-09-18T11:33:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318891+0000", 2024-09-18T11:33:02.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318894+0000", 2024-09-18T11:33:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1379999999999999e-06 2024-09-18T11:33:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318900+0000", 2024-09-18T11:33:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6239999999999999e-06 2024-09-18T11:33:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318903+0000", 2024-09-18T11:33:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8249999999999998e-06 2024-09-18T11:33:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.330916+0000", 2024-09-18T11:33:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012012063999999999 2024-09-18T11:33:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.331011+0000", 2024-09-18T11:33:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5388000000000002e-05 2024-09-18T11:33:02.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.336385+0000", 2024-09-18T11:33:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005374429 2024-09-18T11:33:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4684 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865643,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:33:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:45.318908+0000", 2024-09-18T11:33:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 16.637339056999998, 2024-09-18T11:33:02.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.021211049999999999, 2024-09-18T11:33:02.054 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.054 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:02.054 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:02.054 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:02.054 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:02.054 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4684 2024-09-18T11:33:02.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.054 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318908+0000", 2024-09-18T11:33:02.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318908+0000", 2024-09-18T11:33:02.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318907+0000", 2024-09-18T11:33:02.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318910+0000", 2024-09-18T11:33:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.79e-06 2024-09-18T11:33:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318917+0000", 2024-09-18T11:33:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5740000000000001e-06 2024-09-18T11:33:02.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318920+0000", 2024-09-18T11:33:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5729999999999999e-06 2024-09-18T11:33:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.336370+0000", 2024-09-18T11:33:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01745025 2024-09-18T11:33:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.336506+0000", 2024-09-18T11:33:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013603700000000001 2024-09-18T11:33:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.340119+0000", 2024-09-18T11:33:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00361218 2024-09-18T11:33:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.058 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.058 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:02.059 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:33:02.060 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:33:02.061 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:33:02.061 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:33:02.061 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:33:02.062 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:33:02.062 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:33:02.062 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:33:02.062 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:33:02.063 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:33:02.063 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:33:02.064 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.6 to 1 (10000) 2024-09-18T11:33:02.064 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is unset 2024-09-18T11:33:02.064 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:33:02.058+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:33:02.073 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-18T11:33:02.160 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:33:02.159+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:33:02.220 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:02.220 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:33:02.220 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:33:02.220 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:02.229 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:33:02.273 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:02.271+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:33:02.327 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:02.327 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:33:02.327 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:33:02.328 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:33:02.328 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:02.336 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:33:02.377 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:02.375+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:33:02.462 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:02.460+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:33:02.476 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:02.476 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:33:02.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:33:02.477 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:33:02.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.477 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:33:02.477 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:33:02.477 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 162.491294305, 2024-09-18T11:33:02.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:33:02.477 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.477 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:02.477 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:33:02.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:33:02.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:33:02.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:02.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:33:02.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:33:02.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:33:02.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:33:02.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:33:02.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:33:02.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:33:02.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:33:02.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:33:02.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:33:02.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:02.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:33:02.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:33:02.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:33:02.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:33:02.481 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.481 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.482 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.482 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:33:02.482 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:33:02.482 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 153.256463352, 2024-09-18T11:33:02.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:33:02.482 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.482 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:02.482 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:33:02.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:33:02.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.483 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:33:02.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:33:02.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:33:02.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:33:02.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:33:02.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:33:02.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:33:02.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:33:02.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:33:02.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:33:02.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:33:02.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:33:02.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:02.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:33:02.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:33:02.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:33:02.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:33:02.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.486 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.487 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:33:02.487 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:33:02.487 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 148.08255706700001, 2024-09-18T11:33:02.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:33:02.487 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.487 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:02.487 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:33:02.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:33:02.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.488 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:33:02.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:02.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.488 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:33:02.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:33:02.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:33:02.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:33:02.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:33:02.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:33:02.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:33:02.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:33:02.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:33:02.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:33:02.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:02.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:33:02.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:33:02.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:33:02.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:33:02.491 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.491 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.491 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.491 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:304 4.6 e69/67 4:67c1765a:::smithi11939506-30:17 v 69'35, mlcod=58'31)", 2024-09-18T11:33:02.491 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:33:02.491 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 131.128715973, 2024-09-18T11:33:02.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006377611, 2024-09-18T11:33:02.491 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.492 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:02.492 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:33:02.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:33:02.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:33:02.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:02.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345961+0000", 2024-09-18T11:33:02.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2050000000000008e-06 2024-09-18T11:33:02.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345969+0000", 2024-09-18T11:33:02.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0339999999999997e-06 2024-09-18T11:33:02.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345976+0000", 2024-09-18T11:33:02.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-18T11:33:02.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346026+0000", 2024-09-18T11:33:02.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9286999999999998e-05 2024-09-18T11:33:02.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346050+0000", 2024-09-18T11:33:02.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4397e-05 2024-09-18T11:33:02.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:02.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352293+0000", 2024-09-18T11:33:02.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062427170000000001 2024-09-18T11:33:02.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352330+0000", 2024-09-18T11:33:02.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7348999999999998e-05 2024-09-18T11:33:02.495 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.495 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.495 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.496 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:347 4.d e69/22 4:b44229a1:::smithi11939506-42:19 v 69'99, mlcod=65'93)", 2024-09-18T11:33:02.496 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:33:02.496 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 131.12298277799999, 2024-09-18T11:33:02.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056998760000000004, 2024-09-18T11:33:02.496 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.496 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:02.496 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:33:02.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:33:02.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351684+0000", 2024-09-18T11:33:02.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:02.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351693+0000", 2024-09-18T11:33:02.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-18T11:33:02.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351704+0000", 2024-09-18T11:33:02.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1077000000000001e-05 2024-09-18T11:33:02.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351709+0000", 2024-09-18T11:33:02.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5419999999999999e-06 2024-09-18T11:33:02.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351956+0000", 2024-09-18T11:33:02.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000247252 2024-09-18T11:33:02.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351975+0000", 2024-09-18T11:33:02.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9201000000000001e-05 2024-09-18T11:33:02.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:02.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357334+0000", 2024-09-18T11:33:02.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053588749999999999 2024-09-18T11:33:02.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357385+0000", 2024-09-18T11:33:02.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1406999999999998e-05 2024-09-18T11:33:02.500 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.500 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.500 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.500 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1956 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'37, mlcod=58'31)", 2024-09-18T11:33:02.500 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:33:02.500 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 131.12203786500001, 2024-09-18T11:33:02.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010958842, 2024-09-18T11:33:02.500 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.500 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:02.500 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:33:02.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:33:02.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:33:02.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:02.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352996+0000", 2024-09-18T11:33:02.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036622 2024-09-18T11:33:02.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353011+0000", 2024-09-18T11:33:02.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4849e-05 2024-09-18T11:33:02.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353017+0000", 2024-09-18T11:33:02.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:33:02.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353068+0000", 2024-09-18T11:33:02.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0695000000000002e-05 2024-09-18T11:33:02.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353092+0000", 2024-09-18T11:33:02.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4260000000000002e-05 2024-09-18T11:33:02.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:02.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363551+0000", 2024-09-18T11:33:02.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010459154999999999 2024-09-18T11:33:02.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.504 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363589+0000", 2024-09-18T11:33:02.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8161000000000003e-05 2024-09-18T11:33:02.504 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.504 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.504 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.504 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1957 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'38, mlcod=65'33)", 2024-09-18T11:33:02.505 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:33:02.505 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 131.11562199799999, 2024-09-18T11:33:02.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050499400000000002, 2024-09-18T11:33:02.505 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.505 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:02.505 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:33:02.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:33:02.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:33:02.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:02.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359214+0000", 2024-09-18T11:33:02.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000168047 2024-09-18T11:33:02.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359227+0000", 2024-09-18T11:33:02.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3124e-05 2024-09-18T11:33:02.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359235+0000", 2024-09-18T11:33:02.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0760000000000003e-06 2024-09-18T11:33:02.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359287+0000", 2024-09-18T11:33:02.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.151e-05 2024-09-18T11:33:02.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359310+0000", 2024-09-18T11:33:02.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-09-18T11:33:02.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:02.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364065+0000", 2024-09-18T11:33:02.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047549439999999997 2024-09-18T11:33:02.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364096+0000", 2024-09-18T11:33:02.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1423000000000001e-05 2024-09-18T11:33:02.509 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.509 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.509 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.509 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2300 4.4 e77/22 4:2c93196f:::smithi11939506-28:head v 77'94, mlcod=77'90)", 2024-09-18T11:33:02.509 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:33:02.509 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 121.48655977999999, 2024-09-18T11:33:02.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050425380000000001, 2024-09-18T11:33:02.509 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.509 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:02.510 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:33:02.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:33:02.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988108+0000", 2024-09-18T11:33:02.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:02.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988388+0000", 2024-09-18T11:33:02.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028087700000000001 2024-09-18T11:33:02.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988405+0000", 2024-09-18T11:33:02.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6565000000000001e-05 2024-09-18T11:33:02.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988412+0000", 2024-09-18T11:33:02.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9709999999999998e-06 2024-09-18T11:33:02.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988464+0000", 2024-09-18T11:33:02.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1949e-05 2024-09-18T11:33:02.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988490+0000", 2024-09-18T11:33:02.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5780999999999999e-05 2024-09-18T11:33:02.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:02.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993112+0000", 2024-09-18T11:33:02.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046228399999999996 2024-09-18T11:33:02.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993151+0000", 2024-09-18T11:33:02.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8587999999999997e-05 2024-09-18T11:33:02.513 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.513 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.514 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.514 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.13:63 3.0 e78/21 3:24ebb740:::smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 78'80, mlcod=73'78)", 2024-09-18T11:33:02.514 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:33:02.514 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 121.069570364, 2024-09-18T11:33:02.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052262230000000003, 2024-09-18T11:33:02.514 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.514 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:02.514 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:33:02.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:33:02.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.515 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405097+0000", 2024-09-18T11:33:02.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:02.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405103+0000", 2024-09-18T11:33:02.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8780000000000003e-06 2024-09-18T11:33:02.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405117+0000", 2024-09-18T11:33:02.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3441000000000001e-05 2024-09-18T11:33:02.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405121+0000", 2024-09-18T11:33:02.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7099999999999998e-06 2024-09-18T11:33:02.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405158+0000", 2024-09-18T11:33:02.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6477999999999999e-05 2024-09-18T11:33:02.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405173+0000", 2024-09-18T11:33:02.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4796e-05 2024-09-18T11:33:02.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:02.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410285+0000", 2024-09-18T11:33:02.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051118919999999998 2024-09-18T11:33:02.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410324+0000", 2024-09-18T11:33:02.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9616999999999997e-05 2024-09-18T11:33:02.518 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.518 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.518 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.518 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.13:18 3.0 e91/21 3:35e3c617:::2e64dd273da7619bd64b294f52905c1b9405a1ced2a8fd3c0b5bc481bc9732a6:head v 91'89, mlcod=85'87)", 2024-09-18T11:33:02.518 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:33:02.519 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.45279495299999, 2024-09-18T11:33:02.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048558539999999997, 2024-09-18T11:33:02.519 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.519 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:02.519 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:33:02.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:33:02.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:33:02.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:02.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021880+0000", 2024-09-18T11:33:02.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5860000000000001e-06 2024-09-18T11:33:02.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021890+0000", 2024-09-18T11:33:02.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8290000000000003e-06 2024-09-18T11:33:02.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021895+0000", 2024-09-18T11:33:02.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2480000000000004e-06 2024-09-18T11:33:02.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022166+0000", 2024-09-18T11:33:02.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027073499999999998 2024-09-18T11:33:02.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022184+0000", 2024-09-18T11:33:02.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8082000000000001e-05 2024-09-18T11:33:02.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:02.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026693+0000", 2024-09-18T11:33:02.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045093479999999998 2024-09-18T11:33:02.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026729+0000", 2024-09-18T11:33:02.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6282999999999998e-05 2024-09-18T11:33:02.523 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.523 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.523 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.523 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:573 4.4 e91/22 4:2c93196f:::smithi11939506-28:26 v 91'135, mlcod=90'131)", 2024-09-18T11:33:02.523 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:33:02.523 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.450328364, 2024-09-18T11:33:02.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050722739999999999, 2024-09-18T11:33:02.523 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.523 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:02.523 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:33:02.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:33:02.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024339+0000", 2024-09-18T11:33:02.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:02.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024349+0000", 2024-09-18T11:33:02.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6339999999999996e-06 2024-09-18T11:33:02.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024362+0000", 2024-09-18T11:33:02.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-18T11:33:02.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024368+0000", 2024-09-18T11:33:02.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8649999999999998e-06 2024-09-18T11:33:02.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024431+0000", 2024-09-18T11:33:02.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2780999999999994e-05 2024-09-18T11:33:02.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024448+0000", 2024-09-18T11:33:02.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7388999999999999e-05 2024-09-18T11:33:02.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:02.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029385+0000", 2024-09-18T11:33:02.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004936523 2024-09-18T11:33:02.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029412+0000", 2024-09-18T11:33:02.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7702999999999999e-05 2024-09-18T11:33:02.527 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.527 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.527 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.528 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:574 4.4 e91/22 4:2cb3e6a5:::smithi11939506-3:1e v 91'137, mlcod=90'131)", 2024-09-18T11:33:02.528 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:33:02.528 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.449890321, 2024-09-18T11:33:02.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070802850000000004, 2024-09-18T11:33:02.528 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.528 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:02.528 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:33:02.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:33:02.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024777+0000", 2024-09-18T11:33:02.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:02.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024785+0000", 2024-09-18T11:33:02.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8099999999999998e-06 2024-09-18T11:33:02.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024794+0000", 2024-09-18T11:33:02.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5839999999999995e-06 2024-09-18T11:33:02.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024798+0000", 2024-09-18T11:33:02.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.031e-06 2024-09-18T11:33:02.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024830+0000", 2024-09-18T11:33:02.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2268999999999999e-05 2024-09-18T11:33:02.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024847+0000", 2024-09-18T11:33:02.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7306999999999999e-05 2024-09-18T11:33:02.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:02.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031834+0000", 2024-09-18T11:33:02.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069868380000000004 2024-09-18T11:33:02.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031858+0000", 2024-09-18T11:33:02.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4082000000000001e-05 2024-09-18T11:33:02.532 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.532 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.532 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.532 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:541 4.d e91/22 4:b9bb642b:::smithi11939506-27:19 v 91'158, mlcod=91'154)", 2024-09-18T11:33:02.532 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:33:02.532 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.442509629, 2024-09-18T11:33:02.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052185, 2024-09-18T11:33:02.532 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.533 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:02.533 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:33:02.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:33:02.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032158+0000", 2024-09-18T11:33:02.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:02.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032165+0000", 2024-09-18T11:33:02.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-09-18T11:33:02.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032176+0000", 2024-09-18T11:33:02.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1460999999999999e-05 2024-09-18T11:33:02.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032182+0000", 2024-09-18T11:33:02.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0870000000000003e-06 2024-09-18T11:33:02.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032224+0000", 2024-09-18T11:33:02.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1372999999999998e-05 2024-09-18T11:33:02.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032246+0000", 2024-09-18T11:33:02.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2158e-05 2024-09-18T11:33:02.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:02.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037348+0000", 2024-09-18T11:33:02.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051025319999999999 2024-09-18T11:33:02.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037377+0000", 2024-09-18T11:33:02.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8937000000000001e-05 2024-09-18T11:33:02.536 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.536 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.536 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.537 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4178.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-18T11:33:02.537 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:33:02.537 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 80.533847563999998, 2024-09-18T11:33:02.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048960139999999997, 2024-09-18T11:33:02.537 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.537 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:02.537 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:02.537 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4178", 2024-09-18T11:33:02.537 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/490761693", 2024-09-18T11:33:02.537 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4 2024-09-18T11:33:02.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.538 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:33:02.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:33:02.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940819+0000", 2024-09-18T11:33:02.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:02.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940832+0000", 2024-09-18T11:33:02.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3155e-05 2024-09-18T11:33:02.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940848+0000", 2024-09-18T11:33:02.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6288999999999999e-05 2024-09-18T11:33:02.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940856+0000", 2024-09-18T11:33:02.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4919999999999996e-06 2024-09-18T11:33:02.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945613+0000", 2024-09-18T11:33:02.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047574999999999996 2024-09-18T11:33:02.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945717+0000", 2024-09-18T11:33:02.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103553 2024-09-18T11:33:02.540 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.540 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.541 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.541 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3482 4.6 e117/67 4:67c1765a:::smithi11939506-30:head v 117'120, mlcod=115'116)", 2024-09-18T11:33:02.541 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:33:02.541 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 75.789993340999999, 2024-09-18T11:33:02.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050037479999999997, 2024-09-18T11:33:02.541 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.541 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:02.541 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:33:02.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:33:02.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684674+0000", 2024-09-18T11:33:02.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:02.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684950+0000", 2024-09-18T11:33:02.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027621200000000002 2024-09-18T11:33:02.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684968+0000", 2024-09-18T11:33:02.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8e-05 2024-09-18T11:33:02.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684984+0000", 2024-09-18T11:33:02.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5775000000000001e-05 2024-09-18T11:33:02.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685033+0000", 2024-09-18T11:33:02.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8878000000000002e-05 2024-09-18T11:33:02.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685059+0000", 2024-09-18T11:33:02.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6801999999999999e-05 2024-09-18T11:33:02.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:02.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689640+0000", 2024-09-18T11:33:02.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045809029999999999 2024-09-18T11:33:02.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689679+0000", 2024-09-18T11:33:02.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8507000000000002e-05 2024-09-18T11:33:02.545 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.545 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.545 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.545 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:976 4.4 e132/22 4:2cb3e6a5:::smithi11939506-3:30 v 132'195, mlcod=128'192)", 2024-09-18T11:33:02.545 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:33:02.545 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.456022269999998, 2024-09-18T11:33:02.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0058526510000000004, 2024-09-18T11:33:02.545 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.546 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:02.546 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:33:02.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:33:02.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:33:02.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:02.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018661+0000", 2024-09-18T11:33:02.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5565000000000001e-05 2024-09-18T11:33:02.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018672+0000", 2024-09-18T11:33:02.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0210000000000001e-05 2024-09-18T11:33:02.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018678+0000", 2024-09-18T11:33:02.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1539999999999999e-06 2024-09-18T11:33:02.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018716+0000", 2024-09-18T11:33:02.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8389000000000001e-05 2024-09-18T11:33:02.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018737+0000", 2024-09-18T11:33:02.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0611e-05 2024-09-18T11:33:02.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:02.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.024458+0000", 2024-09-18T11:33:02.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057208629999999996 2024-09-18T11:33:02.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.024499+0000", 2024-09-18T11:33:02.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1100999999999998e-05 2024-09-18T11:33:02.549 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.549 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.549 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.550 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:823 4.d e137/22 4:b44229a1:::smithi11939506-42:49 v 137'266, mlcod=132'260)", 2024-09-18T11:33:02.550 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:33:02.550 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 54.197373714000001, 2024-09-18T11:33:02.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054497770000000003, 2024-09-18T11:33:02.550 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.550 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:02.550 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:33:02.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:33:02.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277294+0000", 2024-09-18T11:33:02.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:33:02.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277299+0000", 2024-09-18T11:33:02.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6280000000000004e-06 2024-09-18T11:33:02.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277306+0000", 2024-09-18T11:33:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.533e-06 2024-09-18T11:33:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277316+0000", 2024-09-18T11:33:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0148e-05 2024-09-18T11:33:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277868+0000", 2024-09-18T11:33:02.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00055199999999999997 2024-09-18T11:33:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277898+0000", 2024-09-18T11:33:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0264999999999999e-05 2024-09-18T11:33:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.282715+0000", 2024-09-18T11:33:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048165619999999999 2024-09-18T11:33:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.282744+0000", 2024-09-18T11:33:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9855000000000001e-05 2024-09-18T11:33:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:163 3.0 e138/21 3:02e8b30e:::4456d047c09ae2a9a25ccebfd83e30cf78df6779d63bfba9365afd7301eaeb6a:head v 138'132, mlcod=134'130)", 2024-09-18T11:33:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:33:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.892510971, 2024-09-18T11:33:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051351770000000003, 2024-09-18T11:33:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:33:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:33:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:33:02.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582232+0000", 2024-09-18T11:33:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5333999999999994e-05 2024-09-18T11:33:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582242+0000", 2024-09-18T11:33:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5200000000000003e-06 2024-09-18T11:33:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582248+0000", 2024-09-18T11:33:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8000000000000004e-06 2024-09-18T11:33:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582305+0000", 2024-09-18T11:33:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7813999999999999e-05 2024-09-18T11:33:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582329+0000", 2024-09-18T11:33:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3414999999999999e-05 2024-09-18T11:33:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587257+0000", 2024-09-18T11:33:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049279880000000003 2024-09-18T11:33:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587293+0000", 2024-09-18T11:33:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5802e-05 2024-09-18T11:33:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.558 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3974 4.d e138/22 4:b5a463bf:::smithi11939506-20:head v 138'270, mlcod=137'266)", 2024-09-18T11:33:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:33:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.888629700000003, 2024-09-18T11:33:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011228458, 2024-09-18T11:33:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:33:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:33:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586038+0000", 2024-09-18T11:33:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587638+0000", 2024-09-18T11:33:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001599184 2024-09-18T11:33:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587647+0000", 2024-09-18T11:33:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3719999999999994e-06 2024-09-18T11:33:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587653+0000", 2024-09-18T11:33:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.105e-06 2024-09-18T11:33:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587712+0000", 2024-09-18T11:33:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9252000000000001e-05 2024-09-18T11:33:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587735+0000", 2024-09-18T11:33:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3132000000000001e-05 2024-09-18T11:33:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597228+0000", 2024-09-18T11:33:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0094930090000000002 2024-09-18T11:33:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597267+0000", 2024-09-18T11:33:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8693999999999999e-05 2024-09-18T11:33:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3975 4.d e138/22 4:b5a463bf:::smithi11939506-20:head v 138'271, mlcod=137'266)", 2024-09-18T11:33:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:33:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 52.882428994000001, 2024-09-18T11:33:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053620450000000002, 2024-09-18T11:33:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:33:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:33:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:33:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592927+0000", 2024-09-18T11:33:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00068878699999999995 2024-09-18T11:33:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592940+0000", 2024-09-18T11:33:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:33:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592951+0000", 2024-09-18T11:33:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1046000000000001e-05 2024-09-18T11:33:02.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593008+0000", 2024-09-18T11:33:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6875e-05 2024-09-18T11:33:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593033+0000", 2024-09-18T11:33:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5341999999999999e-05 2024-09-18T11:33:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597565+0000", 2024-09-18T11:33:02.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045323150000000003 2024-09-18T11:33:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597601+0000", 2024-09-18T11:33:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6071000000000001e-05 2024-09-18T11:33:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:02.567 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:02.567 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:02.569 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:33:02.569 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:02.567+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:33:02.670 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:02.668+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:33:02.707 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:02.707 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:33:02.708 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:33:02.708 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:02.716 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:33:02.800 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:33:02.798+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:33:02.855 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:33:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:33:02.855 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:33:02.855 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:02.864 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:33:02.871 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:33:02.870+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:33:02.972 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:02.969+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:33:03.005 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:03.005 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:33:03.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:33:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:33:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:33:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:33:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 171.833681066, 2024-09-18T11:33:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:33:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:03.006 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:33:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:33:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:33:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:33:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:33:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:33:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:33:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:33:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:33:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:33:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:33:03.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:33:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:33:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:33:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:33:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:33:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:03.010 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:33:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:33:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 167.83850307200001, 2024-09-18T11:33:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:33:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.011 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:33:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:33:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:33:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:33:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:33:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:33:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:33:03.013 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:33:03.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:33:03.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:33:03.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:33:03.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:33:03.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:03.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:33:03.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:33:03.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:33:03.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:33:03.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:03.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:33:03.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:33:03.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:33:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:33:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:33:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:33:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 167.83848478300001, 2024-09-18T11:33:03.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:33:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:33:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:33:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:33:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:33:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:33:03.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:33:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:33:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:33:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:33:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:33:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.019 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:33:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:33:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:33:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:33:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:33:03.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:33:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:33:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:33:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:33:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:33:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.021 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:33:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:33:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 167.838424014, 2024-09-18T11:33:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:33:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:33:03.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:33:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:33:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:03.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:33:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:33:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:33:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:33:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:33:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:33:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:33:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:33:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:33:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:33:03.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:33:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:33:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:33:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:33:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:33:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:03.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:33:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:33:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:33:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:33:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 167.83840169300001, 2024-09-18T11:33:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:33:03.027 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:33:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:33:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:33:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:33:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:33:03.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:33:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:33:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:33:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:33:03.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:33:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:33:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:33:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:33:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:33:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:33:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:33:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:33:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:33:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:33:03.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:33:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:33:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:33:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 163.05727680999999, 2024-09-18T11:33:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:33:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:03.033 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:33:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:33:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:33:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:33:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:33:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:33:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:33:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:03.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:33:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:33:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:33:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:33:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:33:03.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:33:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:33:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:33:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:33:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:33:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:33:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:33:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:33:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:33:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:33:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:33:03.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:33:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:33:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2503 3.1 e85/67 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head v 85'117, mlcod=84'115)", 2024-09-18T11:33:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:33:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 114.025878123, 2024-09-18T11:33:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086527619999999996, 2024-09-18T11:33:03.039 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:33:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:33:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977220+0000", 2024-09-18T11:33:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:33:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980771+0000", 2024-09-18T11:33:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035505200000000002 2024-09-18T11:33:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:03.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980797+0000", 2024-09-18T11:33:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.618e-05 2024-09-18T11:33:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980807+0000", 2024-09-18T11:33:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0433e-05 2024-09-18T11:33:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980861+0000", 2024-09-18T11:33:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3415999999999998e-05 2024-09-18T11:33:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980885+0000", 2024-09-18T11:33:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4212000000000001e-05 2024-09-18T11:33:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985826+0000", 2024-09-18T11:33:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049412450000000004 2024-09-18T11:33:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985875+0000", 2024-09-18T11:33:03.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9116000000000002e-05 2024-09-18T11:33:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2649 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:33:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:33:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.671734754, 2024-09-18T11:33:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566240000000007, 2024-09-18T11:33:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:03.044 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2649 2024-09-18T11:33:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:33:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:33:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331365+0000", 2024-09-18T11:33:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331369+0000", 2024-09-18T11:33:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.551e-06 2024-09-18T11:33:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331377+0000", 2024-09-18T11:33:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8769999999999993e-06 2024-09-18T11:33:03.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331381+0000", 2024-09-18T11:33:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.16e-06 2024-09-18T11:33:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338973+0000", 2024-09-18T11:33:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075918460000000002 2024-09-18T11:33:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:03.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339048+0000", 2024-09-18T11:33:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5813000000000001e-05 2024-09-18T11:33:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339322+0000", 2024-09-18T11:33:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027399899999999998 2024-09-18T11:33:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2650 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:33:03.048 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:33:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 108.671714536, 2024-09-18T11:33:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084392180000000001, 2024-09-18T11:33:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2650 2024-09-18T11:33:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:03.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:33:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:33:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:33:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331388+0000", 2024-09-18T11:33:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3429999999999998e-06 2024-09-18T11:33:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331394+0000", 2024-09-18T11:33:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.19e-06 2024-09-18T11:33:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331398+0000", 2024-09-18T11:33:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.495e-06 2024-09-18T11:33:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339362+0000", 2024-09-18T11:33:03.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079641799999999995 2024-09-18T11:33:03.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:03.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339460+0000", 2024-09-18T11:33:03.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8329e-05 2024-09-18T11:33:03.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:03.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339825+0000", 2024-09-18T11:33:03.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036510500000000001 2024-09-18T11:33:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:127 3.2 3:4d3b277a:::3266c7aaa53a7d2c10c3c54d852d6cbbed49fb041bc0df41a3ac83b56e2b0615:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:33:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:33:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.958699597000006, 2024-09-18T11:33:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093148009999999993, 2024-09-18T11:33:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:03.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:33:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:33:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044400+0000", 2024-09-18T11:33:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:03.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044442+0000", 2024-09-18T11:33:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2247999999999999e-05 2024-09-18T11:33:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044453+0000", 2024-09-18T11:33:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0617e-05 2024-09-18T11:33:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044457+0000", 2024-09-18T11:33:03.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2520000000000001e-06 2024-09-18T11:33:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044495+0000", 2024-09-18T11:33:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7710000000000003e-05 2024-09-18T11:33:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044499+0000", 2024-09-18T11:33:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6269999999999999e-06 2024-09-18T11:33:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048239+0000", 2024-09-18T11:33:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037396719999999999 2024-09-18T11:33:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048351+0000", 2024-09-18T11:33:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112342 2024-09-18T11:33:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048545+0000", 2024-09-18T11:33:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000193881 2024-09-18T11:33:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.052689+0000", 2024-09-18T11:33:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041435780000000002 2024-09-18T11:33:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053654+0000", 2024-09-18T11:33:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096558199999999999 2024-09-18T11:33:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053683+0000", 2024-09-18T11:33:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8850999999999999e-05 2024-09-18T11:33:03.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053716+0000", 2024-09-18T11:33:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2468000000000002e-05 2024-09-18T11:33:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3209 4.0 4:019b6e09:::smithi11939506-40:head [write 638732~557432] snapc 42=[42,41,3f,3e,3d,3b,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-18T11:33:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:33:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.905017557999997, 2024-09-18T11:33:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091774999999999999, 2024-09-18T11:33:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3209 2024-09-18T11:33:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.060 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:33:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:33:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098080+0000", 2024-09-18T11:33:03.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:33:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098676+0000", 2024-09-18T11:33:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059598099999999998 2024-09-18T11:33:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098706+0000", 2024-09-18T11:33:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0020000000000001e-05 2024-09-18T11:33:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098716+0000", 2024-09-18T11:33:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0225e-05 2024-09-18T11:33:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098782+0000", 2024-09-18T11:33:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5530000000000004e-05 2024-09-18T11:33:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098883+0000", 2024-09-18T11:33:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101609 2024-09-18T11:33:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:33:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.099098+0000", 2024-09-18T11:33:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000215052 2024-09-18T11:33:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.102511+0000", 2024-09-18T11:33:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034129429999999999 2024-09-18T11:33:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107165+0000", 2024-09-18T11:33:03.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046531480000000002 2024-09-18T11:33:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107217+0000", 2024-09-18T11:33:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2058000000000002e-05 2024-09-18T11:33:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107260+0000", 2024-09-18T11:33:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3865000000000001e-05 2024-09-18T11:33:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:03.065 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:162 3.1 e138/67 3:8c826a1f:::e682cfc1e82c88472b9acf1cd97978bc105aa035f10b685bb4cace95809b52bf:head v 138'176, mlcod=136'174)", 2024-09-18T11:33:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:33:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.419543812999997, 2024-09-18T11:33:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010445024000000001, 2024-09-18T11:33:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:33:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:33:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583556+0000", 2024-09-18T11:33:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584899+0000", 2024-09-18T11:33:03.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001343056 2024-09-18T11:33:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584929+0000", 2024-09-18T11:33:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0133000000000001e-05 2024-09-18T11:33:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584937+0000", 2024-09-18T11:33:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6380000000000004e-06 2024-09-18T11:33:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.068 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584990+0000", 2024-09-18T11:33:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3186000000000002e-05 2024-09-18T11:33:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585012+0000", 2024-09-18T11:33:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1474000000000001e-05 2024-09-18T11:33:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593963+0000", 2024-09-18T11:33:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089512859999999993 2024-09-18T11:33:03.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594002+0000", 2024-09-18T11:33:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8859999999999997e-05 2024-09-18T11:33:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:166 3.1 e138/67 3:a73e635b:::b6ee478f0f661ec7bac3d17c2ca2fdaff71a47570f64844a7ba79bbb2a0e8d9d:head v 138'177, mlcod=136'174)", 2024-09-18T11:33:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:33:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.417973928999999, 2024-09-18T11:33:03.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089018680000000003, 2024-09-18T11:33:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:33:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:33:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585126+0000", 2024-09-18T11:33:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585211+0000", 2024-09-18T11:33:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5432999999999993e-05 2024-09-18T11:33:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:03.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585220+0000", 2024-09-18T11:33:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5879999999999998e-06 2024-09-18T11:33:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585225+0000", 2024-09-18T11:33:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8999999999999997e-06 2024-09-18T11:33:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585705+0000", 2024-09-18T11:33:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00048042799999999997 2024-09-18T11:33:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585724+0000", 2024-09-18T11:33:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8352e-05 2024-09-18T11:33:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594008+0000", 2024-09-18T11:33:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082840919999999998 2024-09-18T11:33:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594028+0000", 2024-09-18T11:33:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0869e-05 2024-09-18T11:33:03.074 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:168 3.1 e138/67 3:b64d7bad:::7feb22256bdcec217720fd46184ab43ced8923ed66ec6319c57f078d9f123262:head v 138'178, mlcod=136'174)", 2024-09-18T11:33:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:33:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 53.416982195999999, 2024-09-18T11:33:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079290650000000008, 2024-09-18T11:33:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:33:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:33:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:33:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:03.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586191+0000", 2024-09-18T11:33:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3627000000000005e-05 2024-09-18T11:33:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586202+0000", 2024-09-18T11:33:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0488e-05 2024-09-18T11:33:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586207+0000", 2024-09-18T11:33:03.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1719999999999998e-06 2024-09-18T11:33:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586239+0000", 2024-09-18T11:33:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2570000000000002e-05 2024-09-18T11:33:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586271+0000", 2024-09-18T11:33:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1341999999999999e-05 2024-09-18T11:33:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594031+0000", 2024-09-18T11:33:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077597459999999997 2024-09-18T11:33:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594047+0000", 2024-09-18T11:33:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6858e-05 2024-09-18T11:33:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.079 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4267 3.1 3:8d8d13e9:::92886a4340d03039afb2c877fad76e3afe7a80ee04388d55256c386022cd5d63:head [write 0~2203648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:33:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:33:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 35.945950216, 2024-09-18T11:33:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0092856740000000007, 2024-09-18T11:33:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4267 2024-09-18T11:33:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.080 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:33:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:33:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:03.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057149+0000", 2024-09-18T11:33:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058175+0000", 2024-09-18T11:33:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001026765 2024-09-18T11:33:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058190+0000", 2024-09-18T11:33:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4167e-05 2024-09-18T11:33:03.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058195+0000", 2024-09-18T11:33:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.823e-06 2024-09-18T11:33:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058247+0000", 2024-09-18T11:33:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1304999999999998e-05 2024-09-18T11:33:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058422+0000", 2024-09-18T11:33:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017527100000000001 2024-09-18T11:33:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058609+0000", 2024-09-18T11:33:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018714299999999999 2024-09-18T11:33:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.062721+0000", 2024-09-18T11:33:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041118869999999998 2024-09-18T11:33:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066291+0000", 2024-09-18T11:33:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035694860000000002 2024-09-18T11:33:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066328+0000", 2024-09-18T11:33:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7864000000000002e-05 2024-09-18T11:33:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066436+0000", 2024-09-18T11:33:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010762 2024-09-18T11:33:03.085 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:90 3.1 3:a224eaa1:::22b064a083114a2756d85a07d2762a393d3e92dd898a5334bc045c9031244d76:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:33:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:40.057559+0000", 2024-09-18T11:33:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.945541436999999, 2024-09-18T11:33:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085024620000000006, 2024-09-18T11:33:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057559+0000", 2024-09-18T11:33:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057559+0000", 2024-09-18T11:33:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057558+0000", 2024-09-18T11:33:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:03.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057563+0000", 2024-09-18T11:33:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7990000000000001e-06 2024-09-18T11:33:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057572+0000", 2024-09-18T11:33:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0510000000000002e-06 2024-09-18T11:33:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057576+0000", 2024-09-18T11:33:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2509999999999996e-06 2024-09-18T11:33:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057624+0000", 2024-09-18T11:33:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7636000000000003e-05 2024-09-18T11:33:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057722+0000", 2024-09-18T11:33:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7676999999999993e-05 2024-09-18T11:33:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057883+0000", 2024-09-18T11:33:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000160922 2024-09-18T11:33:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059927+0000", 2024-09-18T11:33:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020447350000000002 2024-09-18T11:33:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.065977+0000", 2024-09-18T11:33:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060497679999999996 2024-09-18T11:33:03.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066023+0000", 2024-09-18T11:33:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6041000000000002e-05 2024-09-18T11:33:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066062+0000", 2024-09-18T11:33:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8469e-05 2024-09-18T11:33:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.5.13:42 3.1 3:8372521c:::d4b4fff22a71c591f3e871de3fede050f6721df41a8c639a614c0b94ba6762c2:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:33:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:40.058327+0000", 2024-09-18T11:33:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.944773764000001, 2024-09-18T11:33:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079034699999999992, 2024-09-18T11:33:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058327+0000", 2024-09-18T11:33:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058327+0000", 2024-09-18T11:33:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058325+0000", 2024-09-18T11:33:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058339+0000", 2024-09-18T11:33:03.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3337e-05 2024-09-18T11:33:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058357+0000", 2024-09-18T11:33:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8615e-05 2024-09-18T11:33:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058366+0000", 2024-09-18T11:33:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9460000000000004e-06 2024-09-18T11:33:03.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058658+0000", 2024-09-18T11:33:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029212200000000002 2024-09-18T11:33:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058757+0000", 2024-09-18T11:33:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9025999999999997e-05 2024-09-18T11:33:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:03.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058914+0000", 2024-09-18T11:33:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000156236 2024-09-18T11:33:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.060326+0000", 2024-09-18T11:33:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014118710000000001 2024-09-18T11:33:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066122+0000", 2024-09-18T11:33:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057962029999999998 2024-09-18T11:33:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066192+0000", 2024-09-18T11:33:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0533999999999999e-05 2024-09-18T11:33:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066230+0000", 2024-09-18T11:33:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7914000000000003e-05 2024-09-18T11:33:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.5.13:43 3.1 3:b44664f3:::7f31751f088e4f7577561c03b406e8a8bc48ebfdd310b40206061655307aebe3:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:33:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:40.058386+0000", 2024-09-18T11:33:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.944714736000002, 2024-09-18T11:33:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079575740000000002, 2024-09-18T11:33:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058386+0000", 2024-09-18T11:33:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058386+0000", 2024-09-18T11:33:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058385+0000", 2024-09-18T11:33:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058392+0000", 2024-09-18T11:33:03.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6200000000000001e-06 2024-09-18T11:33:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058400+0000", 2024-09-18T11:33:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7849999999999997e-06 2024-09-18T11:33:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058408+0000", 2024-09-18T11:33:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1770000000000001e-06 2024-09-18T11:33:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059302+0000", 2024-09-18T11:33:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000894851 2024-09-18T11:33:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059400+0000", 2024-09-18T11:33:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7460000000000005e-05 2024-09-18T11:33:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059520+0000", 2024-09-18T11:33:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000120457 2024-09-18T11:33:03.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.060333+0000", 2024-09-18T11:33:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00081255599999999998 2024-09-18T11:33:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066274+0000", 2024-09-18T11:33:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059409700000000003 2024-09-18T11:33:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:03.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066306+0000", 2024-09-18T11:33:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2020000000000002e-05 2024-09-18T11:33:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066343+0000", 2024-09-18T11:33:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7438999999999998e-05 2024-09-18T11:33:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.4.13:145 3.2 3:435dd00a:::773f1f20723e27798deecfe7048ee8e76eeec814e6d6ad791ed305ff28c69be2:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:33:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:45.321055+0000", 2024-09-18T11:33:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 17.682045931000001, 2024-09-18T11:33:03.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079066459999999998, 2024-09-18T11:33:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321055+0000", 2024-09-18T11:33:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321055+0000", 2024-09-18T11:33:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321053+0000", 2024-09-18T11:33:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321062+0000", 2024-09-18T11:33:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0399999999999998e-06 2024-09-18T11:33:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321076+0000", 2024-09-18T11:33:03.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3869999999999999e-05 2024-09-18T11:33:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321083+0000", 2024-09-18T11:33:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6159999999999998e-06 2024-09-18T11:33:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321156+0000", 2024-09-18T11:33:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3093e-05 2024-09-18T11:33:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321274+0000", 2024-09-18T11:33:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000118353 2024-09-18T11:33:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321443+0000", 2024-09-18T11:33:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016836600000000001 2024-09-18T11:33:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.324281+0000", 2024-09-18T11:33:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028381270000000002 2024-09-18T11:33:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.328785+0000", 2024-09-18T11:33:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045048370000000003 2024-09-18T11:33:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.328910+0000", 2024-09-18T11:33:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012438600000000001 2024-09-18T11:33:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.328961+0000", 2024-09-18T11:33:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1375000000000002e-05 2024-09-18T11:33:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4790 3.2 3:79129b11:::2f02c59c01a730562a65f21445c8af0e4a8aab3b969b8a3abf39e7b1bc84f918:head [write 0~2177536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-18T11:33:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.041705+0000", 2024-09-18T11:33:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.9613953179999992, 2024-09-18T11:33:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011619633000000001, 2024-09-18T11:33:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4790 2024-09-18T11:33:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.041705+0000", 2024-09-18T11:33:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.041705+0000", 2024-09-18T11:33:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.041703+0000", 2024-09-18T11:33:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.042939+0000", 2024-09-18T11:33:03.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012361570000000001 2024-09-18T11:33:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.042961+0000", 2024-09-18T11:33:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1531000000000001e-05 2024-09-18T11:33:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.042969+0000", 2024-09-18T11:33:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.892e-06 2024-09-18T11:33:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:03.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.043013+0000", 2024-09-18T11:33:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3921000000000002e-05 2024-09-18T11:33:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.043147+0000", 2024-09-18T11:33:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013394599999999999 2024-09-18T11:33:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.043291+0000", 2024-09-18T11:33:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014399500000000001 2024-09-18T11:33:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.048224+0000", 2024-09-18T11:33:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049331160000000004 2024-09-18T11:33:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053239+0000", 2024-09-18T11:33:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050155720000000003 2024-09-18T11:33:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053287+0000", 2024-09-18T11:33:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8126e-05 2024-09-18T11:33:03.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053325+0000", 2024-09-18T11:33:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7499999999999997e-05 2024-09-18T11:33:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:03.115 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:03.115 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:03.117 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:33:03.118 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.7 to 1 (10000) 2024-09-18T11:33:03.118 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T11:33:03.118 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:33:03.115+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:33:03.217 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:03.215+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:33:03.225 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:03.226 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:03.226 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:33:03.226 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:03.235 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:33:03.318 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:33:03.316+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:33:03.332 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:03.332 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:03.332 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:33:03.332 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:33:03.332 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:03.340 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:33:03.367 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:33:03.366+0000 7f6bf3052640 -1 osd.5 191 handle_osd_map injecting map crc failure 2024-09-18T11:33:03.402 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:33:03.401+0000 7fd27659f640 -1 osd.6 191 handle_osd_map injecting map crc failure 2024-09-18T11:33:03.418 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:03.416+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:33:03.441 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:03.441 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:33:03.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:33:03.441 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:33:03.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.441 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:33:03.441 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:33:03.441 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 149.022664292, 2024-09-18T11:33:03.441 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:33:03.442 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.442 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.442 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:03.442 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:03.442 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.442 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:33:03.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.442 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.442 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.442 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:33:03.442 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:33:03.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:33:03.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:03.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.443 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.443 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:33:03.443 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:33:03.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:33:03.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:33:03.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.444 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:33:03.444 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:33:03.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.444 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:33:03.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:33:03.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:33:03.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:33:03.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.445 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:03.445 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:33:03.445 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:33:03.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:03.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:33:03.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:33:03.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:33:03.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:33:03.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.446 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:03.446 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:33:03.446 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:33:03.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.447 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.447 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:33:03.447 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:33:03.447 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.447 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.447 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.447 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:33:03.447 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:33:03.447 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 122.052434886, 2024-09-18T11:33:03.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:33:03.448 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.448 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.448 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:33:03.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.448 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.448 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.448 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:33:03.448 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:33:03.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:03.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.449 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:33:03.449 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:33:03.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.449 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:33:03.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:33:03.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:33:03.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:33:03.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.450 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.450 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.450 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:33:03.450 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:33:03.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:33:03.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:33:03.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:03.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:33:03.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:33:03.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.451 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.451 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:33:03.451 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:33:03.452 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.452 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.452 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.452 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:33:03.452 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:33:03.452 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 114.46632228, 2024-09-18T11:33:03.452 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:33:03.452 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.452 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.452 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:03.452 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:03.453 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.453 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:33:03.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.453 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:33:03.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.453 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.453 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.453 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:33:03.453 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:33:03.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:03.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:33:03.454 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:33:03.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.454 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.454 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:33:03.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:33:03.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:33:03.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:33:03.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.455 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:33:03.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:33:03.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.455 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:33:03.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:33:03.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:03.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:33:03.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:33:03.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:03.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:33:03.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:33:03.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:33:03.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:33:03.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:03.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:33:03.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:33:03.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:33:03.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:33:03.458 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.458 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.458 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.458 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2956 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~2100908,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:33:03.458 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:33:03.458 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 99.411526030000005, 2024-09-18T11:33:03.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011863831, 2024-09-18T11:33:03.458 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.458 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:03.458 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:03.459 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:03.459 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.459 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2956 2024-09-18T11:33:03.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.459 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:33:03.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:33:03.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:33:03.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:03.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027228+0000", 2024-09-18T11:33:03.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:33:03.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027235+0000", 2024-09-18T11:33:03.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:33:03.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027239+0000", 2024-09-18T11:33:03.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.535e-06 2024-09-18T11:33:03.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027831+0000", 2024-09-18T11:33:03.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059244699999999996 2024-09-18T11:33:03.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:03.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027879+0000", 2024-09-18T11:33:03.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:33:03.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.462 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038098+0000", 2024-09-18T11:33:03.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010219137 2024-09-18T11:33:03.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038207+0000", 2024-09-18T11:33:03.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010951999999999999 2024-09-18T11:33:03.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039089+0000", 2024-09-18T11:33:03.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00088151699999999996 2024-09-18T11:33:03.463 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.463 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.463 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.464 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2957 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:33:03.464 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:33:03.464 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 99.411508245999997, 2024-09-18T11:33:03.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012041979, 2024-09-18T11:33:03.464 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.464 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:03.464 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:03.464 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:03.464 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.464 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2957 2024-09-18T11:33:03.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.464 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:33:03.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:33:03.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:33:03.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:03.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027245+0000", 2024-09-18T11:33:03.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:33:03.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027251+0000", 2024-09-18T11:33:03.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0090000000000004e-06 2024-09-18T11:33:03.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027255+0000", 2024-09-18T11:33:03.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:33:03.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027895+0000", 2024-09-18T11:33:03.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064076400000000005 2024-09-18T11:33:03.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.467 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:03.467 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027930+0000", 2024-09-18T11:33:03.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4334e-05 2024-09-18T11:33:03.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039111+0000", 2024-09-18T11:33:03.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181455999999999 2024-09-18T11:33:03.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039156+0000", 2024-09-18T11:33:03.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5256000000000002e-05 2024-09-18T11:33:03.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039285+0000", 2024-09-18T11:33:03.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012839200000000001 2024-09-18T11:33:03.469 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.469 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.469 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.469 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:978 4.b e132/67 4:d8d14c58:::smithi11939506-49:30 v 132'417, mlcod=128'414)", 2024-09-18T11:33:03.469 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:33:03.469 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 60.413476963999997, 2024-09-18T11:33:03.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011942183, 2024-09-18T11:33:03.469 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.469 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.470 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:33:03.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:33:03.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:33:03.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:33:03.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025278+0000", 2024-09-18T11:33:03.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9060000000000004e-06 2024-09-18T11:33:03.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025288+0000", 2024-09-18T11:33:03.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.028e-05 2024-09-18T11:33:03.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025293+0000", 2024-09-18T11:33:03.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2700000000000004e-06 2024-09-18T11:33:03.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025346+0000", 2024-09-18T11:33:03.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2862999999999999e-05 2024-09-18T11:33:03.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025370+0000", 2024-09-18T11:33:03.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4005000000000001e-05 2024-09-18T11:33:03.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:03.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037159+0000", 2024-09-18T11:33:03.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011788343 2024-09-18T11:33:03.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037216+0000", 2024-09-18T11:33:03.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7695000000000002e-05 2024-09-18T11:33:03.473 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.473 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.473 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.474 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3930 4.b e132/67 4:d3ae8215:::smithi11939506-41:head v 132'421, mlcod=128'414)", 2024-09-18T11:33:03.474 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:33:03.474 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 60.409928716000003, 2024-09-18T11:33:03.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015093554, 2024-09-18T11:33:03.474 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.474 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.474 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:33:03.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:33:03.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028821+0000", 2024-09-18T11:33:03.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:03.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029805+0000", 2024-09-18T11:33:03.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00098410899999999994 2024-09-18T11:33:03.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029817+0000", 2024-09-18T11:33:03.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2058e-05 2024-09-18T11:33:03.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029823+0000", 2024-09-18T11:33:03.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6450000000000002e-06 2024-09-18T11:33:03.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032330+0000", 2024-09-18T11:33:03.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025067710000000001 2024-09-18T11:33:03.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032349+0000", 2024-09-18T11:33:03.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9729e-05 2024-09-18T11:33:03.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:03.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043878+0000", 2024-09-18T11:33:03.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011528514 2024-09-18T11:33:03.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043916+0000", 2024-09-18T11:33:03.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8049000000000001e-05 2024-09-18T11:33:03.478 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.478 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.478 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.478 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:162 3.1 3:8c826a1f:::e682cfc1e82c88472b9acf1cd97978bc105aa035f10b685bb4cace95809b52bf:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:03.478 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:33:03.478 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.857806558, 2024-09-18T11:33:03.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013234571000000001, 2024-09-18T11:33:03.479 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.479 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.479 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:33:03.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:33:03.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580943+0000", 2024-09-18T11:33:03.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:03.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582556+0000", 2024-09-18T11:33:03.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016132970000000001 2024-09-18T11:33:03.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582572+0000", 2024-09-18T11:33:03.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5846999999999999e-05 2024-09-18T11:33:03.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582578+0000", 2024-09-18T11:33:03.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4810000000000003e-06 2024-09-18T11:33:03.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582632+0000", 2024-09-18T11:33:03.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4156000000000001e-05 2024-09-18T11:33:03.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582819+0000", 2024-09-18T11:33:03.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000186431 2024-09-18T11:33:03.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:03.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583118+0000", 2024-09-18T11:33:03.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029880100000000001 2024-09-18T11:33:03.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:03.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590676+0000", 2024-09-18T11:33:03.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075588670000000004 2024-09-18T11:33:03.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594111+0000", 2024-09-18T11:33:03.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034348550000000001 2024-09-18T11:33:03.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:03.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594142+0000", 2024-09-18T11:33:03.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.099e-05 2024-09-18T11:33:03.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594179+0000", 2024-09-18T11:33:03.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6831e-05 2024-09-18T11:33:03.483 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.483 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.484 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.484 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3974 4.d 4:b5a463bf:::smithi11939506-20:head [write 665643~697786,stat] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:03.484 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:33:03.484 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.856719875000003, 2024-09-18T11:33:03.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016621174999999998, 2024-09-18T11:33:03.484 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.484 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.484 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:03.484 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:03.484 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.484 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3974 2024-09-18T11:33:03.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.485 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:33:03.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:33:03.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582030+0000", 2024-09-18T11:33:03.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:03.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585185+0000", 2024-09-18T11:33:03.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003155644 2024-09-18T11:33:03.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585208+0000", 2024-09-18T11:33:03.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3013000000000001e-05 2024-09-18T11:33:03.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585215+0000", 2024-09-18T11:33:03.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7569999999999998e-06 2024-09-18T11:33:03.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585278+0000", 2024-09-18T11:33:03.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2699000000000004e-05 2024-09-18T11:33:03.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585374+0000", 2024-09-18T11:33:03.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.6062000000000003e-05 2024-09-18T11:33:03.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:33:03.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585577+0000", 2024-09-18T11:33:03.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000202926 2024-09-18T11:33:03.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:03.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596544+0000", 2024-09-18T11:33:03.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010966848 2024-09-18T11:33:03.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598594+0000", 2024-09-18T11:33:03.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020502049999999998 2024-09-18T11:33:03.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:03.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598625+0000", 2024-09-18T11:33:03.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1412999999999999e-05 2024-09-18T11:33:03.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598652+0000", 2024-09-18T11:33:03.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7166999999999999e-05 2024-09-18T11:33:03.490 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.490 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.490 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.490 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:166 3.1 3:a73e635b:::b6ee478f0f661ec7bac3d17c2ca2fdaff71a47570f64844a7ba79bbb2a0e8d9d:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:03.490 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:33:03.490 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.856160508999999, 2024-09-18T11:33:03.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011711037000000001, 2024-09-18T11:33:03.490 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.490 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.491 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:33:03.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:33:03.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.491 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.491 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582590+0000", 2024-09-18T11:33:03.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:03.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582673+0000", 2024-09-18T11:33:03.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2861000000000005e-05 2024-09-18T11:33:03.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582683+0000", 2024-09-18T11:33:03.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0081999999999999e-05 2024-09-18T11:33:03.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.493 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582688+0000", 2024-09-18T11:33:03.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6870000000000002e-06 2024-09-18T11:33:03.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.493 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584266+0000", 2024-09-18T11:33:03.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001578423 2024-09-18T11:33:03.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.493 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584407+0000", 2024-09-18T11:33:03.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014144599999999999 2024-09-18T11:33:03.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:03.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584599+0000", 2024-09-18T11:33:03.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000191881 2024-09-18T11:33:03.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:03.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590680+0000", 2024-09-18T11:33:03.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060801479999999996 2024-09-18T11:33:03.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594224+0000", 2024-09-18T11:33:03.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035441700000000001 2024-09-18T11:33:03.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:03.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594269+0000", 2024-09-18T11:33:03.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5297999999999999e-05 2024-09-18T11:33:03.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594302+0000", 2024-09-18T11:33:03.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2703000000000002e-05 2024-09-18T11:33:03.496 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.496 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.496 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.496 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:168 3.1 3:b64d7bad:::7feb22256bdcec217720fd46184ab43ced8923ed66ec6319c57f078d9f123262:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:03.496 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:33:03.496 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.855933057000001, 2024-09-18T11:33:03.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011589809, 2024-09-18T11:33:03.496 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.497 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.497 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:33:03.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:33:03.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582817+0000", 2024-09-18T11:33:03.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:03.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582903+0000", 2024-09-18T11:33:03.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5915999999999995e-05 2024-09-18T11:33:03.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582913+0000", 2024-09-18T11:33:03.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1409999999999994e-06 2024-09-18T11:33:03.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582917+0000", 2024-09-18T11:33:03.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6419999999999998e-06 2024-09-18T11:33:03.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585540+0000", 2024-09-18T11:33:03.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026228940000000002 2024-09-18T11:33:03.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585721+0000", 2024-09-18T11:33:03.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018086299999999999 2024-09-18T11:33:03.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:03.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585918+0000", 2024-09-18T11:33:03.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000196758 2024-09-18T11:33:03.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:03.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590683+0000", 2024-09-18T11:33:03.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047657020000000001 2024-09-18T11:33:03.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594354+0000", 2024-09-18T11:33:03.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003671068 2024-09-18T11:33:03.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:03.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594386+0000", 2024-09-18T11:33:03.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1426e-05 2024-09-18T11:33:03.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594408+0000", 2024-09-18T11:33:03.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1987000000000001e-05 2024-09-18T11:33:03.502 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.502 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.502 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.502 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:175 3.1 3:b5e86b88:::e55cf3d5cf782ed327a67e746b287b76d05f74f58a4f3726e399a564440ceaf6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:03.502 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:33:03.502 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 53.855084116, 2024-09-18T11:33:03.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013709627, 2024-09-18T11:33:03.502 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.502 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.502 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:33:03.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:33:03.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583666+0000", 2024-09-18T11:33:03.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:03.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.504 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584041+0000", 2024-09-18T11:33:03.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037511499999999998 2024-09-18T11:33:03.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.504 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584055+0000", 2024-09-18T11:33:03.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3295999999999999e-05 2024-09-18T11:33:03.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584061+0000", 2024-09-18T11:33:03.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1630000000000001e-06 2024-09-18T11:33:03.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.589980+0000", 2024-09-18T11:33:03.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005918998 2024-09-18T11:33:03.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590117+0000", 2024-09-18T11:33:03.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013715400000000001 2024-09-18T11:33:03.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:03.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590270+0000", 2024-09-18T11:33:03.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015245600000000001 2024-09-18T11:33:03.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:03.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596513+0000", 2024-09-18T11:33:03.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062438729999999996 2024-09-18T11:33:03.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597328+0000", 2024-09-18T11:33:03.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081489599999999996 2024-09-18T11:33:03.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:03.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597358+0000", 2024-09-18T11:33:03.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9291000000000001e-05 2024-09-18T11:33:03.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597377+0000", 2024-09-18T11:33:03.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:33:03.507 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.507 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.508 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.508 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4061 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:03.508 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:33:03.508 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 51.141905426999998, 2024-09-18T11:33:03.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013340967, 2024-09-18T11:33:03.508 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.508 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:03.508 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:03.508 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:03.508 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.508 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4061 2024-09-18T11:33:03.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.509 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:33:03.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:33:03.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296844+0000", 2024-09-18T11:33:03.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:03.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296854+0000", 2024-09-18T11:33:03.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0037000000000001e-05 2024-09-18T11:33:03.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296873+0000", 2024-09-18T11:33:03.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:33:03.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296881+0000", 2024-09-18T11:33:03.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3790000000000006e-06 2024-09-18T11:33:03.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296975+0000", 2024-09-18T11:33:03.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3536999999999996e-05 2024-09-18T11:33:03.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:33:03.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296982+0000", 2024-09-18T11:33:03.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6399999999999997e-06 2024-09-18T11:33:03.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301933+0000", 2024-09-18T11:33:03.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049507190000000001 2024-09-18T11:33:03.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301984+0000", 2024-09-18T11:33:03.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1439e-05 2024-09-18T11:33:03.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:33:03.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082022050000000006 2024-09-18T11:33:03.513 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.513 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.513 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.513 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4062 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:03.513 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:33:03.513 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 51.141856871999998, 2024-09-18T11:33:03.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016297147000000001, 2024-09-18T11:33:03.513 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.514 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:03.514 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:03.514 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:03.514 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.514 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4062 2024-09-18T11:33:03.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.514 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:33:03.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:33:03.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:33:03.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:03.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296898+0000", 2024-09-18T11:33:03.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9419999999999997e-06 2024-09-18T11:33:03.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296906+0000", 2024-09-18T11:33:03.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3520000000000007e-06 2024-09-18T11:33:03.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296910+0000", 2024-09-18T11:33:03.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4469999999999999e-06 2024-09-18T11:33:03.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297032+0000", 2024-09-18T11:33:03.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012168900000000001 2024-09-18T11:33:03.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:03.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297038+0000", 2024-09-18T11:33:03.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9320000000000001e-06 2024-09-18T11:33:03.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310221+0000", 2024-09-18T11:33:03.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013182887000000001 2024-09-18T11:33:03.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310323+0000", 2024-09-18T11:33:03.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010170000000000001 2024-09-18T11:33:03.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313191+0000", 2024-09-18T11:33:03.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002868808 2024-09-18T11:33:03.518 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.518 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.518 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.518 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4063 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:03.518 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:33:03.519 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 51.141835980000003, 2024-09-18T11:33:03.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017648712, 2024-09-18T11:33:03.519 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.519 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:03.519 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:03.519 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:03.519 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.519 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4063 2024-09-18T11:33:03.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.519 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.519 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:33:03.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:33:03.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:33:03.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:03.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296918+0000", 2024-09-18T11:33:03.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8100000000000002e-06 2024-09-18T11:33:03.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296924+0000", 2024-09-18T11:33:03.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2519999999999996e-06 2024-09-18T11:33:03.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296928+0000", 2024-09-18T11:33:03.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0500000000000002e-06 2024-09-18T11:33:03.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297062+0000", 2024-09-18T11:33:03.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013425500000000001 2024-09-18T11:33:03.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:03.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297069+0000", 2024-09-18T11:33:03.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5239999999999997e-06 2024-09-18T11:33:03.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313424+0000", 2024-09-18T11:33:03.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016355277000000001 2024-09-18T11:33:03.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313514+0000", 2024-09-18T11:33:03.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0581999999999994e-05 2024-09-18T11:33:03.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.524 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.524 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.314564+0000", 2024-09-18T11:33:03.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001049384 2024-09-18T11:33:03.524 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.524 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.524 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.524 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4065 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:03.525 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:33:03.525 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 51.141790538999999, 2024-09-18T11:33:03.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013225997, 2024-09-18T11:33:03.525 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.525 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:03.525 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:03.525 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:03.525 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.525 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4065 2024-09-18T11:33:03.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.525 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:33:03.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:33:03.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296960+0000", 2024-09-18T11:33:03.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:03.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296963+0000", 2024-09-18T11:33:03.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2270000000000002e-06 2024-09-18T11:33:03.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296970+0000", 2024-09-18T11:33:03.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6220000000000003e-06 2024-09-18T11:33:03.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296973+0000", 2024-09-18T11:33:03.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3730000000000001e-06 2024-09-18T11:33:03.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297097+0000", 2024-09-18T11:33:03.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012352399999999999 2024-09-18T11:33:03.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:03.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297103+0000", 2024-09-18T11:33:03.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7119999999999997e-06 2024-09-18T11:33:03.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307831+0000", 2024-09-18T11:33:03.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010728794 2024-09-18T11:33:03.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307924+0000", 2024-09-18T11:33:03.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2099e-05 2024-09-18T11:33:03.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:33:03.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022630469999999998 2024-09-18T11:33:03.529 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.529 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.529 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.530 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4066 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:03.530 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:33:03.530 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 51.141773464000003, 2024-09-18T11:33:03.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015656389999999999, 2024-09-18T11:33:03.530 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.530 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:03.530 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:03.530 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:03.530 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.530 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4066 2024-09-18T11:33:03.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.530 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:33:03.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:33:03.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.531 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296977+0000", 2024-09-18T11:33:03.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:03.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296980+0000", 2024-09-18T11:33:03.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8830000000000002e-06 2024-09-18T11:33:03.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296985+0000", 2024-09-18T11:33:03.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:33:03.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296988+0000", 2024-09-18T11:33:03.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0419999999999998e-06 2024-09-18T11:33:03.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297120+0000", 2024-09-18T11:33:03.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131693 2024-09-18T11:33:03.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:03.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297125+0000", 2024-09-18T11:33:03.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1100000000000002e-06 2024-09-18T11:33:03.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310212+0000", 2024-09-18T11:33:03.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013087395999999999 2024-09-18T11:33:03.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310292+0000", 2024-09-18T11:33:03.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9980000000000003e-05 2024-09-18T11:33:03.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312634+0000", 2024-09-18T11:33:03.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002341756 2024-09-18T11:33:03.534 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.535 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.535 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.535 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4068 3.1 3:87e79ed0:::3526641c07d3efa989be9684355570f39c331008b8a02c9fa22b38a94b6c6b9f:head [write 0~1015808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:03.535 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:33:03.535 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 51.137997247000001, 2024-09-18T11:33:03.535 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016392648999999999, 2024-09-18T11:33:03.535 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.535 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.535 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:03.535 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:03.535 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.536 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4068 2024-09-18T11:33:03.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.536 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.536 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.536 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:33:03.536 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.536 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.536 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:33:03.536 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.537 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300753+0000", 2024-09-18T11:33:03.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:03.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.537 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301922+0000", 2024-09-18T11:33:03.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001169432 2024-09-18T11:33:03.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.537 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301936+0000", 2024-09-18T11:33:03.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3574000000000001e-05 2024-09-18T11:33:03.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.538 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301941+0000", 2024-09-18T11:33:03.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6069999999999998e-06 2024-09-18T11:33:03.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310234+0000", 2024-09-18T11:33:03.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082924820000000003 2024-09-18T11:33:03.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.540 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310337+0000", 2024-09-18T11:33:03.540 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102673 2024-09-18T11:33:03.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.540 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:03.540 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310469+0000", 2024-09-18T11:33:03.540 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131982 2024-09-18T11:33:03.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.540 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:03.540 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312844+0000", 2024-09-18T11:33:03.540 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023758680000000002 2024-09-18T11:33:03.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317092+0000", 2024-09-18T11:33:03.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042477549999999998 2024-09-18T11:33:03.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:03.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317124+0000", 2024-09-18T11:33:03.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1968999999999998e-05 2024-09-18T11:33:03.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317147+0000", 2024-09-18T11:33:03.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2339000000000001e-05 2024-09-18T11:33:03.542 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.542 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.542 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.542 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4671 4.1 4:87078dfe:::smithi11939506-14:head [rollback 6f] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:33:03.542 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:33:03.542 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.120034872000002, 2024-09-18T11:33:03.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013540905000000001, 2024-09-18T11:33:03.542 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.542 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.542 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:03.543 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:03.543 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.543 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4671 2024-09-18T11:33:03.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.543 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:33:03.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:33:03.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:33:03.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:03.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318718+0000", 2024-09-18T11:33:03.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3300000000000001e-06 2024-09-18T11:33:03.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318723+0000", 2024-09-18T11:33:03.545 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1590000000000001e-06 2024-09-18T11:33:03.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.545 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318727+0000", 2024-09-18T11:33:03.545 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3529999999999999e-06 2024-09-18T11:33:03.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.545 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.320735+0000", 2024-09-18T11:33:03.545 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020079730000000001 2024-09-18T11:33:03.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.546 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.320843+0000", 2024-09-18T11:33:03.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000108488 2024-09-18T11:33:03.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.546 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:03.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.329522+0000", 2024-09-18T11:33:03.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086787979999999997 2024-09-18T11:33:03.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.546 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:03.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331769+0000", 2024-09-18T11:33:03.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022471269999999998 2024-09-18T11:33:03.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.547 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332189+0000", 2024-09-18T11:33:03.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00041940400000000001 2024-09-18T11:33:03.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.547 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:03.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332228+0000", 2024-09-18T11:33:03.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9298000000000002e-05 2024-09-18T11:33:03.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.547 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332257+0000", 2024-09-18T11:33:03.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.934e-05 2024-09-18T11:33:03.548 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.548 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.548 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.548 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4672 4.1 4:87078dfe:::smithi11939506-14:head [append 0~0] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:33:03.548 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.318763+0000", 2024-09-18T11:33:03.548 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.119988468999999, 2024-09-18T11:33:03.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016099992, 2024-09-18T11:33:03.548 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.548 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.548 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:03.548 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:03.548 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.549 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4672 2024-09-18T11:33:03.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.549 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318763+0000", 2024-09-18T11:33:03.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318763+0000", 2024-09-18T11:33:03.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318762+0000", 2024-09-18T11:33:03.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:03.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318765+0000", 2024-09-18T11:33:03.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.43e-06 2024-09-18T11:33:03.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318770+0000", 2024-09-18T11:33:03.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1479999999999997e-06 2024-09-18T11:33:03.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.551 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318773+0000", 2024-09-18T11:33:03.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.18e-06 2024-09-18T11:33:03.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.551 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.321043+0000", 2024-09-18T11:33:03.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022696819999999999 2024-09-18T11:33:03.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.551 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for blocked object", 2024-09-18T11:33:03.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.321104+0000", 2024-09-18T11:33:03.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1050999999999995e-05 2024-09-18T11:33:03.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.552 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.552 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331070+0000", 2024-09-18T11:33:03.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0099659610000000006 2024-09-18T11:33:03.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.552 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.552 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331176+0000", 2024-09-18T11:33:03.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010582 2024-09-18T11:33:03.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:03.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331377+0000", 2024-09-18T11:33:03.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000201208 2024-09-18T11:33:03.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:03.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332278+0000", 2024-09-18T11:33:03.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00090079999999999999 2024-09-18T11:33:03.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.334798+0000", 2024-09-18T11:33:03.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025200190000000001 2024-09-18T11:33:03.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.554 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:03.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.334833+0000", 2024-09-18T11:33:03.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5678e-05 2024-09-18T11:33:03.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.554 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.334863+0000", 2024-09-18T11:33:03.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9374e-05 2024-09-18T11:33:03.554 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.554 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.554 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.554 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.554 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.554 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:03.556 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:33:03.556 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:03.554+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:03.657 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:03.656+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:03.695 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:03.695 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:03.695 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:33:03.695 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:03.703 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:33:03.758 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:33:03.757+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:33:03.842 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:03.842 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:03.842 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:33:03.842 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:33:03.842 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:03.850 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:33:03.859 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:33:03.858+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:33:03.950 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:03.950 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:33:03.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:33:03.950 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:33:03.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.950 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1788 4.8 4:1898b15d:::smithi11939506-25:head [write 453822~797061] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:33:03.950 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:33:03.951 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 141.03192289399999, 2024-09-18T11:33:03.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094305159999999999, 2024-09-18T11:33:03.951 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.951 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.951 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:03.951 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:03.951 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.951 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1788 2024-09-18T11:33:03.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.951 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:33:03.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:33:03.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915811+0000", 2024-09-18T11:33:03.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:03.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916608+0000", 2024-09-18T11:33:03.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079635000000000003 2024-09-18T11:33:03.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916624+0000", 2024-09-18T11:33:03.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6735000000000002e-05 2024-09-18T11:33:03.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916630+0000", 2024-09-18T11:33:03.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5709999999999997e-06 2024-09-18T11:33:03.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916674+0000", 2024-09-18T11:33:03.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4540999999999999e-05 2024-09-18T11:33:03.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916751+0000", 2024-09-18T11:33:03.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6695999999999999e-05 2024-09-18T11:33:03.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:33:03.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916872+0000", 2024-09-18T11:33:03.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120648 2024-09-18T11:33:03.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:03.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.919076+0000", 2024-09-18T11:33:03.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022046829999999998 2024-09-18T11:33:03.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925151+0000", 2024-09-18T11:33:03.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060740239999999999 2024-09-18T11:33:03.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:03.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925198+0000", 2024-09-18T11:33:03.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7896999999999999e-05 2024-09-18T11:33:03.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925244+0000", 2024-09-18T11:33:03.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5398999999999997e-05 2024-09-18T11:33:03.956 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.956 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.956 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.957 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3963 4.3 4:c70c3162:::smithi11939506-35:head [tier-promote] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:03.957 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:33:03.957 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.571912973000003, 2024-09-18T11:33:03.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010675591999999999, 2024-09-18T11:33:03.957 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.957 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:03.957 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:03.957 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:03.957 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.957 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3963 2024-09-18T11:33:03.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.957 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:33:03.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:33:03.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375821+0000", 2024-09-18T11:33:03.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:03.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375829+0000", 2024-09-18T11:33:03.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7989999999999994e-06 2024-09-18T11:33:03.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375843+0000", 2024-09-18T11:33:03.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:33:03.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375849+0000", 2024-09-18T11:33:03.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1419999999999998e-06 2024-09-18T11:33:03.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375889+0000", 2024-09-18T11:33:03.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0197999999999997e-05 2024-09-18T11:33:03.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:33:03.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375894+0000", 2024-09-18T11:33:03.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0139999999999998e-06 2024-09-18T11:33:03.960 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.377699+0000", 2024-09-18T11:33:03.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001804688 2024-09-18T11:33:03.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.377776+0000", 2024-09-18T11:33:03.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7081000000000003e-05 2024-09-18T11:33:03.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.386499+0000", 2024-09-18T11:33:03.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008722746 2024-09-18T11:33:03.962 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.962 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.962 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.962 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4259 4.b 4:dba5bd39:::smithi11939506-17:head [write 685818~644423] snapc 69=[69,67,61,55,51,4e,4b,45,42,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:33:03.962 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.040083+0000", 2024-09-18T11:33:03.962 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.907652794000001, 2024-09-18T11:33:03.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015748635, 2024-09-18T11:33:03.962 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.962 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.962 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:03.963 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:03.963 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.963 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4259 2024-09-18T11:33:03.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.963 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.040083+0000", 2024-09-18T11:33:03.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.040083+0000", 2024-09-18T11:33:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.040081+0000", 2024-09-18T11:33:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041258+0000", 2024-09-18T11:33:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011766540000000001 2024-09-18T11:33:03.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041286+0000", 2024-09-18T11:33:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8087000000000001e-05 2024-09-18T11:33:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041362+0000", 2024-09-18T11:33:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6155999999999994e-05 2024-09-18T11:33:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.044285+0000", 2024-09-18T11:33:03.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002922437 2024-09-18T11:33:03.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.044360+0000", 2024-09-18T11:33:03.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5201e-05 2024-09-18T11:33:03.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:03.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.044489+0000", 2024-09-18T11:33:03.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012950599999999999 2024-09-18T11:33:03.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052904+0000", 2024-09-18T11:33:03.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084149910000000001 2024-09-18T11:33:03.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:03.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055695+0000", 2024-09-18T11:33:03.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027902040000000001 2024-09-18T11:33:03.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:03.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055765+0000", 2024-09-18T11:33:03.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0720000000000001e-05 2024-09-18T11:33:03.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055832+0000", 2024-09-18T11:33:03.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6736999999999995e-05 2024-09-18T11:33:03.968 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.968 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.968 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.968 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.5.0:195 4.b e156/152 ondisk, result = 0)", 2024-09-18T11:33:03.968 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.041863+0000", 2024-09-18T11:33:03.968 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.905873274000001, 2024-09-18T11:33:03.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010991236, 2024-09-18T11:33:03.969 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.969 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:03.969 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041863+0000", 2024-09-18T11:33:03.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041863+0000", 2024-09-18T11:33:03.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041862+0000", 2024-09-18T11:33:03.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:03.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041868+0000", 2024-09-18T11:33:03.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2979999999999997e-06 2024-09-18T11:33:03.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041879+0000", 2024-09-18T11:33:03.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0522e-05 2024-09-18T11:33:03.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041885+0000", 2024-09-18T11:33:03.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7830000000000004e-06 2024-09-18T11:33:03.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052841+0000", 2024-09-18T11:33:03.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010956604 2024-09-18T11:33:03.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052848+0000", 2024-09-18T11:33:03.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2249999999999997e-06 2024-09-18T11:33:03.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052854+0000", 2024-09-18T11:33:03.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6050000000000003e-06 2024-09-18T11:33:03.972 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.972 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.972 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.972 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.5.0:196 4.b e156/152 ondisk, result = 0)", 2024-09-18T11:33:03.972 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.042994+0000", 2024-09-18T11:33:03.972 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.904742612, 2024-09-18T11:33:03.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098926020000000003, 2024-09-18T11:33:03.973 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.973 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:03.973 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.042994+0000", 2024-09-18T11:33:03.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.042994+0000", 2024-09-18T11:33:03.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.042993+0000", 2024-09-18T11:33:03.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:03.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043000+0000", 2024-09-18T11:33:03.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.525e-06 2024-09-18T11:33:03.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043011+0000", 2024-09-18T11:33:03.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0335e-05 2024-09-18T11:33:03.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043015+0000", 2024-09-18T11:33:03.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2899999999999996e-06 2024-09-18T11:33:03.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052868+0000", 2024-09-18T11:33:03.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098528130000000002 2024-09-18T11:33:03.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052871+0000", 2024-09-18T11:33:03.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5080000000000001e-06 2024-09-18T11:33:03.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052886+0000", 2024-09-18T11:33:03.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4781e-05 2024-09-18T11:33:03.976 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.976 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.976 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.976 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.5.0:194 4.3 e156/67 ondisk, result = 0)", 2024-09-18T11:33:03.976 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.043892+0000", 2024-09-18T11:33:03.976 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.903843821999999, 2024-09-18T11:33:03.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010237263999999999, 2024-09-18T11:33:03.977 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.977 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:03.977 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043892+0000", 2024-09-18T11:33:03.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043892+0000", 2024-09-18T11:33:03.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043891+0000", 2024-09-18T11:33:03.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:03.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043898+0000", 2024-09-18T11:33:03.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1190000000000002e-06 2024-09-18T11:33:03.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043910+0000", 2024-09-18T11:33:03.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.29e-05 2024-09-18T11:33:03.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043916+0000", 2024-09-18T11:33:03.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9499999999999998e-06 2024-09-18T11:33:03.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052830+0000", 2024-09-18T11:33:03.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089137480000000009 2024-09-18T11:33:03.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052839+0000", 2024-09-18T11:33:03.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9560000000000003e-06 2024-09-18T11:33:03.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054130+0000", 2024-09-18T11:33:03.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012905639999999999 2024-09-18T11:33:03.980 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.980 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.980 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.981 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4260 4.b 4:dba5bd39:::smithi11939506-17:head [write 2069089~216729] snapc 69=[69,67,61,55,51,4e,4b,45,42,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:33:03.981 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045444+0000", 2024-09-18T11:33:03.981 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.902292299999999, 2024-09-18T11:33:03.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011166715000000001, 2024-09-18T11:33:03.981 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.981 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.981 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:03.981 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:03.981 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.981 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4260 2024-09-18T11:33:03.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.982 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045444+0000", 2024-09-18T11:33:03.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045444+0000", 2024-09-18T11:33:03.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045443+0000", 2024-09-18T11:33:03.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:03.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045649+0000", 2024-09-18T11:33:03.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020605499999999999 2024-09-18T11:33:03.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045662+0000", 2024-09-18T11:33:03.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3386e-05 2024-09-18T11:33:03.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045667+0000", 2024-09-18T11:33:03.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6750000000000001e-06 2024-09-18T11:33:03.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052921+0000", 2024-09-18T11:33:03.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007254613 2024-09-18T11:33:03.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052994+0000", 2024-09-18T11:33:03.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2721999999999999e-05 2024-09-18T11:33:03.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:03.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.053116+0000", 2024-09-18T11:33:03.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121972 2024-09-18T11:33:03.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:03.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055857+0000", 2024-09-18T11:33:03.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027414539999999999 2024-09-18T11:33:03.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056502+0000", 2024-09-18T11:33:03.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064412499999999995 2024-09-18T11:33:03.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:03.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056555+0000", 2024-09-18T11:33:03.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3467999999999997e-05 2024-09-18T11:33:03.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056611+0000", 2024-09-18T11:33:03.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5577e-05 2024-09-18T11:33:03.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.987 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.988 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4261 4.b 4:dba5bd39:::smithi11939506-17:head [setxattr _header (57) in=7b,truncate 2285818] snapc 69=[69,67,61,55,51,4e,4b,45,42,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:33:03.988 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045672+0000", 2024-09-18T11:33:03.988 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.902064201999998, 2024-09-18T11:33:03.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011289933, 2024-09-18T11:33:03.988 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.988 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:03.988 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:03.988 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:03.988 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.988 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4261 2024-09-18T11:33:03.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.989 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045672+0000", 2024-09-18T11:33:03.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045672+0000", 2024-09-18T11:33:03.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045671+0000", 2024-09-18T11:33:03.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:03.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045675+0000", 2024-09-18T11:33:03.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.563e-06 2024-09-18T11:33:03.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045681+0000", 2024-09-18T11:33:03.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5210000000000002e-06 2024-09-18T11:33:03.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045684+0000", 2024-09-18T11:33:03.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4149999999999999e-06 2024-09-18T11:33:03.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054143+0000", 2024-09-18T11:33:03.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084590810000000002 2024-09-18T11:33:03.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054214+0000", 2024-09-18T11:33:03.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1061000000000003e-05 2024-09-18T11:33:03.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:03.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054354+0000", 2024-09-18T11:33:03.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000140035 2024-09-18T11:33:03.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:03.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056208+0000", 2024-09-18T11:33:03.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018539229999999999 2024-09-18T11:33:03.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:03.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056885+0000", 2024-09-18T11:33:03.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00067688000000000004 2024-09-18T11:33:03.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:03.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056922+0000", 2024-09-18T11:33:03.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6693999999999998e-05 2024-09-18T11:33:03.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:03.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056962+0000", 2024-09-18T11:33:03.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0299000000000002e-05 2024-09-18T11:33:03.994 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.994 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:03.994 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:03.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.994 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4262 4.b 4:dba5bd39:::smithi11939506-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:33:03.994 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:33:03.994 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.902048059999998, 2024-09-18T11:33:03.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011764513000000001, 2024-09-18T11:33:03.995 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:03.995 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:03.995 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:03.995 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:03.995 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:03.995 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4262 2024-09-18T11:33:03.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.995 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:03.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:03.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:33:03.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:03.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:33:03.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:03.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:03.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:33:03.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:03.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:03.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045690+0000", 2024-09-18T11:33:03.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.745e-06 2024-09-18T11:33:03.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:03.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045697+0000", 2024-09-18T11:33:03.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7229999999999998e-06 2024-09-18T11:33:03.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:03.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045704+0000", 2024-09-18T11:33:03.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5719999999999999e-06 2024-09-18T11:33:03.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055555+0000", 2024-09-18T11:33:03.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098515430000000008 2024-09-18T11:33:03.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:03.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055675+0000", 2024-09-18T11:33:03.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012000799999999999 2024-09-18T11:33:03.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:03.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.057004+0000", 2024-09-18T11:33:03.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013286179999999999 2024-09-18T11:33:03.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:03.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:03.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:03.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.057357+0000", 2024-09-18T11:33:03.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00035355100000000002 2024-09-18T11:33:03.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:04.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.057453+0000", 2024-09-18T11:33:04.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5154000000000004e-05 2024-09-18T11:33:04.000 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.000 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:04.000 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.000 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4264 4.7 4:e0d95e10:::smithi11939506-22:head [read 0~3008486,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:33:04.000 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:33:04.000 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.901998073000001, 2024-09-18T11:33:04.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095333889999999998, 2024-09-18T11:33:04.001 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:04.001 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:04.001 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:04.001 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:04.001 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:04.001 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4264 2024-09-18T11:33:04.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.001 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:04.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:04.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:33:04.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:04.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:33:04.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:04.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:33:04.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:04.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:04.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045747+0000", 2024-09-18T11:33:04.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2469999999999997e-06 2024-09-18T11:33:04.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:04.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045755+0000", 2024-09-18T11:33:04.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1499999999999999e-06 2024-09-18T11:33:04.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:04.003 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045759+0000", 2024-09-18T11:33:04.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0849999999999999e-06 2024-09-18T11:33:04.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:04.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045803+0000", 2024-09-18T11:33:04.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3621000000000001e-05 2024-09-18T11:33:04.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:33:04.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045807+0000", 2024-09-18T11:33:04.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7290000000000002e-06 2024-09-18T11:33:04.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:04.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.053475+0000", 2024-09-18T11:33:04.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076680100000000003 2024-09-18T11:33:04.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:04.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.053644+0000", 2024-09-18T11:33:04.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016971299999999999 2024-09-18T11:33:04.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:04.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055272+0000", 2024-09-18T11:33:04.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001627245 2024-09-18T11:33:04.005 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.005 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:04.005 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.006 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4265 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~2163506] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:33:04.006 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045792+0000", 2024-09-18T11:33:04.006 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 36.901944182999998, 2024-09-18T11:33:04.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010085024, 2024-09-18T11:33:04.006 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:04.006 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:04.006 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:04.006 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:04.006 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:04.006 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4265 2024-09-18T11:33:04.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.007 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:04.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:04.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045792+0000", 2024-09-18T11:33:04.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:04.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045792+0000", 2024-09-18T11:33:04.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:04.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045791+0000", 2024-09-18T11:33:04.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:04.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:04.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045797+0000", 2024-09-18T11:33:04.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0209999999999996e-06 2024-09-18T11:33:04.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:04.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045805+0000", 2024-09-18T11:33:04.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9349999999999994e-06 2024-09-18T11:33:04.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:04.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045811+0000", 2024-09-18T11:33:04.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2240000000000003e-06 2024-09-18T11:33:04.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:04.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045858+0000", 2024-09-18T11:33:04.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7565999999999998e-05 2024-09-18T11:33:04.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:33:04.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045869+0000", 2024-09-18T11:33:04.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0852e-05 2024-09-18T11:33:04.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:04.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055337+0000", 2024-09-18T11:33:04.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094678729999999999 2024-09-18T11:33:04.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:04.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055417+0000", 2024-09-18T11:33:04.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0092000000000005e-05 2024-09-18T11:33:04.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:04.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055877+0000", 2024-09-18T11:33:04.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00046001800000000003 2024-09-18T11:33:04.011 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.011 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:04.011 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.011 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4349 4.b 4:d13decd8:::smithi11939506-2:head [read 0~2302625,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:33:04.011 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.382910+0000", 2024-09-18T11:33:04.011 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.564826119000003, 2024-09-18T11:33:04.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098226930000000004, 2024-09-18T11:33:04.011 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:04.011 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:04.011 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:04.012 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:04.012 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:04.012 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4349 2024-09-18T11:33:04.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.012 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:04.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:04.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382910+0000", 2024-09-18T11:33:04.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:04.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382910+0000", 2024-09-18T11:33:04.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:04.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382909+0000", 2024-09-18T11:33:04.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:04.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:04.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382915+0000", 2024-09-18T11:33:04.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6049999999999997e-06 2024-09-18T11:33:04.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:04.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382924+0000", 2024-09-18T11:33:04.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6289999999999999e-06 2024-09-18T11:33:04.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:04.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382929+0000", 2024-09-18T11:33:04.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0479999999999998e-06 2024-09-18T11:33:04.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:04.014 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383030+0000", 2024-09-18T11:33:04.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101346 2024-09-18T11:33:04.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:04.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383036+0000", 2024-09-18T11:33:04.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2720000000000003e-06 2024-09-18T11:33:04.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:04.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.390597+0000", 2024-09-18T11:33:04.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075606880000000003 2024-09-18T11:33:04.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:04.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.390696+0000", 2024-09-18T11:33:04.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8740000000000007e-05 2024-09-18T11:33:04.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:04.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392733+0000", 2024-09-18T11:33:04.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002037025 2024-09-18T11:33:04.016 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.016 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:04.016 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.017 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4350 4.b 4:d13decd8:::smithi11939506-2:head [sparse-read 0~1463453] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:33:04.017 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.382978+0000", 2024-09-18T11:33:04.017 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.564758556000001, 2024-09-18T11:33:04.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01020132, 2024-09-18T11:33:04.017 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:04.017 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:04.017 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:04.017 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:04.017 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:04.017 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4350 2024-09-18T11:33:04.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.018 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:04.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:04.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382978+0000", 2024-09-18T11:33:04.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:04.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382978+0000", 2024-09-18T11:33:04.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:04.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382976+0000", 2024-09-18T11:33:04.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:04.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:04.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382984+0000", 2024-09-18T11:33:04.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.328e-06 2024-09-18T11:33:04.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:04.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382994+0000", 2024-09-18T11:33:04.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0801999999999999e-05 2024-09-18T11:33:04.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:04.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383001+0000", 2024-09-18T11:33:04.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6390000000000003e-06 2024-09-18T11:33:04.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:04.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383138+0000", 2024-09-18T11:33:04.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013676899999999999 2024-09-18T11:33:04.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:04.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383144+0000", 2024-09-18T11:33:04.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6359999999999997e-06 2024-09-18T11:33:04.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:04.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392791+0000", 2024-09-18T11:33:04.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096461659999999994 2024-09-18T11:33:04.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:04.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392897+0000", 2024-09-18T11:33:04.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000106087 2024-09-18T11:33:04.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.022 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:04.022 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.393179+0000", 2024-09-18T11:33:04.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00028227500000000002 2024-09-18T11:33:04.022 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.022 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:04.022 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.022 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4375 4.b 4:dba5bd39:::smithi11939506-17:head [write 437081~638017,stat] snapc 6d=[6d,6c,69,67,61,55,51,4b,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:33:04.022 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.549471+0000", 2024-09-18T11:33:04.022 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.398264853999997, 2024-09-18T11:33:04.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097774179999999995, 2024-09-18T11:33:04.022 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:04.023 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:04.023 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:04.023 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:04.023 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:04.023 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4375 2024-09-18T11:33:04.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.023 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:04.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:04.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.549471+0000", 2024-09-18T11:33:04.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:04.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.549471+0000", 2024-09-18T11:33:04.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:04.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.549470+0000", 2024-09-18T11:33:04.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:04.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:04.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553463+0000", 2024-09-18T11:33:04.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039932520000000001 2024-09-18T11:33:04.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:04.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553478+0000", 2024-09-18T11:33:04.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4647999999999999e-05 2024-09-18T11:33:04.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:04.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553484+0000", 2024-09-18T11:33:04.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3799999999999999e-06 2024-09-18T11:33:04.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:04.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553536+0000", 2024-09-18T11:33:04.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1938999999999999e-05 2024-09-18T11:33:04.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:04.026 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553637+0000", 2024-09-18T11:33:04.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010074899999999999 2024-09-18T11:33:04.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.026 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:04.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553824+0000", 2024-09-18T11:33:04.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000187524 2024-09-18T11:33:04.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.027 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:04.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.556653+0000", 2024-09-18T11:33:04.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028285879999999999 2024-09-18T11:33:04.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.027 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:04.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.559176+0000", 2024-09-18T11:33:04.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025229369999999998 2024-09-18T11:33:04.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:04.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.559212+0000", 2024-09-18T11:33:04.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5750000000000002e-05 2024-09-18T11:33:04.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:04.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.559249+0000", 2024-09-18T11:33:04.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7020999999999997e-05 2024-09-18T11:33:04.028 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.028 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:04.028 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.029 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4430 4.b 4:d3ae8215:::smithi11939506-41:head [read 0~3832029,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T11:33:04.029 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:33:04.029 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 28.919225893, 2024-09-18T11:33:04.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010918034, 2024-09-18T11:33:04.029 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:04.029 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:04.029 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:04.029 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:04.029 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:04.029 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4430 2024-09-18T11:33:04.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.029 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:04.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:04.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:33:04.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:04.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:33:04.030 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.030 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:04.030 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:33:04.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:04.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:04.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028520+0000", 2024-09-18T11:33:04.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.005e-05 2024-09-18T11:33:04.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.031 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:04.031 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028538+0000", 2024-09-18T11:33:04.031 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8020999999999999e-05 2024-09-18T11:33:04.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.032 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:04.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028543+0000", 2024-09-18T11:33:04.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.485e-06 2024-09-18T11:33:04.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.032 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:04.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028634+0000", 2024-09-18T11:33:04.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1120999999999997e-05 2024-09-18T11:33:04.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.032 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:04.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028639+0000", 2024-09-18T11:33:04.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1989999999999997e-06 2024-09-18T11:33:04.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:04.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.036054+0000", 2024-09-18T11:33:04.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074147559999999998 2024-09-18T11:33:04.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:04.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.036158+0000", 2024-09-18T11:33:04.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103564 2024-09-18T11:33:04.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:04.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039428+0000", 2024-09-18T11:33:04.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032706390000000001 2024-09-18T11:33:04.034 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.034 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:04.034 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.034 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4431 4.b 4:d3ae8215:::smithi11939506-41:head [sparse-read 0~2481209] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T11:33:04.034 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:33:04.034 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 28.919187104999999, 2024-09-18T11:33:04.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011399179000000001, 2024-09-18T11:33:04.034 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:04.034 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:04.034 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:04.034 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:04.035 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:04.035 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4431 2024-09-18T11:33:04.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.035 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:04.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:04.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:33:04.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:04.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:33:04.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.036 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:04.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:33:04.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:04.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.036 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:04.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028552+0000", 2024-09-18T11:33:04.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5880000000000002e-06 2024-09-18T11:33:04.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.036 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:04.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028558+0000", 2024-09-18T11:33:04.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.84e-06 2024-09-18T11:33:04.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:04.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028562+0000", 2024-09-18T11:33:04.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6600000000000001e-06 2024-09-18T11:33:04.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:04.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028676+0000", 2024-09-18T11:33:04.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011408000000000001 2024-09-18T11:33:04.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:04.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028681+0000", 2024-09-18T11:33:04.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5350000000000004e-06 2024-09-18T11:33:04.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:04.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039458+0000", 2024-09-18T11:33:04.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010777231 2024-09-18T11:33:04.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:04.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039564+0000", 2024-09-18T11:33:04.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000105524 2024-09-18T11:33:04.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:04.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039948+0000", 2024-09-18T11:33:04.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00038430099999999998 2024-09-18T11:33:04.039 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.039 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:04.039 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.039 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4518 4.7 4:e0d95e10:::smithi11939506-22:head [rollback 6f] snapc 74=[74,73,71,6f,6c,69,67,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:33:04.039 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.055692+0000", 2024-09-18T11:33:04.039 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.892043844, 2024-09-18T11:33:04.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010093540999999999, 2024-09-18T11:33:04.039 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:04.040 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:04.040 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:04.040 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:04.040 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:04.040 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4518 2024-09-18T11:33:04.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.040 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:04.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:04.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055692+0000", 2024-09-18T11:33:04.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:04.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055692+0000", 2024-09-18T11:33:04.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:04.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055692+0000", 2024-09-18T11:33:04.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:04.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:04.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055697+0000", 2024-09-18T11:33:04.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.101e-06 2024-09-18T11:33:04.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:04.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055707+0000", 2024-09-18T11:33:04.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0316000000000001e-05 2024-09-18T11:33:04.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:04.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055713+0000", 2024-09-18T11:33:04.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8900000000000004e-06 2024-09-18T11:33:04.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.042 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:04.042 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057453+0000", 2024-09-18T11:33:04.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001739953 2024-09-18T11:33:04.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.043 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:04.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057537+0000", 2024-09-18T11:33:04.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4287000000000001e-05 2024-09-18T11:33:04.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.043 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:33:04.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057760+0000", 2024-09-18T11:33:04.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022294700000000001 2024-09-18T11:33:04.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.044 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:04.044 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065674+0000", 2024-09-18T11:33:04.044 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079140110000000003 2024-09-18T11:33:04.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.044 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:04.044 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065681+0000", 2024-09-18T11:33:04.044 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2289999999999997e-06 2024-09-18T11:33:04.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.044 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:04.044 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065734+0000", 2024-09-18T11:33:04.044 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3136000000000001e-05 2024-09-18T11:33:04.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.045 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:04.045 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065786+0000", 2024-09-18T11:33:04.045 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1419999999999999e-05 2024-09-18T11:33:04.045 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.045 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:04.045 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.045 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4519 4.7 4:e0d95e10:::smithi11939506-22:head [append 0~0] snapc 74=[74,73,71,6f,6c,69,67,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:33:04.045 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.055719+0000", 2024-09-18T11:33:04.045 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 23.892017681999999, 2024-09-18T11:33:04.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010210589000000001, 2024-09-18T11:33:04.046 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:04.046 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:04.046 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:04.046 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:04.046 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:04.046 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4519 2024-09-18T11:33:04.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.046 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:04.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.046 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:04.046 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055719+0000", 2024-09-18T11:33:04.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.047 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:04.047 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055719+0000", 2024-09-18T11:33:04.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.047 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:04.047 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055718+0000", 2024-09-18T11:33:04.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:04.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.048 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:04.048 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055721+0000", 2024-09-18T11:33:04.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7039999999999999e-06 2024-09-18T11:33:04.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.048 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:04.048 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055728+0000", 2024-09-18T11:33:04.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1620000000000002e-06 2024-09-18T11:33:04.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.048 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:04.048 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055732+0000", 2024-09-18T11:33:04.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8410000000000002e-06 2024-09-18T11:33:04.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.049 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:04.049 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.059646+0000", 2024-09-18T11:33:04.049 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039138489999999996 2024-09-18T11:33:04.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.049 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:04.049 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.059733+0000", 2024-09-18T11:33:04.049 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7163999999999994e-05 2024-09-18T11:33:04.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.049 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:33:04.050 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.059891+0000", 2024-09-18T11:33:04.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000158102 2024-09-18T11:33:04.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.050 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:04.050 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065798+0000", 2024-09-18T11:33:04.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059067649999999996 2024-09-18T11:33:04.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.050 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:04.050 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065840+0000", 2024-09-18T11:33:04.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2515999999999997e-05 2024-09-18T11:33:04.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.051 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:04.051 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065890+0000", 2024-09-18T11:33:04.051 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0055000000000001e-05 2024-09-18T11:33:04.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.051 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:04.051 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065929+0000", 2024-09-18T11:33:04.051 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8831000000000001e-05 2024-09-18T11:33:04.051 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.051 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:04.051 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.051 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4756 4.7 4:e301aefd:::smithi11939506-19:head [write 591712~786646,stat] snapc 7e=[7e,7c,7b,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e179)", 2024-09-18T11:33:04.052 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:50.036231+0000", 2024-09-18T11:33:04.052 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 13.911505038, 2024-09-18T11:33:04.052 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010177099, 2024-09-18T11:33:04.052 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:04.052 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:04.052 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:04.052 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:04.052 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:04.052 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4756 2024-09-18T11:33:04.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.052 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:04.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.052 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:04.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.036231+0000", 2024-09-18T11:33:04.053 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.053 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:04.053 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.036231+0000", 2024-09-18T11:33:04.053 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.053 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:04.053 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.036229+0000", 2024-09-18T11:33:04.053 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:04.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.054 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:04.054 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.037165+0000", 2024-09-18T11:33:04.054 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00093574399999999996 2024-09-18T11:33:04.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.054 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:04.054 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.037183+0000", 2024-09-18T11:33:04.054 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7431999999999998e-05 2024-09-18T11:33:04.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.054 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:04.054 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.037189+0000", 2024-09-18T11:33:04.054 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4590000000000003e-06 2024-09-18T11:33:04.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:04.055 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.037243+0000", 2024-09-18T11:33:04.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4237999999999998e-05 2024-09-18T11:33:04.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:33:04.055 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.037249+0000", 2024-09-18T11:33:04.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.699e-06 2024-09-18T11:33:04.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:04.055 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.038725+0000", 2024-09-18T11:33:04.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014756420000000001 2024-09-18T11:33:04.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.056 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:04.056 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.038797+0000", 2024-09-18T11:33:04.056 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2446000000000004e-05 2024-09-18T11:33:04.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.056 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:04.056 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.038915+0000", 2024-09-18T11:33:04.056 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011769000000000001 2024-09-18T11:33:04.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.056 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:04.057 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.041337+0000", 2024-09-18T11:33:04.057 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024226339999999999 2024-09-18T11:33:04.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.057 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:04.057 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.046311+0000", 2024-09-18T11:33:04.057 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049738359999999997 2024-09-18T11:33:04.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.057 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:04.057 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.046359+0000", 2024-09-18T11:33:04.057 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7586999999999997e-05 2024-09-18T11:33:04.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.058 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:04.058 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.046408+0000", 2024-09-18T11:33:04.058 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9478000000000003e-05 2024-09-18T11:33:04.058 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.058 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:04.058 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.058 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.058 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:4790 3.2 e182/21 3:79129b11:::2f02c59c01a730562a65f21445c8af0e4a8aab3b969b8a3abf39e7b1bc84f918:head v 182'203, mlcod=176'201)", 2024-09-18T11:33:04.058 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:53.043420+0000", 2024-09-18T11:33:04.058 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 10.904316194, 2024-09-18T11:33:04.059 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096982140000000001, 2024-09-18T11:33:04.059 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:04.059 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:04.059 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:04.059 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.059 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:04.059 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.043420+0000", 2024-09-18T11:33:04.059 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.059 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.059 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.059 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:04.059 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.043420+0000", 2024-09-18T11:33:04.059 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:04.059 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.059 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.060 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:04.060 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.043420+0000", 2024-09-18T11:33:04.060 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:04.060 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.060 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.060 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:04.060 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.046026+0000", 2024-09-18T11:33:04.060 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026064590000000002 2024-09-18T11:33:04.060 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.060 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.060 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:04.060 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.046040+0000", 2024-09-18T11:33:04.060 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.364e-05 2024-09-18T11:33:04.061 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.061 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.061 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:04.061 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.046045+0000", 2024-09-18T11:33:04.061 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6010000000000002e-06 2024-09-18T11:33:04.061 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.061 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.061 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:04.061 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.046080+0000", 2024-09-18T11:33:04.061 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4702000000000001e-05 2024-09-18T11:33:04.061 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.061 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.061 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:04.061 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.046095+0000", 2024-09-18T11:33:04.062 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5265e-05 2024-09-18T11:33:04.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.062 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.062 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:04.062 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.053085+0000", 2024-09-18T11:33:04.062 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0069894559999999998 2024-09-18T11:33:04.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:04.062 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:04.062 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:04.062 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.053118+0000", 2024-09-18T11:33:04.062 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3565999999999997e-05 2024-09-18T11:33:04.062 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.062 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:04.062 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.063 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:04.063 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:04.063 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:04.065 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T11:33:04.065 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:33:04.066 INFO:tasks.thrashosds.thrasher:test_map_discontinuity 2024-09-18T11:33:04.066 INFO:tasks.thrashosds.thrasher:Waiting for recovery 2024-09-18T11:33:04.066 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-18T11:33:04.066 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T11:33:04.066 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:04.062+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:33:04.067 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:33:04.073 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T11:33:04.100 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:04.098+0000 7f2d429e4640 -1 osd.1 192 handle_osd_map injecting map crc failure 2024-09-18T11:33:04.165 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:04.165 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:04.165 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:33:04.166 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:04.166 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:04.164+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:04.174 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:33:04.231 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:33:04.231 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:33:04.262 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:33:04.263 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:33:04.266 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:33:04.265+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:33:04.291 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:33:04.292 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:33:04.311 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:33:04.311 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:33:04.313 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:04.313 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:04.313 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:33:04.313 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:33:04.313 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:04.322 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:33:04.342 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:33:04.342 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:33:04.363 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:33:04.364 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:33:04.366 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:33:04.365+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:33:04.378 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:33:04.379 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:33:04.395 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:33:04.395 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:33:04.441 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:33:04.441 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":193,"fsid":"f4cca259-b87a-45cc-8995-7e9fca891a9f","created":"2024-09-18T11:29:38.205487+0000","modified":"2024-09-18T11:33:04.026229+0000","last_up_change":"2024-09-18T11:31:49.142260+0000","last_in_change":"2024-09-18T11:32:45.440256+0000","flags":"nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5804032,"flags_set":["nodeep-scrub","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T11:29:47.951262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T11:29:48.866238+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"low_tier","create_time":"2024-09-18T11:29:59.343964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}},{"pool":4,"pool_name":"unique_pool_0","create_time":"2024-09-18T11:30:00.523818+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":131,"snap_epoch":184,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"dedup_cdc_chunk_size":131072,"dedup_chunk_algorithm":"fastcdc","dedup_tier":3,"fingerprint_algorithm":"sha256"},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"12faced2-7888-4265-ad12-815efe36466c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":65,"up_from":119,"up_thru":189,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":56607},{"type":"v1","addr":"172.21.15.119:6801","nonce":56607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":56607},{"type":"v1","addr":"172.21.15.119:6803","nonce":56607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":56607},{"type":"v1","addr":"172.21.15.119:6807","nonce":56607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":56607},{"type":"v1","addr":"172.21.15.119:6805","nonce":56607}]},"public_addr":"172.21.15.119:6801/56607","cluster_addr":"172.21.15.119:6803/56607","heartbeat_back_addr":"172.21.15.119:6807/56607","heartbeat_front_addr":"172.21.15.119:6805/56607","state":["exists","up"]},{"osd":1,"uuid":"20fee29d-b047-46ae-aa60-88e897fbb0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":33350},{"type":"v1","addr":"172.21.15.119:6825","nonce":33350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":33350},{"type":"v1","addr":"172.21.15.119:6827","nonce":33350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":33350},{"type":"v1","addr":"172.21.15.119:6831","nonce":33350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":33350},{"type":"v1","addr":"172.21.15.119:6829","nonce":33350}]},"public_addr":"172.21.15.119:6825/33350","cluster_addr":"172.21.15.119:6827/33350","heartbeat_back_addr":"172.21.15.119:6831/33350","heartbeat_front_addr":"172.21.15.119:6829/33350","state":["exists","up"]},{"osd":2,"uuid":"21a296f3-b259-4841-9a17-4cd21b302b36","up":1,"in":1,"weight":1,"primary_affinity":0,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":33351},{"type":"v1","addr":"172.21.15.119:6809","nonce":33351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":33351},{"type":"v1","addr":"172.21.15.119:6813","nonce":33351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6819","nonce":33351},{"type":"v1","addr":"172.21.15.119:6821","nonce":33351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":33351},{"type":"v1","addr":"172.21.15.119:6817","nonce":33351}]},"public_addr":"172.21.15.119:6809/33351","cluster_addr":"172.21.15.119:6813/33351","heartbeat_back_addr":"172.21.15.119:6821/33351","heartbeat_front_addr":"172.21.15.119:6817/33351","state":["exists","up"]},{"osd":3,"uuid":"59044f30-32e1-4942-9a8e-202c1def221a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":33349},{"type":"v1","addr":"172.21.15.119:6812","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":33349},{"type":"v1","addr":"172.21.15.119:6816","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":33349},{"type":"v1","addr":"172.21.15.119:6823","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":33349},{"type":"v1","addr":"172.21.15.119:6820","nonce":33349}]},"public_addr":"172.21.15.119:6812/33349","cluster_addr":"172.21.15.119:6816/33349","heartbeat_back_addr":"172.21.15.119:6823/33349","heartbeat_front_addr":"172.21.15.119:6820/33349","state":["exists","up"]},{"osd":4,"uuid":"7d1ae3d0-8d69-4d3c-9e51-24fdd77a22a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":33088},{"type":"v1","addr":"172.21.15.148:6809","nonce":33088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":33088},{"type":"v1","addr":"172.21.15.148:6811","nonce":33088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":33088},{"type":"v1","addr":"172.21.15.148:6815","nonce":33088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":33088},{"type":"v1","addr":"172.21.15.148:6813","nonce":33088}]},"public_addr":"172.21.15.148:6809/33088","cluster_addr":"172.21.15.148:6811/33088","heartbeat_back_addr":"172.21.15.148:6815/33088","heartbeat_front_addr":"172.21.15.148:6813/33088","state":["exists","up"]},{"osd":5,"uuid":"b0dd3d30-b273-428c-a4a9-4c238abf45fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":33089},{"type":"v1","addr":"172.21.15.148:6801","nonce":33089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":33089},{"type":"v1","addr":"172.21.15.148:6803","nonce":33089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":33089},{"type":"v1","addr":"172.21.15.148:6807","nonce":33089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":33089},{"type":"v1","addr":"172.21.15.148:6805","nonce":33089}]},"public_addr":"172.21.15.148:6801/33089","cluster_addr":"172.21.15.148:6803/33089","heartbeat_back_addr":"172.21.15.148:6807/33089","heartbeat_front_addr":"172.21.15.148:6805/33089","state":["exists","up"]},{"osd":6,"uuid":"40290cb3-2334-4c68-8e6c-59d0d5455171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":33107},{"type":"v1","addr":"172.21.15.148:6817","nonce":33107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":33107},{"type":"v1","addr":"172.21.15.148:6819","nonce":33107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":33107},{"type":"v1","addr":"172.21.15.148:6823","nonce":33107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":33107},{"type":"v1","addr":"172.21.15.148:6821","nonce":33107}]},"public_addr":"172.21.15.148:6817/33107","cluster_addr":"172.21.15.148:6819/33107","heartbeat_back_addr":"172.21.15.148:6823/33107","heartbeat_front_addr":"172.21.15.148:6821/33107","state":["exists","up"]},{"osd":7,"uuid":"54802abe-ea1a-448e-85af-d1f14a45be27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":33119},{"type":"v1","addr":"172.21.15.148:6825","nonce":33119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":33119},{"type":"v1","addr":"172.21.15.148:6827","nonce":33119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":33119},{"type":"v1","addr":"172.21.15.148:6831","nonce":33119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":33119},{"type":"v1","addr":"172.21.15.148:6829","nonce":33119}]},"public_addr":"172.21.15.148:6825/33119","cluster_addr":"172.21.15.148:6827/33119","heartbeat_back_addr":"172.21.15.148:6831/33119","heartbeat_front_addr":"172.21.15.148:6829/33119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:31:49.015287+0000","dead_epoch":66},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:32:14.430356+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:32:27.493785+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:32:12.546041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:32:13.027907+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:32:28.500831+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:32:14.951013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:32:27.373504+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"4.b","osds":[5,3]}],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":4,"snaps":[{"begin":108,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T11:33:04.454 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-18T11:33:04.461 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:04.461 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:04.470 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:33:04.494 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:04.492+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:33:04.568 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:04.568 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:04.568 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:33:04.567+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:33:04.576 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:33:04.669 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:04.667+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:33:04.673 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:04.673 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:04.682 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:33:04.770 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:04.768+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:33:04.779 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:04.779 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:04.788 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:33:04.871 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:04.869+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:33:04.889 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:04.890 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:04.899 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:33:04.972 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:04.971+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:05.037 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:05.037 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:05.046 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:33:05.070 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:05.068+0000 7fb6950ff640 -1 osd.2 193 handle_osd_map injecting map crc failure 2024-09-18T11:33:05.072 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is unset 2024-09-18T11:33:05.072 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:33:05.071+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:33:05.149 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:05.149 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:05.158 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:33:05.201 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:33:05.199+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:33:05.257 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:05.257 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:05.265 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:33:05.273 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:33:05.272+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:33:05.375 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:33:05.374+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:33:05.403 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:05.403 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:05.412 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:33:05.475 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:33:05.473+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:33:05.510 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:05.510 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:05.519 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:33:05.603 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:05.601+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:33:05.658 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:05.658 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:05.666 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:33:05.707 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:05.705+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:33:05.777 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:05.776+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:05.804 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:05.804 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:05.812 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:33:05.878 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:05.876+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:33:05.910 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:05.910 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:05.918 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:33:05.979 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:33:05.978+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:33:06.056 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:06.056 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:06.064 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:33:06.079 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:06.078+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:06.162 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:06.162 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:06.171 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:33:06.215 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:06.213+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:33:06.270 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:06.270 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:06.279 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:33:06.282 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:33:06.281+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:33:06.383 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:06.381+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:33:06.385 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:06.385 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:06.395 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:33:06.407 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:33:06.484 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:06.482+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:33:06.493 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:06.493 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:06.502 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:33:06.522 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:33:06.522 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:33:06.537 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:33:06.537 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:33:06.552 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:33:06.552 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:33:06.566 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:33:06.567 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:33:06.584 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:33:06.585 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:33:06.585 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:33:06.584+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:33:06.597 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:33:06.598 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:33:06.601 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:06.601 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:33:06.601 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:33:06.601 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:33:06.601 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.602 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:33:06.602 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:33:06.602 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 152.18302102000001, 2024-09-18T11:33:06.602 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:33:06.602 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:06.602 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:06.602 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:06.602 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:06.602 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:06.602 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:33:06.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.602 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:06.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.603 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:06.603 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:33:06.603 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.603 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:06.603 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:33:06.603 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.603 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:06.603 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:33:06.604 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:06.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.604 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.604 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:06.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:33:06.604 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:33:06.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.604 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.604 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:06.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:33:06.604 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:33:06.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.604 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.605 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:06.605 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:33:06.605 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:33:06.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.605 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.605 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:33:06.605 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:33:06.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.605 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:06.605 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:33:06.605 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:33:06.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.606 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:06.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:33:06.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:33:06.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.606 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:06.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:33:06.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:33:06.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.607 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:06.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:33:06.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:33:06.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.607 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:06.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:33:06.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:33:06.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.607 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:06.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:33:06.608 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:33:06.608 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.608 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:06.608 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.608 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:33:06.608 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:33:06.608 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 125.212791614, 2024-09-18T11:33:06.608 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:33:06.608 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:06.608 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:06.609 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:06.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.609 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:06.609 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:33:06.609 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.609 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:06.609 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:33:06.609 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.610 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:06.610 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:33:06.610 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:06.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.610 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.610 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:06.610 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:33:06.610 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:33:06.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.610 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.610 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:06.610 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:33:06.610 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:33:06.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:06.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:33:06.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:33:06.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:33:06.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:33:06.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:06.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:33:06.612 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:33:06.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.612 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:06.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:33:06.612 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:33:06.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.612 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:06.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:33:06.612 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:33:06.612 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.612 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:06.613 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.613 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:33:06.613 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:33:06.613 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 117.626679008, 2024-09-18T11:33:06.613 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:33:06.613 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:06.613 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:06.613 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:06.613 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:06.613 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:06.613 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:33:06.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.614 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:06.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.614 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:06.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:33:06.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.614 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:06.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:33:06.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.615 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:06.615 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:33:06.615 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:06.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.615 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:06.615 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:33:06.615 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:33:06.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.615 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:06.615 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:33:06.615 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:33:06.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.616 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:06.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:33:06.616 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:33:06.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.616 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:33:06.616 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:33:06.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.617 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:06.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:33:06.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:33:06.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.617 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:06.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:33:06.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:33:06.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:06.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:33:06.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:33:06.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:06.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:33:06.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:33:06.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:06.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:33:06.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:33:06.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.619 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:06.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:33:06.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:33:06.619 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.619 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:06.619 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.619 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2956 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~2100908,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:33:06.620 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:33:06.620 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 102.571882758, 2024-09-18T11:33:06.620 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011863831, 2024-09-18T11:33:06.620 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:06.620 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:06.620 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:06.620 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:06.620 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:06.620 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2956 2024-09-18T11:33:06.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.620 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:06.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:06.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:33:06.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:06.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:33:06.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:06.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:33:06.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:06.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:06.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027228+0000", 2024-09-18T11:33:06.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:33:06.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:06.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027235+0000", 2024-09-18T11:33:06.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:33:06.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:06.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027239+0000", 2024-09-18T11:33:06.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.535e-06 2024-09-18T11:33:06.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.623 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027831+0000", 2024-09-18T11:33:06.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059244699999999996 2024-09-18T11:33:06.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.623 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:06.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027879+0000", 2024-09-18T11:33:06.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:33:06.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.623 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038098+0000", 2024-09-18T11:33:06.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010219137 2024-09-18T11:33:06.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.624 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:06.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038207+0000", 2024-09-18T11:33:06.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010951999999999999 2024-09-18T11:33:06.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.624 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:06.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039089+0000", 2024-09-18T11:33:06.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00088151699999999996 2024-09-18T11:33:06.624 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.624 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:06.624 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.625 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2957 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:33:06.625 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:33:06.625 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 102.57186497399999, 2024-09-18T11:33:06.625 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012041979, 2024-09-18T11:33:06.625 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:06.625 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:06.625 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:06.625 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:06.625 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:06.625 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2957 2024-09-18T11:33:06.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.625 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:06.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.625 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:06.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:33:06.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:06.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:33:06.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:06.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:33:06.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:06.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:06.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027245+0000", 2024-09-18T11:33:06.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:33:06.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:06.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027251+0000", 2024-09-18T11:33:06.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0090000000000004e-06 2024-09-18T11:33:06.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.628 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:06.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027255+0000", 2024-09-18T11:33:06.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:33:06.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.628 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027895+0000", 2024-09-18T11:33:06.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064076400000000005 2024-09-18T11:33:06.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:06.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027930+0000", 2024-09-18T11:33:06.629 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4334e-05 2024-09-18T11:33:06.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039111+0000", 2024-09-18T11:33:06.629 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181455999999999 2024-09-18T11:33:06.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:06.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039156+0000", 2024-09-18T11:33:06.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5256000000000002e-05 2024-09-18T11:33:06.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:06.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039285+0000", 2024-09-18T11:33:06.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012839200000000001 2024-09-18T11:33:06.630 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.630 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:06.631 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.631 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:978 4.b e132/67 4:d8d14c58:::smithi11939506-49:30 v 132'417, mlcod=128'414)", 2024-09-18T11:33:06.631 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:33:06.631 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 63.573833692000001, 2024-09-18T11:33:06.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011942183, 2024-09-18T11:33:06.631 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:06.631 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:06.631 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:06.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:06.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:33:06.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.632 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:06.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:33:06.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.632 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:06.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:33:06.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:33:06.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:06.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025278+0000", 2024-09-18T11:33:06.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9060000000000004e-06 2024-09-18T11:33:06.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:06.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025288+0000", 2024-09-18T11:33:06.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.028e-05 2024-09-18T11:33:06.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:06.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025293+0000", 2024-09-18T11:33:06.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2700000000000004e-06 2024-09-18T11:33:06.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025346+0000", 2024-09-18T11:33:06.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2862999999999999e-05 2024-09-18T11:33:06.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:06.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025370+0000", 2024-09-18T11:33:06.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4005000000000001e-05 2024-09-18T11:33:06.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:06.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037159+0000", 2024-09-18T11:33:06.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011788343 2024-09-18T11:33:06.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.635 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:06.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037216+0000", 2024-09-18T11:33:06.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7695000000000002e-05 2024-09-18T11:33:06.635 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.635 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:06.635 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.635 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3930 4.b e132/67 4:d3ae8215:::smithi11939506-41:head v 132'421, mlcod=128'414)", 2024-09-18T11:33:06.635 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:33:06.635 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 63.570285444, 2024-09-18T11:33:06.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015093554, 2024-09-18T11:33:06.636 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:06.636 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:06.636 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:06.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.636 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:06.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:33:06.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.636 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:06.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:33:06.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:06.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028821+0000", 2024-09-18T11:33:06.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:06.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:06.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029805+0000", 2024-09-18T11:33:06.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00098410899999999994 2024-09-18T11:33:06.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:06.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029817+0000", 2024-09-18T11:33:06.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2058e-05 2024-09-18T11:33:06.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:06.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029823+0000", 2024-09-18T11:33:06.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6450000000000002e-06 2024-09-18T11:33:06.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032330+0000", 2024-09-18T11:33:06.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025067710000000001 2024-09-18T11:33:06.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.639 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:06.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032349+0000", 2024-09-18T11:33:06.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9729e-05 2024-09-18T11:33:06.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.639 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:06.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043878+0000", 2024-09-18T11:33:06.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011528514 2024-09-18T11:33:06.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.640 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:06.640 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043916+0000", 2024-09-18T11:33:06.640 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8049000000000001e-05 2024-09-18T11:33:06.640 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.640 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:06.640 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.640 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:162 3.1 3:8c826a1f:::e682cfc1e82c88472b9acf1cd97978bc105aa035f10b685bb4cace95809b52bf:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:06.640 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:33:06.640 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.018163285999997, 2024-09-18T11:33:06.640 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013234571000000001, 2024-09-18T11:33:06.641 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:06.641 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:06.641 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:06.641 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.641 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:06.641 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:33:06.641 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.641 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.641 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:06.641 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:33:06.641 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:06.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580943+0000", 2024-09-18T11:33:06.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:06.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:06.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582556+0000", 2024-09-18T11:33:06.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016132970000000001 2024-09-18T11:33:06.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:06.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582572+0000", 2024-09-18T11:33:06.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5846999999999999e-05 2024-09-18T11:33:06.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:06.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582578+0000", 2024-09-18T11:33:06.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4810000000000003e-06 2024-09-18T11:33:06.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582632+0000", 2024-09-18T11:33:06.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4156000000000001e-05 2024-09-18T11:33:06.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:06.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582819+0000", 2024-09-18T11:33:06.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000186431 2024-09-18T11:33:06.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.644 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:06.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583118+0000", 2024-09-18T11:33:06.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029880100000000001 2024-09-18T11:33:06.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.644 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:06.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590676+0000", 2024-09-18T11:33:06.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075588670000000004 2024-09-18T11:33:06.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.645 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:06.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594111+0000", 2024-09-18T11:33:06.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034348550000000001 2024-09-18T11:33:06.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.645 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:06.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594142+0000", 2024-09-18T11:33:06.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.099e-05 2024-09-18T11:33:06.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.645 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:06.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594179+0000", 2024-09-18T11:33:06.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6831e-05 2024-09-18T11:33:06.646 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.646 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:06.646 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.646 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3974 4.d 4:b5a463bf:::smithi11939506-20:head [write 665643~697786,stat] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:06.646 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:33:06.646 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.017076603, 2024-09-18T11:33:06.646 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016621174999999998, 2024-09-18T11:33:06.646 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:06.646 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:06.646 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:06.646 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:06.646 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:06.646 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3974 2024-09-18T11:33:06.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.647 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:06.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:06.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:33:06.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:06.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:33:06.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:06.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582030+0000", 2024-09-18T11:33:06.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:06.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:06.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585185+0000", 2024-09-18T11:33:06.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003155644 2024-09-18T11:33:06.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.649 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:06.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585208+0000", 2024-09-18T11:33:06.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3013000000000001e-05 2024-09-18T11:33:06.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.649 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:06.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585215+0000", 2024-09-18T11:33:06.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7569999999999998e-06 2024-09-18T11:33:06.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.649 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585278+0000", 2024-09-18T11:33:06.650 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2699000000000004e-05 2024-09-18T11:33:06.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.650 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:06.650 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585374+0000", 2024-09-18T11:33:06.650 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.6062000000000003e-05 2024-09-18T11:33:06.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.650 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:33:06.650 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585577+0000", 2024-09-18T11:33:06.650 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000202926 2024-09-18T11:33:06.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.650 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:06.650 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596544+0000", 2024-09-18T11:33:06.651 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010966848 2024-09-18T11:33:06.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.651 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:06.651 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598594+0000", 2024-09-18T11:33:06.651 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020502049999999998 2024-09-18T11:33:06.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.651 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:06.651 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598625+0000", 2024-09-18T11:33:06.651 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1412999999999999e-05 2024-09-18T11:33:06.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.651 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:06.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598652+0000", 2024-09-18T11:33:06.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7166999999999999e-05 2024-09-18T11:33:06.652 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.652 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:06.652 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.652 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:166 3.1 3:a73e635b:::b6ee478f0f661ec7bac3d17c2ca2fdaff71a47570f64844a7ba79bbb2a0e8d9d:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:06.652 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:33:06.652 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.016517237000002, 2024-09-18T11:33:06.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011711037000000001, 2024-09-18T11:33:06.652 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:06.652 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:06.652 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:06.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:06.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:33:06.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:06.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:33:06.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:06.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582590+0000", 2024-09-18T11:33:06.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:06.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:06.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582673+0000", 2024-09-18T11:33:06.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2861000000000005e-05 2024-09-18T11:33:06.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:06.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582683+0000", 2024-09-18T11:33:06.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0081999999999999e-05 2024-09-18T11:33:06.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:06.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582688+0000", 2024-09-18T11:33:06.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6870000000000002e-06 2024-09-18T11:33:06.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584266+0000", 2024-09-18T11:33:06.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001578423 2024-09-18T11:33:06.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:06.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584407+0000", 2024-09-18T11:33:06.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014144599999999999 2024-09-18T11:33:06.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:06.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584599+0000", 2024-09-18T11:33:06.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000191881 2024-09-18T11:33:06.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.656 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:06.656 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590680+0000", 2024-09-18T11:33:06.656 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060801479999999996 2024-09-18T11:33:06.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.656 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:06.656 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594224+0000", 2024-09-18T11:33:06.656 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035441700000000001 2024-09-18T11:33:06.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.656 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:06.656 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594269+0000", 2024-09-18T11:33:06.657 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5297999999999999e-05 2024-09-18T11:33:06.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.657 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:06.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594302+0000", 2024-09-18T11:33:06.657 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2703000000000002e-05 2024-09-18T11:33:06.657 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.657 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:06.657 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.657 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:168 3.1 3:b64d7bad:::7feb22256bdcec217720fd46184ab43ced8923ed66ec6319c57f078d9f123262:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:06.657 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:33:06.658 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.016289784999998, 2024-09-18T11:33:06.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011589809, 2024-09-18T11:33:06.658 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:06.658 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:06.658 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:06.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:06.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:33:06.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:06.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:33:06.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:06.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582817+0000", 2024-09-18T11:33:06.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:06.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:06.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582903+0000", 2024-09-18T11:33:06.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5915999999999995e-05 2024-09-18T11:33:06.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:06.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582913+0000", 2024-09-18T11:33:06.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1409999999999994e-06 2024-09-18T11:33:06.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:06.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582917+0000", 2024-09-18T11:33:06.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6419999999999998e-06 2024-09-18T11:33:06.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585540+0000", 2024-09-18T11:33:06.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026228940000000002 2024-09-18T11:33:06.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:06.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585721+0000", 2024-09-18T11:33:06.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018086299999999999 2024-09-18T11:33:06.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:06.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585918+0000", 2024-09-18T11:33:06.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000196758 2024-09-18T11:33:06.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:06.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590683+0000", 2024-09-18T11:33:06.662 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047657020000000001 2024-09-18T11:33:06.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.662 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:06.662 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594354+0000", 2024-09-18T11:33:06.662 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003671068 2024-09-18T11:33:06.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.662 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:06.662 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594386+0000", 2024-09-18T11:33:06.662 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1426e-05 2024-09-18T11:33:06.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.662 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:06.662 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594408+0000", 2024-09-18T11:33:06.663 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1987000000000001e-05 2024-09-18T11:33:06.663 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.663 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:06.663 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.663 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.663 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:175 3.1 3:b5e86b88:::e55cf3d5cf782ed327a67e746b287b76d05f74f58a4f3726e399a564440ceaf6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:06.663 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:33:06.663 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.015440843999997, 2024-09-18T11:33:06.663 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013709627, 2024-09-18T11:33:06.663 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:06.663 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:06.663 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:06.663 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:06.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:33:06.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:06.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:33:06.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:06.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583666+0000", 2024-09-18T11:33:06.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:06.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.665 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:06.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584041+0000", 2024-09-18T11:33:06.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037511499999999998 2024-09-18T11:33:06.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.665 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:06.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584055+0000", 2024-09-18T11:33:06.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3295999999999999e-05 2024-09-18T11:33:06.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.665 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:06.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584061+0000", 2024-09-18T11:33:06.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1630000000000001e-06 2024-09-18T11:33:06.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.589980+0000", 2024-09-18T11:33:06.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005918998 2024-09-18T11:33:06.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:06.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590117+0000", 2024-09-18T11:33:06.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013715400000000001 2024-09-18T11:33:06.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:06.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590270+0000", 2024-09-18T11:33:06.667 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015245600000000001 2024-09-18T11:33:06.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.667 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.667 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:06.667 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596513+0000", 2024-09-18T11:33:06.667 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062438729999999996 2024-09-18T11:33:06.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.667 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.667 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:06.667 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597328+0000", 2024-09-18T11:33:06.667 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081489599999999996 2024-09-18T11:33:06.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.667 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.667 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:06.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597358+0000", 2024-09-18T11:33:06.668 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9291000000000001e-05 2024-09-18T11:33:06.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.668 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:06.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597377+0000", 2024-09-18T11:33:06.668 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:33:06.668 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.668 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:06.669 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.669 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.669 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4061 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:06.669 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:33:06.669 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.302262155000001, 2024-09-18T11:33:06.669 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013340967, 2024-09-18T11:33:06.669 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:06.669 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:06.669 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:06.669 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:06.669 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:06.669 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4061 2024-09-18T11:33:06.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.670 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:06.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.670 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:06.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:33:06.670 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.670 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:06.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:33:06.670 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.670 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:06.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296844+0000", 2024-09-18T11:33:06.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:06.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.671 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:06.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296854+0000", 2024-09-18T11:33:06.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0037000000000001e-05 2024-09-18T11:33:06.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.671 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:06.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296873+0000", 2024-09-18T11:33:06.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:33:06.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.671 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:06.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296881+0000", 2024-09-18T11:33:06.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3790000000000006e-06 2024-09-18T11:33:06.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296975+0000", 2024-09-18T11:33:06.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3536999999999996e-05 2024-09-18T11:33:06.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:33:06.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296982+0000", 2024-09-18T11:33:06.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6399999999999997e-06 2024-09-18T11:33:06.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.673 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.673 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301933+0000", 2024-09-18T11:33:06.673 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049507190000000001 2024-09-18T11:33:06.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.673 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.673 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:06.673 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301984+0000", 2024-09-18T11:33:06.673 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1439e-05 2024-09-18T11:33:06.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.674 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:06.674 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:33:06.674 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082022050000000006 2024-09-18T11:33:06.674 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.674 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:06.674 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.674 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4062 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:06.674 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:33:06.674 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.302213600000002, 2024-09-18T11:33:06.674 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016297147000000001, 2024-09-18T11:33:06.674 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:06.675 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:06.675 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:06.675 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:06.675 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:06.675 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4062 2024-09-18T11:33:06.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.675 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:06.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:06.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:33:06.675 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.676 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:06.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:33:06.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.676 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:06.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:33:06.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:06.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.676 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:06.676 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296898+0000", 2024-09-18T11:33:06.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9419999999999997e-06 2024-09-18T11:33:06.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.677 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:06.677 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296906+0000", 2024-09-18T11:33:06.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3520000000000007e-06 2024-09-18T11:33:06.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.677 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:06.677 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296910+0000", 2024-09-18T11:33:06.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4469999999999999e-06 2024-09-18T11:33:06.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.678 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297032+0000", 2024-09-18T11:33:06.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012168900000000001 2024-09-18T11:33:06.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.678 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:06.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297038+0000", 2024-09-18T11:33:06.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9320000000000001e-06 2024-09-18T11:33:06.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.678 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310221+0000", 2024-09-18T11:33:06.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013182887000000001 2024-09-18T11:33:06.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.679 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:06.679 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310323+0000", 2024-09-18T11:33:06.679 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010170000000000001 2024-09-18T11:33:06.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.679 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:06.679 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313191+0000", 2024-09-18T11:33:06.679 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002868808 2024-09-18T11:33:06.679 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.679 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:06.679 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.680 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4063 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:06.680 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:33:06.680 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.302192708, 2024-09-18T11:33:06.680 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017648712, 2024-09-18T11:33:06.680 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:06.680 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:06.680 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:06.680 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:06.680 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:06.680 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4063 2024-09-18T11:33:06.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.681 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:06.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.681 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:06.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:33:06.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.681 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:06.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:33:06.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.681 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:06.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:33:06.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:06.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.682 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:06.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296918+0000", 2024-09-18T11:33:06.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8100000000000002e-06 2024-09-18T11:33:06.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.682 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:06.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296924+0000", 2024-09-18T11:33:06.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2519999999999996e-06 2024-09-18T11:33:06.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.683 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:06.683 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296928+0000", 2024-09-18T11:33:06.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0500000000000002e-06 2024-09-18T11:33:06.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.683 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.683 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297062+0000", 2024-09-18T11:33:06.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013425500000000001 2024-09-18T11:33:06.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.683 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:06.683 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297069+0000", 2024-09-18T11:33:06.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5239999999999997e-06 2024-09-18T11:33:06.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.684 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.684 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.684 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313424+0000", 2024-09-18T11:33:06.684 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016355277000000001 2024-09-18T11:33:06.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.684 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.684 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:06.684 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313514+0000", 2024-09-18T11:33:06.684 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0581999999999994e-05 2024-09-18T11:33:06.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.684 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.684 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:06.685 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.314564+0000", 2024-09-18T11:33:06.685 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001049384 2024-09-18T11:33:06.685 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.685 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:06.685 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.685 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.685 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4065 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:06.685 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:33:06.685 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.302147267000002, 2024-09-18T11:33:06.685 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013225997, 2024-09-18T11:33:06.685 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:06.685 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:06.686 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:06.686 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:06.686 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:06.686 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4065 2024-09-18T11:33:06.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.686 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:06.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:06.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:33:06.686 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:06.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:33:06.687 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.687 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:06.687 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296960+0000", 2024-09-18T11:33:06.687 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:06.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.687 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:06.687 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296963+0000", 2024-09-18T11:33:06.687 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2270000000000002e-06 2024-09-18T11:33:06.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.688 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:06.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296970+0000", 2024-09-18T11:33:06.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6220000000000003e-06 2024-09-18T11:33:06.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.688 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:06.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296973+0000", 2024-09-18T11:33:06.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3730000000000001e-06 2024-09-18T11:33:06.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.688 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297097+0000", 2024-09-18T11:33:06.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012352399999999999 2024-09-18T11:33:06.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.689 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:06.689 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297103+0000", 2024-09-18T11:33:06.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7119999999999997e-06 2024-09-18T11:33:06.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.689 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.689 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307831+0000", 2024-09-18T11:33:06.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010728794 2024-09-18T11:33:06.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.690 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:06.690 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307924+0000", 2024-09-18T11:33:06.690 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2099e-05 2024-09-18T11:33:06.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.690 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.690 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:06.690 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:33:06.690 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022630469999999998 2024-09-18T11:33:06.690 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.690 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:06.690 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.691 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4066 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:06.691 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:33:06.691 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.302130192, 2024-09-18T11:33:06.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015656389999999999, 2024-09-18T11:33:06.691 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:06.691 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:06.691 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:06.691 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:06.691 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:06.691 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4066 2024-09-18T11:33:06.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.691 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:06.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:06.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:33:06.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:06.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:33:06.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:06.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296977+0000", 2024-09-18T11:33:06.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:06.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.693 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:06.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296980+0000", 2024-09-18T11:33:06.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8830000000000002e-06 2024-09-18T11:33:06.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.693 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:06.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296985+0000", 2024-09-18T11:33:06.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:33:06.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.694 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:06.694 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296988+0000", 2024-09-18T11:33:06.694 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0419999999999998e-06 2024-09-18T11:33:06.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.694 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.694 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297120+0000", 2024-09-18T11:33:06.694 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131693 2024-09-18T11:33:06.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.694 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:06.694 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297125+0000", 2024-09-18T11:33:06.694 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1100000000000002e-06 2024-09-18T11:33:06.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310212+0000", 2024-09-18T11:33:06.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013087395999999999 2024-09-18T11:33:06.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:06.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310292+0000", 2024-09-18T11:33:06.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9980000000000003e-05 2024-09-18T11:33:06.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:06.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312634+0000", 2024-09-18T11:33:06.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002341756 2024-09-18T11:33:06.696 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.696 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:06.696 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.696 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.696 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4068 3.1 3:87e79ed0:::3526641c07d3efa989be9684355570f39c331008b8a02c9fa22b38a94b6c6b9f:head [write 0~1015808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:06.696 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:33:06.696 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 54.298353974999998, 2024-09-18T11:33:06.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016392648999999999, 2024-09-18T11:33:06.696 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:06.696 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:06.696 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:06.697 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:06.697 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:06.697 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4068 2024-09-18T11:33:06.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.697 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:06.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.697 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:06.697 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:33:06.697 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.697 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:06.698 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:33:06.698 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.698 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.698 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:06.698 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300753+0000", 2024-09-18T11:33:06.698 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:06.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.698 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.698 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:06.698 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301922+0000", 2024-09-18T11:33:06.698 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001169432 2024-09-18T11:33:06.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.698 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.699 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:06.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301936+0000", 2024-09-18T11:33:06.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3574000000000001e-05 2024-09-18T11:33:06.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.699 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.699 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:06.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301941+0000", 2024-09-18T11:33:06.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6069999999999998e-06 2024-09-18T11:33:06.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.699 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.699 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310234+0000", 2024-09-18T11:33:06.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082924820000000003 2024-09-18T11:33:06.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.700 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:06.700 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310337+0000", 2024-09-18T11:33:06.700 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102673 2024-09-18T11:33:06.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.700 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:06.700 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310469+0000", 2024-09-18T11:33:06.700 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131982 2024-09-18T11:33:06.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.701 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:06.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312844+0000", 2024-09-18T11:33:06.701 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023758680000000002 2024-09-18T11:33:06.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.701 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:06.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317092+0000", 2024-09-18T11:33:06.701 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042477549999999998 2024-09-18T11:33:06.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.701 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:06.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317124+0000", 2024-09-18T11:33:06.701 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1968999999999998e-05 2024-09-18T11:33:06.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.702 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.702 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:06.702 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317147+0000", 2024-09-18T11:33:06.702 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2339000000000001e-05 2024-09-18T11:33:06.702 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.702 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:06.702 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.702 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.702 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4671 4.1 4:87078dfe:::smithi11939506-14:head [rollback 6f] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:33:06.702 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:33:06.702 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.280391600000002, 2024-09-18T11:33:06.702 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013540905000000001, 2024-09-18T11:33:06.703 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:06.703 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:06.703 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:06.703 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:06.703 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:06.703 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4671 2024-09-18T11:33:06.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.703 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:06.703 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.703 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:06.703 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:33:06.703 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.703 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.704 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:06.704 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:33:06.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.704 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.704 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:06.704 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:33:06.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:06.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.704 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.704 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:06.704 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318718+0000", 2024-09-18T11:33:06.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3300000000000001e-06 2024-09-18T11:33:06.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.705 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:06.705 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318723+0000", 2024-09-18T11:33:06.705 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1590000000000001e-06 2024-09-18T11:33:06.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.705 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:06.705 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318727+0000", 2024-09-18T11:33:06.705 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3529999999999999e-06 2024-09-18T11:33:06.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.705 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.705 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.320735+0000", 2024-09-18T11:33:06.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020079730000000001 2024-09-18T11:33:06.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:06.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.320843+0000", 2024-09-18T11:33:06.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000108488 2024-09-18T11:33:06.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:06.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.329522+0000", 2024-09-18T11:33:06.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086787979999999997 2024-09-18T11:33:06.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:06.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331769+0000", 2024-09-18T11:33:06.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022471269999999998 2024-09-18T11:33:06.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:06.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332189+0000", 2024-09-18T11:33:06.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00041940400000000001 2024-09-18T11:33:06.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:06.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332228+0000", 2024-09-18T11:33:06.708 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9298000000000002e-05 2024-09-18T11:33:06.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.708 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.708 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:06.708 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332257+0000", 2024-09-18T11:33:06.708 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.934e-05 2024-09-18T11:33:06.708 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.708 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:06.708 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.708 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.708 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4672 4.1 4:87078dfe:::smithi11939506-14:head [append 0~0] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:33:06.709 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.318763+0000", 2024-09-18T11:33:06.709 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 21.280345196999999, 2024-09-18T11:33:06.709 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016099992, 2024-09-18T11:33:06.709 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:06.709 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:06.709 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:06.709 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:06.709 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:06.709 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4672 2024-09-18T11:33:06.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.709 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:06.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.709 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:06.709 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318763+0000", 2024-09-18T11:33:06.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.710 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:06.710 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318763+0000", 2024-09-18T11:33:06.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:06.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.710 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:06.710 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318762+0000", 2024-09-18T11:33:06.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:06.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:06.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318765+0000", 2024-09-18T11:33:06.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.43e-06 2024-09-18T11:33:06.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:06.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318770+0000", 2024-09-18T11:33:06.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1479999999999997e-06 2024-09-18T11:33:06.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:06.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318773+0000", 2024-09-18T11:33:06.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.18e-06 2024-09-18T11:33:06.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.712 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.321043+0000", 2024-09-18T11:33:06.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022696819999999999 2024-09-18T11:33:06.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.712 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for blocked object", 2024-09-18T11:33:06.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.321104+0000", 2024-09-18T11:33:06.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1050999999999995e-05 2024-09-18T11:33:06.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.712 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:06.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331070+0000", 2024-09-18T11:33:06.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0099659610000000006 2024-09-18T11:33:06.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:06.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331176+0000", 2024-09-18T11:33:06.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010582 2024-09-18T11:33:06.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:06.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331377+0000", 2024-09-18T11:33:06.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000201208 2024-09-18T11:33:06.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:06.714 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332278+0000", 2024-09-18T11:33:06.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00090079999999999999 2024-09-18T11:33:06.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.714 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:06.714 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.334798+0000", 2024-09-18T11:33:06.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025200190000000001 2024-09-18T11:33:06.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.714 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:06.714 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.334833+0000", 2024-09-18T11:33:06.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5678e-05 2024-09-18T11:33:06.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:06.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:06.715 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:06.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.334863+0000", 2024-09-18T11:33:06.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9374e-05 2024-09-18T11:33:06.715 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.715 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:06.715 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.715 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:06.715 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:06.715 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:06.716 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:33:06.717 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:33:06.717 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:33:06.718 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:33:06.718 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:33:06.719 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:33:06.715+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:33:06.817 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:06.816+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:33:06.858 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:06.858 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:06.858 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:33:06.858 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:06.865 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:33:06.918 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:33:06.917+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:33:06.963 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:06.963 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:06.963 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:33:06.964 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:33:06.964 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:06.972 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:33:07.019 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:07.018+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:33:07.086 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T11:33:07.112 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:07.112 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:33:07.112 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:33:07.113 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:33:07.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.113 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1788 4.8 4:1898b15d:::smithi11939506-25:head [write 453822~797061] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:33:07.113 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:33:07.113 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 144.19434146399999, 2024-09-18T11:33:07.113 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094305159999999999, 2024-09-18T11:33:07.113 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.113 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.113 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.113 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.113 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.113 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1788 2024-09-18T11:33:07.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.114 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.114 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.114 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:33:07.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.114 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.114 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:33:07.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.114 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915811+0000", 2024-09-18T11:33:07.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:07.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916608+0000", 2024-09-18T11:33:07.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079635000000000003 2024-09-18T11:33:07.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916624+0000", 2024-09-18T11:33:07.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6735000000000002e-05 2024-09-18T11:33:07.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916630+0000", 2024-09-18T11:33:07.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5709999999999997e-06 2024-09-18T11:33:07.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916674+0000", 2024-09-18T11:33:07.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4540999999999999e-05 2024-09-18T11:33:07.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916751+0000", 2024-09-18T11:33:07.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6695999999999999e-05 2024-09-18T11:33:07.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:33:07.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916872+0000", 2024-09-18T11:33:07.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120648 2024-09-18T11:33:07.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.118 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:07.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.919076+0000", 2024-09-18T11:33:07.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022046829999999998 2024-09-18T11:33:07.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.118 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:07.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925151+0000", 2024-09-18T11:33:07.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060740239999999999 2024-09-18T11:33:07.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.118 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925198+0000", 2024-09-18T11:33:07.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7896999999999999e-05 2024-09-18T11:33:07.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.119 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.119 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925244+0000", 2024-09-18T11:33:07.119 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5398999999999997e-05 2024-09-18T11:33:07.119 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.119 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.119 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.120 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3963 4.3 4:c70c3162:::smithi11939506-35:head [tier-promote] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:07.120 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:33:07.120 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 57.734331543000003, 2024-09-18T11:33:07.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010675591999999999, 2024-09-18T11:33:07.120 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.120 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:07.120 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.120 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.120 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.120 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3963 2024-09-18T11:33:07.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.121 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:33:07.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:33:07.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.122 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.122 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375821+0000", 2024-09-18T11:33:07.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:07.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.122 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.122 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375829+0000", 2024-09-18T11:33:07.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7989999999999994e-06 2024-09-18T11:33:07.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.122 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.122 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375843+0000", 2024-09-18T11:33:07.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:33:07.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375849+0000", 2024-09-18T11:33:07.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1419999999999998e-06 2024-09-18T11:33:07.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375889+0000", 2024-09-18T11:33:07.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0197999999999997e-05 2024-09-18T11:33:07.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:33:07.124 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375894+0000", 2024-09-18T11:33:07.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0139999999999998e-06 2024-09-18T11:33:07.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.124 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.124 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.377699+0000", 2024-09-18T11:33:07.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001804688 2024-09-18T11:33:07.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.125 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.125 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.377776+0000", 2024-09-18T11:33:07.125 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7081000000000003e-05 2024-09-18T11:33:07.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.125 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.126 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.386499+0000", 2024-09-18T11:33:07.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008722746 2024-09-18T11:33:07.126 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.126 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.126 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.126 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4259 4.b 4:dba5bd39:::smithi11939506-17:head [write 685818~644423] snapc 69=[69,67,61,55,51,4e,4b,45,42,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:33:07.126 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.040083+0000", 2024-09-18T11:33:07.126 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.070071364, 2024-09-18T11:33:07.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015748635, 2024-09-18T11:33:07.126 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.126 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.127 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.127 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.127 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.127 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4259 2024-09-18T11:33:07.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.127 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.127 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.040083+0000", 2024-09-18T11:33:07.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.040083+0000", 2024-09-18T11:33:07.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.040081+0000", 2024-09-18T11:33:07.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:07.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041258+0000", 2024-09-18T11:33:07.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011766540000000001 2024-09-18T11:33:07.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041286+0000", 2024-09-18T11:33:07.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8087000000000001e-05 2024-09-18T11:33:07.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041362+0000", 2024-09-18T11:33:07.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6155999999999994e-05 2024-09-18T11:33:07.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.044285+0000", 2024-09-18T11:33:07.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002922437 2024-09-18T11:33:07.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.130 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.044360+0000", 2024-09-18T11:33:07.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5201e-05 2024-09-18T11:33:07.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.130 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:07.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.044489+0000", 2024-09-18T11:33:07.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012950599999999999 2024-09-18T11:33:07.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.131 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:07.131 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052904+0000", 2024-09-18T11:33:07.131 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084149910000000001 2024-09-18T11:33:07.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.131 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:07.131 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055695+0000", 2024-09-18T11:33:07.131 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027902040000000001 2024-09-18T11:33:07.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.131 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.131 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055765+0000", 2024-09-18T11:33:07.132 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0720000000000001e-05 2024-09-18T11:33:07.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.132 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.132 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055832+0000", 2024-09-18T11:33:07.132 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6736999999999995e-05 2024-09-18T11:33:07.132 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.132 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.132 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.132 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.5.0:195 4.b e156/152 ondisk, result = 0)", 2024-09-18T11:33:07.133 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.041863+0000", 2024-09-18T11:33:07.133 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.068291844000001, 2024-09-18T11:33:07.133 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010991236, 2024-09-18T11:33:07.133 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.133 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:07.133 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.133 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.133 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.133 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041863+0000", 2024-09-18T11:33:07.133 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.133 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.133 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041863+0000", 2024-09-18T11:33:07.134 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041862+0000", 2024-09-18T11:33:07.134 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:07.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041868+0000", 2024-09-18T11:33:07.134 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2979999999999997e-06 2024-09-18T11:33:07.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.135 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.135 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041879+0000", 2024-09-18T11:33:07.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0522e-05 2024-09-18T11:33:07.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.135 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.135 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041885+0000", 2024-09-18T11:33:07.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7830000000000004e-06 2024-09-18T11:33:07.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.135 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.135 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052841+0000", 2024-09-18T11:33:07.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010956604 2024-09-18T11:33:07.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.136 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.136 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052848+0000", 2024-09-18T11:33:07.136 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2249999999999997e-06 2024-09-18T11:33:07.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.136 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.136 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052854+0000", 2024-09-18T11:33:07.136 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6050000000000003e-06 2024-09-18T11:33:07.136 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.136 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.136 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.137 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.137 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.5.0:196 4.b e156/152 ondisk, result = 0)", 2024-09-18T11:33:07.137 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.042994+0000", 2024-09-18T11:33:07.137 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.067161182, 2024-09-18T11:33:07.137 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098926020000000003, 2024-09-18T11:33:07.137 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.137 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:07.137 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.137 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.137 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.137 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.042994+0000", 2024-09-18T11:33:07.137 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.138 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.042994+0000", 2024-09-18T11:33:07.138 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.138 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.042993+0000", 2024-09-18T11:33:07.138 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:07.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.138 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043000+0000", 2024-09-18T11:33:07.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.525e-06 2024-09-18T11:33:07.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.139 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043011+0000", 2024-09-18T11:33:07.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0335e-05 2024-09-18T11:33:07.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.139 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043015+0000", 2024-09-18T11:33:07.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2899999999999996e-06 2024-09-18T11:33:07.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.140 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.140 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052868+0000", 2024-09-18T11:33:07.140 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098528130000000002 2024-09-18T11:33:07.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.140 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.140 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052871+0000", 2024-09-18T11:33:07.140 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5080000000000001e-06 2024-09-18T11:33:07.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.140 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.140 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052886+0000", 2024-09-18T11:33:07.141 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4781e-05 2024-09-18T11:33:07.141 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.141 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.141 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.141 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.5.0:194 4.3 e156/67 ondisk, result = 0)", 2024-09-18T11:33:07.141 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.043892+0000", 2024-09-18T11:33:07.141 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.066262391999999, 2024-09-18T11:33:07.141 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010237263999999999, 2024-09-18T11:33:07.141 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.141 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:07.141 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.142 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043892+0000", 2024-09-18T11:33:07.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.142 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043892+0000", 2024-09-18T11:33:07.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.142 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043891+0000", 2024-09-18T11:33:07.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:07.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.143 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.143 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043898+0000", 2024-09-18T11:33:07.143 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1190000000000002e-06 2024-09-18T11:33:07.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.143 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.143 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043910+0000", 2024-09-18T11:33:07.143 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.29e-05 2024-09-18T11:33:07.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.143 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.143 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043916+0000", 2024-09-18T11:33:07.144 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9499999999999998e-06 2024-09-18T11:33:07.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.144 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.144 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052830+0000", 2024-09-18T11:33:07.144 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089137480000000009 2024-09-18T11:33:07.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.144 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.144 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052839+0000", 2024-09-18T11:33:07.144 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9560000000000003e-06 2024-09-18T11:33:07.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.145 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.145 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054130+0000", 2024-09-18T11:33:07.145 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012905639999999999 2024-09-18T11:33:07.145 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.145 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.145 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.145 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4260 4.b 4:dba5bd39:::smithi11939506-17:head [write 2069089~216729] snapc 69=[69,67,61,55,51,4e,4b,45,42,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:33:07.145 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045444+0000", 2024-09-18T11:33:07.145 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.064710869999999, 2024-09-18T11:33:07.145 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011166715000000001, 2024-09-18T11:33:07.146 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.146 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.146 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.146 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.146 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.146 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4260 2024-09-18T11:33:07.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.146 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.146 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.146 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.146 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045444+0000", 2024-09-18T11:33:07.146 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.146 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.147 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.147 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045444+0000", 2024-09-18T11:33:07.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.147 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.147 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045443+0000", 2024-09-18T11:33:07.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:07.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.147 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.147 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045649+0000", 2024-09-18T11:33:07.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020605499999999999 2024-09-18T11:33:07.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045662+0000", 2024-09-18T11:33:07.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3386e-05 2024-09-18T11:33:07.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045667+0000", 2024-09-18T11:33:07.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6750000000000001e-06 2024-09-18T11:33:07.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052921+0000", 2024-09-18T11:33:07.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007254613 2024-09-18T11:33:07.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.149 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052994+0000", 2024-09-18T11:33:07.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2721999999999999e-05 2024-09-18T11:33:07.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.149 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:07.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.053116+0000", 2024-09-18T11:33:07.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121972 2024-09-18T11:33:07.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.150 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:07.150 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055857+0000", 2024-09-18T11:33:07.150 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027414539999999999 2024-09-18T11:33:07.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.150 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.150 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:07.150 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056502+0000", 2024-09-18T11:33:07.150 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064412499999999995 2024-09-18T11:33:07.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.150 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.150 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.150 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056555+0000", 2024-09-18T11:33:07.150 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3467999999999997e-05 2024-09-18T11:33:07.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.151 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.151 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.151 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056611+0000", 2024-09-18T11:33:07.151 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5577e-05 2024-09-18T11:33:07.151 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.151 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.151 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.152 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.152 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4261 4.b 4:dba5bd39:::smithi11939506-17:head [setxattr _header (57) in=7b,truncate 2285818] snapc 69=[69,67,61,55,51,4e,4b,45,42,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:33:07.152 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045672+0000", 2024-09-18T11:33:07.152 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.064482771999998, 2024-09-18T11:33:07.152 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011289933, 2024-09-18T11:33:07.152 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.152 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.152 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.152 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.152 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.152 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4261 2024-09-18T11:33:07.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.152 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.153 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045672+0000", 2024-09-18T11:33:07.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.153 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045672+0000", 2024-09-18T11:33:07.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.153 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045671+0000", 2024-09-18T11:33:07.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:07.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045675+0000", 2024-09-18T11:33:07.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.563e-06 2024-09-18T11:33:07.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045681+0000", 2024-09-18T11:33:07.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5210000000000002e-06 2024-09-18T11:33:07.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045684+0000", 2024-09-18T11:33:07.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4149999999999999e-06 2024-09-18T11:33:07.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.155 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054143+0000", 2024-09-18T11:33:07.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084590810000000002 2024-09-18T11:33:07.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.155 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054214+0000", 2024-09-18T11:33:07.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1061000000000003e-05 2024-09-18T11:33:07.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.156 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:07.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054354+0000", 2024-09-18T11:33:07.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000140035 2024-09-18T11:33:07.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.156 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:07.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056208+0000", 2024-09-18T11:33:07.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018539229999999999 2024-09-18T11:33:07.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.156 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:07.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056885+0000", 2024-09-18T11:33:07.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00067688000000000004 2024-09-18T11:33:07.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.157 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.157 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056922+0000", 2024-09-18T11:33:07.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6693999999999998e-05 2024-09-18T11:33:07.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.157 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.157 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056962+0000", 2024-09-18T11:33:07.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0299000000000002e-05 2024-09-18T11:33:07.157 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.157 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.157 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.158 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4262 4.b 4:dba5bd39:::smithi11939506-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:33:07.158 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:33:07.158 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.064466629999998, 2024-09-18T11:33:07.158 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011764513000000001, 2024-09-18T11:33:07.158 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.158 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:07.158 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.158 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.158 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.158 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4262 2024-09-18T11:33:07.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.158 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.159 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:33:07.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.159 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:33:07.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.159 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:33:07.160 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:07.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.160 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.160 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045690+0000", 2024-09-18T11:33:07.160 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.745e-06 2024-09-18T11:33:07.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.160 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.160 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045697+0000", 2024-09-18T11:33:07.160 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7229999999999998e-06 2024-09-18T11:33:07.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045704+0000", 2024-09-18T11:33:07.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5719999999999999e-06 2024-09-18T11:33:07.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055555+0000", 2024-09-18T11:33:07.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098515430000000008 2024-09-18T11:33:07.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:07.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055675+0000", 2024-09-18T11:33:07.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012000799999999999 2024-09-18T11:33:07.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.057004+0000", 2024-09-18T11:33:07.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013286179999999999 2024-09-18T11:33:07.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.057357+0000", 2024-09-18T11:33:07.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00035355100000000002 2024-09-18T11:33:07.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.057453+0000", 2024-09-18T11:33:07.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5154000000000004e-05 2024-09-18T11:33:07.163 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.163 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.163 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.163 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4264 4.7 4:e0d95e10:::smithi11939506-22:head [read 0~3008486,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:33:07.163 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:33:07.163 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.064416643000001, 2024-09-18T11:33:07.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095333889999999998, 2024-09-18T11:33:07.163 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.163 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:07.163 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.163 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4264 2024-09-18T11:33:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:33:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:33:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:33:07.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:07.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045747+0000", 2024-09-18T11:33:07.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2469999999999997e-06 2024-09-18T11:33:07.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045755+0000", 2024-09-18T11:33:07.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1499999999999999e-06 2024-09-18T11:33:07.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045759+0000", 2024-09-18T11:33:07.166 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0849999999999999e-06 2024-09-18T11:33:07.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045803+0000", 2024-09-18T11:33:07.166 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3621000000000001e-05 2024-09-18T11:33:07.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:33:07.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045807+0000", 2024-09-18T11:33:07.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7290000000000002e-06 2024-09-18T11:33:07.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.053475+0000", 2024-09-18T11:33:07.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076680100000000003 2024-09-18T11:33:07.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.053644+0000", 2024-09-18T11:33:07.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016971299999999999 2024-09-18T11:33:07.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055272+0000", 2024-09-18T11:33:07.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001627245 2024-09-18T11:33:07.168 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.168 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.168 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.168 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4265 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~2163506] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:33:07.168 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045792+0000", 2024-09-18T11:33:07.168 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 40.064362752999997, 2024-09-18T11:33:07.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010085024, 2024-09-18T11:33:07.168 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.168 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:07.169 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.169 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.169 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.169 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4265 2024-09-18T11:33:07.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.169 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.169 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045792+0000", 2024-09-18T11:33:07.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.169 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045792+0000", 2024-09-18T11:33:07.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045791+0000", 2024-09-18T11:33:07.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:07.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045797+0000", 2024-09-18T11:33:07.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0209999999999996e-06 2024-09-18T11:33:07.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045805+0000", 2024-09-18T11:33:07.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9349999999999994e-06 2024-09-18T11:33:07.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045811+0000", 2024-09-18T11:33:07.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2240000000000003e-06 2024-09-18T11:33:07.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045858+0000", 2024-09-18T11:33:07.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7565999999999998e-05 2024-09-18T11:33:07.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.172 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:33:07.172 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045869+0000", 2024-09-18T11:33:07.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0852e-05 2024-09-18T11:33:07.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.172 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055337+0000", 2024-09-18T11:33:07.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094678729999999999 2024-09-18T11:33:07.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.173 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055417+0000", 2024-09-18T11:33:07.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0092000000000005e-05 2024-09-18T11:33:07.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.173 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055877+0000", 2024-09-18T11:33:07.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00046001800000000003 2024-09-18T11:33:07.174 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.174 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.174 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.174 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4349 4.b 4:d13decd8:::smithi11939506-2:head [read 0~2302625,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:33:07.174 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.382910+0000", 2024-09-18T11:33:07.174 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.727244689000003, 2024-09-18T11:33:07.174 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098226930000000004, 2024-09-18T11:33:07.174 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.174 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:07.175 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.175 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.175 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.175 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4349 2024-09-18T11:33:07.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.175 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.175 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.175 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382910+0000", 2024-09-18T11:33:07.175 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.175 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.175 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382910+0000", 2024-09-18T11:33:07.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382909+0000", 2024-09-18T11:33:07.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:07.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382915+0000", 2024-09-18T11:33:07.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6049999999999997e-06 2024-09-18T11:33:07.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382924+0000", 2024-09-18T11:33:07.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6289999999999999e-06 2024-09-18T11:33:07.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382929+0000", 2024-09-18T11:33:07.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0479999999999998e-06 2024-09-18T11:33:07.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383030+0000", 2024-09-18T11:33:07.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101346 2024-09-18T11:33:07.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:07.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383036+0000", 2024-09-18T11:33:07.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2720000000000003e-06 2024-09-18T11:33:07.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.390597+0000", 2024-09-18T11:33:07.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075606880000000003 2024-09-18T11:33:07.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.179 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.390696+0000", 2024-09-18T11:33:07.179 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8740000000000007e-05 2024-09-18T11:33:07.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.179 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.179 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.179 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392733+0000", 2024-09-18T11:33:07.179 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002037025 2024-09-18T11:33:07.179 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.179 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.179 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.179 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.179 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4350 4.b 4:d13decd8:::smithi11939506-2:head [sparse-read 0~1463453] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:33:07.180 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.382978+0000", 2024-09-18T11:33:07.180 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.727177126000001, 2024-09-18T11:33:07.180 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01020132, 2024-09-18T11:33:07.180 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.180 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:07.180 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.180 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.180 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.180 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4350 2024-09-18T11:33:07.180 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.180 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.180 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382978+0000", 2024-09-18T11:33:07.181 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.181 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382978+0000", 2024-09-18T11:33:07.181 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.181 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382976+0000", 2024-09-18T11:33:07.181 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:07.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.182 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.182 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.182 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382984+0000", 2024-09-18T11:33:07.182 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.328e-06 2024-09-18T11:33:07.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.182 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.182 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.182 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382994+0000", 2024-09-18T11:33:07.182 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0801999999999999e-05 2024-09-18T11:33:07.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.182 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.182 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.182 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383001+0000", 2024-09-18T11:33:07.183 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6390000000000003e-06 2024-09-18T11:33:07.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.183 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.183 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.183 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383138+0000", 2024-09-18T11:33:07.183 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013676899999999999 2024-09-18T11:33:07.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.183 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.183 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:07.183 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383144+0000", 2024-09-18T11:33:07.183 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6359999999999997e-06 2024-09-18T11:33:07.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.183 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.183 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.184 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392791+0000", 2024-09-18T11:33:07.184 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096461659999999994 2024-09-18T11:33:07.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.184 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.184 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.184 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392897+0000", 2024-09-18T11:33:07.184 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000106087 2024-09-18T11:33:07.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.184 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.184 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.184 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.393179+0000", 2024-09-18T11:33:07.184 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00028227500000000002 2024-09-18T11:33:07.184 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.185 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.185 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.185 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.185 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4375 4.b 4:dba5bd39:::smithi11939506-17:head [write 437081~638017,stat] snapc 6d=[6d,6c,69,67,61,55,51,4b,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:33:07.185 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.549471+0000", 2024-09-18T11:33:07.185 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 35.560683423999997, 2024-09-18T11:33:07.185 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097774179999999995, 2024-09-18T11:33:07.185 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.185 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.185 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.185 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.185 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.185 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4375 2024-09-18T11:33:07.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.186 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.186 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.549471+0000", 2024-09-18T11:33:07.186 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.186 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.549471+0000", 2024-09-18T11:33:07.186 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.186 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.549470+0000", 2024-09-18T11:33:07.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:07.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553463+0000", 2024-09-18T11:33:07.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039932520000000001 2024-09-18T11:33:07.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553478+0000", 2024-09-18T11:33:07.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4647999999999999e-05 2024-09-18T11:33:07.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.188 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553484+0000", 2024-09-18T11:33:07.188 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3799999999999999e-06 2024-09-18T11:33:07.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.188 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.188 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553536+0000", 2024-09-18T11:33:07.188 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1938999999999999e-05 2024-09-18T11:33:07.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.188 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.188 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553637+0000", 2024-09-18T11:33:07.188 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010074899999999999 2024-09-18T11:33:07.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.189 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.189 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:07.189 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553824+0000", 2024-09-18T11:33:07.189 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000187524 2024-09-18T11:33:07.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.189 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.189 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:07.189 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.556653+0000", 2024-09-18T11:33:07.189 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028285879999999999 2024-09-18T11:33:07.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.189 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.189 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:07.190 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.559176+0000", 2024-09-18T11:33:07.190 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025229369999999998 2024-09-18T11:33:07.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.190 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.190 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.190 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.559212+0000", 2024-09-18T11:33:07.190 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5750000000000002e-05 2024-09-18T11:33:07.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.190 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.190 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.190 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.559249+0000", 2024-09-18T11:33:07.190 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7020999999999997e-05 2024-09-18T11:33:07.191 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.191 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.191 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.191 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.191 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4430 4.b 4:d3ae8215:::smithi11939506-41:head [read 0~3832029,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T11:33:07.191 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:33:07.191 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.081644463000003, 2024-09-18T11:33:07.191 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010918034, 2024-09-18T11:33:07.191 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.191 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:07.191 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.191 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.191 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.192 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4430 2024-09-18T11:33:07.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.192 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.192 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.192 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.192 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:33:07.192 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.192 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.192 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.192 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:33:07.192 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.193 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.193 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.193 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:33:07.193 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:07.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.193 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.193 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.193 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028520+0000", 2024-09-18T11:33:07.193 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.005e-05 2024-09-18T11:33:07.194 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.194 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.194 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.194 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028538+0000", 2024-09-18T11:33:07.194 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8020999999999999e-05 2024-09-18T11:33:07.196 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.196 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.196 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.196 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028543+0000", 2024-09-18T11:33:07.196 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.485e-06 2024-09-18T11:33:07.196 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.196 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.196 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.196 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028634+0000", 2024-09-18T11:33:07.196 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1120999999999997e-05 2024-09-18T11:33:07.197 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.197 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.197 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:07.197 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028639+0000", 2024-09-18T11:33:07.197 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1989999999999997e-06 2024-09-18T11:33:07.197 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.197 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.197 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.197 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.036054+0000", 2024-09-18T11:33:07.197 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074147559999999998 2024-09-18T11:33:07.197 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.197 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.197 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.197 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.036158+0000", 2024-09-18T11:33:07.198 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103564 2024-09-18T11:33:07.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.198 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.198 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.198 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039428+0000", 2024-09-18T11:33:07.198 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032706390000000001 2024-09-18T11:33:07.198 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.198 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.198 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.198 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.198 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4431 4.b 4:d3ae8215:::smithi11939506-41:head [sparse-read 0~2481209] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T11:33:07.199 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:33:07.199 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.081605674999999, 2024-09-18T11:33:07.199 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011399179000000001, 2024-09-18T11:33:07.199 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.199 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:07.199 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.199 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.199 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.199 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4431 2024-09-18T11:33:07.199 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.199 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.199 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.199 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.199 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:33:07.200 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.200 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.200 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.200 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.200 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:33:07.200 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.200 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.200 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.200 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.200 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:33:07.200 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:07.200 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.200 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.201 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.201 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028552+0000", 2024-09-18T11:33:07.201 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5880000000000002e-06 2024-09-18T11:33:07.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.201 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.201 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.201 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028558+0000", 2024-09-18T11:33:07.201 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.84e-06 2024-09-18T11:33:07.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.201 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.201 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.201 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028562+0000", 2024-09-18T11:33:07.201 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6600000000000001e-06 2024-09-18T11:33:07.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.202 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.202 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.202 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028676+0000", 2024-09-18T11:33:07.202 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011408000000000001 2024-09-18T11:33:07.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.202 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.202 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:07.202 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028681+0000", 2024-09-18T11:33:07.202 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5350000000000004e-06 2024-09-18T11:33:07.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.202 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.202 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.202 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039458+0000", 2024-09-18T11:33:07.203 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010777231 2024-09-18T11:33:07.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.203 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.203 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.203 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039564+0000", 2024-09-18T11:33:07.203 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000105524 2024-09-18T11:33:07.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.203 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.203 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.203 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039948+0000", 2024-09-18T11:33:07.203 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00038430099999999998 2024-09-18T11:33:07.203 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.203 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.203 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.204 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.204 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4518 4.7 4:e0d95e10:::smithi11939506-22:head [rollback 6f] snapc 74=[74,73,71,6f,6c,69,67,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:33:07.204 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.055692+0000", 2024-09-18T11:33:07.204 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.054462414, 2024-09-18T11:33:07.204 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010093540999999999, 2024-09-18T11:33:07.204 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.204 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.204 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.204 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.204 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.204 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4518 2024-09-18T11:33:07.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.205 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.205 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.205 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.205 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055692+0000", 2024-09-18T11:33:07.205 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.205 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.205 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.205 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.205 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055692+0000", 2024-09-18T11:33:07.205 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.205 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.205 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.205 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.205 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055692+0000", 2024-09-18T11:33:07.206 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:07.206 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.206 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.206 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.206 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055697+0000", 2024-09-18T11:33:07.206 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.101e-06 2024-09-18T11:33:07.206 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.206 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.206 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.206 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055707+0000", 2024-09-18T11:33:07.206 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0316000000000001e-05 2024-09-18T11:33:07.206 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.206 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.207 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.207 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055713+0000", 2024-09-18T11:33:07.207 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8900000000000004e-06 2024-09-18T11:33:07.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.207 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.207 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.207 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057453+0000", 2024-09-18T11:33:07.207 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001739953 2024-09-18T11:33:07.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.207 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.207 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.207 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057537+0000", 2024-09-18T11:33:07.207 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4287000000000001e-05 2024-09-18T11:33:07.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.208 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.208 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:33:07.208 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057760+0000", 2024-09-18T11:33:07.208 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022294700000000001 2024-09-18T11:33:07.208 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.208 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.208 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:07.208 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065674+0000", 2024-09-18T11:33:07.208 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079140110000000003 2024-09-18T11:33:07.208 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.208 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.208 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:07.209 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065681+0000", 2024-09-18T11:33:07.209 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2289999999999997e-06 2024-09-18T11:33:07.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.209 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.209 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.209 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065734+0000", 2024-09-18T11:33:07.209 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3136000000000001e-05 2024-09-18T11:33:07.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.209 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.209 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.209 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065786+0000", 2024-09-18T11:33:07.209 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1419999999999999e-05 2024-09-18T11:33:07.209 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.210 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.210 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.210 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.210 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4519 4.7 4:e0d95e10:::smithi11939506-22:head [append 0~0] snapc 74=[74,73,71,6f,6c,69,67,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:33:07.210 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.055719+0000", 2024-09-18T11:33:07.210 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.054436251999999, 2024-09-18T11:33:07.210 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010210589000000001, 2024-09-18T11:33:07.210 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.210 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.210 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.210 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.210 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.211 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4519 2024-09-18T11:33:07.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.211 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.211 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.211 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.211 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055719+0000", 2024-09-18T11:33:07.211 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.211 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.211 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.211 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055719+0000", 2024-09-18T11:33:07.211 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.211 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.212 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.212 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055718+0000", 2024-09-18T11:33:07.212 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:07.212 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.212 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.212 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.212 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055721+0000", 2024-09-18T11:33:07.212 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7039999999999999e-06 2024-09-18T11:33:07.212 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.212 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.212 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.212 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055728+0000", 2024-09-18T11:33:07.212 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1620000000000002e-06 2024-09-18T11:33:07.213 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.213 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.213 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.213 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055732+0000", 2024-09-18T11:33:07.213 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8410000000000002e-06 2024-09-18T11:33:07.213 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.213 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.213 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.213 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.059646+0000", 2024-09-18T11:33:07.213 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039138489999999996 2024-09-18T11:33:07.213 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.213 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.213 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.214 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.059733+0000", 2024-09-18T11:33:07.214 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7163999999999994e-05 2024-09-18T11:33:07.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.214 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.214 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:33:07.214 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.059891+0000", 2024-09-18T11:33:07.214 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000158102 2024-09-18T11:33:07.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.214 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.214 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:07.214 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065798+0000", 2024-09-18T11:33:07.214 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059067649999999996 2024-09-18T11:33:07.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.215 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.215 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:07.215 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065840+0000", 2024-09-18T11:33:07.215 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2515999999999997e-05 2024-09-18T11:33:07.215 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.215 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.215 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.215 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065890+0000", 2024-09-18T11:33:07.215 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0055000000000001e-05 2024-09-18T11:33:07.215 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.215 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.215 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.215 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065929+0000", 2024-09-18T11:33:07.216 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8831000000000001e-05 2024-09-18T11:33:07.216 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.216 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.216 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.216 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.216 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.216 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4756 4.7 4:e301aefd:::smithi11939506-19:head [write 591712~786646,stat] snapc 7e=[7e,7c,7b,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e179)", 2024-09-18T11:33:07.216 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:50.036231+0000", 2024-09-18T11:33:07.216 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 17.073923608000001, 2024-09-18T11:33:07.216 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010177099, 2024-09-18T11:33:07.216 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.216 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.216 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.216 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.217 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.217 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4756 2024-09-18T11:33:07.217 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.217 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.217 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.217 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.217 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.036231+0000", 2024-09-18T11:33:07.217 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.217 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.217 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.217 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.217 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.036231+0000", 2024-09-18T11:33:07.217 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.218 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.218 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.218 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.218 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.036229+0000", 2024-09-18T11:33:07.218 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:07.218 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.218 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.218 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.218 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.037165+0000", 2024-09-18T11:33:07.218 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00093574399999999996 2024-09-18T11:33:07.218 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.218 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.218 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.219 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.037183+0000", 2024-09-18T11:33:07.219 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7431999999999998e-05 2024-09-18T11:33:07.219 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.219 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.219 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.219 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.037189+0000", 2024-09-18T11:33:07.219 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4590000000000003e-06 2024-09-18T11:33:07.219 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.219 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.219 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.219 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.037243+0000", 2024-09-18T11:33:07.219 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4237999999999998e-05 2024-09-18T11:33:07.219 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.220 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.220 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:33:07.220 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.037249+0000", 2024-09-18T11:33:07.220 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.699e-06 2024-09-18T11:33:07.220 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.220 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.220 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.220 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.038725+0000", 2024-09-18T11:33:07.220 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014756420000000001 2024-09-18T11:33:07.220 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.220 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.220 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.220 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.038797+0000", 2024-09-18T11:33:07.220 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2446000000000004e-05 2024-09-18T11:33:07.221 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.221 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.221 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:07.221 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.038915+0000", 2024-09-18T11:33:07.221 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011769000000000001 2024-09-18T11:33:07.221 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.221 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.221 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:07.221 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.041337+0000", 2024-09-18T11:33:07.221 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024226339999999999 2024-09-18T11:33:07.221 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.221 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.221 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:07.222 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.046311+0000", 2024-09-18T11:33:07.222 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049738359999999997 2024-09-18T11:33:07.222 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.222 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.222 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.222 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.046359+0000", 2024-09-18T11:33:07.222 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7586999999999997e-05 2024-09-18T11:33:07.222 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.222 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.222 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.222 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.046408+0000", 2024-09-18T11:33:07.222 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9478000000000003e-05 2024-09-18T11:33:07.222 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.223 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.223 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.223 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.223 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.223 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:4790 3.2 e182/21 3:79129b11:::2f02c59c01a730562a65f21445c8af0e4a8aab3b969b8a3abf39e7b1bc84f918:head v 182'203, mlcod=176'201)", 2024-09-18T11:33:07.223 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:53.043420+0000", 2024-09-18T11:33:07.223 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.066734764, 2024-09-18T11:33:07.223 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096982140000000001, 2024-09-18T11:33:07.223 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.223 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.223 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.223 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.223 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.223 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.043420+0000", 2024-09-18T11:33:07.223 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.224 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.224 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.224 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.224 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.043420+0000", 2024-09-18T11:33:07.224 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.224 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.224 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.224 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.224 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.043420+0000", 2024-09-18T11:33:07.224 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:07.224 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.224 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.224 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.224 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.046026+0000", 2024-09-18T11:33:07.224 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026064590000000002 2024-09-18T11:33:07.225 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.225 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.225 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.225 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.046040+0000", 2024-09-18T11:33:07.225 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.364e-05 2024-09-18T11:33:07.225 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.225 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.225 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.225 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.046045+0000", 2024-09-18T11:33:07.225 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6010000000000002e-06 2024-09-18T11:33:07.226 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.226 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.226 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.226 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.046080+0000", 2024-09-18T11:33:07.226 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4702000000000001e-05 2024-09-18T11:33:07.226 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.226 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.226 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.226 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.046095+0000", 2024-09-18T11:33:07.226 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5265e-05 2024-09-18T11:33:07.226 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.226 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.226 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.226 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.053085+0000", 2024-09-18T11:33:07.227 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0069894559999999998 2024-09-18T11:33:07.227 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.227 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.227 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.227 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.053118+0000", 2024-09-18T11:33:07.227 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3565999999999997e-05 2024-09-18T11:33:07.227 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.227 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.227 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.227 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.227 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.227 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:07.229 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:33:07.229 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:07.227+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:33:07.327 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:07.327 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:07.327 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:33:07.327 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:07.329 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:33:07.328+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:33:07.335 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:33:07.430 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:07.429+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:07.474 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:07.474 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:07.474 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:33:07.474 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:33:07.474 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:07.483 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:33:07.568 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:07.567+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:07.624 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:07.624 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:33:07.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:33:07.624 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:33:07.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.624 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:33:07.624 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:33:07.624 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 176.50535570700001, 2024-09-18T11:33:07.625 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:33:07.625 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.625 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:07.625 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.625 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.625 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.625 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:33:07.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.625 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.625 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.625 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:33:07.625 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:33:07.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:33:07.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:07.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:33:07.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:33:07.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:33:07.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:33:07.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:33:07.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:33:07.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:33:07.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:33:07.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.628 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:33:07.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:33:07.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:33:07.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.628 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:33:07.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:33:07.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:33:07.629 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:33:07.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:33:07.629 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:33:07.629 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.629 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.629 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.630 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:33:07.630 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:33:07.630 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 166.95233030599999, 2024-09-18T11:33:07.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:33:07.630 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.630 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:07.630 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.630 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.630 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.630 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:33:07.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.630 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:33:07.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:33:07.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.631 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:33:07.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:07.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.632 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:33:07.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:33:07.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.632 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:33:07.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:33:07.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.632 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:33:07.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:33:07.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:33:07.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:33:07.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:33:07.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:33:07.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:33:07.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:33:07.634 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.634 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.634 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.634 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2339 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:33:07.634 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:33:07.634 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 126.234873014, 2024-09-18T11:33:07.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011809992, 2024-09-18T11:33:07.634 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.634 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:07.635 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.635 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.635 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.635 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2339 2024-09-18T11:33:07.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.635 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.635 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:33:07.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.635 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:33:07.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.636 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:33:07.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:07.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.636 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.636 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:33:07.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4710000000000001e-06 2024-09-18T11:33:07.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386824+0000", 2024-09-18T11:33:07.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6104e-05 2024-09-18T11:33:07.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386836+0000", 2024-09-18T11:33:07.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1827999999999999e-05 2024-09-18T11:33:07.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388390+0000", 2024-09-18T11:33:07.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015539499999999999 2024-09-18T11:33:07.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:07.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388443+0000", 2024-09-18T11:33:07.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3785000000000001e-05 2024-09-18T11:33:07.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398238+0000", 2024-09-18T11:33:07.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097947989999999999 2024-09-18T11:33:07.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398315+0000", 2024-09-18T11:33:07.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6909999999999994e-05 2024-09-18T11:33:07.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.639 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398612+0000", 2024-09-18T11:33:07.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029671999999999999 2024-09-18T11:33:07.639 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.639 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.639 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.640 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2340 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:33:07.640 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:33:07.640 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 126.234829702, 2024-09-18T11:33:07.640 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012086448, 2024-09-18T11:33:07.640 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.640 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:07.640 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.640 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.640 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.640 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2340 2024-09-18T11:33:07.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.640 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.641 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.641 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:33:07.641 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.641 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.641 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.641 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:33:07.641 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.641 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.641 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.641 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:33:07.641 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:07.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386848+0000", 2024-09-18T11:33:07.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3069999999999998e-06 2024-09-18T11:33:07.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386855+0000", 2024-09-18T11:33:07.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0979999999999996e-06 2024-09-18T11:33:07.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386859+0000", 2024-09-18T11:33:07.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9360000000000001e-06 2024-09-18T11:33:07.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388466+0000", 2024-09-18T11:33:07.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001607154 2024-09-18T11:33:07.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:07.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388507+0000", 2024-09-18T11:33:07.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1093000000000001e-05 2024-09-18T11:33:07.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398647+0000", 2024-09-18T11:33:07.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010139686 2024-09-18T11:33:07.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.644 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398718+0000", 2024-09-18T11:33:07.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1138999999999998e-05 2024-09-18T11:33:07.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.644 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398931+0000", 2024-09-18T11:33:07.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021356899999999999 2024-09-18T11:33:07.644 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.644 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.645 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.645 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2341 4.5 4:aa575018:::smithi11939506-11:head [read 0~2487436,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:33:07.645 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:33:07.645 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 126.23481148, 2024-09-18T11:33:07.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013751648999999999, 2024-09-18T11:33:07.645 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.645 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:07.645 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.645 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.645 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.645 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2341 2024-09-18T11:33:07.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.646 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.646 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:33:07.646 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.646 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:33:07.646 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.646 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:33:07.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:07.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386866+0000", 2024-09-18T11:33:07.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4249999999999998e-06 2024-09-18T11:33:07.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386872+0000", 2024-09-18T11:33:07.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.003e-06 2024-09-18T11:33:07.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:33:07.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.507e-06 2024-09-18T11:33:07.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388520+0000", 2024-09-18T11:33:07.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001644146 2024-09-18T11:33:07.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:07.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388561+0000", 2024-09-18T11:33:07.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1486999999999998e-05 2024-09-18T11:33:07.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.649 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398963+0000", 2024-09-18T11:33:07.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010401353 2024-09-18T11:33:07.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.649 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.399046+0000", 2024-09-18T11:33:07.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3141999999999998e-05 2024-09-18T11:33:07.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.650 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.650 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.400615+0000", 2024-09-18T11:33:07.650 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015690960000000001 2024-09-18T11:33:07.650 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.650 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.650 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.650 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3105 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 3e=[3e,3d,3b,37,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-18T11:33:07.650 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:33:07.650 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 94.603463735999995, 2024-09-18T11:33:07.650 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011735906000000001, 2024-09-18T11:33:07.650 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.651 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:07.651 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.651 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.651 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.651 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3105 2024-09-18T11:33:07.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.651 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.651 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.651 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:33:07.651 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.651 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:33:07.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:33:07.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:07.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018212+0000", 2024-09-18T11:33:07.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6759999999999999e-06 2024-09-18T11:33:07.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018216+0000", 2024-09-18T11:33:07.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-06 2024-09-18T11:33:07.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018218+0000", 2024-09-18T11:33:07.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6110000000000001e-06 2024-09-18T11:33:07.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018246+0000", 2024-09-18T11:33:07.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7557000000000001e-05 2024-09-18T11:33:07.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:33:07.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018250+0000", 2024-09-18T11:33:07.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4299999999999999e-06 2024-09-18T11:33:07.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021364+0000", 2024-09-18T11:33:07.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031135389999999998 2024-09-18T11:33:07.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021494+0000", 2024-09-18T11:33:07.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000130333 2024-09-18T11:33:07.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.029947+0000", 2024-09-18T11:33:07.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084526150000000001 2024-09-18T11:33:07.655 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.655 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.655 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.656 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3324 4.5 4:a087a566:::smithi11939506-46:head [rollback 46] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:33:07.656 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:33:07.656 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.504733133000002, 2024-09-18T11:33:07.656 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010927767999999999, 2024-09-18T11:33:07.656 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.656 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.656 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.656 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.656 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.656 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3324 2024-09-18T11:33:07.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.656 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.657 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:33:07.657 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.657 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:33:07.657 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.657 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116941+0000", 2024-09-18T11:33:07.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:07.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116949+0000", 2024-09-18T11:33:07.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4660000000000002e-06 2024-09-18T11:33:07.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116960+0000", 2024-09-18T11:33:07.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1484999999999999e-05 2024-09-18T11:33:07.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116964+0000", 2024-09-18T11:33:07.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2810000000000002e-06 2024-09-18T11:33:07.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117053+0000", 2024-09-18T11:33:07.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8747999999999998e-05 2024-09-18T11:33:07.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:07.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117059+0000", 2024-09-18T11:33:07.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.246e-06 2024-09-18T11:33:07.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123604+0000", 2024-09-18T11:33:07.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065445319999999996 2024-09-18T11:33:07.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123663+0000", 2024-09-18T11:33:07.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8740999999999999e-05 2024-09-18T11:33:07.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:33:07.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123803+0000", 2024-09-18T11:33:07.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014049200000000001 2024-09-18T11:33:07.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:07.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127600+0000", 2024-09-18T11:33:07.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037971179999999999 2024-09-18T11:33:07.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:07.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127814+0000", 2024-09-18T11:33:07.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021333600000000001 2024-09-18T11:33:07.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.661 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127847+0000", 2024-09-18T11:33:07.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3188999999999997e-05 2024-09-18T11:33:07.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.661 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.662 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127869+0000", 2024-09-18T11:33:07.662 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2515e-05 2024-09-18T11:33:07.662 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.662 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.662 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.662 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3325 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:33:07.662 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:33:07.662 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 85.504685946999999, 2024-09-18T11:33:07.662 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181030999999999, 2024-09-18T11:33:07.662 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.663 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.663 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.663 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.663 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.663 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3325 2024-09-18T11:33:07.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.663 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.663 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.663 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.663 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:33:07.663 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.663 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.663 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:33:07.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116988+0000", 2024-09-18T11:33:07.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:07.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116991+0000", 2024-09-18T11:33:07.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9890000000000001e-06 2024-09-18T11:33:07.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.665 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116998+0000", 2024-09-18T11:33:07.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5309999999999998e-06 2024-09-18T11:33:07.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.665 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117005+0000", 2024-09-18T11:33:07.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6420000000000001e-06 2024-09-18T11:33:07.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.665 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117091+0000", 2024-09-18T11:33:07.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6835999999999993e-05 2024-09-18T11:33:07.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:07.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117097+0000", 2024-09-18T11:33:07.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:33:07.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124717+0000", 2024-09-18T11:33:07.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076204899999999997 2024-09-18T11:33:07.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.667 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.667 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124773+0000", 2024-09-18T11:33:07.667 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5470999999999998e-05 2024-09-18T11:33:07.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.667 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.667 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:33:07.667 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124877+0000", 2024-09-18T11:33:07.667 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010426500000000001 2024-09-18T11:33:07.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.667 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.667 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:07.667 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127901+0000", 2024-09-18T11:33:07.667 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030243449999999999 2024-09-18T11:33:07.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.668 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:07.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128115+0000", 2024-09-18T11:33:07.668 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021355300000000001 2024-09-18T11:33:07.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.668 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128145+0000", 2024-09-18T11:33:07.668 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0290999999999998e-05 2024-09-18T11:33:07.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.669 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.669 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128170+0000", 2024-09-18T11:33:07.669 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4811e-05 2024-09-18T11:33:07.669 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.669 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.669 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.669 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.669 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3474 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1872245~637761] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:33:07.669 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:33:07.669 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 81.177909270000001, 2024-09-18T11:33:07.669 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0092038580000000005, 2024-09-18T11:33:07.669 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.670 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.670 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.670 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.670 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.670 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3474 2024-09-18T11:33:07.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.670 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.670 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:33:07.670 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.671 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:33:07.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.671 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443764+0000", 2024-09-18T11:33:07.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:07.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.671 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444359+0000", 2024-09-18T11:33:07.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059519 2024-09-18T11:33:07.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444374+0000", 2024-09-18T11:33:07.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4949e-05 2024-09-18T11:33:07.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444382+0000", 2024-09-18T11:33:07.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1369999999999996e-06 2024-09-18T11:33:07.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.673 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.673 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444428+0000", 2024-09-18T11:33:07.673 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6783000000000003e-05 2024-09-18T11:33:07.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.673 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.673 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.673 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444507+0000", 2024-09-18T11:33:07.673 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8415000000000001e-05 2024-09-18T11:33:07.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.673 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.673 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:33:07.673 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444638+0000", 2024-09-18T11:33:07.673 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013129 2024-09-18T11:33:07.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.674 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:07.674 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.449059+0000", 2024-09-18T11:33:07.674 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004420719 2024-09-18T11:33:07.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.674 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:07.674 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452909+0000", 2024-09-18T11:33:07.674 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003850773 2024-09-18T11:33:07.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.674 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452944+0000", 2024-09-18T11:33:07.675 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-05 2024-09-18T11:33:07.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.675 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.675 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452969+0000", 2024-09-18T11:33:07.675 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5795999999999998e-05 2024-09-18T11:33:07.675 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.675 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.675 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.675 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4251 4.5 4:af4ba75a:::smithi11939506-37:head [delete] snapc 69=[69,67,61,55,51,4e,4c,4b,45,42,3e,3d] ondisk+write+known_if_redirected+full_force+supports_pool_eio e155)", 2024-09-18T11:33:07.676 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:26.039209+0000", 2024-09-18T11:33:07.676 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 41.582465249999998, 2024-09-18T11:33:07.676 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010258422, 2024-09-18T11:33:07.676 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.676 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.676 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.676 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.676 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.676 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4251 2024-09-18T11:33:07.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.676 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.676 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.677 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039209+0000", 2024-09-18T11:33:07.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.677 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.677 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039209+0000", 2024-09-18T11:33:07.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.677 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.677 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039208+0000", 2024-09-18T11:33:07.677 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:07.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.678 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039217+0000", 2024-09-18T11:33:07.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2650000000000002e-06 2024-09-18T11:33:07.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.678 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039236+0000", 2024-09-18T11:33:07.678 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8734000000000001e-05 2024-09-18T11:33:07.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.678 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.678 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039244+0000", 2024-09-18T11:33:07.679 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2900000000000002e-06 2024-09-18T11:33:07.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.679 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.679 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039289+0000", 2024-09-18T11:33:07.679 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4657999999999998e-05 2024-09-18T11:33:07.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.679 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:33:07.679 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039294+0000", 2024-09-18T11:33:07.679 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0719999999999999e-06 2024-09-18T11:33:07.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.680 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.680 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.042052+0000", 2024-09-18T11:33:07.680 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027585399999999999 2024-09-18T11:33:07.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.680 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.680 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.042162+0000", 2024-09-18T11:33:07.680 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011005699999999999 2024-09-18T11:33:07.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.680 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:33:07.680 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.042341+0000", 2024-09-18T11:33:07.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000178285 2024-09-18T11:33:07.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.681 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:07.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.044140+0000", 2024-09-18T11:33:07.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001799092 2024-09-18T11:33:07.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.681 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:07.681 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.049396+0000", 2024-09-18T11:33:07.681 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005256213 2024-09-18T11:33:07.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.682 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.049431+0000", 2024-09-18T11:33:07.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.536e-05 2024-09-18T11:33:07.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.682 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.682 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.049468+0000", 2024-09-18T11:33:07.682 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6548999999999999e-05 2024-09-18T11:33:07.682 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.682 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.682 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.683 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1241 4.d e169/157 4:b44229a1:::smithi11939506-42:51 v 169'384, mlcod=168'380)", 2024-09-18T11:33:07.683 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.053467+0000", 2024-09-18T11:33:07.683 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.568207424000001, 2024-09-18T11:33:07.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010417658, 2024-09-18T11:33:07.683 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.683 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.683 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.683 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.683 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053467+0000", 2024-09-18T11:33:07.683 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.683 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.684 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053467+0000", 2024-09-18T11:33:07.684 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.684 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.684 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.684 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053467+0000", 2024-09-18T11:33:07.684 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:07.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.684 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.684 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.684 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053474+0000", 2024-09-18T11:33:07.684 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9430000000000003e-06 2024-09-18T11:33:07.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.684 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.684 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.685 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053483+0000", 2024-09-18T11:33:07.685 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3819999999999993e-06 2024-09-18T11:33:07.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.685 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.685 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.685 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053494+0000", 2024-09-18T11:33:07.685 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0609e-05 2024-09-18T11:33:07.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.685 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.685 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.685 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053532+0000", 2024-09-18T11:33:07.685 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8405000000000002e-05 2024-09-18T11:33:07.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053554+0000", 2024-09-18T11:33:07.686 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1758000000000001e-05 2024-09-18T11:33:07.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.063852+0000", 2024-09-18T11:33:07.686 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010298185 2024-09-18T11:33:07.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.686 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.686 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.063885+0000", 2024-09-18T11:33:07.687 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2838e-05 2024-09-18T11:33:07.687 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.687 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.687 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.687 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4513 4.a 4:552523b6:::smithi11939506-38:73 [set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:33:07.687 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.054905+0000", 2024-09-18T11:33:07.687 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.566769547, 2024-09-18T11:33:07.687 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.014365329, 2024-09-18T11:33:07.687 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.687 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.687 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.688 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.688 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.688 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4513 2024-09-18T11:33:07.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.688 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.688 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054905+0000", 2024-09-18T11:33:07.688 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.688 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.688 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054905+0000", 2024-09-18T11:33:07.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.689 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.689 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054904+0000", 2024-09-18T11:33:07.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:07.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.689 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.689 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054912+0000", 2024-09-18T11:33:07.689 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4300000000000006e-06 2024-09-18T11:33:07.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.690 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.690 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054927+0000", 2024-09-18T11:33:07.690 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4588e-05 2024-09-18T11:33:07.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.690 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.690 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.690 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054933+0000", 2024-09-18T11:33:07.690 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9009999999999999e-06 2024-09-18T11:33:07.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.690 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.690 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.690 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054975+0000", 2024-09-18T11:33:07.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2588e-05 2024-09-18T11:33:07.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.691 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.691 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055079+0000", 2024-09-18T11:33:07.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103508 2024-09-18T11:33:07.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.691 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:33:07.691 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.064787+0000", 2024-09-18T11:33:07.691 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097086719999999998 2024-09-18T11:33:07.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:07.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.066147+0000", 2024-09-18T11:33:07.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013597660000000001 2024-09-18T11:33:07.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:07.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.069210+0000", 2024-09-18T11:33:07.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030630430000000001 2024-09-18T11:33:07.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.692 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.692 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.069240+0000", 2024-09-18T11:33:07.692 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9628000000000002e-05 2024-09-18T11:33:07.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.693 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.693 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.069271+0000", 2024-09-18T11:33:07.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0682000000000002e-05 2024-09-18T11:33:07.693 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.693 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.693 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.693 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1242 4.d e169/157 4:b9bb642b:::smithi11939506-27:67 v 169'386, mlcod=168'380)", 2024-09-18T11:33:07.693 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.054970+0000", 2024-09-18T11:33:07.693 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.566704818000002, 2024-09-18T11:33:07.693 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089404059999999997, 2024-09-18T11:33:07.694 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.694 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.694 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.694 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.694 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054970+0000", 2024-09-18T11:33:07.694 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.694 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.694 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054970+0000", 2024-09-18T11:33:07.694 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054969+0000", 2024-09-18T11:33:07.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:07.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054977+0000", 2024-09-18T11:33:07.695 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6240000000000002e-06 2024-09-18T11:33:07.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.695 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.695 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054987+0000", 2024-09-18T11:33:07.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0186e-05 2024-09-18T11:33:07.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.696 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.696 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.696 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054993+0000", 2024-09-18T11:33:07.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5160000000000002e-06 2024-09-18T11:33:07.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.696 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.696 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.696 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055563+0000", 2024-09-18T11:33:07.696 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00057030399999999999 2024-09-18T11:33:07.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.696 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.696 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.697 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055576+0000", 2024-09-18T11:33:07.697 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3366e-05 2024-09-18T11:33:07.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.697 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.697 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.063887+0000", 2024-09-18T11:33:07.697 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083112240000000007 2024-09-18T11:33:07.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.697 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.697 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.063910+0000", 2024-09-18T11:33:07.697 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2932999999999999e-05 2024-09-18T11:33:07.697 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.698 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.698 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.698 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.698 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4665 4.a 4:588debc2:::smithi11939506-18:head [rollback 71] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:33:07.698 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.316572+0000", 2024-09-18T11:33:07.698 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.305102560000002, 2024-09-18T11:33:07.698 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012421599, 2024-09-18T11:33:07.698 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.698 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.698 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.698 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.698 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.699 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4665 2024-09-18T11:33:07.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.699 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.699 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.699 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316572+0000", 2024-09-18T11:33:07.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.699 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.699 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.699 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316572+0000", 2024-09-18T11:33:07.699 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.700 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.700 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316572+0000", 2024-09-18T11:33:07.700 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:07.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.700 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.700 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316574+0000", 2024-09-18T11:33:07.700 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3599999999999999e-06 2024-09-18T11:33:07.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.700 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.700 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316578+0000", 2024-09-18T11:33:07.701 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0840000000000002e-06 2024-09-18T11:33:07.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.701 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.701 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316581+0000", 2024-09-18T11:33:07.701 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8739999999999999e-06 2024-09-18T11:33:07.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.701 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.702 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318695+0000", 2024-09-18T11:33:07.702 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002113382 2024-09-18T11:33:07.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.702 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.702 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.702 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318805+0000", 2024-09-18T11:33:07.702 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000110511 2024-09-18T11:33:07.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.702 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.702 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:33:07.702 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.321575+0000", 2024-09-18T11:33:07.702 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027700730000000001 2024-09-18T11:33:07.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.702 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.703 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:07.703 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.324576+0000", 2024-09-18T11:33:07.703 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030004599999999999 2024-09-18T11:33:07.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.703 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.703 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:07.703 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.328914+0000", 2024-09-18T11:33:07.703 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043382159999999998 2024-09-18T11:33:07.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.703 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.703 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.703 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.328951+0000", 2024-09-18T11:33:07.703 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7036999999999998e-05 2024-09-18T11:33:07.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.704 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.704 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.704 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.328994+0000", 2024-09-18T11:33:07.704 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2979999999999998e-05 2024-09-18T11:33:07.704 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.704 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.704 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.704 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.704 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4666 4.a 4:588debc2:::smithi11939506-18:head [append 0~0] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:33:07.704 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.316584+0000", 2024-09-18T11:33:07.704 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 22.305090929999999, 2024-09-18T11:33:07.705 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012596499000000001, 2024-09-18T11:33:07.705 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.705 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.705 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.705 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.705 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.705 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4666 2024-09-18T11:33:07.705 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.705 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.705 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.705 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316584+0000", 2024-09-18T11:33:07.705 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316584+0000", 2024-09-18T11:33:07.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316584+0000", 2024-09-18T11:33:07.706 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:33:07.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.706 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.706 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316585+0000", 2024-09-18T11:33:07.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8109999999999999e-06 2024-09-18T11:33:07.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316588+0000", 2024-09-18T11:33:07.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0769999999999999e-06 2024-09-18T11:33:07.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.707 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.707 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316591+0000", 2024-09-18T11:33:07.707 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1320000000000001e-06 2024-09-18T11:33:07.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.708 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.708 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.319126+0000", 2024-09-18T11:33:07.708 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002535266 2024-09-18T11:33:07.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.708 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.708 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for blocked object", 2024-09-18T11:33:07.708 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.319180+0000", 2024-09-18T11:33:07.708 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4457999999999999e-05 2024-09-18T11:33:07.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.708 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.708 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.708 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.323839+0000", 2024-09-18T11:33:07.708 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046581720000000004 2024-09-18T11:33:07.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.709 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.709 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.324024+0000", 2024-09-18T11:33:07.709 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018519599999999999 2024-09-18T11:33:07.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.709 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:33:07.709 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.324246+0000", 2024-09-18T11:33:07.709 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022207800000000001 2024-09-18T11:33:07.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.709 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:07.709 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.325067+0000", 2024-09-18T11:33:07.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00082141699999999996 2024-09-18T11:33:07.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.710 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:07.710 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.329117+0000", 2024-09-18T11:33:07.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040501210000000003 2024-09-18T11:33:07.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.710 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.710 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.329151+0000", 2024-09-18T11:33:07.710 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4133e-05 2024-09-18T11:33:07.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.711 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.711 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.329180+0000", 2024-09-18T11:33:07.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.887e-05 2024-09-18T11:33:07.711 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.711 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.711 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.711 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4754 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2739607,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:33:07.711 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:49.234755+0000", 2024-09-18T11:33:07.711 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 18.386919271, 2024-09-18T11:33:07.711 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0093043659999999997, 2024-09-18T11:33:07.711 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.712 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:07.712 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:07.712 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:07.712 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:07.712 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4754 2024-09-18T11:33:07.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.712 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.712 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.712 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234755+0000", 2024-09-18T11:33:07.712 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234755+0000", 2024-09-18T11:33:07.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234755+0000", 2024-09-18T11:33:07.713 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:07.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.713 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.713 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234760+0000", 2024-09-18T11:33:07.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6380000000000003e-06 2024-09-18T11:33:07.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.714 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.714 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234766+0000", 2024-09-18T11:33:07.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0159999999999997e-06 2024-09-18T11:33:07.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.714 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.714 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234771+0000", 2024-09-18T11:33:07.714 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1399999999999999e-06 2024-09-18T11:33:07.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.715 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234846+0000", 2024-09-18T11:33:07.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5308999999999993e-05 2024-09-18T11:33:07.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.715 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:07.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234848+0000", 2024-09-18T11:33:07.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4260000000000002e-06 2024-09-18T11:33:07.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.715 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.715 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.241890+0000", 2024-09-18T11:33:07.715 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070411859999999996 2024-09-18T11:33:07.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.716 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.716 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.241983+0000", 2024-09-18T11:33:07.716 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3843000000000003e-05 2024-09-18T11:33:07.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.716 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.716 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.244060+0000", 2024-09-18T11:33:07.716 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020763679999999999 2024-09-18T11:33:07.716 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.716 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.716 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.717 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.717 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1293 4.d e182/157 4:b9bb642b:::smithi11939506-27:6d v 182'415, mlcod=182'411)", 2024-09-18T11:33:07.717 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:53.041205+0000", 2024-09-18T11:33:07.717 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.580470008000001, 2024-09-18T11:33:07.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0091437189999999998, 2024-09-18T11:33:07.717 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.717 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.717 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.717 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.717 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041205+0000", 2024-09-18T11:33:07.717 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041205+0000", 2024-09-18T11:33:07.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.718 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041204+0000", 2024-09-18T11:33:07.718 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:07.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.718 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041210+0000", 2024-09-18T11:33:07.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3280000000000002e-06 2024-09-18T11:33:07.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041222+0000", 2024-09-18T11:33:07.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1686e-05 2024-09-18T11:33:07.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.719 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.719 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.719 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041227+0000", 2024-09-18T11:33:07.719 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2680000000000002e-06 2024-09-18T11:33:07.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.720 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.720 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.720 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041257+0000", 2024-09-18T11:33:07.720 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9869000000000002e-05 2024-09-18T11:33:07.720 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.720 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.720 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.720 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041279+0000", 2024-09-18T11:33:07.720 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1651999999999999e-05 2024-09-18T11:33:07.720 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.720 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.720 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.720 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.050305+0000", 2024-09-18T11:33:07.720 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090263770000000004 2024-09-18T11:33:07.721 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.721 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.721 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.721 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.050348+0000", 2024-09-18T11:33:07.721 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3173999999999997e-05 2024-09-18T11:33:07.721 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.721 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.721 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.721 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.721 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.721 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1294 4.d e182/157 4:b30010d9:::smithi11939506-43:7e v 182'417, mlcod=182'411)", 2024-09-18T11:33:07.721 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:53.041747+0000", 2024-09-18T11:33:07.721 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.579927775, 2024-09-18T11:33:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011293531000000001, 2024-09-18T11:33:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041747+0000", 2024-09-18T11:33:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.722 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041747+0000", 2024-09-18T11:33:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041746+0000", 2024-09-18T11:33:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041753+0000", 2024-09-18T11:33:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0010000000000004e-06 2024-09-18T11:33:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.723 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041766+0000", 2024-09-18T11:33:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.291e-05 2024-09-18T11:33:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041772+0000", 2024-09-18T11:33:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8390000000000003e-06 2024-09-18T11:33:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041810+0000", 2024-09-18T11:33:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7864000000000002e-05 2024-09-18T11:33:07.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041828+0000", 2024-09-18T11:33:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8748000000000001e-05 2024-09-18T11:33:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.053002+0000", 2024-09-18T11:33:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011173819 2024-09-18T11:33:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.053041+0000", 2024-09-18T11:33:07.725 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8260000000000003e-05 2024-09-18T11:33:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.4.0:1048 4.e e182/22 4:78eefc4b:::smithi11939506-7:6d v 182'218, mlcod=173'215)", 2024-09-18T11:33:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:53.042298+0000", 2024-09-18T11:33:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.579376662, 2024-09-18T11:33:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010780692999999999, 2024-09-18T11:33:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.726 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042298+0000", 2024-09-18T11:33:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042298+0000", 2024-09-18T11:33:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042297+0000", 2024-09-18T11:33:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.727 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042304+0000", 2024-09-18T11:33:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3249999999999996e-06 2024-09-18T11:33:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042316+0000", 2024-09-18T11:33:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2108e-05 2024-09-18T11:33:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042322+0000", 2024-09-18T11:33:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8490000000000002e-06 2024-09-18T11:33:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.728 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042678+0000", 2024-09-18T11:33:07.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00035671900000000002 2024-09-18T11:33:07.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.729 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.729 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042709+0000", 2024-09-18T11:33:07.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0518000000000002e-05 2024-09-18T11:33:07.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.729 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.729 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.053041+0000", 2024-09-18T11:33:07.729 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010331965 2024-09-18T11:33:07.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.053079+0000", 2024-09-18T11:33:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7805000000000001e-05 2024-09-18T11:33:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.4.0:1049 4.e e182/22 4:7410c0e0:::smithi11939506-47:6d v 182'220, mlcod=173'215)", 2024-09-18T11:33:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:53.042878+0000", 2024-09-18T11:33:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 14.578796398, 2024-09-18T11:33:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012621894999999999, 2024-09-18T11:33:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:07.730 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042878+0000", 2024-09-18T11:33:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042878+0000", 2024-09-18T11:33:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.731 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:07.732 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042878+0000", 2024-09-18T11:33:07.732 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:07.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.732 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:07.732 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042883+0000", 2024-09-18T11:33:07.732 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7699999999999998e-06 2024-09-18T11:33:07.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.732 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:07.732 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042896+0000", 2024-09-18T11:33:07.732 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.296e-05 2024-09-18T11:33:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042902+0000", 2024-09-18T11:33:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4600000000000002e-06 2024-09-18T11:33:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.047389+0000", 2024-09-18T11:33:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0044867010000000001 2024-09-18T11:33:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.733 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.047411+0000", 2024-09-18T11:33:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2076e-05 2024-09-18T11:33:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.055464+0000", 2024-09-18T11:33:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080533370000000007 2024-09-18T11:33:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.055500+0000", 2024-09-18T11:33:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6251000000000002e-05 2024-09-18T11:33:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.734 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.735 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.735 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:07.735 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:07.735 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:07.736 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:33:07.737 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:33:07.735+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:33:07.837 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:07.836+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:33:07.874 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:07.875 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:07.875 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:33:07.875 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:07.884 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:33:07.937 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:07.936+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:33:07.982 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:07.982 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:07.982 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:33:07.982 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:33:07.982 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:07.991 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:33:08.028 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:33:08.026+0000 7f6bf5056640 -1 osd.5 194 handle_osd_map injecting map crc failure 2024-09-18T11:33:08.032 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is set 2024-09-18T11:33:08.038 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:33:08.037+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:33:08.091 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:08.091 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:33:08.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:33:08.092 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:33:08.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.092 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:08.092 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:33:08.092 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 198.93172235899999, 2024-09-18T11:33:08.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:33:08.092 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:08.092 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.092 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:08.092 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:08.093 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:08.093 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:33:08.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.093 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:08.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.093 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:08.093 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:33:08.093 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.093 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:08.093 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:33:08.093 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.094 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:08.094 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:33:08.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:08.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.094 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:08.094 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:33:08.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:33:08.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.094 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:08.094 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:33:08.094 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:33:08.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.095 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.095 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:33:08.095 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:33:08.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.095 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:08.095 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:33:08.095 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:33:08.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:08.096 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:33:08.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:33:08.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.096 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:08.096 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:33:08.096 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:33:08.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:08.097 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:33:08.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:33:08.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.097 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:08.097 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:33:08.097 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:33:08.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.098 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:08.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:33:08.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:33:08.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.098 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:08.098 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:33:08.098 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:33:08.098 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.098 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:08.098 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.099 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:08.099 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:33:08.099 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 198.93166742899999, 2024-09-18T11:33:08.099 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:33:08.099 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:08.099 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.099 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:08.099 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:08.099 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:08.099 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:33:08.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.099 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:08.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:08.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:33:08.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:08.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:33:08.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.100 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:08.100 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:33:08.100 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:08.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.101 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:08.101 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:33:08.101 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:33:08.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.101 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:08.101 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:33:08.101 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:33:08.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.102 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:33:08.102 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:33:08.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.102 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:08.102 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:33:08.102 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:33:08.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.103 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:08.103 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:33:08.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:33:08.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.103 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:08.103 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:33:08.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:33:08.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.103 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:08.103 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:33:08.103 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:33:08.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:08.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:33:08.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:33:08.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:08.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:33:08.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:33:08.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.104 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:08.104 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:33:08.104 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:33:08.105 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.105 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:08.105 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.105 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:08.105 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:33:08.105 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 198.931657449, 2024-09-18T11:33:08.105 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:33:08.105 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:08.105 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.105 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:08.105 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:08.105 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:08.106 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:33:08.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.106 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:08.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:08.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:33:08.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.106 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:08.106 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:33:08.106 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:08.107 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:33:08.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:33:08.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:08.107 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:33:08.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:33:08.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.107 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:08.107 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:33:08.107 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:33:08.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.108 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.108 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:33:08.108 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:33:08.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.108 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:08.108 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:33:08.108 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:33:08.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.108 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:08.109 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:33:08.109 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:33:08.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.109 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:08.109 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:33:08.109 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:33:08.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.109 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:08.109 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:33:08.109 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:33:08.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.110 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:08.110 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:33:08.110 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:33:08.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.110 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:08.110 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:33:08.110 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:33:08.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.110 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:08.110 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:33:08.110 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:33:08.110 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.111 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:08.111 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.111 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:08.111 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:33:08.111 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 198.93164585, 2024-09-18T11:33:08.111 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:33:08.111 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:08.111 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.111 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:08.111 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:08.111 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:08.111 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:33:08.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.112 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:08.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.112 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:08.112 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:33:08.112 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.112 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:08.112 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:33:08.112 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.113 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:08.113 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:33:08.113 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:33:08.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.113 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:08.113 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:33:08.113 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:33:08.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.113 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:08.113 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:33:08.113 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:33:08.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.114 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.114 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:33:08.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:33:08.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.114 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:08.114 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:33:08.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:33:08.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.114 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:08.114 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:33:08.114 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:33:08.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:08.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:33:08.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:33:08.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:08.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:33:08.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:33:08.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.115 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:08.115 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:33:08.115 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:33:08.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:08.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:33:08.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:33:08.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.116 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:08.116 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:33:08.116 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:33:08.116 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.116 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:08.116 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.116 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:08.116 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:33:08.117 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 198.93158936899999, 2024-09-18T11:33:08.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:33:08.117 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:08.117 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.117 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:08.117 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:08.117 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:08.117 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:33:08.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.117 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:08.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.117 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:08.117 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:33:08.117 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.118 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:08.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:33:08.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.118 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:08.118 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:33:08.118 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:08.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.118 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:08.119 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:33:08.119 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:33:08.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.119 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:08.119 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:33:08.119 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:33:08.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.119 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.119 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:33:08.119 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:33:08.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:08.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:33:08.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:33:08.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:08.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:33:08.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:33:08.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.120 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:08.120 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:33:08.120 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:33:08.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:08.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:33:08.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:33:08.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:08.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:33:08.121 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:33:08.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.121 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:08.121 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:33:08.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:33:08.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.122 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:08.122 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:33:08.122 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:33:08.122 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.122 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:08.122 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.122 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:08.122 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:33:08.122 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 198.93157554499999, 2024-09-18T11:33:08.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:33:08.123 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:08.123 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.123 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:08.123 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:08.123 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:08.123 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:33:08.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.123 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:08.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:08.123 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:33:08.123 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.123 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:08.124 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:33:08.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.124 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:08.124 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:33:08.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:33:08.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.124 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:08.124 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:33:08.124 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:33:08.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.125 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:08.125 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:33:08.125 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:33:08.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.125 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.125 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:33:08.125 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:33:08.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.126 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:08.126 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:33:08.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:33:08.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.126 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:08.126 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:33:08.126 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:33:08.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.126 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:08.126 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:33:08.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:33:08.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.127 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:08.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:33:08.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:33:08.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.127 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:08.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:33:08.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:33:08.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.127 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:08.127 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:33:08.127 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:33:08.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.128 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:08.128 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:33:08.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:33:08.128 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.128 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:08.128 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.128 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:08.128 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:33:08.128 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 198.515396638, 2024-09-18T11:33:08.128 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:33:08.129 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:08.129 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.129 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:08.129 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:08.129 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:08.129 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:33:08.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.129 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:08.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:08.129 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:33:08.129 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.129 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:08.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:33:08.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.130 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:08.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:33:08.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:33:08.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.130 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:08.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:33:08.130 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:33:08.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.130 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:08.130 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:33:08.131 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:33:08.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.131 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.131 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:33:08.131 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:33:08.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.131 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:08.131 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:33:08.131 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:33:08.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.131 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:08.131 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:33:08.131 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:33:08.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.132 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:08.132 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:33:08.132 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:33:08.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.132 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:08.132 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:33:08.132 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:33:08.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.132 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:08.132 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:33:08.132 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:33:08.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.133 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.133 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:08.133 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:33:08.133 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:33:08.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.133 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.133 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:08.133 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:33:08.133 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:33:08.133 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.133 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:08.133 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.133 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.133 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:08.134 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:33:08.134 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 198.51537333799999, 2024-09-18T11:33:08.134 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:33:08.134 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:08.134 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.134 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:08.134 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:08.134 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:08.134 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:33:08.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.134 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:08.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.134 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:08.134 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:33:08.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.135 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:08.135 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:33:08.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.135 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:08.135 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:33:08.135 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:08.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.136 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:08.136 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:33:08.136 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:33:08.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.136 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:08.136 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:33:08.136 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:33:08.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.136 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.137 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:33:08.137 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:33:08.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.137 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.137 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:08.137 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:33:08.137 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:33:08.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.137 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.137 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:08.137 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:33:08.137 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:33:08.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.138 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:08.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:33:08.138 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:33:08.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.138 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:08.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:33:08.138 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:33:08.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.138 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:08.138 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:33:08.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:33:08.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.139 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:08.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:33:08.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:33:08.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.139 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:08.139 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:33:08.139 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:33:08.139 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.139 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:08.139 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.140 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:08.140 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:33:08.140 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 198.51528772899999, 2024-09-18T11:33:08.140 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:33:08.140 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:08.140 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.140 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:08.140 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:08.140 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:08.140 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:33:08.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.140 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:08.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.141 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:08.141 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:33:08.141 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.141 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:08.141 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:33:08.141 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.141 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:08.141 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:33:08.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:08.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.142 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:08.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:33:08.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:33:08.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.142 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:08.142 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:33:08.142 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:33:08.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.142 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.143 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:33:08.143 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:33:08.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.143 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:08.143 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:33:08.143 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:33:08.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.143 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:08.143 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:33:08.143 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:33:08.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.143 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:08.144 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:33:08.144 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:33:08.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.144 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:08.144 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:33:08.144 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:33:08.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.144 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:08.144 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:33:08.144 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:33:08.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.145 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:08.145 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:33:08.145 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:33:08.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.145 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:08.145 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:33:08.145 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:33:08.145 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.145 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:08.145 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.146 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:08.146 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:33:08.146 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 198.51527676399999, 2024-09-18T11:33:08.146 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:33:08.146 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:08.146 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.146 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:08.146 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:08.146 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:08.146 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:33:08.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.146 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:08.146 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.146 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:08.146 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:33:08.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.147 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:08.147 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:33:08.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.147 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:08.147 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:33:08.147 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:33:08.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.147 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:08.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:33:08.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:33:08.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:08.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:33:08.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:33:08.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.148 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.148 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:33:08.148 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:33:08.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.149 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:08.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:33:08.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:33:08.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.149 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:08.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:33:08.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:33:08.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.149 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:08.149 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:33:08.149 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:33:08.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.150 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:08.150 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:33:08.150 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:33:08.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.150 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.150 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:08.150 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:33:08.150 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:33:08.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.150 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.150 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:08.150 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:33:08.150 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:33:08.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.151 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.151 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:08.151 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:33:08.151 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:33:08.151 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.151 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:08.151 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.151 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.151 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:08.151 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:33:08.151 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 198.51526559600001, 2024-09-18T11:33:08.151 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:33:08.152 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:08.152 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.152 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:08.152 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:08.152 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:08.152 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:33:08.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.152 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:08.152 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.152 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:08.152 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:33:08.152 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.152 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.153 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:08.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:33:08.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.153 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:08.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:33:08.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:33:08.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.153 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:08.153 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:33:08.153 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:33:08.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:08.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:33:08.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:33:08.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:33:08.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:33:08.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.154 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:08.154 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:33:08.154 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:33:08.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.155 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:08.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:33:08.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:33:08.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.155 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:08.155 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:33:08.155 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:33:08.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.155 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:08.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:33:08.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:33:08.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.156 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:08.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:33:08.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:33:08.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.156 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:08.156 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:33:08.156 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:33:08.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.157 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:08.157 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:33:08.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:33:08.157 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.157 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:08.157 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.157 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:08.157 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:33:08.157 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 198.51520026700001, 2024-09-18T11:33:08.157 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:33:08.157 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:08.158 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.158 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:08.158 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:08.158 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:08.158 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:33:08.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.158 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:08.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.158 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:08.158 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:33:08.158 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.158 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:08.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:33:08.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.159 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:08.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:33:08.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:08.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.159 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:08.159 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:33:08.159 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:33:08.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.160 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:08.160 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:33:08.160 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:33:08.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.160 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.160 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:33:08.160 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:33:08.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.160 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:08.160 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:33:08.160 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:33:08.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:08.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:33:08.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:33:08.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:08.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:33:08.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:33:08.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.161 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:08.161 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:33:08.161 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:33:08.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:08.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:33:08.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:33:08.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:08.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:33:08.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:33:08.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.162 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:08.162 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:33:08.162 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:33:08.163 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.163 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:08.163 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.163 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:08.163 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:33:08.163 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 198.505038289, 2024-09-18T11:33:08.163 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:33:08.163 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:08.163 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.163 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:08.163 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:08.164 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:08.164 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:33:08.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.164 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:08.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.164 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:08.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:33:08.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.164 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:08.164 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:33:08.164 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:08.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:33:08.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:08.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:08.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:33:08.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:33:08.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.165 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:08.165 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:33:08.165 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:33:08.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:33:08.166 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:33:08.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:08.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:33:08.166 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:33:08.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.166 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:08.166 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:33:08.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:33:08.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:08.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:33:08.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:33:08.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:08.167 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:33:08.167 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:33:08.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.167 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:08.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:33:08.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:33:08.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:08.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:33:08.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:33:08.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.168 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:08.168 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:33:08.168 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:33:08.168 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.168 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:08.169 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.169 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:08.169 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:33:08.169 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 198.50494581000001, 2024-09-18T11:33:08.169 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:33:08.169 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:08.169 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.169 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:08.169 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:08.169 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:08.169 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:33:08.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.170 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:08.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:08.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:33:08.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:08.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:33:08.170 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.170 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:08.170 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:33:08.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:08.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:08.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:33:08.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:33:08.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:08.171 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:33:08.171 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:33:08.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.171 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.172 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:33:08.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:33:08.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.172 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:08.172 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:33:08.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:33:08.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.172 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:08.172 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:33:08.172 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:33:08.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.173 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:08.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:33:08.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:33:08.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.173 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:08.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:33:08.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:33:08.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.173 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:08.173 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:33:08.173 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:33:08.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.174 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:08.174 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:33:08.174 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:33:08.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.174 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:08.174 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:33:08.174 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:33:08.174 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.174 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:08.174 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.175 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:123 4.3 e132/67 4:cc46532a:::smithi11939506-33:30 v 132'171, mlcod=128'168)", 2024-09-18T11:33:08.175 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:33:08.175 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 65.061639185000004, 2024-09-18T11:33:08.175 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087627039999999996, 2024-09-18T11:33:08.175 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:08.175 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.175 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:08.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.175 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:08.175 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:33:08.175 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:08.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:33:08.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:08.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027851+0000", 2024-09-18T11:33:08.176 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:08.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.176 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:08.176 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027871+0000", 2024-09-18T11:33:08.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9749e-05 2024-09-18T11:33:08.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:08.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027882+0000", 2024-09-18T11:33:08.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1242e-05 2024-09-18T11:33:08.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.177 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027889+0000", 2024-09-18T11:33:08.177 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7649999999999996e-06 2024-09-18T11:33:08.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.177 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:08.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027929+0000", 2024-09-18T11:33:08.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9984999999999997e-05 2024-09-18T11:33:08.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:08.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027951+0000", 2024-09-18T11:33:08.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2047000000000001e-05 2024-09-18T11:33:08.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.178 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:08.178 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.036565+0000", 2024-09-18T11:33:08.178 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086135110000000008 2024-09-18T11:33:08.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.179 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:08.179 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.036614+0000", 2024-09-18T11:33:08.179 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9747000000000003e-05 2024-09-18T11:33:08.179 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.179 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:08.179 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.179 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.179 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:124 4.3 e132/67 4:ca2b2309:::smithi11939506-44:30 v 132'173, mlcod=128'168)", 2024-09-18T11:33:08.179 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:33:08.179 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 65.060947202999998, 2024-09-18T11:33:08.179 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0088947309999999995, 2024-09-18T11:33:08.179 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:08.179 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.180 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:08.180 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.180 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:08.180 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:33:08.180 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.180 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.180 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.180 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:08.180 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:33:08.180 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.180 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.180 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.180 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:08.180 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028543+0000", 2024-09-18T11:33:08.181 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:08.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.181 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:08.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028551+0000", 2024-09-18T11:33:08.181 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8720000000000002e-06 2024-09-18T11:33:08.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.181 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:08.181 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028562+0000", 2024-09-18T11:33:08.181 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1039000000000001e-05 2024-09-18T11:33:08.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.181 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.181 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.182 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028569+0000", 2024-09-18T11:33:08.182 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0350000000000004e-06 2024-09-18T11:33:08.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.182 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.182 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:08.182 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028848+0000", 2024-09-18T11:33:08.182 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00027935600000000003 2024-09-18T11:33:08.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.182 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.182 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:08.182 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028868+0000", 2024-09-18T11:33:08.182 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9947999999999999e-05 2024-09-18T11:33:08.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.182 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.183 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:08.183 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037405+0000", 2024-09-18T11:33:08.183 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085369920000000002 2024-09-18T11:33:08.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.183 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.183 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:08.183 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037438+0000", 2024-09-18T11:33:08.183 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3244000000000002e-05 2024-09-18T11:33:08.183 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.183 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:08.183 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.183 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.183 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:154 4.3 e147/67 4:ca2b2309:::smithi11939506-44:5d v 147'216, mlcod=145'210)", 2024-09-18T11:33:08.184 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:33:08.184 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 49.752160754000002, 2024-09-18T11:33:08.184 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066667829999999999, 2024-09-18T11:33:08.184 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:08.184 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.184 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:08.184 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.184 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:08.184 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:33:08.184 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.184 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.184 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:08.184 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:33:08.185 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.185 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.185 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:08.185 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:33:08.185 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:08.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.185 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.185 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:08.185 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337335+0000", 2024-09-18T11:33:08.185 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1479999999999997e-06 2024-09-18T11:33:08.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.185 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.185 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:08.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337344+0000", 2024-09-18T11:33:08.186 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5640000000000002e-06 2024-09-18T11:33:08.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.186 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337349+0000", 2024-09-18T11:33:08.186 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9470000000000002e-06 2024-09-18T11:33:08.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.186 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:08.186 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337410+0000", 2024-09-18T11:33:08.186 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0448999999999997e-05 2024-09-18T11:33:08.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:08.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337425+0000", 2024-09-18T11:33:08.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.504e-05 2024-09-18T11:33:08.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:08.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.343972+0000", 2024-09-18T11:33:08.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065474169999999998 2024-09-18T11:33:08.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.187 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:08.187 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.343997+0000", 2024-09-18T11:33:08.187 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4672999999999999e-05 2024-09-18T11:33:08.188 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.188 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:08.188 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.188 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.188 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1238 4.6 e169/143 4:6f86ffcc:::smithi11939506-15:51 v 169'193, mlcod=166'187)", 2024-09-18T11:33:08.188 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.044089+0000", 2024-09-18T11:33:08.188 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 28.045401685000002, 2024-09-18T11:33:08.188 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072870620000000004, 2024-09-18T11:33:08.188 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:08.188 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.188 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:08.188 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.188 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:08.188 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044089+0000", 2024-09-18T11:33:08.189 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.189 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.189 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:08.189 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044089+0000", 2024-09-18T11:33:08.189 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.189 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.189 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:08.189 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044089+0000", 2024-09-18T11:33:08.189 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:08.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.190 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.190 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:08.190 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044097+0000", 2024-09-18T11:33:08.190 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9060000000000002e-06 2024-09-18T11:33:08.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.190 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.190 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:08.190 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044105+0000", 2024-09-18T11:33:08.190 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3590000000000008e-06 2024-09-18T11:33:08.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.190 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.190 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.191 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044112+0000", 2024-09-18T11:33:08.191 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0639999999999996e-06 2024-09-18T11:33:08.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.191 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.191 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:08.191 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044851+0000", 2024-09-18T11:33:08.191 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00073857800000000004 2024-09-18T11:33:08.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.191 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.191 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:08.191 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044871+0000", 2024-09-18T11:33:08.191 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0140999999999999e-05 2024-09-18T11:33:08.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.191 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.192 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:08.192 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.051338+0000", 2024-09-18T11:33:08.192 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064673760000000004 2024-09-18T11:33:08.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.192 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.192 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:08.192 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.051376+0000", 2024-09-18T11:33:08.192 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8222000000000001e-05 2024-09-18T11:33:08.192 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.192 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:08.192 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.192 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.193 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:281 4.3 e169/67 4:ca2b2309:::smithi11939506-44:5d v 169'279, mlcod=168'273)", 2024-09-18T11:33:08.193 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.047201+0000", 2024-09-18T11:33:08.193 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 28.04228955, 2024-09-18T11:33:08.193 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007110346, 2024-09-18T11:33:08.193 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:08.193 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.193 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:08.193 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.193 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:08.193 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047201+0000", 2024-09-18T11:33:08.193 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.193 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.193 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:08.193 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047201+0000", 2024-09-18T11:33:08.194 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.194 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.194 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.194 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:08.194 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047201+0000", 2024-09-18T11:33:08.194 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:08.194 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.194 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.194 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:08.194 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047206+0000", 2024-09-18T11:33:08.194 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7309999999999998e-06 2024-09-18T11:33:08.194 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.194 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.194 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:08.195 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047217+0000", 2024-09-18T11:33:08.195 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0577000000000001e-05 2024-09-18T11:33:08.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.195 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.195 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.195 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047221+0000", 2024-09-18T11:33:08.195 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0509999999999998e-06 2024-09-18T11:33:08.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.195 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.195 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:08.195 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.048977+0000", 2024-09-18T11:33:08.195 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017558179999999999 2024-09-18T11:33:08.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.195 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.195 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:08.196 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.048999+0000", 2024-09-18T11:33:08.196 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.23e-05 2024-09-18T11:33:08.196 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.196 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.196 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:08.196 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054275+0000", 2024-09-18T11:33:08.196 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0052754150000000003 2024-09-18T11:33:08.196 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.196 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.196 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:08.196 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054312+0000", 2024-09-18T11:33:08.196 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7163999999999999e-05 2024-09-18T11:33:08.196 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.196 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:08.196 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.197 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.197 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.197 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:4518 4.7 e169/155 4:e0d95e10:::smithi11939506-22:head v 169'394, mlcod=169'389)", 2024-09-18T11:33:08.197 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.057943+0000", 2024-09-18T11:33:08.197 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 28.031548300000001, 2024-09-18T11:33:08.197 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075088719999999998, 2024-09-18T11:33:08.197 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:08.197 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.197 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:08.197 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.197 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:08.197 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057943+0000", 2024-09-18T11:33:08.197 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.198 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.198 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:08.198 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057943+0000", 2024-09-18T11:33:08.198 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:08.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.198 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.198 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:08.198 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057942+0000", 2024-09-18T11:33:08.198 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:08.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.198 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.198 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:08.198 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057953+0000", 2024-09-18T11:33:08.198 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1399e-05 2024-09-18T11:33:08.199 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.199 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.199 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:08.199 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057969+0000", 2024-09-18T11:33:08.199 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5860999999999999e-05 2024-09-18T11:33:08.199 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.199 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.199 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.199 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057975+0000", 2024-09-18T11:33:08.199 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8610000000000003e-06 2024-09-18T11:33:08.199 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.199 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.199 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:08.199 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.058039+0000", 2024-09-18T11:33:08.199 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4313000000000006e-05 2024-09-18T11:33:08.200 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.200 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.200 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:08.200 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.058059+0000", 2024-09-18T11:33:08.200 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0205999999999999e-05 2024-09-18T11:33:08.200 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.200 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.200 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:08.200 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065410+0000", 2024-09-18T11:33:08.200 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073512020000000003 2024-09-18T11:33:08.200 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:08.200 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:08.200 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:08.200 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065451+0000", 2024-09-18T11:33:08.201 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1032999999999998e-05 2024-09-18T11:33:08.201 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.201 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:08.201 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.201 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:08.201 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:08.201 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:08.202 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:33:08.203 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:08.201+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:08.303 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:08.302+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:08.304 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:08.304 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:33:08.304 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:33:08.304 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:08.312 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:33:08.404 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:33:08.403+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:33:08.411 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:08.411 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:33:08.411 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:33:08.411 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:33:08.411 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:08.420 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:33:08.505 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:33:08.503+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:33:08.519 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:08.519 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:33:08.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:33:08.520 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:33:08.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.520 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4736 4.3 4:c70c3162:::smithi11939506-35:head [write 574905~711397,stat] snapc 7c=[7c,7b,79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:33:08.520 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.031772+0000", 2024-09-18T11:33:08.520 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.485666523999999, 2024-09-18T11:33:08.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0092375840000000001, 2024-09-18T11:33:08.520 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:08.520 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.520 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:08.521 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:08.521 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:08.521 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4736 2024-09-18T11:33:08.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.521 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:08.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:08.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.031772+0000", 2024-09-18T11:33:08.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:08.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.031772+0000", 2024-09-18T11:33:08.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:08.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.031771+0000", 2024-09-18T11:33:08.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:08.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:08.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032335+0000", 2024-09-18T11:33:08.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00056435800000000005 2024-09-18T11:33:08.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:08.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032349+0000", 2024-09-18T11:33:08.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4353e-05 2024-09-18T11:33:08.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032356+0000", 2024-09-18T11:33:08.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3550000000000001e-06 2024-09-18T11:33:08.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032487+0000", 2024-09-18T11:33:08.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013154900000000001 2024-09-18T11:33:08.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:33:08.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032495+0000", 2024-09-18T11:33:08.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7519999999999996e-06 2024-09-18T11:33:08.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.035010+0000", 2024-09-18T11:33:08.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025153530000000001 2024-09-18T11:33:08.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:08.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.035142+0000", 2024-09-18T11:33:08.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013133599999999999 2024-09-18T11:33:08.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.525 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:08.525 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.035330+0000", 2024-09-18T11:33:08.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018873499999999999 2024-09-18T11:33:08.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:08.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040882+0000", 2024-09-18T11:33:08.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055514689999999998 2024-09-18T11:33:08.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:08.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040917+0000", 2024-09-18T11:33:08.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5305999999999997e-05 2024-09-18T11:33:08.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:08.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040968+0000", 2024-09-18T11:33:08.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0720000000000002e-05 2024-09-18T11:33:08.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:08.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041010+0000", 2024-09-18T11:33:08.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1893e-05 2024-09-18T11:33:08.527 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.527 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:08.527 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.528 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4737 4.3 4:c70c3162:::smithi11939506-35:head [write 1885086~525140] snapc 7c=[7c,7b,79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:33:08.528 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.032373+0000", 2024-09-18T11:33:08.528 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.485065471999999, 2024-09-18T11:33:08.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0088738949999999997, 2024-09-18T11:33:08.528 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:08.528 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.528 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:08.528 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:08.528 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:08.528 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4737 2024-09-18T11:33:08.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.528 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:08.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:08.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032373+0000", 2024-09-18T11:33:08.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:08.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032373+0000", 2024-09-18T11:33:08.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:08.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032372+0000", 2024-09-18T11:33:08.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:08.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:08.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032651+0000", 2024-09-18T11:33:08.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027865499999999999 2024-09-18T11:33:08.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:08.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032661+0000", 2024-09-18T11:33:08.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5440000000000004e-06 2024-09-18T11:33:08.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032665+0000", 2024-09-18T11:33:08.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9899999999999999e-06 2024-09-18T11:33:08.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032701+0000", 2024-09-18T11:33:08.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6587000000000001e-05 2024-09-18T11:33:08.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:33:08.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032708+0000", 2024-09-18T11:33:08.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.742e-06 2024-09-18T11:33:08.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.037089+0000", 2024-09-18T11:33:08.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043810639999999996 2024-09-18T11:33:08.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:08.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.037176+0000", 2024-09-18T11:33:08.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7266000000000001e-05 2024-09-18T11:33:08.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:08.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.037317+0000", 2024-09-18T11:33:08.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014054200000000001 2024-09-18T11:33:08.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:08.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041074+0000", 2024-09-18T11:33:08.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037575709999999999 2024-09-18T11:33:08.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:08.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041165+0000", 2024-09-18T11:33:08.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0785999999999994e-05 2024-09-18T11:33:08.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:08.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041208+0000", 2024-09-18T11:33:08.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2611999999999998e-05 2024-09-18T11:33:08.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:08.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041247+0000", 2024-09-18T11:33:08.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9538e-05 2024-09-18T11:33:08.534 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.535 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:08.535 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.535 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4738 4.3 4:c70c3162:::smithi11939506-35:head [write 2930189~44716] snapc 7c=[7c,7b,79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:33:08.535 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.032671+0000", 2024-09-18T11:33:08.535 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.484767978000001, 2024-09-18T11:33:08.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086678190000000002, 2024-09-18T11:33:08.535 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:08.535 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.535 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:08.535 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:08.536 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:08.536 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4738 2024-09-18T11:33:08.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.536 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:08.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:08.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032671+0000", 2024-09-18T11:33:08.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:08.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032671+0000", 2024-09-18T11:33:08.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:08.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032670+0000", 2024-09-18T11:33:08.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:08.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:08.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032908+0000", 2024-09-18T11:33:08.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023724099999999999 2024-09-18T11:33:08.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:08.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032917+0000", 2024-09-18T11:33:08.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1719999999999996e-06 2024-09-18T11:33:08.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032920+0000", 2024-09-18T11:33:08.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9270000000000001e-06 2024-09-18T11:33:08.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:33:08.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9938e-05 2024-09-18T11:33:08.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:33:08.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032957+0000", 2024-09-18T11:33:08.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7789999999999996e-06 2024-09-18T11:33:08.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.038135+0000", 2024-09-18T11:33:08.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051778789999999998 2024-09-18T11:33:08.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:08.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.038219+0000", 2024-09-18T11:33:08.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4047000000000004e-05 2024-09-18T11:33:08.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:08.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.038413+0000", 2024-09-18T11:33:08.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019343600000000001 2024-09-18T11:33:08.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:08.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041104+0000", 2024-09-18T11:33:08.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026916190000000001 2024-09-18T11:33:08.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:08.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041256+0000", 2024-09-18T11:33:08.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015182100000000001 2024-09-18T11:33:08.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:08.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041300+0000", 2024-09-18T11:33:08.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3915999999999998e-05 2024-09-18T11:33:08.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:08.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041339+0000", 2024-09-18T11:33:08.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8510000000000002e-05 2024-09-18T11:33:08.542 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.542 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:08.542 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.542 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4739 4.3 4:c70c3162:::smithi11939506-35:head [setxattr _header (57) in=7b,truncate 2974905] snapc 7c=[7c,7b,79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:33:08.542 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.032924+0000", 2024-09-18T11:33:08.542 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.484514351000001, 2024-09-18T11:33:08.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085165610000000006, 2024-09-18T11:33:08.542 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:08.543 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.543 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:08.543 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:08.543 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:08.543 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4739 2024-09-18T11:33:08.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.543 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:08.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:08.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032924+0000", 2024-09-18T11:33:08.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:08.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032924+0000", 2024-09-18T11:33:08.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:08.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032924+0000", 2024-09-18T11:33:08.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:08.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:08.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032926+0000", 2024-09-18T11:33:08.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0619999999999999e-06 2024-09-18T11:33:08.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:08.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032935+0000", 2024-09-18T11:33:08.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.371e-06 2024-09-18T11:33:08.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032937+0000", 2024-09-18T11:33:08.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6769999999999999e-06 2024-09-18T11:33:08.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032979+0000", 2024-09-18T11:33:08.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1810000000000001e-05 2024-09-18T11:33:08.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:33:08.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032984+0000", 2024-09-18T11:33:08.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1660000000000002e-06 2024-09-18T11:33:08.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039042+0000", 2024-09-18T11:33:08.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060573399999999996 2024-09-18T11:33:08.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:08.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039127+0000", 2024-09-18T11:33:08.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5494999999999994e-05 2024-09-18T11:33:08.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:08.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039293+0000", 2024-09-18T11:33:08.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016590999999999999 2024-09-18T11:33:08.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:08.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041146+0000", 2024-09-18T11:33:08.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018531369999999999 2024-09-18T11:33:08.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:08.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041345+0000", 2024-09-18T11:33:08.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000199218 2024-09-18T11:33:08.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:08.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041390+0000", 2024-09-18T11:33:08.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4892000000000003e-05 2024-09-18T11:33:08.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.549 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:08.549 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041441+0000", 2024-09-18T11:33:08.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0815000000000001e-05 2024-09-18T11:33:08.550 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.550 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:08.550 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.550 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4740 4.3 4:c70c3162:::smithi11939506-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:33:08.550 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:33:08.550 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.484488689999999, 2024-09-18T11:33:08.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0087685269999999999, 2024-09-18T11:33:08.550 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:08.550 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:08.550 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:08.550 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:08.551 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:08.551 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4740 2024-09-18T11:33:08.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.551 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:08.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:08.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:33:08.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:08.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:33:08.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:08.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:33:08.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:08.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:08.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032952+0000", 2024-09-18T11:33:08.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.215e-06 2024-09-18T11:33:08.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:08.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032956+0000", 2024-09-18T11:33:08.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8850000000000001e-06 2024-09-18T11:33:08.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032958+0000", 2024-09-18T11:33:08.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1840000000000002e-06 2024-09-18T11:33:08.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032997+0000", 2024-09-18T11:33:08.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9161e-05 2024-09-18T11:33:08.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:33:08.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.033000+0000", 2024-09-18T11:33:08.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1030000000000002e-06 2024-09-18T11:33:08.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039536+0000", 2024-09-18T11:33:08.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006535201 2024-09-18T11:33:08.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:08.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039616+0000", 2024-09-18T11:33:08.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0684000000000002e-05 2024-09-18T11:33:08.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041536+0000", 2024-09-18T11:33:08.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019196440000000001 2024-09-18T11:33:08.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:08.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041610+0000", 2024-09-18T11:33:08.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4295999999999995e-05 2024-09-18T11:33:08.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.555 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:08.555 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041719+0000", 2024-09-18T11:33:08.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108465 2024-09-18T11:33:08.556 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.556 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:08.556 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.556 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:4756 4.7 e179/174 4:e301aefd:::smithi11939506-19:head v 179'405, mlcod=173'401)", 2024-09-18T11:33:08.556 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:50.039068+0000", 2024-09-18T11:33:08.556 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.478370591000001, 2024-09-18T11:33:08.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055292249999999996, 2024-09-18T11:33:08.556 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:08.556 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.556 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:08.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:08.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.039068+0000", 2024-09-18T11:33:08.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:08.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.039068+0000", 2024-09-18T11:33:08.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:08.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.039068+0000", 2024-09-18T11:33:08.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:08.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:08.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040038+0000", 2024-09-18T11:33:08.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096998500000000003 2024-09-18T11:33:08.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:08.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040053+0000", 2024-09-18T11:33:08.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5481999999999998e-05 2024-09-18T11:33:08.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040062+0000", 2024-09-18T11:33:08.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7080000000000003e-06 2024-09-18T11:33:08.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040109+0000", 2024-09-18T11:33:08.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6752000000000002e-05 2024-09-18T11:33:08.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:08.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040124+0000", 2024-09-18T11:33:08.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5715000000000001e-05 2024-09-18T11:33:08.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:08.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044568+0000", 2024-09-18T11:33:08.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044430600000000004 2024-09-18T11:33:08.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:08.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044597+0000", 2024-09-18T11:33:08.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9887999999999999e-05 2024-09-18T11:33:08.560 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.560 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:08.560 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.560 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:4757 4.7 e179/174 4:e301aefd:::smithi11939506-19:head v 179'406, mlcod=173'401)", 2024-09-18T11:33:08.560 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:50.044255+0000", 2024-09-18T11:33:08.561 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.473184029999999, 2024-09-18T11:33:08.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002073166, 2024-09-18T11:33:08.561 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:08.561 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.561 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:08.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:08.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044255+0000", 2024-09-18T11:33:08.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:08.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044255+0000", 2024-09-18T11:33:08.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:08.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044254+0000", 2024-09-18T11:33:08.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:08.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.562 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:08.562 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044875+0000", 2024-09-18T11:33:08.562 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00062136400000000001 2024-09-18T11:33:08.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:08.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044886+0000", 2024-09-18T11:33:08.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0709e-05 2024-09-18T11:33:08.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044891+0000", 2024-09-18T11:33:08.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8019999999999999e-06 2024-09-18T11:33:08.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.563 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.563 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044911+0000", 2024-09-18T11:33:08.563 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0729000000000001e-05 2024-09-18T11:33:08.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:08.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044930+0000", 2024-09-18T11:33:08.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8890999999999999e-05 2024-09-18T11:33:08.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.564 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:08.564 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046295+0000", 2024-09-18T11:33:08.564 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013645420000000001 2024-09-18T11:33:08.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.565 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:08.565 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046328+0000", 2024-09-18T11:33:08.565 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3089000000000001e-05 2024-09-18T11:33:08.565 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.565 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:08.565 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.565 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:4764 3.3 e179/174 3:ec4378e2:::259b372f30a46a47845d5d9c58303dcbeccaeb2db18e8ed0152f1c66341c4df2:head v 179'156, mlcod=172'154)", 2024-09-18T11:33:08.566 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:50.059675+0000", 2024-09-18T11:33:08.566 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.457763677999999, 2024-09-18T11:33:08.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056090439999999997, 2024-09-18T11:33:08.566 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:08.566 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:08.566 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:08.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:08.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.059675+0000", 2024-09-18T11:33:08.566 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.566 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:08.566 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.059675+0000", 2024-09-18T11:33:08.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:08.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.059675+0000", 2024-09-18T11:33:08.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:08.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:08.567 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062252+0000", 2024-09-18T11:33:08.567 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025772859999999998 2024-09-18T11:33:08.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.567 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:08.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062265+0000", 2024-09-18T11:33:08.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3059e-05 2024-09-18T11:33:08.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062272+0000", 2024-09-18T11:33:08.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3589999999999996e-06 2024-09-18T11:33:08.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.568 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.568 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062317+0000", 2024-09-18T11:33:08.568 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5188999999999997e-05 2024-09-18T11:33:08.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:08.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062334+0000", 2024-09-18T11:33:08.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.73e-05 2024-09-18T11:33:08.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:08.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.065251+0000", 2024-09-18T11:33:08.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029169539999999998 2024-09-18T11:33:08.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.569 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:08.569 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.065284+0000", 2024-09-18T11:33:08.569 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3206e-05 2024-09-18T11:33:08.569 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.570 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:08.570 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.570 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4792 4.6 4:67c1765a:::smithi11939506-30:head [sparse-read 0~1153827,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-18T11:33:08.570 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.053543+0000", 2024-09-18T11:33:08.570 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.463895957, 2024-09-18T11:33:08.570 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003529337, 2024-09-18T11:33:08.571 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:08.571 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:08.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:08.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:08.571 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:08.571 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4792 2024-09-18T11:33:08.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.571 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:08.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.571 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:08.571 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053543+0000", 2024-09-18T11:33:08.571 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:08.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053543+0000", 2024-09-18T11:33:08.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:08.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053541+0000", 2024-09-18T11:33:08.572 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:08.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.572 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.572 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:08.572 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053550+0000", 2024-09-18T11:33:08.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2070000000000001e-06 2024-09-18T11:33:08.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:08.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053566+0000", 2024-09-18T11:33:08.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6098000000000001e-05 2024-09-18T11:33:08.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.573 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053575+0000", 2024-09-18T11:33:08.573 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3769999999999996e-06 2024-09-18T11:33:08.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.573 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.573 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053620+0000", 2024-09-18T11:33:08.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5404000000000001e-05 2024-09-18T11:33:08.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:33:08.574 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053634+0000", 2024-09-18T11:33:08.574 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4161e-05 2024-09-18T11:33:08.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.574 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.055502+0000", 2024-09-18T11:33:08.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018678810000000001 2024-09-18T11:33:08.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:08.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.055629+0000", 2024-09-18T11:33:08.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012652 2024-09-18T11:33:08.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.575 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.575 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:08.575 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057072+0000", 2024-09-18T11:33:08.575 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001443567 2024-09-18T11:33:08.576 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.576 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:08.576 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.576 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.576 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4793 4.6 4:67c1765a:::smithi11939506-30:head [sparse-read 0~1153827] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-18T11:33:08.576 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.053593+0000", 2024-09-18T11:33:08.576 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.463846091000001, 2024-09-18T11:33:08.576 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037961539999999999, 2024-09-18T11:33:08.576 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:08.576 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:08.576 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:08.577 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:08.577 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:08.577 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4793 2024-09-18T11:33:08.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.577 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:08.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:08.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053593+0000", 2024-09-18T11:33:08.577 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.577 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.577 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:08.577 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053593+0000", 2024-09-18T11:33:08.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:08.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053592+0000", 2024-09-18T11:33:08.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:08.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.578 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:08.578 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053595+0000", 2024-09-18T11:33:08.578 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2179999999999999e-06 2024-09-18T11:33:08.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.578 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:08.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053603+0000", 2024-09-18T11:33:08.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.576e-06 2024-09-18T11:33:08.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053607+0000", 2024-09-18T11:33:08.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2309999999999998e-06 2024-09-18T11:33:08.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.579 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.579 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053696+0000", 2024-09-18T11:33:08.579 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9130999999999998e-05 2024-09-18T11:33:08.579 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:08.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053702+0000", 2024-09-18T11:33:08.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6370000000000004e-06 2024-09-18T11:33:08.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057115+0000", 2024-09-18T11:33:08.580 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034128100000000001 2024-09-18T11:33:08.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.580 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:08.580 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057176+0000", 2024-09-18T11:33:08.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.143e-05 2024-09-18T11:33:08.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.581 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:08.581 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057389+0000", 2024-09-18T11:33:08.581 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021267700000000001 2024-09-18T11:33:08.581 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.581 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:08.581 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.581 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4794 4.6 4:67c1765a:::smithi11939506-30:head [sparse-read 0~1153827] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-18T11:33:08.581 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.053613+0000", 2024-09-18T11:33:08.582 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 15.463825839, 2024-09-18T11:33:08.582 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040685349999999999, 2024-09-18T11:33:08.582 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:08.582 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:08.582 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:08.582 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:08.582 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:08.582 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4794 2024-09-18T11:33:08.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.582 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:08.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:08.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053613+0000", 2024-09-18T11:33:08.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.583 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:08.583 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053613+0000", 2024-09-18T11:33:08.583 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:08.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053612+0000", 2024-09-18T11:33:08.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:08.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:08.584 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053615+0000", 2024-09-18T11:33:08.584 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5730000000000002e-06 2024-09-18T11:33:08.584 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.584 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.584 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:08.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053628+0000", 2024-09-18T11:33:08.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3084e-05 2024-09-18T11:33:08.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053632+0000", 2024-09-18T11:33:08.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9620000000000004e-06 2024-09-18T11:33:08.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.585 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.585 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053731+0000", 2024-09-18T11:33:08.585 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8547000000000004e-05 2024-09-18T11:33:08.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:08.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053736+0000", 2024-09-18T11:33:08.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.767e-06 2024-09-18T11:33:08.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057461+0000", 2024-09-18T11:33:08.586 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037242780000000001 2024-09-18T11:33:08.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.586 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:08.586 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057538+0000", 2024-09-18T11:33:08.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7008999999999994e-05 2024-09-18T11:33:08.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.587 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:08.587 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057681+0000", 2024-09-18T11:33:08.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000143786 2024-09-18T11:33:08.587 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.587 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:08.587 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.587 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e301aefd:::smithi11939506-19:3f)])", 2024-09-18T11:33:08.587 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.083144+0000", 2024-09-18T11:33:08.587 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.43429523, 2024-09-18T11:33:08.587 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024893160000000001, 2024-09-18T11:33:08.588 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:08.588 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:08.588 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:08.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:08.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.083144+0000", 2024-09-18T11:33:08.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.588 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:08.588 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.083144+0000", 2024-09-18T11:33:08.588 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:08.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.083143+0000", 2024-09-18T11:33:08.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:08.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:08.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.083149+0000", 2024-09-18T11:33:08.589 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0549999999999996e-06 2024-09-18T11:33:08.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.589 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:08.589 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.083161+0000", 2024-09-18T11:33:08.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2668e-05 2024-09-18T11:33:08.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.083169+0000", 2024-09-18T11:33:08.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2830000000000004e-06 2024-09-18T11:33:08.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.590 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.083208+0000", 2024-09-18T11:33:08.590 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9749999999999997e-05 2024-09-18T11:33:08.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.590 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:08.591 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.085633+0000", 2024-09-18T11:33:08.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024244639999999999 2024-09-18T11:33:08.591 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.591 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:08.591 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.591 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e0d95e10:::smithi11939506-22:49)])", 2024-09-18T11:33:08.591 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.102811+0000", 2024-09-18T11:33:08.591 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4146281179999995, 2024-09-18T11:33:08.591 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00260509, 2024-09-18T11:33:08.591 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:08.591 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:08.591 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:08.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:08.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.102811+0000", 2024-09-18T11:33:08.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:08.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.102811+0000", 2024-09-18T11:33:08.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.592 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:08.592 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.102809+0000", 2024-09-18T11:33:08.592 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:08.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:08.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.102818+0000", 2024-09-18T11:33:08.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4089999999999992e-06 2024-09-18T11:33:08.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:08.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.102836+0000", 2024-09-18T11:33:08.593 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7881999999999999e-05 2024-09-18T11:33:08.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.593 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.593 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.102844+0000", 2024-09-18T11:33:08.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8900000000000007e-06 2024-09-18T11:33:08.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.102892+0000", 2024-09-18T11:33:08.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7756999999999997e-05 2024-09-18T11:33:08.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.594 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:08.594 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.105416+0000", 2024-09-18T11:33:08.594 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025237710000000002 2024-09-18T11:33:08.594 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.594 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:08.594 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.595 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e73ba68f:::smithi11939506-1:42)])", 2024-09-18T11:33:08.595 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.114167+0000", 2024-09-18T11:33:08.595 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.4032722129999993, 2024-09-18T11:33:08.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002907303, 2024-09-18T11:33:08.595 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:08.595 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:08.595 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:08.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:08.595 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.114167+0000", 2024-09-18T11:33:08.595 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.595 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:08.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.114167+0000", 2024-09-18T11:33:08.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:08.596 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.114165+0000", 2024-09-18T11:33:08.596 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:08.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.596 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:08.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.114174+0000", 2024-09-18T11:33:08.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4419999999999998e-06 2024-09-18T11:33:08.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:08.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.114192+0000", 2024-09-18T11:33:08.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8346e-05 2024-09-18T11:33:08.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.597 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.597 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.114200+0000", 2024-09-18T11:33:08.597 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9680000000000006e-06 2024-09-18T11:33:08.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.114249+0000", 2024-09-18T11:33:08.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8974000000000002e-05 2024-09-18T11:33:08.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.598 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:08.598 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.117074+0000", 2024-09-18T11:33:08.598 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028247530000000002 2024-09-18T11:33:08.598 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.598 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:08.599 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.599 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e5707c18:::smithi11939506-10:5d)])", 2024-09-18T11:33:08.599 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.126498+0000", 2024-09-18T11:33:08.599 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.3909409359999998, 2024-09-18T11:33:08.599 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0033596170000000001, 2024-09-18T11:33:08.599 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:08.599 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:08.599 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:08.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.599 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:08.599 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.126498+0000", 2024-09-18T11:33:08.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:08.600 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.126498+0000", 2024-09-18T11:33:08.600 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.600 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:08.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.126497+0000", 2024-09-18T11:33:08.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:08.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:08.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.126508+0000", 2024-09-18T11:33:08.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1898e-05 2024-09-18T11:33:08.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.601 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.601 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:08.601 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.126525+0000", 2024-09-18T11:33:08.601 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6761000000000001e-05 2024-09-18T11:33:08.601 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.126533+0000", 2024-09-18T11:33:08.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9629999999999998e-06 2024-09-18T11:33:08.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.602 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.126581+0000", 2024-09-18T11:33:08.602 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7787000000000002e-05 2024-09-18T11:33:08.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.602 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.602 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:08.603 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.129857+0000", 2024-09-18T11:33:08.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003276476 2024-09-18T11:33:08.603 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.603 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:08.603 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.603 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.603 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e73ba68f:::smithi11939506-1:5d)])", 2024-09-18T11:33:08.603 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.141236+0000", 2024-09-18T11:33:08.603 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.3762023750000001, 2024-09-18T11:33:08.603 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002457187, 2024-09-18T11:33:08.603 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:08.603 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:08.603 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:08.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:08.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.141236+0000", 2024-09-18T11:33:08.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:08.604 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.141236+0000", 2024-09-18T11:33:08.604 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.604 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.604 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:08.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.141235+0000", 2024-09-18T11:33:08.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:08.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:08.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.141248+0000", 2024-09-18T11:33:08.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3131e-05 2024-09-18T11:33:08.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.605 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.605 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:08.605 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.141268+0000", 2024-09-18T11:33:08.605 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0689000000000001e-05 2024-09-18T11:33:08.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.141277+0000", 2024-09-18T11:33:08.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7280000000000001e-06 2024-09-18T11:33:08.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.141331+0000", 2024-09-18T11:33:08.606 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4098000000000003e-05 2024-09-18T11:33:08.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.606 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.606 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:08.606 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.143694+0000", 2024-09-18T11:33:08.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023623810000000002 2024-09-18T11:33:08.607 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.607 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:08.607 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.607 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.607 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.607 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:ef60946b:::smithi11939506-8:head)])", 2024-09-18T11:33:08.607 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.151354+0000", 2024-09-18T11:33:08.607 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.3660843440000008, 2024-09-18T11:33:08.607 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026499959999999999, 2024-09-18T11:33:08.607 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:08.607 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:08.608 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:08.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:08.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.151354+0000", 2024-09-18T11:33:08.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.608 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:08.608 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.151354+0000", 2024-09-18T11:33:08.608 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.608 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:08.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.151354+0000", 2024-09-18T11:33:08.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:08.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:08.609 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.151360+0000", 2024-09-18T11:33:08.609 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8120000000000002e-06 2024-09-18T11:33:08.609 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.609 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.609 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:08.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.151371+0000", 2024-09-18T11:33:08.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0324e-05 2024-09-18T11:33:08.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.151375+0000", 2024-09-18T11:33:08.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7049999999999998e-06 2024-09-18T11:33:08.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.610 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.610 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.151420+0000", 2024-09-18T11:33:08.610 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4496000000000002e-05 2024-09-18T11:33:08.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.610 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.611 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:08.611 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.154004+0000", 2024-09-18T11:33:08.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025845579999999998 2024-09-18T11:33:08.611 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.611 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:08.611 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.611 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.611 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e0d95e10:::smithi11939506-22:74)])", 2024-09-18T11:33:08.611 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.169218+0000", 2024-09-18T11:33:08.611 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.3482209639999994, 2024-09-18T11:33:08.611 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023236060000000002, 2024-09-18T11:33:08.611 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:08.611 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:08.611 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:08.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:08.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.169218+0000", 2024-09-18T11:33:08.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:08.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.169218+0000", 2024-09-18T11:33:08.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.612 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:08.612 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.169217+0000", 2024-09-18T11:33:08.612 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:08.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:08.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.169224+0000", 2024-09-18T11:33:08.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9129999999999997e-06 2024-09-18T11:33:08.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.613 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:08.613 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.169235+0000", 2024-09-18T11:33:08.613 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1032e-05 2024-09-18T11:33:08.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.613 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.169242+0000", 2024-09-18T11:33:08.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8589999999999999e-06 2024-09-18T11:33:08.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.614 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.169284+0000", 2024-09-18T11:33:08.614 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2293000000000003e-05 2024-09-18T11:33:08.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.614 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.614 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:08.615 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.171541+0000", 2024-09-18T11:33:08.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022575939999999999 2024-09-18T11:33:08.615 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.615 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:08.615 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.615 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.615 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e301aefd:::smithi11939506-19:7e)])", 2024-09-18T11:33:08.615 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.179260+0000", 2024-09-18T11:33:08.615 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.3381788120000007, 2024-09-18T11:33:08.615 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025394670000000001, 2024-09-18T11:33:08.615 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:08.615 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:08.616 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:08.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:08.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.179260+0000", 2024-09-18T11:33:08.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:08.616 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.179260+0000", 2024-09-18T11:33:08.616 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.616 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.616 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:08.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.179259+0000", 2024-09-18T11:33:08.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:08.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:08.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.179265+0000", 2024-09-18T11:33:08.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1879999999999999e-06 2024-09-18T11:33:08.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.617 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:08.617 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.179276+0000", 2024-09-18T11:33:08.617 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1070000000000001e-05 2024-09-18T11:33:08.617 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.617 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.179283+0000", 2024-09-18T11:33:08.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6059999999999999e-06 2024-09-18T11:33:08.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.179320+0000", 2024-09-18T11:33:08.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6847999999999997e-05 2024-09-18T11:33:08.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.618 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.618 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:08.618 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.181799+0000", 2024-09-18T11:33:08.618 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002479738 2024-09-18T11:33:08.618 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.619 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:08.619 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.619 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e73ba68f:::smithi11939506-1:head)])", 2024-09-18T11:33:08.619 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.201696+0000", 2024-09-18T11:33:08.619 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 9.3157424940000002, 2024-09-18T11:33:08.619 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026547110000000001, 2024-09-18T11:33:08.619 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:08.619 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:08.619 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:08.619 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.619 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:08.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.201696+0000", 2024-09-18T11:33:08.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:08.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.201696+0000", 2024-09-18T11:33:08.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:08.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.620 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.620 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:08.620 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.201695+0000", 2024-09-18T11:33:08.620 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:08.620 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:08.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.201704+0000", 2024-09-18T11:33:08.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5779999999999999e-06 2024-09-18T11:33:08.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:08.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.201717+0000", 2024-09-18T11:33:08.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3019e-05 2024-09-18T11:33:08.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.621 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:08.621 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.201723+0000", 2024-09-18T11:33:08.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2299999999999996e-06 2024-09-18T11:33:08.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:08.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.201771+0000", 2024-09-18T11:33:08.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7652999999999999e-05 2024-09-18T11:33:08.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:08.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:08.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:08.622 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.204351+0000", 2024-09-18T11:33:08.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025802009999999998 2024-09-18T11:33:08.622 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.623 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:08.623 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.623 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:08.623 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:08.623 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:08.624 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:33:08.625 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:33:08.623+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:33:08.719 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:33:08.724 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:08.724 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:33:08.724 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:33:08.724 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:08.725 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:33:08.724+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:33:08.733 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:33:08.826 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:08.825+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:33:08.871 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:08.872 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:33:08.872 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:33:08.872 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:33:08.872 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:08.880 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:33:08.965 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:08.964+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:33:09.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:33:09.015+0000 7fd17affd640 0 --2- 172.21.15.119:0/3720245688 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd17c0b1350 0x7fd17c13dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-18T11:33:09.022 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:09.022 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:33:09.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:33:09.022 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:33:09.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.022 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:262 4.9 e53/22 4:99feffe9:::smithi11939506-45:13 v 53'17, mlcod=51'14)", 2024-09-18T11:33:09.022 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:33:09.022 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 155.02067578200001, 2024-09-18T11:33:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013878679, 2024-09-18T11:33:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:33:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:33:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:33:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998997+0000", 2024-09-18T11:33:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1099999999999999e-06 2024-09-18T11:33:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999008+0000", 2024-09-18T11:33:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1321000000000001e-05 2024-09-18T11:33:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999015+0000", 2024-09-18T11:33:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3000000000000004e-06 2024-09-18T11:33:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999055+0000", 2024-09-18T11:33:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0058000000000002e-05 2024-09-18T11:33:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999070+0000", 2024-09-18T11:33:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4328000000000001e-05 2024-09-18T11:33:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012465+0000", 2024-09-18T11:33:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013395638 2024-09-18T11:33:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012870+0000", 2024-09-18T11:33:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040423400000000001 2024-09-18T11:33:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:205 4.b e53/22 4:dba5bd39:::smithi11939506-17:13 v 53'77, mlcod=51'74)", 2024-09-18T11:33:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:33:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 155.020626521, 2024-09-18T11:33:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013359586, 2024-09-18T11:33:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:33:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:33:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:33:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:09.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999048+0000", 2024-09-18T11:33:09.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3550000000000005e-06 2024-09-18T11:33:09.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:09.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999056+0000", 2024-09-18T11:33:09.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3259999999999996e-06 2024-09-18T11:33:09.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:09.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999063+0000", 2024-09-18T11:33:09.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6409999999999996e-06 2024-09-18T11:33:09.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999098+0000", 2024-09-18T11:33:09.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5203999999999997e-05 2024-09-18T11:33:09.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:09.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999115+0000", 2024-09-18T11:33:09.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6478999999999999e-05 2024-09-18T11:33:09.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:09.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012362+0000", 2024-09-18T11:33:09.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01324768 2024-09-18T11:33:09.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:09.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012400+0000", 2024-09-18T11:33:09.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.731e-05 2024-09-18T11:33:09.031 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.031 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:09.031 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.031 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2937 4.7 4:e301aefd:::smithi11939506-19:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:33:09.032 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:33:09.032 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 105.984826208, 2024-09-18T11:33:09.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019287667000000001, 2024-09-18T11:33:09.032 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:09.032 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:09.032 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:09.032 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:09.032 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:09.032 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2937 2024-09-18T11:33:09.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.032 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:09.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:09.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:33:09.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:09.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:33:09.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:09.033 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:33:09.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:33:09.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:09.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034842+0000", 2024-09-18T11:33:09.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.372e-06 2024-09-18T11:33:09.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:09.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034846+0000", 2024-09-18T11:33:09.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3919999999999996e-06 2024-09-18T11:33:09.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:09.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034848+0000", 2024-09-18T11:33:09.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3539999999999998e-06 2024-09-18T11:33:09.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034893+0000", 2024-09-18T11:33:09.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4861000000000003e-05 2024-09-18T11:33:09.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:09.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034965+0000", 2024-09-18T11:33:09.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1860999999999995e-05 2024-09-18T11:33:09.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:09.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.054128+0000", 2024-09-18T11:33:09.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019163039 2024-09-18T11:33:09.036 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.036 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:09.036 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.036 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3453 4.f 4:f5d7df41:::smithi11939506-16:head [write 1813152~490434] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:33:09.036 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:33:09.037 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 82.996914279999999, 2024-09-18T11:33:09.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050685840000000003, 2024-09-18T11:33:09.037 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:09.037 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:09.037 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:09.037 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:09.037 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:09.037 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3453 2024-09-18T11:33:09.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.037 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:09.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:09.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:33:09.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:09.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:33:09.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:09.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022751+0000", 2024-09-18T11:33:09.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:09.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:09.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022871+0000", 2024-09-18T11:33:09.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000119781 2024-09-18T11:33:09.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:09.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022882+0000", 2024-09-18T11:33:09.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1627000000000001e-05 2024-09-18T11:33:09.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:09.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022887+0000", 2024-09-18T11:33:09.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2390000000000004e-06 2024-09-18T11:33:09.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023123+0000", 2024-09-18T11:33:09.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000236598 2024-09-18T11:33:09.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:09.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023207+0000", 2024-09-18T11:33:09.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4303999999999998e-05 2024-09-18T11:33:09.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:09.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023367+0000", 2024-09-18T11:33:09.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015983600000000001 2024-09-18T11:33:09.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:09.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073237+0000", 2024-09-18T11:33:09.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049869400000000001 2024-09-18T11:33:09.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.043 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:09.043 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073352+0000", 2024-09-18T11:33:09.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011579899999999999 2024-09-18T11:33:09.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:09.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073394+0000", 2024-09-18T11:33:09.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1445e-05 2024-09-18T11:33:09.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:09.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073438+0000", 2024-09-18T11:33:09.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4261999999999997e-05 2024-09-18T11:33:09.044 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.044 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:09.044 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.045 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3454 4.f 4:f5d7df41:::smithi11939506-16:head [setxattr _header (57) in=7b,truncate 2991679] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:33:09.045 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:33:09.045 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 82.996215837999998, 2024-09-18T11:33:09.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050365656000000002, 2024-09-18T11:33:09.045 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:09.045 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:09.045 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:09.045 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:09.045 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:09.045 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3454 2024-09-18T11:33:09.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.045 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:09.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:09.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:33:09.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:09.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:33:09.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:09.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023450+0000", 2024-09-18T11:33:09.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:09.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:09.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023456+0000", 2024-09-18T11:33:09.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8329999999999999e-06 2024-09-18T11:33:09.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:09.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023465+0000", 2024-09-18T11:33:09.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1130000000000007e-06 2024-09-18T11:33:09.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:09.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023470+0000", 2024-09-18T11:33:09.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9139999999999999e-06 2024-09-18T11:33:09.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072611+0000", 2024-09-18T11:33:09.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141643999999998 2024-09-18T11:33:09.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:09.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072729+0000", 2024-09-18T11:33:09.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117667 2024-09-18T11:33:09.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:09.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072887+0000", 2024-09-18T11:33:09.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015760000000000001 2024-09-18T11:33:09.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:09.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073558+0000", 2024-09-18T11:33:09.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00067171699999999995 2024-09-18T11:33:09.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:09.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073764+0000", 2024-09-18T11:33:09.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000205749 2024-09-18T11:33:09.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:09.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073790+0000", 2024-09-18T11:33:09.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5643000000000002e-05 2024-09-18T11:33:09.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:09.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073816+0000", 2024-09-18T11:33:09.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6588e-05 2024-09-18T11:33:09.050 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.050 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:09.050 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.050 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3455 4.f 4:f5d7df41:::smithi11939506-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:33:09.050 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:33:09.050 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 82.996186219999998, 2024-09-18T11:33:09.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050460466000000002, 2024-09-18T11:33:09.051 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:09.051 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:09.051 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:09.051 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:09.051 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:09.051 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3455 2024-09-18T11:33:09.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.051 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:09.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:09.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:33:09.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:09.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:33:09.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:09.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:33:09.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:09.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:09.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023483+0000", 2024-09-18T11:33:09.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.695e-06 2024-09-18T11:33:09.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:09.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023489+0000", 2024-09-18T11:33:09.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1079999999999998e-06 2024-09-18T11:33:09.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:09.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023492+0000", 2024-09-18T11:33:09.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.478e-06 2024-09-18T11:33:09.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073273+0000", 2024-09-18T11:33:09.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049780638000000002 2024-09-18T11:33:09.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:09.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073343+0000", 2024-09-18T11:33:09.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9770000000000005e-05 2024-09-18T11:33:09.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073846+0000", 2024-09-18T11:33:09.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050289499999999995 2024-09-18T11:33:09.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:09.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073886+0000", 2024-09-18T11:33:09.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0658999999999998e-05 2024-09-18T11:33:09.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:09.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073941+0000", 2024-09-18T11:33:09.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4642000000000003e-05 2024-09-18T11:33:09.055 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.055 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:09.055 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.055 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3456 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:33:09.055 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:33:09.055 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 82.996161154999996, 2024-09-18T11:33:09.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049859120999999999, 2024-09-18T11:33:09.056 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:09.056 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:09.056 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:09.056 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:09.056 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:09.056 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3456 2024-09-18T11:33:09.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.056 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:09.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:09.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:33:09.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:09.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:33:09.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:09.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:33:09.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:09.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:09.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023508+0000", 2024-09-18T11:33:09.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9490000000000001e-06 2024-09-18T11:33:09.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:09.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023513+0000", 2024-09-18T11:33:09.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3800000000000002e-06 2024-09-18T11:33:09.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:09.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023517+0000", 2024-09-18T11:33:09.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5300000000000001e-06 2024-09-18T11:33:09.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073262+0000", 2024-09-18T11:33:09.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049745194999999999 2024-09-18T11:33:09.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:09.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073365+0000", 2024-09-18T11:33:09.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102462 2024-09-18T11:33:09.059 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.059 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:09.059 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.060 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3457 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:33:09.060 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:33:09.060 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 82.996144059000002, 2024-09-18T11:33:09.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049928567, 2024-09-18T11:33:09.060 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:09.060 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:09.060 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:09.060 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:09.060 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:09.060 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3457 2024-09-18T11:33:09.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.060 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:09.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:09.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:33:09.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:09.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:33:09.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:09.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023522+0000", 2024-09-18T11:33:09.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:09.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:09.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023524+0000", 2024-09-18T11:33:09.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8500000000000001e-06 2024-09-18T11:33:09.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:09.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023529+0000", 2024-09-18T11:33:09.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7509999999999999e-06 2024-09-18T11:33:09.062 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.062 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.062 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:09.062 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023532+0000", 2024-09-18T11:33:09.062 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.163e-06 2024-09-18T11:33:09.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073386+0000", 2024-09-18T11:33:09.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049853899 2024-09-18T11:33:09.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.063 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.063 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:09.063 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073451+0000", 2024-09-18T11:33:09.063 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5197000000000006e-05 2024-09-18T11:33:09.063 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.063 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:09.063 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.064 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.064 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3458 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:33:09.064 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:33:09.064 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 82.996131501999997, 2024-09-18T11:33:09.064 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.05001659, 2024-09-18T11:33:09.064 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:09.064 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:09.064 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:09.064 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:09.064 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:09.064 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3458 2024-09-18T11:33:09.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.064 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:09.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:09.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:33:09.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:09.065 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:33:09.065 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.065 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.065 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:09.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:33:09.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:09.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:09.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023537+0000", 2024-09-18T11:33:09.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.64e-06 2024-09-18T11:33:09.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.066 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.066 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:09.066 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023543+0000", 2024-09-18T11:33:09.066 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3369999999999996e-06 2024-09-18T11:33:09.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:09.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023546+0000", 2024-09-18T11:33:09.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:33:09.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.067 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073466+0000", 2024-09-18T11:33:09.067 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049919719000000001 2024-09-18T11:33:09.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.067 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.067 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:09.068 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073552+0000", 2024-09-18T11:33:09.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6168999999999994e-05 2024-09-18T11:33:09.068 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.068 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:09.068 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.068 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.068 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(client.4590.0:3453 4.f e116/22 ondisk, result = 0)", 2024-09-18T11:33:09.068 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:33:09.068 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 82.994526418000007, 2024-09-18T11:33:09.068 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048099441, 2024-09-18T11:33:09.068 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:09.069 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:09.069 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:09.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:09.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:33:09.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:09.069 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:33:09.069 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.069 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.069 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:09.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025139+0000", 2024-09-18T11:33:09.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:09.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:09.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025144+0000", 2024-09-18T11:33:09.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4059999999999998e-06 2024-09-18T11:33:09.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.070 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.070 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:09.070 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072212+0000", 2024-09-18T11:33:09.070 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047068482000000002 2024-09-18T11:33:09.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:09.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072230+0000", 2024-09-18T11:33:09.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7632999999999999e-05 2024-09-18T11:33:09.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073224+0000", 2024-09-18T11:33:09.071 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099436899999999994 2024-09-18T11:33:09.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.071 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.071 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:09.071 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073231+0000", 2024-09-18T11:33:09.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-09-18T11:33:09.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.072 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:09.072 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073240+0000", 2024-09-18T11:33:09.072 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5699999999999993e-06 2024-09-18T11:33:09.072 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.072 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:09.072 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.072 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.072 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3930 4.b 4:d3ae8215:::smithi11939506-41:head [write 632029~547288] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:33:09.072 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:33:09.073 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 65.993610738000001, 2024-09-18T11:33:09.073 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018962711, 2024-09-18T11:33:09.073 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:09.073 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:09.073 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:09.073 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:09.073 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:09.073 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3930 2024-09-18T11:33:09.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.073 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:09.073 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.073 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:09.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:33:09.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:09.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:33:09.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:09.074 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026055+0000", 2024-09-18T11:33:09.074 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:09.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.074 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.074 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:09.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026334+0000", 2024-09-18T11:33:09.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000278931 2024-09-18T11:33:09.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:09.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026344+0000", 2024-09-18T11:33:09.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0716999999999999e-05 2024-09-18T11:33:09.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:09.075 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026350+0000", 2024-09-18T11:33:09.075 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7620000000000001e-06 2024-09-18T11:33:09.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.075 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.075 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028429+0000", 2024-09-18T11:33:09.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020787319999999998 2024-09-18T11:33:09.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:09.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028525+0000", 2024-09-18T11:33:09.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6638999999999994e-05 2024-09-18T11:33:09.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.076 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:33:09.076 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028720+0000", 2024-09-18T11:33:09.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000194525 2024-09-18T11:33:09.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:09.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.036996+0000", 2024-09-18T11:33:09.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082756140000000006 2024-09-18T11:33:09.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:09.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044952+0000", 2024-09-18T11:33:09.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566740000000004 2024-09-18T11:33:09.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:09.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044985+0000", 2024-09-18T11:33:09.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3081999999999999e-05 2024-09-18T11:33:09.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:09.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045019+0000", 2024-09-18T11:33:09.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3257999999999999e-05 2024-09-18T11:33:09.078 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.078 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:09.078 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.078 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3932 4.b 4:d3ae8215:::smithi11939506-41:head [write 2796249~437859] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:33:09.078 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:33:09.078 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 65.981604098000005, 2024-09-18T11:33:09.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01458329, 2024-09-18T11:33:09.079 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:09.079 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:09.079 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:09.079 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:09.079 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:09.079 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3932 2024-09-18T11:33:09.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.079 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:09.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:09.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:33:09.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:09.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:33:09.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:09.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038061+0000", 2024-09-18T11:33:09.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:09.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:09.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038341+0000", 2024-09-18T11:33:09.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027989099999999999 2024-09-18T11:33:09.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:09.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038353+0000", 2024-09-18T11:33:09.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2247e-05 2024-09-18T11:33:09.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:09.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038358+0000", 2024-09-18T11:33:09.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-18T11:33:09.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043226+0000", 2024-09-18T11:33:09.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048673049999999997 2024-09-18T11:33:09.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.081 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:09.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043284+0000", 2024-09-18T11:33:09.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8338999999999998e-05 2024-09-18T11:33:09.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:33:09.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043367+0000", 2024-09-18T11:33:09.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3120000000000004e-05 2024-09-18T11:33:09.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:09.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.051544+0000", 2024-09-18T11:33:09.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00817734 2024-09-18T11:33:09.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:09.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052528+0000", 2024-09-18T11:33:09.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098318499999999992 2024-09-18T11:33:09.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:09.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052610+0000", 2024-09-18T11:33:09.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2300999999999997e-05 2024-09-18T11:33:09.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:09.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052646+0000", 2024-09-18T11:33:09.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5955999999999999e-05 2024-09-18T11:33:09.084 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.084 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:09.084 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.084 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3935 4.b 4:d3ae8215:::smithi11939506-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:33:09.084 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:33:09.084 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 65.978266990999998, 2024-09-18T11:33:09.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013142333000000001, 2024-09-18T11:33:09.084 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:09.084 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:09.085 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:09.085 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:09.085 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:09.085 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3935 2024-09-18T11:33:09.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.085 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:09.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:09.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:33:09.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:09.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:33:09.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:09.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041399+0000", 2024-09-18T11:33:09.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:09.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:09.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041401+0000", 2024-09-18T11:33:09.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4890000000000001e-06 2024-09-18T11:33:09.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:09.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041404+0000", 2024-09-18T11:33:09.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5240000000000001e-06 2024-09-18T11:33:09.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:09.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041407+0000", 2024-09-18T11:33:09.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2230000000000001e-06 2024-09-18T11:33:09.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045699+0000", 2024-09-18T11:33:09.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042918970000000002 2024-09-18T11:33:09.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:09.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045750+0000", 2024-09-18T11:33:09.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1332999999999998e-05 2024-09-18T11:33:09.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054438+0000", 2024-09-18T11:33:09.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086882569999999996 2024-09-18T11:33:09.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:09.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054485+0000", 2024-09-18T11:33:09.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7151000000000003e-05 2024-09-18T11:33:09.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:09.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054542+0000", 2024-09-18T11:33:09.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6706999999999998e-05 2024-09-18T11:33:09.089 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.089 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:09.089 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.089 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3969 4.4 4:2cb3e6a5:::smithi11939506-3:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:09.089 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:33:09.089 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.446026725999999, 2024-09-18T11:33:09.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.024248042000000001, 2024-09-18T11:33:09.090 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:09.090 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:09.090 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:09.090 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:09.090 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:09.090 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3969 2024-09-18T11:33:09.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.090 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:09.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:09.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:33:09.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:09.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:33:09.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:09.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573639+0000", 2024-09-18T11:33:09.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:09.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:09.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573646+0000", 2024-09-18T11:33:09.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6569999999999997e-06 2024-09-18T11:33:09.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:09.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573662+0000", 2024-09-18T11:33:09.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5187999999999999e-05 2024-09-18T11:33:09.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:09.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573670+0000", 2024-09-18T11:33:09.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5399999999999996e-06 2024-09-18T11:33:09.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:33:09.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8592999999999999e-05 2024-09-18T11:33:09.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:09.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573818+0000", 2024-09-18T11:33:09.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9388999999999995e-05 2024-09-18T11:33:09.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:09.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597888+0000", 2024-09-18T11:33:09.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.024069859999999998 2024-09-18T11:33:09.093 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.093 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:09.093 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.093 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3973 4.7 4:e73ba68f:::smithi11939506-1:head [tier-promote] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:09.093 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:33:09.094 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 59.445927646999998, 2024-09-18T11:33:09.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.023911297000000001, 2024-09-18T11:33:09.094 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:09.094 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:09.094 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:09.094 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:09.094 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:09.094 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3973 2024-09-18T11:33:09.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.094 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:09.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.094 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:09.094 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:33:09.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:09.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:33:09.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:09.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573738+0000", 2024-09-18T11:33:09.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:09.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:09.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573743+0000", 2024-09-18T11:33:09.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0839999999999999e-06 2024-09-18T11:33:09.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:09.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573752+0000", 2024-09-18T11:33:09.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3080000000000008e-06 2024-09-18T11:33:09.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:09.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573758+0000", 2024-09-18T11:33:09.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2659999999999998e-06 2024-09-18T11:33:09.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573803+0000", 2024-09-18T11:33:09.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5243e-05 2024-09-18T11:33:09.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:09.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573897+0000", 2024-09-18T11:33:09.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3398000000000003e-05 2024-09-18T11:33:09.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:09.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597650+0000", 2024-09-18T11:33:09.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.023753672999999999 2024-09-18T11:33:09.097 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.097 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:09.097 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.098 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4443 4.d 4:b9bb642b:::smithi11939506-27:67 [read 0~3512696,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T11:33:09.098 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:36.030743+0000", 2024-09-18T11:33:09.098 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 32.988924017999999, 2024-09-18T11:33:09.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013696302, 2024-09-18T11:33:09.098 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:09.098 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:09.098 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:09.098 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:09.099 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:09.099 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4443 2024-09-18T11:33:09.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.099 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:09.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:09.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030743+0000", 2024-09-18T11:33:09.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:09.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030743+0000", 2024-09-18T11:33:09.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:09.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030741+0000", 2024-09-18T11:33:09.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:09.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:09.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030751+0000", 2024-09-18T11:33:09.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6979999999999993e-06 2024-09-18T11:33:09.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:09.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030763+0000", 2024-09-18T11:33:09.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2311e-05 2024-09-18T11:33:09.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:09.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030769+0000", 2024-09-18T11:33:09.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6060000000000002e-06 2024-09-18T11:33:09.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.036777+0000", 2024-09-18T11:33:09.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060079089999999996 2024-09-18T11:33:09.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:09.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.036945+0000", 2024-09-18T11:33:09.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016867100000000001 2024-09-18T11:33:09.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:09.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044439+0000", 2024-09-18T11:33:09.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074937149999999998 2024-09-18T11:33:09.102 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.102 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:09.103 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.103 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4444 4.d 4:b9bb642b:::smithi11939506-27:67 [sparse-read 0~1262403] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T11:33:09.103 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:33:09.103 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 32.988889225000001, 2024-09-18T11:33:09.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.014179987, 2024-09-18T11:33:09.103 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:09.103 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:09.103 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:09.103 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:09.103 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:09.103 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4444 2024-09-18T11:33:09.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.104 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:09.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:09.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:33:09.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:09.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:33:09.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:09.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:33:09.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:09.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:09.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030780+0000", 2024-09-18T11:33:09.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.163e-06 2024-09-18T11:33:09.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.105 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:09.105 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030786+0000", 2024-09-18T11:33:09.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879999999999998e-06 2024-09-18T11:33:09.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:09.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030791+0000", 2024-09-18T11:33:09.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2579999999999997e-06 2024-09-18T11:33:09.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044477+0000", 2024-09-18T11:33:09.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013686188 2024-09-18T11:33:09.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:09.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044614+0000", 2024-09-18T11:33:09.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013727300000000001 2024-09-18T11:33:09.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:09.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044957+0000", 2024-09-18T11:33:09.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000343331 2024-09-18T11:33:09.107 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.107 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:09.107 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.107 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4445 4.d 4:b9bb642b:::smithi11939506-27:67 [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T11:33:09.107 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:36.030796+0000", 2024-09-18T11:33:09.107 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 32.988870949000002, 2024-09-18T11:33:09.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.015916046999999999, 2024-09-18T11:33:09.107 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:09.107 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:09.107 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:09.108 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:09.108 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:09.108 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4445 2024-09-18T11:33:09.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.108 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:09.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:09.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030796+0000", 2024-09-18T11:33:09.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:09.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030796+0000", 2024-09-18T11:33:09.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:09.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030795+0000", 2024-09-18T11:33:09.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:09.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:09.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030799+0000", 2024-09-18T11:33:09.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5209999999999998e-06 2024-09-18T11:33:09.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:09.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030806+0000", 2024-09-18T11:33:09.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7599999999999997e-06 2024-09-18T11:33:09.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:09.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030810+0000", 2024-09-18T11:33:09.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0489999999999997e-06 2024-09-18T11:33:09.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.045277+0000", 2024-09-18T11:33:09.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.014467683 2024-09-18T11:33:09.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:09.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.045394+0000", 2024-09-18T11:33:09.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116969 2024-09-18T11:33:09.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:09.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.046712+0000", 2024-09-18T11:33:09.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013174669999999999 2024-09-18T11:33:09.111 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.111 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:09.111 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.111 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4683 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865643,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:33:09.112 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:45.318891+0000", 2024-09-18T11:33:09.112 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.700775331999999, 2024-09-18T11:33:09.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.017494045999999999, 2024-09-18T11:33:09.112 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:09.112 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:09.112 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:09.112 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:09.112 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:09.112 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4683 2024-09-18T11:33:09.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.112 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:09.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:09.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318891+0000", 2024-09-18T11:33:09.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:09.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318891+0000", 2024-09-18T11:33:09.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.113 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:09.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318891+0000", 2024-09-18T11:33:09.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:09.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:09.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318894+0000", 2024-09-18T11:33:09.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1379999999999999e-06 2024-09-18T11:33:09.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:09.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318900+0000", 2024-09-18T11:33:09.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6239999999999999e-06 2024-09-18T11:33:09.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.114 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:09.114 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318903+0000", 2024-09-18T11:33:09.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8249999999999998e-06 2024-09-18T11:33:09.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.330916+0000", 2024-09-18T11:33:09.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012012063999999999 2024-09-18T11:33:09.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:09.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.331011+0000", 2024-09-18T11:33:09.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5388000000000002e-05 2024-09-18T11:33:09.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:09.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.336385+0000", 2024-09-18T11:33:09.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005374429 2024-09-18T11:33:09.116 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.116 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:09.116 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.116 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4684 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865643,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:33:09.116 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:45.318908+0000", 2024-09-18T11:33:09.116 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.700759027, 2024-09-18T11:33:09.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.021211049999999999, 2024-09-18T11:33:09.116 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:09.117 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:09.117 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:09.117 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:09.117 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:09.117 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4684 2024-09-18T11:33:09.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.117 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:09.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:09.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318908+0000", 2024-09-18T11:33:09.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:09.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318908+0000", 2024-09-18T11:33:09.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:09.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:09.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318907+0000", 2024-09-18T11:33:09.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:09.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:09.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318910+0000", 2024-09-18T11:33:09.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.79e-06 2024-09-18T11:33:09.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:09.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318917+0000", 2024-09-18T11:33:09.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5740000000000001e-06 2024-09-18T11:33:09.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:09.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318920+0000", 2024-09-18T11:33:09.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5729999999999999e-06 2024-09-18T11:33:09.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:09.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.336370+0000", 2024-09-18T11:33:09.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01745025 2024-09-18T11:33:09.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:09.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.336506+0000", 2024-09-18T11:33:09.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013603700000000001 2024-09-18T11:33:09.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:09.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:09.120 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:09.120 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.340119+0000", 2024-09-18T11:33:09.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00361218 2024-09-18T11:33:09.120 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.120 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:09.120 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.120 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:09.120 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:09.120 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:09.122 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:33:09.122 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:33:09.123 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:33:09.123 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:33:09.123 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:33:09.124 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:33:09.124 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:33:09.124 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:33:09.124 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:33:09.125 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:33:09.125 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:33:09.125 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:33:09.126 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:33:09.126 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:33:09.121+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:33:09.130 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:33:09.131 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:33:09.146 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:33:09.146 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:33:09.223 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:09.222+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:09.228 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:09.229 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:09.237 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:33:09.324 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:33:09.323+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:33:09.335 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:09.335 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:09.344 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:33:09.425 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:33:09.424+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:33:09.442 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:09.442 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:09.451 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:33:09.526 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:33:09.525+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:33:09.549 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:09.549 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:09.557 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:33:09.627 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:09.626+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:09.655 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:09.655 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:09.663 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:33:09.728 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:09.727+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:33:09.762 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:09.762 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:09.770 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:33:09.829 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:09.828+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:33:09.871 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:09.872 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:09.880 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:33:09.930 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:09.929+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:33:09.977 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:09.977 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:09.986 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:33:10.031 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:33:10.030+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:33:10.047 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T11:33:10.085 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:10.085 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:10.094 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:33:10.131 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:10.130+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:33:10.232 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:10.231+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:33:10.233 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:10.233 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:10.241 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:33:10.333 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:10.332+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:33:10.338 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:10.338 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:10.347 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:33:10.433 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:33:10.432+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:33:10.445 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:10.446 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:10.454 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:33:10.534 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:10.533+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:33:10.593 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:10.593 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:10.601 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:33:10.645 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:10.644+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:10.699 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:10.699 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:10.707 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:33:10.736 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:33:10.735+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:33:10.837 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:10.836+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:10.845 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:10.845 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:10.853 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:33:10.938 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:33:10.937+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:33:10.991 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:10.992 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:11.000 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:33:11.041 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:11.040+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:11.087 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is set 2024-09-18T11:33:11.138 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:11.139 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:11.145 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:11.144+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:33:11.147 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:33:11.158 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:33:11.240 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:33:11.238+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:33:11.245 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:11.245 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:11.253 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:33:11.281 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:33:11.282 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:33:11.295 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:11.294+0000 7fb6930fb640 -1 osd.2 195 handle_osd_map injecting map crc failure 2024-09-18T11:33:11.298 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:33:11.298 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:33:11.313 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:33:11.314 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:33:11.328 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:33:11.329 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:33:11.341 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:33:11.340+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:33:11.344 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:33:11.344 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:33:11.356 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:11.356 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:33:11.356 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:33:11.356 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:11.360 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:33:11.360 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:33:11.364 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:33:11.376 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:33:11.376 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:33:11.391 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:33:11.392 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:33:11.444 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:11.443+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:33:11.474 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:11.474 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:33:11.474 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:33:11.474 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:33:11.474 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:11.482 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:33:11.566 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:11.565+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:33:11.621 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:11.621 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:33:11.621 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:33:11.621 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:33:11.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.621 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4736 4.3 4:c70c3162:::smithi11939506-35:head [write 574905~711397,stat] snapc 7c=[7c,7b,79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:33:11.622 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.031772+0000", 2024-09-18T11:33:11.622 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.5872919, 2024-09-18T11:33:11.622 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0092375840000000001, 2024-09-18T11:33:11.622 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:11.622 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:11.622 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:11.622 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:11.622 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:11.622 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4736 2024-09-18T11:33:11.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.622 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:11.622 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.622 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:11.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.031772+0000", 2024-09-18T11:33:11.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:11.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.031772+0000", 2024-09-18T11:33:11.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.623 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.623 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:11.623 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.031771+0000", 2024-09-18T11:33:11.623 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:11.623 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:11.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032335+0000", 2024-09-18T11:33:11.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00056435800000000005 2024-09-18T11:33:11.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:11.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032349+0000", 2024-09-18T11:33:11.624 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4353e-05 2024-09-18T11:33:11.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.624 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.624 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:11.624 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032356+0000", 2024-09-18T11:33:11.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3550000000000001e-06 2024-09-18T11:33:11.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.625 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032487+0000", 2024-09-18T11:33:11.625 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013154900000000001 2024-09-18T11:33:11.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.625 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:33:11.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032495+0000", 2024-09-18T11:33:11.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7519999999999996e-06 2024-09-18T11:33:11.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.035010+0000", 2024-09-18T11:33:11.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025153530000000001 2024-09-18T11:33:11.626 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.626 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.626 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:11.626 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.035142+0000", 2024-09-18T11:33:11.626 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013133599999999999 2024-09-18T11:33:11.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:11.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.035330+0000", 2024-09-18T11:33:11.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018873499999999999 2024-09-18T11:33:11.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:11.627 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040882+0000", 2024-09-18T11:33:11.627 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055514689999999998 2024-09-18T11:33:11.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.627 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.627 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:11.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040917+0000", 2024-09-18T11:33:11.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5305999999999997e-05 2024-09-18T11:33:11.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:11.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040968+0000", 2024-09-18T11:33:11.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0720000000000002e-05 2024-09-18T11:33:11.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.628 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.628 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:11.628 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041010+0000", 2024-09-18T11:33:11.628 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1893e-05 2024-09-18T11:33:11.629 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.629 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:11.629 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.629 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.629 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4737 4.3 4:c70c3162:::smithi11939506-35:head [write 1885086~525140] snapc 7c=[7c,7b,79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:33:11.629 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.032373+0000", 2024-09-18T11:33:11.629 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.586690848, 2024-09-18T11:33:11.629 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0088738949999999997, 2024-09-18T11:33:11.629 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:11.629 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:11.629 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:11.630 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:11.630 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:11.630 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4737 2024-09-18T11:33:11.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.630 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:11.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:11.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032373+0000", 2024-09-18T11:33:11.630 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.630 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:11.630 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032373+0000", 2024-09-18T11:33:11.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:11.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032372+0000", 2024-09-18T11:33:11.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:11.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:11.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032651+0000", 2024-09-18T11:33:11.631 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027865499999999999 2024-09-18T11:33:11.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.631 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:11.631 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032661+0000", 2024-09-18T11:33:11.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5440000000000004e-06 2024-09-18T11:33:11.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:11.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032665+0000", 2024-09-18T11:33:11.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9899999999999999e-06 2024-09-18T11:33:11.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.632 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.632 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032701+0000", 2024-09-18T11:33:11.632 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6587000000000001e-05 2024-09-18T11:33:11.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:33:11.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032708+0000", 2024-09-18T11:33:11.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.742e-06 2024-09-18T11:33:11.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.037089+0000", 2024-09-18T11:33:11.633 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043810639999999996 2024-09-18T11:33:11.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.633 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:11.633 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.037176+0000", 2024-09-18T11:33:11.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7266000000000001e-05 2024-09-18T11:33:11.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:11.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.037317+0000", 2024-09-18T11:33:11.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014054200000000001 2024-09-18T11:33:11.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.634 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:11.634 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041074+0000", 2024-09-18T11:33:11.634 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037575709999999999 2024-09-18T11:33:11.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041165+0000", 2024-09-18T11:33:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0785999999999994e-05 2024-09-18T11:33:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041208+0000", 2024-09-18T11:33:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2611999999999998e-05 2024-09-18T11:33:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041247+0000", 2024-09-18T11:33:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9538e-05 2024-09-18T11:33:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4738 4.3 4:c70c3162:::smithi11939506-35:head [write 2930189~44716] snapc 7c=[7c,7b,79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:33:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.032671+0000", 2024-09-18T11:33:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.586393353999998, 2024-09-18T11:33:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086678190000000002, 2024-09-18T11:33:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:11.637 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:11.637 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:11.637 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:11.637 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4738 2024-09-18T11:33:11.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.637 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:11.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.637 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:11.637 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032671+0000", 2024-09-18T11:33:11.637 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:11.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032671+0000", 2024-09-18T11:33:11.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:11.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032670+0000", 2024-09-18T11:33:11.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:11.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.638 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:11.638 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032908+0000", 2024-09-18T11:33:11.638 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023724099999999999 2024-09-18T11:33:11.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:11.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032917+0000", 2024-09-18T11:33:11.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1719999999999996e-06 2024-09-18T11:33:11.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:11.639 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032920+0000", 2024-09-18T11:33:11.639 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9270000000000001e-06 2024-09-18T11:33:11.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.639 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:33:11.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9938e-05 2024-09-18T11:33:11.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:33:11.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032957+0000", 2024-09-18T11:33:11.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7789999999999996e-06 2024-09-18T11:33:11.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.640 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.640 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.038135+0000", 2024-09-18T11:33:11.640 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051778789999999998 2024-09-18T11:33:11.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:11.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.038219+0000", 2024-09-18T11:33:11.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4047000000000004e-05 2024-09-18T11:33:11.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:11.641 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.038413+0000", 2024-09-18T11:33:11.641 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019343600000000001 2024-09-18T11:33:11.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.641 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:11.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041104+0000", 2024-09-18T11:33:11.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026916190000000001 2024-09-18T11:33:11.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:11.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041256+0000", 2024-09-18T11:33:11.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015182100000000001 2024-09-18T11:33:11.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.642 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:11.642 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041300+0000", 2024-09-18T11:33:11.642 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3915999999999998e-05 2024-09-18T11:33:11.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.643 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:11.643 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041339+0000", 2024-09-18T11:33:11.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8510000000000002e-05 2024-09-18T11:33:11.643 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.643 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:11.643 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.643 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4739 4.3 4:c70c3162:::smithi11939506-35:head [setxattr _header (57) in=7b,truncate 2974905] snapc 7c=[7c,7b,79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:33:11.643 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.032924+0000", 2024-09-18T11:33:11.643 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.586139726999999, 2024-09-18T11:33:11.643 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085165610000000006, 2024-09-18T11:33:11.644 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:11.644 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:11.644 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:11.644 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:11.644 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:11.644 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4739 2024-09-18T11:33:11.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.644 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:11.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.644 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:11.644 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032924+0000", 2024-09-18T11:33:11.644 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:11.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032924+0000", 2024-09-18T11:33:11.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:11.645 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032924+0000", 2024-09-18T11:33:11.645 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:11.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.645 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:11.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032926+0000", 2024-09-18T11:33:11.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0619999999999999e-06 2024-09-18T11:33:11.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:11.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032935+0000", 2024-09-18T11:33:11.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.371e-06 2024-09-18T11:33:11.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.646 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:11.646 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032937+0000", 2024-09-18T11:33:11.646 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6769999999999999e-06 2024-09-18T11:33:11.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032979+0000", 2024-09-18T11:33:11.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1810000000000001e-05 2024-09-18T11:33:11.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:33:11.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032984+0000", 2024-09-18T11:33:11.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1660000000000002e-06 2024-09-18T11:33:11.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.647 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.647 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039042+0000", 2024-09-18T11:33:11.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060573399999999996 2024-09-18T11:33:11.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:11.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039127+0000", 2024-09-18T11:33:11.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5494999999999994e-05 2024-09-18T11:33:11.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:11.648 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039293+0000", 2024-09-18T11:33:11.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016590999999999999 2024-09-18T11:33:11.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:11.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041146+0000", 2024-09-18T11:33:11.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018531369999999999 2024-09-18T11:33:11.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:11.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041345+0000", 2024-09-18T11:33:11.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000199218 2024-09-18T11:33:11.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:11.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041390+0000", 2024-09-18T11:33:11.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4892000000000003e-05 2024-09-18T11:33:11.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:11.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041441+0000", 2024-09-18T11:33:11.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0815000000000001e-05 2024-09-18T11:33:11.650 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.650 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:11.650 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.650 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4740 4.3 4:c70c3162:::smithi11939506-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:33:11.650 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:33:11.651 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 22.586114066, 2024-09-18T11:33:11.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0087685269999999999, 2024-09-18T11:33:11.651 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:11.651 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:11.651 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:11.651 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:11.651 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:11.651 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4740 2024-09-18T11:33:11.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.651 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:11.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:11.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:33:11.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:11.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:33:11.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:11.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:33:11.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:11.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:11.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032952+0000", 2024-09-18T11:33:11.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.215e-06 2024-09-18T11:33:11.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:11.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032956+0000", 2024-09-18T11:33:11.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8850000000000001e-06 2024-09-18T11:33:11.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:11.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032958+0000", 2024-09-18T11:33:11.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1840000000000002e-06 2024-09-18T11:33:11.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032997+0000", 2024-09-18T11:33:11.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9161e-05 2024-09-18T11:33:11.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:33:11.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.033000+0000", 2024-09-18T11:33:11.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1030000000000002e-06 2024-09-18T11:33:11.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039536+0000", 2024-09-18T11:33:11.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006535201 2024-09-18T11:33:11.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:11.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039616+0000", 2024-09-18T11:33:11.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0684000000000002e-05 2024-09-18T11:33:11.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041536+0000", 2024-09-18T11:33:11.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019196440000000001 2024-09-18T11:33:11.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:11.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041610+0000", 2024-09-18T11:33:11.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4295999999999995e-05 2024-09-18T11:33:11.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:11.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041719+0000", 2024-09-18T11:33:11.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108465 2024-09-18T11:33:11.656 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.656 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:11.656 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.657 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:4756 4.7 e179/174 4:e301aefd:::smithi11939506-19:head v 179'405, mlcod=173'401)", 2024-09-18T11:33:11.657 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:50.039068+0000", 2024-09-18T11:33:11.657 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.579995966999999, 2024-09-18T11:33:11.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055292249999999996, 2024-09-18T11:33:11.657 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:11.657 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:11.657 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:11.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:11.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.039068+0000", 2024-09-18T11:33:11.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:11.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.039068+0000", 2024-09-18T11:33:11.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:11.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.039068+0000", 2024-09-18T11:33:11.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:11.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:11.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040038+0000", 2024-09-18T11:33:11.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096998500000000003 2024-09-18T11:33:11.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:11.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040053+0000", 2024-09-18T11:33:11.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5481999999999998e-05 2024-09-18T11:33:11.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:11.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040062+0000", 2024-09-18T11:33:11.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7080000000000003e-06 2024-09-18T11:33:11.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040109+0000", 2024-09-18T11:33:11.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6752000000000002e-05 2024-09-18T11:33:11.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:11.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040124+0000", 2024-09-18T11:33:11.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5715000000000001e-05 2024-09-18T11:33:11.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:11.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044568+0000", 2024-09-18T11:33:11.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044430600000000004 2024-09-18T11:33:11.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:11.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044597+0000", 2024-09-18T11:33:11.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9887999999999999e-05 2024-09-18T11:33:11.661 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.661 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:11.661 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.662 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:4757 4.7 e179/174 4:e301aefd:::smithi11939506-19:head v 179'406, mlcod=173'401)", 2024-09-18T11:33:11.662 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:50.044255+0000", 2024-09-18T11:33:11.662 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.574809406, 2024-09-18T11:33:11.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002073166, 2024-09-18T11:33:11.662 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:11.662 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:11.662 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:11.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:11.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044255+0000", 2024-09-18T11:33:11.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:11.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044255+0000", 2024-09-18T11:33:11.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:11.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044254+0000", 2024-09-18T11:33:11.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:11.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:11.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044875+0000", 2024-09-18T11:33:11.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00062136400000000001 2024-09-18T11:33:11.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:11.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044886+0000", 2024-09-18T11:33:11.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0709e-05 2024-09-18T11:33:11.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:11.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044891+0000", 2024-09-18T11:33:11.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8019999999999999e-06 2024-09-18T11:33:11.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044911+0000", 2024-09-18T11:33:11.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0729000000000001e-05 2024-09-18T11:33:11.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:11.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044930+0000", 2024-09-18T11:33:11.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8890999999999999e-05 2024-09-18T11:33:11.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:11.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046295+0000", 2024-09-18T11:33:11.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013645420000000001 2024-09-18T11:33:11.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:11.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046328+0000", 2024-09-18T11:33:11.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3089000000000001e-05 2024-09-18T11:33:11.666 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.666 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:11.666 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.666 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:4764 3.3 e179/174 3:ec4378e2:::259b372f30a46a47845d5d9c58303dcbeccaeb2db18e8ed0152f1c66341c4df2:head v 179'156, mlcod=172'154)", 2024-09-18T11:33:11.667 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:50.059675+0000", 2024-09-18T11:33:11.667 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 21.559389054, 2024-09-18T11:33:11.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056090439999999997, 2024-09-18T11:33:11.667 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:11.667 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:11.667 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:11.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:11.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.059675+0000", 2024-09-18T11:33:11.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:11.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.059675+0000", 2024-09-18T11:33:11.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:11.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.059675+0000", 2024-09-18T11:33:11.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:11.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:11.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062252+0000", 2024-09-18T11:33:11.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025772859999999998 2024-09-18T11:33:11.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:11.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062265+0000", 2024-09-18T11:33:11.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3059e-05 2024-09-18T11:33:11.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:11.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062272+0000", 2024-09-18T11:33:11.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3589999999999996e-06 2024-09-18T11:33:11.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062317+0000", 2024-09-18T11:33:11.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5188999999999997e-05 2024-09-18T11:33:11.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:11.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062334+0000", 2024-09-18T11:33:11.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.73e-05 2024-09-18T11:33:11.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:11.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.065251+0000", 2024-09-18T11:33:11.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029169539999999998 2024-09-18T11:33:11.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:11.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.065284+0000", 2024-09-18T11:33:11.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3206e-05 2024-09-18T11:33:11.671 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.671 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:11.671 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.671 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4792 4.6 4:67c1765a:::smithi11939506-30:head [sparse-read 0~1153827,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-18T11:33:11.671 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.053543+0000", 2024-09-18T11:33:11.671 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.565521333, 2024-09-18T11:33:11.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003529337, 2024-09-18T11:33:11.671 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:11.671 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:11.672 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:11.672 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:11.672 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:11.672 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4792 2024-09-18T11:33:11.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.672 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:11.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:11.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053543+0000", 2024-09-18T11:33:11.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:11.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053543+0000", 2024-09-18T11:33:11.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:11.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053541+0000", 2024-09-18T11:33:11.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:11.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:11.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053550+0000", 2024-09-18T11:33:11.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2070000000000001e-06 2024-09-18T11:33:11.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:11.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053566+0000", 2024-09-18T11:33:11.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6098000000000001e-05 2024-09-18T11:33:11.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:11.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053575+0000", 2024-09-18T11:33:11.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3769999999999996e-06 2024-09-18T11:33:11.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053620+0000", 2024-09-18T11:33:11.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5404000000000001e-05 2024-09-18T11:33:11.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:33:11.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053634+0000", 2024-09-18T11:33:11.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4161e-05 2024-09-18T11:33:11.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.055502+0000", 2024-09-18T11:33:11.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018678810000000001 2024-09-18T11:33:11.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:11.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.055629+0000", 2024-09-18T11:33:11.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012652 2024-09-18T11:33:11.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:11.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057072+0000", 2024-09-18T11:33:11.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001443567 2024-09-18T11:33:11.677 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.677 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:11.677 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.677 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4793 4.6 4:67c1765a:::smithi11939506-30:head [sparse-read 0~1153827] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-18T11:33:11.677 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.053593+0000", 2024-09-18T11:33:11.677 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.565471466999998, 2024-09-18T11:33:11.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037961539999999999, 2024-09-18T11:33:11.677 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:11.677 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:11.678 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:11.678 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:11.678 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:11.678 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4793 2024-09-18T11:33:11.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.678 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:11.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:11.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053593+0000", 2024-09-18T11:33:11.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:11.678 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053593+0000", 2024-09-18T11:33:11.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:11.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053592+0000", 2024-09-18T11:33:11.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:11.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:11.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053595+0000", 2024-09-18T11:33:11.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2179999999999999e-06 2024-09-18T11:33:11.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:11.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053603+0000", 2024-09-18T11:33:11.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.576e-06 2024-09-18T11:33:11.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:11.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053607+0000", 2024-09-18T11:33:11.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2309999999999998e-06 2024-09-18T11:33:11.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053696+0000", 2024-09-18T11:33:11.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9130999999999998e-05 2024-09-18T11:33:11.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:11.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053702+0000", 2024-09-18T11:33:11.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6370000000000004e-06 2024-09-18T11:33:11.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057115+0000", 2024-09-18T11:33:11.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034128100000000001 2024-09-18T11:33:11.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:11.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057176+0000", 2024-09-18T11:33:11.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.143e-05 2024-09-18T11:33:11.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:11.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057389+0000", 2024-09-18T11:33:11.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021267700000000001 2024-09-18T11:33:11.682 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.682 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:11.682 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.682 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4794 4.6 4:67c1765a:::smithi11939506-30:head [sparse-read 0~1153827] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-18T11:33:11.682 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.053613+0000", 2024-09-18T11:33:11.683 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 18.565451215, 2024-09-18T11:33:11.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040685349999999999, 2024-09-18T11:33:11.683 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:11.683 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:11.683 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:11.683 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:11.683 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:11.683 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4794 2024-09-18T11:33:11.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.683 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:11.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:11.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053613+0000", 2024-09-18T11:33:11.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:11.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053613+0000", 2024-09-18T11:33:11.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:11.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053612+0000", 2024-09-18T11:33:11.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:11.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:11.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053615+0000", 2024-09-18T11:33:11.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5730000000000002e-06 2024-09-18T11:33:11.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:11.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053628+0000", 2024-09-18T11:33:11.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3084e-05 2024-09-18T11:33:11.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:11.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053632+0000", 2024-09-18T11:33:11.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9620000000000004e-06 2024-09-18T11:33:11.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053731+0000", 2024-09-18T11:33:11.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8547000000000004e-05 2024-09-18T11:33:11.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:11.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053736+0000", 2024-09-18T11:33:11.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.767e-06 2024-09-18T11:33:11.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057461+0000", 2024-09-18T11:33:11.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037242780000000001 2024-09-18T11:33:11.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:11.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057538+0000", 2024-09-18T11:33:11.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7008999999999994e-05 2024-09-18T11:33:11.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:11.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057681+0000", 2024-09-18T11:33:11.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000143786 2024-09-18T11:33:11.687 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.687 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:11.687 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.688 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e301aefd:::smithi11939506-19:3f)])", 2024-09-18T11:33:11.688 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.083144+0000", 2024-09-18T11:33:11.688 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.535920605999999, 2024-09-18T11:33:11.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024893160000000001, 2024-09-18T11:33:11.688 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:11.688 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:11.688 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:11.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.688 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:11.688 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.083144+0000", 2024-09-18T11:33:11.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:11.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.083144+0000", 2024-09-18T11:33:11.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:11.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.083143+0000", 2024-09-18T11:33:11.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:11.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:11.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.083149+0000", 2024-09-18T11:33:11.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0549999999999996e-06 2024-09-18T11:33:11.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:11.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.083161+0000", 2024-09-18T11:33:11.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2668e-05 2024-09-18T11:33:11.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:11.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.083169+0000", 2024-09-18T11:33:11.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2830000000000004e-06 2024-09-18T11:33:11.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.083208+0000", 2024-09-18T11:33:11.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9749999999999997e-05 2024-09-18T11:33:11.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:11.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.085633+0000", 2024-09-18T11:33:11.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024244639999999999 2024-09-18T11:33:11.691 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.691 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:11.691 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.691 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e0d95e10:::smithi11939506-22:49)])", 2024-09-18T11:33:11.692 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.102811+0000", 2024-09-18T11:33:11.692 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.516253494000001, 2024-09-18T11:33:11.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00260509, 2024-09-18T11:33:11.692 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:11.692 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:11.692 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:11.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:11.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.102811+0000", 2024-09-18T11:33:11.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:11.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.102811+0000", 2024-09-18T11:33:11.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:11.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.102809+0000", 2024-09-18T11:33:11.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:11.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:11.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.102818+0000", 2024-09-18T11:33:11.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4089999999999992e-06 2024-09-18T11:33:11.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:11.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.102836+0000", 2024-09-18T11:33:11.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7881999999999999e-05 2024-09-18T11:33:11.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:11.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.102844+0000", 2024-09-18T11:33:11.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8900000000000007e-06 2024-09-18T11:33:11.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.102892+0000", 2024-09-18T11:33:11.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7756999999999997e-05 2024-09-18T11:33:11.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:11.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.105416+0000", 2024-09-18T11:33:11.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025237710000000002 2024-09-18T11:33:11.695 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.695 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:11.695 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.695 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e73ba68f:::smithi11939506-1:42)])", 2024-09-18T11:33:11.695 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.114167+0000", 2024-09-18T11:33:11.695 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.504897589, 2024-09-18T11:33:11.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002907303, 2024-09-18T11:33:11.695 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:11.695 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:11.695 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:11.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:11.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.114167+0000", 2024-09-18T11:33:11.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:11.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.114167+0000", 2024-09-18T11:33:11.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:11.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.114165+0000", 2024-09-18T11:33:11.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:11.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:11.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.114174+0000", 2024-09-18T11:33:11.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4419999999999998e-06 2024-09-18T11:33:11.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:11.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.114192+0000", 2024-09-18T11:33:11.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8346e-05 2024-09-18T11:33:11.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.114200+0000", 2024-09-18T11:33:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9680000000000006e-06 2024-09-18T11:33:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.114249+0000", 2024-09-18T11:33:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8974000000000002e-05 2024-09-18T11:33:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.117074+0000", 2024-09-18T11:33:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028247530000000002 2024-09-18T11:33:11.698 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e5707c18:::smithi11939506-10:5d)])", 2024-09-18T11:33:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.126498+0000", 2024-09-18T11:33:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.492566311999999, 2024-09-18T11:33:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0033596170000000001, 2024-09-18T11:33:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:11.699 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.126498+0000", 2024-09-18T11:33:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.126498+0000", 2024-09-18T11:33:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.126497+0000", 2024-09-18T11:33:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.126508+0000", 2024-09-18T11:33:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1898e-05 2024-09-18T11:33:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.126525+0000", 2024-09-18T11:33:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6761000000000001e-05 2024-09-18T11:33:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.126533+0000", 2024-09-18T11:33:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9629999999999998e-06 2024-09-18T11:33:11.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.126581+0000", 2024-09-18T11:33:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7787000000000002e-05 2024-09-18T11:33:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.129857+0000", 2024-09-18T11:33:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003276476 2024-09-18T11:33:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e73ba68f:::smithi11939506-1:5d)])", 2024-09-18T11:33:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.141236+0000", 2024-09-18T11:33:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.477827751, 2024-09-18T11:33:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002457187, 2024-09-18T11:33:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.141236+0000", 2024-09-18T11:33:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.141236+0000", 2024-09-18T11:33:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.141235+0000", 2024-09-18T11:33:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.141248+0000", 2024-09-18T11:33:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3131e-05 2024-09-18T11:33:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.141268+0000", 2024-09-18T11:33:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0689000000000001e-05 2024-09-18T11:33:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:11.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.141277+0000", 2024-09-18T11:33:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7280000000000001e-06 2024-09-18T11:33:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.141331+0000", 2024-09-18T11:33:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4098000000000003e-05 2024-09-18T11:33:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.143694+0000", 2024-09-18T11:33:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023623810000000002 2024-09-18T11:33:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.706 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:ef60946b:::smithi11939506-8:head)])", 2024-09-18T11:33:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.151354+0000", 2024-09-18T11:33:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.46770972, 2024-09-18T11:33:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026499959999999999, 2024-09-18T11:33:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:11.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.151354+0000", 2024-09-18T11:33:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.151354+0000", 2024-09-18T11:33:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.151354+0000", 2024-09-18T11:33:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.151360+0000", 2024-09-18T11:33:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8120000000000002e-06 2024-09-18T11:33:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.151371+0000", 2024-09-18T11:33:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0324e-05 2024-09-18T11:33:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.151375+0000", 2024-09-18T11:33:11.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7049999999999998e-06 2024-09-18T11:33:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.151420+0000", 2024-09-18T11:33:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4496000000000002e-05 2024-09-18T11:33:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.154004+0000", 2024-09-18T11:33:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025845579999999998 2024-09-18T11:33:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e0d95e10:::smithi11939506-22:74)])", 2024-09-18T11:33:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.169218+0000", 2024-09-18T11:33:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.449846340000001, 2024-09-18T11:33:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023236060000000002, 2024-09-18T11:33:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.169218+0000", 2024-09-18T11:33:11.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.169218+0000", 2024-09-18T11:33:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.169217+0000", 2024-09-18T11:33:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.169224+0000", 2024-09-18T11:33:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9129999999999997e-06 2024-09-18T11:33:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.169235+0000", 2024-09-18T11:33:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1032e-05 2024-09-18T11:33:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.169242+0000", 2024-09-18T11:33:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8589999999999999e-06 2024-09-18T11:33:11.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.169284+0000", 2024-09-18T11:33:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2293000000000003e-05 2024-09-18T11:33:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.171541+0000", 2024-09-18T11:33:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022575939999999999 2024-09-18T11:33:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:11.714 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e301aefd:::smithi11939506-19:7e)])", 2024-09-18T11:33:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.179260+0000", 2024-09-18T11:33:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.439804188, 2024-09-18T11:33:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025394670000000001, 2024-09-18T11:33:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.179260+0000", 2024-09-18T11:33:11.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.179260+0000", 2024-09-18T11:33:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.179259+0000", 2024-09-18T11:33:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.179265+0000", 2024-09-18T11:33:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1879999999999999e-06 2024-09-18T11:33:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.179276+0000", 2024-09-18T11:33:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1070000000000001e-05 2024-09-18T11:33:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.179283+0000", 2024-09-18T11:33:11.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6059999999999999e-06 2024-09-18T11:33:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.179320+0000", 2024-09-18T11:33:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6847999999999997e-05 2024-09-18T11:33:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.181799+0000", 2024-09-18T11:33:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002479738 2024-09-18T11:33:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:11.718 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e73ba68f:::smithi11939506-1:head)])", 2024-09-18T11:33:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.201696+0000", 2024-09-18T11:33:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 12.41736787, 2024-09-18T11:33:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026547110000000001, 2024-09-18T11:33:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.201696+0000", 2024-09-18T11:33:11.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.201696+0000", 2024-09-18T11:33:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.201695+0000", 2024-09-18T11:33:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.201704+0000", 2024-09-18T11:33:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5779999999999999e-06 2024-09-18T11:33:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.201717+0000", 2024-09-18T11:33:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3019e-05 2024-09-18T11:33:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.201723+0000", 2024-09-18T11:33:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2299999999999996e-06 2024-09-18T11:33:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.201771+0000", 2024-09-18T11:33:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7652999999999999e-05 2024-09-18T11:33:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.204351+0000", 2024-09-18T11:33:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025802009999999998 2024-09-18T11:33:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:11.722 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:11.722 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:11.724 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:33:11.724 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:33:11.723+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:33:11.825 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:33:11.824+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:33:11.862 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:11.862 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:33:11.862 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:33:11.862 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:11.870 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:33:11.926 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:11.925+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:12.008 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:12.008 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:33:12.008 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:33:12.009 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:33:12.009 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:12.017 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:33:12.061 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:12.060+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:33:12.120 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:12.120 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:33:12.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:33:12.120 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:33:12.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.121 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:262 4.9 e53/22 4:99feffe9:::smithi11939506-45:13 v 53'17, mlcod=51'14)", 2024-09-18T11:33:12.121 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:33:12.121 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 158.11817107600001, 2024-09-18T11:33:12.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013878679, 2024-09-18T11:33:12.121 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.122 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.122 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:33:12.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:33:12.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998991+0000", 2024-09-18T11:33:12.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:12.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.998997+0000", 2024-09-18T11:33:12.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1099999999999999e-06 2024-09-18T11:33:12.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999008+0000", 2024-09-18T11:33:12.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1321000000000001e-05 2024-09-18T11:33:12.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999015+0000", 2024-09-18T11:33:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3000000000000004e-06 2024-09-18T11:33:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999055+0000", 2024-09-18T11:33:12.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0058000000000002e-05 2024-09-18T11:33:12.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999070+0000", 2024-09-18T11:33:12.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4328000000000001e-05 2024-09-18T11:33:12.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012465+0000", 2024-09-18T11:33:12.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013395638 2024-09-18T11:33:12.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012870+0000", 2024-09-18T11:33:12.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00040423400000000001 2024-09-18T11:33:12.130 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.131 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.131 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.131 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.0.0:205 4.b e53/22 4:dba5bd39:::smithi11939506-17:13 v 53'77, mlcod=51'74)", 2024-09-18T11:33:12.131 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:33:12.132 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 158.11812181499999, 2024-09-18T11:33:12.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013359586, 2024-09-18T11:33:12.132 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.133 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.133 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:33:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:33:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999040+0000", 2024-09-18T11:33:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999048+0000", 2024-09-18T11:33:12.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3550000000000005e-06 2024-09-18T11:33:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999056+0000", 2024-09-18T11:33:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3259999999999996e-06 2024-09-18T11:33:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999063+0000", 2024-09-18T11:33:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6409999999999996e-06 2024-09-18T11:33:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999098+0000", 2024-09-18T11:33:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5203999999999997e-05 2024-09-18T11:33:12.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:33.999115+0000", 2024-09-18T11:33:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6478999999999999e-05 2024-09-18T11:33:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012362+0000", 2024-09-18T11:33:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01324768 2024-09-18T11:33:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.012400+0000", 2024-09-18T11:33:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.731e-05 2024-09-18T11:33:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2937 4.7 4:e301aefd:::smithi11939506-19:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:33:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:33:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 109.082321502, 2024-09-18T11:33:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019287667000000001, 2024-09-18T11:33:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2937 2024-09-18T11:33:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:33:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:33:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034840+0000", 2024-09-18T11:33:12.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:33:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034842+0000", 2024-09-18T11:33:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.372e-06 2024-09-18T11:33:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034846+0000", 2024-09-18T11:33:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3919999999999996e-06 2024-09-18T11:33:12.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034848+0000", 2024-09-18T11:33:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3539999999999998e-06 2024-09-18T11:33:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034893+0000", 2024-09-18T11:33:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4861000000000003e-05 2024-09-18T11:33:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.034965+0000", 2024-09-18T11:33:12.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1860999999999995e-05 2024-09-18T11:33:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.054128+0000", 2024-09-18T11:33:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.019163039 2024-09-18T11:33:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3453 4.f 4:f5d7df41:::smithi11939506-16:head [write 1813152~490434] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:33:12.144 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:33:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 86.094409573999997, 2024-09-18T11:33:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050685840000000003, 2024-09-18T11:33:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3453 2024-09-18T11:33:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:33:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022752+0000", 2024-09-18T11:33:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022751+0000", 2024-09-18T11:33:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022871+0000", 2024-09-18T11:33:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000119781 2024-09-18T11:33:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022882+0000", 2024-09-18T11:33:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1627000000000001e-05 2024-09-18T11:33:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.022887+0000", 2024-09-18T11:33:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2390000000000004e-06 2024-09-18T11:33:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023123+0000", 2024-09-18T11:33:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000236598 2024-09-18T11:33:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023207+0000", 2024-09-18T11:33:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4303999999999998e-05 2024-09-18T11:33:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023367+0000", 2024-09-18T11:33:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015983600000000001 2024-09-18T11:33:12.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073237+0000", 2024-09-18T11:33:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049869400000000001 2024-09-18T11:33:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073352+0000", 2024-09-18T11:33:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00011579899999999999 2024-09-18T11:33:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073394+0000", 2024-09-18T11:33:12.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1445e-05 2024-09-18T11:33:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073438+0000", 2024-09-18T11:33:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4261999999999997e-05 2024-09-18T11:33:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3454 4.f 4:f5d7df41:::smithi11939506-16:head [setxattr _header (57) in=7b,truncate 2991679] snapc 49=[49,48,47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:33:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:33:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 86.093711131999996, 2024-09-18T11:33:12.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050365656000000002, 2024-09-18T11:33:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3454 2024-09-18T11:33:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:33:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023451+0000", 2024-09-18T11:33:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023450+0000", 2024-09-18T11:33:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023456+0000", 2024-09-18T11:33:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8329999999999999e-06 2024-09-18T11:33:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023465+0000", 2024-09-18T11:33:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1130000000000007e-06 2024-09-18T11:33:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023470+0000", 2024-09-18T11:33:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9139999999999999e-06 2024-09-18T11:33:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072611+0000", 2024-09-18T11:33:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049141643999999998 2024-09-18T11:33:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072729+0000", 2024-09-18T11:33:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117667 2024-09-18T11:33:12.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072887+0000", 2024-09-18T11:33:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015760000000000001 2024-09-18T11:33:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073558+0000", 2024-09-18T11:33:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00067171699999999995 2024-09-18T11:33:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073764+0000", 2024-09-18T11:33:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000205749 2024-09-18T11:33:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073790+0000", 2024-09-18T11:33:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5643000000000002e-05 2024-09-18T11:33:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073816+0000", 2024-09-18T11:33:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6588e-05 2024-09-18T11:33:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3455 4.f 4:f5d7df41:::smithi11939506-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:33:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:33:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 86.093681513999996, 2024-09-18T11:33:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.050460466000000002, 2024-09-18T11:33:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.157 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:12.158 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:12.158 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:12.158 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:12.158 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3455 2024-09-18T11:33:12.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.158 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:33:12.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:33:12.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023480+0000", 2024-09-18T11:33:12.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:12.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023483+0000", 2024-09-18T11:33:12.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.695e-06 2024-09-18T11:33:12.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023489+0000", 2024-09-18T11:33:12.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1079999999999998e-06 2024-09-18T11:33:12.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023492+0000", 2024-09-18T11:33:12.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.478e-06 2024-09-18T11:33:12.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073273+0000", 2024-09-18T11:33:12.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049780638000000002 2024-09-18T11:33:12.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:12.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073343+0000", 2024-09-18T11:33:12.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9770000000000005e-05 2024-09-18T11:33:12.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073846+0000", 2024-09-18T11:33:12.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00050289499999999995 2024-09-18T11:33:12.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073886+0000", 2024-09-18T11:33:12.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0658999999999998e-05 2024-09-18T11:33:12.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073941+0000", 2024-09-18T11:33:12.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4642000000000003e-05 2024-09-18T11:33:12.162 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.162 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.162 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.162 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3456 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:33:12.163 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:33:12.163 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 86.093656448999994, 2024-09-18T11:33:12.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049859120999999999, 2024-09-18T11:33:12.163 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.163 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:12.163 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:12.163 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:12.164 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:12.164 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3456 2024-09-18T11:33:12.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.164 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:33:12.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:33:12.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023505+0000", 2024-09-18T11:33:12.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:12.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023508+0000", 2024-09-18T11:33:12.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9490000000000001e-06 2024-09-18T11:33:12.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023513+0000", 2024-09-18T11:33:12.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3800000000000002e-06 2024-09-18T11:33:12.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023517+0000", 2024-09-18T11:33:12.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5300000000000001e-06 2024-09-18T11:33:12.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073262+0000", 2024-09-18T11:33:12.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049745194999999999 2024-09-18T11:33:12.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073365+0000", 2024-09-18T11:33:12.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000102462 2024-09-18T11:33:12.171 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.171 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.171 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.171 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3457 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:33:12.171 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:33:12.172 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 86.093639353, 2024-09-18T11:33:12.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049928567, 2024-09-18T11:33:12.172 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.172 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:12.172 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:12.172 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:12.173 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:12.173 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3457 2024-09-18T11:33:12.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.173 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:33:12.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023523+0000", 2024-09-18T11:33:12.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023522+0000", 2024-09-18T11:33:12.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:12.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023524+0000", 2024-09-18T11:33:12.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8500000000000001e-06 2024-09-18T11:33:12.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023529+0000", 2024-09-18T11:33:12.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7509999999999999e-06 2024-09-18T11:33:12.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023532+0000", 2024-09-18T11:33:12.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.163e-06 2024-09-18T11:33:12.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073386+0000", 2024-09-18T11:33:12.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049853899 2024-09-18T11:33:12.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073451+0000", 2024-09-18T11:33:12.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5197000000000006e-05 2024-09-18T11:33:12.180 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.180 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.181 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.181 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3458 4.7 4:ef60946b:::smithi11939506-8:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-18T11:33:12.181 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:33:12.182 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 86.093626795999995, 2024-09-18T11:33:12.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.05001659, 2024-09-18T11:33:12.182 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.182 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:12.182 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:12.182 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:12.183 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:12.183 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3458 2024-09-18T11:33:12.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.183 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:33:12.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:33:12.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023535+0000", 2024-09-18T11:33:12.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:12.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023537+0000", 2024-09-18T11:33:12.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.64e-06 2024-09-18T11:33:12.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023543+0000", 2024-09-18T11:33:12.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3369999999999996e-06 2024-09-18T11:33:12.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.023546+0000", 2024-09-18T11:33:12.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9790000000000002e-06 2024-09-18T11:33:12.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073466+0000", 2024-09-18T11:33:12.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.049919719000000001 2024-09-18T11:33:12.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073552+0000", 2024-09-18T11:33:12.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6168999999999994e-05 2024-09-18T11:33:12.190 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.190 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.190 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.191 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop_reply(client.4590.0:3453 4.f e116/22 ondisk, result = 0)", 2024-09-18T11:33:12.191 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:33:12.191 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 86.092021712000005, 2024-09-18T11:33:12.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.048099441, 2024-09-18T11:33:12.191 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.192 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:12.192 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:33:12.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025140+0000", 2024-09-18T11:33:12.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025139+0000", 2024-09-18T11:33:12.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:12.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.025144+0000", 2024-09-18T11:33:12.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4059999999999998e-06 2024-09-18T11:33:12.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072212+0000", 2024-09-18T11:33:12.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.047068482000000002 2024-09-18T11:33:12.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.072230+0000", 2024-09-18T11:33:12.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7632999999999999e-05 2024-09-18T11:33:12.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073224+0000", 2024-09-18T11:33:12.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00099436899999999994 2024-09-18T11:33:12.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073231+0000", 2024-09-18T11:33:12.196 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7719999999999997e-06 2024-09-18T11:33:12.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.196 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.196 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.073240+0000", 2024-09-18T11:33:12.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5699999999999993e-06 2024-09-18T11:33:12.197 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.197 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.197 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.197 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.197 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3930 4.b 4:d3ae8215:::smithi11939506-41:head [write 632029~547288] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:33:12.197 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:33:12.197 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.091106031999999, 2024-09-18T11:33:12.197 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.018962711, 2024-09-18T11:33:12.197 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.198 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.198 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:12.198 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:12.198 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:12.198 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3930 2024-09-18T11:33:12.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.198 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.198 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:33:12.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026056+0000", 2024-09-18T11:33:12.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.199 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.199 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026055+0000", 2024-09-18T11:33:12.199 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:12.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.199 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026334+0000", 2024-09-18T11:33:12.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000278931 2024-09-18T11:33:12.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026344+0000", 2024-09-18T11:33:12.200 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0716999999999999e-05 2024-09-18T11:33:12.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.200 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.200 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.026350+0000", 2024-09-18T11:33:12.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7620000000000001e-06 2024-09-18T11:33:12.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028429+0000", 2024-09-18T11:33:12.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020787319999999998 2024-09-18T11:33:12.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.201 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.201 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028525+0000", 2024-09-18T11:33:12.201 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6638999999999994e-05 2024-09-18T11:33:12.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:33:12.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.028720+0000", 2024-09-18T11:33:12.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000194525 2024-09-18T11:33:12.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:12.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.036996+0000", 2024-09-18T11:33:12.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082756140000000006 2024-09-18T11:33:12.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.202 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:12.202 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044952+0000", 2024-09-18T11:33:12.202 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566740000000004 2024-09-18T11:33:12.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.044985+0000", 2024-09-18T11:33:12.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3081999999999999e-05 2024-09-18T11:33:12.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.203 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.203 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045019+0000", 2024-09-18T11:33:12.203 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3257999999999999e-05 2024-09-18T11:33:12.203 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.203 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.203 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.203 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.204 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3932 4.b 4:d3ae8215:::smithi11939506-41:head [write 2796249~437859] snapc 55=[55,54,53,51,4f,4e,4c,4b,49,48,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:33:12.204 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:33:12.204 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.079099392000003, 2024-09-18T11:33:12.204 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01458329, 2024-09-18T11:33:12.204 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.204 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.204 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:12.204 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:12.204 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:12.204 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3932 2024-09-18T11:33:12.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.204 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:33:12.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038063+0000", 2024-09-18T11:33:12.205 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.205 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.205 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038061+0000", 2024-09-18T11:33:12.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:12.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038341+0000", 2024-09-18T11:33:12.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027989099999999999 2024-09-18T11:33:12.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.206 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038353+0000", 2024-09-18T11:33:12.206 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2247e-05 2024-09-18T11:33:12.206 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.206 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.206 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.038358+0000", 2024-09-18T11:33:12.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1599999999999997e-06 2024-09-18T11:33:12.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043226+0000", 2024-09-18T11:33:12.207 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048673049999999997 2024-09-18T11:33:12.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.207 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.207 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043284+0000", 2024-09-18T11:33:12.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8338999999999998e-05 2024-09-18T11:33:12.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 4", 2024-09-18T11:33:12.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.043367+0000", 2024-09-18T11:33:12.208 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3120000000000004e-05 2024-09-18T11:33:12.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.208 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.208 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:12.208 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.051544+0000", 2024-09-18T11:33:12.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00817734 2024-09-18T11:33:12.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:12.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052528+0000", 2024-09-18T11:33:12.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00098318499999999992 2024-09-18T11:33:12.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.209 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052610+0000", 2024-09-18T11:33:12.209 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.2300999999999997e-05 2024-09-18T11:33:12.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.209 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.210 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.052646+0000", 2024-09-18T11:33:12.210 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5955999999999999e-05 2024-09-18T11:33:12.210 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.210 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.210 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.210 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.210 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.210 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3935 4.b 4:d3ae8215:::smithi11939506-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e132)", 2024-09-18T11:33:12.211 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:33:12.211 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.075762284999996, 2024-09-18T11:33:12.211 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013142333000000001, 2024-09-18T11:33:12.211 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.211 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:12.211 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:12.212 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:12.212 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:12.212 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3935 2024-09-18T11:33:12.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.212 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.212 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:33:12.213 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.213 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.213 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041400+0000", 2024-09-18T11:33:12.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.214 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.214 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.214 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041399+0000", 2024-09-18T11:33:12.214 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:12.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.215 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041401+0000", 2024-09-18T11:33:12.215 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4890000000000001e-06 2024-09-18T11:33:12.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.215 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.215 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041404+0000", 2024-09-18T11:33:12.216 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5240000000000001e-06 2024-09-18T11:33:12.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.216 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.216 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.216 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.041407+0000", 2024-09-18T11:33:12.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2230000000000001e-06 2024-09-18T11:33:12.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.217 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.217 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.217 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045699+0000", 2024-09-18T11:33:12.217 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0042918970000000002 2024-09-18T11:33:12.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.218 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.218 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:12.218 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.045750+0000", 2024-09-18T11:33:12.218 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1332999999999998e-05 2024-09-18T11:33:12.218 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054438+0000", 2024-09-18T11:33:12.219 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086882569999999996 2024-09-18T11:33:12.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.219 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.219 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054485+0000", 2024-09-18T11:33:12.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7151000000000003e-05 2024-09-18T11:33:12.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.220 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.220 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.054542+0000", 2024-09-18T11:33:12.220 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6706999999999998e-05 2024-09-18T11:33:12.220 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.220 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.220 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.220 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.221 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3969 4.4 4:2cb3e6a5:::smithi11939506-3:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:12.221 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:33:12.221 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 62.543522019999997, 2024-09-18T11:33:12.221 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.024248042000000001, 2024-09-18T11:33:12.221 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.221 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:12.221 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:12.221 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:12.221 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:12.221 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3969 2024-09-18T11:33:12.221 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.221 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.221 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:33:12.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573640+0000", 2024-09-18T11:33:12.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.222 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.222 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.222 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573639+0000", 2024-09-18T11:33:12.222 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:12.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573646+0000", 2024-09-18T11:33:12.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6569999999999997e-06 2024-09-18T11:33:12.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573662+0000", 2024-09-18T11:33:12.223 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5187999999999999e-05 2024-09-18T11:33:12.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.223 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.223 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.223 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573670+0000", 2024-09-18T11:33:12.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5399999999999996e-06 2024-09-18T11:33:12.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:33:12.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8592999999999999e-05 2024-09-18T11:33:12.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.224 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.224 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573818+0000", 2024-09-18T11:33:12.224 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9388999999999995e-05 2024-09-18T11:33:12.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.224 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.225 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.225 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597888+0000", 2024-09-18T11:33:12.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.024069859999999998 2024-09-18T11:33:12.225 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.225 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.225 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.225 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3973 4.7 4:e73ba68f:::smithi11939506-1:head [tier-promote] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:12.225 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:33:12.225 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 62.543422941000003, 2024-09-18T11:33:12.225 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.023911297000000001, 2024-09-18T11:33:12.225 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.225 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:12.226 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:12.226 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:12.226 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:12.226 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3973 2024-09-18T11:33:12.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.226 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.226 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:33:12.226 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.226 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.226 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573739+0000", 2024-09-18T11:33:12.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573738+0000", 2024-09-18T11:33:12.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:12.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.227 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.227 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573743+0000", 2024-09-18T11:33:12.227 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0839999999999999e-06 2024-09-18T11:33:12.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.227 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573752+0000", 2024-09-18T11:33:12.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3080000000000008e-06 2024-09-18T11:33:12.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573758+0000", 2024-09-18T11:33:12.228 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2659999999999998e-06 2024-09-18T11:33:12.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.228 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.228 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573803+0000", 2024-09-18T11:33:12.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5243e-05 2024-09-18T11:33:12.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.573897+0000", 2024-09-18T11:33:12.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3398000000000003e-05 2024-09-18T11:33:12.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.229 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.229 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597650+0000", 2024-09-18T11:33:12.229 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.023753672999999999 2024-09-18T11:33:12.229 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.229 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.229 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.230 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.230 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4443 4.d 4:b9bb642b:::smithi11939506-27:67 [read 0~3512696,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T11:33:12.230 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:36.030743+0000", 2024-09-18T11:33:12.230 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.086419311999997, 2024-09-18T11:33:12.230 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013696302, 2024-09-18T11:33:12.230 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.230 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:12.230 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:12.230 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:12.230 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:12.230 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4443 2024-09-18T11:33:12.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.231 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030743+0000", 2024-09-18T11:33:12.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030743+0000", 2024-09-18T11:33:12.231 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.231 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.231 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.231 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030741+0000", 2024-09-18T11:33:12.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:12.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030751+0000", 2024-09-18T11:33:12.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6979999999999993e-06 2024-09-18T11:33:12.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.232 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.232 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.232 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030763+0000", 2024-09-18T11:33:12.232 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2311e-05 2024-09-18T11:33:12.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030769+0000", 2024-09-18T11:33:12.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6060000000000002e-06 2024-09-18T11:33:12.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.036777+0000", 2024-09-18T11:33:12.233 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060079089999999996 2024-09-18T11:33:12.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.233 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.233 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.233 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.036945+0000", 2024-09-18T11:33:12.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016867100000000001 2024-09-18T11:33:12.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.234 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.234 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044439+0000", 2024-09-18T11:33:12.234 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0074937149999999998 2024-09-18T11:33:12.234 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.234 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.234 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.234 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.234 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.234 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4444 4.d 4:b9bb642b:::smithi11939506-27:67 [sparse-read 0~1262403] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T11:33:12.234 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:33:12.234 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.086384518999999, 2024-09-18T11:33:12.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.014179987, 2024-09-18T11:33:12.235 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.235 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:12.235 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:12.235 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:12.235 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:12.235 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4444 2024-09-18T11:33:12.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.235 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.235 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.235 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.235 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:33:12.235 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:33:12.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030777+0000", 2024-09-18T11:33:12.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:12.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.236 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.236 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.236 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030780+0000", 2024-09-18T11:33:12.236 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.163e-06 2024-09-18T11:33:12.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030786+0000", 2024-09-18T11:33:12.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2879999999999998e-06 2024-09-18T11:33:12.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.237 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030791+0000", 2024-09-18T11:33:12.237 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2579999999999997e-06 2024-09-18T11:33:12.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.237 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044477+0000", 2024-09-18T11:33:12.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.013686188 2024-09-18T11:33:12.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044614+0000", 2024-09-18T11:33:12.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013727300000000001 2024-09-18T11:33:12.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.238 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.238 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.238 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.044957+0000", 2024-09-18T11:33:12.238 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000343331 2024-09-18T11:33:12.238 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.238 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.239 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.239 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4445 4.d 4:b9bb642b:::smithi11939506-27:67 [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-18T11:33:12.239 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:36.030796+0000", 2024-09-18T11:33:12.239 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.086366243000001, 2024-09-18T11:33:12.239 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.015916046999999999, 2024-09-18T11:33:12.239 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.239 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:12.239 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:12.239 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:12.239 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:12.239 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4445 2024-09-18T11:33:12.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.240 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030796+0000", 2024-09-18T11:33:12.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.240 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030796+0000", 2024-09-18T11:33:12.240 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.240 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030795+0000", 2024-09-18T11:33:12.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:12.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030799+0000", 2024-09-18T11:33:12.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5209999999999998e-06 2024-09-18T11:33:12.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.241 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.241 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030806+0000", 2024-09-18T11:33:12.241 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7599999999999997e-06 2024-09-18T11:33:12.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.030810+0000", 2024-09-18T11:33:12.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0489999999999997e-06 2024-09-18T11:33:12.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.242 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.045277+0000", 2024-09-18T11:33:12.242 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.014467683 2024-09-18T11:33:12.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.242 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.045394+0000", 2024-09-18T11:33:12.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000116969 2024-09-18T11:33:12.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.243 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.243 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:36.046712+0000", 2024-09-18T11:33:12.243 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013174669999999999 2024-09-18T11:33:12.243 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.243 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.243 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.244 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.244 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4683 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865643,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:33:12.244 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:45.318891+0000", 2024-09-18T11:33:12.244 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.798270626000001, 2024-09-18T11:33:12.244 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.017494045999999999, 2024-09-18T11:33:12.244 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.244 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:12.244 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:12.244 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:12.244 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:12.244 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4683 2024-09-18T11:33:12.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.245 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318891+0000", 2024-09-18T11:33:12.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.245 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318891+0000", 2024-09-18T11:33:12.245 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.245 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.245 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318891+0000", 2024-09-18T11:33:12.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:12.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318894+0000", 2024-09-18T11:33:12.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1379999999999999e-06 2024-09-18T11:33:12.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.246 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318900+0000", 2024-09-18T11:33:12.246 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6239999999999999e-06 2024-09-18T11:33:12.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.246 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.246 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318903+0000", 2024-09-18T11:33:12.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8249999999999998e-06 2024-09-18T11:33:12.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.330916+0000", 2024-09-18T11:33:12.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.012012063999999999 2024-09-18T11:33:12.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.247 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.331011+0000", 2024-09-18T11:33:12.247 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5388000000000002e-05 2024-09-18T11:33:12.247 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.247 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.247 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.248 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.336385+0000", 2024-09-18T11:33:12.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005374429 2024-09-18T11:33:12.248 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.248 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.248 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.248 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4684 4.d 4:b5a463bf:::smithi11939506-20:head [read 0~3865643,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:33:12.248 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:45.318908+0000", 2024-09-18T11:33:12.248 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 26.798254321000002, 2024-09-18T11:33:12.248 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.021211049999999999, 2024-09-18T11:33:12.248 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.248 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:12.248 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:12.248 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:12.249 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:12.249 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4684 2024-09-18T11:33:12.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.249 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318908+0000", 2024-09-18T11:33:12.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.249 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.249 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.249 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.249 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318908+0000", 2024-09-18T11:33:12.249 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318907+0000", 2024-09-18T11:33:12.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:12.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318910+0000", 2024-09-18T11:33:12.250 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.79e-06 2024-09-18T11:33:12.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.250 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.250 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.250 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318917+0000", 2024-09-18T11:33:12.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5740000000000001e-06 2024-09-18T11:33:12.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.318920+0000", 2024-09-18T11:33:12.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5729999999999999e-06 2024-09-18T11:33:12.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.251 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.251 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.336370+0000", 2024-09-18T11:33:12.251 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01745025 2024-09-18T11:33:12.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.251 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.336506+0000", 2024-09-18T11:33:12.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013603700000000001 2024-09-18T11:33:12.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.252 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.252 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.252 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.340119+0000", 2024-09-18T11:33:12.252 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00361218 2024-09-18T11:33:12.252 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.252 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.252 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.252 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.252 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.252 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:12.254 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:33:12.254 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:33:12.253+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:33:12.355 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:12.354+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:33:12.393 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:12.393 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:33:12.393 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:33:12.394 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:12.402 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:33:12.456 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:33:12.454+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:33:12.540 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:12.540 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:33:12.540 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:33:12.541 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:33:12.541 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:12.548 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:33:12.557 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:33:12.556+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:33:12.646 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:12.647 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:33:12.647 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:33:12.647 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:33:12.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.647 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:33:12.648 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:33:12.648 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 172.661187144, 2024-09-18T11:33:12.648 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:33:12.648 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.648 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.648 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.648 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.649 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:33:12.649 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.649 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:33:12.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:33:12.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:12.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.650 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.650 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:33:12.650 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:33:12.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:33:12.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:33:12.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:33:12.651 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:33:12.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.651 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.651 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:33:12.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:33:12.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:33:12.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:33:12.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.652 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.652 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:33:12.652 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:33:12.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.653 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.653 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:33:12.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:33:12.653 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.653 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.653 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.653 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:33:12.653 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:33:12.653 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 163.426356191, 2024-09-18T11:33:12.653 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:33:12.654 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.654 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.654 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.654 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:33:12.654 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.654 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:33:12.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:33:12.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:33:12.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.655 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.655 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:33:12.655 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:33:12.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:33:12.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:33:12.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.656 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.656 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:33:12.656 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:33:12.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:33:12.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:33:12.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.657 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.657 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:33:12.657 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:33:12.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:33:12.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:33:12.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.658 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.658 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:33:12.658 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:33:12.658 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.659 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.659 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.659 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:33:12.659 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:33:12.659 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 158.25244990600001, 2024-09-18T11:33:12.659 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:33:12.659 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.659 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.659 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.659 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.659 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:33:12.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:33:12.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.660 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.660 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:33:12.660 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:12.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:33:12.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:33:12.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:33:12.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:33:12.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.661 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.661 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:33:12.661 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:33:12.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:33:12.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:33:12.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.662 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.662 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:33:12.662 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:33:12.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:33:12.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:33:12.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.663 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.663 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:33:12.663 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:33:12.663 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.663 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.663 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.664 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:304 4.6 e69/67 4:67c1765a:::smithi11939506-30:17 v 69'35, mlcod=58'31)", 2024-09-18T11:33:12.664 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:33:12.664 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 141.298608812, 2024-09-18T11:33:12.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006377611, 2024-09-18T11:33:12.664 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.664 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.664 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.664 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.664 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:33:12.664 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:33:12.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:33:12.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:12.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.665 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.665 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345961+0000", 2024-09-18T11:33:12.665 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2050000000000008e-06 2024-09-18T11:33:12.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345969+0000", 2024-09-18T11:33:12.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0339999999999997e-06 2024-09-18T11:33:12.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.666 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.666 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345976+0000", 2024-09-18T11:33:12.666 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-18T11:33:12.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346026+0000", 2024-09-18T11:33:12.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9286999999999998e-05 2024-09-18T11:33:12.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.667 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346050+0000", 2024-09-18T11:33:12.667 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4397e-05 2024-09-18T11:33:12.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.667 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352293+0000", 2024-09-18T11:33:12.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062427170000000001 2024-09-18T11:33:12.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.668 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.668 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352330+0000", 2024-09-18T11:33:12.668 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7348999999999998e-05 2024-09-18T11:33:12.668 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.668 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.668 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.668 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:347 4.d e69/22 4:b44229a1:::smithi11939506-42:19 v 69'99, mlcod=65'93)", 2024-09-18T11:33:12.669 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:33:12.669 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 141.29287561699999, 2024-09-18T11:33:12.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056998760000000004, 2024-09-18T11:33:12.669 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.669 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.669 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.669 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.669 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:33:12.669 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:33:12.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351684+0000", 2024-09-18T11:33:12.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:12.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.670 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.670 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351693+0000", 2024-09-18T11:33:12.670 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-18T11:33:12.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351704+0000", 2024-09-18T11:33:12.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1077000000000001e-05 2024-09-18T11:33:12.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351709+0000", 2024-09-18T11:33:12.671 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5419999999999999e-06 2024-09-18T11:33:12.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.671 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.671 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351956+0000", 2024-09-18T11:33:12.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000247252 2024-09-18T11:33:12.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351975+0000", 2024-09-18T11:33:12.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9201000000000001e-05 2024-09-18T11:33:12.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.672 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.672 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357334+0000", 2024-09-18T11:33:12.672 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053588749999999999 2024-09-18T11:33:12.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.673 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.673 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357385+0000", 2024-09-18T11:33:12.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1406999999999998e-05 2024-09-18T11:33:12.673 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.673 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.673 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.673 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1956 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'37, mlcod=58'31)", 2024-09-18T11:33:12.673 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:33:12.673 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 141.29193070400001, 2024-09-18T11:33:12.673 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010958842, 2024-09-18T11:33:12.673 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.674 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.674 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:33:12.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.674 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:33:12.674 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.674 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:33:12.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:12.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352996+0000", 2024-09-18T11:33:12.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036622 2024-09-18T11:33:12.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.675 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.675 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353011+0000", 2024-09-18T11:33:12.675 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4849e-05 2024-09-18T11:33:12.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353017+0000", 2024-09-18T11:33:12.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:33:12.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353068+0000", 2024-09-18T11:33:12.676 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0695000000000002e-05 2024-09-18T11:33:12.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.676 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.676 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353092+0000", 2024-09-18T11:33:12.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4260000000000002e-05 2024-09-18T11:33:12.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363551+0000", 2024-09-18T11:33:12.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010459154999999999 2024-09-18T11:33:12.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.677 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.677 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363589+0000", 2024-09-18T11:33:12.677 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8161000000000003e-05 2024-09-18T11:33:12.677 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.678 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.678 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.678 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1957 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'38, mlcod=65'33)", 2024-09-18T11:33:12.678 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:33:12.678 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 141.28551483699999, 2024-09-18T11:33:12.678 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050499400000000002, 2024-09-18T11:33:12.678 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.678 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.678 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.678 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:33:12.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:33:12.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.679 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.679 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:33:12.679 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:12.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359214+0000", 2024-09-18T11:33:12.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000168047 2024-09-18T11:33:12.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359227+0000", 2024-09-18T11:33:12.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3124e-05 2024-09-18T11:33:12.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.680 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.680 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359235+0000", 2024-09-18T11:33:12.680 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0760000000000003e-06 2024-09-18T11:33:12.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359287+0000", 2024-09-18T11:33:12.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.151e-05 2024-09-18T11:33:12.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359310+0000", 2024-09-18T11:33:12.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-09-18T11:33:12.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.681 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.681 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364065+0000", 2024-09-18T11:33:12.681 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047549439999999997 2024-09-18T11:33:12.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.682 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.682 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364096+0000", 2024-09-18T11:33:12.682 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1423000000000001e-05 2024-09-18T11:33:12.682 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.682 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.682 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.682 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2300 4.4 e77/22 4:2c93196f:::smithi11939506-28:head v 77'94, mlcod=77'90)", 2024-09-18T11:33:12.682 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:33:12.682 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 131.65645261899999, 2024-09-18T11:33:12.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050425380000000001, 2024-09-18T11:33:12.683 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.683 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.683 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:33:12.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.683 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.683 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:33:12.683 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988108+0000", 2024-09-18T11:33:12.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:12.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.684 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.684 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988388+0000", 2024-09-18T11:33:12.684 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028087700000000001 2024-09-18T11:33:12.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988405+0000", 2024-09-18T11:33:12.685 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6565000000000001e-05 2024-09-18T11:33:12.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.685 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.685 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988412+0000", 2024-09-18T11:33:12.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9709999999999998e-06 2024-09-18T11:33:12.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988464+0000", 2024-09-18T11:33:12.686 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1949e-05 2024-09-18T11:33:12.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.686 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.686 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.686 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988490+0000", 2024-09-18T11:33:12.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5780999999999999e-05 2024-09-18T11:33:12.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993112+0000", 2024-09-18T11:33:12.687 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046228399999999996 2024-09-18T11:33:12.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.687 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.687 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993151+0000", 2024-09-18T11:33:12.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8587999999999997e-05 2024-09-18T11:33:12.688 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.688 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.688 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.688 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.13:63 3.0 e78/21 3:24ebb740:::smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 78'80, mlcod=73'78)", 2024-09-18T11:33:12.688 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:33:12.688 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 131.23946320300001, 2024-09-18T11:33:12.688 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052262230000000003, 2024-09-18T11:33:12.688 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.688 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.688 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:33:12.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:33:12.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.689 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.689 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405097+0000", 2024-09-18T11:33:12.689 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:12.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405103+0000", 2024-09-18T11:33:12.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8780000000000003e-06 2024-09-18T11:33:12.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.690 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.690 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405117+0000", 2024-09-18T11:33:12.690 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3441000000000001e-05 2024-09-18T11:33:12.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.690 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405121+0000", 2024-09-18T11:33:12.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7099999999999998e-06 2024-09-18T11:33:12.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405158+0000", 2024-09-18T11:33:12.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6477999999999999e-05 2024-09-18T11:33:12.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.691 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.691 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405173+0000", 2024-09-18T11:33:12.691 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4796e-05 2024-09-18T11:33:12.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410285+0000", 2024-09-18T11:33:12.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051118919999999998 2024-09-18T11:33:12.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.692 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.692 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410324+0000", 2024-09-18T11:33:12.692 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9616999999999997e-05 2024-09-18T11:33:12.692 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.692 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.692 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.692 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.692 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.13:18 3.0 e91/21 3:35e3c617:::2e64dd273da7619bd64b294f52905c1b9405a1ced2a8fd3c0b5bc481bc9732a6:head v 91'89, mlcod=85'87)", 2024-09-18T11:33:12.693 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:33:12.693 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.62268779199999, 2024-09-18T11:33:12.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048558539999999997, 2024-09-18T11:33:12.693 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.693 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.693 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:33:12.693 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.693 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.693 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.693 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:33:12.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:33:12.694 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:12.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.694 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.694 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021880+0000", 2024-09-18T11:33:12.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5860000000000001e-06 2024-09-18T11:33:12.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.695 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.695 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.695 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021890+0000", 2024-09-18T11:33:12.695 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8290000000000003e-06 2024-09-18T11:33:12.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021895+0000", 2024-09-18T11:33:12.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2480000000000004e-06 2024-09-18T11:33:12.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.696 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.696 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.696 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022166+0000", 2024-09-18T11:33:12.696 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027073499999999998 2024-09-18T11:33:12.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.697 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022184+0000", 2024-09-18T11:33:12.697 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8082000000000001e-05 2024-09-18T11:33:12.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.697 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.697 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026693+0000", 2024-09-18T11:33:12.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045093479999999998 2024-09-18T11:33:12.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.698 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.698 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.698 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026729+0000", 2024-09-18T11:33:12.698 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6282999999999998e-05 2024-09-18T11:33:12.699 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.699 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.699 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.699 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.699 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:573 4.4 e91/22 4:2c93196f:::smithi11939506-28:26 v 91'135, mlcod=90'131)", 2024-09-18T11:33:12.699 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:33:12.699 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.620221203, 2024-09-18T11:33:12.699 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050722739999999999, 2024-09-18T11:33:12.699 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.700 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.700 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:33:12.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.700 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.700 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.700 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:33:12.700 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024339+0000", 2024-09-18T11:33:12.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:12.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.701 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.701 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.701 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024349+0000", 2024-09-18T11:33:12.701 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6339999999999996e-06 2024-09-18T11:33:12.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.704 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.704 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024362+0000", 2024-09-18T11:33:12.704 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-18T11:33:12.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.705 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.705 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.705 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024368+0000", 2024-09-18T11:33:12.705 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8649999999999998e-06 2024-09-18T11:33:12.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.706 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.706 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024431+0000", 2024-09-18T11:33:12.706 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2780999999999994e-05 2024-09-18T11:33:12.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.707 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.707 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024448+0000", 2024-09-18T11:33:12.707 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7388999999999999e-05 2024-09-18T11:33:12.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.708 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.708 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029385+0000", 2024-09-18T11:33:12.708 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004936523 2024-09-18T11:33:12.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.709 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.709 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029412+0000", 2024-09-18T11:33:12.709 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7702999999999999e-05 2024-09-18T11:33:12.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.709 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.710 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:574 4.4 e91/22 4:2cb3e6a5:::smithi11939506-3:1e v 91'137, mlcod=90'131)", 2024-09-18T11:33:12.710 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:33:12.710 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.61978316, 2024-09-18T11:33:12.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070802850000000004, 2024-09-18T11:33:12.711 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.711 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.711 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.711 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.711 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:33:12.711 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:33:12.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024777+0000", 2024-09-18T11:33:12.712 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:12.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.712 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.712 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024785+0000", 2024-09-18T11:33:12.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8099999999999998e-06 2024-09-18T11:33:12.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024794+0000", 2024-09-18T11:33:12.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5839999999999995e-06 2024-09-18T11:33:12.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.713 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.713 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024798+0000", 2024-09-18T11:33:12.713 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.031e-06 2024-09-18T11:33:12.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024830+0000", 2024-09-18T11:33:12.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2268999999999999e-05 2024-09-18T11:33:12.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.714 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.714 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024847+0000", 2024-09-18T11:33:12.714 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7306999999999999e-05 2024-09-18T11:33:12.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031834+0000", 2024-09-18T11:33:12.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069868380000000004 2024-09-18T11:33:12.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.715 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.715 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031858+0000", 2024-09-18T11:33:12.715 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4082000000000001e-05 2024-09-18T11:33:12.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.715 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.716 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:541 4.d e91/22 4:b9bb642b:::smithi11939506-27:19 v 91'158, mlcod=91'154)", 2024-09-18T11:33:12.716 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:33:12.716 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.612402468, 2024-09-18T11:33:12.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052185, 2024-09-18T11:33:12.716 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.716 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.716 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.716 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.716 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:33:12.716 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.717 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.717 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:33:12.717 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.718 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.718 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032158+0000", 2024-09-18T11:33:12.718 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:12.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032165+0000", 2024-09-18T11:33:12.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-09-18T11:33:12.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.719 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.719 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032176+0000", 2024-09-18T11:33:12.719 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1460999999999999e-05 2024-09-18T11:33:12.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032182+0000", 2024-09-18T11:33:12.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0870000000000003e-06 2024-09-18T11:33:12.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.720 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.720 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032224+0000", 2024-09-18T11:33:12.720 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1372999999999998e-05 2024-09-18T11:33:12.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032246+0000", 2024-09-18T11:33:12.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2158e-05 2024-09-18T11:33:12.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037348+0000", 2024-09-18T11:33:12.721 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051025319999999999 2024-09-18T11:33:12.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.721 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.721 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037377+0000", 2024-09-18T11:33:12.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8937000000000001e-05 2024-09-18T11:33:12.722 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.722 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.722 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.722 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4178.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-18T11:33:12.722 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:33:12.722 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 90.703740402999998, 2024-09-18T11:33:12.722 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048960139999999997, 2024-09-18T11:33:12.722 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.722 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:12.722 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:12.723 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4178", 2024-09-18T11:33:12.723 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/490761693", 2024-09-18T11:33:12.723 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4 2024-09-18T11:33:12.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.723 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:33:12.723 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.723 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.723 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:33:12.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940819+0000", 2024-09-18T11:33:12.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:12.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.724 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.724 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940832+0000", 2024-09-18T11:33:12.724 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3155e-05 2024-09-18T11:33:12.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940848+0000", 2024-09-18T11:33:12.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6288999999999999e-05 2024-09-18T11:33:12.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940856+0000", 2024-09-18T11:33:12.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4919999999999996e-06 2024-09-18T11:33:12.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.725 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.725 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945613+0000", 2024-09-18T11:33:12.725 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047574999999999996 2024-09-18T11:33:12.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.726 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.726 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945717+0000", 2024-09-18T11:33:12.726 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103553 2024-09-18T11:33:12.726 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.726 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.726 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.726 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3482 4.6 e117/67 4:67c1765a:::smithi11939506-30:head v 117'120, mlcod=115'116)", 2024-09-18T11:33:12.726 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:33:12.726 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 85.959886179999998, 2024-09-18T11:33:12.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050037479999999997, 2024-09-18T11:33:12.727 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.727 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.727 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.727 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:33:12.727 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.727 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:33:12.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684674+0000", 2024-09-18T11:33:12.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:12.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.728 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.728 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684950+0000", 2024-09-18T11:33:12.728 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027621200000000002 2024-09-18T11:33:12.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684968+0000", 2024-09-18T11:33:12.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8e-05 2024-09-18T11:33:12.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.729 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684984+0000", 2024-09-18T11:33:12.729 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5775000000000001e-05 2024-09-18T11:33:12.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.729 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685033+0000", 2024-09-18T11:33:12.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8878000000000002e-05 2024-09-18T11:33:12.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.730 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685059+0000", 2024-09-18T11:33:12.730 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6801999999999999e-05 2024-09-18T11:33:12.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.730 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689640+0000", 2024-09-18T11:33:12.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045809029999999999 2024-09-18T11:33:12.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.731 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.731 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689679+0000", 2024-09-18T11:33:12.731 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8507000000000002e-05 2024-09-18T11:33:12.731 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.731 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.731 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.731 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:976 4.4 e132/22 4:2cb3e6a5:::smithi11939506-3:30 v 132'195, mlcod=128'192)", 2024-09-18T11:33:12.732 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:33:12.732 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 69.625915109000005, 2024-09-18T11:33:12.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0058526510000000004, 2024-09-18T11:33:12.732 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.732 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.732 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:33:12.732 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.732 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.732 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:33:12.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:33:12.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:12.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.733 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.733 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018661+0000", 2024-09-18T11:33:12.733 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5565000000000001e-05 2024-09-18T11:33:12.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018672+0000", 2024-09-18T11:33:12.734 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0210000000000001e-05 2024-09-18T11:33:12.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.734 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.734 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.734 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018678+0000", 2024-09-18T11:33:12.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1539999999999999e-06 2024-09-18T11:33:12.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018716+0000", 2024-09-18T11:33:12.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8389000000000001e-05 2024-09-18T11:33:12.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.735 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.735 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.735 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018737+0000", 2024-09-18T11:33:12.735 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0611e-05 2024-09-18T11:33:12.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.024458+0000", 2024-09-18T11:33:12.736 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057208629999999996 2024-09-18T11:33:12.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.736 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.736 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.024499+0000", 2024-09-18T11:33:12.737 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1100999999999998e-05 2024-09-18T11:33:12.737 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.737 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.737 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.737 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.738 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:823 4.d e137/22 4:b44229a1:::smithi11939506-42:49 v 137'266, mlcod=132'260)", 2024-09-18T11:33:12.738 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:33:12.738 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 64.367266552999993, 2024-09-18T11:33:12.738 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054497770000000003, 2024-09-18T11:33:12.738 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.738 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.738 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.738 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.738 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.738 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:33:12.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:33:12.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.739 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.739 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277294+0000", 2024-09-18T11:33:12.739 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:33:12.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.739 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.740 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277299+0000", 2024-09-18T11:33:12.740 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6280000000000004e-06 2024-09-18T11:33:12.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.740 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.740 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277306+0000", 2024-09-18T11:33:12.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.533e-06 2024-09-18T11:33:12.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277316+0000", 2024-09-18T11:33:12.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0148e-05 2024-09-18T11:33:12.741 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.741 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.741 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.741 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277868+0000", 2024-09-18T11:33:12.741 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00055199999999999997 2024-09-18T11:33:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277898+0000", 2024-09-18T11:33:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0264999999999999e-05 2024-09-18T11:33:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.282715+0000", 2024-09-18T11:33:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048165619999999999 2024-09-18T11:33:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.742 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.282744+0000", 2024-09-18T11:33:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9855000000000001e-05 2024-09-18T11:33:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:163 3.0 e138/21 3:02e8b30e:::4456d047c09ae2a9a25ccebfd83e30cf78df6779d63bfba9365afd7301eaeb6a:head v 138'132, mlcod=134'130)", 2024-09-18T11:33:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:33:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.062403809999999, 2024-09-18T11:33:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051351770000000003, 2024-09-18T11:33:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.743 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:33:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:33:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.744 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:33:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582232+0000", 2024-09-18T11:33:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5333999999999994e-05 2024-09-18T11:33:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582242+0000", 2024-09-18T11:33:12.745 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5200000000000003e-06 2024-09-18T11:33:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582248+0000", 2024-09-18T11:33:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8000000000000004e-06 2024-09-18T11:33:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582305+0000", 2024-09-18T11:33:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7813999999999999e-05 2024-09-18T11:33:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.746 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.747 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.747 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582329+0000", 2024-09-18T11:33:12.747 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3414999999999999e-05 2024-09-18T11:33:12.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.747 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587257+0000", 2024-09-18T11:33:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049279880000000003 2024-09-18T11:33:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587293+0000", 2024-09-18T11:33:12.748 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5802e-05 2024-09-18T11:33:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3974 4.d e138/22 4:b5a463bf:::smithi11939506-20:head v 138'270, mlcod=137'266)", 2024-09-18T11:33:12.749 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:33:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.058522539000002, 2024-09-18T11:33:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011228458, 2024-09-18T11:33:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.750 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:33:12.751 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.751 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.751 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.751 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:33:12.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586038+0000", 2024-09-18T11:33:12.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:12.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.752 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.752 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.752 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587638+0000", 2024-09-18T11:33:12.752 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001599184 2024-09-18T11:33:12.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587647+0000", 2024-09-18T11:33:12.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3719999999999994e-06 2024-09-18T11:33:12.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.753 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587653+0000", 2024-09-18T11:33:12.753 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.105e-06 2024-09-18T11:33:12.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.753 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.753 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587712+0000", 2024-09-18T11:33:12.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9252000000000001e-05 2024-09-18T11:33:12.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587735+0000", 2024-09-18T11:33:12.754 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3132000000000001e-05 2024-09-18T11:33:12.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.754 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.754 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.754 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597228+0000", 2024-09-18T11:33:12.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0094930090000000002 2024-09-18T11:33:12.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.755 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.755 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597267+0000", 2024-09-18T11:33:12.755 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8693999999999999e-05 2024-09-18T11:33:12.755 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.755 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.755 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.755 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.755 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3975 4.d e138/22 4:b5a463bf:::smithi11939506-20:head v 138'271, mlcod=137'266)", 2024-09-18T11:33:12.755 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:33:12.755 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.052321833000001, 2024-09-18T11:33:12.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053620450000000002, 2024-09-18T11:33:12.756 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:12.756 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:12.756 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:12.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:12.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:33:12.756 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.756 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.756 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:12.756 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:33:12.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:12.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:12.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:33:12.757 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:12.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.757 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.757 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:12.757 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592927+0000", 2024-09-18T11:33:12.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00068878699999999995 2024-09-18T11:33:12.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:12.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592940+0000", 2024-09-18T11:33:12.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:33:12.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.758 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:12.758 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592951+0000", 2024-09-18T11:33:12.758 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1046000000000001e-05 2024-09-18T11:33:12.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:12.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593008+0000", 2024-09-18T11:33:12.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6875e-05 2024-09-18T11:33:12.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:12.759 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593033+0000", 2024-09-18T11:33:12.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5341999999999999e-05 2024-09-18T11:33:12.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.759 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:12.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597565+0000", 2024-09-18T11:33:12.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045323150000000003 2024-09-18T11:33:12.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:12.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:12.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:12.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597601+0000", 2024-09-18T11:33:12.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6071000000000001e-05 2024-09-18T11:33:12.760 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.760 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.760 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.760 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:12.760 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:12.760 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:12.762 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:33:12.763 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:33:12.761+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:33:12.861 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:12.861 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:33:12.861 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:33:12.861 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:12.863 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:33:12.861+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:33:12.869 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:33:12.964 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:33:12.963+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:33:12.967 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:33:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:33:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:33:12.967 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:12.976 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:33:13.065 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:13.064+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:13.076 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:13.076 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:33:13.076 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:33:13.076 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:33:13.076 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.076 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:33:13.076 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:33:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 181.904368619, 2024-09-18T11:33:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:33:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:33:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:13.077 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:33:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:33:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:33:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.078 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:33:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:33:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:33:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:33:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:33:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:33:13.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:33:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:33:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:33:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:33:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.080 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:33:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:33:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:33:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:33:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 177.90919062500001, 2024-09-18T11:33:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:33:13.081 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:33:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:33:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.082 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:33:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:33:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:33:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:33:13.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:33:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:33:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:33:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:33:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.084 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:33:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:33:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:33:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:33:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:33:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:33:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.085 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:33:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:33:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:33:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:33:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:33:13.086 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:33:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 177.90917233600001, 2024-09-18T11:33:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:33:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:33:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:33:13.087 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:33:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:33:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:13.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.089 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.089 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:13.089 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:33:13.089 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:33:13.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.090 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:13.090 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:33:13.090 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:33:13.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.090 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:33:13.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:33:13.091 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.091 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.091 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.091 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:33:13.091 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:33:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:33:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:33:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:33:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:33:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:33:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:33:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:33:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:33:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.093 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:33:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:33:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 177.909111567, 2024-09-18T11:33:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:33:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:33:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:13.094 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:33:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:33:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:33:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.095 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:33:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:33:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:33:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:33:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.096 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:33:13.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:33:13.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.097 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.097 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.097 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:33:13.097 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:33:13.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:13.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:33:13.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:33:13.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.098 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:33:13.098 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:33:13.098 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.098 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.098 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:13.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:33:13.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:33:13.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.099 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:13.099 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:33:13.099 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:33:13.099 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.099 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:13.099 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.099 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.099 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:33:13.099 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:33:13.100 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 177.90908924600001, 2024-09-18T11:33:13.100 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:33:13.100 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:13.100 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:13.100 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:13.100 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:13.100 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.100 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:33:13.100 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.100 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:13.100 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.100 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:13.100 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:33:13.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:13.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:33:13.101 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.101 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.101 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:13.101 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:33:13.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:13.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:13.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:33:13.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:33:13.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:13.102 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:33:13.102 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:33:13.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.102 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.102 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:33:13.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:33:13.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:33:13.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:33:13.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:13.103 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:33:13.103 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:33:13.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.103 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.103 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:33:13.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:33:13.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:13.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:33:13.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:33:13.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.104 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.104 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:13.104 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:33:13.104 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:33:13.105 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.105 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:13.105 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.105 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:33:13.105 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:33:13.105 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 173.12796436299999, 2024-09-18T11:33:13.105 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:33:13.105 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:13.105 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.105 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:13.106 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:13.106 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.106 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:33:13.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.106 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:13.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:13.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:33:13.106 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.106 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.106 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.106 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:13.106 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:33:13.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:13.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:33:13.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:33:13.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.107 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:13.107 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:33:13.107 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:33:13.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.107 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:13.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:33:13.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:33:13.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:33:13.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:33:13.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.108 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.108 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:33:13.108 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:33:13.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:13.109 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:33:13.109 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:33:13.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.109 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.109 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:13.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:33:13.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:33:13.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.110 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.110 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:13.110 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:33:13.110 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:33:13.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.111 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:13.111 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:33:13.111 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:33:13.111 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.111 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:13.112 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:33:13.112 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:33:13.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.112 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:13.113 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:33:13.113 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:33:13.113 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.113 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:13.113 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.114 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2503 3.1 e85/67 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head v 85'117, mlcod=84'115)", 2024-09-18T11:33:13.114 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:33:13.114 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 124.096565676, 2024-09-18T11:33:13.114 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086527619999999996, 2024-09-18T11:33:13.114 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:13.114 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.115 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:13.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:13.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:33:13.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.115 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:13.115 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:33:13.115 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:13.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977220+0000", 2024-09-18T11:33:13.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:33:13.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.116 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:13.116 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980771+0000", 2024-09-18T11:33:13.116 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035505200000000002 2024-09-18T11:33:13.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:13.117 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980797+0000", 2024-09-18T11:33:13.117 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.618e-05 2024-09-18T11:33:13.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.117 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980807+0000", 2024-09-18T11:33:13.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0433e-05 2024-09-18T11:33:13.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980861+0000", 2024-09-18T11:33:13.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3415999999999998e-05 2024-09-18T11:33:13.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.118 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:13.118 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980885+0000", 2024-09-18T11:33:13.118 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4212000000000001e-05 2024-09-18T11:33:13.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:13.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985826+0000", 2024-09-18T11:33:13.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049412450000000004 2024-09-18T11:33:13.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.119 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:13.119 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985875+0000", 2024-09-18T11:33:13.119 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9116000000000002e-05 2024-09-18T11:33:13.119 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.119 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:13.119 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.120 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2649 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:33:13.120 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:33:13.120 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.742422307, 2024-09-18T11:33:13.120 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566240000000007, 2024-09-18T11:33:13.120 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:13.120 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:13.120 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:13.120 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:13.120 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.120 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2649 2024-09-18T11:33:13.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.120 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:13.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:13.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:33:13.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:13.121 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:33:13.121 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.121 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:13.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331365+0000", 2024-09-18T11:33:13.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:13.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:13.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331369+0000", 2024-09-18T11:33:13.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.551e-06 2024-09-18T11:33:13.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:13.122 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331377+0000", 2024-09-18T11:33:13.122 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8769999999999993e-06 2024-09-18T11:33:13.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.122 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331381+0000", 2024-09-18T11:33:13.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.16e-06 2024-09-18T11:33:13.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.123 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338973+0000", 2024-09-18T11:33:13.123 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075918460000000002 2024-09-18T11:33:13.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.123 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:13.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339048+0000", 2024-09-18T11:33:13.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5813000000000001e-05 2024-09-18T11:33:13.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.124 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:13.124 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339322+0000", 2024-09-18T11:33:13.124 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027399899999999998 2024-09-18T11:33:13.124 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.124 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:13.124 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.124 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2650 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:33:13.125 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:33:13.125 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 118.742402089, 2024-09-18T11:33:13.125 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084392180000000001, 2024-09-18T11:33:13.125 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:13.125 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:13.125 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:13.125 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:13.125 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.125 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2650 2024-09-18T11:33:13.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.125 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:13.125 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.125 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:13.125 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:33:13.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:13.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:33:13.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.126 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:13.126 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:33:13.126 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:13.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:13.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331388+0000", 2024-09-18T11:33:13.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3429999999999998e-06 2024-09-18T11:33:13.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:13.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331394+0000", 2024-09-18T11:33:13.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.19e-06 2024-09-18T11:33:13.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.127 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.127 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.127 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331398+0000", 2024-09-18T11:33:13.127 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.495e-06 2024-09-18T11:33:13.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339362+0000", 2024-09-18T11:33:13.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079641799999999995 2024-09-18T11:33:13.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.128 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.128 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:13.128 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339460+0000", 2024-09-18T11:33:13.128 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8329e-05 2024-09-18T11:33:13.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.129 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:13.129 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339825+0000", 2024-09-18T11:33:13.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036510500000000001 2024-09-18T11:33:13.129 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.129 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:13.129 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.129 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.129 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:127 3.2 3:4d3b277a:::3266c7aaa53a7d2c10c3c54d852d6cbbed49fb041bc0df41a3ac83b56e2b0615:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:33:13.129 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:33:13.129 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 110.02938715000001, 2024-09-18T11:33:13.129 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093148009999999993, 2024-09-18T11:33:13.130 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:13.130 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.130 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:13.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:13.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:33:13.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.130 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.130 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:13.130 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:33:13.130 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:13.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044400+0000", 2024-09-18T11:33:13.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:13.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:13.131 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044442+0000", 2024-09-18T11:33:13.131 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2247999999999999e-05 2024-09-18T11:33:13.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.131 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.131 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:13.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044453+0000", 2024-09-18T11:33:13.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0617e-05 2024-09-18T11:33:13.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044457+0000", 2024-09-18T11:33:13.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2520000000000001e-06 2024-09-18T11:33:13.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.132 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.132 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.132 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044495+0000", 2024-09-18T11:33:13.132 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7710000000000003e-05 2024-09-18T11:33:13.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:13.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044499+0000", 2024-09-18T11:33:13.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6269999999999999e-06 2024-09-18T11:33:13.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.133 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.133 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048239+0000", 2024-09-18T11:33:13.133 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037396719999999999 2024-09-18T11:33:13.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:13.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048351+0000", 2024-09-18T11:33:13.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112342 2024-09-18T11:33:13.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:13.134 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048545+0000", 2024-09-18T11:33:13.134 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000193881 2024-09-18T11:33:13.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.134 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.134 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:13.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.052689+0000", 2024-09-18T11:33:13.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041435780000000002 2024-09-18T11:33:13.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:13.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053654+0000", 2024-09-18T11:33:13.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096558199999999999 2024-09-18T11:33:13.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.135 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.135 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:13.135 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053683+0000", 2024-09-18T11:33:13.135 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8850999999999999e-05 2024-09-18T11:33:13.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.136 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:13.136 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053716+0000", 2024-09-18T11:33:13.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2468000000000002e-05 2024-09-18T11:33:13.136 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.136 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:13.136 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.136 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3209 4.0 4:019b6e09:::smithi11939506-40:head [write 638732~557432] snapc 42=[42,41,3f,3e,3d,3b,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-18T11:33:13.136 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:33:13.136 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 95.975705110999996, 2024-09-18T11:33:13.136 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091774999999999999, 2024-09-18T11:33:13.136 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:13.137 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.137 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:13.137 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:13.137 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.137 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3209 2024-09-18T11:33:13.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.137 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:13.137 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.137 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:13.137 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:33:13.137 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:13.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:33:13.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:13.138 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098080+0000", 2024-09-18T11:33:13.138 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:33:13.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.138 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.138 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:13.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098676+0000", 2024-09-18T11:33:13.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059598099999999998 2024-09-18T11:33:13.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:13.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098706+0000", 2024-09-18T11:33:13.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0020000000000001e-05 2024-09-18T11:33:13.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.139 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.139 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.139 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098716+0000", 2024-09-18T11:33:13.139 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0225e-05 2024-09-18T11:33:13.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098782+0000", 2024-09-18T11:33:13.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5530000000000004e-05 2024-09-18T11:33:13.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:13.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098883+0000", 2024-09-18T11:33:13.140 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101609 2024-09-18T11:33:13.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.140 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:33:13.140 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.099098+0000", 2024-09-18T11:33:13.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000215052 2024-09-18T11:33:13.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.141 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:13.141 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.102511+0000", 2024-09-18T11:33:13.141 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034129429999999999 2024-09-18T11:33:13.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:13.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107165+0000", 2024-09-18T11:33:13.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046531480000000002 2024-09-18T11:33:13.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.142 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.142 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:13.142 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107217+0000", 2024-09-18T11:33:13.142 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2058000000000002e-05 2024-09-18T11:33:13.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.143 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:13.143 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107260+0000", 2024-09-18T11:33:13.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3865000000000001e-05 2024-09-18T11:33:13.143 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.143 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:13.143 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.143 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.143 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:162 3.1 e138/67 3:8c826a1f:::e682cfc1e82c88472b9acf1cd97978bc105aa035f10b685bb4cace95809b52bf:head v 138'176, mlcod=136'174)", 2024-09-18T11:33:13.143 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:33:13.143 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.490231366000003, 2024-09-18T11:33:13.143 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010445024000000001, 2024-09-18T11:33:13.144 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:13.144 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.144 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:13.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:13.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:33:13.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.144 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.144 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:13.144 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:33:13.144 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.144 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:13.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583556+0000", 2024-09-18T11:33:13.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:13.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:13.145 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584899+0000", 2024-09-18T11:33:13.145 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001343056 2024-09-18T11:33:13.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.145 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.145 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:13.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584929+0000", 2024-09-18T11:33:13.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0133000000000001e-05 2024-09-18T11:33:13.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584937+0000", 2024-09-18T11:33:13.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6380000000000004e-06 2024-09-18T11:33:13.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.146 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.146 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584990+0000", 2024-09-18T11:33:13.146 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3186000000000002e-05 2024-09-18T11:33:13.146 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.146 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:13.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585012+0000", 2024-09-18T11:33:13.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1474000000000001e-05 2024-09-18T11:33:13.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:13.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593963+0000", 2024-09-18T11:33:13.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089512859999999993 2024-09-18T11:33:13.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.147 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:13.147 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594002+0000", 2024-09-18T11:33:13.147 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8859999999999997e-05 2024-09-18T11:33:13.148 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.148 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:13.148 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.148 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.148 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:166 3.1 e138/67 3:a73e635b:::b6ee478f0f661ec7bac3d17c2ca2fdaff71a47570f64844a7ba79bbb2a0e8d9d:head v 138'177, mlcod=136'174)", 2024-09-18T11:33:13.148 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:33:13.148 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.488661481999998, 2024-09-18T11:33:13.148 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089018680000000003, 2024-09-18T11:33:13.148 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:13.148 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.148 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:13.148 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:13.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:33:13.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:13.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:33:13.149 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.149 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.149 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:13.149 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585126+0000", 2024-09-18T11:33:13.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:13.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:13.150 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585211+0000", 2024-09-18T11:33:13.150 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5432999999999993e-05 2024-09-18T11:33:13.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.150 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.150 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:13.151 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585220+0000", 2024-09-18T11:33:13.151 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5879999999999998e-06 2024-09-18T11:33:13.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.151 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.151 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585225+0000", 2024-09-18T11:33:13.152 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8999999999999997e-06 2024-09-18T11:33:13.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.152 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.152 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.152 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585705+0000", 2024-09-18T11:33:13.153 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00048042799999999997 2024-09-18T11:33:13.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.153 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.153 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:13.153 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585724+0000", 2024-09-18T11:33:13.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8352e-05 2024-09-18T11:33:13.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:13.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594008+0000", 2024-09-18T11:33:13.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082840919999999998 2024-09-18T11:33:13.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.154 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.154 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:13.154 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594028+0000", 2024-09-18T11:33:13.154 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0869e-05 2024-09-18T11:33:13.155 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.155 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:13.155 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.155 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.155 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.155 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:168 3.1 e138/67 3:b64d7bad:::7feb22256bdcec217720fd46184ab43ced8923ed66ec6319c57f078d9f123262:head v 138'178, mlcod=136'174)", 2024-09-18T11:33:13.156 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:33:13.156 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 63.487669748999998, 2024-09-18T11:33:13.156 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079290650000000008, 2024-09-18T11:33:13.156 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:13.156 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.156 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:13.156 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.156 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:13.156 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:33:13.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:13.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:33:13.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.157 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:13.157 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:33:13.157 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:13.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.157 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:13.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586191+0000", 2024-09-18T11:33:13.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3627000000000005e-05 2024-09-18T11:33:13.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:13.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586202+0000", 2024-09-18T11:33:13.158 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0488e-05 2024-09-18T11:33:13.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.158 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.158 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.158 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586207+0000", 2024-09-18T11:33:13.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1719999999999998e-06 2024-09-18T11:33:13.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.159 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.159 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586239+0000", 2024-09-18T11:33:13.159 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2570000000000002e-05 2024-09-18T11:33:13.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:13.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586271+0000", 2024-09-18T11:33:13.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1341999999999999e-05 2024-09-18T11:33:13.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.160 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.160 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:13.160 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594031+0000", 2024-09-18T11:33:13.160 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077597459999999997 2024-09-18T11:33:13.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.161 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:13.161 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594047+0000", 2024-09-18T11:33:13.161 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6858e-05 2024-09-18T11:33:13.161 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.161 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:13.161 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.161 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.161 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.161 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4267 3.1 3:8d8d13e9:::92886a4340d03039afb2c877fad76e3afe7a80ee04388d55256c386022cd5d63:head [write 0~2203648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:33:13.161 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:33:13.161 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 46.016637768999999, 2024-09-18T11:33:13.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0092856740000000007, 2024-09-18T11:33:13.162 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:13.162 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.162 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:13.162 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:13.162 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.162 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4267 2024-09-18T11:33:13.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.162 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:13.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.162 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:13.162 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:33:13.162 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:13.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:33:13.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:13.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057149+0000", 2024-09-18T11:33:13.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:13.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.163 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.163 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:13.163 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058175+0000", 2024-09-18T11:33:13.163 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001026765 2024-09-18T11:33:13.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:13.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058190+0000", 2024-09-18T11:33:13.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4167e-05 2024-09-18T11:33:13.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.164 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.164 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058195+0000", 2024-09-18T11:33:13.164 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.823e-06 2024-09-18T11:33:13.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.164 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058247+0000", 2024-09-18T11:33:13.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1304999999999998e-05 2024-09-18T11:33:13.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:13.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058422+0000", 2024-09-18T11:33:13.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017527100000000001 2024-09-18T11:33:13.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.165 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.165 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:13.165 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058609+0000", 2024-09-18T11:33:13.165 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018714299999999999 2024-09-18T11:33:13.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:13.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.062721+0000", 2024-09-18T11:33:13.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041118869999999998 2024-09-18T11:33:13.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:13.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066291+0000", 2024-09-18T11:33:13.166 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035694860000000002 2024-09-18T11:33:13.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.166 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.166 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:13.166 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066328+0000", 2024-09-18T11:33:13.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7864000000000002e-05 2024-09-18T11:33:13.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.167 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:13.167 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066436+0000", 2024-09-18T11:33:13.167 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010762 2024-09-18T11:33:13.167 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.167 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:13.167 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.167 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.167 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.167 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:90 3.1 3:a224eaa1:::22b064a083114a2756d85a07d2762a393d3e92dd898a5334bc045c9031244d76:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:33:13.167 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:40.057559+0000", 2024-09-18T11:33:13.167 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.016228990000002, 2024-09-18T11:33:13.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085024620000000006, 2024-09-18T11:33:13.168 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:13.168 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.168 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:13.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:13.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057559+0000", 2024-09-18T11:33:13.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.168 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:13.168 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057559+0000", 2024-09-18T11:33:13.168 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:13.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057558+0000", 2024-09-18T11:33:13.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:13.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:13.169 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057563+0000", 2024-09-18T11:33:13.169 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7990000000000001e-06 2024-09-18T11:33:13.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.169 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:13.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057572+0000", 2024-09-18T11:33:13.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0510000000000002e-06 2024-09-18T11:33:13.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057576+0000", 2024-09-18T11:33:13.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2509999999999996e-06 2024-09-18T11:33:13.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.170 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.170 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057624+0000", 2024-09-18T11:33:13.170 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7636000000000003e-05 2024-09-18T11:33:13.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:13.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057722+0000", 2024-09-18T11:33:13.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7676999999999993e-05 2024-09-18T11:33:13.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:13.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057883+0000", 2024-09-18T11:33:13.171 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000160922 2024-09-18T11:33:13.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.171 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:13.171 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059927+0000", 2024-09-18T11:33:13.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020447350000000002 2024-09-18T11:33:13.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:13.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.065977+0000", 2024-09-18T11:33:13.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060497679999999996 2024-09-18T11:33:13.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.172 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:13.172 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066023+0000", 2024-09-18T11:33:13.172 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6041000000000002e-05 2024-09-18T11:33:13.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.173 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:13.173 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066062+0000", 2024-09-18T11:33:13.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8469e-05 2024-09-18T11:33:13.173 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.173 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:13.173 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.173 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.5.13:42 3.1 3:8372521c:::d4b4fff22a71c591f3e871de3fede050f6721df41a8c639a614c0b94ba6762c2:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:33:13.173 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:40.058327+0000", 2024-09-18T11:33:13.173 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.015461317000003, 2024-09-18T11:33:13.173 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079034699999999992, 2024-09-18T11:33:13.173 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:13.174 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.174 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:13.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:13.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058327+0000", 2024-09-18T11:33:13.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:13.174 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058327+0000", 2024-09-18T11:33:13.174 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.174 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:13.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058325+0000", 2024-09-18T11:33:13.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:13.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:13.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058339+0000", 2024-09-18T11:33:13.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3337e-05 2024-09-18T11:33:13.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.175 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:13.175 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058357+0000", 2024-09-18T11:33:13.175 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8615e-05 2024-09-18T11:33:13.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058366+0000", 2024-09-18T11:33:13.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9460000000000004e-06 2024-09-18T11:33:13.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058658+0000", 2024-09-18T11:33:13.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029212200000000002 2024-09-18T11:33:13.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.176 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.176 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:13.176 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058757+0000", 2024-09-18T11:33:13.176 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9025999999999997e-05 2024-09-18T11:33:13.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:13.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058914+0000", 2024-09-18T11:33:13.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000156236 2024-09-18T11:33:13.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:13.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.060326+0000", 2024-09-18T11:33:13.177 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014118710000000001 2024-09-18T11:33:13.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.177 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.177 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:13.177 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066122+0000", 2024-09-18T11:33:13.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057962029999999998 2024-09-18T11:33:13.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:13.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066192+0000", 2024-09-18T11:33:13.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0533999999999999e-05 2024-09-18T11:33:13.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.178 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:13.178 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066230+0000", 2024-09-18T11:33:13.178 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7914000000000003e-05 2024-09-18T11:33:13.178 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.178 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:13.178 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.179 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.5.13:43 3.1 3:b44664f3:::7f31751f088e4f7577561c03b406e8a8bc48ebfdd310b40206061655307aebe3:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:33:13.179 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:40.058386+0000", 2024-09-18T11:33:13.179 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 33.015402289000001, 2024-09-18T11:33:13.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079575740000000002, 2024-09-18T11:33:13.179 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:13.179 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.179 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:13.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.179 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:13.179 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058386+0000", 2024-09-18T11:33:13.179 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:13.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058386+0000", 2024-09-18T11:33:13.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:13.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058385+0000", 2024-09-18T11:33:13.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:13.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.180 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:13.180 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058392+0000", 2024-09-18T11:33:13.180 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6200000000000001e-06 2024-09-18T11:33:13.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:13.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058400+0000", 2024-09-18T11:33:13.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7849999999999997e-06 2024-09-18T11:33:13.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058408+0000", 2024-09-18T11:33:13.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1770000000000001e-06 2024-09-18T11:33:13.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.181 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.181 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059302+0000", 2024-09-18T11:33:13.181 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000894851 2024-09-18T11:33:13.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:13.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059400+0000", 2024-09-18T11:33:13.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7460000000000005e-05 2024-09-18T11:33:13.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:13.182 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059520+0000", 2024-09-18T11:33:13.182 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000120457 2024-09-18T11:33:13.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.182 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:13.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.060333+0000", 2024-09-18T11:33:13.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00081255599999999998 2024-09-18T11:33:13.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:13.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066274+0000", 2024-09-18T11:33:13.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059409700000000003 2024-09-18T11:33:13.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.183 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:13.183 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066306+0000", 2024-09-18T11:33:13.183 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2020000000000002e-05 2024-09-18T11:33:13.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.184 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:13.184 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066343+0000", 2024-09-18T11:33:13.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7438999999999998e-05 2024-09-18T11:33:13.184 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.184 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:13.184 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.184 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.184 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.4.13:145 3.2 3:435dd00a:::773f1f20723e27798deecfe7048ee8e76eeec814e6d6ad791ed305ff28c69be2:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:33:13.184 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:45.321055+0000", 2024-09-18T11:33:13.184 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 27.752733484, 2024-09-18T11:33:13.184 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079066459999999998, 2024-09-18T11:33:13.185 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:13.185 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.185 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:13.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:13.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321055+0000", 2024-09-18T11:33:13.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.185 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:13.185 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321055+0000", 2024-09-18T11:33:13.185 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:13.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321053+0000", 2024-09-18T11:33:13.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:13.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:13.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321062+0000", 2024-09-18T11:33:13.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0399999999999998e-06 2024-09-18T11:33:13.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.186 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:13.186 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321076+0000", 2024-09-18T11:33:13.186 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3869999999999999e-05 2024-09-18T11:33:13.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321083+0000", 2024-09-18T11:33:13.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6159999999999998e-06 2024-09-18T11:33:13.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321156+0000", 2024-09-18T11:33:13.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3093e-05 2024-09-18T11:33:13.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:13.187 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321274+0000", 2024-09-18T11:33:13.187 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000118353 2024-09-18T11:33:13.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.187 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:13.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321443+0000", 2024-09-18T11:33:13.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016836600000000001 2024-09-18T11:33:13.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:13.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.324281+0000", 2024-09-18T11:33:13.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028381270000000002 2024-09-18T11:33:13.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.188 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:13.188 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.328785+0000", 2024-09-18T11:33:13.188 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045048370000000003 2024-09-18T11:33:13.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:13.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.328910+0000", 2024-09-18T11:33:13.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012438600000000001 2024-09-18T11:33:13.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.189 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:13.189 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.328961+0000", 2024-09-18T11:33:13.189 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1375000000000002e-05 2024-09-18T11:33:13.189 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.189 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:13.189 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.189 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4790 3.2 3:79129b11:::2f02c59c01a730562a65f21445c8af0e4a8aab3b969b8a3abf39e7b1bc84f918:head [write 0~2177536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-18T11:33:13.190 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.041705+0000", 2024-09-18T11:33:13.190 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 20.032082871, 2024-09-18T11:33:13.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011619633000000001, 2024-09-18T11:33:13.190 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:13.190 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.190 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:13.190 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:13.190 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.190 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4790 2024-09-18T11:33:13.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.190 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:13.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.190 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:13.190 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.041705+0000", 2024-09-18T11:33:13.190 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:13.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.041705+0000", 2024-09-18T11:33:13.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:13.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:13.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.041703+0000", 2024-09-18T11:33:13.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:13.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.191 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:13.191 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.042939+0000", 2024-09-18T11:33:13.191 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012361570000000001 2024-09-18T11:33:13.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:13.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.042961+0000", 2024-09-18T11:33:13.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1531000000000001e-05 2024-09-18T11:33:13.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.192 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.192 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.042969+0000", 2024-09-18T11:33:13.192 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.892e-06 2024-09-18T11:33:13.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:13.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.043013+0000", 2024-09-18T11:33:13.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3921000000000002e-05 2024-09-18T11:33:13.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:13.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.043147+0000", 2024-09-18T11:33:13.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013394599999999999 2024-09-18T11:33:13.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.193 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:13.193 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.043291+0000", 2024-09-18T11:33:13.193 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014399500000000001 2024-09-18T11:33:13.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:13.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.048224+0000", 2024-09-18T11:33:13.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049331160000000004 2024-09-18T11:33:13.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:13.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053239+0000", 2024-09-18T11:33:13.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050155720000000003 2024-09-18T11:33:13.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.194 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:13.194 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053287+0000", 2024-09-18T11:33:13.194 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8126e-05 2024-09-18T11:33:13.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:13.195 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:13.195 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:13.195 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053325+0000", 2024-09-18T11:33:13.195 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7499999999999997e-05 2024-09-18T11:33:13.195 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.195 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:13.195 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.195 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:13.195 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:13.195 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:13.196 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T11:33:13.198 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:33:13.198 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:13.196+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:33:13.298 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:33:13.297+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:33:13.308 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:13.308 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:13.308 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:33:13.308 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:13.317 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:33:13.399 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:13.398+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:33:13.404 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T11:33:13.456 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:13.456 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:13.457 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:33:13.457 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:33:13.457 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:13.465 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:33:13.510 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:33:13.509+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:33:13.519 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:33:13.519 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:33:13.534 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:33:13.534 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:33:13.549 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:33:13.549 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:33:13.563 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:33:13.564 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:33:13.565 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:13.565 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:33:13.565 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:33:13.565 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:33:13.565 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.565 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:33:13.566 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:33:13.566 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 159.145986456, 2024-09-18T11:33:13.566 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:33:13.566 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:13.566 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.566 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:13.566 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:13.566 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.566 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:33:13.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.566 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:13.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.566 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:13.567 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:33:13.567 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.567 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.567 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:13.567 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:33:13.567 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.567 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.567 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:13.567 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:33:13.567 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:13.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.567 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.568 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:13.568 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:33:13.568 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:33:13.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.568 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.568 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:13.568 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:33:13.568 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:33:13.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.568 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.568 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.568 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:33:13.569 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:33:13.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.569 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.569 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:33:13.569 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:33:13.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.569 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:13.569 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:33:13.569 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:33:13.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.569 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:13.570 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:33:13.570 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:33:13.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.570 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.570 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:13.570 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:33:13.570 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:33:13.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.571 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.571 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:13.571 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:33:13.571 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:33:13.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.571 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.571 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:13.572 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:33:13.572 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:33:13.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.572 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.572 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:13.572 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:33:13.572 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:33:13.572 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.572 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:13.572 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.572 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.572 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:33:13.572 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:33:13.573 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 132.17575704999999, 2024-09-18T11:33:13.573 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:33:13.573 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:13.573 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.573 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:13.573 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.573 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:13.573 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:33:13.573 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.573 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.573 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:13.573 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:33:13.574 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.574 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.574 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:13.574 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:33:13.574 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:13.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.574 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.574 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:13.574 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:33:13.574 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:33:13.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.574 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.575 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:13.575 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:33:13.575 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:33:13.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.575 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.575 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.575 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:33:13.575 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:33:13.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.575 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.575 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.575 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:33:13.575 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:33:13.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.576 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:13.576 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:33:13.576 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:33:13.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.576 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:13.576 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:33:13.576 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:33:13.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.576 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:13.577 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:33:13.577 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:33:13.577 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.577 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:13.577 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.577 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:33:13.577 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:33:13.577 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 124.589644444, 2024-09-18T11:33:13.577 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:33:13.577 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:13.577 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.577 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:13.578 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:13.578 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.578 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:33:13.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.578 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:13.578 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.578 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:13.578 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:33:13.578 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.578 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.578 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:13.579 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:33:13.579 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.579 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.579 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:13.579 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:33:13.579 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:13.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.579 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.579 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:13.579 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:33:13.579 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:33:13.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.579 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.580 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:13.580 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:33:13.580 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:33:13.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.580 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.580 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.580 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:33:13.580 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:33:13.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.580 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.580 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.580 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:33:13.580 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:33:13.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.581 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.581 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:13.581 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:33:13.581 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:33:13.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.581 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.581 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:13.581 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:33:13.581 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:33:13.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.581 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.581 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:13.582 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:33:13.582 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:33:13.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.582 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.582 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:13.582 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:33:13.582 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:33:13.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.582 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.582 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:13.582 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:33:13.582 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:33:13.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.582 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.583 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:13.583 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:33:13.583 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:33:13.583 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.583 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:13.583 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.583 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.583 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2956 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~2100908,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:33:13.583 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:33:13.583 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 109.53484819400001, 2024-09-18T11:33:13.583 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011863831, 2024-09-18T11:33:13.583 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:13.584 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:13.584 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:13.584 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:13.584 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.584 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2956 2024-09-18T11:33:13.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.584 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:13.584 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.584 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:13.584 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:33:13.584 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.584 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.585 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:13.585 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:33:13.585 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.585 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.585 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:13.585 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:33:13.585 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:13.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.585 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.585 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:13.585 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027228+0000", 2024-09-18T11:33:13.585 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:33:13.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.586 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.586 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:13.586 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027235+0000", 2024-09-18T11:33:13.586 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:33:13.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.586 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.586 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.586 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027239+0000", 2024-09-18T11:33:13.586 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.535e-06 2024-09-18T11:33:13.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.586 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.586 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.586 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027831+0000", 2024-09-18T11:33:13.587 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059244699999999996 2024-09-18T11:33:13.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.587 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:13.587 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027879+0000", 2024-09-18T11:33:13.587 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:33:13.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.587 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.587 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038098+0000", 2024-09-18T11:33:13.587 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010219137 2024-09-18T11:33:13.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.588 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:13.588 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038207+0000", 2024-09-18T11:33:13.588 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010951999999999999 2024-09-18T11:33:13.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.588 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.588 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:13.588 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039089+0000", 2024-09-18T11:33:13.588 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00088151699999999996 2024-09-18T11:33:13.588 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.588 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:13.588 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.588 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.589 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2957 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:33:13.589 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:33:13.589 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 109.53483041, 2024-09-18T11:33:13.589 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012041979, 2024-09-18T11:33:13.589 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:13.589 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:13.589 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:13.589 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:13.589 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.589 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2957 2024-09-18T11:33:13.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.589 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:13.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.590 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:13.590 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:33:13.590 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.590 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:13.590 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:33:13.590 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.590 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:13.590 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:33:13.590 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:13.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.591 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:13.591 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027245+0000", 2024-09-18T11:33:13.591 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:33:13.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.591 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.591 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:13.591 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027251+0000", 2024-09-18T11:33:13.591 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0090000000000004e-06 2024-09-18T11:33:13.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.591 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.591 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.591 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027255+0000", 2024-09-18T11:33:13.591 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:33:13.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.592 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.592 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.592 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027895+0000", 2024-09-18T11:33:13.592 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064076400000000005 2024-09-18T11:33:13.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.592 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.592 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:13.592 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027930+0000", 2024-09-18T11:33:13.592 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4334e-05 2024-09-18T11:33:13.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.592 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.592 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.592 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039111+0000", 2024-09-18T11:33:13.592 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181455999999999 2024-09-18T11:33:13.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.593 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.593 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:13.593 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039156+0000", 2024-09-18T11:33:13.593 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5256000000000002e-05 2024-09-18T11:33:13.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.593 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.593 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:13.593 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039285+0000", 2024-09-18T11:33:13.593 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012839200000000001 2024-09-18T11:33:13.593 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.593 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:13.593 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.594 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.594 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:978 4.b e132/67 4:d8d14c58:::smithi11939506-49:30 v 132'417, mlcod=128'414)", 2024-09-18T11:33:13.594 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:33:13.594 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 70.536799127999998, 2024-09-18T11:33:13.594 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011942183, 2024-09-18T11:33:13.594 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:13.594 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.594 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:13.594 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.594 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:13.594 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:33:13.594 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.595 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.595 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:13.595 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:33:13.595 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.595 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.595 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:13.595 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:33:13.595 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:33:13.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.595 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.595 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:13.595 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025278+0000", 2024-09-18T11:33:13.595 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9060000000000004e-06 2024-09-18T11:33:13.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.596 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.596 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:13.596 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025288+0000", 2024-09-18T11:33:13.596 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.028e-05 2024-09-18T11:33:13.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.596 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.596 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.596 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025293+0000", 2024-09-18T11:33:13.596 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2700000000000004e-06 2024-09-18T11:33:13.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.596 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.596 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.596 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025346+0000", 2024-09-18T11:33:13.597 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2862999999999999e-05 2024-09-18T11:33:13.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.597 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.597 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:13.597 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025370+0000", 2024-09-18T11:33:13.597 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4005000000000001e-05 2024-09-18T11:33:13.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.597 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.597 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:13.597 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037159+0000", 2024-09-18T11:33:13.597 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011788343 2024-09-18T11:33:13.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.597 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.598 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:13.598 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037216+0000", 2024-09-18T11:33:13.598 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7695000000000002e-05 2024-09-18T11:33:13.598 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.598 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:13.598 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.598 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.598 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3930 4.b e132/67 4:d3ae8215:::smithi11939506-41:head v 132'421, mlcod=128'414)", 2024-09-18T11:33:13.598 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:33:13.598 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 70.533250879999997, 2024-09-18T11:33:13.598 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015093554, 2024-09-18T11:33:13.598 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:13.599 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.599 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:13.599 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.599 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:13.599 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:33:13.599 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.599 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.599 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:13.599 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:33:13.599 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.599 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.600 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:13.600 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028821+0000", 2024-09-18T11:33:13.600 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:13.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.600 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.600 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:13.600 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029805+0000", 2024-09-18T11:33:13.600 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00098410899999999994 2024-09-18T11:33:13.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.600 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.600 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:13.600 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029817+0000", 2024-09-18T11:33:13.600 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2058e-05 2024-09-18T11:33:13.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.601 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.601 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.601 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029823+0000", 2024-09-18T11:33:13.601 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6450000000000002e-06 2024-09-18T11:33:13.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.601 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.601 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.601 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032330+0000", 2024-09-18T11:33:13.601 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025067710000000001 2024-09-18T11:33:13.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.601 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.602 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:13.602 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032349+0000", 2024-09-18T11:33:13.602 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9729e-05 2024-09-18T11:33:13.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.602 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.602 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:13.602 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043878+0000", 2024-09-18T11:33:13.602 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011528514 2024-09-18T11:33:13.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.602 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.602 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:13.602 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043916+0000", 2024-09-18T11:33:13.602 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8049000000000001e-05 2024-09-18T11:33:13.603 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.603 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:13.603 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.603 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:162 3.1 3:8c826a1f:::e682cfc1e82c88472b9acf1cd97978bc105aa035f10b685bb4cace95809b52bf:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:13.603 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:33:13.603 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 63.981128722000001, 2024-09-18T11:33:13.603 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013234571000000001, 2024-09-18T11:33:13.603 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:13.603 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.603 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:13.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.604 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:13.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:33:13.604 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.604 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.604 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:13.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:33:13.604 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.604 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.604 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:13.604 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580943+0000", 2024-09-18T11:33:13.604 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:13.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.605 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:13.605 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582556+0000", 2024-09-18T11:33:13.605 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016132970000000001 2024-09-18T11:33:13.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.605 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:13.605 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582572+0000", 2024-09-18T11:33:13.605 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5846999999999999e-05 2024-09-18T11:33:13.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.605 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582578+0000", 2024-09-18T11:33:13.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4810000000000003e-06 2024-09-18T11:33:13.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.606 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582632+0000", 2024-09-18T11:33:13.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4156000000000001e-05 2024-09-18T11:33:13.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.606 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:13.606 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582819+0000", 2024-09-18T11:33:13.606 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000186431 2024-09-18T11:33:13.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.607 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:13.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583118+0000", 2024-09-18T11:33:13.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029880100000000001 2024-09-18T11:33:13.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.607 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:13.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590676+0000", 2024-09-18T11:33:13.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075588670000000004 2024-09-18T11:33:13.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.607 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:13.607 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594111+0000", 2024-09-18T11:33:13.607 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034348550000000001 2024-09-18T11:33:13.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.608 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:13.608 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594142+0000", 2024-09-18T11:33:13.608 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.099e-05 2024-09-18T11:33:13.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.608 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:13.608 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594179+0000", 2024-09-18T11:33:13.608 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6831e-05 2024-09-18T11:33:13.608 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.608 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:13.608 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.609 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3974 4.d 4:b5a463bf:::smithi11939506-20:head [write 665643~697786,stat] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:13.609 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:33:13.609 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 63.980042038999997, 2024-09-18T11:33:13.609 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016621174999999998, 2024-09-18T11:33:13.609 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:13.609 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.609 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:13.609 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:13.609 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.609 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3974 2024-09-18T11:33:13.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.610 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:13.610 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.610 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:13.610 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:33:13.610 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.610 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.610 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:13.610 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:33:13.610 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.610 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.610 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:13.610 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582030+0000", 2024-09-18T11:33:13.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:13.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:13.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585185+0000", 2024-09-18T11:33:13.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003155644 2024-09-18T11:33:13.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.611 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:13.611 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585208+0000", 2024-09-18T11:33:13.611 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3013000000000001e-05 2024-09-18T11:33:13.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.612 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585215+0000", 2024-09-18T11:33:13.612 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7569999999999998e-06 2024-09-18T11:33:13.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.612 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585278+0000", 2024-09-18T11:33:13.612 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2699000000000004e-05 2024-09-18T11:33:13.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.612 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:13.612 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585374+0000", 2024-09-18T11:33:13.612 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.6062000000000003e-05 2024-09-18T11:33:13.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.613 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:33:13.613 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585577+0000", 2024-09-18T11:33:13.613 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000202926 2024-09-18T11:33:13.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.613 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:13.613 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596544+0000", 2024-09-18T11:33:13.613 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010966848 2024-09-18T11:33:13.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.613 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:13.613 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598594+0000", 2024-09-18T11:33:13.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020502049999999998 2024-09-18T11:33:13.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.614 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:13.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598625+0000", 2024-09-18T11:33:13.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1412999999999999e-05 2024-09-18T11:33:13.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.614 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:13.614 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598652+0000", 2024-09-18T11:33:13.614 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7166999999999999e-05 2024-09-18T11:33:13.614 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.614 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:13.614 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.615 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:166 3.1 3:a73e635b:::b6ee478f0f661ec7bac3d17c2ca2fdaff71a47570f64844a7ba79bbb2a0e8d9d:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:13.615 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:33:13.615 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 63.979482673, 2024-09-18T11:33:13.615 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011711037000000001, 2024-09-18T11:33:13.615 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:13.615 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.615 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:13.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.615 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:13.615 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:33:13.615 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.615 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:13.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:33:13.616 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.616 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:13.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582590+0000", 2024-09-18T11:33:13.616 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:13.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.616 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:13.616 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582673+0000", 2024-09-18T11:33:13.616 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2861000000000005e-05 2024-09-18T11:33:13.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.617 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:13.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582683+0000", 2024-09-18T11:33:13.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0081999999999999e-05 2024-09-18T11:33:13.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.617 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582688+0000", 2024-09-18T11:33:13.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6870000000000002e-06 2024-09-18T11:33:13.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.617 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.617 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584266+0000", 2024-09-18T11:33:13.617 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001578423 2024-09-18T11:33:13.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:13.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584407+0000", 2024-09-18T11:33:13.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014144599999999999 2024-09-18T11:33:13.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:13.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584599+0000", 2024-09-18T11:33:13.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000191881 2024-09-18T11:33:13.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.618 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:13.618 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590680+0000", 2024-09-18T11:33:13.618 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060801479999999996 2024-09-18T11:33:13.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.619 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:13.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594224+0000", 2024-09-18T11:33:13.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035441700000000001 2024-09-18T11:33:13.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.619 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:13.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594269+0000", 2024-09-18T11:33:13.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5297999999999999e-05 2024-09-18T11:33:13.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.619 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:13.619 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594302+0000", 2024-09-18T11:33:13.619 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2703000000000002e-05 2024-09-18T11:33:13.619 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.620 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:13.620 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.620 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:168 3.1 3:b64d7bad:::7feb22256bdcec217720fd46184ab43ced8923ed66ec6319c57f078d9f123262:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:13.620 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:33:13.620 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 63.979255221000003, 2024-09-18T11:33:13.620 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011589809, 2024-09-18T11:33:13.620 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:13.620 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.620 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:13.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.620 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:13.620 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:33:13.620 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:13.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:33:13.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:13.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582817+0000", 2024-09-18T11:33:13.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:13.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.621 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:13.621 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582903+0000", 2024-09-18T11:33:13.621 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5915999999999995e-05 2024-09-18T11:33:13.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:13.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582913+0000", 2024-09-18T11:33:13.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1409999999999994e-06 2024-09-18T11:33:13.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582917+0000", 2024-09-18T11:33:13.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6419999999999998e-06 2024-09-18T11:33:13.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.622 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.622 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585540+0000", 2024-09-18T11:33:13.622 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026228940000000002 2024-09-18T11:33:13.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.623 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:13.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585721+0000", 2024-09-18T11:33:13.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018086299999999999 2024-09-18T11:33:13.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.623 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:13.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585918+0000", 2024-09-18T11:33:13.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000196758 2024-09-18T11:33:13.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.623 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:13.623 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590683+0000", 2024-09-18T11:33:13.623 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047657020000000001 2024-09-18T11:33:13.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.624 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:13.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594354+0000", 2024-09-18T11:33:13.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003671068 2024-09-18T11:33:13.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.624 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:13.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594386+0000", 2024-09-18T11:33:13.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1426e-05 2024-09-18T11:33:13.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.624 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:13.624 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594408+0000", 2024-09-18T11:33:13.624 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1987000000000001e-05 2024-09-18T11:33:13.624 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.625 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:13.625 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.625 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:175 3.1 3:b5e86b88:::e55cf3d5cf782ed327a67e746b287b76d05f74f58a4f3726e399a564440ceaf6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:13.625 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:33:13.625 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 63.978406280000002, 2024-09-18T11:33:13.625 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013709627, 2024-09-18T11:33:13.625 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:13.625 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.625 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:13.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.625 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:13.625 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:33:13.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:13.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:33:13.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:13.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583666+0000", 2024-09-18T11:33:13.626 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:13.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.626 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:13.626 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584041+0000", 2024-09-18T11:33:13.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037511499999999998 2024-09-18T11:33:13.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:13.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584055+0000", 2024-09-18T11:33:13.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3295999999999999e-05 2024-09-18T11:33:13.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.627 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.627 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584061+0000", 2024-09-18T11:33:13.627 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1630000000000001e-06 2024-09-18T11:33:13.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.628 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.589980+0000", 2024-09-18T11:33:13.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005918998 2024-09-18T11:33:13.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.628 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:13.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590117+0000", 2024-09-18T11:33:13.628 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013715400000000001 2024-09-18T11:33:13.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.628 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:13.628 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590270+0000", 2024-09-18T11:33:13.629 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015245600000000001 2024-09-18T11:33:13.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:13.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596513+0000", 2024-09-18T11:33:13.629 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062438729999999996 2024-09-18T11:33:13.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.629 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:13.629 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597328+0000", 2024-09-18T11:33:13.629 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081489599999999996 2024-09-18T11:33:13.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:13.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597358+0000", 2024-09-18T11:33:13.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9291000000000001e-05 2024-09-18T11:33:13.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.630 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:13.630 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597377+0000", 2024-09-18T11:33:13.630 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:33:13.630 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.630 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:13.630 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.631 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4061 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:13.631 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:33:13.631 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 61.265227590999999, 2024-09-18T11:33:13.631 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013340967, 2024-09-18T11:33:13.631 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:13.631 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:13.631 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:13.631 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:13.631 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.631 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4061 2024-09-18T11:33:13.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.631 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:13.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.631 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:13.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:33:13.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.632 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:13.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:33:13.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.632 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:13.632 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296844+0000", 2024-09-18T11:33:13.632 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:13.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:13.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296854+0000", 2024-09-18T11:33:13.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0037000000000001e-05 2024-09-18T11:33:13.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:13.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296873+0000", 2024-09-18T11:33:13.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:33:13.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.633 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.633 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296881+0000", 2024-09-18T11:33:13.633 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3790000000000006e-06 2024-09-18T11:33:13.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296975+0000", 2024-09-18T11:33:13.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3536999999999996e-05 2024-09-18T11:33:13.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:33:13.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296982+0000", 2024-09-18T11:33:13.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6399999999999997e-06 2024-09-18T11:33:13.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.634 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.634 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301933+0000", 2024-09-18T11:33:13.634 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049507190000000001 2024-09-18T11:33:13.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.635 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:13.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301984+0000", 2024-09-18T11:33:13.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1439e-05 2024-09-18T11:33:13.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.635 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:13.635 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:33:13.635 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082022050000000006 2024-09-18T11:33:13.635 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.635 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:13.635 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.636 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4062 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:13.636 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:33:13.636 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 61.265179035999999, 2024-09-18T11:33:13.636 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016297147000000001, 2024-09-18T11:33:13.636 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:13.636 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:13.636 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:13.636 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:13.636 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.636 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4062 2024-09-18T11:33:13.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.636 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:13.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.636 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:13.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:33:13.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:13.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:33:13.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:13.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:33:13.637 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:13.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.637 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:13.637 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296898+0000", 2024-09-18T11:33:13.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9419999999999997e-06 2024-09-18T11:33:13.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:13.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296906+0000", 2024-09-18T11:33:13.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3520000000000007e-06 2024-09-18T11:33:13.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.638 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.638 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296910+0000", 2024-09-18T11:33:13.638 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4469999999999999e-06 2024-09-18T11:33:13.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.639 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297032+0000", 2024-09-18T11:33:13.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012168900000000001 2024-09-18T11:33:13.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.639 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:13.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297038+0000", 2024-09-18T11:33:13.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9320000000000001e-06 2024-09-18T11:33:13.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.639 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.639 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310221+0000", 2024-09-18T11:33:13.639 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013182887000000001 2024-09-18T11:33:13.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.640 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:13.640 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310323+0000", 2024-09-18T11:33:13.640 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010170000000000001 2024-09-18T11:33:13.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.640 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:13.640 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313191+0000", 2024-09-18T11:33:13.640 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002868808 2024-09-18T11:33:13.640 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.640 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:13.640 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.640 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4063 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:13.641 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:33:13.641 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 61.265158143999997, 2024-09-18T11:33:13.641 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017648712, 2024-09-18T11:33:13.641 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:13.641 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:13.641 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:13.641 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:13.641 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.641 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4063 2024-09-18T11:33:13.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.641 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:13.641 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.641 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:13.641 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:33:13.641 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:13.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:33:13.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:13.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:33:13.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:13.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.642 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:13.642 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296918+0000", 2024-09-18T11:33:13.642 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8100000000000002e-06 2024-09-18T11:33:13.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:13.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296924+0000", 2024-09-18T11:33:13.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2519999999999996e-06 2024-09-18T11:33:13.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296928+0000", 2024-09-18T11:33:13.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0500000000000002e-06 2024-09-18T11:33:13.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.643 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.643 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297062+0000", 2024-09-18T11:33:13.643 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013425500000000001 2024-09-18T11:33:13.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.644 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:13.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297069+0000", 2024-09-18T11:33:13.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5239999999999997e-06 2024-09-18T11:33:13.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.644 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313424+0000", 2024-09-18T11:33:13.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016355277000000001 2024-09-18T11:33:13.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.644 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:13.644 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313514+0000", 2024-09-18T11:33:13.644 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0581999999999994e-05 2024-09-18T11:33:13.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.645 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:13.645 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.314564+0000", 2024-09-18T11:33:13.645 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001049384 2024-09-18T11:33:13.645 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.645 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:13.645 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.645 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4065 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:13.645 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:33:13.645 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 61.265112703, 2024-09-18T11:33:13.646 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013225997, 2024-09-18T11:33:13.646 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:13.646 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:13.646 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:13.646 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:13.646 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.646 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4065 2024-09-18T11:33:13.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.646 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:13.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.646 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:13.646 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:33:13.646 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.646 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:13.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:33:13.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:13.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296960+0000", 2024-09-18T11:33:13.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:13.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:13.647 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296963+0000", 2024-09-18T11:33:13.647 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2270000000000002e-06 2024-09-18T11:33:13.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.647 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:13.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296970+0000", 2024-09-18T11:33:13.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6220000000000003e-06 2024-09-18T11:33:13.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296973+0000", 2024-09-18T11:33:13.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3730000000000001e-06 2024-09-18T11:33:13.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.648 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.648 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297097+0000", 2024-09-18T11:33:13.648 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012352399999999999 2024-09-18T11:33:13.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.649 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:13.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297103+0000", 2024-09-18T11:33:13.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7119999999999997e-06 2024-09-18T11:33:13.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.649 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307831+0000", 2024-09-18T11:33:13.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010728794 2024-09-18T11:33:13.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.649 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:13.649 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307924+0000", 2024-09-18T11:33:13.649 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2099e-05 2024-09-18T11:33:13.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.650 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:13.650 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:33:13.650 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022630469999999998 2024-09-18T11:33:13.650 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.650 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:13.650 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.650 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4066 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:13.650 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:33:13.650 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 61.265095627999997, 2024-09-18T11:33:13.651 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015656389999999999, 2024-09-18T11:33:13.651 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:13.651 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:13.651 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:13.651 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:13.651 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.651 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4066 2024-09-18T11:33:13.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.651 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:13.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.651 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:13.651 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:33:13.651 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:13.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:33:13.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:13.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296977+0000", 2024-09-18T11:33:13.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:13.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.652 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:13.652 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296980+0000", 2024-09-18T11:33:13.652 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8830000000000002e-06 2024-09-18T11:33:13.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:13.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296985+0000", 2024-09-18T11:33:13.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:33:13.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296988+0000", 2024-09-18T11:33:13.653 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0419999999999998e-06 2024-09-18T11:33:13.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.653 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.653 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297120+0000", 2024-09-18T11:33:13.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131693 2024-09-18T11:33:13.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:13.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297125+0000", 2024-09-18T11:33:13.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1100000000000002e-06 2024-09-18T11:33:13.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310212+0000", 2024-09-18T11:33:13.654 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013087395999999999 2024-09-18T11:33:13.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.654 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:13.654 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310292+0000", 2024-09-18T11:33:13.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9980000000000003e-05 2024-09-18T11:33:13.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.655 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:13.655 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312634+0000", 2024-09-18T11:33:13.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002341756 2024-09-18T11:33:13.655 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.655 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:13.655 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.655 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4068 3.1 3:87e79ed0:::3526641c07d3efa989be9684355570f39c331008b8a02c9fa22b38a94b6c6b9f:head [write 0~1015808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:13.655 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:33:13.655 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 61.261319411000002, 2024-09-18T11:33:13.655 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016392648999999999, 2024-09-18T11:33:13.656 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:13.656 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.656 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:13.656 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:13.656 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.656 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4068 2024-09-18T11:33:13.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.656 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:13.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.656 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:13.656 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:33:13.656 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.656 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:13.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:33:13.657 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.657 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:13.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300753+0000", 2024-09-18T11:33:13.657 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:13.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.657 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:13.657 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301922+0000", 2024-09-18T11:33:13.657 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001169432 2024-09-18T11:33:13.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.657 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:13.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301936+0000", 2024-09-18T11:33:13.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3574000000000001e-05 2024-09-18T11:33:13.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301941+0000", 2024-09-18T11:33:13.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6069999999999998e-06 2024-09-18T11:33:13.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.658 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.658 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310234+0000", 2024-09-18T11:33:13.658 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082924820000000003 2024-09-18T11:33:13.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:13.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310337+0000", 2024-09-18T11:33:13.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102673 2024-09-18T11:33:13.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:13.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310469+0000", 2024-09-18T11:33:13.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131982 2024-09-18T11:33:13.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:13.659 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312844+0000", 2024-09-18T11:33:13.659 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023758680000000002 2024-09-18T11:33:13.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.659 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:13.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317092+0000", 2024-09-18T11:33:13.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042477549999999998 2024-09-18T11:33:13.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:13.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317124+0000", 2024-09-18T11:33:13.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1968999999999998e-05 2024-09-18T11:33:13.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.660 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:13.660 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317147+0000", 2024-09-18T11:33:13.660 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2339000000000001e-05 2024-09-18T11:33:13.660 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.660 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:13.661 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.661 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4671 4.1 4:87078dfe:::smithi11939506-14:head [rollback 6f] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:33:13.661 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:33:13.661 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 28.243357035999999, 2024-09-18T11:33:13.661 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013540905000000001, 2024-09-18T11:33:13.661 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:13.661 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.661 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:13.661 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:13.661 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.661 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4671 2024-09-18T11:33:13.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.662 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:13.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.662 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:13.662 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:33:13.662 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.662 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:13.662 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:33:13.662 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.662 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:13.662 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:33:13.663 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:13.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.663 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.663 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:13.663 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318718+0000", 2024-09-18T11:33:13.663 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3300000000000001e-06 2024-09-18T11:33:13.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.663 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.663 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:13.663 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318723+0000", 2024-09-18T11:33:13.663 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1590000000000001e-06 2024-09-18T11:33:13.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.663 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318727+0000", 2024-09-18T11:33:13.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3529999999999999e-06 2024-09-18T11:33:13.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.320735+0000", 2024-09-18T11:33:13.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020079730000000001 2024-09-18T11:33:13.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.664 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:13.664 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.320843+0000", 2024-09-18T11:33:13.664 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000108488 2024-09-18T11:33:13.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.665 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:13.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.329522+0000", 2024-09-18T11:33:13.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086787979999999997 2024-09-18T11:33:13.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.665 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:13.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331769+0000", 2024-09-18T11:33:13.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022471269999999998 2024-09-18T11:33:13.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.665 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:13.665 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332189+0000", 2024-09-18T11:33:13.665 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00041940400000000001 2024-09-18T11:33:13.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:13.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332228+0000", 2024-09-18T11:33:13.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9298000000000002e-05 2024-09-18T11:33:13.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.666 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:13.666 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332257+0000", 2024-09-18T11:33:13.666 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.934e-05 2024-09-18T11:33:13.666 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.666 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:13.666 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.667 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.667 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4672 4.1 4:87078dfe:::smithi11939506-14:head [append 0~0] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:33:13.667 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.318763+0000", 2024-09-18T11:33:13.667 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 28.243310633, 2024-09-18T11:33:13.667 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016099992, 2024-09-18T11:33:13.667 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:13.667 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:13.667 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:13.667 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:13.667 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:13.667 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4672 2024-09-18T11:33:13.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.667 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:13.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.668 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:13.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318763+0000", 2024-09-18T11:33:13.668 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.668 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:13.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318763+0000", 2024-09-18T11:33:13.668 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:13.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.668 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:13.668 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318762+0000", 2024-09-18T11:33:13.668 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:13.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.669 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.669 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:13.669 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318765+0000", 2024-09-18T11:33:13.669 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.43e-06 2024-09-18T11:33:13.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.669 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.669 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:13.669 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318770+0000", 2024-09-18T11:33:13.669 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1479999999999997e-06 2024-09-18T11:33:13.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.669 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.669 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:13.669 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318773+0000", 2024-09-18T11:33:13.670 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.18e-06 2024-09-18T11:33:13.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.670 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.321043+0000", 2024-09-18T11:33:13.670 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022696819999999999 2024-09-18T11:33:13.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.670 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for blocked object", 2024-09-18T11:33:13.670 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.321104+0000", 2024-09-18T11:33:13.670 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1050999999999995e-05 2024-09-18T11:33:13.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.670 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:13.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331070+0000", 2024-09-18T11:33:13.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0099659610000000006 2024-09-18T11:33:13.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.671 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:13.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331176+0000", 2024-09-18T11:33:13.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010582 2024-09-18T11:33:13.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.671 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:13.671 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331377+0000", 2024-09-18T11:33:13.671 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000201208 2024-09-18T11:33:13.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:13.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332278+0000", 2024-09-18T11:33:13.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00090079999999999999 2024-09-18T11:33:13.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:13.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.334798+0000", 2024-09-18T11:33:13.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025200190000000001 2024-09-18T11:33:13.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.672 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:13.672 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.334833+0000", 2024-09-18T11:33:13.672 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5678e-05 2024-09-18T11:33:13.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:13.673 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:13.673 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:13.673 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.334863+0000", 2024-09-18T11:33:13.673 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9374e-05 2024-09-18T11:33:13.673 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.673 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:13.673 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.673 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:13.673 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:13.673 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:13.674 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:33:13.674 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:33:13.675 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:33:13.675 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:33:13.676 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:33:13.676 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:33:13.673+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:33:13.676 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:33:13.676 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:33:13.677 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:33:13.677 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:33:13.775 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:33:13.774+0000 7fe4189f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 56602) UID: 0 2024-09-18T11:33:13.814 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:13.814 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:13.822 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:33:13.876 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:33:13.874+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:33:13.920 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:13.920 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:13.928 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:33:13.977 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:13.975+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:33:14.067 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:14.067 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:14.076 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:33:14.077 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:33:14.076+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:33:14.103 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is unset 2024-09-18T11:33:14.174 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:14.174 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:14.178 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:33:14.177+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:33:14.182 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:33:14.279 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:14.278+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:33:14.320 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:14.320 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:14.329 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:33:14.380 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:14.379+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:14.426 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:14.426 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:14.434 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:33:14.519 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:33:14.518+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:33:14.573 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:14.573 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:14.582 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:33:14.582 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:33:14.580+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:33:14.683 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:14.682+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:14.721 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:14.721 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:14.730 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:33:14.784 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:14.783+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:33:14.868 INFO:teuthology.orchestra.run.smithi148.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:14.868 INFO:teuthology.orchestra.run.smithi148.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:14.876 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:33:14.885 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:33:14.883+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:33:14.985 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:14.984+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:33:14.987 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:14.987 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:14.996 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:33:15.002 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:33:15.000+0000 7fd27659f640 -1 osd.6 196 handle_osd_map injecting map crc failure 2024-09-18T11:33:15.086 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:33:15.085+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:33:15.136 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:15.136 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:15.145 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:33:15.187 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:33:15.186+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:33:15.243 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:15.243 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:15.251 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T11:33:15.289 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:33:15.288+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:33:15.349 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:15.349 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:15.358 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T11:33:15.403 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:15.402+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:33:15.461 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:15.461 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:15.470 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T11:33:15.480 INFO:tasks.rados.rados.0.smithi119.stderr:0 errors. 2024-09-18T11:33:15.480 INFO:tasks.rados.rados.0.smithi119.stderr: 2024-09-18T11:33:15.482 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T11:33:15.491 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:33:15.490+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:33:15.579 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:15.579 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:15.588 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T11:33:15.591 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:33:15.590+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:33:15.678 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T11:33:15.690 INFO:teuthology.orchestra.run.smithi119.stdout:ERROR: (22) Invalid argument 2024-09-18T11:33:15.690 INFO:teuthology.orchestra.run.smithi119.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T11:33:15.691 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:33:15.691+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:33:15.700 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T11:33:15.792 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:15.791+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:33:15.796 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0: {} 2024-09-18T11:33:15.796 INFO:teuthology.orchestra.run.smithi119.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T11:33:15.797 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:33:15.797 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":197,"fsid":"f4cca259-b87a-45cc-8995-7e9fca891a9f","created":"2024-09-18T11:29:38.205487+0000","modified":"2024-09-18T11:33:14.094738+0000","last_up_change":"2024-09-18T11:31:49.142260+0000","last_in_change":"2024-09-18T11:32:45.440256+0000","flags":"nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5804032,"flags_set":["nodeep-scrub","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T11:29:47.951262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T11:29:48.866238+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"low_tier","create_time":"2024-09-18T11:29:59.343964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}},{"pool":4,"pool_name":"unique_pool_0","create_time":"2024-09-18T11:30:00.523818+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":131,"snap_epoch":184,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"dedup_cdc_chunk_size":131072,"dedup_chunk_algorithm":"fastcdc","dedup_tier":3,"fingerprint_algorithm":"sha256"},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"12faced2-7888-4265-ad12-815efe36466c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":65,"up_from":119,"up_thru":189,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":56607},{"type":"v1","addr":"172.21.15.119:6801","nonce":56607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":56607},{"type":"v1","addr":"172.21.15.119:6803","nonce":56607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":56607},{"type":"v1","addr":"172.21.15.119:6807","nonce":56607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":56607},{"type":"v1","addr":"172.21.15.119:6805","nonce":56607}]},"public_addr":"172.21.15.119:6801/56607","cluster_addr":"172.21.15.119:6803/56607","heartbeat_back_addr":"172.21.15.119:6807/56607","heartbeat_front_addr":"172.21.15.119:6805/56607","state":["exists","up"]},{"osd":1,"uuid":"20fee29d-b047-46ae-aa60-88e897fbb0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":33350},{"type":"v1","addr":"172.21.15.119:6825","nonce":33350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":33350},{"type":"v1","addr":"172.21.15.119:6827","nonce":33350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":33350},{"type":"v1","addr":"172.21.15.119:6831","nonce":33350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":33350},{"type":"v1","addr":"172.21.15.119:6829","nonce":33350}]},"public_addr":"172.21.15.119:6825/33350","cluster_addr":"172.21.15.119:6827/33350","heartbeat_back_addr":"172.21.15.119:6831/33350","heartbeat_front_addr":"172.21.15.119:6829/33350","state":["exists","up"]},{"osd":2,"uuid":"21a296f3-b259-4841-9a17-4cd21b302b36","up":1,"in":1,"weight":1,"primary_affinity":0,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":33351},{"type":"v1","addr":"172.21.15.119:6809","nonce":33351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":33351},{"type":"v1","addr":"172.21.15.119:6813","nonce":33351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6819","nonce":33351},{"type":"v1","addr":"172.21.15.119:6821","nonce":33351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":33351},{"type":"v1","addr":"172.21.15.119:6817","nonce":33351}]},"public_addr":"172.21.15.119:6809/33351","cluster_addr":"172.21.15.119:6813/33351","heartbeat_back_addr":"172.21.15.119:6821/33351","heartbeat_front_addr":"172.21.15.119:6817/33351","state":["exists","up"]},{"osd":3,"uuid":"59044f30-32e1-4942-9a8e-202c1def221a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":33349},{"type":"v1","addr":"172.21.15.119:6812","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":33349},{"type":"v1","addr":"172.21.15.119:6816","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":33349},{"type":"v1","addr":"172.21.15.119:6823","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":33349},{"type":"v1","addr":"172.21.15.119:6820","nonce":33349}]},"public_addr":"172.21.15.119:6812/33349","cluster_addr":"172.21.15.119:6816/33349","heartbeat_back_addr":"172.21.15.119:6823/33349","heartbeat_front_addr":"172.21.15.119:6820/33349","state":["exists","up"]},{"osd":4,"uuid":"7d1ae3d0-8d69-4d3c-9e51-24fdd77a22a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":33088},{"type":"v1","addr":"172.21.15.148:6809","nonce":33088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":33088},{"type":"v1","addr":"172.21.15.148:6811","nonce":33088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":33088},{"type":"v1","addr":"172.21.15.148:6815","nonce":33088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":33088},{"type":"v1","addr":"172.21.15.148:6813","nonce":33088}]},"public_addr":"172.21.15.148:6809/33088","cluster_addr":"172.21.15.148:6811/33088","heartbeat_back_addr":"172.21.15.148:6815/33088","heartbeat_front_addr":"172.21.15.148:6813/33088","state":["exists","up"]},{"osd":5,"uuid":"b0dd3d30-b273-428c-a4a9-4c238abf45fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":33089},{"type":"v1","addr":"172.21.15.148:6801","nonce":33089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":33089},{"type":"v1","addr":"172.21.15.148:6803","nonce":33089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":33089},{"type":"v1","addr":"172.21.15.148:6807","nonce":33089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":33089},{"type":"v1","addr":"172.21.15.148:6805","nonce":33089}]},"public_addr":"172.21.15.148:6801/33089","cluster_addr":"172.21.15.148:6803/33089","heartbeat_back_addr":"172.21.15.148:6807/33089","heartbeat_front_addr":"172.21.15.148:6805/33089","state":["exists","up"]},{"osd":6,"uuid":"40290cb3-2334-4c68-8e6c-59d0d5455171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":33107},{"type":"v1","addr":"172.21.15.148:6817","nonce":33107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":33107},{"type":"v1","addr":"172.21.15.148:6819","nonce":33107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":33107},{"type":"v1","addr":"172.21.15.148:6823","nonce":33107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":33107},{"type":"v1","addr":"172.21.15.148:6821","nonce":33107}]},"public_addr":"172.21.15.148:6817/33107","cluster_addr":"172.21.15.148:6819/33107","heartbeat_back_addr":"172.21.15.148:6823/33107","heartbeat_front_addr":"172.21.15.148:6821/33107","state":["exists","up"]},{"osd":7,"uuid":"54802abe-ea1a-448e-85af-d1f14a45be27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":33119},{"type":"v1","addr":"172.21.15.148:6825","nonce":33119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":33119},{"type":"v1","addr":"172.21.15.148:6827","nonce":33119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":33119},{"type":"v1","addr":"172.21.15.148:6831","nonce":33119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":33119},{"type":"v1","addr":"172.21.15.148:6829","nonce":33119}]},"public_addr":"172.21.15.148:6825/33119","cluster_addr":"172.21.15.148:6827/33119","heartbeat_back_addr":"172.21.15.148:6831/33119","heartbeat_front_addr":"172.21.15.148:6829/33119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:33:10.971461+0000","dead_epoch":66},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:32:14.430356+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:32:27.493785+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:32:12.546041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:32:13.027907+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:32:28.500831+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:32:14.951013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:32:27.373504+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"4.b","osds":[5,3]}],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T11:33:15.811 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:33:15.812 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1: {} 2024-09-18T11:33:15.812 INFO:teuthology.orchestra.run.smithi119.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T11:33:15.827 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2: {} 2024-09-18T11:33:15.827 INFO:teuthology.orchestra.run.smithi119.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T11:33:15.843 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3: {} 2024-09-18T11:33:15.843 INFO:teuthology.orchestra.run.smithi119.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T11:33:15.844 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:15.844 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:33:15.844 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:33:15.844 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:33:15.844 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:15.853 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T11:33:15.858 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4: {} 2024-09-18T11:33:15.859 INFO:teuthology.orchestra.run.smithi119.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T11:33:15.875 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5: {} 2024-09-18T11:33:15.875 INFO:teuthology.orchestra.run.smithi119.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T11:33:15.891 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6: {} 2024-09-18T11:33:15.892 INFO:teuthology.orchestra.run.smithi119.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T11:33:15.893 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:33:15.891+0000 7f31619f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33079) UID: 0 2024-09-18T11:33:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7: {} 2024-09-18T11:33:15.908 INFO:teuthology.orchestra.run.smithi119.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T11:33:15.963 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:15.963 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:33:15.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:33:15.963 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:33:15.963 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.963 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1126 3.0 e38/21 3:03262160:::95fae96e43c0ae9ef1724977b0e46fd3c44c64be9ca68918ce33977459439e40:head v 38'56, mlcod=38'54)", 2024-09-18T11:33:15.963 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:33:15.963 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 175.97728610499999, 2024-09-18T11:33:15.963 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069312189999999997, 2024-09-18T11:33:15.963 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:15.964 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:15.964 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:15.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:15.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:33:15.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:15.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.964 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:15.964 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983374+0000", 2024-09-18T11:33:15.964 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:15.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.964 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:15.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.983373+0000", 2024-09-18T11:33:15.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:15.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:15.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984876+0000", 2024-09-18T11:33:15.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00150308 2024-09-18T11:33:15.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.965 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.965 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:15.965 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984897+0000", 2024-09-18T11:33:15.965 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0803999999999999e-05 2024-09-18T11:33:15.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:15.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984906+0000", 2024-09-18T11:33:15.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1330000000000005e-06 2024-09-18T11:33:15.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.966 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.966 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:15.966 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984954+0000", 2024-09-18T11:33:15.966 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8106000000000003e-05 2024-09-18T11:33:15.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:15.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.984981+0000", 2024-09-18T11:33:15.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6872999999999998e-05 2024-09-18T11:33:15.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:15.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990261+0000", 2024-09-18T11:33:15.967 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052798690000000004 2024-09-18T11:33:15.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.967 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:15.967 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.990305+0000", 2024-09-18T11:33:15.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4725000000000003e-05 2024-09-18T11:33:15.968 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:15.968 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:15.968 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:15.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.968 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.968 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1421 4.d e48/22 4:b5a463bf:::smithi11939506-20:head v 48'63, mlcod=47'59)", 2024-09-18T11:33:15.968 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:33:15.968 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 166.74245515199999, 2024-09-18T11:33:15.968 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005025341, 2024-09-18T11:33:15.968 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:15.968 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:15.968 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:15.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:15.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:33:15.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:15.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:15.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:33:15.969 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:15.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.969 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.969 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:15.969 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.218205+0000", 2024-09-18T11:33:15.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:33:15.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:15.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219066+0000", 2024-09-18T11:33:15.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00086098700000000002 2024-09-18T11:33:15.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.970 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:15.970 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219076+0000", 2024-09-18T11:33:15.970 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0081000000000001e-05 2024-09-18T11:33:15.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.970 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:15.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219081+0000", 2024-09-18T11:33:15.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3299999999999998e-06 2024-09-18T11:33:15.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:15.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219135+0000", 2024-09-18T11:33:15.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4268000000000003e-05 2024-09-18T11:33:15.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.971 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.971 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:15.971 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.219159+0000", 2024-09-18T11:33:15.971 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4053000000000002e-05 2024-09-18T11:33:15.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:15.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223204+0000", 2024-09-18T11:33:15.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040448009999999998 2024-09-18T11:33:15.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.972 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.972 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:15.972 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:29.223230+0000", 2024-09-18T11:33:15.972 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6020000000000002e-05 2024-09-18T11:33:15.972 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:15.972 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:15.972 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:15.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.973 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1530 4.d e53/22 4:b81644f2:::smithi11939506-34:head v 53'72, mlcod=51'69)", 2024-09-18T11:33:15.973 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:33:15.973 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 161.568548867, 2024-09-18T11:33:15.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050101679999999997, 2024-09-18T11:33:15.973 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:15.973 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:15.973 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:15.973 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.973 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:15.973 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:33:15.973 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:15.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:15.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392111+0000", 2024-09-18T11:33:15.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:15.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:15.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392110+0000", 2024-09-18T11:33:15.974 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:15.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.974 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.974 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:15.974 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.392994+0000", 2024-09-18T11:33:15.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000884033 2024-09-18T11:33:15.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:15.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393012+0000", 2024-09-18T11:33:15.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7659000000000002e-05 2024-09-18T11:33:15.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.975 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:15.975 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393018+0000", 2024-09-18T11:33:15.975 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.015e-06 2024-09-18T11:33:15.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.975 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:15.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393063+0000", 2024-09-18T11:33:15.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5160999999999997e-05 2024-09-18T11:33:15.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:15.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.393087+0000", 2024-09-18T11:33:15.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.408e-05 2024-09-18T11:33:15.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.976 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.976 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:15.976 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397069+0000", 2024-09-18T11:33:15.976 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0039817200000000002 2024-09-18T11:33:15.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.977 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:15.977 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:34.397121+0000", 2024-09-18T11:33:15.977 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2711999999999999e-05 2024-09-18T11:33:15.977 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:15.977 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:15.977 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:15.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.977 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.977 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:304 4.6 e69/67 4:67c1765a:::smithi11939506-30:17 v 69'35, mlcod=58'31)", 2024-09-18T11:33:15.977 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:33:15.977 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 144.61470777299999, 2024-09-18T11:33:15.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006377611, 2024-09-18T11:33:15.978 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:15.978 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:15.978 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:15.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:15.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:33:15.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:15.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.978 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.978 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:15.978 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:33:15.978 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:15.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:15.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345952+0000", 2024-09-18T11:33:15.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:15.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:15.979 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345961+0000", 2024-09-18T11:33:15.979 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2050000000000008e-06 2024-09-18T11:33:15.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.979 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.979 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:15.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345969+0000", 2024-09-18T11:33:15.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0339999999999997e-06 2024-09-18T11:33:15.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:15.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.345976+0000", 2024-09-18T11:33:15.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0140000000000001e-06 2024-09-18T11:33:15.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.980 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.980 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:15.980 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346026+0000", 2024-09-18T11:33:15.980 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9286999999999998e-05 2024-09-18T11:33:15.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:15.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.346050+0000", 2024-09-18T11:33:15.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4397e-05 2024-09-18T11:33:15.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:15.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352293+0000", 2024-09-18T11:33:15.981 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0062427170000000001 2024-09-18T11:33:15.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.981 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:15.981 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352330+0000", 2024-09-18T11:33:15.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7348999999999998e-05 2024-09-18T11:33:15.982 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:15.982 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:15.982 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:15.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.982 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:347 4.d e69/22 4:b44229a1:::smithi11939506-42:19 v 69'99, mlcod=65'93)", 2024-09-18T11:33:15.982 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:33:15.982 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 144.60897457799999, 2024-09-18T11:33:15.982 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056998760000000004, 2024-09-18T11:33:15.982 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:15.982 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:15.982 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:15.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:15.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:33:15.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:15.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:15.983 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351686+0000", 2024-09-18T11:33:15.983 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:15.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.983 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:15.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351684+0000", 2024-09-18T11:33:15.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:15.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:15.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351693+0000", 2024-09-18T11:33:15.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.6230000000000003e-06 2024-09-18T11:33:15.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.984 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:15.984 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351704+0000", 2024-09-18T11:33:15.984 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1077000000000001e-05 2024-09-18T11:33:15.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:15.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351709+0000", 2024-09-18T11:33:15.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5419999999999999e-06 2024-09-18T11:33:15.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:15.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351956+0000", 2024-09-18T11:33:15.985 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000247252 2024-09-18T11:33:15.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.985 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:15.985 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.351975+0000", 2024-09-18T11:33:15.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.9201000000000001e-05 2024-09-18T11:33:15.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:15.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357334+0000", 2024-09-18T11:33:15.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053588749999999999 2024-09-18T11:33:15.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.986 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.986 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:15.986 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.357385+0000", 2024-09-18T11:33:15.986 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1406999999999998e-05 2024-09-18T11:33:15.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:15.986 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:15.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:15.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.987 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1956 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'37, mlcod=58'31)", 2024-09-18T11:33:15.987 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:33:15.987 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 144.608029665, 2024-09-18T11:33:15.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010958842, 2024-09-18T11:33:15.987 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:15.987 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:15.987 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:15.987 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.987 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:15.987 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:33:15.987 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:15.987 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:15.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:33:15.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:15.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:15.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352630+0000", 2024-09-18T11:33:15.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:15.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.988 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.988 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:15.988 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.352996+0000", 2024-09-18T11:33:15.988 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036622 2024-09-18T11:33:15.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:15.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353011+0000", 2024-09-18T11:33:15.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4849e-05 2024-09-18T11:33:15.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:15.989 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353017+0000", 2024-09-18T11:33:15.989 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.4150000000000004e-06 2024-09-18T11:33:15.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.989 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.989 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:15.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353068+0000", 2024-09-18T11:33:15.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0695000000000002e-05 2024-09-18T11:33:15.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:15.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.353092+0000", 2024-09-18T11:33:15.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4260000000000002e-05 2024-09-18T11:33:15.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.990 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.990 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:15.990 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363551+0000", 2024-09-18T11:33:15.990 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010459154999999999 2024-09-18T11:33:15.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.991 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:15.991 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.363589+0000", 2024-09-18T11:33:15.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8161000000000003e-05 2024-09-18T11:33:15.991 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:15.991 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:15.991 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:15.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.991 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.991 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:1957 4.6 e69/67 4:67c1765a:::smithi11939506-30:head v 69'38, mlcod=65'33)", 2024-09-18T11:33:15.991 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:33:15.991 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 144.60161379799999, 2024-09-18T11:33:15.991 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050499400000000002, 2024-09-18T11:33:15.992 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:15.992 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:15.992 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:15.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:15.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:33:15.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:15.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.992 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:15.992 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:33:15.992 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:15.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.992 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:15.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359046+0000", 2024-09-18T11:33:15.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:15.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:15.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359214+0000", 2024-09-18T11:33:15.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000168047 2024-09-18T11:33:15.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.993 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.993 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:15.993 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359227+0000", 2024-09-18T11:33:15.993 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3124e-05 2024-09-18T11:33:15.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:15.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359235+0000", 2024-09-18T11:33:15.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0760000000000003e-06 2024-09-18T11:33:15.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:15.994 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359287+0000", 2024-09-18T11:33:15.994 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.151e-05 2024-09-18T11:33:15.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.994 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.994 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:15.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.359310+0000", 2024-09-18T11:33:15.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3218999999999999e-05 2024-09-18T11:33:15.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:15.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364065+0000", 2024-09-18T11:33:15.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047549439999999997 2024-09-18T11:33:15.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.995 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.995 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:15.995 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:51.364096+0000", 2024-09-18T11:33:15.995 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1423000000000001e-05 2024-09-18T11:33:15.995 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:15.996 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:15.996 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:15.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.996 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2300 4.4 e77/22 4:2c93196f:::smithi11939506-28:head v 77'94, mlcod=77'90)", 2024-09-18T11:33:15.996 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:33:15.996 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 134.97255157999999, 2024-09-18T11:33:15.996 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050425380000000001, 2024-09-18T11:33:15.996 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:15.996 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:15.996 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:15.996 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:15.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:33:15.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:15.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:15.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988109+0000", 2024-09-18T11:33:15.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:15.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.997 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.997 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:15.997 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988108+0000", 2024-09-18T11:33:15.997 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:15.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:15.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988388+0000", 2024-09-18T11:33:15.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028087700000000001 2024-09-18T11:33:15.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:15.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988405+0000", 2024-09-18T11:33:15.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6565000000000001e-05 2024-09-18T11:33:15.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.998 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.998 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:15.998 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988412+0000", 2024-09-18T11:33:15.998 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9709999999999998e-06 2024-09-18T11:33:15.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:15.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988464+0000", 2024-09-18T11:33:15.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1949e-05 2024-09-18T11:33:15.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:15.999 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.988490+0000", 2024-09-18T11:33:15.999 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5780999999999999e-05 2024-09-18T11:33:15.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:15.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:15.999 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993112+0000", 2024-09-18T11:33:16.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046228399999999996 2024-09-18T11:33:16.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.000 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.000 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:00.993151+0000", 2024-09-18T11:33:16.000 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8587999999999997e-05 2024-09-18T11:33:16.000 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.000 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.000 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.001 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.6.13:63 3.0 e78/21 3:24ebb740:::smithi11939506-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 78'80, mlcod=73'78)", 2024-09-18T11:33:16.001 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:33:16.001 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 134.55556216400001, 2024-09-18T11:33:16.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052262230000000003, 2024-09-18T11:33:16.001 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.001 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.001 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.001 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:33:16.001 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.001 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405098+0000", 2024-09-18T11:33:16.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405097+0000", 2024-09-18T11:33:16.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:16.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.002 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.002 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405103+0000", 2024-09-18T11:33:16.002 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8780000000000003e-06 2024-09-18T11:33:16.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405117+0000", 2024-09-18T11:33:16.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3441000000000001e-05 2024-09-18T11:33:16.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405121+0000", 2024-09-18T11:33:16.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7099999999999998e-06 2024-09-18T11:33:16.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.003 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.003 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405158+0000", 2024-09-18T11:33:16.003 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6477999999999999e-05 2024-09-18T11:33:16.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.405173+0000", 2024-09-18T11:33:16.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4796e-05 2024-09-18T11:33:16.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.004 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410285+0000", 2024-09-18T11:33:16.004 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051118919999999998 2024-09-18T11:33:16.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.004 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.005 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:01.410324+0000", 2024-09-18T11:33:16.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9616999999999997e-05 2024-09-18T11:33:16.005 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.005 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.005 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.005 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.5.13:18 3.0 e91/21 3:35e3c617:::2e64dd273da7619bd64b294f52905c1b9405a1ced2a8fd3c0b5bc481bc9732a6:head v 91'89, mlcod=85'87)", 2024-09-18T11:33:16.005 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:33:16.005 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 121.938786753, 2024-09-18T11:33:16.005 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048558539999999997, 2024-09-18T11:33:16.005 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.005 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.005 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:33:16.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:33:16.006 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.006 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.006 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021873+0000", 2024-09-18T11:33:16.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:16.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021880+0000", 2024-09-18T11:33:16.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5860000000000001e-06 2024-09-18T11:33:16.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.007 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.007 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021890+0000", 2024-09-18T11:33:16.007 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8290000000000003e-06 2024-09-18T11:33:16.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.021895+0000", 2024-09-18T11:33:16.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2480000000000004e-06 2024-09-18T11:33:16.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022166+0000", 2024-09-18T11:33:16.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027073499999999998 2024-09-18T11:33:16.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.008 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.008 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.022184+0000", 2024-09-18T11:33:16.008 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8082000000000001e-05 2024-09-18T11:33:16.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.009 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.009 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026693+0000", 2024-09-18T11:33:16.009 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045093479999999998 2024-09-18T11:33:16.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.014 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.014 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.026729+0000", 2024-09-18T11:33:16.014 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6282999999999998e-05 2024-09-18T11:33:16.014 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.014 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.014 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.014 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:573 4.4 e91/22 4:2c93196f:::smithi11939506-28:26 v 91'135, mlcod=90'131)", 2024-09-18T11:33:16.014 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:33:16.014 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 121.93632016399999, 2024-09-18T11:33:16.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050722739999999999, 2024-09-18T11:33:16.015 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.015 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.015 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:33:16.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.015 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.015 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.015 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024340+0000", 2024-09-18T11:33:16.015 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024339+0000", 2024-09-18T11:33:16.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:16.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.016 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024349+0000", 2024-09-18T11:33:16.016 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.6339999999999996e-06 2024-09-18T11:33:16.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.016 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.016 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024362+0000", 2024-09-18T11:33:16.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3305999999999999e-05 2024-09-18T11:33:16.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024368+0000", 2024-09-18T11:33:16.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8649999999999998e-06 2024-09-18T11:33:16.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.017 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.017 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024431+0000", 2024-09-18T11:33:16.017 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2780999999999994e-05 2024-09-18T11:33:16.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.017 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024448+0000", 2024-09-18T11:33:16.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7388999999999999e-05 2024-09-18T11:33:16.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029385+0000", 2024-09-18T11:33:16.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.004936523 2024-09-18T11:33:16.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.018 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.018 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.018 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.029412+0000", 2024-09-18T11:33:16.018 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.7702999999999999e-05 2024-09-18T11:33:16.019 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.019 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.019 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.019 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:574 4.4 e91/22 4:2cb3e6a5:::smithi11939506-3:1e v 91'137, mlcod=90'131)", 2024-09-18T11:33:16.019 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:33:16.019 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 121.93588212100001, 2024-09-18T11:33:16.019 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0070802850000000004, 2024-09-18T11:33:16.019 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.019 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.019 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.019 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:33:16.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024778+0000", 2024-09-18T11:33:16.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.020 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.020 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.020 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024777+0000", 2024-09-18T11:33:16.020 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:16.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024785+0000", 2024-09-18T11:33:16.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8099999999999998e-06 2024-09-18T11:33:16.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024794+0000", 2024-09-18T11:33:16.021 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5839999999999995e-06 2024-09-18T11:33:16.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.021 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.021 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024798+0000", 2024-09-18T11:33:16.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.031e-06 2024-09-18T11:33:16.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024830+0000", 2024-09-18T11:33:16.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2268999999999999e-05 2024-09-18T11:33:16.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.022 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.024847+0000", 2024-09-18T11:33:16.022 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7306999999999999e-05 2024-09-18T11:33:16.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.022 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.022 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031834+0000", 2024-09-18T11:33:16.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0069868380000000004 2024-09-18T11:33:16.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.023 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.023 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.031858+0000", 2024-09-18T11:33:16.023 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4082000000000001e-05 2024-09-18T11:33:16.023 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.023 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.023 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.023 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.023 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:541 4.d e91/22 4:b9bb642b:::smithi11939506-27:19 v 91'158, mlcod=91'154)", 2024-09-18T11:33:16.024 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:33:16.024 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 121.92850142899999, 2024-09-18T11:33:16.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0052185, 2024-09-18T11:33:16.024 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.024 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.024 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:33:16.024 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.024 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.024 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.024 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032159+0000", 2024-09-18T11:33:16.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032158+0000", 2024-09-18T11:33:16.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:16.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.025 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032165+0000", 2024-09-18T11:33:16.025 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7460000000000003e-06 2024-09-18T11:33:16.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.025 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.025 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032176+0000", 2024-09-18T11:33:16.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1460999999999999e-05 2024-09-18T11:33:16.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032182+0000", 2024-09-18T11:33:16.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0870000000000003e-06 2024-09-18T11:33:16.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.026 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.026 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.026 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032224+0000", 2024-09-18T11:33:16.026 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1372999999999998e-05 2024-09-18T11:33:16.026 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.032246+0000", 2024-09-18T11:33:16.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2158e-05 2024-09-18T11:33:16.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037348+0000", 2024-09-18T11:33:16.027 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051025319999999999 2024-09-18T11:33:16.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.027 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.027 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.027 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.037377+0000", 2024-09-18T11:33:16.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8937000000000001e-05 2024-09-18T11:33:16.028 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.028 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.028 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.028 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4178.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-18T11:33:16.028 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:33:16.028 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 94.019839364000006, 2024-09-18T11:33:16.028 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048960139999999997, 2024-09-18T11:33:16.028 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.028 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:16.028 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:16.029 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4178", 2024-09-18T11:33:16.029 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/490761693", 2024-09-18T11:33:16.029 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4 2024-09-18T11:33:16.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.029 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:33:16.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.029 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.029 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.029 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940821+0000", 2024-09-18T11:33:16.029 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940819+0000", 2024-09-18T11:33:16.030 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:16.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.030 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.030 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.030 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940832+0000", 2024-09-18T11:33:16.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3155e-05 2024-09-18T11:33:16.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.031 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940848+0000", 2024-09-18T11:33:16.031 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6288999999999999e-05 2024-09-18T11:33:16.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.031 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.031 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.940856+0000", 2024-09-18T11:33:16.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4919999999999996e-06 2024-09-18T11:33:16.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945613+0000", 2024-09-18T11:33:16.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0047574999999999996 2024-09-18T11:33:16.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.032 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.032 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.032 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:41.945717+0000", 2024-09-18T11:33:16.032 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000103553 2024-09-18T11:33:16.032 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.033 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.033 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.033 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3482 4.6 e117/67 4:67c1765a:::smithi11939506-30:head v 117'120, mlcod=115'116)", 2024-09-18T11:33:16.033 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:33:16.033 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 89.275985141000007, 2024-09-18T11:33:16.033 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050037479999999997, 2024-09-18T11:33:16.033 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.033 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.033 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.033 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.033 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:33:16.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684675+0000", 2024-09-18T11:33:16.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.034 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.034 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.034 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684674+0000", 2024-09-18T11:33:16.034 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:16.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684950+0000", 2024-09-18T11:33:16.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027621200000000002 2024-09-18T11:33:16.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684968+0000", 2024-09-18T11:33:16.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8e-05 2024-09-18T11:33:16.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.035 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.035 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.035 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.684984+0000", 2024-09-18T11:33:16.035 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5775000000000001e-05 2024-09-18T11:33:16.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685033+0000", 2024-09-18T11:33:16.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8878000000000002e-05 2024-09-18T11:33:16.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.036 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.685059+0000", 2024-09-18T11:33:16.036 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6801999999999999e-05 2024-09-18T11:33:16.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.036 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689640+0000", 2024-09-18T11:33:16.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045809029999999999 2024-09-18T11:33:16.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.037 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.037 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:46.689679+0000", 2024-09-18T11:33:16.037 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8507000000000002e-05 2024-09-18T11:33:16.037 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.037 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.037 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.037 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.037 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.0:976 4.4 e132/22 4:2cb3e6a5:::smithi11939506-3:30 v 132'195, mlcod=128'192)", 2024-09-18T11:33:16.037 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:33:16.038 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 72.942014069999999, 2024-09-18T11:33:16.038 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0058526510000000004, 2024-09-18T11:33:16.038 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.038 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.038 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.038 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.038 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:33:16.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.039 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.039 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:33:16.039 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018646+0000", 2024-09-18T11:33:16.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:16.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.040 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018661+0000", 2024-09-18T11:33:16.040 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5565000000000001e-05 2024-09-18T11:33:16.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.040 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.040 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018672+0000", 2024-09-18T11:33:16.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0210000000000001e-05 2024-09-18T11:33:16.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018678+0000", 2024-09-18T11:33:16.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1539999999999999e-06 2024-09-18T11:33:16.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.041 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.041 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.041 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018716+0000", 2024-09-18T11:33:16.041 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8389000000000001e-05 2024-09-18T11:33:16.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.018737+0000", 2024-09-18T11:33:16.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0611e-05 2024-09-18T11:33:16.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.024458+0000", 2024-09-18T11:33:16.042 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057208629999999996 2024-09-18T11:33:16.042 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.042 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.042 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.042 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:03.024499+0000", 2024-09-18T11:33:16.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1100999999999998e-05 2024-09-18T11:33:16.043 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.043 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.043 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.043 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.4.0:823 4.d e137/22 4:b44229a1:::smithi11939506-42:49 v 137'266, mlcod=132'260)", 2024-09-18T11:33:16.043 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:33:16.043 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 67.683365514000002, 2024-09-18T11:33:16.043 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0054497770000000003, 2024-09-18T11:33:16.043 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.043 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.043 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.043 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:33:16.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.044 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.044 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277295+0000", 2024-09-18T11:33:16.044 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.044 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.044 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277294+0000", 2024-09-18T11:33:16.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967296 2024-09-18T11:33:16.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277299+0000", 2024-09-18T11:33:16.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6280000000000004e-06 2024-09-18T11:33:16.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.045 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.045 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277306+0000", 2024-09-18T11:33:16.045 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.533e-06 2024-09-18T11:33:16.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277316+0000", 2024-09-18T11:33:16.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0148e-05 2024-09-18T11:33:16.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277868+0000", 2024-09-18T11:33:16.046 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00055199999999999997 2024-09-18T11:33:16.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.046 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.046 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.046 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.277898+0000", 2024-09-18T11:33:16.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0264999999999999e-05 2024-09-18T11:33:16.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.282715+0000", 2024-09-18T11:33:16.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0048165619999999999 2024-09-18T11:33:16.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.047 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.047 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.047 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:08.282744+0000", 2024-09-18T11:33:16.047 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9855000000000001e-05 2024-09-18T11:33:16.047 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.047 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.048 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.048 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:163 3.0 e138/21 3:02e8b30e:::4456d047c09ae2a9a25ccebfd83e30cf78df6779d63bfba9365afd7301eaeb6a:head v 138'132, mlcod=134'130)", 2024-09-18T11:33:16.048 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:33:16.048 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 66.378502771000001, 2024-09-18T11:33:16.048 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051351770000000003, 2024-09-18T11:33:16.048 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.048 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.048 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.048 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.048 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.048 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:33:16.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.049 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:33:16.049 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.049 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.049 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582157+0000", 2024-09-18T11:33:16.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:16.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582232+0000", 2024-09-18T11:33:16.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5333999999999994e-05 2024-09-18T11:33:16.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.050 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.050 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.050 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582242+0000", 2024-09-18T11:33:16.050 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5200000000000003e-06 2024-09-18T11:33:16.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582248+0000", 2024-09-18T11:33:16.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.8000000000000004e-06 2024-09-18T11:33:16.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582305+0000", 2024-09-18T11:33:16.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.7813999999999999e-05 2024-09-18T11:33:16.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.051 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.051 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.051 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.582329+0000", 2024-09-18T11:33:16.051 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3414999999999999e-05 2024-09-18T11:33:16.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587257+0000", 2024-09-18T11:33:16.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049279880000000003 2024-09-18T11:33:16.052 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.052 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.052 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.052 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587293+0000", 2024-09-18T11:33:16.052 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5802e-05 2024-09-18T11:33:16.052 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.052 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.052 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.053 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3974 4.d e138/22 4:b5a463bf:::smithi11939506-20:head v 138'270, mlcod=137'266)", 2024-09-18T11:33:16.053 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:33:16.053 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 66.374621500000003, 2024-09-18T11:33:16.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011228458, 2024-09-18T11:33:16.053 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.053 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.053 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.053 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.053 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.053 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:33:16.053 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586039+0000", 2024-09-18T11:33:16.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586038+0000", 2024-09-18T11:33:16.054 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:16.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.054 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.054 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.054 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587638+0000", 2024-09-18T11:33:16.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001599184 2024-09-18T11:33:16.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587647+0000", 2024-09-18T11:33:16.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.3719999999999994e-06 2024-09-18T11:33:16.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.055 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587653+0000", 2024-09-18T11:33:16.055 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.105e-06 2024-09-18T11:33:16.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.055 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587712+0000", 2024-09-18T11:33:16.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9252000000000001e-05 2024-09-18T11:33:16.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.587735+0000", 2024-09-18T11:33:16.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3132000000000001e-05 2024-09-18T11:33:16.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.056 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.056 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.056 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597228+0000", 2024-09-18T11:33:16.056 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0094930090000000002 2024-09-18T11:33:16.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.057 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.057 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597267+0000", 2024-09-18T11:33:16.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8693999999999999e-05 2024-09-18T11:33:16.057 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.057 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.057 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.057 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.057 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:3975 4.d e138/22 4:b5a463bf:::smithi11939506-20:head v 138'271, mlcod=137'266)", 2024-09-18T11:33:16.057 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:33:16.057 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 66.368420794000002, 2024-09-18T11:33:16.057 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0053620450000000002, 2024-09-18T11:33:16.057 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.058 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.058 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:33:16.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.058 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.058 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:33:16.058 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.058 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592239+0000", 2024-09-18T11:33:16.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:16.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592927+0000", 2024-09-18T11:33:16.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00068878699999999995 2024-09-18T11:33:16.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.059 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.059 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.059 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592940+0000", 2024-09-18T11:33:16.059 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2337999999999999e-05 2024-09-18T11:33:16.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.592951+0000", 2024-09-18T11:33:16.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1046000000000001e-05 2024-09-18T11:33:16.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.060 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593008+0000", 2024-09-18T11:33:16.060 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6875e-05 2024-09-18T11:33:16.060 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.060 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.060 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593033+0000", 2024-09-18T11:33:16.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5341999999999999e-05 2024-09-18T11:33:16.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597565+0000", 2024-09-18T11:33:16.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045323150000000003 2024-09-18T11:33:16.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.061 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.061 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.061 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.597601+0000", 2024-09-18T11:33:16.061 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6071000000000001e-05 2024-09-18T11:33:16.061 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.062 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.062 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.062 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.062 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.062 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:16.063 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T11:33:16.064 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:16.062+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:33:16.083 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:33:16.084 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:33:16.095 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-18T11:33:15.956072+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":442677650,"num_objects":323,"num_object_clones":102,"num_object_copies":646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":321,"num_whiteouts":4,"num_read":5643,"num_read_kb":5786198,"num_write":2728,"num_write_kb":758986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":733,"num_bytes_recovered":358556754,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":83,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5582,"ondisk_log_size":5582,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":3013624,"kb_used_data":591768,"kb_used_omap":28,"kb_used_meta":2421731,"kb_avail":751961096,"statfs":{"total":773094113280,"available":770008162304,"internally_reserved":0,"allocated":605970432,"data_stored":725593302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29629,"internal_metadata":2479852611},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.926128"},"pg_stats":[{"pgid":"4.f","version":"182'436","reported_seq":969,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.396384+0000","last_change":"2024-09-18T11:32:53.044075+0000","last_active":"2024-09-18T11:33:08.396384+0000","last_peered":"2024-09-18T11:33:08.396384+0000","last_clean":"2024-09-18T11:33:08.396384+0000","last_became_active":"2024-09-18T11:30:02.074391+0000","last_became_peered":"2024-09-18T11:30:02.074391+0000","last_unstale":"2024-09-18T11:33:08.396384+0000","last_undegraded":"2024-09-18T11:33:08.396384+0000","last_fullsized":"2024-09-18T11:33:08.396384+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"169'408","last_scrub_stamp":"2024-09-18T11:32:40.754566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:32:40.754566+0000","objects_scrubbed":9,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:44.296253+0000","scrub_duration":0.0037238530000000001,"objects_trimmed":0,"snaptrim_duration":0.010150497,"stat_sum":{"num_bytes":24390584,"num_objects":12,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":373,"num_read_kb":402862,"num_write":177,"num_write_kb":47119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":10,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"182'222","reported_seq":786,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.396053+0000","last_change":"2024-09-18T11:32:53.058981+0000","last_active":"2024-09-18T11:33:08.396053+0000","last_peered":"2024-09-18T11:33:08.396053+0000","last_clean":"2024-09-18T11:33:08.396053+0000","last_became_active":"2024-09-18T11:30:02.073185+0000","last_became_peered":"2024-09-18T11:30:02.073185+0000","last_unstale":"2024-09-18T11:33:08.396053+0000","last_undegraded":"2024-09-18T11:33:08.396053+0000","last_fullsized":"2024-09-18T11:33:08.396053+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"169'204","last_scrub_stamp":"2024-09-18T11:32:40.769398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:32:40.769398+0000","objects_scrubbed":7,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:01.753029+0000","scrub_duration":0.0021695009999999999,"objects_trimmed":0,"snaptrim_duration":0.017097318,"stat_sum":{"num_bytes":15873222,"num_objects":8,"num_object_clones":5,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":369,"num_read_kb":414869,"num_write":74,"num_write_kb":20255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"184'422","reported_seq":1243,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.396276+0000","last_change":"2024-09-18T11:33:04.037550+0000","last_active":"2024-09-18T11:33:08.396276+0000","last_peered":"2024-09-18T11:33:08.396276+0000","last_clean":"2024-09-18T11:33:08.396276+0000","last_became_active":"2024-09-18T11:33:04.037100+0000","last_became_peered":"2024-09-18T11:33:04.037100+0000","last_unstale":"2024-09-18T11:33:08.396276+0000","last_undegraded":"2024-09-18T11:33:08.396276+0000","last_fullsized":"2024-09-18T11:33:08.396276+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'346","last_scrub_stamp":"2024-09-18T11:32:29.706752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:32:29.706752+0000","objects_scrubbed":12,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:49.851840+0000","scrub_duration":0.003481076,"objects_trimmed":0,"snaptrim_duration":0.019540651999999999,"stat_sum":{"num_bytes":31062436,"num_objects":15,"num_object_clones":10,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":643,"num_read_kb":828166,"num_write":153,"num_write_kb":40845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":74215778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":9,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.c","version":"182'163","reported_seq":652,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.396034+0000","last_change":"2024-09-18T11:32:54.758480+0000","last_active":"2024-09-18T11:33:08.396034+0000","last_peered":"2024-09-18T11:33:08.396034+0000","last_clean":"2024-09-18T11:33:08.396034+0000","last_became_active":"2024-09-18T11:30:02.073883+0000","last_became_peered":"2024-09-18T11:30:02.073883+0000","last_unstale":"2024-09-18T11:33:08.396034+0000","last_undegraded":"2024-09-18T11:33:08.396034+0000","last_fullsized":"2024-09-18T11:33:08.396034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"182'163","last_scrub_stamp":"2024-09-18T11:32:54.758391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:32:54.758391+0000","objects_scrubbed":7,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:20.063140+0000","scrub_duration":0.0024865780000000001,"objects_trimmed":0,"snaptrim_duration":0.022452152999999999,"stat_sum":{"num_bytes":14392894,"num_objects":7,"num_object_clones":5,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":148,"num_read_kb":171779,"num_write":52,"num_write_kb":11924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":5,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"184'628","reported_seq":1281,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.033217+0000","last_change":"2024-09-18T11:33:05.160887+0000","last_active":"2024-09-18T11:33:08.033217+0000","last_peered":"2024-09-18T11:33:08.033217+0000","last_clean":"2024-09-18T11:33:08.033217+0000","last_became_active":"2024-09-18T11:32:24.024084+0000","last_became_peered":"2024-09-18T11:32:24.024084+0000","last_unstale":"2024-09-18T11:33:08.033217+0000","last_undegraded":"2024-09-18T11:33:08.033217+0000","last_fullsized":"2024-09-18T11:33:08.033217+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'628","last_scrub_stamp":"2024-09-18T11:33:05.160811+0000","last_deep_scrub":"95'286","last_deep_scrub_stamp":"2024-09-18T11:31:21.818375+0000","last_clean_scrub_stamp":"2024-09-18T11:33:05.160811+0000","objects_scrubbed":19,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:23.689827+0000","scrub_duration":0.003814123,"objects_trimmed":1,"snaptrim_duration":0.013533854999999999,"stat_sum":{"num_bytes":42386329,"num_objects":19,"num_object_clones":13,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":1,"num_read":638,"num_read_kb":752049,"num_write":255,"num_write_kb":70306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":46933733,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":6,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"182'317","reported_seq":829,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.396150+0000","last_change":"2024-09-18T11:32:53.046945+0000","last_active":"2024-09-18T11:33:08.396150+0000","last_peered":"2024-09-18T11:33:08.396150+0000","last_clean":"2024-09-18T11:33:08.396150+0000","last_became_active":"2024-09-18T11:30:02.075538+0000","last_became_peered":"2024-09-18T11:30:02.075538+0000","last_unstale":"2024-09-18T11:33:08.396150+0000","last_undegraded":"2024-09-18T11:33:08.396150+0000","last_fullsized":"2024-09-18T11:33:08.396150+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'313","last_scrub_stamp":"2024-09-18T11:32:50.511572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:32:50.511572+0000","objects_scrubbed":8,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:56.258219+0000","scrub_duration":0.0025077400000000001,"objects_trimmed":0,"snaptrim_duration":0.010874983,"stat_sum":{"num_bytes":15405144,"num_objects":8,"num_object_clones":6,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":304,"num_read_kb":434105,"num_write":103,"num_write_kb":28167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":7,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"169'120","reported_seq":601,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.396021+0000","last_change":"2024-09-18T11:32:53.041965+0000","last_active":"2024-09-18T11:33:08.396021+0000","last_peered":"2024-09-18T11:33:08.396021+0000","last_clean":"2024-09-18T11:33:08.396021+0000","last_became_active":"2024-09-18T11:30:02.074414+0000","last_became_peered":"2024-09-18T11:30:02.074414+0000","last_unstale":"2024-09-18T11:33:08.396021+0000","last_undegraded":"2024-09-18T11:33:08.396021+0000","last_fullsized":"2024-09-18T11:33:08.396021+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'111","last_scrub_stamp":"2024-09-18T11:32:28.794020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:32:28.794020+0000","objects_scrubbed":3,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:54.739238+0000","scrub_duration":0.0019747829999999999,"objects_trimmed":0,"snaptrim_duration":0.00054662899999999997,"stat_sum":{"num_bytes":3451605,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":109583,"num_write":43,"num_write_kb":12483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"194'103","reported_seq":561,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.033091+0000","last_change":"2024-09-18T11:33:06.188749+0000","last_active":"2024-09-18T11:33:08.033091+0000","last_peered":"2024-09-18T11:33:08.033091+0000","last_clean":"2024-09-18T11:33:08.033091+0000","last_became_active":"2024-09-18T11:30:02.076172+0000","last_became_peered":"2024-09-18T11:30:02.076172+0000","last_unstale":"2024-09-18T11:33:08.033091+0000","last_undegraded":"2024-09-18T11:33:08.033091+0000","last_fullsized":"2024-09-18T11:33:08.033091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'103","last_scrub_stamp":"2024-09-18T11:33:06.188666+0000","last_deep_scrub":"194'103","last_deep_scrub_stamp":"2024-09-18T11:33:06.188666+0000","last_clean_scrub_stamp":"2024-09-18T11:33:06.188666+0000","objects_scrubbed":5,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:21.248405+0000","scrub_duration":0.022988136999999999,"objects_trimmed":0,"snaptrim_duration":0.0055251049999999998,"stat_sum":{"num_bytes":10810459,"num_objects":5,"num_object_clones":4,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":48,"num_read_kb":39286,"num_write":42,"num_write_kb":10414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"182'405","reported_seq":948,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.396350+0000","last_change":"2024-09-18T11:32:53.046942+0000","last_active":"2024-09-18T11:33:08.396350+0000","last_peered":"2024-09-18T11:33:08.396350+0000","last_clean":"2024-09-18T11:33:08.396350+0000","last_became_active":"2024-09-18T11:30:02.074697+0000","last_became_peered":"2024-09-18T11:30:02.074697+0000","last_unstale":"2024-09-18T11:33:08.396350+0000","last_undegraded":"2024-09-18T11:33:08.396350+0000","last_fullsized":"2024-09-18T11:33:08.396350+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"169'389","last_scrub_stamp":"2024-09-18T11:32:40.598391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:32:40.598391+0000","objects_scrubbed":12,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:58.717506+0000","scrub_duration":0.0028905850000000002,"objects_trimmed":0,"snaptrim_duration":0.010580518000000001,"stat_sum":{"num_bytes":28591305,"num_objects":13,"num_object_clones":9,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":389,"num_read_kb":425070,"num_write":150,"num_write_kb":38085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":6,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":326,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.396873+0000","last_change":"2024-09-18T11:32:46.462271+0000","last_active":"2024-09-18T11:33:08.396873+0000","last_peered":"2024-09-18T11:33:08.396873+0000","last_clean":"2024-09-18T11:33:08.396873+0000","last_became_active":"2024-09-18T11:32:46.456653+0000","last_became_peered":"2024-09-18T11:32:46.456653+0000","last_unstale":"2024-09-18T11:33:08.396873+0000","last_undegraded":"2024-09-18T11:33:08.396873+0000","last_fullsized":"2024-09-18T11:33:08.396873+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'72","last_scrub_stamp":"2024-09-18T11:32:29.173114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_clean_scrub_stamp":"2024-09-18T11:32:29.173114+0000","objects_scrubbed":2,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:47.098458+0000","scrub_duration":0.0016253019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":236,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.033192+0000","last_change":"2024-09-18T11:32:42.123653+0000","last_active":"2024-09-18T11:33:08.033192+0000","last_peered":"2024-09-18T11:33:08.033192+0000","last_clean":"2024-09-18T11:33:08.033192+0000","last_became_active":"2024-09-18T11:32:27.079772+0000","last_became_peered":"2024-09-18T11:32:27.079772+0000","last_unstale":"2024-09-18T11:33:08.033192+0000","last_undegraded":"2024-09-18T11:33:08.033192+0000","last_fullsized":"2024-09-18T11:33:08.033192+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'1","last_scrub_stamp":"2024-09-18T11:32:42.123530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:32:42.123530+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:04.343044+0000","scrub_duration":0.0016360910000000001,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"182'203","reported_seq":419,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.395935+0000","last_change":"2024-09-18T11:32:39.596000+0000","last_active":"2024-09-18T11:33:08.395935+0000","last_peered":"2024-09-18T11:33:08.395935+0000","last_clean":"2024-09-18T11:33:08.395935+0000","last_became_active":"2024-09-18T11:30:01.077612+0000","last_became_peered":"2024-09-18T11:30:01.077612+0000","last_unstale":"2024-09-18T11:33:08.395935+0000","last_undegraded":"2024-09-18T11:33:08.395935+0000","last_fullsized":"2024-09-18T11:33:08.395935+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'194","last_scrub_stamp":"2024-09-18T11:32:39.595890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:32:39.595890+0000","objects_scrubbed":49,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:05.467884+0000","scrub_duration":0.0058412239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42108006,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":159,"num_read_kb":4762,"num_write":230,"num_write_kb":78760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"182'197","reported_seq":753,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-18T11:33:11.085435+0000","last_change":"2024-09-18T11:32:53.059438+0000","last_active":"2024-09-18T11:33:11.085435+0000","last_peered":"2024-09-18T11:33:11.085435+0000","last_clean":"2024-09-18T11:33:11.085435+0000","last_became_active":"2024-09-18T11:32:49.035340+0000","last_became_peered":"2024-09-18T11:32:49.035340+0000","last_unstale":"2024-09-18T11:33:11.085435+0000","last_undegraded":"2024-09-18T11:33:11.085435+0000","last_fullsized":"2024-09-18T11:33:11.085435+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"145'152","last_scrub_stamp":"2024-09-18T11:32:18.824714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:32:18.824714+0000","objects_scrubbed":7,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:27.399835+0000","scrub_duration":0.0028774880000000001,"objects_trimmed":0,"snaptrim_duration":0.0040844449999999999,"stat_sum":{"num_bytes":13681512,"num_objects":6,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":166,"num_read_kb":149008,"num_write":70,"num_write_kb":17901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":11402529,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":208,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.396808+0000","last_change":"2024-09-18T11:32:18.028783+0000","last_active":"2024-09-18T11:33:08.396808+0000","last_peered":"2024-09-18T11:33:08.396808+0000","last_clean":"2024-09-18T11:33:08.396808+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:33:08.396808+0000","last_undegraded":"2024-09-18T11:33:08.396808+0000","last_fullsized":"2024-09-18T11:33:08.396808+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:32:18.028662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:32:18.028662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:20.190152+0000","scrub_duration":0.001470486,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"173'206","reported_seq":845,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-18T11:33:11.085216+0000","last_change":"2024-09-18T11:33:01.261204+0000","last_active":"2024-09-18T11:33:11.085216+0000","last_peered":"2024-09-18T11:33:11.085216+0000","last_clean":"2024-09-18T11:33:11.085216+0000","last_became_active":"2024-09-18T11:33:01.060996+0000","last_became_peered":"2024-09-18T11:33:01.060996+0000","last_unstale":"2024-09-18T11:33:11.085216+0000","last_undegraded":"2024-09-18T11:33:11.085216+0000","last_fullsized":"2024-09-18T11:33:11.085216+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'206","last_scrub_stamp":"2024-09-18T11:32:51.136969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:32:51.136969+0000","objects_scrubbed":46,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:00.923271+0000","scrub_duration":0.0050590330000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43270431,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":179,"num_read_kb":11045,"num_write":225,"num_write_kb":76253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":74278671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"182'415","reported_seq":1148,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.396206+0000","last_change":"2024-09-18T11:33:03.876983+0000","last_active":"2024-09-18T11:33:08.396206+0000","last_peered":"2024-09-18T11:33:08.396206+0000","last_clean":"2024-09-18T11:33:08.396206+0000","last_became_active":"2024-09-18T11:33:01.058086+0000","last_became_peered":"2024-09-18T11:33:01.058086+0000","last_unstale":"2024-09-18T11:33:08.396206+0000","last_undegraded":"2024-09-18T11:33:08.396206+0000","last_fullsized":"2024-09-18T11:33:08.396206+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"182'415","last_scrub_stamp":"2024-09-18T11:33:03.876908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:33:03.876908+0000","objects_scrubbed":16,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:05.886279+0000","scrub_duration":0.0036095860000000001,"objects_trimmed":0,"snaptrim_duration":0.0021604340000000001,"stat_sum":{"num_bytes":24208503,"num_objects":16,"num_object_clones":11,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":506,"num_read_kb":525306,"num_write":148,"num_write_kb":40019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":49742791,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":11,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":123,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.396009+0000","last_change":"2024-09-18T11:32:59.475277+0000","last_active":"2024-09-18T11:33:08.396009+0000","last_peered":"2024-09-18T11:33:08.396009+0000","last_clean":"2024-09-18T11:33:08.396009+0000","last_became_active":"2024-09-18T11:32:59.474959+0000","last_became_peered":"2024-09-18T11:32:59.474959+0000","last_unstale":"2024-09-18T11:33:08.396009+0000","last_undegraded":"2024-09-18T11:33:08.396009+0000","last_fullsized":"2024-09-18T11:33:08.396009+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:32:32.611633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:32:32.611633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:43.724465+0000","scrub_duration":0.001364355,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"172'146","reported_seq":611,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.395993+0000","last_change":"2024-09-18T11:32:59.712706+0000","last_active":"2024-09-18T11:33:08.395993+0000","last_peered":"2024-09-18T11:33:08.395993+0000","last_clean":"2024-09-18T11:33:08.395993+0000","last_became_active":"2024-09-18T11:32:59.476088+0000","last_became_peered":"2024-09-18T11:32:59.476088+0000","last_unstale":"2024-09-18T11:33:08.395993+0000","last_undegraded":"2024-09-18T11:33:08.395993+0000","last_fullsized":"2024-09-18T11:33:08.395993+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"144'136","last_scrub_stamp":"2024-09-18T11:32:20.808182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:32:20.808182+0000","objects_scrubbed":29,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:24.436066+0000","scrub_duration":0.0050914910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24142412,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":132,"num_read_kb":1719,"num_write":158,"num_write_kb":46975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":29291946,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"178'242","reported_seq":868,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.395883+0000","last_change":"2024-09-18T11:32:59.595509+0000","last_active":"2024-09-18T11:33:08.395883+0000","last_peered":"2024-09-18T11:33:08.395883+0000","last_clean":"2024-09-18T11:33:08.395883+0000","last_became_active":"2024-09-18T11:32:59.475868+0000","last_became_peered":"2024-09-18T11:32:59.475868+0000","last_unstale":"2024-09-18T11:33:08.395883+0000","last_undegraded":"2024-09-18T11:33:08.395883+0000","last_fullsized":"2024-09-18T11:33:08.395883+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"169'235","last_scrub_stamp":"2024-09-18T11:32:42.761151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:32:42.761151+0000","objects_scrubbed":6,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:07.314558+0000","scrub_duration":0.0020700279999999998,"objects_trimmed":0,"snaptrim_duration":0.000395702,"stat_sum":{"num_bytes":15274901,"num_objects":7,"num_object_clones":4,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":286,"num_read_kb":341631,"num_write":118,"num_write_kb":31261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":18638112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":211,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.033103+0000","last_change":"2024-09-18T11:33:04.151627+0000","last_active":"2024-09-18T11:33:08.033103+0000","last_peered":"2024-09-18T11:33:08.033103+0000","last_clean":"2024-09-18T11:33:08.033103+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:33:08.033103+0000","last_undegraded":"2024-09-18T11:33:08.033103+0000","last_fullsized":"2024-09-18T11:33:08.033103+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"18'2","last_scrub_stamp":"2024-09-18T11:33:04.151495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:04.151495+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:24.817971+0000","scrub_duration":0.0017471489999999999,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"179'157","reported_seq":587,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.395970+0000","last_change":"2024-09-18T11:32:46.707704+0000","last_active":"2024-09-18T11:33:08.395970+0000","last_peered":"2024-09-18T11:33:08.395970+0000","last_clean":"2024-09-18T11:33:08.395970+0000","last_became_active":"2024-09-18T11:32:46.456572+0000","last_became_peered":"2024-09-18T11:32:46.456572+0000","last_unstale":"2024-09-18T11:33:08.395970+0000","last_undegraded":"2024-09-18T11:33:08.395970+0000","last_fullsized":"2024-09-18T11:33:08.395970+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"166'152","last_scrub_stamp":"2024-09-18T11:32:42.852232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:32:42.852232+0000","objects_scrubbed":33,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:04.691723+0000","scrub_duration":0.0039393850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31988294,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":127,"num_read_kb":7849,"num_write":171,"num_write_kb":62728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":36817156,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"179'137","reported_seq":647,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.395883+0000","last_change":"2024-09-18T11:33:01.057644+0000","last_active":"2024-09-18T11:33:08.395883+0000","last_peered":"2024-09-18T11:33:08.395883+0000","last_clean":"2024-09-18T11:33:08.395883+0000","last_became_active":"2024-09-18T11:33:01.057128+0000","last_became_peered":"2024-09-18T11:33:01.057128+0000","last_unstale":"2024-09-18T11:33:08.395883+0000","last_undegraded":"2024-09-18T11:33:08.395883+0000","last_fullsized":"2024-09-18T11:33:08.395883+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"169'136","last_scrub_stamp":"2024-09-18T11:32:41.161334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:32:41.161334+0000","objects_scrubbed":2,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:05.049209+0000","scrub_duration":0.001814244,"objects_trimmed":0,"snaptrim_duration":0.00037324100000000001,"stat_sum":{"num_bytes":5310084,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":144270,"num_write":54,"num_write_kb":14782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":8422554,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":232,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.396155+0000","last_change":"2024-09-18T11:32:46.457112+0000","last_active":"2024-09-18T11:33:08.396155+0000","last_peered":"2024-09-18T11:33:08.396155+0000","last_clean":"2024-09-18T11:33:08.396155+0000","last_became_active":"2024-09-18T11:32:46.456819+0000","last_became_peered":"2024-09-18T11:32:46.456819+0000","last_unstale":"2024-09-18T11:33:08.396155+0000","last_undegraded":"2024-09-18T11:33:08.396155+0000","last_fullsized":"2024-09-18T11:33:08.396155+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:32:30.705704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:32:30.705704+0000","last_clean_scrub_stamp":"2024-09-18T11:32:30.705704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:42.193160+0000","scrub_duration":0.0039850399999999996,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"182'305","reported_seq":859,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-18T11:33:11.085333+0000","last_change":"2024-09-18T11:33:03.236656+0000","last_active":"2024-09-18T11:33:11.085333+0000","last_peered":"2024-09-18T11:33:11.085333+0000","last_clean":"2024-09-18T11:33:11.085333+0000","last_became_active":"2024-09-18T11:32:49.034281+0000","last_became_peered":"2024-09-18T11:32:49.034281+0000","last_unstale":"2024-09-18T11:33:11.085333+0000","last_undegraded":"2024-09-18T11:33:11.085333+0000","last_fullsized":"2024-09-18T11:33:11.085333+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"182'305","last_scrub_stamp":"2024-09-18T11:33:03.236586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:33:03.236586+0000","objects_scrubbed":7,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:09.351032+0000","scrub_duration":0.0027275789999999999,"objects_trimmed":0,"snaptrim_duration":0.0087594539999999999,"stat_sum":{"num_bytes":13511520,"num_objects":7,"num_object_clones":4,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":234,"num_read_kb":245979,"num_write":119,"num_write_kb":31575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":7755628,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":5,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":228,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.396101+0000","last_change":"2024-09-18T11:32:46.456500+0000","last_active":"2024-09-18T11:33:08.396101+0000","last_peered":"2024-09-18T11:33:08.396101+0000","last_clean":"2024-09-18T11:33:08.396101+0000","last_became_active":"2024-09-18T11:32:46.456211+0000","last_became_peered":"2024-09-18T11:32:46.456211+0000","last_unstale":"2024-09-18T11:33:08.396101+0000","last_undegraded":"2024-09-18T11:33:08.396101+0000","last_fullsized":"2024-09-18T11:33:08.396101+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:32:42.223963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:32:42.223963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:12.013506+0000","scrub_duration":0.001097408,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"184'286","reported_seq":805,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.396003+0000","last_change":"2024-09-18T11:32:53.033485+0000","last_active":"2024-09-18T11:33:08.396003+0000","last_peered":"2024-09-18T11:33:08.396003+0000","last_clean":"2024-09-18T11:33:08.396003+0000","last_became_active":"2024-09-18T11:30:02.072165+0000","last_became_peered":"2024-09-18T11:30:02.072165+0000","last_unstale":"2024-09-18T11:33:08.396003+0000","last_undegraded":"2024-09-18T11:33:08.396003+0000","last_fullsized":"2024-09-18T11:33:08.396003+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"145'219","last_scrub_stamp":"2024-09-18T11:32:18.546181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:32:18.546181+0000","objects_scrubbed":8,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:45.217135+0000","scrub_duration":0.0027375450000000001,"objects_trimmed":0,"snaptrim_duration":0.0032416960000000001,"stat_sum":{"num_bytes":21157839,"num_objects":12,"num_object_clones":9,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":279,"num_read_kb":315300,"num_write":126,"num_write_kb":35637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":8,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.396372+0000","last_change":"2024-09-18T11:32:59.475270+0000","last_active":"2024-09-18T11:33:08.396372+0000","last_peered":"2024-09-18T11:33:08.396372+0000","last_clean":"2024-09-18T11:33:08.396372+0000","last_became_active":"2024-09-18T11:32:59.474991+0000","last_became_peered":"2024-09-18T11:32:59.474991+0000","last_unstale":"2024-09-18T11:33:08.396372+0000","last_undegraded":"2024-09-18T11:33:08.396372+0000","last_fullsized":"2024-09-18T11:33:08.396372+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:32:28.708882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:32:28.708882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:55.766077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"182'397","reported_seq":941,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.396137+0000","last_change":"2024-09-18T11:32:53.056310+0000","last_active":"2024-09-18T11:33:08.396137+0000","last_peered":"2024-09-18T11:33:08.396137+0000","last_clean":"2024-09-18T11:33:08.396137+0000","last_became_active":"2024-09-18T11:30:02.075529+0000","last_became_peered":"2024-09-18T11:30:02.075529+0000","last_unstale":"2024-09-18T11:33:08.396137+0000","last_undegraded":"2024-09-18T11:33:08.396137+0000","last_fullsized":"2024-09-18T11:33:08.396137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'391","last_scrub_stamp":"2024-09-18T11:32:50.713250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:01.063520+0000","last_clean_scrub_stamp":"2024-09-18T11:32:50.713250+0000","objects_scrubbed":12,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:59.180972+0000","scrub_duration":0.00290735,"objects_trimmed":0,"snaptrim_duration":0.01426404,"stat_sum":{"num_bytes":21069783,"num_objects":12,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":390,"num_read_kb":461505,"num_write":157,"num_write_kb":41867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":6,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":208,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-18T11:33:08.396087+0000","last_change":"2024-09-18T11:32:06.213366+0000","last_active":"2024-09-18T11:33:08.396087+0000","last_peered":"2024-09-18T11:33:08.396087+0000","last_clean":"2024-09-18T11:33:08.396087+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:33:08.396087+0000","last_undegraded":"2024-09-18T11:33:08.396087+0000","last_fullsized":"2024-09-18T11:33:08.396087+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:32:06.213281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:32:06.213281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:23.965948+0000","scrub_duration":0.0013619280000000001,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":300578120,"num_objects":151,"num_object_clones":102,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":4,"num_read":4980,"num_read_kb":5760768,"num_write":1841,"num_write_kb":492640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":217111125,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":83,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405327872,"data_stored":529914958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4795,"ondisk_log_size":4795,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":141509143,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":597,"num_read_kb":25375,"num_write":784,"num_write_kb":264716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":140387773,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":205524992,"data_stored":205034291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4182,"internal_metadata":0},"log_size":712,"ondisk_log_size":712,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":323576,"kb_used_data":22060,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94048264,"statfs":{"total":96636764160,"available":96305422336,"internally_reserved":0,"allocated":22589440,"data_stored":30379250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.34000000000000002}]},{"osd":1,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45300000000000001,"15min":0.46300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.65300000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.505,"15min":0.51800000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.754,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.45700000000000002}]},{"osd":2,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.29199999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.378},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42899999999999999,"15min":0.435},"min":{"1min":0.317,"5min":0.314,"15min":0.314},"max":{"1min":0.52100000000000002,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.40999999999999998}]},{"osd":3,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.439,"15min":0.442},"min":{"1min":0.36499999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.55100000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.47099999999999997,"15min":0.48599999999999999},"min":{"1min":0.311,"5min":0.308,"15min":0.308},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.39900000000000002}]},{"osd":4,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.439,"15min":0.45700000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.626,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.442},{"interface":"front","average":{"1min":0.437,"5min":0.47799999999999998,"15min":0.498},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.624,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.433}]},{"osd":5,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.41899999999999998,"15min":0.44},"min":{"1min":0.24399999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.71199999999999997,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.442,"15min":0.46500000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.72399999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.38800000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.5,"15min":0.52300000000000002},"min":{"1min":0.28699999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48599999999999999,"15min":0.502},"min":{"1min":0.29399999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69099999999999995,"5min":0.75,"15min":0.75},"last":0.45000000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":343704,"kb_used_data":42188,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94028136,"statfs":{"total":96636764160,"available":96284811264,"internally_reserved":0,"allocated":43200512,"data_stored":56067714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.24099999999999999}]},{"osd":1,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.46600000000000003,"15min":0.46899999999999997},"min":{"1min":0.379,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.61399999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.45400000000000001,"15min":0.45600000000000002},"min":{"1min":0.32800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.69499999999999995,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.36099999999999999}]},{"osd":2,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46100000000000002,"15min":0.46300000000000002},"min":{"1min":0.312,"5min":0.221,"15min":0.221},"max":{"1min":0.64700000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.45300000000000001,"15min":0.44700000000000001},"min":{"1min":0.34300000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.37}]},{"osd":3,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.49299999999999999,"15min":0.51100000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.67100000000000004,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.505,"15min":0.51900000000000002},"min":{"1min":0.33900000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69699999999999995,"5min":1.032,"15min":1.032},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.40100000000000002,"15min":0.39300000000000002},"min":{"1min":0.316,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.61299999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.437,"15min":0.42499999999999999},"min":{"1min":0.41299999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65400000000000003,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.34799999999999998}]},{"osd":5,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.41499999999999998,"15min":0.40899999999999997},"min":{"1min":0.29599999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66700000000000004,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.317},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.441,"15min":0.433},"min":{"1min":0.34799999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.33200000000000002}]},{"osd":7,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.46500000000000002,"15min":0.46400000000000002},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.48399999999999999,"15min":0.47799999999999998},"min":{"1min":0.34599999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76600000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.30299999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413452,"kb_used_data":107776,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93958388,"statfs":{"total":96636764160,"available":96213389312,"internally_reserved":0,"allocated":110362624,"data_stored":130932585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.45800000000000002}]},{"osd":1,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42399999999999999,"15min":0.42999999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.44700000000000001,"15min":0.44900000000000001},"min":{"1min":0.248,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.47499999999999998}]},{"osd":2,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47999999999999998,"15min":0.496},"min":{"1min":0.316,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66600000000000004,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.47199999999999998,"15min":0.49399999999999999},"min":{"1min":0.32400000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.59599999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.35599999999999998}]},{"osd":3,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.46700000000000003,"15min":0.48899999999999999},"min":{"1min":0.29899999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.51300000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.38},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.48399999999999999,"15min":0.496},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.61299999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.49199999999999999}]},{"osd":4,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42999999999999999,"15min":0.44400000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.435,"15min":0.45700000000000002},"min":{"1min":0.22900000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.65700000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.39900000000000002}]},{"osd":6,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.25600000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.44},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.46700000000000003,"15min":0.48599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.57899999999999996}]},{"osd":7,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.47499999999999998,"15min":0.499},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.72899999999999998,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.499,"5min":0.503,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.53900000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":429752,"kb_used_data":124076,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93942088,"statfs":{"total":96636764160,"available":96196698112,"internally_reserved":0,"allocated":127053824,"data_stored":147754368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.314},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.373}]},{"osd":1,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.47899999999999998,"15min":0.501},"min":{"1min":0.315,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.65300000000000002,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48399999999999999,"15min":0.498},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73299999999999998,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.40400000000000003}]},{"osd":2,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48099999999999998,"15min":0.49299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.748,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.49199999999999999,"15min":0.51600000000000001},"min":{"1min":0.35399999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72599999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.36299999999999999}]},{"osd":3,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.48899999999999999,"15min":0.51300000000000001},"min":{"1min":0.35799999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.61399999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.50600000000000001,"15min":0.52700000000000002},"min":{"1min":0.378,"5min":0.316,"15min":0.316},"max":{"1min":0.68799999999999994,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.34499999999999997}]},{"osd":5,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.442,"15min":0.46200000000000002},"min":{"1min":0.27700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.70699999999999996,"5min":1.002,"15min":1.002},"last":0.25},{"interface":"front","average":{"1min":0.442,"5min":0.47999999999999998,"15min":0.50800000000000001},"min":{"1min":0.30199999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.70299999999999996,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.29899999999999999}]},{"osd":6,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.45000000000000001,"15min":0.46899999999999997},"min":{"1min":0.28199999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.69399999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48399999999999999,"15min":0.504},"min":{"1min":0.33200000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.746,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.32500000000000001}]},{"osd":7,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.49199999999999999,"15min":0.51000000000000001},"min":{"1min":0.34899999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.54500000000000004},"min":{"1min":0.34100000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66700000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.42199999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":398212,"kb_used_data":96696,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93973628,"statfs":{"total":96636764160,"available":96228995072,"internally_reserved":0,"allocated":99016704,"data_stored":118781867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.315},{"interface":"front","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.30099999999999999}]},{"osd":1,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.33100000000000002,"15min":0.34100000000000003},"min":{"1min":0.246,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.45500000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.36799999999999999},"min":{"1min":0.224,"5min":0.214,"15min":0.214},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.34000000000000002}]},{"osd":2,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.27800000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.377,"15min":0.39900000000000002},"min":{"1min":0.28100000000000003,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.48699999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.28299999999999997}]},{"osd":4,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.50700000000000001,"15min":0.51300000000000001},"min":{"1min":0.39300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.64600000000000002,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52000000000000002,"15min":0.52400000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.57999999999999996}]},{"osd":5,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.49199999999999999,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69399999999999995,"5min":0.752,"15min":0.752},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53100000000000003,"15min":0.55500000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67000000000000004,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.54400000000000004}]},{"osd":6,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.51300000000000001,"15min":0.53300000000000003},"min":{"1min":0.35999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.60899999999999999,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.54200000000000004,"15min":0.56100000000000005},"min":{"1min":0.39800000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.66500000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.55500000000000005}]},{"osd":7,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56999999999999995,"15min":0.60099999999999998},"min":{"1min":0.41599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68899999999999995,"5min":1.26,"15min":1.26},"last":0.502},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.54500000000000004,"15min":0.57199999999999995},"min":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.63,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.56599999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574891,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":348648,"kb_used_data":47132,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94023192,"statfs":{"total":96636764160,"available":96279748608,"internally_reserved":0,"allocated":48263168,"data_stored":56232964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.45700000000000002}]},{"osd":1,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.35599999999999998,"15min":0.375},"min":{"1min":0.23200000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.57099999999999995,"5min":1.621,"15min":1.621},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.377,"15min":0.39900000000000002},"min":{"1min":0.27100000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.70599999999999996,"5min":1.589,"15min":1.589},"last":0.41299999999999998}]},{"osd":3,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35799999999999998,"15min":0.36799999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.72399999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.441},{"interface":"front","average":{"1min":0.379,"5min":0.39200000000000002,"15min":0.41399999999999998},"min":{"1min":0.27500000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52900000000000003,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.42599999999999999}]},{"osd":4,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53100000000000003,"15min":0.55100000000000005},"min":{"1min":0.373,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.755,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.504,"15min":0.51700000000000002},"min":{"1min":0.34300000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73199999999999998,"5min":1.012,"15min":1.012},"last":0.60699999999999998}]},{"osd":5,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53000000000000003,"15min":0.54200000000000004},"min":{"1min":0.36799999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70799999999999996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.628},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.56100000000000005},"min":{"1min":0.377,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77900000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.63800000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52400000000000002,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82399999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.622},{"interface":"front","average":{"1min":0.5,"5min":0.53600000000000003,"15min":0.56100000000000005},"min":{"1min":0.35499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65200000000000002,"5min":1.165,"15min":1.165},"last":0.56499999999999995}]},{"osd":7,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54400000000000004,"15min":0.56000000000000005},"min":{"1min":0.371,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70099999999999996,"5min":1.073,"15min":1.073},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.39700000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91600000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.59799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":388932,"kb_used_data":83256,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93982908,"statfs":{"total":96636764160,"available":96238497792,"internally_reserved":0,"allocated":85254144,"data_stored":113978107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.26800000000000002}]},{"osd":2,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.38100000000000001},"min":{"1min":0.28699999999999998,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.871,"5min":1.333,"15min":1.333},"last":0.316},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.40000000000000002,"15min":0.42099999999999999},"min":{"1min":0.23899999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.89200000000000002,"5min":1.298,"15min":1.298},"last":0.32600000000000001}]},{"osd":3,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.371,"15min":0.38200000000000001},"min":{"1min":0.27300000000000002,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.35599999999999998,"15min":0.36199999999999999},"min":{"1min":0.22500000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.29399999999999998}]},{"osd":4,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51500000000000001,"15min":0.52500000000000002},"min":{"1min":0.307,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.34999999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.96899999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.51100000000000001}]},{"osd":5,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.50700000000000001,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.91800000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.52500000000000002},"min":{"1min":0.32300000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.47899999999999998}]},{"osd":6,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.34000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53200000000000003,"15min":0.54300000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.94499999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56899999999999995}]},{"osd":7,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53700000000000003,"15min":0.54700000000000004},"min":{"1min":0.36299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56599999999999995,"15min":0.57899999999999996},"min":{"1min":0.34200000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.48999999999999999}]}]},{"osd":0,"up_from":119,"seq":511101108243,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":367348,"kb_used_data":68584,"kb_used_omap":17,"kb_used_meta":298734,"kb_avail":94004492,"statfs":{"total":96636764160,"available":96260599808,"internally_reserved":0,"allocated":70230016,"data_stored":71466447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17701,"internal_metadata":305904347},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.249},{"interface":"front","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.42099999999999999}]},{"osd":2,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.37,"5min":0.37,"15min":0.37},"last":0.26800000000000002}]},{"osd":3,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.311,"15min":0.311},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.38700000000000001}]},{"osd":4,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.379},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.40100000000000002}]},{"osd":5,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.41299999999999998}]},{"osd":6,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.46600000000000003}]},{"osd":7,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.44},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.45600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4182,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17088512,"data_stored":18954830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":67264512,"data_stored":96612579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":30281728,"data_stored":38867775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":71155712,"data_stored":91590273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":74366976,"data_stored":95760172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":82558976,"data_stored":103750068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":41881600,"data_stored":55299662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20729856,"data_stored":29079599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:33:16.098 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-18T11:33:16.099 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-18T11:33:16.117 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T11:33:16.180 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:33:16.179+0000 7f2d4e5f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33321) UID: 0 2024-09-18T11:33:16.187 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:16.188 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:33:16.188 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:33:16.188 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:16.198 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T11:33:16.265 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:16.264+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:16.339 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:16.339 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:33:16.340 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:33:16.340 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:33:16.340 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:16.348 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T11:33:16.367 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:33:16.366+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:33:16.448 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:33:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:33:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:33:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:925 4.0 4:019b6e09:::smithi11939506-40:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:33:16.448 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:33:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 185.27626600900001, 2024-09-18T11:33:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011488124000000001, 2024-09-18T11:33:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 925 2024-09-18T11:33:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.449 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:33:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169419+0000", 2024-09-18T11:33:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169417+0000", 2024-09-18T11:33:16.450 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169431+0000", 2024-09-18T11:33:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4032000000000001e-05 2024-09-18T11:33:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169450+0000", 2024-09-18T11:33:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8768000000000001e-05 2024-09-18T11:33:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.451 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169458+0000", 2024-09-18T11:33:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8860000000000004e-06 2024-09-18T11:33:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169513+0000", 2024-09-18T11:33:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.5108000000000002e-05 2024-09-18T11:33:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.452 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.169597+0000", 2024-09-18T11:33:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3438999999999999e-05 2024-09-18T11:33:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:11.180908+0000", 2024-09-18T11:33:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011310908 2024-09-18T11:33:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1002 4.0 4:019b6e09:::smithi11939506-40:6 [sparse-read 0~1274797] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:33:16.453 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:33:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 181.28108801499999, 2024-09-18T11:33:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0078758679999999994, 2024-09-18T11:33:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1002 2024-09-18T11:33:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:33:16.454 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:33:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164597+0000", 2024-09-18T11:33:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.455 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164601+0000", 2024-09-18T11:33:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.749e-06 2024-09-18T11:33:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164607+0000", 2024-09-18T11:33:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.9359999999999996e-06 2024-09-18T11:33:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164611+0000", 2024-09-18T11:33:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.0069999999999999e-06 2024-09-18T11:33:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.456 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164740+0000", 2024-09-18T11:33:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129438 2024-09-18T11:33:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164744+0000", 2024-09-18T11:33:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9870000000000001e-06 2024-09-18T11:33:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.457 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172070+0000", 2024-09-18T11:33:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0073258589999999997 2024-09-18T11:33:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172185+0000", 2024-09-18T11:33:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000114799 2024-09-18T11:33:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172473+0000", 2024-09-18T11:33:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028857600000000002 2024-09-18T11:33:16.458 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1003 4.0 4:059d8d95:::smithi11939506-32:6 [read 0~3075648,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:33:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:33:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 181.281069726, 2024-09-18T11:33:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010454586, 2024-09-18T11:33:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:16.459 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1003 2024-09-18T11:33:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:33:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164616+0000", 2024-09-18T11:33:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.460 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164615+0000", 2024-09-18T11:33:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164627+0000", 2024-09-18T11:33:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1528e-05 2024-09-18T11:33:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164634+0000", 2024-09-18T11:33:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7689999999999999e-06 2024-09-18T11:33:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164638+0000", 2024-09-18T11:33:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7720000000000002e-06 2024-09-18T11:33:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164768+0000", 2024-09-18T11:33:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000129851 2024-09-18T11:33:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.462 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164773+0000", 2024-09-18T11:33:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.729e-06 2024-09-18T11:33:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172518+0000", 2024-09-18T11:33:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077451439999999998 2024-09-18T11:33:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.463 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.172636+0000", 2024-09-18T11:33:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000117701 2024-09-18T11:33:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175070+0000", 2024-09-18T11:33:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024345119999999998 2024-09-18T11:33:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.464 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1004 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:33:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:33:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 181.28100895700001, 2024-09-18T11:33:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010808834999999999, 2024-09-18T11:33:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:16.465 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1004 2024-09-18T11:33:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:33:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164677+0000", 2024-09-18T11:33:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164676+0000", 2024-09-18T11:33:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164680+0000", 2024-09-18T11:33:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5860000000000001e-06 2024-09-18T11:33:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164689+0000", 2024-09-18T11:33:16.467 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2359999999999993e-06 2024-09-18T11:33:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164694+0000", 2024-09-18T11:33:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9099999999999996e-06 2024-09-18T11:33:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164793+0000", 2024-09-18T11:33:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9415999999999998e-05 2024-09-18T11:33:16.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164797+0000", 2024-09-18T11:33:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7900000000000001e-06 2024-09-18T11:33:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175110+0000", 2024-09-18T11:33:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010313521000000001 2024-09-18T11:33:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.469 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175205+0000", 2024-09-18T11:33:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4652000000000001e-05 2024-09-18T11:33:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175485+0000", 2024-09-18T11:33:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00028032699999999997 2024-09-18T11:33:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.470 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1005 4.0 4:059d8d95:::smithi11939506-32:6 [sparse-read 0~1630974] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-18T11:33:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:33:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 181.28098663599999, 2024-09-18T11:33:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011205054000000001, 2024-09-18T11:33:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1005 2024-09-18T11:33:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.471 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:33:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164699+0000", 2024-09-18T11:33:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164698+0000", 2024-09-18T11:33:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:16.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164701+0000", 2024-09-18T11:33:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5179999999999999e-06 2024-09-18T11:33:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164708+0000", 2024-09-18T11:33:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.007e-06 2024-09-18T11:33:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.473 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164715+0000", 2024-09-18T11:33:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:33:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164819+0000", 2024-09-18T11:33:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000104802 2024-09-18T11:33:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:16.474 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.164824+0000", 2024-09-18T11:33:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.916e-06 2024-09-18T11:33:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175509+0000", 2024-09-18T11:33:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.01068475 2024-09-18T11:33:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.475 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175594+0000", 2024-09-18T11:33:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5211999999999997e-05 2024-09-18T11:33:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:15.175904+0000", 2024-09-18T11:33:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00030955799999999998 2024-09-18T11:33:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:1118 3.2 3:6a581379:::afc9a51cddcf32e15f6dde6b67a9509a7f12db3d041549e5eb56bd3e1b55f11b:head [write 0~1835520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T11:33:16.476 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:33:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 176.499861753, 2024-09-18T11:33:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084296490000000009, 2024-09-18T11:33:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 1118 2024-09-18T11:33:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:33:16.477 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945824+0000", 2024-09-18T11:33:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.945820+0000", 2024-09-18T11:33:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999967 2024-09-18T11:33:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.478 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947330+0000", 2024-09-18T11:33:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0015095989999999999 2024-09-18T11:33:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947352+0000", 2024-09-18T11:33:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.2354999999999999e-05 2024-09-18T11:33:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947360+0000", 2024-09-18T11:33:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4000000000000003e-06 2024-09-18T11:33:16.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947414+0000", 2024-09-18T11:33:16.480 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4317999999999997e-05 2024-09-18T11:33:16.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.480 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.480 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947561+0000", 2024-09-18T11:33:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014703700000000001 2024-09-18T11:33:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.947724+0000", 2024-09-18T11:33:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016331399999999999 2024-09-18T11:33:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.952756+0000", 2024-09-18T11:33:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.005031939 2024-09-18T11:33:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.481 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954158+0000", 2024-09-18T11:33:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014020860000000001 2024-09-18T11:33:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954210+0000", 2024-09-18T11:33:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1799999999999999e-05 2024-09-18T11:33:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:30:19.954253+0000", 2024-09-18T11:33:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3213000000000001e-05 2024-09-18T11:33:16.482 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:2503 3.1 e85/67 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head v 85'117, mlcod=84'115)", 2024-09-18T11:33:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:33:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 127.468463066, 2024-09-18T11:33:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086527619999999996, 2024-09-18T11:33:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.483 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:33:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977222+0000", 2024-09-18T11:33:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.977220+0000", 2024-09-18T11:33:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999976 2024-09-18T11:33:16.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980771+0000", 2024-09-18T11:33:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035505200000000002 2024-09-18T11:33:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980797+0000", 2024-09-18T11:33:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.618e-05 2024-09-18T11:33:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.485 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980807+0000", 2024-09-18T11:33:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0433e-05 2024-09-18T11:33:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980861+0000", 2024-09-18T11:33:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3415999999999998e-05 2024-09-18T11:33:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.980885+0000", 2024-09-18T11:33:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.4212000000000001e-05 2024-09-18T11:33:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985826+0000", 2024-09-18T11:33:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049412450000000004 2024-09-18T11:33:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:08.985875+0000", 2024-09-18T11:33:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.9116000000000002e-05 2024-09-18T11:33:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.487 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.488 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2649 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:33:16.488 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:33:16.488 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 122.114319697, 2024-09-18T11:33:16.488 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079566240000000007, 2024-09-18T11:33:16.488 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.488 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:16.488 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:16.488 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2649 2024-09-18T11:33:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:33:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.489 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331366+0000", 2024-09-18T11:33:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331365+0000", 2024-09-18T11:33:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.490 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331369+0000", 2024-09-18T11:33:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.551e-06 2024-09-18T11:33:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331377+0000", 2024-09-18T11:33:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8769999999999993e-06 2024-09-18T11:33:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331381+0000", 2024-09-18T11:33:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.16e-06 2024-09-18T11:33:16.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.338973+0000", 2024-09-18T11:33:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0075918460000000002 2024-09-18T11:33:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339048+0000", 2024-09-18T11:33:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.5813000000000001e-05 2024-09-18T11:33:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339322+0000", 2024-09-18T11:33:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027399899999999998 2024-09-18T11:33:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:2650 4.0 4:059d8d95:::smithi11939506-32:head [sparse-read 0~2327051] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-18T11:33:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:33:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 122.114299479, 2024-09-18T11:33:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0084392180000000001, 2024-09-18T11:33:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.493 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 2650 2024-09-18T11:33:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:33:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.494 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:33:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331386+0000", 2024-09-18T11:33:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331388+0000", 2024-09-18T11:33:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.3429999999999998e-06 2024-09-18T11:33:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.495 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331394+0000", 2024-09-18T11:33:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.19e-06 2024-09-18T11:33:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.331398+0000", 2024-09-18T11:33:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.495e-06 2024-09-18T11:33:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339362+0000", 2024-09-18T11:33:16.496 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079641799999999995 2024-09-18T11:33:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339460+0000", 2024-09-18T11:33:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8329e-05 2024-09-18T11:33:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:14.339825+0000", 2024-09-18T11:33:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00036510500000000001 2024-09-18T11:33:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.1.13:127 3.2 3:4d3b277a:::3266c7aaa53a7d2c10c3c54d852d6cbbed49fb041bc0df41a3ac83b56e2b0615:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-18T11:33:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:33:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 113.40128454000001, 2024-09-18T11:33:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0093148009999999993, 2024-09-18T11:33:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:33:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044401+0000", 2024-09-18T11:33:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044400+0000", 2024-09-18T11:33:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044442+0000", 2024-09-18T11:33:16.499 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2247999999999999e-05 2024-09-18T11:33:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044453+0000", 2024-09-18T11:33:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0617e-05 2024-09-18T11:33:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044457+0000", 2024-09-18T11:33:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2520000000000001e-06 2024-09-18T11:33:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.500 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044495+0000", 2024-09-18T11:33:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7710000000000003e-05 2024-09-18T11:33:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.044499+0000", 2024-09-18T11:33:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6269999999999999e-06 2024-09-18T11:33:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048239+0000", 2024-09-18T11:33:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037396719999999999 2024-09-18T11:33:16.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048351+0000", 2024-09-18T11:33:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000112342 2024-09-18T11:33:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.048545+0000", 2024-09-18T11:33:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000193881 2024-09-18T11:33:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:16.502 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.052689+0000", 2024-09-18T11:33:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041435780000000002 2024-09-18T11:33:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053654+0000", 2024-09-18T11:33:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096558199999999999 2024-09-18T11:33:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053683+0000", 2024-09-18T11:33:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.8850999999999999e-05 2024-09-18T11:33:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.503 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:23.053716+0000", 2024-09-18T11:33:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2468000000000002e-05 2024-09-18T11:33:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:3209 4.0 4:019b6e09:::smithi11939506-40:head [write 638732~557432] snapc 42=[42,41,3f,3e,3d,3b,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-18T11:33:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:33:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 99.347602500999997, 2024-09-18T11:33:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0091774999999999999, 2024-09-18T11:33:16.504 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 3209 2024-09-18T11:33:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:33:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.505 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098083+0000", 2024-09-18T11:33:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098080+0000", 2024-09-18T11:33:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999971 2024-09-18T11:33:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098676+0000", 2024-09-18T11:33:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00059598099999999998 2024-09-18T11:33:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.506 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098706+0000", 2024-09-18T11:33:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0020000000000001e-05 2024-09-18T11:33:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098716+0000", 2024-09-18T11:33:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0225e-05 2024-09-18T11:33:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098782+0000", 2024-09-18T11:33:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.5530000000000004e-05 2024-09-18T11:33:16.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.098883+0000", 2024-09-18T11:33:16.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000101609 2024-09-18T11:33:16.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 7", 2024-09-18T11:33:16.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.099098+0000", 2024-09-18T11:33:16.508 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000215052 2024-09-18T11:33:16.508 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.508 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.508 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:16.508 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.102511+0000", 2024-09-18T11:33:16.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034129429999999999 2024-09-18T11:33:16.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:16.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107165+0000", 2024-09-18T11:33:16.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0046531480000000002 2024-09-18T11:33:16.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.509 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.509 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107217+0000", 2024-09-18T11:33:16.509 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.2058000000000002e-05 2024-09-18T11:33:16.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.510 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.510 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:31:37.107260+0000", 2024-09-18T11:33:16.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3865000000000001e-05 2024-09-18T11:33:16.510 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.510 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.510 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.510 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:162 3.1 e138/67 3:8c826a1f:::e682cfc1e82c88472b9acf1cd97978bc105aa035f10b685bb4cace95809b52bf:head v 138'176, mlcod=136'174)", 2024-09-18T11:33:16.510 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:33:16.510 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 66.862128756000004, 2024-09-18T11:33:16.510 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.010445024000000001, 2024-09-18T11:33:16.510 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.511 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.511 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:33:16.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.511 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.511 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583557+0000", 2024-09-18T11:33:16.511 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.583556+0000", 2024-09-18T11:33:16.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:16.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584899+0000", 2024-09-18T11:33:16.512 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001343056 2024-09-18T11:33:16.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.512 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.512 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584929+0000", 2024-09-18T11:33:16.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.0133000000000001e-05 2024-09-18T11:33:16.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584937+0000", 2024-09-18T11:33:16.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.6380000000000004e-06 2024-09-18T11:33:16.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.513 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.584990+0000", 2024-09-18T11:33:16.513 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.3186000000000002e-05 2024-09-18T11:33:16.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.513 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585012+0000", 2024-09-18T11:33:16.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1474000000000001e-05 2024-09-18T11:33:16.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.593963+0000", 2024-09-18T11:33:16.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089512859999999993 2024-09-18T11:33:16.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.514 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.514 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594002+0000", 2024-09-18T11:33:16.514 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8859999999999997e-05 2024-09-18T11:33:16.514 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.515 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.515 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.515 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:166 3.1 e138/67 3:a73e635b:::b6ee478f0f661ec7bac3d17c2ca2fdaff71a47570f64844a7ba79bbb2a0e8d9d:head v 138'177, mlcod=136'174)", 2024-09-18T11:33:16.515 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:33:16.515 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 66.860558871999999, 2024-09-18T11:33:16.515 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0089018680000000003, 2024-09-18T11:33:16.515 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.515 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.515 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.515 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:33:16.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585127+0000", 2024-09-18T11:33:16.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.516 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.516 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585126+0000", 2024-09-18T11:33:16.516 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:16.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585211+0000", 2024-09-18T11:33:16.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5432999999999993e-05 2024-09-18T11:33:16.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585220+0000", 2024-09-18T11:33:16.517 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5879999999999998e-06 2024-09-18T11:33:16.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.517 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.517 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585225+0000", 2024-09-18T11:33:16.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8999999999999997e-06 2024-09-18T11:33:16.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585705+0000", 2024-09-18T11:33:16.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00048042799999999997 2024-09-18T11:33:16.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.518 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.518 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.585724+0000", 2024-09-18T11:33:16.518 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8352e-05 2024-09-18T11:33:16.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594008+0000", 2024-09-18T11:33:16.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0082840919999999998 2024-09-18T11:33:16.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.519 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.519 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594028+0000", 2024-09-18T11:33:16.519 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0869e-05 2024-09-18T11:33:16.519 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.519 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.519 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.519 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(osd.1.13:168 3.1 e138/67 3:b64d7bad:::7feb22256bdcec217720fd46184ab43ced8923ed66ec6319c57f078d9f123262:head v 138'178, mlcod=136'174)", 2024-09-18T11:33:16.520 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:33:16.520 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 66.859567139000006, 2024-09-18T11:33:16.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079290650000000008, 2024-09-18T11:33:16.520 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.520 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.520 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.520 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.520 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:33:16.520 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:33:16.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586118+0000", 2024-09-18T11:33:16.521 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:16.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.521 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.521 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586191+0000", 2024-09-18T11:33:16.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3627000000000005e-05 2024-09-18T11:33:16.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.522 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586202+0000", 2024-09-18T11:33:16.522 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0488e-05 2024-09-18T11:33:16.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.522 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586207+0000", 2024-09-18T11:33:16.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1719999999999998e-06 2024-09-18T11:33:16.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586239+0000", 2024-09-18T11:33:16.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2570000000000002e-05 2024-09-18T11:33:16.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.523 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.523 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.586271+0000", 2024-09-18T11:33:16.523 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1341999999999999e-05 2024-09-18T11:33:16.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594031+0000", 2024-09-18T11:33:16.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0077597459999999997 2024-09-18T11:33:16.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.524 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.524 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:09.594047+0000", 2024-09-18T11:33:16.524 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6858e-05 2024-09-18T11:33:16.524 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.524 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.524 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.525 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4267 3.1 3:8d8d13e9:::92886a4340d03039afb2c877fad76e3afe7a80ee04388d55256c386022cd5d63:head [write 0~2203648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:33:16.525 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:33:16.525 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 49.388535159, 2024-09-18T11:33:16.525 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0092856740000000007, 2024-09-18T11:33:16.525 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.525 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.525 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:16.525 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:16.525 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:16.525 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4267 2024-09-18T11:33:16.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.526 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:33:16.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.526 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.526 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057150+0000", 2024-09-18T11:33:16.526 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.526 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.057149+0000", 2024-09-18T11:33:16.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:16.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058175+0000", 2024-09-18T11:33:16.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001026765 2024-09-18T11:33:16.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.527 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.527 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.527 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058190+0000", 2024-09-18T11:33:16.527 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4167e-05 2024-09-18T11:33:16.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058195+0000", 2024-09-18T11:33:16.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.823e-06 2024-09-18T11:33:16.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.528 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058247+0000", 2024-09-18T11:33:16.528 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1304999999999998e-05 2024-09-18T11:33:16.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.528 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.528 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058422+0000", 2024-09-18T11:33:16.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00017527100000000001 2024-09-18T11:33:16.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:16.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.058609+0000", 2024-09-18T11:33:16.529 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018714299999999999 2024-09-18T11:33:16.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.529 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.529 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:16.529 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.062721+0000", 2024-09-18T11:33:16.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0041118869999999998 2024-09-18T11:33:16.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:16.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066291+0000", 2024-09-18T11:33:16.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0035694860000000002 2024-09-18T11:33:16.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.530 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.530 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066328+0000", 2024-09-18T11:33:16.530 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7864000000000002e-05 2024-09-18T11:33:16.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.530 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.531 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.531 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:27.066436+0000", 2024-09-18T11:33:16.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00010762 2024-09-18T11:33:16.531 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.531 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.531 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.531 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.531 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.6.13:90 3.1 3:a224eaa1:::22b064a083114a2756d85a07d2762a393d3e92dd898a5334bc045c9031244d76:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:33:16.531 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:40.057559+0000", 2024-09-18T11:33:16.531 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.388126380000003, 2024-09-18T11:33:16.531 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085024620000000006, 2024-09-18T11:33:16.532 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.532 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.532 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057559+0000", 2024-09-18T11:33:16.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.532 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.532 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.532 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057559+0000", 2024-09-18T11:33:16.532 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057558+0000", 2024-09-18T11:33:16.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:16.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.533 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.533 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057563+0000", 2024-09-18T11:33:16.533 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7990000000000001e-06 2024-09-18T11:33:16.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.533 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057572+0000", 2024-09-18T11:33:16.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0510000000000002e-06 2024-09-18T11:33:16.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.534 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057576+0000", 2024-09-18T11:33:16.534 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2509999999999996e-06 2024-09-18T11:33:16.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.534 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057624+0000", 2024-09-18T11:33:16.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7636000000000003e-05 2024-09-18T11:33:16.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.535 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.535 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057722+0000", 2024-09-18T11:33:16.535 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7676999999999993e-05 2024-09-18T11:33:16.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:16.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.057883+0000", 2024-09-18T11:33:16.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000160922 2024-09-18T11:33:16.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:16.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059927+0000", 2024-09-18T11:33:16.536 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0020447350000000002 2024-09-18T11:33:16.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.536 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:16.536 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.065977+0000", 2024-09-18T11:33:16.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060497679999999996 2024-09-18T11:33:16.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066023+0000", 2024-09-18T11:33:16.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6041000000000002e-05 2024-09-18T11:33:16.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.537 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.537 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066062+0000", 2024-09-18T11:33:16.537 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8469e-05 2024-09-18T11:33:16.537 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.537 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.538 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.538 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.5.13:42 3.1 3:8372521c:::d4b4fff22a71c591f3e871de3fede050f6721df41a8c639a614c0b94ba6762c2:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:33:16.538 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:40.058327+0000", 2024-09-18T11:33:16.538 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.387358706999997, 2024-09-18T11:33:16.538 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079034699999999992, 2024-09-18T11:33:16.538 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.538 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.538 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.538 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.538 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058327+0000", 2024-09-18T11:33:16.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058327+0000", 2024-09-18T11:33:16.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.539 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058325+0000", 2024-09-18T11:33:16.539 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:16.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.539 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058339+0000", 2024-09-18T11:33:16.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3337e-05 2024-09-18T11:33:16.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058357+0000", 2024-09-18T11:33:16.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8615e-05 2024-09-18T11:33:16.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.540 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.540 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058366+0000", 2024-09-18T11:33:16.540 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9460000000000004e-06 2024-09-18T11:33:16.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058658+0000", 2024-09-18T11:33:16.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00029212200000000002 2024-09-18T11:33:16.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.541 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058757+0000", 2024-09-18T11:33:16.541 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.9025999999999997e-05 2024-09-18T11:33:16.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.541 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:16.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058914+0000", 2024-09-18T11:33:16.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000156236 2024-09-18T11:33:16.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:16.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.060326+0000", 2024-09-18T11:33:16.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0014118710000000001 2024-09-18T11:33:16.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.542 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:16.542 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066122+0000", 2024-09-18T11:33:16.542 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0057962029999999998 2024-09-18T11:33:16.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066192+0000", 2024-09-18T11:33:16.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.0533999999999999e-05 2024-09-18T11:33:16.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.543 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.543 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066230+0000", 2024-09-18T11:33:16.543 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7914000000000003e-05 2024-09-18T11:33:16.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.543 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.543 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.5.13:43 3.1 3:b44664f3:::7f31751f088e4f7577561c03b406e8a8bc48ebfdd310b40206061655307aebe3:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:33:16.543 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:40.058386+0000", 2024-09-18T11:33:16.544 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 36.387299679000002, 2024-09-18T11:33:16.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079575740000000002, 2024-09-18T11:33:16.544 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.544 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.544 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058386+0000", 2024-09-18T11:33:16.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.544 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.544 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058386+0000", 2024-09-18T11:33:16.544 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058385+0000", 2024-09-18T11:33:16.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:16.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058392+0000", 2024-09-18T11:33:16.545 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6200000000000001e-06 2024-09-18T11:33:16.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.545 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.545 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058400+0000", 2024-09-18T11:33:16.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7849999999999997e-06 2024-09-18T11:33:16.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.058408+0000", 2024-09-18T11:33:16.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.1770000000000001e-06 2024-09-18T11:33:16.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.546 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.546 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059302+0000", 2024-09-18T11:33:16.546 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000894851 2024-09-18T11:33:16.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059400+0000", 2024-09-18T11:33:16.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.7460000000000005e-05 2024-09-18T11:33:16.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:16.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.059520+0000", 2024-09-18T11:33:16.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000120457 2024-09-18T11:33:16.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.547 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:16.547 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.060333+0000", 2024-09-18T11:33:16.547 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00081255599999999998 2024-09-18T11:33:16.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:16.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066274+0000", 2024-09-18T11:33:16.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0059409700000000003 2024-09-18T11:33:16.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066306+0000", 2024-09-18T11:33:16.548 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2020000000000002e-05 2024-09-18T11:33:16.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.548 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.548 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:40.066343+0000", 2024-09-18T11:33:16.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7438999999999998e-05 2024-09-18T11:33:16.549 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.549 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.549 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.549 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(osd.4.13:145 3.2 3:435dd00a:::773f1f20723e27798deecfe7048ee8e76eeec814e6d6ad791ed305ff28c69be2:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:33:16.549 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:45.321055+0000", 2024-09-18T11:33:16.549 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 31.124630874000001, 2024-09-18T11:33:16.549 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0079066459999999998, 2024-09-18T11:33:16.549 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.549 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.549 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321055+0000", 2024-09-18T11:33:16.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321055+0000", 2024-09-18T11:33:16.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.550 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.550 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321053+0000", 2024-09-18T11:33:16.550 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:16.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321062+0000", 2024-09-18T11:33:16.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0399999999999998e-06 2024-09-18T11:33:16.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.551 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321076+0000", 2024-09-18T11:33:16.551 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3869999999999999e-05 2024-09-18T11:33:16.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.551 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.551 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321083+0000", 2024-09-18T11:33:16.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6159999999999998e-06 2024-09-18T11:33:16.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321156+0000", 2024-09-18T11:33:16.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.3093e-05 2024-09-18T11:33:16.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.552 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.552 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321274+0000", 2024-09-18T11:33:16.552 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000118353 2024-09-18T11:33:16.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.552 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:16.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.321443+0000", 2024-09-18T11:33:16.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016836600000000001 2024-09-18T11:33:16.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:16.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.324281+0000", 2024-09-18T11:33:16.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028381270000000002 2024-09-18T11:33:16.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.553 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.553 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:16.553 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.328785+0000", 2024-09-18T11:33:16.553 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0045048370000000003 2024-09-18T11:33:16.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.328910+0000", 2024-09-18T11:33:16.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012438600000000001 2024-09-18T11:33:16.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.554 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.554 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.554 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:45.328961+0000", 2024-09-18T11:33:16.554 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1375000000000002e-05 2024-09-18T11:33:16.554 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.554 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.554 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.555 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4790 3.2 3:79129b11:::2f02c59c01a730562a65f21445c8af0e4a8aab3b969b8a3abf39e7b1bc84f918:head [write 0~2177536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-18T11:33:16.555 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.041705+0000", 2024-09-18T11:33:16.555 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 23.403980261000001, 2024-09-18T11:33:16.555 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.011619633000000001, 2024-09-18T11:33:16.555 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:16.555 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.555 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:16.555 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:16.555 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:16.555 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4790 2024-09-18T11:33:16.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.556 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:16.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:16.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.041705+0000", 2024-09-18T11:33:16.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.556 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.556 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.556 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:16.556 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.041705+0000", 2024-09-18T11:33:16.556 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:16.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:16.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.041703+0000", 2024-09-18T11:33:16.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:16.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.557 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:16.557 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.042939+0000", 2024-09-18T11:33:16.557 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0012361570000000001 2024-09-18T11:33:16.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.557 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:16.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.042961+0000", 2024-09-18T11:33:16.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1531000000000001e-05 2024-09-18T11:33:16.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.042969+0000", 2024-09-18T11:33:16.558 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.892e-06 2024-09-18T11:33:16.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.558 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.558 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:16.558 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.043013+0000", 2024-09-18T11:33:16.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3921000000000002e-05 2024-09-18T11:33:16.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:16.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.043147+0000", 2024-09-18T11:33:16.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013394599999999999 2024-09-18T11:33:16.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.559 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:16.559 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.043291+0000", 2024-09-18T11:33:16.559 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014399500000000001 2024-09-18T11:33:16.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:16.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.048224+0000", 2024-09-18T11:33:16.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0049331160000000004 2024-09-18T11:33:16.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:16.560 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053239+0000", 2024-09-18T11:33:16.560 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0050155720000000003 2024-09-18T11:33:16.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.560 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:16.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053287+0000", 2024-09-18T11:33:16.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8126e-05 2024-09-18T11:33:16.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:16.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:16.561 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:16.561 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053325+0000", 2024-09-18T11:33:16.561 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.7499999999999997e-05 2024-09-18T11:33:16.561 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.561 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.561 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.561 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:16.561 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:16.561 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:16.563 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T11:33:16.563 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:16.562+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:33:16.664 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:33:16.663+0000 7f6bfebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33081) UID: 0 2024-09-18T11:33:16.673 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:16.673 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:16.673 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:33:16.673 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:16.682 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T11:33:16.767 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:33:16.766+0000 7fd2821f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33083) UID: 0 2024-09-18T11:33:16.821 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:16.821 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:16.821 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:33:16.821 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:33:16.822 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:16.829 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T11:33:16.866 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:16.865+0000 7fb69ebf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33331) UID: 0 2024-09-18T11:33:16.967 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:16.966+0000 7fcb71df8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33086) UID: 0 2024-09-18T11:33:16.968 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:16.968 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:33:16.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:33:16.968 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:33:16.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.968 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1554 3.3 3:d31352ae:::287d3322ad9acbb2fed5e144df2b9442fe9984d13cfcecfb068578b1c5bddd03:head [write 0~1404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T11:33:16.968 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:33:16.969 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 162.549665275, 2024-09-18T11:33:16.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015251242999999999, 2024-09-18T11:33:16.969 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:16.969 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.969 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:16.969 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:16.969 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:16.969 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1554 2024-09-18T11:33:16.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.970 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:16.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:16.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:33:16.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:16.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:16.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416087+0000", 2024-09-18T11:33:16.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:16.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:16.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.416085+0000", 2024-09-18T11:33:16.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:16.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:16.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419280+0000", 2024-09-18T11:33:16.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031946050000000001 2024-09-18T11:33:16.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:16.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419305+0000", 2024-09-18T11:33:16.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4692e-05 2024-09-18T11:33:16.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419315+0000", 2024-09-18T11:33:16.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0917999999999999e-05 2024-09-18T11:33:16.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:16.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419398+0000", 2024-09-18T11:33:16.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2616999999999999e-05 2024-09-18T11:33:16.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:16.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419527+0000", 2024-09-18T11:33:16.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012912199999999999 2024-09-18T11:33:16.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:16.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.419631+0000", 2024-09-18T11:33:16.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010373 2024-09-18T11:33:16.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:16.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.421899+0000", 2024-09-18T11:33:16.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022677460000000002 2024-09-18T11:33:16.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:16.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431272+0000", 2024-09-18T11:33:16.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.009373167 2024-09-18T11:33:16.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:16.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431309+0000", 2024-09-18T11:33:16.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7162000000000001e-05 2024-09-18T11:33:16.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.977 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:16.977 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:34.431338+0000", 2024-09-18T11:33:16.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9074999999999999e-05 2024-09-18T11:33:16.977 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:16.977 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:16.977 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:16.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.977 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.6.0:174 4.5 e78/22 4:aa575018:::smithi11939506-11:17 v 78'137, mlcod=77'132)", 2024-09-18T11:33:16.977 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:33:16.977 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 135.57943586900001, 2024-09-18T11:33:16.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011566945, 2024-09-18T11:33:16.978 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:16.978 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.978 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:16.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:16.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:33:16.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:16.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:16.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:33:16.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:16.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:16.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386316+0000", 2024-09-18T11:33:16.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:16.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:16.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386322+0000", 2024-09-18T11:33:16.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2790000000000004e-06 2024-09-18T11:33:16.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:16.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386334+0000", 2024-09-18T11:33:16.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.199e-05 2024-09-18T11:33:16.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386339+0000", 2024-09-18T11:33:16.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4269999999999999e-06 2024-09-18T11:33:16.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:16.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387668+0000", 2024-09-18T11:33:16.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001328409 2024-09-18T11:33:16.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:16.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.387694+0000", 2024-09-18T11:33:16.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6514000000000001e-05 2024-09-18T11:33:16.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:16.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397849+0000", 2024-09-18T11:33:16.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0101552 2024-09-18T11:33:16.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:16.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.397883+0000", 2024-09-18T11:33:16.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3846000000000001e-05 2024-09-18T11:33:16.981 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:16.982 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:16.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:16.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.982 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2503 3.1 3:8d1352af:::e49b021157e61768915a03c1e9ab0db1da5ae05ef12a6cd224ee50d26fdfb2c6:head [write 0~2110976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-18T11:33:16.982 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:33:16.982 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 127.99332326299999, 2024-09-18T11:33:16.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01367556, 2024-09-18T11:33:16.982 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:16.982 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.982 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:16.982 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:16.983 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:16.983 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2503 2024-09-18T11:33:16.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.983 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:16.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:16.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:33:16.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:16.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:16.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972429+0000", 2024-09-18T11:33:16.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:16.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:16.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.972427+0000", 2024-09-18T11:33:16.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:16.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:16.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976767+0000", 2024-09-18T11:33:16.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043397519999999997 2024-09-18T11:33:16.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:16.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976791+0000", 2024-09-18T11:33:16.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4294000000000002e-05 2024-09-18T11:33:16.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976800+0000", 2024-09-18T11:33:16.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4079999999999998e-06 2024-09-18T11:33:16.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:16.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976876+0000", 2024-09-18T11:33:16.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6830999999999997e-05 2024-09-18T11:33:16.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:16.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.976990+0000", 2024-09-18T11:33:16.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000113904 2024-09-18T11:33:16.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:16.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.977094+0000", 2024-09-18T11:33:16.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103303 2024-09-18T11:33:16.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:16.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.979868+0000", 2024-09-18T11:33:16.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027744499999999999 2024-09-18T11:33:16.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:16.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986044+0000", 2024-09-18T11:33:16.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0061763219999999997 2024-09-18T11:33:16.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:16.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986077+0000", 2024-09-18T11:33:16.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2064000000000003e-05 2024-09-18T11:33:16.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.987 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:16.987 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:08.986104+0000", 2024-09-18T11:33:16.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7908000000000001e-05 2024-09-18T11:33:16.988 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:16.988 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:16.988 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:16.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.988 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2956 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~2100908,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:33:16.988 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:33:16.988 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.938527013, 2024-09-18T11:33:16.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011863831, 2024-09-18T11:33:16.988 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:16.988 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:16.989 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:16.989 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:16.989 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:16.989 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2956 2024-09-18T11:33:16.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.989 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:16.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:16.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:33:16.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:16.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:16.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:33:16.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:16.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:16.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027225+0000", 2024-09-18T11:33:16.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:16.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:16.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027228+0000", 2024-09-18T11:33:16.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6789999999999998e-06 2024-09-18T11:33:16.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:16.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027235+0000", 2024-09-18T11:33:16.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7000000000000002e-06 2024-09-18T11:33:16.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027239+0000", 2024-09-18T11:33:16.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.535e-06 2024-09-18T11:33:16.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:16.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027831+0000", 2024-09-18T11:33:16.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059244699999999996 2024-09-18T11:33:16.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:16.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027879+0000", 2024-09-18T11:33:16.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7766000000000003e-05 2024-09-18T11:33:16.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:16.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038098+0000", 2024-09-18T11:33:16.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010219137 2024-09-18T11:33:16.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:16.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.038207+0000", 2024-09-18T11:33:16.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010951999999999999 2024-09-18T11:33:16.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:16.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039089+0000", 2024-09-18T11:33:16.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00088151699999999996 2024-09-18T11:33:16.993 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:16.993 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:16.993 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:16.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.993 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2957 4.d 4:b9bb642b:::smithi11939506-27:head [sparse-read 0~1480638] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-18T11:33:16.993 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:33:16.993 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 112.938509229, 2024-09-18T11:33:16.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012041979, 2024-09-18T11:33:16.994 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:16.994 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:16.994 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:16.994 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:16.994 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:16.994 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2957 2024-09-18T11:33:16.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.994 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:16.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:16.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:33:16.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:16.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:16.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:33:16.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:16.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:16.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027243+0000", 2024-09-18T11:33:16.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:16.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:16.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027245+0000", 2024-09-18T11:33:16.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4119999999999999e-06 2024-09-18T11:33:16.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:16.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027251+0000", 2024-09-18T11:33:16.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0090000000000004e-06 2024-09-18T11:33:16.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:16.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027255+0000", 2024-09-18T11:33:16.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7390000000000001e-06 2024-09-18T11:33:16.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:16.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027895+0000", 2024-09-18T11:33:16.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064076400000000005 2024-09-18T11:33:16.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:16.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.027930+0000", 2024-09-18T11:33:16.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4334e-05 2024-09-18T11:33:16.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:16.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039111+0000", 2024-09-18T11:33:16.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181455999999999 2024-09-18T11:33:16.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:16.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039156+0000", 2024-09-18T11:33:16.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5256000000000002e-05 2024-09-18T11:33:16.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:16.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:24.039285+0000", 2024-09-18T11:33:16.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012839200000000001 2024-09-18T11:33:16.998 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:16.998 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:16.998 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:16.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.998 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:978 4.b e132/67 4:d8d14c58:::smithi11939506-49:30 v 132'417, mlcod=128'414)", 2024-09-18T11:33:16.998 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:33:16.998 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 73.940477947000005, 2024-09-18T11:33:16.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011942183, 2024-09-18T11:33:16.999 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:16.999 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:16.999 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:16.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:16.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:33:16.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:16.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:16.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:33:16.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:16.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:16.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:16.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025274+0000", 2024-09-18T11:33:17.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:33:17.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025278+0000", 2024-09-18T11:33:17.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9060000000000004e-06 2024-09-18T11:33:17.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025288+0000", 2024-09-18T11:33:17.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.028e-05 2024-09-18T11:33:17.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025293+0000", 2024-09-18T11:33:17.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2700000000000004e-06 2024-09-18T11:33:17.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.001 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025346+0000", 2024-09-18T11:33:17.001 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2862999999999999e-05 2024-09-18T11:33:17.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.001 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.025370+0000", 2024-09-18T11:33:17.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4005000000000001e-05 2024-09-18T11:33:17.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037159+0000", 2024-09-18T11:33:17.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011788343 2024-09-18T11:33:17.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.002 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.002 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037216+0000", 2024-09-18T11:33:17.002 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7695000000000002e-05 2024-09-18T11:33:17.002 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.003 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.003 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.003 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:3930 4.b e132/67 4:d3ae8215:::smithi11939506-41:head v 132'421, mlcod=128'414)", 2024-09-18T11:33:17.003 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:33:17.003 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 73.936929699000004, 2024-09-18T11:33:17.003 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015093554, 2024-09-18T11:33:17.003 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.003 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.003 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.003 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:33:17.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.004 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028822+0000", 2024-09-18T11:33:17.004 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.004 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028821+0000", 2024-09-18T11:33:17.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:17.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029805+0000", 2024-09-18T11:33:17.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00098410899999999994 2024-09-18T11:33:17.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.005 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.005 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029817+0000", 2024-09-18T11:33:17.005 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2058e-05 2024-09-18T11:33:17.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.029823+0000", 2024-09-18T11:33:17.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6450000000000002e-06 2024-09-18T11:33:17.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032330+0000", 2024-09-18T11:33:17.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025067710000000001 2024-09-18T11:33:17.006 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.006 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.006 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.032349+0000", 2024-09-18T11:33:17.006 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9729e-05 2024-09-18T11:33:17.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043878+0000", 2024-09-18T11:33:17.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011528514 2024-09-18T11:33:17.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.007 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.007 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.043916+0000", 2024-09-18T11:33:17.007 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8049000000000001e-05 2024-09-18T11:33:17.007 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.007 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.007 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.008 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:162 3.1 3:8c826a1f:::e682cfc1e82c88472b9acf1cd97978bc105aa035f10b685bb4cace95809b52bf:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:17.008 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:33:17.008 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 67.384807541000001, 2024-09-18T11:33:17.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013234571000000001, 2024-09-18T11:33:17.008 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.008 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.008 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.008 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.008 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.008 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:33:17.008 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580945+0000", 2024-09-18T11:33:17.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.009 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.580943+0000", 2024-09-18T11:33:17.009 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:17.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.009 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582556+0000", 2024-09-18T11:33:17.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016132970000000001 2024-09-18T11:33:17.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582572+0000", 2024-09-18T11:33:17.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5846999999999999e-05 2024-09-18T11:33:17.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.010 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.010 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582578+0000", 2024-09-18T11:33:17.010 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4810000000000003e-06 2024-09-18T11:33:17.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582632+0000", 2024-09-18T11:33:17.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4156000000000001e-05 2024-09-18T11:33:17.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582819+0000", 2024-09-18T11:33:17.011 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000186431 2024-09-18T11:33:17.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.011 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.011 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:17.011 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583118+0000", 2024-09-18T11:33:17.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029880100000000001 2024-09-18T11:33:17.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590676+0000", 2024-09-18T11:33:17.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075588670000000004 2024-09-18T11:33:17.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.012 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.012 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594111+0000", 2024-09-18T11:33:17.012 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0034348550000000001 2024-09-18T11:33:17.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594142+0000", 2024-09-18T11:33:17.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.099e-05 2024-09-18T11:33:17.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.013 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.013 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594179+0000", 2024-09-18T11:33:17.013 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6831e-05 2024-09-18T11:33:17.013 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.013 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.013 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.014 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3974 4.d 4:b5a463bf:::smithi11939506-20:head [write 665643~697786,stat] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:17.014 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:33:17.014 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 67.383720858000004, 2024-09-18T11:33:17.014 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016621174999999998, 2024-09-18T11:33:17.014 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.014 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.014 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.014 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.014 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.014 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3974 2024-09-18T11:33:17.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.014 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.014 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:33:17.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582031+0000", 2024-09-18T11:33:17.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.015 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.015 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582030+0000", 2024-09-18T11:33:17.015 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:17.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585185+0000", 2024-09-18T11:33:17.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003155644 2024-09-18T11:33:17.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585208+0000", 2024-09-18T11:33:17.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3013000000000001e-05 2024-09-18T11:33:17.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.016 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.016 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585215+0000", 2024-09-18T11:33:17.016 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7569999999999998e-06 2024-09-18T11:33:17.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585278+0000", 2024-09-18T11:33:17.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2699000000000004e-05 2024-09-18T11:33:17.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.017 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585374+0000", 2024-09-18T11:33:17.017 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.6062000000000003e-05 2024-09-18T11:33:17.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.017 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 2", 2024-09-18T11:33:17.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585577+0000", 2024-09-18T11:33:17.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000202926 2024-09-18T11:33:17.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596544+0000", 2024-09-18T11:33:17.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010966848 2024-09-18T11:33:17.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.018 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.018 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598594+0000", 2024-09-18T11:33:17.018 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020502049999999998 2024-09-18T11:33:17.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598625+0000", 2024-09-18T11:33:17.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1412999999999999e-05 2024-09-18T11:33:17.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.019 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.019 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.598652+0000", 2024-09-18T11:33:17.019 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7166999999999999e-05 2024-09-18T11:33:17.019 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.019 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.019 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.020 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:166 3.1 3:a73e635b:::b6ee478f0f661ec7bac3d17c2ca2fdaff71a47570f64844a7ba79bbb2a0e8d9d:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:17.020 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:33:17.020 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 67.383161491999999, 2024-09-18T11:33:17.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011711037000000001, 2024-09-18T11:33:17.020 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.020 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.020 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.020 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.020 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:33:17.020 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582591+0000", 2024-09-18T11:33:17.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582590+0000", 2024-09-18T11:33:17.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.021 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.021 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582673+0000", 2024-09-18T11:33:17.021 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2861000000000005e-05 2024-09-18T11:33:17.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.022 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.022 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582683+0000", 2024-09-18T11:33:17.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0081999999999999e-05 2024-09-18T11:33:17.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.022 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.022 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582688+0000", 2024-09-18T11:33:17.022 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6870000000000002e-06 2024-09-18T11:33:17.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.022 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584266+0000", 2024-09-18T11:33:17.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001578423 2024-09-18T11:33:17.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.023 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.023 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584407+0000", 2024-09-18T11:33:17.023 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014144599999999999 2024-09-18T11:33:17.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:17.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584599+0000", 2024-09-18T11:33:17.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000191881 2024-09-18T11:33:17.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.024 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.024 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590680+0000", 2024-09-18T11:33:17.024 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060801479999999996 2024-09-18T11:33:17.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594224+0000", 2024-09-18T11:33:17.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0035441700000000001 2024-09-18T11:33:17.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594269+0000", 2024-09-18T11:33:17.025 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5297999999999999e-05 2024-09-18T11:33:17.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.025 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.025 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594302+0000", 2024-09-18T11:33:17.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2703000000000002e-05 2024-09-18T11:33:17.026 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.026 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.026 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.026 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:168 3.1 3:b64d7bad:::7feb22256bdcec217720fd46184ab43ced8923ed66ec6319c57f078d9f123262:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:17.026 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:33:17.026 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 67.382934039999995, 2024-09-18T11:33:17.026 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011589809, 2024-09-18T11:33:17.026 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.026 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.026 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.027 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:33:17.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.027 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582818+0000", 2024-09-18T11:33:17.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.027 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.027 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582817+0000", 2024-09-18T11:33:17.027 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582903+0000", 2024-09-18T11:33:17.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5915999999999995e-05 2024-09-18T11:33:17.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582913+0000", 2024-09-18T11:33:17.028 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1409999999999994e-06 2024-09-18T11:33:17.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.028 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.028 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.582917+0000", 2024-09-18T11:33:17.029 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6419999999999998e-06 2024-09-18T11:33:17.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.029 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.032 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585540+0000", 2024-09-18T11:33:17.032 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026228940000000002 2024-09-18T11:33:17.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585721+0000", 2024-09-18T11:33:17.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018086299999999999 2024-09-18T11:33:17.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.033 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:17.033 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.585918+0000", 2024-09-18T11:33:17.033 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000196758 2024-09-18T11:33:17.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590683+0000", 2024-09-18T11:33:17.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0047657020000000001 2024-09-18T11:33:17.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594354+0000", 2024-09-18T11:33:17.034 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003671068 2024-09-18T11:33:17.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.034 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.034 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594386+0000", 2024-09-18T11:33:17.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1426e-05 2024-09-18T11:33:17.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.035 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.035 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.594408+0000", 2024-09-18T11:33:17.035 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1987000000000001e-05 2024-09-18T11:33:17.035 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.035 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.035 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.035 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(osd.1.13:175 3.1 3:b5e86b88:::e55cf3d5cf782ed327a67e746b287b76d05f74f58a4f3726e399a564440ceaf6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:17.035 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:33:17.035 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 67.382085098999994, 2024-09-18T11:33:17.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013709627, 2024-09-18T11:33:17.036 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.036 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.036 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.036 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:33:17.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.036 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.036 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583667+0000", 2024-09-18T11:33:17.036 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.583666+0000", 2024-09-18T11:33:17.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.037 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.037 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584041+0000", 2024-09-18T11:33:17.037 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00037511499999999998 2024-09-18T11:33:17.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584055+0000", 2024-09-18T11:33:17.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3295999999999999e-05 2024-09-18T11:33:17.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.038 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.038 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.584061+0000", 2024-09-18T11:33:17.038 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1630000000000001e-06 2024-09-18T11:33:17.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.589980+0000", 2024-09-18T11:33:17.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005918998 2024-09-18T11:33:17.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.039 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590117+0000", 2024-09-18T11:33:17.039 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013715400000000001 2024-09-18T11:33:17.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.039 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:17.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.590270+0000", 2024-09-18T11:33:17.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00015245600000000001 2024-09-18T11:33:17.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.596513+0000", 2024-09-18T11:33:17.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0062438729999999996 2024-09-18T11:33:17.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.040 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.040 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597328+0000", 2024-09-18T11:33:17.040 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081489599999999996 2024-09-18T11:33:17.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597358+0000", 2024-09-18T11:33:17.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9291000000000001e-05 2024-09-18T11:33:17.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.041 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.041 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.597377+0000", 2024-09-18T11:33:17.041 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:33:17.041 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.041 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.041 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.042 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4061 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:17.042 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:33:17.042 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 64.668906410000005, 2024-09-18T11:33:17.042 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013340967, 2024-09-18T11:33:17.042 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.042 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.042 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.042 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.042 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.042 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4061 2024-09-18T11:33:17.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.042 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.043 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:33:17.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.043 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296846+0000", 2024-09-18T11:33:17.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.043 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.043 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296844+0000", 2024-09-18T11:33:17.043 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:17.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.044 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.044 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296854+0000", 2024-09-18T11:33:17.044 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0037000000000001e-05 2024-09-18T11:33:17.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.044 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.044 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296873+0000", 2024-09-18T11:33:17.044 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9063e-05 2024-09-18T11:33:17.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.045 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.045 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296881+0000", 2024-09-18T11:33:17.045 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3790000000000006e-06 2024-09-18T11:33:17.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.045 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.045 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296975+0000", 2024-09-18T11:33:17.045 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3536999999999996e-05 2024-09-18T11:33:17.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.046 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:33:17.046 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296982+0000", 2024-09-18T11:33:17.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6399999999999997e-06 2024-09-18T11:33:17.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.046 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.046 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301933+0000", 2024-09-18T11:33:17.046 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049507190000000001 2024-09-18T11:33:17.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.046 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.046 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301984+0000", 2024-09-18T11:33:17.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1439e-05 2024-09-18T11:33:17.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.047 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.047 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:33:17.047 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082022050000000006 2024-09-18T11:33:17.047 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.047 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.047 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.047 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4062 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:17.047 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:33:17.048 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 64.668857854999999, 2024-09-18T11:33:17.048 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016297147000000001, 2024-09-18T11:33:17.048 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.048 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.048 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.048 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.048 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.048 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4062 2024-09-18T11:33:17.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.048 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.048 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.048 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:33:17.049 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.049 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.049 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:33:17.049 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.049 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.049 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296894+0000", 2024-09-18T11:33:17.049 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.050 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.050 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296898+0000", 2024-09-18T11:33:17.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9419999999999997e-06 2024-09-18T11:33:17.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.050 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.050 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296906+0000", 2024-09-18T11:33:17.050 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3520000000000007e-06 2024-09-18T11:33:17.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.051 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.051 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296910+0000", 2024-09-18T11:33:17.051 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4469999999999999e-06 2024-09-18T11:33:17.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.051 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.051 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297032+0000", 2024-09-18T11:33:17.051 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012168900000000001 2024-09-18T11:33:17.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.051 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:17.051 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297038+0000", 2024-09-18T11:33:17.051 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9320000000000001e-06 2024-09-18T11:33:17.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.052 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310221+0000", 2024-09-18T11:33:17.052 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013182887000000001 2024-09-18T11:33:17.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.052 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310323+0000", 2024-09-18T11:33:17.052 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010170000000000001 2024-09-18T11:33:17.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.052 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.052 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313191+0000", 2024-09-18T11:33:17.053 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002868808 2024-09-18T11:33:17.053 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.053 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.053 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.053 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4063 4.d 4:b9bb642b:::smithi11939506-27:head [read 0~3512696,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:17.053 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:33:17.053 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 64.668836963000004, 2024-09-18T11:33:17.053 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017648712, 2024-09-18T11:33:17.053 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.053 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.054 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.054 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.054 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.054 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4063 2024-09-18T11:33:17.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.054 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.054 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.054 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:33:17.054 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.055 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:33:17.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.055 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296915+0000", 2024-09-18T11:33:17.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.055 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.055 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296918+0000", 2024-09-18T11:33:17.055 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8100000000000002e-06 2024-09-18T11:33:17.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.056 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.056 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296924+0000", 2024-09-18T11:33:17.056 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2519999999999996e-06 2024-09-18T11:33:17.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.056 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.056 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296928+0000", 2024-09-18T11:33:17.056 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0500000000000002e-06 2024-09-18T11:33:17.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.056 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.057 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297062+0000", 2024-09-18T11:33:17.057 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013425500000000001 2024-09-18T11:33:17.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.057 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:17.057 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297069+0000", 2024-09-18T11:33:17.057 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5239999999999997e-06 2024-09-18T11:33:17.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.057 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.057 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313424+0000", 2024-09-18T11:33:17.057 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016355277000000001 2024-09-18T11:33:17.058 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.058 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.058 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.313514+0000", 2024-09-18T11:33:17.058 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.0581999999999994e-05 2024-09-18T11:33:17.058 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.058 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.058 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.314564+0000", 2024-09-18T11:33:17.059 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001049384 2024-09-18T11:33:17.059 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.059 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.059 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.059 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.059 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.059 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4065 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:17.059 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:33:17.059 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 64.668791522000006, 2024-09-18T11:33:17.059 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013225997, 2024-09-18T11:33:17.059 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.059 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.059 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.060 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.060 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.060 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4065 2024-09-18T11:33:17.060 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.060 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.060 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.060 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.060 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:33:17.060 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.060 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.060 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.060 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.061 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296961+0000", 2024-09-18T11:33:17.061 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.061 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.061 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.061 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.061 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296960+0000", 2024-09-18T11:33:17.061 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.062 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.062 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.062 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296963+0000", 2024-09-18T11:33:17.062 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2270000000000002e-06 2024-09-18T11:33:17.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.062 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.062 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.062 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296970+0000", 2024-09-18T11:33:17.062 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6220000000000003e-06 2024-09-18T11:33:17.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.062 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.062 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.062 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296973+0000", 2024-09-18T11:33:17.062 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3730000000000001e-06 2024-09-18T11:33:17.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.063 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.063 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.063 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297097+0000", 2024-09-18T11:33:17.063 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012352399999999999 2024-09-18T11:33:17.063 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.063 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.063 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:17.063 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297103+0000", 2024-09-18T11:33:17.063 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7119999999999997e-06 2024-09-18T11:33:17.063 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.063 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.063 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.063 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307831+0000", 2024-09-18T11:33:17.063 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010728794 2024-09-18T11:33:17.064 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.064 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.064 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.307924+0000", 2024-09-18T11:33:17.064 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2099e-05 2024-09-18T11:33:17.064 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.064 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.064 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310187+0000", 2024-09-18T11:33:17.064 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022630469999999998 2024-09-18T11:33:17.064 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.064 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.064 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.064 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.065 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4066 4.1 4:87078dfe:::smithi11939506-14:head [read 0~3916064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:17.065 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:33:17.065 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 64.668774447000004, 2024-09-18T11:33:17.065 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015656389999999999, 2024-09-18T11:33:17.065 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.065 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.065 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.065 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.065 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.065 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4066 2024-09-18T11:33:17.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.065 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.065 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.065 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.066 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:33:17.066 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.066 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.066 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296978+0000", 2024-09-18T11:33:17.066 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.066 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.066 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296977+0000", 2024-09-18T11:33:17.066 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.066 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.067 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296980+0000", 2024-09-18T11:33:17.067 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8830000000000002e-06 2024-09-18T11:33:17.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.067 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.067 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296985+0000", 2024-09-18T11:33:17.067 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8840000000000002e-06 2024-09-18T11:33:17.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.067 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.067 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.296988+0000", 2024-09-18T11:33:17.067 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0419999999999998e-06 2024-09-18T11:33:17.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.067 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.068 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297120+0000", 2024-09-18T11:33:17.068 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131693 2024-09-18T11:33:17.068 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.068 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.068 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:17.068 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.297125+0000", 2024-09-18T11:33:17.068 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1100000000000002e-06 2024-09-18T11:33:17.068 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.068 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.068 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.068 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310212+0000", 2024-09-18T11:33:17.068 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013087395999999999 2024-09-18T11:33:17.068 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.069 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.069 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.069 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310292+0000", 2024-09-18T11:33:17.069 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9980000000000003e-05 2024-09-18T11:33:17.069 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.069 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.069 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.069 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312634+0000", 2024-09-18T11:33:17.069 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002341756 2024-09-18T11:33:17.069 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.069 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.070 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.070 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4068 3.1 3:87e79ed0:::3526641c07d3efa989be9684355570f39c331008b8a02c9fa22b38a94b6c6b9f:head [write 0~1015808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-18T11:33:17.070 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:33:17.070 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 64.664998229999995, 2024-09-18T11:33:17.070 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016392648999999999, 2024-09-18T11:33:17.070 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.070 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.070 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.070 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.070 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.070 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4068 2024-09-18T11:33:17.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.071 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.071 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.071 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.071 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:33:17.071 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.071 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.071 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.071 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300754+0000", 2024-09-18T11:33:17.071 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.071 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.071 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.072 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.300753+0000", 2024-09-18T11:33:17.072 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:17.072 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.072 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.072 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.072 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301922+0000", 2024-09-18T11:33:17.072 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001169432 2024-09-18T11:33:17.072 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.072 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.072 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.072 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301936+0000", 2024-09-18T11:33:17.072 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3574000000000001e-05 2024-09-18T11:33:17.072 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.072 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.073 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.073 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.301941+0000", 2024-09-18T11:33:17.073 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6069999999999998e-06 2024-09-18T11:33:17.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.073 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.073 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.073 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310234+0000", 2024-09-18T11:33:17.073 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0082924820000000003 2024-09-18T11:33:17.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.073 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.074 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.074 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310337+0000", 2024-09-18T11:33:17.074 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102673 2024-09-18T11:33:17.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.074 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:17.074 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.310469+0000", 2024-09-18T11:33:17.074 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000131982 2024-09-18T11:33:17.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.074 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.074 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.312844+0000", 2024-09-18T11:33:17.074 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0023758680000000002 2024-09-18T11:33:17.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.075 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.075 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317092+0000", 2024-09-18T11:33:17.075 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0042477549999999998 2024-09-18T11:33:17.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.075 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.075 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317124+0000", 2024-09-18T11:33:17.075 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1968999999999998e-05 2024-09-18T11:33:17.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.075 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.075 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:12.317147+0000", 2024-09-18T11:33:17.076 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2339000000000001e-05 2024-09-18T11:33:17.076 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.076 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.076 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.076 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4671 4.1 4:87078dfe:::smithi11939506-14:head [rollback 6f] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:33:17.076 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:33:17.076 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.647035854999999, 2024-09-18T11:33:17.076 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013540905000000001, 2024-09-18T11:33:17.076 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.077 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.077 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.077 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.077 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.077 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4671 2024-09-18T11:33:17.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.077 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.077 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.077 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:33:17.077 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.077 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:33:17.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318716+0000", 2024-09-18T11:33:17.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.078 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.078 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318718+0000", 2024-09-18T11:33:17.078 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3300000000000001e-06 2024-09-18T11:33:17.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.079 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318723+0000", 2024-09-18T11:33:17.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1590000000000001e-06 2024-09-18T11:33:17.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.079 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.079 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318727+0000", 2024-09-18T11:33:17.079 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3529999999999999e-06 2024-09-18T11:33:17.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.080 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.080 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.320735+0000", 2024-09-18T11:33:17.080 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020079730000000001 2024-09-18T11:33:17.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.081 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.081 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.320843+0000", 2024-09-18T11:33:17.081 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000108488 2024-09-18T11:33:17.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.081 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:17.082 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.329522+0000", 2024-09-18T11:33:17.082 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086787979999999997 2024-09-18T11:33:17.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.082 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.082 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331769+0000", 2024-09-18T11:33:17.082 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022471269999999998 2024-09-18T11:33:17.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.082 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.082 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332189+0000", 2024-09-18T11:33:17.082 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00041940400000000001 2024-09-18T11:33:17.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.083 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.083 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332228+0000", 2024-09-18T11:33:17.083 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9298000000000002e-05 2024-09-18T11:33:17.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.083 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.083 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332257+0000", 2024-09-18T11:33:17.083 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.934e-05 2024-09-18T11:33:17.083 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.083 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.083 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.084 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4672 4.1 4:87078dfe:::smithi11939506-14:head [append 0~0] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:33:17.084 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.318763+0000", 2024-09-18T11:33:17.084 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 31.646989452, 2024-09-18T11:33:17.084 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.016099992, 2024-09-18T11:33:17.084 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.084 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.084 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.084 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.084 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.084 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4672 2024-09-18T11:33:17.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.085 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318763+0000", 2024-09-18T11:33:17.085 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.085 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.085 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318763+0000", 2024-09-18T11:33:17.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.086 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.086 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318762+0000", 2024-09-18T11:33:17.086 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.087 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.087 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318765+0000", 2024-09-18T11:33:17.087 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.43e-06 2024-09-18T11:33:17.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.087 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.087 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318770+0000", 2024-09-18T11:33:17.088 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1479999999999997e-06 2024-09-18T11:33:17.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.088 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.088 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318773+0000", 2024-09-18T11:33:17.088 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.18e-06 2024-09-18T11:33:17.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.088 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.088 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.321043+0000", 2024-09-18T11:33:17.088 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022696819999999999 2024-09-18T11:33:17.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.089 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for blocked object", 2024-09-18T11:33:17.089 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.321104+0000", 2024-09-18T11:33:17.089 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1050999999999995e-05 2024-09-18T11:33:17.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.089 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.089 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331070+0000", 2024-09-18T11:33:17.089 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0099659610000000006 2024-09-18T11:33:17.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331176+0000", 2024-09-18T11:33:17.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010582 2024-09-18T11:33:17.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:17.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.331377+0000", 2024-09-18T11:33:17.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000201208 2024-09-18T11:33:17.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.090 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.090 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.332278+0000", 2024-09-18T11:33:17.090 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00090079999999999999 2024-09-18T11:33:17.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.334798+0000", 2024-09-18T11:33:17.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025200190000000001 2024-09-18T11:33:17.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.091 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.091 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.334833+0000", 2024-09-18T11:33:17.091 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5678e-05 2024-09-18T11:33:17.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.092 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.092 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.334863+0000", 2024-09-18T11:33:17.092 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9374e-05 2024-09-18T11:33:17.092 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.092 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.092 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.092 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.092 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.092 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:17.094 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T11:33:17.094 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:17.093+0000 7f69869f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33335) UID: 0 2024-09-18T11:33:17.107 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:17.106+0000 7fb6950ff640 -1 osd.2 197 handle_osd_map injecting map crc failure 2024-09-18T11:33:17.109 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is unset 2024-09-18T11:33:17.109 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'unique_pool_0' removed 2024-09-18T11:33:17.131 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-18T11:33:17.141 INFO:tasks.thrashosds:joining thrashosds 2024-09-18T11:33:17.239 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:17.240 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:17.240 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:33:17.240 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:17.248 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T11:33:17.346 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:17.346 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:17.346 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:33:17.346 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:33:17.346 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:17.354 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T11:33:17.455 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:17.455 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:33:17.455 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:33:17.455 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:33:17.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.455 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1788 4.8 4:1898b15d:::smithi11939506-25:head [write 453822~797061] snapc 1c=[1c,1b,19,18,17,16,14,13,10] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T11:33:17.455 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:33:17.456 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 154.537007206, 2024-09-18T11:33:17.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094305159999999999, 2024-09-18T11:33:17.456 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.456 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.456 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.456 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.456 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.456 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1788 2024-09-18T11:33:17.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.456 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.456 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.456 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.456 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:33:17.456 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915813+0000", 2024-09-18T11:33:17.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.457 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.457 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.915811+0000", 2024-09-18T11:33:17.457 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:17.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916608+0000", 2024-09-18T11:33:17.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00079635000000000003 2024-09-18T11:33:17.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916624+0000", 2024-09-18T11:33:17.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6735000000000002e-05 2024-09-18T11:33:17.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.458 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.458 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916630+0000", 2024-09-18T11:33:17.458 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5709999999999997e-06 2024-09-18T11:33:17.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916674+0000", 2024-09-18T11:33:17.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4540999999999999e-05 2024-09-18T11:33:17.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.459 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916751+0000", 2024-09-18T11:33:17.459 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6695999999999999e-05 2024-09-18T11:33:17.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.459 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:33:17.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.916872+0000", 2024-09-18T11:33:17.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000120648 2024-09-18T11:33:17.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.919076+0000", 2024-09-18T11:33:17.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022046829999999998 2024-09-18T11:33:17.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.460 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.460 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925151+0000", 2024-09-18T11:33:17.460 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060740239999999999 2024-09-18T11:33:17.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925198+0000", 2024-09-18T11:33:17.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7896999999999999e-05 2024-09-18T11:33:17.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.461 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.461 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:42.925244+0000", 2024-09-18T11:33:17.461 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5398999999999997e-05 2024-09-18T11:33:17.461 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.461 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.461 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.462 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3963 4.3 4:c70c3162:::smithi11939506-35:head [tier-promote] snapc 5a=[59,57,55,54,51,4f,4e,4c,4b,49,47,45,43,42,3e,3d,3b] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-18T11:33:17.462 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:33:17.462 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 68.076997285000004, 2024-09-18T11:33:17.462 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010675591999999999, 2024-09-18T11:33:17.462 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.462 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.462 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.462 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.462 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.462 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3963 2024-09-18T11:33:17.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.462 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.462 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:33:17.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375823+0000", 2024-09-18T11:33:17.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.463 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375821+0000", 2024-09-18T11:33:17.463 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:17.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.463 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375829+0000", 2024-09-18T11:33:17.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7989999999999994e-06 2024-09-18T11:33:17.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375843+0000", 2024-09-18T11:33:17.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.377e-05 2024-09-18T11:33:17.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.464 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.464 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375849+0000", 2024-09-18T11:33:17.464 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1419999999999998e-06 2024-09-18T11:33:17.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375889+0000", 2024-09-18T11:33:17.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0197999999999997e-05 2024-09-18T11:33:17.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:33:17.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.375894+0000", 2024-09-18T11:33:17.465 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0139999999999998e-06 2024-09-18T11:33:17.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.465 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.465 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.377699+0000", 2024-09-18T11:33:17.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001804688 2024-09-18T11:33:17.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.377776+0000", 2024-09-18T11:33:17.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.7081000000000003e-05 2024-09-18T11:33:17.466 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.466 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.466 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:09.386499+0000", 2024-09-18T11:33:17.466 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.008722746 2024-09-18T11:33:17.466 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.466 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.466 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.467 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4259 4.b 4:dba5bd39:::smithi11939506-17:head [write 685818~644423] snapc 69=[69,67,61,55,51,4e,4b,45,42,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:33:17.467 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.040083+0000", 2024-09-18T11:33:17.467 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 50.412737106000002, 2024-09-18T11:33:17.467 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.015748635, 2024-09-18T11:33:17.467 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.467 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.467 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.467 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.467 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.467 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4259 2024-09-18T11:33:17.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.468 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.040083+0000", 2024-09-18T11:33:17.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.468 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.040083+0000", 2024-09-18T11:33:17.468 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.468 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.040081+0000", 2024-09-18T11:33:17.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:17.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041258+0000", 2024-09-18T11:33:17.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011766540000000001 2024-09-18T11:33:17.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.469 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.469 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041286+0000", 2024-09-18T11:33:17.469 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8087000000000001e-05 2024-09-18T11:33:17.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041362+0000", 2024-09-18T11:33:17.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6155999999999994e-05 2024-09-18T11:33:17.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.044285+0000", 2024-09-18T11:33:17.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002922437 2024-09-18T11:33:17.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.470 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.470 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.044360+0000", 2024-09-18T11:33:17.470 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5201e-05 2024-09-18T11:33:17.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:17.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.044489+0000", 2024-09-18T11:33:17.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012950599999999999 2024-09-18T11:33:17.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052904+0000", 2024-09-18T11:33:17.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084149910000000001 2024-09-18T11:33:17.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.471 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.471 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055695+0000", 2024-09-18T11:33:17.471 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027902040000000001 2024-09-18T11:33:17.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055765+0000", 2024-09-18T11:33:17.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0720000000000001e-05 2024-09-18T11:33:17.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.472 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.472 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055832+0000", 2024-09-18T11:33:17.472 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6736999999999995e-05 2024-09-18T11:33:17.472 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.472 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.472 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.473 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.5.0:195 4.b e156/152 ondisk, result = 0)", 2024-09-18T11:33:17.473 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.041863+0000", 2024-09-18T11:33:17.473 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 50.410957586000002, 2024-09-18T11:33:17.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010991236, 2024-09-18T11:33:17.473 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.473 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.473 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.473 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041863+0000", 2024-09-18T11:33:17.473 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.473 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041863+0000", 2024-09-18T11:33:17.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041862+0000", 2024-09-18T11:33:17.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:17.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.474 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041868+0000", 2024-09-18T11:33:17.474 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2979999999999997e-06 2024-09-18T11:33:17.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.474 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041879+0000", 2024-09-18T11:33:17.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0522e-05 2024-09-18T11:33:17.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.041885+0000", 2024-09-18T11:33:17.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7830000000000004e-06 2024-09-18T11:33:17.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.475 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052841+0000", 2024-09-18T11:33:17.475 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010956604 2024-09-18T11:33:17.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.475 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052848+0000", 2024-09-18T11:33:17.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2249999999999997e-06 2024-09-18T11:33:17.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.476 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.476 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052854+0000", 2024-09-18T11:33:17.476 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6050000000000003e-06 2024-09-18T11:33:17.476 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.476 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.476 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.476 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.5.0:196 4.b e156/152 ondisk, result = 0)", 2024-09-18T11:33:17.476 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.042994+0000", 2024-09-18T11:33:17.476 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 50.409826924000001, 2024-09-18T11:33:17.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098926020000000003, 2024-09-18T11:33:17.477 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.477 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.477 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.042994+0000", 2024-09-18T11:33:17.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.477 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.477 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.042994+0000", 2024-09-18T11:33:17.477 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.042993+0000", 2024-09-18T11:33:17.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043000+0000", 2024-09-18T11:33:17.478 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.525e-06 2024-09-18T11:33:17.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.478 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.478 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043011+0000", 2024-09-18T11:33:17.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0335e-05 2024-09-18T11:33:17.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043015+0000", 2024-09-18T11:33:17.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2899999999999996e-06 2024-09-18T11:33:17.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052868+0000", 2024-09-18T11:33:17.479 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098528130000000002 2024-09-18T11:33:17.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.479 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.479 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052871+0000", 2024-09-18T11:33:17.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5080000000000001e-06 2024-09-18T11:33:17.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.480 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.480 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052886+0000", 2024-09-18T11:33:17.480 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4781e-05 2024-09-18T11:33:17.480 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.480 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.480 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.480 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop_reply(osd.5.0:194 4.3 e156/67 ondisk, result = 0)", 2024-09-18T11:33:17.480 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.043892+0000", 2024-09-18T11:33:17.480 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 50.408928134, 2024-09-18T11:33:17.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010237263999999999, 2024-09-18T11:33:17.481 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.481 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.481 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043892+0000", 2024-09-18T11:33:17.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.481 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.481 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043892+0000", 2024-09-18T11:33:17.481 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043891+0000", 2024-09-18T11:33:17.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:17.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043898+0000", 2024-09-18T11:33:17.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.1190000000000002e-06 2024-09-18T11:33:17.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.482 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.482 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043910+0000", 2024-09-18T11:33:17.482 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.29e-05 2024-09-18T11:33:17.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.043916+0000", 2024-09-18T11:33:17.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9499999999999998e-06 2024-09-18T11:33:17.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052830+0000", 2024-09-18T11:33:17.483 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089137480000000009 2024-09-18T11:33:17.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.483 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.483 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052839+0000", 2024-09-18T11:33:17.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.9560000000000003e-06 2024-09-18T11:33:17.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.484 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.484 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054130+0000", 2024-09-18T11:33:17.484 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0012905639999999999 2024-09-18T11:33:17.484 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.484 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.484 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.484 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4260 4.b 4:dba5bd39:::smithi11939506-17:head [write 2069089~216729] snapc 69=[69,67,61,55,51,4e,4b,45,42,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:33:17.484 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045444+0000", 2024-09-18T11:33:17.485 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 50.407376612, 2024-09-18T11:33:17.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011166715000000001, 2024-09-18T11:33:17.485 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.485 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.485 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.485 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.485 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.485 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4260 2024-09-18T11:33:17.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.485 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.485 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.485 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045444+0000", 2024-09-18T11:33:17.485 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045444+0000", 2024-09-18T11:33:17.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.486 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045443+0000", 2024-09-18T11:33:17.486 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:17.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.486 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045649+0000", 2024-09-18T11:33:17.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00020605499999999999 2024-09-18T11:33:17.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045662+0000", 2024-09-18T11:33:17.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3386e-05 2024-09-18T11:33:17.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.487 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.487 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045667+0000", 2024-09-18T11:33:17.487 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6750000000000001e-06 2024-09-18T11:33:17.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052921+0000", 2024-09-18T11:33:17.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007254613 2024-09-18T11:33:17.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.052994+0000", 2024-09-18T11:33:17.488 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2721999999999999e-05 2024-09-18T11:33:17.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.488 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:17.488 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.053116+0000", 2024-09-18T11:33:17.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121972 2024-09-18T11:33:17.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055857+0000", 2024-09-18T11:33:17.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027414539999999999 2024-09-18T11:33:17.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.489 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.489 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056502+0000", 2024-09-18T11:33:17.489 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00064412499999999995 2024-09-18T11:33:17.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056555+0000", 2024-09-18T11:33:17.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3467999999999997e-05 2024-09-18T11:33:17.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.490 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.490 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056611+0000", 2024-09-18T11:33:17.490 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5577e-05 2024-09-18T11:33:17.490 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.490 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.490 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.491 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4261 4.b 4:dba5bd39:::smithi11939506-17:head [setxattr _header (57) in=7b,truncate 2285818] snapc 69=[69,67,61,55,51,4e,4b,45,42,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:33:17.491 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045672+0000", 2024-09-18T11:33:17.491 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 50.407148513999999, 2024-09-18T11:33:17.491 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011289933, 2024-09-18T11:33:17.491 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.491 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.492 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.492 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.492 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.492 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4261 2024-09-18T11:33:17.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.492 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045672+0000", 2024-09-18T11:33:17.492 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.492 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.492 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045672+0000", 2024-09-18T11:33:17.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.493 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045671+0000", 2024-09-18T11:33:17.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.493 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.493 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045675+0000", 2024-09-18T11:33:17.493 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.563e-06 2024-09-18T11:33:17.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045681+0000", 2024-09-18T11:33:17.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5210000000000002e-06 2024-09-18T11:33:17.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045684+0000", 2024-09-18T11:33:17.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4149999999999999e-06 2024-09-18T11:33:17.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.494 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.494 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054143+0000", 2024-09-18T11:33:17.494 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084590810000000002 2024-09-18T11:33:17.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054214+0000", 2024-09-18T11:33:17.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1061000000000003e-05 2024-09-18T11:33:17.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:17.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.054354+0000", 2024-09-18T11:33:17.495 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000140035 2024-09-18T11:33:17.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.495 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.495 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056208+0000", 2024-09-18T11:33:17.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0018539229999999999 2024-09-18T11:33:17.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056885+0000", 2024-09-18T11:33:17.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00067688000000000004 2024-09-18T11:33:17.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.496 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.496 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056922+0000", 2024-09-18T11:33:17.496 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6693999999999998e-05 2024-09-18T11:33:17.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.497 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.497 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.056962+0000", 2024-09-18T11:33:17.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0299000000000002e-05 2024-09-18T11:33:17.497 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.497 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.497 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.497 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4262 4.b 4:dba5bd39:::smithi11939506-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:33:17.497 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:33:17.497 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 50.407132372, 2024-09-18T11:33:17.497 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011764513000000001, 2024-09-18T11:33:17.497 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.498 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.498 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.498 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.498 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.498 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4262 2024-09-18T11:33:17.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.498 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.498 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.498 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:33:17.498 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:33:17.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045688+0000", 2024-09-18T11:33:17.499 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.499 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.499 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045690+0000", 2024-09-18T11:33:17.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.745e-06 2024-09-18T11:33:17.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045697+0000", 2024-09-18T11:33:17.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7229999999999998e-06 2024-09-18T11:33:17.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.500 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045704+0000", 2024-09-18T11:33:17.500 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5719999999999999e-06 2024-09-18T11:33:17.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.500 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055555+0000", 2024-09-18T11:33:17.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098515430000000008 2024-09-18T11:33:17.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:17.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055675+0000", 2024-09-18T11:33:17.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012000799999999999 2024-09-18T11:33:17.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.501 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.501 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.057004+0000", 2024-09-18T11:33:17.501 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013286179999999999 2024-09-18T11:33:17.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.057357+0000", 2024-09-18T11:33:17.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00035355100000000002 2024-09-18T11:33:17.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.502 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.502 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.057453+0000", 2024-09-18T11:33:17.502 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5154000000000004e-05 2024-09-18T11:33:17.502 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.502 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.502 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.502 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4264 4.7 4:e0d95e10:::smithi11939506-22:head [read 0~3008486,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:33:17.503 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:33:17.503 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 50.407082385000002, 2024-09-18T11:33:17.503 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095333889999999998, 2024-09-18T11:33:17.503 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.503 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.503 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.503 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.503 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.503 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4264 2024-09-18T11:33:17.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.503 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.503 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.503 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:33:17.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.504 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:33:17.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.504 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.504 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045738+0000", 2024-09-18T11:33:17.504 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045747+0000", 2024-09-18T11:33:17.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2469999999999997e-06 2024-09-18T11:33:17.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045755+0000", 2024-09-18T11:33:17.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1499999999999999e-06 2024-09-18T11:33:17.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.505 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.505 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045759+0000", 2024-09-18T11:33:17.505 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0849999999999999e-06 2024-09-18T11:33:17.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045803+0000", 2024-09-18T11:33:17.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3621000000000001e-05 2024-09-18T11:33:17.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:33:17.506 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045807+0000", 2024-09-18T11:33:17.506 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7290000000000002e-06 2024-09-18T11:33:17.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.506 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.053475+0000", 2024-09-18T11:33:17.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076680100000000003 2024-09-18T11:33:17.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.053644+0000", 2024-09-18T11:33:17.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00016971299999999999 2024-09-18T11:33:17.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.507 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.507 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055272+0000", 2024-09-18T11:33:17.507 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001627245 2024-09-18T11:33:17.507 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.507 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.508 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.508 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4265 4.7 4:e0d95e10:::smithi11939506-22:head [sparse-read 0~2163506] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e156)", 2024-09-18T11:33:17.508 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:27.045792+0000", 2024-09-18T11:33:17.508 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 50.407028494999999, 2024-09-18T11:33:17.508 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010085024, 2024-09-18T11:33:17.508 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.508 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.508 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.508 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.508 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.508 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4265 2024-09-18T11:33:17.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.509 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045792+0000", 2024-09-18T11:33:17.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.509 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.509 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045792+0000", 2024-09-18T11:33:17.509 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045791+0000", 2024-09-18T11:33:17.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045797+0000", 2024-09-18T11:33:17.510 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0209999999999996e-06 2024-09-18T11:33:17.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.510 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.510 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045805+0000", 2024-09-18T11:33:17.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9349999999999994e-06 2024-09-18T11:33:17.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045811+0000", 2024-09-18T11:33:17.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2240000000000003e-06 2024-09-18T11:33:17.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.511 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.511 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045858+0000", 2024-09-18T11:33:17.511 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7565999999999998e-05 2024-09-18T11:33:17.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for peered", 2024-09-18T11:33:17.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.045869+0000", 2024-09-18T11:33:17.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0852e-05 2024-09-18T11:33:17.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055337+0000", 2024-09-18T11:33:17.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0094678729999999999 2024-09-18T11:33:17.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.512 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.512 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055417+0000", 2024-09-18T11:33:17.512 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.0092000000000005e-05 2024-09-18T11:33:17.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.513 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.513 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:27.055877+0000", 2024-09-18T11:33:17.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00046001800000000003 2024-09-18T11:33:17.513 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.513 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.513 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.513 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4349 4.b 4:d13decd8:::smithi11939506-2:head [read 0~2302625,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:33:17.513 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.382910+0000", 2024-09-18T11:33:17.513 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 46.069910430999997, 2024-09-18T11:33:17.513 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0098226930000000004, 2024-09-18T11:33:17.513 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.514 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.514 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.514 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.514 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.514 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4349 2024-09-18T11:33:17.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.514 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.514 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.514 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382910+0000", 2024-09-18T11:33:17.514 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382910+0000", 2024-09-18T11:33:17.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382909+0000", 2024-09-18T11:33:17.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.515 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.515 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382915+0000", 2024-09-18T11:33:17.515 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6049999999999997e-06 2024-09-18T11:33:17.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382924+0000", 2024-09-18T11:33:17.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6289999999999999e-06 2024-09-18T11:33:17.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382929+0000", 2024-09-18T11:33:17.516 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0479999999999998e-06 2024-09-18T11:33:17.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.516 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.516 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383030+0000", 2024-09-18T11:33:17.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000101346 2024-09-18T11:33:17.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:17.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383036+0000", 2024-09-18T11:33:17.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.2720000000000003e-06 2024-09-18T11:33:17.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.517 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.390597+0000", 2024-09-18T11:33:17.517 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075606880000000003 2024-09-18T11:33:17.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.517 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.390696+0000", 2024-09-18T11:33:17.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8740000000000007e-05 2024-09-18T11:33:17.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.518 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.518 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392733+0000", 2024-09-18T11:33:17.518 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002037025 2024-09-18T11:33:17.518 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.518 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.518 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.518 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4350 4.b 4:d13decd8:::smithi11939506-2:head [sparse-read 0~1463453] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:33:17.518 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.382978+0000", 2024-09-18T11:33:17.519 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 46.069842868000002, 2024-09-18T11:33:17.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.01020132, 2024-09-18T11:33:17.519 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.519 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.519 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.519 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.519 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.519 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4350 2024-09-18T11:33:17.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.519 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.519 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.519 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382978+0000", 2024-09-18T11:33:17.519 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382978+0000", 2024-09-18T11:33:17.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382976+0000", 2024-09-18T11:33:17.520 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:17.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.520 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.520 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382984+0000", 2024-09-18T11:33:17.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.328e-06 2024-09-18T11:33:17.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.382994+0000", 2024-09-18T11:33:17.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0801999999999999e-05 2024-09-18T11:33:17.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.521 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383001+0000", 2024-09-18T11:33:17.521 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6390000000000003e-06 2024-09-18T11:33:17.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.521 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383138+0000", 2024-09-18T11:33:17.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013676899999999999 2024-09-18T11:33:17.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:17.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.383144+0000", 2024-09-18T11:33:17.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6359999999999997e-06 2024-09-18T11:33:17.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.522 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.522 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392791+0000", 2024-09-18T11:33:17.522 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096461659999999994 2024-09-18T11:33:17.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.392897+0000", 2024-09-18T11:33:17.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000106087 2024-09-18T11:33:17.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.523 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.523 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.393179+0000", 2024-09-18T11:33:17.523 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00028227500000000002 2024-09-18T11:33:17.523 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.523 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.523 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.524 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4375 4.b 4:dba5bd39:::smithi11939506-17:head [write 437081~638017,stat] snapc 6d=[6d,6c,69,67,61,55,51,4b,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-18T11:33:17.524 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:31.549471+0000", 2024-09-18T11:33:17.524 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 45.903349165999998, 2024-09-18T11:33:17.524 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097774179999999995, 2024-09-18T11:33:17.524 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.524 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.524 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.524 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.524 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.524 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4375 2024-09-18T11:33:17.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.524 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.549471+0000", 2024-09-18T11:33:17.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.525 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.525 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.525 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.549471+0000", 2024-09-18T11:33:17.525 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.526 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.549470+0000", 2024-09-18T11:33:17.526 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:17.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.526 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553463+0000", 2024-09-18T11:33:17.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039932520000000001 2024-09-18T11:33:17.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553478+0000", 2024-09-18T11:33:17.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4647999999999999e-05 2024-09-18T11:33:17.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.527 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.527 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553484+0000", 2024-09-18T11:33:17.527 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3799999999999999e-06 2024-09-18T11:33:17.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553536+0000", 2024-09-18T11:33:17.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1938999999999999e-05 2024-09-18T11:33:17.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553637+0000", 2024-09-18T11:33:17.528 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010074899999999999 2024-09-18T11:33:17.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.528 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 3", 2024-09-18T11:33:17.528 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.553824+0000", 2024-09-18T11:33:17.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000187524 2024-09-18T11:33:17.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.556653+0000", 2024-09-18T11:33:17.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0028285879999999999 2024-09-18T11:33:17.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.529 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.559176+0000", 2024-09-18T11:33:17.529 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025229369999999998 2024-09-18T11:33:17.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.529 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.530 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.559212+0000", 2024-09-18T11:33:17.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5750000000000002e-05 2024-09-18T11:33:17.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.530 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.530 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:31.559249+0000", 2024-09-18T11:33:17.530 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7020999999999997e-05 2024-09-18T11:33:17.530 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.530 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.530 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.530 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4430 4.b 4:d3ae8215:::smithi11939506-41:head [read 0~3832029,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T11:33:17.531 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:33:17.531 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 42.424310204999998, 2024-09-18T11:33:17.531 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010918034, 2024-09-18T11:33:17.531 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.531 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.531 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.531 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.531 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.531 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4430 2024-09-18T11:33:17.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.531 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.531 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:33:17.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:33:17.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.532 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.532 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028510+0000", 2024-09-18T11:33:17.532 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:17.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028520+0000", 2024-09-18T11:33:17.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.005e-05 2024-09-18T11:33:17.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.533 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.533 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028538+0000", 2024-09-18T11:33:17.533 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8020999999999999e-05 2024-09-18T11:33:17.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028543+0000", 2024-09-18T11:33:17.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.485e-06 2024-09-18T11:33:17.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028634+0000", 2024-09-18T11:33:17.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.1120999999999997e-05 2024-09-18T11:33:17.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.534 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:17.534 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028639+0000", 2024-09-18T11:33:17.534 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1989999999999997e-06 2024-09-18T11:33:17.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.535 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.535 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.036054+0000", 2024-09-18T11:33:17.535 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074147559999999998 2024-09-18T11:33:17.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.535 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.535 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.036158+0000", 2024-09-18T11:33:17.535 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103564 2024-09-18T11:33:17.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.535 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.535 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039428+0000", 2024-09-18T11:33:17.536 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0032706390000000001 2024-09-18T11:33:17.536 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.536 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.536 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.536 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4431 4.b 4:d3ae8215:::smithi11939506-41:head [sparse-read 0~2481209] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e164)", 2024-09-18T11:33:17.536 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:33:17.536 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 42.424271417, 2024-09-18T11:33:17.536 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011399179000000001, 2024-09-18T11:33:17.536 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.536 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.536 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.537 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.537 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.537 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4431 2024-09-18T11:33:17.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.537 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.537 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:33:17.537 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.537 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.537 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:33:17.538 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.538 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.538 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028549+0000", 2024-09-18T11:33:17.538 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.538 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.538 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028552+0000", 2024-09-18T11:33:17.538 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5880000000000002e-06 2024-09-18T11:33:17.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028558+0000", 2024-09-18T11:33:17.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.84e-06 2024-09-18T11:33:17.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028562+0000", 2024-09-18T11:33:17.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6600000000000001e-06 2024-09-18T11:33:17.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.539 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.539 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028676+0000", 2024-09-18T11:33:17.539 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011408000000000001 2024-09-18T11:33:17.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.540 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:17.540 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.028681+0000", 2024-09-18T11:33:17.540 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5350000000000004e-06 2024-09-18T11:33:17.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.540 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.540 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039458+0000", 2024-09-18T11:33:17.540 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010777231 2024-09-18T11:33:17.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.540 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039564+0000", 2024-09-18T11:33:17.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000105524 2024-09-18T11:33:17.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.541 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.541 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:35.039948+0000", 2024-09-18T11:33:17.541 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00038430099999999998 2024-09-18T11:33:17.541 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.541 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.541 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.541 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4518 4.7 4:e0d95e10:::smithi11939506-22:head [rollback 6f] snapc 74=[74,73,71,6f,6c,69,67,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:33:17.541 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.055692+0000", 2024-09-18T11:33:17.542 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 37.397128156000001, 2024-09-18T11:33:17.542 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010093540999999999, 2024-09-18T11:33:17.542 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.542 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.542 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.542 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.542 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.542 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4518 2024-09-18T11:33:17.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.542 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.542 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.542 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055692+0000", 2024-09-18T11:33:17.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055692+0000", 2024-09-18T11:33:17.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.543 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.543 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055692+0000", 2024-09-18T11:33:17.543 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:17.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055697+0000", 2024-09-18T11:33:17.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.101e-06 2024-09-18T11:33:17.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055707+0000", 2024-09-18T11:33:17.544 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0316000000000001e-05 2024-09-18T11:33:17.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.544 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.544 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055713+0000", 2024-09-18T11:33:17.545 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8900000000000004e-06 2024-09-18T11:33:17.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.545 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057453+0000", 2024-09-18T11:33:17.545 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001739953 2024-09-18T11:33:17.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.545 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.545 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057537+0000", 2024-09-18T11:33:17.545 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4287000000000001e-05 2024-09-18T11:33:17.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.546 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:33:17.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057760+0000", 2024-09-18T11:33:17.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022294700000000001 2024-09-18T11:33:17.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.546 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065674+0000", 2024-09-18T11:33:17.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0079140110000000003 2024-09-18T11:33:17.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.546 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.546 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065681+0000", 2024-09-18T11:33:17.546 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2289999999999997e-06 2024-09-18T11:33:17.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.547 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065734+0000", 2024-09-18T11:33:17.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3136000000000001e-05 2024-09-18T11:33:17.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.547 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.547 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065786+0000", 2024-09-18T11:33:17.547 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1419999999999999e-05 2024-09-18T11:33:17.547 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.547 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.547 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.548 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4519 4.7 4:e0d95e10:::smithi11939506-22:head [append 0~0] snapc 74=[74,73,71,6f,6c,69,67,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:33:17.548 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.055719+0000", 2024-09-18T11:33:17.548 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 37.397101994000003, 2024-09-18T11:33:17.548 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010210589000000001, 2024-09-18T11:33:17.548 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.548 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.548 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.548 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.548 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.548 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4519 2024-09-18T11:33:17.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.548 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055719+0000", 2024-09-18T11:33:17.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055719+0000", 2024-09-18T11:33:17.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.549 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.549 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055718+0000", 2024-09-18T11:33:17.549 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055721+0000", 2024-09-18T11:33:17.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7039999999999999e-06 2024-09-18T11:33:17.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055728+0000", 2024-09-18T11:33:17.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1620000000000002e-06 2024-09-18T11:33:17.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.550 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.550 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055732+0000", 2024-09-18T11:33:17.550 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8410000000000002e-06 2024-09-18T11:33:17.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.551 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.059646+0000", 2024-09-18T11:33:17.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0039138489999999996 2024-09-18T11:33:17.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.551 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.059733+0000", 2024-09-18T11:33:17.551 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7163999999999994e-05 2024-09-18T11:33:17.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.551 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 7", 2024-09-18T11:33:17.551 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.059891+0000", 2024-09-18T11:33:17.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000158102 2024-09-18T11:33:17.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.552 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.552 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065798+0000", 2024-09-18T11:33:17.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059067649999999996 2024-09-18T11:33:17.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.552 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.552 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065840+0000", 2024-09-18T11:33:17.552 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2515999999999997e-05 2024-09-18T11:33:17.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.552 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065890+0000", 2024-09-18T11:33:17.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0055000000000001e-05 2024-09-18T11:33:17.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.553 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.553 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065929+0000", 2024-09-18T11:33:17.553 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8831000000000001e-05 2024-09-18T11:33:17.553 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.553 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.553 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.553 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4756 4.7 4:e301aefd:::smithi11939506-19:head [write 591712~786646,stat] snapc 7e=[7e,7c,7b,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e179)", 2024-09-18T11:33:17.554 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:50.036231+0000", 2024-09-18T11:33:17.554 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 27.416589349999999, 2024-09-18T11:33:17.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010177099, 2024-09-18T11:33:17.554 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.554 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.554 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.554 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.554 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.554 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4756 2024-09-18T11:33:17.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.554 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.554 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.554 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.036231+0000", 2024-09-18T11:33:17.554 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.036231+0000", 2024-09-18T11:33:17.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.555 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.036229+0000", 2024-09-18T11:33:17.555 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:17.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.555 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.037165+0000", 2024-09-18T11:33:17.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00093574399999999996 2024-09-18T11:33:17.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.556 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.037183+0000", 2024-09-18T11:33:17.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.7431999999999998e-05 2024-09-18T11:33:17.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.556 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.556 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.037189+0000", 2024-09-18T11:33:17.556 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4590000000000003e-06 2024-09-18T11:33:17.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.557 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.037243+0000", 2024-09-18T11:33:17.557 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4237999999999998e-05 2024-09-18T11:33:17.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.557 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:33:17.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.037249+0000", 2024-09-18T11:33:17.557 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.699e-06 2024-09-18T11:33:17.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.557 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.557 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.038725+0000", 2024-09-18T11:33:17.557 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0014756420000000001 2024-09-18T11:33:17.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.558 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.038797+0000", 2024-09-18T11:33:17.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2446000000000004e-05 2024-09-18T11:33:17.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.558 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:17.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.038915+0000", 2024-09-18T11:33:17.558 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011769000000000001 2024-09-18T11:33:17.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.558 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.558 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.041337+0000", 2024-09-18T11:33:17.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0024226339999999999 2024-09-18T11:33:17.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.046311+0000", 2024-09-18T11:33:17.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049738359999999997 2024-09-18T11:33:17.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.046359+0000", 2024-09-18T11:33:17.559 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7586999999999997e-05 2024-09-18T11:33:17.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.559 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.559 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:50.046408+0000", 2024-09-18T11:33:17.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9478000000000003e-05 2024-09-18T11:33:17.560 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.560 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.560 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.560 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:4790 3.2 e182/21 3:79129b11:::2f02c59c01a730562a65f21445c8af0e4a8aab3b969b8a3abf39e7b1bc84f918:head v 182'203, mlcod=176'201)", 2024-09-18T11:33:17.560 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:53.043420+0000", 2024-09-18T11:33:17.560 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 24.409400506000001, 2024-09-18T11:33:17.560 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096982140000000001, 2024-09-18T11:33:17.560 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.560 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.560 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.560 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.043420+0000", 2024-09-18T11:33:17.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.561 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.043420+0000", 2024-09-18T11:33:17.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.561 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.561 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.043420+0000", 2024-09-18T11:33:17.561 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.562 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.046026+0000", 2024-09-18T11:33:17.562 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0026064590000000002 2024-09-18T11:33:17.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.562 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.046040+0000", 2024-09-18T11:33:17.562 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.364e-05 2024-09-18T11:33:17.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.562 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.562 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.046045+0000", 2024-09-18T11:33:17.562 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6010000000000002e-06 2024-09-18T11:33:17.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.563 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.563 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.563 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.046080+0000", 2024-09-18T11:33:17.563 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4702000000000001e-05 2024-09-18T11:33:17.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.563 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.563 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.563 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.046095+0000", 2024-09-18T11:33:17.563 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5265e-05 2024-09-18T11:33:17.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.563 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.563 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.563 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.053085+0000", 2024-09-18T11:33:17.563 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0069894559999999998 2024-09-18T11:33:17.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.564 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.564 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.564 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.053118+0000", 2024-09-18T11:33:17.564 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3565999999999997e-05 2024-09-18T11:33:17.564 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.564 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.564 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.564 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.564 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.564 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:17.565 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T11:33:17.663 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:17.663 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:17.663 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:33:17.663 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:17.671 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T11:33:17.768 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:17.769 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:17.769 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:33:17.769 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:33:17.769 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:17.781 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T11:33:17.887 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:17.887 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:33:17.887 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:33:17.887 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:33:17.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:883 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T11:33:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:33:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 186.76877708000001, 2024-09-18T11:33:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.022411547, 2024-09-18T11:33:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 883 2024-09-18T11:33:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.888 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:33:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116319+0000", 2024-09-18T11:33:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.889 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116317+0000", 2024-09-18T11:33:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116325+0000", 2024-09-18T11:33:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4479999999999994e-06 2024-09-18T11:33:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116345+0000", 2024-09-18T11:33:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0083000000000001e-05 2024-09-18T11:33:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116354+0000", 2024-09-18T11:33:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5380000000000002e-06 2024-09-18T11:33:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116421+0000", 2024-09-18T11:33:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6711999999999994e-05 2024-09-18T11:33:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:33:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.116428+0000", 2024-09-18T11:33:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.3089999999999998e-06 2024-09-18T11:33:17.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118225+0000", 2024-09-18T11:33:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001796944 2024-09-18T11:33:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.118313+0000", 2024-09-18T11:33:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.7791e-05 2024-09-18T11:33:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.892 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:11.138731+0000", 2024-09-18T11:33:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.020417798000000001 2024-09-18T11:33:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:1170 4.a 4:5975a5ec:::smithi11939506-36:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-18T11:33:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:33:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 177.21575167899999, 2024-09-18T11:33:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017688285000000002, 2024-09-18T11:33:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.893 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 1170 2024-09-18T11:33:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:33:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.894 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669344+0000", 2024-09-18T11:33:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669343+0000", 2024-09-18T11:33:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669351+0000", 2024-09-18T11:33:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8290000000000008e-06 2024-09-18T11:33:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669366+0000", 2024-09-18T11:33:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5165e-05 2024-09-18T11:33:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669373+0000", 2024-09-18T11:33:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4370000000000001e-06 2024-09-18T11:33:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669420+0000", 2024-09-18T11:33:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6711000000000001e-05 2024-09-18T11:33:17.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.669490+0000", 2024-09-18T11:33:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9647000000000006e-05 2024-09-18T11:33:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:30:20.687033+0000", 2024-09-18T11:33:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.017543025 2024-09-18T11:33:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.897 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2339 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:33:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:33:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 136.49829438699999, 2024-09-18T11:33:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011809992, 2024-09-18T11:33:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2339 2024-09-18T11:33:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.898 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:33:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386802+0000", 2024-09-18T11:33:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386800+0000", 2024-09-18T11:33:17.899 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386808+0000", 2024-09-18T11:33:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4710000000000001e-06 2024-09-18T11:33:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386824+0000", 2024-09-18T11:33:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6104e-05 2024-09-18T11:33:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.900 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386836+0000", 2024-09-18T11:33:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1827999999999999e-05 2024-09-18T11:33:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388390+0000", 2024-09-18T11:33:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015539499999999999 2024-09-18T11:33:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388443+0000", 2024-09-18T11:33:17.901 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3785000000000001e-05 2024-09-18T11:33:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398238+0000", 2024-09-18T11:33:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097947989999999999 2024-09-18T11:33:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398315+0000", 2024-09-18T11:33:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6909999999999994e-05 2024-09-18T11:33:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.902 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398612+0000", 2024-09-18T11:33:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00029671999999999999 2024-09-18T11:33:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2340 4.5 4:aa575018:::smithi11939506-11:head [sparse-read 0~1204594] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:33:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:33:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 136.49825107500001, 2024-09-18T11:33:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012086448, 2024-09-18T11:33:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.903 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2340 2024-09-18T11:33:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:33:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.904 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:33:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386845+0000", 2024-09-18T11:33:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386848+0000", 2024-09-18T11:33:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3069999999999998e-06 2024-09-18T11:33:17.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386855+0000", 2024-09-18T11:33:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0979999999999996e-06 2024-09-18T11:33:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386859+0000", 2024-09-18T11:33:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9360000000000001e-06 2024-09-18T11:33:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388466+0000", 2024-09-18T11:33:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001607154 2024-09-18T11:33:17.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388507+0000", 2024-09-18T11:33:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1093000000000001e-05 2024-09-18T11:33:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398647+0000", 2024-09-18T11:33:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010139686 2024-09-18T11:33:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398718+0000", 2024-09-18T11:33:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1138999999999998e-05 2024-09-18T11:33:17.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398931+0000", 2024-09-18T11:33:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021356899999999999 2024-09-18T11:33:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:2341 4.5 4:aa575018:::smithi11939506-11:head [read 0~2487436,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T11:33:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:33:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 136.49823285299999, 2024-09-18T11:33:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.013751648999999999, 2024-09-18T11:33:17.908 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 2341 2024-09-18T11:33:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:33:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:33:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386863+0000", 2024-09-18T11:33:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386866+0000", 2024-09-18T11:33:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4249999999999998e-06 2024-09-18T11:33:17.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386872+0000", 2024-09-18T11:33:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.003e-06 2024-09-18T11:33:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.386876+0000", 2024-09-18T11:33:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.507e-06 2024-09-18T11:33:17.911 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.912 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388520+0000", 2024-09-18T11:33:17.912 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001644146 2024-09-18T11:33:17.912 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.912 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.912 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.388561+0000", 2024-09-18T11:33:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1486999999999998e-05 2024-09-18T11:33:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.398963+0000", 2024-09-18T11:33:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010401353 2024-09-18T11:33:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.399046+0000", 2024-09-18T11:33:17.913 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3141999999999998e-05 2024-09-18T11:33:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:01.400615+0000", 2024-09-18T11:33:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0015690960000000001 2024-09-18T11:33:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3105 4.a 4:552523b6:::smithi11939506-38:head [tier-promote] snapc 3e=[3e,3d,3b,37,35,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-18T11:33:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:33:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 104.86688510899999, 2024-09-18T11:33:17.914 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011735906000000001, 2024-09-18T11:33:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3105 2024-09-18T11:33:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:33:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:33:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018211+0000", 2024-09-18T11:33:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.916 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018212+0000", 2024-09-18T11:33:17.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.6759999999999999e-06 2024-09-18T11:33:17.917 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.917 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.917 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018216+0000", 2024-09-18T11:33:17.917 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-06 2024-09-18T11:33:17.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.918 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018218+0000", 2024-09-18T11:33:17.918 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6110000000000001e-06 2024-09-18T11:33:17.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.918 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018246+0000", 2024-09-18T11:33:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7557000000000001e-05 2024-09-18T11:33:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:33:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.018250+0000", 2024-09-18T11:33:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4299999999999999e-06 2024-09-18T11:33:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021364+0000", 2024-09-18T11:33:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0031135389999999998 2024-09-18T11:33:17.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.021494+0000", 2024-09-18T11:33:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000130333 2024-09-18T11:33:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:33.029947+0000", 2024-09-18T11:33:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084526150000000001 2024-09-18T11:33:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3324 4.5 4:a087a566:::smithi11939506-46:head [rollback 46] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:33:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:33:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 95.768154506000002, 2024-09-18T11:33:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010927767999999999, 2024-09-18T11:33:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3324 2024-09-18T11:33:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:33:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116942+0000", 2024-09-18T11:33:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.922 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116941+0000", 2024-09-18T11:33:17.923 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.923 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.923 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.923 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116949+0000", 2024-09-18T11:33:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4660000000000002e-06 2024-09-18T11:33:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116960+0000", 2024-09-18T11:33:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1484999999999999e-05 2024-09-18T11:33:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116964+0000", 2024-09-18T11:33:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2810000000000002e-06 2024-09-18T11:33:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117053+0000", 2024-09-18T11:33:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.8747999999999998e-05 2024-09-18T11:33:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117059+0000", 2024-09-18T11:33:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.246e-06 2024-09-18T11:33:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123604+0000", 2024-09-18T11:33:17.925 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065445319999999996 2024-09-18T11:33:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123663+0000", 2024-09-18T11:33:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8740999999999999e-05 2024-09-18T11:33:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:33:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.123803+0000", 2024-09-18T11:33:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014049200000000001 2024-09-18T11:33:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.926 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127600+0000", 2024-09-18T11:33:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0037971179999999999 2024-09-18T11:33:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127814+0000", 2024-09-18T11:33:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021333600000000001 2024-09-18T11:33:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.927 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127847+0000", 2024-09-18T11:33:17.928 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3188999999999997e-05 2024-09-18T11:33:17.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.928 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.928 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127869+0000", 2024-09-18T11:33:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2515e-05 2024-09-18T11:33:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3325 4.5 4:a087a566:::smithi11939506-46:head [append 0~0] snapc 47=[47,46,45,43,42,41,3f,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-18T11:33:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:33:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 95.768107319999999, 2024-09-18T11:33:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011181030999999999, 2024-09-18T11:33:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.929 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3325 2024-09-18T11:33:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:33:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.930 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116989+0000", 2024-09-18T11:33:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116988+0000", 2024-09-18T11:33:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116991+0000", 2024-09-18T11:33:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9890000000000001e-06 2024-09-18T11:33:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.931 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.116998+0000", 2024-09-18T11:33:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.5309999999999998e-06 2024-09-18T11:33:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117005+0000", 2024-09-18T11:33:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6420000000000001e-06 2024-09-18T11:33:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117091+0000", 2024-09-18T11:33:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.6835999999999993e-05 2024-09-18T11:33:17.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.117097+0000", 2024-09-18T11:33:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2569999999999998e-06 2024-09-18T11:33:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124717+0000", 2024-09-18T11:33:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0076204899999999997 2024-09-18T11:33:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124773+0000", 2024-09-18T11:33:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5470999999999998e-05 2024-09-18T11:33:17.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:33:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.124877+0000", 2024-09-18T11:33:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00010426500000000001 2024-09-18T11:33:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.127901+0000", 2024-09-18T11:33:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030243449999999999 2024-09-18T11:33:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128115+0000", 2024-09-18T11:33:17.934 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00021355300000000001 2024-09-18T11:33:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128145+0000", 2024-09-18T11:33:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0290999999999998e-05 2024-09-18T11:33:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:42.128170+0000", 2024-09-18T11:33:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4811e-05 2024-09-18T11:33:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:3474 4.5 4:a56ea1fb:::smithi11939506-48:head [write 1872245~637761] snapc 49=[49,48,47,46,45,43,42,41,3e,3d,3b,30,2a] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-18T11:33:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:33:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 91.441330643000001, 2024-09-18T11:33:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0092038580000000005, 2024-09-18T11:33:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.936 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 3474 2024-09-18T11:33:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:33:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443765+0000", 2024-09-18T11:33:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.937 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.443764+0000", 2024-09-18T11:33:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444359+0000", 2024-09-18T11:33:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00059519 2024-09-18T11:33:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444374+0000", 2024-09-18T11:33:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4949e-05 2024-09-18T11:33:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444382+0000", 2024-09-18T11:33:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.1369999999999996e-06 2024-09-18T11:33:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444428+0000", 2024-09-18T11:33:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6783000000000003e-05 2024-09-18T11:33:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444507+0000", 2024-09-18T11:33:17.939 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8415000000000001e-05 2024-09-18T11:33:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:33:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.444638+0000", 2024-09-18T11:33:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013129 2024-09-18T11:33:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.449059+0000", 2024-09-18T11:33:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.004420719 2024-09-18T11:33:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452909+0000", 2024-09-18T11:33:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003850773 2024-09-18T11:33:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452944+0000", 2024-09-18T11:33:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4039999999999999e-05 2024-09-18T11:33:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:31:46.452969+0000", 2024-09-18T11:33:17.941 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5795999999999998e-05 2024-09-18T11:33:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4251 4.5 4:af4ba75a:::smithi11939506-37:head [delete] snapc 69=[69,67,61,55,51,4e,4c,4b,45,42,3e,3d] ondisk+write+known_if_redirected+full_force+supports_pool_eio e155)", 2024-09-18T11:33:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:26.039209+0000", 2024-09-18T11:33:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 51.845886622999998, 2024-09-18T11:33:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010258422, 2024-09-18T11:33:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.942 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4251 2024-09-18T11:33:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039209+0000", 2024-09-18T11:33:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039209+0000", 2024-09-18T11:33:17.943 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039208+0000", 2024-09-18T11:33:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039217+0000", 2024-09-18T11:33:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.2650000000000002e-06 2024-09-18T11:33:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.944 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039236+0000", 2024-09-18T11:33:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8734000000000001e-05 2024-09-18T11:33:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039244+0000", 2024-09-18T11:33:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2900000000000002e-06 2024-09-18T11:33:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039289+0000", 2024-09-18T11:33:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4657999999999998e-05 2024-09-18T11:33:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.945 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op must wait for map", 2024-09-18T11:33:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.039294+0000", 2024-09-18T11:33:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0719999999999999e-06 2024-09-18T11:33:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.042052+0000", 2024-09-18T11:33:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027585399999999999 2024-09-18T11:33:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.042162+0000", 2024-09-18T11:33:17.946 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00011005699999999999 2024-09-18T11:33:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 4", 2024-09-18T11:33:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.042341+0000", 2024-09-18T11:33:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000178285 2024-09-18T11:33:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.044140+0000", 2024-09-18T11:33:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001799092 2024-09-18T11:33:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.947 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.049396+0000", 2024-09-18T11:33:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.005256213 2024-09-18T11:33:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.049431+0000", 2024-09-18T11:33:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.536e-05 2024-09-18T11:33:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:26.049468+0000", 2024-09-18T11:33:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6548999999999999e-05 2024-09-18T11:33:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.948 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1241 4.d e169/157 4:b44229a1:::smithi11939506-42:51 v 169'384, mlcod=168'380)", 2024-09-18T11:33:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.053467+0000", 2024-09-18T11:33:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 37.831628797, 2024-09-18T11:33:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010417658, 2024-09-18T11:33:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.949 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053467+0000", 2024-09-18T11:33:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053467+0000", 2024-09-18T11:33:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053467+0000", 2024-09-18T11:33:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053474+0000", 2024-09-18T11:33:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9430000000000003e-06 2024-09-18T11:33:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053483+0000", 2024-09-18T11:33:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3819999999999993e-06 2024-09-18T11:33:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053494+0000", 2024-09-18T11:33:17.951 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0609e-05 2024-09-18T11:33:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053532+0000", 2024-09-18T11:33:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8405000000000002e-05 2024-09-18T11:33:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.053554+0000", 2024-09-18T11:33:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1758000000000001e-05 2024-09-18T11:33:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.952 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.063852+0000", 2024-09-18T11:33:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010298185 2024-09-18T11:33:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.063885+0000", 2024-09-18T11:33:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2838e-05 2024-09-18T11:33:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.953 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4513 4.a 4:552523b6:::smithi11939506-38:73 [set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e169)", 2024-09-18T11:33:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.054905+0000", 2024-09-18T11:33:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 37.83019092, 2024-09-18T11:33:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.014365329, 2024-09-18T11:33:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4513 2024-09-18T11:33:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.954 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054905+0000", 2024-09-18T11:33:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054905+0000", 2024-09-18T11:33:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054904+0000", 2024-09-18T11:33:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.955 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054912+0000", 2024-09-18T11:33:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.4300000000000006e-06 2024-09-18T11:33:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054927+0000", 2024-09-18T11:33:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.4588e-05 2024-09-18T11:33:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054933+0000", 2024-09-18T11:33:17.956 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.9009999999999999e-06 2024-09-18T11:33:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054975+0000", 2024-09-18T11:33:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2588e-05 2024-09-18T11:33:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055079+0000", 2024-09-18T11:33:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000103508 2024-09-18T11:33:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.957 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:33:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.064787+0000", 2024-09-18T11:33:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0097086719999999998 2024-09-18T11:33:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.066147+0000", 2024-09-18T11:33:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0013597660000000001 2024-09-18T11:33:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.069210+0000", 2024-09-18T11:33:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030630430000000001 2024-09-18T11:33:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.069240+0000", 2024-09-18T11:33:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9628000000000002e-05 2024-09-18T11:33:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.069271+0000", 2024-09-18T11:33:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0682000000000002e-05 2024-09-18T11:33:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1242 4.d e169/157 4:b9bb642b:::smithi11939506-27:67 v 169'386, mlcod=168'380)", 2024-09-18T11:33:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.054970+0000", 2024-09-18T11:33:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 37.830126190999998, 2024-09-18T11:33:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0089404059999999997, 2024-09-18T11:33:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054970+0000", 2024-09-18T11:33:17.960 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054970+0000", 2024-09-18T11:33:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054969+0000", 2024-09-18T11:33:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054977+0000", 2024-09-18T11:33:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6240000000000002e-06 2024-09-18T11:33:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054987+0000", 2024-09-18T11:33:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0186e-05 2024-09-18T11:33:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054993+0000", 2024-09-18T11:33:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5160000000000002e-06 2024-09-18T11:33:17.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055563+0000", 2024-09-18T11:33:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00057030399999999999 2024-09-18T11:33:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.055576+0000", 2024-09-18T11:33:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.3366e-05 2024-09-18T11:33:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.963 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.063887+0000", 2024-09-18T11:33:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083112240000000007 2024-09-18T11:33:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.063910+0000", 2024-09-18T11:33:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2932999999999999e-05 2024-09-18T11:33:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.964 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4665 4.a 4:588debc2:::smithi11939506-18:head [rollback 71] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:33:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.316572+0000", 2024-09-18T11:33:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.568523933000002, 2024-09-18T11:33:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012421599, 2024-09-18T11:33:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4665 2024-09-18T11:33:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.965 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316572+0000", 2024-09-18T11:33:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316572+0000", 2024-09-18T11:33:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316572+0000", 2024-09-18T11:33:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.966 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316574+0000", 2024-09-18T11:33:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3599999999999999e-06 2024-09-18T11:33:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316578+0000", 2024-09-18T11:33:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0840000000000002e-06 2024-09-18T11:33:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316581+0000", 2024-09-18T11:33:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8739999999999999e-06 2024-09-18T11:33:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318695+0000", 2024-09-18T11:33:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002113382 2024-09-18T11:33:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.318805+0000", 2024-09-18T11:33:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000110511 2024-09-18T11:33:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:33:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.321575+0000", 2024-09-18T11:33:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0027700730000000001 2024-09-18T11:33:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.324576+0000", 2024-09-18T11:33:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0030004599999999999 2024-09-18T11:33:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.328914+0000", 2024-09-18T11:33:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0043382159999999998 2024-09-18T11:33:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.328951+0000", 2024-09-18T11:33:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7036999999999998e-05 2024-09-18T11:33:17.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.328994+0000", 2024-09-18T11:33:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2979999999999998e-05 2024-09-18T11:33:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4666 4.a 4:588debc2:::smithi11939506-18:head [append 0~0] snapc 79=[79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-18T11:33:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:45.316584+0000", 2024-09-18T11:33:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 32.568512302999999, 2024-09-18T11:33:17.970 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012596499000000001, 2024-09-18T11:33:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4666 2024-09-18T11:33:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316584+0000", 2024-09-18T11:33:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316584+0000", 2024-09-18T11:33:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316584+0000", 2024-09-18T11:33:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:33:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.972 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316585+0000", 2024-09-18T11:33:17.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8109999999999999e-06 2024-09-18T11:33:17.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316588+0000", 2024-09-18T11:33:17.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0769999999999999e-06 2024-09-18T11:33:17.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.973 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.316591+0000", 2024-09-18T11:33:17.973 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1320000000000001e-06 2024-09-18T11:33:17.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.973 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.319126+0000", 2024-09-18T11:33:17.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002535266 2024-09-18T11:33:17.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for blocked object", 2024-09-18T11:33:17.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.319180+0000", 2024-09-18T11:33:17.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4457999999999999e-05 2024-09-18T11:33:17.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.974 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.974 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.323839+0000", 2024-09-18T11:33:17.974 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046581720000000004 2024-09-18T11:33:17.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.324024+0000", 2024-09-18T11:33:17.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00018519599999999999 2024-09-18T11:33:17.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 1", 2024-09-18T11:33:17.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.324246+0000", 2024-09-18T11:33:17.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00022207800000000001 2024-09-18T11:33:17.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.975 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:17.975 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.325067+0000", 2024-09-18T11:33:17.975 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00082141699999999996 2024-09-18T11:33:17.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:17.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.329117+0000", 2024-09-18T11:33:17.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0040501210000000003 2024-09-18T11:33:17.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.329151+0000", 2024-09-18T11:33:17.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4133e-05 2024-09-18T11:33:17.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.976 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.976 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:45.329180+0000", 2024-09-18T11:33:17.976 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.887e-05 2024-09-18T11:33:17.977 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.977 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.977 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.977 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4590.0:4754 4.5 4:af4ba75a:::smithi11939506-37:head [read 0~2739607,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:33:17.977 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:49.234755+0000", 2024-09-18T11:33:17.977 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 28.650340644, 2024-09-18T11:33:17.977 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0093043659999999997, 2024-09-18T11:33:17.977 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.977 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "started", 2024-09-18T11:33:17.977 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:17.978 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4590", 2024-09-18T11:33:17.978 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:17.978 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 4754 2024-09-18T11:33:17.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.978 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234755+0000", 2024-09-18T11:33:17.978 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.978 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.978 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234755+0000", 2024-09-18T11:33:17.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234755+0000", 2024-09-18T11:33:17.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.979 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234760+0000", 2024-09-18T11:33:17.979 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6380000000000003e-06 2024-09-18T11:33:17.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.979 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234766+0000", 2024-09-18T11:33:17.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0159999999999997e-06 2024-09-18T11:33:17.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234771+0000", 2024-09-18T11:33:17.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1399999999999999e-06 2024-09-18T11:33:17.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.980 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.980 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234846+0000", 2024-09-18T11:33:17.980 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.5308999999999993e-05 2024-09-18T11:33:17.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:17.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.234848+0000", 2024-09-18T11:33:17.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4260000000000002e-06 2024-09-18T11:33:17.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.241890+0000", 2024-09-18T11:33:17.981 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0070411859999999996 2024-09-18T11:33:17.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.981 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.981 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.241983+0000", 2024-09-18T11:33:17.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.3843000000000003e-05 2024-09-18T11:33:17.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.982 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.982 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:49.244060+0000", 2024-09-18T11:33:17.982 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020763679999999999 2024-09-18T11:33:17.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.982 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.982 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1293 4.d e182/157 4:b9bb642b:::smithi11939506-27:6d v 182'415, mlcod=182'411)", 2024-09-18T11:33:17.982 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:53.041205+0000", 2024-09-18T11:33:17.982 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 24.843891380999999, 2024-09-18T11:33:17.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0091437189999999998, 2024-09-18T11:33:17.983 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.983 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.983 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041205+0000", 2024-09-18T11:33:17.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.983 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.983 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041205+0000", 2024-09-18T11:33:17.983 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041204+0000", 2024-09-18T11:33:17.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.984 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041210+0000", 2024-09-18T11:33:17.984 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3280000000000002e-06 2024-09-18T11:33:17.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.984 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041222+0000", 2024-09-18T11:33:17.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1686e-05 2024-09-18T11:33:17.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041227+0000", 2024-09-18T11:33:17.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2680000000000002e-06 2024-09-18T11:33:17.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.985 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.985 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041257+0000", 2024-09-18T11:33:17.985 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.9869000000000002e-05 2024-09-18T11:33:17.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041279+0000", 2024-09-18T11:33:17.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1651999999999999e-05 2024-09-18T11:33:17.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.050305+0000", 2024-09-18T11:33:17.986 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090263770000000004 2024-09-18T11:33:17.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.986 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.986 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.050348+0000", 2024-09-18T11:33:17.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.3173999999999997e-05 2024-09-18T11:33:17.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.987 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.987 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1294 4.d e182/157 4:b30010d9:::smithi11939506-43:7e v 182'417, mlcod=182'411)", 2024-09-18T11:33:17.987 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:53.041747+0000", 2024-09-18T11:33:17.987 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 24.843349148000001, 2024-09-18T11:33:17.987 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011293531000000001, 2024-09-18T11:33:17.987 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.987 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.987 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041747+0000", 2024-09-18T11:33:17.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041747+0000", 2024-09-18T11:33:17.988 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.988 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.988 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041746+0000", 2024-09-18T11:33:17.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:17.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041753+0000", 2024-09-18T11:33:17.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0010000000000004e-06 2024-09-18T11:33:17.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.989 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041766+0000", 2024-09-18T11:33:17.989 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.291e-05 2024-09-18T11:33:17.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.989 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041772+0000", 2024-09-18T11:33:17.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8390000000000003e-06 2024-09-18T11:33:17.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041810+0000", 2024-09-18T11:33:17.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7864000000000002e-05 2024-09-18T11:33:17.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.990 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.990 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.041828+0000", 2024-09-18T11:33:17.990 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8748000000000001e-05 2024-09-18T11:33:17.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.053002+0000", 2024-09-18T11:33:17.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.011173819 2024-09-18T11:33:17.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.991 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.991 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.053041+0000", 2024-09-18T11:33:17.991 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8260000000000003e-05 2024-09-18T11:33:17.991 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.991 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.991 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.991 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.4.0:1048 4.e e182/22 4:78eefc4b:::smithi11939506-7:6d v 182'218, mlcod=173'215)", 2024-09-18T11:33:17.992 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:53.042298+0000", 2024-09-18T11:33:17.992 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 24.842798035000001, 2024-09-18T11:33:17.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010780692999999999, 2024-09-18T11:33:17.992 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.992 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.992 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.992 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042298+0000", 2024-09-18T11:33:17.992 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.992 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042298+0000", 2024-09-18T11:33:17.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042297+0000", 2024-09-18T11:33:17.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.993 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.993 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042304+0000", 2024-09-18T11:33:17.993 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.3249999999999996e-06 2024-09-18T11:33:17.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042316+0000", 2024-09-18T11:33:17.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.2108e-05 2024-09-18T11:33:17.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.994 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.994 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042322+0000", 2024-09-18T11:33:17.994 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8490000000000002e-06 2024-09-18T11:33:17.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042678+0000", 2024-09-18T11:33:17.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00035671900000000002 2024-09-18T11:33:17.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042709+0000", 2024-09-18T11:33:17.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0518000000000002e-05 2024-09-18T11:33:17.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.995 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.995 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:17.995 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.053041+0000", 2024-09-18T11:33:17.995 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.010331965 2024-09-18T11:33:17.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.996 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:17.996 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.053079+0000", 2024-09-18T11:33:17.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7805000000000001e-05 2024-09-18T11:33:17.996 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.996 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:17.996 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:17.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.996 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.4.0:1049 4.e e182/22 4:7410c0e0:::smithi11939506-47:6d v 182'220, mlcod=173'215)", 2024-09-18T11:33:17.996 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:53.042878+0000", 2024-09-18T11:33:17.996 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 24.842217771000001, 2024-09-18T11:33:17.996 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012621894999999999, 2024-09-18T11:33:17.997 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:17.997 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:17.997 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:17.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:17.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042878+0000", 2024-09-18T11:33:17.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.997 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:17.997 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042878+0000", 2024-09-18T11:33:17.997 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:17.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:17.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042878+0000", 2024-09-18T11:33:17.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:17.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:17.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042883+0000", 2024-09-18T11:33:17.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7699999999999998e-06 2024-09-18T11:33:17.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.998 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:17.998 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042896+0000", 2024-09-18T11:33:17.998 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.296e-05 2024-09-18T11:33:17.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.998 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:17.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.042902+0000", 2024-09-18T11:33:17.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4600000000000002e-06 2024-09-18T11:33:17.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:17.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.047389+0000", 2024-09-18T11:33:17.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0044867010000000001 2024-09-18T11:33:17.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:17.999 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:17.999 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.047411+0000", 2024-09-18T11:33:17.999 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2076e-05 2024-09-18T11:33:17.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:17.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:18.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.055464+0000", 2024-09-18T11:33:18.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080533370000000007 2024-09-18T11:33:18.000 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.000 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:18.000 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:53.055500+0000", 2024-09-18T11:33:18.000 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6251000000000002e-05 2024-09-18T11:33:18.000 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.000 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.000 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.000 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.000 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.000 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:18.002 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T11:33:18.102 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:18.103 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:18.103 INFO:teuthology.orchestra.run.smithi148.stdout: "num_ops": 0 2024-09-18T11:33:18.103 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:18.111 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T11:33:18.209 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:18.209 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [], 2024-09-18T11:33:18.209 INFO:teuthology.orchestra.run.smithi148.stdout: "complaint_time": 30, 2024-09-18T11:33:18.210 INFO:teuthology.orchestra.run.smithi148.stdout: "num_blocked_ops": 0 2024-09-18T11:33:18.210 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:18.218 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T11:33:18.316 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-18T11:33:18.316 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 20, 2024-09-18T11:33:18.316 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 600, 2024-09-18T11:33:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: "ops": [ 2024-09-18T11:33:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:33:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 209.15633476599999, 2024-09-18T11:33:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0084708089999999993, 2024-09-18T11:33:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 47 2024-09-18T11:33:18.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:33:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157769+0000", 2024-09-18T11:33:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:18.318 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157768+0000", 2024-09-18T11:33:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157810+0000", 2024-09-18T11:33:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2357000000000001e-05 2024-09-18T11:33:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157816+0000", 2024-09-18T11:33:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.6119999999999998e-06 2024-09-18T11:33:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.319 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157819+0000", 2024-09-18T11:33:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8669999999999998e-06 2024-09-18T11:33:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163388+0000", 2024-09-18T11:33:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055688020000000003 2024-09-18T11:33:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:18.320 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163439+0000", 2024-09-18T11:33:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.0773999999999999e-05 2024-09-18T11:33:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163522+0000", 2024-09-18T11:33:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3257999999999995e-05 2024-09-18T11:33:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166137+0000", 2024-09-18T11:33:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.002615362 2024-09-18T11:33:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166144+0000", 2024-09-18T11:33:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.9099999999999999e-06 2024-09-18T11:33:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166199+0000", 2024-09-18T11:33:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.4987000000000001e-05 2024-09-18T11:33:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166239+0000", 2024-09-18T11:33:18.322 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0297999999999999e-05 2024-09-18T11:33:18.323 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.323 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.323 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.323 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:18.323 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:33:18.323 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 209.15627983600001, 2024-09-18T11:33:18.323 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085563340000000005, 2024-09-18T11:33:18.323 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:18.323 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.323 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:18.323 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:18.323 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 48 2024-09-18T11:33:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:33:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:33:18.324 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:18.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157823+0000", 2024-09-18T11:33:18.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:18.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:18.325 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157825+0000", 2024-09-18T11:33:18.325 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.1179999999999999e-06 2024-09-18T11:33:18.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.325 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:18.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157829+0000", 2024-09-18T11:33:18.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.433e-06 2024-09-18T11:33:18.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157831+0000", 2024-09-18T11:33:18.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.075e-06 2024-09-18T11:33:18.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.326 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:18.326 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163822+0000", 2024-09-18T11:33:18.326 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0059913249999999996 2024-09-18T11:33:18.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:18.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.163903+0000", 2024-09-18T11:33:18.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1143000000000006e-05 2024-09-18T11:33:18.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:18.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164026+0000", 2024-09-18T11:33:18.327 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00012254599999999999 2024-09-18T11:33:18.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.327 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:18.327 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166250+0000", 2024-09-18T11:33:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0022246570000000001 2024-09-18T11:33:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166299+0000", 2024-09-18T11:33:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.8798999999999998e-05 2024-09-18T11:33:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166344+0000", 2024-09-18T11:33:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4962000000000001e-05 2024-09-18T11:33:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.329 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:18.329 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166380+0000", 2024-09-18T11:33:18.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5596000000000003e-05 2024-09-18T11:33:18.329 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.329 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.329 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.329 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:18.329 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:33:18.329 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 209.15626985599999, 2024-09-18T11:33:18.329 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087046840000000007, 2024-09-18T11:33:18.330 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:18.330 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.330 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:18.330 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:18.330 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:18.330 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 49 2024-09-18T11:33:18.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.330 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:18.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.330 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:18.330 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:33:18.330 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:33:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157833+0000", 2024-09-18T11:33:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:33:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.331 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157837+0000", 2024-09-18T11:33:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.58e-06 2024-09-18T11:33:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157840+0000", 2024-09-18T11:33:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1810000000000001e-06 2024-09-18T11:33:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157842+0000", 2024-09-18T11:33:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2390000000000001e-06 2024-09-18T11:33:18.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164347+0000", 2024-09-18T11:33:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065045520000000003 2024-09-18T11:33:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164441+0000", 2024-09-18T11:33:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.4080999999999996e-05 2024-09-18T11:33:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.164580+0000", 2024-09-18T11:33:18.333 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000138904 2024-09-18T11:33:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166439+0000", 2024-09-18T11:33:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00185929 2024-09-18T11:33:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166447+0000", 2024-09-18T11:33:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8359999999999992e-06 2024-09-18T11:33:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.334 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166488+0000", 2024-09-18T11:33:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1539999999999999e-05 2024-09-18T11:33:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166538+0000", 2024-09-18T11:33:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.969e-05 2024-09-18T11:33:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.335 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:33:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 209.15625825699999, 2024-09-18T11:33:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096321789999999994, 2024-09-18T11:33:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 50 2024-09-18T11:33:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:18.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:33:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:33:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157845+0000", 2024-09-18T11:33:18.337 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:33:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157888+0000", 2024-09-18T11:33:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.2821000000000002e-05 2024-09-18T11:33:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157894+0000", 2024-09-18T11:33:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0889999999999996e-06 2024-09-18T11:33:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.338 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157897+0000", 2024-09-18T11:33:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.0910000000000001e-06 2024-09-18T11:33:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165408+0000", 2024-09-18T11:33:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075112379999999999 2024-09-18T11:33:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165487+0000", 2024-09-18T11:33:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8461000000000007e-05 2024-09-18T11:33:18.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:18.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.165621+0000", 2024-09-18T11:33:18.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013495 2024-09-18T11:33:18.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:18.340 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167392+0000", 2024-09-18T11:33:18.340 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017705640000000001 2024-09-18T11:33:18.340 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.340 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167428+0000", 2024-09-18T11:33:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6424999999999998e-05 2024-09-18T11:33:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167456+0000", 2024-09-18T11:33:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7861e-05 2024-09-18T11:33:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167477+0000", 2024-09-18T11:33:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0882000000000001e-05 2024-09-18T11:33:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.341 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:33:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 209.15620177599999, 2024-09-18T11:33:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0096586869999999991, 2024-09-18T11:33:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 51 2024-09-18T11:33:18.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:33:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157902+0000", 2024-09-18T11:33:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.343 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157901+0000", 2024-09-18T11:33:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157904+0000", 2024-09-18T11:33:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7690000000000001e-06 2024-09-18T11:33:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157907+0000", 2024-09-18T11:33:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.5360000000000001e-06 2024-09-18T11:33:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157911+0000", 2024-09-18T11:33:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7189999999999999e-06 2024-09-18T11:33:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166577+0000", 2024-09-18T11:33:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086654460000000003 2024-09-18T11:33:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166643+0000", 2024-09-18T11:33:18.345 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.6054000000000001e-05 2024-09-18T11:33:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166722+0000", 2024-09-18T11:33:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9505000000000005e-05 2024-09-18T11:33:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167508+0000", 2024-09-18T11:33:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00078607099999999995 2024-09-18T11:33:18.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167512+0000", 2024-09-18T11:33:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7230000000000002e-06 2024-09-18T11:33:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167540+0000", 2024-09-18T11:33:18.347 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8296000000000001e-05 2024-09-18T11:33:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167560+0000", 2024-09-18T11:33:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9978000000000001e-05 2024-09-18T11:33:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:18.348 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:33:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 209.15618795200001, 2024-09-18T11:33:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.012418024, 2024-09-18T11:33:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 52 2024-09-18T11:33:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:18.349 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:33:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:33:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157915+0000", 2024-09-18T11:33:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:33:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157918+0000", 2024-09-18T11:33:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5229999999999998e-06 2024-09-18T11:33:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157921+0000", 2024-09-18T11:33:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4620000000000001e-06 2024-09-18T11:33:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.351 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.157923+0000", 2024-09-18T11:33:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.306e-06 2024-09-18T11:33:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.166970+0000", 2024-09-18T11:33:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0090470059999999998 2024-09-18T11:33:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167056+0000", 2024-09-18T11:33:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.5798999999999996e-05 2024-09-18T11:33:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.167190+0000", 2024-09-18T11:33:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013353799999999999 2024-09-18T11:33:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.168333+0000", 2024-09-18T11:33:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011435180000000001 2024-09-18T11:33:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170254+0000", 2024-09-18T11:33:18.353 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0019210939999999999 2024-09-18T11:33:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170310+0000", 2024-09-18T11:33:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.5872000000000003e-05 2024-09-18T11:33:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.170333+0000", 2024-09-18T11:33:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3125999999999998e-05 2024-09-18T11:33:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.355 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:18.355 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:33:18.355 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 208.74000904499999, 2024-09-18T11:33:18.355 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007377074, 2024-09-18T11:33:18.355 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:18.355 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.355 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:18.355 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:18.355 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:18.355 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 71 2024-09-18T11:33:18.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.355 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:18.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:18.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:33:18.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:18.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:33:18.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.356 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:18.356 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574094+0000", 2024-09-18T11:33:18.356 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:33:18.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:18.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574096+0000", 2024-09-18T11:33:18.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.742e-06 2024-09-18T11:33:18.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.357 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:18.357 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574099+0000", 2024-09-18T11:33:18.357 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1700000000000001e-06 2024-09-18T11:33:18.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574101+0000", 2024-09-18T11:33:18.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9999999999999999e-06 2024-09-18T11:33:18.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:18.358 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578738+0000", 2024-09-18T11:33:18.358 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0046370140000000001 2024-09-18T11:33:18.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.358 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:18.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578789+0000", 2024-09-18T11:33:18.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1331e-05 2024-09-18T11:33:18.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:18.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.578864+0000", 2024-09-18T11:33:18.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4580999999999996e-05 2024-09-18T11:33:18.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.359 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:18.359 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581365+0000", 2024-09-18T11:33:18.359 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0025008589999999998 2024-09-18T11:33:18.359 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:18.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581409+0000", 2024-09-18T11:33:18.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4644000000000001e-05 2024-09-18T11:33:18.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:18.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581435+0000", 2024-09-18T11:33:18.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.5826999999999999e-05 2024-09-18T11:33:18.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.360 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:18.360 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581471+0000", 2024-09-18T11:33:18.360 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6114e-05 2024-09-18T11:33:18.361 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.361 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.361 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.361 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:18.361 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:33:18.361 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 208.73998574500001, 2024-09-18T11:33:18.361 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0074827569999999996, 2024-09-18T11:33:18.361 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:18.361 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.361 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:18.361 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:18.362 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:18.362 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 72 2024-09-18T11:33:18.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.362 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:18.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:18.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:33:18.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.362 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.362 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:18.362 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574118+0000", 2024-09-18T11:33:18.362 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:18.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574117+0000", 2024-09-18T11:33:18.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:18.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:18.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574188+0000", 2024-09-18T11:33:18.363 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0636999999999995e-05 2024-09-18T11:33:18.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.363 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:18.363 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574194+0000", 2024-09-18T11:33:18.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.122e-06 2024-09-18T11:33:18.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574197+0000", 2024-09-18T11:33:18.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.7590000000000002e-06 2024-09-18T11:33:18.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.364 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:18.364 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579190+0000", 2024-09-18T11:33:18.364 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0049930030000000002 2024-09-18T11:33:18.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:18.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579234+0000", 2024-09-18T11:33:18.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4653000000000001e-05 2024-09-18T11:33:18.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:18.365 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579317+0000", 2024-09-18T11:33:18.365 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.2440000000000004e-05 2024-09-18T11:33:18.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.365 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:18.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581374+0000", 2024-09-18T11:33:18.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0020567670000000001 2024-09-18T11:33:18.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:18.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581522+0000", 2024-09-18T11:33:18.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00014808200000000001 2024-09-18T11:33:18.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.366 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:18.366 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581567+0000", 2024-09-18T11:33:18.366 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.5021999999999997e-05 2024-09-18T11:33:18.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.367 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:18.367 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581600+0000", 2024-09-18T11:33:18.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3661000000000002e-05 2024-09-18T11:33:18.367 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.367 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.367 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.367 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:18.367 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:33:18.367 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 208.73990013599999, 2024-09-18T11:33:18.367 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0080496400000000003, 2024-09-18T11:33:18.367 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:18.367 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.368 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:18.368 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:18.368 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:18.368 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 73 2024-09-18T11:33:18.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.368 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:18.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.368 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:18.368 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:33:18.368 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:18.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:33:18.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:18.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574203+0000", 2024-09-18T11:33:18.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:18.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.369 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:18.369 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574205+0000", 2024-09-18T11:33:18.369 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.3329999999999999e-06 2024-09-18T11:33:18.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:18.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574209+0000", 2024-09-18T11:33:18.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.506e-06 2024-09-18T11:33:18.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574211+0000", 2024-09-18T11:33:18.370 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.582e-06 2024-09-18T11:33:18.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.370 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:18.370 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579792+0000", 2024-09-18T11:33:18.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0055810909999999998 2024-09-18T11:33:18.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:18.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579861+0000", 2024-09-18T11:33:18.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.8770999999999996e-05 2024-09-18T11:33:18.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:18.371 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.579982+0000", 2024-09-18T11:33:18.371 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000121239 2024-09-18T11:33:18.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.371 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:18.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581611+0000", 2024-09-18T11:33:18.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001628261 2024-09-18T11:33:18.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:18.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582178+0000", 2024-09-18T11:33:18.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00056782100000000004 2024-09-18T11:33:18.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.372 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:18.372 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582219+0000", 2024-09-18T11:33:18.372 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1004000000000002e-05 2024-09-18T11:33:18.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.373 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:18.373 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582253+0000", 2024-09-18T11:33:18.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3420999999999997e-05 2024-09-18T11:33:18.373 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.373 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.373 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.373 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:18.373 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:33:18.373 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 208.73988917099999, 2024-09-18T11:33:18.373 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0083116230000000006, 2024-09-18T11:33:18.374 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:18.374 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.374 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:18.374 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:18.374 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:18.374 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 74 2024-09-18T11:33:18.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.374 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:18.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.374 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:18.374 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:33:18.374 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:18.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:33:18.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:18.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574214+0000", 2024-09-18T11:33:18.375 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:33:18.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.375 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:18.375 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574216+0000", 2024-09-18T11:33:18.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.8369999999999999e-06 2024-09-18T11:33:18.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:18.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574219+0000", 2024-09-18T11:33:18.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.174e-06 2024-09-18T11:33:18.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.376 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.376 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574223+0000", 2024-09-18T11:33:18.376 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.6169999999999998e-06 2024-09-18T11:33:18.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:18.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580275+0000", 2024-09-18T11:33:18.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0060519770000000001 2024-09-18T11:33:18.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:18.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580319+0000", 2024-09-18T11:33:18.377 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4515000000000003e-05 2024-09-18T11:33:18.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.377 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:18.377 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580395+0000", 2024-09-18T11:33:18.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.6137999999999995e-05 2024-09-18T11:33:18.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:18.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581983+0000", 2024-09-18T11:33:18.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.001587576 2024-09-18T11:33:18.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.378 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:18.378 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582467+0000", 2024-09-18T11:33:18.378 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00048379199999999998 2024-09-18T11:33:18.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:18.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582504+0000", 2024-09-18T11:33:18.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7104999999999998e-05 2024-09-18T11:33:18.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.379 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:18.379 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582526+0000", 2024-09-18T11:33:18.379 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2095000000000001e-05 2024-09-18T11:33:18.379 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.379 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.379 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.380 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:18.380 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:33:18.380 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 208.739878003, 2024-09-18T11:33:18.380 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086273040000000006, 2024-09-18T11:33:18.380 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:18.380 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.380 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:18.380 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:18.380 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:18.380 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 75 2024-09-18T11:33:18.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.380 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:18.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:18.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:33:18.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:18.381 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:33:18.381 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.381 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.381 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:18.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574225+0000", 2024-09-18T11:33:18.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967296 2024-09-18T11:33:18.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:18.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574279+0000", 2024-09-18T11:33:18.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3701e-05 2024-09-18T11:33:18.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.382 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:18.382 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574283+0000", 2024-09-18T11:33:18.382 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.6619999999999996e-06 2024-09-18T11:33:18.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574286+0000", 2024-09-18T11:33:18.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.847e-06 2024-09-18T11:33:18.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:18.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580618+0000", 2024-09-18T11:33:18.383 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063321510000000003 2024-09-18T11:33:18.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.383 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:18.383 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580693+0000", 2024-09-18T11:33:18.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4753999999999996e-05 2024-09-18T11:33:18.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:18.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.580826+0000", 2024-09-18T11:33:18.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00013265799999999999 2024-09-18T11:33:18.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.384 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:18.384 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582568+0000", 2024-09-18T11:33:18.384 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017417489999999999 2024-09-18T11:33:18.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:18.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582779+0000", 2024-09-18T11:33:18.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000211134 2024-09-18T11:33:18.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:18.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582819+0000", 2024-09-18T11:33:18.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0040999999999998e-05 2024-09-18T11:33:18.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.385 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:18.385 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582853+0000", 2024-09-18T11:33:18.385 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3819999999999998e-05 2024-09-18T11:33:18.385 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.386 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.386 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.386 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:18.386 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:33:18.386 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 208.73981267400001, 2024-09-18T11:33:18.386 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0095302909999999998, 2024-09-18T11:33:18.386 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:18.386 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.386 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:18.386 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:18.386 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:18.386 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 76 2024-09-18T11:33:18.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.387 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:18.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:18.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:33:18.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:18.387 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574291+0000", 2024-09-18T11:33:18.387 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.387 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:18.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574290+0000", 2024-09-18T11:33:18.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:18.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:18.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574292+0000", 2024-09-18T11:33:18.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2369999999999999e-06 2024-09-18T11:33:18.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.388 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:18.388 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574296+0000", 2024-09-18T11:33:18.388 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.2669999999999998e-06 2024-09-18T11:33:18.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.574298+0000", 2024-09-18T11:33:18.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0959999999999999e-06 2024-09-18T11:33:18.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:18.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581628+0000", 2024-09-18T11:33:18.389 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073304709999999999 2024-09-18T11:33:18.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.389 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:18.389 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581673+0000", 2024-09-18T11:33:18.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.4428000000000002e-05 2024-09-18T11:33:18.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:18.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.581775+0000", 2024-09-18T11:33:18.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000102147 2024-09-18T11:33:18.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:18.390 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.582590+0000", 2024-09-18T11:33:18.390 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00081471500000000004 2024-09-18T11:33:18.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.390 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:18.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583749+0000", 2024-09-18T11:33:18.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011596600000000001 2024-09-18T11:33:18.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:18.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583789+0000", 2024-09-18T11:33:18.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0244000000000003e-05 2024-09-18T11:33:18.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.391 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.391 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:18.391 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.583821+0000", 2024-09-18T11:33:18.391 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.1414000000000001e-05 2024-09-18T11:33:18.391 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.391 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.392 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.392 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:79 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:18.392 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:33:18.392 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 208.72965069599999, 2024-09-18T11:33:18.392 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063752069999999999, 2024-09-18T11:33:18.392 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:18.392 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.392 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:18.392 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:18.392 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:18.392 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 79 2024-09-18T11:33:18.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.393 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:18.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:18.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:33:18.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.393 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:18.393 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584453+0000", 2024-09-18T11:33:18.393 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:18.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584452+0000", 2024-09-18T11:33:18.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:18.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:18.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584526+0000", 2024-09-18T11:33:18.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.4062999999999999e-05 2024-09-18T11:33:18.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.394 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:18.394 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584536+0000", 2024-09-18T11:33:18.394 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.8439999999999993e-06 2024-09-18T11:33:18.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584540+0000", 2024-09-18T11:33:18.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9110000000000003e-06 2024-09-18T11:33:18.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.395 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:18.395 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585729+0000", 2024-09-18T11:33:18.395 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0011890850000000001 2024-09-18T11:33:18.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:18.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585811+0000", 2024-09-18T11:33:18.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.1797000000000003e-05 2024-09-18T11:33:18.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:18.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.585934+0000", 2024-09-18T11:33:18.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.000123499 2024-09-18T11:33:18.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.396 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.396 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:18.396 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586834+0000", 2024-09-18T11:33:18.396 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00089981000000000002 2024-09-18T11:33:18.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:18.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590740+0000", 2024-09-18T11:33:18.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003906021 2024-09-18T11:33:18.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:18.397 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590793+0000", 2024-09-18T11:33:18.397 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3152000000000002e-05 2024-09-18T11:33:18.397 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.397 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:18.398 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590828+0000", 2024-09-18T11:33:18.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.4486000000000002e-05 2024-09-18T11:33:18.398 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.398 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.398 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.398 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_op(client.4206.0:80 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T11:33:18.398 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:33:18.398 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 208.729558217, 2024-09-18T11:33:18.398 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0063772150000000003, 2024-09-18T11:33:18.398 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:18.398 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.399 INFO:teuthology.orchestra.run.smithi148.stdout: "client_info": { 2024-09-18T11:33:18.399 INFO:teuthology.orchestra.run.smithi148.stdout: "client": "client.4206", 2024-09-18T11:33:18.399 INFO:teuthology.orchestra.run.smithi148.stdout: "client_addr": "172.21.15.119:0/3584115318", 2024-09-18T11:33:18.399 INFO:teuthology.orchestra.run.smithi148.stdout: "tid": 80 2024-09-18T11:33:18.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.399 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:18.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:18.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:33:18.399 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.399 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:18.399 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:33:18.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:18.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584545+0000", 2024-09-18T11:33:18.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:18.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.400 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:18.400 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584597+0000", 2024-09-18T11:33:18.400 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.2413000000000001e-05 2024-09-18T11:33:18.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:18.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584602+0000", 2024-09-18T11:33:18.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.7140000000000001e-06 2024-09-18T11:33:18.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.584604+0000", 2024-09-18T11:33:18.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.6869999999999999e-06 2024-09-18T11:33:18.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.401 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.401 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:18.401 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586290+0000", 2024-09-18T11:33:18.401 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0016857980000000001 2024-09-18T11:33:18.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:18.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586344+0000", 2024-09-18T11:33:18.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.3989000000000001e-05 2024-09-18T11:33:18.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "waiting for subops from 0", 2024-09-18T11:33:18.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.586417+0000", 2024-09-18T11:33:18.402 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.2292000000000005e-05 2024-09-18T11:33:18.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.402 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.402 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "op_commit", 2024-09-18T11:33:18.402 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.587019+0000", 2024-09-18T11:33:18.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00060287000000000003 2024-09-18T11:33:18.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:18.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590873+0000", 2024-09-18T11:33:18.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.003853885 2024-09-18T11:33:18.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.403 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:18.403 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590902+0000", 2024-09-18T11:33:18.403 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.8326999999999998e-05 2024-09-18T11:33:18.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.404 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:18.404 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:29:49.590922+0000", 2024-09-18T11:33:18.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0650999999999999e-05 2024-09-18T11:33:18.404 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.404 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.404 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.404 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.404 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:123 4.3 e132/67 4:cc46532a:::smithi11939506-33:30 v 132'171, mlcod=128'168)", 2024-09-18T11:33:18.404 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:33:18.404 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 75.286251591999999, 2024-09-18T11:33:18.404 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0087627039999999996, 2024-09-18T11:33:18.405 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:18.405 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.405 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:18.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:18.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:33:18.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.405 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.405 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:18.405 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027852+0000", 2024-09-18T11:33:18.405 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:18.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027851+0000", 2024-09-18T11:33:18.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:18.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:18.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027871+0000", 2024-09-18T11:33:18.406 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9749e-05 2024-09-18T11:33:18.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.406 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:18.406 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027882+0000", 2024-09-18T11:33:18.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1242e-05 2024-09-18T11:33:18.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027889+0000", 2024-09-18T11:33:18.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.7649999999999996e-06 2024-09-18T11:33:18.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:18.407 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027929+0000", 2024-09-18T11:33:18.407 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.9984999999999997e-05 2024-09-18T11:33:18.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.407 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:18.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.027951+0000", 2024-09-18T11:33:18.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.2047000000000001e-05 2024-09-18T11:33:18.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:18.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.036565+0000", 2024-09-18T11:33:18.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0086135110000000008 2024-09-18T11:33:18.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.408 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.408 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:18.408 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.036614+0000", 2024-09-18T11:33:18.408 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9747000000000003e-05 2024-09-18T11:33:18.409 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.409 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.409 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.409 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:124 4.3 e132/67 4:ca2b2309:::smithi11939506-44:30 v 132'173, mlcod=128'168)", 2024-09-18T11:33:18.409 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:33:18.409 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 75.285559610000007, 2024-09-18T11:33:18.409 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0088947309999999995, 2024-09-18T11:33:18.409 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:18.409 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.409 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:18.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:18.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:33:18.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:18.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028544+0000", 2024-09-18T11:33:18.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.410 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:18.410 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028543+0000", 2024-09-18T11:33:18.410 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:18.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:18.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028551+0000", 2024-09-18T11:33:18.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.8720000000000002e-06 2024-09-18T11:33:18.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:18.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028562+0000", 2024-09-18T11:33:18.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1039000000000001e-05 2024-09-18T11:33:18.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.411 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.411 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028569+0000", 2024-09-18T11:33:18.411 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0350000000000004e-06 2024-09-18T11:33:18.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:18.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028848+0000", 2024-09-18T11:33:18.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00027935600000000003 2024-09-18T11:33:18.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.412 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:18.412 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.028868+0000", 2024-09-18T11:33:18.412 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.9947999999999999e-05 2024-09-18T11:33:18.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:18.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037405+0000", 2024-09-18T11:33:18.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0085369920000000002 2024-09-18T11:33:18.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.413 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:18.413 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:03.037438+0000", 2024-09-18T11:33:18.413 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.3244000000000002e-05 2024-09-18T11:33:18.413 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.413 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.413 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.413 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.414 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:154 4.3 e147/67 4:ca2b2309:::smithi11939506-44:5d v 147'216, mlcod=145'210)", 2024-09-18T11:33:18.414 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:33:18.414 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 59.976773160999997, 2024-09-18T11:33:18.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0066667829999999999, 2024-09-18T11:33:18.414 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:18.414 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.414 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:18.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:18.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:33:18.414 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.414 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:18.414 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:33:18.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:18.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337330+0000", 2024-09-18T11:33:18.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:18.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:18.415 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337335+0000", 2024-09-18T11:33:18.415 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.1479999999999997e-06 2024-09-18T11:33:18.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.415 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:18.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337344+0000", 2024-09-18T11:33:18.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 9.5640000000000002e-06 2024-09-18T11:33:18.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337349+0000", 2024-09-18T11:33:18.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.9470000000000002e-06 2024-09-18T11:33:18.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.416 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:18.416 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337410+0000", 2024-09-18T11:33:18.416 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.0448999999999997e-05 2024-09-18T11:33:18.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.416 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:18.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.337425+0000", 2024-09-18T11:33:18.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.504e-05 2024-09-18T11:33:18.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:18.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.343972+0000", 2024-09-18T11:33:18.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0065474169999999998 2024-09-18T11:33:18.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.417 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.417 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:18.417 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:18.343997+0000", 2024-09-18T11:33:18.417 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.4672999999999999e-05 2024-09-18T11:33:18.417 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.418 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.418 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.418 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.1.0:1238 4.6 e169/143 4:6f86ffcc:::smithi11939506-15:51 v 169'193, mlcod=166'187)", 2024-09-18T11:33:18.418 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.044089+0000", 2024-09-18T11:33:18.418 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 38.270014091999997, 2024-09-18T11:33:18.418 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0072870620000000004, 2024-09-18T11:33:18.418 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:18.418 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.418 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:18.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.418 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:18.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044089+0000", 2024-09-18T11:33:18.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:18.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044089+0000", 2024-09-18T11:33:18.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.419 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.419 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:18.419 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044089+0000", 2024-09-18T11:33:18.419 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:18.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:18.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044097+0000", 2024-09-18T11:33:18.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.9060000000000002e-06 2024-09-18T11:33:18.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:18.420 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044105+0000", 2024-09-18T11:33:18.420 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 8.3590000000000008e-06 2024-09-18T11:33:18.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.420 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044112+0000", 2024-09-18T11:33:18.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 7.0639999999999996e-06 2024-09-18T11:33:18.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:18.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044851+0000", 2024-09-18T11:33:18.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.00073857800000000004 2024-09-18T11:33:18.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.421 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.421 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:18.421 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.044871+0000", 2024-09-18T11:33:18.421 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0140999999999999e-05 2024-09-18T11:33:18.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:18.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.051338+0000", 2024-09-18T11:33:18.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0064673760000000004 2024-09-18T11:33:18.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.422 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:18.422 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.051376+0000", 2024-09-18T11:33:18.422 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.8222000000000001e-05 2024-09-18T11:33:18.422 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.422 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.422 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.423 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(osd.5.0:281 4.3 e169/67 4:ca2b2309:::smithi11939506-44:5d v 169'279, mlcod=168'273)", 2024-09-18T11:33:18.423 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.047201+0000", 2024-09-18T11:33:18.423 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 38.266901957000002, 2024-09-18T11:33:18.423 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.007110346, 2024-09-18T11:33:18.423 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:18.423 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.423 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:18.423 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.423 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:18.423 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047201+0000", 2024-09-18T11:33:18.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:18.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047201+0000", 2024-09-18T11:33:18.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:18.424 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047201+0000", 2024-09-18T11:33:18.424 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:18.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.424 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:18.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047206+0000", 2024-09-18T11:33:18.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.7309999999999998e-06 2024-09-18T11:33:18.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:18.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047217+0000", 2024-09-18T11:33:18.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.0577000000000001e-05 2024-09-18T11:33:18.425 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.425 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.425 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.425 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.047221+0000", 2024-09-18T11:33:18.425 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.0509999999999998e-06 2024-09-18T11:33:18.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:18.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.048977+0000", 2024-09-18T11:33:18.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0017558179999999999 2024-09-18T11:33:18.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:18.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.048999+0000", 2024-09-18T11:33:18.426 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.23e-05 2024-09-18T11:33:18.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.426 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:18.426 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054275+0000", 2024-09-18T11:33:18.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0052754150000000003 2024-09-18T11:33:18.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.427 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:18.427 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.054312+0000", 2024-09-18T11:33:18.427 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 3.7163999999999999e-05 2024-09-18T11:33:18.427 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.427 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.427 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.427 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.427 INFO:teuthology.orchestra.run.smithi148.stdout: "description": "osd_repop(client.4590.0:4518 4.7 e169/155 4:e0d95e10:::smithi11939506-22:head v 169'394, mlcod=169'389)", 2024-09-18T11:33:18.427 INFO:teuthology.orchestra.run.smithi148.stdout: "initiated_at": "2024-09-18T11:32:40.057943+0000", 2024-09-18T11:33:18.427 INFO:teuthology.orchestra.run.smithi148.stdout: "age": 38.256160706999999, 2024-09-18T11:33:18.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0075088719999999998, 2024-09-18T11:33:18.428 INFO:teuthology.orchestra.run.smithi148.stdout: "type_data": { 2024-09-18T11:33:18.428 INFO:teuthology.orchestra.run.smithi148.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.428 INFO:teuthology.orchestra.run.smithi148.stdout: "events": [ 2024-09-18T11:33:18.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "initiated", 2024-09-18T11:33:18.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057943+0000", 2024-09-18T11:33:18.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.428 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.428 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "throttled", 2024-09-18T11:33:18.428 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057943+0000", 2024-09-18T11:33:18.428 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0 2024-09-18T11:33:18.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "header_read", 2024-09-18T11:33:18.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057942+0000", 2024-09-18T11:33:18.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4294967295.999999 2024-09-18T11:33:18.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "all_read", 2024-09-18T11:33:18.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057953+0000", 2024-09-18T11:33:18.429 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.1399e-05 2024-09-18T11:33:18.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.429 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "dispatched", 2024-09-18T11:33:18.429 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057969+0000", 2024-09-18T11:33:18.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 1.5860999999999999e-05 2024-09-18T11:33:18.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.057975+0000", 2024-09-18T11:33:18.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 5.8610000000000003e-06 2024-09-18T11:33:18.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.430 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "reached_pg", 2024-09-18T11:33:18.430 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.058039+0000", 2024-09-18T11:33:18.430 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 6.4313000000000006e-05 2024-09-18T11:33:18.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.430 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "started", 2024-09-18T11:33:18.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.058059+0000", 2024-09-18T11:33:18.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 2.0205999999999999e-05 2024-09-18T11:33:18.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "commit_sent", 2024-09-18T11:33:18.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065410+0000", 2024-09-18T11:33:18.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 0.0073512020000000003 2024-09-18T11:33:18.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-18T11:33:18.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-09-18T11:33:18.431 INFO:teuthology.orchestra.run.smithi148.stdout: "event": "done", 2024-09-18T11:33:18.431 INFO:teuthology.orchestra.run.smithi148.stdout: "time": "2024-09-18T11:32:40.065451+0000", 2024-09-18T11:33:18.431 INFO:teuthology.orchestra.run.smithi148.stdout: "duration": 4.1032999999999998e-05 2024-09-18T11:33:18.432 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.432 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.432 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.432 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-18T11:33:18.432 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-18T11:33:18.432 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-18T11:33:18.433 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:33:18.544 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:18.544 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:33:18.544 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:33:18.544 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:18.553 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T11:33:18.651 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:33:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "complaint_time": 30, 2024-09-18T11:33:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "num_blocked_ops": 0 2024-09-18T11:33:18.652 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:18.659 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T11:33:18.758 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:33:18.758 INFO:teuthology.orchestra.run.smithi119.stdout: "size": 20, 2024-09-18T11:33:18.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 600, 2024-09-18T11:33:18.759 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [ 2024-09-18T11:33:18.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.759 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4736 4.3 4:c70c3162:::smithi11939506-35:head [write 574905~711397,stat] snapc 7c=[7c,7b,79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:33:18.759 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.031772+0000", 2024-09-18T11:33:18.759 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 29.724513529999999, 2024-09-18T11:33:18.759 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0092375840000000001, 2024-09-18T11:33:18.759 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:18.759 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.759 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:18.759 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:18.759 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:18.759 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4736 2024-09-18T11:33:18.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.760 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:18.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:18.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.031772+0000", 2024-09-18T11:33:18.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:18.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.031772+0000", 2024-09-18T11:33:18.760 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.760 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.760 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.760 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:18.760 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.031771+0000", 2024-09-18T11:33:18.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:18.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:18.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032335+0000", 2024-09-18T11:33:18.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00056435800000000005 2024-09-18T11:33:18.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:18.761 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032349+0000", 2024-09-18T11:33:18.761 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4353e-05 2024-09-18T11:33:18.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.761 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.761 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032356+0000", 2024-09-18T11:33:18.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3550000000000001e-06 2024-09-18T11:33:18.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032487+0000", 2024-09-18T11:33:18.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013154900000000001 2024-09-18T11:33:18.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:33:18.762 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032495+0000", 2024-09-18T11:33:18.762 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7519999999999996e-06 2024-09-18T11:33:18.762 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.762 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.762 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.035010+0000", 2024-09-18T11:33:18.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025153530000000001 2024-09-18T11:33:18.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:18.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.035142+0000", 2024-09-18T11:33:18.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00013133599999999999 2024-09-18T11:33:18.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:18.763 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.035330+0000", 2024-09-18T11:33:18.763 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00018873499999999999 2024-09-18T11:33:18.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.763 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.763 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:18.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040882+0000", 2024-09-18T11:33:18.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055514689999999998 2024-09-18T11:33:18.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:18.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040917+0000", 2024-09-18T11:33:18.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.5305999999999997e-05 2024-09-18T11:33:18.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.764 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:18.764 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.040968+0000", 2024-09-18T11:33:18.764 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0720000000000002e-05 2024-09-18T11:33:18.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.764 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.765 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:18.765 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041010+0000", 2024-09-18T11:33:18.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1893e-05 2024-09-18T11:33:18.765 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.765 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:18.765 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.765 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.765 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.765 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4737 4.3 4:c70c3162:::smithi11939506-35:head [write 1885086~525140] snapc 7c=[7c,7b,79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:33:18.765 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.032373+0000", 2024-09-18T11:33:18.765 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 29.723912477999999, 2024-09-18T11:33:18.765 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0088738949999999997, 2024-09-18T11:33:18.765 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:18.765 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.765 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:18.766 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:18.766 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:18.766 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4737 2024-09-18T11:33:18.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.766 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:18.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:18.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032373+0000", 2024-09-18T11:33:18.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.766 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.766 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:18.766 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032373+0000", 2024-09-18T11:33:18.766 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:18.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032372+0000", 2024-09-18T11:33:18.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:18.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:18.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032651+0000", 2024-09-18T11:33:18.767 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00027865499999999999 2024-09-18T11:33:18.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.767 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.767 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:18.767 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032661+0000", 2024-09-18T11:33:18.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.5440000000000004e-06 2024-09-18T11:33:18.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032665+0000", 2024-09-18T11:33:18.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9899999999999999e-06 2024-09-18T11:33:18.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.768 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.768 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032701+0000", 2024-09-18T11:33:18.768 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6587000000000001e-05 2024-09-18T11:33:18.768 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.768 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:33:18.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032708+0000", 2024-09-18T11:33:18.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.742e-06 2024-09-18T11:33:18.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.037089+0000", 2024-09-18T11:33:18.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0043810639999999996 2024-09-18T11:33:18.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.769 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.769 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:18.769 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.037176+0000", 2024-09-18T11:33:18.769 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7266000000000001e-05 2024-09-18T11:33:18.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:18.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.037317+0000", 2024-09-18T11:33:18.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00014054200000000001 2024-09-18T11:33:18.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.770 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:18.770 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041074+0000", 2024-09-18T11:33:18.770 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037575709999999999 2024-09-18T11:33:18.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.770 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:18.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041165+0000", 2024-09-18T11:33:18.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.0785999999999994e-05 2024-09-18T11:33:18.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:18.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041208+0000", 2024-09-18T11:33:18.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2611999999999998e-05 2024-09-18T11:33:18.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.771 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.771 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:18.771 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041247+0000", 2024-09-18T11:33:18.771 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9538e-05 2024-09-18T11:33:18.771 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.771 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:18.772 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.772 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4738 4.3 4:c70c3162:::smithi11939506-35:head [write 2930189~44716] snapc 7c=[7c,7b,79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:33:18.772 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.032671+0000", 2024-09-18T11:33:18.772 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 29.723614984000001, 2024-09-18T11:33:18.772 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0086678190000000002, 2024-09-18T11:33:18.772 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:18.772 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.772 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:18.772 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:18.772 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:18.772 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4738 2024-09-18T11:33:18.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.773 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:18.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:18.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032671+0000", 2024-09-18T11:33:18.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:18.773 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032671+0000", 2024-09-18T11:33:18.773 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.773 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.773 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:18.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032670+0000", 2024-09-18T11:33:18.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:18.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:18.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032908+0000", 2024-09-18T11:33:18.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00023724099999999999 2024-09-18T11:33:18.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.774 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:18.774 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032917+0000", 2024-09-18T11:33:18.774 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.1719999999999996e-06 2024-09-18T11:33:18.774 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.774 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032920+0000", 2024-09-18T11:33:18.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9270000000000001e-06 2024-09-18T11:33:18.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:33:18.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9938e-05 2024-09-18T11:33:18.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.775 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.775 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:33:18.775 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032957+0000", 2024-09-18T11:33:18.775 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7789999999999996e-06 2024-09-18T11:33:18.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.038135+0000", 2024-09-18T11:33:18.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0051778789999999998 2024-09-18T11:33:18.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:18.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.038219+0000", 2024-09-18T11:33:18.776 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4047000000000004e-05 2024-09-18T11:33:18.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.776 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.776 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:18.776 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.038413+0000", 2024-09-18T11:33:18.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00019343600000000001 2024-09-18T11:33:18.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:18.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041104+0000", 2024-09-18T11:33:18.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026916190000000001 2024-09-18T11:33:18.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.777 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.777 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:18.777 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041256+0000", 2024-09-18T11:33:18.777 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00015182100000000001 2024-09-18T11:33:18.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:18.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041300+0000", 2024-09-18T11:33:18.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.3915999999999998e-05 2024-09-18T11:33:18.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.778 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.778 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:18.778 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041339+0000", 2024-09-18T11:33:18.778 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8510000000000002e-05 2024-09-18T11:33:18.778 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.778 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:18.778 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.779 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.779 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4739 4.3 4:c70c3162:::smithi11939506-35:head [setxattr _header (57) in=7b,truncate 2974905] snapc 7c=[7c,7b,79,78,77,74,73,71,6f,6c,69,61,45,3e,3d] ondisk+write+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:33:18.779 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.032924+0000", 2024-09-18T11:33:18.779 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 29.723361357000002, 2024-09-18T11:33:18.779 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0085165610000000006, 2024-09-18T11:33:18.779 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:18.779 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.779 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:18.779 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:18.779 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:18.779 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4739 2024-09-18T11:33:18.779 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.779 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:18.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:18.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032924+0000", 2024-09-18T11:33:18.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:18.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032924+0000", 2024-09-18T11:33:18.780 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.780 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:18.780 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032924+0000", 2024-09-18T11:33:18.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:18.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:18.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032926+0000", 2024-09-18T11:33:18.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0619999999999999e-06 2024-09-18T11:33:18.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:18.781 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032935+0000", 2024-09-18T11:33:18.781 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.371e-06 2024-09-18T11:33:18.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.781 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.781 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032937+0000", 2024-09-18T11:33:18.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.6769999999999999e-06 2024-09-18T11:33:18.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032979+0000", 2024-09-18T11:33:18.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.1810000000000001e-05 2024-09-18T11:33:18.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.782 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:33:18.782 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032984+0000", 2024-09-18T11:33:18.782 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.1660000000000002e-06 2024-09-18T11:33:18.782 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039042+0000", 2024-09-18T11:33:18.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0060573399999999996 2024-09-18T11:33:18.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:18.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039127+0000", 2024-09-18T11:33:18.783 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5494999999999994e-05 2024-09-18T11:33:18.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.783 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.783 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for subops from 5", 2024-09-18T11:33:18.783 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039293+0000", 2024-09-18T11:33:18.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00016590999999999999 2024-09-18T11:33:18.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "sub_op_commit_rec", 2024-09-18T11:33:18.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041146+0000", 2024-09-18T11:33:18.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018531369999999999 2024-09-18T11:33:18.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.784 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op_commit", 2024-09-18T11:33:18.784 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041345+0000", 2024-09-18T11:33:18.784 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000199218 2024-09-18T11:33:18.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.784 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:18.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041390+0000", 2024-09-18T11:33:18.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4892000000000003e-05 2024-09-18T11:33:18.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.785 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.785 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:18.785 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041441+0000", 2024-09-18T11:33:18.785 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.0815000000000001e-05 2024-09-18T11:33:18.785 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.785 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:18.785 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.786 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.786 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4740 4.3 4:c70c3162:::smithi11939506-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e178)", 2024-09-18T11:33:18.786 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:33:18.786 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 29.723335695999999, 2024-09-18T11:33:18.786 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0087685269999999999, 2024-09-18T11:33:18.786 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:18.786 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:18.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:18.786 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:18.856 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:18.856 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4740 2024-09-18T11:33:18.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.856 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:18.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:33:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:33:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032950+0000", 2024-09-18T11:33:18.857 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032952+0000", 2024-09-18T11:33:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.215e-06 2024-09-18T11:33:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032956+0000", 2024-09-18T11:33:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.8850000000000001e-06 2024-09-18T11:33:18.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032958+0000", 2024-09-18T11:33:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.1840000000000002e-06 2024-09-18T11:33:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.032997+0000", 2024-09-18T11:33:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9161e-05 2024-09-18T11:33:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for peered", 2024-09-18T11:33:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.033000+0000", 2024-09-18T11:33:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.1030000000000002e-06 2024-09-18T11:33:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039536+0000", 2024-09-18T11:33:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.006535201 2024-09-18T11:33:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting for rw locks", 2024-09-18T11:33:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.039616+0000", 2024-09-18T11:33:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.0684000000000002e-05 2024-09-18T11:33:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041536+0000", 2024-09-18T11:33:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0019196440000000001 2024-09-18T11:33:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041610+0000", 2024-09-18T11:33:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.4295999999999995e-05 2024-09-18T11:33:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:18.861 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:49.041719+0000", 2024-09-18T11:33:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000108465 2024-09-18T11:33:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:4756 4.7 e179/174 4:e301aefd:::smithi11939506-19:head v 179'405, mlcod=173'401)", 2024-09-18T11:33:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:50.039068+0000", 2024-09-18T11:33:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.717217597000001, 2024-09-18T11:33:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0055292249999999996, 2024-09-18T11:33:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:18.862 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.039068+0000", 2024-09-18T11:33:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.039068+0000", 2024-09-18T11:33:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.863 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.039068+0000", 2024-09-18T11:33:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040038+0000", 2024-09-18T11:33:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00096998500000000003 2024-09-18T11:33:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040053+0000", 2024-09-18T11:33:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5481999999999998e-05 2024-09-18T11:33:18.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040062+0000", 2024-09-18T11:33:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7080000000000003e-06 2024-09-18T11:33:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040109+0000", 2024-09-18T11:33:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.6752000000000002e-05 2024-09-18T11:33:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:18.865 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.040124+0000", 2024-09-18T11:33:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.5715000000000001e-05 2024-09-18T11:33:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044568+0000", 2024-09-18T11:33:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0044430600000000004 2024-09-18T11:33:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044597+0000", 2024-09-18T11:33:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.9887999999999999e-05 2024-09-18T11:33:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:18.866 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:4757 4.7 e179/174 4:e301aefd:::smithi11939506-19:head v 179'406, mlcod=173'401)", 2024-09-18T11:33:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:50.044255+0000", 2024-09-18T11:33:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.712031035999999, 2024-09-18T11:33:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002073166, 2024-09-18T11:33:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044255+0000", 2024-09-18T11:33:18.867 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044255+0000", 2024-09-18T11:33:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044254+0000", 2024-09-18T11:33:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044875+0000", 2024-09-18T11:33:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00062136400000000001 2024-09-18T11:33:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044886+0000", 2024-09-18T11:33:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0709e-05 2024-09-18T11:33:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.869 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044891+0000", 2024-09-18T11:33:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8019999999999999e-06 2024-09-18T11:33:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044911+0000", 2024-09-18T11:33:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0729000000000001e-05 2024-09-18T11:33:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.044930+0000", 2024-09-18T11:33:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8890999999999999e-05 2024-09-18T11:33:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.870 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046295+0000", 2024-09-18T11:33:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0013645420000000001 2024-09-18T11:33:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.046328+0000", 2024-09-18T11:33:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3089000000000001e-05 2024-09-18T11:33:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_repop(client.4590.0:4764 3.3 e179/174 3:ec4378e2:::259b372f30a46a47845d5d9c58303dcbeccaeb2db18e8ed0152f1c66341c4df2:head v 179'156, mlcod=172'154)", 2024-09-18T11:33:18.871 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:50.059675+0000", 2024-09-18T11:33:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 28.696610683999999, 2024-09-18T11:33:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0056090439999999997, 2024-09-18T11:33:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T11:33:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.059675+0000", 2024-09-18T11:33:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.059675+0000", 2024-09-18T11:33:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.059675+0000", 2024-09-18T11:33:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062252+0000", 2024-09-18T11:33:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025772859999999998 2024-09-18T11:33:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062265+0000", 2024-09-18T11:33:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3059e-05 2024-09-18T11:33:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062272+0000", 2024-09-18T11:33:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.3589999999999996e-06 2024-09-18T11:33:18.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062317+0000", 2024-09-18T11:33:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5188999999999997e-05 2024-09-18T11:33:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.062334+0000", 2024-09-18T11:33:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.73e-05 2024-09-18T11:33:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.876 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "commit_sent", 2024-09-18T11:33:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.065251+0000", 2024-09-18T11:33:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0029169539999999998 2024-09-18T11:33:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:50.065284+0000", 2024-09-18T11:33:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.3206e-05 2024-09-18T11:33:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.877 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4792 4.6 4:67c1765a:::smithi11939506-30:head [sparse-read 0~1153827,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-18T11:33:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.053543+0000", 2024-09-18T11:33:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 25.702742962999999, 2024-09-18T11:33:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003529337, 2024-09-18T11:33:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4792 2024-09-18T11:33:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.878 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053543+0000", 2024-09-18T11:33:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053543+0000", 2024-09-18T11:33:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053541+0000", 2024-09-18T11:33:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.879 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053550+0000", 2024-09-18T11:33:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.2070000000000001e-06 2024-09-18T11:33:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053566+0000", 2024-09-18T11:33:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6098000000000001e-05 2024-09-18T11:33:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053575+0000", 2024-09-18T11:33:18.880 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.3769999999999996e-06 2024-09-18T11:33:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053620+0000", 2024-09-18T11:33:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.5404000000000001e-05 2024-09-18T11:33:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "op must wait for map", 2024-09-18T11:33:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053634+0000", 2024-09-18T11:33:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.4161e-05 2024-09-18T11:33:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.881 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.055502+0000", 2024-09-18T11:33:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0018678810000000001 2024-09-18T11:33:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.055629+0000", 2024-09-18T11:33:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00012652 2024-09-18T11:33:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057072+0000", 2024-09-18T11:33:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.001443567 2024-09-18T11:33:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.882 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4793 4.6 4:67c1765a:::smithi11939506-30:head [sparse-read 0~1153827] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-18T11:33:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.053593+0000", 2024-09-18T11:33:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 25.702693097000001, 2024-09-18T11:33:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037961539999999999, 2024-09-18T11:33:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:18.883 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4793 2024-09-18T11:33:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053593+0000", 2024-09-18T11:33:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053593+0000", 2024-09-18T11:33:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053592+0000", 2024-09-18T11:33:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053595+0000", 2024-09-18T11:33:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.2179999999999999e-06 2024-09-18T11:33:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053603+0000", 2024-09-18T11:33:18.885 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.576e-06 2024-09-18T11:33:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053607+0000", 2024-09-18T11:33:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2309999999999998e-06 2024-09-18T11:33:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053696+0000", 2024-09-18T11:33:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.9130999999999998e-05 2024-09-18T11:33:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.886 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053702+0000", 2024-09-18T11:33:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.6370000000000004e-06 2024-09-18T11:33:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057115+0000", 2024-09-18T11:33:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0034128100000000001 2024-09-18T11:33:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057176+0000", 2024-09-18T11:33:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.143e-05 2024-09-18T11:33:18.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057389+0000", 2024-09-18T11:33:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00021267700000000001 2024-09-18T11:33:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "osd_op(client.4590.0:4794 4.6 4:67c1765a:::smithi11939506-30:head [sparse-read 0~1153827] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-18T11:33:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:53.053613+0000", 2024-09-18T11:33:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 25.702672844999999, 2024-09-18T11:33:18.888 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0040685349999999999, 2024-09-18T11:33:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "started", 2024-09-18T11:33:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: "client_info": { 2024-09-18T11:33:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: "client": "client.4590", 2024-09-18T11:33:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: "client_addr": "172.21.15.119:0/3243439143", 2024-09-18T11:33:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: "tid": 4794 2024-09-18T11:33:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053613+0000", 2024-09-18T11:33:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053613+0000", 2024-09-18T11:33:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053612+0000", 2024-09-18T11:33:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999995 2024-09-18T11:33:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053615+0000", 2024-09-18T11:33:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.5730000000000002e-06 2024-09-18T11:33:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053628+0000", 2024-09-18T11:33:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3084e-05 2024-09-18T11:33:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053632+0000", 2024-09-18T11:33:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9620000000000004e-06 2024-09-18T11:33:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.891 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053731+0000", 2024-09-18T11:33:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.8547000000000004e-05 2024-09-18T11:33:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "waiting_for_map not empty", 2024-09-18T11:33:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.053736+0000", 2024-09-18T11:33:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.767e-06 2024-09-18T11:33:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057461+0000", 2024-09-18T11:33:18.892 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0037242780000000001 2024-09-18T11:33:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "started", 2024-09-18T11:33:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057538+0000", 2024-09-18T11:33:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.7008999999999994e-05 2024-09-18T11:33:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:53.057681+0000", 2024-09-18T11:33:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.000143786 2024-09-18T11:33:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:18.893 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e301aefd:::smithi11939506-19:3f)])", 2024-09-18T11:33:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.083144+0000", 2024-09-18T11:33:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.673142236, 2024-09-18T11:33:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024893160000000001, 2024-09-18T11:33:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.083144+0000", 2024-09-18T11:33:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.083144+0000", 2024-09-18T11:33:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.083143+0000", 2024-09-18T11:33:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:18.895 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.083149+0000", 2024-09-18T11:33:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.0549999999999996e-06 2024-09-18T11:33:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.083161+0000", 2024-09-18T11:33:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.2668e-05 2024-09-18T11:33:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.083169+0000", 2024-09-18T11:33:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.2830000000000004e-06 2024-09-18T11:33:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.896 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.083208+0000", 2024-09-18T11:33:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.9749999999999997e-05 2024-09-18T11:33:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.085633+0000", 2024-09-18T11:33:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0024244639999999999 2024-09-18T11:33:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.897 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e0d95e10:::smithi11939506-22:49)])", 2024-09-18T11:33:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.102811+0000", 2024-09-18T11:33:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.653475124, 2024-09-18T11:33:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.00260509, 2024-09-18T11:33:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.102811+0000", 2024-09-18T11:33:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.102811+0000", 2024-09-18T11:33:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.102809+0000", 2024-09-18T11:33:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.102818+0000", 2024-09-18T11:33:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 9.4089999999999992e-06 2024-09-18T11:33:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:18.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.102836+0000", 2024-09-18T11:33:18.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.7881999999999999e-05 2024-09-18T11:33:18.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.102844+0000", 2024-09-18T11:33:18.900 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.8900000000000007e-06 2024-09-18T11:33:18.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.900 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.900 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.900 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.102892+0000", 2024-09-18T11:33:18.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7756999999999997e-05 2024-09-18T11:33:18.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.901 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:18.901 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.105416+0000", 2024-09-18T11:33:18.901 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025237710000000002 2024-09-18T11:33:18.901 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.901 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:18.901 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.901 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.901 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e73ba68f:::smithi11939506-1:42)])", 2024-09-18T11:33:18.901 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.114167+0000", 2024-09-18T11:33:18.902 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.642119219000001, 2024-09-18T11:33:18.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002907303, 2024-09-18T11:33:18.902 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:18.902 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:18.902 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:18.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:18.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.114167+0000", 2024-09-18T11:33:18.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.902 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.902 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:18.902 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.114167+0000", 2024-09-18T11:33:18.902 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:18.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.114165+0000", 2024-09-18T11:33:18.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:18.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:18.903 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.114174+0000", 2024-09-18T11:33:18.903 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.4419999999999998e-06 2024-09-18T11:33:18.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.903 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.903 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:18.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.114192+0000", 2024-09-18T11:33:18.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.8346e-05 2024-09-18T11:33:18.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.114200+0000", 2024-09-18T11:33:18.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9680000000000006e-06 2024-09-18T11:33:18.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.904 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.904 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.114249+0000", 2024-09-18T11:33:18.904 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.8974000000000002e-05 2024-09-18T11:33:18.904 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.904 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.905 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:18.905 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.117074+0000", 2024-09-18T11:33:18.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0028247530000000002 2024-09-18T11:33:18.905 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.905 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:18.905 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.905 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e5707c18:::smithi11939506-10:5d)])", 2024-09-18T11:33:18.905 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.126498+0000", 2024-09-18T11:33:18.905 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.629787942, 2024-09-18T11:33:18.905 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0033596170000000001, 2024-09-18T11:33:18.905 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:18.905 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:18.906 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:18.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:18.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.126498+0000", 2024-09-18T11:33:18.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:18.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.126498+0000", 2024-09-18T11:33:18.906 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.906 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.906 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:18.906 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.126497+0000", 2024-09-18T11:33:18.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999986 2024-09-18T11:33:18.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:18.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.126508+0000", 2024-09-18T11:33:18.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1898e-05 2024-09-18T11:33:18.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.907 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:18.907 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.126525+0000", 2024-09-18T11:33:18.907 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.6761000000000001e-05 2024-09-18T11:33:18.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.907 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.126533+0000", 2024-09-18T11:33:18.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 7.9629999999999998e-06 2024-09-18T11:33:18.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.126581+0000", 2024-09-18T11:33:18.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7787000000000002e-05 2024-09-18T11:33:18.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.908 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:18.908 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.129857+0000", 2024-09-18T11:33:18.908 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.003276476 2024-09-18T11:33:18.908 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.909 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:18.909 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.909 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.909 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e73ba68f:::smithi11939506-1:5d)])", 2024-09-18T11:33:18.909 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.141236+0000", 2024-09-18T11:33:18.909 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.615049380999999, 2024-09-18T11:33:18.909 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002457187, 2024-09-18T11:33:18.909 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:18.909 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:18.909 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:18.909 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.909 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:18.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.141236+0000", 2024-09-18T11:33:18.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:18.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.141236+0000", 2024-09-18T11:33:18.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.910 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.910 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:18.910 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.141235+0000", 2024-09-18T11:33:18.910 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.9999981 2024-09-18T11:33:18.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:18.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.141248+0000", 2024-09-18T11:33:18.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3131e-05 2024-09-18T11:33:18.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:18.911 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.141268+0000", 2024-09-18T11:33:18.911 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 2.0689000000000001e-05 2024-09-18T11:33:18.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.911 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.911 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.141277+0000", 2024-09-18T11:33:18.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.7280000000000001e-06 2024-09-18T11:33:18.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.141331+0000", 2024-09-18T11:33:18.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 5.4098000000000003e-05 2024-09-18T11:33:18.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.912 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.912 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:18.912 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.143694+0000", 2024-09-18T11:33:18.912 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023623810000000002 2024-09-18T11:33:18.913 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.913 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:18.913 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.913 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.913 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:ef60946b:::smithi11939506-8:head)])", 2024-09-18T11:33:18.913 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.151354+0000", 2024-09-18T11:33:18.913 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.604931350000001, 2024-09-18T11:33:18.913 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026499959999999999, 2024-09-18T11:33:18.913 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:18.913 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:18.913 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:18.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:18.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.151354+0000", 2024-09-18T11:33:18.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:18.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.151354+0000", 2024-09-18T11:33:18.914 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.914 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.914 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.914 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:18.914 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.151354+0000", 2024-09-18T11:33:18.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:18.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:18.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.151360+0000", 2024-09-18T11:33:18.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8120000000000002e-06 2024-09-18T11:33:18.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:18.915 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.151371+0000", 2024-09-18T11:33:18.915 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.0324e-05 2024-09-18T11:33:18.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.915 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.915 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.151375+0000", 2024-09-18T11:33:18.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7049999999999998e-06 2024-09-18T11:33:18.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.151420+0000", 2024-09-18T11:33:18.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.4496000000000002e-05 2024-09-18T11:33:18.916 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.916 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.916 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:18.916 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.154004+0000", 2024-09-18T11:33:18.916 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025845579999999998 2024-09-18T11:33:18.917 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.917 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:18.917 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.917 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e0d95e10:::smithi11939506-22:74)])", 2024-09-18T11:33:18.917 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.169218+0000", 2024-09-18T11:33:18.917 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.58706797, 2024-09-18T11:33:18.917 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0023236060000000002, 2024-09-18T11:33:18.917 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:18.917 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:18.917 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:18.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:18.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.169218+0000", 2024-09-18T11:33:18.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:18.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.169218+0000", 2024-09-18T11:33:18.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.918 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.918 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:18.918 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.169217+0000", 2024-09-18T11:33:18.918 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:18.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:18.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.169224+0000", 2024-09-18T11:33:18.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.9129999999999997e-06 2024-09-18T11:33:18.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:18.919 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.169235+0000", 2024-09-18T11:33:18.919 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1032e-05 2024-09-18T11:33:18.919 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.919 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.919 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.169242+0000", 2024-09-18T11:33:18.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.8589999999999999e-06 2024-09-18T11:33:18.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.169284+0000", 2024-09-18T11:33:18.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.2293000000000003e-05 2024-09-18T11:33:18.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.920 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.920 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:18.920 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.171541+0000", 2024-09-18T11:33:18.920 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0022575939999999999 2024-09-18T11:33:18.920 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.921 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:18.921 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.921 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e301aefd:::smithi11939506-19:7e)])", 2024-09-18T11:33:18.921 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.179260+0000", 2024-09-18T11:33:18.921 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.577025817999999, 2024-09-18T11:33:18.921 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025394670000000001, 2024-09-18T11:33:18.921 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:18.921 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:18.921 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:18.921 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.921 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:18.921 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.179260+0000", 2024-09-18T11:33:18.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:18.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.179260+0000", 2024-09-18T11:33:18.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:18.922 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.179259+0000", 2024-09-18T11:33:18.922 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:18.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.922 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.922 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:18.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.179265+0000", 2024-09-18T11:33:18.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.1879999999999999e-06 2024-09-18T11:33:18.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:18.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.179276+0000", 2024-09-18T11:33:18.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.1070000000000001e-05 2024-09-18T11:33:18.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.923 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.923 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.179283+0000", 2024-09-18T11:33:18.923 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.6059999999999999e-06 2024-09-18T11:33:18.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.923 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.179320+0000", 2024-09-18T11:33:18.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 3.6847999999999997e-05 2024-09-18T11:33:18.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.924 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:18.924 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.181799+0000", 2024-09-18T11:33:18.924 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.002479738 2024-09-18T11:33:18.924 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.924 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:18.924 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.924 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.925 INFO:teuthology.orchestra.run.smithi119.stdout: "description": "MOSDPGPushReply(4.7 188/187 [PushReplyOp(4:e73ba68f:::smithi11939506-1:head)])", 2024-09-18T11:33:18.925 INFO:teuthology.orchestra.run.smithi119.stdout: "initiated_at": "2024-09-18T11:32:59.201696+0000", 2024-09-18T11:33:18.925 INFO:teuthology.orchestra.run.smithi119.stdout: "age": 19.554589499999999, 2024-09-18T11:33:18.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0026547110000000001, 2024-09-18T11:33:18.925 INFO:teuthology.orchestra.run.smithi119.stdout: "type_data": { 2024-09-18T11:33:18.925 INFO:teuthology.orchestra.run.smithi119.stdout: "flag_point": "reached pg", 2024-09-18T11:33:18.925 INFO:teuthology.orchestra.run.smithi119.stdout: "events": [ 2024-09-18T11:33:18.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.925 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "initiated", 2024-09-18T11:33:18.925 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.201696+0000", 2024-09-18T11:33:18.925 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "throttled", 2024-09-18T11:33:18.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.201696+0000", 2024-09-18T11:33:18.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0 2024-09-18T11:33:18.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "header_read", 2024-09-18T11:33:18.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.201695+0000", 2024-09-18T11:33:18.926 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4294967295.999999 2024-09-18T11:33:18.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.926 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.926 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "all_read", 2024-09-18T11:33:18.926 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.201704+0000", 2024-09-18T11:33:18.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 8.5779999999999999e-06 2024-09-18T11:33:18.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "dispatched", 2024-09-18T11:33:18.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.201717+0000", 2024-09-18T11:33:18.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 1.3019e-05 2024-09-18T11:33:18.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.927 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.927 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "queued_for_pg", 2024-09-18T11:33:18.927 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.201723+0000", 2024-09-18T11:33:18.927 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 6.2299999999999996e-06 2024-09-18T11:33:18.927 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "reached_pg", 2024-09-18T11:33:18.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.201771+0000", 2024-09-18T11:33:18.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 4.7652999999999999e-05 2024-09-18T11:33:18.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-18T11:33:18.928 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-09-18T11:33:18.928 INFO:teuthology.orchestra.run.smithi119.stdout: "event": "done", 2024-09-18T11:33:18.928 INFO:teuthology.orchestra.run.smithi119.stdout: "time": "2024-09-18T11:32:59.204351+0000", 2024-09-18T11:33:18.928 INFO:teuthology.orchestra.run.smithi119.stdout: "duration": 0.0025802009999999998 2024-09-18T11:33:18.928 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.928 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:18.928 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.929 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-18T11:33:18.929 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-18T11:33:18.929 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:33:19.131 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T11:33:20.133 INFO:teuthology.orchestra.run.smithi119.stderr:noscrub is unset 2024-09-18T11:33:20.150 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T11:33:20.192 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:33:20.190+0000 7fcb662ad640 -1 osd.7 198 handle_osd_map injecting map crc failure 2024-09-18T11:33:21.135 INFO:teuthology.orchestra.run.smithi119.stderr:nodeep-scrub is unset 2024-09-18T11:33:23.032 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:23.030+0000 7f697ceb1640 -1 osd.3 200 handle_osd_map injecting map crc failure 2024-09-18T11:33:23.032 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:33:23.031+0000 7fb6950ff640 -1 osd.2 200 handle_osd_map injecting map crc failure 2024-09-18T11:33:24.454 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T11:33:24.454 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:33:24.724 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:33:24.724 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:33:24.735 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-18T11:33:23.957922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":142099530,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":748,"num_read_kb":25515,"num_write":887,"num_write_kb":266346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":141445629,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2613796,"kb_used_data":191676,"kb_used_omap":28,"kb_used_meta":2421731,"kb_avail":752360924,"statfs":{"total":773094113280,"available":770417586176,"internally_reserved":0,"allocated":196276224,"data_stored":193066817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29629,"internal_metadata":2479852611},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001434"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":330,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.193757+0000","last_change":"2024-09-18T11:32:46.462271+0000","last_active":"2024-09-18T11:33:20.193757+0000","last_peered":"2024-09-18T11:33:20.193757+0000","last_clean":"2024-09-18T11:33:20.193757+0000","last_became_active":"2024-09-18T11:32:46.456653+0000","last_became_peered":"2024-09-18T11:32:46.456653+0000","last_unstale":"2024-09-18T11:33:20.193757+0000","last_undegraded":"2024-09-18T11:33:20.193757+0000","last_fullsized":"2024-09-18T11:33:20.193757+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'72","last_scrub_stamp":"2024-09-18T11:32:29.173114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_clean_scrub_stamp":"2024-09-18T11:32:29.173114+0000","objects_scrubbed":2,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:47.098458+0000","scrub_duration":0.0016253019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":240,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.908038+0000","last_change":"2024-09-18T11:32:42.123653+0000","last_active":"2024-09-18T11:33:20.908038+0000","last_peered":"2024-09-18T11:33:20.908038+0000","last_clean":"2024-09-18T11:33:20.908038+0000","last_became_active":"2024-09-18T11:32:27.079772+0000","last_became_peered":"2024-09-18T11:32:27.079772+0000","last_unstale":"2024-09-18T11:33:20.908038+0000","last_undegraded":"2024-09-18T11:33:20.908038+0000","last_fullsized":"2024-09-18T11:33:20.908038+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'1","last_scrub_stamp":"2024-09-18T11:32:42.123530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:32:42.123530+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:04.343044+0000","scrub_duration":0.0016360910000000001,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"182'203","reported_seq":449,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.132212+0000","last_change":"2024-09-18T11:32:39.596000+0000","last_active":"2024-09-18T11:33:20.132212+0000","last_peered":"2024-09-18T11:33:20.132212+0000","last_clean":"2024-09-18T11:33:20.132212+0000","last_became_active":"2024-09-18T11:30:01.077612+0000","last_became_peered":"2024-09-18T11:30:01.077612+0000","last_unstale":"2024-09-18T11:33:20.132212+0000","last_undegraded":"2024-09-18T11:33:20.132212+0000","last_fullsized":"2024-09-18T11:33:20.132212+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'194","last_scrub_stamp":"2024-09-18T11:32:39.595890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:32:39.595890+0000","objects_scrubbed":49,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:05.467884+0000","scrub_duration":0.0058412239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42108006,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":185,"num_read_kb":4788,"num_write":230,"num_write_kb":78760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":222,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-18T11:33:21.778887+0000","last_change":"2024-09-18T11:33:21.778858+0000","last_active":"2024-09-18T11:33:21.778887+0000","last_peered":"2024-09-18T11:33:21.778887+0000","last_clean":"2024-09-18T11:33:21.778887+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:33:21.778887+0000","last_undegraded":"2024-09-18T11:33:21.778887+0000","last_fullsized":"2024-09-18T11:33:21.778887+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:21.778728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:21.778728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:33.603349+0000","scrub_duration":0.0020274260000000001,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"173'206","reported_seq":882,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T11:33:15.460786+0000","last_change":"2024-09-18T11:33:01.261204+0000","last_active":"2024-09-18T11:33:15.460786+0000","last_peered":"2024-09-18T11:33:15.460786+0000","last_clean":"2024-09-18T11:33:15.460786+0000","last_became_active":"2024-09-18T11:33:01.060996+0000","last_became_peered":"2024-09-18T11:33:01.060996+0000","last_unstale":"2024-09-18T11:33:15.460786+0000","last_undegraded":"2024-09-18T11:33:15.460786+0000","last_fullsized":"2024-09-18T11:33:15.460786+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'206","last_scrub_stamp":"2024-09-18T11:32:51.136969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:32:51.136969+0000","objects_scrubbed":46,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:00.923271+0000","scrub_duration":0.0050590330000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43270431,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":215,"num_read_kb":11081,"num_write":225,"num_write_kb":76253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":74278671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":127,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.132197+0000","last_change":"2024-09-18T11:32:59.475277+0000","last_active":"2024-09-18T11:33:20.132197+0000","last_peered":"2024-09-18T11:33:20.132197+0000","last_clean":"2024-09-18T11:33:20.132197+0000","last_became_active":"2024-09-18T11:32:59.474959+0000","last_became_peered":"2024-09-18T11:32:59.474959+0000","last_unstale":"2024-09-18T11:33:20.132197+0000","last_undegraded":"2024-09-18T11:33:20.132197+0000","last_fullsized":"2024-09-18T11:33:20.132197+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:32:32.611633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:32:32.611633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:43.724465+0000","scrub_duration":0.001364355,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"172'146","reported_seq":616,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.203502+0000","last_change":"2024-09-18T11:32:59.712706+0000","last_active":"2024-09-18T11:33:20.203502+0000","last_peered":"2024-09-18T11:33:20.203502+0000","last_clean":"2024-09-18T11:33:20.203502+0000","last_became_active":"2024-09-18T11:32:59.476088+0000","last_became_peered":"2024-09-18T11:32:59.476088+0000","last_unstale":"2024-09-18T11:33:20.203502+0000","last_undegraded":"2024-09-18T11:33:20.203502+0000","last_fullsized":"2024-09-18T11:33:20.203502+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"144'136","last_scrub_stamp":"2024-09-18T11:32:20.808182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:32:20.808182+0000","objects_scrubbed":29,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:24.436066+0000","scrub_duration":0.0050914910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24142412,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":133,"num_read_kb":1720,"num_write":158,"num_write_kb":46975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":29291946,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":215,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.908071+0000","last_change":"2024-09-18T11:33:04.151627+0000","last_active":"2024-09-18T11:33:20.908071+0000","last_peered":"2024-09-18T11:33:20.908071+0000","last_clean":"2024-09-18T11:33:20.908071+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:33:20.908071+0000","last_undegraded":"2024-09-18T11:33:20.908071+0000","last_fullsized":"2024-09-18T11:33:20.908071+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"18'2","last_scrub_stamp":"2024-09-18T11:33:04.151495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:04.151495+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:24.817971+0000","scrub_duration":0.0017471489999999999,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"179'157","reported_seq":613,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.905163+0000","last_change":"2024-09-18T11:32:46.707704+0000","last_active":"2024-09-18T11:33:20.905163+0000","last_peered":"2024-09-18T11:33:20.905163+0000","last_clean":"2024-09-18T11:33:20.905163+0000","last_became_active":"2024-09-18T11:32:46.456572+0000","last_became_peered":"2024-09-18T11:32:46.456572+0000","last_unstale":"2024-09-18T11:33:20.905163+0000","last_undegraded":"2024-09-18T11:33:20.905163+0000","last_fullsized":"2024-09-18T11:33:20.905163+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"166'152","last_scrub_stamp":"2024-09-18T11:32:42.852232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:32:42.852232+0000","objects_scrubbed":33,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:04.691723+0000","scrub_duration":0.0039393850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31988294,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":149,"num_read_kb":7871,"num_write":171,"num_write_kb":62728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":36817156,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":236,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.203483+0000","last_change":"2024-09-18T11:32:46.457112+0000","last_active":"2024-09-18T11:33:20.203483+0000","last_peered":"2024-09-18T11:33:20.203483+0000","last_clean":"2024-09-18T11:33:20.203483+0000","last_became_active":"2024-09-18T11:32:46.456819+0000","last_became_peered":"2024-09-18T11:32:46.456819+0000","last_unstale":"2024-09-18T11:33:20.203483+0000","last_undegraded":"2024-09-18T11:33:20.203483+0000","last_fullsized":"2024-09-18T11:33:20.203483+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:32:30.705704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:32:30.705704+0000","last_clean_scrub_stamp":"2024-09-18T11:32:30.705704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:42.193160+0000","scrub_duration":0.0039850399999999996,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":232,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.193709+0000","last_change":"2024-09-18T11:32:46.456500+0000","last_active":"2024-09-18T11:33:20.193709+0000","last_peered":"2024-09-18T11:33:20.193709+0000","last_clean":"2024-09-18T11:33:20.193709+0000","last_became_active":"2024-09-18T11:32:46.456211+0000","last_became_peered":"2024-09-18T11:32:46.456211+0000","last_unstale":"2024-09-18T11:33:20.193709+0000","last_undegraded":"2024-09-18T11:33:20.193709+0000","last_fullsized":"2024-09-18T11:33:20.193709+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:32:42.223963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:32:42.223963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:12.013506+0000","scrub_duration":0.001097408,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.130106+0000","last_change":"2024-09-18T11:32:59.475270+0000","last_active":"2024-09-18T11:33:20.130106+0000","last_peered":"2024-09-18T11:33:20.130106+0000","last_clean":"2024-09-18T11:33:20.130106+0000","last_became_active":"2024-09-18T11:32:59.474991+0000","last_became_peered":"2024-09-18T11:32:59.474991+0000","last_unstale":"2024-09-18T11:33:20.130106+0000","last_undegraded":"2024-09-18T11:33:20.130106+0000","last_fullsized":"2024-09-18T11:33:20.130106+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:32:28.708882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:32:28.708882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:55.766077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":212,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.130121+0000","last_change":"2024-09-18T11:32:06.213366+0000","last_active":"2024-09-18T11:33:20.130121+0000","last_peered":"2024-09-18T11:33:20.130121+0000","last_clean":"2024-09-18T11:33:20.130121+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:33:20.130121+0000","last_undegraded":"2024-09-18T11:33:20.130121+0000","last_fullsized":"2024-09-18T11:33:20.130121+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:32:06.213281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:32:06.213281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:23.965948+0000","scrub_duration":0.0013619280000000001,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":141509143,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":682,"num_read_kb":25460,"num_write":784,"num_write_kb":264716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":140387773,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":205524992,"data_stored":205034291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4182,"internal_metadata":0},"log_size":712,"ondisk_log_size":712,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302812,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069028,"statfs":{"total":96636764160,"available":96326684672,"internally_reserved":0,"allocated":1323008,"data_stored":975864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.54300000000000004}]},{"osd":1,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45300000000000001,"15min":0.46300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.65300000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.505,"15min":0.51800000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.754,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61199999999999999}]},{"osd":2,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.29199999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.503},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42899999999999999,"15min":0.435},"min":{"1min":0.317,"5min":0.314,"15min":0.314},"max":{"1min":0.52100000000000002,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.58399999999999996}]},{"osd":3,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.439,"15min":0.442},"min":{"1min":0.36499999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.55100000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.47099999999999997,"15min":0.48599999999999999},"min":{"1min":0.311,"5min":0.308,"15min":0.308},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.626}]},{"osd":4,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.439,"15min":0.45700000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.626,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.437,"5min":0.47799999999999998,"15min":0.498},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.624,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.66100000000000003}]},{"osd":5,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.41899999999999998,"15min":0.44},"min":{"1min":0.24399999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.71199999999999997,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.442,"15min":0.46500000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.72399999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.68100000000000005}]},{"osd":6,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.5,"15min":0.52300000000000002},"min":{"1min":0.28699999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48599999999999999,"15min":0.502},"min":{"1min":0.29399999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69099999999999995,"5min":0.75,"15min":0.75},"last":0.69199999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302264,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069576,"statfs":{"total":96636764160,"available":96327245824,"internally_reserved":0,"allocated":765952,"data_stored":441689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.52700000000000002}]},{"osd":1,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.46600000000000003,"15min":0.46899999999999997},"min":{"1min":0.379,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.61399999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.45400000000000001,"15min":0.45600000000000002},"min":{"1min":0.32800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.69499999999999995,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.58399999999999996}]},{"osd":2,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46100000000000002,"15min":0.46300000000000002},"min":{"1min":0.312,"5min":0.221,"15min":0.221},"max":{"1min":0.64700000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.45300000000000001,"15min":0.44700000000000001},"min":{"1min":0.34300000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.59899999999999998}]},{"osd":3,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.49299999999999999,"15min":0.51100000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.67100000000000004,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.505,"15min":0.51900000000000002},"min":{"1min":0.33900000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69699999999999995,"5min":1.032,"15min":1.032},"last":0.69499999999999995}]},{"osd":4,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.40100000000000002,"15min":0.39300000000000002},"min":{"1min":0.316,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.61299999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.437,"15min":0.42499999999999999},"min":{"1min":0.41299999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65400000000000003,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.73599999999999999}]},{"osd":5,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.41499999999999998,"15min":0.40899999999999997},"min":{"1min":0.29599999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66700000000000004,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.441,"15min":0.433},"min":{"1min":0.34799999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.66500000000000004}]},{"osd":7,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.46500000000000002,"15min":0.46400000000000002},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.48399999999999999,"15min":0.47799999999999998},"min":{"1min":0.34599999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76600000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.75900000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332320,"kb_used_data":26640,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94039520,"statfs":{"total":96636764160,"available":96296468480,"internally_reserved":0,"allocated":27279360,"data_stored":26860362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.56599999999999995}]},{"osd":1,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42399999999999999,"15min":0.42999999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.44700000000000001,"15min":0.44900000000000001},"min":{"1min":0.248,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.44900000000000001}]},{"osd":2,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47999999999999998,"15min":0.496},"min":{"1min":0.316,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66600000000000004,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.47199999999999998,"15min":0.49399999999999999},"min":{"1min":0.32400000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.59599999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.52900000000000003}]},{"osd":3,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.46700000000000003,"15min":0.48899999999999999},"min":{"1min":0.29899999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.51300000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.48399999999999999,"15min":0.496},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.61299999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.46899999999999997}]},{"osd":4,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42999999999999999,"15min":0.44400000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.435,"15min":0.45700000000000002},"min":{"1min":0.22900000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.65700000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.48199999999999998}]},{"osd":6,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.25600000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.46700000000000003,"15min":0.48599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.49399999999999999}]},{"osd":7,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.47499999999999998,"15min":0.499},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.72899999999999998,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.503,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.57699999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356576,"kb_used_data":50900,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94015264,"statfs":{"total":96636764160,"available":96271630336,"internally_reserved":0,"allocated":52121600,"data_stored":51665786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.57599999999999996}]},{"osd":1,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.47899999999999998,"15min":0.501},"min":{"1min":0.315,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.65300000000000002,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48399999999999999,"15min":0.498},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73299999999999998,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.73599999999999999}]},{"osd":2,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48099999999999998,"15min":0.49299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.748,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.49199999999999999,"15min":0.51600000000000001},"min":{"1min":0.35399999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72599999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.63600000000000001}]},{"osd":3,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.48899999999999999,"15min":0.51300000000000001},"min":{"1min":0.35799999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.61399999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.50600000000000001,"15min":0.52700000000000002},"min":{"1min":0.378,"5min":0.316,"15min":0.316},"max":{"1min":0.68799999999999994,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.59099999999999997}]},{"osd":5,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.442,"15min":0.46200000000000002},"min":{"1min":0.27700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.70699999999999996,"5min":1.002,"15min":1.002},"last":0.439},{"interface":"front","average":{"1min":0.442,"5min":0.47999999999999998,"15min":0.50800000000000001},"min":{"1min":0.30199999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.70299999999999996,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.41999999999999998}]},{"osd":6,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.45000000000000001,"15min":0.46899999999999997},"min":{"1min":0.28199999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.69399999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48399999999999999,"15min":0.504},"min":{"1min":0.33200000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.746,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.50600000000000001}]},{"osd":7,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.49199999999999999,"15min":0.51000000000000001},"min":{"1min":0.34899999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.54500000000000004},"min":{"1min":0.34100000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66700000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":328180,"kb_used_data":26660,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94043660,"statfs":{"total":96636764160,"available":96300707840,"internally_reserved":0,"allocated":27299840,"data_stored":26863829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.51200000000000001}]},{"osd":1,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.33100000000000002,"15min":0.34100000000000003},"min":{"1min":0.246,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.45500000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.36799999999999999},"min":{"1min":0.224,"5min":0.214,"15min":0.214},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.63800000000000001}]},{"osd":2,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.27800000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.377,"15min":0.39900000000000002},"min":{"1min":0.28100000000000003,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.48699999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.73399999999999999}]},{"osd":4,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.50700000000000001,"15min":0.51300000000000001},"min":{"1min":0.39300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.64600000000000002,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52000000000000002,"15min":0.52400000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.88900000000000001}]},{"osd":5,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.49199999999999999,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69399999999999995,"5min":0.752,"15min":0.752},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53100000000000003,"15min":0.55500000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67000000000000004,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.94899999999999995}]},{"osd":6,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.51300000000000001,"15min":0.53300000000000003},"min":{"1min":0.35999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.60899999999999999,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.504,"5min":0.54200000000000004,"15min":0.56100000000000005},"min":{"1min":0.39800000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.66500000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.91800000000000004}]},{"osd":7,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56999999999999995,"15min":0.60099999999999998},"min":{"1min":0.41599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68899999999999995,"5min":1.26,"15min":1.26},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.54500000000000004,"15min":0.57199999999999995},"min":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.63,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.86099999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318532,"kb_used_data":17016,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94053308,"statfs":{"total":96636764160,"available":96310587392,"internally_reserved":0,"allocated":17424384,"data_stored":17038411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.48599999999999999}]},{"osd":1,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.35599999999999998,"15min":0.375},"min":{"1min":0.23200000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.57099999999999995,"5min":1.621,"15min":1.621},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.377,"15min":0.39900000000000002},"min":{"1min":0.27100000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.70599999999999996,"5min":1.589,"15min":1.589},"last":0.52200000000000002}]},{"osd":3,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35799999999999998,"15min":0.36799999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.72399999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.39200000000000002,"15min":0.41399999999999998},"min":{"1min":0.27500000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52900000000000003,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.61899999999999999}]},{"osd":4,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53100000000000003,"15min":0.55100000000000005},"min":{"1min":0.373,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.755,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.504,"15min":0.51700000000000002},"min":{"1min":0.34300000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73199999999999998,"5min":1.012,"15min":1.012},"last":0.66500000000000004}]},{"osd":5,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53000000000000003,"15min":0.54200000000000004},"min":{"1min":0.36799999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70799999999999996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.56100000000000005},"min":{"1min":0.377,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77900000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.71899999999999997}]},{"osd":6,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52400000000000002,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82399999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.5,"5min":0.53600000000000003,"15min":0.56100000000000005},"min":{"1min":0.35499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65200000000000002,"5min":1.165,"15min":1.165},"last":0.77300000000000002}]},{"osd":7,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54400000000000004,"15min":0.56000000000000005},"min":{"1min":0.371,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70099999999999996,"5min":1.073,"15min":1.073},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.39700000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91600000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.82399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322692,"kb_used_data":17016,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94049148,"statfs":{"total":96636764160,"available":96306327552,"internally_reserved":0,"allocated":17424384,"data_stored":17037068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.24199999999999999}]},{"osd":2,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.38100000000000001},"min":{"1min":0.28699999999999998,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.871,"5min":1.333,"15min":1.333},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.40000000000000002,"15min":0.42099999999999999},"min":{"1min":0.23899999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.89200000000000002,"5min":1.298,"15min":1.298},"last":0.39900000000000002}]},{"osd":3,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.371,"15min":0.38200000000000001},"min":{"1min":0.27300000000000002,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.35599999999999998,"15min":0.36199999999999999},"min":{"1min":0.22500000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.35199999999999998}]},{"osd":4,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51500000000000001,"15min":0.52500000000000002},"min":{"1min":0.307,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.34999999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.96899999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.48399999999999999}]},{"osd":5,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.50700000000000001,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.91800000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.52500000000000002},"min":{"1min":0.32300000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.436}]},{"osd":6,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.34000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53200000000000003,"15min":0.54300000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.94499999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.45900000000000002}]},{"osd":7,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53700000000000003,"15min":0.54700000000000004},"min":{"1min":0.36299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56599999999999995,"15min":0.57899999999999996},"min":{"1min":0.34200000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.55800000000000005}]}]},{"osd":0,"up_from":119,"seq":511101108244,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":350420,"kb_used_data":51404,"kb_used_omap":17,"kb_used_meta":298734,"kb_avail":94021420,"statfs":{"total":96636764160,"available":96277934080,"internally_reserved":0,"allocated":52637696,"data_stored":52183808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17701,"internal_metadata":305904347},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.39200000000000002}]},{"osd":2,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.37,"5min":0.37,"15min":0.37},"last":0.31}]},{"osd":3,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.311,"15min":0.311},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.54400000000000004}]},{"osd":4,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.75900000000000001}]},{"osd":5,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.68600000000000005}]},{"osd":6,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.56499999999999995}]},{"osd":7,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4182,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:33:24.737 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:33:25.034 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:33:25.033+0000 7ff47d3d0640 0 --2- 172.21.15.119:0/393670286 >> v2:172.21.15.119:3300/0 conn(0x7ff4780aabb0 0x7ff4781376c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-18T11:33:25.201 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:33:25.201 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:33:25.213 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-18T11:33:23.957922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":142099530,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":748,"num_read_kb":25515,"num_write":887,"num_write_kb":266346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":141445629,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2613796,"kb_used_data":191676,"kb_used_omap":28,"kb_used_meta":2421731,"kb_avail":752360924,"statfs":{"total":773094113280,"available":770417586176,"internally_reserved":0,"allocated":196276224,"data_stored":193066817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29629,"internal_metadata":2479852611},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001434"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":330,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.193757+0000","last_change":"2024-09-18T11:32:46.462271+0000","last_active":"2024-09-18T11:33:20.193757+0000","last_peered":"2024-09-18T11:33:20.193757+0000","last_clean":"2024-09-18T11:33:20.193757+0000","last_became_active":"2024-09-18T11:32:46.456653+0000","last_became_peered":"2024-09-18T11:32:46.456653+0000","last_unstale":"2024-09-18T11:33:20.193757+0000","last_undegraded":"2024-09-18T11:33:20.193757+0000","last_fullsized":"2024-09-18T11:33:20.193757+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'72","last_scrub_stamp":"2024-09-18T11:32:29.173114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_clean_scrub_stamp":"2024-09-18T11:32:29.173114+0000","objects_scrubbed":2,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:47.098458+0000","scrub_duration":0.0016253019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":240,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.908038+0000","last_change":"2024-09-18T11:32:42.123653+0000","last_active":"2024-09-18T11:33:20.908038+0000","last_peered":"2024-09-18T11:33:20.908038+0000","last_clean":"2024-09-18T11:33:20.908038+0000","last_became_active":"2024-09-18T11:32:27.079772+0000","last_became_peered":"2024-09-18T11:32:27.079772+0000","last_unstale":"2024-09-18T11:33:20.908038+0000","last_undegraded":"2024-09-18T11:33:20.908038+0000","last_fullsized":"2024-09-18T11:33:20.908038+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'1","last_scrub_stamp":"2024-09-18T11:32:42.123530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:32:42.123530+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:04.343044+0000","scrub_duration":0.0016360910000000001,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"182'203","reported_seq":449,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.132212+0000","last_change":"2024-09-18T11:32:39.596000+0000","last_active":"2024-09-18T11:33:20.132212+0000","last_peered":"2024-09-18T11:33:20.132212+0000","last_clean":"2024-09-18T11:33:20.132212+0000","last_became_active":"2024-09-18T11:30:01.077612+0000","last_became_peered":"2024-09-18T11:30:01.077612+0000","last_unstale":"2024-09-18T11:33:20.132212+0000","last_undegraded":"2024-09-18T11:33:20.132212+0000","last_fullsized":"2024-09-18T11:33:20.132212+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'194","last_scrub_stamp":"2024-09-18T11:32:39.595890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:32:39.595890+0000","objects_scrubbed":49,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:05.467884+0000","scrub_duration":0.0058412239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42108006,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":185,"num_read_kb":4788,"num_write":230,"num_write_kb":78760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":222,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-18T11:33:21.778887+0000","last_change":"2024-09-18T11:33:21.778858+0000","last_active":"2024-09-18T11:33:21.778887+0000","last_peered":"2024-09-18T11:33:21.778887+0000","last_clean":"2024-09-18T11:33:21.778887+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:33:21.778887+0000","last_undegraded":"2024-09-18T11:33:21.778887+0000","last_fullsized":"2024-09-18T11:33:21.778887+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:21.778728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:21.778728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:33.603349+0000","scrub_duration":0.0020274260000000001,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"173'206","reported_seq":882,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-18T11:33:15.460786+0000","last_change":"2024-09-18T11:33:01.261204+0000","last_active":"2024-09-18T11:33:15.460786+0000","last_peered":"2024-09-18T11:33:15.460786+0000","last_clean":"2024-09-18T11:33:15.460786+0000","last_became_active":"2024-09-18T11:33:01.060996+0000","last_became_peered":"2024-09-18T11:33:01.060996+0000","last_unstale":"2024-09-18T11:33:15.460786+0000","last_undegraded":"2024-09-18T11:33:15.460786+0000","last_fullsized":"2024-09-18T11:33:15.460786+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'206","last_scrub_stamp":"2024-09-18T11:32:51.136969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:32:51.136969+0000","objects_scrubbed":46,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:00.923271+0000","scrub_duration":0.0050590330000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43270431,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":215,"num_read_kb":11081,"num_write":225,"num_write_kb":76253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":74278671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":127,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.132197+0000","last_change":"2024-09-18T11:32:59.475277+0000","last_active":"2024-09-18T11:33:20.132197+0000","last_peered":"2024-09-18T11:33:20.132197+0000","last_clean":"2024-09-18T11:33:20.132197+0000","last_became_active":"2024-09-18T11:32:59.474959+0000","last_became_peered":"2024-09-18T11:32:59.474959+0000","last_unstale":"2024-09-18T11:33:20.132197+0000","last_undegraded":"2024-09-18T11:33:20.132197+0000","last_fullsized":"2024-09-18T11:33:20.132197+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:32:32.611633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:32:32.611633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:43.724465+0000","scrub_duration":0.001364355,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"172'146","reported_seq":616,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.203502+0000","last_change":"2024-09-18T11:32:59.712706+0000","last_active":"2024-09-18T11:33:20.203502+0000","last_peered":"2024-09-18T11:33:20.203502+0000","last_clean":"2024-09-18T11:33:20.203502+0000","last_became_active":"2024-09-18T11:32:59.476088+0000","last_became_peered":"2024-09-18T11:32:59.476088+0000","last_unstale":"2024-09-18T11:33:20.203502+0000","last_undegraded":"2024-09-18T11:33:20.203502+0000","last_fullsized":"2024-09-18T11:33:20.203502+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"144'136","last_scrub_stamp":"2024-09-18T11:32:20.808182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:32:20.808182+0000","objects_scrubbed":29,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:24.436066+0000","scrub_duration":0.0050914910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24142412,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":133,"num_read_kb":1720,"num_write":158,"num_write_kb":46975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":29291946,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":215,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.908071+0000","last_change":"2024-09-18T11:33:04.151627+0000","last_active":"2024-09-18T11:33:20.908071+0000","last_peered":"2024-09-18T11:33:20.908071+0000","last_clean":"2024-09-18T11:33:20.908071+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:33:20.908071+0000","last_undegraded":"2024-09-18T11:33:20.908071+0000","last_fullsized":"2024-09-18T11:33:20.908071+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"18'2","last_scrub_stamp":"2024-09-18T11:33:04.151495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:04.151495+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:24.817971+0000","scrub_duration":0.0017471489999999999,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"179'157","reported_seq":613,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.905163+0000","last_change":"2024-09-18T11:32:46.707704+0000","last_active":"2024-09-18T11:33:20.905163+0000","last_peered":"2024-09-18T11:33:20.905163+0000","last_clean":"2024-09-18T11:33:20.905163+0000","last_became_active":"2024-09-18T11:32:46.456572+0000","last_became_peered":"2024-09-18T11:32:46.456572+0000","last_unstale":"2024-09-18T11:33:20.905163+0000","last_undegraded":"2024-09-18T11:33:20.905163+0000","last_fullsized":"2024-09-18T11:33:20.905163+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"166'152","last_scrub_stamp":"2024-09-18T11:32:42.852232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:32:42.852232+0000","objects_scrubbed":33,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:04.691723+0000","scrub_duration":0.0039393850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31988294,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":149,"num_read_kb":7871,"num_write":171,"num_write_kb":62728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":36817156,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":236,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.203483+0000","last_change":"2024-09-18T11:32:46.457112+0000","last_active":"2024-09-18T11:33:20.203483+0000","last_peered":"2024-09-18T11:33:20.203483+0000","last_clean":"2024-09-18T11:33:20.203483+0000","last_became_active":"2024-09-18T11:32:46.456819+0000","last_became_peered":"2024-09-18T11:32:46.456819+0000","last_unstale":"2024-09-18T11:33:20.203483+0000","last_undegraded":"2024-09-18T11:33:20.203483+0000","last_fullsized":"2024-09-18T11:33:20.203483+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:32:30.705704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:32:30.705704+0000","last_clean_scrub_stamp":"2024-09-18T11:32:30.705704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:42.193160+0000","scrub_duration":0.0039850399999999996,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":232,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.193709+0000","last_change":"2024-09-18T11:32:46.456500+0000","last_active":"2024-09-18T11:33:20.193709+0000","last_peered":"2024-09-18T11:33:20.193709+0000","last_clean":"2024-09-18T11:33:20.193709+0000","last_became_active":"2024-09-18T11:32:46.456211+0000","last_became_peered":"2024-09-18T11:32:46.456211+0000","last_unstale":"2024-09-18T11:33:20.193709+0000","last_undegraded":"2024-09-18T11:33:20.193709+0000","last_fullsized":"2024-09-18T11:33:20.193709+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:32:42.223963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:32:42.223963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:12.013506+0000","scrub_duration":0.001097408,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.130106+0000","last_change":"2024-09-18T11:32:59.475270+0000","last_active":"2024-09-18T11:33:20.130106+0000","last_peered":"2024-09-18T11:33:20.130106+0000","last_clean":"2024-09-18T11:33:20.130106+0000","last_became_active":"2024-09-18T11:32:59.474991+0000","last_became_peered":"2024-09-18T11:32:59.474991+0000","last_unstale":"2024-09-18T11:33:20.130106+0000","last_undegraded":"2024-09-18T11:33:20.130106+0000","last_fullsized":"2024-09-18T11:33:20.130106+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:32:28.708882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:32:28.708882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:55.766077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":212,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-18T11:33:20.130121+0000","last_change":"2024-09-18T11:32:06.213366+0000","last_active":"2024-09-18T11:33:20.130121+0000","last_peered":"2024-09-18T11:33:20.130121+0000","last_clean":"2024-09-18T11:33:20.130121+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:33:20.130121+0000","last_undegraded":"2024-09-18T11:33:20.130121+0000","last_fullsized":"2024-09-18T11:33:20.130121+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:32:06.213281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:32:06.213281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:23.965948+0000","scrub_duration":0.0013619280000000001,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":141509143,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":682,"num_read_kb":25460,"num_write":784,"num_write_kb":264716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":140387773,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":205524992,"data_stored":205034291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4182,"internal_metadata":0},"log_size":712,"ondisk_log_size":712,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302812,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069028,"statfs":{"total":96636764160,"available":96326684672,"internally_reserved":0,"allocated":1323008,"data_stored":975864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.54300000000000004}]},{"osd":1,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45300000000000001,"15min":0.46300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.65300000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.505,"15min":0.51800000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.754,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61199999999999999}]},{"osd":2,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.29199999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.503},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42899999999999999,"15min":0.435},"min":{"1min":0.317,"5min":0.314,"15min":0.314},"max":{"1min":0.52100000000000002,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.58399999999999996}]},{"osd":3,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.439,"15min":0.442},"min":{"1min":0.36499999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.55100000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.47099999999999997,"15min":0.48599999999999999},"min":{"1min":0.311,"5min":0.308,"15min":0.308},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.626}]},{"osd":4,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.439,"15min":0.45700000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.626,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.437,"5min":0.47799999999999998,"15min":0.498},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.624,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.66100000000000003}]},{"osd":5,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.41899999999999998,"15min":0.44},"min":{"1min":0.24399999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.71199999999999997,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.442,"15min":0.46500000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.72399999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.68100000000000005}]},{"osd":6,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.5,"15min":0.52300000000000002},"min":{"1min":0.28699999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48599999999999999,"15min":0.502},"min":{"1min":0.29399999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69099999999999995,"5min":0.75,"15min":0.75},"last":0.69199999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302264,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069576,"statfs":{"total":96636764160,"available":96327245824,"internally_reserved":0,"allocated":765952,"data_stored":441689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.52700000000000002}]},{"osd":1,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.46600000000000003,"15min":0.46899999999999997},"min":{"1min":0.379,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.61399999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.45400000000000001,"15min":0.45600000000000002},"min":{"1min":0.32800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.69499999999999995,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.58399999999999996}]},{"osd":2,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46100000000000002,"15min":0.46300000000000002},"min":{"1min":0.312,"5min":0.221,"15min":0.221},"max":{"1min":0.64700000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.45300000000000001,"15min":0.44700000000000001},"min":{"1min":0.34300000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.59899999999999998}]},{"osd":3,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.49299999999999999,"15min":0.51100000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.67100000000000004,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.505,"15min":0.51900000000000002},"min":{"1min":0.33900000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69699999999999995,"5min":1.032,"15min":1.032},"last":0.69499999999999995}]},{"osd":4,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.40100000000000002,"15min":0.39300000000000002},"min":{"1min":0.316,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.61299999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.437,"15min":0.42499999999999999},"min":{"1min":0.41299999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65400000000000003,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.73599999999999999}]},{"osd":5,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.41499999999999998,"15min":0.40899999999999997},"min":{"1min":0.29599999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66700000000000004,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.441,"15min":0.433},"min":{"1min":0.34799999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.66500000000000004}]},{"osd":7,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.46500000000000002,"15min":0.46400000000000002},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.48399999999999999,"15min":0.47799999999999998},"min":{"1min":0.34599999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76600000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.75900000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332320,"kb_used_data":26640,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94039520,"statfs":{"total":96636764160,"available":96296468480,"internally_reserved":0,"allocated":27279360,"data_stored":26860362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.56599999999999995}]},{"osd":1,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42399999999999999,"15min":0.42999999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.44700000000000001,"15min":0.44900000000000001},"min":{"1min":0.248,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.44900000000000001}]},{"osd":2,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47999999999999998,"15min":0.496},"min":{"1min":0.316,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66600000000000004,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.47199999999999998,"15min":0.49399999999999999},"min":{"1min":0.32400000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.59599999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.52900000000000003}]},{"osd":3,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.46700000000000003,"15min":0.48899999999999999},"min":{"1min":0.29899999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.51300000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.48399999999999999,"15min":0.496},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.61299999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.46899999999999997}]},{"osd":4,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42999999999999999,"15min":0.44400000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.435,"15min":0.45700000000000002},"min":{"1min":0.22900000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.65700000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.48199999999999998}]},{"osd":6,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.25600000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.46700000000000003,"15min":0.48599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.49399999999999999}]},{"osd":7,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.47499999999999998,"15min":0.499},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.72899999999999998,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.503,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.57699999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356576,"kb_used_data":50900,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94015264,"statfs":{"total":96636764160,"available":96271630336,"internally_reserved":0,"allocated":52121600,"data_stored":51665786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.57599999999999996}]},{"osd":1,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.47899999999999998,"15min":0.501},"min":{"1min":0.315,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.65300000000000002,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48399999999999999,"15min":0.498},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73299999999999998,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.73599999999999999}]},{"osd":2,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48099999999999998,"15min":0.49299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.748,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.49199999999999999,"15min":0.51600000000000001},"min":{"1min":0.35399999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72599999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.63600000000000001}]},{"osd":3,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.48899999999999999,"15min":0.51300000000000001},"min":{"1min":0.35799999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.61399999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.50600000000000001,"15min":0.52700000000000002},"min":{"1min":0.378,"5min":0.316,"15min":0.316},"max":{"1min":0.68799999999999994,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.59099999999999997}]},{"osd":5,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.442,"15min":0.46200000000000002},"min":{"1min":0.27700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.70699999999999996,"5min":1.002,"15min":1.002},"last":0.439},{"interface":"front","average":{"1min":0.442,"5min":0.47999999999999998,"15min":0.50800000000000001},"min":{"1min":0.30199999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.70299999999999996,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.41999999999999998}]},{"osd":6,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.45000000000000001,"15min":0.46899999999999997},"min":{"1min":0.28199999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.69399999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48399999999999999,"15min":0.504},"min":{"1min":0.33200000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.746,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.50600000000000001}]},{"osd":7,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.49199999999999999,"15min":0.51000000000000001},"min":{"1min":0.34899999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.54500000000000004},"min":{"1min":0.34100000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66700000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":328180,"kb_used_data":26660,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94043660,"statfs":{"total":96636764160,"available":96300707840,"internally_reserved":0,"allocated":27299840,"data_stored":26863829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.51200000000000001}]},{"osd":1,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.33100000000000002,"15min":0.34100000000000003},"min":{"1min":0.246,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.45500000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.36799999999999999},"min":{"1min":0.224,"5min":0.214,"15min":0.214},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.63800000000000001}]},{"osd":2,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.36699999999999999,"15min":0.377},"min":{"1min":0.27800000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.377,"15min":0.39900000000000002},"min":{"1min":0.28100000000000003,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.48699999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.73399999999999999}]},{"osd":4,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.50700000000000001,"15min":0.51300000000000001},"min":{"1min":0.39300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.64600000000000002,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52000000000000002,"15min":0.52400000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.88900000000000001}]},{"osd":5,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.49199999999999999,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69399999999999995,"5min":0.752,"15min":0.752},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53100000000000003,"15min":0.55500000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67000000000000004,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.94899999999999995}]},{"osd":6,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.51300000000000001,"15min":0.53300000000000003},"min":{"1min":0.35999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.60899999999999999,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.504,"5min":0.54200000000000004,"15min":0.56100000000000005},"min":{"1min":0.39800000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.66500000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.91800000000000004}]},{"osd":7,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56999999999999995,"15min":0.60099999999999998},"min":{"1min":0.41599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68899999999999995,"5min":1.26,"15min":1.26},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.54500000000000004,"15min":0.57199999999999995},"min":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.63,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.86099999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318532,"kb_used_data":17016,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94053308,"statfs":{"total":96636764160,"available":96310587392,"internally_reserved":0,"allocated":17424384,"data_stored":17038411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.48599999999999999}]},{"osd":1,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.35599999999999998,"15min":0.375},"min":{"1min":0.23200000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.57099999999999995,"5min":1.621,"15min":1.621},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.377,"15min":0.39900000000000002},"min":{"1min":0.27100000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.70599999999999996,"5min":1.589,"15min":1.589},"last":0.52200000000000002}]},{"osd":3,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35799999999999998,"15min":0.36799999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.72399999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.39200000000000002,"15min":0.41399999999999998},"min":{"1min":0.27500000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52900000000000003,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.61899999999999999}]},{"osd":4,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53100000000000003,"15min":0.55100000000000005},"min":{"1min":0.373,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.755,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.504,"15min":0.51700000000000002},"min":{"1min":0.34300000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73199999999999998,"5min":1.012,"15min":1.012},"last":0.66500000000000004}]},{"osd":5,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53000000000000003,"15min":0.54200000000000004},"min":{"1min":0.36799999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70799999999999996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.56100000000000005},"min":{"1min":0.377,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77900000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.71899999999999997}]},{"osd":6,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52400000000000002,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82399999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.5,"5min":0.53600000000000003,"15min":0.56100000000000005},"min":{"1min":0.35499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65200000000000002,"5min":1.165,"15min":1.165},"last":0.77300000000000002}]},{"osd":7,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54400000000000004,"15min":0.56000000000000005},"min":{"1min":0.371,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70099999999999996,"5min":1.073,"15min":1.073},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.39700000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91600000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.82399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322692,"kb_used_data":17016,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94049148,"statfs":{"total":96636764160,"available":96306327552,"internally_reserved":0,"allocated":17424384,"data_stored":17037068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.24199999999999999}]},{"osd":2,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.38100000000000001},"min":{"1min":0.28699999999999998,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.871,"5min":1.333,"15min":1.333},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.40000000000000002,"15min":0.42099999999999999},"min":{"1min":0.23899999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.89200000000000002,"5min":1.298,"15min":1.298},"last":0.39900000000000002}]},{"osd":3,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.371,"15min":0.38200000000000001},"min":{"1min":0.27300000000000002,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.35599999999999998,"15min":0.36199999999999999},"min":{"1min":0.22500000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.35199999999999998}]},{"osd":4,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51500000000000001,"15min":0.52500000000000002},"min":{"1min":0.307,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.34999999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.96899999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.48399999999999999}]},{"osd":5,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.50700000000000001,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.91800000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.52500000000000002},"min":{"1min":0.32300000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.436}]},{"osd":6,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.34000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53200000000000003,"15min":0.54300000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.94499999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.45900000000000002}]},{"osd":7,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53700000000000003,"15min":0.54700000000000004},"min":{"1min":0.36299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56599999999999995,"15min":0.57899999999999996},"min":{"1min":0.34200000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.55800000000000005}]}]},{"osd":0,"up_from":119,"seq":511101108244,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":350420,"kb_used_data":51404,"kb_used_omap":17,"kb_used_meta":298734,"kb_avail":94021420,"statfs":{"total":96636764160,"available":96277934080,"internally_reserved":0,"allocated":52637696,"data_stored":52183808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17701,"internal_metadata":305904347},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.39200000000000002}]},{"osd":2,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.37,"5min":0.37,"15min":0.37},"last":0.31}]},{"osd":3,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.311,"15min":0.311},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.54400000000000004}]},{"osd":4,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.75900000000000001}]},{"osd":5,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.68600000000000005}]},{"osd":6,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.56499999999999995}]},{"osd":7,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4182,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:33:25.216 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T11:33:45.217 INFO:tasks.thrashosds.thrasher:Recovered, killing an osd 2024-09-18T11:33:45.217 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-18T11:33:45.217 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-18T11:33:45.218 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T11:33:45.256 INFO:tasks.ceph.osd.0:Stopped 2024-09-18T11:33:45.256 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2024-09-18T11:33:45.576 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-09-18T11:33:45.574+0000 7ff6f6bf1640 -1 mon.a@0(leader).osd e205 definitely_dead 0 2024-09-18T11:33:46.213 INFO:teuthology.orchestra.run.smithi119.stderr:marked down osd.0. 2024-09-18T11:33:46.229 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-18T11:33:46.229 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-09-18T11:33:46.503 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-09-18T11:33:46.501+0000 7ff6f6bf1640 -1 mon.a@0(leader).osd e206 definitely_dead 0 2024-09-18T11:33:47.214 INFO:teuthology.orchestra.run.smithi119.stderr:marked out osd.0. 2024-09-18T11:33:47.229 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.0 2024-09-18T11:33:47.229 INFO:tasks.thrashosds.thrasher:remote for osd 0 is ubuntu@smithi119.front.sepia.ceph.com 2024-09-18T11:33:47.229 DEBUG:teuthology.orchestra.run.smithi119:> type ceph-objectstore-tool 2024-09-18T11:33:47.245 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T11:33:47.245 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op list-pgs 2024-09-18T11:33:47.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:33:47.487+0000 7fee2480e200 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file) 2024-09-18T11:33:48.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-18T11:33:48.516+0000 7fee2480e200 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file) 2024-09-18T11:33:48.784 INFO:teuthology.orchestra.run.smithi119.stdout:3.3 2024-09-18T11:33:48.784 INFO:teuthology.orchestra.run.smithi119.stdout:3.1 2024-09-18T11:33:48.784 INFO:teuthology.orchestra.run.smithi119.stdout:2.5 2024-09-18T11:33:48.784 INFO:teuthology.orchestra.run.smithi119.stdout:2.4 2024-09-18T11:33:48.784 INFO:teuthology.orchestra.run.smithi119.stdout:1.0 2024-09-18T11:33:49.305 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op export --pgid 2.5 --file /var/log/ceph/exp.2.5.0 2024-09-18T11:33:50.394 INFO:teuthology.orchestra.run.smithi119.stderr:Exporting 2.5 info 2.5( empty local-lis/les=174/175 n=0 ec=15/15 lis/c=174/174 les/c/f=175/175/0 sis=174) 2024-09-18T11:33:50.402 INFO:teuthology.orchestra.run.smithi119.stderr:Export successful 2024-09-18T11:33:50.913 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --force --op remove --pgid 2.5 2024-09-18T11:33:52.003 INFO:teuthology.orchestra.run.smithi119.stdout: marking collection for removal 2024-09-18T11:33:52.003 INFO:teuthology.orchestra.run.smithi119.stdout:setting '_remove' omap key 2024-09-18T11:33:52.003 INFO:teuthology.orchestra.run.smithi119.stdout:finish_remove_pgs 2.5_head removing 2.5 2024-09-18T11:33:52.003 INFO:teuthology.orchestra.run.smithi119.stdout:Remove successful 2024-09-18T11:33:52.520 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.2.5.0 2024-09-18T11:33:53.619 INFO:teuthology.orchestra.run.smithi119.stdout:get_pg_num_history pg_num_history pg_num_history(e205 pg_nums {1={14=1},2={15=8},3={21=4},4={22=16}} deleted_pools 198,4) 2024-09-18T11:33:53.619 INFO:teuthology.orchestra.run.smithi119.stdout:Importing pgid 2.5 2024-09-18T11:33:53.619 INFO:teuthology.orchestra.run.smithi119.stdout:write_pg epoch 175 info 2.5( empty local-lis/les=174/175 n=0 ec=15/15 lis/c=174/174 les/c/f=175/175/0 sis=174) 2024-09-18T11:33:53.627 INFO:teuthology.orchestra.run.smithi119.stdout:Import successful 2024-09-18T11:33:54.137 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f /var/log/ceph/exp.2.5.0 2024-09-18T11:33:54.167 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T11:33:54.475 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:33:54.475 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":209,"fsid":"f4cca259-b87a-45cc-8995-7e9fca891a9f","created":"2024-09-18T11:29:38.205487+0000","modified":"2024-09-18T11:33:49.218232+0000","last_up_change":"2024-09-18T11:33:46.203364+0000","last_in_change":"2024-09-18T11:33:47.210102+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T11:29:47.951262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T11:29:48.866238+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"low_tier","create_time":"2024-09-18T11:29:59.343964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"12faced2-7888-4265-ad12-815efe36466c","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":65,"up_from":119,"up_thru":189,"down_at":206,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":56607},{"type":"v1","addr":"172.21.15.119:6801","nonce":56607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":56607},{"type":"v1","addr":"172.21.15.119:6803","nonce":56607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":56607},{"type":"v1","addr":"172.21.15.119:6807","nonce":56607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":56607},{"type":"v1","addr":"172.21.15.119:6805","nonce":56607}]},"public_addr":"172.21.15.119:6801/56607","cluster_addr":"172.21.15.119:6803/56607","heartbeat_back_addr":"172.21.15.119:6807/56607","heartbeat_front_addr":"172.21.15.119:6805/56607","state":["exists"]},{"osd":1,"uuid":"20fee29d-b047-46ae-aa60-88e897fbb0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":33350},{"type":"v1","addr":"172.21.15.119:6825","nonce":33350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":33350},{"type":"v1","addr":"172.21.15.119:6827","nonce":33350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":33350},{"type":"v1","addr":"172.21.15.119:6831","nonce":33350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":33350},{"type":"v1","addr":"172.21.15.119:6829","nonce":33350}]},"public_addr":"172.21.15.119:6825/33350","cluster_addr":"172.21.15.119:6827/33350","heartbeat_back_addr":"172.21.15.119:6831/33350","heartbeat_front_addr":"172.21.15.119:6829/33350","state":["exists","up"]},{"osd":2,"uuid":"21a296f3-b259-4841-9a17-4cd21b302b36","up":1,"in":1,"weight":1,"primary_affinity":0,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":33351},{"type":"v1","addr":"172.21.15.119:6809","nonce":33351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":33351},{"type":"v1","addr":"172.21.15.119:6813","nonce":33351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6819","nonce":33351},{"type":"v1","addr":"172.21.15.119:6821","nonce":33351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":33351},{"type":"v1","addr":"172.21.15.119:6817","nonce":33351}]},"public_addr":"172.21.15.119:6809/33351","cluster_addr":"172.21.15.119:6813/33351","heartbeat_back_addr":"172.21.15.119:6821/33351","heartbeat_front_addr":"172.21.15.119:6817/33351","state":["exists","up"]},{"osd":3,"uuid":"59044f30-32e1-4942-9a8e-202c1def221a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":33349},{"type":"v1","addr":"172.21.15.119:6812","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":33349},{"type":"v1","addr":"172.21.15.119:6816","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":33349},{"type":"v1","addr":"172.21.15.119:6823","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":33349},{"type":"v1","addr":"172.21.15.119:6820","nonce":33349}]},"public_addr":"172.21.15.119:6812/33349","cluster_addr":"172.21.15.119:6816/33349","heartbeat_back_addr":"172.21.15.119:6823/33349","heartbeat_front_addr":"172.21.15.119:6820/33349","state":["exists","up"]},{"osd":4,"uuid":"7d1ae3d0-8d69-4d3c-9e51-24fdd77a22a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":33088},{"type":"v1","addr":"172.21.15.148:6809","nonce":33088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":33088},{"type":"v1","addr":"172.21.15.148:6811","nonce":33088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":33088},{"type":"v1","addr":"172.21.15.148:6815","nonce":33088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":33088},{"type":"v1","addr":"172.21.15.148:6813","nonce":33088}]},"public_addr":"172.21.15.148:6809/33088","cluster_addr":"172.21.15.148:6811/33088","heartbeat_back_addr":"172.21.15.148:6815/33088","heartbeat_front_addr":"172.21.15.148:6813/33088","state":["exists","up"]},{"osd":5,"uuid":"b0dd3d30-b273-428c-a4a9-4c238abf45fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":33089},{"type":"v1","addr":"172.21.15.148:6801","nonce":33089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":33089},{"type":"v1","addr":"172.21.15.148:6803","nonce":33089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":33089},{"type":"v1","addr":"172.21.15.148:6807","nonce":33089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":33089},{"type":"v1","addr":"172.21.15.148:6805","nonce":33089}]},"public_addr":"172.21.15.148:6801/33089","cluster_addr":"172.21.15.148:6803/33089","heartbeat_back_addr":"172.21.15.148:6807/33089","heartbeat_front_addr":"172.21.15.148:6805/33089","state":["exists","up"]},{"osd":6,"uuid":"40290cb3-2334-4c68-8e6c-59d0d5455171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":33107},{"type":"v1","addr":"172.21.15.148:6817","nonce":33107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":33107},{"type":"v1","addr":"172.21.15.148:6819","nonce":33107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":33107},{"type":"v1","addr":"172.21.15.148:6823","nonce":33107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":33107},{"type":"v1","addr":"172.21.15.148:6821","nonce":33107}]},"public_addr":"172.21.15.148:6817/33107","cluster_addr":"172.21.15.148:6819/33107","heartbeat_back_addr":"172.21.15.148:6823/33107","heartbeat_front_addr":"172.21.15.148:6821/33107","state":["exists","up"]},{"osd":7,"uuid":"54802abe-ea1a-448e-85af-d1f14a45be27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":33119},{"type":"v1","addr":"172.21.15.148:6825","nonce":33119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":33119},{"type":"v1","addr":"172.21.15.148:6827","nonce":33119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":33119},{"type":"v1","addr":"172.21.15.148:6831","nonce":33119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":33119},{"type":"v1","addr":"172.21.15.148:6829","nonce":33119}]},"public_addr":"172.21.15.148:6825/33119","cluster_addr":"172.21.15.148:6827/33119","heartbeat_back_addr":"172.21.15.148:6831/33119","heartbeat_front_addr":"172.21.15.148:6829/33119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-18T11:33:10.971461+0000","dead_epoch":206},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:33:32.688806+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:32:27.493785+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:33:30.547974+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:33:31.230982+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:33:47.139246+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:33:26.841646+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:33:48.628932+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T11:33:54.487 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T11:29:47.951262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T11:29:48.866238+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'low_tier', 'create_time': '2024-09-18T11:29:59.343964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}}] 2024-09-18T11:33:54.487 DEBUG:teuthology.orchestra.run.smithi119:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op apply-layout-settings --pool .mgr 2024-09-18T11:33:55.566 INFO:teuthology.orchestra.run.smithi119.stderr:Nothing to do for non-filestore backend 2024-09-18T11:33:56.083 DEBUG:teuthology.orchestra.run.smithi119:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op apply-layout-settings --pool rbd 2024-09-18T11:33:56.241 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:33:56.240+0000 7f697ceb1640 -1 osd.3 209 handle_osd_map injecting map crc failure 2024-09-18T11:33:57.172 INFO:teuthology.orchestra.run.smithi119.stderr:Nothing to do for non-filestore backend 2024-09-18T11:33:57.694 DEBUG:teuthology.orchestra.run.smithi119:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op apply-layout-settings --pool low_tier 2024-09-18T11:33:58.789 INFO:teuthology.orchestra.run.smithi119.stderr:Nothing to do for non-filestore backend 2024-09-18T11:33:59.313 INFO:tasks.thrashosds.thrasher:Waiting for clean again 2024-09-18T11:33:59.313 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T11:33:59.313 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:33:59.584 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:33:59.584 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:33:59.596 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-09-18T11:33:57.965494+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":142099530,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":712,"num_read_kb":25479,"num_write":887,"num_write_kb":266346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":449,"num_bytes_recovered":164988368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":2313624,"kb_used_data":190532,"kb_used_omap":11,"kb_used_meta":2122996,"kb_avail":658289256,"statfs":{"total":676457349120,"available":674088198144,"internally_reserved":0,"allocated":195104768,"data_stored":192338948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2173948264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":51748116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.741666"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":367,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:50.797940+0000","last_change":"2024-09-18T11:33:50.797882+0000","last_active":"2024-09-18T11:33:50.797940+0000","last_peered":"2024-09-18T11:33:50.797940+0000","last_clean":"2024-09-18T11:33:50.797940+0000","last_became_active":"2024-09-18T11:33:48.598488+0000","last_became_peered":"2024-09-18T11:33:48.598488+0000","last_unstale":"2024-09-18T11:33:50.797940+0000","last_undegraded":"2024-09-18T11:33:50.797940+0000","last_fullsized":"2024-09-18T11:33:50.797940+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'72","last_scrub_stamp":"2024-09-18T11:33:50.797790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_clean_scrub_stamp":"2024-09-18T11:33:50.797790+0000","objects_scrubbed":2,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:20.554098+0000","scrub_duration":0.001837497,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":250,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.360793+0000","last_change":"2024-09-18T11:32:42.123653+0000","last_active":"2024-09-18T11:33:49.360793+0000","last_peered":"2024-09-18T11:33:49.360793+0000","last_clean":"2024-09-18T11:33:49.360793+0000","last_became_active":"2024-09-18T11:32:27.079772+0000","last_became_peered":"2024-09-18T11:32:27.079772+0000","last_unstale":"2024-09-18T11:33:49.360793+0000","last_undegraded":"2024-09-18T11:33:49.360793+0000","last_fullsized":"2024-09-18T11:33:49.360793+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'1","last_scrub_stamp":"2024-09-18T11:32:42.123530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:32:42.123530+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:04.343044+0000","scrub_duration":0.0016360910000000001,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"182'203","reported_seq":459,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.225219+0000","last_change":"2024-09-18T11:32:39.596000+0000","last_active":"2024-09-18T11:33:49.225219+0000","last_peered":"2024-09-18T11:33:49.225219+0000","last_clean":"2024-09-18T11:33:49.225219+0000","last_became_active":"2024-09-18T11:30:01.077612+0000","last_became_peered":"2024-09-18T11:30:01.077612+0000","last_unstale":"2024-09-18T11:33:49.225219+0000","last_undegraded":"2024-09-18T11:33:49.225219+0000","last_fullsized":"2024-09-18T11:33:49.225219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'194","last_scrub_stamp":"2024-09-18T11:32:39.595890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:32:39.595890+0000","objects_scrubbed":49,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:05.467884+0000","scrub_duration":0.0058412239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42108006,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":185,"num_read_kb":4788,"num_write":230,"num_write_kb":78760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":232,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.360995+0000","last_change":"2024-09-18T11:33:21.778858+0000","last_active":"2024-09-18T11:33:49.360995+0000","last_peered":"2024-09-18T11:33:49.360995+0000","last_clean":"2024-09-18T11:33:49.360995+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:33:49.360995+0000","last_undegraded":"2024-09-18T11:33:49.360995+0000","last_fullsized":"2024-09-18T11:33:49.360995+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:21.778728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:21.778728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:33.603349+0000","scrub_duration":0.0020274260000000001,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"173'206","reported_seq":847,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.224471+0000","last_change":"2024-09-18T11:33:48.450575+0000","last_active":"2024-09-18T11:33:49.224471+0000","last_peered":"2024-09-18T11:33:49.224471+0000","last_clean":"2024-09-18T11:33:49.224471+0000","last_became_active":"2024-09-18T11:33:48.227585+0000","last_became_peered":"2024-09-18T11:33:48.227585+0000","last_unstale":"2024-09-18T11:33:49.224471+0000","last_undegraded":"2024-09-18T11:33:49.224471+0000","last_fullsized":"2024-09-18T11:33:49.224471+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'206","last_scrub_stamp":"2024-09-18T11:32:51.136969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:32:51.136969+0000","objects_scrubbed":46,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:01.520694+0000","scrub_duration":0.0050590330000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43270431,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":179,"num_read_kb":11045,"num_write":225,"num_write_kb":76253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":74278671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":147,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.225189+0000","last_change":"2024-09-18T11:33:45.869084+0000","last_active":"2024-09-18T11:33:49.225189+0000","last_peered":"2024-09-18T11:33:49.225189+0000","last_clean":"2024-09-18T11:33:49.225189+0000","last_became_active":"2024-09-18T11:32:59.474959+0000","last_became_peered":"2024-09-18T11:32:59.474959+0000","last_unstale":"2024-09-18T11:33:49.225189+0000","last_undegraded":"2024-09-18T11:33:49.225189+0000","last_fullsized":"2024-09-18T11:33:49.225189+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:45.869005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:45.869005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:47.117862+0000","scrub_duration":0.0012218940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"172'146","reported_seq":637,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.360879+0000","last_change":"2024-09-18T11:33:24.900812+0000","last_active":"2024-09-18T11:33:49.360879+0000","last_peered":"2024-09-18T11:33:49.360879+0000","last_clean":"2024-09-18T11:33:49.360879+0000","last_became_active":"2024-09-18T11:32:59.476088+0000","last_became_peered":"2024-09-18T11:32:59.476088+0000","last_unstale":"2024-09-18T11:33:49.360879+0000","last_undegraded":"2024-09-18T11:33:49.360879+0000","last_fullsized":"2024-09-18T11:33:49.360879+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"172'146","last_scrub_stamp":"2024-09-18T11:33:24.900735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:33:24.900735+0000","objects_scrubbed":35,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:35.491129+0000","scrub_duration":0.0037680470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24142412,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":133,"num_read_kb":1720,"num_write":158,"num_write_kb":46975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":29291946,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":225,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.360856+0000","last_change":"2024-09-18T11:33:04.151627+0000","last_active":"2024-09-18T11:33:49.360856+0000","last_peered":"2024-09-18T11:33:49.360856+0000","last_clean":"2024-09-18T11:33:49.360856+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:33:49.360856+0000","last_undegraded":"2024-09-18T11:33:49.360856+0000","last_fullsized":"2024-09-18T11:33:49.360856+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"18'2","last_scrub_stamp":"2024-09-18T11:33:04.151495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:04.151495+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:24.817971+0000","scrub_duration":0.0017471489999999999,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"179'157","reported_seq":769,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.858858+0000","last_change":"2024-09-18T11:33:49.858786+0000","last_active":"2024-09-18T11:33:49.858858+0000","last_peered":"2024-09-18T11:33:49.858858+0000","last_clean":"2024-09-18T11:33:49.858858+0000","last_became_active":"2024-09-18T11:33:48.598474+0000","last_became_peered":"2024-09-18T11:33:48.598474+0000","last_unstale":"2024-09-18T11:33:49.858858+0000","last_undegraded":"2024-09-18T11:33:49.858858+0000","last_fullsized":"2024-09-18T11:33:49.858858+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"179'157","last_scrub_stamp":"2024-09-18T11:33:49.858681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:33:49.858681+0000","objects_scrubbed":36,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:06.886975+0000","scrub_duration":0.0046045019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31988294,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":149,"num_read_kb":7871,"num_write":171,"num_write_kb":62728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":59830967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":267,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.360881+0000","last_change":"2024-09-18T11:33:48.226469+0000","last_active":"2024-09-18T11:33:49.360881+0000","last_peered":"2024-09-18T11:33:49.360881+0000","last_clean":"2024-09-18T11:33:49.360881+0000","last_became_active":"2024-09-18T11:33:48.226262+0000","last_became_peered":"2024-09-18T11:33:48.226262+0000","last_unstale":"2024-09-18T11:33:49.360881+0000","last_undegraded":"2024-09-18T11:33:49.360881+0000","last_fullsized":"2024-09-18T11:33:49.360881+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:44.054611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:32:30.705704+0000","last_clean_scrub_stamp":"2024-09-18T11:33:44.054611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:49.819708+0000","scrub_duration":0.0015037430000000001,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":256,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.360997+0000","last_change":"2024-09-18T11:33:48.773158+0000","last_active":"2024-09-18T11:33:49.360997+0000","last_peered":"2024-09-18T11:33:49.360997+0000","last_clean":"2024-09-18T11:33:49.360997+0000","last_became_active":"2024-09-18T11:33:48.226249+0000","last_became_peered":"2024-09-18T11:33:48.226249+0000","last_unstale":"2024-09-18T11:33:49.360997+0000","last_undegraded":"2024-09-18T11:33:49.360997+0000","last_fullsized":"2024-09-18T11:33:49.360997+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:32:42.223963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:32:42.223963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001097408,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-18T11:33:56.614457+0000","last_change":"2024-09-18T11:33:56.614431+0000","last_active":"2024-09-18T11:33:56.614457+0000","last_peered":"2024-09-18T11:33:56.614457+0000","last_clean":"2024-09-18T11:33:56.614457+0000","last_became_active":"2024-09-18T11:32:59.474991+0000","last_became_peered":"2024-09-18T11:32:59.474991+0000","last_unstale":"2024-09-18T11:33:56.614457+0000","last_undegraded":"2024-09-18T11:33:56.614457+0000","last_fullsized":"2024-09-18T11:33:56.614457+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:56.614347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:56.614347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:19.182426+0000","scrub_duration":0.0015834950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":232,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.360897+0000","last_change":"2024-09-18T11:33:24.650387+0000","last_active":"2024-09-18T11:33:49.360897+0000","last_peered":"2024-09-18T11:33:49.360897+0000","last_clean":"2024-09-18T11:33:49.360897+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:33:49.360897+0000","last_undegraded":"2024-09-18T11:33:49.360897+0000","last_fullsized":"2024-09-18T11:33:49.360897+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:24.650294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:24.650294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:34.919021+0000","scrub_duration":0.001363908,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":141509143,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":646,"num_read_kb":25424,"num_write":784,"num_write_kb":264716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":441,"num_bytes_recovered":163401584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":240234496,"data_stored":239663954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4182,"internal_metadata":0},"log_size":712,"ondisk_log_size":712,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302752,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069088,"statfs":{"total":96636764160,"available":96326746112,"internally_reserved":0,"allocated":1265664,"data_stored":934202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.50900000000000001}]},{"osd":1,"last update":"Wed Sep 18 11:33:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.46600000000000003,"15min":0.46700000000000003},"min":{"1min":0.36599999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.64600000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51100000000000001,"15min":0.52000000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.68300000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.66000000000000003}]},{"osd":2,"last update":"Wed Sep 18 11:33:55 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.436,"15min":0.42599999999999999},"min":{"1min":0.378,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.63400000000000001,"5min":1.085,"15min":1.085},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.44600000000000001,"15min":0.441},"min":{"1min":0.40999999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.50600000000000001}]},{"osd":3,"last update":"Wed Sep 18 11:33:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.46000000000000002,"15min":0.44900000000000001},"min":{"1min":0.35499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.47899999999999998,"15min":0.48899999999999999},"min":{"1min":0.39900000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.70199999999999996,"5min":1.177,"15min":1.177},"last":0.52200000000000002}]},{"osd":4,"last update":"Wed Sep 18 11:33:55 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.43099999999999999,"15min":0.45500000000000002},"min":{"1min":0.28599999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.61199999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47199999999999998,"15min":0.496},"min":{"1min":0.33600000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69399999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.69399999999999995}]},{"osd":5,"last update":"Wed Sep 18 11:33:55 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40699999999999997,"15min":0.436},"min":{"1min":0.29699999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.52700000000000002,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.432},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.437,"15min":0.46300000000000002},"min":{"1min":0.33800000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.68100000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.47299999999999998}]},{"osd":6,"last update":"Wed Sep 18 11:33:55 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48699999999999999,"15min":0.51900000000000002},"min":{"1min":0.32000000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.70399999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.48499999999999999,"15min":0.501},"min":{"1min":0.375,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69199999999999995,"5min":0.75,"15min":0.75},"last":0.57299999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325260,"kb_used_data":23744,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94046580,"statfs":{"total":96636764160,"available":96303697920,"internally_reserved":0,"allocated":24313856,"data_stored":23943675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.55400000000000005}]},{"osd":1,"last update":"Wed Sep 18 11:33:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.26300000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.35899999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.58399999999999996,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.434}]},{"osd":2,"last update":"Wed Sep 18 11:33:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.47099999999999997,"15min":0.46700000000000003},"min":{"1min":0.38100000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.66300000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.46800000000000003,"15min":0.45200000000000001},"min":{"1min":0.36899999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.42399999999999999}]},{"osd":3,"last update":"Wed Sep 18 11:33:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48399999999999999,"15min":0.50800000000000001},"min":{"1min":0.28599999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.64100000000000001,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.50700000000000001,"15min":0.52000000000000002},"min":{"1min":0.307,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69499999999999995,"5min":1.032,"15min":1.032},"last":0.504}]},{"osd":4,"last update":"Wed Sep 18 11:33:53 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.41199999999999998,"15min":0.39700000000000002},"min":{"1min":0.21099999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.45900000000000002,"15min":0.433},"min":{"1min":0.34300000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.58499999999999996}]},{"osd":5,"last update":"Wed Sep 18 11:33:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.435,"15min":0.41599999999999998},"min":{"1min":0.317,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.45000000000000001,"15min":0.436},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66500000000000004,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.52700000000000002}]},{"osd":7,"last update":"Wed Sep 18 11:33:53 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47099999999999997,"15min":0.46600000000000003},"min":{"1min":0.31,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48799999999999999,"15min":0.47999999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78600000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.54800000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332264,"kb_used_data":26588,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94039576,"statfs":{"total":96636764160,"available":96296525824,"internally_reserved":0,"allocated":27226112,"data_stored":26819345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.501},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.47899999999999998}]},{"osd":1,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42399999999999999,"15min":0.42999999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.44700000000000001,"15min":0.44900000000000001},"min":{"1min":0.248,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47999999999999998,"15min":0.496},"min":{"1min":0.316,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66600000000000004,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.627},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.47199999999999998,"15min":0.49399999999999999},"min":{"1min":0.32400000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.59599999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.57999999999999996}]},{"osd":3,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.46700000000000003,"15min":0.48899999999999999},"min":{"1min":0.29899999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.51300000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.48399999999999999,"15min":0.496},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.61299999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.59499999999999997}]},{"osd":4,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42999999999999999,"15min":0.44400000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.435,"15min":0.45700000000000002},"min":{"1min":0.22900000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.65700000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.42699999999999999}]},{"osd":6,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.25600000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.46700000000000003,"15min":0.48599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.501}]},{"osd":7,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.47499999999999998,"15min":0.499},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.72899999999999998,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.503,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.53700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356520,"kb_used_data":50844,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94015320,"statfs":{"total":96636764160,"available":96271687680,"internally_reserved":0,"allocated":52064256,"data_stored":51624124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.748},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.66600000000000004}]},{"osd":1,"last update":"Wed Sep 18 11:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.47799999999999998,"15min":0.5},"min":{"1min":0.28899999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67400000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.48299999999999998,"15min":0.498},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.73599999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed Sep 18 11:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.48299999999999998,"15min":0.49299999999999999},"min":{"1min":0.30499999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71699999999999997,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48999999999999999,"15min":0.51600000000000001},"min":{"1min":0.33400000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72199999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.51600000000000001}]},{"osd":3,"last update":"Wed Sep 18 11:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48699999999999999,"15min":0.51200000000000001},"min":{"1min":0.32800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65800000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.504,"15min":0.52600000000000002},"min":{"1min":0.34499999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.71899999999999997,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.70699999999999996}]},{"osd":5,"last update":"Wed Sep 18 11:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.42599999999999999,"15min":0.45700000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.70499999999999996,"5min":1.002,"15min":1.002},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.46100000000000002,"15min":0.501},"min":{"1min":0.29899999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.627,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.61299999999999999}]},{"osd":6,"last update":"Wed Sep 18 11:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.443,"15min":0.46700000000000003},"min":{"1min":0.252,"5min":0.245,"15min":0.245},"max":{"1min":0.68200000000000005,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46999999999999997,"15min":0.499},"min":{"1min":0.32500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.625,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.441}]},{"osd":7,"last update":"Wed Sep 18 11:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.48399999999999999,"15min":0.50700000000000001},"min":{"1min":0.28599999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69399999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.54300000000000004},"min":{"1min":0.28499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.77000000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.505}]}]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":355724,"kb_used_data":54208,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94016116,"statfs":{"total":96636764160,"available":96272502784,"internally_reserved":0,"allocated":55508992,"data_stored":55026793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.34999999999999998}]},{"osd":1,"last update":"Wed Sep 18 11:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.33400000000000002,"15min":0.34200000000000003},"min":{"1min":0.22500000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.56999999999999995,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.36799999999999999,"15min":0.372},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.69499999999999995,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.379}]},{"osd":2,"last update":"Wed Sep 18 11:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.38100000000000001,"15min":0.38200000000000001},"min":{"1min":0.23699999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.67800000000000005,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.38,"15min":0.40000000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.73399999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.36499999999999999}]},{"osd":4,"last update":"Wed Sep 18 11:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.52900000000000003,"15min":0.52000000000000002},"min":{"1min":0.375,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.54600000000000004,"15min":0.53200000000000003},"min":{"1min":0.45700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56999999999999995}]},{"osd":5,"last update":"Wed Sep 18 11:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.504,"15min":0.50800000000000001},"min":{"1min":0.34399999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.54200000000000004,"15min":0.55900000000000005},"min":{"1min":0.44,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94899999999999995,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.51600000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.51800000000000002,"15min":0.53400000000000003},"min":{"1min":0.36799999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.55100000000000005,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.61199999999999999}]},{"osd":7,"last update":"Wed Sep 18 11:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.57999999999999996,"15min":0.60399999999999998},"min":{"1min":0.438,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.99099999999999999,"5min":1.26,"15min":1.26},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.55200000000000005,"15min":0.57399999999999995},"min":{"1min":0.40400000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.86099999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.46400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318472,"kb_used_data":16956,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94053368,"statfs":{"total":96636764160,"available":96310648832,"internally_reserved":0,"allocated":17362944,"data_stored":16996076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.315}]},{"osd":1,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.35599999999999998,"15min":0.375},"min":{"1min":0.23200000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.57099999999999995,"5min":1.621,"15min":1.621},"last":0.315},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.377,"15min":0.39900000000000002},"min":{"1min":0.27100000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.70599999999999996,"5min":1.589,"15min":1.589},"last":0.33800000000000002}]},{"osd":3,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35799999999999998,"15min":0.36799999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.72399999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.39200000000000002,"15min":0.41399999999999998},"min":{"1min":0.27500000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52900000000000003,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.36199999999999999}]},{"osd":4,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53100000000000003,"15min":0.55100000000000005},"min":{"1min":0.373,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.755,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.504,"15min":0.51700000000000002},"min":{"1min":0.34300000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73199999999999998,"5min":1.012,"15min":1.012},"last":0.50700000000000001}]},{"osd":5,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53000000000000003,"15min":0.54200000000000004},"min":{"1min":0.36799999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70799999999999996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.56100000000000005},"min":{"1min":0.377,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77900000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.63600000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52400000000000002,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82399999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.53600000000000003,"15min":0.56100000000000005},"min":{"1min":0.35499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65200000000000002,"5min":1.165,"15min":1.165},"last":0.51800000000000002}]},{"osd":7,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54400000000000004,"15min":0.56000000000000005},"min":{"1min":0.371,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70099999999999996,"5min":1.073,"15min":1.073},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.39700000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91600000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.55600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322632,"kb_used_data":16956,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94049208,"statfs":{"total":96636764160,"available":96306388992,"internally_reserved":0,"allocated":17362944,"data_stored":16994733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.501}]},{"osd":2,"last update":"Wed Sep 18 11:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.379},"min":{"1min":0.26800000000000002,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.54500000000000004,"5min":1.333,"15min":1.333},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.38800000000000001,"15min":0.41699999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.57699999999999996,"5min":1.298,"15min":1.298},"last":0.32400000000000001}]},{"osd":3,"last update":"Wed Sep 18 11:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36699999999999999,"15min":0.38100000000000001},"min":{"1min":0.28100000000000003,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.503,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.35599999999999998,"15min":0.36199999999999999},"min":{"1min":0.248,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.52400000000000002,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.308}]},{"osd":4,"last update":"Wed Sep 18 11:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51400000000000001,"15min":0.52500000000000002},"min":{"1min":0.33700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.67300000000000004,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.504,"15min":0.50900000000000001},"min":{"1min":0.40500000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65300000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.45300000000000001}]},{"osd":5,"last update":"Wed Sep 18 11:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.50900000000000001,"15min":0.51900000000000002},"min":{"1min":0.38800000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.72499999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.51600000000000001,"15min":0.52600000000000002},"min":{"1min":0.436,"5min":0.308,"15min":0.308},"max":{"1min":0.747,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.51700000000000002}]},{"osd":6,"last update":"Wed Sep 18 11:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.41099999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72599999999999998,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.44},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53000000000000003,"15min":0.54200000000000004},"min":{"1min":0.35199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70899999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.42899999999999999}]},{"osd":7,"last update":"Wed Sep 18 11:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53600000000000003,"15min":0.54700000000000004},"min":{"1min":0.40899999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76800000000000002,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.57799999999999996},"min":{"1min":0.39800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.74299999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.41999999999999998}]}]},{"osd":0,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4182,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":54759424,"data_stored":54619047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23064576,"data_stored":23013811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:33:59.598 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:33:59.855 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:33:59.856 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:33:59.867 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-09-18T11:33:57.965494+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":142099530,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":712,"num_read_kb":25479,"num_write":887,"num_write_kb":266346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":449,"num_bytes_recovered":164988368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":2313624,"kb_used_data":190532,"kb_used_omap":11,"kb_used_meta":2122996,"kb_avail":658289256,"statfs":{"total":676457349120,"available":674088198144,"internally_reserved":0,"allocated":195104768,"data_stored":192338948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2173948264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":51748116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.741666"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":367,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:50.797940+0000","last_change":"2024-09-18T11:33:50.797882+0000","last_active":"2024-09-18T11:33:50.797940+0000","last_peered":"2024-09-18T11:33:50.797940+0000","last_clean":"2024-09-18T11:33:50.797940+0000","last_became_active":"2024-09-18T11:33:48.598488+0000","last_became_peered":"2024-09-18T11:33:48.598488+0000","last_unstale":"2024-09-18T11:33:50.797940+0000","last_undegraded":"2024-09-18T11:33:50.797940+0000","last_fullsized":"2024-09-18T11:33:50.797940+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'72","last_scrub_stamp":"2024-09-18T11:33:50.797790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_clean_scrub_stamp":"2024-09-18T11:33:50.797790+0000","objects_scrubbed":2,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:20.554098+0000","scrub_duration":0.001837497,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":250,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.360793+0000","last_change":"2024-09-18T11:32:42.123653+0000","last_active":"2024-09-18T11:33:49.360793+0000","last_peered":"2024-09-18T11:33:49.360793+0000","last_clean":"2024-09-18T11:33:49.360793+0000","last_became_active":"2024-09-18T11:32:27.079772+0000","last_became_peered":"2024-09-18T11:32:27.079772+0000","last_unstale":"2024-09-18T11:33:49.360793+0000","last_undegraded":"2024-09-18T11:33:49.360793+0000","last_fullsized":"2024-09-18T11:33:49.360793+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'1","last_scrub_stamp":"2024-09-18T11:32:42.123530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:32:42.123530+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:04.343044+0000","scrub_duration":0.0016360910000000001,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"182'203","reported_seq":459,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.225219+0000","last_change":"2024-09-18T11:32:39.596000+0000","last_active":"2024-09-18T11:33:49.225219+0000","last_peered":"2024-09-18T11:33:49.225219+0000","last_clean":"2024-09-18T11:33:49.225219+0000","last_became_active":"2024-09-18T11:30:01.077612+0000","last_became_peered":"2024-09-18T11:30:01.077612+0000","last_unstale":"2024-09-18T11:33:49.225219+0000","last_undegraded":"2024-09-18T11:33:49.225219+0000","last_fullsized":"2024-09-18T11:33:49.225219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'194","last_scrub_stamp":"2024-09-18T11:32:39.595890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:32:39.595890+0000","objects_scrubbed":49,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:05.467884+0000","scrub_duration":0.0058412239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42108006,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":185,"num_read_kb":4788,"num_write":230,"num_write_kb":78760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":232,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.360995+0000","last_change":"2024-09-18T11:33:21.778858+0000","last_active":"2024-09-18T11:33:49.360995+0000","last_peered":"2024-09-18T11:33:49.360995+0000","last_clean":"2024-09-18T11:33:49.360995+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:33:49.360995+0000","last_undegraded":"2024-09-18T11:33:49.360995+0000","last_fullsized":"2024-09-18T11:33:49.360995+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:21.778728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:21.778728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:33.603349+0000","scrub_duration":0.0020274260000000001,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"173'206","reported_seq":847,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.224471+0000","last_change":"2024-09-18T11:33:48.450575+0000","last_active":"2024-09-18T11:33:49.224471+0000","last_peered":"2024-09-18T11:33:49.224471+0000","last_clean":"2024-09-18T11:33:49.224471+0000","last_became_active":"2024-09-18T11:33:48.227585+0000","last_became_peered":"2024-09-18T11:33:48.227585+0000","last_unstale":"2024-09-18T11:33:49.224471+0000","last_undegraded":"2024-09-18T11:33:49.224471+0000","last_fullsized":"2024-09-18T11:33:49.224471+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'206","last_scrub_stamp":"2024-09-18T11:32:51.136969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:32:51.136969+0000","objects_scrubbed":46,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:01.520694+0000","scrub_duration":0.0050590330000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43270431,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":179,"num_read_kb":11045,"num_write":225,"num_write_kb":76253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":74278671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":147,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.225189+0000","last_change":"2024-09-18T11:33:45.869084+0000","last_active":"2024-09-18T11:33:49.225189+0000","last_peered":"2024-09-18T11:33:49.225189+0000","last_clean":"2024-09-18T11:33:49.225189+0000","last_became_active":"2024-09-18T11:32:59.474959+0000","last_became_peered":"2024-09-18T11:32:59.474959+0000","last_unstale":"2024-09-18T11:33:49.225189+0000","last_undegraded":"2024-09-18T11:33:49.225189+0000","last_fullsized":"2024-09-18T11:33:49.225189+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:45.869005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:45.869005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:47.117862+0000","scrub_duration":0.0012218940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"172'146","reported_seq":637,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.360879+0000","last_change":"2024-09-18T11:33:24.900812+0000","last_active":"2024-09-18T11:33:49.360879+0000","last_peered":"2024-09-18T11:33:49.360879+0000","last_clean":"2024-09-18T11:33:49.360879+0000","last_became_active":"2024-09-18T11:32:59.476088+0000","last_became_peered":"2024-09-18T11:32:59.476088+0000","last_unstale":"2024-09-18T11:33:49.360879+0000","last_undegraded":"2024-09-18T11:33:49.360879+0000","last_fullsized":"2024-09-18T11:33:49.360879+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"172'146","last_scrub_stamp":"2024-09-18T11:33:24.900735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:33:24.900735+0000","objects_scrubbed":35,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:35.491129+0000","scrub_duration":0.0037680470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24142412,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":133,"num_read_kb":1720,"num_write":158,"num_write_kb":46975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":29291946,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":225,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.360856+0000","last_change":"2024-09-18T11:33:04.151627+0000","last_active":"2024-09-18T11:33:49.360856+0000","last_peered":"2024-09-18T11:33:49.360856+0000","last_clean":"2024-09-18T11:33:49.360856+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:33:49.360856+0000","last_undegraded":"2024-09-18T11:33:49.360856+0000","last_fullsized":"2024-09-18T11:33:49.360856+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"18'2","last_scrub_stamp":"2024-09-18T11:33:04.151495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:04.151495+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:24.817971+0000","scrub_duration":0.0017471489999999999,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"179'157","reported_seq":769,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.858858+0000","last_change":"2024-09-18T11:33:49.858786+0000","last_active":"2024-09-18T11:33:49.858858+0000","last_peered":"2024-09-18T11:33:49.858858+0000","last_clean":"2024-09-18T11:33:49.858858+0000","last_became_active":"2024-09-18T11:33:48.598474+0000","last_became_peered":"2024-09-18T11:33:48.598474+0000","last_unstale":"2024-09-18T11:33:49.858858+0000","last_undegraded":"2024-09-18T11:33:49.858858+0000","last_fullsized":"2024-09-18T11:33:49.858858+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"179'157","last_scrub_stamp":"2024-09-18T11:33:49.858681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:33:49.858681+0000","objects_scrubbed":36,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:06.886975+0000","scrub_duration":0.0046045019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31988294,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":149,"num_read_kb":7871,"num_write":171,"num_write_kb":62728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":59830967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":267,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.360881+0000","last_change":"2024-09-18T11:33:48.226469+0000","last_active":"2024-09-18T11:33:49.360881+0000","last_peered":"2024-09-18T11:33:49.360881+0000","last_clean":"2024-09-18T11:33:49.360881+0000","last_became_active":"2024-09-18T11:33:48.226262+0000","last_became_peered":"2024-09-18T11:33:48.226262+0000","last_unstale":"2024-09-18T11:33:49.360881+0000","last_undegraded":"2024-09-18T11:33:49.360881+0000","last_fullsized":"2024-09-18T11:33:49.360881+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:44.054611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:32:30.705704+0000","last_clean_scrub_stamp":"2024-09-18T11:33:44.054611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:49.819708+0000","scrub_duration":0.0015037430000000001,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":256,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.360997+0000","last_change":"2024-09-18T11:33:48.773158+0000","last_active":"2024-09-18T11:33:49.360997+0000","last_peered":"2024-09-18T11:33:49.360997+0000","last_clean":"2024-09-18T11:33:49.360997+0000","last_became_active":"2024-09-18T11:33:48.226249+0000","last_became_peered":"2024-09-18T11:33:48.226249+0000","last_unstale":"2024-09-18T11:33:49.360997+0000","last_undegraded":"2024-09-18T11:33:49.360997+0000","last_fullsized":"2024-09-18T11:33:49.360997+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:32:42.223963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:32:42.223963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001097408,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-18T11:33:56.614457+0000","last_change":"2024-09-18T11:33:56.614431+0000","last_active":"2024-09-18T11:33:56.614457+0000","last_peered":"2024-09-18T11:33:56.614457+0000","last_clean":"2024-09-18T11:33:56.614457+0000","last_became_active":"2024-09-18T11:32:59.474991+0000","last_became_peered":"2024-09-18T11:32:59.474991+0000","last_unstale":"2024-09-18T11:33:56.614457+0000","last_undegraded":"2024-09-18T11:33:56.614457+0000","last_fullsized":"2024-09-18T11:33:56.614457+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:56.614347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:56.614347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:19.182426+0000","scrub_duration":0.0015834950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":232,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-18T11:33:49.360897+0000","last_change":"2024-09-18T11:33:24.650387+0000","last_active":"2024-09-18T11:33:49.360897+0000","last_peered":"2024-09-18T11:33:49.360897+0000","last_clean":"2024-09-18T11:33:49.360897+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:33:49.360897+0000","last_undegraded":"2024-09-18T11:33:49.360897+0000","last_fullsized":"2024-09-18T11:33:49.360897+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:24.650294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:24.650294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:34.919021+0000","scrub_duration":0.001363908,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":141509143,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":646,"num_read_kb":25424,"num_write":784,"num_write_kb":264716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":441,"num_bytes_recovered":163401584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":240234496,"data_stored":239663954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4182,"internal_metadata":0},"log_size":712,"ondisk_log_size":712,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302752,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069088,"statfs":{"total":96636764160,"available":96326746112,"internally_reserved":0,"allocated":1265664,"data_stored":934202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.50900000000000001}]},{"osd":1,"last update":"Wed Sep 18 11:33:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.46600000000000003,"15min":0.46700000000000003},"min":{"1min":0.36599999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.64600000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51100000000000001,"15min":0.52000000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.68300000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.66000000000000003}]},{"osd":2,"last update":"Wed Sep 18 11:33:55 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.436,"15min":0.42599999999999999},"min":{"1min":0.378,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.63400000000000001,"5min":1.085,"15min":1.085},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.44600000000000001,"15min":0.441},"min":{"1min":0.40999999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.50600000000000001}]},{"osd":3,"last update":"Wed Sep 18 11:33:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.46000000000000002,"15min":0.44900000000000001},"min":{"1min":0.35499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.47899999999999998,"15min":0.48899999999999999},"min":{"1min":0.39900000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.70199999999999996,"5min":1.177,"15min":1.177},"last":0.52200000000000002}]},{"osd":4,"last update":"Wed Sep 18 11:33:55 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.43099999999999999,"15min":0.45500000000000002},"min":{"1min":0.28599999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.61199999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47199999999999998,"15min":0.496},"min":{"1min":0.33600000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69399999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.69399999999999995}]},{"osd":5,"last update":"Wed Sep 18 11:33:55 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40699999999999997,"15min":0.436},"min":{"1min":0.29699999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.52700000000000002,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.432},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.437,"15min":0.46300000000000002},"min":{"1min":0.33800000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.68100000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.47299999999999998}]},{"osd":6,"last update":"Wed Sep 18 11:33:55 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48699999999999999,"15min":0.51900000000000002},"min":{"1min":0.32000000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.70399999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.48499999999999999,"15min":0.501},"min":{"1min":0.375,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69199999999999995,"5min":0.75,"15min":0.75},"last":0.57299999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325260,"kb_used_data":23744,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94046580,"statfs":{"total":96636764160,"available":96303697920,"internally_reserved":0,"allocated":24313856,"data_stored":23943675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.55400000000000005}]},{"osd":1,"last update":"Wed Sep 18 11:33:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.26300000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.35899999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.58399999999999996,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.434}]},{"osd":2,"last update":"Wed Sep 18 11:33:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.47099999999999997,"15min":0.46700000000000003},"min":{"1min":0.38100000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.66300000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.46800000000000003,"15min":0.45200000000000001},"min":{"1min":0.36899999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.42399999999999999}]},{"osd":3,"last update":"Wed Sep 18 11:33:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48399999999999999,"15min":0.50800000000000001},"min":{"1min":0.28599999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.64100000000000001,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.50700000000000001,"15min":0.52000000000000002},"min":{"1min":0.307,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69499999999999995,"5min":1.032,"15min":1.032},"last":0.504}]},{"osd":4,"last update":"Wed Sep 18 11:33:53 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.41199999999999998,"15min":0.39700000000000002},"min":{"1min":0.21099999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.45900000000000002,"15min":0.433},"min":{"1min":0.34300000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.58499999999999996}]},{"osd":5,"last update":"Wed Sep 18 11:33:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.435,"15min":0.41599999999999998},"min":{"1min":0.317,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.45000000000000001,"15min":0.436},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66500000000000004,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.52700000000000002}]},{"osd":7,"last update":"Wed Sep 18 11:33:53 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47099999999999997,"15min":0.46600000000000003},"min":{"1min":0.31,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48799999999999999,"15min":0.47999999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78600000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.54800000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332264,"kb_used_data":26588,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94039576,"statfs":{"total":96636764160,"available":96296525824,"internally_reserved":0,"allocated":27226112,"data_stored":26819345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.501},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.47899999999999998}]},{"osd":1,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42399999999999999,"15min":0.42999999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.44700000000000001,"15min":0.44900000000000001},"min":{"1min":0.248,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47999999999999998,"15min":0.496},"min":{"1min":0.316,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66600000000000004,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.627},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.47199999999999998,"15min":0.49399999999999999},"min":{"1min":0.32400000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.59599999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.57999999999999996}]},{"osd":3,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.46700000000000003,"15min":0.48899999999999999},"min":{"1min":0.29899999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.51300000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.48399999999999999,"15min":0.496},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.61299999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.59499999999999997}]},{"osd":4,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42999999999999999,"15min":0.44400000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.435,"15min":0.45700000000000002},"min":{"1min":0.22900000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.65700000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.42699999999999999}]},{"osd":6,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.25600000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.46700000000000003,"15min":0.48599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.501}]},{"osd":7,"last update":"Wed Sep 18 11:32:53 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.47499999999999998,"15min":0.499},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.72899999999999998,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.503,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.53700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356520,"kb_used_data":50844,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94015320,"statfs":{"total":96636764160,"available":96271687680,"internally_reserved":0,"allocated":52064256,"data_stored":51624124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.748},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.66600000000000004}]},{"osd":1,"last update":"Wed Sep 18 11:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.47799999999999998,"15min":0.5},"min":{"1min":0.28899999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67400000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.48299999999999998,"15min":0.498},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.73599999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed Sep 18 11:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.48299999999999998,"15min":0.49299999999999999},"min":{"1min":0.30499999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71699999999999997,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48999999999999999,"15min":0.51600000000000001},"min":{"1min":0.33400000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72199999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.51600000000000001}]},{"osd":3,"last update":"Wed Sep 18 11:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48699999999999999,"15min":0.51200000000000001},"min":{"1min":0.32800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65800000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.504,"15min":0.52600000000000002},"min":{"1min":0.34499999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.71899999999999997,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.70699999999999996}]},{"osd":5,"last update":"Wed Sep 18 11:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.42599999999999999,"15min":0.45700000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.70499999999999996,"5min":1.002,"15min":1.002},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.46100000000000002,"15min":0.501},"min":{"1min":0.29899999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.627,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.61299999999999999}]},{"osd":6,"last update":"Wed Sep 18 11:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.443,"15min":0.46700000000000003},"min":{"1min":0.252,"5min":0.245,"15min":0.245},"max":{"1min":0.68200000000000005,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46999999999999997,"15min":0.499},"min":{"1min":0.32500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.625,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.441}]},{"osd":7,"last update":"Wed Sep 18 11:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.48399999999999999,"15min":0.50700000000000001},"min":{"1min":0.28599999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69399999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.54300000000000004},"min":{"1min":0.28499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.77000000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.505}]}]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":355724,"kb_used_data":54208,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94016116,"statfs":{"total":96636764160,"available":96272502784,"internally_reserved":0,"allocated":55508992,"data_stored":55026793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.34999999999999998}]},{"osd":1,"last update":"Wed Sep 18 11:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.33400000000000002,"15min":0.34200000000000003},"min":{"1min":0.22500000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.56999999999999995,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.36799999999999999,"15min":0.372},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.69499999999999995,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.379}]},{"osd":2,"last update":"Wed Sep 18 11:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.38100000000000001,"15min":0.38200000000000001},"min":{"1min":0.23699999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.67800000000000005,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.38,"15min":0.40000000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.73399999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.36499999999999999}]},{"osd":4,"last update":"Wed Sep 18 11:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.52900000000000003,"15min":0.52000000000000002},"min":{"1min":0.375,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.54600000000000004,"15min":0.53200000000000003},"min":{"1min":0.45700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56999999999999995}]},{"osd":5,"last update":"Wed Sep 18 11:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.504,"15min":0.50800000000000001},"min":{"1min":0.34399999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.54200000000000004,"15min":0.55900000000000005},"min":{"1min":0.44,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94899999999999995,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.51600000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.51800000000000002,"15min":0.53400000000000003},"min":{"1min":0.36799999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.55100000000000005,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.61199999999999999}]},{"osd":7,"last update":"Wed Sep 18 11:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.57999999999999996,"15min":0.60399999999999998},"min":{"1min":0.438,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.99099999999999999,"5min":1.26,"15min":1.26},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.55200000000000005,"15min":0.57399999999999995},"min":{"1min":0.40400000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.86099999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.46400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318472,"kb_used_data":16956,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94053368,"statfs":{"total":96636764160,"available":96310648832,"internally_reserved":0,"allocated":17362944,"data_stored":16996076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.315}]},{"osd":1,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.35599999999999998,"15min":0.375},"min":{"1min":0.23200000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.57099999999999995,"5min":1.621,"15min":1.621},"last":0.315},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.377,"15min":0.39900000000000002},"min":{"1min":0.27100000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.70599999999999996,"5min":1.589,"15min":1.589},"last":0.33800000000000002}]},{"osd":3,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35799999999999998,"15min":0.36799999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.72399999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.39200000000000002,"15min":0.41399999999999998},"min":{"1min":0.27500000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52900000000000003,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.36199999999999999}]},{"osd":4,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53100000000000003,"15min":0.55100000000000005},"min":{"1min":0.373,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.755,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.504,"15min":0.51700000000000002},"min":{"1min":0.34300000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73199999999999998,"5min":1.012,"15min":1.012},"last":0.50700000000000001}]},{"osd":5,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53000000000000003,"15min":0.54200000000000004},"min":{"1min":0.36799999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70799999999999996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.56100000000000005},"min":{"1min":0.377,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77900000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.63600000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52400000000000002,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82399999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.53600000000000003,"15min":0.56100000000000005},"min":{"1min":0.35499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65200000000000002,"5min":1.165,"15min":1.165},"last":0.51800000000000002}]},{"osd":7,"last update":"Wed Sep 18 11:32:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54400000000000004,"15min":0.56000000000000005},"min":{"1min":0.371,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70099999999999996,"5min":1.073,"15min":1.073},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.39700000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91600000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.55600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322632,"kb_used_data":16956,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94049208,"statfs":{"total":96636764160,"available":96306388992,"internally_reserved":0,"allocated":17362944,"data_stored":16994733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.501}]},{"osd":2,"last update":"Wed Sep 18 11:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.379},"min":{"1min":0.26800000000000002,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.54500000000000004,"5min":1.333,"15min":1.333},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.38800000000000001,"15min":0.41699999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.57699999999999996,"5min":1.298,"15min":1.298},"last":0.32400000000000001}]},{"osd":3,"last update":"Wed Sep 18 11:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36699999999999999,"15min":0.38100000000000001},"min":{"1min":0.28100000000000003,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.503,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.35599999999999998,"15min":0.36199999999999999},"min":{"1min":0.248,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.52400000000000002,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.308}]},{"osd":4,"last update":"Wed Sep 18 11:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51400000000000001,"15min":0.52500000000000002},"min":{"1min":0.33700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.67300000000000004,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.504,"15min":0.50900000000000001},"min":{"1min":0.40500000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65300000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.45300000000000001}]},{"osd":5,"last update":"Wed Sep 18 11:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.50900000000000001,"15min":0.51900000000000002},"min":{"1min":0.38800000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.72499999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.51600000000000001,"15min":0.52600000000000002},"min":{"1min":0.436,"5min":0.308,"15min":0.308},"max":{"1min":0.747,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.51700000000000002}]},{"osd":6,"last update":"Wed Sep 18 11:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.41099999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72599999999999998,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.44},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53000000000000003,"15min":0.54200000000000004},"min":{"1min":0.35199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70899999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.42899999999999999}]},{"osd":7,"last update":"Wed Sep 18 11:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53600000000000003,"15min":0.54700000000000004},"min":{"1min":0.40899999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76800000000000002,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.57799999999999996},"min":{"1min":0.39800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.74299999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.41999999999999998}]}]},{"osd":0,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4182,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":54759424,"data_stored":54619047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23064576,"data_stored":23013811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:33:59.869 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T11:33:59.870 INFO:tasks.thrashosds.thrasher:Waiting for trim 2024-09-18T11:34:39.871 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2024-09-18T11:34:39.871 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-18T11:34:39.872 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-18T11:34:39.875 INFO:tasks.ceph.osd.0:Started 2024-09-18T11:34:39.875 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:34:39.980 INFO:teuthology.orchestra.run.smithi119.stderr:no valid command found; 10 closest matches: 2024-09-18T11:34:39.980 INFO:teuthology.orchestra.run.smithi119.stderr:0 2024-09-18T11:34:39.980 INFO:teuthology.orchestra.run.smithi119.stderr:1 2024-09-18T11:34:39.980 INFO:teuthology.orchestra.run.smithi119.stderr:2 2024-09-18T11:34:39.980 INFO:teuthology.orchestra.run.smithi119.stderr:abort 2024-09-18T11:34:39.980 INFO:teuthology.orchestra.run.smithi119.stderr:assert 2024-09-18T11:34:39.981 INFO:teuthology.orchestra.run.smithi119.stderr:config diff 2024-09-18T11:34:39.981 INFO:teuthology.orchestra.run.smithi119.stderr:config diff get 2024-09-18T11:34:39.981 INFO:teuthology.orchestra.run.smithi119.stderr:config get 2024-09-18T11:34:39.981 INFO:teuthology.orchestra.run.smithi119.stderr:config help [] 2024-09-18T11:34:39.981 INFO:teuthology.orchestra.run.smithi119.stderr:config set ... 2024-09-18T11:34:39.981 INFO:teuthology.orchestra.run.smithi119.stderr:admin_socket: invalid command 2024-09-18T11:34:39.982 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T11:34:39.983 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-18T11:34:40.468 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:34:40.466+0000 7f85c2b23440 -1 Falling back to public interface 2024-09-18T11:34:40.713 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:34:40.712+0000 7fcb662ad640 -1 osd.7 210 handle_osd_map injecting map crc failure 2024-09-18T11:34:42.095 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:34:42.093+0000 7f6bf5056640 -1 osd.5 211 handle_osd_map injecting map crc failure 2024-09-18T11:34:43.096 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:34:43.095+0000 7f3157e90640 -1 osd.4 211 handle_osd_map injecting map crc failure 2024-09-18T11:34:43.936 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:34:43.935+0000 7f85c2b23440 -1 osd.0 205 log_to_monitors true 2024-09-18T11:34:44.124 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:34:44.123+0000 7f85b85ec640 -1 osd.0 205 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T11:34:44.908 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-09-18T11:34:44.906+0000 7f85b3abd640 -1 osd.0 205 handle_osd_map injecting map crc failure 2024-09-18T11:34:44.984 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T11:34:45.095 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:34:45.095 INFO:teuthology.orchestra.run.smithi119.stdout: "ops": [], 2024-09-18T11:34:45.095 INFO:teuthology.orchestra.run.smithi119.stdout: "num_ops": 0 2024-09-18T11:34:45.096 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:34:45.104 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T11:34:45.105 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-18T11:34:45.105 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-18T11:34:45.378 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-09-18T11:34:45.377+0000 7ff6f6bf1640 -1 mon.a@0(leader).osd e215 definitely_dead 0 2024-09-18T11:34:46.132 INFO:teuthology.orchestra.run.smithi119.stderr:marked in osd.0. 2024-09-18T11:34:46.136 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:34:46.135+0000 7f2d449e8640 -1 osd.1 214 handle_osd_map injecting map crc failure 2024-09-18T11:34:46.150 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-18T11:34:49.142 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:34:49.140+0000 7f2d449e8640 -1 osd.1 217 handle_osd_map injecting map crc failure 2024-09-18T11:34:50.150 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:34:50.147+0000 7fcb662ad640 -1 osd.7 219 handle_osd_map injecting map crc failure 2024-09-18T11:34:51.151 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-18T11:34:51.269 INFO:teuthology.orchestra.run.smithi119.stdout:mon.a: {} 2024-09-18T11:34:51.269 INFO:teuthology.orchestra.run.smithi119.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T11:34:51.278 INFO:teuthology.orchestra.run.smithi119.stdout:mon.b: {} 2024-09-18T11:34:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T11:34:51.286 INFO:teuthology.orchestra.run.smithi119.stdout:mon.c: {} 2024-09-18T11:34:51.287 INFO:teuthology.orchestra.run.smithi119.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T11:34:51.297 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T11:34:52.156 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T11:34:52.172 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-18T11:34:53.158 INFO:teuthology.orchestra.run.smithi119.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-18T11:34:53.173 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T11:34:54.169 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T11:34:54.184 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-18T11:34:54.685 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:34:54.683+0000 7fcb682b1640 -1 osd.7 221 handle_osd_map injecting map crc failure 2024-09-18T11:34:54.885 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:34:54.883+0000 7f6bf5056640 -1 osd.5 221 handle_osd_map injecting map crc failure 2024-09-18T11:34:55.173 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-18T11:34:55.171+0000 7f3157e90640 -1 osd.4 223 handle_osd_map injecting map crc failure 2024-09-18T11:34:55.176 INFO:teuthology.orchestra.run.smithi119.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-18T11:34:55.191 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T11:34:56.150 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:34:56.147+0000 7f6bf3052640 -1 osd.5 223 handle_osd_map injecting map crc failure 2024-09-18T11:34:56.186 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T11:34:56.200 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-18T11:34:56.668 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:34:56.665+0000 7fd27659f640 -1 osd.6 224 handle_osd_map injecting map crc failure 2024-09-18T11:34:57.186 INFO:teuthology.orchestra.run.smithi119.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-18T11:34:57.201 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T11:34:58.189 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:34:58.187+0000 7f697ceb1640 -1 osd.3 226 handle_osd_map injecting map crc failure 2024-09-18T11:34:58.192 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T11:34:58.207 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-18T11:34:59.197 INFO:teuthology.orchestra.run.smithi119.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-18T11:34:59.214 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-18T11:35:00.201 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:35:00.198+0000 7f2d449e8640 -1 osd.1 228 handle_osd_map injecting map crc failure 2024-09-18T11:35:00.203 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.5 to 1 (10000) 2024-09-18T11:35:00.218 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-18T11:35:01.206 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-09-18T11:35:01.204+0000 7f697ceb1640 -1 osd.3 229 handle_osd_map injecting map crc failure 2024-09-18T11:35:01.207 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:35:01.205+0000 7f2d449e8640 -1 osd.1 229 handle_osd_map injecting map crc failure 2024-09-18T11:35:01.208 INFO:teuthology.orchestra.run.smithi119.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-18T11:35:01.224 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-18T11:35:02.217 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.6 to 1 (10000) 2024-09-18T11:35:02.233 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-18T11:35:03.224 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:35:03.222+0000 7f2d449e8640 -1 osd.1 231 handle_osd_map injecting map crc failure 2024-09-18T11:35:03.225 INFO:teuthology.orchestra.run.smithi119.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-18T11:35:03.240 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-18T11:35:03.717 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-18T11:35:03.716+0000 7fcb662ad640 -1 osd.7 231 handle_osd_map injecting map crc failure 2024-09-18T11:35:03.717 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:35:03.716+0000 7f6bf3052640 -1 osd.5 230 handle_osd_map injecting map crc failure 2024-09-18T11:35:04.232 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-09-18T11:35:04.229+0000 7fb6950ff640 -1 osd.2 232 handle_osd_map injecting map crc failure 2024-09-18T11:35:04.234 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.7 to 1 (10000) 2024-09-18T11:35:04.247 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-18T11:35:04.595 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-18T11:35:04.594+0000 7f6bf3052640 -1 osd.5 232 handle_osd_map injecting map crc failure 2024-09-18T11:35:05.235 INFO:teuthology.orchestra.run.smithi119.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-18T11:35:05.251 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T11:35:06.241 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-18T11:35:06.239+0000 7fd2785a3640 -1 osd.6 234 handle_osd_map injecting map crc failure 2024-09-18T11:35:06.248 INFO:teuthology.orchestra.run.smithi119.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T11:35:06.263 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-18T11:35:07.257 INFO:teuthology.orchestra.run.smithi119.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-18T11:35:07.272 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-18T11:35:07.272 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-18T11:35:07.272 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-18T11:35:07.272 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-18T11:35:07.272 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-18T11:35:07.272 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T11:35:07.534 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:35:07.534 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":236,"fsid":"f4cca259-b87a-45cc-8995-7e9fca891a9f","created":"2024-09-18T11:29:38.205487+0000","modified":"2024-09-18T11:35:07.250663+0000","last_up_change":"2024-09-18T11:34:45.106232+0000","last_in_change":"2024-09-18T11:34:46.116722+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T11:29:47.951262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T11:29:48.866238+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"low_tier","create_time":"2024-09-18T11:29:59.343964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"12faced2-7888-4265-ad12-815efe36466c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":119,"last_clean_end":205,"up_from":215,"up_thru":219,"down_at":206,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":74987},{"type":"v1","addr":"172.21.15.119:6801","nonce":74987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":74987},{"type":"v1","addr":"172.21.15.119:6803","nonce":74987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":74987},{"type":"v1","addr":"172.21.15.119:6807","nonce":74987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":74987},{"type":"v1","addr":"172.21.15.119:6805","nonce":74987}]},"public_addr":"172.21.15.119:6801/74987","cluster_addr":"172.21.15.119:6803/74987","heartbeat_back_addr":"172.21.15.119:6807/74987","heartbeat_front_addr":"172.21.15.119:6805/74987","state":["exists","up"]},{"osd":1,"uuid":"20fee29d-b047-46ae-aa60-88e897fbb0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":33350},{"type":"v1","addr":"172.21.15.119:6825","nonce":33350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":33350},{"type":"v1","addr":"172.21.15.119:6827","nonce":33350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":33350},{"type":"v1","addr":"172.21.15.119:6831","nonce":33350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":33350},{"type":"v1","addr":"172.21.15.119:6829","nonce":33350}]},"public_addr":"172.21.15.119:6825/33350","cluster_addr":"172.21.15.119:6827/33350","heartbeat_back_addr":"172.21.15.119:6831/33350","heartbeat_front_addr":"172.21.15.119:6829/33350","state":["exists","up"]},{"osd":2,"uuid":"21a296f3-b259-4841-9a17-4cd21b302b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":33351},{"type":"v1","addr":"172.21.15.119:6809","nonce":33351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":33351},{"type":"v1","addr":"172.21.15.119:6813","nonce":33351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6819","nonce":33351},{"type":"v1","addr":"172.21.15.119:6821","nonce":33351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":33351},{"type":"v1","addr":"172.21.15.119:6817","nonce":33351}]},"public_addr":"172.21.15.119:6809/33351","cluster_addr":"172.21.15.119:6813/33351","heartbeat_back_addr":"172.21.15.119:6821/33351","heartbeat_front_addr":"172.21.15.119:6817/33351","state":["exists","up"]},{"osd":3,"uuid":"59044f30-32e1-4942-9a8e-202c1def221a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":33349},{"type":"v1","addr":"172.21.15.119:6812","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":33349},{"type":"v1","addr":"172.21.15.119:6816","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":33349},{"type":"v1","addr":"172.21.15.119:6823","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":33349},{"type":"v1","addr":"172.21.15.119:6820","nonce":33349}]},"public_addr":"172.21.15.119:6812/33349","cluster_addr":"172.21.15.119:6816/33349","heartbeat_back_addr":"172.21.15.119:6823/33349","heartbeat_front_addr":"172.21.15.119:6820/33349","state":["exists","up"]},{"osd":4,"uuid":"7d1ae3d0-8d69-4d3c-9e51-24fdd77a22a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":33088},{"type":"v1","addr":"172.21.15.148:6809","nonce":33088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":33088},{"type":"v1","addr":"172.21.15.148:6811","nonce":33088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":33088},{"type":"v1","addr":"172.21.15.148:6815","nonce":33088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":33088},{"type":"v1","addr":"172.21.15.148:6813","nonce":33088}]},"public_addr":"172.21.15.148:6809/33088","cluster_addr":"172.21.15.148:6811/33088","heartbeat_back_addr":"172.21.15.148:6815/33088","heartbeat_front_addr":"172.21.15.148:6813/33088","state":["exists","up"]},{"osd":5,"uuid":"b0dd3d30-b273-428c-a4a9-4c238abf45fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":33089},{"type":"v1","addr":"172.21.15.148:6801","nonce":33089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":33089},{"type":"v1","addr":"172.21.15.148:6803","nonce":33089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":33089},{"type":"v1","addr":"172.21.15.148:6807","nonce":33089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":33089},{"type":"v1","addr":"172.21.15.148:6805","nonce":33089}]},"public_addr":"172.21.15.148:6801/33089","cluster_addr":"172.21.15.148:6803/33089","heartbeat_back_addr":"172.21.15.148:6807/33089","heartbeat_front_addr":"172.21.15.148:6805/33089","state":["exists","up"]},{"osd":6,"uuid":"40290cb3-2334-4c68-8e6c-59d0d5455171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":33107},{"type":"v1","addr":"172.21.15.148:6817","nonce":33107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":33107},{"type":"v1","addr":"172.21.15.148:6819","nonce":33107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":33107},{"type":"v1","addr":"172.21.15.148:6823","nonce":33107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":33107},{"type":"v1","addr":"172.21.15.148:6821","nonce":33107}]},"public_addr":"172.21.15.148:6817/33107","cluster_addr":"172.21.15.148:6819/33107","heartbeat_back_addr":"172.21.15.148:6823/33107","heartbeat_front_addr":"172.21.15.148:6821/33107","state":["exists","up"]},{"osd":7,"uuid":"54802abe-ea1a-448e-85af-d1f14a45be27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":33119},{"type":"v1","addr":"172.21.15.148:6825","nonce":33119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":33119},{"type":"v1","addr":"172.21.15.148:6827","nonce":33119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":33119},{"type":"v1","addr":"172.21.15.148:6831","nonce":33119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":33119},{"type":"v1","addr":"172.21.15.148:6829","nonce":33119}]},"public_addr":"172.21.15.148:6825/33119","cluster_addr":"172.21.15.148:6827/33119","heartbeat_back_addr":"172.21.15.148:6831/33119","heartbeat_front_addr":"172.21.15.148:6829/33119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:34:44.910546+0000","dead_epoch":206},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:34:41.761116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:33:55.714008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:34:39.587239+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:35:00.703150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:35:02.069253+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:34:42.647805+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:34:51.482687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T11:35:07.547 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-18T11:35:07.547 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T11:35:07.811 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:35:07.811 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":236,"fsid":"f4cca259-b87a-45cc-8995-7e9fca891a9f","created":"2024-09-18T11:29:38.205487+0000","modified":"2024-09-18T11:35:07.250663+0000","last_up_change":"2024-09-18T11:34:45.106232+0000","last_in_change":"2024-09-18T11:34:46.116722+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T11:29:47.951262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T11:29:48.866238+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"low_tier","create_time":"2024-09-18T11:29:59.343964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"12faced2-7888-4265-ad12-815efe36466c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":119,"last_clean_end":205,"up_from":215,"up_thru":219,"down_at":206,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":74987},{"type":"v1","addr":"172.21.15.119:6801","nonce":74987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":74987},{"type":"v1","addr":"172.21.15.119:6803","nonce":74987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":74987},{"type":"v1","addr":"172.21.15.119:6807","nonce":74987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":74987},{"type":"v1","addr":"172.21.15.119:6805","nonce":74987}]},"public_addr":"172.21.15.119:6801/74987","cluster_addr":"172.21.15.119:6803/74987","heartbeat_back_addr":"172.21.15.119:6807/74987","heartbeat_front_addr":"172.21.15.119:6805/74987","state":["exists","up"]},{"osd":1,"uuid":"20fee29d-b047-46ae-aa60-88e897fbb0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":33350},{"type":"v1","addr":"172.21.15.119:6825","nonce":33350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":33350},{"type":"v1","addr":"172.21.15.119:6827","nonce":33350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":33350},{"type":"v1","addr":"172.21.15.119:6831","nonce":33350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":33350},{"type":"v1","addr":"172.21.15.119:6829","nonce":33350}]},"public_addr":"172.21.15.119:6825/33350","cluster_addr":"172.21.15.119:6827/33350","heartbeat_back_addr":"172.21.15.119:6831/33350","heartbeat_front_addr":"172.21.15.119:6829/33350","state":["exists","up"]},{"osd":2,"uuid":"21a296f3-b259-4841-9a17-4cd21b302b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":33351},{"type":"v1","addr":"172.21.15.119:6809","nonce":33351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":33351},{"type":"v1","addr":"172.21.15.119:6813","nonce":33351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6819","nonce":33351},{"type":"v1","addr":"172.21.15.119:6821","nonce":33351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":33351},{"type":"v1","addr":"172.21.15.119:6817","nonce":33351}]},"public_addr":"172.21.15.119:6809/33351","cluster_addr":"172.21.15.119:6813/33351","heartbeat_back_addr":"172.21.15.119:6821/33351","heartbeat_front_addr":"172.21.15.119:6817/33351","state":["exists","up"]},{"osd":3,"uuid":"59044f30-32e1-4942-9a8e-202c1def221a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":33349},{"type":"v1","addr":"172.21.15.119:6812","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":33349},{"type":"v1","addr":"172.21.15.119:6816","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":33349},{"type":"v1","addr":"172.21.15.119:6823","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":33349},{"type":"v1","addr":"172.21.15.119:6820","nonce":33349}]},"public_addr":"172.21.15.119:6812/33349","cluster_addr":"172.21.15.119:6816/33349","heartbeat_back_addr":"172.21.15.119:6823/33349","heartbeat_front_addr":"172.21.15.119:6820/33349","state":["exists","up"]},{"osd":4,"uuid":"7d1ae3d0-8d69-4d3c-9e51-24fdd77a22a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":33088},{"type":"v1","addr":"172.21.15.148:6809","nonce":33088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":33088},{"type":"v1","addr":"172.21.15.148:6811","nonce":33088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":33088},{"type":"v1","addr":"172.21.15.148:6815","nonce":33088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":33088},{"type":"v1","addr":"172.21.15.148:6813","nonce":33088}]},"public_addr":"172.21.15.148:6809/33088","cluster_addr":"172.21.15.148:6811/33088","heartbeat_back_addr":"172.21.15.148:6815/33088","heartbeat_front_addr":"172.21.15.148:6813/33088","state":["exists","up"]},{"osd":5,"uuid":"b0dd3d30-b273-428c-a4a9-4c238abf45fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":33089},{"type":"v1","addr":"172.21.15.148:6801","nonce":33089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":33089},{"type":"v1","addr":"172.21.15.148:6803","nonce":33089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":33089},{"type":"v1","addr":"172.21.15.148:6807","nonce":33089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":33089},{"type":"v1","addr":"172.21.15.148:6805","nonce":33089}]},"public_addr":"172.21.15.148:6801/33089","cluster_addr":"172.21.15.148:6803/33089","heartbeat_back_addr":"172.21.15.148:6807/33089","heartbeat_front_addr":"172.21.15.148:6805/33089","state":["exists","up"]},{"osd":6,"uuid":"40290cb3-2334-4c68-8e6c-59d0d5455171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":33107},{"type":"v1","addr":"172.21.15.148:6817","nonce":33107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":33107},{"type":"v1","addr":"172.21.15.148:6819","nonce":33107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":33107},{"type":"v1","addr":"172.21.15.148:6823","nonce":33107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":33107},{"type":"v1","addr":"172.21.15.148:6821","nonce":33107}]},"public_addr":"172.21.15.148:6817/33107","cluster_addr":"172.21.15.148:6819/33107","heartbeat_back_addr":"172.21.15.148:6823/33107","heartbeat_front_addr":"172.21.15.148:6821/33107","state":["exists","up"]},{"osd":7,"uuid":"54802abe-ea1a-448e-85af-d1f14a45be27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":33119},{"type":"v1","addr":"172.21.15.148:6825","nonce":33119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":33119},{"type":"v1","addr":"172.21.15.148:6827","nonce":33119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":33119},{"type":"v1","addr":"172.21.15.148:6831","nonce":33119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":33119},{"type":"v1","addr":"172.21.15.148:6829","nonce":33119}]},"public_addr":"172.21.15.148:6825/33119","cluster_addr":"172.21.15.148:6827/33119","heartbeat_back_addr":"172.21.15.148:6831/33119","heartbeat_front_addr":"172.21.15.148:6829/33119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:34:44.910546+0000","dead_epoch":206},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:34:41.761116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:33:55.714008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:34:39.587239+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:35:00.703150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:35:02.069253+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:34:42.647805+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:34:51.482687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T11:35:07.825 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-18T11:35:07.825 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-18T11:35:07.826 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-18T11:35:07.826 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-18T11:35:07.827 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-18T11:35:07.827 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-18T11:35:07.827 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-18T11:35:07.828 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-18T11:35:08.039 INFO:teuthology.orchestra.run.smithi119.stdout:55834574917 2024-09-18T11:35:08.039 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T11:35:08.051 INFO:teuthology.orchestra.run.smithi119.stdout:55834574916 2024-09-18T11:35:08.052 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T11:35:08.054 INFO:teuthology.orchestra.run.smithi119.stdout:55834574915 2024-09-18T11:35:08.054 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T11:35:08.058 INFO:teuthology.orchestra.run.smithi119.stdout:55834574915 2024-09-18T11:35:08.058 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T11:35:08.060 INFO:teuthology.orchestra.run.smithi119.stdout:55834574915 2024-09-18T11:35:08.060 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T11:35:08.064 INFO:teuthology.orchestra.run.smithi119.stdout:923417968647 2024-09-18T11:35:08.065 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T11:35:08.065 INFO:teuthology.orchestra.run.smithi119.stdout:55834574916 2024-09-18T11:35:08.066 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T11:35:08.067 INFO:teuthology.orchestra.run.smithi119.stdout:55834574915 2024-09-18T11:35:08.067 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T11:35:08.562 INFO:teuthology.orchestra.run.smithi119.stdout:55834574916 2024-09-18T11:35:08.581 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574917 got 55834574916 for osd.3 2024-09-18T11:35:08.583 INFO:teuthology.orchestra.run.smithi119.stdout:55834574914 2024-09-18T11:35:08.588 INFO:teuthology.orchestra.run.smithi119.stdout:55834574915 2024-09-18T11:35:08.589 INFO:teuthology.orchestra.run.smithi119.stdout:55834574914 2024-09-18T11:35:08.604 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574915 got 55834574914 for osd.2 2024-09-18T11:35:08.609 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574915 got 55834574914 for osd.6 2024-09-18T11:35:08.610 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574916 got 55834574915 for osd.1 2024-09-18T11:35:08.621 INFO:teuthology.orchestra.run.smithi119.stdout:55834574915 2024-09-18T11:35:08.626 INFO:teuthology.orchestra.run.smithi119.stdout:55834574914 2024-09-18T11:35:08.632 INFO:teuthology.orchestra.run.smithi119.stdout:55834574914 2024-09-18T11:35:08.634 INFO:teuthology.orchestra.run.smithi119.stdout:923417968646 2024-09-18T11:35:08.637 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574916 got 55834574915 for osd.5 2024-09-18T11:35:08.640 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574915 got 55834574914 for osd.7 2024-09-18T11:35:08.650 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574915 got 55834574914 for osd.4 2024-09-18T11:35:08.650 INFO:tasks.ceph.ceph_manager.ceph:need seq 923417968647 got 923417968646 for osd.0 2024-09-18T11:35:09.582 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T11:35:09.604 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T11:35:09.610 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T11:35:09.610 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T11:35:09.638 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T11:35:09.642 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T11:35:09.650 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T11:35:09.651 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T11:35:10.067 INFO:teuthology.orchestra.run.smithi119.stdout:55834574916 2024-09-18T11:35:10.086 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574917 got 55834574916 for osd.3 2024-09-18T11:35:10.137 INFO:teuthology.orchestra.run.smithi119.stdout:55834574914 2024-09-18T11:35:10.142 INFO:teuthology.orchestra.run.smithi119.stdout:55834574915 2024-09-18T11:35:10.148 INFO:teuthology.orchestra.run.smithi119.stdout:55834574915 2024-09-18T11:35:10.150 INFO:teuthology.orchestra.run.smithi119.stdout:55834574914 2024-09-18T11:35:10.159 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574915 got 55834574914 for osd.6 2024-09-18T11:35:10.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574916 got 55834574915 for osd.5 2024-09-18T11:35:10.168 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574915 got 55834574914 for osd.2 2024-09-18T11:35:10.169 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574916 got 55834574915 for osd.1 2024-09-18T11:35:10.188 INFO:teuthology.orchestra.run.smithi119.stdout:55834574914 2024-09-18T11:35:10.188 INFO:teuthology.orchestra.run.smithi119.stdout:923417968646 2024-09-18T11:35:10.196 INFO:teuthology.orchestra.run.smithi119.stdout:55834574914 2024-09-18T11:35:10.201 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574915 got 55834574914 for osd.7 2024-09-18T11:35:10.202 INFO:tasks.ceph.ceph_manager.ceph:need seq 923417968647 got 923417968646 for osd.0 2024-09-18T11:35:10.209 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574915 got 55834574914 for osd.4 2024-09-18T11:35:11.087 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T11:35:11.160 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T11:35:11.162 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T11:35:11.169 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T11:35:11.170 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T11:35:11.202 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T11:35:11.203 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T11:35:11.210 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T11:35:11.523 INFO:teuthology.orchestra.run.smithi119.stdout:55834574917 2024-09-18T11:35:11.540 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574917 got 55834574917 for osd.3 2024-09-18T11:35:11.540 DEBUG:teuthology.parallel:result is None 2024-09-18T11:35:11.607 INFO:teuthology.orchestra.run.smithi119.stdout:55834574916 2024-09-18T11:35:11.620 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574916 got 55834574916 for osd.5 2024-09-18T11:35:11.620 DEBUG:teuthology.parallel:result is None 2024-09-18T11:35:11.648 INFO:teuthology.orchestra.run.smithi119.stdout:55834574915 2024-09-18T11:35:11.665 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574915 got 55834574915 for osd.6 2024-09-18T11:35:11.665 DEBUG:teuthology.parallel:result is None 2024-09-18T11:35:11.710 INFO:teuthology.orchestra.run.smithi119.stdout:55834574916 2024-09-18T11:35:11.712 INFO:teuthology.orchestra.run.smithi119.stdout:55834574915 2024-09-18T11:35:11.713 INFO:teuthology.orchestra.run.smithi119.stdout:55834574915 2024-09-18T11:35:11.726 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574916 got 55834574916 for osd.1 2024-09-18T11:35:11.726 DEBUG:teuthology.parallel:result is None 2024-09-18T11:35:11.727 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574915 got 55834574915 for osd.2 2024-09-18T11:35:11.727 DEBUG:teuthology.parallel:result is None 2024-09-18T11:35:11.728 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574915 got 55834574915 for osd.7 2024-09-18T11:35:11.728 DEBUG:teuthology.parallel:result is None 2024-09-18T11:35:11.755 INFO:teuthology.orchestra.run.smithi119.stdout:923417968647 2024-09-18T11:35:11.768 INFO:tasks.ceph.ceph_manager.ceph:need seq 923417968647 got 923417968647 for osd.0 2024-09-18T11:35:11.768 DEBUG:teuthology.parallel:result is None 2024-09-18T11:35:11.782 INFO:teuthology.orchestra.run.smithi119.stdout:55834574915 2024-09-18T11:35:11.794 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574915 got 55834574915 for osd.4 2024-09-18T11:35:11.794 DEBUG:teuthology.parallel:result is None 2024-09-18T11:35:11.794 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T11:35:11.795 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:35:12.096 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:35:12.097 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:35:12.108 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-18T11:35:11.983477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":142099530,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":712,"num_read_kb":25479,"num_write":887,"num_write_kb":266346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":432,"num_bytes_recovered":175750418,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2613008,"kb_used_data":190724,"kb_used_omap":36,"kb_used_meta":2422107,"kb_avail":752361712,"statfs":{"total":773094113280,"available":770418393088,"internally_reserved":0,"allocated":195301376,"data_stored":192265495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37868,"internal_metadata":2480237588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.750774"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":421,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.604491+0000","last_change":"2024-09-18T11:34:50.157763+0000","last_active":"2024-09-18T11:35:06.604491+0000","last_peered":"2024-09-18T11:35:06.604491+0000","last_clean":"2024-09-18T11:35:06.604491+0000","last_became_active":"2024-09-18T11:34:50.157413+0000","last_became_peered":"2024-09-18T11:34:50.157413+0000","last_unstale":"2024-09-18T11:35:06.604491+0000","last_undegraded":"2024-09-18T11:35:06.604491+0000","last_fullsized":"2024-09-18T11:35:06.604491+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'72","last_scrub_stamp":"2024-09-18T11:33:50.797790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_clean_scrub_stamp":"2024-09-18T11:33:50.797790+0000","objects_scrubbed":2,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001837497,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2115712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":282,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641913+0000","last_change":"2024-09-18T11:34:08.163615+0000","last_active":"2024-09-18T11:35:06.641913+0000","last_peered":"2024-09-18T11:35:06.641913+0000","last_clean":"2024-09-18T11:35:06.641913+0000","last_became_active":"2024-09-18T11:32:27.079772+0000","last_became_peered":"2024-09-18T11:32:27.079772+0000","last_unstale":"2024-09-18T11:35:06.641913+0000","last_undegraded":"2024-09-18T11:35:06.641913+0000","last_fullsized":"2024-09-18T11:35:06.641913+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'1","last_scrub_stamp":"2024-09-18T11:34:08.163526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:34:08.163526+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:18.531722+0000","scrub_duration":0.001981956,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"182'203","reported_seq":497,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.246545+0000","last_change":"2024-09-18T11:34:05.671877+0000","last_active":"2024-09-18T11:35:06.246545+0000","last_peered":"2024-09-18T11:35:06.246545+0000","last_clean":"2024-09-18T11:35:06.246545+0000","last_became_active":"2024-09-18T11:30:01.077612+0000","last_became_peered":"2024-09-18T11:30:01.077612+0000","last_unstale":"2024-09-18T11:35:06.246545+0000","last_undegraded":"2024-09-18T11:35:06.246545+0000","last_fullsized":"2024-09-18T11:35:06.246545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"182'203","last_scrub_stamp":"2024-09-18T11:34:05.671790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:34:05.671790+0000","objects_scrubbed":51,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:12.454118+0000","scrub_duration":0.0089221579999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42108006,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":185,"num_read_kb":4788,"num_write":230,"num_write_kb":78760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":268,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.604491+0000","last_change":"2024-09-18T11:34:42.697160+0000","last_active":"2024-09-18T11:35:06.604491+0000","last_peered":"2024-09-18T11:35:06.604491+0000","last_clean":"2024-09-18T11:35:06.604491+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:35:06.604491+0000","last_undegraded":"2024-09-18T11:35:06.604491+0000","last_fullsized":"2024-09-18T11:35:06.604491+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:34:42.697075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:34:42.697075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:05.947061+0000","scrub_duration":0.001876314,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"173'206","reported_seq":779,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.246359+0000","last_change":"2024-09-18T11:34:50.152896+0000","last_active":"2024-09-18T11:35:06.246359+0000","last_peered":"2024-09-18T11:35:06.246359+0000","last_clean":"2024-09-18T11:35:06.246359+0000","last_became_active":"2024-09-18T11:34:50.152112+0000","last_became_peered":"2024-09-18T11:34:50.152112+0000","last_unstale":"2024-09-18T11:35:06.246359+0000","last_undegraded":"2024-09-18T11:35:06.246359+0000","last_fullsized":"2024-09-18T11:35:06.246359+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'206","last_scrub_stamp":"2024-09-18T11:34:01.824422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:34:01.824422+0000","objects_scrubbed":46,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:10.875581+0000","scrub_duration":0.0050590330000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43270431,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":179,"num_read_kb":11045,"num_write":225,"num_write_kb":76253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":64589501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":128,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641509+0000","last_change":"2024-09-18T11:34:56.185524+0000","last_active":"2024-09-18T11:35:06.641509+0000","last_peered":"2024-09-18T11:35:06.641509+0000","last_clean":"2024-09-18T11:35:06.641509+0000","last_became_active":"2024-09-18T11:34:56.185242+0000","last_became_peered":"2024-09-18T11:34:56.185242+0000","last_unstale":"2024-09-18T11:35:06.641509+0000","last_undegraded":"2024-09-18T11:35:06.641509+0000","last_fullsized":"2024-09-18T11:35:06.641509+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:34:48.411852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:34:48.411852+0000","last_clean_scrub_stamp":"2024-09-18T11:34:48.411852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:51.892915+0000","scrub_duration":0.001364355,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"172'146","reported_seq":673,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641439+0000","last_change":"2024-09-18T11:34:35.993294+0000","last_active":"2024-09-18T11:35:06.641439+0000","last_peered":"2024-09-18T11:35:06.641439+0000","last_clean":"2024-09-18T11:35:06.641439+0000","last_became_active":"2024-09-18T11:32:59.476088+0000","last_became_peered":"2024-09-18T11:32:59.476088+0000","last_unstale":"2024-09-18T11:35:06.641439+0000","last_undegraded":"2024-09-18T11:35:06.641439+0000","last_fullsized":"2024-09-18T11:35:06.641439+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"172'146","last_scrub_stamp":"2024-09-18T11:34:35.993193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:34:35.993193+0000","objects_scrubbed":35,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:51.068138+0000","scrub_duration":0.0051455880000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24142412,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":133,"num_read_kb":1720,"num_write":158,"num_write_kb":46975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":29291946,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"210'3","reported_seq":258,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641920+0000","last_change":"2024-09-18T11:34:25.106232+0000","last_active":"2024-09-18T11:35:06.641920+0000","last_peered":"2024-09-18T11:35:06.641920+0000","last_clean":"2024-09-18T11:35:06.641920+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:35:06.641920+0000","last_undegraded":"2024-09-18T11:35:06.641920+0000","last_fullsized":"2024-09-18T11:35:06.641920+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'3","last_scrub_stamp":"2024-09-18T11:34:25.106157+0000","last_deep_scrub":"210'3","last_deep_scrub_stamp":"2024-09-18T11:34:25.106157+0000","last_clean_scrub_stamp":"2024-09-18T11:34:25.106157+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:29.899803+0000","scrub_duration":0.0034758250000000001,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"179'157","reported_seq":875,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641856+0000","last_change":"2024-09-18T11:34:50.152718+0000","last_active":"2024-09-18T11:35:06.641856+0000","last_peered":"2024-09-18T11:35:06.641856+0000","last_clean":"2024-09-18T11:35:06.641856+0000","last_became_active":"2024-09-18T11:34:50.152231+0000","last_became_peered":"2024-09-18T11:34:50.152231+0000","last_unstale":"2024-09-18T11:35:06.641856+0000","last_undegraded":"2024-09-18T11:35:06.641856+0000","last_fullsized":"2024-09-18T11:35:06.641856+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"179'157","last_scrub_stamp":"2024-09-18T11:33:49.858681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:33:49.858681+0000","objects_scrubbed":36,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:11.960501+0000","scrub_duration":0.0046045019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31988294,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":149,"num_read_kb":7871,"num_write":171,"num_write_kb":62728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":79753259,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":301,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641431+0000","last_change":"2024-09-18T11:34:47.138275+0000","last_active":"2024-09-18T11:35:06.641431+0000","last_peered":"2024-09-18T11:35:06.641431+0000","last_clean":"2024-09-18T11:35:06.641431+0000","last_became_active":"2024-09-18T11:34:47.137971+0000","last_became_peered":"2024-09-18T11:34:47.137971+0000","last_unstale":"2024-09-18T11:35:06.641431+0000","last_undegraded":"2024-09-18T11:35:06.641431+0000","last_fullsized":"2024-09-18T11:35:06.641431+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:44.054611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:32:30.705704+0000","last_clean_scrub_stamp":"2024-09-18T11:33:44.054611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:12.058086+0000","scrub_duration":0.0015037430000000001,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":301,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.604496+0000","last_change":"2024-09-18T11:34:47.136279+0000","last_active":"2024-09-18T11:35:06.604496+0000","last_peered":"2024-09-18T11:35:06.604496+0000","last_clean":"2024-09-18T11:35:06.604496+0000","last_became_active":"2024-09-18T11:34:47.135985+0000","last_became_peered":"2024-09-18T11:34:47.135985+0000","last_unstale":"2024-09-18T11:35:06.604496+0000","last_undegraded":"2024-09-18T11:35:06.604496+0000","last_fullsized":"2024-09-18T11:35:06.604496+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:59.647914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:59.647914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:27.634628+0000","scrub_duration":0.001373326,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641508+0000","last_change":"2024-09-18T11:34:56.673422+0000","last_active":"2024-09-18T11:35:06.641508+0000","last_peered":"2024-09-18T11:35:06.641508+0000","last_clean":"2024-09-18T11:35:06.641508+0000","last_became_active":"2024-09-18T11:34:56.673243+0000","last_became_peered":"2024-09-18T11:34:56.673243+0000","last_unstale":"2024-09-18T11:35:06.641508+0000","last_undegraded":"2024-09-18T11:35:06.641508+0000","last_fullsized":"2024-09-18T11:35:06.641508+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:56.614347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:56.614347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:24.409305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":263,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.051971+0000","last_change":"2024-09-18T11:34:36.653574+0000","last_active":"2024-09-18T11:35:06.051971+0000","last_peered":"2024-09-18T11:35:06.051971+0000","last_clean":"2024-09-18T11:35:06.051971+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:35:06.051971+0000","last_undegraded":"2024-09-18T11:35:06.051971+0000","last_fullsized":"2024-09-18T11:35:06.051971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:34:36.653477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:34:36.653477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:51.408009+0000","scrub_duration":0.001348115,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":141509143,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":646,"num_read_kb":25424,"num_write":784,"num_write_kb":264716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":422,"num_bytes_recovered":173634706,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":211963904,"data_stored":211458577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9210,"internal_metadata":0},"log_size":712,"ondisk_log_size":712,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2115712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574915,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302688,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069152,"statfs":{"total":96636764160,"available":96326811648,"internally_reserved":0,"allocated":1191936,"data_stored":871593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.432},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.61799999999999999}]},{"osd":1,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.48499999999999999,"15min":0.47299999999999998},"min":{"1min":0.44800000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.50800000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.71699999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.56000000000000005}]},{"osd":2,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.45900000000000002,"15min":0.434},"min":{"1min":0.38700000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.75800000000000001,"5min":1.085,"15min":1.085},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.46100000000000002,"15min":0.44600000000000001},"min":{"1min":0.39900000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.58399999999999996}]},{"osd":3,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.47799999999999998,"15min":0.45500000000000002},"min":{"1min":0.42199999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.48899999999999999,"15min":0.49199999999999999},"min":{"1min":0.432,"5min":0.308,"15min":0.308},"max":{"1min":0.66000000000000003,"5min":1.177,"15min":1.177},"last":0.66100000000000003}]},{"osd":4,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.45100000000000001},"min":{"1min":0.30099999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.60999999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.46000000000000002,"15min":0.49199999999999999},"min":{"1min":0.28199999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.62,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.72499999999999998}]},{"osd":5,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.39900000000000002,"15min":0.433},"min":{"1min":0.30199999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68999999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.46100000000000002},"min":{"1min":0.35899999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63200000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.76200000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47599999999999998,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65600000000000003,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47699999999999998,"15min":0.499},"min":{"1min":0.32500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.629,"5min":0.75,"15min":0.75},"last":0.77900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302168,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069672,"statfs":{"total":96636764160,"available":96327344128,"internally_reserved":0,"allocated":659456,"data_stored":340981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.56200000000000006}]},{"osd":1,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.48399999999999999,"15min":0.47499999999999998},"min":{"1min":0.35699999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.72499999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.47099999999999997,"15min":0.46200000000000002},"min":{"1min":0.25800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.70099999999999996,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.46400000000000002}]},{"osd":2,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.27000000000000002,"5min":0.221,"15min":0.221},"max":{"1min":0.75700000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.48499999999999999,"15min":0.45700000000000002},"min":{"1min":0.40899999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68100000000000005,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.64600000000000002}]},{"osd":3,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48299999999999998,"15min":0.50800000000000001},"min":{"1min":0.29899999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59099999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.52000000000000002},"min":{"1min":0.31,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.71899999999999997,"5min":1.032,"15min":1.032},"last":0.63300000000000001}]},{"osd":4,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42799999999999999,"15min":0.40200000000000002},"min":{"1min":0.31900000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.70999999999999996,"5min":0.755,"15min":0.755},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.47999999999999998,"15min":0.44},"min":{"1min":0.375,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.749,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.35899999999999999}]},{"osd":5,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44800000000000001,"15min":0.41999999999999998},"min":{"1min":0.34499999999999997,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67000000000000004,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.46700000000000003,"15min":0.441},"min":{"1min":0.33200000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.374}]},{"osd":7,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47999999999999998,"15min":0.46899999999999997},"min":{"1min":0.28999999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.72099999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48999999999999999,"15min":0.47999999999999998},"min":{"1min":0.308,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63400000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.40999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574916,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332192,"kb_used_data":26508,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94039648,"statfs":{"total":96636764160,"available":96296599552,"internally_reserved":0,"allocated":27144192,"data_stored":26761132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.56899999999999995}]},{"osd":1,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.46700000000000003,"15min":0.44500000000000001},"min":{"1min":0.40200000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.47799999999999998,"15min":0.46000000000000002},"min":{"1min":0.46100000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.65200000000000002,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.52000000000000002}]},{"osd":2,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.503,"15min":0.503},"min":{"1min":0.443,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73999999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.47099999999999997,"15min":0.49299999999999999},"min":{"1min":0.40300000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.63400000000000001,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.49299999999999999}]},{"osd":3,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.47299999999999998,"15min":0.49099999999999999},"min":{"1min":0.38200000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.499,"15min":0.501},"min":{"1min":0.41399999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68899999999999995,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.55900000000000005}]},{"osd":4,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41899999999999998,"15min":0.44},"min":{"1min":0.32100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.68500000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.41599999999999998,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.222,"15min":0.222},"max":{"1min":0.63300000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.35699999999999998}]},{"osd":6,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41399999999999998,"15min":0.42799999999999999},"min":{"1min":0.32600000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.317},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.45300000000000001,"15min":0.48099999999999998},"min":{"1min":0.30599999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.33600000000000002}]},{"osd":7,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.45800000000000002,"15min":0.49399999999999999},"min":{"1min":0.377,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.70599999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.497,"15min":0.51800000000000002},"min":{"1min":0.39700000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.40799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574915,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356460,"kb_used_data":50784,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94015380,"statfs":{"total":96636764160,"available":96271749120,"internally_reserved":0,"allocated":52002816,"data_stored":51562431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.29299999999999998}]},{"osd":1,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.48299999999999998,"15min":0.502},"min":{"1min":0.42599999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76500000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48099999999999998,"15min":0.497},"min":{"1min":0.34399999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.67300000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.378}]},{"osd":2,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.497,"15min":0.498},"min":{"1min":0.40000000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73199999999999998,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.50700000000000001,"15min":0.52100000000000002},"min":{"1min":0.51800000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.80900000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.57599999999999996}]},{"osd":3,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.48699999999999999,"15min":0.51200000000000001},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72999999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.502,"15min":0.52600000000000002},"min":{"1min":0.379,"5min":0.316,"15min":0.316},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.47999999999999998}]},{"osd":5,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41599999999999998,"15min":0.45400000000000001},"min":{"1min":0.27700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.67900000000000005,"5min":1.002,"15min":1.002},"last":0.504},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.45800000000000002,"15min":0.5},"min":{"1min":0.35099999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.73099999999999998,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.55700000000000005}]},{"osd":6,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.44,"15min":0.46600000000000003},"min":{"1min":0.33500000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.66900000000000004,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.496},"min":{"1min":0.318,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.70599999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.46600000000000003}]},{"osd":7,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48299999999999998,"15min":0.50700000000000001},"min":{"1min":0.33400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.754,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.52500000000000002,"15min":0.54500000000000004},"min":{"1min":0.41099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83999999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.66100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574917,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":328056,"kb_used_data":26532,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94043784,"statfs":{"total":96636764160,"available":96300834816,"internally_reserved":0,"allocated":27168768,"data_stored":26762930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.19400000000000001},{"interface":"front","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.24299999999999999}]},{"osd":1,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.33900000000000002,"15min":0.34399999999999997},"min":{"1min":0.27500000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.55900000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.375,"15min":0.375},"min":{"1min":0.22500000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.57099999999999995,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.313}]},{"osd":2,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.38900000000000001,"15min":0.38500000000000001},"min":{"1min":0.32400000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.53000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.374,"15min":0.39800000000000002},"min":{"1min":0.23799999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.46600000000000003,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.27300000000000002}]},{"osd":4,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.55100000000000005,"15min":0.52700000000000002},"min":{"1min":0.38600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79900000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56899999999999995,"15min":0.54000000000000004},"min":{"1min":0.47499999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78700000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58999999999999997}]},{"osd":5,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.51300000000000001},"min":{"1min":0.46100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77000000000000002,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.56000000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.755,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.63}]},{"osd":6,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52400000000000002,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81100000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.47699999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.74299999999999999,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.58099999999999996}]},{"osd":7,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57599999999999996,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77900000000000003,"5min":1.26,"15min":1.26},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55500000000000005,"15min":0.57499999999999996},"min":{"1min":0.44500000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.749,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.61899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318412,"kb_used_data":16892,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94053428,"statfs":{"total":96636764160,"available":96310710272,"internally_reserved":0,"allocated":17297408,"data_stored":16938301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.378}]},{"osd":1,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34899999999999998,"15min":0.373},"min":{"1min":0.20599999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.42499999999999999,"5min":1.621,"15min":1.621},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.36299999999999999,"15min":0.39400000000000002},"min":{"1min":0.251,"5min":0.218,"15min":0.218},"max":{"1min":0.63600000000000001,"5min":1.589,"15min":1.589},"last":0.40200000000000002}]},{"osd":3,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.35499999999999998,"15min":0.36699999999999999},"min":{"1min":0.28299999999999997,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.42099999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.38200000000000001,"15min":0.41099999999999998},"min":{"1min":0.24099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.56000000000000005,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.52700000000000002}]},{"osd":4,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53500000000000003,"15min":0.55200000000000005},"min":{"1min":0.42899999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70399999999999996,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.52500000000000002},"min":{"1min":0.46400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80000000000000004,"5min":1.012,"15min":1.012},"last":0.73599999999999999}]},{"osd":5,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79000000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.749},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.48499999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.749,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.79100000000000004}]},{"osd":6,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.45600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76300000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.56499999999999995},"min":{"1min":0.48699999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79800000000000004,"5min":1.165,"15min":1.165},"last":0.77700000000000002}]},{"osd":7,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.46000000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82899999999999996,"5min":1.073,"15min":1.073},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.45500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82299999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.85899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322560,"kb_used_data":16884,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94049280,"statfs":{"total":96636764160,"available":96306462720,"internally_reserved":0,"allocated":17289216,"data_stored":16931921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.40600000000000003}]},{"osd":2,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.35599999999999998,"15min":0.378},"min":{"1min":0.255,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.50700000000000001,"5min":1.333,"15min":1.333},"last":0.374},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38600000000000001,"15min":0.41599999999999998},"min":{"1min":0.28599999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.65700000000000003,"5min":1.298,"15min":1.298},"last":0.316}]},{"osd":3,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36399999999999999,"15min":0.38},"min":{"1min":0.24199999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.60499999999999998,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.36099999999999999,"15min":0.36399999999999999},"min":{"1min":0.27100000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55500000000000005,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.35699999999999998}]},{"osd":4,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.496,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.876,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.37,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.74299999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56699999999999995}]},{"osd":5,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40600000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78100000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.52900000000000003,"15min":0.53000000000000003},"min":{"1min":0.48699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.85299999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.44,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82899999999999996,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81599999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57799999999999996}]},{"osd":7,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41799999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85499999999999998,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56200000000000006,"15min":0.57799999999999996},"min":{"1min":0.41999999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83499999999999996,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.52100000000000002}]}]},{"osd":0,"up_from":215,"seq":923417968648,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":350472,"kb_used_data":51316,"kb_used_omap":25,"kb_used_meta":299110,"kb_avail":94021368,"statfs":{"total":96636764160,"available":96277880832,"internally_reserved":0,"allocated":52547584,"data_stored":52096206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25940,"internal_metadata":306289324},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9210,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23064576,"data_stored":23013811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:35:12.110 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:35:12.371 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:35:12.372 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:35:12.383 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-18T11:35:11.983477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":142099530,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":712,"num_read_kb":25479,"num_write":887,"num_write_kb":266346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":432,"num_bytes_recovered":175750418,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2613008,"kb_used_data":190724,"kb_used_omap":36,"kb_used_meta":2422107,"kb_avail":752361712,"statfs":{"total":773094113280,"available":770418393088,"internally_reserved":0,"allocated":195301376,"data_stored":192265495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37868,"internal_metadata":2480237588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.750774"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":421,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.604491+0000","last_change":"2024-09-18T11:34:50.157763+0000","last_active":"2024-09-18T11:35:06.604491+0000","last_peered":"2024-09-18T11:35:06.604491+0000","last_clean":"2024-09-18T11:35:06.604491+0000","last_became_active":"2024-09-18T11:34:50.157413+0000","last_became_peered":"2024-09-18T11:34:50.157413+0000","last_unstale":"2024-09-18T11:35:06.604491+0000","last_undegraded":"2024-09-18T11:35:06.604491+0000","last_fullsized":"2024-09-18T11:35:06.604491+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'72","last_scrub_stamp":"2024-09-18T11:33:50.797790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_clean_scrub_stamp":"2024-09-18T11:33:50.797790+0000","objects_scrubbed":2,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001837497,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2115712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":282,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641913+0000","last_change":"2024-09-18T11:34:08.163615+0000","last_active":"2024-09-18T11:35:06.641913+0000","last_peered":"2024-09-18T11:35:06.641913+0000","last_clean":"2024-09-18T11:35:06.641913+0000","last_became_active":"2024-09-18T11:32:27.079772+0000","last_became_peered":"2024-09-18T11:32:27.079772+0000","last_unstale":"2024-09-18T11:35:06.641913+0000","last_undegraded":"2024-09-18T11:35:06.641913+0000","last_fullsized":"2024-09-18T11:35:06.641913+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'1","last_scrub_stamp":"2024-09-18T11:34:08.163526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:34:08.163526+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:18.531722+0000","scrub_duration":0.001981956,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"182'203","reported_seq":497,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.246545+0000","last_change":"2024-09-18T11:34:05.671877+0000","last_active":"2024-09-18T11:35:06.246545+0000","last_peered":"2024-09-18T11:35:06.246545+0000","last_clean":"2024-09-18T11:35:06.246545+0000","last_became_active":"2024-09-18T11:30:01.077612+0000","last_became_peered":"2024-09-18T11:30:01.077612+0000","last_unstale":"2024-09-18T11:35:06.246545+0000","last_undegraded":"2024-09-18T11:35:06.246545+0000","last_fullsized":"2024-09-18T11:35:06.246545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"182'203","last_scrub_stamp":"2024-09-18T11:34:05.671790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:34:05.671790+0000","objects_scrubbed":51,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:12.454118+0000","scrub_duration":0.0089221579999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42108006,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":185,"num_read_kb":4788,"num_write":230,"num_write_kb":78760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":268,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.604491+0000","last_change":"2024-09-18T11:34:42.697160+0000","last_active":"2024-09-18T11:35:06.604491+0000","last_peered":"2024-09-18T11:35:06.604491+0000","last_clean":"2024-09-18T11:35:06.604491+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:35:06.604491+0000","last_undegraded":"2024-09-18T11:35:06.604491+0000","last_fullsized":"2024-09-18T11:35:06.604491+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:34:42.697075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:34:42.697075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:05.947061+0000","scrub_duration":0.001876314,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"173'206","reported_seq":779,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.246359+0000","last_change":"2024-09-18T11:34:50.152896+0000","last_active":"2024-09-18T11:35:06.246359+0000","last_peered":"2024-09-18T11:35:06.246359+0000","last_clean":"2024-09-18T11:35:06.246359+0000","last_became_active":"2024-09-18T11:34:50.152112+0000","last_became_peered":"2024-09-18T11:34:50.152112+0000","last_unstale":"2024-09-18T11:35:06.246359+0000","last_undegraded":"2024-09-18T11:35:06.246359+0000","last_fullsized":"2024-09-18T11:35:06.246359+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'206","last_scrub_stamp":"2024-09-18T11:34:01.824422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:34:01.824422+0000","objects_scrubbed":46,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:10.875581+0000","scrub_duration":0.0050590330000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43270431,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":179,"num_read_kb":11045,"num_write":225,"num_write_kb":76253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":64589501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":128,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641509+0000","last_change":"2024-09-18T11:34:56.185524+0000","last_active":"2024-09-18T11:35:06.641509+0000","last_peered":"2024-09-18T11:35:06.641509+0000","last_clean":"2024-09-18T11:35:06.641509+0000","last_became_active":"2024-09-18T11:34:56.185242+0000","last_became_peered":"2024-09-18T11:34:56.185242+0000","last_unstale":"2024-09-18T11:35:06.641509+0000","last_undegraded":"2024-09-18T11:35:06.641509+0000","last_fullsized":"2024-09-18T11:35:06.641509+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:34:48.411852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:34:48.411852+0000","last_clean_scrub_stamp":"2024-09-18T11:34:48.411852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:51.892915+0000","scrub_duration":0.001364355,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"172'146","reported_seq":673,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641439+0000","last_change":"2024-09-18T11:34:35.993294+0000","last_active":"2024-09-18T11:35:06.641439+0000","last_peered":"2024-09-18T11:35:06.641439+0000","last_clean":"2024-09-18T11:35:06.641439+0000","last_became_active":"2024-09-18T11:32:59.476088+0000","last_became_peered":"2024-09-18T11:32:59.476088+0000","last_unstale":"2024-09-18T11:35:06.641439+0000","last_undegraded":"2024-09-18T11:35:06.641439+0000","last_fullsized":"2024-09-18T11:35:06.641439+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"172'146","last_scrub_stamp":"2024-09-18T11:34:35.993193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:34:35.993193+0000","objects_scrubbed":35,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:51.068138+0000","scrub_duration":0.0051455880000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24142412,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":133,"num_read_kb":1720,"num_write":158,"num_write_kb":46975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":29291946,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"210'3","reported_seq":258,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641920+0000","last_change":"2024-09-18T11:34:25.106232+0000","last_active":"2024-09-18T11:35:06.641920+0000","last_peered":"2024-09-18T11:35:06.641920+0000","last_clean":"2024-09-18T11:35:06.641920+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:35:06.641920+0000","last_undegraded":"2024-09-18T11:35:06.641920+0000","last_fullsized":"2024-09-18T11:35:06.641920+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'3","last_scrub_stamp":"2024-09-18T11:34:25.106157+0000","last_deep_scrub":"210'3","last_deep_scrub_stamp":"2024-09-18T11:34:25.106157+0000","last_clean_scrub_stamp":"2024-09-18T11:34:25.106157+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:29.899803+0000","scrub_duration":0.0034758250000000001,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"179'157","reported_seq":875,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641856+0000","last_change":"2024-09-18T11:34:50.152718+0000","last_active":"2024-09-18T11:35:06.641856+0000","last_peered":"2024-09-18T11:35:06.641856+0000","last_clean":"2024-09-18T11:35:06.641856+0000","last_became_active":"2024-09-18T11:34:50.152231+0000","last_became_peered":"2024-09-18T11:34:50.152231+0000","last_unstale":"2024-09-18T11:35:06.641856+0000","last_undegraded":"2024-09-18T11:35:06.641856+0000","last_fullsized":"2024-09-18T11:35:06.641856+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"179'157","last_scrub_stamp":"2024-09-18T11:33:49.858681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:33:49.858681+0000","objects_scrubbed":36,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:11.960501+0000","scrub_duration":0.0046045019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31988294,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":149,"num_read_kb":7871,"num_write":171,"num_write_kb":62728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":79753259,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":301,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641431+0000","last_change":"2024-09-18T11:34:47.138275+0000","last_active":"2024-09-18T11:35:06.641431+0000","last_peered":"2024-09-18T11:35:06.641431+0000","last_clean":"2024-09-18T11:35:06.641431+0000","last_became_active":"2024-09-18T11:34:47.137971+0000","last_became_peered":"2024-09-18T11:34:47.137971+0000","last_unstale":"2024-09-18T11:35:06.641431+0000","last_undegraded":"2024-09-18T11:35:06.641431+0000","last_fullsized":"2024-09-18T11:35:06.641431+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:44.054611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:32:30.705704+0000","last_clean_scrub_stamp":"2024-09-18T11:33:44.054611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:12.058086+0000","scrub_duration":0.0015037430000000001,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":301,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.604496+0000","last_change":"2024-09-18T11:34:47.136279+0000","last_active":"2024-09-18T11:35:06.604496+0000","last_peered":"2024-09-18T11:35:06.604496+0000","last_clean":"2024-09-18T11:35:06.604496+0000","last_became_active":"2024-09-18T11:34:47.135985+0000","last_became_peered":"2024-09-18T11:34:47.135985+0000","last_unstale":"2024-09-18T11:35:06.604496+0000","last_undegraded":"2024-09-18T11:35:06.604496+0000","last_fullsized":"2024-09-18T11:35:06.604496+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:59.647914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:59.647914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:27.634628+0000","scrub_duration":0.001373326,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641508+0000","last_change":"2024-09-18T11:34:56.673422+0000","last_active":"2024-09-18T11:35:06.641508+0000","last_peered":"2024-09-18T11:35:06.641508+0000","last_clean":"2024-09-18T11:35:06.641508+0000","last_became_active":"2024-09-18T11:34:56.673243+0000","last_became_peered":"2024-09-18T11:34:56.673243+0000","last_unstale":"2024-09-18T11:35:06.641508+0000","last_undegraded":"2024-09-18T11:35:06.641508+0000","last_fullsized":"2024-09-18T11:35:06.641508+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:56.614347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:56.614347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:24.409305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":263,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.051971+0000","last_change":"2024-09-18T11:34:36.653574+0000","last_active":"2024-09-18T11:35:06.051971+0000","last_peered":"2024-09-18T11:35:06.051971+0000","last_clean":"2024-09-18T11:35:06.051971+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:35:06.051971+0000","last_undegraded":"2024-09-18T11:35:06.051971+0000","last_fullsized":"2024-09-18T11:35:06.051971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:34:36.653477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:34:36.653477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:51.408009+0000","scrub_duration":0.001348115,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":141509143,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":646,"num_read_kb":25424,"num_write":784,"num_write_kb":264716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":422,"num_bytes_recovered":173634706,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":211963904,"data_stored":211458577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9210,"internal_metadata":0},"log_size":712,"ondisk_log_size":712,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2115712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574915,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302688,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069152,"statfs":{"total":96636764160,"available":96326811648,"internally_reserved":0,"allocated":1191936,"data_stored":871593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.432},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.61799999999999999}]},{"osd":1,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.48499999999999999,"15min":0.47299999999999998},"min":{"1min":0.44800000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.50800000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.71699999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.56000000000000005}]},{"osd":2,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.45900000000000002,"15min":0.434},"min":{"1min":0.38700000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.75800000000000001,"5min":1.085,"15min":1.085},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.46100000000000002,"15min":0.44600000000000001},"min":{"1min":0.39900000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.58399999999999996}]},{"osd":3,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.47799999999999998,"15min":0.45500000000000002},"min":{"1min":0.42199999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.48899999999999999,"15min":0.49199999999999999},"min":{"1min":0.432,"5min":0.308,"15min":0.308},"max":{"1min":0.66000000000000003,"5min":1.177,"15min":1.177},"last":0.66100000000000003}]},{"osd":4,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.45100000000000001},"min":{"1min":0.30099999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.60999999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.46000000000000002,"15min":0.49199999999999999},"min":{"1min":0.28199999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.62,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.72499999999999998}]},{"osd":5,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.39900000000000002,"15min":0.433},"min":{"1min":0.30199999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68999999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.46100000000000002},"min":{"1min":0.35899999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63200000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.76200000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47599999999999998,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65600000000000003,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47699999999999998,"15min":0.499},"min":{"1min":0.32500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.629,"5min":0.75,"15min":0.75},"last":0.77900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302168,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069672,"statfs":{"total":96636764160,"available":96327344128,"internally_reserved":0,"allocated":659456,"data_stored":340981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.56200000000000006}]},{"osd":1,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.48399999999999999,"15min":0.47499999999999998},"min":{"1min":0.35699999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.72499999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.47099999999999997,"15min":0.46200000000000002},"min":{"1min":0.25800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.70099999999999996,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.46400000000000002}]},{"osd":2,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.27000000000000002,"5min":0.221,"15min":0.221},"max":{"1min":0.75700000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.48499999999999999,"15min":0.45700000000000002},"min":{"1min":0.40899999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68100000000000005,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.64600000000000002}]},{"osd":3,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48299999999999998,"15min":0.50800000000000001},"min":{"1min":0.29899999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59099999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.52000000000000002},"min":{"1min":0.31,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.71899999999999997,"5min":1.032,"15min":1.032},"last":0.63300000000000001}]},{"osd":4,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42799999999999999,"15min":0.40200000000000002},"min":{"1min":0.31900000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.70999999999999996,"5min":0.755,"15min":0.755},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.47999999999999998,"15min":0.44},"min":{"1min":0.375,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.749,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.35899999999999999}]},{"osd":5,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44800000000000001,"15min":0.41999999999999998},"min":{"1min":0.34499999999999997,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67000000000000004,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.46700000000000003,"15min":0.441},"min":{"1min":0.33200000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.374}]},{"osd":7,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47999999999999998,"15min":0.46899999999999997},"min":{"1min":0.28999999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.72099999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48999999999999999,"15min":0.47999999999999998},"min":{"1min":0.308,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63400000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.40999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574916,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332192,"kb_used_data":26508,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94039648,"statfs":{"total":96636764160,"available":96296599552,"internally_reserved":0,"allocated":27144192,"data_stored":26761132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.56899999999999995}]},{"osd":1,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.46700000000000003,"15min":0.44500000000000001},"min":{"1min":0.40200000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.47799999999999998,"15min":0.46000000000000002},"min":{"1min":0.46100000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.65200000000000002,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.52000000000000002}]},{"osd":2,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.503,"15min":0.503},"min":{"1min":0.443,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73999999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.47099999999999997,"15min":0.49299999999999999},"min":{"1min":0.40300000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.63400000000000001,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.49299999999999999}]},{"osd":3,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.47299999999999998,"15min":0.49099999999999999},"min":{"1min":0.38200000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.499,"15min":0.501},"min":{"1min":0.41399999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68899999999999995,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.55900000000000005}]},{"osd":4,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41899999999999998,"15min":0.44},"min":{"1min":0.32100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.68500000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.41599999999999998,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.222,"15min":0.222},"max":{"1min":0.63300000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.35699999999999998}]},{"osd":6,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41399999999999998,"15min":0.42799999999999999},"min":{"1min":0.32600000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.317},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.45300000000000001,"15min":0.48099999999999998},"min":{"1min":0.30599999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.33600000000000002}]},{"osd":7,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.45800000000000002,"15min":0.49399999999999999},"min":{"1min":0.377,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.70599999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.497,"15min":0.51800000000000002},"min":{"1min":0.39700000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.40799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574915,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356460,"kb_used_data":50784,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94015380,"statfs":{"total":96636764160,"available":96271749120,"internally_reserved":0,"allocated":52002816,"data_stored":51562431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.29299999999999998}]},{"osd":1,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.48299999999999998,"15min":0.502},"min":{"1min":0.42599999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76500000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48099999999999998,"15min":0.497},"min":{"1min":0.34399999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.67300000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.378}]},{"osd":2,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.497,"15min":0.498},"min":{"1min":0.40000000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73199999999999998,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.50700000000000001,"15min":0.52100000000000002},"min":{"1min":0.51800000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.80900000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.57599999999999996}]},{"osd":3,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.48699999999999999,"15min":0.51200000000000001},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72999999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.502,"15min":0.52600000000000002},"min":{"1min":0.379,"5min":0.316,"15min":0.316},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.47999999999999998}]},{"osd":5,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41599999999999998,"15min":0.45400000000000001},"min":{"1min":0.27700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.67900000000000005,"5min":1.002,"15min":1.002},"last":0.504},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.45800000000000002,"15min":0.5},"min":{"1min":0.35099999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.73099999999999998,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.55700000000000005}]},{"osd":6,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.44,"15min":0.46600000000000003},"min":{"1min":0.33500000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.66900000000000004,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.496},"min":{"1min":0.318,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.70599999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.46600000000000003}]},{"osd":7,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48299999999999998,"15min":0.50700000000000001},"min":{"1min":0.33400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.754,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.52500000000000002,"15min":0.54500000000000004},"min":{"1min":0.41099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83999999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.66100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574917,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":328056,"kb_used_data":26532,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94043784,"statfs":{"total":96636764160,"available":96300834816,"internally_reserved":0,"allocated":27168768,"data_stored":26762930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.19400000000000001},{"interface":"front","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.24299999999999999}]},{"osd":1,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.33900000000000002,"15min":0.34399999999999997},"min":{"1min":0.27500000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.55900000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.375,"15min":0.375},"min":{"1min":0.22500000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.57099999999999995,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.313}]},{"osd":2,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.38900000000000001,"15min":0.38500000000000001},"min":{"1min":0.32400000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.53000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.374,"15min":0.39800000000000002},"min":{"1min":0.23799999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.46600000000000003,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.27300000000000002}]},{"osd":4,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.55100000000000005,"15min":0.52700000000000002},"min":{"1min":0.38600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79900000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56899999999999995,"15min":0.54000000000000004},"min":{"1min":0.47499999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78700000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58999999999999997}]},{"osd":5,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.51300000000000001},"min":{"1min":0.46100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77000000000000002,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.56000000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.755,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.63}]},{"osd":6,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52400000000000002,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81100000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.47699999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.74299999999999999,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.58099999999999996}]},{"osd":7,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57599999999999996,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77900000000000003,"5min":1.26,"15min":1.26},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55500000000000005,"15min":0.57499999999999996},"min":{"1min":0.44500000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.749,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.61899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318412,"kb_used_data":16892,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94053428,"statfs":{"total":96636764160,"available":96310710272,"internally_reserved":0,"allocated":17297408,"data_stored":16938301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.378}]},{"osd":1,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34899999999999998,"15min":0.373},"min":{"1min":0.20599999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.42499999999999999,"5min":1.621,"15min":1.621},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.36299999999999999,"15min":0.39400000000000002},"min":{"1min":0.251,"5min":0.218,"15min":0.218},"max":{"1min":0.63600000000000001,"5min":1.589,"15min":1.589},"last":0.40200000000000002}]},{"osd":3,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.35499999999999998,"15min":0.36699999999999999},"min":{"1min":0.28299999999999997,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.42099999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.38200000000000001,"15min":0.41099999999999998},"min":{"1min":0.24099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.56000000000000005,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.52700000000000002}]},{"osd":4,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53500000000000003,"15min":0.55200000000000005},"min":{"1min":0.42899999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70399999999999996,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.52500000000000002},"min":{"1min":0.46400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80000000000000004,"5min":1.012,"15min":1.012},"last":0.73599999999999999}]},{"osd":5,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79000000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.749},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.48499999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.749,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.79100000000000004}]},{"osd":6,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.45600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76300000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.56499999999999995},"min":{"1min":0.48699999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79800000000000004,"5min":1.165,"15min":1.165},"last":0.77700000000000002}]},{"osd":7,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.46000000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82899999999999996,"5min":1.073,"15min":1.073},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.45500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82299999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.85899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322560,"kb_used_data":16884,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94049280,"statfs":{"total":96636764160,"available":96306462720,"internally_reserved":0,"allocated":17289216,"data_stored":16931921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.40600000000000003}]},{"osd":2,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.35599999999999998,"15min":0.378},"min":{"1min":0.255,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.50700000000000001,"5min":1.333,"15min":1.333},"last":0.374},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38600000000000001,"15min":0.41599999999999998},"min":{"1min":0.28599999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.65700000000000003,"5min":1.298,"15min":1.298},"last":0.316}]},{"osd":3,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36399999999999999,"15min":0.38},"min":{"1min":0.24199999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.60499999999999998,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.36099999999999999,"15min":0.36399999999999999},"min":{"1min":0.27100000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55500000000000005,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.35699999999999998}]},{"osd":4,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.496,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.876,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.37,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.74299999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56699999999999995}]},{"osd":5,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40600000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78100000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.52900000000000003,"15min":0.53000000000000003},"min":{"1min":0.48699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.85299999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.44,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82899999999999996,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81599999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57799999999999996}]},{"osd":7,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41799999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85499999999999998,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56200000000000006,"15min":0.57799999999999996},"min":{"1min":0.41999999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83499999999999996,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.52100000000000002}]}]},{"osd":0,"up_from":215,"seq":923417968648,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":350472,"kb_used_data":51316,"kb_used_omap":25,"kb_used_meta":299110,"kb_avail":94021368,"statfs":{"total":96636764160,"available":96277880832,"internally_reserved":0,"allocated":52547584,"data_stored":52096206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25940,"internal_metadata":306289324},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9210,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23064576,"data_stored":23013811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:35:12.385 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T11:35:12.385 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-18T11:35:12.684 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-18T11:35:12.694 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-09-18T11:35:12.694 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T11:35:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:35:12.954 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":236,"fsid":"f4cca259-b87a-45cc-8995-7e9fca891a9f","created":"2024-09-18T11:29:38.205487+0000","modified":"2024-09-18T11:35:07.250663+0000","last_up_change":"2024-09-18T11:34:45.106232+0000","last_in_change":"2024-09-18T11:34:46.116722+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T11:29:47.951262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T11:29:48.866238+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"low_tier","create_time":"2024-09-18T11:29:59.343964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"12faced2-7888-4265-ad12-815efe36466c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":119,"last_clean_end":205,"up_from":215,"up_thru":219,"down_at":206,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":74987},{"type":"v1","addr":"172.21.15.119:6801","nonce":74987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":74987},{"type":"v1","addr":"172.21.15.119:6803","nonce":74987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":74987},{"type":"v1","addr":"172.21.15.119:6807","nonce":74987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":74987},{"type":"v1","addr":"172.21.15.119:6805","nonce":74987}]},"public_addr":"172.21.15.119:6801/74987","cluster_addr":"172.21.15.119:6803/74987","heartbeat_back_addr":"172.21.15.119:6807/74987","heartbeat_front_addr":"172.21.15.119:6805/74987","state":["exists","up"]},{"osd":1,"uuid":"20fee29d-b047-46ae-aa60-88e897fbb0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":33350},{"type":"v1","addr":"172.21.15.119:6825","nonce":33350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":33350},{"type":"v1","addr":"172.21.15.119:6827","nonce":33350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":33350},{"type":"v1","addr":"172.21.15.119:6831","nonce":33350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":33350},{"type":"v1","addr":"172.21.15.119:6829","nonce":33350}]},"public_addr":"172.21.15.119:6825/33350","cluster_addr":"172.21.15.119:6827/33350","heartbeat_back_addr":"172.21.15.119:6831/33350","heartbeat_front_addr":"172.21.15.119:6829/33350","state":["exists","up"]},{"osd":2,"uuid":"21a296f3-b259-4841-9a17-4cd21b302b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":33351},{"type":"v1","addr":"172.21.15.119:6809","nonce":33351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":33351},{"type":"v1","addr":"172.21.15.119:6813","nonce":33351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6819","nonce":33351},{"type":"v1","addr":"172.21.15.119:6821","nonce":33351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":33351},{"type":"v1","addr":"172.21.15.119:6817","nonce":33351}]},"public_addr":"172.21.15.119:6809/33351","cluster_addr":"172.21.15.119:6813/33351","heartbeat_back_addr":"172.21.15.119:6821/33351","heartbeat_front_addr":"172.21.15.119:6817/33351","state":["exists","up"]},{"osd":3,"uuid":"59044f30-32e1-4942-9a8e-202c1def221a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":33349},{"type":"v1","addr":"172.21.15.119:6812","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":33349},{"type":"v1","addr":"172.21.15.119:6816","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":33349},{"type":"v1","addr":"172.21.15.119:6823","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":33349},{"type":"v1","addr":"172.21.15.119:6820","nonce":33349}]},"public_addr":"172.21.15.119:6812/33349","cluster_addr":"172.21.15.119:6816/33349","heartbeat_back_addr":"172.21.15.119:6823/33349","heartbeat_front_addr":"172.21.15.119:6820/33349","state":["exists","up"]},{"osd":4,"uuid":"7d1ae3d0-8d69-4d3c-9e51-24fdd77a22a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":33088},{"type":"v1","addr":"172.21.15.148:6809","nonce":33088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":33088},{"type":"v1","addr":"172.21.15.148:6811","nonce":33088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":33088},{"type":"v1","addr":"172.21.15.148:6815","nonce":33088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":33088},{"type":"v1","addr":"172.21.15.148:6813","nonce":33088}]},"public_addr":"172.21.15.148:6809/33088","cluster_addr":"172.21.15.148:6811/33088","heartbeat_back_addr":"172.21.15.148:6815/33088","heartbeat_front_addr":"172.21.15.148:6813/33088","state":["exists","up"]},{"osd":5,"uuid":"b0dd3d30-b273-428c-a4a9-4c238abf45fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":33089},{"type":"v1","addr":"172.21.15.148:6801","nonce":33089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":33089},{"type":"v1","addr":"172.21.15.148:6803","nonce":33089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":33089},{"type":"v1","addr":"172.21.15.148:6807","nonce":33089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":33089},{"type":"v1","addr":"172.21.15.148:6805","nonce":33089}]},"public_addr":"172.21.15.148:6801/33089","cluster_addr":"172.21.15.148:6803/33089","heartbeat_back_addr":"172.21.15.148:6807/33089","heartbeat_front_addr":"172.21.15.148:6805/33089","state":["exists","up"]},{"osd":6,"uuid":"40290cb3-2334-4c68-8e6c-59d0d5455171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":33107},{"type":"v1","addr":"172.21.15.148:6817","nonce":33107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":33107},{"type":"v1","addr":"172.21.15.148:6819","nonce":33107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":33107},{"type":"v1","addr":"172.21.15.148:6823","nonce":33107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":33107},{"type":"v1","addr":"172.21.15.148:6821","nonce":33107}]},"public_addr":"172.21.15.148:6817/33107","cluster_addr":"172.21.15.148:6819/33107","heartbeat_back_addr":"172.21.15.148:6823/33107","heartbeat_front_addr":"172.21.15.148:6821/33107","state":["exists","up"]},{"osd":7,"uuid":"54802abe-ea1a-448e-85af-d1f14a45be27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":33119},{"type":"v1","addr":"172.21.15.148:6825","nonce":33119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":33119},{"type":"v1","addr":"172.21.15.148:6827","nonce":33119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":33119},{"type":"v1","addr":"172.21.15.148:6831","nonce":33119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":33119},{"type":"v1","addr":"172.21.15.148:6829","nonce":33119}]},"public_addr":"172.21.15.148:6825/33119","cluster_addr":"172.21.15.148:6827/33119","heartbeat_back_addr":"172.21.15.148:6831/33119","heartbeat_front_addr":"172.21.15.148:6829/33119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:34:44.910546+0000","dead_epoch":206},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:34:41.761116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:33:55.714008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:34:39.587239+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:35:00.703150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:35:02.069253+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:34:42.647805+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:34:51.482687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T11:35:12.967 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T11:35:12.967 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:35:13.224 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:35:13.225 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:35:13.236 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-18T11:35:11.983477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":142099530,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":712,"num_read_kb":25479,"num_write":887,"num_write_kb":266346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":432,"num_bytes_recovered":175750418,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2613008,"kb_used_data":190724,"kb_used_omap":36,"kb_used_meta":2422107,"kb_avail":752361712,"statfs":{"total":773094113280,"available":770418393088,"internally_reserved":0,"allocated":195301376,"data_stored":192265495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37868,"internal_metadata":2480237588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.750774"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":421,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.604491+0000","last_change":"2024-09-18T11:34:50.157763+0000","last_active":"2024-09-18T11:35:06.604491+0000","last_peered":"2024-09-18T11:35:06.604491+0000","last_clean":"2024-09-18T11:35:06.604491+0000","last_became_active":"2024-09-18T11:34:50.157413+0000","last_became_peered":"2024-09-18T11:34:50.157413+0000","last_unstale":"2024-09-18T11:35:06.604491+0000","last_undegraded":"2024-09-18T11:35:06.604491+0000","last_fullsized":"2024-09-18T11:35:06.604491+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'72","last_scrub_stamp":"2024-09-18T11:33:50.797790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_clean_scrub_stamp":"2024-09-18T11:33:50.797790+0000","objects_scrubbed":2,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001837497,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2115712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":282,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641913+0000","last_change":"2024-09-18T11:34:08.163615+0000","last_active":"2024-09-18T11:35:06.641913+0000","last_peered":"2024-09-18T11:35:06.641913+0000","last_clean":"2024-09-18T11:35:06.641913+0000","last_became_active":"2024-09-18T11:32:27.079772+0000","last_became_peered":"2024-09-18T11:32:27.079772+0000","last_unstale":"2024-09-18T11:35:06.641913+0000","last_undegraded":"2024-09-18T11:35:06.641913+0000","last_fullsized":"2024-09-18T11:35:06.641913+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'1","last_scrub_stamp":"2024-09-18T11:34:08.163526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:34:08.163526+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:18.531722+0000","scrub_duration":0.001981956,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"182'203","reported_seq":497,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.246545+0000","last_change":"2024-09-18T11:34:05.671877+0000","last_active":"2024-09-18T11:35:06.246545+0000","last_peered":"2024-09-18T11:35:06.246545+0000","last_clean":"2024-09-18T11:35:06.246545+0000","last_became_active":"2024-09-18T11:30:01.077612+0000","last_became_peered":"2024-09-18T11:30:01.077612+0000","last_unstale":"2024-09-18T11:35:06.246545+0000","last_undegraded":"2024-09-18T11:35:06.246545+0000","last_fullsized":"2024-09-18T11:35:06.246545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"182'203","last_scrub_stamp":"2024-09-18T11:34:05.671790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:34:05.671790+0000","objects_scrubbed":51,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:12.454118+0000","scrub_duration":0.0089221579999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42108006,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":185,"num_read_kb":4788,"num_write":230,"num_write_kb":78760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":268,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.604491+0000","last_change":"2024-09-18T11:34:42.697160+0000","last_active":"2024-09-18T11:35:06.604491+0000","last_peered":"2024-09-18T11:35:06.604491+0000","last_clean":"2024-09-18T11:35:06.604491+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:35:06.604491+0000","last_undegraded":"2024-09-18T11:35:06.604491+0000","last_fullsized":"2024-09-18T11:35:06.604491+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:34:42.697075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:34:42.697075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:05.947061+0000","scrub_duration":0.001876314,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"173'206","reported_seq":779,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.246359+0000","last_change":"2024-09-18T11:34:50.152896+0000","last_active":"2024-09-18T11:35:06.246359+0000","last_peered":"2024-09-18T11:35:06.246359+0000","last_clean":"2024-09-18T11:35:06.246359+0000","last_became_active":"2024-09-18T11:34:50.152112+0000","last_became_peered":"2024-09-18T11:34:50.152112+0000","last_unstale":"2024-09-18T11:35:06.246359+0000","last_undegraded":"2024-09-18T11:35:06.246359+0000","last_fullsized":"2024-09-18T11:35:06.246359+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'206","last_scrub_stamp":"2024-09-18T11:34:01.824422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:34:01.824422+0000","objects_scrubbed":46,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:10.875581+0000","scrub_duration":0.0050590330000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43270431,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":179,"num_read_kb":11045,"num_write":225,"num_write_kb":76253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":64589501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":128,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641509+0000","last_change":"2024-09-18T11:34:56.185524+0000","last_active":"2024-09-18T11:35:06.641509+0000","last_peered":"2024-09-18T11:35:06.641509+0000","last_clean":"2024-09-18T11:35:06.641509+0000","last_became_active":"2024-09-18T11:34:56.185242+0000","last_became_peered":"2024-09-18T11:34:56.185242+0000","last_unstale":"2024-09-18T11:35:06.641509+0000","last_undegraded":"2024-09-18T11:35:06.641509+0000","last_fullsized":"2024-09-18T11:35:06.641509+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:34:48.411852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:34:48.411852+0000","last_clean_scrub_stamp":"2024-09-18T11:34:48.411852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:51.892915+0000","scrub_duration":0.001364355,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"172'146","reported_seq":673,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641439+0000","last_change":"2024-09-18T11:34:35.993294+0000","last_active":"2024-09-18T11:35:06.641439+0000","last_peered":"2024-09-18T11:35:06.641439+0000","last_clean":"2024-09-18T11:35:06.641439+0000","last_became_active":"2024-09-18T11:32:59.476088+0000","last_became_peered":"2024-09-18T11:32:59.476088+0000","last_unstale":"2024-09-18T11:35:06.641439+0000","last_undegraded":"2024-09-18T11:35:06.641439+0000","last_fullsized":"2024-09-18T11:35:06.641439+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"172'146","last_scrub_stamp":"2024-09-18T11:34:35.993193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:34:35.993193+0000","objects_scrubbed":35,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:51.068138+0000","scrub_duration":0.0051455880000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24142412,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":133,"num_read_kb":1720,"num_write":158,"num_write_kb":46975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":29291946,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"210'3","reported_seq":258,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641920+0000","last_change":"2024-09-18T11:34:25.106232+0000","last_active":"2024-09-18T11:35:06.641920+0000","last_peered":"2024-09-18T11:35:06.641920+0000","last_clean":"2024-09-18T11:35:06.641920+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:35:06.641920+0000","last_undegraded":"2024-09-18T11:35:06.641920+0000","last_fullsized":"2024-09-18T11:35:06.641920+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'3","last_scrub_stamp":"2024-09-18T11:34:25.106157+0000","last_deep_scrub":"210'3","last_deep_scrub_stamp":"2024-09-18T11:34:25.106157+0000","last_clean_scrub_stamp":"2024-09-18T11:34:25.106157+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:29.899803+0000","scrub_duration":0.0034758250000000001,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"179'157","reported_seq":875,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641856+0000","last_change":"2024-09-18T11:34:50.152718+0000","last_active":"2024-09-18T11:35:06.641856+0000","last_peered":"2024-09-18T11:35:06.641856+0000","last_clean":"2024-09-18T11:35:06.641856+0000","last_became_active":"2024-09-18T11:34:50.152231+0000","last_became_peered":"2024-09-18T11:34:50.152231+0000","last_unstale":"2024-09-18T11:35:06.641856+0000","last_undegraded":"2024-09-18T11:35:06.641856+0000","last_fullsized":"2024-09-18T11:35:06.641856+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"179'157","last_scrub_stamp":"2024-09-18T11:33:49.858681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:33:49.858681+0000","objects_scrubbed":36,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:11.960501+0000","scrub_duration":0.0046045019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31988294,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":149,"num_read_kb":7871,"num_write":171,"num_write_kb":62728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":79753259,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":301,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641431+0000","last_change":"2024-09-18T11:34:47.138275+0000","last_active":"2024-09-18T11:35:06.641431+0000","last_peered":"2024-09-18T11:35:06.641431+0000","last_clean":"2024-09-18T11:35:06.641431+0000","last_became_active":"2024-09-18T11:34:47.137971+0000","last_became_peered":"2024-09-18T11:34:47.137971+0000","last_unstale":"2024-09-18T11:35:06.641431+0000","last_undegraded":"2024-09-18T11:35:06.641431+0000","last_fullsized":"2024-09-18T11:35:06.641431+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:44.054611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:32:30.705704+0000","last_clean_scrub_stamp":"2024-09-18T11:33:44.054611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:12.058086+0000","scrub_duration":0.0015037430000000001,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":301,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.604496+0000","last_change":"2024-09-18T11:34:47.136279+0000","last_active":"2024-09-18T11:35:06.604496+0000","last_peered":"2024-09-18T11:35:06.604496+0000","last_clean":"2024-09-18T11:35:06.604496+0000","last_became_active":"2024-09-18T11:34:47.135985+0000","last_became_peered":"2024-09-18T11:34:47.135985+0000","last_unstale":"2024-09-18T11:35:06.604496+0000","last_undegraded":"2024-09-18T11:35:06.604496+0000","last_fullsized":"2024-09-18T11:35:06.604496+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:59.647914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:59.647914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:27.634628+0000","scrub_duration":0.001373326,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641508+0000","last_change":"2024-09-18T11:34:56.673422+0000","last_active":"2024-09-18T11:35:06.641508+0000","last_peered":"2024-09-18T11:35:06.641508+0000","last_clean":"2024-09-18T11:35:06.641508+0000","last_became_active":"2024-09-18T11:34:56.673243+0000","last_became_peered":"2024-09-18T11:34:56.673243+0000","last_unstale":"2024-09-18T11:35:06.641508+0000","last_undegraded":"2024-09-18T11:35:06.641508+0000","last_fullsized":"2024-09-18T11:35:06.641508+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:56.614347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:56.614347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:24.409305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":263,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.051971+0000","last_change":"2024-09-18T11:34:36.653574+0000","last_active":"2024-09-18T11:35:06.051971+0000","last_peered":"2024-09-18T11:35:06.051971+0000","last_clean":"2024-09-18T11:35:06.051971+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:35:06.051971+0000","last_undegraded":"2024-09-18T11:35:06.051971+0000","last_fullsized":"2024-09-18T11:35:06.051971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:34:36.653477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:34:36.653477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:51.408009+0000","scrub_duration":0.001348115,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":141509143,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":646,"num_read_kb":25424,"num_write":784,"num_write_kb":264716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":422,"num_bytes_recovered":173634706,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":211963904,"data_stored":211458577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9210,"internal_metadata":0},"log_size":712,"ondisk_log_size":712,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2115712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574915,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302688,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069152,"statfs":{"total":96636764160,"available":96326811648,"internally_reserved":0,"allocated":1191936,"data_stored":871593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.432},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.61799999999999999}]},{"osd":1,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.48499999999999999,"15min":0.47299999999999998},"min":{"1min":0.44800000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.50800000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.71699999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.56000000000000005}]},{"osd":2,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.45900000000000002,"15min":0.434},"min":{"1min":0.38700000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.75800000000000001,"5min":1.085,"15min":1.085},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.46100000000000002,"15min":0.44600000000000001},"min":{"1min":0.39900000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.58399999999999996}]},{"osd":3,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.47799999999999998,"15min":0.45500000000000002},"min":{"1min":0.42199999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.48899999999999999,"15min":0.49199999999999999},"min":{"1min":0.432,"5min":0.308,"15min":0.308},"max":{"1min":0.66000000000000003,"5min":1.177,"15min":1.177},"last":0.66100000000000003}]},{"osd":4,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.45100000000000001},"min":{"1min":0.30099999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.60999999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.46000000000000002,"15min":0.49199999999999999},"min":{"1min":0.28199999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.62,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.72499999999999998}]},{"osd":5,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.39900000000000002,"15min":0.433},"min":{"1min":0.30199999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68999999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.46100000000000002},"min":{"1min":0.35899999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63200000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.76200000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47599999999999998,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65600000000000003,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47699999999999998,"15min":0.499},"min":{"1min":0.32500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.629,"5min":0.75,"15min":0.75},"last":0.77900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302168,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069672,"statfs":{"total":96636764160,"available":96327344128,"internally_reserved":0,"allocated":659456,"data_stored":340981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.56200000000000006}]},{"osd":1,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.48399999999999999,"15min":0.47499999999999998},"min":{"1min":0.35699999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.72499999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.47099999999999997,"15min":0.46200000000000002},"min":{"1min":0.25800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.70099999999999996,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.46400000000000002}]},{"osd":2,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.27000000000000002,"5min":0.221,"15min":0.221},"max":{"1min":0.75700000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.48499999999999999,"15min":0.45700000000000002},"min":{"1min":0.40899999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68100000000000005,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.64600000000000002}]},{"osd":3,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48299999999999998,"15min":0.50800000000000001},"min":{"1min":0.29899999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59099999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.52000000000000002},"min":{"1min":0.31,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.71899999999999997,"5min":1.032,"15min":1.032},"last":0.63300000000000001}]},{"osd":4,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42799999999999999,"15min":0.40200000000000002},"min":{"1min":0.31900000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.70999999999999996,"5min":0.755,"15min":0.755},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.47999999999999998,"15min":0.44},"min":{"1min":0.375,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.749,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.35899999999999999}]},{"osd":5,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44800000000000001,"15min":0.41999999999999998},"min":{"1min":0.34499999999999997,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67000000000000004,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.46700000000000003,"15min":0.441},"min":{"1min":0.33200000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.374}]},{"osd":7,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47999999999999998,"15min":0.46899999999999997},"min":{"1min":0.28999999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.72099999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48999999999999999,"15min":0.47999999999999998},"min":{"1min":0.308,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63400000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.40999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574916,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332192,"kb_used_data":26508,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94039648,"statfs":{"total":96636764160,"available":96296599552,"internally_reserved":0,"allocated":27144192,"data_stored":26761132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.56899999999999995}]},{"osd":1,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.46700000000000003,"15min":0.44500000000000001},"min":{"1min":0.40200000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.47799999999999998,"15min":0.46000000000000002},"min":{"1min":0.46100000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.65200000000000002,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.52000000000000002}]},{"osd":2,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.503,"15min":0.503},"min":{"1min":0.443,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73999999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.47099999999999997,"15min":0.49299999999999999},"min":{"1min":0.40300000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.63400000000000001,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.49299999999999999}]},{"osd":3,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.47299999999999998,"15min":0.49099999999999999},"min":{"1min":0.38200000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.499,"15min":0.501},"min":{"1min":0.41399999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68899999999999995,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.55900000000000005}]},{"osd":4,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41899999999999998,"15min":0.44},"min":{"1min":0.32100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.68500000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.41599999999999998,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.222,"15min":0.222},"max":{"1min":0.63300000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.35699999999999998}]},{"osd":6,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41399999999999998,"15min":0.42799999999999999},"min":{"1min":0.32600000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.317},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.45300000000000001,"15min":0.48099999999999998},"min":{"1min":0.30599999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.33600000000000002}]},{"osd":7,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.45800000000000002,"15min":0.49399999999999999},"min":{"1min":0.377,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.70599999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.497,"15min":0.51800000000000002},"min":{"1min":0.39700000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.40799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574915,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356460,"kb_used_data":50784,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94015380,"statfs":{"total":96636764160,"available":96271749120,"internally_reserved":0,"allocated":52002816,"data_stored":51562431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.29299999999999998}]},{"osd":1,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.48299999999999998,"15min":0.502},"min":{"1min":0.42599999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76500000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48099999999999998,"15min":0.497},"min":{"1min":0.34399999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.67300000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.378}]},{"osd":2,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.497,"15min":0.498},"min":{"1min":0.40000000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73199999999999998,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.50700000000000001,"15min":0.52100000000000002},"min":{"1min":0.51800000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.80900000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.57599999999999996}]},{"osd":3,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.48699999999999999,"15min":0.51200000000000001},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72999999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.502,"15min":0.52600000000000002},"min":{"1min":0.379,"5min":0.316,"15min":0.316},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.47999999999999998}]},{"osd":5,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41599999999999998,"15min":0.45400000000000001},"min":{"1min":0.27700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.67900000000000005,"5min":1.002,"15min":1.002},"last":0.504},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.45800000000000002,"15min":0.5},"min":{"1min":0.35099999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.73099999999999998,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.55700000000000005}]},{"osd":6,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.44,"15min":0.46600000000000003},"min":{"1min":0.33500000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.66900000000000004,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.496},"min":{"1min":0.318,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.70599999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.46600000000000003}]},{"osd":7,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48299999999999998,"15min":0.50700000000000001},"min":{"1min":0.33400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.754,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.52500000000000002,"15min":0.54500000000000004},"min":{"1min":0.41099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83999999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.66100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574917,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":328056,"kb_used_data":26532,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94043784,"statfs":{"total":96636764160,"available":96300834816,"internally_reserved":0,"allocated":27168768,"data_stored":26762930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.19400000000000001},{"interface":"front","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.24299999999999999}]},{"osd":1,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.33900000000000002,"15min":0.34399999999999997},"min":{"1min":0.27500000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.55900000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.375,"15min":0.375},"min":{"1min":0.22500000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.57099999999999995,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.313}]},{"osd":2,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.38900000000000001,"15min":0.38500000000000001},"min":{"1min":0.32400000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.53000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.374,"15min":0.39800000000000002},"min":{"1min":0.23799999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.46600000000000003,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.27300000000000002}]},{"osd":4,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.55100000000000005,"15min":0.52700000000000002},"min":{"1min":0.38600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79900000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56899999999999995,"15min":0.54000000000000004},"min":{"1min":0.47499999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78700000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58999999999999997}]},{"osd":5,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.51300000000000001},"min":{"1min":0.46100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77000000000000002,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.56000000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.755,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.63}]},{"osd":6,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52400000000000002,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81100000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.47699999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.74299999999999999,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.58099999999999996}]},{"osd":7,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57599999999999996,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77900000000000003,"5min":1.26,"15min":1.26},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55500000000000005,"15min":0.57499999999999996},"min":{"1min":0.44500000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.749,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.61899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318412,"kb_used_data":16892,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94053428,"statfs":{"total":96636764160,"available":96310710272,"internally_reserved":0,"allocated":17297408,"data_stored":16938301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.378}]},{"osd":1,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34899999999999998,"15min":0.373},"min":{"1min":0.20599999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.42499999999999999,"5min":1.621,"15min":1.621},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.36299999999999999,"15min":0.39400000000000002},"min":{"1min":0.251,"5min":0.218,"15min":0.218},"max":{"1min":0.63600000000000001,"5min":1.589,"15min":1.589},"last":0.40200000000000002}]},{"osd":3,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.35499999999999998,"15min":0.36699999999999999},"min":{"1min":0.28299999999999997,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.42099999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.38200000000000001,"15min":0.41099999999999998},"min":{"1min":0.24099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.56000000000000005,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.52700000000000002}]},{"osd":4,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53500000000000003,"15min":0.55200000000000005},"min":{"1min":0.42899999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70399999999999996,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.52500000000000002},"min":{"1min":0.46400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80000000000000004,"5min":1.012,"15min":1.012},"last":0.73599999999999999}]},{"osd":5,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79000000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.749},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.48499999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.749,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.79100000000000004}]},{"osd":6,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.45600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76300000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.56499999999999995},"min":{"1min":0.48699999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79800000000000004,"5min":1.165,"15min":1.165},"last":0.77700000000000002}]},{"osd":7,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.46000000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82899999999999996,"5min":1.073,"15min":1.073},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.45500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82299999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.85899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322560,"kb_used_data":16884,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94049280,"statfs":{"total":96636764160,"available":96306462720,"internally_reserved":0,"allocated":17289216,"data_stored":16931921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.40600000000000003}]},{"osd":2,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.35599999999999998,"15min":0.378},"min":{"1min":0.255,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.50700000000000001,"5min":1.333,"15min":1.333},"last":0.374},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38600000000000001,"15min":0.41599999999999998},"min":{"1min":0.28599999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.65700000000000003,"5min":1.298,"15min":1.298},"last":0.316}]},{"osd":3,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36399999999999999,"15min":0.38},"min":{"1min":0.24199999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.60499999999999998,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.36099999999999999,"15min":0.36399999999999999},"min":{"1min":0.27100000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55500000000000005,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.35699999999999998}]},{"osd":4,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.496,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.876,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.37,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.74299999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56699999999999995}]},{"osd":5,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40600000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78100000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.52900000000000003,"15min":0.53000000000000003},"min":{"1min":0.48699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.85299999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.44,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82899999999999996,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81599999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57799999999999996}]},{"osd":7,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41799999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85499999999999998,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56200000000000006,"15min":0.57799999999999996},"min":{"1min":0.41999999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83499999999999996,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.52100000000000002}]}]},{"osd":0,"up_from":215,"seq":923417968648,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":350472,"kb_used_data":51316,"kb_used_omap":25,"kb_used_meta":299110,"kb_avail":94021368,"statfs":{"total":96636764160,"available":96277880832,"internally_reserved":0,"allocated":52547584,"data_stored":52096206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25940,"internal_metadata":306289324},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9210,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23064576,"data_stored":23013811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:35:13.239 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:35:13.499 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:35:13.500 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:35:13.512 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-18T11:35:11.983477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":142099530,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":712,"num_read_kb":25479,"num_write":887,"num_write_kb":266346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":432,"num_bytes_recovered":175750418,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2613008,"kb_used_data":190724,"kb_used_omap":36,"kb_used_meta":2422107,"kb_avail":752361712,"statfs":{"total":773094113280,"available":770418393088,"internally_reserved":0,"allocated":195301376,"data_stored":192265495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37868,"internal_metadata":2480237588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.750774"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":421,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.604491+0000","last_change":"2024-09-18T11:34:50.157763+0000","last_active":"2024-09-18T11:35:06.604491+0000","last_peered":"2024-09-18T11:35:06.604491+0000","last_clean":"2024-09-18T11:35:06.604491+0000","last_became_active":"2024-09-18T11:34:50.157413+0000","last_became_peered":"2024-09-18T11:34:50.157413+0000","last_unstale":"2024-09-18T11:35:06.604491+0000","last_undegraded":"2024-09-18T11:35:06.604491+0000","last_fullsized":"2024-09-18T11:35:06.604491+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'72","last_scrub_stamp":"2024-09-18T11:33:50.797790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_clean_scrub_stamp":"2024-09-18T11:33:50.797790+0000","objects_scrubbed":2,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001837497,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2115712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":282,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641913+0000","last_change":"2024-09-18T11:34:08.163615+0000","last_active":"2024-09-18T11:35:06.641913+0000","last_peered":"2024-09-18T11:35:06.641913+0000","last_clean":"2024-09-18T11:35:06.641913+0000","last_became_active":"2024-09-18T11:32:27.079772+0000","last_became_peered":"2024-09-18T11:32:27.079772+0000","last_unstale":"2024-09-18T11:35:06.641913+0000","last_undegraded":"2024-09-18T11:35:06.641913+0000","last_fullsized":"2024-09-18T11:35:06.641913+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'1","last_scrub_stamp":"2024-09-18T11:34:08.163526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:34:08.163526+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:18.531722+0000","scrub_duration":0.001981956,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"182'203","reported_seq":497,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.246545+0000","last_change":"2024-09-18T11:34:05.671877+0000","last_active":"2024-09-18T11:35:06.246545+0000","last_peered":"2024-09-18T11:35:06.246545+0000","last_clean":"2024-09-18T11:35:06.246545+0000","last_became_active":"2024-09-18T11:30:01.077612+0000","last_became_peered":"2024-09-18T11:30:01.077612+0000","last_unstale":"2024-09-18T11:35:06.246545+0000","last_undegraded":"2024-09-18T11:35:06.246545+0000","last_fullsized":"2024-09-18T11:35:06.246545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"182'203","last_scrub_stamp":"2024-09-18T11:34:05.671790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:34:05.671790+0000","objects_scrubbed":51,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:12.454118+0000","scrub_duration":0.0089221579999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42108006,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":185,"num_read_kb":4788,"num_write":230,"num_write_kb":78760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":268,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.604491+0000","last_change":"2024-09-18T11:34:42.697160+0000","last_active":"2024-09-18T11:35:06.604491+0000","last_peered":"2024-09-18T11:35:06.604491+0000","last_clean":"2024-09-18T11:35:06.604491+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:35:06.604491+0000","last_undegraded":"2024-09-18T11:35:06.604491+0000","last_fullsized":"2024-09-18T11:35:06.604491+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:34:42.697075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:34:42.697075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:05.947061+0000","scrub_duration":0.001876314,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"173'206","reported_seq":779,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.246359+0000","last_change":"2024-09-18T11:34:50.152896+0000","last_active":"2024-09-18T11:35:06.246359+0000","last_peered":"2024-09-18T11:35:06.246359+0000","last_clean":"2024-09-18T11:35:06.246359+0000","last_became_active":"2024-09-18T11:34:50.152112+0000","last_became_peered":"2024-09-18T11:34:50.152112+0000","last_unstale":"2024-09-18T11:35:06.246359+0000","last_undegraded":"2024-09-18T11:35:06.246359+0000","last_fullsized":"2024-09-18T11:35:06.246359+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'206","last_scrub_stamp":"2024-09-18T11:34:01.824422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:34:01.824422+0000","objects_scrubbed":46,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:10.875581+0000","scrub_duration":0.0050590330000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43270431,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":179,"num_read_kb":11045,"num_write":225,"num_write_kb":76253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":64589501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":128,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641509+0000","last_change":"2024-09-18T11:34:56.185524+0000","last_active":"2024-09-18T11:35:06.641509+0000","last_peered":"2024-09-18T11:35:06.641509+0000","last_clean":"2024-09-18T11:35:06.641509+0000","last_became_active":"2024-09-18T11:34:56.185242+0000","last_became_peered":"2024-09-18T11:34:56.185242+0000","last_unstale":"2024-09-18T11:35:06.641509+0000","last_undegraded":"2024-09-18T11:35:06.641509+0000","last_fullsized":"2024-09-18T11:35:06.641509+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:34:48.411852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:34:48.411852+0000","last_clean_scrub_stamp":"2024-09-18T11:34:48.411852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:51.892915+0000","scrub_duration":0.001364355,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"172'146","reported_seq":673,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641439+0000","last_change":"2024-09-18T11:34:35.993294+0000","last_active":"2024-09-18T11:35:06.641439+0000","last_peered":"2024-09-18T11:35:06.641439+0000","last_clean":"2024-09-18T11:35:06.641439+0000","last_became_active":"2024-09-18T11:32:59.476088+0000","last_became_peered":"2024-09-18T11:32:59.476088+0000","last_unstale":"2024-09-18T11:35:06.641439+0000","last_undegraded":"2024-09-18T11:35:06.641439+0000","last_fullsized":"2024-09-18T11:35:06.641439+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"172'146","last_scrub_stamp":"2024-09-18T11:34:35.993193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:34:35.993193+0000","objects_scrubbed":35,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:51.068138+0000","scrub_duration":0.0051455880000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24142412,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":133,"num_read_kb":1720,"num_write":158,"num_write_kb":46975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":29291946,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"210'3","reported_seq":258,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641920+0000","last_change":"2024-09-18T11:34:25.106232+0000","last_active":"2024-09-18T11:35:06.641920+0000","last_peered":"2024-09-18T11:35:06.641920+0000","last_clean":"2024-09-18T11:35:06.641920+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:35:06.641920+0000","last_undegraded":"2024-09-18T11:35:06.641920+0000","last_fullsized":"2024-09-18T11:35:06.641920+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'3","last_scrub_stamp":"2024-09-18T11:34:25.106157+0000","last_deep_scrub":"210'3","last_deep_scrub_stamp":"2024-09-18T11:34:25.106157+0000","last_clean_scrub_stamp":"2024-09-18T11:34:25.106157+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:29.899803+0000","scrub_duration":0.0034758250000000001,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"179'157","reported_seq":875,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641856+0000","last_change":"2024-09-18T11:34:50.152718+0000","last_active":"2024-09-18T11:35:06.641856+0000","last_peered":"2024-09-18T11:35:06.641856+0000","last_clean":"2024-09-18T11:35:06.641856+0000","last_became_active":"2024-09-18T11:34:50.152231+0000","last_became_peered":"2024-09-18T11:34:50.152231+0000","last_unstale":"2024-09-18T11:35:06.641856+0000","last_undegraded":"2024-09-18T11:35:06.641856+0000","last_fullsized":"2024-09-18T11:35:06.641856+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"179'157","last_scrub_stamp":"2024-09-18T11:33:49.858681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:33:49.858681+0000","objects_scrubbed":36,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:11.960501+0000","scrub_duration":0.0046045019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31988294,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":149,"num_read_kb":7871,"num_write":171,"num_write_kb":62728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":79753259,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":301,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641431+0000","last_change":"2024-09-18T11:34:47.138275+0000","last_active":"2024-09-18T11:35:06.641431+0000","last_peered":"2024-09-18T11:35:06.641431+0000","last_clean":"2024-09-18T11:35:06.641431+0000","last_became_active":"2024-09-18T11:34:47.137971+0000","last_became_peered":"2024-09-18T11:34:47.137971+0000","last_unstale":"2024-09-18T11:35:06.641431+0000","last_undegraded":"2024-09-18T11:35:06.641431+0000","last_fullsized":"2024-09-18T11:35:06.641431+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:44.054611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:32:30.705704+0000","last_clean_scrub_stamp":"2024-09-18T11:33:44.054611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:12.058086+0000","scrub_duration":0.0015037430000000001,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":301,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.604496+0000","last_change":"2024-09-18T11:34:47.136279+0000","last_active":"2024-09-18T11:35:06.604496+0000","last_peered":"2024-09-18T11:35:06.604496+0000","last_clean":"2024-09-18T11:35:06.604496+0000","last_became_active":"2024-09-18T11:34:47.135985+0000","last_became_peered":"2024-09-18T11:34:47.135985+0000","last_unstale":"2024-09-18T11:35:06.604496+0000","last_undegraded":"2024-09-18T11:35:06.604496+0000","last_fullsized":"2024-09-18T11:35:06.604496+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:59.647914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:59.647914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:27.634628+0000","scrub_duration":0.001373326,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641508+0000","last_change":"2024-09-18T11:34:56.673422+0000","last_active":"2024-09-18T11:35:06.641508+0000","last_peered":"2024-09-18T11:35:06.641508+0000","last_clean":"2024-09-18T11:35:06.641508+0000","last_became_active":"2024-09-18T11:34:56.673243+0000","last_became_peered":"2024-09-18T11:34:56.673243+0000","last_unstale":"2024-09-18T11:35:06.641508+0000","last_undegraded":"2024-09-18T11:35:06.641508+0000","last_fullsized":"2024-09-18T11:35:06.641508+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:56.614347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:56.614347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:24.409305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":263,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.051971+0000","last_change":"2024-09-18T11:34:36.653574+0000","last_active":"2024-09-18T11:35:06.051971+0000","last_peered":"2024-09-18T11:35:06.051971+0000","last_clean":"2024-09-18T11:35:06.051971+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:35:06.051971+0000","last_undegraded":"2024-09-18T11:35:06.051971+0000","last_fullsized":"2024-09-18T11:35:06.051971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:34:36.653477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:34:36.653477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:51.408009+0000","scrub_duration":0.001348115,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":141509143,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":646,"num_read_kb":25424,"num_write":784,"num_write_kb":264716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":422,"num_bytes_recovered":173634706,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":211963904,"data_stored":211458577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9210,"internal_metadata":0},"log_size":712,"ondisk_log_size":712,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2115712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574915,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302688,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069152,"statfs":{"total":96636764160,"available":96326811648,"internally_reserved":0,"allocated":1191936,"data_stored":871593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.432},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.61799999999999999}]},{"osd":1,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.48499999999999999,"15min":0.47299999999999998},"min":{"1min":0.44800000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.50800000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.71699999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.56000000000000005}]},{"osd":2,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.45900000000000002,"15min":0.434},"min":{"1min":0.38700000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.75800000000000001,"5min":1.085,"15min":1.085},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.46100000000000002,"15min":0.44600000000000001},"min":{"1min":0.39900000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.58399999999999996}]},{"osd":3,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.47799999999999998,"15min":0.45500000000000002},"min":{"1min":0.42199999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.48899999999999999,"15min":0.49199999999999999},"min":{"1min":0.432,"5min":0.308,"15min":0.308},"max":{"1min":0.66000000000000003,"5min":1.177,"15min":1.177},"last":0.66100000000000003}]},{"osd":4,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.45100000000000001},"min":{"1min":0.30099999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.60999999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.46000000000000002,"15min":0.49199999999999999},"min":{"1min":0.28199999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.62,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.72499999999999998}]},{"osd":5,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.39900000000000002,"15min":0.433},"min":{"1min":0.30199999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68999999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.46100000000000002},"min":{"1min":0.35899999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63200000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.76200000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47599999999999998,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65600000000000003,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47699999999999998,"15min":0.499},"min":{"1min":0.32500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.629,"5min":0.75,"15min":0.75},"last":0.77900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302168,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069672,"statfs":{"total":96636764160,"available":96327344128,"internally_reserved":0,"allocated":659456,"data_stored":340981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.56200000000000006}]},{"osd":1,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.48399999999999999,"15min":0.47499999999999998},"min":{"1min":0.35699999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.72499999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.47099999999999997,"15min":0.46200000000000002},"min":{"1min":0.25800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.70099999999999996,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.46400000000000002}]},{"osd":2,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.27000000000000002,"5min":0.221,"15min":0.221},"max":{"1min":0.75700000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.48499999999999999,"15min":0.45700000000000002},"min":{"1min":0.40899999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68100000000000005,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.64600000000000002}]},{"osd":3,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48299999999999998,"15min":0.50800000000000001},"min":{"1min":0.29899999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59099999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.52000000000000002},"min":{"1min":0.31,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.71899999999999997,"5min":1.032,"15min":1.032},"last":0.63300000000000001}]},{"osd":4,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42799999999999999,"15min":0.40200000000000002},"min":{"1min":0.31900000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.70999999999999996,"5min":0.755,"15min":0.755},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.47999999999999998,"15min":0.44},"min":{"1min":0.375,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.749,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.35899999999999999}]},{"osd":5,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44800000000000001,"15min":0.41999999999999998},"min":{"1min":0.34499999999999997,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67000000000000004,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.46700000000000003,"15min":0.441},"min":{"1min":0.33200000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.374}]},{"osd":7,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47999999999999998,"15min":0.46899999999999997},"min":{"1min":0.28999999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.72099999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48999999999999999,"15min":0.47999999999999998},"min":{"1min":0.308,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63400000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.40999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574916,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332192,"kb_used_data":26508,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94039648,"statfs":{"total":96636764160,"available":96296599552,"internally_reserved":0,"allocated":27144192,"data_stored":26761132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.56899999999999995}]},{"osd":1,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.46700000000000003,"15min":0.44500000000000001},"min":{"1min":0.40200000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.47799999999999998,"15min":0.46000000000000002},"min":{"1min":0.46100000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.65200000000000002,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.52000000000000002}]},{"osd":2,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.503,"15min":0.503},"min":{"1min":0.443,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73999999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.47099999999999997,"15min":0.49299999999999999},"min":{"1min":0.40300000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.63400000000000001,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.49299999999999999}]},{"osd":3,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.47299999999999998,"15min":0.49099999999999999},"min":{"1min":0.38200000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.499,"15min":0.501},"min":{"1min":0.41399999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68899999999999995,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.55900000000000005}]},{"osd":4,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41899999999999998,"15min":0.44},"min":{"1min":0.32100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.68500000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.41599999999999998,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.222,"15min":0.222},"max":{"1min":0.63300000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.35699999999999998}]},{"osd":6,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41399999999999998,"15min":0.42799999999999999},"min":{"1min":0.32600000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.317},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.45300000000000001,"15min":0.48099999999999998},"min":{"1min":0.30599999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.33600000000000002}]},{"osd":7,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.45800000000000002,"15min":0.49399999999999999},"min":{"1min":0.377,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.70599999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.497,"15min":0.51800000000000002},"min":{"1min":0.39700000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.40799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574915,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356460,"kb_used_data":50784,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94015380,"statfs":{"total":96636764160,"available":96271749120,"internally_reserved":0,"allocated":52002816,"data_stored":51562431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.29299999999999998}]},{"osd":1,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.48299999999999998,"15min":0.502},"min":{"1min":0.42599999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76500000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48099999999999998,"15min":0.497},"min":{"1min":0.34399999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.67300000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.378}]},{"osd":2,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.497,"15min":0.498},"min":{"1min":0.40000000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73199999999999998,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.50700000000000001,"15min":0.52100000000000002},"min":{"1min":0.51800000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.80900000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.57599999999999996}]},{"osd":3,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.48699999999999999,"15min":0.51200000000000001},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72999999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.502,"15min":0.52600000000000002},"min":{"1min":0.379,"5min":0.316,"15min":0.316},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.47999999999999998}]},{"osd":5,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41599999999999998,"15min":0.45400000000000001},"min":{"1min":0.27700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.67900000000000005,"5min":1.002,"15min":1.002},"last":0.504},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.45800000000000002,"15min":0.5},"min":{"1min":0.35099999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.73099999999999998,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.55700000000000005}]},{"osd":6,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.44,"15min":0.46600000000000003},"min":{"1min":0.33500000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.66900000000000004,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.496},"min":{"1min":0.318,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.70599999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.46600000000000003}]},{"osd":7,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48299999999999998,"15min":0.50700000000000001},"min":{"1min":0.33400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.754,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.52500000000000002,"15min":0.54500000000000004},"min":{"1min":0.41099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83999999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.66100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574917,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":328056,"kb_used_data":26532,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94043784,"statfs":{"total":96636764160,"available":96300834816,"internally_reserved":0,"allocated":27168768,"data_stored":26762930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.19400000000000001},{"interface":"front","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.24299999999999999}]},{"osd":1,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.33900000000000002,"15min":0.34399999999999997},"min":{"1min":0.27500000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.55900000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.375,"15min":0.375},"min":{"1min":0.22500000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.57099999999999995,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.313}]},{"osd":2,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.38900000000000001,"15min":0.38500000000000001},"min":{"1min":0.32400000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.53000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.374,"15min":0.39800000000000002},"min":{"1min":0.23799999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.46600000000000003,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.27300000000000002}]},{"osd":4,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.55100000000000005,"15min":0.52700000000000002},"min":{"1min":0.38600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79900000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56899999999999995,"15min":0.54000000000000004},"min":{"1min":0.47499999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78700000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58999999999999997}]},{"osd":5,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.51300000000000001},"min":{"1min":0.46100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77000000000000002,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.56000000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.755,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.63}]},{"osd":6,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52400000000000002,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81100000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.47699999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.74299999999999999,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.58099999999999996}]},{"osd":7,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57599999999999996,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77900000000000003,"5min":1.26,"15min":1.26},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55500000000000005,"15min":0.57499999999999996},"min":{"1min":0.44500000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.749,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.61899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318412,"kb_used_data":16892,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94053428,"statfs":{"total":96636764160,"available":96310710272,"internally_reserved":0,"allocated":17297408,"data_stored":16938301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.378}]},{"osd":1,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34899999999999998,"15min":0.373},"min":{"1min":0.20599999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.42499999999999999,"5min":1.621,"15min":1.621},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.36299999999999999,"15min":0.39400000000000002},"min":{"1min":0.251,"5min":0.218,"15min":0.218},"max":{"1min":0.63600000000000001,"5min":1.589,"15min":1.589},"last":0.40200000000000002}]},{"osd":3,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.35499999999999998,"15min":0.36699999999999999},"min":{"1min":0.28299999999999997,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.42099999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.38200000000000001,"15min":0.41099999999999998},"min":{"1min":0.24099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.56000000000000005,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.52700000000000002}]},{"osd":4,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53500000000000003,"15min":0.55200000000000005},"min":{"1min":0.42899999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70399999999999996,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.52500000000000002},"min":{"1min":0.46400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80000000000000004,"5min":1.012,"15min":1.012},"last":0.73599999999999999}]},{"osd":5,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79000000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.749},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.48499999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.749,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.79100000000000004}]},{"osd":6,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.45600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76300000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.56499999999999995},"min":{"1min":0.48699999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79800000000000004,"5min":1.165,"15min":1.165},"last":0.77700000000000002}]},{"osd":7,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.46000000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82899999999999996,"5min":1.073,"15min":1.073},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.45500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82299999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.85899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322560,"kb_used_data":16884,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94049280,"statfs":{"total":96636764160,"available":96306462720,"internally_reserved":0,"allocated":17289216,"data_stored":16931921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.40600000000000003}]},{"osd":2,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.35599999999999998,"15min":0.378},"min":{"1min":0.255,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.50700000000000001,"5min":1.333,"15min":1.333},"last":0.374},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38600000000000001,"15min":0.41599999999999998},"min":{"1min":0.28599999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.65700000000000003,"5min":1.298,"15min":1.298},"last":0.316}]},{"osd":3,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36399999999999999,"15min":0.38},"min":{"1min":0.24199999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.60499999999999998,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.36099999999999999,"15min":0.36399999999999999},"min":{"1min":0.27100000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55500000000000005,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.35699999999999998}]},{"osd":4,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.496,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.876,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.37,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.74299999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56699999999999995}]},{"osd":5,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40600000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78100000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.52900000000000003,"15min":0.53000000000000003},"min":{"1min":0.48699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.85299999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.44,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82899999999999996,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81599999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57799999999999996}]},{"osd":7,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41799999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85499999999999998,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56200000000000006,"15min":0.57799999999999996},"min":{"1min":0.41999999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83499999999999996,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.52100000000000002}]}]},{"osd":0,"up_from":215,"seq":923417968648,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":350472,"kb_used_data":51316,"kb_used_omap":25,"kb_used_meta":299110,"kb_avail":94021368,"statfs":{"total":96636764160,"available":96277880832,"internally_reserved":0,"allocated":52547584,"data_stored":52096206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25940,"internal_metadata":306289324},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9210,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23064576,"data_stored":23013811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:35:13.513 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T11:35:13.514 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:35:13.576 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-09-18T11:35:13.574+0000 7f2d449e8640 -1 osd.1 236 handle_osd_map injecting map crc failure 2024-09-18T11:35:13.777 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:35:13.778 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:35:13.789 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-09-18T11:35:13.574799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":142099530,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":712,"num_read_kb":25479,"num_write":887,"num_write_kb":266346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":432,"num_bytes_recovered":175750418,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":839,"ondisk_log_size":839,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2613044,"kb_used_data":190772,"kb_used_omap":36,"kb_used_meta":2422107,"kb_avail":752361676,"statfs":{"total":773094113280,"available":770418356224,"internally_reserved":0,"allocated":195350528,"data_stored":192289387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37868,"internal_metadata":2480237588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.594465"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":431,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-18T11:35:10.515786+0000","last_change":"2024-09-18T11:35:10.515732+0000","last_active":"2024-09-18T11:35:10.515786+0000","last_peered":"2024-09-18T11:35:10.515786+0000","last_clean":"2024-09-18T11:35:10.515786+0000","last_became_active":"2024-09-18T11:34:50.157413+0000","last_became_peered":"2024-09-18T11:34:50.157413+0000","last_unstale":"2024-09-18T11:35:10.515786+0000","last_undegraded":"2024-09-18T11:35:10.515786+0000","last_fullsized":"2024-09-18T11:35:10.515786+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'72","last_scrub_stamp":"2024-09-18T11:35:10.515634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_clean_scrub_stamp":"2024-09-18T11:35:10.515634+0000","objects_scrubbed":2,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:30.216617+0000","scrub_duration":0.0021162030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2115712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":282,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641913+0000","last_change":"2024-09-18T11:34:08.163615+0000","last_active":"2024-09-18T11:35:06.641913+0000","last_peered":"2024-09-18T11:35:06.641913+0000","last_clean":"2024-09-18T11:35:06.641913+0000","last_became_active":"2024-09-18T11:32:27.079772+0000","last_became_peered":"2024-09-18T11:32:27.079772+0000","last_unstale":"2024-09-18T11:35:06.641913+0000","last_undegraded":"2024-09-18T11:35:06.641913+0000","last_fullsized":"2024-09-18T11:35:06.641913+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'1","last_scrub_stamp":"2024-09-18T11:34:08.163526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:34:08.163526+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:18.531722+0000","scrub_duration":0.001981956,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"236'254","reported_seq":560,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-18T11:35:12.570775+0000","last_change":"2024-09-18T11:35:12.570752+0000","last_active":"2024-09-18T11:35:12.570775+0000","last_peered":"2024-09-18T11:35:12.570775+0000","last_clean":"2024-09-18T11:35:12.570775+0000","last_became_active":"2024-09-18T11:30:01.077612+0000","last_became_peered":"2024-09-18T11:30:01.077612+0000","last_unstale":"2024-09-18T11:35:12.570775+0000","last_undegraded":"2024-09-18T11:35:12.570775+0000","last_fullsized":"2024-09-18T11:35:12.570775+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'254","last_scrub_stamp":"2024-09-18T11:35:12.570681+0000","last_deep_scrub":"236'254","last_deep_scrub_stamp":"2024-09-18T11:35:12.570681+0000","last_clean_scrub_stamp":"2024-09-18T11:35:12.570681+0000","objects_scrubbed":51,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:14.062587+0000","scrub_duration":0.084234940999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42108006,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":185,"num_read_kb":4788,"num_write":230,"num_write_kb":78760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":268,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.604491+0000","last_change":"2024-09-18T11:34:42.697160+0000","last_active":"2024-09-18T11:35:06.604491+0000","last_peered":"2024-09-18T11:35:06.604491+0000","last_clean":"2024-09-18T11:35:06.604491+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:35:06.604491+0000","last_undegraded":"2024-09-18T11:35:06.604491+0000","last_fullsized":"2024-09-18T11:35:06.604491+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:34:42.697075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:34:42.697075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:05.947061+0000","scrub_duration":0.001876314,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"173'206","reported_seq":779,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.246359+0000","last_change":"2024-09-18T11:34:50.152896+0000","last_active":"2024-09-18T11:35:06.246359+0000","last_peered":"2024-09-18T11:35:06.246359+0000","last_clean":"2024-09-18T11:35:06.246359+0000","last_became_active":"2024-09-18T11:34:50.152112+0000","last_became_peered":"2024-09-18T11:34:50.152112+0000","last_unstale":"2024-09-18T11:35:06.246359+0000","last_undegraded":"2024-09-18T11:35:06.246359+0000","last_fullsized":"2024-09-18T11:35:06.246359+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'206","last_scrub_stamp":"2024-09-18T11:34:01.824422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:34:01.824422+0000","objects_scrubbed":46,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:10.875581+0000","scrub_duration":0.0050590330000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43270431,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":179,"num_read_kb":11045,"num_write":225,"num_write_kb":76253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":64589501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":128,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641509+0000","last_change":"2024-09-18T11:34:56.185524+0000","last_active":"2024-09-18T11:35:06.641509+0000","last_peered":"2024-09-18T11:35:06.641509+0000","last_clean":"2024-09-18T11:35:06.641509+0000","last_became_active":"2024-09-18T11:34:56.185242+0000","last_became_peered":"2024-09-18T11:34:56.185242+0000","last_unstale":"2024-09-18T11:35:06.641509+0000","last_undegraded":"2024-09-18T11:35:06.641509+0000","last_fullsized":"2024-09-18T11:35:06.641509+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:34:48.411852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:34:48.411852+0000","last_clean_scrub_stamp":"2024-09-18T11:34:48.411852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:51.892915+0000","scrub_duration":0.001364355,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"172'146","reported_seq":673,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641439+0000","last_change":"2024-09-18T11:34:35.993294+0000","last_active":"2024-09-18T11:35:06.641439+0000","last_peered":"2024-09-18T11:35:06.641439+0000","last_clean":"2024-09-18T11:35:06.641439+0000","last_became_active":"2024-09-18T11:32:59.476088+0000","last_became_peered":"2024-09-18T11:32:59.476088+0000","last_unstale":"2024-09-18T11:35:06.641439+0000","last_undegraded":"2024-09-18T11:35:06.641439+0000","last_fullsized":"2024-09-18T11:35:06.641439+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"172'146","last_scrub_stamp":"2024-09-18T11:34:35.993193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:34:35.993193+0000","objects_scrubbed":35,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:51.068138+0000","scrub_duration":0.0051455880000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24142412,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":133,"num_read_kb":1720,"num_write":158,"num_write_kb":46975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":29291946,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"210'3","reported_seq":258,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641920+0000","last_change":"2024-09-18T11:34:25.106232+0000","last_active":"2024-09-18T11:35:06.641920+0000","last_peered":"2024-09-18T11:35:06.641920+0000","last_clean":"2024-09-18T11:35:06.641920+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:35:06.641920+0000","last_undegraded":"2024-09-18T11:35:06.641920+0000","last_fullsized":"2024-09-18T11:35:06.641920+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'3","last_scrub_stamp":"2024-09-18T11:34:25.106157+0000","last_deep_scrub":"210'3","last_deep_scrub_stamp":"2024-09-18T11:34:25.106157+0000","last_clean_scrub_stamp":"2024-09-18T11:34:25.106157+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:29.899803+0000","scrub_duration":0.0034758250000000001,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"179'157","reported_seq":875,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641856+0000","last_change":"2024-09-18T11:34:50.152718+0000","last_active":"2024-09-18T11:35:06.641856+0000","last_peered":"2024-09-18T11:35:06.641856+0000","last_clean":"2024-09-18T11:35:06.641856+0000","last_became_active":"2024-09-18T11:34:50.152231+0000","last_became_peered":"2024-09-18T11:34:50.152231+0000","last_unstale":"2024-09-18T11:35:06.641856+0000","last_undegraded":"2024-09-18T11:35:06.641856+0000","last_fullsized":"2024-09-18T11:35:06.641856+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"179'157","last_scrub_stamp":"2024-09-18T11:33:49.858681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:33:49.858681+0000","objects_scrubbed":36,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:11.960501+0000","scrub_duration":0.0046045019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31988294,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":149,"num_read_kb":7871,"num_write":171,"num_write_kb":62728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":79753259,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":311,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-18T11:35:12.141110+0000","last_change":"2024-09-18T11:35:12.141068+0000","last_active":"2024-09-18T11:35:12.141110+0000","last_peered":"2024-09-18T11:35:12.141110+0000","last_clean":"2024-09-18T11:35:12.141110+0000","last_became_active":"2024-09-18T11:34:47.137971+0000","last_became_peered":"2024-09-18T11:34:47.137971+0000","last_unstale":"2024-09-18T11:35:12.141110+0000","last_undegraded":"2024-09-18T11:35:12.141110+0000","last_fullsized":"2024-09-18T11:35:12.141110+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:35:12.140948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:32:30.705704+0000","last_clean_scrub_stamp":"2024-09-18T11:35:12.140948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:24.078144+0000","scrub_duration":0.001329036,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":301,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.604496+0000","last_change":"2024-09-18T11:34:47.136279+0000","last_active":"2024-09-18T11:35:06.604496+0000","last_peered":"2024-09-18T11:35:06.604496+0000","last_clean":"2024-09-18T11:35:06.604496+0000","last_became_active":"2024-09-18T11:34:47.135985+0000","last_became_peered":"2024-09-18T11:34:47.135985+0000","last_unstale":"2024-09-18T11:35:06.604496+0000","last_undegraded":"2024-09-18T11:35:06.604496+0000","last_fullsized":"2024-09-18T11:35:06.604496+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:59.647914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:59.647914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:27.634628+0000","scrub_duration":0.001373326,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.641508+0000","last_change":"2024-09-18T11:34:56.673422+0000","last_active":"2024-09-18T11:35:06.641508+0000","last_peered":"2024-09-18T11:35:06.641508+0000","last_clean":"2024-09-18T11:35:06.641508+0000","last_became_active":"2024-09-18T11:34:56.673243+0000","last_became_peered":"2024-09-18T11:34:56.673243+0000","last_unstale":"2024-09-18T11:35:06.641508+0000","last_undegraded":"2024-09-18T11:35:06.641508+0000","last_fullsized":"2024-09-18T11:35:06.641508+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:56.614347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:56.614347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:24.409305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":264,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-18T11:35:06.248756+0000","last_change":"2024-09-18T11:34:36.653574+0000","last_active":"2024-09-18T11:35:06.248756+0000","last_peered":"2024-09-18T11:35:06.248756+0000","last_clean":"2024-09-18T11:35:06.248756+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:35:06.248756+0000","last_undegraded":"2024-09-18T11:35:06.248756+0000","last_fullsized":"2024-09-18T11:35:06.248756+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:34:36.653477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:34:36.653477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:51.408009+0000","scrub_duration":0.001348115,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":141509143,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":646,"num_read_kb":25424,"num_write":784,"num_write_kb":264716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":422,"num_bytes_recovered":173634706,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":211963904,"data_stored":211458577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9210,"internal_metadata":0},"log_size":763,"ondisk_log_size":763,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2115712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574916,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302688,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069152,"statfs":{"total":96636764160,"available":96326811648,"internally_reserved":0,"allocated":1204224,"data_stored":877566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.315},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.377}]},{"osd":1,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.48499999999999999,"15min":0.47299999999999998},"min":{"1min":0.44800000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.50800000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.71699999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.41499999999999998}]},{"osd":2,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.45900000000000002,"15min":0.434},"min":{"1min":0.38700000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.75800000000000001,"5min":1.085,"15min":1.085},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.46100000000000002,"15min":0.44600000000000001},"min":{"1min":0.39900000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.47199999999999998}]},{"osd":3,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.47799999999999998,"15min":0.45500000000000002},"min":{"1min":0.42199999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.48899999999999999,"15min":0.49199999999999999},"min":{"1min":0.432,"5min":0.308,"15min":0.308},"max":{"1min":0.66000000000000003,"5min":1.177,"15min":1.177},"last":0.54900000000000004}]},{"osd":4,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.45100000000000001},"min":{"1min":0.30099999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.60999999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.46000000000000002,"15min":0.49199999999999999},"min":{"1min":0.28199999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.62,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.58899999999999997}]},{"osd":5,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.39900000000000002,"15min":0.433},"min":{"1min":0.30199999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68999999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.46100000000000002},"min":{"1min":0.35899999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63200000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.57699999999999996}]},{"osd":6,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47599999999999998,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65600000000000003,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.629},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47699999999999998,"15min":0.499},"min":{"1min":0.32500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.629,"5min":0.75,"15min":0.75},"last":0.62}]}]},{"osd":6,"up_from":13,"seq":55834574916,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302172,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069668,"statfs":{"total":96636764160,"available":96327340032,"internally_reserved":0,"allocated":671744,"data_stored":346954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.501}]},{"osd":1,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.48399999999999999,"15min":0.47499999999999998},"min":{"1min":0.35699999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.72499999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.47099999999999997,"15min":0.46200000000000002},"min":{"1min":0.25800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.70099999999999996,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.48899999999999999}]},{"osd":2,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.27000000000000002,"5min":0.221,"15min":0.221},"max":{"1min":0.75700000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.48499999999999999,"15min":0.45700000000000002},"min":{"1min":0.40899999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68100000000000005,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.52900000000000003}]},{"osd":3,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48299999999999998,"15min":0.50800000000000001},"min":{"1min":0.29899999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59099999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.52000000000000002},"min":{"1min":0.31,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.71899999999999997,"5min":1.032,"15min":1.032},"last":0.47099999999999997}]},{"osd":4,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42799999999999999,"15min":0.40200000000000002},"min":{"1min":0.31900000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.70999999999999996,"5min":0.755,"15min":0.755},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.47999999999999998,"15min":0.44},"min":{"1min":0.375,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.749,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.54400000000000004}]},{"osd":5,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44800000000000001,"15min":0.41999999999999998},"min":{"1min":0.34499999999999997,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67000000000000004,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.441},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.46700000000000003,"15min":0.441},"min":{"1min":0.33200000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.41699999999999998}]},{"osd":7,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47999999999999998,"15min":0.46899999999999997},"min":{"1min":0.28999999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.72099999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48999999999999999,"15min":0.47999999999999998},"min":{"1min":0.308,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63400000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.35399999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574917,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332192,"kb_used_data":26508,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94039648,"statfs":{"total":96636764160,"available":96296599552,"internally_reserved":0,"allocated":27144192,"data_stored":26761132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.505}]},{"osd":1,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.46700000000000003,"15min":0.44500000000000001},"min":{"1min":0.40200000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.47799999999999998,"15min":0.46000000000000002},"min":{"1min":0.46100000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.65200000000000002,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.53100000000000003}]},{"osd":2,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.503,"15min":0.503},"min":{"1min":0.443,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73999999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.437},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.47099999999999997,"15min":0.49299999999999999},"min":{"1min":0.40300000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.63400000000000001,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.47299999999999998}]},{"osd":3,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.47299999999999998,"15min":0.49099999999999999},"min":{"1min":0.38200000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.499,"15min":0.501},"min":{"1min":0.41399999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68899999999999995,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.48799999999999999}]},{"osd":4,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41899999999999998,"15min":0.44},"min":{"1min":0.32100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.68500000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.41599999999999998,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.222,"15min":0.222},"max":{"1min":0.63300000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.372}]},{"osd":6,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41399999999999998,"15min":0.42799999999999999},"min":{"1min":0.32600000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.45300000000000001,"15min":0.48099999999999998},"min":{"1min":0.30599999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.46200000000000002}]},{"osd":7,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.45800000000000002,"15min":0.49399999999999999},"min":{"1min":0.377,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.70599999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.497,"15min":0.51800000000000002},"min":{"1min":0.39700000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.40400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574916,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356476,"kb_used_data":50796,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94015364,"statfs":{"total":96636764160,"available":96271732736,"internally_reserved":0,"allocated":52015104,"data_stored":51568404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.55700000000000005}]},{"osd":1,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.48299999999999998,"15min":0.502},"min":{"1min":0.42599999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76500000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48099999999999998,"15min":0.497},"min":{"1min":0.34399999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.67300000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.54300000000000004}]},{"osd":2,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.497,"15min":0.498},"min":{"1min":0.40000000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73199999999999998,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.50700000000000001,"15min":0.52100000000000002},"min":{"1min":0.51800000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.80900000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.47999999999999998}]},{"osd":3,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.48699999999999999,"15min":0.51200000000000001},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72999999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.502,"15min":0.52600000000000002},"min":{"1min":0.379,"5min":0.316,"15min":0.316},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.59799999999999998}]},{"osd":5,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41599999999999998,"15min":0.45400000000000001},"min":{"1min":0.27700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.67900000000000005,"5min":1.002,"15min":1.002},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.45800000000000002,"15min":0.5},"min":{"1min":0.35099999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.73099999999999998,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.496}]},{"osd":6,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.44,"15min":0.46600000000000003},"min":{"1min":0.33500000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.66900000000000004,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.37},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.496},"min":{"1min":0.318,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.70599999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.441}]},{"osd":7,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48299999999999998,"15min":0.50700000000000001},"min":{"1min":0.33400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.754,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.52500000000000002,"15min":0.54500000000000004},"min":{"1min":0.41099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83999999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.62}]}]},{"osd":3,"up_from":13,"seq":55834574918,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":328056,"kb_used_data":26532,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94043784,"statfs":{"total":96636764160,"available":96300834816,"internally_reserved":0,"allocated":27168768,"data_stored":26762930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.20399999999999999}]},{"osd":1,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.33900000000000002,"15min":0.34399999999999997},"min":{"1min":0.27500000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.55900000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.375,"15min":0.375},"min":{"1min":0.22500000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.57099999999999995,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.314}]},{"osd":2,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.38900000000000001,"15min":0.38500000000000001},"min":{"1min":0.32400000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.53000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.374,"15min":0.39800000000000002},"min":{"1min":0.23799999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.46600000000000003,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.30399999999999999}]},{"osd":4,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.55100000000000005,"15min":0.52700000000000002},"min":{"1min":0.38600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79900000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56899999999999995,"15min":0.54000000000000004},"min":{"1min":0.47499999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78700000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.70599999999999996}]},{"osd":5,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.51300000000000001},"min":{"1min":0.46100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77000000000000002,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.56000000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.755,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.69099999999999995}]},{"osd":6,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52400000000000002,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81100000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.47699999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.74299999999999999,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.65800000000000003}]},{"osd":7,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57599999999999996,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77900000000000003,"5min":1.26,"15min":1.26},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55500000000000005,"15min":0.57499999999999996},"min":{"1min":0.44500000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.749,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574916,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318412,"kb_used_data":16892,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94053428,"statfs":{"total":96636764160,"available":96310710272,"internally_reserved":0,"allocated":17297408,"data_stored":16938301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.33100000000000002}]},{"osd":1,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34899999999999998,"15min":0.373},"min":{"1min":0.20599999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.42499999999999999,"5min":1.621,"15min":1.621},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.36299999999999999,"15min":0.39400000000000002},"min":{"1min":0.251,"5min":0.218,"15min":0.218},"max":{"1min":0.63600000000000001,"5min":1.589,"15min":1.589},"last":0.34499999999999997}]},{"osd":3,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.35499999999999998,"15min":0.36699999999999999},"min":{"1min":0.28299999999999997,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.42099999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.38200000000000001,"15min":0.41099999999999998},"min":{"1min":0.24099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.56000000000000005,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.36699999999999999}]},{"osd":4,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53500000000000003,"15min":0.55200000000000005},"min":{"1min":0.42899999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70399999999999996,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.52500000000000002},"min":{"1min":0.46400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80000000000000004,"5min":1.012,"15min":1.012},"last":0.68300000000000005}]},{"osd":5,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79000000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.48499999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.749,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.60899999999999999}]},{"osd":6,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.45600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76300000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.56499999999999995},"min":{"1min":0.48699999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79800000000000004,"5min":1.165,"15min":1.165},"last":0.70299999999999996}]},{"osd":7,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.46000000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82899999999999996,"5min":1.073,"15min":1.073},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.45500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82299999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.56100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322576,"kb_used_data":16896,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94049264,"statfs":{"total":96636764160,"available":96306446336,"internally_reserved":0,"allocated":17301504,"data_stored":16937894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.27500000000000002}]},{"osd":2,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.35599999999999998,"15min":0.378},"min":{"1min":0.255,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.50700000000000001,"5min":1.333,"15min":1.333},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38600000000000001,"15min":0.41599999999999998},"min":{"1min":0.28599999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.65700000000000003,"5min":1.298,"15min":1.298},"last":0.30599999999999999}]},{"osd":3,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36399999999999999,"15min":0.38},"min":{"1min":0.24199999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.60499999999999998,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.36099999999999999,"15min":0.36399999999999999},"min":{"1min":0.27100000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55500000000000005,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.25900000000000001}]},{"osd":4,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.496,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.876,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.37,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.74299999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.48199999999999998}]},{"osd":5,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40600000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78100000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.52900000000000003,"15min":0.53000000000000003},"min":{"1min":0.48699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.85299999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.498}]},{"osd":6,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.44,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82899999999999996,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.622},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81599999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.63400000000000001}]},{"osd":7,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41799999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85499999999999998,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56200000000000006,"15min":0.57799999999999996},"min":{"1min":0.41999999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83499999999999996,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.54900000000000004}]}]},{"osd":0,"up_from":215,"seq":923417968648,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":350472,"kb_used_data":51316,"kb_used_omap":25,"kb_used_meta":299110,"kb_avail":94021368,"statfs":{"total":96636764160,"available":96277880832,"internally_reserved":0,"allocated":52547584,"data_stored":52096206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25940,"internal_metadata":306289324},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9210,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23064576,"data_stored":23013811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:35:13.791 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T11:35:14.053 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:35:14.053 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":237,"fsid":"f4cca259-b87a-45cc-8995-7e9fca891a9f","created":"2024-09-18T11:29:38.205487+0000","modified":"2024-09-18T11:35:13.569521+0000","last_up_change":"2024-09-18T11:34:45.106232+0000","last_in_change":"2024-09-18T11:34:46.116722+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T11:29:47.951262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T11:29:48.866238+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"low_tier","create_time":"2024-09-18T11:29:59.343964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"12faced2-7888-4265-ad12-815efe36466c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":119,"last_clean_end":205,"up_from":215,"up_thru":219,"down_at":206,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":74987},{"type":"v1","addr":"172.21.15.119:6801","nonce":74987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":74987},{"type":"v1","addr":"172.21.15.119:6803","nonce":74987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":74987},{"type":"v1","addr":"172.21.15.119:6807","nonce":74987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":74987},{"type":"v1","addr":"172.21.15.119:6805","nonce":74987}]},"public_addr":"172.21.15.119:6801/74987","cluster_addr":"172.21.15.119:6803/74987","heartbeat_back_addr":"172.21.15.119:6807/74987","heartbeat_front_addr":"172.21.15.119:6805/74987","state":["exists","up"]},{"osd":1,"uuid":"20fee29d-b047-46ae-aa60-88e897fbb0fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":33350},{"type":"v1","addr":"172.21.15.119:6825","nonce":33350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":33350},{"type":"v1","addr":"172.21.15.119:6827","nonce":33350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":33350},{"type":"v1","addr":"172.21.15.119:6831","nonce":33350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":33350},{"type":"v1","addr":"172.21.15.119:6829","nonce":33350}]},"public_addr":"172.21.15.119:6825/33350","cluster_addr":"172.21.15.119:6827/33350","heartbeat_back_addr":"172.21.15.119:6831/33350","heartbeat_front_addr":"172.21.15.119:6829/33350","state":["exists","up"]},{"osd":2,"uuid":"21a296f3-b259-4841-9a17-4cd21b302b36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":33351},{"type":"v1","addr":"172.21.15.119:6809","nonce":33351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":33351},{"type":"v1","addr":"172.21.15.119:6813","nonce":33351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6819","nonce":33351},{"type":"v1","addr":"172.21.15.119:6821","nonce":33351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":33351},{"type":"v1","addr":"172.21.15.119:6817","nonce":33351}]},"public_addr":"172.21.15.119:6809/33351","cluster_addr":"172.21.15.119:6813/33351","heartbeat_back_addr":"172.21.15.119:6821/33351","heartbeat_front_addr":"172.21.15.119:6817/33351","state":["exists","up"]},{"osd":3,"uuid":"59044f30-32e1-4942-9a8e-202c1def221a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":33349},{"type":"v1","addr":"172.21.15.119:6812","nonce":33349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":33349},{"type":"v1","addr":"172.21.15.119:6816","nonce":33349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":33349},{"type":"v1","addr":"172.21.15.119:6823","nonce":33349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":33349},{"type":"v1","addr":"172.21.15.119:6820","nonce":33349}]},"public_addr":"172.21.15.119:6812/33349","cluster_addr":"172.21.15.119:6816/33349","heartbeat_back_addr":"172.21.15.119:6823/33349","heartbeat_front_addr":"172.21.15.119:6820/33349","state":["exists","up"]},{"osd":4,"uuid":"7d1ae3d0-8d69-4d3c-9e51-24fdd77a22a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":33088},{"type":"v1","addr":"172.21.15.148:6809","nonce":33088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":33088},{"type":"v1","addr":"172.21.15.148:6811","nonce":33088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":33088},{"type":"v1","addr":"172.21.15.148:6815","nonce":33088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":33088},{"type":"v1","addr":"172.21.15.148:6813","nonce":33088}]},"public_addr":"172.21.15.148:6809/33088","cluster_addr":"172.21.15.148:6811/33088","heartbeat_back_addr":"172.21.15.148:6815/33088","heartbeat_front_addr":"172.21.15.148:6813/33088","state":["exists","up"]},{"osd":5,"uuid":"b0dd3d30-b273-428c-a4a9-4c238abf45fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":33089},{"type":"v1","addr":"172.21.15.148:6801","nonce":33089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":33089},{"type":"v1","addr":"172.21.15.148:6803","nonce":33089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":33089},{"type":"v1","addr":"172.21.15.148:6807","nonce":33089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":33089},{"type":"v1","addr":"172.21.15.148:6805","nonce":33089}]},"public_addr":"172.21.15.148:6801/33089","cluster_addr":"172.21.15.148:6803/33089","heartbeat_back_addr":"172.21.15.148:6807/33089","heartbeat_front_addr":"172.21.15.148:6805/33089","state":["exists","up"]},{"osd":6,"uuid":"40290cb3-2334-4c68-8e6c-59d0d5455171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":33107},{"type":"v1","addr":"172.21.15.148:6817","nonce":33107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":33107},{"type":"v1","addr":"172.21.15.148:6819","nonce":33107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":33107},{"type":"v1","addr":"172.21.15.148:6823","nonce":33107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":33107},{"type":"v1","addr":"172.21.15.148:6821","nonce":33107}]},"public_addr":"172.21.15.148:6817/33107","cluster_addr":"172.21.15.148:6819/33107","heartbeat_back_addr":"172.21.15.148:6823/33107","heartbeat_front_addr":"172.21.15.148:6821/33107","state":["exists","up"]},{"osd":7,"uuid":"54802abe-ea1a-448e-85af-d1f14a45be27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":33119},{"type":"v1","addr":"172.21.15.148:6825","nonce":33119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":33119},{"type":"v1","addr":"172.21.15.148:6827","nonce":33119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":33119},{"type":"v1","addr":"172.21.15.148:6831","nonce":33119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":33119},{"type":"v1","addr":"172.21.15.148:6829","nonce":33119}]},"public_addr":"172.21.15.148:6825/33119","cluster_addr":"172.21.15.148:6827/33119","heartbeat_back_addr":"172.21.15.148:6831/33119","heartbeat_front_addr":"172.21.15.148:6829/33119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:34:44.910546+0000","dead_epoch":206},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:34:41.761116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:35:12.657573+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:34:39.587239+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:35:00.703150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:35:02.069253+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:34:42.647805+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T11:34:51.482687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T11:35:15.067 INFO:tasks.ceph:Scrubbing osd.0 2024-09-18T11:35:15.067 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-18T11:35:15.192 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:35:15.192 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T11:35:15.192 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:35:15.202 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-18T11:35:15.468 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 0 to deep-scrub 2024-09-18T11:35:15.480 INFO:tasks.ceph:Scrubbing osd.1 2024-09-18T11:35:15.481 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-18T11:35:15.597 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:35:15.597 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T11:35:15.597 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:35:15.607 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-18T11:35:15.867 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 1 to deep-scrub 2024-09-18T11:35:15.880 INFO:tasks.ceph:Scrubbing osd.2 2024-09-18T11:35:15.880 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-18T11:35:15.996 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:35:15.996 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T11:35:15.996 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:35:16.006 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-18T11:35:16.267 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 2 to deep-scrub 2024-09-18T11:35:16.283 INFO:tasks.ceph:Scrubbing osd.3 2024-09-18T11:35:16.283 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-18T11:35:16.399 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:35:16.399 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T11:35:16.399 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:35:16.409 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-18T11:35:16.667 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 3 to deep-scrub 2024-09-18T11:35:16.679 INFO:tasks.ceph:Scrubbing osd.4 2024-09-18T11:35:16.679 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-18T11:35:16.797 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:35:16.797 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T11:35:16.797 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:35:16.807 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-18T11:35:17.069 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 4 to deep-scrub 2024-09-18T11:35:17.082 INFO:tasks.ceph:Scrubbing osd.5 2024-09-18T11:35:17.082 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-18T11:35:17.199 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:35:17.199 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T11:35:17.199 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:35:17.209 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-18T11:35:17.511 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 5 to deep-scrub 2024-09-18T11:35:17.524 INFO:tasks.ceph:Scrubbing osd.6 2024-09-18T11:35:17.524 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-18T11:35:17.642 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:35:17.642 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T11:35:17.642 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:35:17.652 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-18T11:35:17.912 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 6 to deep-scrub 2024-09-18T11:35:17.924 INFO:tasks.ceph:Scrubbing osd.7 2024-09-18T11:35:17.924 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-18T11:35:18.043 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-18T11:35:18.044 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T11:35:18.044 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-18T11:35:18.054 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-18T11:35:18.312 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 7 to deep-scrub 2024-09-18T11:35:18.324 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:35:18.582 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:35:18.583 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:35:18.594 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-18T11:35:17.984314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":142099530,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":712,"num_read_kb":25479,"num_write":887,"num_write_kb":266346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":432,"num_bytes_recovered":175750418,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":839,"ondisk_log_size":839,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2617260,"kb_used_data":190864,"kb_used_omap":36,"kb_used_meta":2426267,"kb_avail":752357460,"statfs":{"total":773094113280,"available":770414039040,"internally_reserved":0,"allocated":195444736,"data_stored":192337221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37868,"internal_metadata":2484497428},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001642"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":432,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-18T11:35:10.515820+0000","last_change":"2024-09-18T11:35:10.515732+0000","last_active":"2024-09-18T11:35:10.515820+0000","last_peered":"2024-09-18T11:35:10.515820+0000","last_clean":"2024-09-18T11:35:10.515820+0000","last_became_active":"2024-09-18T11:34:50.157413+0000","last_became_peered":"2024-09-18T11:34:50.157413+0000","last_unstale":"2024-09-18T11:35:10.515820+0000","last_undegraded":"2024-09-18T11:35:10.515820+0000","last_fullsized":"2024-09-18T11:35:10.515820+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'72","last_scrub_stamp":"2024-09-18T11:35:10.515634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:47.951437+0000","last_clean_scrub_stamp":"2024-09-18T11:35:10.515634+0000","objects_scrubbed":2,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:30.216617+0000","scrub_duration":0.0021162030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2115712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":284,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:13.575842+0000","last_change":"2024-09-18T11:34:08.163615+0000","last_active":"2024-09-18T11:35:13.575842+0000","last_peered":"2024-09-18T11:35:13.575842+0000","last_clean":"2024-09-18T11:35:13.575842+0000","last_became_active":"2024-09-18T11:32:27.079772+0000","last_became_peered":"2024-09-18T11:32:27.079772+0000","last_unstale":"2024-09-18T11:35:13.575842+0000","last_undegraded":"2024-09-18T11:35:13.575842+0000","last_fullsized":"2024-09-18T11:35:13.575842+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'1","last_scrub_stamp":"2024-09-18T11:34:08.163526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:34:08.163526+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for deep scrub","scrub_duration":0.001981956,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"236'254","reported_seq":574,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:17.445660+0000","last_change":"2024-09-18T11:35:17.445629+0000","last_active":"2024-09-18T11:35:17.445660+0000","last_peered":"2024-09-18T11:35:17.445660+0000","last_clean":"2024-09-18T11:35:17.445660+0000","last_became_active":"2024-09-18T11:30:01.077612+0000","last_became_peered":"2024-09-18T11:30:01.077612+0000","last_unstale":"2024-09-18T11:35:17.445660+0000","last_undegraded":"2024-09-18T11:35:17.445660+0000","last_fullsized":"2024-09-18T11:35:17.445660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'254","last_scrub_stamp":"2024-09-18T11:35:17.445529+0000","last_deep_scrub":"236'254","last_deep_scrub_stamp":"2024-09-18T11:35:17.445529+0000","last_clean_scrub_stamp":"2024-09-18T11:35:17.445529+0000","objects_scrubbed":51,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:34.833116+0000","scrub_duration":0.059874783000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42108006,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":185,"num_read_kb":4788,"num_write":230,"num_write_kb":78760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":269,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-18T11:35:07.705135+0000","last_change":"2024-09-18T11:34:42.697160+0000","last_active":"2024-09-18T11:35:07.705135+0000","last_peered":"2024-09-18T11:35:07.705135+0000","last_clean":"2024-09-18T11:35:07.705135+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:35:07.705135+0000","last_undegraded":"2024-09-18T11:35:07.705135+0000","last_fullsized":"2024-09-18T11:35:07.705135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:34:42.697075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:34:42.697075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:05.947061+0000","scrub_duration":0.001876314,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"173'206","reported_seq":780,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-18T11:35:07.256010+0000","last_change":"2024-09-18T11:34:50.152896+0000","last_active":"2024-09-18T11:35:07.256010+0000","last_peered":"2024-09-18T11:35:07.256010+0000","last_clean":"2024-09-18T11:35:07.256010+0000","last_became_active":"2024-09-18T11:34:50.152112+0000","last_became_peered":"2024-09-18T11:34:50.152112+0000","last_unstale":"2024-09-18T11:35:07.256010+0000","last_undegraded":"2024-09-18T11:35:07.256010+0000","last_fullsized":"2024-09-18T11:35:07.256010+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'206","last_scrub_stamp":"2024-09-18T11:34:01.824422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:34:01.824422+0000","objects_scrubbed":46,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050590330000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43270431,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":179,"num_read_kb":11045,"num_write":225,"num_write_kb":76253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":64589501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":130,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:13.586228+0000","last_change":"2024-09-18T11:34:56.185524+0000","last_active":"2024-09-18T11:35:13.586228+0000","last_peered":"2024-09-18T11:35:13.586228+0000","last_clean":"2024-09-18T11:35:13.586228+0000","last_became_active":"2024-09-18T11:34:56.185242+0000","last_became_peered":"2024-09-18T11:34:56.185242+0000","last_unstale":"2024-09-18T11:35:13.586228+0000","last_undegraded":"2024-09-18T11:35:13.586228+0000","last_fullsized":"2024-09-18T11:35:13.586228+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:34:48.411852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:34:48.411852+0000","last_clean_scrub_stamp":"2024-09-18T11:34:48.411852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for deep scrub","scrub_duration":0.001364355,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"172'146","reported_seq":675,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:13.578177+0000","last_change":"2024-09-18T11:34:35.993294+0000","last_active":"2024-09-18T11:35:13.578177+0000","last_peered":"2024-09-18T11:35:13.578177+0000","last_clean":"2024-09-18T11:35:13.578177+0000","last_became_active":"2024-09-18T11:32:59.476088+0000","last_became_peered":"2024-09-18T11:32:59.476088+0000","last_unstale":"2024-09-18T11:35:13.578177+0000","last_undegraded":"2024-09-18T11:35:13.578177+0000","last_fullsized":"2024-09-18T11:35:13.578177+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"172'146","last_scrub_stamp":"2024-09-18T11:34:35.993193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:34:35.993193+0000","objects_scrubbed":35,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for deep scrub","scrub_duration":0.0051455880000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24142412,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":133,"num_read_kb":1720,"num_write":158,"num_write_kb":46975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":29291946,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"210'3","reported_seq":260,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:13.575868+0000","last_change":"2024-09-18T11:34:25.106232+0000","last_active":"2024-09-18T11:35:13.575868+0000","last_peered":"2024-09-18T11:35:13.575868+0000","last_clean":"2024-09-18T11:35:13.575868+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:35:13.575868+0000","last_undegraded":"2024-09-18T11:35:13.575868+0000","last_fullsized":"2024-09-18T11:35:13.575868+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'3","last_scrub_stamp":"2024-09-18T11:34:25.106157+0000","last_deep_scrub":"210'3","last_deep_scrub_stamp":"2024-09-18T11:34:25.106157+0000","last_clean_scrub_stamp":"2024-09-18T11:34:25.106157+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for deep scrub","scrub_duration":0.0034758250000000001,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"179'157","reported_seq":888,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:13.586254+0000","last_change":"2024-09-18T11:35:13.484645+0000","last_active":"2024-09-18T11:35:13.586254+0000","last_peered":"2024-09-18T11:35:13.586254+0000","last_clean":"2024-09-18T11:35:13.586254+0000","last_became_active":"2024-09-18T11:34:50.152231+0000","last_became_peered":"2024-09-18T11:34:50.152231+0000","last_unstale":"2024-09-18T11:35:13.586254+0000","last_undegraded":"2024-09-18T11:35:13.586254+0000","last_fullsized":"2024-09-18T11:35:13.586254+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"179'157","last_scrub_stamp":"2024-09-18T11:35:13.484555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:30:00.059457+0000","last_clean_scrub_stamp":"2024-09-18T11:35:13.484555+0000","objects_scrubbed":36,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for deep scrub","scrub_duration":0.0047268429999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31988294,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":149,"num_read_kb":7871,"num_write":171,"num_write_kb":62728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":79753259,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":323,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:16.177128+0000","last_change":"2024-09-18T11:35:16.177109+0000","last_active":"2024-09-18T11:35:16.177128+0000","last_peered":"2024-09-18T11:35:16.177128+0000","last_clean":"2024-09-18T11:35:16.177128+0000","last_became_active":"2024-09-18T11:34:47.137971+0000","last_became_peered":"2024-09-18T11:34:47.137971+0000","last_unstale":"2024-09-18T11:35:16.177128+0000","last_undegraded":"2024-09-18T11:35:16.177128+0000","last_fullsized":"2024-09-18T11:35:16.177128+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:35:16.177039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:35:16.177039+0000","last_clean_scrub_stamp":"2024-09-18T11:35:16.177039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:37.797997+0000","scrub_duration":0.001167874,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":302,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-18T11:35:07.705138+0000","last_change":"2024-09-18T11:34:47.136279+0000","last_active":"2024-09-18T11:35:07.705138+0000","last_peered":"2024-09-18T11:35:07.705138+0000","last_clean":"2024-09-18T11:35:07.705138+0000","last_became_active":"2024-09-18T11:34:47.135985+0000","last_became_peered":"2024-09-18T11:34:47.135985+0000","last_unstale":"2024-09-18T11:35:07.705138+0000","last_undegraded":"2024-09-18T11:35:07.705138+0000","last_fullsized":"2024-09-18T11:35:07.705138+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:59.647914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:59.647914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:27.634628+0000","scrub_duration":0.001373326,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:13.586233+0000","last_change":"2024-09-18T11:34:56.673422+0000","last_active":"2024-09-18T11:35:13.586233+0000","last_peered":"2024-09-18T11:35:13.586233+0000","last_clean":"2024-09-18T11:35:13.586233+0000","last_became_active":"2024-09-18T11:34:56.673243+0000","last_became_peered":"2024-09-18T11:34:56.673243+0000","last_unstale":"2024-09-18T11:35:13.586233+0000","last_undegraded":"2024-09-18T11:35:13.586233+0000","last_fullsized":"2024-09-18T11:35:13.586233+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:33:56.614347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:33:56.614347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":266,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:13.586481+0000","last_change":"2024-09-18T11:34:36.653574+0000","last_active":"2024-09-18T11:35:13.586481+0000","last_peered":"2024-09-18T11:35:13.586481+0000","last_clean":"2024-09-18T11:35:13.586481+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:35:13.586481+0000","last_undegraded":"2024-09-18T11:35:13.586481+0000","last_fullsized":"2024-09-18T11:35:13.586481+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:34:36.653477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:29:48.952943+0000","last_clean_scrub_stamp":"2024-09-18T11:34:36.653477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for deep scrub","scrub_duration":0.001348115,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":141509143,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":646,"num_read_kb":25424,"num_write":784,"num_write_kb":264716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":422,"num_bytes_recovered":173634706,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":211963904,"data_stored":211458577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9210,"internal_metadata":0},"log_size":763,"ondisk_log_size":763,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2115712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574917,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302704,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069136,"statfs":{"total":96636764160,"available":96326795264,"internally_reserved":0,"allocated":1216512,"data_stored":883581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.219},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.27500000000000002}]},{"osd":1,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.48499999999999999,"15min":0.47299999999999998},"min":{"1min":0.44800000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.50800000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.71699999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.33300000000000002}]},{"osd":2,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.45900000000000002,"15min":0.434},"min":{"1min":0.38700000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.75800000000000001,"5min":1.085,"15min":1.085},"last":0.315},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.46100000000000002,"15min":0.44600000000000001},"min":{"1min":0.39900000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.42899999999999999}]},{"osd":3,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.47799999999999998,"15min":0.45500000000000002},"min":{"1min":0.42199999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.48899999999999999,"15min":0.49199999999999999},"min":{"1min":0.432,"5min":0.308,"15min":0.308},"max":{"1min":0.66000000000000003,"5min":1.177,"15min":1.177},"last":0.40200000000000002}]},{"osd":4,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.45100000000000001},"min":{"1min":0.30099999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.60999999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.46000000000000002,"15min":0.49199999999999999},"min":{"1min":0.28199999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.62,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.36899999999999999}]},{"osd":5,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.39900000000000002,"15min":0.433},"min":{"1min":0.30199999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68999999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.46100000000000002},"min":{"1min":0.35899999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63200000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54900000000000004}]},{"osd":6,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47599999999999998,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65600000000000003,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.503},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47699999999999998,"15min":0.499},"min":{"1min":0.32500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.629,"5min":0.75,"15min":0.75},"last":0.45100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574917,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302184,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069656,"statfs":{"total":96636764160,"available":96327327744,"internally_reserved":0,"allocated":684032,"data_stored":352969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.379}]},{"osd":1,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.48399999999999999,"15min":0.47499999999999998},"min":{"1min":0.35699999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.72499999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.432},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.47099999999999997,"15min":0.46200000000000002},"min":{"1min":0.25800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.70099999999999996,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.35799999999999998}]},{"osd":2,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.27000000000000002,"5min":0.221,"15min":0.221},"max":{"1min":0.75700000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.443},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.48499999999999999,"15min":0.45700000000000002},"min":{"1min":0.40899999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68100000000000005,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.39800000000000002}]},{"osd":3,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48299999999999998,"15min":0.50800000000000001},"min":{"1min":0.29899999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59099999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.52000000000000002},"min":{"1min":0.31,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.71899999999999997,"5min":1.032,"15min":1.032},"last":0.47299999999999998}]},{"osd":4,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42799999999999999,"15min":0.40200000000000002},"min":{"1min":0.31900000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.70999999999999996,"5min":0.755,"15min":0.755},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.47999999999999998,"15min":0.44},"min":{"1min":0.375,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.749,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.251}]},{"osd":5,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44800000000000001,"15min":0.41999999999999998},"min":{"1min":0.34499999999999997,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67000000000000004,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.46700000000000003,"15min":0.441},"min":{"1min":0.33200000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.28199999999999997}]},{"osd":7,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47999999999999998,"15min":0.46899999999999997},"min":{"1min":0.28999999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.72099999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48999999999999999,"15min":0.47999999999999998},"min":{"1min":0.308,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63400000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.33600000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574918,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332196,"kb_used_data":26520,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94039644,"statfs":{"total":96636764160,"available":96296595456,"internally_reserved":0,"allocated":27156480,"data_stored":26767147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.36399999999999999}]},{"osd":1,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.46700000000000003,"15min":0.44500000000000001},"min":{"1min":0.40200000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.47799999999999998,"15min":0.46000000000000002},"min":{"1min":0.46100000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.65200000000000002,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.52600000000000002}]},{"osd":2,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.503,"15min":0.503},"min":{"1min":0.443,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73999999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.47099999999999997,"15min":0.49299999999999999},"min":{"1min":0.40300000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.63400000000000001,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.55400000000000005}]},{"osd":3,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.47299999999999998,"15min":0.49099999999999999},"min":{"1min":0.38200000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.499,"15min":0.501},"min":{"1min":0.41399999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68899999999999995,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.57399999999999995}]},{"osd":4,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41899999999999998,"15min":0.44},"min":{"1min":0.32100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.68500000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.41599999999999998,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.222,"15min":0.222},"max":{"1min":0.63300000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.503}]},{"osd":6,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41399999999999998,"15min":0.42799999999999999},"min":{"1min":0.32600000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.45300000000000001,"15min":0.48099999999999998},"min":{"1min":0.30599999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.56599999999999995}]},{"osd":7,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.45800000000000002,"15min":0.49399999999999999},"min":{"1min":0.377,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.70599999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.497,"15min":0.51800000000000002},"min":{"1min":0.39700000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.622}]}]},{"osd":4,"up_from":13,"seq":55834574917,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356484,"kb_used_data":50808,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94015356,"statfs":{"total":96636764160,"available":96271724544,"internally_reserved":0,"allocated":52027392,"data_stored":51574419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.5}]},{"osd":1,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.48299999999999998,"15min":0.502},"min":{"1min":0.42599999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76500000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48099999999999998,"15min":0.497},"min":{"1min":0.34399999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.67300000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.36699999999999999}]},{"osd":2,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.497,"15min":0.498},"min":{"1min":0.40000000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73199999999999998,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.438},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.50700000000000001,"15min":0.52100000000000002},"min":{"1min":0.51800000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.80900000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.53800000000000003}]},{"osd":3,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.48699999999999999,"15min":0.51200000000000001},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72999999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.502,"15min":0.52600000000000002},"min":{"1min":0.379,"5min":0.316,"15min":0.316},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.47299999999999998}]},{"osd":5,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41599999999999998,"15min":0.45400000000000001},"min":{"1min":0.27700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.67900000000000005,"5min":1.002,"15min":1.002},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.45800000000000002,"15min":0.5},"min":{"1min":0.35099999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.73099999999999998,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.42899999999999999}]},{"osd":6,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.44,"15min":0.46600000000000003},"min":{"1min":0.33500000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.66900000000000004,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.496},"min":{"1min":0.318,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.70599999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.40100000000000002}]},{"osd":7,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48299999999999998,"15min":0.50700000000000001},"min":{"1min":0.33400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.754,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.52500000000000002,"15min":0.54500000000000004},"min":{"1min":0.41099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83999999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55700000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574919,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332220,"kb_used_data":26544,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94039620,"statfs":{"total":96636764160,"available":96296570880,"internally_reserved":0,"allocated":27181056,"data_stored":26768945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.35399999999999998}]},{"osd":1,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.33900000000000002,"15min":0.34399999999999997},"min":{"1min":0.27500000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.55900000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.375,"15min":0.375},"min":{"1min":0.22500000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.57099999999999995,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.32200000000000001}]},{"osd":2,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.38900000000000001,"15min":0.38500000000000001},"min":{"1min":0.32400000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.53000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.375},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.374,"15min":0.39800000000000002},"min":{"1min":0.23799999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.46600000000000003,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.31}]},{"osd":4,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.55100000000000005,"15min":0.52700000000000002},"min":{"1min":0.38600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79900000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56899999999999995,"15min":0.54000000000000004},"min":{"1min":0.47499999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78700000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.44600000000000001}]},{"osd":5,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.51300000000000001},"min":{"1min":0.46100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77000000000000002,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.438},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.56000000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.755,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.39900000000000002}]},{"osd":6,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52400000000000002,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81100000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.47699999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.74299999999999999,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.39000000000000001}]},{"osd":7,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57599999999999996,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77900000000000003,"5min":1.26,"15min":1.26},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55500000000000005,"15min":0.57499999999999996},"min":{"1min":0.44500000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.749,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.40699999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574917,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318420,"kb_used_data":16904,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94053420,"statfs":{"total":96636764160,"available":96310702080,"internally_reserved":0,"allocated":17309696,"data_stored":16944316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.26500000000000001}]},{"osd":1,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34899999999999998,"15min":0.373},"min":{"1min":0.20599999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.42499999999999999,"5min":1.621,"15min":1.621},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.36299999999999999,"15min":0.39400000000000002},"min":{"1min":0.251,"5min":0.218,"15min":0.218},"max":{"1min":0.63600000000000001,"5min":1.589,"15min":1.589},"last":0.247}]},{"osd":3,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.35499999999999998,"15min":0.36699999999999999},"min":{"1min":0.28299999999999997,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.42099999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.38200000000000001,"15min":0.41099999999999998},"min":{"1min":0.24099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.56000000000000005,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.33100000000000002}]},{"osd":4,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53500000000000003,"15min":0.55200000000000005},"min":{"1min":0.42899999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70399999999999996,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.52500000000000002},"min":{"1min":0.46400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80000000000000004,"5min":1.012,"15min":1.012},"last":0.45100000000000001}]},{"osd":5,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79000000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.48499999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.749,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.59199999999999997}]},{"osd":6,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.45600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76300000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.56499999999999995},"min":{"1min":0.48699999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79800000000000004,"5min":1.165,"15min":1.165},"last":0.49099999999999999}]},{"osd":7,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.46000000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82899999999999996,"5min":1.073,"15min":1.073},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.45500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82299999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.55000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322576,"kb_used_data":16904,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94049264,"statfs":{"total":96636764160,"available":96306446336,"internally_reserved":0,"allocated":17309696,"data_stored":16943623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.34699999999999998}]},{"osd":2,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.35599999999999998,"15min":0.378},"min":{"1min":0.255,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.50700000000000001,"5min":1.333,"15min":1.333},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38600000000000001,"15min":0.41599999999999998},"min":{"1min":0.28599999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.65700000000000003,"5min":1.298,"15min":1.298},"last":0.41299999999999998}]},{"osd":3,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36399999999999999,"15min":0.38},"min":{"1min":0.24199999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.60499999999999998,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.36099999999999999,"15min":0.36399999999999999},"min":{"1min":0.27100000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55500000000000005,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.26800000000000002}]},{"osd":4,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.496,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.876,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.308},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.37,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.74299999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.33200000000000002}]},{"osd":5,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40600000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78100000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.52900000000000003,"15min":0.53000000000000003},"min":{"1min":0.48699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.85299999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.44}]},{"osd":6,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.44,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82899999999999996,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81599999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.37}]},{"osd":7,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41799999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85499999999999998,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56200000000000006,"15min":0.57799999999999996},"min":{"1min":0.41999999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83499999999999996,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.39500000000000002}]}]},{"osd":0,"up_from":215,"seq":923417968649,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":350476,"kb_used_data":51328,"kb_used_omap":25,"kb_used_meta":299110,"kb_avail":94021364,"statfs":{"total":96636764160,"available":96277876736,"internally_reserved":0,"allocated":52559872,"data_stored":52102221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25940,"internal_metadata":306289324},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9210,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23064576,"data_stored":23013811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:35:18.599 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-18T11:35:10.515634+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=35, tm_sec=10, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=35, tm_sec=14, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T11:35:18.599 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-18T11:34:08.163526+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=34, tm_sec=8, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=35, tm_sec=14, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T11:35:18.600 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-18T11:34:42.697075+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=34, tm_sec=42, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=35, tm_sec=14, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T11:35:18.600 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-18T11:34:01.824422+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=34, tm_sec=1, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=35, tm_sec=14, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T11:35:18.600 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-18T11:34:48.411852+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=34, tm_sec=48, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=35, tm_sec=14, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T11:35:18.600 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-18T11:34:35.993193+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=34, tm_sec=35, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=35, tm_sec=14, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T11:35:18.600 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-18T11:34:25.106157+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=34, tm_sec=25, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=35, tm_sec=14, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T11:35:18.600 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-18T11:35:13.484555+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=35, tm_sec=13, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=35, tm_sec=14, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T11:35:18.600 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-18T11:33:59.647914+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=33, tm_sec=59, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=35, tm_sec=14, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T11:35:18.600 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-18T11:33:56.614347+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=33, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=35, tm_sec=14, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T11:35:18.601 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-18T11:34:36.653477+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=34, tm_sec=36, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=11, tm_min=35, tm_sec=14, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T11:35:18.601 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-18T11:35:38.602 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T11:35:38.874 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-18T11:35:38.875 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-09-18T11:35:38.887 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-18T11:35:37.988441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":142099530,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":712,"num_read_kb":25479,"num_write":887,"num_write_kb":266346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":432,"num_bytes_recovered":175750418,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":959,"ondisk_log_size":959,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2617232,"kb_used_data":190864,"kb_used_omap":36,"kb_used_meta":2426267,"kb_avail":752357488,"statfs":{"total":773094113280,"available":770414067712,"internally_reserved":0,"allocated":195444736,"data_stored":192337221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37868,"internal_metadata":2484497428},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002491"},"pg_stats":[{"pgid":"1.0","version":"237'74","reported_seq":445,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:28.707788+0000","last_change":"2024-09-18T11:35:28.707761+0000","last_active":"2024-09-18T11:35:28.707788+0000","last_peered":"2024-09-18T11:35:28.707788+0000","last_clean":"2024-09-18T11:35:28.707788+0000","last_became_active":"2024-09-18T11:34:50.157413+0000","last_became_peered":"2024-09-18T11:34:50.157413+0000","last_unstale":"2024-09-18T11:35:28.707788+0000","last_undegraded":"2024-09-18T11:35:28.707788+0000","last_fullsized":"2024-09-18T11:35:28.707788+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'74","last_scrub_stamp":"2024-09-18T11:35:28.707665+0000","last_deep_scrub":"237'74","last_deep_scrub_stamp":"2024-09-18T11:35:28.707665+0000","last_clean_scrub_stamp":"2024-09-18T11:35:28.707665+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:44.157986+0000","scrub_duration":0.0052967359999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2115712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"237'2","reported_seq":295,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:20.077266+0000","last_change":"2024-09-18T11:35:20.077236+0000","last_active":"2024-09-18T11:35:20.077266+0000","last_peered":"2024-09-18T11:35:20.077266+0000","last_clean":"2024-09-18T11:35:20.077266+0000","last_became_active":"2024-09-18T11:32:27.079772+0000","last_became_peered":"2024-09-18T11:32:27.079772+0000","last_unstale":"2024-09-18T11:35:20.077266+0000","last_undegraded":"2024-09-18T11:35:20.077266+0000","last_fullsized":"2024-09-18T11:35:20.077266+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'2","last_scrub_stamp":"2024-09-18T11:35:20.077151+0000","last_deep_scrub":"237'2","last_deep_scrub_stamp":"2024-09-18T11:35:20.077151+0000","last_clean_scrub_stamp":"2024-09-18T11:35:20.077151+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:49.428461+0000","scrub_duration":0.0030963639999999999,"objects_trimmed":0,"snaptrim_duration":0.00029570200000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"236'254","reported_seq":574,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:17.445660+0000","last_change":"2024-09-18T11:35:17.445629+0000","last_active":"2024-09-18T11:35:17.445660+0000","last_peered":"2024-09-18T11:35:17.445660+0000","last_clean":"2024-09-18T11:35:17.445660+0000","last_became_active":"2024-09-18T11:30:01.077612+0000","last_became_peered":"2024-09-18T11:30:01.077612+0000","last_unstale":"2024-09-18T11:35:17.445660+0000","last_undegraded":"2024-09-18T11:35:17.445660+0000","last_fullsized":"2024-09-18T11:35:17.445660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'254","last_scrub_stamp":"2024-09-18T11:35:17.445529+0000","last_deep_scrub":"236'254","last_deep_scrub_stamp":"2024-09-18T11:35:17.445529+0000","last_clean_scrub_stamp":"2024-09-18T11:35:17.445529+0000","objects_scrubbed":51,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:34.833116+0000","scrub_duration":0.059874783000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42108006,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":185,"num_read_kb":4788,"num_write":230,"num_write_kb":78760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":280,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:30.624062+0000","last_change":"2024-09-18T11:35:30.624035+0000","last_active":"2024-09-18T11:35:30.624062+0000","last_peered":"2024-09-18T11:35:30.624062+0000","last_clean":"2024-09-18T11:35:30.624062+0000","last_became_active":"2024-09-18T11:29:50.029199+0000","last_became_peered":"2024-09-18T11:29:50.029199+0000","last_unstale":"2024-09-18T11:35:30.624062+0000","last_undegraded":"2024-09-18T11:35:30.624062+0000","last_fullsized":"2024-09-18T11:35:30.624062+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:35:30.623947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:35:30.623947+0000","last_clean_scrub_stamp":"2024-09-18T11:35:30.623947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:32.517392+0000","scrub_duration":0.0024234930000000001,"objects_trimmed":0,"snaptrim_duration":0.00032710200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"237'252","reported_seq":849,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:17.511315+0000","last_change":"2024-09-18T11:35:17.511270+0000","last_active":"2024-09-18T11:35:17.511315+0000","last_peered":"2024-09-18T11:35:17.511315+0000","last_clean":"2024-09-18T11:35:17.511315+0000","last_became_active":"2024-09-18T11:34:50.152112+0000","last_became_peered":"2024-09-18T11:34:50.152112+0000","last_unstale":"2024-09-18T11:35:17.511315+0000","last_undegraded":"2024-09-18T11:35:17.511315+0000","last_fullsized":"2024-09-18T11:35:17.511315+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'252","last_scrub_stamp":"2024-09-18T11:35:17.511202+0000","last_deep_scrub":"237'252","last_deep_scrub_stamp":"2024-09-18T11:35:17.511202+0000","last_clean_scrub_stamp":"2024-09-18T11:35:17.511202+0000","objects_scrubbed":46,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:40.303579+0000","scrub_duration":0.076404630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43270431,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":179,"num_read_kb":11045,"num_write":225,"num_write_kb":76253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":64589501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":140,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:18.742021+0000","last_change":"2024-09-18T11:35:18.741989+0000","last_active":"2024-09-18T11:35:18.742021+0000","last_peered":"2024-09-18T11:35:18.742021+0000","last_clean":"2024-09-18T11:35:18.742021+0000","last_became_active":"2024-09-18T11:34:56.185242+0000","last_became_peered":"2024-09-18T11:34:56.185242+0000","last_unstale":"2024-09-18T11:35:18.742021+0000","last_undegraded":"2024-09-18T11:35:18.742021+0000","last_fullsized":"2024-09-18T11:35:18.742021+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:35:18.741910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:35:18.741910+0000","last_clean_scrub_stamp":"2024-09-18T11:35:18.741910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:36.894301+0000","scrub_duration":0.0011553990000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"237'181","reported_seq":721,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:19.180441+0000","last_change":"2024-09-18T11:35:19.180408+0000","last_active":"2024-09-18T11:35:19.180441+0000","last_peered":"2024-09-18T11:35:19.180441+0000","last_clean":"2024-09-18T11:35:19.180441+0000","last_became_active":"2024-09-18T11:32:59.476088+0000","last_became_peered":"2024-09-18T11:32:59.476088+0000","last_unstale":"2024-09-18T11:35:19.180441+0000","last_undegraded":"2024-09-18T11:35:19.180441+0000","last_fullsized":"2024-09-18T11:35:19.180441+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'181","last_scrub_stamp":"2024-09-18T11:35:19.180300+0000","last_deep_scrub":"237'181","last_deep_scrub_stamp":"2024-09-18T11:35:19.180300+0000","last_clean_scrub_stamp":"2024-09-18T11:35:19.180300+0000","objects_scrubbed":35,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:44.339887+0000","scrub_duration":0.057544008000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24142412,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":133,"num_read_kb":1720,"num_write":158,"num_write_kb":46975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":29291946,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"210'3","reported_seq":270,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:17.999507+0000","last_change":"2024-09-18T11:35:17.999483+0000","last_active":"2024-09-18T11:35:17.999507+0000","last_peered":"2024-09-18T11:35:17.999507+0000","last_clean":"2024-09-18T11:35:17.999507+0000","last_became_active":"2024-09-18T11:29:50.028867+0000","last_became_peered":"2024-09-18T11:29:50.028867+0000","last_unstale":"2024-09-18T11:35:17.999507+0000","last_undegraded":"2024-09-18T11:35:17.999507+0000","last_fullsized":"2024-09-18T11:35:17.999507+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'3","last_scrub_stamp":"2024-09-18T11:35:17.999404+0000","last_deep_scrub":"210'3","last_deep_scrub_stamp":"2024-09-18T11:35:17.999404+0000","last_clean_scrub_stamp":"2024-09-18T11:35:17.999404+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:31.920308+0000","scrub_duration":0.0018571799999999999,"objects_trimmed":0,"snaptrim_duration":0.00086144300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"237'193","reported_seq":935,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:19.433994+0000","last_change":"2024-09-18T11:35:19.433970+0000","last_active":"2024-09-18T11:35:19.433994+0000","last_peered":"2024-09-18T11:35:19.433994+0000","last_clean":"2024-09-18T11:35:19.433994+0000","last_became_active":"2024-09-18T11:34:50.152231+0000","last_became_peered":"2024-09-18T11:34:50.152231+0000","last_unstale":"2024-09-18T11:35:19.433994+0000","last_undegraded":"2024-09-18T11:35:19.433994+0000","last_fullsized":"2024-09-18T11:35:19.433994+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'193","last_scrub_stamp":"2024-09-18T11:35:19.433897+0000","last_deep_scrub":"237'193","last_deep_scrub_stamp":"2024-09-18T11:35:19.433897+0000","last_clean_scrub_stamp":"2024-09-18T11:35:19.433897+0000","objects_scrubbed":36,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:39.937123+0000","scrub_duration":0.058071631999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31988294,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":149,"num_read_kb":7871,"num_write":171,"num_write_kb":62728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":79753259,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":323,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:16.177128+0000","last_change":"2024-09-18T11:35:16.177109+0000","last_active":"2024-09-18T11:35:16.177128+0000","last_peered":"2024-09-18T11:35:16.177128+0000","last_clean":"2024-09-18T11:35:16.177128+0000","last_became_active":"2024-09-18T11:34:47.137971+0000","last_became_peered":"2024-09-18T11:34:47.137971+0000","last_unstale":"2024-09-18T11:35:16.177128+0000","last_undegraded":"2024-09-18T11:35:16.177128+0000","last_fullsized":"2024-09-18T11:35:16.177128+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:35:16.177039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:35:16.177039+0000","last_clean_scrub_stamp":"2024-09-18T11:35:16.177039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:37.797997+0000","scrub_duration":0.001167874,"objects_trimmed":0,"snaptrim_duration":0.000325373,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":313,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:33.521656+0000","last_change":"2024-09-18T11:35:33.521602+0000","last_active":"2024-09-18T11:35:33.521656+0000","last_peered":"2024-09-18T11:35:33.521656+0000","last_clean":"2024-09-18T11:35:33.521656+0000","last_became_active":"2024-09-18T11:34:47.135985+0000","last_became_peered":"2024-09-18T11:34:47.135985+0000","last_unstale":"2024-09-18T11:35:33.521656+0000","last_undegraded":"2024-09-18T11:35:33.521656+0000","last_fullsized":"2024-09-18T11:35:33.521656+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:35:33.521506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:35:33.521506+0000","last_clean_scrub_stamp":"2024-09-18T11:35:33.521506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:33.787233+0000","scrub_duration":0.0017849540000000001,"objects_trimmed":0,"snaptrim_duration":0.000434428,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:26.686383+0000","last_change":"2024-09-18T11:35:26.686351+0000","last_active":"2024-09-18T11:35:26.686383+0000","last_peered":"2024-09-18T11:35:26.686383+0000","last_clean":"2024-09-18T11:35:26.686383+0000","last_became_active":"2024-09-18T11:34:56.673243+0000","last_became_peered":"2024-09-18T11:34:56.673243+0000","last_unstale":"2024-09-18T11:35:26.686383+0000","last_undegraded":"2024-09-18T11:35:26.686383+0000","last_fullsized":"2024-09-18T11:35:26.686383+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:35:26.686259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:35:26.686259+0000","last_clean_scrub_stamp":"2024-09-18T11:35:26.686259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:52.750606+0000","scrub_duration":0.0019893089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":276,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-18T11:35:18.573706+0000","last_change":"2024-09-18T11:35:18.573672+0000","last_active":"2024-09-18T11:35:18.573706+0000","last_peered":"2024-09-18T11:35:18.573706+0000","last_clean":"2024-09-18T11:35:18.573706+0000","last_became_active":"2024-09-18T11:29:50.034100+0000","last_became_peered":"2024-09-18T11:29:50.034100+0000","last_unstale":"2024-09-18T11:35:18.573706+0000","last_undegraded":"2024-09-18T11:35:18.573706+0000","last_fullsized":"2024-09-18T11:35:18.573706+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T11:35:18.573577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T11:35:18.573577+0000","last_clean_scrub_stamp":"2024-09-18T11:35:18.573577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:28.976999+0000","scrub_duration":0.0014022469999999999,"objects_trimmed":0,"snaptrim_duration":0.00030577000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":141509143,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":646,"num_read_kb":25424,"num_write":784,"num_write_kb":264716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":422,"num_bytes_recovered":173634706,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":211963904,"data_stored":211458577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9210,"internal_metadata":0},"log_size":880,"ondisk_log_size":880,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2115712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574921,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302700,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069140,"statfs":{"total":96636764160,"available":96326799360,"internally_reserved":0,"allocated":1216512,"data_stored":883581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.70999999999999996}]},{"osd":1,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.48499999999999999,"15min":0.47299999999999998},"min":{"1min":0.44800000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.621},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.50800000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.71699999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.65700000000000003}]},{"osd":2,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.45900000000000002,"15min":0.434},"min":{"1min":0.38700000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.75800000000000001,"5min":1.085,"15min":1.085},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.46100000000000002,"15min":0.44600000000000001},"min":{"1min":0.39900000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.68899999999999995}]},{"osd":3,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.47799999999999998,"15min":0.45500000000000002},"min":{"1min":0.42199999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.48899999999999999,"15min":0.49199999999999999},"min":{"1min":0.432,"5min":0.308,"15min":0.308},"max":{"1min":0.66000000000000003,"5min":1.177,"15min":1.177},"last":0.502}]},{"osd":4,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.45100000000000001},"min":{"1min":0.30099999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.60999999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.46000000000000002,"15min":0.49199999999999999},"min":{"1min":0.28199999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.62,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.42099999999999999}]},{"osd":5,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.39900000000000002,"15min":0.433},"min":{"1min":0.30199999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68999999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.46100000000000002},"min":{"1min":0.35899999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63200000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.44500000000000001}]},{"osd":6,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47599999999999998,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65600000000000003,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47699999999999998,"15min":0.499},"min":{"1min":0.32500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.629,"5min":0.75,"15min":0.75},"last":0.46800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302180,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069660,"statfs":{"total":96636764160,"available":96327331840,"internally_reserved":0,"allocated":684032,"data_stored":352969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.624},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.67500000000000004}]},{"osd":1,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.48399999999999999,"15min":0.47499999999999998},"min":{"1min":0.35699999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.72499999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.47099999999999997,"15min":0.46200000000000002},"min":{"1min":0.25800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.70099999999999996,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.68600000000000005}]},{"osd":2,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.27000000000000002,"5min":0.221,"15min":0.221},"max":{"1min":0.75700000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.745},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.48499999999999999,"15min":0.45700000000000002},"min":{"1min":0.40899999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68100000000000005,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.755}]},{"osd":3,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48299999999999998,"15min":0.50800000000000001},"min":{"1min":0.29899999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59099999999999997,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.52000000000000002},"min":{"1min":0.31,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.71899999999999997,"5min":1.032,"15min":1.032},"last":0.77200000000000002}]},{"osd":4,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42799999999999999,"15min":0.40200000000000002},"min":{"1min":0.31900000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.70999999999999996,"5min":0.755,"15min":0.755},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.47999999999999998,"15min":0.44},"min":{"1min":0.375,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.749,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.70899999999999996}]},{"osd":5,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44800000000000001,"15min":0.41999999999999998},"min":{"1min":0.34499999999999997,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67000000000000004,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.46700000000000003,"15min":0.441},"min":{"1min":0.33200000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.53600000000000003}]},{"osd":7,"last update":"Wed Sep 18 11:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47999999999999998,"15min":0.46899999999999997},"min":{"1min":0.28999999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.72099999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48999999999999999,"15min":0.47999999999999998},"min":{"1min":0.308,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63400000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.66600000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332192,"kb_used_data":26520,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94039648,"statfs":{"total":96636764160,"available":96296599552,"internally_reserved":0,"allocated":27156480,"data_stored":26767147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.65400000000000003}]},{"osd":1,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.46700000000000003,"15min":0.44500000000000001},"min":{"1min":0.40200000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.47799999999999998,"15min":0.46000000000000002},"min":{"1min":0.46100000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.65200000000000002,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.60799999999999998}]},{"osd":2,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.503,"15min":0.503},"min":{"1min":0.443,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73999999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.47099999999999997,"15min":0.49299999999999999},"min":{"1min":0.40300000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.63400000000000001,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.53600000000000003}]},{"osd":3,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.47299999999999998,"15min":0.49099999999999999},"min":{"1min":0.38200000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.499,"15min":0.501},"min":{"1min":0.41399999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68899999999999995,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.59799999999999998}]},{"osd":4,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41899999999999998,"15min":0.44},"min":{"1min":0.32100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.68500000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.41599999999999998,"15min":0.45000000000000001},"min":{"1min":0.315,"5min":0.222,"15min":0.222},"max":{"1min":0.63300000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.433}]},{"osd":6,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41399999999999998,"15min":0.42799999999999999},"min":{"1min":0.32600000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.45300000000000001,"15min":0.48099999999999998},"min":{"1min":0.30599999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.502}]},{"osd":7,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.45800000000000002,"15min":0.49399999999999999},"min":{"1min":0.377,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.70599999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.497,"15min":0.51800000000000002},"min":{"1min":0.39700000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.52000000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574921,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356480,"kb_used_data":50808,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94015360,"statfs":{"total":96636764160,"available":96271728640,"internally_reserved":0,"allocated":52027392,"data_stored":51574419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.67700000000000005}]},{"osd":1,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.48299999999999998,"15min":0.502},"min":{"1min":0.42599999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76500000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48099999999999998,"15min":0.497},"min":{"1min":0.34399999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.67300000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.441}]},{"osd":2,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.497,"15min":0.498},"min":{"1min":0.40000000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73199999999999998,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.50700000000000001,"15min":0.52100000000000002},"min":{"1min":0.51800000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.80900000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.63600000000000001}]},{"osd":3,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.48699999999999999,"15min":0.51200000000000001},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72999999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.502,"15min":0.52600000000000002},"min":{"1min":0.379,"5min":0.316,"15min":0.316},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.69599999999999995}]},{"osd":5,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41599999999999998,"15min":0.45400000000000001},"min":{"1min":0.27700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.67900000000000005,"5min":1.002,"15min":1.002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.45800000000000002,"15min":0.5},"min":{"1min":0.35099999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.73099999999999998,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.54200000000000004}]},{"osd":6,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.44,"15min":0.46600000000000003},"min":{"1min":0.33500000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.66900000000000004,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.496},"min":{"1min":0.318,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.70599999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.496}]},{"osd":7,"last update":"Wed Sep 18 11:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48299999999999998,"15min":0.50700000000000001},"min":{"1min":0.33400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.754,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.52500000000000002,"15min":0.54500000000000004},"min":{"1min":0.41099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83999999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.65700000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332216,"kb_used_data":26544,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94039624,"statfs":{"total":96636764160,"available":96296574976,"internally_reserved":0,"allocated":27181056,"data_stored":26768945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.374}]},{"osd":1,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.33900000000000002,"15min":0.34399999999999997},"min":{"1min":0.27500000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.55900000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.375,"15min":0.375},"min":{"1min":0.22500000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.57099999999999995,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.43099999999999999}]},{"osd":2,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.38900000000000001,"15min":0.38500000000000001},"min":{"1min":0.32400000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.53000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.374,"15min":0.39800000000000002},"min":{"1min":0.23799999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.46600000000000003,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.33200000000000002}]},{"osd":4,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.55100000000000005,"15min":0.52700000000000002},"min":{"1min":0.38600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79900000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56899999999999995,"15min":0.54000000000000004},"min":{"1min":0.47499999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78700000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.73399999999999999}]},{"osd":5,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.51300000000000001},"min":{"1min":0.46100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77000000000000002,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.56000000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.755,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.70699999999999996}]},{"osd":6,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52400000000000002,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81100000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.47699999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.74299999999999999,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.56100000000000005}]},{"osd":7,"last update":"Wed Sep 18 11:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57599999999999996,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77900000000000003,"5min":1.26,"15min":1.26},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55500000000000005,"15min":0.57499999999999996},"min":{"1min":0.44500000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.749,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.57399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574921,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318416,"kb_used_data":16904,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94053424,"statfs":{"total":96636764160,"available":96310706176,"internally_reserved":0,"allocated":17309696,"data_stored":16944316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.372},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.38600000000000001}]},{"osd":1,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34899999999999998,"15min":0.373},"min":{"1min":0.20599999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.42499999999999999,"5min":1.621,"15min":1.621},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.36299999999999999,"15min":0.39400000000000002},"min":{"1min":0.251,"5min":0.218,"15min":0.218},"max":{"1min":0.63600000000000001,"5min":1.589,"15min":1.589},"last":0.41399999999999998}]},{"osd":3,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.35499999999999998,"15min":0.36699999999999999},"min":{"1min":0.28299999999999997,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.42099999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.38200000000000001,"15min":0.41099999999999998},"min":{"1min":0.24099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.56000000000000005,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.443}]},{"osd":4,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53500000000000003,"15min":0.55200000000000005},"min":{"1min":0.42899999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70399999999999996,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.52500000000000002},"min":{"1min":0.46400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80000000000000004,"5min":1.012,"15min":1.012},"last":0.64600000000000002}]},{"osd":5,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79000000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.48499999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.749,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.70699999999999996}]},{"osd":6,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.45600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76300000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.745},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.56499999999999995},"min":{"1min":0.48699999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79800000000000004,"5min":1.165,"15min":1.165},"last":0.55000000000000004}]},{"osd":7,"last update":"Wed Sep 18 11:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.46000000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82899999999999996,"5min":1.073,"15min":1.073},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.45500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82299999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.69199999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322576,"kb_used_data":16904,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94049264,"statfs":{"total":96636764160,"available":96306446336,"internally_reserved":0,"allocated":17309696,"data_stored":16943623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 11:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.35199999999999998}]},{"osd":2,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.35599999999999998,"15min":0.378},"min":{"1min":0.255,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.50700000000000001,"5min":1.333,"15min":1.333},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38600000000000001,"15min":0.41599999999999998},"min":{"1min":0.28599999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.65700000000000003,"5min":1.298,"15min":1.298},"last":0.435}]},{"osd":3,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36399999999999999,"15min":0.38},"min":{"1min":0.24199999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.60499999999999998,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.36099999999999999,"15min":0.36399999999999999},"min":{"1min":0.27100000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55500000000000005,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.36899999999999999}]},{"osd":4,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.496,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.876,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.37,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.74299999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.44700000000000001}]},{"osd":5,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40600000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78100000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.52900000000000003,"15min":0.53000000000000003},"min":{"1min":0.48699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.85299999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.56699999999999995}]},{"osd":6,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.44,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82899999999999996,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81599999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57099999999999995}]},{"osd":7,"last update":"Wed Sep 18 11:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41799999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85499999999999998,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56200000000000006,"15min":0.57799999999999996},"min":{"1min":0.41999999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83499999999999996,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.46200000000000002}]}]},{"osd":0,"up_from":215,"seq":923417968653,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":350472,"kb_used_data":51328,"kb_used_omap":25,"kb_used_meta":299110,"kb_avail":94021368,"statfs":{"total":96636764160,"available":96277880832,"internally_reserved":0,"allocated":52559872,"data_stored":52102221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25940,"internal_metadata":306289324},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9210,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16625664,"data_stored":16589525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":51335168,"data_stored":51219188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":26488832,"data_stored":26413670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23064576,"data_stored":23013811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T11:35:38.889 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-18T11:35:39.168 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-18T11:35:39.168 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-18T11:35:39.168 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-18T11:35:39.169 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T11:35:39.207 INFO:tasks.ceph.osd.0:Stopped 2024-09-18T11:35:39.207 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-18T11:35:39.207 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T11:35:39.283 INFO:tasks.ceph.osd.1:Stopped 2024-09-18T11:35:39.283 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-18T11:35:39.283 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T11:35:39.351 INFO:tasks.ceph.osd.2:Stopped 2024-09-18T11:35:39.351 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-18T11:35:39.351 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T11:35:39.419 INFO:tasks.ceph.osd.3:Stopped 2024-09-18T11:35:39.419 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-18T11:35:39.419 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T11:35:39.453 INFO:tasks.ceph.osd.4:Stopped 2024-09-18T11:35:39.453 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-18T11:35:39.453 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T11:35:39.526 INFO:tasks.ceph.osd.5:Stopped 2024-09-18T11:35:39.527 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-18T11:35:39.527 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T11:35:39.599 INFO:tasks.ceph.osd.6:Stopped 2024-09-18T11:35:39.600 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-18T11:35:39.600 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T11:35:39.670 INFO:tasks.ceph.osd.7:Stopped 2024-09-18T11:35:39.670 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-18T11:35:39.671 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-18T11:35:39.671 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T11:35:39.695 INFO:tasks.ceph.mgr.y:Stopped 2024-09-18T11:35:39.695 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-18T11:35:39.695 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T11:35:39.736 INFO:tasks.ceph.mgr.x:Stopped 2024-09-18T11:35:39.736 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-18T11:35:39.737 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-18T11:35:39.737 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T11:35:39.748 INFO:tasks.ceph.mon.a:Stopped 2024-09-18T11:35:39.749 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-18T11:35:39.749 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T11:35:39.801 INFO:tasks.ceph.mon.c:Stopped 2024-09-18T11:35:39.801 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-18T11:35:39.801 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T11:35:39.814 INFO:tasks.ceph.mon.b:Stopped 2024-09-18T11:35:39.814 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-18T11:35:39.814 DEBUG:teuthology.orchestra.run.smithi119:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(REQUEST_STUCK\)' | egrep -v 'failed to encode map' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v osd_map_cache_size | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2024-09-18T11:35:39.873 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-18T11:35:39.873 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-18T11:35:40.133 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-18T11:35:40.133 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-18T11:35:40.226 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-18T11:35:40.226 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-18T11:35:40.306 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi119.front.sepia.ceph.com 2024-09-18T11:35:40.306 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-18T11:35:40.363 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi148.front.sepia.ceph.com 2024-09-18T11:35:40.363 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-18T11:35:40.480 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi148.front.sepia.ceph.com 2024-09-18T11:35:40.480 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-18T11:35:40.569 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi148.front.sepia.ceph.com 2024-09-18T11:35:40.569 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-18T11:35:40.625 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi148.front.sepia.ceph.com 2024-09-18T11:35:40.625 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-18T11:35:40.692 INFO:tasks.ceph:Archiving mon data... 2024-09-18T11:35:40.693 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910958/data/mon.a.tgz 2024-09-18T11:35:40.693 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2024-09-18T11:35:40.709 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.gVN0e4O9xd 2024-09-18T11:35:40.709 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.gVN0e4O9xd 2024-09-18T11:35:40.909 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.gVN0e4O9xd 2024-09-18T11:35:40.994 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.gVN0e4O9xd is 563KB 2024-09-18T11:35:41.092 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.gVN0e4O9xd 2024-09-18T11:35:41.106 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910958/data/mon.c.tgz 2024-09-18T11:35:41.107 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2024-09-18T11:35:41.161 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.spOYmERGWe 2024-09-18T11:35:41.161 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.spOYmERGWe 2024-09-18T11:35:41.329 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.spOYmERGWe 2024-09-18T11:35:41.412 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.spOYmERGWe is 564KB 2024-09-18T11:35:41.510 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.spOYmERGWe 2024-09-18T11:35:41.525 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910958/data/mon.b.tgz 2024-09-18T11:35:41.525 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2024-09-18T11:35:41.544 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.Eu8NteuuOA 2024-09-18T11:35:41.544 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Eu8NteuuOA 2024-09-18T11:35:41.735 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.Eu8NteuuOA 2024-09-18T11:35:41.819 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.Eu8NteuuOA is 564KB 2024-09-18T11:35:41.891 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.Eu8NteuuOA 2024-09-18T11:35:41.906 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-18T11:35:41.906 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T11:35:41.908 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T11:35:41.970 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-18T11:35:41.970 DEBUG:teuthology.orchestra.run.smithi119:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-18T11:35:41.973 DEBUG:teuthology.orchestra.run.smithi148:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-18T11:35:41.997 INFO:teuthology.orchestra.run.smithi119.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-18T11:35:42.037 INFO:teuthology.orchestra.run.smithi148.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-18T11:35:42.041 INFO:tasks.ceph:Archiving crash dumps... 2024-09-18T11:35:42.042 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910958/remote/smithi119/crash 2024-09-18T11:35:42.043 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T11:35:42.067 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910958/remote/smithi148/crash 2024-09-18T11:35:42.068 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T11:35:42.105 INFO:tasks.ceph:Compressing logs... 2024-09-18T11:35:42.105 DEBUG:teuthology.orchestra.run.smithi119:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-18T11:35:42.111 DEBUG:teuthology.orchestra.run.smithi148:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-18T11:35:42.188 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32778.log 2024-09-18T11:35:42.188 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-18T11:35:42.188 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.32778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32858.log 2024-09-18T11:35:42.188 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32778.log.gz 2024-09-18T11:35:42.188 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32989.log 2024-09-18T11:35:42.189 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.32858.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-18T11:35:42.189 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32858.log.gz 2024-09-18T11:35:42.189 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.32989.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-18T11:35:42.189 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32989.log.gz 2024-09-18T11:35:42.189 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32698.log 2024-09-18T11:35:42.189 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32818.log 2024-09-18T11:35:42.189 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.32698.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32698.log.gz 2024-09-18T11:35:42.189 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-18T11:35:42.189 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-18T11:35:42.189 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.32818.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32818.log.gz 2024-09-18T11:35:42.189 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32949.log 2024-09-18T11:35:42.190 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-18T11:35:42.190 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.32949.log: gzip -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32949.log.gz 2024-09-18T11:35:42.190 INFO:teuthology.orchestra.run.smithi148.stderr: /var/log/ceph/ceph-mgr.x.log 88.2% 2024-09-18T11:35:42.190 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-18T11:35:42.190 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32738.log 2024-09-18T11:35:42.190 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-18T11:35:42.190 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.32738.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32900.log 2024-09-18T11:35:42.190 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32738.log.gz 2024-09-18T11:35:42.190 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mon.b.log: 87.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-18T11:35:42.190 INFO:teuthology.orchestra.run.smithi148.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-18T11:35:42.190 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.32900.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32900.log.gz 2024-09-18T11:35:42.251 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-18T11:35:42.251 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62308.log 2024-09-18T11:35:42.251 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37880.log 2024-09-18T11:35:42.251 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53555.log 2024-09-18T11:35:42.251 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62308.log.gz 2024-09-18T11:35:42.251 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76830.log 2024-09-18T11:35:42.251 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37880.log.gz 2024-09-18T11:35:42.251 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53555.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.67435.log -- replaced with /var/log/ceph/ceph-client.admin.53555.log.gz 2024-09-18T11:35:42.251 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-18T11:35:42.251 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60273.log 2024-09-18T11:35:42.251 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76830.log.gz 2024-09-18T11:35:42.252 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45639.log 2024-09-18T11:35:42.252 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67435.log.gz 2024-09-18T11:35:42.252 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.60273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58689.log 2024-09-18T11:35:42.252 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60273.log.gz 2024-09-18T11:35:42.252 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64387.log 2024-09-18T11:35:42.252 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45639.log.gz 2024-09-18T11:35:42.252 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78168.log 2024-09-18T11:35:42.252 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58689.log.gz 2024-09-18T11:35:42.252 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54889.log 2024-09-18T11:35:42.252 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64387.log.gz 2024-09-18T11:35:42.252 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.78168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60471.log 2024-09-18T11:35:42.252 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78168.log.gz 2024-09-18T11:35:42.252 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66447.log 2024-09-18T11:35:42.253 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54889.log.gz 2024-09-18T11:35:42.253 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.60471.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60471.log.gz 2024-09-18T11:35:42.253 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65263.log 2024-09-18T11:35:42.253 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.66447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78798.log 2024-09-18T11:35:42.253 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66447.log.gz 2024-09-18T11:35:42.253 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37442.log 2024-09-18T11:35:42.253 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65263.log.gz 2024-09-18T11:35:42.253 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.78798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68633.log 2024-09-18T11:35:42.253 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78798.log.gz 2024-09-18T11:35:42.253 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58607.log 2024-09-18T11:35:42.253 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37442.log.gz 2024-09-18T11:35:42.253 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75853.log 2024-09-18T11:35:42.253 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68633.log.gz 2024-09-18T11:35:42.253 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37542.log 2024-09-18T11:35:42.254 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58607.log.gz 2024-09-18T11:35:42.254 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48668.log 2024-09-18T11:35:42.254 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75853.log.gz 2024-09-18T11:35:42.254 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66911.log 2024-09-18T11:35:42.254 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37542.log.gz 2024-09-18T11:35:42.254 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67293.log 2024-09-18T11:35:42.254 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48668.log.gz 2024-09-18T11:35:42.254 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.66911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60690.log 2024-09-18T11:35:42.254 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66911.log.gz 2024-09-18T11:35:42.254 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77173.log 2024-09-18T11:35:42.254 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67293.log.gz 2024-09-18T11:35:42.254 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.60690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65063.log 2024-09-18T11:35:42.254 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60690.log.gz 2024-09-18T11:35:42.255 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77173.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.77173.log.gz 2024-09-18T11:35:42.255 INFO:teuthology.orchestra.run.smithi119.stderr: /var/log/ceph/ceph-client.admin.43432.log 2024-09-18T11:35:42.255 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76488.log 2024-09-18T11:35:42.255 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65063.log.gz 2024-09-18T11:35:42.255 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76614.log 2024-09-18T11:35:42.255 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43432.log.gz 2024-09-18T11:35:42.255 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38321.log 2024-09-18T11:35:42.255 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76488.log.gz 2024-09-18T11:35:42.255 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78672.log 2024-09-18T11:35:42.255 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76614.log.gz 2024-09-18T11:35:42.255 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38551.log 2024-09-18T11:35:42.255 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38321.log.gz 2024-09-18T11:35:42.255 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.78672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65907.log 2024-09-18T11:35:42.256 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78672.log.gz 2024-09-18T11:35:42.256 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77706.log 2024-09-18T11:35:42.256 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38551.log.gz 2024-09-18T11:35:42.256 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58989.log 2024-09-18T11:35:42.256 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65907.log.gz 2024-09-18T11:35:42.256 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39484.log 2024-09-18T11:35:42.256 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77706.log.gz 2024-09-18T11:35:42.256 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78042.log 2024-09-18T11:35:42.256 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58989.log.gz 2024-09-18T11:35:42.256 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39484.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.39484.log.gz /var/log/ceph/ceph-client.admin.61467.log 2024-09-18T11:35:42.256 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-18T11:35:42.256 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.78042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38200.log 2024-09-18T11:35:42.256 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78042.log.gz 2024-09-18T11:35:42.256 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61467.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.61467.log.gz 2024-09-18T11:35:42.257 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/objectstore_tool.69483.log 2024-09-18T11:35:42.257 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38200.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-18T11:35:42.257 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38200.log.gz 2024-09-18T11:35:42.257 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/objectstore_tool.69483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61989.log 2024-09-18T11:35:42.257 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79050.log 2024-09-18T11:35:42.257 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61989.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61989.log.gz 2024-09-18T11:35:42.257 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78546.log 2024-09-18T11:35:42.257 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63766.log 2024-09-18T11:35:42.257 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.79050.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79050.log.gz 2024-09-18T11:35:42.257 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.78546.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78546.log.gz 2024-09-18T11:35:42.257 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42020.log 2024-09-18T11:35:42.257 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76278.log 2024-09-18T11:35:42.257 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63766.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63766.log.gz 2024-09-18T11:35:42.257 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42020.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42020.log.gz 2024-09-18T11:35:42.258 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64265.log 2024-09-18T11:35:42.258 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59935.log 2024-09-18T11:35:42.258 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76278.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76278.log.gz 2024-09-18T11:35:42.258 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61565.log 2024-09-18T11:35:42.258 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64265.log.gz 2024-09-18T11:35:42.258 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59935.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59935.log.gz 2024-09-18T11:35:42.258 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37241.log 2024-09-18T11:35:42.258 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64428.log 2024-09-18T11:35:42.258 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61565.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61565.log.gz 2024-09-18T11:35:42.258 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37241.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37241.log.gz 2024-09-18T11:35:42.258 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56646.log 2024-09-18T11:35:42.258 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38241.log 2024-09-18T11:35:42.258 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64428.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64428.log.gz 2024-09-18T11:35:42.259 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.56646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65206.log 2024-09-18T11:35:42.259 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56646.log.gz 2024-09-18T11:35:42.259 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38241.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38241.log.gz 2024-09-18T11:35:42.259 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38405.log 2024-09-18T11:35:42.259 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61507.log 2024-09-18T11:35:42.259 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65206.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65206.log.gz 2024-09-18T11:35:42.259 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38405.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38405.log.gz 2024-09-18T11:35:42.259 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77169.log 2024-09-18T11:35:42.259 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40201.log 2024-09-18T11:35:42.259 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61507.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61507.log.gz 2024-09-18T11:35:42.259 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65431.log 2024-09-18T11:35:42.259 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77169.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77169.log.gz 2024-09-18T11:35:42.259 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78966.log 2024-09-18T11:35:42.260 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40201.log.gz 2024-09-18T11:35:42.260 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63315.log 2024-09-18T11:35:42.260 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65431.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65431.log.gz 2024-09-18T11:35:42.260 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.78966.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78966.log.gz 2024-09-18T11:35:42.260 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66487.log 2024-09-18T11:35:42.260 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60767.log 2024-09-18T11:35:42.260 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63315.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63315.log.gz 2024-09-18T11:35:42.260 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.66487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78882.log 2024-09-18T11:35:42.260 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66487.log.gz 2024-09-18T11:35:42.260 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.60767.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60767.log.gz 2024-09-18T11:35:42.260 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62511.log 2024-09-18T11:35:42.260 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53929.log 2024-09-18T11:35:42.261 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.78882.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78882.log.gz 2024-09-18T11:35:42.261 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39146.log 2024-09-18T11:35:42.261 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62511.log.gz 2024-09-18T11:35:42.261 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74925.log 2024-09-18T11:35:42.261 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53929.log.gz 2024-09-18T11:35:42.261 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39146.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39146.log.gz 2024-09-18T11:35:42.261 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54558.log 2024-09-18T11:35:42.261 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.74925.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74925.log.gz 2024-09-18T11:35:42.261 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76838.log 2024-09-18T11:35:42.261 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54558.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54558.log.gz 2024-09-18T11:35:42.261 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76530.log 2024-09-18T11:35:42.261 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79132.log 2024-09-18T11:35:42.261 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76838.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76838.log.gz 2024-09-18T11:35:42.262 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76446.log 2024-09-18T11:35:42.262 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76530.log.gz 2024-09-18T11:35:42.262 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.79132.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79132.log.gz 2024-09-18T11:35:42.262 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77090.log 2024-09-18T11:35:42.262 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37244.log 2024-09-18T11:35:42.262 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76446.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76446.log.gz 2024-09-18T11:35:42.262 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37964.log 2024-09-18T11:35:42.262 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77090.log.gz 2024-09-18T11:35:42.262 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37244.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37244.log.gz 2024-09-18T11:35:42.262 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63704.log 2024-09-18T11:35:42.262 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59306.log 2024-09-18T11:35:42.262 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37964.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37964.log.gz 2024-09-18T11:35:42.262 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63704.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.66008.log 2024-09-18T11:35:42.262 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63704.log.gz 2024-09-18T11:35:42.263 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59306.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59306.log.gz 2024-09-18T11:35:42.263 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37877.log 2024-09-18T11:35:42.263 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5/var/log/ceph/ceph-client.admin.66008.log: --verbose -- /var/log/ceph/ceph-client.admin.62573.log 2024-09-18T11:35:42.263 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66008.log.gz 2024-09-18T11:35:42.263 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69053.log 2024-09-18T11:35:42.263 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37877.log.gz 2024-09-18T11:35:42.263 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39506.log 2024-09-18T11:35:42.263 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62573.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62573.log.gz 2024-09-18T11:35:42.263 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39798.log 2024-09-18T11:35:42.263 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69053.log.gz 2024-09-18T11:35:42.263 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.39506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63888.log 2024-09-18T11:35:42.263 INFO:teuthology.orchestra.run.smithi119.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.0.39506.log.gz 2024-09-18T11:35:42.263 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78294.log 2024-09-18T11:35:42.263 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39798.log.gz 2024-09-18T11:35:42.264 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63888.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63888.log.gz 2024-09-18T11:35:42.264 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36982.log 2024-09-18T11:35:42.264 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42219.log 2024-09-18T11:35:42.264 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.78294.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78294.log.gz 2024-09-18T11:35:42.264 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.36982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40296.log 2024-09-18T11:35:42.264 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36982.log.gz 2024-09-18T11:35:42.264 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42219.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42219.log.gz 2024-09-18T11:35:42.264 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69217.log 2024-09-18T11:35:42.264 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42121.log 2024-09-18T11:35:42.264 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40296.log.gz 2024-09-18T11:35:42.264 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44891.log 2024-09-18T11:35:42.264 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69217.log.gz 2024-09-18T11:35:42.265 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42121.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.42121.log.gz 2024-09-18T11:35:42.265 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61070.log 2024-09-18T11:35:42.265 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43038.log 2024-09-18T11:35:42.265 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44891.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44891.log.gz 2024-09-18T11:35:42.265 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77080.log 2024-09-18T11:35:42.265 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61070.log.gz 2024-09-18T11:35:42.265 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43038.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43038.log.gz 2024-09-18T11:35:42.265 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78714.log 2024-09-18T11:35:42.265 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77747.log 2024-09-18T11:35:42.265 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77080.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77080.log.gz 2024-09-18T11:35:42.265 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.78714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39736.log 2024-09-18T11:35:42.265 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78714.log.gz 2024-09-18T11:35:42.266 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77747.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77747.log.gz 2024-09-18T11:35:42.266 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76837.log 2024-09-18T11:35:42.266 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78378.log 2024-09-18T11:35:42.266 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39736.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39736.log.gz 2024-09-18T11:35:42.266 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76837.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76837.log.gz 2024-09-18T11:35:42.266 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53083.log 2024-09-18T11:35:42.266 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56310.log 2024-09-18T11:35:42.266 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.78378.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78378.log.gz 2024-09-18T11:35:42.266 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58525.log 2024-09-18T11:35:42.266 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53083.log.gz 2024-09-18T11:35:42.267 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.56310.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.71769.log 2024-09-18T11:35:42.267 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56310.log.gz 2024-09-18T11:35:42.267 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63580.log 2024-09-18T11:35:42.267 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58525.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58525.log.gz 2024-09-18T11:35:42.267 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/objectstore_tool.71769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52627.log 2024-09-18T11:35:42.267 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56235.log 2024-09-18T11:35:42.267 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63580.log.gz 2024-09-18T11:35:42.267 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.52627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77430.log 2024-09-18T11:35:42.267 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52627.log.gz 2024-09-18T11:35:42.267 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.56235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37535.log 2024-09-18T11:35:42.267 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56235.log.gz 2024-09-18T11:35:42.267 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62735.log 2024-09-18T11:35:42.267 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77430.log.gz 2024-09-18T11:35:42.268 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37247.log 2024-09-18T11:35:42.268 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37535.log.gz 2024-09-18T11:35:42.268 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49300.log 2024-09-18T11:35:42.268 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62735.log.gz 2024-09-18T11:35:42.268 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53319.log 2024-09-18T11:35:42.268 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37247.log.gz 2024-09-18T11:35:42.268 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.49300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67012.log 2024-09-18T11:35:42.268 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49300.log.gz 2024-09-18T11:35:42.268 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55796.log 2024-09-18T11:35:42.268 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53319.log.gz 2024-09-18T11:35:42.268 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76836.log 2024-09-18T11:35:42.268 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67012.log.gz 2024-09-18T11:35:42.269 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37541.log 2024-09-18T11:35:42.269 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55796.log.gz 2024-09-18T11:35:42.269 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37245.log 2024-09-18T11:35:42.269 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76836.log.gz 2024-09-18T11:35:42.269 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76404.log 2024-09-18T11:35:42.269 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37541.log.gz 2024-09-18T11:35:42.269 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37245.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-18T11:35:42.269 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37245.log.gz 2024-09-18T11:35:42.269 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77529.log 2024-09-18T11:35:42.269 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76404.log.gz 2024-09-18T11:35:42.269 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55701.log 2024-09-18T11:35:42.269 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77529.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77529.log.gz 2024-09-18T11:35:42.270 INFO:teuthology.orchestra.run.smithi119.stderr: -- /var/log/ceph/ceph-client.admin.69259.log 2024-09-18T11:35:42.270 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55701.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55701.log.gz 2024-09-18T11:35:42.270 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59470.log 2024-09-18T11:35:42.270 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69259.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69259.log.gz 2024-09-18T11:35:42.270 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77750.log 2024-09-18T11:35:42.270 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59470.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59470.log.gz 2024-09-18T11:35:42.270 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.77428.log 2024-09-18T11:35:42.270 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77750.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77750.log.gz 2024-09-18T11:35:42.270 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39319.log 2024-09-18T11:35:42.270 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77428.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77428.log.gz 2024-09-18T11:35:42.270 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55504.log 2024-09-18T11:35:42.270 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39319.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39319.log.gz 2024-09-18T11:35:42.270 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.71007.log 2024-09-18T11:35:42.270 INFO:teuthology.orchestra.run.smithi119.stderr: 93.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-18T11:35:42.271 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55504.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55504.log.gz 2024-09-18T11:35:42.271 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/objectstore_tool.44982.log 2024-09-18T11:35:42.271 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/objectstore_tool.71007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41563.log 2024-09-18T11:35:42.271 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/objectstore_tool.44982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78588.log 2024-09-18T11:35:42.271 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.41563.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39920.log 2024-09-18T11:35:42.271 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41563.log.gz 2024-09-18T11:35:42.271 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.78588.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78588.log.gz 2024-09-18T11:35:42.271 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38690.log 2024-09-18T11:35:42.271 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36791.log 2024-09-18T11:35:42.271 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39920.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39920.log.gz 2024-09-18T11:35:42.271 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38690.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38690.log.gz 2024-09-18T11:35:42.271 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76320.log 2024-09-18T11:35:42.271 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.36791.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36791.log.gz 2024-09-18T11:35:42.271 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37540.log 2024-09-18T11:35:42.271 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39167.log 2024-09-18T11:35:42.272 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37540.log: /var/log/ceph/ceph-client.admin.76320.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76320.log.gz 2024-09-18T11:35:42.272 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37540.log.gz 2024-09-18T11:35:42.272 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58468.log 2024-09-18T11:35:42.272 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58948.log 2024-09-18T11:35:42.272 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39167.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39167.log.gz 2024-09-18T11:35:42.272 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58468.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58468.log.gz 2024-09-18T11:35:42.272 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77877.log 2024-09-18T11:35:42.272 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58948.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58948.log.gz 2024-09-18T11:35:42.272 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76068.log 2024-09-18T11:35:42.272 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37539.log 2024-09-18T11:35:42.272 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77877.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77877.log.gz 2024-09-18T11:35:42.272 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76068.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52669.log 2024-09-18T11:35:42.272 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76068.log.gz 2024-09-18T11:35:42.272 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37539.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37539.log.gz 2024-09-18T11:35:42.273 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54653.log 2024-09-18T11:35:42.273 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69385.log 2024-09-18T11:35:42.273 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.52669.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52669.log.gz 2024-09-18T11:35:42.273 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54653.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54653.log.gz 2024-09-18T11:35:42.273 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59853.log 2024-09-18T11:35:42.273 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69385.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69385.log.gz 2024-09-18T11:35:42.273 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67194.log 2024-09-18T11:35:42.273 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78504.log 2024-09-18T11:35:42.273 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59853.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59853.log.gz 2024-09-18T11:35:42.273 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67194.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67194.log.gz 2024-09-18T11:35:42.273 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36847.log 2024-09-18T11:35:42.273 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.78504.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78504.log.gz 2024-09-18T11:35:42.273 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39318.log 2024-09-18T11:35:42.274 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37780.log 2024-09-18T11:35:42.274 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.36847.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36847.log.gz 2024-09-18T11:35:42.274 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39318.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39318.log.gz 2024-09-18T11:35:42.274 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53001.log 2024-09-18T11:35:42.274 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37780.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37780.log.gz 2024-09-18T11:35:42.274 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52886.log 2024-09-18T11:35:42.274 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78630.log 2024-09-18T11:35:42.274 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53001.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53001.log.gz 2024-09-18T11:35:42.274 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.52886.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52886.log.gz 2024-09-18T11:35:42.274 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59552.log 2024-09-18T11:35:42.274 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.78630.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78630.log.gz 2024-09-18T11:35:42.274 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67516.log 2024-09-18T11:35:42.274 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66606.log 2024-09-18T11:35:42.274 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59552.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59552.log.gz 2024-09-18T11:35:42.275 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67516.log.gz 2024-09-18T11:35:42.275 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78336.log 2024-09-18T11:35:42.275 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.66606.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66606.log.gz 2024-09-18T11:35:42.275 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42538.log 2024-09-18T11:35:42.275 INFO:teuthology.orchestra.run.smithi119.stderr: 93.6% -- replaced with /var/log/ceph/objectstore_tool.71769.log.gz 2024-09-18T11:35:42.275 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61269.log 2024-09-18T11:35:42.275 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.78336.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78336.log.gz 2024-09-18T11:35:42.275 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42538.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42538.log.gz 2024-09-18T11:35:42.275 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37242.log 2024-09-18T11:35:42.275 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57971.log 2024-09-18T11:35:42.275 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61269.log: gzip/var/log/ceph/ceph-client.admin.37242.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.60908.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37242.log.gz 2024-09-18T11:35:42.275 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-18T11:35:42.275 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61269.log.gz/var/log/ceph/ceph-client.admin.57971.log: 2024-09-18T11:35:42.275 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57971.log.gz 2024-09-18T11:35:42.276 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77174.log 2024-09-18T11:35:42.276 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.60908.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60908.log.gz 2024-09-18T11:35:42.276 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41084.log 2024-09-18T11:35:42.276 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77174.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77174.log.gz 2024-09-18T11:35:42.276 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-18T11:35:42.276 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.41084.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41084.log.gz 2024-09-18T11:35:42.276 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64006.log 2024-09-18T11:35:42.276 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76835.log 2024-09-18T11:35:42.276 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64006.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42316.log 2024-09-18T11:35:42.276 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64006.log.gz 2024-09-18T11:35:42.276 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76572.log 2024-09-18T11:35:42.276 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76835.log: /var/log/ceph/ceph-client.admin.42316.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76835.log.gz 2024-09-18T11:35:42.276 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42316.log.gz 2024-09-18T11:35:42.277 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.70245.log 2024-09-18T11:35:42.277 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75984.log 2024-09-18T11:35:42.277 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76572.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76572.log.gz 2024-09-18T11:35:42.277 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/objectstore_tool.70245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59714.log 2024-09-18T11:35:42.277 INFO:teuthology.orchestra.run.smithi119.stderr: 93.6% -- replaced with /var/log/ceph/objectstore_tool.69483.log.gz 2024-09-18T11:35:42.277 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75984.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75984.log.gz 2024-09-18T11:35:42.277 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64825.log 2024-09-18T11:35:42.277 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59714.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69343.log 2024-09-18T11:35:42.277 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59714.log.gz 2024-09-18T11:35:42.277 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44890.log 2024-09-18T11:35:42.277 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69343.log: /var/log/ceph/ceph-client.admin.64825.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64825.log.gz 2024-09-18T11:35:42.277 INFO:teuthology.orchestra.run.smithi119.stderr: 25.5% -- replaced with /var/log/ceph/ceph-client.admin.69343.log.gz 2024-09-18T11:35:42.278 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76656.log 2024-09-18T11:35:42.278 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44890.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44890.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51753.log 2024-09-18T11:35:42.278 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-18T11:35:42.278 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38692.log 2024-09-18T11:35:42.278 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.51753.log: /var/log/ceph/ceph-client.admin.76656.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76656.log.gz 2024-09-18T11:35:42.278 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51753.log.gz 2024-09-18T11:35:42.278 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67517.log 2024-09-18T11:35:42.278 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38692.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39147.log 2024-09-18T11:35:42.278 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38692.log.gz 2024-09-18T11:35:42.278 INFO:teuthology.orchestra.run.smithi119.stderr: 93.5% -- replaced with /var/log/ceph/objectstore_tool.71007.log.gz 2024-09-18T11:35:42.278 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67517.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.67915.log 2024-09-18T11:35:42.278 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67517.log.gz 2024-09-18T11:35:42.278 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39147.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38363.log 2024-09-18T11:35:42.278 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39147.log.gz 2024-09-18T11:35:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67858.log 2024-09-18T11:35:42.279 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67915.log.gz 2024-09-18T11:35:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66970.log 2024-09-18T11:35:42.279 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38363.log.gz 2024-09-18T11:35:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.67858.log: /var/log/ceph/ceph-client.admin.53867.log 2024-09-18T11:35:42.279 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67858.log.gz/var/log/ceph/ceph-client.admin.66970.log: 2024-09-18T11:35:42.279 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66970.log.gz 2024-09-18T11:35:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37705.log 2024-09-18T11:35:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53867.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.76833.log 2024-09-18T11:35:42.279 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53867.log.gz 2024-09-18T11:35:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37705.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37705.log.gz 2024-09-18T11:35:42.279 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.78462.log 2024-09-18T11:35:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76833.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76833.log.gz 2024-09-18T11:35:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54815.log 2024-09-18T11:35:42.280 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5/var/log/ceph/ceph-client.admin.78462.log: --verbose -- /var/log/ceph/ceph-client.admin.66626.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78462.log.gz 2024-09-18T11:35:42.280 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-18T11:35:42.280 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54815.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54815.log.gz 2024-09-18T11:35:42.280 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75895.log 2024-09-18T11:35:42.280 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5/var/log/ceph/ceph-client.admin.66626.log: --verbose -- /var/log/ceph/ceph-client.admin.40703.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66626.log.gz 2024-09-18T11:35:42.280 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-18T11:35:42.280 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75895.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75895.log.gz 2024-09-18T11:35:42.280 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69052.log 2024-09-18T11:35:42.280 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5/var/log/ceph/ceph-client.admin.40703.log: --verbose -- /var/log/ceph/ceph-client.admin.59428.log 0.0% 2024-09-18T11:35:42.280 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40703.log.gz 2024-09-18T11:35:42.280 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69052.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69052.log.gz 2024-09-18T11:35:42.280 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44092.log 2024-09-18T11:35:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59428.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59428.log.gz 2024-09-18T11:35:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65608.log 2024-09-18T11:35:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44092.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44092.log.gz 2024-09-18T11:35:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59386.log 2024-09-18T11:35:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65608.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65608.log.gz 2024-09-18T11:35:42.281 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.69301.log 2024-09-18T11:35:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59386.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59386.log.gz 2024-09-18T11:35:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48747.log 2024-09-18T11:35:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69301.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69301.log.gz 2024-09-18T11:35:42.281 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41362.log 2024-09-18T11:35:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48747.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48747.log.gz 2024-09-18T11:35:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77846.log 2024-09-18T11:35:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.41362.log: gzip -5 0.0% --verbose -- /var/log/ceph/ceph-client.admin.77844.log 2024-09-18T11:35:42.281 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41362.log.gz 2024-09-18T11:35:42.282 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77846.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77846.log.gz 2024-09-18T11:35:42.282 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-18T11:35:42.282 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78420.log/var/log/ceph/ceph-client.admin.77844.log: 2024-09-18T11:35:42.282 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77844.log.gz/var/log/ceph/ceph-mon.a.log: 2024-09-18T11:35:42.282 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50628.log 2024-09-18T11:35:42.282 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.78420.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78420.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77370.log 2024-09-18T11:35:42.282 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-18T11:35:42.282 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50628.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50628.log.gz 2024-09-18T11:35:42.282 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77091.log 2024-09-18T11:35:42.282 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62469.log 2024-09-18T11:35:42.282 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77370.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77370.log.gz 2024-09-18T11:35:42.282 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77091.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77091.log.gz 2024-09-18T11:35:42.282 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61128.log 2024-09-18T11:35:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62469.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62469.log.gz 2024-09-18T11:35:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55444.log 2024-09-18T11:35:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78252.log 2024-09-18T11:35:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61128.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61128.log.gz 2024-09-18T11:35:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55444.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55444.log.gz 2024-09-18T11:35:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33348.log 2024-09-18T11:35:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.78252.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78252.log.gz 2024-09-18T11:35:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56949.log 2024-09-18T11:35:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58770.log 2024-09-18T11:35:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33348.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33348.log.gz 2024-09-18T11:35:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.56949.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56949.log.gz 2024-09-18T11:35:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56370.log 2024-09-18T11:35:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58770.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58770.log.gz 2024-09-18T11:35:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.46828.log 2024-09-18T11:35:42.284 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.47823.log 2024-09-18T11:35:42.284 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.56370.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56370.log.gz 2024-09-18T11:35:42.284 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/objectstore_tool.46828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78840.log 2024-09-18T11:35:42.284 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/objectstore_tool.47823.log: 93.5% -- replaced with /var/log/ceph/objectstore_tool.70245.log.gz 2024-09-18T11:35:42.284 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43616.log 2024-09-18T11:35:42.284 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.78840.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78840.log.gz 2024-09-18T11:35:42.284 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54064.log 2024-09-18T11:35:42.284 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43616.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43616.log.gz 2024-09-18T11:35:42.284 INFO:teuthology.orchestra.run.smithi119.stderr: 94.0% -- replaced with /var/log/ceph/objectstore_tool.44982.log.gz 2024-09-18T11:35:42.284 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42600.log 2024-09-18T11:35:42.284 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54064.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54064.log.gz 2024-09-18T11:35:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37937.log 2024-09-18T11:35:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37582.log 2024-09-18T11:35:42.285 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42600.log.gz 2024-09-18T11:35:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37937.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37937.log.gz 2024-09-18T11:35:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63518.log 2024-09-18T11:35:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37582.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37582.log.gz 2024-09-18T11:35:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-18T11:35:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59307.log 2024-09-18T11:35:42.285 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63518.log.gz 2024-09-18T11:35:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38980.log 2024-09-18T11:35:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59307.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59307.log.gz 2024-09-18T11:35:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52012.log 2024-09-18T11:35:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38980.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38980.log.gz 2024-09-18T11:35:42.286 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55147.log 2024-09-18T11:35:42.286 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.52012.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52012.log.gz 2024-09-18T11:35:42.286 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39778.log 2024-09-18T11:35:42.286 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55147.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55147.log.gz 2024-09-18T11:35:42.286 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66810.log 2024-09-18T11:35:42.286 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39778.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39778.log.gz 2024-09-18T11:35:42.286 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53712.log 2024-09-18T11:35:42.286 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.66810.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66810.log.gz 2024-09-18T11:35:42.286 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68872.log 2024-09-18T11:35:42.286 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53712.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53712.log.gz 2024-09-18T11:35:42.286 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-18T11:35:42.286 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68872.log.gz 2024-09-18T11:35:42.286 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54206.log 2024-09-18T11:35:42.314 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76110.log 2024-09-18T11:35:42.314 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54206.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54206.log.gz 93.6% -- replaced with /var/log/ceph/objectstore_tool.46828.log.gz 2024-09-18T11:35:42.314 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-18T11:35:42.314 INFO:teuthology.orchestra.run.smithi119.stderr: 93.4% -- replaced with /var/log/ceph/objectstore_tool.47823.log.gz 2024-09-18T11:35:42.314 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60132.log 2024-09-18T11:35:42.314 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77530.log 2024-09-18T11:35:42.314 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76110.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76110.log.gz 2024-09-18T11:35:42.314 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.60132.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60132.log.gz 2024-09-18T11:35:42.314 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68812.log 2024-09-18T11:35:42.314 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77530.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77530.log.gz 2024-09-18T11:35:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68037.log 2024-09-18T11:35:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76194.log 2024-09-18T11:35:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68812.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68812.log.gz 2024-09-18T11:35:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68037.log: 27.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77524.log 2024-09-18T11:35:42.315 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68037.log.gz 2024-09-18T11:35:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76194.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76194.log.gz 2024-09-18T11:35:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43773.log 2024-09-18T11:35:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55368.log 2024-09-18T11:35:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77524.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77524.log.gz 2024-09-18T11:35:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43773.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43773.log.gz 2024-09-18T11:35:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64048.log 2024-09-18T11:35:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55368.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55368.log.gz 2024-09-18T11:35:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62051.log 2024-09-18T11:35:42.316 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-18T11:35:42.316 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64048.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64048.log.gz 2024-09-18T11:35:42.316 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62051.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62051.log.gz 2024-09-18T11:35:42.316 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52724.log 2024-09-18T11:35:42.316 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.log: 92.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-18T11:35:42.316 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42881.log 2024-09-18T11:35:42.316 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44271.log 2024-09-18T11:35:42.316 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.52724.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52724.log.gz 2024-09-18T11:35:42.316 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42881.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42881.log.gz 2024-09-18T11:35:42.316 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55978.log 2024-09-18T11:35:42.316 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44271.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44271.log.gz 2024-09-18T11:35:42.316 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37066.log 2024-09-18T11:35:42.316 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62390.log 2024-09-18T11:35:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55978.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55978.log.gz 2024-09-18T11:35:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37066.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37066.log.gz 2024-09-18T11:35:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67515.log 2024-09-18T11:35:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63101.log 2024-09-18T11:35:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62390.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62390.log.gz 2024-09-18T11:35:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67515.log.gz 2024-09-18T11:35:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37024.log 2024-09-18T11:35:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79008.log 2024-09-18T11:35:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63101.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63101.log.gz 2024-09-18T11:35:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37024.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37024.log.gz 2024-09-18T11:35:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62718.log 2024-09-18T11:35:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36872.log 2024-09-18T11:35:42.318 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.79008.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79008.log.gz 2024-09-18T11:35:42.318 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62718.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62718.log.gz 2024-09-18T11:35:42.318 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66154.log 2024-09-18T11:35:42.318 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38281.log 2024-09-18T11:35:42.318 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.36872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36872.log.gz 2024-09-18T11:35:42.318 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.66154.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66154.log.gz 2024-09-18T11:35:42.318 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38694.log 2024-09-18T11:35:42.318 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38281.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38281.log.gz 2024-09-18T11:35:42.318 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33051.log 2024-09-18T11:35:42.318 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47272.log 2024-09-18T11:35:42.318 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38694.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38694.log.gz 2024-09-18T11:35:42.318 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33051.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33051.log.gz 2024-09-18T11:35:42.318 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74883.log 2024-09-18T11:35:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77429.log 2024-09-18T11:35:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.47272.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47272.log.gz 2024-09-18T11:35:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.74883.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74883.log.gz 2024-09-18T11:35:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38158.log 2024-09-18T11:35:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77429.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77429.log.gz 2024-09-18T11:35:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37820.log 2024-09-18T11:35:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46786.log 2024-09-18T11:35:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38158.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38158.log.gz 2024-09-18T11:35:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37820.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37820.log.gz 2024-09-18T11:35:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57810.log 2024-09-18T11:35:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.46786.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46786.log.gz 2024-09-18T11:35:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.46020.log 2024-09-18T11:35:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56053.log 2024-09-18T11:35:42.320 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.57810.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57810.log.gz 2024-09-18T11:35:42.320 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/objectstore_tool.46020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39039.log 2024-09-18T11:35:42.320 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.56053.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56053.log.gz 2024-09-18T11:35:42.320 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78210.log 2024-09-18T11:35:42.320 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39039.log.gz 2024-09-18T11:35:42.320 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42719.log 2024-09-18T11:35:42.320 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.78210.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78210.log.gz 2024-09-18T11:35:42.320 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37233.log 2024-09-18T11:35:42.320 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42719.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42719.log.gz 2024-09-18T11:35:42.320 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27193.log 2024-09-18T11:35:42.320 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37233.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37233.log.gz 2024-09-18T11:35:42.320 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38447.log 2024-09-18T11:35:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.tmp-client.admin.27193.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27193.log.gz 2024-09-18T11:35:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64865.log 2024-09-18T11:35:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38447.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38447.log.gz 2024-09-18T11:35:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75942.log 2024-09-18T11:35:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64865.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64865.log.gz 2024-09-18T11:35:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77094.log 2024-09-18T11:35:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75942.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75942.log.gz 2024-09-18T11:35:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79092.log 2024-09-18T11:35:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77094.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77094.log.gz 2024-09-18T11:35:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53278.log 2024-09-18T11:35:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.79092.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79092.log.gz 2024-09-18T11:35:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41761.log 2024-09-18T11:35:42.322 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53278.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53278.log.gz 2024-09-18T11:35:42.322 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65650.log 2024-09-18T11:35:42.322 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.41761.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41761.log.gz 2024-09-18T11:35:42.322 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55092.log 2024-09-18T11:35:42.322 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65650.log.gz 2024-09-18T11:35:42.322 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33466.log 2024-09-18T11:35:42.322 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55092.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55092.log.gz 2024-09-18T11:35:42.322 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63036.log 2024-09-18T11:35:42.322 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33466.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33466.log.gz 2024-09-18T11:35:42.322 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38691.log 2024-09-18T11:35:42.322 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63036.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63036.log.gz 2024-09-18T11:35:42.322 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78126.log 2024-09-18T11:35:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38691.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38691.log.gz 2024-09-18T11:35:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44550.log 2024-09-18T11:35:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.78126.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78126.log.gz 2024-09-18T11:35:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61730.log 2024-09-18T11:35:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44550.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44550.log.gz 2024-09-18T11:35:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77172.log 2024-09-18T11:35:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61730.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61730.log.gz 2024-09-18T11:35:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45906.log 2024-09-18T11:35:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77172.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77172.log.gz 2024-09-18T11:35:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66364.log 2024-09-18T11:35:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45906.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45906.log.gz 2024-09-18T11:35:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72558.log 2024-09-18T11:35:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.66364.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66364.log.gz 2024-09-18T11:35:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78084.log 2024-09-18T11:35:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.72558.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72558.log.gz 2024-09-18T11:35:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59654.log 2024-09-18T11:35:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.78084.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78084.log.gz 2024-09-18T11:35:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78756.log 2024-09-18T11:35:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59654.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59654.log.gz 2024-09-18T11:35:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77843.log 2024-09-18T11:35:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.78756.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78756.log.gz 2024-09-18T11:35:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44388.log 2024-09-18T11:35:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77843.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77843.log.gz 2024-09-18T11:35:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37248.log 2024-09-18T11:35:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44388.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44388.log.gz 2024-09-18T11:35:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62370.log 2024-09-18T11:35:42.325 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37248.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37248.log.gz 2024-09-18T11:35:42.331 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40646.log 2024-09-18T11:35:42.331 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62370.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62370.log.gz 2024-09-18T11:35:42.331 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43815.log 2024-09-18T11:35:42.331 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40646.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40646.log.gz 2024-09-18T11:35:42.331 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76832.log 2024-09-18T11:35:42.331 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43815.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43815.log.gz 2024-09-18T11:35:42.331 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76026.log 2024-09-18T11:35:42.331 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76832.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76832.log.gz 2024-09-18T11:35:42.331 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61648.log 2024-09-18T11:35:42.331 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76026.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76026.log.gz 2024-09-18T11:35:42.331 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67032.log 2024-09-18T11:35:42.331 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61648.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61648.log.gz 2024-09-18T11:35:42.332 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50379.log 2024-09-18T11:35:42.332 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67032.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67032.log.gz 2024-09-18T11:35:42.332 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56728.log 2024-09-18T11:35:42.332 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50379.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50379.log.gz 2024-09-18T11:35:42.332 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52565.log 2024-09-18T11:35:42.332 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.56728.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56728.log.gz 2024-09-18T11:35:42.332 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53610.log 2024-09-18T11:35:42.332 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.52565.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52565.log.gz 2024-09-18T11:35:42.332 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63414.log 2024-09-18T11:35:42.332 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53610.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53610.log.gz 2024-09-18T11:35:42.332 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63456.log 2024-09-18T11:35:42.332 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63414.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63414.log.gz 2024-09-18T11:35:42.332 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42642.log 2024-09-18T11:35:42.332 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63456.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63456.log.gz 2024-09-18T11:35:42.333 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77527.log 2024-09-18T11:35:42.333 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42642.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42642.log.gz 2024-09-18T11:35:42.333 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58150.log 2024-09-18T11:35:42.333 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77527.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77527.log.gz 2024-09-18T11:35:42.333 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool..log 2024-09-18T11:35:42.333 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58150.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58150.log.gz 2024-09-18T11:35:42.333 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43377.log 2024-09-18T11:35:42.337 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/objectstore_tool..log: 93.9% -- replaced with /var/log/ceph/objectstore_tool.46020.log.gz 2024-09-18T11:35:42.346 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60411.log 2024-09-18T11:35:42.354 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43377.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43377.log.gz 2024-09-18T11:35:42.354 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39617.log 2024-09-18T11:35:42.354 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.60411.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60411.log.gz 2024-09-18T11:35:42.354 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38489.log 2024-09-18T11:35:42.354 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39617.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39617.log.gz 2024-09-18T11:35:42.354 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68254.log 2024-09-18T11:35:42.354 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38489.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38489.log.gz 2024-09-18T11:35:42.354 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38116.log 2024-09-18T11:35:42.354 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68254.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68254.log.gz 2024-09-18T11:35:42.355 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37246.log 2024-09-18T11:35:42.355 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38116.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38116.log.gz 2024-09-18T11:35:42.355 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68296.log 2024-09-18T11:35:42.355 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37246.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37246.log.gz 2024-09-18T11:35:42.355 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58129.log 2024-09-18T11:35:42.355 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68296.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68296.log.gz 2024-09-18T11:35:42.372 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77845.log 2024-09-18T11:35:42.372 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58129.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58129.log.gz 2024-09-18T11:35:42.372 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63642.log 2024-09-18T11:35:42.373 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.77845.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77845.log.gz 2024-09-18T11:35:42.373 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68941.log 2024-09-18T11:35:42.373 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63642.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63642.log.gz 2024-09-18T11:35:42.373 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43099.log 2024-09-18T11:35:42.373 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68941.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68941.log.gz 2024-09-18T11:35:42.373 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61932.log 2024-09-18T11:35:42.373 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43099.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43099.log.gz 2024-09-18T11:35:42.373 INFO:teuthology.orchestra.run.smithi119.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-18T11:35:42.373 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64586.log 2024-09-18T11:35:42.373 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41439.log 2024-09-18T11:35:42.373 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61932.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61932.log.gz 2024-09-18T11:35:42.373 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64586.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64586.log.gz 2024-09-18T11:35:42.373 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76236.log 2024-09-18T11:35:42.373 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.41439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68593.log 2024-09-18T11:35:42.374 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41439.log.gz 2024-09-18T11:35:42.374 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69427.log 2024-09-18T11:35:42.374 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68593.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76236.log.gz 2024-09-18T11:35:42.374 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68593.log.gz 2024-09-18T11:35:42.374 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37876.log 2024-09-18T11:35:42.374 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.69427.log: -- /var/log/ceph/ceph-client.admin.40039.log 2024-09-18T11:35:42.374 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69427.log.gz 2024-09-18T11:35:42.374 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.37876.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37876.log.gz 2024-09-18T11:35:42.374 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58062.log 2024-09-18T11:35:42.374 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40039.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66166.log 2024-09-18T11:35:42.374 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40039.log.gz 2024-09-18T11:35:42.374 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41043.log 2024-09-18T11:35:42.374 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58062.log: /var/log/ceph/ceph-client.admin.66166.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58062.log.gz 2024-09-18T11:35:42.374 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66166.log.gz 2024-09-18T11:35:42.374 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48223.log 2024-09-18T11:35:42.375 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.41043.log: -- /var/log/ceph/ceph-client.admin.62876.log 2024-09-18T11:35:42.375 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41043.log.gz 2024-09-18T11:35:42.375 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48223.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48223.log.gz 2024-09-18T11:35:42.375 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76834.log 2024-09-18T11:35:42.375 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62876.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37940.log 2024-09-18T11:35:42.375 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62876.log.gz 2024-09-18T11:35:42.375 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63370.log 2024-09-18T11:35:42.375 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76834.log: /var/log/ceph/ceph-client.admin.37940.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76834.log.gz 2024-09-18T11:35:42.375 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37940.log.gz 2024-09-18T11:35:42.375 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64667.log 2024-09-18T11:35:42.375 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78924.log 2024-09-18T11:35:42.375 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63370.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63370.log.gz 2024-09-18T11:35:42.375 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64667.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64667.log.gz 2024-09-18T11:35:42.375 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-18T11:35:42.375 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.78924.log: /var/log/ceph/ceph-client.admin.76152.log 2024-09-18T11:35:42.376 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78924.log.gz 2024-09-18T11:35:42.383 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54462.log 2024-09-18T11:35:42.384 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76152.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76152.log.gz 2024-09-18T11:35:42.392 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44788.log 2024-09-18T11:35:42.393 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54462.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54462.log.gz 2024-09-18T11:35:42.393 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40243.log 2024-09-18T11:35:42.393 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44788.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44788.log.gz 2024-09-18T11:35:42.399 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40805.log 2024-09-18T11:35:42.400 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40243.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40243.log.gz 2024-09-18T11:35:42.400 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54360.log 2024-09-18T11:35:42.400 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40805.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40805.log.gz 2024-09-18T11:35:42.400 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44686.log 2024-09-18T11:35:42.400 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54360.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54360.log.gz 2024-09-18T11:35:42.400 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76362.log 2024-09-18T11:35:42.400 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44686.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44686.log.gz 2024-09-18T11:35:42.400 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44153.log 2024-09-18T11:35:42.400 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76362.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76362.log.gz 2024-09-18T11:35:42.400 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41823.log 2024-09-18T11:35:42.400 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44153.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44153.log.gz 2024-09-18T11:35:42.400 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.41823.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41823.log.gz 2024-09-18T11:35:42.422 INFO:teuthology.orchestra.run.smithi148.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-18T11:35:42.498 INFO:teuthology.orchestra.run.smithi119.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-18T11:35:42.541 INFO:teuthology.orchestra.run.smithi119.stderr: 93.6% -- replaced with /var/log/ceph/objectstore_tool..log.gz 2024-09-18T11:35:42.632 INFO:teuthology.orchestra.run.smithi119.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-18T11:35:42.843 INFO:teuthology.orchestra.run.smithi148.stderr: 91.1% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-18T11:35:42.999 INFO:teuthology.orchestra.run.smithi119.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-18T11:35:43.129 INFO:teuthology.orchestra.run.smithi148.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-18T11:35:43.142 INFO:teuthology.orchestra.run.smithi119.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-18T11:35:43.479 INFO:teuthology.orchestra.run.smithi148.stderr: 91.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-18T11:35:43.544 INFO:teuthology.orchestra.run.smithi119.stderr: 91.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-18T11:35:43.978 INFO:teuthology.orchestra.run.smithi148.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-18T11:35:43.979 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-09-18T11:35:43.979 INFO:teuthology.orchestra.run.smithi148.stderr:real 0m1.815s 2024-09-18T11:35:43.980 INFO:teuthology.orchestra.run.smithi148.stderr:user 0m4.835s 2024-09-18T11:35:43.980 INFO:teuthology.orchestra.run.smithi148.stderr:sys 0m0.167s 2024-09-18T11:35:44.219 INFO:teuthology.orchestra.run.smithi119.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-18T11:35:44.221 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-18T11:35:44.221 INFO:teuthology.orchestra.run.smithi119.stderr:real 0m2.097s 2024-09-18T11:35:44.221 INFO:teuthology.orchestra.run.smithi119.stderr:user 0m6.289s 2024-09-18T11:35:44.221 INFO:teuthology.orchestra.run.smithi119.stderr:sys 0m0.400s 2024-09-18T11:35:44.221 INFO:tasks.ceph:Archiving logs... 2024-09-18T11:35:44.221 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910958/remote/smithi119/log 2024-09-18T11:35:44.222 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T11:35:45.401 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910958/remote/smithi148/log 2024-09-18T11:35:45.401 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T11:35:46.239 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-18T11:35:46.248 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-18T11:35:46.248 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T11:35:46.251 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T11:35:46.325 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-18T11:35:46.334 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-18T11:35:46.334 DEBUG:teuthology.orchestra.run.smithi119:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T11:35:46.337 DEBUG:teuthology.orchestra.run.smithi148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T11:35:46.351 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-09-18T11:35:46.354 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-09-18T11:35:46.356 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T11:35:46.356 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-09-18T11:35:46.356 INFO:teuthology.orchestra.run.smithi119.stdout:^* hv01.front.sepia.ceph.com 3 6 377 10 -307us[ -300us] +/- 55ms 2024-09-18T11:35:46.356 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 13 +149us[ +156us] +/- 62ms 2024-09-18T11:35:46.356 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +298us[ +305us] +/- 68ms 2024-09-18T11:35:46.356 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T11:35:46.359 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T11:35:46.359 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-09-18T11:35:46.359 INFO:teuthology.orchestra.run.smithi148.stdout:^* hv01.front.sepia.ceph.com 3 6 377 11 -296us[ -266us] +/- 55ms 2024-09-18T11:35:46.359 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 12 +124us[ +154us] +/- 62ms 2024-09-18T11:35:46.359 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +313us[ +343us] +/- 68ms 2024-09-18T11:35:46.359 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T11:35:46.359 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-18T11:35:46.368 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-18T11:35:46.368 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-18T11:35:46.377 DEBUG:teuthology.orchestra.run.smithi119:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-18T11:35:46.486 DEBUG:teuthology.orchestra.run.smithi148:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-18T11:35:46.567 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-18T11:35:46.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T11:35:46.598 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-18T11:35:46.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T11:35:46.628 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-18T11:35:46.637 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-18T11:35:46.646 INFO:teuthology.task.internal:Duration was 763.686613 seconds 2024-09-18T11:35:46.647 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-18T11:35:46.655 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-18T11:35:46.655 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T11:35:46.658 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T11:35:46.695 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T11:35:46.708 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T11:35:47.032 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-18T11:35:47.033 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi119.front.sepia.ceph.com 2024-09-18T11:35:47.033 DEBUG:teuthology.orchestra.run.smithi119:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T11:35:47.094 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi148.front.sepia.ceph.com 2024-09-18T11:35:47.095 DEBUG:teuthology.orchestra.run.smithi148:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T11:35:47.125 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-18T11:35:47.125 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T11:35:47.137 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T11:35:47.192 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-18T11:35:47.192 DEBUG:teuthology.orchestra.run.smithi119:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T11:35:47.204 DEBUG:teuthology.orchestra.run.smithi148:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T11:35:47.358 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-18T11:35:47.367 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-18T11:35:47.368 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T11:35:47.392 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T11:35:47.417 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-18T11:35:47.427 DEBUG:teuthology.orchestra.run.smithi119:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T11:35:47.435 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T11:35:47.456 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = core 2024-09-18T11:35:47.481 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = core 2024-09-18T11:35:47.531 DEBUG:teuthology.orchestra.run.smithi119:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T11:35:47.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T11:35:47.546 DEBUG:teuthology.orchestra.run.smithi148:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T11:35:47.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T11:35:47.586 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-18T11:35:47.596 INFO:teuthology.task.internal:Transferring archived files... 2024-09-18T11:35:47.597 DEBUG:teuthology.misc:Transferring archived files from smithi119:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910958/remote/smithi119 2024-09-18T11:35:47.597 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T11:35:47.633 DEBUG:teuthology.misc:Transferring archived files from smithi148:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910958/remote/smithi148 2024-09-18T11:35:47.633 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T11:35:47.666 INFO:teuthology.task.internal:Removing archive directory... 2024-09-18T11:35:47.666 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T11:35:47.669 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T11:35:47.716 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-18T11:35:47.727 INFO:teuthology.task.internal:Not uploading archives. 2024-09-18T11:35:47.727 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-18T11:35:47.736 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-18T11:35:47.737 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T11:35:47.739 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T11:35:47.751 INFO:teuthology.orchestra.run.smithi119.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 11:35 /home/ubuntu/cephtest 2024-09-18T11:35:47.771 INFO:teuthology.orchestra.run.smithi148.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 11:35 /home/ubuntu/cephtest 2024-09-18T11:35:47.772 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-18T11:35:47.782 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-18T11:35:47.805 INFO:teuthology.run:Summary data: description: rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/dedup-io-snaps} duration: 763.6866128444672 flavor: default owner: scheduled_skanta@teuthology success: true 2024-09-18T11:35:47.805 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-18T11:35:47.888 INFO:teuthology.run:pass